Class ActivationRoutines

Namespace

NeuralNetworks.Core

Assembly

NeuralNetworks.dll

Provides activation routines for neural network computations.

public static class ActivationRoutines

Inheritance

object

ActivationRoutines

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Includes element-wise transformations and probability normalization operations for float arrays, with row-wise softmax and log-softmax variants and numerically stable implementations where applicable. Methods return new arrays and do not mutate inputs.

Methods

Gelu(float[,])

Applies the Gaussian Error Linear Unit (GELU) activation function to each element of the specified two-dimensional array.

public static float[,] Gelu(this float[,] source)

Parameters

source float[,]

A two-dimensional array of single-precision floating-point values to which the GELU function will be applied.

Returns

float[,]

A two-dimensional array of the same dimensions as the input, where each element is the result of applying the GELU function to the corresponding element in the input array.

Remarks

The GELU activation function is commonly used in neural networks and is defined as GELU(x) = 0.5 * x * (1 + tanh(√(2/π) * (x + 0.044715 * x^3))). This method does not modify the input array; it returns a new array containing the transformed values.

LogSoftmax(float[,])

Computes the log-softmax of each row in a two-dimensional array of single-precision floating-point values.

public static float[,] LogSoftmax(this float[,] source)

Parameters

source float[,]

The two-dimensional array to transform (log-softmax applied per row).

Returns

float[,]

A new source with log-softmax-applied values.

Remarks

The log-softmax function is useful for numerical stability when working with probabilities in machine learning models. It is computed as the logarithm of the softmax function, which ensures that the output values are normalized and sum to 1 in the probability space.

Sigmoid(float[,])

Applies the sigmoid function to each element of the source.

public static float[,] Sigmoid(this float[,] source)

Parameters

source float[,]

The two-dimensional array to transform.

Returns

float[,]

A new source with each element transformed by the sigmoid function with the same dimensions as the original source.

SigmoidDerivative(float[,])

Calculates the derivative of the sigmoid function for each element of the source.

public static float[,] SigmoidDerivative(this float[,] source)

Parameters

source float[,]

The two-dimensional array to transform.

Returns

float[,]

A new source with each element transformed by the derivative of the sigmoid function with the same dimensions as the original source.

Remarks

The derivative of the sigmoid function is calculated as: sigmoid(x) * (1 - sigmoid(x)).

SoftmaxBasic(float[,])

Applies the softmax function to the source.

public static float[,] SoftmaxBasic(this float[,] source)

Parameters

source float[,]

The two-dimensional array to transform (softmax applied per row).

Returns

float[,]

A new source with softmax-applied values.

Remarks

Softmax formula: exp(x) / sum(exp(x)).

SoftmaxStable(float[,])

Computes the numerically stable softmax of each row in a two-dimensional array of single-precision floating-point values.

public static float[,] SoftmaxStable(this float[,] source)

Parameters

source float[,]

A two-dimensional array of type float representing the input data. Each row is treated as a separate vector for which the softmax will be computed.

Returns

float[,]

A two-dimensional array of type float containing the softmax probabilities for each row of the input array. Each row sums to 1.0.

Remarks

This method applies the softmax function to each row independently, using a numerically stable algorithm to prevent overflow or underflow. The input array is not modified.

SoftmaxStableWithTemperature(float[], float)

public static float[] SoftmaxStableWithTemperature(this float[] source, float temperature)

Parameters

source float[]

temperature float

Returns

float[]

Softplus(float[,])

Applies the Softplus activation function to each element of the specified two-dimensional array.

public static float[,] Softplus(this float[,] source)

Parameters

source float[,]

A two-dimensional array of single-precision floating-point values to which the Softplus function will be applied.

Returns

float[,]

A two-dimensional array of the same dimensions as source, where each element is the result of applying the Softplus function to the corresponding element in source.

Remarks

The Softplus function is defined as log(1 + exp(x)) and is commonly used as a smooth approximation to the ReLU activation in machine learning applications. The returned array is a new instance; the input array is not modified.

Tanh(float[,,,])

Applies the hyperbolic tangent function element-wise to the source.

public static float[,,,] Tanh(this float[,,,] source)

Parameters

source float[,,,]

The four-dimensional array to transform.

Returns

float[,,,]

A new source with the hyperbolic tangent applied element-wise.

Tanh(float[,])

Applies the hyperbolic tangent function element-wise to the source.

public static float[,] Tanh(this float[,] source)

Parameters

source float[,]

The two-dimensional array to transform.

Returns

float[,]

A new source with the hyperbolic tangent applied element-wise.