Class Sigmoid

Namespace

NeuralNetworks.Operations.ActivationFunctions

Assembly

NeuralNetworks.dll

Sigmoid (Logistic) activation function.

public class Sigmoid : ActivationFunction<float[,], float[,]>

Inheritance

object

Operation

Operation<float[,], float[,]>

ActivationFunction<float[,], float[,]>

Sigmoid

Inherited Members

Operation<float[,], float[,]>.Input

Operation<float[,], float[,]>.Output

Operation<float[,], float[,]>.Forward(float[,], bool)

Operation<float[,], float[,]>.Backward(float[,])

Operation<float[,], float[,]>.CalcOutput(bool)

Operation<float[,], float[,]>.CalcInputGradient(float[,])

Operation<float[,], float[,]>.Forward(object, bool)

Operation<float[,], float[,]>.Backward(object)

Operation<float[,], float[,]>.GetOutputType()

Operation<float[,], float[,]>.GetInputType()

Operation.SetRegistered()

Operation.GetOutputType()

Operation.GetInputType()

Operation.Forward(object, bool)

Operation.Backward(object)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

Formula: f(x) = 1 / (1 + exp(-x)) = σ(x)

Input Gradient Formula: ∂L/∂x = ∂L/∂y · f(x) · (1 - f(x)) = ∂L/∂y · σ(x) · (1 - σ(x))

Output Range: (0, 1)

Description: The sigmoid function is a smooth, S-shaped curve that squashes input values into the range (0, 1). It is commonly used in binary classification tasks, especially in the output layer for predicting probabilities. The function is differentiable everywhere, making it suitable for gradient-based optimization methods.

Remarks: The sigmoid function suffers from vanishing gradient problems for large positive or negative inputs, as the gradient approaches zero at the extremes. This can slow down training in deep networks. For hidden layers, alternatives like ReLU2D or Tanh2D are often preferred. The sigmoid is computationally more expensive than ReLU due to the exponential operation.

Methods

CalcInputGradient(float[,])

Calculates input gradient.

protected override float[,] CalcInputGradient(float[,] outputGradient)

Parameters

outputGradient float[,]

Returns

float[,]

Remarks

Based on outputGradient, calculates changes in input.

CalcOutput(bool)

Calculates the output of the Sigmoid activation function. The range is (0, 1).

protected override float[,] CalcOutput(bool inference)

Parameters

inference bool

Indicates whether the calculation is for inference or training.

Returns

float[,]

A two-dimensional array of floating-point values representing the output of the Sigmoid activation function.

ToString()

Returns a string that represents the current operation.

public override string ToString()

Returns

string

A string that represents the current operation.

Remarks

It is used in a layer/model description.