Class BipolarSigmoid

Namespace

NeuralNetworks.Operations.ActivationFunctions

Assembly

NeuralNetworks.dll

Bipolar (Symmetric) Sigmoid activation function.

public class BipolarSigmoid : ActivationFunction<float[,], float[,]>

Inheritance

object

Operation

Operation<float[,], float[,]>

ActivationFunction<float[,], float[,]>

BipolarSigmoid

Inherited Members

Operation<float[,], float[,]>.Input

Operation<float[,], float[,]>.Output

Operation<float[,], float[,]>.Forward(float[,], bool)

Operation<float[,], float[,]>.Backward(float[,])

Operation<float[,], float[,]>.CalcOutput(bool)

Operation<float[,], float[,]>.CalcInputGradient(float[,])

Operation<float[,], float[,]>.Forward(object, bool)

Operation<float[,], float[,]>.Backward(object)

Operation<float[,], float[,]>.GetOutputType()

Operation<float[,], float[,]>.GetInputType()

Operation.SetRegistered()

Operation.GetOutputType()

Operation.GetInputType()

Operation.Forward(object, bool)

Operation.Backward(object)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

Formula: f(x) = scale · (σ(x) - 0.5) = scale · (1/(1 + e^(-x)) - 0.5)

Special case (scale=2): f(x) = 2σ(x) - 1, which is mathematically equivalent to tanh(x/2)

Input Gradient Formula: ∂L/∂x = ∂L/∂y · scale · σ(x) · (1 - σ(x))

Output Range: (-scale/2, scale/2)

Description: A variant of the Sigmoid function that centers the output around zero by subtracting 0.5 and applying a scale factor. This creates a zero-centered activation function, which can improve gradient flow compared to the standard sigmoid. The scale parameter allows control over the output range, enabling flexibility in network design.

Remarks: With scale = 2, this produces the classic bipolar sigmoid f(x) = 2σ(x) - 1, ranging from (-1, 1), which is mathematically identical to tanh(x/2). For general scale s, it produces a scaled tanh: f(x) = (s/2) · tanh(x/2). The zero-centered nature helps mitigate the zigzag gradient updates that can occur with non-zero-centered activations like standard sigmoid. However, it still inherits the vanishing gradient problem from the sigmoid function.

Constructors

BipolarSigmoid(float)

Bipolar (Symmetric) Sigmoid activation function.

public BipolarSigmoid(float scale = 1)

Parameters

scale float

The scaling factor applied to the output of the sigmoid function. Must be non-zero.

Remarks

Formula: f(x) = scale · (σ(x) - 0.5) = scale · (1/(1 + e^(-x)) - 0.5)

Special case (scale=2): f(x) = 2σ(x) - 1, which is mathematically equivalent to tanh(x/2)

Input Gradient Formula: ∂L/∂x = ∂L/∂y · scale · σ(x) · (1 - σ(x))

Output Range: (-scale/2, scale/2)

Description: A variant of the Sigmoid function that centers the output around zero by subtracting 0.5 and applying a scale factor. This creates a zero-centered activation function, which can improve gradient flow compared to the standard sigmoid. The scale parameter allows control over the output range, enabling flexibility in network design.

Remarks: With scale = 2, this produces the classic bipolar sigmoid f(x) = 2σ(x) - 1, ranging from (-1, 1), which is mathematically identical to tanh(x/2). For general scale s, it produces a scaled tanh: f(x) = (s/2) · tanh(x/2). The zero-centered nature helps mitigate the zigzag gradient updates that can occur with non-zero-centered activations like standard sigmoid. However, it still inherits the vanishing gradient problem from the sigmoid function.

Methods

CalcInputGradient(float[,])

Calculates input gradient.

protected override float[,] CalcInputGradient(float[,] outputGradient)

Parameters

outputGradient float[,]

Returns

float[,]

Remarks

Based on outputGradient, calculates changes in input.

CalcOutput(bool)

Computes output.

protected override float[,] CalcOutput(bool inference)

Parameters

inference bool

Returns

float[,]

ToString()

Returns a string that represents the current operation.

public override string ToString()

Returns

string

A string that represents the current operation.

Remarks

It is used in a layer/model description.