Class Tanh2D

Namespace

NeuralNetworks.Operations.ActivationFunctions

Assembly

NeuralNetworks.dll

Hyperbolic Tangent (Tanh) activation function for 2D tensors.

public class Tanh2D : ActivationFunction<float[,], float[,]>

Inheritance

object

Operation

Operation<float[,], float[,]>

ActivationFunction<float[,], float[,]>

Tanh2D

Inherited Members

Operation<float[,], float[,]>.Input

Operation<float[,], float[,]>.Output

Operation<float[,], float[,]>.Forward(float[,], bool)

Operation<float[,], float[,]>.Backward(float[,])

Operation<float[,], float[,]>.CalcOutput(bool)

Operation<float[,], float[,]>.CalcInputGradient(float[,])

Operation<float[,], float[,]>.Forward(object, bool)

Operation<float[,], float[,]>.Backward(object)

Operation<float[,], float[,]>.GetOutputType()

Operation<float[,], float[,]>.GetInputType()

Operation.SetRegistered()

Operation.GetOutputType()

Operation.GetInputType()

Operation.Forward(object, bool)

Operation.Backward(object)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

Formula: f(x) = tanh(x) = (exp(x) - exp(-x)) / (exp(x) + exp(-x)) = (e^(2x) - 1) / (e^(2x) + 1)

Input Gradient Formula: ∂L/∂x = ∂L/∂y · (1 - f(x)²) = ∂L/∂y · (1 - tanh²(x))

Output Range: (-1, 1)

Description: The hyperbolic tangent function is a smooth, S-shaped curve that maps input values to the range (-1, 1). Unlike Sigmoid, it is zero-centered, which can help with gradient flow during training. Commonly used in recurrent neural networks (RNNs) and as a hidden layer activation function.

Remarks: Tanh is essentially a scaled and shifted version of the sigmoid: tanh(x) = 2σ(2x) - 1. While it addresses the non-zero-centered issue of sigmoid, it still suffers from vanishing gradients for large absolute input values. The zero-centered output can lead to faster convergence compared to sigmoid in many cases.

Methods

CalcInputGradient(float[,])

Calculates input gradient.

protected override float[,] CalcInputGradient(float[,] outputGradient)

Parameters

outputGradient float[,]

Returns

float[,]

Remarks

Based on outputGradient, calculates changes in input.

CalcOutput(bool)

Computes output.

protected override float[,] CalcOutput(bool inference)

Parameters

inference bool

Returns

float[,]

ToString()

Returns a string that represents the current operation.

public override string ToString()

Returns

string

A string that represents the current operation.

Remarks

It is used in a layer/model description.