Class Linear

Namespace

NeuralNetworks.Operations.ActivationFunctions

Assembly

NeuralNetworks.dll

Linear (Identity) activation function.

public class Linear : ActivationFunction<float[,], float[,]>

Inheritance

object

Operation

Operation<float[,], float[,]>

ActivationFunction<float[,], float[,]>

Linear

Inherited Members

Operation<float[,], float[,]>.Input

Operation<float[,], float[,]>.Output

Operation<float[,], float[,]>.Forward(float[,], bool)

Operation<float[,], float[,]>.Backward(float[,])

Operation<float[,], float[,]>.CalcOutput(bool)

Operation<float[,], float[,]>.CalcInputGradient(float[,])

Operation<float[,], float[,]>.Forward(object, bool)

Operation<float[,], float[,]>.Backward(object)

Operation<float[,], float[,]>.GetOutputType()

Operation<float[,], float[,]>.GetInputType()

Operation.SetRegistered()

Operation.GetOutputType()

Operation.GetInputType()

Operation.Forward(object, bool)

Operation.Backward(object)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

Formula: f(x) = x

Input Gradient Formula: ∂L/∂x = ∂L/∂y · 1 = ∂L/∂y

Output Range: (-∞, ∞)

Description: The linear activation function simply returns its input without any transformation. It is primarily used in regression tasks where the output layer needs to predict continuous values without bounds. Since it provides no non-linearity, using linear activations in all layers would collapse a deep network into a single-layer linear model.

Remarks: The linear activation is also known as the "identity" function. It has a constant gradient of 1, which means it does not suffer from vanishing or exploding gradient problems by itself. However, stacking multiple linear layers without non-linear activations in between provides no benefit over a single linear layer, as the composition of linear functions is still linear. Linear activation is typically used only in the output layer for regression tasks or in specific architectural components where no transformation is desired (e.g., residual connections in ResNets use identity mappings).

Methods

CalcInputGradient(float[,])

Calculates input gradient.

protected override float[,] CalcInputGradient(float[,] outputGradient)

Parameters

outputGradient float[,]

Returns

float[,]

Remarks

Based on outputGradient, calculates changes in input.

CalcOutput(bool)

Computes output.

protected override float[,] CalcOutput(bool inference)

Parameters

inference bool

Returns

float[,]

ToString()

Returns a string that represents the current operation.

public override string ToString()

Returns

string

A string that represents the current operation.

Remarks

It is used in a layer/model description.