Class ReLU3D

Namespace

NeuralNetworks.Operations.ActivationFunctions

Assembly

NeuralNetworks.dll

Rectified Linear Unit (ReLU) activation function for 3D tensors.

public class ReLU3D : ActivationFunction<float[,,], float[,,]>

Inheritance

object

Operation

Operation<float[,,], float[,,]>

ActivationFunction<float[,,], float[,,]>

ReLU3D

Inherited Members

Operation<float[,,], float[,,]>.Input

Operation<float[,,], float[,,]>.Output

Operation<float[,,], float[,,]>.Forward(float[,,], bool)

Operation<float[,,], float[,,]>.Backward(float[,,])

Operation<float[,,], float[,,]>.CalcOutput(bool)

Operation<float[,,], float[,,]>.CalcInputGradient(float[,,])

Operation<float[,,], float[,,]>.Forward(object, bool)

Operation<float[,,], float[,,]>.Backward(object)

Operation<float[,,], float[,,]>.GetOutputType()

Operation<float[,,], float[,,]>.GetInputType()

Operation.SetRegistered()

Operation.GetOutputType()

Operation.GetInputType()

Operation.Forward(object, bool)

Operation.Backward(object)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

Formula: f(x) = max(0, beta · x)

Input Gradient Formula: ∂L/∂x = ∂L/∂y · beta if x > 0, else 0

Output Range: [0, ∞) when beta > 0

Description: ReLU variant for 3D tensors, commonly used in sequence models or 3D convolutional networks. Maintains the same computational efficiency and gradient flow benefits as the standard ReLU while operating on 3D data structures such as sequences with features [batch, timesteps, features] or volumetric data.

Remarks: Functionally identical to ReLU2D but designed for 3D tensor operations. The beta parameter (default 1) scales the positive region. Inherits the same advantages (no vanishing gradients, computational efficiency) and disadvantages (dying ReLU problem) as the standard ReLU.

Constructors

ReLU3D(float)

Rectified Linear Unit (ReLU) activation function for 3D tensors.

public ReLU3D(float beta = 1)

Parameters

beta float

The scaling factor for positive inputs. Default is 1.

Remarks

Formula: f(x) = max(0, beta · x)

Input Gradient Formula: ∂L/∂x = ∂L/∂y · beta if x > 0, else 0

Output Range: [0, ∞) when beta > 0

Description: ReLU variant for 3D tensors, commonly used in sequence models or 3D convolutional networks. Maintains the same computational efficiency and gradient flow benefits as the standard ReLU while operating on 3D data structures such as sequences with features [batch, timesteps, features] or volumetric data.

Remarks: Functionally identical to ReLU2D but designed for 3D tensor operations. The beta parameter (default 1) scales the positive region. Inherits the same advantages (no vanishing gradients, computational efficiency) and disadvantages (dying ReLU problem) as the standard ReLU.

Methods

CalcInputGradient(float[,,])

Calculates input gradient.

protected override float[,,] CalcInputGradient(float[,,] outputGradient)

Parameters

outputGradient float[,,]

Returns

float[,,]

Remarks

Based on outputGradient, calculates changes in input.

CalcOutput(bool)

Computes output.

protected override float[,,] CalcOutput(bool inference)

Parameters

inference bool

Returns

float[,,]

ToString()

Returns a string that represents the current operation.

public override string ToString()

Returns

string

A string that represents the current operation.

Remarks

It is used in a layer/model description.