Class GradientDescentMomentumOptimizer

Namespace

NeuralNetworks.Optimizers

Assembly

NeuralNetworks.dll

Implements the classic "Stochastic" Gradient Descent (SGD) optimizer with momentum for neural network training. Updates parameters by applying momentum to the previous update and subtracting the scaled gradient using a specified learning rate.

public class GradientDescentMomentumOptimizer : Optimizer

Inheritance

object

Optimizer

GradientDescentMomentumOptimizer

Inherited Members

Optimizer.LearningRate

Optimizer.UpdateLearningRate(int, int, int)

Optimizer.Update<T>(T, T)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

The momentum term helps accelerate gradients in the relevant direction and dampens oscillations.

Constructors

GradientDescentMomentumOptimizer(LearningRate, float)

Implements the classic "Stochastic" Gradient Descent (SGD) optimizer with momentum for neural network training. Updates parameters by applying momentum to the previous update and subtracting the scaled gradient using a specified learning rate.

public GradientDescentMomentumOptimizer(LearningRate learningRate, float momentum)

Parameters

learningRate LearningRate

momentum float

Remarks

The momentum term helps accelerate gradients in the relevant direction and dampens oscillations.

Methods

ToString()

Returns a string that represents the current object.

public override string ToString()

Returns

string

A string that represents the current object.

Update(object, Span<float>, ReadOnlySpan<float>)

Updates the parameter values in-place using the provided gradients and the optimizer's momentum and learning rate settings.

protected override void Update(object paramsKey, Span<float> paramsToUpdate, ReadOnlySpan<float> paramGradients)

Parameters

paramsKey object

The key that uniquely identifies the parameter set to update. Used to maintain optimizer state for each parameter group.

paramsToUpdate Span<float>

A span containing the parameter values to be updated. The values are modified in-place based on the computed updates.

paramGradients ReadOnlySpan<float>

A read-only span containing the gradients corresponding to each parameter in paramsToUpdate. Must have the same length as paramsToUpdate.

Remarks

This method applies a momentum-based update to each parameter. The optimizer maintains a separate velocity vector for each parameter key, which is updated and used to compute the parameter adjustments. The method modifies the contents of paramsToUpdate directly.