Class SigmoidBinaryCrossEntropyLoss

Namespace

NeuralNetworks.Losses

Assembly

NeuralNetworks.dll

Binary Cross-Entropy loss combined with Sigmoid activation function.

public class SigmoidBinaryCrossEntropyLoss : Loss<float[,]>

Inheritance

object

Loss<float[,]>

SigmoidBinaryCrossEntropyLoss

Inherited Members

Loss<float[,]>.Prediction

Loss<float[,]>.Target

Loss<float[,]>.Forward(float[,], float[,])

Loss<float[,]>.Backward()

Loss<float[,]>.CalculateLoss()

Loss<float[,]>.CalculateLossGradient()

Loss<float[,]>.CloneBase()

Loss<float[,]>.Clone()

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

Input: The predicted values (logits) and the target values (binary labels) as 2D tensors.

Formula: L = -[y · log(σ(ŷ)) + (1 - y) · log(1 - σ(ŷ))] where σ(ŷ) = 1/(1 + e^(-ŷ))

Gradient Formula: ∂L/∂ŷᵢ = σ(ŷᵢ) - yᵢ (elegantly simple due to combined sigmoid+BCE derivative)

Description: This loss function combines the sigmoid activation with binary cross-entropy loss, making it the standard choice for binary classification problems. It expects raw logits (unnormalized scores) as predictions and binary labels (0 or 1) as targets. The sigmoid converts logits to probabilities in the range (0, 1), and the binary cross-entropy measures how well the predicted probabilities match the true binary labels.

Remarks: Combining sigmoid and binary cross-entropy into a single operation provides numerical stability by avoiding potential issues with log(0) or extreme sigmoid values. The gradient simplifies to (sigmoid_output - target), which is computationally efficient and mirrors the elegant gradient of softmax cross-entropy for multi-class problems. This loss is widely used in binary classification tasks such as spam detection, medical diagnosis (disease/no disease), sentiment analysis (positive/negative), and any two-class classification problem. The eps (epsilon) parameter adds numerical stability. For multi-label classification (where each sample can belong to multiple classes simultaneously), this loss can be applied independently to each label. Unlike multi-class cross-entropy which uses one-hot encoding, binary cross-entropy works directly with scalar 0/1 targets. This loss naturally handles probability outputs and encourages confident predictions toward 0 or 1.

Constructors

SigmoidBinaryCrossEntropyLoss(float)

Binary Cross-Entropy loss combined with Sigmoid activation function.

public SigmoidBinaryCrossEntropyLoss(float eps = 1E-07)

Parameters

eps float

Small epsilon value added for numerical stability to prevent log(0) and log(1). Default is 1e-7.

Remarks

Input: The predicted values (logits) and the target values (binary labels) as 2D tensors.

Formula: L = -[y · log(σ(ŷ)) + (1 - y) · log(1 - σ(ŷ))] where σ(ŷ) = 1/(1 + e^(-ŷ))

Gradient Formula: ∂L/∂ŷᵢ = σ(ŷᵢ) - yᵢ (elegantly simple due to combined sigmoid+BCE derivative)

Description: This loss function combines the sigmoid activation with binary cross-entropy loss, making it the standard choice for binary classification problems. It expects raw logits (unnormalized scores) as predictions and binary labels (0 or 1) as targets. The sigmoid converts logits to probabilities in the range (0, 1), and the binary cross-entropy measures how well the predicted probabilities match the true binary labels.

Remarks: Combining sigmoid and binary cross-entropy into a single operation provides numerical stability by avoiding potential issues with log(0) or extreme sigmoid values. The gradient simplifies to (sigmoid_output - target), which is computationally efficient and mirrors the elegant gradient of softmax cross-entropy for multi-class problems. This loss is widely used in binary classification tasks such as spam detection, medical diagnosis (disease/no disease), sentiment analysis (positive/negative), and any two-class classification problem. The eps (epsilon) parameter adds numerical stability. For multi-label classification (where each sample can belong to multiple classes simultaneously), this loss can be applied independently to each label. Unlike multi-class cross-entropy which uses one-hot encoding, binary cross-entropy works directly with scalar 0/1 targets. This loss naturally handles probability outputs and encourages confident predictions toward 0 or 1.

Methods

CalculateLoss()

protected override float CalculateLoss()

Returns

float

CalculateLossGradient()

protected override float[,] CalculateLossGradient()

Returns

float[,]

ToString()

Returns a string that represents the current object.

public override string ToString()

Returns

string

A string that represents the current object.