noether.modeling.modules.layers.layer_scale

Classes

LayerScale

LayerScale module scales the input tensor by a learnable parameter gamma.

Module Contents

class noether.modeling.modules.layers.layer_scale.LayerScale(config)

Bases: torch.nn.Module

LayerScale module scales the input tensor by a learnable parameter gamma.

Initialize the LayerScale module. :param config: Configuration for the LayerScale module. See LayerScaleConfig for details.

Parameters:

config (noether.core.schemas.modules.layers.LayerScaleConfig)

forward(x)

Forward function of the LayerScale module.

Parameters:

x (torch.Tensor) – Input tensor to be scaled.

Returns:

Tensor scaled by the gamma parameter.

Return type:

torch.Tensor