noether.modeling.modules.mlp¶

Submodules¶

Classes¶

`MLP`	Implements a Multi-Layer Perceptron (MLP) with configurable number of layers, hidden dimension activation functions and weight initialization methods.
`MLPConfig`
`UpActDownMlp`	UpActDownMlp is a vanilla MLP with an up-projection followed by an GELU activation function and a
`UpActDownMLPConfig`

Package Contents¶

class noether.modeling.modules.mlp.MLP(config)¶

Bases: torch.nn.Module

Implements a Multi-Layer Perceptron (MLP) with configurable number of layers, hidden dimension activation functions and weight initialization methods. Only one hidden dimension is supported for simplicity, i.e., all hidden layers have the same dimension. The MLP will always have one input layer and one output layer. When num_layers=0, the MLP is a two layer network with one non-linearity in between. When num_layers>=1, the MLP has additional hidden layers, etc.

Initialize the MLP.

Parameters:: config (MLPConfig) – Configuration object for the MLP. See MLPConfig for available options.

init_weights¶

mlp¶

reset_parameters()¶

Reset the parameters of the MLP with a specific initialization. Options are “torch” (i.e., default), or: “truncnormal002”.

Raises:: NotImplementedError – raised if the specified initialization is not implemented.
Return type:: None

forward(x)¶

Forward function of the MLP.

Parameters:: x (torch.Tensor) – Input tensor to the MLP.
Returns:: Output tensor from the MLP.
Return type:: torch.Tensor

class noether.modeling.modules.mlp.MLPConfig(/, **data)¶

Bases: pydantic.BaseModel

Parameters:: data (Any)

input_dim: int = None¶: Input dimension of the MLP.

output_dim: int = None¶: Output dimension of the MLP.

hidden_dim: int = None¶: Hidden dimension for each layer.

num_layers: int = None¶: Number of hidden layers in the MLP. If 0, the MLP is a two linear layer MLP from input_dim, hidden_dim, activation to output_dim.

activation: Literal['RELU', 'GELU', 'SIGMOID', 'TANH', 'LEAKY_RELU', 'SOFTPLUS', 'ELU', 'SILU'] = 'GELU'¶: Activation function to use between layers.

init_weights: noether.core.types.InitWeightsMode = 'truncnormal002'¶: Weight initialization method.

bias: bool = None¶: Whether to use bias in the linear layers.

class noether.modeling.modules.mlp.UpActDownMlp(config)¶

Bases: torch.nn.Module

UpActDownMlp is a vanilla MLP with an up-projection followed by an GELU activation function and a down-projection to the original input dim.

Initialize the UpActDownMlp.

Parameters:: config (UpActDownMLPConfig) – The configuration of the UpActDownMlp.

init_weights¶

fc1¶

act¶

fc2¶

reset_parameters()¶

Reset the parameters of the MLP with a specific initialization. Options are “torch” (i.e., default), or: “truncnormal002”.

Raises:: NotImplementedError – raised if the specified initialization is not implemented.
Return type:: None

forward(x)¶

Forward function of the UpActDownMlp.

Parameters:: x (torch.Tensor) – Input tensor to the MLP.
Returns:: Output tensor from the MLP.
Return type:: torch.Tensor

class noether.modeling.modules.mlp.UpActDownMLPConfig(/, **data)¶

Bases: pydantic.BaseModel

Parameters:: data (Any)

input_dim: int = None¶: Input dimension of the MLP.

hidden_dim: int = None¶: Hidden dimension of the MLP.

bias: bool = None¶: Whether to use bias in the MLP.

init_weights: noether.core.types.InitWeightsMode = None¶: Initialization method of the weights of the MLP. Options are “torch” (i.e., similar to the module) or ‘truncnormal002’. Defaults to ‘truncnormal002’.

check_dims()¶

Validator to check that hidden_dim is greater than input_dim.

Raises:: ValueError – raised if hidden_dim is not greater than input_dim.
Return type:: UpActDownMLPConfig