noether.modeling.modules.mlp¶

Submodules¶

Classes¶

`MLP`	Implements a Multi-Layer Perceptron (MLP) with configurable number of layers, hidden dimension activation functions and weight initialization methods.
`UpActDownMlp`	UpActDownMlp is a vanilla MLP with an up-projection followed by an GELU activation function and a

Package Contents¶

class noether.modeling.modules.mlp.MLP(config)¶

Bases: torch.nn.Module

Implements a Multi-Layer Perceptron (MLP) with configurable number of layers, hidden dimension activation functions and weight initialization methods. Only one hidden dimension is supported for simplicity, i.e., all hidden layers have the same dimension. The MLP will always have one input layer and one output layer. When num_layers=0, the MLP is a two layer network with one non-linearity in between. When num_layers>=1, the MLP has additional hidden layers, etc.

Initialize the MLP.

Parameters:: config (noether.core.schemas.modules.mlp.MLPConfig) – Configuration object for the MLP. See MLPConfig for available options.

init_weights¶

mlp¶

reset_parameters()¶

Reset the parameters of the MLP with a specific initialization. Options are “torch” (i.e., default), or: “truncnormal002”.

Raises:: NotImplementedError – raised if the specified initialization is not implemented.
Return type:: None

forward(x)¶

Forward function of the MLP.

Parameters:: x (torch.Tensor) – Input tensor to the MLP.
Returns:: Output tensor from the MLP.
Return type:: torch.Tensor

class noether.modeling.modules.mlp.UpActDownMlp(config)¶

Bases: torch.nn.Module

UpActDownMlp is a vanilla MLP with an up-projection followed by an GELU activation function and a down-projection to the original input dim.

Initialize the UpActDownMlp.

Parameters:: config (noether.core.schemas.modules.mlp.UpActDownMLPConfig) – The configuration of the UpActDownMlp.

init_weights¶

fc1¶

act¶

fc2¶

reset_parameters()¶

Reset the parameters of the MLP with a specific initialization. Options are “torch” (i.e., default), or: “truncnormal002”.

Raises:: NotImplementedError – raised if the specified initialization is not implemented.
Return type:: None

forward(x)¶

Forward function of the UpActDownMlp.

Parameters:: x (torch.Tensor) – Input tensor to the MLP.
Returns:: Output tensor from the MLP.
Return type:: torch.Tensor