noether.core.initializers¶

Submodules¶

Attributes¶

AnyInitializer

Classes¶

`InitializerBase`	Helper class that provides a standard way to create an ABC using
`InitializerConfig`
`CheckpointInitializer`	Base class to initialize models from checkpoints of previous runs. Should not be used directly, but inherited by other initializers such as PreviousRunInitializer or ResumeInitializer.
`CheckpointInitializerConfig`
`PreviousRunInitializer`	Initializes a model from a checkpoint of a previous run (specified by the run_id), this initializers hence only loads model weights.
`PreviousRunInitializerConfig`
`ResumeInitializer`	Initializes models/optimizers from a checkpoint ready for resuming training.
`ResumeInitializerConfig`

Package Contents¶

class noether.core.initializers.InitializerBase(path_provider)¶

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

Base class for model initializers.

Parameters:: path_provider (noether.core.providers.PathProvider) – PathProvider instance to access paths to load models from.

logger¶

path_provider¶

abstractmethod init_weights(model)¶

Initialize the model weights from the checkpoint.

Parameters:: model (noether.core.models.base.ModelBase) – the model to load the weights into.
Return type:: None

abstractmethod init_optimizer(model)¶

Initialize the optimizer for the model.

Parameters:: model (noether.core.models.base.ModelBase) – a model to initialize the optimizer for. Assumes the model has an attribute optim.
Return type:: None

init_trainer(trainer)¶

Initialize the trainer from the checkpoint.

By default, does nothing. Can be overridden by child classes.

Parameters:: trainer – the trainer to initialize.
Return type:: None

init_callbacks(callbacks, model)¶

Initialize the callbacks from the checkpoint.

By default, does nothing. Can be overridden by child classes.

Parameters:

callbacks (list[noether.core.callbacks.base.CallbackBase]) – the list of callbacks to initialize.
model (noether.core.models.base.ModelBase) – the model associated with the callbacks.

Return type:

None

start_checkpoint()¶

Get the start checkpoint for the model.

By default , returns a TrainingIteration starting from zero.

Returns:: the start checkpoint for the model.
Return type:: TrainingIteration

class noether.core.initializers.InitializerConfig(/, **data)¶

Bases: pydantic.BaseModel

Parameters:: data (Any)

kind: str = None¶

kwargs: dict[str, Any] | None = None¶: Additional keyword arguments to pass to the initializer.

run_id: str¶: A unique identifier for the training stage. This is used to find the correct checkpoint.

stage_name: str | None = None¶: The name of the stage training stage if defined. When training, the stage name is usually “train”.

model_name: str | None = None¶: The name of the model to load. This is the model_name used in CheckpointCallback.

checkpoint_tag: str | None | dict = None¶: Which checkpoint to load. Checkpoint is usually “latest” or “best_loss”, or “E*_U*_S*”, depending on which checkpoint you want to load.

model_info: str | None = None¶: Optional string to provide additional info about the model weights in the checkpoint filename. E.g., the stored weights are the EMA, or in a different precision.

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class noether.core.initializers.CheckpointInitializer(initializer_config, **kwargs)¶

Bases: noether.core.initializers.base.InitializerBase

Base class to initialize models from checkpoints of previous runs. Should not be used directly, but inherited by other initializers such as PreviousRunInitializer or ResumeInitializer.

Parameters:

initializer_config (CheckpointInitializerConfig) – configuration for the initializer. See CheckpointInitializerConfig for available options.
**kwargs – additional arguments to pass to the parent class.

checkpoint_tag: str | noether.core.utils.training.training_iteration.TrainingIteration¶

run_id¶

model_name¶

load_optim¶

model_info¶

pop_ckpt_kwargs_keys¶

stage_name¶

init_optimizer(model)¶

Initialize the optimizer for the model if it is derived from Model.

If model is a CompositeModel, nothing happens. This is expected as CompositeModels can be arbitrarily nested and do not have an optimizer. Instead, a CompositeModel calls init_optim with all its submodels which can be of type Model or a nested CompositeModel.

Parameters:: model (noether.core.models.ModelBase) – a model to initialize the optimizer for. Assumes the model has an attribute optim.
Return type:: None

class noether.core.initializers.CheckpointInitializerConfig(/, **data)¶

Bases: noether.core.initializers.base.InitializerConfig

Parameters:: data (Any)

kind: Literal['noether.core.initializers.CheckpointInitializer'] = None¶

load_optim: bool = None¶: Whether or not to load the optimizer state from the checkpoint. Default is True, as this is usually used to resume a training run

pop_ckpt_kwargs_keys: list[str] | None = None¶: which checkpoint to load. If a string is provided, must be one of (“latest”, “best_loss”). If a dictionary is provided, must contain keys “epoch”, “update”, “sample” to identify the checkpoint.

output_path: pathlib.Path | None = None¶: Output root where the source run (identified by run_id/stage_name) lives. When None, the current run’s path provider is used to locate it, which assumes the source shares this run’s output_path. Set explicitly by noether-eval so that overriding output_path for the eval run doesn’t redirect source-checkpoint lookup.

class noether.core.initializers.PreviousRunInitializer(initializer_config, **kwargs)¶

Bases: noether.core.initializers.checkpoint.CheckpointInitializer

Initializes a model from a checkpoint of a previous run (specified by the run_id), this initializers hence only loads model weights. When a previous run should be resumed for further training, use ResumeInitializer instead. This initializer needs to be initialized as part of a model config. It is possible to remove certain keys or patterns from the checkpoint before loading it into the model, or to rename certain patterns.

For example:

model:
  kind: path.to.MyModelClass
  param1: value1
  name: my_model
  initializers:
    - kind: noether.core.initializers.PreviousRunInitializer
    run_id: <run_id>
    model_name: transformer
    stage_name: train
    checkpoint_tag: last
    keys_to_remove:
        - encoder.block1.weight

Parameters:

initializer_config (PreviousRunInitializerConfig) – Configuration for the initializer. See PreviousRunInitializerConfig for available options.
**kwargs (dict) – additional arguments to pass to the parent class.

keys_to_remove¶

patterns_to_remove¶

patterns_to_rename¶

patterns_to_instantiate¶

init_weights(model, model_name=None)¶

Initialize the model weights from the checkpoint.

Parameters:

model (noether.core.models.ModelBase) – the model to load the weights into.
model_name (str | None)

Return type:

None

init_callbacks(callbacks, model)¶

Initialize the callbacks from the checkpoint.

Parameters:

callbacks (list[noether.core.callbacks.base.CallbackBase]) – the callbacks to initialize.
model (noether.core.models.ModelBase) – the model to initialize the callbacks for.

Return type:

None

class noether.core.initializers.PreviousRunInitializerConfig(/, **data)¶

Bases: noether.core.initializers.checkpoint.CheckpointInitializerConfig

Parameters:: data (Any)

kind: Literal['noether.core.initializers.PreviousRunInitializer'] = None¶

load_optim: bool = None¶: Whether or not to load the optimizer state from the checkpoint. Default is True, as this is usually used to resume a training run

keys_to_remove: list[str] | None = None¶: List of keys to remove from the checkpoint.

patterns_to_remove: list[str] | None = None¶: List of patterns to remove from the checkpoint.

patterns_to_rename: list[dict] | None = None¶: List of patterns to rename in the checkpoint.

patterns_to_instantiate: list[str] | None = None¶: List of patterns to instantiate in the checkpoint.

class noether.core.initializers.ResumeInitializer(initializer_config, **kwargs)¶

Bases: noether.core.initializers.checkpoint.CheckpointInitializer

Initializes models/optimizers from a checkpoint ready for resuming training. Needs to be configured as part of the config by setting the resume_run_id in the root config (assumg the same output path is used). This initializer assumes that the previous run is going to be resumed for further training (i.e., trianing is not finished yet)

For example (config snippet is part of the trainer config):

resume_run_id: <previous_run_id>

Parameters:

initializer_config (ResumeInitializerConfig) – configuration for the initializer. See ResumeInitializerConfig for available options.
**kwargs – additional arguments to pass to the parent class.

init_weights(model)¶

Initialize the model weights from the checkpoint.

Parameters:: model (noether.core.models.ModelBase) – the model to load the weights into.
Return type:: None

init_optimizer(model)¶

Initialize the optimizer for the model.

Parameters:: model (noether.core.models.ModelBase) – a model to initialize the optimizer for.
Return type:: None

start_checkpoint()¶

Get the start checkpoint for the model.

Returns:: the start checkpoint for the model.
Return type:: TrainingIteration

init_trainer(trainer)¶

Initialize the trainer from the checkpoint.

Parameters:: trainer (noether.training.trainers.BaseTrainer) – the trainer to initialize.
Return type:: None

init_callbacks(callbacks, model)¶

Initialize the callbacks from the checkpoint.

Parameters:

callbacks (list[noether.core.callbacks.CallbackBase]) – the callbacks to initialize.
model (noether.core.models.ModelBase) – the model to initialize the callbacks for.

Return type:

None

class noether.core.initializers.ResumeInitializerConfig(/, **data)¶

Bases: noether.core.initializers.checkpoint.CheckpointInitializerConfig

Parameters:: data (Any)

kind: Literal['noether.core.initializers.ResumeInitializer'] = None¶

load_optim: bool = None¶: Whether or not to load the optimizer state from the checkpoint. Default is True, as this is usually used to resume a training run

model_name: str = None¶: The name of the model to load. This is the model_name used in CheckpointCallback.

noether.core.initializers.AnyInitializer¶