noether.core.schemas¶

Submodules¶

Attributes¶

`CallbacksConfig`
`AnyInitializer`
`AnyNormalizer`
`AnyOptimizerConfig`
`AnyScheduleConfig`

Classes¶

`BestCheckpointCallbackConfig`	Internal base class for all registry-based configs.
`BestMetricCallbackConfig`	Internal base class for all registry-based configs.
`CallBackBaseConfig`	Internal base class for all registry-based configs.
`CheckpointCallbackConfig`	Internal base class for all registry-based configs.
`EmaCallbackConfig`	Internal base class for all registry-based configs.
`FixedEarlyStopperConfig`
`MetricEarlyStopperConfig`	Internal base class for all registry-based configs.
`OfflineLossCallbackConfig`	Internal base class for all registry-based configs.
`OnlineLossCallbackConfig`	Internal base class for all registry-based configs.
`TrackAdditionalOutputsCallbackConfig`	Internal base class for all registry-based configs.
`DatasetBaseConfig`	Internal base class for all registry-based configs.
`StandardDatasetConfig`	Base config for datasets with fixed splits.
`CheckpointInitializerConfig`
`InitializerConfig`
`PreviousRunInitializerConfig`
`ResumeInitializerConfig`
`ModelBaseConfig`	Internal base class for all registry-based configs.
`FieldNormalizerConfig`	Declarative normalizer config that references dataset statistics by convention.
`AdamOptimizerConfig`	Configuration for Adam-family optimizers (AdamW, Lion).
`MuonOptimizerConfig`	Configuration for `MuonComposite`.
`OptimizerConfig`	Base configuration for optimizers.
`ParamGroupModifierConfig`	Configuration for a parameter group modifier. Both for the LrScaleByNameModifier and the WeightDecayByNameModifier,
`SGDOptimizerConfig`	Configuration for SGD.
`ConstantScheduleConfig`
`CustomScheduleConfig`
`DecreasingProgressScheduleConfig`
`IncreasingProgressScheduleConfig`
`LinearWarmupCosineDecayScheduleConfig`
`PeriodicBoolScheduleConfig`
`PolynomialDecreasingScheduleConfig`
`PolynomialIncreasingScheduleConfig`
`ProgressScheduleConfig`
`ScheduleBaseConfig`
`SchedulerConfig`
`StepDecreasingScheduleConfig`
`StepFixedScheduleConfig`
`StepIntervalScheduleConfig`
`ConfigSchema`	Root configuration schema for all experiments in Noether.
`SlurmConfig`	Configuration for SLURM job submission via `submitit`.
`WandBTrackerSchema`	Base configuration for experiment trackers. All tracker configs should inherit from this class.
`BaseTrainerConfig`	Internal base class for all registry-based configs.

Package Contents¶

class noether.core.schemas.BestCheckpointCallbackConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['BestCheckpointCallback'] = None¶

metric_key: str = None¶: “The key of the metric to be used for checking the best model.

save_frozen_weights: bool = None¶: Whether to also save the frozen weights of the model.

tolerances: list[int] | None = None¶: “If provided, this callback will produce multiple best models which differ in the amount of intervals they allow the metric to not improve. For example, tolerance=[5] with every_n_epochs=1 will store a checkpoint where at most 5 epochs have passed until the metric improved. Additionally, the best checkpoint over the whole training will always be stored (i.e., tolerance=infinite). When setting different tolerances, one can evaluate different early stopping configurations with one training run.

model_names: list[str] | None = None¶: Which model name to save (e.g., if only the encoder of an autoencoder should be stored, one could pass model_name=’encoder’ here). This only applies when training a CompositeModel. If None, all models are saved.

class noether.core.schemas.BestMetricCallbackConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['BestMetricCallback'] = None¶: The metric to use to dermine whether the current model obtained a new best (e.g., loss/valid/total)

source_metric_key: str = None¶: The metrics to keep track of (e.g., loss/test/total)

target_metric_keys: list[str] | None = None¶: The metrics to keep track of if they are present (useful when different model configurations log different evaluation metrics to avoid reconfiguring the callback).

optional_target_metric_keys: list[str] | None = None¶

class noether.core.schemas.CallBackBaseConfig(/, **data)¶

Bases: noether.core.schemas.lib._RegistryBase

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: str | None = None¶

id: str | None = None¶: Optional unique identifier for this callback instance. Required when multiple stateful callbacks of the same type exist (e.g., two BestCheckpointCallbacks tracking different metrics). Used as the key when saving/loading callback state dicts to ensure correct matching on resume.

every_n_epochs: int | None = None¶: Epoch-based interval. Invokes the callback after every n epochs. Mutually exclusive with other intervals.

every_n_updates: int | None = None¶: Update-based interval. Invokes the callback after every n updates. Mutually exclusive with other intervals.

every_n_samples: int | None = None¶: Sample-based interval. Invokes the callback after every n samples. Mutually exclusive with other intervals.

batch_size: int | None = None¶

None (use the same batch_size as for training).

Type:: Batch size to use for this callback. Default

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

validate_callback_frequency()¶

Ensures that exactly one frequency (‘every_n_*’) is specified and that ‘batch_size’ is present if ‘every_n_samples’ is used.

Return type:: CallBackBaseConfig

classmethod check_positive_values(v)¶

Ensures that all integer-based frequency and batch size fields are positive.

Parameters:: v (int | None)
Return type:: int | None

classmethod check_kind_is_not_empty(v)¶

Ensures the ‘kind’ field is a non-empty string.

Parameters:: v (str)
Return type:: str

noether.core.schemas.CallbacksConfig¶

class noether.core.schemas.CheckpointCallbackConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['CheckpointCallback'] = None¶

save_weights: bool = None¶: Whether to save the weights of the model every time this callback is invoked. The checkpoint name will contain the training iteration (e.g., epoch/update/sample) at which the checkpoint was saved.

save_optim: bool = None¶: Whether to save the optimizer state every time this callback is invoked. The checkpoint name will contain the training iteration (e.g., epoch/update/sample) at which the checkpoint was saved.

save_latest_weights: bool = None¶: Whether to save the latest weights of the model every time this callback is invoked. Note that the latest weights are always overwritten on the next invocation of this callback.

save_latest_optim: bool = None¶: Whether to save the latest optimizer state every time this callback is invoked. Note that the latest optimizer state is always overwritten on the next invocation of this callback

model_names: list[str] | None = None¶: The name of the model to save. If None, all models are saved.

class noether.core.schemas.EmaCallbackConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['EmaCallback'] = None¶

target_factors: list[float] = None¶: The factors for the EMA.

model_paths: list[str | None] | None = None¶: The paths to the models to apply the EMA to (i.e., composite_model.encoder/composite_model.decoder, path of the PyTorch nn.Modules in the checkpoint). If None, the EMA is applied to the whole model. When training with a CompositeModel, the paths on the submodules (i.e., ‘encoder’, ‘decoder’, etc.) should be provided via this field, otherwise the EMA will be applied to the CompositeModel as a whole which is not possible to restore later on.

save_weights: bool = None¶: Whether to save the EMA weights.

save_last_weights: bool = None¶: Save the weights of the model when training is over (i.e., at the end of training, save the EMA weights).

save_latest_weights: bool = None¶: Save the latest EMA weights. Note that the latest weights are always overwritten on the next invocation of this callback.

eval_callbacks: list[Annotated[Any, Discriminated(CallBackBaseConfig)]] | None = None¶: Optional nested periodic callbacks to run against EMA weights. Each child retains its own schedule (every_n_epochs etc.); the EMA callback swaps its stored EMA parameters into the live model around eval-time hooks (after_epoch, after_update, at_eval) and restores the live weights on exit. Children are dispatched once per target_factor and their metric keys are automatically prefixed with ema=<factor>/ to avoid collisions with live-model metrics. Note: before_training and after_training are forwarded without swapping, so EMA initialization and the final save see live weights.

class noether.core.schemas.FixedEarlyStopperConfig(/, **data)¶

Bases: pydantic.BaseModel

Parameters:: data (Any)

kind: str | None = None¶

name: Literal['FixedEarlyStopper'] = None¶

stop_at_sample: int | None = None¶

stop_at_update: int | None = None¶

stop_at_epoch: int | None = None¶

validate_callback_frequency()¶

Ensures that exactly one stop (‘stop_at_*’) is specified

Return type:: FixedEarlyStopperConfig

class noether.core.schemas.MetricEarlyStopperConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['MetricEarlyStopper'] = None¶

metric_key: str¶: The key of the metric to monitor

tolerance: int¶: The number of times the metric can stagnate before stopping training

classmethod check_tolerance_positive(v)¶

Ensures that tolerance is at least 1.

Parameters:: v (int)
Return type:: int

class noether.core.schemas.OfflineLossCallbackConfig(/, **data)¶

Bases: PeriodicDataIteratorCallbackConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['OfflineLossCallback'] = None¶

output_patterns_to_log: list[str] | None = None¶

additional arguments passed to the parent class.

Type:: For instance, if the output key is ‘some_loss’ and the pattern is [‘loss’]. **kwargs

class noether.core.schemas.OnlineLossCallbackConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['OnlineLossCallback'] = None¶

verbose: bool = None¶: Whether to also log to the (console) logger. If False, the loss will only logged to the experiment tracker.

class noether.core.schemas.TrackAdditionalOutputsCallbackConfig(/, **data)¶

Bases: CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['TrackAdditionalOutputsCallback'] = None¶

keys: list[str] | None = None¶: List of keys to track in the additional_outputs of the TrainerResult returned by the trainer’s update step.

patterns: list[str] | None = None¶: List of patterns to track in the additional_outputs of the TrainerResult returned by the trainer’s update step. Matched if it is contained in one of the update_outputs keys.

verbose: bool = None¶: If True uses the logger to print the tracked values otherwise uses no logger.

reduce: Literal['mean', 'last'] = None¶: The reduction method to be applied to the tracked values to reduce to scalar. Currently supports ‘mean’ and ‘last’.

log_output: bool = None¶: Whether to log the tracked scalar values.

save_output: bool = None¶: Whether to save the tracked scalar values to disk.

class noether.core.schemas.DatasetBaseConfig[TPipelineConfig: PipelineConfig](/, **data)¶

Bases: noether.core.schemas.lib._RegistryBase

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: str | None = None¶: Kind of dataset to use.

pipeline: Annotated[TPipelineConfig | None, Discriminated(PipelineConfig)] = None¶: Config of the pipeline to use for the dataset.

dataset_normalizers: dict[str, list[Annotated[Any, Discriminated(NormalizerConfig)]] | Annotated[Any, Discriminated(NormalizerConfig)]] | None = None¶: List of normalizers to apply to the dataset. The key is the data source name.

dataset_wrappers: list[DatasetWrappers] | None = None¶

included_properties: set[str] | None = None¶: Set of properties (i.e., getitem_* methods that are called) of this dataset that will be loaded, if not set all properties are loaded

excluded_properties: set[str] | None = None¶: Set of properties of this dataset that will NOT be loaded, even if they are present in the included list

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class noether.core.schemas.StandardDatasetConfig(/, **data)¶

Bases: DatasetBaseConfig, abc.ABC

Base config for datasets with fixed splits.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

root: str¶: Root directory of the dataset.

split: Literal['train', 'val', 'test']¶: Which split of the dataset to use. Must be one of “train”, “val”, or “test”.

noether.core.schemas.AnyInitializer¶

class noether.core.schemas.CheckpointInitializerConfig(/, **data)¶

Bases: InitializerConfig

Parameters:: data (Any)

kind: Literal['noether.core.initializers.CheckpointInitializer'] = None¶

load_optim: bool = None¶: Whether or not to load the optimizer state from the checkpoint. Default is True, as this is usually used to resume a training run

pop_ckpt_kwargs_keys: list[str] | None = None¶: which checkpoint to load. If a string is provided, must be one of (“latest”, “best_loss”). If a dictionary is provided, must contain keys “epoch”, “update”, “sample” to identify the checkpoint.

class noether.core.schemas.InitializerConfig(/, **data)¶

Bases: pydantic.BaseModel

Parameters:: data (Any)

kind: str = None¶

kwargs: dict[str, Any] | None = None¶: Additional keyword arguments to pass to the initializer.

run_id: str¶: A unique identifier for the training stage. This is used to find the correct checkpoint.

stage_name: str | None = None¶: The name of the stage training stage if defined. When training, the stage name is usually “train”.

model_name: str | None = None¶: The name of the model to load. This is the model_name used in CheckpointCallback.

checkpoint_tag: str | None | dict = None¶: Which checkpoint to load. Checkpoint is usually “latest” or “best_loss”, or “E*_U*_S*”, depending on which checkpoint you want to load.

model_info: str | None = None¶: Optional string to provide additional info about the model weights in the checkpoint filename. E.g., the stored weights are the EMA, or in a different precision.

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class noether.core.schemas.PreviousRunInitializerConfig(/, **data)¶

Bases: CheckpointInitializerConfig

Parameters:: data (Any)

kind: Literal['noether.core.initializers.PreviousRunInitializer'] = None¶

load_optim: bool = None¶: Whether or not to load the optimizer state from the checkpoint. Default is True, as this is usually used to resume a training run

keys_to_remove: list[str] | None = None¶: List of keys to remove from the checkpoint.

patterns_to_remove: list[str] | None = None¶: List of patterns to remove from the checkpoint.

patterns_to_rename: list[dict] | None = None¶: List of patterns to rename in the checkpoint.

patterns_to_instantiate: list[str] | None = None¶: List of patterns to instantiate in the checkpoint.

class noether.core.schemas.ResumeInitializerConfig(/, **data)¶

Bases: CheckpointInitializerConfig

Parameters:: data (Any)

kind: Literal['noether.core.initializers.ResumeInitializer'] = None¶

load_optim: bool = None¶: Whether or not to load the optimizer state from the checkpoint. Default is True, as this is usually used to resume a training run

model_name: str = None¶: The name of the model to load. This is the model_name used in CheckpointCallback.

class noether.core.schemas.ModelBaseConfig(/, **data)¶

Bases: noether.core.schemas.lib._RegistryBase

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: str | None = None¶: Kind of model to use, i.e. class path

name: str¶: Name of the model. Needs to be unique

optimizer_config: noether.core.schemas.optimizers.AnyOptimizerConfig | None = None¶: The optimizer configuration to use for training the model. When a model is used for inference only, this can be left as None.

initializers: list[Annotated[noether.core.schemas.initializers.AnyInitializer, Field(discriminator='kind')]] | None = None¶: List of initializers configs to use for the model.

is_frozen: bool | None = False¶: Whether to freeze the model parameters (i.e., not trainable).

forward_properties: list[str] | None = []¶: List of properties to be used as inputs for the forward pass of the model. Only relevant when the train_step of the BaseTrainer is used. When overridden in a class method, this property is ignored.

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property config_kind: str¶

The fully qualified import path for the configuration class.

Return type:: str

noether.core.schemas.AnyNormalizer¶

class noether.core.schemas.FieldNormalizerConfig(/, **data)¶

Bases: NormalizerConfig

Declarative normalizer config that references dataset statistics by convention.

Instead of embedding numeric values (mean, std, etc.) directly, this config declares how to normalize a field. The actual statistics are resolved at runtime from the dataset’s statistics file.

For "mean_std" normalization, the builder looks up {field}_mean and {field}_std in the dataset statistics (customizable via stat_keys).

For "position" normalization, the builder looks up raw_pos_min and raw_pos_max (customizable via stat_keys).

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: str | None = 'noether.data.preprocessors.normalizers.FieldNormalizer'¶: Kind of normalizer to use, i.e. class path

strategy: Literal['mean_std', 'position'] = 'mean_std'¶: Normalization strategy. "mean_std" for mean/std normalization, "position" for position normalization.

logscale: bool = False¶: If true, the input data is converted to log scale before normalization. Only used for "mean_std".

stat_keys: dict[str, str] | None = None¶

{"mean": "custom_mean_key", "std": "custom_std_key"}. For "position": {"min": "custom_min_key", "max": "custom_max_key"}.

Type:: Optional overrides for statistic key lookup. For "mean_std"

scale: float = None¶: Scaling factor for position normalization. Coordinates are scaled to [0, scale]. Only used for "position".

class noether.core.schemas.AdamOptimizerConfig(/, **data)¶

Bases: OptimizerConfig

Configuration for Adam-family optimizers (AdamW, Lion).

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: Literal['torch.optim.AdamW', 'noether.core.optimizer.Lion'] = 'torch.optim.AdamW'¶: The class path of the torch optimizer to use. E.g., ‘torch.optim.AdamW’.

betas: tuple[float, float] | None = None¶: Beta coefficients for Adam-style optimizers.

noether.core.schemas.AnyOptimizerConfig¶

class noether.core.schemas.MuonOptimizerConfig(/, **data)¶

Bases: OptimizerConfig

Configuration for MuonComposite.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: Literal['noether.core.optimizer.MuonComposite'] = 'noether.core.optimizer.MuonComposite'¶: The class path of the torch optimizer to use. E.g., ‘torch.optim.AdamW’.

momentum: float | None = None¶: Momentum factor for the Muon optimizer.

secondary: MuonSecondaryOptimizerConfig | None = None¶: Configuration of the secondary optimizer in MuonComposite.

nesterov: bool | None = None¶: Enable Nesterov momentum in Muon. None uses Muon’s default (True).

ns_steps: int | None = None¶: Number of Newton-Schulz iteration steps. None uses Muon’s default (5).

adjust_lr_fn: Literal['original', 'match_rms_adamw'] | None = None¶: Per-matrix LR adjustment strategy. None uses Muon’s default ("original").

class noether.core.schemas.OptimizerConfig(/, **data)¶

Bases: pydantic.BaseModel

Base configuration for optimizers.

Holds fields common to all optimizers plus the wrapper-level options. Optimizer-specific fields live on the dedicated subclasses.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

kind: str | None = None¶: The class path of the torch optimizer to use. E.g., ‘torch.optim.AdamW’.

lr: float | None = None¶: The learning rate for the optimizer.

weight_decay: float | None = None¶: The weight decay. Falls back to the primary weight_decay if not set.

clip_grad_value: float | None = None¶: The maximum value for gradient clipping.

clip_grad_norm: float | None = None¶: The maximum norm for gradient clipping.

param_group_modifiers_config: list[ParamGroupModifierConfig] | None = None¶: List of parameter group modifiers to apply. These can modify the learning rate or weight decay for specific parameters.

exclude_bias_from_weight_decay: bool = True¶: If true, excludes the bias parameters (i.e., parameters that end with ‘.bias’) from the weight decay. Default true.

exclude_normalization_params_from_weight_decay: bool = True¶: If true, excludes the weights of normalization layers from the weight decay. This is implemented by excluding all 1D tensors from the weight decay. Default true.

weight_decay_schedule: noether.core.schemas.schedules.AnyScheduleConfig | None = None¶

schedule_config: noether.core.schemas.schedules.AnyScheduleConfig | None = None¶

return_optim_wrapper_args()¶

Return type:: dict

class noether.core.schemas.ParamGroupModifierConfig(/, **data)¶

Bases: pydantic.BaseModel

Configuration for a parameter group modifier. Both for the LrScaleByNameModifier and the WeightDecayByNameModifier,

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: str | None = None¶: The class path of the parameter group modifier. Either noether.core.optimizer.param_group_modifiers.LrScaleByNameModifier or noether.core.optimizer.param_group_modifiers.WeightDecayByNameModifier.

scale: float | None = None¶: The scaling factor for the learning rate. Must be greater than 0.0. Only for the LrScaleByNameModifier.

value: float | None = None¶: The weight decay value. With 0.0 the parameter is excluded from the weight decay. Only for the WeightDecayByNameModifier.

name: str¶: The name of the parameter within the model. E.g., ‘backbone.cls_token’.

check_scale_or_value_exclusive()¶

Validates that either ‘scale’ or ‘value’ is provided, but not both. This is a model-level validator that runs after individual field validation.

Return type:: Self

class noether.core.schemas.SGDOptimizerConfig(/, **data)¶

Bases: OptimizerConfig

Configuration for SGD.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: Literal['torch.optim.SGD'] = 'torch.optim.SGD'¶: The class path of the torch optimizer to use. E.g., ‘torch.optim.AdamW’.

momentum: float | None = None¶: Momentum factor.

noether.core.schemas.AnyScheduleConfig¶

class noether.core.schemas.ConstantScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.ConstantSchedule'] = 'noether.core.schedules.ConstantSchedule'¶: The fully qualified class name of the scheduler.

value: float¶: The constant value that will be returned for all steps. Value should be equal to the learning rate defined in the optimizer.

class noether.core.schemas.CustomScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.CustomSchedule'] = 'noether.core.schedules.CustomSchedule'¶: The fully qualified class name of the scheduler.

values: list[float]¶: The list of values that will be returned for each step. Values show ben as long as the number of steps.

class noether.core.schemas.DecreasingProgressScheduleConfig(/, **data)¶

Bases: ProgressScheduleConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.DecreasingProgressSchedule'] = 'noether.core.schedules.DecreasingProgressSchedule'¶: The fully qualified class name of the scheduler.

max_value: float = None¶: Maximum (starting) value of the schedule.

end_value: float = None¶: Minimum (ending) value of the schedule.

class noether.core.schemas.IncreasingProgressScheduleConfig(/, **data)¶

Bases: ProgressScheduleConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.IncreasingProgressSchedule'] = 'noether.core.schedules.IncreasingProgressSchedule'¶: The fully qualified class name of the scheduler.

start_value: float = None¶

max_value: float | None = None¶: Minimum (starting) value of the schedule.

class noether.core.schemas.LinearWarmupCosineDecayScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

max_value: float = None¶: The maximum value of the scheduler from which to start the cosine decay phase. This should be equal to the learning rate defined in the optimizer. I.e., max value is learning rate

kind: Literal['noether.core.schedules.LinearWarmupCosineDecaySchedule'] = 'noether.core.schedules.LinearWarmupCosineDecaySchedule'¶: The fully qualified class name of the scheduler.

warmup_steps: int | None = None¶: The number of steps to linearly increase the value from start to max.

warmup_percent: float | None = None¶: The percentage of steps to linearly increase the value from start to max.

validate_warmup()¶

Ensures that exactly one of ‘warmup_steps’ or ‘warmup_percent’ is specified.

Return type:: LinearWarmupCosineDecayScheduleConfig

class noether.core.schemas.PeriodicBoolScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.PeriodicBoolSchedule'] = 'noether.core.schedules.PeriodicBoolSchedule'¶: The fully qualified class name of the scheduler.

initial_state: bool¶: The initial (boolean) state of the scheduler (on or off).

off_value: float = None¶: The value to return when the scheduler is in the off state.

on_value: float = None¶: The value to return when the scheduler is in the on state.

off_duration: int = None¶: The number of steps the scheduler is in the off state.

on_duration: int = None¶: The number of steps the scheduler is in the on state.

invert: bool = None¶: Whether to invert the scheduler, i.e. return off_value when on and vice versa.

class noether.core.schemas.PolynomialDecreasingScheduleConfig(/, **data)¶

Bases: DecreasingProgressScheduleConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.PolynomialDecreasingSchedule'] = 'noether.core.schedules.PolynomialDecreasingSchedule'¶: The fully qualified class name of the scheduler.

power: float = None¶: The power of the polynomial function.

class noether.core.schemas.PolynomialIncreasingScheduleConfig(/, **data)¶

Bases: IncreasingProgressScheduleConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.PolynomialIncreasingSchedule'] = 'noether.core.schedules.PolynomialIncreasingSchedule'¶: The fully qualified class name of the scheduler.

power: float = None¶: The power of the polynomial function.

class noether.core.schemas.ProgressScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.ProgressSchedule'] = 'noether.core.schedules.ProgressSchedule'¶: The fully qualified class name of the scheduler.

exclude_first: bool = None¶: Whether to exclude the first value of the schedule.

exclude_last: bool = None¶: Whether to exclude the last value of the schedule.

class noether.core.schemas.ScheduleBaseConfig(/, **data)¶

Bases: pydantic.BaseModel

Parameters:: data (Any)

kind: str | None = None¶: The fully qualified class name of the scheduler.

overhang_percent: float | None = None¶: The percentage by which the schedule is artificially prolonged. Mutually exclusive with overhang_steps.

overhang_steps: int | None = None¶: The number of steps by which the schedule is artificially prolonged. Mutually exclusive with overhang_percent.

start_value: float = None¶

end_value: float = None¶

weight_decay: float | None = None¶

start_percent: float | None = None¶: The percentage of steps at which the schedule starts.

end_percent: float | None = None¶: The percentage of steps at which the schedule ends.

start_step: int | None = None¶: The step at which the schedule starts.

end_step: int | None = None¶: The step at which the schedule ends.

interval: Literal['update', 'epoch'] = None¶: Whether the schedule is based on updates or epochs. Interval should be either “update” or “epoch”. Default is “update”. Under the hood steps is always used. However, when “epoch” is selected here, the step count is derived from epochs via the UpdateCounter.

check_mutual_exclusion()¶

Ensures that ‘overhang_percent’ and ‘overhang_steps’ are mutually exclusive.

Return type:: ScheduleBaseConfig

validate_start_end_steps()¶

Return type:: ScheduleBaseConfig

validate_start_end_percents()¶

Return type:: ScheduleBaseConfig

class noether.core.schemas.SchedulerConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.scheduler.SchedulerConfig'] = 'noether.core.schedules.scheduler.SchedulerConfig'¶: The fully qualified class name of the scheduler.

warmup_percent: float = None¶

end_value: float = None¶

class noether.core.schemas.StepDecreasingScheduleConfig(/, **data)¶

Bases: DecreasingProgressScheduleConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.StepDecreasingSchedule'] = 'noether.core.schedules.StepDecreasingSchedule'¶: The fully qualified class name of the scheduler.

factor: float = None¶: The factor by which the value decreases.

decreases_interval: float = None¶: The interval in range [0, 1] at which the value decreases.

check_interval()¶

Ensures that ‘interval’ is a float in the range (0, 1).

Return type:: StepDecreasingScheduleConfig

class noether.core.schemas.StepFixedScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.StepFixedSchedule'] = 'noether.core.schedules.StepFixedSchedule'¶: The fully qualified class name of the scheduler.

start_value: float = None¶: The initial value of the scheduler.

factor: float = None¶: The factor by which the value is multiplied after reaching the next step provided in steps.

steps: list[float] = None¶: The steps at which the value changes, must be a list of floats in the range (0, 1).

validate_steps()¶

Ensures that ‘steps’ is a non-empty list of floats in the range (0, 1).

Return type:: StepFixedScheduleConfig

class noether.core.schemas.StepIntervalScheduleConfig(/, **data)¶

Bases: ScheduleBaseConfig

Parameters:: data (Any)

kind: Literal['noether.core.schedules.StepIntervalSchedule'] = 'noether.core.schedules.StepIntervalSchedule'¶: The fully qualified class name of the scheduler.

start_value: float = None¶: The initial value of the scheduler. I.e, the learning rate at step 0.

factor: float = None¶: The factor by which the value is multiplied after reaching the next interval.

update_interval: float = None¶: The interval in range (0, 1) at which the value changes.

check_update_interval(v)¶

Ensures that ‘update_interval’ is a float in the range (0, 1).

Parameters:: v (float)
Return type:: float

class noether.core.schemas.ConfigSchema[TModelConfig: noether.core.schemas.models.ModelBaseConfig, TDatasetConfig: noether.core.schemas.dataset.DatasetBaseConfig, TTrainerConfig: noether.core.schemas.trainers.BaseTrainerConfig](/, **data)¶

Bases: pydantic.BaseModel

Root configuration schema for all experiments in Noether.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: str | None = None¶: Name of the experiment.

accelerator: ACCELERATOR_TYPES = None¶: Type of accelerator to use. By default the system choose the best available accelerator. GPU > MPS > CPU.

stage_name: str | None = None¶: Name of the current stage. I.e., train, finetune, test, etc. When None, the run_id directory is used as output directory. Otherwise, run_id/stage_name is used.

dataset_kind: str | None = None¶: Kind of dataset to use i.e., class path.

dataset_root: str | None = None¶: Root directory of the dataset.

resume_run_id: str | None = None¶: Run ID to resume from. If None, start a new run. This can be used to resume training from the last checkpoint of a previous run when training was interrupted/failed.

resume_stage_name: str | None = None¶: Stage name to resume from. If None, resume from the default stage.

resume_checkpoint: str | None = None¶: Path to checkpoint to resume from. If None, the ‘latest’ checkpoint will be used.

seed: int = None¶: Random seed for reproducibility.

dataset_statistics: dict[str, list[float | int]] | None = None¶: Pre-computed dataset statistics, e.g., mean and std for normalization. Since some tensors are multi-dimensional, the statistics are stored as lists.

tracker: Annotated[noether.core.schemas.trackers.BaseTrackerConfig, Discriminated(BaseTrackerConfig)] | None = None¶: Configuration for experiment tracking. If None, no tracking is used. If “disabled”, tracking is explicitly disabled. WandB is currently the only supported tracker.

run_id: str | None = None¶: Unique identifier for the run. When running under SLURM and not set explicitly, defaults to the SLURM job ID (e.g. 12345_2 for array task 2 of job 12345). Otherwise a new ID is generated at runtime.

devices: str | None = None¶: Comma-separated list of device IDs to use. If None, all available devices will be used.

num_workers: int | None = None¶: Number of worker threads for data loading. If None, will use (#CPUs / #GPUs - 1) workers

cudnn_benchmark: bool = True¶: Whether to enable cudnn benchmark mode for this run.

cudnn_deterministic: bool = False¶: Whether to enable cudnn deterministic mode for this run.

datasets: dict[str, Annotated[TDatasetConfig, Discriminated(DatasetBaseConfig)]] = None¶: Configuration for datasets. The key is the dataset and value is the configuration for that dataset. See DatasetBaseConfig for available options. The key “train” is reserved for the training dataset, but if not provided, the first dataset will be used as training dataset by default, other keys are arbitrary and can be used to identify datasets for different stages, e.g., “train”, “val”, “test”, etc. or different datasets for the same stage, e.g., “train_cfd”, “train_wind_turbine”, etc.

model: Annotated[TModelConfig, Discriminated(ModelBaseConfig)] = None¶: Configuration for the model. See ModelBaseConfig for available options.

trainer: Annotated[TTrainerConfig, Discriminated(BaseTrainerConfig)] = None¶: Configuration for the trainer. See BaseTrainerConfig for available options.

debug: bool = False¶: If True, enables debug mode with more verbose logging, no WandB logging and output written to debug directory.

store_code_in_output: bool = True¶: If True, store a copy of the current code in the output directory for reproducibility.

output_path: pathlib.Path | None = None¶: Path to output directory. When omitted, defaults to slurm.folder if a slurm section is present. Raises a validation error when neither output_path nor slurm is provided.

master_port: int = None¶: Port for distributed master node. If None, will be set from environment variable MASTER_PORT if available.

slurm: noether.core.schemas.slurm.SlurmConfig | None = None¶: Configuration for SLURM job submission.

classmethod empty_dict_is_none(v)¶

Pre-processes tracker input before validation.

Parameters:: v (Any)
Return type:: Any

serialize_output_path(value)¶

Parameters:: value (Any)
Return type:: str | None

classmethod get_env_master_port(value)¶

Sets master_port from environment variable if available.

Parameters:: value (Any)
Return type:: Any

property config_schema_kind: str¶

The fully qualified import path for the configuration class.

Return type:: str

class noether.core.schemas.SlurmConfig(/, **data)¶

Bases: pydantic.BaseModel

Configuration for SLURM job submission via submitit.

Field names mirror the keyword arguments accepted by submitit.AutoExecutor.update_parameters(). All fields are optional and default to None, meaning the cluster default is used.

Note

Job stdout/stderr is owned by submitit and written to <folder>/<job_id>_log.out / <folder>/<job_id>_log.err. Use the folder field to control where these files land. SLURM --output/--error directives are intentionally not exposed; pass them via slurm_additional_parameters if you really need to override submitit’s defaults (this disables job.stdout() helpers).

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

folder: str = 'submitit_logs'¶

Directory where submitit writes the job script, pickled task, and stdout/stderr logs. Per-job files are named <job_id>_log.out etc. inside this directory. This is also used as the default output_path for training runs (see ConfigSchema.output_path).

Supports %u (current username) interpolation, e.g. /home/%u/logs/experiment. SLURM job-time patterns like %j are not supported because submitit needs the directory to exist before submission.

name: str | None = None¶: Job name (SLURM --job-name).

nodes: int | None = None¶: Number of nodes to allocate.

tasks_per_node: int | None = None¶: Number of tasks per allocated node.

cpus_per_task: int | None = None¶: Number of CPUs per task.

gpus_per_node: int | str | None = None¶: GPUs per node. Accepts a count or type:count (e.g. "a100:4").

mem_gb: float | None = None¶: Memory per node in gigabytes.

timeout_min: int = 0¶: Wall-clock limit in minutes. Use 0 for no time limit

stderr_to_stdout: bool | None = None¶: If True, merge stderr into stdout.

slurm_partition: str | None = None¶: Partition to submit the job to.

slurm_array_parallelism: int | None = None¶: Maximum number of array tasks running concurrently (SLURM %N in --array).

slurm_setup: list[str] | None = None¶: Shell commands run inside the job before the main command, e.g. ["source .venv/bin/activate"].

slurm_additional_parameters: dict[str, Any] | None = None¶: Escape hatch for SLURM directives not exposed as first-class fields, e.g. {"nice": 0, "reservation": "my_res", "chdir": "/work"}. Keys are passed as --key=value to sbatch.

to_executor_kwargs()¶

Return (folder, update_parameters_kwargs) for submitit.AutoExecutor.

Generic fields are passed under their bare name; everything else keeps its slurm_ prefix so submitit routes it to the slurm executor.

Returns:: A tuple (folder, kwargs) where folder is the executor’s log directory and kwargs is the dict to splat into executor.update_parameters(**kwargs).
Return type:: tuple[str, dict[str, Any]]

class noether.core.schemas.WandBTrackerSchema(/, **data)¶

Bases: BaseTrackerConfig

Base configuration for experiment trackers. All tracker configs should inherit from this class.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

entity: str | None = None¶: The entity name for the W&B project.

project: str | None = None¶: The project name for the W&B project.

mode: Literal['disabled', 'online', 'offline'] | None = None¶: Tracking mode. Can be ‘disabled’, ‘online’, or ‘offline’.

class noether.core.schemas.BaseTrainerConfig[TCallbackConfig: noether.core.schemas.callbacks.CallBackBaseConfig](/, **data)¶

Bases: noether.core.schemas.lib._RegistryBase

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

kind: str¶

max_epochs: int | None = None¶: The maximum number of epochs to train for. Mutually exclusive with max_updates and max_samples. If set to 0, training will be skipped and all callbacks will be invoked once (useful for evaluation-only runs).

max_updates: int | None = None¶: The maximum number of updates to train for. Mutually exclusive with max_epochs and max_samples. If set to 0, training will be skipped and all callbacks will be invoked once (useful for evaluation-only runs).

max_samples: int | None = None¶: The maximum number of samples to train for. Mutually exclusive with max_epochs and max_updates. If set to 0, training will be skipped and all callbacks will be invoked once (useful for evaluation-only runs).

start_at_epoch: int | None = None¶: The epoch to start training at. This means that the trainer will skip all epochs before this epoch. Learning rate and other schedulers will be stepped accordingly. Useful for resuming training from a specific epoch.

add_default_callbacks: bool | None = None¶: Whether to add default callbacks. Default callbacks log things like simple dataset statistics or the current value of the learning rate if it is scheduled.

add_trainer_callbacks: bool | None = None¶: Whether to add trainer specific callbacks (e.g., a callback to log the training accuracy for a classification task).

effective_batch_size: int = None¶

the “global batch size”. In multi-GPU setups, the batch size per device, (“local batch size”) is effective_batch_size / number of devices. If gradient accumulation is used, the forward-pass batch size is derived by dividing by the number of gradient accumulation steps.

Type:: The effective batch size used for optimization. This is the number of samples that are processed before an update step is taken

precision: Literal['float32', 'fp32', 'float16', 'fp16', 'bfloat16', 'bf16'] = None¶: The precision to use for training (e.g., “float32”). Mixed precision training (e.g., “float16” or “bfloat16”) can be used to speed up training and reduce memory usage on supported hardware (e.g., NVIDIA GPUs).

callbacks: list[Annotated[TCallbackConfig, Discriminated(CallBackBaseConfig)]] | None = None¶: The callbacks to use for training.

initializer: noether.core.schemas.initializers.InitializerConfig | None = None¶: The initializer to use for training. Mainly used for resuming training via ResumeInitializer.

log_every_n_epochs: int | None = None¶: The integer number of epochs to periodically log at.

log_every_n_updates: int | None = None¶: The integer number of updates to periodically log at.

log_every_n_samples: int | None = None¶: The integer number of samples to periodically log at.

track_every_n_epochs: int | None = None¶: The integer number of epochs to periodically track metrics at.

track_every_n_updates: int | None = None¶: The integer number of updates to periodically track metrics at.

track_every_n_samples: int | None = None¶: The integer number of samples to periodically track metrics at.

max_batch_size: int | None = None¶: The maximum batch size to use for model forward pass in training. If the effective_batch_size is larger than max_batch_size, gradient accumulation will be used to simulate the larger batch size. For example, if effective_batch_size=8 and max_batch_size=2, 4 gradient accumulation steps will be taken before each optimizer step.

skip_nan_loss: bool = None¶: Whether to skip NaN losses. These can sometimes occur due to unlucky coincidences. If true, NaN losses will be skipped without terminating the training up until 100 NaN losses occurred in a row.

skip_nan_loss_max_count: int = None¶

disable_gradient_accumulation: bool = None¶: Whether to disable gradient accumulation. Gradient accumulation is sometimes used to simulate larger batch sizes, but can lead to worse generalization.

save_on_sigint: bool = None¶: Whether to save a checkpoint on SIGINT (Ctrl+C). SIGTERM always triggers a checkpoint save. When False (default), Ctrl+C will stop training immediately without saving.

use_torch_compile: bool = None¶: Whether to use torch.compile to compile the model for faster training.

find_unused_params: bool = None¶: Sets the find_unused_parameters flag of DistributedDataParallel.

static_graph: bool = None¶: Sets the static_graph flag of DistributedDataParallel.

forward_properties: list[str] | None = []¶: Properties (i.e., keys from the batch dict) from the input batch that are used as inputs to the model during the forward pass.

target_properties: list[str] | None = []¶: Properties (i.e., keys from the batch dict) from the input batch that are used as targets for the model during the forward pass.

model_config¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

dataloader_prefetch_factor: int | None = None¶: The prefetch_factor to use for the training dataloader. This controls how many batches are prefetched by each worker process in the dataloader. Increasing this can speed up training if data loading is a bottleneck, but also increases memory usage.

validate_callback_frequency()¶

Ensures that exactly one frequency (‘every_n_*’) is specified and that ‘batch_size’ is present if ‘every_n_samples’ is used.

Return type:: BaseTrainerConfig

validate_max_training_criteria()¶

Ensures that exactly one of max_epochs, max_updates, or max_samples is specified.

Return type:: BaseTrainerConfig