noether.core.callbacks.online.best_metric¶

Classes¶

`BestMetricCallbackConfig`	Internal base class for all registry-based configs.
`BestMetricCallback`	A callback that keeps track of the best metric value over a training run for a certain metric (i.e., source_metric_key) while also logging one or more target metrics.

Module Contents¶

class noether.core.callbacks.online.best_metric.BestMetricCallbackConfig(/, **data)¶

Bases: noether.core.callbacks.base.CallBackBaseConfig

Internal base class for all registry-based configs.

Provides auto-registration via __init_subclass__. Not meant to be used directly - use specific config base classes instead.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

name: Literal['BestMetricCallback'] = None¶: The metric to use to dermine whether the current model obtained a new best (e.g., loss/valid/total)

source_metric_key: str = None¶: The metrics to keep track of (e.g., loss/test/total)

target_metric_keys: list[str] | None = None¶: The metrics to keep track of if they are present (useful when different model configurations log different evaluation metrics to avoid reconfiguring the callback).

optional_target_metric_keys: list[str] | None = None¶

class noether.core.callbacks.online.best_metric.BestMetricCallback(callback_config, **kwargs)¶

Bases: noether.core.callbacks.periodic.PeriodicCallback

A callback that keeps track of the best metric value over a training run for a certain metric (i.e., source_metric_key) while also logging one or more target metrics.

For example, track the test loss the epoch with the best validation loss to simulate early stopping.

Example config:

- kind: noether.core.callbacks.BestMetricCallback
  every_n_epochs: 1
  source_metric_key: loss/val/total
  target_metric_keys:
    -  loss/test/total

In this example, whenever a new best validation loss is found, the corresponding test loss is logged under the key loss/test/total/at_best/loss/val/total.

Parameters:

callback_config (BestMetricCallbackConfig) – Configuration for the callback. See BestMetricCallbackConfig for available options including source and target metric keys.
**kwargs – Additional keyword arguments provided to the parent class.

source_metric_key¶

target_metric_keys¶

optional_target_metric_keys¶

higher_is_better¶

best_metric_value¶

previous_log_values: dict[str, Any]¶

periodic_callback(**__)¶

Hook called periodically based on the configured intervals.

This method is the primary entry point for periodic actions in subclasses. It is triggered when any of the configured intervals (every_n_epochs, every_n_updates, or every_n_samples) are reached.

Subclasses should override this method to implement periodic logic such as:

Calculating and logging expensive validation metrics
Saving specific model checkpoints or artifacts
Visualizing training progress (e.g., plotting samples)
Adjusting training hyperparameters or model state

Note

This method is executed within a torch.no_grad() context.

Parameters:

interval_type – “epoch”, “update”, “sample” or “eval” indicating which interval triggered this callback.
update_counter – UpdateCounter instance providing details about the current training progress (epoch, update, sample counts).
**kwargs – Additional keyword arguments passed from the triggering hook (e.g., from after_epoch() or after_update()).

Return type:

None