noether.core.utils.logging

Submodules

• noether.core.utils.logging.base
• noether.core.utils.logging.formatting
• noether.core.utils.logging.progress

Classes

CustomFormatter	Formatter instances are used to convert a LogRecord to text.
MessageCounter	A logging handler that counts the number of messages logged at each level starting from WARNING.
NoopTqdm	A no-operation (noop) version of tqdm that does not display a progress bar.

Functions

add_global_handlers([log_file_uri, debug])	Set up logging.getLogger() to log to stdout and optionally to a file.
log_from_all_ranks()	ContextManager that changes the log level to Info for all ranks and resets it to CRITICAL for non-rank0 ranks.
dict_to_string(obj[, item_seperator])	Convert a dictionary to a string representation.
float_to_scientific_notation(value, max_precision[, ...])	Convert a float to scientific notation with a specified precision.
seconds_to_duration_str(total_seconds)	Convert a number of seconds to a duration string.
short_number_str(number[, precision])	Convert a number to a short string with SI prefix, using rounding.
summarize_indices_list(indices)	Summarize a list of indices into ranges.
tensor_like_to_string(tensor_or_list)	Convert a list or a tensor to a string representation.

Package Contents

class noether.core.utils.logging.CustomFormatter(fmt, datefmt=None, colors=True)

Bases: logging.Formatter

Formatter instances are used to convert a LogRecord to text.

Formatters need to know how a LogRecord is constructed. They are responsible for converting a LogRecord to (usually) a string which can be interpreted by either a human or an external system. The base Formatter allows a formatting string to be specified. If none is supplied, the style-dependent default value, “%(message)s”, “{message}”, or “${message}”, is used.

The Formatter can be initialized with a format string which makes use of knowledge of the LogRecord attributes - e.g. the default value mentioned above makes use of the fact that the user’s message and arguments are pre- formatted into a LogRecord’s message attribute. Currently, the useful attributes in a LogRecord are described by:

%(name)s Name of the logger (logging channel) %(levelno)s Numeric logging level for the message (DEBUG, INFO,

WARNING, ERROR, CRITICAL)

%(levelname)s Text logging level for the message (“DEBUG”, “INFO”,

“WARNING”, “ERROR”, “CRITICAL”)

%(pathname)s Full pathname of the source file where the logging

call was issued (if available)

%(filename)s Filename portion of pathname %(module)s Module (name portion of filename) %(lineno)d Source line number where the logging call was issued

(if available)

%(funcName)s Function name %(created)f Time when the LogRecord was created (time.time()

return value)

%(asctime)s Textual time when the LogRecord was created %(msecs)d Millisecond portion of the creation time %(relativeCreated)d Time in milliseconds when the LogRecord was created,

relative to the time the logging module was loaded (typically at application startup time)

%(thread)d Thread ID (if available) %(threadName)s Thread name (if available) %(taskName)s Task name (if available) %(process)d Process ID (if available) %(message)s The result of record.getMessage(), computed just as

the record is emitted

Initialize the formatter with specified format strings.

Initialize the formatter either with the specified format string, or a default as described above. Allow for specialized date formatting with the optional datefmt argument. If datefmt is omitted, you get an ISO8601-like (or RFC 3339-like) format.

Use a style parameter of ‘%’, ‘{’ or ‘$’ to specify that you want to use one of %-formatting, str.format() ({}) formatting or string.Template formatting in your format string.

Changed in version 3.2: Added the style parameter.

Parameters:

• fmt (str)

• datefmt (str | None)

• colors (bool)

GREY = '\x1b[38;20m'

YELLOW = '\x1b[33;20m'

RED = '\x1b[31;20m'

BOLD_RED = '\x1b[31;1m'

RESET = '\x1b[0m'

fmt

FORMATS

datefmt = None

colors = True

format(record)

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

class noether.core.utils.logging.MessageCounter

Bases: logging.Handler

A logging handler that counts the number of messages logged at each level starting from WARNING.

Initializes the instance - basically setting the formatter to None and the filter list to empty.

min_level = 30

counts: dict[int, int]

emit(record)

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

Parameters:

record (logging.LogRecord)

Return type:

None

log()

Return type:

None

noether.core.utils.logging.add_global_handlers(log_file_uri=None, debug=False)

Set up logging.getLogger() to log to stdout and optionally to a file.

Sets up logging for distributed runs: only rank0 logs to console and file, other ranks log only CRITICAL messages to suppress output. This also adds a MessageCounter handler to count the number of messages logged at each level.

Parameters:

• log_file_uri (pathlib.Path | None) – The path to the log file. If None, no file logging is done.

• debug (bool) – Whether to log debug messages to stdout.

Returns:

The MessageCounter handler.

Return type:

MessageCounter

noether.core.utils.logging.log_from_all_ranks()

ContextManager that changes the log level to Info for all ranks and resets it to CRITICAL for non-rank0 ranks.

Return type:

collections.abc.Generator[None, None, None]

noether.core.utils.logging.dict_to_string(obj, item_seperator='-')

Convert a dictionary to a string representation.

Example: {epoch: 5, batch_size: 64} -> epoch=5-batchsize=64

Parameters:

• obj (dict[str, Any]) – The dictionary to convert.

• item_seperator (str) – The separator to use between items.

Returns:

The string representation of the dictionary.

Return type:

str

noether.core.utils.logging.float_to_scientific_notation(value, max_precision, remove_plus=True)

Convert a float to scientific notation with a specified precision.

Example: value = 0.0000032, max_precision = 10 -> “3.2e-6”

Parameters:

• value (float) – The float to convert.

• max_precision (int) – The maximum number of decimal places to include.

• remove_plus (bool) – Whether to remove the ‘+’ sign from the exponent.

Returns:

The scientific notation string representation of the float.

Return type:

str

noether.core.utils.logging.seconds_to_duration_str(total_seconds)

Convert a number of seconds to a duration string.

Example: 60 * 60 * 24 * 14 + 1234 -> “14-10:20:34.00”

Parameters:

total_seconds (float | int) – The number of seconds to convert.

Returns:

The string representation of the duration.

Return type:

str

noether.core.utils.logging.short_number_str(number, precision=1)

Convert a number to a short string with SI prefix, using rounding.

Example: 1234567 -> 1.2M Example: 123 -> 123 Example: 1234 -> 1.2K Example: 0.1234 -> 123.4m Example: 0.000123 -> 123.0µ

Parameters:

• number (float) – The number to convert.

• precision (int) – The number of decimal places to include.

Returns:

The short string representation of the number.

Return type:

str

noether.core.utils.logging.summarize_indices_list(indices)

Summarize a list of indices into ranges.

Example: [0, 1, 2, 3, 6, 7, 8] -> [“0-3”, “6-8”]

Parameters:

indices (list[int]) – The list of indices to summarize.

Returns:

A list of strings representing the summarized indices.

Return type:

list[str]

noether.core.utils.logging.tensor_like_to_string(tensor_or_list)

Convert a list or a tensor to a string representation.

Example: [1, 2, 3] -> “1, 2, 3” Example: [1.1, 2.2, 3.3] -> “1.10, 2.20, 3.30”

Parameters:

• tensor – Tensor or list to be converted to a string.

• tensor_or_list (torch.Tensor | list[Any])

Returns:

The string representation of the tensor or list.

Return type:

str

class noether.core.utils.logging.NoopTqdm(iterable)

A no-operation (noop) version of tqdm that does not display a progress bar.

Parameters:

iterable (collections.abc.Iterable[Any])

iterable

noop(*_, **__)

Parameters:

• _ (Any)

• __ (Any)

Return type:

None