noether.modeling.modules.layers.continuous_sincos_embed

Classes

ContinuousSincosEmbeddingConfig	Configuration for Continuous Sine-Cosine Embedding layer.
ContinuousSincosEmbed	Embedding layer for continuous coordinates using sine and cosine functions.

Module Contents

class noether.modeling.modules.layers.continuous_sincos_embed.ContinuousSincosEmbeddingConfig(/, **data)

Bases: pydantic.BaseModel

Configuration for Continuous Sine-Cosine Embedding layer.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

data (Any)

hidden_dim: int = None

Dimensionality of the output embedding.

input_dim: int = None

Dimensionality of the input coordinates.

mode: Literal['wavelength', 'nerf'] = None

Frequency schedule.

• "wavelength" (default): transformer-style geometric wavelengths from 1 to max_wavelength. Suitable for integer / unnormalized coordinates.

• "nerf": NeRF-style log-spaced frequencies from π to π * max_frequency. Suitable for coordinates normalized to [-1, 1]. The L available bands are distributed evenly in log-frequency across this range.

max_wavelength: int = None

Maximum wavelength. Only used when mode == "wavelength".

max_frequency: float | None = None

Highest frequency band for NeRF mode, in units of π. The L frequencies are log-spaced between π (wavelength 2, spans the [-1, 1] domain) and π * max_frequency (wavelength 2 / max_frequency). Required when mode == "nerf"; pick based on the smallest spatial scale you need to resolve in normalized coordinates (rough heuristic: 1 / typical_point_spacing).

class noether.modeling.modules.layers.continuous_sincos_embed.ContinuousSincosEmbed(config)

Bases: torch.nn.Module

Embedding layer for continuous coordinates using sine and cosine functions. The original implementation from the Attenion is All You Need paper, deals with descrete 1D cordinates (i.e., a sequence). Howerver, this implementation is able to deal with 2D and 3D coordinate systems as well.

Two frequency schedules are supported via config.mode:

• "wavelength" (default): geometric wavelengths from 1 to max_wavelength, matching the original Transformer encoding. Use this for integer / unnormalized coordinates.

• "nerf": log-spaced frequencies from π to π * max_frequency. Use this for coordinates normalized to [-1, 1].

Parameters:

config (ContinuousSincosEmbeddingConfig) – Configuration for the ContinuousSincosEmbed module. See ContinuousSincosEmbeddingConfig for the available options.

omega: torch.Tensor

padding_tensor: torch.Tensor

hidden_dim

input_dim

ndim_padding

sincos_padding

mode

max_wavelength

max_frequency

padding

forward(coords)

Forward method of the ContinuousSincosEmbed layer.

Parameters:

coords (torch.Tensor) – Tensor of coordinates. The shape of the tensor should be [batch size, number of points, coordinate dimension] or [number of points, coordinate dimension].

Raises:

NotImplementedError – Only supports sparse (i.e. [number of points, coordinate dimension]) or dense (i.e. [batch size, number of points, coordinate dimension]) coordinates systems.

Returns:

Tensor with embedded coordinates.

Return type:

torch.Tensor