noether.modeling.modules.attention.anchor_attention.multi_branch

Exceptions

MissingBranchTokensError

Raised when expected tokens for a configured branch are not present.

MissingAnchorTokenError

Raised when a required anchor token is not present.

UnexpectedTokenError

Raised when an unexpected token is present.

Classes

MultiBranchAnchorAttention

A base class for multi-branch anchor-based attention modules with shared parameters between branches.

Module Contents

exception noether.modeling.modules.attention.anchor_attention.multi_branch.MissingBranchTokensError

Bases: ValueError

Raised when expected tokens for a configured branch are not present.

Initialize self. See help(type(self)) for accurate signature.

exception noether.modeling.modules.attention.anchor_attention.multi_branch.MissingAnchorTokenError

Bases: ValueError

Raised when a required anchor token is not present.

Initialize self. See help(type(self)) for accurate signature.

exception noether.modeling.modules.attention.anchor_attention.multi_branch.UnexpectedTokenError

Bases: ValueError

Raised when an unexpected token is present.

Initialize self. See help(type(self)) for accurate signature.

class noether.modeling.modules.attention.anchor_attention.multi_branch.MultiBranchAnchorAttention(config)

Bases: torch.nn.Module

A base class for multi-branch anchor-based attention modules with shared parameters between branches.

Anchor attention limits the self-attention to anchor tokens while other tokens use cross-attention. Multiple branches for different modalities use the same linear-projection parameters. This base class provides a common constructor, validation logic, and forward method implementation. Subclasses only need to implement _create_attention_patterns to define their specific attention patterns.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

config (noether.core.schemas.modules.attention.AttentionConfig)

mixed_attention
branches = None
anchor_suffix = None
forward(x, token_specs, freqs=None)

Apply attention using the patterns defined by the subclass.

Parameters:

  • x (torch.Tensor)

  • token_specs (collections.abc.Sequence[noether.core.schemas.modules.attention.TokenSpec])

  • freqs (torch.Tensor | None)

Return type:

torch.Tensor