noether.core.schedules.linear_warmup_cosine_decay¶

Classes¶

LinearWarmupCosineDecaySchedule

A cosine annealing scheduler with linear increasing warmup phase."

Module Contents¶

class noether.core.schedules.linear_warmup_cosine_decay.LinearWarmupCosineDecaySchedule(config)¶

Bases: noether.core.schedules.base.ScheduleBase

A cosine annealing scheduler with linear increasing warmup phase.”

Example

>>> schedule_config:
>>>   kind: noether.core.schedules.LinearWarmupCosineDecaySchedule
>>>   warmup_percent: 0.05
>>>   end_value: 1.0e-6
>>>   max_value: ${model.optim.lr}

Initialize the scheduler.

Takes either warmup_steps or warmup_percent as argument to determine the length of the warmup phase.

Parameters:: config (noether.core.schemas.schedules.LinearWarmupCosineDecayScheduleConfig) – Configuration for the linear warmup cosine decay schedule.

schedule: noether.core.schedules.base.ScheduleBase¶

warmup_steps¶

warmup_percent¶