noether.core.schedules.linear_warmup_cosine_decay

Classes

LinearWarmupCosineDecaySchedule

A cosine annealing scheduler with linear increasing warmup phase."

Module Contents

class noether.core.schedules.linear_warmup_cosine_decay.LinearWarmupCosineDecaySchedule(config)

Bases: noether.core.schedules.base.ScheduleBase

A cosine annealing scheduler with linear increasing warmup phase.”

Example

Takes either warmup_steps or warmup_percent as argument to determine the length of the warmup phase.

Parameters:

config (noether.core.schemas.schedules.LinearWarmupCosineDecayScheduleConfig) – Configuration for the linear warmup cosine decay schedule. See LinearWarmupCosineDecayScheduleConfig for details.

schedule: noether.core.schedules.base.ScheduleBase

warmup_steps

warmup_percent