Why AdamW matters Adams does exactly that for you: Big steps when the gradients do not change much and small steps when they… Continue Reading decay, regularization, weight