Change optimizers and lr shedulers

The default optimizer of PytorchTabular, WideDeep, and TorchModel is torch.optim.Adam while the default lr scheduler is torch.optim.lr_scheduler.StepLR (but with gamma=1 so that the learning rate does not change).

To set optimizers or lr schedulers for each model in each model base, use the optimizers and lr_schedulers arguments, such as

models = [
    PytorchTabular(
        trainer,
        model_subset=["Category Embedding"],
        optimizers={
            "Category Embedding": ("AdamW", {"lr": None, "weight_decay": None})
        },
        lr_schedulers={
            "Category Embedding": ("StepLR", {"gamma": 0.1, "step_size": 1})
        },
    )
]

Optimizers and lr schedulers are set by their names in torch.optim and torch.optim.lr_scheduler, respectively.

Note that, if names match, the values of parameters (such as the Nones shown above) will be updated by those from AbstractModel._initial_values or those optimized during the bayesian hyperparameter optimization.