tabensemb.model.AbstractNN.configure_gradient_clipping

method

AbstractNN.configure_gradient_clipping(optimizer: Optimizer, optimizer_idx: int, gradient_clip_val: int | float | None = None, gradient_clip_algorithm: str | None = None) → None

Perform gradient clipping for the optimizer parameters. Called before optimizer_step().

Parameters:

• optimizer – Current optimizer being used.

• optimizer_idx – Index of the current optimizer being used.

• gradient_clip_val – The value at which to clip gradients. By default value passed in Trainer will be available here.

• gradient_clip_algorithm – The gradient clipping algorithm to use. By default value passed in Trainer will be available here.

Example:

# Perform gradient clipping on gradients associated with discriminator (optimizer_idx=1) in GAN
def configure_gradient_clipping(self, optimizer, optimizer_idx, gradient_clip_val, gradient_clip_algorithm):
    if optimizer_idx == 1:
        # Lightning will handle the gradient clipping
        self.clip_gradients(
            optimizer,
            gradient_clip_val=gradient_clip_val,
            gradient_clip_algorithm=gradient_clip_algorithm
        )
    else:
        # implement your own custom logic to clip gradients for generator (optimizer_idx=0)