How to increase the weight decay of `tfa.optimizers.AdamW` withtout interfering with the rest of the optimizer parameters

I have a model that I've been training, and I would like to increase the weight decay parameter I original set for AdamW

For convenience here is the optimizer's documentation https://www.tensorflow.org/addons/api_docs/python/tfa/optimizers/AdamW

and sample initialization

tfa.optimizers.AdamW(
    weight_decay: Union[FloatTensorLike, Callable],
    learning_rate: Union[FloatTensorLike, Callable] = 0.001,
    beta_1: Union[FloatTensorLike, Callable] = 0.9,
    beta_2: Union[FloatTensorLike, Callable] = 0.999,
    epsilon: tfa.image.filters.FloatTensorLike = 1e-07,
    amsgrad: bool = False,
    name: str = 'AdamW',
    **kwargs
)

I started training with a certain weight decay, and I wish to increase it now, but I believe I can't change the optimizer parameters without changing the current Adam parameters as well.

I can either load my model after compiling it

model.compile(..., optimizer=tfa.optimizers.AdamW(...
model.load_weights(...

Which will also load the last saved Adam optimizer parameters.

Or I can compile after loading weights

model.load_weights(...
model.compile(..., optimizer=tfa.optimizers.AdamW(...

which will allow a new weight decay, but also wipeout the saved optimizer values.