How to check if an object is a certain PyTorch optimizer?

user_7353970 0
Viewed 409

I want to check the class of objects like this:

from torch.optim import Adam, SGD, AdamW

adam_range = (0.8, 1.0)
adamw_range = (0.6, 0.7)
sgd_range = (0.0, 0.5)

targets = []
for cfg in configs:
    if isinstance(cfg["optimizer"], Adam):
        sample = np.random.uniform(low=adam_range[0], high=adam_range[1], size=1)
    elif isinstance(cfg["optimizer"], AdamW):
        sample = np.random.uniform(low=adamw_range[0], high=adamw_range[1], size=1)
    elif isinstance(cfg["optimizer"], SGD):
        sample = np.random.uniform(low=sgd_range[0], high=sgd_range[1], size=1)

where configs is a list of dicts and cfg["optimizer"] are uninitialized references of from torch.optim import Adam, SGD, AdamW. However, none of the if/elif statements evaluate to True.

How do I properly check the type of these references? Is there a better way than checking cfg["optimizer"].__name__? The latter seems inelegant.

Thank you.

pytorch python
1 Answers

Your cfg["optimizer"] is not an instance of any optimizer, but the type itself.
therefore, you should test it like this:

for cfg in configs:
    if cfg["optimizer"] is Adam:
        sample = np.random.uniform(low=adam_range[0], high=adam_range[1], size=1)
    elif cfg["optimizer"] is AdamW:
        sample = np.random.uniform(low=adamw_range[0], high=adamw_range[1], size=1)
    elif cfg["optimizer"] is SGD:
        sample = np.random.uniform(low=sgd_range[0], high=sgd_range[1], size=1)

To emphasize the difference between an instance of Adam and type Adam, try the following:

opt = Adam(model.parameters(), lr=0.1)  # make an _instance_ of Adam

isinstance(opt, Adam)  # True  - opt is an instance of Adam optimizer

isinstance(Adam, Adam) # False - Adam is a type, not an instance of Adam

isinstnace(Adam, type) # True  - "Adam" is a type not an instance of Adam

type(opt) is Adam      # True  - the type of opt is Adam

type(opt) == Adam      # True

I find this page to summarize this difference quite well.

user_17144100
Related