I stumbled upon the method add_module() in a Pytorch model.
The doc only states
Adds a child module to the current module.
The module can be accessed as an attribute using the given name.
I don't understand what "adding a child module" means.
How is it different from just setting a pointer to the other module using self._other module = other_module?
What are the nuances?
As mentioned here: https://discuss.pytorch.org/t/when-to-use-add-module-function/10534
In general, you won’t need to call add_module. One potential use case is the following:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
modules = [...] # some list of modules
for module in modules:
self.add_module(...)
nn.Modules have a hierarchy of child modules that you can access via methods like module.named_children() or module.children().
As mentioned in the forum post above, doing self._other module = other_module will type check other_module, see it's an nn.Module, and also add it to the child list, so add_module isn't really necessary.
Adding a module as an attribute works fine, as you say. But it can be a bit difficult to do at runtime if you don't know how many modules you have in advance, and you have to construct names programmatically. In such a case, add_module() is a very convenient way to do this. I've just written a short blog post showing this in action:
https://blog.d-and-j.net/deep-learning/2021/04/23/pytorch-add_module.html