Why not accumulate query loss and then take derivative in MAML with Pytorch and Higher?

The only difference is that in second approach you'll have to keep much more stuff in memory - until you call backward you'll have all unrolled parameters fnet.parameters(time=T) (along with intermediate computation tensors) for each of task_num iterations as part of the graph for the aggregated meta_loss. If you call backward on every task then you only need to keep full set of unrolled parameters (and other pieces of the graph) for one task.

So to answer your question's title: because in this case the memory footprint is task_num times bigger.

In a nutshell what you're doing is similar to comparing loopA(N) and loopB(N) in the following code. Here loopA will get as much memory as it can and OOM with sufficiently large N, while loopB will use about same amount of memory for any large N:

import torch
import numpy as np
a = 0
np.random.seed(1)
v = torch.tensor(np.random.randn(1000000))
y = torch.tensor(np.random.randn(1000000))
x = torch.zeros(1000000, requires_grad=True)

def loopA(N=1000):
    a = 0
    for i in range(N):
        a += ((x * v - y)**2).sum()
    a.backward()

def loopB(N=1000):
    for i in range(N):
        a = ((x * v - y)**2).sum()
        a.backward()

Regarding the normalization - two approaches are equivalent (up to numerical precision maybe): if you first sum up individual losses, then divide by task_num, then finally call backward then you'll effectively compute d((Loss_1 + ... + Loss_{task_num})/task_num) / dw (where w is one of the weights meta-optimizer is fitting). On the other hand if you call backward for each loss divided by task_num you'll get d(Loss_1/task_num)/dw + ... + d(Loss_{task_num}/task_num)/dw which is the same because taking gradient operation is linear. So in both cases your meta-optimizer step will start with pretty much same gradients.

I will try to summarize our discussion to Alex's already great answer. It usually helps me when I go through answers to read more than 1 answer.

The main thing is realizing the main functionality of Pytorch. As I understand it really is a framework for doing derivatives automatically for you. Thus, whenever you do a forward pass, Pytorch needs to keep track of the weights and intermediate computations to be able to compute gradients whenever the user wants. However, once the derivatives (wrt to the corresponding variables) have been take with .backward() usually one does not need to keep track of the intermediate computation (unless you put the retain_graph=True flag) so internally Pytorch somehow stops following some references (it's our guess) and then eventually python's Garbage collector cleans things up. This reduces the memory. Thus, if one calls .backward() each time inside the loop then one is potentially saving on memory. On top of that the code in the original post has on top of that a inner loop inside the other loop. Thus, the memory foot print increases even more.

Consider the simplified pseudocode:

def loop_good_memory():
  for outer_i in range(nb_outer_episodes):
    for inner_i in range(nb_inner_steps):
        spt_logits = fnet(x_spt[i])
        spt_loss = F.cross_entropy(spt_logits, y_spt[i])
        diffopt.step(spt_loss)
    qry_logits = fnet(x_qry[i])
    qry_loss = F.cross_entropy(qry_logits, y_qry[I])
    qry_loss.backward()
meta_opt.step()

this one is good in terms of memory footprint because since we are calling .backward()s each time it means each iteration it drops all intermediate computations which here would be even larger due to the inner loop. In total it would be O(nb_outer_episodes * nb_inner_steps) without the inner backward but this is just about O(nb_outer_episodes ).

def loop_bad_memory():
  for outer_i in range(nb_outer_episodes):
    for inner_i in range(nb_inner_steps):
        spt_logits = fnet(x_spt[i])
        spt_loss = F.cross_entropy(spt_logits, y_spt[i])
        diffopt.step(spt_loss)
    qry_logits = fnet(x_qry[i])
    qry_loss += F.cross_entropy(qry_logits, y_qry[I])
qry_loss.backward()
meta_opt.step()

this second one (imho) is easier to read but has worse memory since each loop has O(nb_inner_steps ) since it needs to remember the intermediate computations, in particular the inner weights are part of the graph (i.e. fnet.parameters(time=T)). Note that I didn't write the higher context manger but it should be there of course.

Credits to Alex thanks!