append to request.sessions[list] in Django

user_14118634 0
Viewed 1032

Something is bugging me. I'm following along with this beginner tutorial for django (cs50) and at some point we receive a string back from a form submission and want to add it to a list: https://www.youtube.com/watch?v=w8q0C-C1js4&list=PLhQjrBD2T380xvFSUmToMMzERZ3qB5Ueu&t=5777s

def add(request):
if 'tasklist' not in request.session:
    request.session['tasklist'] = []
if request.method == 'POST':
    form_data = NewTaskForm(request.POST)
    if form_data.is_valid():
        task = form_data.cleaned_data['task']
        request.session['tasklist'] += [task]
        return HttpResponseRedirect(reverse('tasks:index'))

I've checked the type of request.session['tasklist']and python shows it's a list. The task variable is a string.

So why doesn't request.session['tasklist'].append(task) work properly? I can see it being added to the list via some print statements but then it is 'forgotten again' - it doesn't seem to be permanently added to the tasklist.

Why do we use this request.session['tasklist'] += [task] instead?

The only thing I could find is https://ogirardot.wordpress.com/2010/09/17/append-objects-in-request-session-in-django/ but that refers to a site that no longer exists.

The code works fine, but I'm trying to understand why you need to use a different operation and can't / shouldn't use the append method.

Thanks.

django python
3 Answers

The reason why it does not work is because django does not see that you have changed anything in the session by using the append() method on a list that is in the session.

What you are doing here is essentially pulling out the reference to the list and making changes to it without the session backend knowing anything about it. An other way to explain:

  • The append() method is on the list itself not on the session object
  • When you call append() on the list you are only talking to the list and the list's parent (the session) has no idea what you guys are doing
  • When you however do an assignment on the session itself session['whatever'] = 'something' then it knows that something is up and changes are made
  • So the key here is that you need to operate on the session object directly if you want your changes to be updated automatically

Django only thinks it needs to save a changed session item if the item got reassigned to the session. See here: django session base code the __setitem__ method containing a self.modified = True statement.

The session['list'] += [new_element] adds a new list item (mutates the list so reference stays the same) and then gets it reassigned to the session -> thus triggering a __setitem__ call on the session and marking it as modified.

The session['list'] = session['list'] + [new_item] mode of doing the same does create a new list every time it's run so its a bit less efficient, but you should not store hundreds of items in the session anyway. So you're probably fine. This also works exactly as above.

However if you use sub-keys in the session like session['list']['x'] = 'whatever' the session will not see itself as modified so you need to mark it as by request.session.modified = True

user_99841350

Short answer: It's about how Python chooses to implement the dict data structure.

Long answer:

Let's start by saying that request.session is a dictionary.

Quoting Django's documentation, "By default, Django only saves to the session database when the session has been modified – that is if any of its dictionary values have been assigned or deleted". Link

So, the problem is that the session database is not being modified by request.session['tasklist'].append(task)

Seeing the related parts Django's Session base code (as posted by @Csaba K. in an answer), the variable self.modified is to be set True when setitem dunder method is called.

Now, at this step the problem seems like the setitem dunder method is not being called with request.session['tasklist'].append(task) but with request.session['tasklist'] += [task] it gets called. It is not due to if the reference of request.session['tasklist'] is changing or not as pointed out by another answer, because the reference to the underlying list remains the same.

To confirm, let's create a custom dictionary which extends the Python dict, and print something when setitem dunder method is called.

class MyDict(dict):
    def __init__(self, globalVar):
        super().__init__()
        self.globalVar = globalVar

    def __setitem__(self, key, value):
        super().__setitem__(key, value)
        print("Called Set item when: ", end="")

myDict =  MyDict(0)
print("Creating Dict")

print("-----")

myDict["y"] = []
print("Adding a new key-value pair")

print("-----")

myDict["y"] += ["x"]
print(" using +=")

print("-----")

myDict["y"].append("x")
print("append")

print("-----")

myDict["y"].extend(["x"])
print("extend")

print("-----")

myDict["y"] = myDict["y"] + ["x"]
print(" using +",)

print("-----")

It prints:

Creating Dict
-----
Called Set item when: Adding a new key-value pair
-----
Called Set item when:  using +=
-----
append
-----
extend
-----
Called Set item when:  using +
-----

As we can see, setitem dunder method is called and in turn self.modified is set true only when adding a new key-value pair, or using += or using +, but not when initializing, appending or extending an iterable (in this case a list). Now, the operator + and += do very different things in Python, as explained in the other answer. += behaves more like the append method but in this case, I guess it's more about how Python chooses to implement the dict data structure rather than how +, += and append behave on lists.

user_57720080

I found this while doing some more searching: https://code.djangoproject.com/wiki/NewbieMistakes

Scroll to 'Appending to a list in session doesn't work'

Again, it is a very dated entry but still seems to hold true.

Not completely satisfied because this does not answer the question as to 'why' this doesn't work, but at the very least confirms 'something's up' and you should probably still use the recommendations there.

(if anyone out there can actually explain this in a more verbose manner then I'd be happy to hear it)

user_141186340
Related