How CPython detects where to find pool_header after reference count goes down to zero?

user_14021199 0
Viewed 39

Recently I read an article about CPython memory model: https://rushter.com/blog/python-memory-managment/. The article shows the following structure that CPython uses to manage single pool:

struct pool_header {
   union { block *_padding;
           uint count; } ref;          /* number of allocated blocks    */
   block *freeblock;                   /* pool's free list head         */
   struct pool_header *nextpool;       /* next pool of this size class  */
   struct pool_header *prevpool;       /* previous pool       ""        */
   uint arenaindex;                    /* index into arenas of base adr */
   uint szidx;                         /* block size class index        */
   uint nextoffset;                    /* bytes to virgin block         */
   uint maxnextoffset;                 /* largest valid nextoffset      */
};

The thing that I don't get is how CPython obtains this header to update if some block became free? Am I right that it relies on some low level trick that if you allocate a page of 4Kb size then it's pointer is somehow aligned and you can detect the start of the page by zeroing out several bits (possibly 12, because 2^12=4096) of blocks address? Am I right?

cpython garbage-collection python
1 Answers

It just rounds the block address down to the nearest pool-aligned value:

/* Round pointer P down to the closest pool-aligned address <= P, as a poolp */
#define POOL_ADDR(P) ((poolp)_Py_ALIGN_DOWN((P), POOL_SIZE))

While the rounding is essentially as you hypothesized, it is valid not because of the low-level trick you hypothesized, but simply because CPython manually ensures pools have the necessary alignment. When CPython allocates the big chunk of memory used for an arena's pools, it sets the arena's pool_address to the first pool-aligned address in that big chunk of memory:

    /* pool_address <- first pool-aligned address in the arena
       nfreepools <- number of whole pools that fit after alignment */
    arenaobj->pool_address = (block*)arenaobj->address;
    arenaobj->nfreepools = MAX_POOLS_IN_ARENA;
    excess = (uint)(arenaobj->address & POOL_SIZE_MASK);
    if (excess != 0) {
        --arenaobj->nfreepools;
        arenaobj->pool_address += POOL_SIZE - excess;
    }
user_23571120
Related