Boost python not seeing memory is owned by smart pointer

Question

I am getting a seg fault trigger when the destructor below destroys it's vector elements. Originally it was a vector but I changed this to vector> and since then the crash occurs every time:

class Owner
{
    Owner() = default;
    ~Owner() = default;    // Seg faults

    void assignVec(std::vector>& vec)
    {
        _vec = std::move(vec);
    }

    std::vector> _vec;
};

Each vector element subtype is a polymorphic class, also inheriting from boost::python::wrapper

class Child: public Parent, public boost::python::wrapper
{
    Child();
    virtual ~Child() = default;
};

where:

class Parent
{
    Parent() = default;
    virtual ~Parent() = default;
};

So the entire inheritance hierarchy does have virtual destructors.

GDB backtrace is showing:

#0  0x00007ffff636b207 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1  0x00007ffff636c8f8 in __GI_abort () at abort.c:90
#2  0x00007ffff63add27 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff64bf678 "*** Error in `%s': %s: 0x%s ***
") at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
#3  0x00007ffff63b6489 in malloc_printerr (ar_ptr=0x7ffff66fb760 , ptr=, str=0x7ffff64bcd31 "free(): invalid pointer", action=3) at malloc.c:5004
#4  _int_free (av=0x7ffff66fb760 , p=, have_lock=0) at malloc.c:3843
#5  0x00007fffc373972f in Child::~Child (this=0x2742b10, __in_chrg=) at Child.h:23
#6  0x000000000045694e in std::default_delete::operator() (this=0x11922e0, __ptr=0x2742b10) at /opt/gcc-8.2.0/include/c++/8.2.0/bits/unique_ptr.h:81
#7  0x0000000000454c27 in std::unique_ptr >::~unique_ptr (this=0x11922e0, __in_chrg=) at /opt/gcc-8.2.0/include/c++/8.2.0/bits/unique_ptr.h:274
#8  0x000000000045a882 in std::_Destroy > > (__pointer=0x11922e0) at /opt/gcc-8.2.0/include/c++/8.2.0/bits/stl_construct.h:98
#9  0x0000000000458f67 in std::_Destroy_aux::__destroy >*> (__first=0x11922e0, __last=0x11922e8) at /opt/gcc-8.2.0/include/c++/8.2.0/bits/stl_construct.h:108
#10 0x0000000000457636 in std::_Destroy >*> (__first=0x11922e0, __last=0x11922e8) at /opt/gcc-8.2.0/include/c++/8.2.0/bits/stl_construct.h:137
#11 0x000000000045584d in std::_Destroy >*, std::unique_ptr > > (__first=0x11922e0, __last=0x11922e8)
    at /opt/gcc-8.2.0/include/c++/8.2.0/bits/stl_construct.h:206
#12 0x000000000049b53d in std::vector >, std::allocator > > >::~vector (this=0x7fffffffc4a8, __in_chrg=)
    at /opt/gcc-8.2.0/include/c++/8.2.0/bits/stl_vector.h:567
#13 0x000000000048c677 in Owner::~Owner (this=0x7fffffffc4a8, __in_chrg=)

Printing this in frame 5 does show a valid object. Frame 4 source code of free() is:

static void _int_free(mstate av, mchunkptr p, int have_lock)                      
{                                                                     
  INTERNAL_SIZE_T size;        /* its size */                         
  mfastbinptr*    fb;          /* associated fastbin */               
  mchunkptr       nextchunk;   /* next contiguous chunk */            
  INTERNAL_SIZE_T nextsize;    /* its size */                         
  int             nextinuse;   /* true if nextchunk is used */        
  INTERNAL_SIZE_T prevsize;    /* size of previous contiguous chunk */
  mchunkptr       bck;         /* misc temp for linking */            
  mchunkptr       fwd;         /* misc temp for linking */            

  const char *errstr = NULL;
  int locked = 0;

  size = chunksize(p);

  /* Little security check which won't hurt performance: the          
     allocator never wrapps around at the end of the address space.   
     Therefore we can exclude some size values which might appear
     here by accident or by "design" from some intruder.  */        
  if (__builtin_expect ((uintptr_t) p > (uintptr_t) -size, 0)    
      || __builtin_expect (misaligned_chunk (p), 0))
    {  
      errstr = "free(): invalid pointer";
    errout:      
      if (have_lock || locked)
        (void)mutex_unlock(&av->mutex);                               
      malloc_printerr (check_action, errstr, chunk2mem(p), av);      // CRASHES HERE

Does anyone have any advice how to proceed debugging this?

UPDATE:

I have created a small example in a unit test, creating Owner and a vector, calling assignVec() and the problem does not occur. However, once the vector is passed in, nothing else obtains the Parent memory.

UPDATE2:

We think the problem is with boost python needing to be informed of the smart pointer. Apparently Boost Python doesn't support unique_ptr and we're struggling to get it to recognize shared_ptr (both Boost and std), even using the typical register technique.

user_3537677 · Answer

Based on the code you are giving and with the assumption boost::python has not a bug here I would guess your usage of moving semantics might be the cause:

void assignVec(std::vector>& vec)
    {
        _vec = std::move(vec);
    }

Here you are moving from a l-value-reference to a vector, vector& to your member. Problem is: Usually one only moves from r-values references (vector&&) as they ca not accessed any more, because the only bind to temporaries or if the one explicitly acknowledges the moving by creating an r/x-values reference like you did by using std::move.

Problem is: I bet you caller of assignVec might not be aware of this, because then why did you not already use a r-value reference in the signature, so that the caller has to std::move explicitly? My assumption is, you caller does not, and is doing more than the one thing that is legal to moved from values: Destruct them.

Of course you ask yourself, why does it break then in the destructor? In my experience segmentation faults occur some expression after the cause, in this case I would say the undefined behaviour of the caller of assignVec by still using the vector given to assignVec.

Boost python not seeing memory is owned by smart pointer

1 Answers