What tools or processes would you recommend for debugging ctypes memory issues? How is it possible to print the contents of an array but not be able to copy it?
I have narrowed my problem down to the following lines:
[ ... lots of code ...]
valid_data_as_bytes = bytearray(self._output_buffer[:result])
print("valid_data_as_bytes:",valid_data_as_bytes)
print("len(valid_data_as_bytes):", len(valid_data_as_bytes))
assert len(valid_data_as_bytes) == result
Buffer = ctypes.c_ubyte * len(valid_data_as_bytes)
buf2 = Buffer.from_buffer_copy(valid_data_as_bytes) # <--- error occurs here
[ ... lots of code ...]
Below is a print out of the last few lines of output when the tests are run with the command pytest -s:
valid_data_as_bytes: bytearray(b'\xfc\xdf]\'O\xb6\xf4n\xa3p\x87\xcd\xca\xe1\xd53~V\x04\xa3\x977\xd3q[\xf8\x9dh\xe1\x05\x7f\xf3$\xb2S,\xa4\x8d\x04\xe6\xc0\x12\xbf)\xb8\r\xcb&\x9a(\x08\x1c\xca\x1a\x13\x16\xba}\x93e\xf0J\x91\x8eJ\xcbk\xc0\xc2\xc9@{\xcc$\x7f\x1d0\xc58\xcf\xe5\xbf_W\r\x02\xf7\xb8\x7fm\xa1\xd9\xa8\xba6\xed\x17\xdf;\x7f\x8b\x9a\x9f\x03\r\xec\xdc]\x14\xfd\x15B\x98\\\xcb\xc8)\xdd9\xc0\xb2\xb5\xa5Af\x85\x8b\x1a\xac\x89,#\xd9 \xc7\x86SiE|\x84B\xdc"\xcfhG\xef\xec\xffh\x84\x14\xdb\xf9\xc9J2O\xe3[8\x9531Be\xb0p\x87\x0eJ\xa4i\xce)\xbaBXY\xdeP\x88\x13\xdc](_\xd5^m\xdc\xde\x18\x13\xb2H\xe0\xae\xea=#+\x92\xd3\xe6[W\x94X\x96\xfe\xa3\x137\xe9\xfe\xcc6F\xb9\x0b\xc3NO\x93Ksy\xc4A\xcb"\xf0\xc1\xa0\x83\xa3\xc4@\xbb\\\xb8\xfd\xbem\'(A\x9f\x0fV\x9f\xe1\xc0\x97\xd0\x95W"a\x94\xfdymv\xa2>P\x9d\xcc\x80\xc13B\xd0\nu\xd4\x04\xa9\xc4\xd3\xb5q3\x8f\x08:\xacrm\x1bre\x0e\x9a\xd5\x98\xeez \xd3H\xd2\xed\xdd\x12\xd8\xa2,[\xac\xdf\x13\x9f\xed\xa8\xf5\x98\x97\xb3\xc4\x10\xc3\xa6\x10A\x96.>D\x048\x02~\x9c}\xa5\xd1\x93\xd0I\xe94\x1b\x85\'~0\x92\x82lr')
len(valid_data_as_bytes): 354
Windows fatal exception: access violation
This code does not error under macOS or Linux. It also does not error when each of the pytest-based tests in the test suite are run individually. However it consistently throws a Windows fatal exception: access violation error when the entire test suite is run.
I have been looking at what I believe is the implementation of from_buffer_copy(), however I do not know how to step through this; what tools would be best to step through the C-side of Python after it has been running many tests? Or perhaps I should be looking elsewhere?
There are no threads or processes other than the main thread.
On the fact that the code in inefficient: in an attempt to pin down this issue, I forced Python to copy all blocks of memory (hence, for example, the bytearray conversion).
Any suggestions would be very gratefully received!