How can I get a pointer to the data inside a generic PyObject*?

This seems like an X-Y problem (i.e. you think you need to extract the data out of a bunch of Python objects at C level, but actually you would benefit from having a single Python object which exposes all your data).

A Python int can store (almost) arbitrarily large numbers:

>>> 1000**1000  # creates a very big int

i.e. it is not stored internally as a C long. Internally it is stored as an array of integers (ob_digits) of size ob_size which are in a slightly odd format that isn't much use to you. However, if you really wanted to copy it you would case your object pointer to a PyLongObject* and then do a memcpy(&dest, my_int->ob_digit, sizeof(digit)*abs(my_int->ob_size));. I recommend against this because it's pretty hard for you to use this data.

Obviously this only applies if you know you have a Python int. For a "generic PyObject*" this doesn't work, because a generic PyObject* can contain almost any data. This includes pointers which need ownership and/or reference counting (this especially applies to any PyObject that contains other PyObjects).

What I think you actually want is to store your data in a large array of C integers. This can be done with array.array, or numpy.array, or a variety of other classes.

At a C level these objects support the buffer protocol where they expose that internal array to C, allowing each of your values to be accessed, copied, manipulated, etc. from C.

Some quick untested illustrative code:

Py_Buffer view;
view.format = "l"; // request an array of longs
if (PyObject_GetBuffer(obj, &view, PyBUF_CONTIG | PyBUF_FORMAT | PyBUF_WRITABLE ) == -1) {
   // failed
   return NULL;
}

// you want to check that view.ndim == 1 (for a simple 1D array)
long* data = (long*)view.buf;
// At this point you can access data as a C array of length view.len

// When you've finished;
PyBuffer_Release(view);

This seems to be a design issue relating to abstraction of data structures. Often, it is desirable to give the user an opaque data structure or pointer. Access to internal elements would require a method (or function) call.

From https://docs.python.org/3/c-api/long.html,

PyObject* PyLong_FromLong(long v)

Return value: New reference.
Return a new PyLongObject object from v, or NULL on failure.

The current implementation keeps an array of integer objects for all integers between -5 and 256, when you create an int in that range you actually just get back a reference to the existing object.

The call is potentially doing an internal conversion to PyLongObject which can be a link to its internal object. If you pass between -5 to 256, it will replace your integer with its internal object. For others, a new object will be created. Even if you locate the internal memory location, there is no guarantee the behavior will remain consistent.

PyObject seems to be designed to be opaque. Treat it as such.

There is an internal CPython function that does something like what you want, called _PyLong_AsByteArray.

It seems to read the needed bytes from a field called ob_digit, but I don't fully get the whole function.