Return reference to member field in PyO3

I don't think what you say is possible without some rejigging of the interface:

Your XWrapper owns the x and your Y owns its x as well. That means creating an XWrapper will always involve a clone (or a new).

Could we change XWrapper so that it merely contains a reference to an x? Not really, because that would require giving XWrapper a lifetime annotation, and PyO3 afaik doesn't allow pyclasses with lifetime annotation. Makes sense, because passing an object to python puts it on the python heap, at which point rust loses control over the object.

So what can we do?

Some thoughts: Do you really need to expose the composition structure of y to the python module? Just because that's the way it's organized within Rust doesn't mean it needs to be that way in Python. Your YWrapper could provide methods to the python interface that behind the scenes forward the request to the x instance:

#[pymethods]
impl YWrapper{
   pub fn some_method(&self) {
     self.y.x.some_method();
   }
}

This would also be a welcome sight to strict adherents of the Law of Demeter ;)

I'm trying to think of other clever ways. Depending on some of the details of how y.x is accessed and modified by the methods of y itself, it might be possible to add a field x: XWrapper to the YWrapper. Then you create the XWrapper (including a clone of y.x) once when YWrapper is created, and from then on you can return references to that XWrapper in your pub fn x. Of course that becomes much more cumbersome when x gets frequently changed and updated via the methods of y...

In a way, this demonstrates the clash between Python's ref-counted object model and Rust's ownership object model. Rust enforces that you can't arbitrarily mess with objects unless you're their owner.

It is indeed possible to share objects and return them or mutate them whatever. Just as in Python. What Lagerbaer suggested works and it's actually pretty ideal for small codes. However, if the number of methods increases there will be A LOT of repeating and boilerplate needed (and drastically increases when you increase the depth of your nesting).

I have no idea if this is something that we are supposed to do. But from what I understood from the way to do it is using Py. God wish I had a habit of reading the docs thoroughly before experimenting.

In https://docs.rs/pyo3/latest/pyo3/#the-gil-independent-types in the MAIN PAGE of the doc says: When wrapped in Py<...>, like with Py or Py, Python objects no longer have a limited lifetime which makes them easier to store in structs and pass between functions. However, you cannot do much with them without a Python<'py> token, for which you’d need to reacquire the GIL.

A Py is "A GIL-independent reference to an object allocated on the Python heap." https://docs.rs/pyo3/latest/pyo3/prelude/struct.Py.html

In other words, what we need here to return pyclass objects is return Py<pyclass_struct_name>.

Your example is too complicated and to be honest I don't even understand what you are trying to do but here is an alternative version which suits my own usecase more closely. Since this is basically one of the only results that pops in Google I see it fit to paste it here even if it is not an exact response to the example provided above.

So here we go...

Suppose we have a Rust struct X and we cannot modify the lib as you mentioned. We need an XWrapper (let's call it PyX) pyclass to hold it.

So we define them here:

// in lib.rs
pub struct X {}

// in py_bindings.rs
#[pyclass]
struct PyX{
    value: Py<X>,
}

impl_new_for!(PyX);

Then for the usage, all we have to do is to initialize the object with a GIL lock (assuming in the init of the XWrapper) and then define a getter for it. THE IMPORTANT NOTE HERE IS THAT YOU CALL clone_ref ON IT AND DO NOT RETURN THE OBJECT.

This is basically a nested class system afterwards and the nested object is immutable (has interior mutability tho) so it's a fantastic way to nest your code as well.

In the example below, I used my needed X as a PyX in yet another wrapper called Api.

#[pyclass]
struct Api {
    x: PyX,
}

#[pymethods]
impl Api {
    #[new]
    fn __new__() -> PyResult<Self> {
        Python::with_gil(|py| {
            Ok(Self {
                x: Py::new(
                    py,
                    PyX::new(),
                ),
            })
        }
    }

    #[getter(x)]
    fn x(&mut self, py: Python) -> Py<Network> {
        self.x.clone_ref(py)
    }
}