Pointer dereferencing in Julia and writing directly into memory

user_13878502 0
Viewed 208

This question continues story of this one

For some reason I'd like to manually manage memory in my program. So I want to allocate memory, work with it, and free it. The first and the last are quite easy. However there are some troubles with pointer dereferencing.

Suupose I have such a struct

mutable struct Point
    x::Float64
    y::Float64
end

I'd like to allocate some memory and store my Point there. In fact I could just say point = Point(0.,0.) but this prevents me from manually freeing up allocated memory and lets me rely on gc. So I write

ptr = convert(Ptr{Point}, Base.Libc.malloc(sizeof(Point))) # N.B. sizeof(Point) == 16

And at this point I want to write into memory pointer by ptr. I found some hacky ways to do it.

  1. Use unsafe_store!. This method is not good for me, because I have to allocate memory for object to be written (Point(0.,0.)) and only after that copy it into ptr.
unsafe_store!(ptr, Point(0.,0.))
  1. Use unsafe_wrap. This could be a good idea however I cannot just wrap a piece of memory into array of mutable types. That's because... array of mutable structs contains only sequential pointers to this structs, not the structs themselves. So because Point has two fields of type Int, I can wrap with type Tuple{Float64,Float64} and then assign values inside of wrapped array. But I also have to store (0.,0.) somewhere, and it's also just copying. And of course I cannot assign to fields of immutable type, so on the one hand I cannot wrap around mutable type, on the other, I cannot modify immutable (and on the third hand I cannot reinterpret them into each other)
ptr2 = convert(Ptr{Tuple{Float64,Float64}}, ptr)
arr = unsafe_wrap(Array, ptr2, 1)
arr[1] = (0.,0.)
  1. Finally I can brutally convert everything to primitive types and this really allows avoid additional allocating and copying. And at this point it's time to ask myself a question, what for do I need the declaration of type Point if I'm working with it internals, guts and bones, directly. (Means I wanted something like p.x = p.y = 0.)
ptr3 = convert(Ptr{Int}, ptr)
arr = unsafe_wrap(Array, ptr3, 2)
arr[1] = arr[2] = 0.
  1. The least hackish and the least working (not working at all) method. I found such a good function, called unsafe_pointer_to_objref and at first sight it should do all the bussines. But. It doesn't. It works quite strange. As it is said in docs (source code)
unsafe_pointer_to_objref(p::Ptr)

Convert a Ptr to an object reference. Assumes the pointer refers to a valid heap-allocated Julia object. If this is not the case, undefined behavior results, hence this function is considered "unsafe" and should be used with care.

So I have a heap-allocated Point, with ptr pointing on it, I can even unsafe_load value of it (unsafe_load makes a copy, so it is not the solution), and I should be able to make something like

point = unsafe_pointer_to_objref(ptr)

But this leads to

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x278d238 -- typekeyvalue_hash at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:1160 [inlined]
lookup_typevalue at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:722
in expression starting at none:0
typekeyvalue_hash at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:1160 [inlined]
lookup_typevalue at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:722
jl_inst_arg_tuple_type at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:1589
jl_f_tuple at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:786 [inlined]
jl_f_tuple at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:781
indexed_iterate at .\pair.jl:37 [inlined]
indexed_iterate at .\pair.jl:37
jfptr_indexed_iterate_29996.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
print_response at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:281
#45 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:275
jfptr_YY.45_50669.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
with_repl_linfo at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:508
print_response at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:273
do_respond at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:844
jfptr_do_respond_48815.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:757
#invokelatest#2 at .\essentials.jl:716 [inlined]
invokelatest at .\essentials.jl:714 [inlined]                                                                                                                                                                      run_interface at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\LineEdit.jl:2493
jfptr_run_interface_49569.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
run_frontend at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:1230
#49 at .\task.jl:423
jfptr_YY.49_48790.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:877
Allocations: 2721 (Pool: 2712; Big: 9); GC: 0

However it works good in pair with pointer_from_objref (which returns Ptr{Nothing}, so I think unsafe_pointer_to_objref works good only for Julia-owned objects

a === unsafe_pointer_to_objref(pointer_from_objref(a))

Finally it seems that PointerArithmetic.jl uses unsafe_load to dereference pointer, that is not the solution due to mentioned above reason.

So there are 2 questions:

Main: How to dereference pointer on a mutable struct and obtain object located in the place that pointer points on?

Not main: why does not unsafe_pointer_to_objref give a result and what for does it exist?

dereference julia pointers
1 Answers

if you are not requiring a C interface for anything but memory allocation, but do not want the garbage collector to manage your structs, why not just preallocate your own heap?

Consider

struct MyStruct
    i::Int
    j::Float64
    MyStruct(i = 0, j = 0.0) = new(0, 0.0)
end

const heapofMyStruct = [MyStruct() for _ in 1:1000]

m = heapofMyStruct[1]

You will need to keep track of where you are "allocating" the next item in your heap, but heapofMyStruct should not be garbage collected in use.

user_42828470
Related