What's the lifetime of Julia String literals

Viewed 91

For example, is it safe to use the pointer returned by foo until the end of a Julia session? And in this case, there is no difference between foo and bar, right?

foo(s::String="foo") = Base.unsafe_convert(Ptr{UInt8}, Base.cconvert(Ptr{UInt8}, s))
bar() = Base.unsafe_convert(Ptr{UInt8}, Base.cconvert(Ptr{UInt8}, "foo"))

EDIT:

I'd like to add more background info. I'm trying to interface some methods in a C++ library that has a default parameter value that defaults to a C++ string literal. The derived pointer is used as a "reference"(see the link below). It seems to be OK to just use the literal in C++, I'm wondering what's the best way to do this in Julia. (I don't think GC.@preserve is a good solution because it's hard to know when the resource can be safely GC-ed.)

X-ref: Is it safe to init `StringRef` with a string literal?

2 Answers

From the documentation, the answer appears to be a fairly clear "no".

Firstly, as you likely know (and indeed might guess from the name), Base.unsafe_convert provides no safety on this front, with the docstring warning

Be careful to ensure that a Julia reference to x exists as long as the result of this function will be used

The existence of a pointer alone will certainly not provide that. Worse, as the docstring for Base.cconvert notes

Neither convert nor cconvert should take a Julia object and turn it into a Ptr.

What is the recommended way to take a Julia object and turn it into a Ptr? That would seem to be Base.pointer, which provides a bit more useful information in its documentation

help?> Base.pointer
  pointer(array [, index])

  Get the native address of an array or string, optionally at a given location index.

  This function is "unsafe". Be careful to ensure that a Julia reference to array exists
  as long as this pointer will be used. The GC.@preserve macro should be used to protect
  the array argument from garbage collection within a given block of code.

  Calling Ref(array[, index]) is generally preferable to this function as it guarantees
  validity.

Notably, the pointers returned by

julia> s = "some string"
"some string"

julia> Base.pointer(s)
Ptr{UInt8} @0x000000010d4b2838

julia> Base.unsafe_convert(Ptr{UInt8}, s)
Ptr{UInt8} @0x000000010d4b2838

julia> Base.unsafe_convert(Ptr{UInt8}, Base.cconvert(Ptr{UInt8}, s))
Ptr{UInt8} @0x000000010d4b2838

are identical.

A bit more useful information can be found in the Base.GC.@preserve docstring in turn:

help?> Base.GC.@preserve
  GC.@preserve x1 x2 ... xn expr

  Mark the objects x1, x2, ... as being in use during the evaluation of the expression
  expr. This is only required in unsafe code where expr implicitly uses memory or other
  resources owned by one of the xs.

  Implicit use of x covers any indirect use of resources logically owned by x which the
  compiler cannot see. Some examples:

    •  Accessing memory of an object directly via a Ptr

    •  Passing a pointer to x to ccall

    •  Using resources of x which would be cleaned up in the finalizer.

  @preserve should generally not have any performance impact in typical use cases where
  it briefly extends object lifetime. In implementation, @preserve has effects such as
  protecting dynamically allocated objects from garbage collection.

  Examples
  ≡≡≡≡≡≡≡≡≡≡

  When loading from a pointer with unsafe_load, the underlying object is implicitly
  used, for example x is implicitly used by unsafe_load(p) in the following:

  julia> let
             x = Ref{Int}(101)
             p = Base.unsafe_convert(Ptr{Int}, x)
             GC.@preserve x unsafe_load(p)
         end
  101

  When passing pointers to ccall, the pointed-to object is implicitly used and should be
  preserved. (Note however that you should normally just pass x directly to ccall which
  counts as an explicit use.)

  julia> let
             x = "Hello"
             p = pointer(x)
             Int(GC.@preserve x @ccall strlen(p::Cstring)::Csize_t)
             # Preferred alternative
             Int(@ccall strlen(x::Cstring)::Csize_t)
         end
  5

So, some take-home points

  1. A pointer returned by Base.pointer(s) to a string will be safe if and only if either (a) there is a Julia reference to that string s later in the code or (b) you have used Base.GC.@preserve
  2. A pointer returned by Base.unsafe_convert(Ptr{UInt8}, Base.cconvert(Ptr{UInt8}, s)) appears to have all the same caveats as a pointer returned by Base.pointer, with the added bonus of a cryptic and ominous warning.
  3. There is no indication that the type of Julia object (String vs Array vs etc.) matters.

The garbage collector can invalidate the references returned by your functions, if they are relocated by the garbage collector and then referenced. So don't do that. If you must, you can wrap the code in the GC.@preserve macro to mark them as not relocatable. Note I have experienced some weird situations using opaque C compiled libraries where you will crash Julia even then, so avoid such reference taking if you can. The macro is done this way, see https://docs.julialang.org/en/v1/base/base/#Base.GC.@preserve:

GC.@preserve x y begin
    x = foo()
    y = bar()
    @show x

# do stuff that might fire off the garbage collector ...

    @show x
end

The string "foo" itself can be referenced without fear even if it is relocated, because Julia always knows where any given string literal is.

Related