Julia fastest way to read values string to array

user_5824843 0
Viewed 335

I have an array of multiple strings, whos values are actually arrays of various types, e.g.:

Julia> _string
"[(-33.8800966, 151.2069034), (-33.8801202, 151.2071933), (-33.8803442, 151.2083656), (-33.8804469, 151.2088682), (-33.8788247, 151.2104533)]"

Julia> typeof(_string)
String

Julia> _string2
"[1, 2, 3, 4]"

Julia> typeof(_string2)
String

I would like to convert to these into arrays quickly, I know the type of what each string is supposed to be beforehand.

i.e. so

Julia> convert(Array{Tuple(Float64, Float64)}, _string)
(-33.8800966, 151.2069034), (-33.8801202, 151.2071933), (-33.8803442, 151.2083656), (-33.8804469, 151.2088682), (-33.8788247, 151.2104533)]

Julia> convert(Array{Int,1}, _string2)
[1, 2, 3, 4]

Currently i'm using eval(Meta.parse(_string)) which is super slow when repeated millions of times.

What's a fast way to quickly read these strings into arrays?

julia type-conversion optimization string performance
1 Answers

This probably isn't the best answer but one way to speed it up is to parse the strings exploiting any information on their structure you have, e.g. for your second example:

julia> using BenchmarkTools

julia> @btime parse.(Int64, split(replace(replace($_string2, "[" => ""), "]" => ""), ","))
  995.583 ns (19 allocations: 944 bytes)
4-element Array{Int64,1}:
 1
 2
 3
 4

Which compares to 

julia> @btime eval(Meta.parse($_string2))
  135.553 μs (43 allocations: 2.67 KiB)
4-element Array{Int64,1}:
 1
 2
 3
 4

on my machine.

How feasible this is will of course depend on whether you are able to find patterns in that way for all your strings quickly.

user_24998920
Related