I know this way to convert a column of string values to float values:
df = DataFrame(:a => ["2,2", "3,3"])
df.a = parse.(Float64, replace.(df.a, (','=>'.',)))
But how to convert a column with missing values?
df = DataFrame(:a => ["2,2", missing]
I've tried passmissing but it seems not time and memory optimized:
julia> @time passmissing(v->parse(Float64, replace(v, (','=>'.')))).(df.a)
0.140616 seconds (143.84 k allocations: 7.424 MiB)
2-element Array{Float64,1}:
2.2
3.3
It is optimized:
julia> f(x) = passmissing(v->parse(Float64, replace(v, (','=>'.'))))(x)
f (generic function with 1 method)
julia> @time f.(df.a)
0.031789 seconds (108.92 k allocations: 5.402 MiB)
2-element Array{Float64,1}:
2.2
3.3
julia> @time f.(df.a)
0.000047 seconds (11 allocations: 576 bytes)
2-element Array{Float64,1}:
2.2
3.3
The only problem that if you define a new anonymous function each time you want to do a replacement you trigger compilation anew.
If you are bound to use a new anonymous function each time then you can use a comprehension as it is lighter than broadcasting in terms of compilation cost:
julia> @time passmissing(v->parse(Float64, replace(v, (','=>'.')))).(df.a)
0.051069 seconds (143.81 k allocations: 7.420 MiB)
2-element Array{Float64,1}:
2.2
3.3
julia> @time passmissing(v->parse(Float64, replace(v, (','=>'.')))).(df.a)
0.072161 seconds (143.81 k allocations: 7.421 MiB)
2-element Array{Float64,1}:
2.2
3.3
julia> @time [passmissing(v->parse(Float64, replace(v, (','=>'.'))))(v) for v in df.a]
0.039240 seconds (76.63 k allocations: 3.960 MiB)
2-element Array{Float64,1}:
2.2
3.3
julia> @time [passmissing(v->parse(Float64, replace(v, (','=>'.'))))(v) for v in df.a]
0.051859 seconds (76.63 k allocations: 3.961 MiB, 25.99% gc time)
2-element Array{Float64,1}:
2.2
3.3
(you can see this by analyzing the volume of allocations)