Julia dataframe to database ODBC , not slow solution

Viewed 151

I want to insert a Dataframe in a table using ODBC.jl , the table already exists and it seems that i can't use the function ODBC.load with it (even with the append=true). usually i insert DataFrames with copyIn by loading the dataframe as a csv but with just ODBC it seems i can't do that .

The last thing i found is :

stmt = ODBC.prepare(dsn, "INSERT INTO cool_table VALUES(?, ?, ?)")

for row = 1:size(df, 1)
    ODBC.execute!(stmt, [df[row, x] for x = 1:size(df, 2)])
end

but this is line by line , it's incredibly long to insert everything .

I tried also to do it myself like this :

_prepare_field(x::Any) = x
_prepare_field(x::Missing) = ""
_prepare_field(x::AbstractString) = string(''', x, ''')

row_names = join(string.(Tables.columnnames(table)), ",")
row_strings = imap(Tables.eachrow(table)) do row
    join((_prepare_field(x) for x in row), ",") * "\n"
end
query = "INSERT INTO $tableName($row_names) VALUES $row_strings"
@debug "Inserting $(nrow(df)) in $tableName)"
DBInterface.execute(db.conn, query)

but it throw me an error because of some "," not at the right place at like col 9232123 which i can't find because i have too many lines , i guess it's the _prepare_field that doesn't cover all possible strings escape but i can't find another way .

Did i miss something and there is a easier way to do it ?

Thank you

0 Answers
Related