Hi I would like to know.
- what is the best way to copy large data into an sql database using dplyr?
copy_to fails as data is larger than memory. I am using code:
library(DBI)
library(dplyr)
db <- dbConnect(
RPostgreSQL::PostgreSQL(),
dbname = "postgres",
host="localhost",
user="user",
password="password")
Data<-rio::import("Data/data.feather")
Data <- copy_to(db, Data,temporary=FALSE)
This results in error:
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: out of memory
DETAIL: Cannot enlarge string buffer containing 0 bytes by 1305608043 more bytes.
)
- is there a way to do this without having to import the data first?
- is there a way to do this though sparkly since this is using only one core?