I've unfortunately been given a very messy and very large table (csv) by a client. It's in wide format :'(
As an example the columns are:
Name, Date, Usage_Hr1, Usage_Hr2, ..., Usage_Hr24, ... lots more columns
I would normally just load the .csv into R and use gather from the tidyr package, but the data is too big. I've considered loading the data into sparklyr, but there is no gather function in sparklyr yet...
So my question is, once I've COPYed my table into Cassandra (set the PRIMARY KEY to Name and Date), how can I convert these cols to long format? Am I just out of luck? I'm not a DB guy btw so I have no idea here.
Note I'm using the latest version of Cassandra and my current table is about 10 million rows.