How to deal with a CSV file with strings in MATLAB R2016b (mac version)

user_3727139 0
Viewed 59

I am now trying to deal with a large csv (2 million rows) in matlab r2016b mac version. One of the small part of csv is attached as below.

user_id,video_id,session,new_speed,old_speed,new_time,old_time,event_type,event_time
a74fe6d4812fa93a1afa1a6a334ebdda,af7f974d395a4adddc8ab17a83996073,892d85cfeea8298fb7ca8755ac090e90,,,,,play_video,2015-04-06 22:20:58.928268
a74fe6d4812fa93a1afa1a6a334ebdda,af7f974d395a4adddc8ab17a83996073,892d85cfeea8298fb7ca8755ac090e90,,,161.0,72.581,seek_video,2015-04-06 22:20:58.879149
a74fe6d4812fa93a1afa1a6a334ebdda,7fb29805973a8a396c6de3faa8290ac1,892d85cfeea8298fb7ca8755ac090e90,,,,,play_video,2015-04-06 22:24:14.988693
a74fe6d4812fa93a1afa1a6a334ebdda,9ace07b312f206ef7af2f48188360b16,892d85cfeea8298fb7ca8755ac090e90,,,,,load_video,

What I want to do is to is

  1. read the csv file into matlab;
  2. map the user id and video id into simple number id;
  3. delete the session id.

I have tried so many methods but could not get the result I want. csvread is unable to deal with this csv file since there are some non-numerical items inside, while I am not familiar with fopen and always get some strange errors like all data come into a single cell. Is there any solution to this issue?

fopen csv matlab
2 Answers

I can't imagine what you are going to do with that array of strings. If you just need the file to be edited, I suggest using python (which is easiest but not the fastest, although faster than matlab) or C (which is so much faster given your condition of 2 million rows, but not the easiest).

As far as I've seen from Matlab, it is very good at mathematical operations on matrix, but not a good option for what you want to do. Maybe if you say what you're going to do with all that data (I mean math ops after what you said), someone can help!

user_77218660

Since you have different format in your text as you mentioned, it would be a good idea to use textread() function. It returns 1D cell array. So first you have to parse it by regexp() and index the cell in this way :

Thecell{row}{column}  %accessing cell elements

Here we go :

cellarray = textread('Put directory here','%s',9);

for i=1:length(cellarray)
parsed_cell{i} = regexp(cellarray, ',', 'split'); %% Parsing 1d cell
end

%% 3 items which you wanted for e.g. the first line
line1 = parsed_cell{1};
line1{1}{3} = []; %% Deleting session id in line 1 of the text.
                  %%easily can be implemented for others

%% putting user_id,video_id into number id  in line 1. 
numberid = [line1{1}{1},line1{1}{2}];
user_73471960
Related