How to go about the .csv file which contains text lines with commas?

user_6021867 0
Viewed 78

I am using the read.delim function but since the text lines I am reading also contains comments where users have used commas (","), the comments get divided into two or more columns.

Below are two lines from the data set:

@Zillaman u just aite all types of food at Zina crib and didnt even think about me!!!!,0

I must have been only 11 when Mr Peepers started. It was a must see for the whole family, I believe on Sun...,1

The first line gets read properly. The "0" is read in the next column. The second line gets broken into three columns, with the last column containing the "1"

dataset_original = read.delim('TrainingData.csv', 
                              quote = "",
                              row.names = NULL, 
                              stringsAsFactors = FALSE,
                              header = F, as.is = F,
                              colClasses = "character",
                              blank.lines.skip = T,
                              sep = ",")
r
2 Answers

Try reading all the lines individually and then separating text and target columns afterwards.

Try this:

df= read.delim('TrainingData.csv',
               quote = "",
               row.names = NULL,
               stringsAsFactors = FALSE,
               header = F, as.is = F,
               colClasses = "character",
               blank.lines.skip = T,
               sep = "\n")


df$target = regmatches(df$V1, regexpr(pattern = "[^,]*$", text = df$V1))
df$V1 = sub(pattern = ",[^,]*$", replacement = "", x = df$V1)

where df stands for dataset_original

Example:

With a file containing:

hello,0
world,1
not,right,1
this,one,is,even,worse,0

This method returns:

> df
                      V1 target
1                  hello      0
2                  world      1
3              not,right      1
4 this,one,is,even,worse      0
user_17300170

If we read the file by using readLines(), we can afterwards split on the last comma.

write(x="@Zillaman u just aite all types of food at Zina crib and didnt even think about me!!!!,0

I must have been only 11 when Mr Peepers started. It was a must see for the whole family, I believe on Sun...,1", 
file="file.txt")

gg <- readLines("file.txt")

spl <- strsplit(gg, ",(?=[^,]+$)", perl=TRUE)
dtf <- as.data.frame(do.call(rbind, spl), stringsAsFactors=FALSE)

dtf
#                                                     V1  V2
# 1 @Zillaman u just (...) didnt even think about me!!!!   0
# 2 I must have been (...) family, I believe on Sun...     1
user_42727250
Related