How to remove hidden newline or carriage return character in vim

Viewed 559

I have a CSV file on macOS that was created by Windows that looks like this:

"C00115940","","","","","O","","","f","","b",NULL,"92583","","123-456-7890","","","","","","M","W","","","","jc@abc.def","","","","C","5","","S",NULL,"92583","","951","","","m@s.gov","","","","","","","","","",NULL,NULL,"",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"",NULL,NULL,"",NULL,NULL,"",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"",NULL,NULL,"",""," ","1","",NULL,"","","","","",NULL,"","75350","","","",NULL,"","","","","","","","","","",""¬
"C00102747","","","","","R","","","3","","A",NULL,"12207","","5","","","","","","S","D","","","","t@abc.def","","","","R","35 Any St¬
3rd Fl","","A",NULL,"12207","","5","","","S@r.org","","","","","","","","","",NULL,NULL,"",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"",NULL,NULL,"",NULL,NULL,"",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"",NULL,NULL,"",""," T","1","",NULL,"","","","","",NULL,"","75350","","","",NULL,"","","","","","","","","","",""¬

In vim, I have :set list enabled so you can see ¬ where the newlines are.

On the second line I have a newline in the middle of one of the fields that I need to remove. Every line should always end in a double-quote. If I do /[^"]$ I can see, for example, t¬ on the second line where the newline is inside the address field ("35 Any St¬ 3rd Fl")

I have tried :%s/\([^"]\)$/\1/g but that doesn't do anything.

Is there a better substitution command I can run that will remove these bad newlines but not any of the others?

1 Answers

You're close! You can use \n to match a new line in vim's search command, so you can highlight problematic lines with:

/[^"]\n

You could join them like so:

:%s/\([^"]\)\n/\1/g

or in a slightly more elegant way using a global command:

:g/[^"]\n/join

or equivalently:

:g/[^"]\n/normal J
Related