Incomplete final line found error when reading url using readLines() in R

user_8741108 0
Viewed 19753

I am trying to read the html content of a url using the readLines() function in R. However, I am getting a "incomplete final line found" warning message as shown below? How can I skip the final line in such a case? Any suggestions would be very much appreciated.

x <- readLines("https://in.finance.yahoo.com/industries/technology")

Warning message:
In readLines("https://in.finance.yahoo.com/industries/technology") :
  incomplete final line found on 'https://in.finance.yahoo.com/industries/technology'
web-scraping r
2 Answers

Most files are missing an End of Line marker like a new line below, so I would just use warn=FALSE. 

cat("abc\ndef\nhij", file="test.txt")
readLines( "test.txt")
# [1] "abc" "def" "hij"
# Warning message:
# In readLines("test.txt") : incomplete final line found on 'test.txt'
readLines( "test.txt", warn=FALSE)
# [1] "abc" "def" "hij"
user_27004940

I have encountered this issue many time. Most of the cases the following solution helped: "Opene the file in the editor, pressed "Enter" and saved".

However there was one time where this solution didn't work. So I set the option warn=FALSE in readLines() and the final result only stored few lines from the entire file. So here is what I did.

  1. I ran readLines with warn="FALSE". data <- readLines(file, warn=FALSE)
  2. I checked the length of "data" length(data) 7258
  3. I opened the file and went to the line number mentioned above i.e line number 7258. There was a weird looking character in the next line, which I removed. And removed that character from many other lines from the same file. And saved it.

I ran the readLines function again and the error was gone.

user_105680220
Related