How to replace the first few bytes of a file in Ruby without opening the whole file?

user_1493116 0
Viewed 68

I have a 30MB XML file that contains some gibberish in the beginning, and so typically I have to remove that in order for Nokogiri to be able to parse the XML document properly.

Here's what I currently have:

    contents = File.open(file_path).read
    if contents[0..123].include? 'authenticate_response'
      fixed_contents = File.open(file_path).read[123..-1]
      File.open(file_path, 'w') { |f| f.write(fixed_contents) }
    end

However, this actually causes the ruby script to open up the large XML file twice. Once to read the first 123 characters, and another time to read everything but the first 123 characters.

To solve the first issue, I was able to accomplish this:

contents = File.open(file_path).read(123)

However, now I need to remove these characters from the file without reading the entire file. How can I "trim" the beginning of this file without having to open the entire thing in memory?

ruby
1 Answers

You can open the file once, then read and check the "garbage" and finally pass the opened file directly to nokogiri for parsing. That way, you only need read the file once and don't need to write it at all.

File.open(file_path) do |xml_file|
  if xml_file.read(123).include? 'authenticate_response'
    # header found, nothing to do
  else
    # no header found. We rewind and let nokogiri parse the whole file
    xml_file.rewind
  end

  xml = Nokogiri::XML.parse(xml_file)
  # Now to whatever you want with the parsed XML document
end

Please refer to the documentation of IO#read, IO#rewind and Nokigiri::XML::Document.parse for details about those methods.

user_4217050
Related