Scraping and storing in CSV in Ruby

user_8703414 0
Viewed 379

I am a Ruby-newbie and I tried my first scraper today. It's a scraper designed to store recipes in a CSV file. Nevertheless, I can't figure out why it doesn't work. here is my code:

recipe.rb :

require 'csv'
require 'nokogiri'
require 'open-uri'


def write_csv(ingredient)

doc = Nokogiri::HTML(open("http://www.marmiton.org/recettes/recherche.aspx?aqt=#{ingredient}"), nil, 'utf-8')
  doc.search(".m_contenu_resultat").first(10).each do |item|
    name = item.search('.m_titre_resultat a').text
    description = item.search('.m_texte_resultat').text
    cooking_time = item.search('.m_detail_time').text
    diff = item.search('.m_detail_recette').text.split('-')
    difficulty = diff[2]
    recipes = [name, description, cooking_time, difficulty]
    CSV.open('recueil.csv', 'wb') do |csv|
      csv << recipes
    end
  end
end

write_csv('chocolat')

Thank you so much for your answers, it'll help me a lot !

csv ruby
3 Answers

IT WORKED ! I changed my code as below, using a hash :

require 'csv'
require 'nokogiri'
require 'open-uri'


def write_csv(ingredient)
recipes= []
doc = Nokogiri::HTML(open("http://www.marmiton.org/recettes/recherche.aspx?aqt=#{ingredient}"), nil, 'utf-8')
  doc.search(".m_contenu_resultat").first(10).each do |item|
    name = item.search('.m_titre_resultat a').text
    description = item.search('.m_texte_resultat').text
    cooking_time = item.search('.m_detail_time').text
    diff = item.search('.m_detail_recette').text.split('-')
    difficulty = diff[2]
    recipes << {
    name: name,
    description: description,
    difficulty: difficulty
  }
end

  CSV.open('recueil.csv','a') do |csv|
  csv << ["name", "description", "cooking_time", "difficulty"]
  recipes.each do |recipe|
    csv << [
      recipe[:name],
      recipe[:description],
      recipe[:cooking_time],
      recipe[:difficulty]
    ]
  end
end
end

write_csv('chocolat')
user_87034140

When you are opening your CSV file you are overwriting the previous one every time. You should eighter append to the file like this:

CSV.open('recueil.csv', 'a') do |csv|

or you could open it before you start looping like this:

def write_csv(ingredient)
  doc = Nokogiri::HTML(open("http://www.marmiton.org/recettes/recherche.aspx?aqt=#{ingredient}"), nil, 'utf-8')
  csv = CSV.open('recueil.csv', 'wb')
  doc.search(".m_contenu_resultat").first(10).each do |item|
    name = item.search('.m_titre_resultat a').text
    description = item.search('.m_texte_resultat').text
    cooking_time = item.search('.m_detail_time').text
    diff = item.search('.m_detail_recette').text.split('-')
    difficulty = diff[2]
    recipes = [name, description, cooking_time, difficulty]
    csv << recipes
  end
  csv.close
end
user_50103700

You don't specify what doesn't work, what the result of the errors are, so I must speculate.

I tried your script and had difficulties with the encoding, since the site is in french, there are lots of special characters.

Try again with this at the head of your script, it should solve at least that problem.

# encoding: utf-8
Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8
user_9233150
Related