How to generate fixtures based on my development database?

Viewed 10457

I'm lazy and I since my production database has data I could use for testing through on going development, I was wondering if there were any easy methods of generating fixtures.

5 Answers

In case you're creating a script to run under rails runner you can use the following approach:

File.open("#{Rails.root}/spec/fixtures/documents.yml", 'w') do |file|
  file.write Document.all.to_a.map(&:attributes).to_yaml
end

you can create as much blocks as you want, or if you want to go to the full database you can try:

models = defined?(AppicationRecord) ? ApplicationRecord.decendants : ActiveRecord::Base.descendants
models.each do |model|
  model_name = model.name.pluralize.underscore
  File.open("#{Rails.root}/spec/fixtures/#{model_name}.yml", 'w') do |file|
    file.write model.all.to_a.map(&:attributes).to_yaml
  end
end

if you do not want the timestamps you can change the code to: model.all.to_a.map { |m| m.attributes.except('created_at', 'updated_at')}.to_yaml

Building on @nikolasgd's answer I wrote a Rake Task, which may be useful for some:

# Use like "rake custom:create_test_fixtures[model]"
namespace :custom do
  desc 'Re-Creates Fixtures for Testing for a Model'

  task :create_test_fixtures, [:model] => [:environment] do |t, args|
    class ActiveRecord::Base
      def dump_fixture
        fixture_file = "#{Rails.root}/test/fixtures/#{self.class.table_name}.yml"
        File.open(fixture_file, "a+") do |f|
          f.puts({"#{self.class.table_name.singularize}_#{id}" => attributes}.
              to_yaml.sub!(/---\s?/, "\n"))
        end
      end
    end

    begin
      model = args[:model].constantize
      model.all.map(&:dump_fixture)
      puts "OK: Created Fixture for Model \"#{args[:model]}\"."
    rescue NameError
      puts "ERROR: Model \"#{args[:model]}\" not found."
    end

  end
end

I expanded the solution by @nikolasgd to also support ActiveStorage attachment fields/blobs and support naming dumped objects:

# config/initializers/generate_fixture.rb
class ActiveRecord::Base

  # Append this record to the fixture.yml for this record class
  def dump_fixture(name: nil, include_attachments: false)

    # puts "Dumping fixture for #{self.class.name} id=#{id} #{"with name #{name}" if name}"

    attributes_to_exclude = [:updated_at, :created_at, *Rails.application.config.filter_parameters].map(&:to_s)
    attributes_to_exclude << "id" if name != nil

    # puts "  Attributes excluded: #{attributes_to_exclude.inspect}"

    attributes_to_dump = attributes
      .except(*attributes_to_exclude)
      .reject { |k,v| v.blank? }

    name = "#{self.class.table_name.singularize}_#{id}" if name == nil

    self.dump_raw_fixture({ name => attributes_to_dump }.to_yaml.sub(/---\s?/, "\n"))

    if include_attachments != false
      self.class.reflect_on_all_attachments
        .each { |association|

          a_name = association.name
          Array(self.send(a_name.to_sym)).each_with_index { |attachment, index|

            attachment_name = "#{name}_#{a_name.to_s.underscore}_#{index}"
            blob_name = "#{attachment_name}_blob"

            attachment.dump_raw_fixture({ name => {
              "name" => a_name,
              "record" => "#{name} (#{self.class.name})",
              "blob" => blob_name
            }}.to_yaml.sub(/---\s?/, "\n"))

            blob = attachment.blob
            blob.dump_raw_fixture("#{blob_name}: <%= ActiveStorage::Blob.fixture(filename: '#{blob.filename}') %>\n")
            blob_path = "#{Rails.root}/test/fixtures/files/#{blob.filename}"
            File.open(blob_path, "wb+") do |file|
              blob.download { |chunk| file.write(chunk) }
            end
          }
        }
    end
  end

  def dump_raw_fixture(text)
    fixture_file = "#{Rails.root}/test/fixtures/#{self.class.name.underscore.pluralize}.yml"
    File.open(fixture_file, "a+") do |f|
      f.puts(text)
    end
  end

end

It requires the following test_helper (Rails 7 will obsolete this):

# test/test_helper.rb
class ActiveStorage::Blob
  def self.fixture(filename:, **attributes)
    blob = new(
      filename: filename,
      key: generate_unique_secure_token
    )
    io = Rails.root.join("test/fixtures/files/#{filename}").open
    blob.unfurl(io)
    blob.assign_attributes(attributes)
    blob.upload_without_unfurling(io)

    blob.attributes.transform_values { |values| values.is_a?(Hash) ? values.to_json : values }.compact.to_json
  end
end

You can run this from rails console as before:

User.find(1).dump_fixture name: bob, include_attachments: true

I went through the answers provided for this question. I have a particular use case for this (or special treatment if you will) therefore I didn't want to just dump for .all in all cases. I want to load limited records and the relationships will help me ensure foreign keys are preserved. I wrote a class for it that I use in a rake task. Here is the code (may be useful to someone):

module Tools
    class FixturesGenerator

        attr_accessor :tenant, :limit
        # An array of specific model names as symbols.
        # or a hash of record names and relationships as you would pass to <model>.includess(...)

        FIXTURES = [
            
        ].freeze


        def initialize(options = {})
            options.each do |key, value|
                send("#{key}=", value)
            end
        end

        def run
            send(:clear)

            ActsAsTenant.with_tenant(self.tenant) do
                save_model_as_yaml(self.tenant, 'tenants')
                FIXTURES.each do |item|
                    if item.is_a?(Hash)
                        item.each do |key, value|
                        generate_model_yaml(to_model_class(key), value, { limit: self.limit })
                        end
                    else
                        generate_model_yaml(to_model_class(item), nil, { limit: self.limit })
                    end
                end
            end
        end


        

        protected

        def to_model_class(name)
            name.to_s.singularize.split('_').map(&:capitalize).join('').constantize
        end

        def fixture_path(name)
            "#{Rails.root}/test/fixtures/#{name}.yml"
        end


        def clear
            build_fixture_names.each do |name|
                name = name.to_s.pluralize
                path = fixture_path(name)
                if File.exists?(path)
                    File.delete(path)
                end
            end
        end


        def build_fixture_names
            add_fixture_name([], FIXTURES) + [:tenant]
        end

        def add_fixture_name(list, item)
            if item.is_a?(Hash)
                item.each do |key, value|
                    list << key
                    list = add_fixture_name(list, value)
                end
            elsif item.is_a?(Array)
                item.each do |x|
                    list = add_fixture_name(list, x)
                end
            else
                list << item
            end
            list
        end

        def generate_model_yaml(model_class, associations = nil, options = {})
            limit = options.fetch(:limit, 0)
            table_name = model_class.table_name
            query = model_class
            
            unless associations.nil?
                query = query.includes(associations)
            end

            query = query.order(created_at: :desc)

            if limit > 0
                query = query.limit(limit)
            end

            records = query.all

            save_collection_as_yaml(records, table_name)

            save_associations_delegator(records, associations) unless associations.nil?
        end

        def save_associations_delegator(records, associations)
            if associations.is_a?(Hash)
                associations.each do |key, value|
                    save_associations(records, key)
                    local_records = records.collect {|d| d.send(key)}.find_all {|c| !c.nil? }
                    save_associations_delegator(local_records, value)
                end
            elsif associations.is_a?(Array)
                associations.each do |name|
                    save_associations_delegator(records, name)
                end
            else
                save_associations(records, associations)
            end
        end

        def save_associations(relation, name)
            table_name = name.to_s.pluralize

            items = relation.inject([]) do |list, record|
                model = record.send(name)
                if model.nil?
                    list
                else
                    if model.respond_to?(:length)
                        list = list + model
                    else
                        list << model
                    end
                end
                list
            end
            save_collection_as_yaml(items, table_name)
        end

        def model_attributes(model)
            blacklist_attributes = %w[created_at updated_at deleted_at]
            attrs = model.attributes.reject {|k, v| blacklist_attributes.include?(k.to_s)}
            attrs.keys.inject({}) do |env, key|
                env[key] = attrs[key].to_s
                env
            end
        end

        def for_yaml(model, table_name)
            h = {}
            key = "#{table_name.singularize}_#{model.id}"            
            h[key] = model_attributes(model)
            h
        end


        def collection_for_yaml(collection, table_name)
            h = {}
            entry_prefix = table_name.singularize
            collection.each do |model|
                key = "#{entry_prefix}_#{model.id}"
                h[key] = model_attributes(model)

            end
            h
        end

        def save_collection_as_yaml(records, table_name)
            File.open(fixture_path(table_name), "a+") do |f|
                f.write collection_for_yaml(records, table_name).to_yaml
            end
        end

        def save_model_as_yaml(model, table_name)
            File.open(fixture_path(table_name), "a+") do |f|
                f.write for_yaml(model, table_name).to_yaml
            end
        end
    end
end
Related