How can I track down a memory leak in a rails app?

Viewed 425

I have a class in my Rails application that evaluates where in a business process an application is. To do this I have a concept of a Task model, which uses single table inheritance to provide polymorphism. Each sub-class of Task can determine if it is completed by performing its own database queries, usually on associated models through a polymorphic association.

Every time this evaluator is ran more memory is consumed and does not get released even after manual calls to GC.start. Using ObjectSpace.object_counts I have produced a chart that shows the most strongly correlated object types that grow with the TOTAL count. T_IMEMO grows the fastest, with T_STRING coming in second.

Stats from ObjectSpace.object_counts

Memory profilers tell me that most memory is getting allocated to ActiveRecord. But between iterations there are no new ActiveRecord objects that I keep around. The code is roughly:

CustomerApplication.all.in_batches(of: 10).each do |batch|
  batch.each do |customer_application|
    Evaluator.new(customer_application).evaluate!
    GC.start
    # collect statistics
  end
end

To my knowledge I'm not using any class attributes or globals where data could be stored between iterations.

I've also tried disabling ActiveRecord logging and clearing the ActiveRecord cache on each iteration.

The core of the Evaluator code looks like this:

class Evaluator
  # ...

  def stage(name, &block)
    stage = Stage.new(@customer_application)
    stage.instance_eval(&block) if block_given?
    @stages << stage
  end

  def evaluate!
    @stages = []

    stage :first_stage do
      step :first_step do
        task(Task::SpecificTask)
        phantom_task(Task::ReportTask, reference: ..., key: 'report_a')
      end
    end

    stage :second_stage do
      step :first_step do
        task(Task::ReportTask, reference: ..., key: 'report_a')
      end

      step :second_step do
        task(Task::ReportTask, reference: ..., key: 'report_b')
      end
    end

    # A ton more stages/steps/tasks

    Task.transaction do
      self.tasks.each do |task|
        task.enabled = true
        task.evaluate_completion!
        task.evaluate_optional!

        task.save!
      end

      # Disable all other tasks
    end
  end
end

class Stage
  # ...

  private

  def step(name, reference: nil, deps: [], &block)
    step = Step.new(name, @customer_application, reference, deps)
    step.instance_eval(&block) if block_given?
    @steps << step
    step
  end
end

class Step
  # ...

  private

  def task(klass, reference: nil, key: nil)
    existing_task = @customer_application.tasks.find_by(type: klass.name, reference: reference, key: key)

    task = (existing_task || klass.new(customer_application: @customer_application, reference: reference, key: key))

    @tasks << task

    task
  end

  def phantom_task(klass, reference: nil, key: nil)
    readonly_klass =
        Class.new(klass) do
          def self.name
            superclass.name
          end

          def readonly?
            true
          end

          def optional?
            true
          end

          def phantom_task?
            true
          end
        end

    task = readonly_klass.new(customer_application: @customer_application, reference: reference, key: key)

    @tasks << task

    task
  end
end
1 Answers

I managed to figure it out. I was instantiating runtime classes on each iteration which apparently do not get GC'd. Refactoring to not use Class.new fixed the problem.

So if anyone's googling this, Class.new creates memory leaks.

Related