How to enforce eager loading of associations when importing records using Searchkick with Rails4?

user_56082 0
Viewed 494

For this setup with default (unspecified) search_data:

class Item < ActiveRecord::Base
 searchkick
 has_many :quantities, dependent: :destroy
 scope :search_import, -> { includes(:quantities) }

end

When importing all database records by running Item.reindex, each "batch" eager loads the quantities for all of the item ids as expected.

However, if I want to specify the index model document differently than the default attributes using the search_data method including the associated model data with something like this:

class Item < ActiveRecord::Base
 searchkick
 has_many :quantities, dependent: :destroy

 def search_data
   {
     part_number: standard_part_number,
     category_id: category_id.to_i,
     content_source_name: content_source_name,
     standard_price: standard_price.to_i,
     locations: quantities.map {|q| [q.location_code,
       quantity: q.value.to_i,
       bins: q.location_bins]
     }.to_h
   }
 end

 scope :search_import, -> { includes(:quantities) }

end

where I am operating on the quantities using map to define a locations attribute, returning to import using Item.reindex I see that it not only eager loads all of the associated quantities each batch, but it also then loads all quantities per item with a hit to the database again for each item record it is indexing.

Am I missing something here to get Searchkick to eager load the quantities models and be able to customize the search data record using that already loaded associated model without it doing another pull from the database again per item?

Update I've determined there is something interfering with the way the ActiveRecord normally responds to the method name used for the association with the eager loaded models in our app and may not be exclusively related to us using Searchkick, although it did reveal the issue. I am not sure what it is that is interfering at this time but it has something to do with the target method on the association. Perhaps a gem that is installed is causing this problem. I did manage to find a way to work around this (for now) using item.association(:quantities).target as a replacement for item.quantities and now when running the reindex it makes use of the already eager loaded quantities and doesn't hit the db again for each item

searchkick ruby-on-rails
1 Answers

I see that it not only eager loads all of the associated quantities each batch

This is expected behaviour (and very likely performant), since each batch you will get different quantities to load, since they are connecting to different items, so you don't need to keep all quantities in memory.

each batch, but it also then loads all quantities per item with a hit to the database again for each item record it is indexing.

This is actually unexpected, but my guess here is, that in one of the methods in Quantity (#location_code #value or #location_bins) or even one of the methods on Item that you call (#standard_part_number, #category_id, #content_source_name, #standard_price), there is some implementation, that requires reloading of records. Without knowing the code of that methods it is purely speculative, but the presented part of the code looks fine.

user_23896280
Related