As of Rails 5, when is :inverse_of necessary to set manually?

If you do not set the :inverse_of record, the association will do its best to match itself up with the correct inverse. Automatic inverse detection only works on has_many, has_one, and belongs_to associations.
-- Rails API docs - ActiveRecord::Associations::ClassMethods

In cases where you have "non-standard" naming you may however need to provide the option:

The automatic guessing of the inverse association uses a heuristic based on the name of the class, so it may not work for all associations, especially the ones with non-standard names. -- Rails API docs - ActiveRecord::Associations::ClassMethods

For example:

class Pet < ApplicationRecord
  belongs_to :owner, class_name: 'User'
end

class User < ApplicationRecord
  has_many :pets
end

If we call pet.owner it may incur a database hit even if we have already loaded that record.

If we add the inverse_of option:

class Pet < ApplicationRecord
  belongs_to :owner, class_name: 'User', inverse_of: :pets
end

class User < ApplicationRecord
  has_many :pets, inverse_of: :owner
end

Now if we already have that owner record in memory then pet.owner will point to the same owner.

In general there is no harm in explicitly setting the inverse_of so you can set it for each case where you are unsure. You can also manually test if it is needed by seeing if accessing the assocation creates a DB query via the console or by using shoulda-matchers with your test suite.

You can look up exact inferring procedure in ActiveRecord::Reflection::AssociationReflection#automatic_inverse_of.

Rails tries to detect inverse for has_many, has_one and belongs_to that do not have through, foreign_key options and scope and have standard names (this is also described in official guide).