Adding and removing records with a many to many relationship in Ecto

I have 2 ecto schema models that have a man_to_many relationship.

schema "Users" do

  many_to_many :locations, Location, join_through: "locations_users"
end

schema "Locations" do
  many_to_many :users, User, join_through: "locations_users"
end

So in my User.ex schema model file how should I go about writing a function to add and also to remove a user?

def add_location(location) do
  # ???
end

def remove_location(location) do
  # ???
end

In the ecto docs it says you shouldn't have to load all the associated data before adding/removing an association item, you can just set the association. But since I am using a many to many, I'm not sure how I can just set the association since it is a join table only.