Rails 5.1.: destroy records in "has_many: through" association with restriction

I have classic has_many: through relationship:

class UserGroup < ApplicationRecord
  has_many :user_groups_users
  has_many :users, through: :user_groups_users
end

class UserGroupsUser < ApplicationRecord
  belongs_to :user_group
  belongs_to :user
end

class User < ApplicationRecord
  has_many :user_groups_users
  has_many :user_groups, through: :user_groups_users
end

and in order to destroy UserGroup record, I need to destroy appropriate records in UserGroupsUser, which both is part of a gem. Otherwise I will get back error that there are Users tied to UserGroups and I cannot destroy particular UserGroup.

At the moment in my Controller I have this:

  def destroy
    @user_group = UserGroup.find(params[:id])
    UserGroupsUser.where(user_group_id: @user_group).destroy_all
    respond_to do |format|
    if @user_group.destroy
      format.js { flash.now[:notice] = "User group #{@user_group.name} deleted!" }
      format.html { redirect_to user_groups_url }
      format.json { head :no_content }
    else
      format.js { flash[:danger] = "User group #{@user_group.name} cannot be deleted because
                                   #{@user_group.users.size} users belong to it" }
    end
    end
  end

however when I click Delete button in View, it destroys a record before I accept that in my modal window. How do I make it do destroy action, after accept in view, please? As I undestand it would require that after accept, it would firs destroy records in through models and then UserGroup.

My "Delete" action in View is quite regular:

<%= link_to 'Delete', user_group, method: :delete, remote: true, 
data: { confirm: "Do you confirm deleting #{user_group.name}?" }, class: 'btn-danger btn btn-xs' %>