Delegate on multiple objects didn't work the way I thought

I had an idea to use delegates for my buttons whenever the button is pressed I can notify everthing listening/subscribed(?) to it and do stuff (I feel like this is a scenario where delegates aren't needed). I have this code attached to 3 objects

public class ClassSelectButton : MonoBehaviour
{
    public delegate void ClassSelectDelegate();
    public event ClassSelectDelegate classSelectedEvent;

    public BaseClass classToGive;

    void Awake()
    {
        var button = GetComponent<Button>();
        if (button)
            button.onClick.AddListener(() => OnButtonPressed());
    }

    void OnButtonPressed()
    {
        if (classSelectedEvent != null)
            classSelectedEvent();
    }
}

public class Player : MonoBehaviour
{
    public BaseClass playerClass;

    void Start()
    {
        FindObjectOfType<ClassSelectButton>().classSelectedEvent += Test;
    }

    void Test(BaseClass newClass)
    {
        playerClass = newClass;
    }
}

Turns out this doesn't work the way I thought it would because when pressing the button only one of the delegates work and I realized that since there's multiple instances of this script there's duplicates of the delegate and so only one of them is being listened to. So what's an alternative way to go about doing this?

p.s: As I was writing this I realized another problem is probably because I'm using FindObjectOfType