Why do event handlers need to be references and not invocations?

In the React tutorial, it says

Doing onClick={alert('click')} would alert immediately instead of when the button is clicked.

class Square extends React.Component {
  render() {
    return (
      <button className="square" onClick={() => alert('click')}>
        {this.props.value}
      </button>
    );
  }
}

However, I cannot understand why that would be... can anyone clarify this for me please? Why can't you pass a function invocation as a handler?