Let's start from the state array...
As argument, use a function instead a direct value. It works the same, but it prevents to recreate uselessly an array (then fill it), which will be trashed on every render.
const [checkedState, setCheckedState] = useState(
() => new Array(toppings.length).fill(false)
);
Then, count how many are currently selected:
const selectedCount = checkedState.reduce((sum, currentState, index) => {
if (currentState === true) {
return sum + 1;
}
return sum;
}, 0);
Finally, prevent further selections on the checkboxes which are still selectable:
<input
type="checkbox"
id={`custom-checkbox-${index}`}
name={name}
value={name}
checked={checkedState[index]}
onChange={(e) => handleOnChange(index, e)}
disabled={!checkedState[index] && selectedCount >= 3}
/>