How can I sum the values from a regexextract on multiple rows

I have 3 rows. All of them have a (number)&(letter) format. For example, Say 5&b is present on 3 rows. My question is:

How do I do sum just the numbers? I have tried:

  =SUM(REGEXEXTRACT(B3:B, "(.)&."))

I am trying to add the first value of all the rows.