Azure Key Vault - Validate JWT signed by 3rd party

I have created a Key on my Azure Key Vault using the same pfx a known 3rd party uses to sign the JWTs it emits.

But I don't know what im doing wrong but the validation is always coming back negative.

my code is:

var client = new CryptographyClient(new Uri(myKeyVaultAddress), clientSecretCredential);

var jwtByParts = jwt.Split(".");
var header = jwtByParts[0];
var body = jwtByParts[1];
var signature = Encoding.UTF8.GetBytes(jwtByParts[2]);


var byteData = Convert.FromBase64String(body.Length % 4 == 0 ? body : body.PadRight(body.Length + 4 - body.Length % 4, '='));
var hasher = new SHA256CryptoServiceProvider();
var digest = hasher.ComputeHash(byteData);

var res = await client.VerifyAsync(SignatureAlgorithm.RS256, digest, signature);

a valid JWT sample is:

eyJhbGciOiJSUzI1NiJ9.eyJhdXRoX3RpbWUiOjE2MzMzNjQxODAyNzksImV4cCI6MTYzNDY5MTg4ODE1Miwic3ViIjoibGFuZXRcL3ZpemVuZXppQGNhcmJvbi5zdXBlciIsImF6cCI6InE4aENMNkM5bU5EZDhBMHBEb3lQT2d5ckJuY2EiLCJhdF9oYXNoIjoiWTJFeVl6TmxNek00TXpZM01qQXpOelUyWVdZek5EUmpOek16WW1NM05tTT0iLCJhdWQiOlsicThoQ0w2QzltTkRkOEEwcERveVBPZ3lyQm5jYSJdLCJpc3MiOiJodHRwczpcL1wvZGV2LWRldi1hcGkudGlja2V0LmNvbS5iclwvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTYzNDY4ODI4ODE1Mn0.QuQaMdaQO9iCuvf6rbkroIXth4XjurxENSriHqnMUniIR2UwVK63JoWorDwTEIWdfBUkJr4unN6DYO0e96L4QmXvMoHLJ12xRzqFblZ_gPUtsREde6Qfw3RIbPO50Sj6t_Q2E61cLqytCwlVNsUQ9sZrlAfYIvQulec5PgMepqk-dQJYdqD3iZYb1Qqoek4LffxcqdYioqFo4--uarKJrFfbv8Kv8ukI7HPZ0TMZadKYmkgpzwWZJQqhYm6Jz4vGiyJqnFuXls7QoasG10V9OQN38zq7IywOdB7eH6q_QAzM2gF57Eg5EJ6U2YMu6BKn6f5Nialol7KcWaLIsHMLnA