SendGrid inbound parse nordic chars

Question

Completely stuck on a problem related to the inbound parse webhook functionality offered by SendGrid: https://sendgrid.com/docs/for-developers/parsing-email/setting-up-the-inbound-parse-webhook/

First off everything is working just fine with retrieving the mail sent to my application endpoint. Using Request.Form I'm able to retrieve the data and work with it.

The problem is that we started noticing question mark symbols instead of letters when recieving some mails (written in swedish using Å Ä and Ö). This occured both when sending plaintext mails, and mails with an HTML-body.

However, this only happens every now and then. After a lot of searching I found out that if the mail is sent from e.g. Postbox or Outlook (or the like), and the application has the charset set to iso-8859-1 that's when Å Ä Ö is replaced by question marks.

To replicate the error and be able to debug it I set up a HTML page with a form using the iso-8859-1 encoding, sending a similar payload as the one seen in the link above (the default one). And after that been through testing a multitude of things trying to get it to work.

As of now I'm trying to recode the input, without success. Code I'm testing:

        Encoding wind1252 = Encoding.GetEncoding(1252);
        Encoding utf8 = Encoding.UTF8;
        byte[] wind1252Bytes = wind1252.GetBytes(Request.Form.["html"]);
        byte[] utf8Bytes = Encoding.Convert(wind1252, utf8,wind1252Bytes);
        string utf8String = Encoding.UTF8.GetString(utf8Bytes);

This only results in the utf8String producing the same result with "???" where Å Ä Ö should be. My guess here is that perhaps it's due to the Request.Form["html"] returning a UTF-16 string, of the content that is encoded already in the wrong encoding iso-8859-1.

The method for fetching the POST is as follows

public async Task FetchMail(IFormCollection form)
{
        InboundParseModel _em = new InboundParseModel
        {
            To = form["to"].SingleOrDefault(),
            From = form["from"].SingleOrDefault(),
            Subject = form["subject"].SingleOrDefault(),
            Html = form["html"].SingleOrDefault(),
            Text = System.Net.WebUtility.HtmlEncode(form["text"].SingleOrDefault()),
            Envelope = form["envelope"].SingleOrDefault()
        };
}

Called from another method that the POST is done to by FetchMail(Request.Form);

Project info: ASP.NET Core 2.2, C#

So as stated earlier, I am completely stuck and don't really have any ideas on how to solve this. Any help would be much appreciated!

SendGrid inbound parse nordic chars

0 Answers