How to prevent Arabic characters from being re-encoded by AntiXssEncoder?

user_6787523 0
Viewed 686

We are using the System.Web.Security.AntiXss.AntiXssEncoder class provided by the .NET Framework (we target .NET Framework 4.5.2) in our application, but are running into issues with fields containing Arabic characters.

The following console application demonstrates the issue that we have:

using System;
using System.Collections.Generic;
using System.Web.Security.AntiXss;

namespace EncodingTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var source = new List<string> { "Hello World", "على", "blöd", "&#1575;&#1604;&#1605;" };

            foreach (var testString in source)
            {
                var antiXssEncoded = AntiXssEncoder.HtmlEncode(testString, false);
                Console.WriteLine($"{testString} => {antiXssEncoded}");

                Console.WriteLine();
            }

            Console.ReadKey();
        }
    }
}

The Arabic characters in the second list item are correctly encoded, but if the already encoded characters in the forth element are passed through the encoder, then the '&' characters are encoded a second time into &amp; which then doesn't display correctly on a webpage.

This output from the application shows this (The unencoded Arabic characters display as '???' in the console):

Hello World => Hello World

??? => &#1593;&#1604;&#1609;

blöd => blöd

&#1575;&#1604;&#1605; => &amp;#1575;&amp;#1604;&amp;#1605;

Is there any way to prevent this?

We currently take user input and pass it through the encoder before it is saved in our database - this encoded data is then sent to the frontend and displayed. If the user edits the string and it is passed back to our backend we encode it again before saving it, so we get the problem with the ampersands.

I've seen some comments on other questions saying that the user input should be saved in the database as it comes in and then passed through the encoder before it is sent for display; we could do this, but would then have to make sure that this was encoded in all of the different places that the data is output. It wouldn't help with the problem of the edited data coming back from the frontend either - we'd still not know if the data was encoded or not and would just re-encode it the next time it was sent for display.

Is there some way to make the encoder not re-encode the characters like &#1575; into &amp;#1575;, or are we just doing something wrong?

asp.net-web-api encoding xss .net c#
2 Answers

This is expected behavior and your should not try to prevent it.

Strings passed to HtmlEncode are normally provided by a user of your program and should end up appearing in HTML in exact way passed to the method. This imply all & (and similar cases) need to be encoded. E.g. if user entered something like "use '&amp;' to represent '&' in HTML" you really would not expect "use '&' to represent '&' in HTML" in the rendered HTML.

How to approach the problem:

You really must know what is encoding of content of the string you are using. There is absolutely no way to know whether string was HTML-encoded at all or how many times. The same applies to url encoding (also known as %-encoding like ?q=search%20alot) and JavaScript-string-value encoding (like I said \"Hi Matt\"\nin this is long & winded post) .

This means you either need to store strings with consistent encoding or store type of encoding applied to the sting along with each string (i.e. next column in DB). Here are descriptions I'd use for different cases: "this is plain text, non-encoded", "this is ready to render HTML, sanitized", "this is raw HTML provided by user, non-sanitized", "this is plain text encoded as JavaScript string (using \n)".

There is significant danger of mis-classifying string as "ready to render HTML, don't HtmlEncode" - you can easily render rogue user's input as HTML and cause cross-site-scripting (XSS) vulnerability. Over-encoding behavior you got is safer also can produce useless output - so demand that string data comes with well defined encoding.

Note that common usage of HtmlEncode method is to eventually render it as raw HTML - depending on framework you are using there could be better way to deal with it. If you are using ASP.Net MVC just regular @Model.Text will provide all necessary encoding.

As one of the ways to specify encoding you can use HtmlString class to declare "the string value represents raw HTML that can be safely rendered in HTML as-is". It could be good idea to store output of HtmlEncode in variable/property of type HtmlString so everyone knows that value is sanitized and should be rendered directly.

user_4774200

For me adding this did the trick.

AntiXssEncoder.MarkAsSafe(LowerCodeCharts.Arabic | LowerCodeCharts.Hebrew,
                LowerMidCodeCharts.None, MidCodeCharts.None, UpperMidCodeCharts.None, UpperCodeCharts.None);

It resolved the "? issue" for me and it did parse the arabic symbols the right way.

Note that this should be added in the "StartUp" (or similar files that are executed on application start time - e.g. Global.asax in old framework projects) to configure it once.

user_118967330
Related