Weird output when parsing special characters using Html Agility Pack

Viewed 291

I'm trying to scrape a french website but the french langage have some special characters like é and à etc and I end up with some weird output for exemple instead of that (Inspect in Google Chrome)

<a href="www.exemple.com/1" title="Soins Anti-âge et Anti-rides">
  <span>Soins Anti-âge et Anti-rides</span>
</a>

<a href="www.exemple.com/2" title="Rides installées">
  <span>Rides installées</span>
</a>

I will get this output (Text Visualizer in Visual Studio)

<a href="www.exemple.com/1" title="Soins Anti-&acirc;ge et Anti-rides">
  <span>Soins Anti-&acirc;ge et Anti-rides</span>
</a>

<a href="www.exemple.com/2" title="Rides install&eacute;es">
  <span>Rides install&eacute;es</span>
</a>

My code is the following

string url = "https://universparadiscount.ma/7-product.html";
HttpClient httpClient = new HttpClient();
string html = await httpClient.GetStringAsync(url);
HtmlDocument htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(html);

string SubCategory = htmlDocument.DocumentNode.Descendants("span")
    .Where(node => node.GetAttributeValue("itemprop", string.Empty)
    .Equals("title")).LastOrDefault()?.InnerText.Trim('\r', '\n', '\t', ' ');
Console.WriteLine(SubCategory);
1 Answers

If I understand the problem, you can use HttpUtility.HtmlDecode

Converts a string that has been HTML-encoded for HTTP transmission into a decoded string.

To encode or decode values outside of a web application, use the WebUtility class.

Example

Console.WriteLine(SubCategory);
Console.WriteLine(HttpUtility.HtmlDecode(SubCategory));

Output

Rides install&eacute;es
Rides installées
Related