Is there a way to server-side detect a website character encoding?

usereKbm1 0
Viewed 28

I need to pull the HTML from any given website with a network request and display it, the problem is that by the differences in character encoding some websites content characters would not display the right way.

My current code look like this: (Have in count that this is a server-side code)

    const response = await fetch(url);
    const buffer = await response.arrayBuffer();
    const decoder = new TextDecoder("iso-8859-15");
    const decoded = decoder.decode(buffer);

However neither iso-8859-15 or utf-8 work for all the websites.

So I would need a dynamic way to know the correct one for each website

node.js javascript
1 Answers

For anyone with the same problem, here is my solution

    const response = await fetch(url);
    const headers = Object.fromEntries(response.headers.entries());
    const contentType = headers["content-type"];
    const charset = contentType.split("=")[1];
    const buffer = await response.arrayBuffer();
    const decoder = new TextDecoder(charset);
    const decoded = decoder.decode(buffer);
usereKbm10
Related