Ruby on Rails - Arabic text sometimes being encoded, and others not - why and how to fix?

user_5079196 0
Viewed 301

I want to save a wikipedia link that has Arabic characters in it.

In my console I can do

> title = "The Broken Wings / الأجنحة المتكسرة"
 => "The Broken Wings / الأجنحة المتكسرة

And it returns properly with English and Arabic. But if I try to save a link, it encodes the Arabic characters.

When I try to enter this link : https://ar.wikipedia.org/wiki/الأجنحة_المتكسرة it changes to https://ar.wikipedia.org/wiki/%D8%A7%D9%84%D8%A3%D8%AC%D9%86%D8%AD%D8%A9_%D8%A7%D9%84%D9%85%D8%AA%D9%83%D8%B3%D8%B1%D8%A9

How can I save the link as-is?

arabic-support arabic urlencode ruby-on-rails
1 Answers

I assume by 'console' you mean the Rails console. Ruby (since 2.0) uses UTF-8 as its default character encoding. That means it can natively handle Universal Character Set (aka Unicode) characters. So, when you work with Arabic strings in the Rails console there is no transformation occurring.

However, for URIs (Uniform Resource Identifier -- simplistically the 'path' part of a URL), the standard (RFC3986) says that only US-ASCII and some special characters are allowed. You can use other characters to specify a location (what is called an Internationalized Resource Identifier), but only some systems natively understand IRIs. Otherwise, they get translated to a byte encoding called 'Percent-Encoding', which is what you see in the wikipedia URL.

This introduction does a more complete job of explaining how multi-lingual web addressing works and how translation between IRIs and URIs works with percent encoding.

user_103696470
Related