Translate Unicode mathematical bold/italic characters to latin-1 in Perl

user_700771 0
Viewed 301

Unicode has separate characters for bold or italic characters, e.g. U+1D43B is an italic H. See https://unicode-search.net/unicode-namesearch.pl?term=mathematical for list of these.

When a user copies a chemical formulae from an electronic textbook, they may actually be copying these characters instead of the Latin-1 characters, so instead of "H2O" they are copying "U+1D43B U+2082 U+1D442". It looks like H2O when they paste it in the search form. But they'll get no results because it's not latin characters.

So, I need to translate these characters to Latin-1 characters in Perl. The Text::Unidecode library doesn't seem to recognise these.

I tried using transliteration,

 y/\x{1d434}-\x{1d467}/A-Za-z/

but this doesn't seem to work at all.

Is there a way to use the translation operator on unicode character ranges? Or is there a library that will do this?

unicode perl
2 Answers

Actually it does work.

use open ':std', ':encoding(UTF-8)';

my $s = "\N{U+1D43B}\N{U+2082}\N{U+1D442}";
say sprintf "%vX", $s;
$s =~ y/\x{1d434}-\x{1d467}/A-Za-z/;
say sprintf "%1\$vX %1\$s", $s;

Output:

1D43B.2082.1D442
48.2082.4F H₂O

Perhaps you don't actually have the three-character string you describe? Perhaps you have the text encoded using UTF-8 instead?

user_5899240

Actually, the transliteration operator does work. It was a matter of ensuring the variable was correctly encoded.

$str =~ y/\x{1d400}-\x{1d6a3}/A-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-zA-Za-z/r
  =~ y/\x{1d7ce}-\x{1d7ff}/0-90-90-90-90-9/r );

Seems to work.

However, Unicode::Normalize::NDKD works better. Thanks.

user_7007710
Related