Convert diacritic or modifier diacritic Unicode character to 'combining form'

user_44389 0
Viewed 230

I have some accents/diacritics in a Unicode string that I want to convert into their 'combining form' (per the Unicode standard). For example, if I have \N{CIRCUMFLEX ACCENT} (\u005E) or \N{MODIFIER LETTER CIRCUMFLEX ACCENT} (\u02C6), I want to convert it to \N{COMBINING CIRCUMFLEX ACCENT} (\u0302). Is there any consistent, reliable method for doing this for any diacritical mark? If the Python (3.9) standard library has in-built support for it (perhaps via the unicodedata module), that would be ideal, but I'm happy to use a pip package for this too.

unicode text python
1 Answers

This is my current best solution. It feels a bit hacky still, but it seems to work in the scenarios I encounter.

from typing import *
import unicodedata

def _strip_prefix(s: str, prefix: str) -> str:
    return s[len(prefix):] if s.startswith(prefix) else s

def make_combining_form(diacritic: str) -> Optional[str]:
    if unicodedata.category(diacritic) not in ("Sk", "Lm"):
        return None

    name = unicodedata.name(diacritic)
    name = _strip_prefix(name, "MODIFIER LETTER ")
    name = _strip_prefix(name, "COMBINING ")
    try:
        return unicodedata.lookup("COMBINING " + name)
    except KeyError:
        return None
user_443890
Related