I am currently working with the Telegram API, and in one of its methods it returns the following information:
- a piece of text
- offset in UTF-16 Code Units
- length in UTF-16 Code Units
In my programming language, Rust, all strings are valid UTF-8. This means that UTF-16 offsets are not immediately useful as they can be off by a variable amount (due to 1 or 3 byte code points). A one-byte code-point in UTF-8 corresponds to a two-byte one in UTF-16, so I cannot simply index the UTF-8 string as I may lay outside of code-point boundaries.
I am wondering now: Is there a way to convert it to valid UTF-8 without iterating through the UTF-8 string or if the information is useless once in UTF-8?