I am trying to solve a problem where I have a string with mixed language as input.
E.g. "Hyundai Motor Company 현대자동차 现代 Some other English words"
And I want to split the string into different language chunks.
E.g. ["Hyundai Motor Company", "현대자동차", "现代", "Some other English words"]
OR (Space/Punctuation marks and order do not matter)
["HyundaiMotorCompany", "현대자동차", "现代", "SomeotherEnglishwords"]
Is there an easy way to solve this problem? Or any assembly/nuget package I can use?
Thanks
Edit: I figured that my "language chunks" is ambiguous. What I want by "language chunks" is language character sets.
For example "Hyundai Motor Company" is in English character set, "현대자동차" in Korean set, "现代" in Chinese set, "Some other English words" in English set.
Additions to clarify the requirement of my problem is:
1: The input can have spaces or any other punctuation marks, but I can always use regular expressions to ignore them.
2: I will pre-process the input to ignore Diacritics. So "å" becomes "a" in my input. So all the English like characters will become English characters.
What I really want is to find a way to parse the inputs into different language-character sets ignoring spaces & punctuation marks.
E.g. From "HyundaiMotorCompany현대자동차现代SomeotherEnglishwords"
To ["HyundaiMotorCompany", "현대자동차", "现代", "SomeotherEnglishwords"]