I have had problems with regular expression and Cyrillic letters in the past so I was wondering whether there was something I am doing wrong?
Here are two reproducible examples:
Example 1 - Issue with lookahead and lookbehind assertions:
latin <- "city New York, Manhattan\n1st Avenue"
cyrilic <- "град Ню Йорк, Манхатън\n1во Авеню"
stringr::str_extract(latin, pattern = "(?<=city New York, )[\\w\\s]+(?=\n)")
#returns: Manhattan
stringr::str_extract(cyrilic, pattern = "(?<=град Ню Йорк, )[\\w\\s]+(?=\n)")
stringr::str_extract(cyrilic, pattern = "(?<=град Ню Йорк, ).+(?=\n)")
#both return: NA
Example 2 - issue with grep's ignore.case = TRUE:
randomWord <- "Човек"
grep(pattern = "човек", x = randomWord, ignore.case = T)
#returns: integer(0)
Any ideas of how to write the regexes in order for them to work in cyrillics?
My default text encoding is UTF-8 and here is my sessionInfo:
> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
locale:
[1] LC_COLLATE=Bulgarian_Bulgaria.1251 LC_CTYPE=Bulgarian_Bulgaria.1251
[3] LC_MONETARY=Bulgarian_Bulgaria.1251 LC_NUMERIC=C
[5] LC_TIME=Bulgarian_Bulgaria.1251