golang sort.Strings vs unix sort, how do I sort according to a given collation?

user_657703 0
Viewed 218

I was puzzled for a while because the two methods of sorting behave differently, or at least in my environment. After a while, I realized it's because of different sorting rules.

A quick experiment illustrates my point:

$ foo() { echo "o'4" "o'neil" o-ciclo-music ó-do-forró  o-1 o-2 o-3 "o'" "o'1"  "o'2" aaa b bb; } 
$ foo | s2n | LC_ALL=C  sort  > /tmp/s.c
$ foo | s2n | LC_ALL="en_US.UTF-8" sort  > /tmp/s.u
$ paste /tmp/s.{c,u}  | awk '{ printf "%16s %s\n", $1 , $2 ; }'
             aaa aaa
               b b
              bb bb
              o' o'
             o'1 o'1
             o'2 o-1
             o'4 o'2
          o'neil o-2
             o-1 o-3
             o-2 o'4
             o-3 o-ciclo-music
   o-ciclo-music ó-do-forró
      ó-do-forró o'neil

The left column is sorted according to ascii-rules while the right column is sorted according to utf8 rules. The golang sort.Strings() behaves according to ascii rules. That function is based on the < and = operators and they in turn operates on integers. So the words are being sorted according to the ascii ordinal values of their letters.

It it possible to influence how sort.Strings() behaves based on some environment variable? I have tried with the LC_ALL variables, but the 'a' < 'b' behavior in Golang is not influenced by that. This is probably for the better.

I am wondering which package provides a function with the same signature (and return type) as strings.Compare(), but will heed some arbitrary collation rule-set (or at least unicode).

I want a function that indicates that o'neil should come after ó-do-forró.

       foobar.SetCollation(unicode)
       foobar.Compare(`o'neil`, `ó-do-forró`) -> 1

What package am I looking for?

go collation sorting
1 Answers
import  "golang.org/x/text/collate"

(...)

cull := collate.New(language.English,
    collate.IgnoreCase, collate.IgnoreDiacritics, collate.IgnoreWidth, 
    collate.Loose,
    collate.Force,
    collate.Numeric)
println(cull.CompareString( "o'neil", "ó-do-forró", )) // prints 1
println(cull.CompareString( "b", "a", )) // prints 1
user_6577030
Related