I'm trying to decode CSV files encoded in UTF-16BE in Golang. What is the charmap ISO character number that I have to call for the new reader ?
I want to invoke
csv.NewReader(charmap.XXXX.NewDecoder().Reader(file))
What should be the value of XXXX ?
Have you tried this?
https://godoc.org/golang.org/x/text/encoding/unicode#UTF16
unicode.UTF16(BigEndian, UseBOM)
After some review, a simple way to decode UTF16 into UTF8 is provided by this code:
https://gist.github.com/bradleypeabody/185b1d7ed6c0c2ab6cec#file-gistfile1-go
You can use golang.org/x/text/encoding/unicode.UTF16 to create a decoder from your target UTF-16 Little/Big-Endian encoding into UTF-8.
The code below shows a working example for UTF-16 LE (Go playground):
dec := unicode.UTF16(unicode.LittleEndian, unicode.UseBOM).NewDecoder()
utf16r := getUTF16LittleEndianCSVReader()
utf8r := transform.NewReader(utf16r, dec)
csvr := csv.NewReader(utf8r)
records, err := csvr.ReadAll()
// TODO: handle err
fmt.Printf("%#v", records)
// [][]string{[]string{"id", "name"}, []string{"1", "foo"}}
Switching to Big-endian should be as simple as below:
enc := unicode.UTF16(unicode.BigEndian, unicode.UseBOM).NewDecoder()