Named field with Umlaut not recognized with Cassava

user_768656 0
Viewed 131

I'm trying to parse a CSV file containing German text, i.e., it is not "comma" separated, but semicolon separated and it may contain Umlauts (äöü etc).

Using Cassava and following the linked tutorial, for a column with a header including an Umlaut, I'm getting the error:

parse error (Failed reading: conversion error: no field named "W\228hrung") at "\nEUR;0,99"

MRE:

{-# LANGUAGE OverloadedStrings, TypeApplications #-}

import Data.Char
import qualified Data.ByteString.Lazy as ByteString
import Data.Csv
import Data.Text

myOpts = defaultDecodeOptions {
      decDelimiter = fromIntegral (ord ';')
  }

data Transaction = Tx
  { waehrung :: Text
  , betrag :: Text
  } deriving Show

instance FromNamedRecord Transaction where
  parseNamedRecord m =
    Tx
      <$> m .: "Währung"
      <*> m .: "Betrag"

main :: IO ()
main =
  ByteString.readFile "bank.csv"
    >>= print . decodeByNameWith @Transaction myOpts

Save this as bank.csv:

Währung;Betrag
EUR;14,12
EUR;0,99

Versions: GHC 8.10.7 cassava ^>=0.5.2.0

csv haskell
1 Answers

You need to write:

import qualified Data.Text.Encoding as Text
instance FromNamedRecord Transaction where
  parseNamedRecord m =
    Tx
      <$> m .: Text.encodeUtf8 "Währung"
      <*> m .: "Betrag"

The problem is that cassava is internally representing field names as the ByteString of the UTF-8 encoding of the text. However, the IsString instance for ByteStrings which is used to encode a string literal to a ByteString does not use UTF-8 encoding but rather encodes each character as the least-significant byte of its code point (which is basically never what you want for non-ASCII strings).

user_72030160
Related