Problem reading ANSII Encoded CSV file with CSVHelper

Viewed 53

I am using the CSVHelper to read a CSV file which contains some characters such as the GBP character (eg £2000)

However when processing the file, the pound symbol is lost and instead a diamond with a question mark appears.

My code is as follows:

        var csvConfig = new CsvConfiguration(new CultureInfo("en-GB"))
        {
            ShouldSkipRecord = (row) =>
            {
                if (row.Row.Parser.Row <= 4)
                {
                    return true;
                } else
                {
                    return false;
                }                    
            }
        };

        using var streamReader = File.OpenText(_fileName);
        using var csvReader = new CsvReader(streamReader, csvConfig);
        
        var records = csvReader.GetRecords<WebApi.Entities.Csv.Transaction();

        foreach (var record in records)
        {
            System.Diagnostics.Debug.WriteLine(record.PaidIn);
        }

The above code outputs the field containing the pound symbol with a diamond question mark instead .. eg : ?2000

I cannot convert the source file to UTF-8 before processing it. However I have tried converting it to UTF-8 in the function above and that did not work.

I'm at a lost and need some guidance.

---- UPDATE ----------------------------------

After reading the comments I have proceeded as suggested by detecting the encoding of the file and then passing to a StreamReader:

    var csvConfig = new CsvConfiguration(new CultureInfo("en-GB"))
    {
        ShouldSkipRecord = (row) =>
        {
            if (row.Row.Parser.Row <= 4)
            {
                return true;
            } else
            {
                return false;
            }                    
        }
    };

    //get current encoding of source csv file
    var reader = new StreamReader(_fileName, Encoding.Default, true);
    if (reader.Peek() >= 0)
        reader.Read();
    Encoding encoding = reader.CurrentEncoding;

    using var streamReader = new StreamReader(_fileName, reader.CurrentEncoding);
    using var csvReader = new CsvReader(streamReader, csvConfig);
    
    var records = csvReader.GetRecords<WebApi.Entities.Csv.Transaction>();

    foreach (var record in records)
    {
        System.Diagnostics.Debug.WriteLine(record.PaidIn);
    }

But all is in vein, this still does not resolve the original problem.

1 Answers

In the end, attempting to detect the encoding of the current file resulted in ain correct encoding being returned by .NET, I therefore specified the encoding manually:

using var streamReader = new StreamReader(_fileName, Encoding.GetEncoding("iso-8859-1"));

The above solved my issue.

Related