Reading avro files which are compressed is blowing up

user_1076111 0
Viewed 113

trying to write a program to read avro files.

var reader = DataFileReader<GenericRecord>.OpenReader(path);
foreach (var record in reader.NextEntries)
{
   var name = record["username"];
}

I downloaded the avro files 'twitter.avro' - this one works fine (it is not compressed), and 'twitter.snappy.avro' (this one is compressed) from here: https://github.com/miguno/avro-cli-examples, and tested them with online viewer which shows both.

However in my program although the uncompressed file works fine, the compressed one fails, the initial record's 'username' comes out to be: �cmigunoFRock: Nerf paper, scissors is fine.���

And the second record blows up on the call to 'NextEntries', with this exception: System.StackOverflowException: 'Exception of type 'System.StackOverflowException' was thrown.'

avro c#
1 Answers

To answer my own question, 'snappy' is a type of compression, which is different from the 'deflate' compression supported by the library, that is why it is failing.

Would have been better to receive an error such as "snappy codec is not supported".

user_10761110
Related