How to check record count in a csv file uploaded in azure blob storage?

Viewed 1123

So i am uploading a 2gb csv file to my BLOB storage, and i want the record count (no of rows) of this file, so that i can validate after it gets loaded to ADW. Is there any way to get record count(like column count) in azure itself.

Thanks in advance

1 Answers

Azure Blobs are not like local files: You'd have to download (or stream) your blob to something that works through the file to perform any calculation you're trying to do.

Alternatively, you could mount your blob storage to something like Databricks (Spark cluster) and write your code there (same basic concept).

Or... you could do your record counts prior to (or during) your upload to blob storage.

Ultimately, how you perform this counting is really up to you. Blob storage is just bulk storage and knows nothing about file formats.

Related