How to read from Azure Blob Storage with Python delta-rs

user_7454972 0
Viewed 686

I'd like to use the Python bindings to the delta-rs to read from my blob storage. https://github.com/delta-io/delta-rs/tree/main/python

Currently I am kind of lost, since I cannot figure out how to configure the filesystem on my local machine. Where do i have to put my credentials?

Can I use adlfs for this?

from adlfs import AzureBlobFileSystem
    
fs = AzureBlobFileSystem(
        account_name="...", 
        account_key='...'
    )

and then use the fs object?

Would be really helpful if someone could provide me with some headstart here.

Best

delta-lake azure-blob-storage rust python
1 Answers

Unfortunately we don't have great documentation around this at the moment. You should be able to set AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_SAS environment variables a la this integration test.

That will ensure the Python bindings can access table metadata, but typically fetching of the data for query is done through Pandas, and I'm not sure if Pandas will handle these variables as well (not an ADLSv2 user myself)..

user_126111890
Related