Queries in delta tables

Viewed 149

I'm trying to read some tables from delta-lake stored in a S3 Bucket, using delta-rs in rust.

When I ran the code, it seems to be opening the table, because when I print the table it returns the following:

        version: 0
        metadata: GUID=36348853-e380-4d3d-986f-034b1cd7bcd2, name=None, description=None, partitionColumns=[], createdTime=1632167494225, configuration={}
        min_version: read=1, write=2
        files count: 1

But when I try to query on it, it returns the following message:

Parquet reader thread terminated due to error: 
 IoError(Os { code: 2, kind: NotFound, message: "No such file or directory" }

Here is my code, and I don't have any idea what I can do to solve this issue:


#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>>  {
    env::set_var("AWS_ACCESS_KEY_ID", AWS_ACCESS_KEY_ID);
    env::set_var("AWS_SECRET_ACCESS_KEY", AWS_SECRET_ACCESS_KEY);
    
    let web_site_request = GetBucketWebsiteRequest{bucket: S3_TEST_BUCKET.to_string(), expected_bucket_owner: None};
    let table_uri = "s3://dev-evandro/common/lakehouse-sync/parquet/payments/chargebee/customer/";
    let be = storage::get_backend_for_uri(table_uri).unwrap();
    let table = deltalake::open_table(table_uri).await.unwrap();
    println!("{}", table);

    let mut ctx = ExecutionContext::new();
    ctx.register_table("test_table", Arc::new(table))?;
    let batches = ctx
    .sql("SELECT * FROM test_table LIMIT 1")?
    .collect()
    .await?;

    let batch = pretty_format_batches(&[batches][0]).unwrap();
    println!("{}", batch);
    Ok(())        
}
0 Answers
Related