Download Sentinel file from S3 using Python boto3

user_7113481 0
Viewed 830

I am using following code for downloading full size Sentinel file from S3

import boto3

s3_client = boto3.Session().client('s3')
response = s3_client.get_object(Bucket='sentinel-s2-l1c',
                        Key='tiles/7/W/FR/2018/3/31/0/B8A.jp2', 
                        RequestPayer='requester')
response_content = response['Body'].read()

with open('./B8A.jp2', 'wb') as file:
     file.write(response_content)

But I don't want to download full size image. Is there any way to download image based on latMax, longMin,LatMin and LatMax? I was using below command but it is not working since when data was made requester-payer on S3

gdal_translate --config CPL_TMPDIR temp -projwin_srs "EPSG:4326" -projwin 23.55 80.32 23.22 80.44 /vsicurl/http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/tiles/43/R/EQ/2020/7/26/0/B02.jp2 /TestScript/B02.jp2

Is there any way to achieve this using Python boto?

gdal gis amazon-web-services amazon-s3 python
1 Answers

You can use rasterio to access sub-windows of the image:

(I'm assuming that the AWS credentials are set up for use with boto3 and you have the necessary permissions)

import boto3
from matplotlib.pyplot import imshow
import rasterio as rio
from rasterio.session import AWSSession
from rasterio.windows import Window

# create AWS session object
aws_session = AWSSession(boto3.Session(), requester_pays=True)

with rio.Env(aws_session):
    with rio.open("s3://sentinel-s2-l1c/tiles/7/W/FR/2018/3/31/0/B8A.jp2") as src:
        profile = src.profile
        win = Window(0, 0, 1024, 1024)
        arr = src.read(1, window=win)

imshow(arr)

raster plot

print(arr.shape)

(1024, 1024)

Explanation:

If the AWS credentials are properly configured for boto3, you can create an AWSSession object based on a boto3.Session(). This will set the necessary credentials for the S3 access. Add the flag requester_pays=True so you can read from the requester-pays bucket.

The AWSSession object can be passed into a rasterio.Env context, so rasterio (and more importantly the underlying gdal functions) has access to the credentials.

Using rasterio.windows.Window I'm reading the arbitrary sub-window (0, 0, 1024, 1024) into memory, but you could also define a window using coordinates, as explained in the documentation.

From there you can process the array or save it to disk.

user_59975550
Related