Is there a way to process PDF files in Blob Storage without downloading them locally using Python?

Viewed 1217

Summarize the problem:

I am trying to process multiple PDFs to an OCR program written in Python. During local development, PDFs are located in a local directory where can be processed but I wasn't able to figure it out a path-like filesystem in Blob Storage. Technically speaking, I know there is no such filesystem in Blob but I need such path to be passed in OCR program. Any ways we can achieve this?

What I've tried:

Currently I have the code below to connect to container and blobs in azure.py:

import os
import glob
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, PublicAccess

# list input PDF files 
def ls_files(client, path, recursive=False):
    if not path == '' and not path.endswith('/'):
        path += '/'

    blob_list = client.list_blobs(name_starts_with=path)
    files = []
    for blob in blob_list:
        relative_path = os.path.relpath(blob.name, path) # blob.name is the name of blobs in containers
        if recursive or not '/' in relative_path:
            files.append(relative_path)
            files = [f for f in files if f.endswith('.pdf')] # look for PDF files 
    return files

# connection string to the storage account
connect_str = '<connection string>'
# same container but different folders for inputs and outputs 
container_name = 'ocr'

blob_service_client = BlobServiceClient.from_connection_string(connect_str)
client = blob_service_client.get_container_client(container_name)

input_files = ls_files(client, '', recursive=True) # This is the input PDF files 

for files in input_files:
    ############################
    # kick off OCR program here#
    ############################ 
    print('Processing ...', files, '\n')

In main.py file:

import azure as az 

input_directory = az.input_files # input_directory was like '/Users/xyz/path/to/local/dir'

# do regular OCR processing next 

After executing the script, Python is unable to recognize file or path in Blob Storage. Is there a way that we can achieve goal here? Thanks in advance.

EDIT 1:

I came across this sample code but I'm afraid that is for older version of Python SDK not for V12. Also been looking at official repo but to no avail.

EDIT 2:

Ok. Opened a ticket here to seek out help from MSFT team and will update here once I know more. The workarounds are 1) download files as memory streams or 2) create a temp file in Python to serve as a placeholder. Any suggestions are welcomed.

1 Answers
Related