Activity function failed :python exited with code 137

venkatesh shetty 0
Viewed 44

I'm facing same issue which was described in the below thread. Python Azure Function exits with code 137? But I am unable to find proper solution to it. Below is what I'm trying to do:

  1. Approach one: (using http trigger functions in azure)
    I passed the parameter to the request (container path for which I want to retrieve the data) and date for which I want to filter out the records, but the http trigger timeout is of 230 seconds. So tried to google another approach and came across durable activity thing.

  2. Approach two: (using durable activity)
    The same as above, but now I'm getting this error (Activity function failed :python exited with code 137). Below is how my code snippet looks. The response will the data displayed in json format. I'm using Function premium plan with EP2 tried EP3 as well but still memory consumption is going 100%

fs = AzureBlobFileSystem(account_name="stgacc",credential=get_secret_value('sec_value'));
dt = DeltaTable(f'{file_name}',file_system=fs);
dt_df = dt.to_pandas();
df = dt_df[dt_df['date']==date[0]].dropna(subset=['date']).to_json(orient='records');
fnl_data = json.loads(df);
return fnl_data

file_name is the container name which is partitioned on date. I'm not doing any operation here, just trying to extract the data and display as an response. Files are in delta tables, so it's parquet formatted. I did check the size in csv for a particular date for which I was facing the issue it's only 63 mbs.

Activity function failed :python exited with code 137

Is it because I'm reading the full container at the start and then filtering the record which is causing the issue?

azure-durable-functions azure-functions pandas rest python
1 Answers

You are running out of memory probably because you are loading a huge file into the memory. Your code seems to be using some library to do this. Is it DeltaLakeReader ??

Then its documentation already tells you what is happening:

To read a DeltaTable, first create a DeltaTable object. This will read the delta transaction log to find the current files, and get the schema. This will, however, not read any data. To read the content of the table, call to_table() to get a pyarrow.Table object, or to_pandas() to get a pandas.DataFrame

Mo Nazemi0
Related