Improve data serialization function runtime memory

Viewed 31

The code below is used for data serialization data coming from (AWS rds) db and zipping it to column name, so it's going basically through every row and column.

The problem I am facing is it storing variables 2 times so if I get data like 500 MB it's converting that data to 1GB and that is not feasible.

I'm trying to minimize the function runtime memory as far as possible, is there any possible way to get data from RDS in key value pair dict? Or can we do the serialization column wise?

RDS response: {
    'result':[
     (datetime.datetime(2020, 1, 2, 0, 0), Decimal('133.340000'),Decimal('-1098.683000'),556,6667,90,'a','b','c','d','e','f'), 
     (datetime.datetime(2020, 1, 2, 0, 0), Decimal('133.340000'),Decimal('-1098.683000'),556,6667,90,'a','b','c','d','e','f'), 
     (datetime.datetime(2020, 1, 2, 0, 0), Decimal('133.340000'),Decimal('-1098.683000'),556,6667,90,'a','b','c','d','e','f'), 
     (datetime.datetime(2020, 1, 2, 0, 0), Decimal('133.340000'),Decimal('-1098.683000'),556,6667,90,'a','b','c','d','e','f'), 
     (datetime.datetime(2020, 1, 2, 0, 0), Decimal('133.340000'),Decimal('-1098.683000'),556,6667,90,'a','b','c','d','e','f'), 
     (datetime.datetime(2020, 1, 2, 0, 0), Decimal('133.340000'),Decimal('-1098.683000'),556,6667,90,'a','b','c','d','e','f'),
]}

@staticmethod
def map_result_set(rds_response, column_metadata):
    """
    Return a list of row objects with parameter-value pairs extracted from RDS response.
    """
    column_names_list = [column[0] for column in column_metadata]
    result_set = []
    for row in rds_response:
        # Create row objects by mapping column names to row values
        result_set.append(dict(zip(column_names_list, row)))
    return result_set
0 Answers
Related