append data to an instance of DataFrame placed in a dictionary

user_230270 0
Viewed 62

In my problem I am trying to define a DataFrame (ex) and to store inside it some values during my computation.

Below is a short example of my problem:

import pandas as pd

def main():
    ex1 = pd.DataFrame(columns=['Timestamp','A'])
    ex2 = pd.DataFrame(columns=['Timestamp','A'])
    str1 = 'hello-1'
    str2 = 'hello-2'
    tcp_hash = { 1 : (ex1, str1), 2 : (ex2, str2) }

    for idx in range(5):
        try:
            (df, str_idx) = tcp_hash.get(idx, None)
            df.append({'Timestamp': idx, 'A': str_idx}, ignore_index=True)
            tcp_hash[idx] = (df, str_idx)
        except TypeError:
            print('TCP Message: {} does not have a parser'.format(idx))

    print(ex1)
    print(ex2)

if __name__== "__main__":
    main()

As result of my ex, i get:

TCP Message: 0 does not have a parser
TCP Message: 3 does not have a parser
TCP Message: 4 does not have a parser
Empty DataFrame
Columns: [Timestamp, A]
Index: []
Empty DataFrame
Columns: [Timestamp, A]
Index: []

instead of:

         Timestamp     A
      0  1             hello-1

and

         Timestamp     A
      0  2             hello-2

If I understand correctly my problem, then i guess that ex inside _tcp_hash_ is a copy of my ex instead of a "reference".

How could I fix my code?

pandas python-3.x
1 Answers

I just fix your code. Pandas df.append will create new object. Therefore, any dataframe result from append is not the original ex1, ex2 anymore. You may use loc to inline modify the ex1, ex2 as follows

def main():
    ex1 = pd.DataFrame(columns=['Timestamp','A'])
    ex2 = pd.DataFrame(columns=['Timestamp','A'])
    str1 = 'hello-1'
    str2 = 'hello-2'
    tcp_hash = { 1 : (ex1, str1), 2 : (ex2, str2) }

    for idx in range(5):
        try:
            (df, str_idx) = tcp_hash.get(idx, None)
            #df.append({'Timestamp': idx, 'A': str_idx}, ignore_index=True)
            df.loc[df.shape[0], :] = {'Timestamp': idx, 'A': str_idx}
            tcp_hash[idx] = (df, str_idx)
        except TypeError:
            print('TCP Message: {} does not have a parser'.format(idx))

    print(ex1)
    print(ex2)

if __name__== "__main__":
    main()

Output

TCP Message: 0 does not have a parser
TCP Message: 3 does not have a parser
TCP Message: 4 does not have a parser
  Timestamp        A
0  1         hello-1
  Timestamp        A
0  2         hello-2
user_101892140
Related