Pandas: Join data to a single row to new columns

Viewed 145

I'm new to pandas and have been having trouble using the merge, join and concatenate functions on a single row of data.

I'm iterating over a handful of rows in a table and in each iteration add some data I've found to the row I'm handling. I know, blasphemy! Thou shall not iterate. Each iteration results in a call to a server, so I need to control flow. There aren't that many rows. It's just for my own use. I promise I'll not iterate when I shouldn't.

That aside, my basic question is this: How do I add data to a given row where the new data has priority over existing data and has new columns?

Let's suppose I have a DataFrame df that I'm iterating over by row:

> df
  c1 c2 c3
0  a  b  c
1  d  e  f

and when iterating on row 0, I get some new data that I want to add to row 0. That new data is in df_a:

> df_a
  c4 c5 c6
0  g  h  i

I want to add data from df_a to row 0 of df so df is now:

> df
  c1 c2 c3  c4  c5  c6
0  a  b  c   g   h   i
1  d  e  f NaN NaN NaN

Next I iterate on row 1 and I get some columns which overlap and some which don't in df_b:

> df_b
  c5 c7 c8
0  j  k  l

And again I want to add this data to row 1 so df now has

> df
  c1 c2 c3  c4  c5  c6  c7  c8
0  a  b  c   g   h   i NaN NaN
1  d  e  f NaN   j NaN   k   l

I can't list columns names because I don't know what they'll be and new ones can appear beyond my control. Rows don't have a key because the whole thing gets thrown away after I disconnect. Data I find during each iteration always overwrites what's currently in df.

Thanks in advance!

0 Answers
Related