Rows not appending properly in Pandas

Viewed 34

I apologize if my formatting isn't correct, this is my first post on here.

I'm trying to webscrape a site, and create a DataFrame with multiple columns of data. I'll just insert the code for one of the columns below:

for name in (soup.findAll('p',{'class':'profile-name'})):
     username=name.text
     df1=pd.DataFrame({'Username':[username]})

When I print(df1), it doesn't append the rows. Instead, it outputs individual tables per name. So for instance:

         Username
   0     Alice
         Username
   0     Bob
         Username
   0     Carl

But I want it to output:

        Username
   0     Alice
   1     Bob
   2     Carl

I've tried df1.append(df1). I've also tried merge, cat, and nothing works. For instance, if I do df2 = df1.append(df1) I get

         Username
   0     Alice
   0     Alice
         Username
   0     Bob
   0     Bob
         Username
   0     Carl
   0     Carl

I should also add that the type is <class 'str'> Does anyone have any suggestions? I'd really appreciate your help.

Thank you.

1 Answers

I think you can try:

pd.DataFrame({'Username': [name.text for name in (soup.findAll('p',{'class':'profile-name'}))]})
Related