Can't get index position from list of Dataframes

user_9284434 0
Viewed 1135

I'm trying to get the position of a dataframe from a list of dataframes, by using the built-in method index in python. My code is below:

df1 = pd.DataFrame([1, 2, 3])
df2 = pd.DataFrame([4, 5, 6])
df3 = pd.DataFrame([7, 8, 9])   

dfs = [df1, df2, df3]
for df in dfs:
    print(dfs.index(df))

Where instead of getting the expected 0, 1 and 2, it only returns 0, followed by ValueError:

ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

I know this error typically comes from data frames comparison, and I've tried adding the .all() but to no avail.

If I change the list to something like a list of strings, ints or a mix of them, no problem whatsoever.

I've tried searching around but found nothing on this specific error.

I know I can easily add an extra variable that keeps on adding 1 for each iteration of the list, but I'd really like to understand what I'm doing wrong.

pandas python
3 Answers

As others have said, you can use enumerate to do what you want.

As to why what you're trying isn't working:

list.index(item) looks for the first element of the list such that element == item. As an example consider df = dfs[0]. When we call dfs.index(df), we first check whether df == the first element of dfs. In other words we are checking whether df == df. If you type this into your interpreter you will find that this gives you a DataFrame of Trues. This is a DataFrame object -- but what Python wants to know is whether it should consider this object as True or not. So it needs to convert this DataFrame into a single bool. It tries to do this via bool(df == df), which relies on pandas implementing a method that converts any DataFrame into a bool. However there is no such method, and for good reason -- because the correct way to do this is ambiguous. So at this point pandas raises the error that you see.

In summary: for index to make sense, the objects must have some notion of equality (==), but DataFrames don't have such a notion, for good reason.

If in a future problem you need to find the index of a DataFrame within a list of DataFrames, you first have to decide on a notion of equality. One sensible such notion is if all the values are the same. Then you can define a function like:

def index(search_dfs, target_df):
    for i, search_df in enumerate(search_dfs):
        if (search_df.values == target_df.values).all():
            return i
    return ValueError('DataFrame not in list')

index(dfs, df[2])
Out: 2
user_55068940

Use enumerate :

for i, df in enumerate(dfs):
    print(i)
user_102015800

I guess you want to do something like this:

dfs = [df1, df2, df3]
for i, df in enumerate(dfs):
    print(i)

And this is not a pandas related question. It's simply Python question.

user_92303740
Related