DataFrame.from_dict fails to change dict with tuple keys into multi-index dataframe

user_10596645 0
Viewed 484

I feel very confused with DataFrame.from_dict. It seems DataFrame.from_dict works right on dict with tuple dict accidentally @_@. Seriously, I have tried some cases below.

case1:

dict_var1 = {('a', 'b'): [43,47], ('a', 'c'): [23,40], ('b', 'c'): [30,35], ('c', 'd'): [10,30], ('a', 'd'): [23,28]}
frame_var1 = pd.DataFrame.from_dict(dict_var1, orient='index', columns=['num', 'cnt'])
print(frame_var1.index)

result1:

Index([('a', 'b'), ('a', 'c'), ('b', 'c'), ('c', 'd'), ('a', 'd')], dtype='object')

case2:

dict_var2 = {('a', 'b'): {'num': 43, 'cnt': 47}, ('a', 'c'): {'num': 23, 'cnt': 40}, ('b', 'c'): {'num': 30, 'cnt': 35}, ('c', 'd'): {'num': 10, 'cnt': 30}, ('a', 'd'): {'num': 23, 'cnt': 28}}
frame_var2 = pd.DataFrame.from_dict(dict_var2, orient='index', columns=['num', 'cnt'])
print(frame_var2.index)

result2:

MultiIndex(levels=[['a', 'b', 'c'], ['b', 'c', 'd']], labels=[[0, 0, 0, 1, 2], [0, 1, 2, 1, 2]])

question:

When I change the type of dict values from list to dict, DataFrame.from_dict can change the tuple dict keys into multi-index right. Could anyone explain how DataFrame.from_dict works leading to these results?

pandas python
2 Answers

Base on the document 

if orient == 'index':
    if len(data) > 0:
        # TODO speed up Series case
        if isinstance(list(data.values())[0], (Series, dict)):
            data = _from_nested_dict(data)
        else:
            data, index = list(data.values()), list(data.keys())

So in your case the second one pass the isinstance, then will 

data = _from_nested_dict(data)

But the first one values type is list , then 

data, index = list(data.values()), list(data.keys())
user_79645270

This isn't 100% answer but just to show that you can get it to work with dict_var1 the same way as dict_var2 works.

The cause for this is an implementation detail. But it looks like pd.DataFrame (or pd.DataFrame.from_dict(..., orient='columns')) favours dict_var1 if you present it in a columnar fashion:

frame_var1 = pd.DataFrame(dict_var1, index=['num', 'cnt']).T
print(frame_var1)
     num  cnt
a b   43   47
  c   23   40
b c   30   35
c d   10   30
a d   23   28

Whereas pd.DataFrame.from_dict(..., orient='index') favours dict_var2 with the keys as the index. 

frame_var2 = pd.DataFrame.from_dict(
    dict_var2, orient='index', columns=['num', 'cnt'])
print(frame_var2)
     num  cnt
a b   43   47
  c   23   40
  d   23   28
b c   30   35
c d   10   30

Let me know if I can explain further.

user_49090870
Related