Pandas dataframe from dict, why?

user_825227 0
Viewed 165

I can create a pandas dataframe from dict as follows:

d = {'Key':['abc','def','xyz'], 'Value':[1,2,3]}
df = pd.DataFrame(d)
df.set_index('Key', inplace=True)

And also by first creating a series like this:

d = {'abc': 1, 'def': 2, 'xyz': 3}
a = pd.Series(d, name='Value')
df = pd.DataFrame(a)

But not directly like this:

d = {'abc': 1, 'def': 2, 'xyz': 3}
df = pd.DataFrame(d)

I'm aware of the from_dict method, and this also gives the desired result:

d = {'abc': 1, 'def': 2, 'xyz': 3}
pd.DataFrame.from_dict(d, orient='index')

but I don't see why:

(1) a separate method is needed to create a dataframe from dict when creating from series or list works without issue;

(2) how/why creating a dataframe from dict/list of lists works, but not creating from dict directly.

Have found several SE answers that offer solutions, but looking for the 'why' as this behavior seems inconsistent. Can anyone shed some light on what I may be missing here.

python-3.7 pandas python-3.x python
1 Answers

There's actually a lot happening here, so let's break it down.

The Problem

There are soooo many different ways to create a DataFrame (from a list of records, dict, csv, ndarray, etc ...) that even for python veterans it can take a long time to understand them all. Hell, within each of those ways, there are EVEN MORE ways to build a DataFrame by tweaking some parameters and whatnot.

For example, for dictionaries (where the values are equal length lists), here are two ways pandas can handle them:

Case 1: You treat each key-value pair as a column title and it's values at each row respectively. In this case, the rows don't have names, and so by default you might just name them by their row index.

Case 2: You treat each key-value pair as the row's name and it's values at each column respectively. In this case, the columns don't have names, and so by default you might just name them by their index.

The Solution

Python's is a weakly typed language (aka variables don't declare a type and functions don't declare a return). As a result, it doesn't have function overloading. So, you basically have two philosophies when you want to create a object class that can have multiple ways of being constructed:

  1. Create only one constructor that checks the input and handles it accordingly, covering all possible options. This can get very bloated and complicated when certain inputs have their own options/parameters and when there's simply just too much variety.
  2. Separate each option into @classmethod's to handle each specific individual way of constructing the object.

The second is generally better, as it really enforces seperation of concerns as a SE design principle, however the user will need to know all the different @classmethod constructor calls as a result. Although, in my opinion, if you're object class is complicated enough to have many different construction options, the user should be aware of that anyways.

The Panda's Way

Pandas adopts a sorta mix between the two solutions. It'll use the default behaviour for each input type, and it you wanna get any extra functionality you'll need to use the respective @classmethod constructor.

For example, for dicts, by default, if you pass a dict into the DataFrame constructor, it will handle it as Case 1. If you want to do the second case, you'll need to use DataFrame.from_dict and pass in orient='index' (without orient='index', it would would use default behaviour described base Case 1).

In my opinion, I'm not a fan of this kind of implementation. Personally, it's more confusing than helpful. Honestly, a lot of pandas is designed like that. There's a reason why pandas is the topic of every other python tagged question on stackoverflow.

user_110704630
Related