When to use pandas series, numpy ndarrays or simply python dictionaries?

If you want to an answer which tells you to stick with just one type of data structures, here goes one: use pandas series/dataframe structures.

The pandas series object can be seen as an enhanced numpy 1D array and the pandas dataframe can be seen as an enhanced numpy 2D array. The main difference is that pandas series and pandas dataframes has explicit index, while numpy arrays has implicit indexation. So, in any python code that you think to use something like

import numpy as np
a = np.array([1,2,3])

you can just use

import pandas as pd
a = pd.Series([1,2,3])

All the functions and methods from numpy arrays will work with pandas series. In analogy, the same can be done with dataframes and numpy 2D arrays.

A further question you might have can be about the performance differences between a numpy array and pandas series. Here is a post that shows the differences in performance using these two tools: performance of pandas series vs numpy arrays.

Please note that even in an explicit way pandas series has a subtle worse in performance when compared to numpy, you can solve this by just calling the values method on a pandas series:

a.values

The result of apply the values method on a pandas series will be a numpy array!

I would say that pandas lets you index and slice off of strings and create data frames directly from dictionaries, whereas numpy is mostly nested lists. Other than that, they are pretty much exactly the same (pandas is built on top of numpy). So pandas "feels" more natural to use for database-like data (e.g. csv, excel, and sql files), whereas numpy "feels" more natural for numeric processing of data (e.g. signals, images, etc.). Granted, you can do many of the same things in both libraries; you can even create pandas data frames from numpy arrays and vice-versa.

One major difference (something to watch out for) is slicing in pandas is inclusive whereas numpy is exclusive (i.e. 0:10 in pandas is "0 up to and including 10" whereas it is "0 up to, but not including 10" in numpy). This is intuitively due to the fact that since pandas permits slicing on strings, it doesn't make much sense to slice, say, "up to but not including a column of name x" (shout out to Corey Schafer for that insight (see about 30 mins in): Python Pandas Tutorial (Part 2)).

Other than that, pandas utilizes the same slicing, indexing, and fancy indexing notation as numpy (minus the ability for strings) and the same kinds of "gotcha's" with respect to different operations creating views vs copies of data. (An excellent numpy tutorial is a Numpy lecture from SciPy 2019 by Alex Chabot-Leclerc).

Ultimately, I would say pandas is a database analyst's best friend while numpy is a data scientists friend. Personally, I use pandas to pull data from the real world, sort it, and preprocess it. Then I convert this data into numpy arrays where necessary to do more serious/intensive numeric computing. PLEASE NOTE: This is purely opinion. There is no right answer.

That being said, I highly recommend getting to know and understand numpy first (highly recommend the Alex Chabot-Leclerc video). Afterwards, pandas will make a lot more sense.