Pivot a Pandas Dataframe into a 3D array

Question

I would like to extend my existing code, so that I can add another dimension to my plot (semi-3D -> semi-4D).

My current code looks like this:

df = df[(['T_id'] == 3)]  # Pick just t_id = 3 data

df_grouped = df.groupby(['Timestamp', 'Pos_id'], as_index = False)[['Error']]
  
df_sum = df_grouped .agg('sum').pivot('Timestamp', 'Pos_id')["Error"] # Counting all errors
df_count = df_grouped .agg('count').pivot('Timestamp', 'Pos_id')["Error"] # Counting all entries
df_error_rate = df_date_sum.div(df_count) # err_rate = errors / total
df_error_rate = df_date_error_rate.fillna(0)

fig, ax = plt.pyplot.subplots()
sns.heatmap(df_error_rate.T)

The code above plots the error rate as a heatmap where the x-axis is the timeline (‘Timestamp’), the y-axis the positions (‘Pos_id’) and the color is representing the error rate (df_date_error_rate - 0% - 100%).

This example is only plotting for T_id = 3 (first line in my code), but I want to remove this first line, add another dimension for the T_id and do the same calculation for all available T_id's as well.

The heatmap would have these dimension then:

  Axis  | Old          |  New
--------+--------------+-------------
  X     | Timestamp    | Timestamp
  Y     | Pos_id       | Pos_id
  Z     | not existing | T_id
  Color | error_rate   | error_rate

There are different approaches for creating 3d heatmaps, like:

My current problem right now is to create the new pivot table. As a first naïve approach, I just added the ‘T_id’ to the pivot-command:

df_grouped = df.groupby(['Timestamp', 'T_id', 'Pos_id'], as_index = False)[['Error']]
df_sum = df_grouped .agg('sum').pivot('Timestamp', 'T_id', 'Pos_id')["Error"]
df_count = df_grouped .agg('count').pivot('Timestamp', 'T_id', 'Pos_id')["Error"]
df_error_rate = df_date_sum.div(df_count)
# …

This does not work though.

I guess pivot is not suitable for this. How can I do it?

Pivot a Pandas Dataframe into a 3D array

0 Answers