How to plot a min-max fill_between plot from multiple files

Question

Thank you in advance for your help! (Code Below) (Link to 1st piece of data) (Link to data I want to add)

I am trying to import data from a second CSV (above) and add a second line to this plot based on that CSVs data. What is the best approach to doing this? (Images below)

The squiggly lines on the plot represent the range of data.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')

raw_data = pd.read_csv('all-deep-soil-temperatures.csv', index_col=1, parse_dates=True)
df_all_stations = raw_data.copy()

selected_soil_station = 'Minot'
df_selected_station = df_all_stations[df_all_stations['Station'] == selected_soil_station]
df_selected_station.fillna(method = 'ffill', inplace=True);
df_selected_station_D=df_selected_station.resample(rule='D').mean()
df_selected_station_D['Day'] = df_selected_station_D.index.dayofyear
mean=df_selected_station_D.groupby(by='Day').mean()
mean['Day']=mean.index

maxx=df_selected_station_D.groupby(by='Day').max()
minn=df_selected_station_D.groupby(by='Day').min()
mean['maxx20']=maxx['20 cm']
mean['minn20']=minn['20 cm']

plt.style.use('ggplot')
bx = mean.plot(x='Day', y='20 cm',color='black')
plt.fill_between(mean['Day'],mean['minn20'],mean['maxx20'],color='blue',alpha = 0.2);
bx.set_xlabel("Day of the year")
bx.set_ylabel("Temperature in Celsius")
bx.set_title("Soil Temp, Air Temp, and Snow Depth for " + str(selected_soil_station))

What I have:

What I want to have:

Sample Data

all-deep-soil-temperatures.csv

Station,Time,5 cm,10 cm,20 cm,30 cm,40 cm,50 cm,60 cm,80 cm,100 cm,125 cm,150 cm,175 cm,200 cm,225 cm
Adams,2018-06-21 1700,32.8,27.74,23.06,20.28,18.16,16.64,15.33,13.07,11.19,9.35,7.919,6.842,6.637,5.686
Adams,2018-06-21 1800,31.78,27.66,23.41,20.52,18.31,16.77,15.46,13.23,11.34,9.51,8.06,6.894,6.681,5.781
Adams,2018-06-21 1900,30.5,27.24,23.61,20.73,18.54,17.02,15.73,13.51,11.63,9.8,8.36,7.262,6.681,5.893
Adams,2018-06-21 2000,29.12,26.74,23.72,20.9,18.66,17.14,15.85,13.62,11.8,10.03,8.69,7.65,6.684,5.904
Adams,2018-06-21 2100,27.5,26.08,23.74,21.07,18.86,17.36,16.12,13.96,12.19,10.43,9.11,8.1,6.823,6.069
Adams,2018-06-21 2200,26.05,25.41,23.66,21.2,18.98,17.43,16.15,13.96,12.15,10.41,9.09,8.11,6.909,6.164
Adams,2018-06-21 2300,24.89,24.75,23.48,21.21,19.01,17.42,16.1,13.9,12.07,10.33,9.01,7.997,6.886,6.132
Adams,2018-06-22 0000,24.09,24.19,23.31,21.22,19.06,17.43,16.1,13.88,12.04,10.31,8.97,7.964,6.887,6.125
Adams,2018-06-22 0100,23.49,23.74,23.11,21.2,19.1,17.49,16.13,13.87,12.01,10.23,8.88,7.89,6.89,6.128
Adams,2018-06-22 0200,22.92,23.3,22.91,21.19,19.15,17.53,16.16,13.88,12.02,10.25,8.91,7.911,6.902,6.14
Adams,2018-06-22 0300,22.32,22.86,22.68,21.11,19.14,17.52,16.14,13.84,11.98,10.21,8.87,7.858,6.892,6.121
Adams,2018-06-22 0400,21.81,22.46,22.44,21.05,19.15,17.55,16.16,13.85,11.99,10.21,8.86,7.84,6.899,6.111
Williston,2020-09-21 0500,14.69,15.29,15.61,15.68,15.48,15.22,14.99,14.7,14.51,14.27,14.06,13.85,,
Williston,2020-09-21 0600,14.39,15.09,15.49,15.61,15.43,15.19,14.99,14.68,14.46,14.2,13.97,13.73,,
Williston,2020-09-21 0700,14.16,14.93,15.39,15.56,15.4,15.18,14.99,14.69,14.47,14.22,13.99,13.74,,
Williston,2020-09-21 0800,13.72,14.54,15.05,15.22,15.05,14.84,14.68,14.37,14.09,13.92,13.64,13.35,,
Williston,2020-09-21 0900,13.64,14.35,14.87,15.08,14.95,14.78,14.63,14.32,14.04,13.88,13.61,13.33,,
Williston,2020-09-21 1000,13.9,14.33,14.79,15.06,14.99,14.85,14.72,14.41,14.14,13.99,13.74,13.51,,
Williston,2020-09-21 1100,14.46,14.43,14.78,15.07,15.04,14.93,14.78,14.49,14.24,14.07,13.84,13.62,,
Williston,2020-09-21 1200,15.34,14.77,14.89,15.15,15.17,15.09,14.97,14.7,14.47,14.28,14.06,13.87,,
Williston,2020-09-21 1300,16.26,15.19,15.03,15.22,15.28,15.24,15.16,14.89,14.69,14.49,14.28,14.06,,
Williston,2020-09-21 1400,17.2,15.74,15.24,15.29,15.35,15.31,15.24,15,14.82,14.62,14.41,14.22,,
Williston,2020-09-21 1500,18.04,16.35,15.54,15.35,15.37,15.32,15.23,14.97,14.77,14.55,14.35,14.15,,
Williston,2020-09-21 1600,18.59,16.89,15.83,15.42,15.36,15.28,15.16,14.89,14.69,14.47,14.28,14.09,,
Williston,2020-09-21 1700,18.68,17.21,16.1,15.52,15.4,15.3,15.23,14.95,14.78,14.54,14.35,14.14,,

allStationsDailyAirTemp1.csv

Station,Date,Temp
Adams,2018-06-21,22.723
Adams,2018-06-22,23.358
Adams,2018-06-23,20.986
Adams,2018-06-24,20.524
Adams,2018-06-25,19.699
Adams,2018-06-26,22.146
Adams,2018-06-27,21.239
Adams,2018-06-28,21.367
Adams,2018-06-29,20.701
Adams,2018-06-30,18.613
Adams,2018-07-01,19.376
Adams,2018-07-02,19.079
Adams,2018-07-03,20.747
Adams,2018-07-04,19.622
Adams,2018-07-05,18.029
Adams,2018-07-06,18.883
Adams,2018-07-07,25.655
Adams,2018-07-08,22.953
Adams,2018-07-09,20.281
Williston,2020-09-05,21.69
Williston,2020-09-06,16.595
Williston,2020-09-07,5.917
Williston,2020-09-08,3.863
Williston,2020-09-09,8.996
Williston,2020-09-10,14.488
Williston,2020-09-11,15.689
Williston,2020-09-12,16.002
Williston,2020-09-13,11.219
Williston,2020-09-14,16.695
Williston,2020-09-15,12.77
Williston,2020-09-16,9.523
Williston,2020-09-17,13.186
Williston,2020-09-18,16.992
Williston,2020-09-19,16.85
Williston,2020-09-20,17.235
Williston,2020-09-21,17.595
Williston,2020-09-22,19.115
Williston,2020-09-23,16.43
Williston,2020-09-24,21.035
Williston,2020-09-25,17.01
Williston,2020-09-26,14.109

user_7758804 · Answer

See inline notations with the new code
Removed plt.style.use('ggplot') because it makes it difficult to see the fill_between colors
Also see How to create a min-max lineplot by month?
Don't use ; in python
Load the data from the other file, into a separate dataframe
Clean, and aggregate the new data as needed
- Set the Date column to a datetime format
- Extract day of year
- groupby day of year and aggregate mean, min, and max temperature
Plot the new data to the same axes as the original plot, bx.

df_all_stations = pd.read_csv('data/so_data/2020-09-29 64128817/all-deep-soil-temperatures.csv', index_col=1, parse_dates=True)

# load air temp data
at = pd.read_csv('data/so_data/2020-09-29 64128817/allStationsDailyAirTemp1.csv')

# set Date to a datetime format
at.Date = pd.to_datetime(at.Date)

# extract day of year
at['doy'] = at.Date.dt.dayofyear

# selet data from Minot
at = at[at.Station == 'Minot']

# groupby the day of year (doy) and aggregate min max and mean
atg = at.groupby('doy')['Temp'].agg([min, max, 'mean'])

selected_soil_station = 'Minot'
df_selected_station = df_all_stations[df_all_stations['Station'] == selected_soil_station].copy()  # make a copy here, otherwise there will be warning
df_selected_station.fillna(method = 'ffill', inplace=True)
df_selected_station_D=df_selected_station.resample(rule='D').mean()
df_selected_station_D['Day'] = df_selected_station_D.index.dayofyear
mean=df_selected_station_D.groupby(by='Day').mean()
mean['Day']=mean.index

maxx=df_selected_station_D.groupby(by='Day').max()
minn=df_selected_station_D.groupby(by='Day').min()
mean['maxx20']=maxx['20 cm']
mean['minn20']=minn['20 cm']

bx = mean.plot(x='Day', y='20 cm', color='black', figsize=(9, 6), label='20 cm Soil Temp')
plt.fill_between(mean['Day'], mean['minn20'], mean['maxx20'], color='blue', alpha = 0.2, label='20 cm Soil Temp Range')

# add air temp plot to the bx plot with ax=bx
atg['mean'].plot(ax=bx, label='Mean Air Temp')

# add air temp fill between plot to the bx plot
bx.fill_between(atg.index, atg['min'], atg['max'], color='cyan', alpha = 0.2, label='Air Temp Range')

bx.set_xlabel("Day of the year")
bx.set_ylabel("Temperature in Celsius")
bx.set_title("Soil Temp, Air Temp, and Snow Depth for " + str(selected_soil_station))

# grid
bx.grid()

# set legend location
bx.legend(bbox_to_anchor=(1.05, 1), loc='upper left')

# remove margin spaces
plt.margins(0, 0)

plt.show()

How to plot a min-max fill_between plot from multiple files

Sample Data

1 Answers