I am working on a analysis of rainfall fall data, using dataframes. The dataframe consists of multiple variables but dropped these and I am now left with only the rainfall and date. The dataset set starts in 1906, I dropped all NaN's. My rainfall data starts at 1947 with about 20 zeros, only after ten years the measurement is restarted and continued till present.
The 20 zeros are outliers as these are the only measurements in 10 year time. I could easily get rid of them for this instance. However I am writing a definition and want it to be automated for other datasets aswell.
here is the code i am working with, could someone please help me out:
file = 'test_vlissingen.txt'
data = pd.read_table("test_vlissingen.txt", skipinitialspace=True, skiprows=46, delimiter=',', parse_dates=True, index_col=1)
data_RH = data.loc(axis=1)['RH']
rain = data_RH.dropna()
display(rain.head(50))
Below is a sample of the dataframe:
YYYYMMDD
1947-08-15 0.0
1947-08-16 0.0
1947-08-17 0.0
1947-08-18 0.0
1947-08-19 0.0
1947-08-20 2.0
1947-08-21 0.0
1947-08-22 0.0
1947-08-23 0.0
1947-08-24 0.0
1947-08-25 0.0
1947-08-26 0.0
1947-08-27 0.0
1947-08-28 0.0
1947-08-29 0.0
1947-08-30 0.0
1947-08-31 0.0
1957-01-01 30.0
1957-01-02 2.0
1957-01-03 17.0
1957-01-04 8.0
1957-01-05 1.0
1957-01-06 56.0
1957-01-07 1.0
1957-01-08 0.0
1957-01-09 7.0
1957-01-10 13.0
1957-01-11 13.0
1957-01-12 44.0
1957-01-13 77.0
1957-01-14 4.0
1957-01-15 0.0
1957-01-16 0.0
1957-01-17 3.0
1957-01-18 4.0
1957-01-19 0.0
1957-01-20 2.0
1957-01-21 0.0
1957-01-22 0.0
1957-01-23 0.0
1957-01-24 0.0
1957-01-25 4.0
1957-01-26 25.0
1957-01-27 42.0
1957-01-28 1.0
1957-01-29 16.0
1957-01-30 2.0
1957-01-31 32.0
1957-02-01 14.0
1957-02-02 30.0
Name: RH, dtype: float64