Split an array with dates in python

user_13183390 0
Viewed 927

I have an array consisting of 1 column and 1980796 rows full of dates. The date type is the following: '2018-01-01 00:00:00 +01:00' I am trying to split the date in two different arrays, one according to each day and the other to the hour. Here is the part of the code which i am working on to extract those two arrays:

    time=np.array(parameters[:,2])
    time1=time.astype(str)
    month=[]
    hour=[]
    for i in (time1):
         month.append(i.split(' ')[0])
         hour.append(i.split(' ')[1])
    month1=np.array(month)
    hour1=np.array(hour)

When i run the code i recieve the following error:

Traceback (most recent call last):
  File "<ipython-input-136-3cf951a10e93>", line 12, in <module>
    hour.append(i.split(' ')[1])

IndexError: list index out of range

Finally, when i ckeck the shape of the new arrays is different that the time1 array. I want just to split the dates and get two new arrays with the same shape as the initial array in all the indices.

numpy arrays python split string
4 Answers

Your method is correct provided that you have the needed data quality. Here is a working example solution: Split data into "months"- and "hours"- numpy arrays (to test your data quality, try printing out all dates that do or do not satisfy the timestamp format you expect):

from datetime import datetime, timedelta

import numpy as np


parameters = [datetime.today() - timedelta(days=x) for x in range(5)]

months, hours = [], []

for dt in parameters:
    month = str(dt).split(' ')[0].split('-')[1]
    hour = str(dt).split(' ')[1].split(':')[1]
    months.append(month)
    hours.append(hour)

months = np.array(months)
hours = np.array(hours)

print(f"{months = }")
print(f"{hours = }")

Returns:

months = array(['06', '06', '06', '06', '06'], dtype='<U2')
hours = array(['14', '14', '14', '14', '14'], dtype='<U2')
user_74455280

Perhaps you can try something like this to keep bad dates out:

import numpy as np
dates = ['2018-01-01 00:00:00 +01:00', '2018-01-02 01:00:00 +01:00', '2018-01-0302:00:00 +01:00']

arr = np.array(dates).astype(str)
month, hour = [], []

for i in arr:
    if i.count(' ')==2:
      m, h, _ = i.split(' ')
      month.append(m)
      hour.append(h)
    else:
      print('Value "{0}" does not contain 2 spaces like the rest!'.format(i))

month = np.array(month)
hour = np.array(hour)
user_128818440 
#You can use try catch to avoid the error    
time=np.array(parameters[:,2])
time1=time.astype(str)
month=[]
hour=[]
for i in (time1):
    try:
        month.append(i.split(' ')[0])
        hour.append(i.split(' ')[1])
    except:
        month.append('')
        hour.append('')
        continue
month1=np.array(month)
hour1=np.array(hour)
user_86665240

You need to decide what to do with bad data entries: add a "Try" and "Except" block around your parsing code.

when you reach Except you should print out the original string which caused the error. Then you should either ignore that string (marking it as bad), and continue the for loop. Or end your program by throwing an error, and manually fix the input dates.

One comment: You should parse everything before you add it to the arrays, because if you error on the "hour", you don't want the other value appended to month. So first parse both values, and put them in a variable. Only once you know that they are both good, append them to the array.

try:
  parsed_month=i.split(' ')[0]    
  parsed_hour=i.split(' ')[1]    
  month.append(parsed_month)
  hour.append(parsed_hour)
except: # catch *all* exceptions
  # print out the string
  # print out the error
  # maybe exit the program with an error (because the input)
user_5084590
Related