Creating new arrays based on the difference of previous and following elements of two other arrays

user_5734797 0
Viewed 121

I have some events with their start and end time steps. Array “start” represents the start time steps of 4 events, array “end” represents the end time steps for these events, and array “prop” contains one numerical property for each event (e.g. the 2nd event (1 index) started at time step 12 and finished at time step 14, and its property is 20). Array “diff” shows the difference between the events (from the end of the previous event to the start of the next one). The time difference between the end of the 1st event and the start of the 2nd event is 7 steps. Array “diff” is smaller than the other arrays (“start”, “end”, "prop") by 1 element.

import numpy as np

start=np.array([3,12,16,30])
end = np.array([5,14,18,32])
prop=np.array([10,20,10,30])

diff=np.zeros(len(start)-1)

for i in range(1,len(start)):
    diff[i-1] = start[i] - end[i-1]

print('diff',diff)


diff [ 7.  2. 12.]

The events which are close timewise need to merge. If the difference between 2 neighboring events is smaller than 3 timesteps, they need to merge. For example the 2nd and 3rd events differ by 2 time steps, so they will merge into a new event whose start is time step:12, and its end time step is 18). As for the “prop” array, the max prop[i] between the merged events need to be kept (prop[1] >prop[2]), so 20 will be assigned to the new merged event (merged_prop[1]=20). I would like to have 3 new arrays with the characteristics of all events (merged and not merged) like those:

merged_start=np.array([3,12,30])
merged_end = np.array([5,18,32])  #2nd and 3rd event have been merged
merged_prop=np.array([10,20,30])

I have attached another larger example as well to be more clear about what I want. The 2nd and 3rd events merged to 1 large event, and so did the 4th up to (included) 7th did. 

start_2=np.array([3,12,16,38,42,46,50,60])
end_2=  np.array([5,14,32,40,44,48,54,70])
prop_2= np.array([10,8,20,10,35,10,10,10])

diff_2=np.zeros(len(start_2)-1)

for i in range(1,len(start_2)):
    diff_2[i-1] = start_2[i] - end_2[i-1]

print('diff_2',diff_2)


diff_2 [7. 2. 6. 2. 2. 2. 6.]

#Desirable outputs
merged_start_2=np.array([3,12,38,60])
merged_end_2 = np.array([5,32,54,70])
merged_prop_2= np.array([10,20,35,10])

Another Example

start_3 = np.array([ 3, 12, 18, 38, 42, 46, 50, 60])
end_3  = np.array([ 5, 14, 32, 40, 44, 48, 54, 70])
prop_3  = np.array([10,  8, 20, 10, 35, 10, 10, 10])

#Desirable outputs
merged_start_3=np.array([3,12,18,38,60])
merged_end_3 = np.array([5,14,32,54,70])
merged_prop_3= np.array([10,8,20,35,10])

How can I do it? I am able to extract the indices from arrays "diff","diff_2" which values are lower than 3 but I do not know how to continue.

numpy arrays python
1 Answers

Here is a way you can do that:

import numpy as np

MERGE_THRESHOLD = 3

start = np.array([ 3, 12, 16, 38, 42, 46, 50, 60])
end   = np.array([ 5, 14, 32, 40, 44, 48, 54, 70])
prop  = np.array([10,  8, 20, 10, 35, 10, 10, 10])

# Gap between events
dists = start[1:] - end[:-1]
# Mask events to merge
m = dists >= MERGE_THRESHOLD
# Find first and last indices of each merged group
first_indices = np.flatnonzero(np.r_[True, m])
last_indices = np.r_[first_indices[1:], len(start)] - 1
# Make results
merged_start    = start[first_indices]
merged_end      = end[last_indices]
merged_prop_max = np.maximum.reduceat(prop, first_indices)
merged_prop_sum = np.add.reduceat(prop, first_indices)
elems_per_merge = last_indices - first_indices + 1
merged_prop_avg = merged_prop_sum / elems_per_merge

print(merged_start)
# [ 3 12 38 60]
print(merged_end)
# [ 5 32 54 70]
print(merged_prop_max)
# [10 20 35 10]
print(merged_prop_sum)
# [10 28 65 10]
print(merged_prop_avg)
# [10.   14.   16.25 10.  ]
user_17827920
Related