NumPy - calculate histogram intersection

user_1115237 0
Viewed 8326

The following data, represent 2 given histograms split into 13 bins:

key 0   1-9 10-18   19-27   28-36   37-45   46-54   55-63   64-72   73-81   82-90   91-99   100
A   1.274580708 2.466224824 5.045757621 7.413716262 8.958855646 10.41325305 11.14150951 10.91949012 11.29095648 10.95054297 10.10976255 8.128781795 1.886568472
B   0   1.700493692 4.059243006 5.320899616 6.747120132 7.899067471 9.434997257 11.24520022 12.94569391 12.83598464 12.6165661  10.80636314 4.388370817

enter image description here

I'm trying to follow this article in order to calculate the intersection between those 2 histograms, using this method:

def histogram_intersection(h1, h2, bins):
   bins = numpy.diff(bins)
   sm = 0
   for i in range(len(bins)):
       sm += min(bins[i]*h1[i], bins[i]*h2[i])
   return sm

Since my data is already calculated as a histogram, I can't use numpy built-in function, so I'm failing to provide the function the necessary data.

How can I process my data to fit the algorithm?

histogram statistics numpy python
3 Answers

Since you have the same number of bins for both of the histograms you can use:

def histogram_intersection(h1, h2):
    sm = 0
    for i in range(13):
        sm += min(h1[i], h2[i])
    return sm
user_11160680

You can calculate it faster and more simply with Numpy:

#!/usr/bin/env python3

import numpy as np

A = np.array([1.274580708,2.466224824,5.045757621,7.413716262,8.958855646,10.41325305,11.14150951,10.91949012,11.29095648,10.95054297,10.10976255,8.128781795,1.886568472])
B = np.array([0,1.700493692,4.059243006,5.320899616,6.747120132,7.899067471,9.434997257,11.24520022,12.94569391,12.83598464,12.6165661,10.80636314,4.388370817])

def histogram_intersection(h1, h2):
    sm = 0
    for i in range(13):
        sm += min(h1[i], h2[i])
    return sm

print(histogram_intersection(A,B))
print(np.sum(np.minimum(A,B)))

Output

88.44792356099998
88.447923561

But if you time it, Numpy only takes 60% of the time:

%timeit histogram_intersection(A,B)
5.02 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

%timeit np.sum(np.minimum(A,B))
3.22 µs ± 11.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
user_28366210

Some caveats first : in your data bins are ranges, in your algorithm they are numbers. You must redefine bins for that.

Furthermore, min(bins[i]*h1[i], bins[i]*h2[i]) is bins[i]*min(h1[i], h2[i]), so the result can be obtained by :

hists=pandas.read_clipboard(index_col=0) # your data
bins=arange(-4,112,9)   #  try for bins but edges are different here
mins=hists.min('rows')
intersection=dot(mins,bins)
user_40162850
Related