I have a point-clould of which I would like to meassure (/approximate) the entropy (density-entropy). The entropy can only be computed if I assign a probability to each point in the point-cloud. What is the common way to do this?
Note:
One Idea I have is to compute the density with a kernel density estimation (wiki):
Is there a common approach, or should I just use kde?
The kernel density estimator is a well established classic https://jakevdp.github.io/PythonDataScienceHandbook/05.13-kernel-density-estimation.html
If you have lots of points or need to be especially fast, a histogram may be a worth a try. The implementation can be quite simple, e.g. python
import numpy as np
hist = np.histogramdd(np.array(data), bins=n_bins)[0]
hist /= hist.sum()
hist = hist.flatten()
hist = hist[hist.nonzero()]
entropy = -0.5 * np.sum(hist * np.log2(hist))
Alternatively, if you are working in high dimensions a gaussian-mixture model may be a helpful proxy. https://scikit-learn.org/stable/modules/density.html