How to perform cluster with weights/density in python? Something like kmeans with weights?

user_971888 0
Viewed 5450

My data is like this:

powerplantname, latitude, longitude, powergenerated
A, -92.3232, 100.99, 50
B, <lat>, <long>, 10
C, <lat>, <long>, 20
D, <lat>, <long>, 40
E, <lat>, <long>, 5

I want to be able to cluster the data into N number of clusters (say 3). Normally I would use a kmeans:

import numpy as np

import matplotlib.pyplot as plt
from scipy.cluster.vq import kmeans2, whiten
coordinates= np.array([
           [lat, long],
           [lat, long],
            ...
           [lat, long]
           ])
x, y = kmeans2(whiten(coordinates), 3, iter = 20)  
plt.scatter(coordinates[:,0], coordinates[:,1], c=y);
plt.show()

The problem with this is it does not account for any weighting (in this case, my powergenerated value) I want to ideally have my clusters taking the value "powergenerated" into account, trying to keep the clusters not only spatially close, but also have close to relatively equal total powergenerated.

Should I be doing this with kmeans (or some other method)? Or is there something else I should be using for this problem that would be better?

scikit-learn cluster-analysis scipy algorithm python
3 Answers

If you are looking for a solution where you form clusters based on coordinates and power being weights to these co-ordinates, you can add sample_weight= power. This will give you clusters based on coordinates and centroid will be leaning towards the higher weights observations in the cluster

user_110823640

Sumup

There appears to be a lot of confusion between OP and answers. A brief sumup:

Input:

  • power plants with lat/lon and generated power [3D-array]

Desired output:

  • clusters (groups of power plants) with similar cumulative generated power
  • power plants in a cluster must be geographically close/coherent

Partial solutions

  • any kmeans implementation (only takes care of geographical proximity and coherence, without weight)
  • SciKit Learn's weighted kmeans (notwithstanding the sample_weight-parameter it cannot weight the data points but instead only moves the cluster centroids to the cluster's point of gravity
  • accepted answer doesn't respect output condition no 2 (geographical coherence)

Solution

The only solution I found is this repo. Confusingly it is also called "weighted k-means" but instead of SciKit Learn's implementation it really does fulfil both criteria above.

To get started clone the repo and run example.py. For my use case the results are pretty good.

Once you get to the point of adding the cluster numbers back to your original dataframe, unfortunately a small hack is needed but it still works.

user_130835830
Related