Looking for a clustering algorithm for highly noisy data

user_19442059 0
Viewed 86

I am trying to find clusters in some data with high noise (see plot below).

enter image description here

I tried using DBSCAN which sort of worked, but it required quite a bit of manually tuning the input parameters to find the clusters properly. Are there any other good clustering algorithms for dealing with this kind of data?

Some considerations:

  • I am using Julia to do my data processing.

  • The data has periodic boundary conditions in both directions.

  • The number of clusters is known a priori.

  • I am planning to process many datasets in this way, so it should run relatively fast and not require too much manual fiddling.

Thanks!

data-science julia cluster-analysis
2 Answers

I think the algorithm presented here https://arxiv.org/abs/1406.7130 could work for your problem. It is implemented in Julia here https://github.com/twMisc/Clustering-ToMaTo I forked the project to refactor it as a package here https://pnavaro.github.io/ClusteringToMaTo.jl with some examples https://pnavaro.github.io/ClusteringToMaTo.jl/dev/demo2/

Perhaps you can put the code inside your project and adapt it. I hope it helps.

My objective is to offer this algorithm in a cleaner package https://github.com/pnavaro/GeometricClusterAnalysis.jl but it is not finished yet.

user_47460100

What about a generative/probabilistic model ? Maybe it doesn't fit your case, but you can try it very quickly:

using Pkg
Pkg.add("BetaML")
using BetaML
m       = GMMClusterModel(nClasses=K,mixtures=[FullGaussian() for i in 1:K])
fit!(m,X)  # X is a n by d matrix
classes = mode(predict(m))

Let me know!

user_15868600
Related