How to avoid memory leak when dealing with KMeans for example in this code, I am trying to find the best WCSS to use in KMean clustering

user_13385039 0
Viewed 4295
wcss = []

for i in range(1,10):
    kmeans = KMeans(i)
    kmeans.fit(x_scaled)
    wcss_iter = kmeans.inertia_
    wcss.append(wcss_iter)

I get the following warning when I run the code: C:\Users\grahv\anaconda3\lib\site-packages\sklearn\cluster_kmeans.py:881: UserWarning: KMeans is known to have a memory leak on Windows with MKL, when there are less chunks than available threads. You can avoid it by setting the environment variable OMP_NUM_THREADS=1. warnings.warn(

k-means python
3 Answers

As the warning says:

UserWarning: KMeans is known to have a memory leak on Windows with MKL, when there are less chunks than available threads. You can avoid it by setting the environment variable OMP_NUM_THREADS=1.

Ignoring such a warning is not a good choice at all! So it's better to try to put effort into solving the issue. to set the environment variable OMP_NUM_THREADS to 1 in Windows 10, You should follow these steps:

  1. Hit Windows Button on your keyboard.
  2. search for "Edit the system environment variables"
  3. click on "Environment variables..."
  4. in the System Variables section, click on "New..." enter image description here
  5. Fill the fields with suitable content(as what mentioned in warning of sklearn): enter image description here
  6. click on "OK" and restart your machine.

Then you can try to run your code without getting the warning.

user_117471480

I was also getting the same warning. I didn't find anything relevant to this. So I used

import warnings
warnings.filterwarnings('ignore')

to ignore the warnings.

user_116031940

you can set the environment variable by

import os
os.environ["OMP_NUM_THREADS"] = '1'

before import kmeans

user_198206120
Related