Understanding Jupyter notebook processes

Viewed 391

I manage a set of machines where users are able to run jupyter notebooks. Users often forget to check their memory consumption, close old notebooks, etc. So I often have to check myself who and what is eating all the memory or CPU.

Jupyter gives me a hard time doing this. I cannot understand why there seems to be so many processes running for what looks like a single kernel. For example, here is the kind of things I see in ps:

someuser     66865  0.0  0.3 17415896 6679588 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66866  0.0  0.3 17407700 6679452 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66867  0.0  0.3 17407700 6679452 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66868  0.0  0.3 17407700 6679472 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66869  0.0  0.3 17407700 6679472 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66870  0.0  0.3 17407700 6679472 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66871  0.0  0.3 17407700 6679472 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser     66872  0.0  0.3 17407700 6679476 ?    S    07:12   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-a6a6d1ce-397a-484e-85fe-64cd8c8c8a57.json
someuser    124515  0.0  0.3 10509344 6360328 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124516  0.0  0.3 10509344 6360332 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124517  0.0  0.3 10509344 6360332 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124518  0.0  0.3 10509344 6360332 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124519  0.0  0.3 10509344 6360320 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124520  0.0  0.3 10509344 6360320 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124521  0.0  0.3 10509344 6360320 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124522  0.0  0.3 10509344 6360324 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124523  0.0  0.3 10509344 6360324 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124524  0.0  0.3 10509344 6360328 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124525  0.0  0.3 10509344 6360332 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124526  0.0  0.3 10509344 6360332 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124527  0.0  0.3 10509344 6360344 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124528  0.0  0.3 10509344 6360344 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124529  0.0  0.3 10509344 6360348 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124530  0.0  0.3 10509344 6360348 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124531  0.0  0.3 10509344 6360348 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124532  0.0  0.3 10509344 6360352 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124533  0.0  0.3 10509344 6360376 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json
someuser    124534  0.0  0.3 10509344 6360352 ?    S    09:04   0:00 /home/someuser/research/bin/python3 -m ipykernel_launcher -f /run/user/3007/jupyter/kernel-10fa5a85-9c4a-472d-9db4-59cb9ca7aa77.json

These are not threads but real processes! How come there are so many of them? Are they all sharing the same memory space? (I guess so).

0 Answers
Related