Rachana
0
I was running a simulation on a terminal and the simulation did not go through due to disk space issue. (It reported "No space left on device") Then we cleaned up some space and ran simulation on the same terminal. However, it still complained the space issue. When we ran on a new terminal, the simulation went through. Hence I want to understand the cause of this. Please help
Thank you.
This is a common issue in linux.
If a process has opened a file and not closed it, removing the file only removes the directory entry (think name) from the directory it is in. Until the file is closed by the process or the process terminates the disk space will not be reclaimed.
To find these files you can look through the /proc file system. Every running process can be found in there by it process id (pid).
Here I'm running a python program that opened a file and is doing nothing. If I use ps to find the pid of the process and cd int /proc/<pid>/fd I can see the open file descriptors and the names of the files that are open:
$ pwd
/proc/38246/fd
$ ls -l
total 0
lrwx------ 1 x x 64 Sep 8 15:39 0 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 1 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 2 -> /dev/pts/0
lr-x------ 1 x x 64 Sep 8 15:39 3 -> /tmp/test
If I remove the file /tmp/test I see this:
$ rm /tmp/test
$ ls -l
total 0
lrwx------ 1 x x 64 Sep 8 15:39 0 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 1 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 2 -> /dev/pts/0
lr-x------ 1 x x 64 Sep 8 15:39 3 -> /tmp/test (deleted)
Search through /proc/*/fd/ for files that say deleted.