So, cerbot was not renewing automatically and the culprit is that the nginx.pid file is empty. Even if I manually add the pid, it gets erased a few hours later.
I decided to run auditctl on the file to see the culprit and I got the following lines:
type=PROCTITLE msg=audit(07/13/22 09:28:44.836:464) : proctitle=/usr/bin/python3 /bin/systemctl
type=PATH msg=audit(07/13/22 09:28:44.836:464) : item=1 name=/run/nginx.pid inode=969508 dev=00:15 mode=file,644 ouid=root ogid=root rdev=00:00 nametype=NORMAL cap_fp=none cap_fi=none cap_fe=0 cap_fver=0
type=PATH msg=audit(07/13/22 09:28:44.836:464) : item=0 name=/run/ inode=8446 dev=00:15 mode=dir,755 ouid=root ogid=root rdev=00:00 nametype=PARENT cap_fp=none cap_fi=none cap_fe=0 cap_fver=0
type=CWD msg=audit(07/13/22 09:28:44.836:464) : cwd=/
type=SYSCALL msg=audit(07/13/22 09:28:44.836:464) : arch=x86_64 syscall=openat success=yes exit=3 a0=0xffffff9c a1=0x7f1b334fcf50 a2=O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC a3=0x1b6 items=2 ppid=18612 pid=18613 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=systemctl exe=/usr/bin/python3.7 subj==unconfined key=(null)
I thought the culprit was certbot, since certbot is the one that uses python3, but I've deleted all certbot cronjobs and this is still happening. The letsencrypt log file had no entries for 9:28am too. I'm not sure how to follow from here, but my reading is some python3 process is creating a new file while truncating it (O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC) and not putting the correct pid inside.
How do I see which process it is? My environment is debian10.