So, I've got a CentOS6 host where some process has cached an old DNS server, and I'm trying to find out what process that is. (Let's say I can't go blindly rebooting servers and I need to physically confirm what program making requests to this old DNS server).
I've been looking at the outgoing DNS traffic from said host to the old DNS server, and it's originating from some short-lived ephemeral port that opens and closes immediately after, and I'm trying to figure out what process requested it.
example:
$ tcpdump -i eth0 port 53 | grep "> old.dns.host"
19:26:33.442632 IP my.host.46133 > old.dns.host.domain: 56541+ PTR? 49.6.123.10.in-addr.arpa. (42)
my.host is still reaching out to the old DNS server old.dns.host, over the ephemeral port 46133. But when checking netstat, that port is gone. Even watching netstat and using -c don't pick anything up.
So in general -- if I have a port that opens only for a few milliseconds for the duration of a request, how do I find out what process opened it?