So I’m starting a bit of a series on my blog. I’ve found there are no good senior level Linux admin questions out there to ask. A lot of companies with a sys admin team have their default questions they ask but when you are hiring your first sys admin and want a senior guy there is no real way to tell how good he is if no one on your team knows Linux all that well.
So maybe some of these will help some companies figure it all out.
The server is running Apache and by mistake one of the log files gets deleted via
Without installing any 3rd party recovery tools how can you recover this file that your boss needs. Also assume that as soon as the log file was deleted that Apache was not restarted.
This is a really tough question to answer. I would think a lot of senior guys might not even know the answer to this. I certainly did not when I got it asked by Google five years ago.
If the person gets it right I would say he has a really great grasp on how Linux /proc system works and I would say he is a pretty solid admin if he answered all your other questions correctly
So the simple answer is the deleted file is still held open by Apache so it can still be recovered in the /proc filesystem.
So here is how to get to it. For example I have a really small access_log
4 -rw-r--r-- 1 root root 2262 Jan 13 12:32 access_log
So lets remove it
rm -f access_log
So the file has been deleted now. So lets find the process number for the main apache process. It will be owned by root
[root@laptop httpd]# ps aux | grep httpd
root 8070 0.0 0.3 38468 10948 ? Ss 12:31 0:00 /usr/sbin/httpd
apache 8072 0.0 0.1 38388 5844 ? S 12:31 0:00 /usr/sbin/httpd
apache 8073 0.0 0.3 40752 11532 ? S 12:31 0:00 /usr/sbin/httpd
So we see that the pid for the main apache process is 8070. So now lets list the file descriptors
ls -lsa /proc/8070/fd
We get something like
0 l-wx------ 1 root root 64 Jan 13 12:34 11 -> /var/log/httpd/access_log (deleted)
As you can see that is our file and its marked as deleted.. so we can do
cp /proc/8070/fd/11 /tmp/access_log
Then from there you can stop apache.. move the access_log in its correct place and start apache back up