Restarting Apache with a funny pid error

A power flux or some other anomaly caused the server to go down this afternoon. Once the power came back up, everything email got going immediately, but apache had some issues. It seems that the pid used by apache was reused by mysql which caused apache to think that it was running when it really wasn't and not want to restart. The resolve the issue I had to kill mysql, then start apache, then start mysql back up.

How did we figure out this was the problem?

1. ps -elf | grep apache did not reveal any apache processes running. This makes sense because the web site was down.
   
2. apachectl start told me that apache was already cranking away on pid XX
   
3. checking out the apache logs located in /opt/apache/logs revealed that no commands were reaching the logs.
   
4. I looked up the pid using ps again and saw that mysql was the process that held that pid.
   
5. Killed mysql using the command ./mysqladmin -u root -p shutdown. mysqladmin is located in /usr/local/mysql/bin/ and the password is the root password for mysql.
   
6. That command killed mysql and freed the pid apache thought it owned. apachectl start got apache rolling again. The only thing left to do is start mysql back up.
   
7. Since I didn't know the command to start mysql, I had to look in the boot.local file stored in /etc/init.d. This file has startup procedures for our server, one of which is mysql.
   
8. The command /usr/local/mysql/bin/safe_mysqld & will kick off mysql and get everything running back to normal.
   

So, to review, the important stuff to remember:

• Where's my apache log?
  
• ps -elf is the easy way to find a process
  
• apachectl is your gateway to manage apache
  
• mysqladmin allows you to stop mySql (among other things)
  
• boot.local houses the command to start mysql, and it is also written above.