I had some maintenance to do on my central syslog server and the server was down for about 30 minutes or so. When the syslog server was back online it was no longer logging syslog messages from my vSphere hosts. First thing I did was check to see if I was receiving syslog messages from the hosts using tcpdump.
tcpdump -i eth0 -A -s 0 udp port 514 and host ESXiHostIpAddress
I thought maybe something changed with the maintenance/upgrade that caused the issue but a quick search resulted in this vSphere 4.x article in the VMware KB. I did not find anything on this issue in vSphere 5 but the 4.x article pointed me in the right direction.
I checked the logging in the Syslog.global.logDir for the hosts and logging had stopped there also.
To correct this you simply have to reload the ESXi syslog service using esxcli.
esxcli –server HostIpAddress –username UserName –password PassWord system syslog reload
Once syslog is reloaded run tcpdump on the system again and you should see syslog messages from the host. I also verified that syslog messages were being logged to Syslog.global.logDir.
I did some quick testing and if I take the syslog server down even briefly (no more than a minute) I have to reload syslog on each of the hosts before log messages a generated.
All is well now. Just have to remember to reload syslog on the ESXi hosts anytime I take down the syslog server.
Anyone know why this happens or if there is a way to prevent it from happening (other than never taking down the syslog server)?