While sharing common components with most of the campus’ VM infrastructure, it appears none of our VMware servers were actually affected. We have verified that all head nodes are fine.
November 16, 2012
Massive network outage requires head node restarts!
Earlier today, a campus wide network outage disrupted communications between the Head Node VMs and their storage. Some of these may be working, however, nothing that is done on them properly saved. We will be restarting these machines shortly, after which, everything will return to normal.
This should not cause already scheduled jobs to fail, but any scripts running on the head nodes will surely fail.
We will send an “all-clear” when we have completed the list.
Comments Off on Massive network outage requires head node restarts!