Greetings,
Our quarterly maintenance is now complete. We have no known outstanding issues affecting general operations, but do have some notes for specific clusters which have been sent separately.
Just a reminder that we have removed the modules for old MPI versions (and applications compiled with them), which are known to be incompatible with the new scheduler servers. Please make sure to check your module lists for compatibility before sending new jobs. Accordingly, we have new default versions of the MPI modules. If you do not explicitly specify a version, mvapich2/1.9 or openmpi/1.6.2 will be loaded by default. Our new repository is almost ready for testing, but it requires a post processing step and migration to shared storage, which may take another couple of days. We will send another communication when this is ready.
Given the delays, we have deferred our stretch goals until a future maintenance window. These filesystems successfully moved to their new locations across campus and are traversing the new dual 40-gigabit path between data centers.
I’ll take another opportunity to apologize for the extended downtime this week. We will be taking a critical look at the events that lead to these delays and learn from the events of this week as much as we can.
–Neil Bright