PACE A Partnership for an Advanced Computing Environment

October 20, 2021

TensorFlow update required due to identified security vulnerability

Filed under: Uncategorized — Michael Weiner @ 8:07 pm

Summary: TensorFlow update required due to identified security vulnerability

What’s happening and what are we doing: A security vulnerability was discovered in TensorFlow. PACE has installed the patched version 2.6.0 of TensorFlow in our software repository, and we will retire the older versions on November 3, 2021, during our maintenance period.

How does this impact me: Both researchers who use PACE’s TensorFlow installation and those who have installed their own are impacted.

The following PACE installations will be retired:

Modules: tensorflow-gpu/2.0.0 and tensorflow-gpu/2.2.0

Virtual envs under anaconda3/2020.02: pace-tensorflow-gpu-2.2.0 and pace-tensorflow-2.2.0

Please use the tensorflow-gpu/2.6.0 module instead of the older versions  identified above. If you were previously using  a PACE-provided virtual env provided  inside the anaconda3 module, please use the separate new module instead. You can find more information about using PACE’s TensorFlow installation in our documentation. You will need to update your PBS scripts to call the new module, and you may need to update python code to ensure compatibility with the latest version of the package.

If you have created your own conda environment on PACE and installed TensorFlow in it, please create a new virtual environment and install the necessary packages. You can build this environment from the tensorflow-gpu/2.6.0 virtual environment as a base if you would like, then install other packages you need, as described in our documentation. In order to protect Georgia Tech’s cybersecurity, please discontinue use of any older environments running prior versions of TensorFlow on PACE.

What we will continue to do: We are happy to assist researchers with the transition to the new version of TensorFlow. PACE will offer support to researchers upgrading TensorFlow at our upcoming consulting sessions. The next sessions are Thursday, October 28, 10:30-12:15, and Tuesday, November 2, 2:00-3:45. Visit our training page for the full schedule and BlueJeans links.

Thank you for your prompt attention to this security update, and please accept our sincere apology for any inconvenience that this may cause you. If you have any questions or concerns, please direct them to pace-support@oit.gatech.edu.

October 15, 2021

Hive scheduler recurring outages

Filed under: Uncategorized — Michael Weiner @ 12:35 pm

[Update 11/5/21 3:15 PM]

During the November maintenance period, PACE separated Torque and Moab, the two components of the Hive scheduler. This two-server setup, mirroring the Phoenix scheduler arrangement, should improve stability of the Hive scheduler under heavy utilization. We will continue to monitor the Hive scheduler. If you have any questions or concerns, please direct them to pace-support@oit.gatech.edu.

[Update 10/15/21 5:15 PM]

The Hive scheduler is functioning at this time. The PACE team disabled several system utilities that may have contributed to earlier issues with the scheduler. We will continue to monitor the scheduler status and to work with our support vendor to improve stability of Hive’s scheduler. Please check this blog post for updates.

[Update 10/15/21 4:15 PM]

The Hive scheduler is again functional. The PACE team and our vendor are continuing our investigation in order to restore stability to the scheduler.

[Original Post 10/15/21 12:35 PM]

Summary: Hive scheduler recurring outages

What’s happening and what are we doing: The Hive scheduler has been experiencing intermittent outages over the past few weeks requiring frequent restarts. At this time, the PACE team is running a diagnostic utility and will restart the scheduler shortly. The PACE team is actively investigating the outages in coordination with our scheduler vendor to restore stability to Hive’s scheduler.

How does this impact me: Hive researchers may be unable to submit or check the status of jobs, and jobs may be unable to start. You may find that the “qsub” and “qstat” commands and/or the “showq” command are not responsive. Already-running jobs will continue.

What we will continue to do: PACE will continue working to restore functionality to the Hive scheduler and coordinating with our support vendor. We will provide updates on our blog, so please check here for current status.

Please accept our sincere apology for any inconvenience that this temporary limitation may cause you. If you have any questions or concerns, please direct them to pace-support@oit.gatech.edu.

 

October 14, 2021

PACE’s centralized OSG service, powered with a new cluster “Buzzard”

Filed under: Uncategorized — Semir Sarajlic @ 5:18 pm

We are happy to announce a new addition to PACE’s service portfolio to support Open Science Grid (OSG) efforts on campus and beyond. This service is kick-started by a brand new cluster, named “Buzzard”, funded by an NSF award* lead by Dr. Mehmet Belgin and Semir Sarajlic of PACE, in collaboration with Drs. Laura Cadonati, Nepomuk Otte, and Ignacio Taboada of the Center for Relativistic Astrophysics (CRA). 

Open Science Grid (OSG) is a unique consortium that provides shared infrastructure and services to unify access to supercomputing sites across the nation, making a vast array of High Throughput Computing (HTC) resources available to US-based researchers. OSG has been instrumental in ground-breaking scientific advancements, including but not limited to the Nobel-winning Gravitational Waves research (LIGO).  

Did you know that all of the GT researchers already qualify for OSG? This means you can join today and start running jobs on this vast resource at no cost. We highly encourage you to register for PACE’s next OSG orientation class, which will get you started with the basics of running on OSG.  As an added resource, PACE offers documentation to get researchers quickly started with OSG. 

In addition to training and documentation, PACE offers resource integration services. More specifically, GT faculty members now have an option to acquire new resources to expand Buzzard with their own OSG projects, similar to the High Performance Computing (HPC) services PACE had been successfully offering since 2009 prior to the new cost model. As a part of the NSF award, PACE already started supporting several exceptional OSG projects, namely LIGO, IceCube and CTA/VERITAS, and we look forward to supporting more OSG projects in the future! 

If you are interested in the OSG service, please feel free to reach out to us (pace-support@oit.gatech.edu) and we’ll be happy to discuss how our new service can transform your research. 

Thank you! 

 

* This material is based upon work supported by the National Science Foundation under grant number 1925541. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. 

October 7, 2021

Announcing the PACE OSG Orientation Class

Filed under: Uncategorized — Semir Sarajlic @ 11:59 am

Dear PACE Researchers, 

PACE is pleased to announce the launch of the PACE Open Science Grid (OSG) Orientation class that introduces Georgia Tech’s research community to OSG and the distributed high throughput computing resources that are available via OSG Connect.   Join us for this virtual orientation to learn about OSG and how it may benefit your research needs. 

Please see below the dates for the sessions and the registration form: 

Dates and times:  October 15, 10:30am – 12:15pm 

                               November 11, 1:30pm – 3:15pm 

Registration:         https://b.gatech.edu/3Bi4Yie 

This class is based in part on the work supported by the NSF CC* award 1925541: “Integrating Georgia Tech into the Open Science Grid for Multi-Messenger Astrophysics”. With this award, PACE, in collaboration with Center for Relativistic Astrophysics, added CPU/GPU/Storage to the existing OSG capacity, as well as the first regional StashCache service that benefits all OSG institutions in the Southeast region, not just Georgia Tech.  

This orientation is the first step into PACE’s longer-term plans to support OSG initiatives on campus. Please be on the lookout for more exciting announcements from our team in the very near future. 

We look forward to you joining us for the OSG orientation. 

Best,

The PACE Team

Powered by WordPress