PACE A Partnership for an Advanced Computing Environment

January 30, 2015

Short duration of unavailability (Feb 3, 2015)

Filed under: tech support — Semir Sarajlic @ 9:56 pm

On Feburary 3, at 7:00am, in coordination with our Network Team:

– Network team will have one of the main PACE firewall appliances rebooted in order to fix some isses we have been having with these appliances. This may cause a disruption in networking from outside of PACE in to the head nodes of all PACE clusters.

– PACE will also reboot all of the head nodes for PACE clusters to enact patches to fix the GHOST vulnerability.

These disruptions will result in a 15 minutes period of unavailability for the head nodes. Any login sessions and data transfers into PACE established prior to that time will be terminated, and any processes running on the head nodes themselves will also be terminated. Submitted and running user jobs, however, will remain unaffected, as this will only affect the connections from outside of PACE to inside — all internal operations will function as normal.

[Intro to MATLAB and Data Analytics Seminars] Coming to Georgia Tech – Feb 5 & 6

Filed under: Uncategorized — Semir Sarajlic @ 9:12 pm

Complimentary MATLAB and Data Analytics seminars. Faculty, staff, researchers and students are all welcome to attend.

MATLAB and Data Analytics Seminars at Georgia Tech

Session 1
Thursday, February 5
Location: Student Success Center, Press Room A/B
Session Time: 2:00 p.m. – 4:00 p.m.

2:00 p.m. – 4:00 p.m.: Session 1: Data Analysis and Visualization in MATLAB
Learn how MATLAB can be used to visualize and analyze data, perform numerical computations, and develop algorithms.


Session 2
Friday, February 6
Location:  Klaus 1116
Session Time:  2:00 p.m. – 4:00 p.m.

2:00 – 4:00 p.m.: Session 2: Data Science/Data Analytics and Scaling to Big Data
Using Data Analytics to turn large volumes of complex data into actionable information can help you improve design and decision-making processes. However, developing effective analytics and integrating them into business systems can be challenging. In this seminar you will learn approaches and techniques available in MATLAB® to tackle these challenges

To view complete session descriptions and register, visit:

Please contact me with any questions at 508-647-4343 or

January 29, 2015

New Parallel Debugger+Profiler (Allinea Forge) is available!

Filed under: Uncategorized — Semir Sarajlic @ 9:37 pm

Allinea Forge is a GUI debugger and profiler for serial and parallel applications, which is now available on all clusters. In the past, PACE had “DDT”, which is a parallel debugger from the same company. Allinea combined DDT with a parallel profiler “MAP”, and rebranded the product as “FORGE”. For more information, please visit

You can use Forge as follows:

module load forge/4.2.2
dtt      (for the debugger)
map   (for the profiler)

We still keep the old versions of DDT 3.1 and 3.2, which are available via module ‘ddt’, but we strongly recommend using forge from now on.

We have a shared ‘floating’ 128-core license for both tools. This means, when a user is debugging a 100-core run using DDT, another user can use up to 28-cores for either DDT or MAP.

If you are developing codes, DDT is a great way to identify bugs, memory leaks, etc (think of it as gdb in parallel with a easy-to-use GUI). If your code is working, but performing poorly, then MAP is the tool to profile the code and find the bottlenecks. PACE offers a debugging and profiling summer class (, which partly covers DDT. MAP is a new tool for us as well, and we will include it in future sessions once we figure out how it works 🙂

These tools provide an intuitive interface for a shallow learning curve and targets all levels of users without requiring any specific expertise. So if you have any problem nodes, we strongly encourage you to give them a shot. If you still have difficulties, PACE team is always here to help you (and learn together in the process).

January 21, 2015

Board of Regents address new HPC center progress

Filed under: Uncategorized — Semir Sarajlic @ 4:37 pm

President G. P. “Bud” Peterson presented on the development of the High Performance Computing Center, which will soon begin the process of selecting a developer.

More information is available at:

Georgia Tech taking next step to expand Tech Square presence

Filed under: Uncategorized — Semir Sarajlic @ 4:35 pm

The Georgia Institute of Technology has taken another step to reinforce its commitment to support and enhance the Institute’s innovation ecosystem and economic development role in Midtown Atlanta’s Technology Square. The Institute, advised by Cushman & Wakefield, issued a request for qualification (RFQ) for private developers interested in the expansion of Technology Square.

Anchored by a high performance computing center, the development will be a transformative project providing a sustainable platform for expanding interdisciplinary research, collaboration and co-location with industry partners, while also supporting Georgia Tech’s academic and research programs in advanced analytics and computing.

More information is available at:

January 15, 2015

PACE clusters ready for research

Filed under: tech support — admin @ 2:03 am
Our quarterly maintenance is now complete, and the clusters are running previously submitted jobs and awaiting new submissions.
No general issues to report, although we do have some notes for the Atlas and Joe users which have been sent separately.  We’ll apply some lessons learned here to the April maintenance.
As always, please contact us ( for any problems or concerns you may have. Your feedback is very important to us!

January 9, 2015

PACE quarterly maintenance – January ’15

Filed under: tech support — admin @ 10:39 pm

Hi everybody,

Our January maintenance window is upon us.  We’ll have PACE clusters down Tuesday and Wednesday next week, January 13 and 14.  We’ll get started at 6:00am on Tuesday, and have things back to you as soon as possible.

Major items this time around include:

  • routine patching on our DDN system that servers project directories for many users.
  • routine patching on the file server that provides the storage for /usr/local and our virtual machine infrastructure (including most head nodes)
  • firmware updates on some NFS project directory servers to address stability issues

Additionally, the Joe and Atlas cluster users have graciously offered to test out an upgraded version of the Moab/Torque scheduler software.  Presuming we have success with these two clusters, we will look to roll out the upgrades to the rest of the PACE universe during our April maintenance period.  If you use clusters other than Atlas and Joe, this the rest of this announcement will not affect you next week. Users of Atlas and Joe can expect the following:

  • The current version uses a different database, so we will not be able to migrate submitted jobs.  The scheduler will start with an empty queue, and you will need to resubmit your jobs after the maintenance day (sorry for this inconvenience).
  • We will start using “node packing” which allocates as many jobs on a node as possible before jumping on the next one. With the current version, users can submit many single-core jobs, each landing on a separate node, making it more difficult for the scheduler to start jobs that require entire nodes.
  • You will be able to use msub for interactive jobs (which is currently broken due to a bug), although the recommendation from the vendor company is to use “qsub” for everything (we confirmed that it’s much faster than msub).
  • There will no longer be a discrepancy between job IDs generated by msub (Moab.###) and qsub (####). You will always see a single job ID (in plain number format) regardless of your msub/qsub preference.

Other improvements included in the scheduler upgrade:

  • Speed – new versions of Moab and Torque are now multithreaded, making it possible for some query commands (e.g. showq) to return instantly regardless of the load on the scheduler. Currently, when a user submits a large job array, these commands usually timeout.
  • Introduction of cpusets. When a user is given X cores, he/she will not be able to use more than that. Currently, users can easily violate the requested limits by spawning more processes and threads and Torque cannot do much to stop that. This will significantly reduce the job interference and allows us to finally use ‘node packing’ as explained above.
  • Several other benefits from bug fixes and improvements including but not limited to, zombie processes, lost output files, missing array jobs, long job allocation times, etc.

We hope these improvements will provide you with a more efficient and productive computing environment. Please let us know ( if you have any concerns or questions regarding this maintenance period.

January 7, 2015

COMSOL 5.0 and Application Builder Workshop in Atlanta, GA (1/29)

Filed under: Uncategorized — Semir Sarajlic @ 4:18 pm

You’re invited to our free workshop focusing on
COMSOL Multiphysics® version 5.0 and its new features
and additional capabilities. This event will take place on
Thursday, January 29th at Georgia Tech in Atlanta. All attendees
will receive a free two-week trial of the software.

During the workshop you will:
– Learn the fundamental modeling steps in COMSOL Multiphysics
– Set up and solve a simulation through a hands-on exercise
– Convert existing COMSOL models into Apps using the
COMSOL Application Builder

AM Session
8:45am – 9:00am   Registration
9:00am – 10:30am  Simulations in COMSOL Multiphysics 5.0
10:30am – 10:45am Coffee Break
10:45am – 12:00pm Hands-on Tutorial

PM Session
12:45pm – 1:00pm Registration
1:00pm – 2:30pm  Simulations in COMSOL Multiphysics 5.0
2:30pm – 2:45pm  Coffee Break
2:45pm – 4:00pm  Hands-on Tutorial

Event details and registration:

Seating is limited, so advance registration is recommended.
Feel free to contact me with any questions.

Contact information:

Miraj Desai
1 New England Executive Park
Burlington, MA 01803


Powered by WordPress