As reported by many of our users, we are experiencing storage related slowness on the majority of login nodes. At this point, we have reason to believe that this is caused by heavy-duty data operations running on login nodes by several users. We are currently working on pinpointing these processes contributing to the problem and the users running them.
We’d like to once again ask all of our users to not perform any data operations (e.g. SFTP connections, rsync, scp, tar, zip/unzip, etc) on login nodes. Instead, please use the data mover machine (iw-dm-4.pace.gatech.edu). This will not only help keep the login nodes responsive, but will provide you with a significantly faster data performance compared to login nodes.
This issue has been recurring for a long while and PACE has been working on an alternative mechanisms to address this issue permanently. We now have an experimental solution in place and looking for a small group of volunteers to test it. If you are experiencing slowness on login nodes and would like to volunteer for some testing, please contact mehmet.belgin@oit.gatech.edu directly.
In the mean time, PACE system engineers will continue to work on this issue and eliminate the slowness as soon as possible.