[Update 4/4/22 12:25 PM]
Summary: [Resolved] Free tier charge account balances did not reset on April 1. A manual reset was performed on April 4.
Details: The deleted Perl library that prevented job submissions last Thursday night and Friday morning also caused an error in the monthly reset of free tier charge account balances at midnight on Friday, April 1. Other accounts that reset on a monthly basis were not impacted. PACE manually reset all free tier account balances just before noon today.
Impact: Job submissions to free tier accounts over the last three days would have succeeded only if sufficient leftover balance from March remained. At this time, all free tier accounts have been reset to their full monthly allocation, and jobs run prior to the reset will not count towards April utilization. All faculty and their teams now have access to their full April free tier allocation. Researchers can run the “pace-quota” command to view their available charge accounts and balances.
We apologize for any disruption this may have caused. Please contact us at pace-support@oit.gatech.edu with any questions.
[Original Message 4/1/22 8:49 AM]
Summary: [Resolved] Phoenix users attempting to submit jobs have received an error message that they are not authorized for their charge account.
Details: Beginning yesterday evening, Phoenix users attempting to submit jobs have at times received an error message indicating that they are not authorized for charge accounts to which they should have access. PACE deployed a temporary repair at 6:45 PM yesterday. The issue recurred at midnight, and the temporary repair was again made at 8:15 AM today. We have now identified the root cause as a deleted Perl library on the scheduler and deployed a permanent fix.
Impact: At this time, researchers are again able to submit jobs. Please resubmit any rejected jobs with the usual charge account. Researchers can run the “pace-quota” command to view their available charge accounts and balances. No running jobs were impacted.
We apologize for this disruption. Please contact us at pace-support@oit.gatech.edu with any questions.