Dear PACE research community,
During our extended grace period, nearly 1M user jobs from nearly 160 PI groups have completed, consuming nearly 40M CPU hours on the Phoenix cluster. The average wait time in queue per job was less than 0.5 hours, confirming the effectiveness of the measures to ensure fair use of the Phoenix cluster to maintain an exceptional level of quality of service.
With the billing for both storage and compute usage in effect as of April 1st, we are following up to provide an update on a few important points.
Compute billing started April 1:
Throughout March, we’ve sent communications to all PIs in accordance with the PACE’s new cost model, including the amount of compute credits based on the refreshed compute equipment as part of the migration to Coda data center and/or recently purchased equipment from FY20 Phase 1/2/3 purchase(s).
PACE has identified and fixed some discrepancies since our initially communicated information as part of our compute audit, which included resources purchased but not provisioned on time. We apologize for this oversight and encourage users to run pace-quota command to verify the updated list of charge accounts. We’ll follow up with the impacted PIs/users in a separate communication.
Please note that most school-owned accounts, as well as those jointly purchased by multiple faculty members, will show a zero balance, but you can still run jobs with them. We are working to make the balances in those accounts visible to you.
As of April 1, all the jobs that run on Phoenix and/or Firebird clusters will be debited/charged to the provided charge account (e.g., GT-gburdell3, GT-gburdell3-CODA20), and a statement will be sent to PIs at the start of May.
This does NOT necessarily mean that you must immediately begin providing funding to use Phoenix. All faculty and their research groups have access to our free tier. Additionally, if you had access to paid resources in Rich, they have been refreshed with an equivalent prepaid account intended to last for 5 years.
Project storage billing started on April 1:
As announced, quotas to Phoenix project storage were applied on March 31 based on PI choices as part of our storage audit. Users may run pace-quota command to check their research group’s utilization and quota at any time. For further information about the Phoenix storage, please see our documentation. April is the first month where storage quotas incur charges for PIs who have chosen quotas above the 1 TB funded by the Institute.
Showback statements sent to PIs:
Throughout March, we sent out “showback” statements for the prior months’ usage on the Phoenix cluster, which covered the usage for Oct 2020 through Feb 2021. We are in a process of sending the March 2021 showback statements that will also include a storage report. Overall, these statements provided PIs with an opportunity to review their group’s usage and follow up with PACE as needed. Explanations for each of the metrics can be found in our documentation.
No charges were incurred for usage during the grace period, so the showback statements are solely for your information and to guide your usage plans going forward.
User account audit completed:
Users of ECE and Prometheus resources migrated in Nov 2020 did not have all their charge accounts provisioned during their groups’ migration. Since then, we have provided access to these additional accounts for the impacted users. We apologize for any inconvenience this may have caused. Also, as part of our preparation to start billing for computation, on Feb 8, the PACE team sent out a notification to PIs to conduct a review of the job submission accounts and corresponding user lists. We appreciate PIs input throughout this process, and if any changes have occurred in your group since then, or if you would like to add a new user(s) to your account(s), please don’t hesitate to send a request to pace-support@oit.gatech.edu. Users may run the pace-whoami command to see a list of charge accounts they may use.
Additionally, we have created a blog page for the frequently asked questions we have received from our community after the end of the extended grace period on March 31, which we would like to share with you at this time.
If you have any questions, concerns or comments about the Phoenix cluster or the new cost model, please direct them to pace-support@oit.gatech.edu.
Thank you,
The PACE Team