HPC Updates and forthcoming Developments: June 2018

posted 20 Jun 2018, 06:28 by Michael K Griffiths
A number of developments to HPC systems used by the research community at the University of Sheffield will be taking place over the next few months and towards the start of 2019.

We will keep the research community updated about developments of the next phase of the Sheffield Advanced research Computer, ShARC2.

In this update we will report on changes to Iceberg and on the closure of the N8 HPC machine (polaris), key dates have been highlighted.

's /fastdata storage is approaching the end of it's service life, and is due to be retired from service on 25th June 2018. 
You must make sure that any files that you require are copied from Iceberg's /fastdata onto another location by this date. 
To minimise disruption to workflows we will be making ShARC's /fastdata storage directly accessible on Iceberg.  This storage area has the same 60 day lifetime policy as Iceberg's /fastdata area, and should only be used for temporary files.  Iceberg's nodes will access ShARC's /fastdata storage using traditional gigabit networking rather than using high-performance Infiniband network, so performance will likely be slower than accessing Iceberg's /fastdata area. 

ShARC's /fastdata storage area is now directly accessible on Iceberg's login nodes under the directory location /fastdata-sharc. You can begin to move any files that you still require into this new location. 
From 9th May 2018 we will make ShARC's /fastdata area available on worker nodes under the directory location /fastdata-sharc. 
On 18th June 2018 Iceberg's /fastdata will become read-only. 
From 25th June 2018 Iceberg's /fastdata will no longer be available.  /fastdata-sharc will continue to be accessible for the remainder of the lifetime of Iceberg
Many of Iceberg's compute nodes are now reaching the end of their service lives.  The oldest nodes on Iceberg will be decommissioned towards the end of the summer, after the next HPC upgrade becomes available.  This will remove approximately ~1/3 of Iceberg's available cpu-cores.  However we expect to keep the remaining newer nodes in service for several more years. 
Further details regarding the next HPC upgrade will be shared once procurement has been completed.
If you have any concerns regarding this work please contact research-it@sheffield.ac.uk.

N8 HPC - Polaris

The current machine, Polaris is five years old and is approaching end of life.  It is due to be switched off on the 17th July 2018 i.e. in 4 weeks. 

We strongly urge all past and present users to remove their data now from the machine ahead of the anticipated rush in July.  If you have any questions about removing your data, you should contact your local helpdesk - https://n8hpc.org.uk/support/

Thank you to those of you who have already removed your data.

Background Information

The 8 member universities of N8 HPC have been in consultation for the past year as to how to proceed after this point.  Three universities (Lancaster, Manchester and York) are planning to jointly fund and procure a new cluster.  This will be hosted at Durham, who will receive 5% of available resources in return.  There will also be an investment in cloud-based resources by Manchester and Leeds.  (Sharing of resources will allow users from Lancaster and York access to cloud cycles and users from Leeds access to the new cluster.)

 To discuss resource availability researchers should contact


All 8 universities will be contributing to a N8 HPC Centre of Excellence which will accelerate progress in areas of research that are of strategic importance to the N8 partners, including some that have not made heavy use of technology in the past.  Led by academics, each area will identify software tools, methodology, expertise etc that could benefit each research area.  A research roadmap will be developed for each selected theme, identifying key opportunities for us to accelerate progress, proposing a joint programme of work, and building a community of practice.

Further announcements will be made on the N8 HPC Users mailing list, website (www.n8hpc.org.uk) and Twitter feed (https://twitter.com/N8HPC), regarding removing user data from Polaris, queue shutdown, etc., which will be before the middle of July.
If you have any concerns regarding this work please contact research-it@sheffield.ac.uk.