Liquid-cooled supercomputing cluster deployed at Lawrence Livermore

Called the “Ruby” cluster, Supermicro’s TwinPro 2U 4 node servers leverage advanced Second Intel^® Xeon^® Platinum processors with built-in AI acceleration and have 192 gigabytes (GB) of onboard memory. The cluster includes a total of almost 85,000 cores that reach an estimated six petaflops of peak performance. With over 1,500 nodes in 26 racks and 16 2U TwinPro servers in each rack (64 nodes), the racks are liquid-cooled using a direct-to-chip approach. These very dense racks, coupled with liquid cooling, enable a significantly smaller data centre footprint and lower energy costs. Liquid cooling can provide up to 40-50% TCO savings by reducing air conditioning and cooling fan usage. Supermicro staged, tested, and orchestrated the rack-level integration and delivered complete plug-n-play systems to LLNL.

“Supermicro’s advanced TwinPro and Ultra 2U dual CPU servers were selected for their extreme density, support for large compute workloads, and the flexible server Building Block Solutions^® approach so that LLNL could build exactly the best cluster configuration for their requirements,” said Charles Liang, president, and CEO of Supermicro. “We recognise the importance of LLNL’s work to support research on the devastating global pandemic and help to discover a vaccine for the COVID-19.”

Funded by the National Nuclear Security Administration’s (NNSA) Advanced Simulation and Computing Program, the Laboratory’s Multi-programmatic and Institutional Computing (M&IC) program, and the Coronavirus Aid, Relief, and Economic Security (CARES) Act, the supercomputing cluster will be used for unclassified programmatic work in support of NNSA’s stockpile stewardship mission, LLNL open science, and the search for therapeutic drugs and designer antibodies against SARS-CoV-2, the virus that causes COVID-19.

“Ruby provides a substantial computing resource in our open collaboration zone, which has experienced a heavy increase in demand due to an uptick in telecommuting and a growth in external collaborations,” said Chris Clouse, acting program director for LLNL’s ASC program. “A resource like Ruby provides a venue for leveraging expertise and tools in the open community for areas that are important to our programmatic missions.”