IMS: Breakthroughs in Molecular Science

Institute for Molecular Science adopted new HPC system to support massively parallel operations and high-speed computations.

At a Glance:

  • Japan’s Institute for Molecular Science (IMS) provides a place for joint research and exchanges researchers through domestic and international relationships.

  • The Molecular Simulator’s two systems run on Intel® Xeon® Scalable processors and Intel® SSDs for massively parallel computations and deliver the speed necessary for more demanding serial operations.

BUILT IN - ARTICLE INTRO SECOND COMPONENT

Executive Summary
The Institute for Molecular Science (IMS) significantly expanded its computing capabilities with a dual-purpose system designed to serve researchers that need high-performance parallel computing and memory-demanding serial processing. The new system was built on Intel® Xeon® Gold 6148 processors and Intel® Xeon® Gold 6154 processors with 800 GB Intel® SSD DC 3520 Series solid-state drives all interconnected by the Intel® Omni-Path Architecture (Intel® OPA).

Challenge
Japan’s Institute for Molecular Science (IMS) is a center for advanced research in the molecular sciences—both theoretical and experimental. IMS hosts four research departments: Theoretical and Computational Molecular Science, Photo- Molecular Science, Materials Molecular Science, and Life and Coordination- Complex Molecular Science. The organization provides a place of joint-research for the molecular science community, and it exchanges researchers through domestic and international relationships. IMS scientists also work collaboratively with a wide range of investigators across Japan and around the world, supporting breakthroughs in molecular science knowledge. IMS supercomputers have been used for important work in quantum chemistry calculations, band calculations, and molecular dynamics simulations. Recent work has appeared in scientific journals, including Nature (25 February 2016, vol. 530, pp. 465–468).

“The biggest challenge for real breakthroughs comes from the huge number of trial-and-error calculations that researchers have to run on our supercomputers to reveal novel structures and behaviors.” —Shinji Saito, director of Research Center for Computational Science (RCCS) at IMS

While molecular dynamics (MD) simulations are typically highly optimized for parallel computing, many quantum chemistry (QC) algorithms tend to run in serial fashion. In both types of computing, the large problems scientists need to study lead to long run times to gather the data they need to further their work. IMS provides enough CPU time for researchers to tackle such challenges, irrespective of the type of computing they need (serial or parallel).

“Our previous supercomputers were installed in 2011,” commented Saito. “They were running on six-year-old technologies. The numbers of cores and the speed of calculations were not enough for our users today.”

Solution
MD calculation can use thousands of cores at a time. More cores with a non-blocking interconnect allow researchers to run their jobs much faster, or run much larger jobs, compared to systems with fewer cores. But the serial processes of QC calculations require massive amounts of memory with the fastest CPU clock speeds to achieve results quickly.

“Since IMS supports research in both types of computational domains, and since CPU core speeds typically are lower with more cores, we needed a solution that offered both configurations—a system with thousands of cores and one with fewer, faster cores and large memory.” —Fumiyasu Mizutani, section chief of RCCS

IMS worked with NEC* to install two clusters with Supermicro* servers interconnected by Intel® Omni-Path Architecture (Intel® OPA). The new machine is called the High Performance Molecular Simulator. It placed 70 on the November 2017 Top500 list with 1.8 petaFLOPS LINPACK* and 3.1 petaFLOPS theoretical peak performance.1 It went into production at IMS on October 1, 2017.

The Molecular Simulator’s two systems run on Intel® Xeon® Gold 6148 processors with 20 cores for MD’s massively parallel computations, while the Intel® Xeon® Gold 6154 processors with 18 cores running at 3.0 to 3.7 GHz (Turbo) deliver the speed necessary for QC’s more demanding serial operations. To meet the requirements of different types of workloads, the 20-core nodes were configured in a full bi-sectional bandwidth (FBB) topology, while the faster nodes were 1:3 oversubscribed, considering they would not be communicating as much while running their memory-demanding jobs.

The Molecular Simulator also uses 800 GB Intel® SSD DC 3520 Series solid-state drives.

Results
Since the Molecular Simulator went into production, it has run many benchmarks using quantum chemistry calculations, molecular dynamics simulation, memory transfer, and disk performance programs. Additionally, users have begun running their research on the new system. A benchmark of a modified Test397, which is the geometry optimization and frequency calculation, with Gaussian09 Rev.d01 on the new system is approximately 2.1 times faster than that on the old system.2 The new system, with 40,588 cores, provides 7.3X the computational capacity of IMS’ previous system.2

“While these Gaussian benchmark results of this memory intensive workload were calculated prior to applying any ‘Spectre’ and ‘Meltdown’ software mitigations and firmware updates,” Mizutani noted, “further testing of the code indicated no impact to performance after the security updates were applied.”

Now, approximately 1000 jobs using one to 1000 cores by 80 active users are running on the new system constantly and efficiently.

Solution Summary
IMS supports a wide range of molecular science research, including computational research, using its new High Performance Molecular Simulator. The new system provides high performance computing (HPC) for both massively parallel operations and high-speed, memory-demanding serial computations. It integrates 40,588 cores of both Intel® Xeon® Gold 6154 processors and Intel® Xeon® Gold 6148 processors interconnected by the Intel® Omni-Path Architecture (Intel® OPA). The system placed 70 in the November 2017 Top500 list.

Solution Configuration

  • 40,588 cores of Intel® Xeon® Gold 6148 processors and Intel® Xeon® Gold 6154 processors
  • Intel® Omni-Path Architecture (Intel® OPA) fabric
  • Intel® SSD DC 3520 Series drives
  • 216,768 GB memory

Avvisi e limitazioni alla responsabilità

Le caratteristiche e i vantaggi delle tecnologie Intel® dipendono dalla configurazione di sistema e potrebbero richiedere hardware e software abilitati o l'attivazione di servizi. Le prestazioni variano in base alla configurazione di sistema. Nessun sistema informatico può essere totalmente sicuro. Rivolgersi al produttore o al rivenditore del proprio sistema oppure consultare il sito Web https://www.intel.it. // Il software e i carichi di lavoro utilizzati nei test delle prestazioni possono essere stati ottimizzati per le prestazioni solo su microprocessori Intel®. I test delle prestazioni, come SYSmark* e MobileMark*, sono calcolati utilizzando specifici sistemi computer, componenti, software, operazioni e funzioni. Qualsiasi modifica a uno di questi fattori può determinare risultati diversi. Gli acquirenti sono tenuti a consultare altre fonti di informazioni e test prestazionali per valutare appieno i prodotti che intendono acquistare, nonché le prestazioni di tali prodotti se abbinati ad altri prodotti.Per informazioni più complete, visitare https://www.intel.it/benchmarks. // I risultati prestazionali si basano sui test eseguiti nella data indicata nei dettagli della configurazione e potrebbero non riflettere tutti gli aggiornamenti sulla sicurezza pubblicamente disponibili. Per i dettagli, consultare le informazioni sulla configurazione. Nessun prodotto o componente è totalmente sicuro. // Gli scenari di riduzione dei costi descritti sono da intendersi come esempio di come un determinato prodotto Intel®, in circostanze e configurazioni specificate, può avere effetto sui costi futuri e consentire risparmi. Le circostanze possono variare. Intel non garantisce alcun costo o diminuzione dei costi. // Intel non controlla né verifica i dati di benchmark o i siti Web di terze parti citati in questo documento. Si consiglia di visitare i siti Web indicati e verificare se i dati riportati sono accurati. // Alcuni risultati sono stati stimati o simulati utilizzando analisi interna Intel o simulazione di architettura o modellazione, e vengono forniti solo a scopo informativo. Qualsiasi differenza nell'hardware del sistema, nel software o nella configurazione potrebbe influire sulle prestazioni effettive.

Informazioni su prodotti e prestazioni

1

Cluster NEC LX, processore Intel® Xeon® Gold 6148/6154, Intel® Omni-Path Architecture (Intel® OPA) con 40.558 core e prestazioni di picco teoriche di 3,1 petaFLOP.

2

Fujitsu PRIMERGY* CX250 e RX300, processore Intel® Xeon® E5-2697v3 2,9 GHz/2,6 Ghz, InfiniBand FDR/QDR con 12,992 core e una prestazione teorica di 0,437427 petaFLOP secondo http://www.top500.org/site/48473. I risultati prestazionali si basano su test eseguiti in data 8 giugno 2018 e potrebbero non riflettere tutti gli aggiornamenti di sicurezza disponibili pubblicamente.