31 March 2020 - Benchmark Q&As updated
For questions and answers regarding the NWSC-3 HPC Benchmarks, refer to the updated NWSC-3 Benchmarks Q&As document.
19 March 2020 - Updated benchmarks released
Please note that the NWSC-3 HPC Benchmarks have been updated to include changes to the GOES and OSU MPI benchmarks. Prospective Offerors for the NWSC-3 Request for Proposal (RFP), which will be released 2 April 2020, should download the updated benchmark code, input cases, and instructions (see below). NCAR does not plan to make any additional changes to the HPC Benchmarks unless there are issues with the ones provided here.
19 July 2019 - Benchmarks released
The NWSC-3 HPC Benchmarks are available ahead of an anticipated release of the NWSC-3 Request for Proposal (RFP) in Q1 of 2020. NCAR does not plan to make any changes to the HPC Benchmarks unless there are issues with the ones provided here.
Page contents
Release Date: July 19, 2019
Last Updated: March 19, 2020
Ensuring that real applications perform well on NCAR computing platforms is critical for getting the best value out of complex and costly high-performance computing and storage resources. Climate and weather applications are large, often with millions of lines of code, and are generally difficult to configure in a way that permits ease of use for things such as system deployments, upgrades, and procurements.
Thus, NCAR has developed a suite of application kernels, micro-benchmarks, and full applications with moderate input cases that can be used as proxies for the full applications and still provide meaningful information and insights into system performance. A few of these are well-known benchmarks that are commonly used in HPC for characterizing system performance.
NCAR's benchmarking applications are listed in the table below, along with file names, sizes, and checksums. These packages include source files, build scripts, and input data sets required to compile and run the applications. In cases where the benchmarks depend on applications and libraries that are not part of the package distributions, you will find version number and download details in the README files.
Documentation and benchmark download packages are available through the Globus-based NCAR Data Sharing Service. Instructions are given below for obtaining a Globus account, installing the required Globus software, and downloading the benchmark packages via the NCAR HPC Benchmarks endpoint. See the Globus instructions below for more information.
Benchmarks
| Name | Description | File Contents | File Name | Size (Bytes) | MD5 Checksum |
|---|---|---|---|---|---|
| CLUBB | Physics Kernel | Instructions | CLUBB_2019-05-19.pdf | 63246 | 006fb83f72d3cc4042c361701f837ea4 |
| Benchmark Files | CLUBB_2019-05-19.tar.gz | 184931304 | 626d0c5108f76c662d8e722fa64cfe82 | ||
| DART_WRF | Model Kernel | Instructions | DART_WRF_2019-05-20.pdf | 59749 | cbe78c13a80073910c0275467c52ee2c |
| Benchmark Files | DART_WRF_2019-05-20.tar.gz | 258607013 | a388e961fce0960d096417cd45e1f00f | ||
| GOES | ML Benchmark | Instructions | GOES16_2020-04-27.pdf | 71665 | 1d9ba301526d25cd768304e14fa5ab14 |
| Benchmark Files | GOES16_2020-04-27.tar.gz | 3578032111 | 5517e495689c75c4f71478d5d3f45e7e | ||
| MG2 | Physics Kernel | Instructions | MG2_2019-05-20.pdf | 61804 | c13a288f425993504ab9ce5db692c008 |
| Benchmark Files | MG2_2019-05-20.tar.gz | 85366943 | 25ebc145a374d3ccd1d410f9d495261a | ||
| MPAS-A* | GPU-capable Atmospheric Model | Instructions | MPAS_2019-06-26.pdf | 274813 | cf523aa8e3a9d889d11817d0d07edca9 |
| Benchmark Files | (For access, follow the MPAS-A instructions below) | ||||
| Input Data | MPAS_2020-04-27_data.tar.gz | 17270465681 | 0ced450ce164b86cb9a6c82a5dcfd966 | ||
| Stream | Memory Bandwidth | Instructions | Stream_2019-05-22.pdf | 69436 | 577d9da38eed93d782d6a046c36d7353 |
| Benchmark Files | Stream_2019-05-22.tar.gz | 20743 | 23d9d58f8d709553c7e409ab1b1e44cc | ||
| WACCM | Physics Kernel | Instructions | WACCM_2019-05-19.pdf | 62391 | 6bb2d8bd1df3471ad93a17540e2c2c17 |
| Benchmark Files | WACCM_2019-05-19.tar.gz | 18289529 | b31c36c58f5ecbbb613caaa39b663b32 | ||
| WRF | Weather Research and Forecasting (WRF) Model | Instructions | WRF_2019-09-06.pdf | 93213 | 2a47071030e4417d4873938333a60af9 |
| Benchmark Files | WRF_2019-09-06.tar.gz | 9778202346 | bec5bf5cc682b14ebb30a2da51d381ab | ||
| OSU MPI | MPI Communications Benchmark | Instructions | osu-micro-benchmarks-5.5_2020-03-12.pdf | 62732 | 4baea167d0698973e751b75e578ac6bb |
| Benchmark Files | osu-micro-benchmarks-5.5_2020-03-12.tar.gz | 765369 | bcb970d5a1f3424e2c7302ff60611008 | ||
Globus instructions
Step 1: Obtain a Globus account
Go to www.globus.org and click the Sign Up button in the upper-right corner.
Step 2: Install Globus Connect Personal
Go to https://www.globus.org/globus-connect-personal and install the version of Globus Connect Personal appropriate for your computer. Versions are available for Mac OS X, Linux, and Windows.
Step 3: Use Globus to download benchmarks
- Access the NCAR HPC Benchmarks folder on Globus. (You will need to log in to Globus with the account created in Step 1.)
- Select the files you wish to download and click Transfer or Sync to in the right-hand pane.
- Select the endpoint you wish to transfer files to. This can be the computer where you installed Globus Connect Personal in Step 2, or another Globus endpoint to which you have access.
- Click on the Start button below the file manager to initiate the transfer.
Your download will be submitted through the Globus Transfer service. You will receive an email when your transfer has completed. You can monitor the transfer by clicking Activity in the left-hand menu to bring up the Globus Activity view.
* Note: Access to the MPAS-A benchmark code is restricted. To obtain access, follow the instructions below.
Instructions for obtaining NWSC-3 MPAS-A benchmark source code
Code releases for the MPAS-A GPU project will occur through the open-source GitHub site. However, before you may access the site, you are required to sign the MPAS-A Confidentiality Agreement. To obtain access to the MPAS-A GPU GitHub site, send both your signed MPAS-A Confidentiality Agreement and your GitHub account/login to Alison Propes, UCAR's Subcontract/Procurement Manager.
Note that all materials (including source code, products derived from source code, and documents) related to NWSC-3 MPAS should not be distributed, either formally or informally, in any form. Publishing any kind of results obtained from the NWSC-3 MPAS-A source code requires written consent from UCAR.