Hardware
The main hardware specs of mc2 are described below, namely:
- Hardware Overview
- Head Node
- Compute Nodes
- Networking
- Storage and Quotas
Overview
The mc2 cluster consists of a Head Node (this is the landing machine), plus a twin pair of Compute Nodes (accessible to users via Slurm mediation).
The Head Node offers 24 CPU-cores for interactive and web compute services, including a JupyterHub server. Compute nodes offer up to 196 CPU-cores for interactive and non-interactive, serial or parallel compute services.
Each node counts with 256 GB of error correction code (ECC) RAM. The whole system has access to 67 TB of user storage, seaminglessly accessible from all nodes via NFS over a 10GBASE-t LAN.
Figure: Architecture of mc2 emphasizing the CPUs, volatile and non-volatile storage, as well as connectivity.
Cluster Nodes
mc2 has a heterogeneous architecture, comprising an Intel® Xeon® Head Node and two AMD EPYC Compute Nodes. Processing, memory and storage specifications (per node) are described in the table below:
| Specifications | Head Node | Compute Nodes 1, 2 |
|---|---|---|
| CPU | ||
| Manufacturer | Intel® | AMD® |
| Designation | Xeon® Gold 6252 | EPYCTM 9454 |
| Part No. | CD8069504194401 | 100-000000478 |
| Code name | Cascade Lake | Genoa (Zen 4) |
| No. CPUs (sockets) | 1 | 2 |
| Cores per CPU | 24 | 48 |
| Max turbo Frequency | 3.70 MHz | 3.80 MHz |
| Base frequency | 2.10 Mhz | 2.75 Mhz |
| Level 3 Cache | 35.75 MB | 256 MB |
| Thermal Design Power (TDP) | 150 W | 290 W |
| Max memory speed | 2933 MHz | 4800 MHz |
| PCI Express Revision | 3.0 | 5.0 |
| Instruction Set Extensions | SSE4.2, AVX, AVX2, AVX-512 | SSE4.2, AVX, AVX2, AVX-512 |
| Memory | ||
| Manufacturer | Samsung® | Micron® |
| Designation | Registered DIMM 64 GB | Registered DIMM 64 GB |
| Part No. | M393A8G40BB4-CWE | MTC40F2046S1RC64BD2 |
| Data rate | DDR4 | DDR5 |
| No. DIMMs | 2 | 4 |
| Density | 64 GB | 64 GB |
| Total capacity | 128 GB | 256 GB |
| Max memory speed | 3200 MHz | 6400 MHz |
| Rank organization | 2R × 4 | 2R × 4 |
| Storage | ||
| Manufacturer | Seagate® | Samsung® |
| Designation | IronWolf Pro 3.5" HDD | V-NAND 990 PRO NVMe 4TB |
| Part No. | ST24000NT002 | MZ-V9P4T0BW |
| Interface | SATA 6 GB/s | PCIe 4.0 x4, NVMe 2.0 |
| Sequential Read | Up to 254 MB/s | Up to 7450 MB/s |
| Sequential Write | Up to 230 MB/s | Up to 6900 MB/s |
| Cache memory | 2 TB NVMe® SSD | 4 GB LPDDR4 |
| Density | 24 TB | 4 TB |
| No. volumes | 4 (RAID-Z1) | 4 (RAID-0) |
| Total capacity | 67 TB | 16 TB |
| Mounting point | /home |
/scratch / |
User storage under /home is provided by a NFS-mounted Storage Node, hosting
4×24 TB SATA disks assembled as a ZFS-Z1 array. The /home mount point on Compute Nodes
hooks to the Head Node via NFS across a 10GBASE-t network.
Compute nodes possess local scratch spaces mounted on /scratch for fast I/O operations.
These comprise an array of four 4 TB SSDs in a striped geometry (RAID-0), totaling 16 TB
each.
Users have a limited storage quota at $HOME, which can be monitored with:
[user@hn ~]$ df --human-readable .
Filesystem Size Used Avail Use% Mounted on
home/user 10G 888M 9.2G 9% /home/user