Hardware

Summary

As of July 2020, the cluster comprises 15 nodes with over 250 CPU cores and 2.25 TiB of RAM. Centralized storage features more than 40 TiB of high performance SSD and 11 TiB of HDD capacity, and is accessed by cluster nodes via 10Gb Ethernet.

Head Node (Medusa)

  • 4x 8-core 2.8 GHz Opteron 6320
  • 256 GiB RAM (16x 16GiB DDR3 ECC reg)
  • 1x 10Gb NIC

Purchased 2013.12. Supermicro's specs: A+ Server 2042G-TRF.

Data Node (Zing)

  • 1x 8-core 3.2 GHz Xeon E5-1660 v4
  • 96 GiB RAM (6x 16GB DDR4 2133 ECC reg)
  • 42 TiB SSD (~29.3 TiB usable) and ~26 TiB HDD (~10.5 TiB usable) storage
  • 2x 10Gb bonded NICs

Purchased in 2016.12. Supermicro's specs: SuperChassis 216BE1C-R920LPB and Mainboard X10SRL-F

Backup Node (Thunk)

The only server in this list that is not hosted in G01. It is instead across campus in the G26 data center.

  • 2x 6-core 2.4 GHz Xeon E5645
  • 96 GiB RAM (12x 8GB DDR3 ECC reg)
  • 76.4 TiB HDD (52.1 TiB usable) storage
  • 1x 1Gb NIC

Purchased 2011.12. Supermicro's specs: SuperChassis 836E16-R1200B and Mainboard X8DTH-iF

snake1-6

  • 2x 6-core 2.4 GHz Xeon E5645
  • 96 GiB RAM (12x 8 GiB DDR3 ECC reg)
  • 1x 1Gb NIC

Purchased 2011.12. Supermicro's specs: Twinserver 6016TT-TF

snake7

  • 4x 16-core 2.4 GHz Opteron 6272
  • 256 GiB RAM (32x 8 GiB DDR3 ECC reg)
  • 1x 1Gb NIC

Purchased 2012.06. Supermicro's specs: H8QG6+-F Motherboard

snake8

  • 1x 4-core 2.67 GHz i7 920
  • 18 GiB RAM
  • 1x 1Gb NIC

Purchased 2010.03; formerly amras. Supermicro's specs: RM21706 Chassis and X8STE Motherboard

snake9

  • 1x 8-core 2.0 GHz Opteron 6128
  • 64 GiB RAM (8x 8GiB DDR3 ECC Reg)
  • 1x 1Gb NIC

Purchased 2010 (estimated); formerly "just laying about" in Toemme's lab. Supermicro's specs: H8DGU-F Motherboard

snake10

  • 4x 8-core 2.8 GHz Opteron 6320
  • 512 GiB RAM (32x 16GiB DDR3 ECC Reg)
  • 1x 1Gb NIC

Purchased 2013.12. Supermicro's specs: A+ Server 1042G-TF

snake11

  • 2x 10-core 2.3 GHz Intel Xeon E5-2650v3
  • 96 GiB RAM (6x 16GiB DDR4 ECC Reg)
  • 1.2 TB NVMe mounted at /tmp
  • 1x 10Gb NIC

Purchased 2015.12. Supermicro's specs: 825TQ-R740LPB Chassis and X10DRi-T Motherboard

Mulder

  • 1x 4-core 3.2 GHz Intel Xeon E3-1230
  • 16 GiB RAM (4x 4GiB DDR3)
  • 5.5 TB mounted at /media/data
  • 2x 1Gb Nic (?)

TODO: Add Mulder's specs

Networking

An SG-8860 running OpenBSD acts as the router/firewall/etc for the cluster. All cluster traffic uses one 1Gb connection. All web services use a separate 1Gb connection. Internally, the data network is 10Gb (though not all hosts have 10Gb cards). Both the management and DMZ networks are physically separate.

Power

We have one zero-U 3-phase PDU (Raritan PX2-2730). It is connected to a red IEC_60309 power plug. For specifics on which machine is plugged into which PDU outlet, consult the rack diagram.

As we have limited battery capacity, only critical equipment is protected by the UPSs. Both UPSs are monitored by zing via NUT; the head node and zing poll this information; if the main UPS (Eaton) reaches low battery, both Medusa and Zing will shutdown immediately.