At IU, what is Quarry?

On this page:

• Introduction
• Hardware configuration
• Networking
• Storage
• More information

---

Introduction

Quarry provides a general-purpose Unix computing environment for academic and instructional use. IU's newest supercomputer, Quarry is an IBM HS21 Bladeserver cluster running Red Hat Linux, with TORQUE and Moab for job management and SoftEnv to simplify application and environment configuration. Quarry is a seven teraflop system built from Intel processors.

The user environment is generally similar to AVIDD's, except that Quarry uses SoftEnv. When you log into Quarry for the first time, a file named .soft will be created for you, defining system defaults, such as Intel's C/C++ and Fortran compilers. You can add more packages using the Nano editor (similar to Pico); for a list of the possibilities, enter the command softenv .

If you have questions about the system, email the High Performance Systems group. For help with migrating applications or optimizing code, email the High Performance Applications team.

Hardware configuration

Quarry consists of 112 IBM HS21 Blade servers, each containing two Intel Xeon 5335 quad-core processors, 8GB of memory, a 36GB locally attached SAS disk for local scratch space, and gigabit Ethernet for system interconnects. The Blade configuration provides electrical power and cooling advantages that make Quarry much more cost effective than AVIDD, while occupying much less space.

Following is information about Quarry's cluster nodes:

• Compute Nodes (112, HS21 Bladeserver):
  
  
  • 2 x 2.0GHz quad-core Intel Xeon 5335 processors
  • 8GB 667MHz DDR2 ECC
  • 36GB locally attached SAS disk (250TB GPFS global scratch, 340TB Lustre global scratch)
  • 2 x 1GB Ethernet card
  
  
• Image Servers (2, E1350 X3650):
  
  
  • 2 x 2.5GHz dual-core PowerPC 970MP processors
  • 8GB 533MHz DDR2 SDRAM
  • 73GB SAS disk (67GB in /scratch)
  • 1 x Myricom M3S-PCIXD-2-I (Lanai XP)
  
  
• Management nodes (3, E1350 X3650):
  
  
  • 2 x 1.65GHz dual-core Power5+ processor
  • 8GB 533MHz DDR2 SDRAM
  • 73GB SAS disk
  • 2 x Emulex LP10000 PCI-X/133MHz FC adapters

Networking

Each Blade chassis has six gigabit-Ethernet connections to a Force10 E1200 switch that connects to the other research systems. Fourteen HS21 Blades, each with gigabit-Ethernet adapters, share those six connections.

Storage

The disk for Quarry's 266TB GPFS volume is hosted on 10 DataDirect Networks S2A 9500 storage controllers, each dual-pathed to 5 SAF 4248 chassis. Physical disks are aggregated in an 8+2 RAID configuration; combined with the dual-pathed controllers and active/active GPFS storage hosts, this provides multiple levels of redundancy for Quarry's storage space. Home directories are available via NFSv3 over gigabit Ethernet.

Quarry currently shares the GPFS volume with Big Red and Libra.

Home directory disk space is allocated on the IBM N5500 NAS storage device. The default 10GB disk quota is shared with Big Red, Libra, and the Research Database Complex (RDC) if you have accounts on those systems.

More information

For more information about Quarry, see:

• The Quarry home page
• Software available on Quarry