Main Component Functions	• Master node – Controls worker nodes – Manages parallel jobs – Exports file storage – Provides stable compilation and synthesis environment – Handles event logging, data acquisition and instrumentation – Always on-line • Worker nodes – Environment for experiments – Ephemeral – can be taken off line by researchers (sliding rails) – Reconfigurable and extensible (4U: plenty of space for add-on boards)
Standard configuration: • 4U form factor • 4x 940-pin socket • 2x Opteron 875 • 2x RPU110-L160 • 8x 1GB DDR DRAM • Dual 1GigE ports • Myrinet NIC • PCIe FPGA board	DRC DS1004 Compute Node
HT-connected FPGA Module	• RPU110-L160 by DRC • FPGA: Xilinx Virtex4 with 152,052 logic cells • RPware logic usage: 9-13% • HyperTransport 0, 1, 2 – 8bit @ 400MHz per link (DDR) – 3 physical links • On-board Memory – RLDRAM • 128MB @1.6GB/s peak • 18bit wide @200MHz (DDR) – DDR2 • up to 2GB @6.4GB/s • 144bit wide @200MHz (DDR) • Motherboard DRAM – Up to 4GB @6.4GB/s peak – 288bit wide @200MHz (DDR)
• Xilinx Virtex4 FX100 – 94,000 logic cells – 6 Mbit on-die block RAM – 160 DSP slices – Rocket I/O MGTs – Two PowerPC 405 cores • Memory – 16Mx32 bit DDR DRAM – 8Mx16 bit flash • Interfaces – 8-lane PCIe edge connector – SATA – USB 2.0 – 1 GigE PHY – RS-232 – Rocket I/O SMAs – JTAG	Avnet FPGA Board
Power Aware Testbed	• Goals – An accessible and controllable hardware environment – Means for determining energy consumption – Tools for associating energy with computational demand • Power instrumentation – Wall plug power meters (120V AC) – Internal DC rail Hall-effect current sensors
• Revenue grade meter, class 0.5 (Yokogawa PR300) – 128 samples per power signal cycle • Measurement capabilities – Momentary RMS voltage and current – Momentary power (active, reactive, apparent) – Power factor – Energy (active, reactive, apparent) – Line frequency – Reset and min/max of most measured parameters • Remote access – RS-485 up to 19.1kbps – 4 protocols, including Modbus RTU – Ethernet connectivity via an external bridge (NET485)	Wall Plug Power Instrumentation
Hall-Effect DC Rail Instrumentation	• Configurable Texmate Tiger 320 meters – Input: 4-channel 50mV DC converter – Dual-slope 17 bit ADCs, up to 10 samples/s – Displays directly any 3 of 4 measured values – Min/max detection – RS-485 capable at up to 38.4kbps; ASCII and binary Modbus protocols • Non-invasive current probes – Closed-loop Hall effect sensors (0.45% accuracy, <0.15% linearity) – 1000:1 current conversion; 1A in circuit under test maps to 1mV at the meter when using 1ohm reference resistor – Four per node • Reconfigurable arrangement of measurement circuits – Any of distinct 3.3V, 5V and 12V power rails (more can be added) – Selection of individual circuits by reconnecting Faston tabs on EPS extension cables – Primarily for monitoring of motherboard power consumption, but can be extended to peripherals as well
Power Software Stack • Meter access library – Modbus RTU protocol implementation – Encapsulation in TCP/IP packets • Power monitor daemon – Synchronizes and coordinates accesses to the meters from multiple (remote) applications – Ensures good bandwidth utilization/communication latency on serial interconnects – Maintains precise timestamp information – Logs power data – FUSE based data export and configuration interface • Simple command line utilities – Handle both single process and MPI applications – Provide summary power consumption info • Power analyzers and monitors – Post-mortem tools, combining information from multiple sources (hardware counters, traces, profiles) – Sophisticated system monitors	Hall-Effect Sensor Installation

ADVANCED COMPUTER ARCHITECTURE LABORATORY
CONFIGURABLE Configurable GPGPU GPGPU LaTech LaTech Univ of Arkansas Univ of Arkansas Main Page	Test Beds	Support	Contact

CONFIGURABLE TESTBED