- Introduction
The problem. A survey of current methods. A brief history of supercomputing. About the book.
Topics:
- Why HPC and what are the motivating driver applications
- What is a supercomputer and its key elements and what makes it go fast as well as degrades its performance
- What is the role of system software, its organization, and its challenges
- How are HPC systems applied to user problems; how are they programmed and what are the basic parallel algorithms employed
- What are key metrics, how are they measured in the real world, and how are benchmarks used
- A short history of supercomputing
- This course: how is it organized and how to make best use of the experience and resources
- Large Scale Applications
Very big problems in science and technology that benefit from high performance computing and what kind of time/space resources they do/will require to achieve their goals.
Topics:
- Major computational problems requiring current and future generation supercomputers
- Resource requirements and challenges implied by such applications
- Examples of specific application codes being used today and the programming languages used to create them
- Enabling Technologies
A brief history of device technology. Current technologies including: logic, storage, and communications.
Topics:
- Major classes of component types required to implement supercomputers
- A historical sequence of technologies used to realize these component classes over the last 60 years
- SIA ITSR roadmap of semiconductor technologies
- Network technology
- Packaging
- Challenges of power consumption, reliability, size, and cost
- Single Node Architecture and Performance
Single node structures including multi-core and an introduction to the use of OpenMP as a programming methodology. Performance metrics and issues related to sustained performance including levels of parallelism and memory structures. External I/O as a segue to using these in larger scalable systems.
Topics:
- Uniprocessor architecture for performance
- Symmetric multiprocessors
- OpenMP programming
- Memory systems and cache latency avoidance
- Parallel Computer Architecture and Performance
The interplay between technology advances and computer architecture. The major forms of HPC system architecture: Vector, SIMD, MPP, DSM, Commodity Clusters. System area networks.
Topics:
- Vector processors
- SIMD arrays
- Distributed shared memory
- MPP
- Commodity clusters
- Network architecture and protocols
- Commodity Clusters: An example of an HPC system
Linux Clusters in detail exposing all aspects of components, organization, system software, operation, and measured capabilities.
Topics:
- Beowulf and NOW commodity clusters
- Specific example
- Loading and running Linux in a distributed environment
- Loading and running Condor
- Loading and running MPI
- Experiments with canned codes and benchmarks
- Benchmarking
Standard benchmarks, metrics, and measurement techniques.
Topics:
- Linpack
- NPB
- Spec
- HPC Challenge
- SPIO
- Metrics for each
- Throughput computing and Condor
Methods for using Condor to run concurrent jobs for capacity processing.
- MPI Programming
Basic programming model, calls, and environment tools.
Topics:
- Communicating Sequential Processes model of computation
- Basic framework for running MPI
- Message passing constructs
- Reduction operators
- Parallel I/O
- Performance Monitoring, Metrics, and Measurement
The tools and methods used to determine what is happening, how well a system is performing, and application, where the bottlenecks are.
Topics:
- Metrics of response time, throughput, bandwidth, latency, delay
- Processor built-in instrumentation and counters
- PAPI
- Parallel Kernel Algorithms and Application Design
A few of the basic algorithm kernels used for a wide range of applications, perhaps based on the 7 dwarves or from Source book. How to design/create a parallel code including issues of distributing data structures, particularly at scale, and how these relate to the algorithms and programming approaches.
Topics:
- Linear solvers
- N-body Barnes-Hut
- Particle in Cell (PIC)
- FFT
- Sparse Matrices
- Domain Specific Programming Environments
Various packages and libraries that go along way in simplifying mainstream parallel application building.
- Visualization
How to assimilate the meaning of output results. Primary packages. What they need to work effectively.
Topics:
- Presentation modes for data
- Tools for visualization
- Interactive visualization and runtime computing steering
- System Software
Basic Operating System and Middleware. Schedulers like Maui and PBS
Topics:
- Linux services for parallel processing
- PBS
- Maui
- LSF
- ROCKS
- Parallel I/O
Getting out to the real world. Achieving persistent storage. Moving massive data between systems. Real time interactive user interfacing.
Topics:
- Checkpoint & Restart
- Parallel file systems
- Grid access to remote shared data
- Performance measurement of I/O
- Beyond the Basics
This briefly mentions all the advanced topics that are not covered in this book with an annotated bibliography and URLs point to good sources. Will try to articulate contending points of view and where the unknowns are giving a sense of where research opportunities are today.
Topics:
- MPI-2 constructs
- Advanced computer architecture
- Alternative programming models
- Other ways of exploiting parallelism
- Towards the Future
The hardware and software challenges facing HPC in the near term and various research strategies for future directions. Will talk about alternative device technologies, architectures, programming and execution models.
Topics:
- The challenges to be overcome to sustain the opportunity of Moore’s Law
- Alternative technologies being explored including nanoscale
- Alternative computer architectures
- Beyond Moore’s Law
- Quantum computing and other alternative paradigms
- Very high level programming models
- Self healing systems