The DECsystem-10 and DECsystem-20 were two of the best-loved computers of their day. The ran operating systems called TENEX, TOPS-10, and TOPS-20. These machines were mainframes that could be used interactively by dozens of people at the same time. The most interesting features about the machine, from today's point of view:
These kinds of machines were used at ISI for many years; JEEVES, the first implementation of the Internet's Domain Name Service, was developed on TOPS-20, and ran one of the Internet's root servers at ISI for a long time.
One interesting feature of TOPS-20 was its support for offline files. To save disk space, files could be migrated out to tape, but still appear in your directory; when a program attempted to open the file, a request was generated on the operator's console to find and mount the correct tape.
Last week, we discussed shared-memory multiprocessors. We began with the Types of Parallel Machines.
Let's look at the basic hardware layout in the last category. First, the shared-memory architecture:
Then, the distributed-memory architecture:
This week, we are focusing on the latter type.
In the diagram above, fairly obviously, the system performance and scalability will be limited by the bus between the processors and the main memory. To achieve better scalability, the memory can be distributed among multiple nodes, and connected to an interconnect, as in the lower picture. If the system allows all CPUs to access the memory at all nodes using a hardware-based mechanism, as if the memory were local, it is called a distributed shared memory architecture. Because the latency to memory depends on the address of the CPU in the network and the address of the memory in the network, these systems can also be called non-uniform memory access (NUMA) architectures.
One of the key issues we discussed last week was cache coherence. We discussed snooping buses and directory-based protocols, focusing on the former. However, snooping buses don't scale well, so DSM systems generally use a directory-based protocol.
But rather than all nodes receiving changes to the state of every cache block, each memory block has a home directory entry in the cache directory. That directory entry must maintain a list of all nodes that currently have the block cached, and send invalidate messages to them as necessary.
The topology of the network determines a number of charactistics that impact performance:
The table below (from my thesis) lists a few topologies:
In addition to the topologies shown above, there are other important ones:
Clos networks are defined by three integers n, m, and r. n represents the number of sources which feed into each of r ingress stage crossbar switches. Each ingress stage crossbar switch has m outlets, and there are m centre stage crossbar switches. There is exactly one connection between each ingress stage switch and each middle stage switch. There are r egress stage switches, each with m inputs and n outputs. Each middle stage switch is connected exactly once to each egress stage switch.
Such switched topologies may be either blocking or non-blocking. If m ≥ n, the Clos network is rearrangeably nonblocking, meaning that an unused input on an ingress switch can always be connected to an unused output on an egress switch, but for this to take place, existing calls may have to be rearranged by assigning them to different centre stage switches in the Clos network. If m ≥ 2n - 1, the Clos network is strict-sense nonblocking, meaning that an unused input on an ingress switch can always be connected to an unused output on an egress switch, without having to re-arrange existing calls.Before our understanding of the network is complete, we must know a few things about each link:
One of the most prominent examples of a MIMD multicomputer, or cluster, today is the supercomputer IBM Blue Gene. MTBF time of the largest system installation is reportedly only about 6.16 days (dominated by memory failure)!
The machine at LLNL is a 32x32x64 3D torus: 106,496 dual-processor nodes, 64 terabytes of RAM, several additional, special-purpose networks for global barriers, interrupts, and data reduction (e.g., adding up all of the results), and 1,024 gigabit per second interface to its file system of 806 terabytes. The largest system is believed to be capable of sustained performance of 280 teraflops (2.8*10^14 floating point operations per second).
Next week, we will continue with the discussion of multiprocessors, focusing on chip multiprocessor (CMP) systems and synchronization primitives.
Next lecture:
第8回 12月10日
Lecture 8, December 17: Systems:Systems: Multi-Core Chips