Notes on Lab #11

This lab will introduce you to MPI (Message-Passing Interface), the most popular software platform for HPC (high-performance computing) on clusters (networks of computers connected by a high-speed switch). Because the Unix operating system is by far the most popular OS for HPC, we will start with a discussion of the history and philosophy of Unix, and a crash course on simple Unix tools for software development.

As we will see in our discussion, the Mac OS X operating system runs a variety of Unix. Since our HBAR cluster also runs Unix (Linux, to be precise), it makes sense to do this lab on a Mac instead of in Windows.

Part I: Your first Unix command

To get started, click on the Applications icon in the dock at the right of the desktop (it looks like this). Then click on the Utilities icon at bottom, and then the Terminal icon at the bottom of the Utilities. This is the "virtual terminal" program that you will use to do this lab and the lab next week. You will see a little window where you can type things, next to a prompt -- typically, a dollar or percent sign or arrow, perhaps with your username and/or the name of the computer and/or directory (Unix term for folder) you're working in.

As we did in Matlab, you should get into the habit of using the up- and down-arrow keys (lower-right of keyboard) to retrieve previous commands, rather than re-typing them, which is prone to error. The first Unix command we will use is ls, which in typically terse (cryptic) Unix fashion stands for list directory. Enter this command and see what's in your home directory (where you always start). Then hit the up arrow to retrieve the same command, and enter it again. If you're a Mac user, you'll note that this directory contains the same sub-directories (folders) that you're used to seeing in the Home folder on your Mac. In fact, I will use the terms folder and directory interchangeably to mean the same thing.

Part II: The vi editor

If Unix is the OS for real programmers, then vi is the text editor for real Unix programmers. (In fact, I'm using it right now to edit this web page.) It has a completely stripped-down, keyboard-only interface that saves wear-and-tear on your wrists and puts the entire set of powerful editing commands literally at your fingertips. Switching from a using modern, over-engineered text-editing program like MS Word to using vi is like switching from the family minivan with built-in GPS and Taylor Swift in the CD player to a 1969 Dodge Dart with a V8 engine, standard transmission, and Led Zeppelin on the AM radio. It's like .... actually, it's just a text editor, I guess.

Really, vi is so simple that you can learn most of it in 10 minutes. So do that on your own now (Parts 1 and 2), while I go listen to some Zeppelin.

Part III: Connecting to the HBAR cluster

Okay, I'm back, and it's time to learn our third Unix command, ssh, which we will use to log on to the hbar cluster. The program that runs in your terminal window, allowing you to talk to Unix, is called a shell, and ssh stands for secure shell: it's a safe way of connecting to other computers with encrypted communication. Before you can connect to hbar, you'll have to retrieve the email sent to you by Dean Tallman, containing your hbar username and password. I asked Dean to set things up so that your hbar username is similar to your W&L AD network username (the one you used to log in today), but it probably is not exactly the same. So to log into hbar I for example would type ssh levys@hbar at the Unix prompt. When asked for your password, type the one from Dean's message. Once you're connected, you can, if you like, change your hbar password to something easier to remember (like your AD password), using the Unix passwd command.

Part IV: A simple MPI program

This page has a simple illustration of how to write an MPI program, using the classic "Hello World!" example taught to generations of computer science students. On the hbar cluster you can use the vi program to create the file hello.c:
  [levys@HBAR ~]$ vi hello.c
To help avoid typos, I suggest copying and pasting the commands from your web browser to your hbar shell, too -- then you can just up-arrow to retrieve them. (In fact, if I see people struggling to type commands into the shell, I'm going to get angry and put Taylor Swift on the speakers or something.) This will open up an empty file, into which you can insert text by typing a single i, putting you in insert mode. Now you can copy-and-paste from the example into your vi session. You can just copy-and-paste everything between the first two paragraphs -- i.e, from the /*The Parallel Hello World Program*/ comment line through the closing }. Once you're done pasting, hit ESC to get out of insert mode, hit : (colon, i.e. shift-semicolon) to get into file mode, and type wq to write (save) the file and quit.

High-performance languages like C use a compiler to convert your human-readable program into low-level commands that can be executed optimally on a particular architecture (like HBAR). To compile your program, issue the following command:
  [levys@HBAR ~]$ mpicc -o hello hello.c
(If you get all kinds of crazy errors, it's probably because you didn't copy and paste the code correctly -- perhaps you left out the initial or final character or line. In typical Unix fashion, if everything's okay you get no message, just the prompt.) The mpicc command invokes the MPI C Compiler. The -o option specifies the name of the output file, which is usually the same as the name of the .c file, but without the .c extension.

Now you are ready to run your program. You will use mpirun to run the program on a specified number of processors; for example:
  [levys@HBAR ~]$ mpirun -np 8 hello
If everything is working, you should see an output something like this:
  Hello World from Node 0
  Hello World from Node 3
  Hello World from Node 4
  Hello World from Node 5
  Hello World from Node 1
  Hello World from Node 2
  Hello World from Node 6
  Hello World from Node 7
(The output may be preceded and followed by some warnings or other messages, but don't worry about those right now.) Repeat the run several times (up-arrow!), and you'll see that the node order isn't always the same. This is because MPI is executing each process (copy of the program) concurrently (simultaneously, independently), with no guarantee of who finishes first. MPI decides internally how to dole out the processes to the processors (individual computing nodes), but to a first approximation we can assume one processor per process.

Part V: Timing

Now we will explore some non-trivial aspects of parallel computing using MPI. Use vi to make a copy of cpi.c, which computes an approximate value of Π. Compile the program as you did with the hello example. To run the new program, you will specify not just the number of processors but also the number of intervals over which to compute the value; for example:
  [levys@HBAR ~]$ mpirun -np 4 cpi 1000
uses four processes and 1000 intervals. Now you can experiment to see how long it takes to execute the program under various combinations of these parameters, using the Unix time command:
  [levys@HBAR ~]$ time mpirun -np 4 cpi 1000
One interesting experiment is to start with just one processor and increase the number of intervals by a factor of 10 until it starts to take a non-trivial amount of time (several seconds) to compute the result. Then add more processors and see what happens. Then, conversely, keep the number of intervals small (like 1000) and see what happens as you increase the number of processors. Does using more processors always give you a faster result? Why not? (Hint: Look at the slides starting at #23 in the lectures notes for this week.)

In your writeup, include plots for these two experiments, as well as your explanation of the results. If you're comfortable using Excel and Word on the Mac, you should be able to copy your PDF document (created via Print/PDF) into the turnin folder on L: drive via the AcadShared L-Drive icon on the desktop, but feel free to launch the Stable and do your Excel and Word and submission in Windows if you prefer. (Just remember to log out of both Windows and Mac when you're done!)