System Monitoring

The goal of this assignment is to "peek under the hood" by using various user level tools that monitor system state in both Linux and Windows. You will be asked to use various tools to see how many processes are running, how much memory those processes are using, etc. The questions themselves should be fairly straightforward (especially if you've used these tools before). Hopefully, everyone will learn at least a few tools that they didn't know before!

You may want to consider making a copy of this document (even a text copy if necessary) and cutting and pasting answers from the screen into the document. We may ask you to submit a hardcopy, but it will still make doing the exercise easier.

The point of this exercise is not really to answer the questions correctly but rather to familiarize you with system monitoring tools that will be helpful in illustrating various aspects of this course. So don't worry too much about the details and don't spend too much time on it! I would expect the entire exercise to take no more than 1 - 1 1/2 hours.

SYSTEM MONITORING ON A LINUX PLATFORM

Unix variants (like Solaris, BSD, etc.) and even Linux distributions can vary immensely. You will find these monitoring tools or similar ones on most Unix/Linux platforms.

For each question, you may need to consult the man page for the command referenced. To view the man page for the uname command you type man uname, to search for a man page for something to do with process you type man -k process.

Linux 1.

Bring up a command window from Applications -> System Tools -> Terminal. Use the hostname command to find the name of the computer you are using. Try variations on the hostname command, like hostname -a or hostname -f or hostname --help to find out what else you can do.

Linux 2.

Use the uname command to determine the following

  1. Which version of Linux are you using? (uname -sr)
  2. What is hardware type of your machine? (uname -m)
  3. What information does uname -n return?
Linux 3.

Using the uptime command determine the following

  1. How long has the system been running?
  2. What is the load average on this machine for the past 1, 5 and 15 minutes?
  3. How many users are currently logged on?
Linux 4.

Use the mount and the df commands (df -h is especially nice if it works) to determine the following:

  1. How much total space is reserved for the root filesystem (/)?
    1. How much of that space is free?
  2. What type of filesystem is the root file system? (mount)
  3. Which mounted file system currently has the most free space? How much?
  4. Which mounted file system currently has the most total space? How much?
Linux 5.

Use the vmstat command to determine the following

  1. How much virtual memory is currently is use? (swpd)
  2. How many processes are in an uninterruptible sleep state (b)?
  3. How many context switches are being performed per second (cs)?
  4. How many interrupts are occurring per second? (in)
  5. Use vmstat 5 5 to report statistics 5 times in a row with a 5 second delay. Which, if any, of the statistics reported change during the time you watch?
Linux 6.

Bring up another command window. In the first, run the command top. (Note: Cntrl-C or q will exit top.)

  1. How many processes are running?
    1. Of these, how many are sleeping?
    2. How many are runnable?
    3. How many are zombie processes?
    4. How many are stopped? (Note the summary information at the top of the display.)
  2. What percentage of time is the CPU spending in user mode, is system mode?
    1. How much of the time is it idle?
  3. In the second command window, experiment with starting a variety (3-4) of processes (mozilla, an editor like emacs, running a grep or a find etc.). Describe changes in the top output.
Linux 7.

Run gnome-system-monitor, a GUI version of top (Note: You will need to be sitting at the machine or able to run a remote Xserver). Experiment with the tabs Process Listing and Resource Monitor; look in the Devices section of the Resource Monitor tab to see free space. Look at the colors used in the various color graph lines.

  1. Change view to All Processes. Which processes are using the most physical memory (i.e. resident size)?
Linux 8.

Use the ps command to determine the following information. (Try ps and ps -efl or maybe ps aux).

  1. Consult the manpage for ps to determine what the process state abbreviations S, R and Z mean? (man ps)
    1. Which process is in state R?
    2. Why must this be the case?
  2. Which process has the smallest process id?
    1. What is that id and how long has the process been running?
  3. Can you find any children of the process with the smallest process id?
    1. List some of them and their process ids. (ps -efl will show process id, PID, as well as parent process id, PPID. pstree described below will also help you with this.)
  4. How many processes are running under your user name? (ps aux | grep your_username) You can find your username using whoami (or combine all the commands like this: ps aux | grep `whoami`).
    1. Do you recognize all these processes?
  5. Start a variety of processes you run regularly (vi, find / -name foo -print, emacs, mozilla etc.). Rerun ps and find them in the output. Note their start time and running time.
Linux 9.

Use the pstree command to examine the family tree of all the processes in the system.

  1. What process is at the root of the tree? Run ps again and find this process in the output. Record all of its attributes.
  2. Record the longest "branch" in the tree.
  3. Make a really long branch by spawning one xterm from another, then another from that one, etc. Rerun pstree and record the branch with the processes you just created.
Linux 10.

Use the netstat command.

  1. Use netstat alone to list the open connections. If you are logged in remotely find the connections to the machine you are connecting from. If you are sitting at the machine, try logging in remotely to another machine and locating that connection. Record the properties of one such connection.
  2. Use netstat -rn to display the routing table. What is the default gateway for this machine?
Linux 11.

Many of these monitoring utilities find the information they provide via the /proc file system. /proc contains file and directories which export some kernel level variables. Type: cd /proc

  1. Examine the contents of the file /proc/stat. What does ctxt mean? (man proc)
  2. Execute ls in /proc (ls /proc). Notice all the numbered directories. Each number corresponds to a process id. Determine the process id of your command shell. Execute ls then ls -al of /proc/PROCESSIDOFYOURSHELL/fd.
    1. What do each of the entries in this directory indicate?
    2. What are the contents of /proc/PROCESSIDOFYOURSHELL/cwd?
  3. How do you think the ps command is implemented?
  4. What information would you expect to find in /proc/net?
OPTIONAL Linux 12.

You'll need xosview to answer the following questions, you can run the script do-xosview.sh to get and compile xosview (this can only be done on a linux machine). Download this script, then run like this chmod +x do-xosview.sh; ./do-xosview.sh

  1. Look at the INTs display line. It shows when various interrupts occur. What happens when you move the mouse?
    1. Type on the keyboard?
  2. One of the interrupts should appear to be constantly on. What do you think that is? Look at the file /proc/interrupts to see if you are right
    3. .
OPTIONAL Linux 13.

You may also want to experiment with some of the following commands: kill (can you make a zombie process?), dmesg, procinfo, hdparm, sysinfo, strace.

OPTIONAL Linux 14.

You may also want to experiment with some interesting network commands: /sbin/route, /sbin/arp, /sbin/ifconfig, /sbin/rarp, /sbin/iptables, /usr/sbin/tcpdump, hostname, traceroute6, nslookup.

SYSTEM MONITORING ON A WINDOWS PLATFORM

These exercises should be done on a Windows NT/2000/XP machine. Most of these tools are not available for Windows 98/ME.
Windows 1.

Open a command prompt (Start..Programs..Accessories..Command Prompt). Run hostname and record the name of the computer you are using.

Windows 2.

Run winmsd and explore the variety of information available.

  1. What is the type and speed of the CPU in your machine?
  2. How much space is formatted into your C:\ drive? (Components..Storage..Drives)
  3. How much space is used on your C:\ drive?
    1. How much of it is free?
  4. How much physical memory is there?
  5. What is the paging file and how big is it?
Windows 3.

Run taskmgr (You can also start the task manager by typing Cntl-Alt-Delete).

  1. What is the process id of the system idle process?
  2. Does the task manager itself appear in the list of processes?
  3. What task has the largest memory usage?
  4. How many processes are running in the system (Note: Look on the performance tab rather than counting)?
    1. Are there that many applications running?
  5. What percentage of the total memory is non-paged kernel memory?
Windows 4.

Run perfmon. Within the chart, right click and select "Add Counter". Notice that there are a variety of performance objects.

  1. Add a counter for SystemCalls/sec from the System object. What happens to this quantity if you Search for a file on the C:\ drive? (Note: you may need to change the maximum value using the Properties of the graph to get a good view.) Why?
  2. For some objects, there are many instances. Note that for the Process object you may choose to graph a quantity based on all instances or on one particular process. Start a program like Microsoft Word or Winzip and open a large file. Find its instance in the process list and graph its working set. Then open a large file. What happens to its working set. Why?
Windows 5.

Download pview, Windows Process Explode and execute pview. Also search for pview.exe on the lab machines (Search..Files or Folders C:\Program Files\).

  1. Find the idle process. What fraction of the time does the idle process spend in Privileged and User time?
  2. What is the priority of the thread running in the idle process?
  3. Find the PVIEW process. How many threads are running in the PVIEW process?
  4. Which process has the most threads?
    1. How many?
  5. Start up a variety of applications that you use regularly (Word, netscape, IE, Excel, etc.) and record how many new threads they have. Do any of those applications cause the creation of more than one new process?
Windows 6.

Run netstat at the command prompt.

  1. Use netstat alone to list the open connections. If you are logged in remotely find the connections to the machine you are connecting from. If you are sitting at the machine, try logging in remotely to another machine and locating that connection. Record the properties of one such connection.
  2. Use netstat -rn to display the routing table. What is the default gateway for this machine?
  3. Did you notice any differences in the output of netstat on Windows versus the Linux version?
OPTIONAL Windows 7.

Explore the following tools if available: sc (service control) for manipulating services, cacls\xcacls for manipulating file permissions, netcps (freeware) for diagnosing networking performance issues, cpustres for creating cpu load, diruse for returning the size of a directory and all its children, top (much like on UNIX), qslice, showacls, srvnay, uptime (try also "uptime \\anothermachine"), where, kernprof, i386kd and windbg.

Consult http://www.microsoft.com/windows2000/techinfo/reskit/rktour/server/S_tools.asp for a list of Microsoft resource kit tools and http://www.sysinternals.com for a nice collection of non-Microsoft monitoring tools like DebugView and Filemon.

FEEDBACK

Feedback 1.
  1. What percentage of these commands were new to you?
Feedback 2.
  1. Do you have suggestions for improving this assignment?