In this lecture we explore the far reaches of what computers can (and cannot) do. We also prove the single deepest result in Computer Science, Turing's Halting Theorem (it's related to Goedel's incompleteness proof, among other things), and talk about bigger and smaller infinities. At the end, if there is time we'll relate it all back to compiler optimizations (it not all theory). * We've been showing you lots of powerful programming tools. * You might think that computer can solve any problem, given the right program. * If you've thought a bit about it, you might realize there could be problems with solving, say, social or philosophical or religious problems. So you might only think that they could solve all mathematical problems. But even that isn't true. Today we'll look at some things that CANNOT be computed. * And prove that they can't be computed. * No matter - what program, - what language, - what machine, - how long you wait - ANYTHING. We will look at TWO reasons: 1. There are more mathematical functions than programs. * Lots more. * There are infinitely many of both, * But there is a *bigger* infinity of functions 2. We'll even show you a function which it would be contradictory to compute (an explicit counterexample to the claim that everything is computable). Some desirable compiler optimizations are uncomputable. So this isn't just a math problem (although it is at heart mathematics). Here's what we'll do: 1. There are *countably* many programs. (We can give all of them integers.) 2. There are *uncountably* many functions. (It's impossible to give them all integers.) 3. That means that there must be some function which isn't programmable. 4. Then we'll show you a particular uncomputable function. ---------------------------------------------------------------------- First we're going to show that the set of all Dylan programs is *countable*. Let's say that a set S is *countable* if there is a way to number the elements of the set. For each integer there is a unique element of S, and every element of S has an integer. More formally, S is countable if there is an onto function f: N-->S. (f is onto if, for each s in S, there is some number n with f(n)=s) (in other words, f "covers" S) For example, N is countable: f(n) = n Z = {..., -3, -2, -1, 0, 1, 2, 3, ...} is countable: f(2n) = n f(2n+1) = -n The *rationals* are countable. 1/1 1/2 1/3 1/4 2/1 2/2 2/3 2/4 3/1 3/2 3/3 3/4 4/1 4/2 4/3 4/4 Number these in a diagonal zigzag: 1 2 4 7 3 5 8 6 9 10 etc... f(n) = the n'th rational in that ordering. You can figure it out. ---------------------------------------------------------------------- There are countably many programs: * Any program is a finite string of ASCII characters. * We can count the finite strings over an alphabet. 1 - a 2 - b ... 26 - z 27 - aa 28 - ab etc. * Now, not all of these are legal Dylan programs, * but all Dylan programs are in this list. * So, there are clearly only countably many Dylan programs. But there are an uncountable number of functions. In fact, we will just consider a small subset of all functions, and see that that is uncountable, S = {f: integers->#T/#F} Any element s of S can be thought of as an infinite sequence, showing the output of s for each each integer INPUTS 1 2 3 4 5 6 odd? T F T F T F ... prime? F T T F T F ... etc. Suppose that S were countable. Then there is some mapping f:N-->S which is onto. In other words, we could arrange the elements of S into a sequence so that f(1) is the first element of the sequence, f(2) is the second, etc, and EVERY element of S is in this sequence (f is onto). We'll show that there is no such f. More precisely, we'll show that for any such supposed list, it is possible to construct an element of S that isn't in the list (hence, f is not onto). We can write the effects of f as a table: INPUTS 1 2 3 4 f(1) T F T F f(2) F F F F f(3) F F F T f(4) T T F T ..... . . . But for any such table, one can construct a function r that does not appear in this table (i.e, there is no integer i such that f(i) = r). How? We *diagonalize* r will be an element of S, hence a function from the integers to #t/#f. What is the value of r applied to j? The opposite of the value of f(j) applied to j! INPUTS 1 2 3 4 r = F T T F ... Now, r cannot appear in the table. It isn't f(1) because f(1) and r give different values when applied to 1. It isn't f(2) because f(2) and r give different values when applied to 2. Etc! So, there are uncountably many functions, and countably many programs. We can't compute all functions. Note that this procedure for constructing r will work for *any* choice of f. * It's not just that we left r(f) off that list -- * For *any* list f' with r(f) on it, there is some other number r(f') left off. ---------------------------------------------------------------------- Note: a function from the integers to #T/#F has a name (it's a real number!) So the real numbers are NOT countable! Cantor discovered this around the turn of the century and flipped out... and turned logic and set theory on their heads. Let I = (0,1) = the Open Unit Interval (not including 0 or 1). Suppose that I were countable. For any f:N-->I, we'll construct an element of I that isn't covered (hence, f is not onto). Write down a big oo x oo table of the digits of the numbers. f(1) = 0 . 3 8 5 9 4 0 5 5 f(2) = 0 . 2 3 6 5 3 5 5 5 f(3) = 0 . 4 3 4 4 3 3 3 4 f(4) = 0 . 2 3 4 1 4 2 3 4 ... Now, can construct a real that is not in the table in the same manner as above, pick the digits on the diagonal and make sure that the new real differs from the i-th element of the list in the i-th digit. The real numbers are UNCOUNTABLE -- * There is NO onto function from the integers to the reals * There are A LOT MORE reals than integers. - AMAZINGLY more. This is called a diagonalization argument, btw. ---------------------------------------------------------------------- Why would you ever want to compute one of these functions that can't be computed ? Well, many interesting questions concerning programs are uncomputable. Here's a real useful one: (safe? prog arg) ---> #T if the function call (prog arg) halts with an answer #F if it doesn't. This would be very, very useful, e.g., in finding infinite loops or programs that get errors. But there's a problem. Consider the following program using safe? (define safify (method ((p )) (if (safe? p p) (not (p p)) #f))) Note that give that safe? exists, it must be that safify is always safe on any argument, because it uses safe? to check whether the computation is safe, and only does the computation if it is. Therefore safify must always halt with an answer. For example, (safify (method (x) 4)) --> (not ((method (x) 4) (method (x) 4))) --> (not 4) --> #f What about (safify safify)? (safify safify) is by definition safe, so what value does it have? (safify safify) = (not (safify safify)) by substitution. But this is impossible, there's no Dylan value that is equal to the negation of itself -- this is a contradiction. All we assumed was the existence of safe?, so there CANNOT be a procedure `safe?'. This is Turing's Halting Theorem. An interesting thing that all the neat tricks we've taught you can't possibly help with. ---------------------------------------------------------------------- There are *lots* of useful uncomputable functions. For example, suppose we have in the compiler (bind ((t1 (f)) (t2 (g)) (t3 (+ t1 t2))) t3) Now we'd like to go common subexpression one better: if F and G compute the same value, then we can optimize out the call to G (ie, bind t2 to t1). So how can we tell if two functions F,G compute the same value? Sounds easy, and various special cases are. (method () 3) vs (method () (+ 1 2)) Can we do it in general? NO. It's uncomputable!