Lecture 28: Turing machines

• We will review the tail end of Kleene's theorem; this material has been added to the lecture 27 notes. This marks the end of the prelim 2 material

• We'll briefly introduce turing machines, which will not be in scope for the second prelim.

Turing machines

Turing machines are a small generalization of finite automata. Like DFA, a Turing machine has a finite set of states, and upon observing a character, changes to a new state. However, the Turing machine is also able to write information back on top if the input, and may move its position to the left as well as the right. At may also move past the end of its input, so that it can keep additional state as it computes. Because running off the end of the input doesn't cause the machine to stop, the machine needs to indicate when to stop processing. In particular, a TM can run forever.

Definitions: Let \(L \subseteq Σ^*\) be a language. If there exists a TM M such that for all \(x \in L\), \(M\) halts and accepts \(x\), and for all \(x \notin L\), \(M\) halts and rejects \(x\), then \(M\) decides \(L\) (and \(L\) is said to be "decidable"). If there is a machine \(M\) such that for all \(x\) in \(L\), \(M\) halts and accepts \(x\), but if \(x \notin L\), \(M\) either halts and rejects or \(M\) runs forever, then we say \(M\) recognizes \(L\) (and \(L\) is said to be "recognizable").

Example: Recognizing \(\{0^n1^n \mid n \in \mathbb{N}\}\)

To show that Turing Machines are more strictly more powerful than DFA, and to give a flavor of how TMs work, let's build a TM that decides the language \(\{0^n1^n \mid n \in \mathbb{N}\}\).

Here's the general strategy: if the string is empty, we will accept. If it is not empty, we will erase a 0 from the beginning of the string, erase a 1 from the end of the string, and then return to the beginning of the string and repeat. If at any point we can't do that, we will reject the string. Here is the machine:

We start in \(q_0\); if we see a blank character, then the string is empty, so we accept. If we see a 1, then the string starts with a 1; it can't be in the language, so we reject. If we see a 0, we replace it with a blank, move to the right, and change to state \(q_1\).

State \(q_1\)'s job is to skip over all the other 0's. If we see a blank, that means the string ends in 0, so we reject. If we see a 0, we leave it and keep going; if we see a 1, then we move to state \(q_2\).

State \(q_3\)'s job is to skip over all the 1's. If we see a 0, then there is some 0 after some 1, so the string should be rejected. If we see a blank, then we've gotten to the end of the string. We step back to the left (so that we are looking at the last 1), and transition to state \(q_3\); from there we replace the 1 with a blank, and then go to state \(q_4\).

\(q_4\)'s job is to rewind back to the beginning of the string: it keeps looping until it finds a blank character, then it moves one space back to the right, so that it is looking at the first character of the string. We then transition back to state \(q_0\) and start the process again.

Interesting facts about Turing machines

We don't have time to go into depth on Turing machines, but let me state several facts about Turing machines without proving them.

• Turing machines are a universal model of computation. Every other "reasonable" model of a computer can be encoded as a TM. For example, you can simulate Java or python or even your computer's processor on a TM, so any software you can run on your computer can be run on a Turing machine. This property is referred to as the "Church-Turing Thesis"

• Turing machines can be encoded as strings. One can write down a string representing a turing machine: the set of states and alphabet must be finite, which means the transition function must also be finite. You can think of the string as the "source code" of the machine. Note that this means that the set of TMs is countable. Since the set of languages is uncountable, this means there are some languages that are not decidable. We give an example of an undecidable problem below.

• Turing machines can "simulate" other Turing machines. For example, there is a "universal turing machine" \(U\), which will accept the string "M,x" if the machine \(M\) would accept \(x\), will reject the string "M,x" if \(M\) would reject \(x\), and will run forever if \(M\) would run forever on input \(x\).

• Turing machines can be encoded in even simpler computational models. For example, we can simulate reading and writing from a tape by pushing and popping elements from two stacks. A stack can in turn be though of as a big number; it turns out that being able to increment and decrement two counters and compare them to 0 is enough to encode a stack. In fact, with two counters, one can encode 4 counters, so with 2 counters, one can simulate a turing machine. One can simulate a counter by the distance between two pebbles, so with the operations "move pebble a to the left", "move pebble b to the left" "move pebble a to the right", "move pebble b to the right", "check whether pebble a is on top of pebble c", "check whether pebble b is on top of pebble c", "accept" and "reject", one can simulate any program that can be written in any computer language.

The halting problem

It is useful to know at least one unsolvable problem. This lets you prove other things are unsolvable by reducing them to the unsolvable problem. Here we will show the halting problem is solvable. In other classes you will make arguments like "if I could build a machine to check that this grammar is unambiguous, then I could use that machine to solve the halting problem; the halting problem is unsolvable, ergo, the ambiguity question is unsolvable as well"

Definition: The halting problem is the language \[L_{HP} = \{"M,x"\mid \text{$M$ on input $x$ halts}\}\]

Informally, a machine \(M_{HP}\) that decides the halting problem would be a machine that looks at the "source code" of another machine, and an input \(x\), and tells you whether \(M\) runs forever on input \(x\) or not.

Claim: The halting problem is recognizable.

Proof: Let \(M_{HP}\) do the following on input "\(M,x\)": first, run the universal Turing machine \(U\) to simulate \(M\) on input \(x\). If \(U\) halts and accepts, accept. If \(U\) halts and rejects, still accept (because \(M\) would have halted). If \(U\) runs forever, \(M\) will be running it forever; this means that \(M_{HP}\) halts and accepts any string \("M,x"\) where \(M\) halts on input \(x\), and on any other string, \(M_{HP}\) either runs forever or rejects (in particular, it runs forever).

Claim: The halting problem is undecidable.

Proof: This means there is no program that always halts and says yes on input "M,x" if and only if \(M\) on input \(x\) would halt. Proof by contradiction. Suppose there were some machine \(M_{HP}\) that decides the halting problem. We can build a diabolical machine \(M_D\) that does the following on input \(x\):

1. Run \(M_{HP}\) on input "x,x". This interprets \(x\) as both the source code of a machine \(M_x\) and as the input. We are guaranteed that \(M_{HP}\) will halt, so we always get to the next step.
2. If \(M_{HP}\) accepted, go into an infinite loop.
3. If \(M_{HP}\) rejects, halt and accept.

Now, what happens if we run \(M_D\) on its own source code? Does it halt? If it does, that means that \(M_{HP}\) on input "\(M_D,M_D\)" will accept; which means \(M_D\) goes to step 2 and goes into an infinite loop. But that's a contradiction, because we said \(M_D\) halts. But if it doesn't halt, then \(M_{HP}\) will reject "\(M_D,M_D\); therefore \(M_D\) will go to step 3, and halt. But this is also a contradiction, because we assumed \(M_D\) will not halt. In either case, we get a contradiction, which means our initial assumption (that \(M_{HP}\) exists) must have been false.