Lecture 22: Automata correctness

Previous semester's notes: automata correctness (see the last section), automata constructions section 1.1
Definitions: $M$ accepts $x$, $L(M)$, $M$ recognizes $L$
Important general proof ideas:
- vacuously true statements
- strengthening the inductive hypothesis
Constructing machines and proving they are correct
Combining machines together
Review exercises:
- do another case or two of the inductive proof below
- build some automata for different problems, and set up the proofs of correctness
- given automata $M_1$ and $M_2$ for $L_1$ and $L_2$, give an automaton $M$ for $L_1 \cup L_2$. Use the idea that $M$ needs to know what state $M_1$ and $M_2$ are in.

Definitions

Last lecture we defined the extended transition function $\hat{δ}$ for a given machine $M$; $\hat{δ}(q,x)$ indicates the state that $M$ would end up in after processing the string $x$.

We say that a string $x$ is accepted by $M$ if $\hat{δ}(q_0,x) \in A$, i.e. if the machine ends in an accept state after processing $x$.

The language of $M$ (denoted $L(M)$) is the set of accepted strings. Formally, $L(M) = \{x \mid \hat{δ}(q_0,x) \in A\}$.

If $L = L(M)$ the we say $M$ recognizes $L$; in this case, we say that $L$ is DFA-recognizable.

Building a machine

Because DFA cannot go back and look at earlier parts of the string after processing them, they must encode everything they need to know about the string in the state.

When building a DFA, it helps to ask "what do I need to know about the string to decide whether to accept it?"

For example, suppose we want to build a machine for the language \[L = \{x \in \Sigma^* \mid \text{the number of 0's in $x$ is even and the number of 1's in $x$ is odd}\}\]

while processing a string, we will at least need to know whether the number of 0's is even or odd, and whether the number of 1's is even or odd. Let's start with a state representing each possibility:

state	specification
$q_{ee}$	even 0's, even 1's.
$q_{eo}$	even 0's, odd 1's.
$q_{oe}$	odd 0's, even 1's.
$q_{oo}$	odd 0's, odd 1's.

Once we have these specifications in mind, we can start to think about the start state and transition function. The empty string has even 0's and even 1's, so we should start in $q_{ee}$.

If we are in $q_{eo}$, we would expect to have processed an even number of 0's and an odd number of 1's. If we then see one more 0, we should transition to $q_{oo}$, since there are now an odd number of 0's. Therefore, we add a 0 transition from $q_{eo}$ to $q_{oo}$. Using the same reasoning, we can add all of the transitions. Finally, we want to accept $x$ if $x$ has an even number of 0's and an odd number of 1's, so we make $q_{eo}$ an accept state. We end up with the following picture:

diagram of $M$ (click for TeX)

Proving the machine correct

The proof of correctness of the machine is similar to the reasoning we used when building it. Simply setting up the induction proof forces us to write specifications and check all of the transitions.

Claim: With $M$ and $L$ as above, $L(M) = L$.

We'll start the proof, get stuck, and then fix the proof.

Proof attempt: expanding the definitions, we want to show that \[\{x \mid \hat{\delta}(q_{ee}, x) \in F\} = \{x \mid x\text{ has an even number of 0's and an odd number of 1's}\}\]

In other words, we need to show that for all $x \in \Sigma^*$, if $\hat{\delta}(q_{ee},x) = q_{eo}$ then $x$ has an even number of 0's and an odd number of 1's, and also the converse.

We'll start on the first direction, by structural induction on $x$. Let $P(x)$ be the statement "if $\hat{\delta}(q_{ee},x) = q_{eo}$ then $x$ has even 0's and odd 1's".

We need to show $P(ε)$ and $P(xa)$, assuming $P(x)$ in the latter case.

$P(ε)$ says that if $\hat{δ}(q_{ee},ε) = q_{eo}$ then $ε$ has even 0's and odd 1's. This statement actually says nothing, because it is not the case that $\hat{δ}(q_{ee},ε) = q_{eo}$, and $P(ε)$ only makes a claim if $\hat{δ}(q_{ee},ε) = q_{eo}$.

A statement of the form "if $P$ then $Q$" is considered to be true if $P$ is false. For example, if I say "if $x$ is even then $x+1$ is odd" is true, even if $x$ is odd. The statement "if I had a million dollars I would give it to you" is considered logically acceptable if I don't have a million dollars, regardless of my intentions; indeed, you have no way to disprove it.

In this situation, we say that the statement "if $P$ then $Q$" is vacuously true: it is true because it doesn't say anything.

In this case, $P(ε)$ is vacuously true, because $\hat{δ}(q_{ee},ε) \neq q_{eo}$.

We now proceed to the inductive step, where we will get stuck. We inductively assume $P(x)$, which says that if $\hat{δ}(q_{ee},x) = q_{eo}$ then $x$ has an even number of 0's and on odd number of 1's. Our goal is to prove $P(xa)$, which says that if $\hat{δ}(q_{ee},xa) = q_{eo}$ then $xa$ has an even number of 0's and on odd number of 1's.

Since we are only making a claim about the case where $\hat{δ}(q_{ee},xa) = q_{eo}$, we assume this. By definition, $\hat{δ}(q_{ee},xa) = \delta(\hat{\delta}(q_{ee},x),a)$. Therefore, $\hat{\delta}(q_{ee},x)$ is either $q_{ee}$ and $a$ is 1, or $\hat{\delta}(q_{ee},x)$ is $q_{oo}$ and $a$ is 0. This is because these are the only two states with transitions to $q_{eo}$.

But now we're stuck, because our inductive hypothesis doesn't tell us anything about either of these two cases. Therefore, we don't know anything about $x$, so we can't prove that $xa$ has an even number of 0's or an odd number of 1's.

Strengthening the inductive hypothesis

This situation often arises in inductive proofs: we get stuck in our proof, because the inductive hypothesis doesn't tell us enough to proceed. Ironically, we can often make the proof possible by trying to prove more; by adding to our inductive hypothesis, we get more information.

In this case, it was unsurprising that we got stuck. In a sense, we were trying to prove that the $q_{eo}$ state behaved correctly, but without knowing that the rest of the machine works properly, we got stuck. Correctness isn't a property of a single state, but of the machine as a whole.

Let's expand the claim to cover the whole machine. Let $P(x)$ be the statement:

if $\hat{δ}(q_{ee},x) = q_{ee}$ then $x$ has an even number of 0's and an even number of 1's, AND
if $\hat{δ}(q_{ee},x) = q_{eo}$ then $x$ has an even number of 0's and an odd number of 1's, AND
if $\hat{δ}(q_{ee},x) = q_{oe}$ then $x$ has an odd number of 0's and an even number of 1's, AND
if $\hat{δ}(q_{ee},x) = q_{oo}$ then $x$ has an odd number of 0's and an odd number of 1's.

Let's refer to the first part as $P_{ee}(x)$, the second as $P_{eo}(x)$, and so on. Then $P(x)$ = "$P_{ee}(x)$ and $P_{eo}(x)$ and $P_{oe}(x)$ and $P_{oo}(x)$".

Claim: For all $x$, $P(x)$.

Proof: By induction on the structure of $x$. We must show $P(ε)$ and $P(xa)$.

To show $P(ε)$, we must show $P_{ee}(ε)$ and $P_{eo}(ε)$ and $P_{oe}(ε)$ and $P_{oo}(ε)$. $P_{ee}(ε)$ is true, because $ε$ has an even number of 0's and an even number of 1's (it has 0 of both and 0 is even). $P_{eo}(ε)$, $P_{oe}(ε)$, and $P_{oo}(ε)$ are all vacuously true, because $\hat{δ}(ε)$ is neither $q_{eo}$ nor $q_{oe}$ nor $q_{oo}$.

We must now show $P(xa)$, assuming $P(x)$. We must prove 4 things: $P_{ee}(xa)$, $P_{eo}(xa)$, $P_{oe}(xa)$ and $P_{oo}(xa)$. I will do one of these; the other three are similar and are left as an exercise. The $P_{eo}$ case is the same as our aborted proof attempt above, so I will finish that one.

We want to show $P_{oe}(xa)$, i.e. that if $\hat{δ}(q_{ee},xa) = q_{eo}$ then $x$ has an even number of 0's and an odd number of 1's. We assume $\hat{δ}(q_{ee},xa) = q_{eo}$. Then either $\hat{\delta}(q_{ee}, x) = q_{ee}$ and $a = 1$, or $\hat{\delta}(q_{ee},x) = q_{oo}$ and $a = 0$ (it helps to look at the picture here).

In the former case, since $\hat{δ}(q_{ee},x) = q_{ee}$, our inductive hypothesis tells us that $x$ has an even number of 1's and of 0's; since $a = 1$, $xa$ has one more 1 and the same number of 0's, so $xa$ has an even number of 0's and an odd number of 1's, as desired.

In the latter case, since $\hat{δ}(q_{ee},x) = q_{oo}$, our inductive hypothesis tells us that $x$ has an odd number of 1's and of 0's; since $a = 0$, $xa$ has one more 0 and the same number of 1's, so $xa$ has an even number of 0's and an odd number of 1's, as desired.

This completes the proof of that case. The other cases are similar.

Looking back at the induction

Although the proof has many cases, it is completely mechanical (indeed, a proof in an automated proof assistant might read something like: "Proof: induction; cases; auto.").

We first give a specification of each state as our inductive hypothesis. The base case amounts to checking that the start state is correct; all of the other cases are vacuously true. In the inductive step, we have to examine each state. The premise breaks down into all the ways we can transition to that state, so we are required to check each transition.

The proof will be exactly as complicated as the automaton. Which makes sense: to check that the machine works, you have to look at every part of the machine.

Finishing the proof

So far, we have shown that each state behaves properly. We haven't quite finished the original claim:

Claim: $L(M) = L$.

But we've already done the heavy lifting.

Proof: Suppose $x \in L(M)$. Then $\hat{δ}(q_{ee},x) = q_{eo}$, so by above, $x$ has an even number of 0's and an odd number of 1's, so by definition of $L$, $x \in L$.

Conversely, suppose $x \notin L(M)$. Then $\hat{δ}(q_{ee},x) \neq q_{eo}$ so it is either $q_{ee}$, $q_{oe}$, or $q_{oo}$. In each of these cases, the proof above tells us that $x$ either does not have an even number of 0's or it does not have an odd number of 1's (or neither); and therefore $x \notin L$.

Thus $x \in L(M)$ if and only if $x \in L$, so $L(M) = L$, as required.

Union of recognizable languages is recognizable

At the end of lecture, we started building a more general automaton.

Claim: if $L_1$ and $L_2$ are DFA-recognizable, then so is $L_1 \cup L_2$. More concisely, the set of DFA-recognizable languages are closed under union.

Proof outline: Suppose $L_1$ is recognized by $M_1$ and $L_2$ is recognized by $M_2$. We want to build a machine that recognizes $L_1 \cup L_2$. As above, we think about what such a machine needs to know about a string while it is being processed.

If the machine knew what state $M_1$ would be in, and what state $M_2$ would be in, then it would have enough information to decide whether to accept. Next lecture, we will use this idea to build $M$.

state	specification
\(q_{ee}\)	even 0's, even 1's.
\(q_{eo}\)	even 0's, odd 1's.
\(q_{oe}\)	odd 0's, even 1's.
\(q_{oo}\)	odd 0's, odd 1's.