Loop invariants

A loop invariant is a condition that is true at the beginning and end of every iteration of a loop. The concept is similar to a class invariant, which must be true at the beginning and end of every public method. When you write a loop that works correctly, you are at least implicitly relying on a loop invariant. Knowing what a loop invariant is and thinking explicitly about loop invariants will help you write correct, efficient code that implements tricky algorithms.

Binary search via iteration

Suppose we want to find an element in a sorted array. We can do much better than scanning from left to right: we can use binary search. Here is the binary search algorithm, written as a loop.

binary_search.java

Conceptually, this algorithm is simple. But it is deceptively tricky to get exactly right. How do we know we got the computation of m right? Why is it k <= a[m] and not k < a[m]? Why m and m+1 in the two updates to r and l respectively? If we change any of these decisions, the algorithm is incorrect and will sometimes fail to find the correct element.

Binary search loop invariant

To convince ourselves that we wrote the correct code, we need a loop invariant that describes the conditions that we want the loop body to preserve. For this example, our loop invariant has two clauses:

1. 0 ≤ l ≤ r ≤ a.length-1
2. k ∈ a[l..r]

We use the notation i..j to denote the set {x | i ≤ x ≤ j} = {i,i+1,...,j-1,j}. We use the notation a[i..j] to denote the subsequence of the array a starting from a[i] and continuing up to and including a[j].

If we have a loop and we know what the loop invariant is for that loop, it is often a good idea to document it. In fact, we can document it in a checkable way by using an assert statement that is executed on every loop iteration.

Using loop invariants to show code is correct

Loop invariants can help us convince ourselves that our code, especially tricky code, is correct. They also help us develop code that is correct in the first place, and they help us write efficient code.

To use a loop invariant to argue that code does what we want, we use the following steps:

1. Initialization. Show that the loop invariant is true just after loop initialization. For while loops while (guard) { body }, this means just before entering the loop. For for loops for (init; guard; incr) { body }, this means just after executing init.
2. Preservation. Show that whenever both the loop invariant and the guard are true just before executing the loop body, then the loop invariant is true just after executing the loop body. The loop invariant may fail to be true at intermediate steps during the execution of the loop body, as long as it is reestablished by the end. (For for loops, the loop body also includes the increment incr.)
3. Postcondition. Show that if the guard is false (so the loop exits) and the loop invariant holds, then the desired result of the loop has been achieved.

Other than coming up with the loop invariant in the first place, the Preservation step is typically the most challenging. The Postcondition step is a crucial step too. If the chosen loop invariant is too weak, this step will not be possible.

These three steps allow us to conclude that the loop satisfies partial correctness, which means that if the loop terminates, it will succeed. To show total correctness, we must show in addition that the loop eventually terminates. To show this, there is a fourth step:

4. Termination. Assuming the loop invariant holds at the start of each iteration, show that some quantity strictly decreases, and that it cannot decrease indefinitely. This quantity is called the decrementing function or loop variant.

Let's try these four steps on the binary search algorithm.

1. Initialization.

   The loop invariant has two parts:

   1. Since a.length is at least 1 by the precondition that k is in the array, and since initially l = 0 and r = a.length-1, we have 0 ≤ l ≤ r ≤ a.length-1.
   2. k ∈ a[l..r] because that is the whole array and the precondition guarantees that k is there.

2. Preservation.

   We use l', r' to represent the values of l, r at the end of the loop. We want to show that if the invariant is true at the beginning of the loop body, that is, if 0 ≤ l ≤ r ≤ a.length-1 and k ∈ a[l..r], then the invariant is true at the end of the loop body, that is, 0 ≤ l' ≤ r' ≤ a.length-1 and k ∈ a[l'..r'].

   Note that m is the average of l and r, rounded down. So we know that l ≤ m ≤ r. In fact, we know something stronger: l ≤ m < r. We cannot have m = r by the assumption that the loop guard l < r is true. We also know that either k ∈ a[l..m] or k ∈ a[m+1..r]. We analyze the two cases separately.

   • Case k ∈ a[l..m]:

     We must have k ≤ a[m], since the array is sorted. In this case, the loop body sets r' = m and l' = l. Since 0 ≤ l ≤ m < r ≤ a.length-1 and k ∈ a[l..m], we have 0 ≤ l' ≤ r' ≤ a.length-1 and k ∈ a[l'..r'] as required.

   • Case k ∉ a[l..m]:

     Since k ∈ a[l..r], we must have k ∈ a[m+1..r]. Also, k ≥ a[m], since the array is sorted. In this case, the loop body sets r' = r and l' = m+1. Since 0 ≤ l < m+1 ≤ r ≤ a.length-1 and k ∈ a[m+1..r], we have 0 ≤ l' ≤ r' ≤ a.length-1 and k ∈ a[l'..r'] as required.

3. Postcondition.

   For the algorithm to be correct, we need a[l] = k. If the loop guard is false, we know l ≥ r. But the invariant (1) guarantees l ≤ r, so we must have l = r. We know from the invariant (2) that k ∈ a[l..r], which has been reduced to a single element, so that must be where k is.

4. Termination.

   The value r − l is guaranteed by the invariant (1) to be nonnegative. In the case where k ∈ a[l..m], we know m < r, so r' − l' < r − l. In the other case, we know l < m+1, so again, r' − l' < r − l. Thus the quantity r − l gets strictly smaller on every loop iteration as long as l < r. Therefore the loop eventually terminates with l = r.

In this case, the loop invariant has two clauses, but it is easy to leave things out of the loop invariant. If clauses are omitted, the invariant may be too weak: Initialization is easier to argue, but it becomes impossible to show Preservation or Postcondition. On the other hand, if the loop invariant is too strong because it contains clauses that shouldn't be there, then Initialization or Preservation become impossible to show.

Let's consider what would have happened had we omitted either of the two clauses from the binary search loop invariant:

1. l ≤ r
   Without this clause, we don't know that we are going to the correct side when we split on m. The Termination argument also fails because the decrementing function is no longer guaranteed to be nonnegative.

2. k ∈ a[l..r]
   Without this clause, we don't know that the loop has found anything when it terminates, so Postcondition fails.

Example: Exponentiation by squaring and multiplication

Here is an implementation of exponentiation that is efficient but whose correctness is not immediately apparent.

Pow.java

Intuitively, this algorithm converts the exponent e into a binary representation, which we can think of as a sum of powers of 2. So e = 2^k₁ + 2^k₂ + ··· and x^e = x^{2k₁+2k₂+···} = x^2k₁·x^2k₂···. (Note that x^2k always means x^(2k), never (x²)^k.)

For example, if e = 11010 in binary, then x^e = x¹¹⁰¹⁰ = x^{10000+1000+10} = x¹⁰⁰⁰⁰·x¹⁰⁰⁰·x¹⁰.

By repeatedly halving y and inspecting the resulting parity, the algorithm finds each of the “1 digits” in the binary representation of e, corresponding to the terms 2^k_i, and for such a digit at position k, multiplies into r the appropriate factor x^2k. However, the loop invariant will help convince us that it really does work. The loop invariant captures that part of the final result has been transferred into r and what remains is b^y.

Let's consider the four steps outlined above.

1. Initialization.

   Initially, r=1, b=x and y=e, so trivially we have r·b^y = x^e. Also y ≥ 0 since initially y = e and e ≥ 0 as a precondition.

2. Preservation.

   Let us use y', b', and r' to refer to the values of these variables at the end of the loop. We need to show that if y > 0 and r·b^y = x^e at the beginning of the loop body, then y' ≥ 0 and r'·b'^y' = x^e at the end of the loop body. It suffices to show that if y > 0, then r'·b'^y' = r·b^y and y' ≥ 0. There are two cases to consider:

   • Case: y is even. In this case, r' = r, y' = y/2, and b'= b². Therefore, r'·b'^y' = r·(b²)^y/2 = r·b^y, as desired.

   • Case: y is odd. Here we have r'=r·b, y' = (y-1)/2, and b' = b². Therefore, r'·b'^y' = r·b·(b²)^(y-1)/2 = r·b·b^y-1 = r·b^y, again.

   In either case, y' ≥ 0, since halving a positive number cannot make it negative.

3. Postcondition.

   If the loop guard is false, then y = 0, since y ≥ 0 and not y > 0. But if y = 0, then r = r·b^y = x^e, and that is the return value.

4. Termination.

   Dividing by two makes the quantity y strictly smaller on every loop iteration, because it is always nonnegative (this is a part of the loop invariant). It can never become negative, so eventually it will become zero and the loop will terminate.

Therefore, the loop terminates and returns the correct value of x^e.

Example: Insertion sort

insertion_sort.java

There are two loops, hence two loop invariants. These loop invariants can be visualized with the following diagram:

Notice that the loop invariant holds in for loops at the point when the loop guard (i.e., i < a.length) is evaluated, and not necessarily at the point when the for statement starts executing. That is, the initialization expression in the for statement can help establish the loop invariant.

Loop invariants in software engineering

Loop invariants capture key facts that explain why code works. If you write code in which the loop invariant is not obvious, you should add a comment that gives the loop invariant. This helps other programmers understand the code, and helps keep them (and you!) from accidentally breaking the invariant with future changes.

It also makes sense to add assert statements that for every iteration check the parts of the loop invariant that are easy to check. Such assertions will catch errors early and expose problems with your understanding of why the code works.