CS 10051 - Chapter 11

Chapter 11 - Models of Computation

(11.1) - Introduction

It sure would be nice if every problem had an algorithmic solution: to solve any problem all we need to do is build a computer with enough memory and give it enough time. Unfortunately, there are some problems that don't have algorithmic solutions: no computer can solve any of these problems no matter how much time and memory it's given. To show this we describe a model of a computing agent with a memory that can be made as large as it wants and then show some problems that it can never solve.

(11.2) - What is a Model?

Science uses models to study many physical and social phenomena: weather systems, creation of stars, movement of land masses, spread of epidemics, the economy, etc. A model of a phenomenon:

Captures the essence - the important properties - of the real thing.
Probably differs in scale from the real thing.
Suppresses unimportant details of the real thing.
Lacks the full functionality of the real thing.

(11.3) - A Model of a Computing Agent

(11.3.1) - Properties of a Computing Agent

A computing agent:

Can accept input.
Can store information in a memory and retrieve it later.
Can take actions according to the instructions of an algorithm: the choice of what action to take next may depend on the present state of the agent and what input item is presently being processed.
Can produce output.

(11.3.2) - The Turing Machine

In 1936, Alan Turing, a British mathematician proposed a model of a computing agent.

A Turing machine uses a tape for input, memory, and output which can be extended as much as required to the left and to the right. The tape is divided into cells and each cell contains a symbol from a finite alphabet. One of the symbols in the alphabet is b standing for blank. At any time, only a finite number of tape cells contain nonblank symbols.

A Turing machine also has a unit with a finite number of states: 1, 2, ..., k, that can read the symbol in one of the tape cells. Depending on its current state and the symbol it is reading, the unit can:

write a new symbol in the cell (or leave the cell alone);
go to some other state (or stay in the same state); and
move to the neighboring tape cell on the right or left.

One can describe an instruction for a Turing machine with five components enclosed in parentheses: (current state, current symbol, next symbol, next state, direction of move). The instruction (i, j, k, s, d) means:

	if you are in state i
and
	you are reading symbol j
then
	write symbol k on the tape
	go into state s
	move in direction d

A Turing machine program is a set of instructions: the program can't be ambiguous so no two instructions in the set can have identical current-state/current-symbol combinations. The machine always starts a program in state 1 reading the leftmost nonblank cell on the tape. It keeps performing instructions in the program until it halts when it reaches a state and reads a tape symbol that doesn't match any of its instructions.

(11.4) - A Model of an Algorithm

Recall from chapter 1 that:

An algorithm is a well ordered collection of unambiguous and effectively computable operations that, when executed, produces a result and halts in a finite amount of time.

A Turing machine program satisfies this definition provided it always halts in a finite amount of time. Consider the program:

(1, 0, 0, 1, R)
(1, b, b, 1, R)

If this program is given a tape with a single nonblank cell containing a 1 then the program will immediately halt. If, on the other hand, the tape has a single nonblank cell containing a 0 then the program will keep moving to the right and never halt. This program will only halt in a finite amount time provided the tape contains at least one nonblank cell containing some symbol other than 0.

(11.5) - Turing Machine Examples

Section 11.5 in the text shows four examples of Turing machines. Only the Parity Bit machine is described here: it scans a string of 0-cells and 1-cells and adds a 0-cell or a 1-cell to the right of the string to make the number of 1-cells an odd number. The machine is in state 1 whenever it has scanned an even number of 1-cells or in state 2 whenever it has scanned an odd number of 1-cells. When the machine reads the blank cell at the end of the string it writes the appropriate parity bit and halts in state 3. There are six instructions in the program:

(1, 1, 1, 2, R)	Even parity state reading 1, change state
(1, 0, 0, 1, R)	Even parity state reading 0, don't change state
(2, 1, 1, 1, R)	Odd parity state reading 1, change state
(2, 0, 0, 2, R)	Odd parity state reading 0, don't change state
(1, b, 1, 3, R)	End of string in even parity state, write 1 and go to state 3
(2, b, 0, 3, R)	End of string in odd parity state, write 0 and go to state 3

(11.6) - The Church-Turing Thesis

A Turing machine manipulates symbols on its tape. Most every algorithm we have examined manipulates symbols and it is generally thought to be true that:

If there is an algorithm to do a symbol manipulation task, then there is a Turing machine to do that task.

The Turing machine may take many orders of magnitude longer to do the task than a conventional computer performing the algorithm, but the time is still finite provided the computer performs the task in a finite amount of time.

(11.7) - Unsolvable Problems

In section 11.4 we saw a Turing machine that would halt given certain input tapes and never halt given other input tapes. A machine that never halts isn't very useful so before actually running some Turing machine with some input tape we should solve its halting problem:

Decide, given a collection of instructions of the Turing machine together with the initial contents of its tape, whether or not that machine will halt if started on that tape.

The halting problem for simple machines like the one shown in section 11.4 can be easily solved by hand but how is the halting problem of a complicated Turing machine with thousands of instructions solved? What would be nice is a Turing machine, P, that would solve the halting problem for any Turing machine and input tape. To solve the halting problem for some Turing machine, T, and input tape, t, we give P a tape that looks like:

... b b T* b t b b ...

where T* is the collection of instructions of T encoded as a sequence of 0 symbols and 1 symbols. We must insist that P always halts and gives the correct answer, say:

P halts with 1 on its tape if T halts when begun with tape t; else
P halts with 0 on its tape if T never halts when begun with tape t.

Should we try to build P? No, our efforts are doomed to fail: no Turing machine can solve the halting problem for any Turing machine and input tape combination. The proof that P can't be built is a proof by contradiction: assuming P could be built leads to an impossible condition:

Assume there exists a Turing machine, P, that solves the halting problem for any Turing machine, T, and input tape, t, combination.
If the T/t combination does halt then P halts in some state s reading a 1-symbol on its tape so P has no instruction of the form (s,1,-,-,-).
Pick some state number, s', that P never enters and construct another Turing machine, Q, that contains all the instructions of P plus these two instructions:
(s,1,1,s',R)
(s',b,b,s',R)
Note that if Q is given the input tape, ... b b T* b t b b ..., then Q halts if the T/t combination never halts or Q never halts if the T/t combination does halt.
Now we construct another Turing machine, S, that first makes a copy of what appears on its input tape, and then uses the same instructions as Q. What happens if S is given an input tape containing S*, the encoded version of its instruction set? Does the S/S* combination halt or not?
- If the S/S* combination halts then Q must halt given the tape: ... b b S* b S* b b ..., but this means that the S/S* combination never halts. A contradiction.
- If the S/S* combination never halts then Q must never halt given the tape: ... b b S* b S* b b ..., but this means that the S/S* combination does halt. Another contradiction.
The assumption is false: it's impossible to build a Turing machine, P, that solves the halting problem for any Turing machine and input tape combination.

Do we really care that no Turing machine can solve the halting problem for any machine/input combination? Yes. Real programs written in real programming languages to be run on real computers manipulate symbols. The Church-Turing thesis tells us that any such program can be simulated by a Turing machine so there is no algorithm that solves the halting problem in any other programming environment either. For example, it sure would be nice to write a C++ program to test whether any C++ program halts or not but that's impossible.

The halting problem is unsolvable. A number of other problems are also unsolvable.

Kenneth E. Batcher - 11/8/2006