(3.2) Attributes of Algorithms

1. An algorithm must be correct. It should:
   • be solving the correct problem;
   • provide the correct answer for every possible input; and
   • provide accurate answers.

2. An algorithm should be easy to understand since some may use it in computer programs and program maintenance personnel may have to fix bugs in the programs and/or enhance the algorithm.

3. A desirable feature of an algorithm is elegance. For example, in elementary school, Carl Friedrich Gauss (1777-1855) found an elegant algorithm for computing:
   1 + 2 + 3 + ... + 98 + 99 + 100

   But sometimes an elegant algorithm is hard to understand.

4. An algorithm should also make efficient use of the main computer resources: time and memory. Computer technology is advancing rapidly with ever larger memories and ever faster processors but problems are also getting more complex so usage of computer time and memory will always be important.

   A space-efficient algorithm doesn't require much computer memory beyond that required to hold the input data.

   A time-efficient algorithm consumes the minimum amount of computer time. Benchmark time measured with a stopwatch depends on the speed of the computer, the size of the problem, and other factors. To compare different algorithms for the same problem one should count how many times each algorithm performs the same fundamental unit of work on problems of the same size.

An important part of computer science is the analysis of algorithms where we examine the efficiency of algorithms.

(3.3.1) Sequential Search: Figure 3.1 shows the sequential search algorithm from Chapter 2 with n entries instead of 10,000 entries. How many times is each step performed? We consider four cases:

• Best Case: The entry we want is at the beginning of the list ( NAME = N₁ ).
• Average Case: The entry we want is exactly in the middle of the list ( NAME = N_(n+1)/2 ).
• Worst Case: The entry we want is at the end of the list ( NAME = N_n ).
• Not-found Case: The entry we want is nowhere in the list.

Suppose we try to determine the running time of this algorithm on some computer. For j = 1, 2, ..., 9, let c_j be the number of microseconds that that computer requires to perform step j one time. The running times for the four cases on that computer are then:

• Best Case: c₁ + c₂ + c₄ + c₅ + c₆ + c₃ + c₉ microseconds.
• Average Case: n*(c₄ + c₇ + c₃)/2 + (c₄ - c₇ + c₃)/2 + c₁ + c₂ + c₅ + c₆ + c₉ microseconds.
• Worst Case: n*(c₄ + c₇ + c₃) + c₁ + c₂ + c₅ + c₆ - c₇ + c₉ microseconds.
• Not-found Case: n*(c₄ + c₇ + c₃) + c₁ + c₂ + c₈ + c₉ microseconds.

• Best Case: c₁ + c₂ + c₄ + c₅ + c₆ + c₃ + c₉ microseconds.
• Average Case: n*(c₄ + c₇ + c₃)/2 microseconds.
• Worst Case: n*(c₄ + c₇ + c₃) microseconds.
• Not-found Case: n*(c₄ + c₇ + c₃) microseconds.

(3.3.2) Order of Magnitude: The worst case running time of the sequential search algorithm depends on the size of the list, n - the larger the list, the more time it takes to search. On a very short list the algorithm runs so fast that we really don't care how much time it takes. On a very long list the algorithm takes so long that we care very much how much time it takes.

This is true of most every algorithm: we don't worry about how much time is required to solve a small problem, only a very large problem. This lets us ignore all but the most dominant term in the running time formula.

When n is very large the worst-case running time for the sequential search algorithm is n*(c₄ + c₇ + c₃) microseconds where the values of c₄, c₇, and c₃ depend on the speed of the machine. We are really only concerned with the algorithm and not the particular machine it is running on so we ignore these machine constants and just say that the worst-case running time of the sequential search algorithm is order of magnitude n - written θ(n).

Note: Does your Web browser show the Greek letter, theta, before (n) in the previous sentence? If not, try to use a different browser.

The worst-case running time of the sequential search algorithm is proportional to the size of the list - doubling the size of the list will double the worst-case time it takes to search the list. The running times for the average-case and the not-found-case are also θ(n). The running time for the best-case is θ(1) since it is independent of list size.

An algorithm to print out every entry in an n-row by n-column rectangular array has a θ(n²) running time because the array has n² entries.

Note that we always ignore any constant factors in the order of magnitude (theta) notation: whether an algorithm takes n, 2n, 100n, or 1,000,000n work units its order of magnitude growth is always θ(n). We ignore the constant factors because they depend on the speed of a machine running the algorithm: the θ notation shows how the running time of an algorithm grows as the problem size gets larger and larger, independent of what machine is running the algorithm.

The effect of any constant factor disappears as the problem gets larger and larger: a slow machine running a time-efficient algorithm will eventually beat a fast machine running an inefficient algorithm (see the Tortoise and the Hare example on page 97 of the text.)

(3.3.3) Selection Sort: A common task is to sort a list of items into alphabetical or numerical order and there are hundreds of algorithms to solve the problem. Selection Sort (shown in Figure 3.6) is a sort algorithm that's easy to understand: find the largest item in the list and exchange it with the last item; then find the largest of the remaining items and exchange it with the next-to-last item; etc., etc.

To find the largest item in a list of n items requires (n - 1) comparisons; to find the next largest item requires (n - 2) comparisons; etc., etc. The total number of comparisons is:

so Selection Sort is a θ(n²) algorithm. It does all its work within the list so its space efficiency is θ(n ).

Most computers don't have an exchange instruction - they can only copy the value at one location into another location. How can a computing agent perform an exchange operation if it can only copy? To exchange the values between two locations, X and Y, the computing agent can use a third location, T, to temporarily hold one of the values:

(3.4) Analysis of Algorithms

Here we calculate the time and space efficiency of a number of algorithms for a variety of problems: θ-notation will be used for the time efficiency.

(3.4.2) Binary Search: Searching for a particular item in a list is much faster if the list is sorted because one can then use the Binary Search Algorithm (Fig. 3-18.)

As an example suppose we are looking for Cora in the following list of seven names sorted alphabetically:

The first time the while loop is entered, beginning = 1, end = 7, and Found = NO. Step 5 sets m to 4 and Steps 6 & 9 compare Cora to N₄ (Devi.) Cora precedes Devi alphabetically so Step 9 sets end to m - 1 = 3 and the while loop is entered again.

The second time the while loop is entered, beginning = 1, end = 3, and Found = NO. Step 5 sets m to 2 and Steps 6 & 10 compare Cora to N₂ (Bob.) Cora follows Bob alphabetically so Step 10 sets beginning to m + 1 = 3 and the body of the while loop is entered again.

The third time the while loop is entered, beginning = 3, end = 3, and Found = NO. Step 5 sets m to 3 and Step 6 discovers that Cora = N₃ so Step 7 prints the telephone number, T₃, and Step 8 sets Found to YES to exit the algorithm.

The worst-case running time for Sequential Search is θ(n).

(3.4.3) Pattern Matching: Figure 2.12 shows a pattern matching algorithm: the worst-case running time depends on the length of the text, n, and the length of the pattern, m : θ(m n). The best-case is θ(n).

For example suppose N₁ = 6, T₁ = 10 seconds, and we want to estimate T₂ for N₂ = 36. If the order of magnitude for the algorithm is:

• θ(1) then T₂ ≅ (10 seconds) * 1/1 = 10 seconds;

• θ(n) then T₂ ≅ (10 seconds) * 36/6 = 60 seconds;

• θ(n²) then T₂ ≅ (10 seconds) * (36/6)² = 360 seconds;

• θ(lg n) then T₂ ≅ (10 seconds) * (lg 36)/(lg 6) = 20 seconds
  (since 36 = 6² so (lg 36) = 2 * (lg 6)); and

• θ(n lg n) then T₂ ≅ (10 seconds) * (36/6) * (lg 36)/(lg 6) = 120 seconds.

The algorithms examined up to now had worst-case running times of θ(lg n), θ(n), and θ(n²). These algorithms are polynomially bounded because at worst their running times is less than some polynomial in the problem size, n.

A graph is a set of nodes and a set of edges where each edge connects two nodes. A Hamiltonian circuit on a graph of n nodes is a loop of n edges that starts at some node, visits every other node, and returns to the starting node.

The problem is to decide whether an arbitrary graph has a Hamiltonian circuit or not. One algorithm to solve this problem is to try all possible cases but there are at least 2ⁿ cases so the worst-case running time is at least θ(2ⁿ). This algorithm is not polynomially bounded and can only be used on very small graphs (see Figure 3.27).

A problem is tractable if there exists a polynomially bounded algorithm that solves the problem, otherwise, the problem is intractable. So far no one has found a polynomially bounded algorithm for the Hamiltonian circuit problem but no one has proven that such an algorithm can't exist: nobody knows whether this problem is tractable or not.

There are hundreds of problems, like the Hamiltonian circuit problem, which may or may not be tractable. Most computer scientists believe they are intractable but nobody has proven that yet.