Lectures‎ > ‎

Lecture 04

For today you should:
1) Swap books, read your predecessor's summary, read another chunk and write a summary.  If you get an unread book, you might have to create the page for it.
2) Start Chapter 2?

1) Swappy books: any ideas for replicable experiments?
2) Results from Chapter 1.
3) Intro to Chapter 2.
4) Studio time.

For next time you should:
1) Read Chapter 2 and prepare the reading questions from Lecture 3.
2) Work on Exercises from Chapter 2.
3) Bring back your swappy book.
4) Extra time?  Catch up, get ahead, skim Complexity Digest for case study ideas.

Erdos-Renyi Random Graphs

Why did Erdos and Renyi study this problem?  Skim the papers and see if you can tell.

Why did I include this example?
1) Chance to practice graph algorithms and see something interesting.
2) First example of critical behavior, which will be a recurring theme.
3) Connection to percolation, which we will come back to (and from there on to fractals).
4) Candidate model of real phenomena.

Caveat from Wikipedia:
Both of the two major assumptions of the G(np) model (that edges are independent and that each edge is equally likely) may be inappropriate for modeling real-life phenomena. In particular, an Erdős–Rényi graph does not have heavy tails, as is the case in many real networks. Moreover, it has low clustering, unlike many social networks. For popular modeling alternatives, see Barabási–Albert model and Watts and Strogatz model. A model for interacting ER networks was developed recently by Buldyrev et al[5].

Analysis of Algorithms

How can we compare algorithms when their performance depends on hardware, size of the input, and contents of the input?

How do you compare two functions to say which is bigger?


Because O(g(x)) is a set, it is more correct to write "element of" rather than "equals".

This definition implies some of the limitations of this kind of analysis:
1) Only relevant for large x.
2) M can be arbitrarily big.

Other issues to think about:
1) Bounded/typical problem size.
2) Space vs. time tradeoffs.  Taken to an extreme: pre-compute and cache all answers.
3) Worst case vs. average case.

Example of #3: simplex algorithm, one of the top 10 algorithms of the 20th century.  

The simplex method is remarkably efficient in practice and was a great improvement over earlier methods such as Fourier–Motzkin elimination. However, in 1972, Klee and Minty[31] gave an example showing that the worst-case complexity of simplex method as formulated by Dantzig is exponential time. Since then, for almost every variation on the method, it has been shown that there is a family of linear programs for which it performs badly. It is an open question if there is a variation with polynomial time, or even sub-exponential worst-case complexity.[32][33]

Analyzing and quantifying the observation that the simplex algorithm is efficient in practice, even though it has exponential worst-case complexity, has led to the development of other measures of complexity. The simplex algorithm has polynomial-time average-case complexity under various probability distributions, with the precise average-case performance of the simplex algorithm depending on the choice of a probability distribution for the random matrices.[33][34] Another approach to studying "typical phenoma" uses Baire category theory from general topology, and to show that (topologically) "most" matrices can be solved by the simplex algorithm in a polynomial number of steps. Another method to analyze the performance of the simplex algorithm studies the behavior of worst-case scenarios under small perturbation – are worst-case scenarios stable under a small change (in the sense of structural stability), or do they become tractable? Formally, this method uses random problems to which is added a Gaussian random vector ("smoothed complexity").[35]