All of Statistics, and more

It's not all of statistics, of course - the title is tongue-in-cheek, and in fact the same author brought out another book, "All of Nonparametric Statistics" a few years later, also available online*.

The phrase "non-parametric statistics" is a bit misleading, though it's hard to think of a more accurate one that's less than ten words long. What it's getting at is that in traditional statistics, as covered in the first book, you generally assume the data comes from a model with a particular structure, which in the simplest case, draws samples from one specified distribution - uniform, binomial, normal or whatever. You then try to estimate the parameters (mean, standard deviation etc) of the distribution from the data. In "non-parametric" statistics, you relax some of those assumptions: perhaps you don't assume a particular distribution (so, no parameters to estimate), or you have a looser kind of model, or any of several other things, some of which might even involve having more parameters rather than fewer. By going in that general direction, you are more likely to avoid the pitfall of assuming an inappropriate model or distribution, but you pay the price of losing some statistical power: if your data does follow a normal distribution, throwing away the assumption that it does so means you might miss drawing some valid conclusions.

Think of the difference as being analogous to adjusting the control on your shower head or hosepipe to change how the water comes out. With traditional statistics (one concentrated jet of water), you're aiming for one specific place; if your aim is good (you guessed the right kind of model), it's really effective, but if you didn't, you get nothing. With nonparametric (wide angle spray), your aim matters less because you cover a wider area, but the effect in any one place will be less.

*Addendum: the status of the books is no longer clear. They used to be downloadable for nothing from the publisher's web site (Springer), but they no longer seem to be. However there are copies available all over the web, including from respectable places like Stanford University, as you will find if you Google the book title in double quotes followed by "filetype:pdf".