S5 The Probability Integral Transform

THEOREM: Suppose X is a continuous random variable with strictly increasing CDF F(x). Then U=F(X) is a Uniform random variable on the interval [0.1].

PROOF: This follows easily from the change of variables formula. The density of U is d(u)=1 for 0<u<1, and 0 elsewhere. The range of U is [0,1]. Let G(U)=X be the INVERSE function of F(X). That is, G(U)=X if and only if U=F(X). Consider how the density of u, which is 1 x du on the range [0,1] transforms to the density of X. The density of U is constant, equal to 1, so substituting u=F(x) makes no difference within the density, which remains equal to 1. The differential is du=F'(x)dx. Now we note that the derivative of the CDF is exactly the probability density function of X. This is because F is the integral of the density f, so the density f is the derivative of the integral F, by the Fundamental Theorem of Calculus. It follows that if X=G(U) then X has density F'(x) and hence CDF F(x), exactly as desired. Note that the mapping transforms the range of probabilities in [0.1] to the set of all possible values of X since F(x)=0 at the smallest possible value of X,and F(x)=1 at the largest possible value of X.

APPLICATION: Suppose I want to generate Gamma Random Variables. In EXCEL, RAND() generates U(0,1) random variables. I need to apply the INVERSE of the Gamma CDF to get Gamma RV's. EXCEL happens to have a BUILT IN function GAMMAINV, which has been renamed to GAMMA.INV in more recent versions. Suppose the shape parameter is K and the scale parameter is S. Then GAMMAINV(RAND(),K,S) will generate a Gamma Random Variable G(K,S).

APPLICATION: If X= -LN(U), the X has an Exponential Density. For details see LINK.