Normalizing and Rescaling Variables

Rescaling means changing a variable to have a different range (ie. differen min and max). Normalizing means converting the variable to a canonical form based on their level of mesurement, such as creating z-scores for interval-level data

Rescaling

Suppose you have a variable that has values between 20 and 80, and you want to rescale it so that it runs between 0 and 1. Here's a formula: 

X* =  (X - Min(X)) / (Max(X) - Min(X))

So if you had an X value of 40, you would it convert to .333 = (40 - 20)/(80 - 20) = 20/60 = .333. Essentially, the 40 was 1/3 of the way from 20 (the old min) to 80 (the old max).

Suppose you have a variable that has values between 0 and 1 and you would like to rescale it so that it has values between -1 and 1, making 0 the midpoint. You would do this:

Z = 2Y - 1

Similarly, the take variable that runs between -1 and 1 to range between 0 and 1, we just use

Y = (Z + 1) / 2

More generally, if you want to rescale a variable to any arbitrary minimum value (call it "a") and any maximum value (call  it "b"), use this:

Y = [a + (x−xmin)*(b−a)] / (xmax - xmin)​ 

If a = 0 and b = 1, then this reduces to our first formula

Normalizing

Ordinal variables

Because ordinal measurements are unique only up to a monotonic transformation, we normalize ordinal variables by converting to rank orders. So the smallest number becomes a 1, the next smallest number becomes a 2, and so on. Tied numbers should receive the same ranks. Once two ordinal variables have been converted to ranks, we can compare the values side by side. 

Interval variables

Interval variables are unique up to a linear transformation. By convention, we normalize interval variables by subtracting their mean and dividing by their standard deviation, which yields a new variable that has mean 0 and standard deviation 1. This is called standardizing, and the resulting scores are called z-scores. 

X* = (X - mx)/sx

Ratio variables

Ratio variables are unique up to congruence or proportionality transformation (which means you can multiply them by a constant and the result is equally valid; like converting dollars to euros). By convention, we normalize ratio variables forcing them to have unit length, which is to say have a Euclidean norm of 1, which means that the square root of their sum of squares is 1. 

X* = X/sqrt(sum of squares of X)

Centering

To center a variable is to subtract its mean from every value, causing the new variabe to have a mean of zero. This is useful in regression, especially when using interaction terms. Also, if you center the X variables, the intercept gives the expected Y value when all the X variables are at their mean.