logit

Degree Of Freedom (DF)

For a hypothesis test on a model as a whole, the DF equals the number of predictor variables

For a hypothesis test on a predictor variable of a model, if the variable is numeric, the DF=1. If the variable is categorical and there are k categories then 

the DF equals k-1.

The reason is that we need k-1 coded variables to represent the categorical variable. 

E.g. Quality = {Good, Bad, Normal}

The we have 2 coded variables as follows

              code1  code2

quality=good    1      0

quality=bad     0      1

quality=normal  0      0

The above coding is also called dummy coding, it can be enabled by setting param=ref

proc logistic data =test;

  class quality / param=ref;

  model price = quality;

run;

If there is no param=ref, the default coding is effect coding, as follows:

              code1  code2

quality=good    1      0

quality=bad     0      1

quality=normal  -1     -1

generally the two coding schemes are the same, but effect coding has some benefits when you have an interaction of two categorical variables.

Model Fit Statistics

The "-2 Log L" is the log of the likelihood (L) of the model. The likelihood score can be derived by algorithms like Fisher's Scoring or Newton-Raphson.

AIC (Akaike Information Criterion) and SC (Schwarz Criterion) are both variants of the "-2 Log L" by adding a penalty factor

AIC/SC = -2 Log L + e, e is formula of small things, so it can return a more accurate likelihood

The model fit statistics is the smaller the better. the statistics itself is meaningful, but comparing the scores of two models does.

In SAS, it compares the socre of the model with intercept only to the score of the model with both intercept and covariates (variables), if the score of the model

with intercept and covariates is smaller then we say the predictor variables help fit the data better.

Testing Global Null Hypothesis: BETA=0

Empty Model (BETA=0). In SAS when testing a model as a whole against an empty model (ie BETA=0)

The Null Hypothesis here is BETA=0  which means the model has intercept only, all other coefficients are zero so the variables are not participating.

The idea of testing global null hypothesis is to compare the variance between the full model and the one with intercept only.

If the Chi-Square statistics is big, with "Pr > ChiSq" (p-value) smaller than 0.05 (the 0.05 alpha level), then we say the full model significantly better

than the NULL model.

Analysis of Maximum Likelihood Estimates

The Analysis here tests on each of the predictor variables individually. The NULL hypothesis = an predictor's regression coefficient is zero, so the predictor is not participating.

The test compares the variance between the full model and the model without the selected predictor to evaluate the importance of the selected predictor. It calculates the Wald Chi-Square (to measure the variation), and if "Pr > ChiSq" (p-value) smaller than 0.05/0.01 (the 0.05/0.01 alpha level), then we say the full model significantly better than the model without the selected predictor.

It also gives the prameter Estimate (something like expectation) and the Standard Error (something like standard deviation).

Therefore the model (log odds): Log[p / (1-p)] = b0 + b1 * predictor1 + b2 * predictor2 + b3 * predictor3 = -12.7772 + 1.4825 * female + 0.1035 * read + 0.0948 * science

We can interpret the Estimate as follows: for a one unit change in the predictor variable, the difference in log-odds for a positive outcome is expected to change by the respective coefficient, given the other variables in the model are held constant.

Odds Ratio Estiamtes

The above analysis is for the log-odds. here we also have the analysis for Odds. (log-odds = log (odds) )

The Point Estimate for each of the predictor variables is obtained by exponentiation the the Estimate in the above. E.g. 4.404 = e ^ 1.4825

Again we can interpret the Point Estimate as: for a one unit change in the predictor variable, the odds ratio for a positive outcome is expected to change by the respective coefficient, given the other variables in the model are held constant.

The 95% Wald Confidence Limits give the interval which would include the true Point Estimate with a confidence of 95%. As 4.404 is within [1.832, 10.584] we say the Estimate is not unusual.

The estimate +/- 1.96 standard error gives the 95% confidence interval, and e^interval gives the confidence limits.