Parameter Estimation for Binary Logistic Regression Using Different Iterative Methods.

Khwazbeen S. Fatah & Rzgar F. Mahmood

College of science / Mathematics Department-Salahaddin University, Ierbil-Iraq

College of Education / Mathematics Department-Garmian University, Kalar-Sulaimania-Iraq

DOI: https://doi.org/10.17656/jzs.10621

Original: 12 July 2016, Revised: 7 November 2016, Accepted: 20 November 2016, Published online: 20 June 2017

Abstract

Logistic Regression Analysis describes how a response variable having two or more categories is associated with a set of predictor variables (continuous or categorical) through a probability function. When the response variable is with only two categories a Binary Logistic Regression Model is the most widely used approach. The main deficiency with this method is in estimating logistic parameters numerically by applying Maximum Likelihood Estimation using Newton Raphson Method. In this paper, in order to improve the efficiency of the parameter estimates, four different modifications D-B-N; C-M-J; A-C-T; and L-W-W-Z, for NRM are introduced; each is an iterative method based on NRM. To specify the efficiency of these approaches, based on the number of iterations, all these procedures are compared with each other and then with NRM to identify the most efficient one. Finally, practical implementations for these procedures are given.

Key Words:

Binary Logistic

Regression,

Maximum Likelihood Estimation,

NRM,

Modified NRM.

References

[1] Abad, M.; Cordero, A. and Torregrosa, J., "Fourth- and Fifth-Order Methods for Solving Nonlinear Systems of Equations: An Application to the Global Positioning System", Abstract and Applied Analysis, Article ID 586708. (2013).

[2] Agresti, A., "Categorical Data Analysis", 3rd edition. John Wiley & Sons, (2013).

[3] Albert, A. and Anderson, J.,"On the Existence of Maximum Likelihood Estimates in Logistic Regression Models", Biometrika, Vol. 71, No. 1, pp. 1-10, (1984).

[4] Ben-Israel, A., "A Newton-Raphson Method for the Solution of Systems of Equations", Journal of Mathematical Analysis and Applications Vol. 15, No. 2, pp. 243-252, (1966).

[5] Cordero, A.; Martínez, E. and Torregrosa, J., "Iterative Methods of Order Four and Five for Systems of Nonlinear Equations", Journal of Computation and Applied Mathematic Vol. 231, No. 2, pp. 541-551, (2009).

[6] Cramer J., "Logit Models from Economics and Other Fields", Cambridge University Press, (2003).

[7] Czepiel, S., "Maximum Likelihood Estimation of Logistic Regression Models: Theory and Implementation", Available at czep.net/stat/mlelr.pdf. (2002).

[8] Darvishi, M. and Barati, A., "A third-order Newton-type method to solve systems of nonlinear equations", Applied Mathematics and Computation Vol. 187, No. 2, pp. 630-635, (2007).

[9] Demira, E. and Akkus, Ö, "An Introductory Study on "How the Genetic Algorithm Works in the Parameter Estimation of Binary Logit Model?", International Journal of Sciences: Basic and Applied Research (IJSBAR) Vol. 19, No, 2, pp. 162-180, (2015).

[10] Givens, G. and Hoeting, J., "Computational Statistics", John Wiley & Sons, (2013).

[11] Green, P., "Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives", Journal of the Royal Statistical Society. Series B (Methodological) Vol. 46, pp. 149-192, (1984).

[12] Hassanien, A.E., Tolba, M. and Azar, A.T. eds., "Advanced Machine Learning Technologies and Applications: Second International Conference, AMLTA 2014, Cairo, Egypt, November 28-30, 2014. Proceedings", Vol. 488, Springer, (2014).

[13] Hosmer, D.; Lemeshow. S. and Sturdivant. R., "Applied Logistic Regression", 3rd edition. John Wiley & Sons. (2013).

[14] Kelley, C., "Iterative Methods for Linear and Nonlinear Equations", Society for Industrial and Applied Mathematics, (1995).

[15] Kleinbaum, D. and Klein, M., "Logistic Regression: A Self-Learning Text", 3rd Edition. Springer Science & Business Media, (2010).

[16] Li, X.; Wu, Z.; Wang, L. and Zhang, Q., "A Ninth-Order Newton-Type Method to solve system of nonlinear equations". International Journal of Research and Reviews in Applied Science Vol. 16, No. 2, pp. 224-228, (2013).

[17] Mak, T., "Solving Non-Linear Estimation Equations", Journal of the Royal Statistical Society. Series B (Methodological) Vol. 55, No. 4, pp. 945-955, (1993).

[18] Menard, S., "Applied Logistic Regression Analysis", 2nd edition. Sage, (2002).

[19] Montgomery D. and Peck E., "Introduction to Linear Regression Analysis", John Wiley & Sons. Inc, (1982).

[20] Pampel, F., "Logistic Regression: A Primer", Sage, (2000).

[21] Rashid, M., "Inference on Logistic Regression Models", (Doctoral dissertation, Bowling Green State University), (2008).

[22] Reilly, S., Onslow, M., Packman, A., Wake, M., Bavin, E., Prior, M., Eadie, P., Cini, E., Bolzonello, C. and Ukoumunne, O., "Predicting stuttering onset by the age of 3 years: A prospective, community cohort study", Pediatrics, Vol. 123, No. 1, pp. 270–277, (2009).

[23] Serfling, R., "Approximation Theorems of Mathematical Statistics", Wiley Series in Probability and Statistics, (1980).

[24] Van Den Berg, C.; Christensen, J. and Ressel, P., "Harmonic Analysis on Semigroups: Theory of Positive Definite and Related Functions", Springer Science & Business Media, (1984).

[25] Yarandi, H. and Simpson, S., "The Logistic Regression Model and the Odds of Testing HIV positive", Nursing Research, Vol. 40, No. 6, pp. 372-373, (1991).

View Full Article