Near-exact distributions

 
    In a 2004 paper in the Journal of Multivariate Analysis, Carlos A. Coelho laid the foundations for what he called ’near-exact distributions’ (see reference [1]). Since then these have been successfully applied to a large number of statistics (see references [1]-[30]). These are asymptotic distributions developed under a new concept of approximating distributions. Based on decompositions of the characteristic function (c.f.) of the test statistic under study, or of the c.f. of its logarithm, these are asymptotic distributions which lay much closer to the exact distribution than common asymptotic distributions. Once a convenient decomposition of the c.f., most often a factorization, has been obtained, one has to identify its component parts which yield a manageable distribution and its parts which have to be asymptotically approximated by an adequate asymptotic replacement, in such a way that together with the part of the c.f. left unchanged yield a manageable distribution from which p-values and quantiles are easy to compute.

    All this is done in order to obtain a manageable and very well-fitting approximation, which may be used to compute sharp near-exact quantiles or p-values. The near-exact distributions lie much closer to the exact distribution than common asymptotic distributions and when correctly developed for statistics used in Multivariate Analysis, besides showing an asymptotic behavior for increasing sample sizes, they also show a marked asymptotic behavior for increasing numbers of variables and/or populations involved.
 
    Although the whole process may seem a bit complicated,  it is usually easy to implement and it enables the development of near-exact distributions for statistics, for which common asymptotic distributions do not perform well or are not easy or not even possible to be obtained.These near-exact distributions are not too hard to obtain and they are much useful in situations where it is not possible to obtain the exact distribution in a manageable form, but sharp but manageable approximations are required. The near-exact distributions may be easily implemented even for tests of highly complicated structures for covariance matrices, by considering the decomposition of the null hypothesis into a set of conditionally independent hypotheses. Very sharp near-exact distributions are available even for statistics for which there are no asymptotic distributions available (see publications [21] and [22]).


Meet the Team

Meet the team that has been writing the papers in the References:

Carlos A. Coelho

Barry C. Arnold

https://sites.google.com/site/profilefilipemarques/

Filipe J. Marques 

http://www.aim.estt.ipt.pt/~lgrilo/PagPessoal/WebPage_Lgrilo.htm

 Luís M. Grilo

Rui P. Alberto 




Near-exact distributions for the most common likelihood ratio test statistics used in Multivariate Analysis - Real r.v.'s
(for complex r.v.'s see here)
 
In the paper “A general near-exact distribution theory for the most common likelihood ratio test statistics used in Multivariate Analysis”, by Filipe J. Marques, Carlos A. Coelho and Barry C. Arnold (TEST 20 (2011), 180-203), the authors develop a general near-exact distribution approach for the most common likelihood ratio test (l.r.t.) statistics used in Multivariate Analysis:
 - the l.r.t. statistic to test the independence of several sets of variables,
 - the l.r.t. statistic to test the equality of several mean vectors
 - the l.r.t. statistic to test sphericity
 - the l.r.t. statistic to test the equality of several covariance matrices
 - and, by extension any l.r.t. statistic whose statistic may be written as the product of any of the above l.r.t. statistics.
 
The Technical Report cited in this paper as:

Coelho CA, Marques FJ (2009b). Near-exact distributions for the likelihood ratio test statistic for testing equality of several variance–covariance matrices (revisited). The New University of Lisbon, Mathematics Department, technical report #8/2009

has changed location and is now available here. The corresponding paper is:

 Coelho, C. A., Marques, F. J. (2012). Near-exact distributions for the likelihood ratio test statistic to test equality of several variance-covariance matrices in elliptically contoured distributionsComputational Statistics, 27, 627-659.

 
The authors provide here computational modules to be used in Mathematica to enable the implementation of the near-exact distributions developed in the above mentioned paper. The modules may be copied directly from the box below and pasted directly into a Mathematica notebook.
 

Modules.txt

  
 The authors provide below complete directions to enable the use of the computational modules:

Users_Guide_for_the_computational_modules.doc


 
Examples of use of the modules, with some execution times are available here: 
examples.pdf (pdf file)
examples.mht (mht file - please click on 'download' on the page that says that no visualization is available)



 References

30. Marques, F. J., Coelho, C. A. de Carvalho, M. (2014). On the distribution of linear combinations of independent Gumbel random variables, Satisticsand Computing (in print).

29. Coelho, C. A. (2014). Near-exact distributions: what are they and why do we need them? In Proceedings 59th ISI World Congress 2013, Special Topics Session 084, International Statistical Institute.

28. Coelho, C. A., Arnold, B. C. (2014). On the exact and near-exact distributions of the product of generalized Gamma random variables and the generalized variance, Communications in Statistics – Theory and Methods,43 2007-2033.

26. Grilo, L. M., Coelho, C. A. (2013).   Near-exact distributions for the likelihood ratio statistic used to test the reality of a covariance matrix AIP ConferencePrceedings, 1558, 797-800. 

25. Marques,F. J.,Coelho, C. A. (2013). The multisample block-diagonal equicorrelation and equivariance test. AIP Conference Proceedings, 1558, 793-796.

24. Coelho, C. A, Marqus, F. J. (2013). Near-exact distrbutions fr the block equicorrelation and equivariance likelihood ratio test statistic. AIP Confeence Proceedings, 1557, 429-433.

23. Marques, F. J., Coelho, C. A. (2013). The multi-sample block-scalar sphericity test under the comlex multivariate Normal case. AIP Conference Poceedings, 1557, 420-423.

22. Coelho, C. A., Marques, F. J. (2013). The multi-sample block-scalar sphericity test: exact and near-exact distributions for its likelihood ratio test statistic. Communications in Statistics – Theory and Methods, 42, 1153–1175.

21. Marques, F. J., Coelho, C. A. (2012). Near-exact distributions for the likelihood ratio test statistic of the multi-sample block-matrix sphericity test. Applied Mathematics and Computation, 219, 2861-2874.

20. Coelho, C. A., Marques, F. J. (2012). Near-exact distributions for the likelihood ratio test statistic to test equality of several variance-covariance matrices in elliptically contoured distributionsComputational Statistics, 27, 627-659.

19. Marques, F. J., Coelho, C. A. (2012). The multi-sample independence test. AIP Conference Proceedings, 1479, 1129-1132.

18. Grilo, L. M., Coelho, C. A. (2012). A family of near-exact distributions based on truncations of the exact distribution for the generalized Wilks Lambda statistic. Communications in Statistics - Theory and Methods, 41, 2321-2341.

17. Marques, F. J., Coelho, C. A. (2012). The block sphericity test – exact and near-exact distributions for the likelihood ratio statisticMathematical Methods in the Applied Sciences, 35, 373–383.

16. Coelho, C. A. (2012). Near-exact Distributions – Needing Them and Building Them, Gaudium Sciendi, 1, 100-122, U.C.P.

15. Marques, F. J., Coelho, C. A., Arnold, B. C. (2011) A general near-exact distribution theory for the most common likelihood ratio test statistics used in Multivariate Analysis. TEST, 20, 1, 180–203.

14. Coelho, C. A., Marques, F. J. (2011) On the exact, asymptotic and near-exact distributions for the likelihood ratio statistics to test equality of several Exponential distributions. AIP Conference Proceedings, 1389, 1471-1474.

13. Marques, F. J., Coelho, C. A. (2011) The multi-sample block-matrix sphericity test. AIP Conference Proceedings, 1389, 1479-1482.

12. Coelho, C. A., Marques, F. J. (2010) Near-exact distributions for the independence and sphericity likelihood ratio test statisticsJournal of Multivariate Analysis, 101, 583-593.

11. Marques, F. J., Coelho, C. A. (2010) The exact and near-exact distributions of the likelihood ratio statistic for the block sphericity testAIP Conference Proceedings, 1281, 1237-1240.
 
10. Coelho, C. A., Arnold, B. C., Marques, F. J. (2010) Near-exact distributions for certain likelihood ratio test statisticsJournal of Statistical Theory and Practice, 4, 4, 711-725 (invited paper for the special memorial issue in honor of H. C. Gupta, guest-edited by C. R. Rao).

9. Grilo, L. M., Coelho, C. A. (2010) Near-exact distributions for the generalized Wilks Lambda statistic. Discussiones Mathematicae - Probability and Statistics, 30, 53-86.
 
8. Grilo, L. M., Coelho, C. A. (2010) The exact and near-exact distributions for the Wilks Lambda statistic used in the test of independence of two sets of variables. American Journal of Mathematical and Management Sciences, 30, 1-2, 111–145.

7. Coelho, C. A., Mexia, J. T. (2010) Product and Ratio of Generalized Gamma-Ratio Random Variables: Exact and Near-exact Distributions - Applications. Lambert Academic Publishing AG & Co. KG, Saarbrücken, Germany, 145+v pp. (isbn: 978-3-8383-5846-8) (near-exact distributions may be found in Chapter 6 of the book).

6. Coelho, C. A., Marques, F. J. (2009) The advantage of decomposing elaborate hypotheses on covariance matrices into conditionally independent hypotheses in building near-exact distributions for the test statistics. Linear Algebra and its Applications, 430, 2592-2606. 

5. Marques, F. J., Coelho, C. A. (2008) Near-exact distributions for the sphericity likelihood ratio test statistic. Journal of Statistical Planning and Inference, 138, 726-741. 

4. Alberto, R. P., Coelho, C. A. (2007). Study of the quality of several asymptotic and near-exact approximations based on moments for the distribution of the Wilks Lambda statistic. Journal of
Statistical 
Planning and Inference, 137, 5, 1612-1626. 

3. Grilo, L. M., Coelho, C. A. (2007). Development and study of two near-exact approximations to the distribution of the product of an odd number of independent Beta random variables. Journal of Statistical Planning and Inference, 137, 5, 1560-1575. 

2. Coelho, C. A. (2006). The exact and near-exact distributions of the product of independent Beta random variables whose second parameter is rational. Journal of Combinatorics, Information & System Sciences, 31, 21-44.

1. Coelho, C. A. (2004). The Generalized Near-Integer Gamma distribution: a basis for ’near-exact’ approximations to the distribution of statistics which are the product of an odd number of independent Beta random variablesJournal of Multivariate Analysis, 89, 191-218.
Comments