[Current editors: Otto Wildi, Miquel De Cáceres]
Comparison of vegetation classifications
Rohlf (1974) and Podani (1989b) provide introductions to the comparison of classification results obtained by multivariate analyses in numerical ecology.
Cross-classification tables
Let U and V be two partitions of the same set of n objects (in our case, vegetation plots) into K and K' groups. A cross-classification table T is a K x K' matrix where the two partitions are compared. Each element tij in T contains the number of objects classified into group i in U and into group j in V. Simple inspection of the cross-classification table T already provides a lot of information of the relationship between the two partitions.
Comparing pairs of clusters
Instead of generating a cross-classification table, one can compare partitions U and V by calculating a correlation measure between pairs of groups. For example, Bruelheide & Chytry (2000) employed the phi coefficient (equivalent to the Pearson linear correlation coefficient for binary variables) to evaluate the correspondence between all the possible pairs of groups of two classifications.
Indices of classification agreement
Indices of classification agreement assess the level of concordance between classification structures. Different approaches have been developed to compare dendrograms, hard partitions and fuzzy partitions:
- Agreement between dendrograms. In numerical taxaonomy, the first attempts to compare pairs of dendrograms were conducted by studying the correlation between their corresponding ultrametric matrices (this was referred to as cophenetic correlation) (Robertson 1979). The correlation measure could be, for example, the Pearson linear correlation coeficient or the Spearman rank correlation coefficient. Podani & Dickinson (1984) suggested that correlations between ultrametrics matrices to be conducted using five different descriptors (see Podani 2000). Alternatively to the comparison of ultrametric matrices, it is possible to compare hierarchical dendrograms by cutting the trees at specified levels and comparing the resulting partitions (e.g. Fowlkes & Mallows 1983). This approach also allows comparing a dendrogram with a hard partition.
- Agreement between hard partitons. One of the most used approaches to assess the similarity between two partitions is the Rand (1971) index, which is defined as the probability that two objects (i.e. vegetation plots) randomly chosen have the same treatment in both classifications. By same treatment, we mean that either the two objects belong to the same group in both classifications or that the two objects belong to different groups in both classifications. Hubert & Arabie (1985) proposed to correct the Rand index to remove the agreement expected just by chance. Another index to compare partitions is the Goodman-Kruskal (1954) lambda index (Podani 1986). Yet another approach to the comparison of two partitions consists in expressing the distance between the partitions as the minimum number of admisible transformations to go from one to the other (Podani 1986).
- Agreement between fuzzy partitions. Although fuzzy partitions can be compared after defuzzification, it is also possible to compare them directly. In this sense, Campello (2007) provided a fuzzy counterpart for the well known Rand index.
Consensus classifications
Constructing a consensus classification can be very useful to assess the level of agreement between alternative classifications of the same set of objects. A consensus classification prevents overtrusting the results of a given classification approach. Generating consensus classifications can be useful not only to compare different methods (algorithms), but also classifications generated using different vegetation attributes. There are several approaches to generate consensus for hierarchical classifications (e.g. Sokal & Rohlf 1981; Leftkovich 1985), as well as for hard or fuzzy partitions (Neuman & Norton 1986; Podani 1986, 1989a, 1990).
Validation of vegetation classifications
Validating a vegetation classification consists in determining whether the classification can be accepted for the intended purpose. There is no single criteria for the validity of vegetation classifications. Rather, several aspects may be evaluated in order to be sure that the classification is valid. In the following, we mention some useful criteria. Aho et al (2009) divide criteria between geometric and non-geometric, and Gauch & Whittaker (1981) divide them between external and internal. We follow a different division here, based on the aspect being evaluated.
Gauch and Whittaker (1981) defined classification stability (robustness) as a resistance against several types of data set modifications: a) simulation of error or noise, b) random division of the data set into subsets, which are classified separately, c) addition, or d) removal of sites. Bootstrap with replacement is the most popular randomization technique in ecological and environmental applications. This method was also proposed for testing the classification stability of community data (Pillar 1999). Alternatively, Tichy et al. (2011) proposed an algorithm to evaluate classification stability, which repeatedly compares the classification of the original data set with classifications of its randomly selected subsets created by the without-replacement bootstrap. They used the Goodman-Kruskal (1954) lamda index as comparison measure.
Isolation and compactness of clusters
Several indices have been devised to help determining the number of clusters that naturally 'arise' from a data set (see Milligan & Cooper 1985 for a comparison). These are normally based on the degree of compactness and isolation of clusters. One of the most frequently used indices is the pseudo-F statistic proposed by Calinsky-Harabasz (1974). Another geometric index that is frequently used is the silhouette (Rousseeuw 1987), which compares, for each object, the distance to the centroid of the cluster where it belongs with the distance to the nearest centroid, among all others. Silhouettes can be used to evaluate the degree of isolation of each cluster separately or the validity of the whole classification.
If vegetation types are required to be separated environmentally, one can use linear or quadratic discriminant analysis with environmental variables as a validation criterion (e.g. Hakes 1994; Pausas & Feoli 1996).
Existence of diagnostic species
Diagnostic species are frequently required for vegetation types, because vegetation scientists need easy-to-use rules for recognition in the field. There are several statistical alternatives to the determination of diagnostic species (see subsection 'Assignment Methods'). OptimClass (Tichý et al. 2010) is a method that allows comparing several clustering results (obtained with different methods and numbers of clusters) to determine which solution is optimal in terms of the amount of diagnostic species.
- Aho, K., Roberts, D.W. & Weaver, T. (2009). Using geometric and non-geometric internal evaluators to compare eight vegetation classification methods. Journal of Vegetation Science, 19, 549–562.
- Bruelheide, H. & Chytrý, M. (2000). Towards unification of national vegetation classifications: A comparison of two methods for analysis of large data sets. Journal of Vegetation Science, 11, 295–306.
- Calinski, T. & Harabasz, J. (1974). A Dendrite Method for Cluster Analysis. Communications in Statistics - Simulation and Computation, 3, 1–27.
- Campello, R.J.G.B. (2007). A fuzzy extension of the Rand index and other related indexes for clustering and classification assessment. Pattern Recognition Letters, 28, 833–841.
- Fowlkes, E.B. & Mallows, C.L. (1983). A method for comparing two hierarchical algorithms. Journal of the American Statistical Association, 78, 553–569.
- Gauch Jr, H.G. & Whittaker, R.H. (1981). Hierarchical classification of community data. Journal of Ecology, 69, 537–557.
- Goodman, L. & Kruskal, W. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732–764.
- Hakes, W. (1994). On the predictive power of numerical and Braun-Blanquet classification: an example from beechwoods. Journal of Vegetation Science, 5, 153–160.
- Hubert, L. & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2, 193–218.
- Lefkovitch, L.P. (1985). Euclidean consensus dendrograms and other classification structures. Mathematical Biosciences, 74, 1–15.
- Milligan, G.W. & Cooper, M.C. (1985). An examination of procedures for determining the number of clusters in a data set. Psychometrika, 50, 159–179.
- Neumann, D.A. & Norton, V.T. (1986). Clustering and isolation in the consensus problem for partitions. Journal of Classification, 3, 281–297.
- Pausas, J.G. & Feoli, E. (1996). Environment-vegetation relationships in the understory of pyrenean Pinus sylvestris forests II: A classification approach. Coenoses, 11, 45–51.
- Pillar, V.D. (1999). How Sharp Are Classifications? Ecology, 80, 2508–2516.
- Podani, J. & Dickinson, T.A. (1984). Comparison of dendrograms: a multivariate approach. Canadian Journal of Botany, 62, 2765–2778.
- Podani, J. (1986). Comparison of partitions in vegetation studies. Abstracta Botanica, 10, 235–290.
- Podani, J. (1989a). A method for generating consensus partitions and its application to community classification. Coenoses, 4, 1–10.
- Podani, J. (1989b). Comparison of ordinations and classifications of vegetation data. Vegetatio, 83, 111–128.
- Podani, J. (1990). Comparison of fuzzy classifications. Coenoses, 5, 17–21.
- Podani, J. (2000). Simulation of random dendrograms and comparison tests: some comments. Journal of Classification, 17, 123–142.
- Rand, W.M. (1971). Objective Criteria for the Evaluation of Clustering Methods. Journal of the American Statistical Association, 66, 846–850.
- Rousseuw, P.J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computation and Applied Mathematics, 20, 53–65.
- Sokal, R.R. & Rohlf, F.J. (1981). Taxonomic Congruence in the Leptopodomorpha Re-Examined. Systematic Zoology, 30, 309–325.
- Tichý, L., Chytrý, M., Hájek, M., Talbot, S.S. & Botta-Dukát, Z. (2010). OptimClass: Using species-to-cluster fidelity to determine the optimal partition in classification of ecological communities. Journal of Vegetation Science, 21, 287–299.
- Tichý, L., Chytrý, M. & S̆marda, P. (2011). Evaluating the stability of the classification of community data. Ecography, 34, 807–813.