Abstract:

Motivated by the practical demands for simplification of data towards being consistent with human thinking and problem solving as well as tolerance of uncertainty, information granules are becoming important entities in data processing at different levels of data abstraction. This paper proposes a method to construct classifiers from multi-resolution hierarchical granular representations (MRHGRC) using hyperbox fuzzy sets. The proposed approach forms a series of granular inferences hierarchically through many levels of abstraction. An attractive characteristic of our classifier is that it can maintain a high accuracy in comparison to other types of fuzzy min-max neural networks at a low degree of granularity based on reusing the knowledge learned from lower levels of abstraction. In addition, our approach can reduce the data size significantly as well as handles the uncertainty and incompleteness associated with data in real-world applications. The construction process of the classifier consists of two phases. The first phase is to formulate the model at the greatest level of granularity, while the later stage aims to reduce the complexity of the constructed model and deduce it from data at higher abstraction levels. Experimental analyses conducted comprehensively on both synthetic and real datasets indicated the efficiency of our method in terms of training time and predictive performance in comparison to other types of fuzzy min-max neural networks and common machine learning algorithms.

Introduction:

Hierarchical problem solving, where the problems are analysed in a variety of granularity degrees, is a typical characteristic of the human brain [1]. Inspired by this ability, granular computing [2] was introduced. One of the critical features of granular computing is to model the data as high-level abstract structures and to tackle problems based on these representations similar to structured human thinking [3]. Information granules (IGs) [4] are underlying constructs of the granular computing. They are abstract entities describing important properties of numeric data and formulating knowledge pieces from data at a higher abstraction level. They play a critical role in the concise description and abstraction of numeric data [5]. Information granules have also contributed to quantifying the shortage of numeric precision in data [6]. Information granule is one of the problem-solving methods based on decomposing a big problem into sub-tasks which can be solved individually. They appear like an evident accomplishment of the critical abstraction model [6]. In the world of big data, one regularly departs from specific entities of data and discover general rules from data via encapsulation and abstraction. The use of information granules is meaningful when tackling the five Vs of big data [7], i.e., volume, variety, velocity, veracity, and value. Granulation process gathering similar data together contributes to reducing the data size, and so the volume issue is addressed. The information from many heterogeneous sources can be granulated into various granular constructs, and then several measures and rules for uniform representation are proposed to fuse base information granules as shown in [8]. Hence, the property of variety is eliminated. Several studies constructed the evolving information granules to adapt to the changes in the streams of data as in [9]. The variations of information granules in a high-speed data stream assist in tackling the velocity problem of big data. The process of forming information granules is often associated with the removal of outliers and dealing with incomplete data [7]; thus the veracity of data is guaranteed. Finally, the multiresolution hierarchical architecture of various granular levels can disregard some irrelevant features but highlight facets of interest [10]. By this way, the granular representation might disclose the values of big data according to different cognitive demands of users.

A multi-dimensional hyperbox fuzzy set is a fundamental conceptual vehicle to represent information granules. Each fuzzy min-max hyperbox is determined by the minimum and maximum points and a fuzzy membership function. A classifier can be built from a set of fuzzy hyperboxes along with an appropriate training algorithm. We can extract a rule set directly from hyperbox fuzzy sets or by using it in combination with other methods such as decision tree [11] to account for the predictive results. However, a limitation of hyperbox-based classifiers is that their accuracy at the low level of granularity (corresponding to large-sized hyperboxes) is not high. In contrast, classifiers at the high granularity level are more accurate, but the building process of classifiers at this level is time-consuming, and it is difficult to extract the rule set interpretable for predictive outcomes because of the high complexity of resulting models. Hence, it is desired to construct a simple classifier with high accuracy. In addition, we expect to observe the change in the predictive results at different data abstraction levels. This paper proposes a method of constructing a high-precision classifier at the high data abstraction level based on the knowledge learned from lower abstraction levels. On the basis of classification errors on the validation set, we can predict the change in the accuracy of the constructed classifier on unseen data, and we can select an abstraction level satisfying both acceptable accuracy and simple architecture on the resulting classifier. Furthermore, our method is likely to expand for large-sized datasets due to the capability of parallel execution during the constructing process of core hyperboxes at the highest level of granularity. In our method, the algorithm starts with a relatively small value of maximum hyperbox size (θ) to produce base hyperbox fuzzy sets, and then this threshold is increased in succeeding levels of abstraction whose inputs are the hyperbox fuzzy sets formed from the previous step. By using many hierarchical resolutions of granularity, the information captured in earlier steps is transferred to the classifier at the next level. Therefore, the classification accuracy is still maintained at an acceptable value when the resolution of training data is low.

Data generated from complex real-world applications frequently change over time, so the machine learning models used to predict behaviors of such systems need the efficient online learning capability. Fuzzy min-max neural networks proposed by Simpson [12] may absorb new information by singlepass through training datasets without forgetting previously learned patterns. However, this original version and many of its improved variants only work on the input data in the form of points. In practice, due to the uncertainty and some abnormal behaviors in the systems, the input data include not only crisp points but also intervals. To address this problem, Gabrys and Bargiela [13] introduced a general fuzzy min-max (GFMM) neural network, which can handle both fuzzy and crisp input samples. By using hyperbox fuzzy sets for the input layer, this model can accept the input patterns in the granular form and process data at a high-level abstract structure. As a result, our proposed method used a similar mechanism as in the general fuzzy min-max neural network to build a series of classifiers through different resolutions, where the small-sized resulting hyperbox fuzzy sets generated in the previous step become the input to be handled at a higher level of abstraction (corresponding to a higher value of the allowable hyperbox size). Going through different resolution degrees, the valuable information in the input data is fuzzified and reduced in size, but our method helps significantly to preserve the amount of knowledge contained in the original datasets. This capability is illustrated via the slow decline in the classification accuracy. In some cases, the predictive accuracy increases at higher levels of abstraction because the noise existing in the detailed levels is eliminated.

Building on the principles of developing GFMM classifiers with good generalization performance discussed in [14], this paper employs different hierarchical representations of granular data with various hyperbox sizes to select a compact classifier with acceptable accuracy at a high level of abstraction. Our main contributions in this paper can be summarized as follows:

• We propose a new data classification model based on the multi-resolution of granular data representations in combination with the online learning ability of the general fuzzy min-max neural network.
• The proposed method is capable of reusing the learned knowledge from the highest granularity level to construct new classifiers at higher abstraction levels with the low trade-off between the simplification and accuracy.
• The efficiency and running time of the general fuzzy minmax classifier are significantly enhanced in the proposed algorithm.
• Our classifier can perform on large-sized datasets because of the parallel execution ability.
• Comprehensive experiments are conducted on synthetic and real datasets to prove the effectiveness of the proposed method compared to other approaches and baselines.

Full content of the paper can be found in ArXiv

Reference:

[1] G. Wang, J. Yang, and J. Xu, “Granular computing: from granularity optimization to multi-granularity joint problem solving,” Granular Computing, vol. 2, no. 3, pp. 105–120, 2017.

[2] A. Bargiela and W. Pedrycz, Granular Computing: An introduction, 1st ed. Kluwer Academic Publishers, 2003.

[3] J. A. Morente-Molinera, J. Mezei, C. Carlsson, and E. Herrera-Viedma, “Improving supervised learning classification methods using multigranular linguistic modeling and fuzzy entropy,” IEEE Transactions on Fuzzy Systems, vol. 25, no. 5, pp. 1078–1089, 2017.

[4] L. A. Zadeh, “Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic,” Fuzzy Sets and Systems, vol. 90, no. 2, pp. 111 – 127, 1997.

[5] W. Pedrycz, G. Succi, A. Sillitti, and J. Iljazi, “Data description: A general framework of information granules,” Knowledge-Based Systems, vol. 80, pp. 98 – 108, 2015.

[6] W. Pedrycz, “Allocation of information granularity in optimization and decision-making models: Towards building the foundations of granular computing,” European Journal of Operational Research, vol. 232, no. 1, pp. 137 – 145, 2014.

[7] J. Xu, G. Wang, T. Li, and W. Pedrycz, “Local-density-based optimal granulation and manifold information granule description,” IEEE Transactions on Cybernetics, vol. 48, no. 10, pp. 2795–2808, 2018.

[8] W. Xu and J. Yu, “A novel approach to information fusion in multisource datasets: A granular computing viewpoint,” Information Sciences, vol. 378, pp. 410 – 423, 2017.

[9] R. Al-Hmouz, W. Pedrycz, A. S. Balamash, and A. Morfeq, “Granular description of data in a non-stationary environment,” Soft Computing, vol. 22, no. 2, pp. 523–540, 2018.

[10] C. P. Chen and C.-Y. Zhang, “Data-intensive applications, challenges, techniques and technologies: A survey on big data,” Information Sciences, vol. 275, pp. 314 – 347, 2014.

[11] T. T. Khuat, D. Ruta, and B. Gabrys, “Hyperbox based machine learning algorithms: A comprehensive survey,” CoRR, vol. abs/1901.11303, 2019.

[12] P. K. Simpson, “Fuzzy min-max neural networks. i. classification,” IEEE Transactions on Neural Networks, vol. 3, no. 5, pp. 776–786, 1992.

[13] B. Gabrys and A. Bargiela, “General fuzzy min-max neural network for clustering and classification,” IEEE Transactions on Neural Networks, vol. 11, no. 3, pp. 769–783, 2000.

[14] B. Gabrys, “Learning hybrid neuro-fuzzy classifier models from data: to combine or not to combine?” Fuzzy Sets and Systems, vol. 147, no. 1, pp. 39–56, 2004.

Demonstration: