Creation of an Ensemble: Diversity Production Based Approach
Hamid Parvin, Zahra Rezaei, Sajad Parvin
Journal of E-Technology, Vol. 3, No. 1, 2012.
DOI: 
Link: http://www.dline.info/jet/fulltext/v3n1/2.pdf
Abstract:
Generally in design of combinational classifier systems, the more diverse the results of the classifiers, the more appropriate final result. In this paper, a new method for combining classifiers is proposed which its main idea is heuristic retraining of classifiers. Specifically, in the new method named Combinational Classifiers using Heuristic Retraining (CCHR) which proposes a new way for generating diversity in ensemble pool, a classifier is first run, then, focusing on the drawbacks of this base classifier, other classifiers are retrained heuristically. Each of these classifiers looks at the data with its own attitude. The main concentration in the retrained classifiers is to leverage the error-prone data. So, retrained classifiers usually have different votes about the sample points which are close to boundaries and may be likely erroneous. Experiments show significant improvements in terms of accuracies of consensus classification. This study also investigates that focusing on which crucial data points can lead to more performance in base classifiers. Also, this study shows that adding the number of all â€œdifficultâ€ data points like boosting method, does not always cause a better performance. The experimental results show that the performance of the proposed algorithm outperforms some of the best methods in the literature. So empirically, the authors claim that forcing crucial data points to the training set as well as eliminating them from the training set can yield to the more accurate results, conditionally.