Classification System Based on Ensemble Methods for Solving Machine Learning Tasks

Abstract

The paper investigates the solution of the classification problem using a two-level structure of model ensembles based on machine learning methods. The general structure of a two-level ensemble for solving classification problems is proposed. Based on the use of the two-level ensemble learning structure in the processing of two datasets, the quality of classification was improved. The procedures for processing the datasets included identifying and describing the key quality characteristics of the models, selecting a metric, selecting the base models, selecting parameters for the base models and ensemble methods. Preliminary data processing was performed. The basic datasets are divided into training and test samples, and input variables are generated. The results of applying simple classifiers and the ensemble of the two-level classification model are presented, and the efficiency of the developed classification models is evaluated. A two-level ensemble structure was used to find a compromise between the bias and variance inherent in machine learning models. At the first level of the ensemble, stacking was used to reduce the bias of the base models. This resulted in a preliminary improvement in classification quality. At the second level, bagging was used to reduce the variance of the base models. The basic classification models and ensemble models based on stacking and bagging, as well as metrics for assessing the quality of using basic classifiers and models of the first and second levels, were studied.