Dynamic Mutual Information Based Feature Selection for Multi-label Learning

Abstract

Feature selection in classification problems is to identify important input features in order to reduce the dimensionality of the input space while improving or maintaining classification performance. Traditional feature selection algorithms were designed to handle single-label learning which has only one target. Recently, however, classification problems emerge in multi-label domains, such as scene annotation, emotions data, gene function prediction, text categorization, healthcare data and so on. Despite this trend, there is still a need for more research on feature selection for multi-label learning. This research mainly focuses on estimating information among features more accurately. For the multi-label learning feature selection algorithms, however, these algorithms are estimated the mutual information just on the whole input space and this cannot exactly express the relevance between features and labels. So, this research proposes a novel feature selection algorithm for classifying multi-label data, based on dynamic mutual information which can handle a redundancy among features controlling input space, called dynamic mutual information based feature selection for multi-label learning (DMIML). This research proposes a novel method that constructs a more accurate feature selection method for multi-label learning. The proposed method consists of two key algorithms. The first objective is to develop a filter ranking method for multi-label learning by extending the dynamic mutual information from single-label learning to multi-label learning. Also, the dynamic mutual information is improved to be applicable to multi-label learning. The second objective is to develop a filter subset selection method for multi-label learning by a specific search strategy with dynamic mutual information and improve the time-complexity of searching the features by minimum spanning tree. Here, this research improves the search strategy for utilizing the information of dynamic mutual information. This research provides three contributions rarely covered in the literature. First, it is the first time to extend the dynamic mutual information to multi-label learning and this research improves original dynamic mutual information to utilize in multi-label learning. Second, in the absence of much research, this research proposes a new method for the filter ranking method and subset selection method in feature selection. Lastly, this research has novelty because a new symmetric uncertainty criterion is proposed for multi-label learning. Since the above contributions, the proposed methods are helpful for users to make the best possible use of the multi-label datasets with the best feature set in a variety of research areas and industries.