Fast incremental learning of logistic model tree using least angle regression

Abstract

Expert and intelligent systems understand the underlying information behind the data by relying on a wide range of machine learning techniques. The interpretation of machine learning models is often the key to success in research areas such as business, finance, medical and health science, and bioinformatics; such research areas demand human understanding of the obtained model. The logistic model tree (LMT) algorithm is a popular classification method that combines a decision tree and logistic regression models. The combination of two complementary algorithms produces an accurate and interpretable classifier by combining the advantages of both logistic regression and tree induction. However, LMT has the disadvantage of high computational cost, which makes the algorithm undesirable in practice. In this paper, we propose an efficient method to learn the logistic regression models in the tree. We employ least angle regression to update the regression model in LogitBoost so that the algorithm efficiently learns sparse logistic regression models composed of relevant input variables. We compare the performance of our proposed method with the original LMT algorithm using 14 benchmark datasets and show that the training time dramatically decreases while the accuracy is preserved. Our proposed algorithm is not only accurate and intuitively interpretable but also computationally efficient. It helps users in making the best possible use of the data that are included in expert and intelligent systems. (C) 2017 Elsevier Ltd. All rights reserved.