대용량 패턴 인식 문제를 위한 서포트 함수 기반 커널 학습 방법론

Abstract

In the last decade, the kernel methods have contributed to significantadvances in research areas such as statistics, probability theory,data mining, pattern recognition and articial intelligence with practicalsuccess in their application to regression, classication, clustering,ranking and visualization problems. By utilizing techniques andtheories from optimization, statistics, and most importantly functionalanalysis in the feature space, kernel machines have provided principledways of solving these problems with exibility, generality and accuracy.Among various kernel methods, Support Vector Machines (SVMs) andGaussian Processes(GPs) have been extensively studied and applied todiverse problems in machine learning with state-of-the-art performance.However, as the size of data to be analyzed by these kernel machinesgrows explosively due to the development of automated datacollecting, processing and storing techniques, the eciency has emergedas a critical issue. In this circumstance, nonlinear training cost of thekernel machines to solve quadratic programming for SVMs and eigendecompositionproblem for GPs inhibits their practical advantages. Andwhen the runtime complexity of the solution is a primary concern as inmany real-time applications, these kernel methods are less attractivethan the conventional linear models since the run time complexity ofkernel machines are generally proportional to the size of training data.To this end, many researches in kernel methods have been recently devotedto the development of fast algorithms for ecient training andsparse kernel machines for online testing. Even though some of themachieved fair success, most of them leads to another issues to be solvedsuch as local minimum, performance degrade and sensitivity to the setting.In this thesis, we aim to provide a new class of methods which enhancethe eciency of conventional kernel machines without compensatingthe performance and robustness. By utilizing important conceptsfrom nonlinear dynamics, the proposed methods are mainly based on themulti-basin system constructed by kernel support functions. These includereduced set construction method for sparser kernel machines, fastlabeling method for clustering, domain described machine for rankingand support learning for pattern denoising. Through various experimentson synthetic and real-world problems, we show the eectiveness of the proposed methods by comparing the performance with other related methods. We expect the proposed methods will pave the wayfor the kernel machines to be applied eciently to the emerging largescaleapplications such as high-denition image analysis, web documentsranking and gene database clustering.