Performance and Efficiency: Recent Advances in Supervised Learning

This paper reviews recent advances in supervised learning with a focus on two most important issues: performance and efficiency.Performance addresses the generalization capability of a learning machine on randomly chosen samples that are notincluded in a training set. Efficiency deals with the complexity of a learning machine in both space and time. As these two issues are general to various learning machines and learning approaches, we focus on a special type of adaptive learning systems with a neural architecture. We discuss four types of learning approaches: training an individual model, combinations of several well-trained models, combinations of many weak models, and evolutionary computation of models. We explore advantages and weaknesses of each approach and their interrelations, and pose open questions for possible future research.