Autonomous Learning of Visual Concept Models

Abstract—As the amount of video data increases, organizing and retrieving video data based on their semantics is becoming more and more important. Traditionally, supervised learning is used to build models for detecting semantic concepts. However, in order to obtain a substantial amount of training data, extensive labeling work is needed with the supervised learning schemes. In this paper, we propose a novel Autonomous Learning mechanism, in which imperfect information extracted from cross-modality information is used for training. This shall, thus, not only reduce the number of examples needed for labeling, as active learning and transductive learning do, but also totally avoid the manual labeling process. First of all, imperfect labels without user involvement are obtained from cross-modality information. Then based on our proposed new schemes, “Generalized Multiple-Instance Learning” and “Uncertain Labeling Density”, the system conjectures relevance score of visual concepts. From these scores, Support Vector Regression is used to build generic visual models. Our proposed algorithm is tested on several concepts in large video databases. Preliminary experiments show promising results in limited number of concepts. This novel Autonomous Learning mechanism can achieve better system average precisions than two supervised algorithms. Currently, we’re investigating the performance of this proposed system by taking large scale experiments, whose results is not in this paper yet.