Autonomous Learning of Visual Concept Models

Abstract—As the amount of video data increases, organizing and retrieving video data based on their semantics is becoming more and more important. Traditionally, supervised learning is used to build models for detecting semantic concepts. However, in order to obtain a substantial amount of training data, extensive labeling work is needed with the supervised learning schemes. In this paper, we propose a novel Autonomous Learning mechanism, in which imperfect information extracted from cross-modality information is used for training. This shall, thus, not only reduce the number of examples needed for labeling, as active learning and transductive learning do, but also totally avoid the manual labeling process. First of all, imperfect labels without user involvement are obtained from cross-modality information. Then based on our proposed new schemes, “Generalized Multiple-Instance Learning” and “Uncertain Labeling Density”, the system conjectures relevance score of visual concepts. From these scores, Support Vector Regression is used to build generic visual models. Our proposed algorithm is tested on several concepts in large video databases. Preliminary experiments show promising results in limited number of concepts. This novel Autonomous Learning mechanism can achieve better system average precisions than two supervised algorithms. Currently, we’re investigating the performance of this proposed system by taking large scale experiments, whose results is not in this paper yet.

By: Xiaodan Song; Ching-Yung Lin; Ming-Ting Sun

Published in: RC23647 in 2005


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to .