Searching Image Databases at Multiple Levels of Abstraction

In the last several years, technological advances have made it possible to create digital image and video libraries that, today, comprise tens of tera bytes of on-line data, and that will significantly grow in size in the near future. Thus, we envision that the need for databases supporting efficient storage, search, retrieval and transmission of this type of data will grow significantly in the next few years. Due to the nature of the data, the search phase cannot rely on traditional methods based on indexing. A solution to this problem is content-based search, where the user specifies a query in terms of the desired content of the retrieved data. In this paper we describe an approach to content-based search used in project in progress at the IBM T.J. Watson Research Center under joint sponsorship by NASA, aimed at facilitating the storage, query and retrieval of images from large satellite digital libraries by a diverse community of users. In our approach, content is specified at three different levels of abstraction: the pixel level, the feature level, and the semantic level. We discuss how to search at each individual level, how to combine results from different levels and we describe the architecture of an image query system that implements the described concepts.