Latent variable analysis for image annotation and retrieval

Abstract

Nowadays, digital photography is very popular type of media used to record important events in everyday’s life. The increasing number of digital images requires a good image database to support image collection and image search. In order to correctly retrieve image according to the user’s query, many researches focus to improve the precision of the retrieval images and the retrieval processing time especially for general image retrieval system using low-level image features where query by words and examples do not have satisfactory retrieval performance. The purpose of this dissertation is thus to develop the image annotation model aimed for higher precision in identifying and retrieving images with acceptable search time and supported query by words or images. The proposed model is called “Two-Probabilistic Latent Semantic Analysis”. The proposed model uses two latent variables of probabilistic latent semantic model, of which the first latent variable is used to group the words in an image database that often occurs, and of which the second latent variable is used to group the visual words usually appear in each word. Based on Bag-of-Feature (BoF) technique applied to image annotation, images in the database are represented by counting the number of visual word of the constructed visual vocabulary. Afterward, the BoF of images corresponding to their meaning is used to construct the annotation models namely naïve Bayes, CMRM, and pLSA, comparing with our proposed model, “Two-pLSA”. Using the automatic image annotation, an unlabeled image is annotated by the words from the constructed models, and then the annotated words of that image are used for a text index for image retrieval task. Moreover, the performance has been evaluated by the precision and speed to find the best model for supporting the annotation and retrieval tasks. The performance values of both tasks are measured by mean Average Precision (mAP) to compare among 4 annotation models. The results showed that our proposed model used to identify meaningful images offers satisfactory performance both in terms of accuracy and speed of annotation and retrieval with appropriate control parameters. In addition, the proposed model also supports query by words and image examples including unlabeled images in the database taken by the photographers without object segmentation and specific meaning.