Sharif Digital Repository

Image annotation approaches need an annotated dataset to learn a model for the relation between images and words. Unfortunately, preparing a labeled dataset is highly time consuming and expensive. In this work, we describe the development of an annotation system in semi-supervised learning framework which by incorporating unlabeled images into training phase reduces the system demand to labeled images. Our approach constructs a generative model for each semantic class in two main steps. First, based on Gamma distribution, a generative model is constructed for each semantic class using labeled images in that class. The second step incorporates the unlabeled images by using a modified EM... 

Automatic image annotation (AIA) is an image retrieval mechanism to extract relative semantic tags from visual content. So far, the improvement of accuracy in newly developed such methods have been about 1 or 2% in the F1-score and the architectures seem to have room for improvement. Therefore, the authors designed a more detailed architecture for AIA and suggested new algorithms for its main parts. The proposed architecture has three main parts: feature extraction, learning, and annotation. They designed a novel learning method using machine learning and probability bases. In the annotation part, they suggest a novel method that gains the maximum benefit from the learning part. The... 

Automatic image annotation has emerged as an important research topic due to the existence of the semantic gap and in addition to its potential application on image retrieval and management. In this paper we present an approach which combines regional contexts and visual topics to automatic image annotation. Regional contexts model the relationship between the regions, whereas visual topics provide the global distribution of topics over an image. Conventional image annotation methods neglected the relationship between the regions in an image, while these regions are exactly explanation of the image semantics, therefore considering the relationship between them are helpful to annotate the... 

Different types of visual features provide multi-modal representation for images in the annotation task. Conventional graph-based image annotation methods integrate various features into a single descriptor and consider one node for each descriptor on the learning graph. However, this graph does not capture the information of individual features, making it unsuitable for propagating the labels of annotated images. In this paper, we address this issue by proposing an approach for fusing the visual features such that a specific subgraph is constructed for each visual modality and then subgraphs are connected to form a supergraph. As the size of supergraph grows linearly with the number of... 

Nowadays, the number of digital images has increased so that the management of this volume of data needs an efficient system for browsing, categorising and searching. Automatic image annotation is designed for assigning tags to images for more accurate retrieval. Non-negative matrix factorisation (NMF) is a traditional machine learning technique for decomposing a matrix into a set of basis and coefficients under the non-negative constraints. In this study, the authors propose a two-step algorithm for designing an automatic image annotation system that employs the NMF framework for its first step and a variant of K-nearest neighbourhood as its second step. In the first step, a new multimodal... 

In automatic image annotation (AIA) different features describe images from different aspects or views. Part of information embedded in some views is common for all views, while other parts are individual and specific. In this paper, we present the Mvg-NMF approach, a multi-view-group non-negative matrix factorization (NMF) method for an AIA system which considers both common and individual factors. The NMF framework discovers a latent space by decomposing data into a set of non-negative basis vectors and coefficients. The views divided into homogeneous groups and latent spaces are extracted for each group. After mapping the test images into these spaces, a unified distance matrix is... 

Automatic image annotation is a mechanism to assign a list of appropriate tags that describe the visual content of a given image. Most methods only focus on the content of the images and ignore the relationship between the tags in vocabulary. In this work, we propose a new deep learning-based automatic image annotation architecture, which considers label dependencies in a graph convolution neural network structure and extracts tag descriptors to re-weight the output class scores based on their relationships. The proposed architecture has three main parts: feature extraction, graph convolutional network, and annotation. In graph convolutional network, we apply one layer convolution on... 

A probabilistic formulation for semantic image annotation and retrieval is proposed. Annotation and retrieval are posed as classification problems where each class is defined as the group of database images labeled with a common semantic label. It is shown that, by establishing this one-to-one correspondence between semantic labels and semantic classes, a minimum probability of error annotation and retrieval are feasible with algorithms that are 1) conceptually simple and 2) computationally efficient. In this article, a content-based image retrieval and annotation architecture is proposed. Its attitude is decreasing the semantic gap by partitioning the image to its semantic regions and using... 

In automatic image annotation (AIA) different features describe images from different aspects or views. Part of information embedded in some views is common for all views, while other parts are individual and specific. In this paper, we present the Mvg-NMF approach, a multi-view-group non-negative matrix factorization (NMF) method for an AIA system which considers both common and individual factors. The NMF framework discovers a latent space by decomposing data into a set of non-negative basis vectors and coefficients. The views divided into homogeneous groups and latent spaces are extracted for each group. After mapping the test images into these spaces, a unified distance matrix is... 

Automatic image annotation methods generate a list of tags for each test image and present it in a matrix structure. To achieve a more accurate annotation, we propose a method with the aim of correcting the tag list. In our method, we detect an indicator for each group of tags and use it to rectify the annotation results. To find a correct indicator, we apply a deep feature vector generated by the “AlexNet” model. Using this indicator, we determine the suitable tags for an image. The purposed method is independent of feature vector, dataset, and annotation method. It can be applied to the currently available annotation methods. Our experiments showed improvement in all annotation methods... 

Automatic image annotation is a challenging problem in the field of image retrieval. Dealing with large databases makes the annotation problem more difficult and therefore an effective approach is needed to manage such databases. In this work, an annotation system has been developed which considers images in separate categories and constructs a profiling model for each category. To describe an image, we propose a new feature extraction method based on color and texture information that describes image content using discrete distribution signatures. Image signatures of one category are partitioned using spectral clustering and a prototype is determined for each cluster by solving an... 

Automatic Image Annotation (AIA) helps image retrieval systems by predicting tags for images. In this paper, we propose an AIA system using Non-negative Matrix Factorization (NMF) framework. The NMF framework discovers a latent space, by factorizing data into a set of non-negative basis and coefficients. To model the images, multiple features are extracted, each one represents images from a specific view. We use multi-view graph regularization NMF and allow NMF to choose a different number of basis vectors for each view. For tag prediction, each test image is mapped onto the multiple latent spaces. The distances of images in these spaces are used to form a unified distance matrix. The... 

The number of digital images uploaded in the virtual world is rapidly growing every day. Therefore, an automatic image annotation system that can retrieve information from these images seems to be in high demand. One of the challenges in this field is the imbalanced data sets and the difficulty of successfully learning tags from them. Even if a nearly balanced data set exists for image annotation, it is unlikely to find a single learner, which could learn all tags with the same accuracy. In this paper, we suggest a novel integration system that selects an elite group of models from all existing annotation models and then combines them to take the best advantage of each model’s learning... 

Considering each of the visual features as one modality in image annotation task, efficient fusion of different modalities is essential in graph-based learning. Traditional graph-based methods consider one node for each image and combine its visual features into a single descriptor before constructing the graph. In this paper, we propose an approach that constructs a subgraph for each modality in such a way that edges of subgraph are determined using a search-based approach that handles class-imbalance challenge in the annotation datasets. Multiple subgraphs are then connected to each other to have a supergraph. This follows by introducing a learning framework to infer the tags of... 

In most image retrieval systems, images include various high-level semantics, called tags or annotations. Virtually all the state-of-the-art image annotation methods that handle imbalanced labeling are search-based techniques which are time-consuming. In this paper, a novel coupled dictionary learning approach is proposed to learn a limited number of visual prototypes and their corresponding semantics simultaneously. This approach leads to a real-time image annotation procedure. Another contribution of this paper is that utilizes a marginalized loss function instead of the squared loss function that is inappropriate for image annotation with imbalanced labels. We have employed a marginalized... 

Anatomical image segmentation is one of the foundations for medical planning. Recently, convolutional neural networks (CNN) have achieved much success in segmenting volumetric (3D) images when a large number of fully annotated 3D samples are available. However, rarely a volumetric medical image dataset containing a sufficient number of segmented 3D images is accessible since providing manual segmentation masks is monotonous and time-consuming. Thus, to alleviate the burden of manual annotation, we attempt to effectively train a 3D CNN using a sparse annotation where ground truth on just one 2D slice of the axial axis of each training 3D image is available. To tackle this problem, we propose...