query: My current job involves efficient image retrieval. The primary issue is that images require substantial storage space, and the constant conversion to and from compact code results in unnecessary time expenditure. Therefore, I am seeking a model capable of retrieving images in compact code format. The challenge for such a model is that images in compact code do not always maintain close local information, leading to increased computational costs for local features. Consequently, I am interested in image retrieval algorithms and innovative loss functions that prioritize global features. Additionally, I require algorithms that can add labels or annotations directly to the images’ compact code without necessitating any format conversion of the image. I prefer methods that have demonstrated effective performance on popular image retrieval benchmarks.

positive:
It has been shown that image descriptors extracted by convolutional neural
networks (CNNs) achieve remarkable results for retrieval problems. In this
paper, we apply attention mechanism to CNN, which aims at enhancing more
relevant features that correspond to important keypoints in the input image.
The generated attention-aware features are then aggregated by the previous
state-of-the-art generalized mean (GeM) pooling followed by normalization to
produce a compact global descriptor, which can be efficiently compared to other
image descriptors by the dot product. An extensive comparison of our proposed
approach with state-of-the-art methods is performed on the new challenging
ROxford5k and RParis6k retrieval benchmarks. Results indicate significant
improvement over previous work. In particular, our attention-aware GeM (AGeM)
descriptor outperforms state-of-the-art method on ROxford5k under the `Hard'
evaluation protocal.

positive:
Several recent works have shown that image descriptors produced by deep
convolutional neural networks provide state-of-the-art performance for image
classification and retrieval problems. It has also been shown that the
activations from the convolutional layers can be interpreted as local features
describing particular image regions. These local features can be aggregated
using aggregation approaches developed for local features (e.g. Fisher
vectors), thus providing new powerful global descriptors.
  In this paper we investigate possible ways to aggregate local deep features
to produce compact global descriptors for image retrieval. First, we show that
deep features and traditional hand-engineered features have quite different
distributions of pairwise similarities, hence existing aggregation methods have
to be carefully re-evaluated. Such re-evaluation reveals that in contrast to
shallow features, the simple aggregation method based on sum pooling provides
arguably the best performance for deep convolutional features. This method is
efficient, has few parameters, and bears little risk of overfitting when e.g.
learning the PCA matrix. Overall, the new compact global descriptor improves
the state-of-the-art on four common benchmarks considerably.

positive:
Hashing methods have been recently found very effective in retrieval of
remote sensing (RS) images due to their computational efficiency and fast
search speed. The traditional hashing methods in RS usually exploit
hand-crafted features to learn hash functions to obtain binary codes, which can
be insufficient to optimally represent the information content of RS images. To
overcome this problem, in this paper we introduce a metric-learning based
hashing network, which learns: 1) a semantic-based metric space for effective
feature representation; and 2) compact binary hash codes for fast archive
search. Our network considers an interplay of multiple loss functions that
allows to jointly learn a metric based semantic space facilitating similar
images to be clustered together in that target space and at the same time
producing compact final activations that lose negligible information when
binarized. Experiments carried out on two benchmark RS archives point out that
the proposed network significantly improves the retrieval performance under the
same retrieval time when compared to the state-of-the-art hashing methods in
RS.

positive:
The objective of this paper is to design an embedding method that maps local
features describing an image (e.g. SIFT) to a higher dimensional representation
useful for the image retrieval problem. First, motivated by the relationship
between the linear approximation of a nonlinear function in high dimensional
space and the stateof-the-art feature representation used in image retrieval,
i.e., VLAD, we propose a new approach for the approximation. The embedded
vectors resulted by the function approximation process are then aggregated to
form a single representation for image retrieval. Second, in order to make the
proposed embedding method applicable to large scale problem, we further derive
its fast version in which the embedded vectors can be efficiently computed,
i.e., in the closed-form. We compare the proposed embedding methods with the
state of the art in the context of image search under various settings: when
the images are represented by medium length vectors, short vectors, or binary
vectors. The experimental results show that the proposed embedding methods
outperform existing the state of the art on the standard public image retrieval
benchmarks.

positive:
Deep hashing methods have been proved to be effective for the large-scale
medical image search assisting reference-based diagnosis for clinicians.
However, when the salient region plays a maximal discriminative role in
ophthalmic image, existing deep hashing methods do not fully exploit the
learning ability of the deep network to capture the features of salient regions
pointedly. The different grades or classes of ophthalmic images may be share
similar overall performance but have subtle differences that can be
differentiated by mining salient regions. To address this issue, we propose a
novel end-to-end network, named Attention-based Saliency Hashing (ASH), for
learning compact hash-code to represent ophthalmic images. ASH embeds a
spatial-attention module to focus more on the representation of salient regions
and highlights their essential role in differentiating ophthalmic images.
Benefiting from the spatial-attention module, the information of salient
regions can be mapped into the hash-code for similarity calculation. In the
training stage, we input the image pairs to share the weights of the network,
and a pairwise loss is designed to maximize the discriminability of the
hash-code. In the retrieval stage, ASH obtains the hash-code by inputting an
image with an end-to-end manner, then the hash-code is used to similarity
calculation to return the most similar images. Extensive experiments on two
different modalities of ophthalmic image datasets demonstrate that the proposed
ASH can further improve the retrieval performance compared to the
state-of-the-art deep hashing methods due to the huge contributions of the
spatial-attention module.

positive:
Content-based image retrieval (CBIR) in large medical image archives is a
challenging and necessary task. Generally, different feature extraction methods
are used to assign expressive and invariant features to each image such that
the search for similar images comes down to feature classification and/or
matching. The present work introduces a new image retrieval method for medical
applications that employs a convolutional neural network (CNN) with recently
introduced Radon barcodes. We combine neural codes for global classification
with Radon barcodes for the final retrieval. We also examine image search based
on regions of interest (ROI) matching after image retrieval. The IRMA dataset
with more than 14,000 x-rays images is used to evaluate the performance of our
method. Experimental results show that our approach is superior to many
published works.

positive:
In this paper, we introduce an approach to overcome the low accuracy of the
Content-Based Image Retrieval (CBIR) (when using the global features). To
increase the accuracy, we use Harris-Laplace detector to identify the interest
regions of image. Then, we build the Region-Based Image Retrieval (RBIR). For
the efficient image storage and retrieval, we encode images into binary
signatures. The binary signature of a image is created from its interest
regions. Furthermore, this paper also provides an algorithm for image retrieval
on S-tree by comparing the images' signatures on a metric similarly to EMD
(earth mover's distance). Finally, we evaluate the created models on COREL's
images.

positive:
Content-based image retrieval (CBIR) systems on pixel domain use low-level
features, such as colour, texture and shape, to retrieve images. In this
context, two types of image representations i.e. local and global image
features have been studied in the literature. Extracting these features from
pixel images and comparing them with images from the database is very
time-consuming. Therefore, in recent years, there has been some effort to
accomplish image analysis directly in the compressed domain with lesser
computations. Furthermore, most of the images in our daily transactions are
stored in the JPEG compressed format. Therefore, it would be ideal if we could
retrieve features directly from the partially decoded or compressed data and
use them for retrieval. Here, we propose a unified model for image retrieval
which takes DCT coefficients as input and efficiently extracts global and local
features directly in the JPEG compressed domain for accurate image retrieval.
The experimental findings indicate that our proposed model performed similarly
to the current DELG model which takes RGB features as an input with reference
to mean average precision while having a faster training and retrieval speed.

positive:
It has been shown that the activations invoked by an image within the top
layers of a large convolutional neural network provide a high-level descriptor
of the visual content of the image. In this paper, we investigate the use of
such descriptors (neural codes) within the image retrieval application. In the
experiments with several standard retrieval benchmarks, we establish that
neural codes perform competitively even when the convolutional neural network
has been trained for an unrelated classification task (e.g.\ Image-Net). We
also evaluate the improvement in the retrieval performance of neural codes,
when the network is retrained on a dataset of images that are similar to images
encountered at test time.
  We further evaluate the performance of the compressed neural codes and show
that a simple PCA compression provides very good short codes that give
state-of-the-art accuracy on a number of datasets. In general, neural codes
turn out to be much more resilient to such compression in comparison other
state-of-the-art descriptors. Finally, we show that discriminative
dimensionality reduction trained on a dataset of pairs of matched photographs
improves the performance of PCA-compressed neural codes even further. Overall,
our quantitative experiments demonstrate the promise of neural codes as visual
descriptors for image retrieval.

positive:
Sketch-based image retrieval (SBIR) has undergone an increasing interest in
the community of computer vision bringing high impact in real applications. For
instance, SBIR brings an increased benefit to eCommerce search engines because
it allows users to formulate a query just by drawing what they need to buy.
However, current methods showing high precision in retrieval work in a high
dimensional space, which negatively affects aspects like memory consumption and
time processing. Although some authors have also proposed compact
representations, these drastically degrade the performance in a low dimension.
Therefore in this work, we present different results of evaluating methods for
producing compact embeddings in the context of sketch-based image retrieval.
Our main interest is in strategies aiming to keep the local structure of the
original space. The recent unsupervised local-topology preserving dimension
reduction method UMAP fits our requirements and shows outstanding performance,
improving even the precision achieved by SOTA methods. We evaluate six methods
in two different datasets. We use Flickr15K and eCommerce datasets; the latter
is another contribution of this work. We show that UMAP allows us to have
feature vectors of 16 bytes improving precision by more than 35%.

positive:
Due to an increase in the number of image achieves, Content-Based Image
Retrieval (CBIR) has gained attention for research community of computer
vision. The image visual contents are represented in a feature space in the
form of numerical values that is considered as a feature vector of image.
Images belonging to different classes may contain the common visuals and shapes
that can result in the closeness of computed feature space of two different
images belonging to separate classes. Due to this reason, feature extraction
and image representation is selected with appropriate features as it directly
affects the performance of image retrieval system. The commonly used visual
features are image spatial layout, color, texture and shape. Image feature
space is combined to achieve the discriminating ability that is not possible to
achieve when the features are used separately. Due to this reason, in this
paper, we aim to explore the low-level feature combination that are based on
color and shape features. We selected color moments and color histogram to
represent color while shape is represented by using invariant moments. We
selected this combination, as these features are reported intuitive, compact
and robust for image representation. We evaluated the performance of our
proposed research by using the Corel, Coil and Ground Truth (GT) image
datasets. We evaluated the proposed low-level feature fusion by calculating the
precision, recall and time required for feature extraction. The precision,
recall and feature extraction values obtained from the proposed low-level
feature fusion outperforms the existing research of CBIR.

positive:
Driven by the urgent demand for managing remote sensing big data, large-scale
remote sensing image retrieval (RSIR) attracts increasing attention in the
remote sensing field. In general, existing retrieval methods can be regarded as
visual-based retrieval approaches which search and return a set of similar
images from a database to a given query image. Although retrieval methods have
achieved great success, there is still a question that needs to be responded
to: Can we obtain the accurate semantic labels of the returned similar images
to further help analyzing and processing imagery? Inspired by the above
question, in this paper, we redefine the image retrieval problem as visual and
semantic retrieval of images. Specifically, we propose a novel deep hashing
convolutional neural network (DHCNN) to simultaneously retrieve the similar
images and classify their semantic labels in a unified framework. In more
detail, a convolutional neural network (CNN) is used to extract
high-dimensional deep features. Then, a hash layer is perfectly inserted into
the network to transfer the deep features into compact hash codes. In addition,
a fully connected layer with a softmax function is performed on hash layer to
generate class distribution. Finally, a loss function is elaborately designed
to simultaneously consider the label loss of each image and similarity loss of
pairs of images. Experimental results on two remote sensing datasets
demonstrate that the proposed method achieves the state-of-art retrieval and
classification performance.

positive:
Extracting informative image features and learning effective approximate
hashing functions are two crucial steps in image retrieval . Conventional
methods often study these two steps separately, e.g., learning hash functions
from a predefined hand-crafted feature space. Meanwhile, the bit lengths of
output hashing codes are preset in most previous methods, neglecting the
significance level of different bits and restricting their practical
flexibility. To address these issues, we propose a supervised learning
framework to generate compact and bit-scalable hashing codes directly from raw
images. We pose hashing learning as a problem of regularized similarity
learning. Specifically, we organize the training images into a batch of triplet
samples, each sample containing two images with the same label and one with a
different label. With these triplet samples, we maximize the margin between
matched pairs and mismatched pairs in the Hamming space. In addition, a
regularization term is introduced to enforce the adjacency consistency, i.e.,
images of similar appearances should have similar codes. The deep convolutional
neural network is utilized to train the model in an end-to-end fashion, where
discriminative image features and hash functions are simultaneously optimized.
Furthermore, each bit of our hashing codes is unequally weighted so that we can
manipulate the code lengths by truncating the insignificant bits. Our framework
outperforms state-of-the-arts on public benchmarks of similar image search and
also achieves promising results in the application of person re-identification
in surveillance. It is also shown that the generated bit-scalable hashing codes
well preserve the discriminative powers with shorter code lengths.

positive:
Hashing has been widely applied to multimodal retrieval on large-scale
multimedia data due to its efficiency in computation and storage. In this
article, we propose a novel deep semantic multimodal hashing network (DSMHN)
for scalable image-text and video-text retrieval. The proposed deep hashing
framework leverages 2-D convolutional neural networks (CNN) as the backbone
network to capture the spatial information for image-text retrieval, while the
3-D CNN as the backbone network to capture the spatial and temporal information
for video-text retrieval. In the DSMHN, two sets of modality-specific hash
functions are jointly learned by explicitly preserving both intermodality
similarities and intramodality semantic labels. Specifically, with the
assumption that the learned hash codes should be optimal for the classification
task, two stream networks are jointly trained to learn the hash functions by
embedding the semantic labels on the resultant hash codes. Moreover, a unified
deep multimodal hashing framework is proposed to learn compact and high-quality
hash codes by exploiting the feature representation learning, intermodality
similarity-preserving learning, semantic label-preserving learning, and hash
function learning with different types of loss functions simultaneously. The
proposed DSMHN method is a generic and scalable deep hashing framework for both
image-text and video-text retrievals, which can be flexibly integrated with
different types of loss functions. We conduct extensive experiments for both
single modal- and cross-modal-retrieval tasks on four widely used
multimodal-retrieval data sets. Experimental results on both image-text- and
video-text-retrieval tasks demonstrate that the DSMHN significantly outperforms
the state-of-the-art methods.

positive:
Most image instance retrieval pipelines are based on comparison of vectors
known as global image descriptors between a query image and the database
images. Due to their success in large scale image classification,
representations extracted from Convolutional Neural Networks (CNN) are quickly
gaining ground on Fisher Vectors (FVs) as state-of-the-art global descriptors
for image instance retrieval. While CNN-based descriptors are generally
remarked for good retrieval performance at lower bitrates, they nevertheless
present a number of drawbacks including the lack of robustness to common object
transformations such as rotations compared with their interest point based FV
counterparts.
  In this paper, we propose a method for computing invariant global descriptors
from CNNs. Our method implements a recently proposed mathematical theory for
invariance in a sensory cortex modeled as a feedforward neural network. The
resulting global descriptors can be made invariant to multiple arbitrary
transformation groups while retaining good discriminativeness.
  Based on a thorough empirical evaluation using several publicly available
datasets, we show that our method is able to significantly and consistently
improve retrieval results every time a new type of invariance is incorporated.
We also show that our method which has few parameters is not prone to
overfitting: improvements generalize well across datasets with different
properties with regard to invariances. Finally, we show that our descriptors
are able to compare favourably to other state-of-the-art compact descriptors in
similar bitranges, exceeding the highest retrieval results reported in the
literature on some datasets. A dedicated dimensionality reduction step
--quantization or hashing-- may be able to further improve the competitiveness
of the descriptors.

positive:
This paper addresses the problem of large-scale image retrieval. We propose a
two-layer fusion method which takes advantage of global and local cues and
ranks database images from coarse to fine (C2F). Departing from the previous
methods fusing multiple image descriptors simultaneously, C2F is featured by a
layered procedure composed by filtering and refining. In particular, C2F
consists of three components. 1) Distractor filtering. With holistic
representations, noise images are filtered out from the database, so the number
of candidate images to be used for comparison with the query can be greatly
reduced. 2) Adaptive weighting. For a certain query, the similarity of
candidate images can be estimated by holistic similarity scores in
complementary to the local ones. 3) Candidate refining. Accurate retrieval is
conducted via local features, combining the pre-computed adaptive weights.
Experiments are presented on two benchmarks, \emph{i.e.,} Holidays and Ukbench
datasets. We show that our method outperforms recent fusion methods in terms of
storage consumption and computation complexity, and that the accuracy is
competitive to the state-of-the-arts.

positive:
The use of high-dimensional features has become a normal practice in many
computer vision applications. The large dimension of these features is a
limiting factor upon the number of data points which may be effectively stored
and processed, however. We address this problem by developing a novel approach
to learning a compact binary encoding, which exploits both pair-wise proximity
and class-label information on training data set. Exploiting this extra
information allows the development of encodings which, although compact,
outperform the original high-dimensional features in terms of final
classification or retrieval performance. The method is general, in that it is
applicable to both non-parametric and parametric learning methods. This
generality means that the embedded features are suitable for a wide variety of
computer vision tasks, such as image classification and content-based image
retrieval. Experimental results demonstrate that the new compact descriptor
achieves an accuracy comparable to, and in some cases better than, the visual
descriptor in the original space despite being significantly more compact.
Moreover, any convex loss function and convex regularization penalty (e.g., $
\ell_p $ norm with $ p \ge 1 $) can be incorporated into the framework, which
provides future flexibility.

positive:
An image retrieval method based on convolution neural network and dimension
reduction is proposed in this paper. Convolution neural network is used to
extract high-level features of images, and to solve the problem that the
extracted feature dimensions are too high and have strong correlation,
multilinear principal component analysis is used to reduce the dimension of
features. The features after dimension reduction are binary hash coded for fast
image retrieval. Experiments show that the method proposed in this paper has
better retrieval effect than the retrieval method based on principal component
analysis on the e-commerce image datasets.

positive:
This paper focuses on network pruning for image retrieval acceleration.
Prevailing image retrieval works target at the discriminative feature learning,
while little attention is paid to how to accelerate the model inference, which
should be taken into consideration in real-world practice. The challenge of
pruning image retrieval models is that the middle-level feature should be
preserved as much as possible. Such different requirements of the retrieval and
classification model make the traditional pruning methods not that suitable for
our task. To solve the problem, we propose a new Progressive Local Filter
Pruning (PLFP) method for image retrieval acceleration. Specifically, layer by
layer, we analyze the local geometric properties of each filter and select the
one that can be replaced by the neighbors. Then we progressively prune the
filter by gradually changing the filter weights. In this way, the
representation ability of the model is preserved. To verify this, we evaluate
our method on two widely-used image retrieval datasets,i.e., Oxford5k and
Paris6K, and one person re-identification dataset,i.e., Market-1501. The
proposed method arrives with superior performance to the conventional pruning
methods, suggesting the effectiveness of the proposed method for image
retrieval.

positive:
One of the key challenges of deep learning based image retrieval remains in
aggregating convolutional activations into one highly representative feature
vector. Ideally, this descriptor should encode semantic, spatial and low level
information. Even though off-the-shelf pre-trained neural networks can already
produce good representations in combination with aggregation methods,
appropriate fine tuning for the task of image retrieval has shown to
significantly boost retrieval performance. In this paper, we present a simple
yet effective supervised aggregation method built on top of existing regional
pooling approaches. In addition to the maximum activation of a given region, we
calculate regional average activations of extracted feature maps. Subsequently,
weights for each of the pooled feature vectors are learned to perform a
weighted aggregation to a single feature vector. Furthermore, we apply our
newly proposed NRA loss function for deep metric learning to fine tune the
backbone neural network and to learn the aggregation weights. Our method
achieves state-of-the-art results for the INRIA Holidays data set and
competitive results for the Oxford Buildings and Paris data sets while reducing
the training time significantly.

positive:
Domain adaptive image retrieval includes single-domain retrieval and
cross-domain retrieval. Most of the existing image retrieval methods only focus
on single-domain retrieval, which assumes that the distributions of retrieval
databases and queries are similar. However, in practical application, the
discrepancies between retrieval databases often taken in ideal
illumination/pose/background/camera conditions and queries usually obtained in
uncontrolled conditions are very large. In this paper, considering the
practical application, we focus on challenging cross-domain retrieval. To
address the problem, we propose an effective method named Probability Weighted
Compact Feature Learning (PWCF), which provides inter-domain correlation
guidance to promote cross-domain retrieval accuracy and learns a series of
compact binary codes to improve the retrieval speed. First, we derive our loss
function through the Maximum A Posteriori Estimation (MAP): Bayesian
Perspective (BP) induced focal-triplet loss, BP induced quantization loss and
BP induced classification loss. Second, we propose a common manifold structure
between domains to explore the potential correlation across domains.
Considering the original feature representation is biased due to the
inter-domain discrepancy, the manifold structure is difficult to be
constructed. Therefore, we propose a new feature named Histogram Feature of
Neighbors (HFON) from the sample statistics perspective. Extensive experiments
on various benchmark databases validate that our method outperforms many
state-of-the-art image retrieval methods for domain adaptive image retrieval.
The source code is available at https://github.com/fuxianghuang1/PWCF

positive:
Fast and scalable Content-Based Image Retrieval using visual features is
required for document analysis, Medical image analysis, etc. in the present
age. Convolutional Neural Network (CNN) activations as features achieved their
outstanding performance in this area. Deep Convolutional representations using
the softmax function in the output layer are also ones among visual features.
However, almost all the image retrieval systems hold their index of visual
features on main memory in order to high responsiveness, limiting their
applicability for big data applications. In this paper, we propose a fast
calculation method of cosine similarity with L2 norm indexed in advance on
Elasticsearch. We evaluate our approach with ImageNet Dataset and VGG-16
pre-trained model. The evaluation results show the effectiveness and efficiency
of our proposed method.

positive:
Representing images by compact hash codes is an attractive approach for
large-scale content-based image retrieval. In most state-of-the-art
hashing-based image retrieval systems, for each image, local descriptors are
first aggregated as a global representation vector. This global vector is then
subjected to a hashing function to generate a binary hash code. In previous
works, the aggregating and the hashing processes are designed independently.
Hence these frameworks may generate suboptimal hash codes. In this paper, we
first propose a novel unsupervised hashing framework in which feature
aggregating and hashing are designed simultaneously and optimized jointly.
Specifically, our joint optimization generates aggregated representations that
can be better reconstructed by some binary codes. This leads to more
discriminative binary hash codes and improved retrieval accuracy. In addition,
the proposed method is flexible. It can be extended for supervised hashing.
When the data label is available, the framework can be adapted to learn binary
codes which minimize the reconstruction loss w.r.t. label vectors. Furthermore,
we also propose a fast version of the state-of-the-art hashing method Binary
Autoencoder to be used in our proposed frameworks. Extensive experiments on
benchmark datasets under various settings show that the proposed methods
outperform state-of-the-art unsupervised and supervised hashing methods.

positive:
With the rapid growing of remotely sensed imagery data, there is a high
demand for effective and efficient image retrieval tools to manage and exploit
such data. In this letter, we present a novel content-based remote sensing
image retrieval method based on Triplet deep metric learning convolutional
neural network (CNN). By constructing a Triplet network with metric learning
objective function, we extract the representative features of the images in a
semantic space in which images from the same class are close to each other
while those from different classes are far apart. In such a semantic space,
simple metric measures such as Euclidean distance can be used directly to
compare the similarity of images and effectively retrieve images of the same
class. We also investigate a supervised and an unsupervised learning methods
for reducing the dimensionality of the learned semantic features. We present
comprehensive experimental results on two publicly available remote sensing
image retrieval datasets and show that our method significantly outperforms
state-of-the-art.

positive:
Image retrieval is the problem of searching an image database for items that
are similar to a query image. To address this task, two main types of image
representations have been studied: global and local image features. In this
work, our key contribution is to unify global and local features into a single
deep model, enabling accurate retrieval with efficient feature extraction. We
refer to the new model as DELG, standing for DEep Local and Global features. We
leverage lessons from recent feature learning work and propose a model that
combines generalized mean pooling for global features and attentive selection
for local features. The entire network can be learned end-to-end by carefully
balancing the gradient flow between two heads -- requiring only image-level
labels. We also introduce an autoencoder-based dimensionality reduction
technique for local features, which is integrated into the model, improving
training efficiency and matching performance. Comprehensive experiments show
that our model achieves state-of-the-art image retrieval on the Revisited
Oxford and Paris datasets, and state-of-the-art single-model instance-level
recognition on the Google Landmarks dataset v2. Code and models are available
at https://github.com/tensorflow/models/tree/master/research/delf .

positive:
Text contained in an image carries high-level semantics that can be exploited
to achieve richer image understanding. In particular, the mere presence of text
provides strong guiding content that should be employed to tackle a diversity
of computer vision tasks such as image retrieval, fine-grained classification,
and visual question answering. In this paper, we address the problem of
fine-grained classification and image retrieval by leveraging textual
information along with visual cues to comprehend the existing intrinsic
relation between the two modalities. The novelty of the proposed model consists
of the usage of a PHOC descriptor to construct a bag of textual words along
with a Fisher Vector Encoding that captures the morphology of text. This
approach provides a stronger multimodal representation for this task and as our
experiments demonstrate, it achieves state-of-the-art results on two different
tasks, fine-grained classification and image retrieval.

positive:
Similarity-preserving hashing is a commonly used method for nearest neighbour
search in large-scale image retrieval. For image retrieval, deep-networks-based
hashing methods are appealing since they can simultaneously learn effective
image representations and compact hash codes. This paper focuses on
deep-networks-based hashing for multi-label images, each of which may contain
objects of multiple categories. In most existing hashing methods, each image is
represented by one piece of hash code, which is referred to as semantic
hashing. This setting may be suboptimal for multi-label image retrieval. To
solve this problem, we propose a deep architecture that learns
\textbf{instance-aware} image representations for multi-label image data, which
are organized in multiple groups, with each group containing the features for
one category. The instance-aware representations not only bring advantages to
semantic hashing, but also can be used in category-aware hashing, in which an
image is represented by multiple pieces of hash codes and each piece of code
corresponds to a category. Extensive evaluations conducted on several benchmark
datasets demonstrate that, for both semantic hashing and category-aware
hashing, the proposed method shows substantial improvement over the
state-of-the-art supervised and unsupervised hashing methods.

positive:
We investigate the problem of image retrieval based on visual queries when
the latter comprise arbitrary regions-of-interest (ROI) rather than entire
images. Our proposal is a compact image descriptor that combines the
state-of-the-art in content-based descriptor extraction with a multi-level,
Voronoi-based spatial partitioning of each dataset image. The proposed
multi-level Voronoi-based encoding uses a spatial hierarchical K-means over
interest-point locations, and computes a content-based descriptor over each
cell. In order to reduce the matching complexity with minimal or no sacrifice
in retrieval performance: (i) we utilize the tree structure of the spatial
hierarchical K-means to perform a top-to-bottom pruning for local similarity
maxima; (ii) we propose a new image similarity score that combines relevant
information from all partition levels into a single measure for similarity;
(iii) we combine our proposal with a novel and efficient approach for optimal
bit allocation within quantized descriptor representations. By deriving both a
Voronoi-based VLAD descriptor (termed as Fast-VVLAD) and a Voronoi-based deep
convolutional neural network (CNN) descriptor (termed as Fast-VDCNN), we
demonstrate that our Voronoi-based framework is agnostic to the descriptor
basis, and can easily be slotted into existing frameworks. Via a range of ROI
queries in two standard datasets, it is shown that the Voronoi-based
descriptors achieve comparable or higher mean Average Precision against
conventional grid-based spatial search, while offering more than two-fold
reduction in complexity. Finally, beyond ROI queries, we show that Voronoi
partitioning improves the geometric invariance of compact CNN descriptors,
thereby resulting in competitive performance to the current state-of-the-art on
whole image retrieval.

positive:
Due to its low storage cost and fast query speed, hashing has been widely
used in large-scale image retrieval tasks. Hash bucket search returns data
points within a given Hamming radius to each query, which can enable search at
a constant or sub-linear time cost. However, existing hashing methods cannot
achieve satisfactory retrieval performance for hash bucket search in complex
scenarios, since they learn only one hash code for each image. More
specifically, by using one hash code to represent one image, existing methods
might fail to put similar image pairs to the buckets with a small Hamming
distance to the query when the semantic information of images is complex. As a
result, a large number of hash buckets need to be visited for retrieving
similar images, based on the learned codes. This will deteriorate the
efficiency of hash bucket search. In this paper, we propose a novel hashing
framework, called multiple code hashing (MCH), to improve the performance of
hash bucket search. The main idea of MCH is to learn multiple hash codes for
each image, with each code representing a different region of the image.
Furthermore, we propose a deep reinforcement learning algorithm to learn the
parameters in MCH. To the best of our knowledge, this is the first work that
proposes to learn multiple hash codes for each image in image retrieval.
Experiments demonstrate that MCH can achieve a significant improvement in hash
bucket search, compared with existing methods that learn only one hash code for
each image.

positive:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

positive:
Similarity-preserving hashing is a widely-used method for nearest neighbour
search in large-scale image retrieval tasks. For most existing hashing methods,
an image is first encoded as a vector of hand-engineering visual features,
followed by another separate projection or quantization step that generates
binary codes. However, such visual feature vectors may not be optimally
compatible with the coding process, thus producing sub-optimal hashing codes.
In this paper, we propose a deep architecture for supervised hashing, in which
images are mapped into binary codes via carefully designed deep neural
networks. The pipeline of the proposed deep architecture consists of three
building blocks: 1) a sub-network with a stack of convolution layers to produce
the effective intermediate image features; 2) a divide-and-encode module to
divide the intermediate image features into multiple branches, each encoded
into one hash bit; and 3) a triplet ranking loss designed to characterize that
one image is more similar to the second image than to the third one. Extensive
evaluations on several benchmark image datasets show that the proposed
simultaneous feature learning and hash coding pipeline brings substantial
improvements over other state-of-the-art supervised or unsupervised hashing
methods.

positive:
With the rapid growth of web images, hashing has received increasing
interests in large scale image retrieval. Research efforts have been devoted to
learning compact binary codes that preserve semantic similarity based on
labels. However, most of these hashing methods are designed to handle simple
binary similarity. The complex multilevel semantic structure of images
associated with multiple labels have not yet been well explored. Here we
propose a deep semantic ranking based method for learning hash functions that
preserve multilevel semantic similarity between multi-label images. In our
approach, deep convolutional neural network is incorporated into hash functions
to jointly learn feature representations and mappings from them to hash codes,
which avoids the limitation of semantic representation power of hand-crafted
features. Meanwhile, a ranking list that encodes the multilevel similarity
information is employed to guide the learning of such deep hash functions. An
effective scheme based on surrogate loss is used to solve the intractable
optimization problem of nonsmooth and multivariate ranking measures involved in
the learning procedure. Experimental results show the superiority of our
proposed approach over several state-of-the-art hashing methods in term of
ranking evaluation metrics when tested on multi-label image datasets.

positive:
In most state-of-the-art hashing-based visual search systems, local image
descriptors of an image are first aggregated as a single feature vector. This
feature vector is then subjected to a hashing function that produces a binary
hash code. In previous work, the aggregating and the hashing processes are
designed independently. In this paper, we propose a novel framework where
feature aggregating and hashing are designed simultaneously and optimized
jointly. Specifically, our joint optimization produces aggregated
representations that can be better reconstructed by some binary codes. This
leads to more discriminative binary hash codes and improved retrieval accuracy.
In addition, we also propose a fast version of the recently-proposed Binary
Autoencoder to be used in our proposed framework. We perform extensive
retrieval experiments on several benchmark datasets with both SIFT and
convolutional features. Our results suggest that the proposed framework
achieves significant improvements over the state of the art.

positive:
Recently, image representation built upon Convolutional Neural Network (CNN)
has been shown to provide effective descriptors for image search, outperforming
pre-CNN features as short-vector representations. Yet such models are not
compatible with geometry-aware re-ranking methods and still outperformed, on
some particular object retrieval benchmarks, by traditional image search
systems relying on precise descriptor matching, geometric re-ranking, or query
expansion. This work revisits both retrieval stages, namely initial search and
re-ranking, by employing the same primitive information derived from the CNN.
We build compact feature vectors that encode several image regions without the
need to feed multiple inputs to the network. Furthermore, we extend integral
images to handle max-pooling on convolutional layer activations, allowing us to
efficiently localize matching objects. The resulting bounding box is finally
used for image re-ranking. As a result, this paper significantly improves
existing CNN-based recognition pipeline: We report for the first time results
competing with traditional methods on the challenging Oxford5k and Paris6k
datasets.

positive:
The convolutional neural network (CNN) features can give a good description
of image content, which usually represent images with unique global vectors.
Although they are compact compared to local descriptors, they still cannot
efficiently deal with large-scale image retrieval due to the cost of the linear
incremental computation and storage. To address this issue, we build a simple
but effective indexing framework based on inverted table, which significantly
decreases both the search time and memory usage. In addition, several
strategies are fully investigated under an indexing framework to adapt it to
CNN features and compensate for quantization errors. First, we use multiple
assignment for the query and database images to increase the probability of
relevant images' co-existing in the same Voronoi cells obtained via the
clustering algorithm. Then, we introduce embedding codes to further improve
precision by removing false matches during a search. We demonstrate that by
using hashing schemes to calculate the embedding codes and by changing the
ranking rule, indexing framework speeds can be greatly improved. Extensive
experiments conducted on several unsupervised and supervised benchmarks support
these results and the superiority of the proposed indexing framework. We also
provide a fair comparison between the popular CNN features.

positive:
Fine-grained image recognition is very challenging due to the difficulty of
capturing both semantic global features and discriminative local features.
Meanwhile, these two features are not easy to be integrated, which are even
conflicting when used simultaneously. In this paper, a retrieval-based
coarse-to-fine framework is proposed, where we re-rank the TopN classification
results by using the local region enhanced embedding features to improve the
Top1 accuracy (based on the observation that the correct category usually
resides in TopN results). To obtain the discriminative regions for
distinguishing the fine-grained images, we introduce a weakly-supervised method
to train a box generating branch with only image-level labels. In addition, to
learn more effective semantic global features, we design a multi-level loss
over an automatically constructed hierarchical category structure. Experimental
results show that our method achieves state-of-the-art performance on three
benchmarks: CUB-200-2011, Stanford Cars, and FGVC Aircraft. Also,
visualizations and analysis are provided for better understanding.

positive:
Medical Image Retrieval is a challenging field in Visual information
retrieval, due to the multi-dimensional and multi-modal context of the
underlying content. Traditional models often fail to take the intrinsic
characteristics of data into consideration, and have thus achieved limited
accuracy when applied to medical images. The Bag of Visual Words (BoVW) is a
technique that can be used to effectively represent intrinsic image features in
vector space, so that applications like image classification and similar-image
search can be optimized. In this paper, we present a MedIR approach based on
the BoVW model for content-based medical image retrieval. As medical images as
multi-dimensional, they exhibit underlying cluster and manifold information
which enhances semantic relevance and allows for label uniformity. Hence, the
BoVW features extracted for each image are used to train a supervised machine
learning classifier based on positive and negative training images, for
extending content based image retrieval. During experimental validation, the
proposed model performed very well, achieving a Mean Average Precision of
88.89% during top-3 image retrieval experiments.

positive:
Remote Sensing Image Retrieval remains a challenging topic due to the special
nature of Remote Sensing Imagery. Such images contain various different
semantic objects, which clearly complicates the retrieval task. In this paper,
we present an image retrieval pipeline that uses attentive, local convolutional
features and aggregates them using the Vector of Locally Aggregated Descriptors
(VLAD) to produce a global descriptor. We study various system parameters such
as the multiplicative and additive attention mechanisms and descriptor
dimensionality. We propose a query expansion method that requires no external
inputs. Experiments demonstrate that even without training, the local
convolutional features and global representation outperform other systems.
After system tuning, we can achieve state-of-the-art or competitive results.
Furthermore, we observe that our query expansion method increases overall
system performance by about 3%, using only the top-three retrieved images.
Finally, we show how dimensionality reduction produces compact descriptors with
increased retrieval performance and fast retrieval computation times, e.g. 50%
faster than the current systems.

positive:
Visual localization to compute 6DoF camera pose from a given image has wide
applications such as in robotics, virtual reality, augmented reality, etc. Two
kinds of descriptors are important for the visual localization. One is global
descriptors that extract the whole feature from each image. The other is local
descriptors that extract the local feature from each image patch usually
enclosing a key point. More and more methods of the visual localization have
two stages: at first to perform image retrieval by global descriptors and then
from the retrieval feedback to make 2D-3D point correspondences by local
descriptors. The two stages are in serial for most of the methods. This simple
combination has not achieved superiority of fusing local and global
descriptors. The 3D points obtained from the retrieval feedback are as the
nearest neighbor candidates of the 2D image points only by global descriptors.
Each of the 2D image points is also called a query local feature when
performing the 2D-3D point correspondences. In this paper, we propose a novel
parallel search framework, which leverages advantages of both local and global
descriptors to get nearest neighbor candidates of a query local feature.
Specifically, besides using deep learning based global descriptors, we also
utilize local descriptors to construct random tree structures for obtaining
nearest neighbor candidates of the query local feature. We propose a new
probabilistic model and a new deep learning based local descriptor when
constructing the random trees. A weighted Hamming regularization term to keep
discriminativeness after binarization is given in the loss function for the
proposed local descriptor. The loss function co-trains both real and binary
descriptors of which the results are integrated into the random trees.

positive:
We propose a novel approach for instance-level image retrieval. It produces a
global and compact fixed-length representation for each image by aggregating
many region-wise descriptors. In contrast to previous works employing
pre-trained deep networks as a black box to produce features, our method
leverages a deep architecture trained for the specific task of image retrieval.
Our contribution is twofold: (i) we leverage a ranking framework to learn
convolution and projection weights that are used to build the region features;
and (ii) we employ a region proposal network to learn which regions should be
pooled to form the final global descriptor. We show that using clean training
data is key to the success of our approach. To that aim, we use a large scale
but noisy landmark dataset and develop an automatic cleaning approach. The
proposed architecture produces a global image representation in a single
forward pass. Our approach significantly outperforms previous approaches based
on global descriptors on standard datasets. It even surpasses most prior works
based on costly local descriptor indexing and spatial verification. Additional
material is available at www.xrce.xerox.com/Deep-Image-Retrieval.

positive:
Query expansion is a popular method to improve the quality of image retrieval
with both conventional and CNN representations. It has been so far limited to
global image similarity. This work focuses on diffusion, a mechanism that
captures the image manifold in the feature space. The diffusion is carried out
on descriptors of overlapping image regions rather than on a global image
descriptor like in previous approaches. An efficient off-line stage allows
optional reduction in the number of stored regions. In the on-line stage, the
proposed handling of unseen queries in the indexing stage removes additional
computation to adjust the precomputed data. We perform diffusion through a
sparse linear system solver, yielding practical query times well below one
second. Experimentally, we observe a significant boost in performance of image
retrieval with compact CNN descriptors on standard benchmarks, especially when
the query object covers only a small part of the image. Small objects have been
a common failure case of CNN-based retrieval.

positive:
In the large-scale image retrieval task, the two most important requirements
are the discriminability of image representations and the efficiency in
computation and storage of representations. Regarding the former requirement,
Convolutional Neural Network (CNN) is proven to be a very powerful tool to
extract highly discriminative local descriptors for effective image search.
Additionally, in order to further improve the discriminative power of the
descriptors, recent works adopt fine-tuned strategies. In this paper, taking a
different approach, we propose a novel, computationally efficient, and
competitive framework. Specifically, we firstly propose various strategies to
compute masks, namely SIFT-mask, SUM-mask, and MAX-mask, to select a
representative subset of local convolutional features and eliminate redundant
features. Our in-depth analyses demonstrate that proposed masking schemes are
effective to address the burstiness drawback and improve retrieval accuracy.
Secondly, we propose to employ recent embedding and aggregating methods which
can significantly boost the feature discriminability. Regarding the computation
and storage efficiency, we include a hashing module to produce very compact
binary image representations. Extensive experiments on six image retrieval
benchmarks demonstrate that our proposed framework achieves the
state-of-the-art retrieval performances.

positive:
Image Retrieval is a fundamental task of obtaining images similar to the
query one from a database. A common image retrieval practice is to firstly
retrieve candidate images via similarity search using global image features and
then re-rank the candidates by leveraging their local features. Previous
learning-based studies mainly focus on either global or local image
representation learning to tackle the retrieval task. In this paper, we abandon
the two-stage paradigm and seek to design an effective single-stage solution by
integrating local and global information inside images into compact image
representations. Specifically, we propose a Deep Orthogonal Local and Global
(DOLG) information fusion framework for end-to-end image retrieval. It
attentively extracts representative local information with multi-atrous
convolutions and self-attention at first. Components orthogonal to the global
image representation are then extracted from the local information. At last,
the orthogonal components are concatenated with the global representation as a
complementary, and then aggregation is performed to generate the final
representation. The whole framework is end-to-end differentiable and can be
trained with image-level labels. Extensive experimental results validate the
effectiveness of our solution and show that our model achieves state-of-the-art
image retrieval performances on Revisited Oxford and Paris datasets.

positive:
Deep hashing has been widely applied to large-scale image retrieval by
encoding high-dimensional data points into binary codes for efficient
retrieval. Compared with pairwise/triplet similarity based hash learning,
central similarity based hashing can more efficiently capture the global data
distribution. For multi-label image retrieval, however, previous methods only
use multiple hash centers with equal weights to generate one centroid as the
learning target, which ignores the relationship between the weights of hash
centers and the proportion of instance regions in the image. To address the
above issue, we propose a two-step alternative optimization approach,
Instance-weighted Central Similarity (ICS), to automatically learn the center
weight corresponding to a hash code. Firstly, we apply the maximum entropy
regularizer to prevent one hash center from dominating the loss function, and
compute the center weights via projection gradient descent. Secondly, we update
neural network parameters by standard back-propagation with fixed center
weights. More importantly, the learned center weights can well reflect the
proportion of foreground instances in the image. Our method achieves the
state-of-the-art performance on the image retrieval benchmarks, and especially
improves the mAP by 1.6%-6.4% on the MS COCO dataset.

positive:
In applications involving matching of image sets, the information from
multiple images must be effectively exploited to represent each set.
State-of-the-art methods use probabilistic distribution or subspace to model a
set and use specific distance measure to compare two sets. These methods are
slow to compute and not compact to use in a large scale scenario.
Learning-based hashing is often used in large scale image retrieval as they
provide a compact representation of each sample and the Hamming distance can be
used to efficiently compare two samples. However, most hashing methods encode
each image separately and discard knowledge that multiple images in the same
set represent the same object or person. We investigate the set hashing problem
by combining both set representation and hashing in a single deep neural
network. An image set is first passed to a CNN module to extract image
features, then these features are aggregated using two types of set feature to
capture both set specific and database-wide distribution information. The
computed set feature is then fed into a multilayer perceptron to learn a
compact binary embedding. Triplet loss is used to train the network by forming
set similarity relations using class labels. We extensively evaluate our
approach on datasets used for image matching and show highly competitive
performance compared to state-of-the-art methods.

positive:
With the explosive growth of image databases, deep hashing, which learns
compact binary descriptors for images, has become critical for fast image
retrieval. Many existing deep hashing methods leverage quantization loss,
defined as distance between the features before and after quantization, to
reduce the error from binarizing features. While minimizing the quantization
loss guarantees that quantization has minimal effect on retrieval accuracy, it
unfortunately significantly reduces the expressiveness of features even before
the quantization. In this paper, we show that the above definition of
quantization loss is too restricted and in fact not necessary for maintaining
high retrieval accuracy. We therefore propose a new form of quantization loss
measured in triplets. The core idea of the triplet quantization loss is to
learn discriminative real-valued descriptors which lead to minimal loss on
retrieval accuracy after quantization. Extensive experiments on two widely used
benchmark data sets of different scales, CIFAR-10 and In-shop, demonstrate that
the proposed method outperforms the state-of-the-art deep hashing methods.
Moreover, we show that the compact binary descriptors obtained with triplet
quantization loss lead to very small performance drop after quantization.

positive:
Vision-based localization of an agent in a map is an important problem in
robotics and computer vision. In that context, localization by learning
matchable image features is gaining popularity due to recent advances in
machine learning. Features that uniquely describe the visual contents of images
have a wide range of applications, including image retrieval and understanding.
In this work, we propose a method that learns image features targeted for
image-retrieval-based localization. Retrieval-based localization has several
benefits, such as easy maintenance and quick computation. However, the
state-of-the-art features only provide visual similarity scores which do not
explicitly reveal the geometric distance between query and retrieved images.
Knowing this distance is highly desirable for accurate localization, especially
when the reference images are sparsely distributed in the scene. Therefore, we
propose a novel loss function for learning image features which are both
visually representative and geometrically relatable. This is achieved by
guiding the learning process such that the feature and geometric distances
between images are directly proportional. In our experiments we show that our
features not only offer significantly better localization accuracy, but also
allow to estimate the trajectory of a query sequence in absence of the
reference images.

positive:
While deep learning has become a key ingredient in the top performing methods
for many computer vision tasks, it has failed so far to bring similar
improvements to instance-level image retrieval. In this article, we argue that
reasons for the underwhelming results of deep methods on image retrieval are
threefold: i) noisy training data, ii) inappropriate deep architecture, and
iii) suboptimal training procedure. We address all three issues.
  First, we leverage a large-scale but noisy landmark dataset and develop an
automatic cleaning method that produces a suitable training set for deep
retrieval. Second, we build on the recent R-MAC descriptor, show that it can be
interpreted as a deep and differentiable architecture, and present improvements
to enhance it. Last, we train this network with a siamese architecture that
combines three streams with a triplet loss. At the end of the training process,
the proposed architecture produces a global image representation in a single
forward pass that is well suited for image retrieval. Extensive experiments
show that our approach significantly outperforms previous retrieval approaches,
including state-of-the-art methods based on costly local descriptor indexing
and spatial verification. On Oxford 5k, Paris 6k and Holidays, we respectively
report 94.7, 96.6, and 94.8 mean average precision. Our representations can
also be heavily compressed using product quantization with little loss in
accuracy. For additional material, please see
www.xrce.xerox.com/Deep-Image-Retrieval.

positive:
Image descriptors based on activations of Convolutional Neural Networks
(CNNs) have become dominant in image retrieval due to their discriminative
power, compactness of representation, and search efficiency. Training of CNNs,
either from scratch or fine-tuning, requires a large amount of annotated data,
where a high quality of annotation is often crucial. In this work, we propose
to fine-tune CNNs for image retrieval on a large collection of unordered images
in a fully automated manner. Reconstructed 3D models obtained by the
state-of-the-art retrieval and structure-from-motion methods guide the
selection of the training data. We show that both hard-positive and
hard-negative examples, selected by exploiting the geometry and the camera
positions available from the 3D models, enhance the performance of
particular-object retrieval. CNN descriptor whitening discriminatively learned
from the same training data outperforms commonly used PCA whitening. We propose
a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and
average pooling and show that it boosts retrieval performance. Applying the
proposed method to the VGG network achieves state-of-the-art performance on the
standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.

positive:
The cost-effective visual representation and fast query-by-example search are
two challenging goals that should be maintained for web-scale visual retrieval
tasks on moderate hardware. This paper introduces a fast and robust method that
ensures both of these goals by obtaining state-of-the-art performance for an
image-to-video search scenario. Hence, we present critical enhancements to
well-known indexing and visual representation techniques by promoting faster,
better and moderate retrieval performance. We also boost the superiority of our
method for some visual challenges by exploiting individual decisions of local
and global descriptors at query time. For instance, local content descriptors
represent copied/duplicated scenes with large geometric deformations such as
scale, orientation and affine transformation. In contrast, the use of global
content descriptors is more practical for near-duplicate and semantic searches.
Experiments are conducted on a large-scale Stanford I2V dataset. The
experimental results show that our method is useful in terms of complexity and
query processing time for large-scale visual retrieval scenarios, even if local
and global representations are used together. The proposed method is superior
and achieves state-of-the-art performance based on the mean average precision
(MAP) score of this dataset. Lastly, we report additional MAP scores after
updating the ground annotations unveiled by retrieval results of the proposed
method, and it shows that the actual performance.

positive:
Image retrieval in realistic scenarios targets large dynamic datasets of
unlabeled images. In these cases, training or fine-tuning a model every time
new images are added to the database is neither efficient nor scalable.
Convolutional neural networks trained for image classification over large
datasets have been proven effective feature extractors for image retrieval. The
most successful approaches are based on encoding the activations of
convolutional layers, as they convey the image spatial information. In this
paper, we go beyond this spatial information and propose a local-aware encoding
of convolutional features based on semantic information predicted in the target
image. To this end, we obtain the most discriminative regions of an image using
Class Activation Maps (CAMs). CAMs are based on the knowledge contained in the
network and therefore, our approach, has the additional advantage of not
requiring external information. In addition, we use CAMs to generate object
proposals during an unsupervised re-ranking stage after a first fast search.
Our experiments on two public available datasets for instance retrieval,
Oxford5k and Paris6k, demonstrate the competitiveness of our approach
outperforming the current state-of-the-art when using off-the-shelf models
trained on ImageNet. The source code and model used in this paper are publicly
available at http://imatge-upc.github.io/retrieval-2017-cam/.

positive:
Visual analytics have played an increasingly critical role in the Internet of
Things, where massive visual signals have to be compressed and fed into
machines. But facing such big data and constrained bandwidth capacity, existing
image/video compression methods lead to very low-quality representations, while
existing feature compression techniques fail to support diversified visual
analytics applications/tasks with low-bit-rate representations. In this paper,
we raise and study the novel problem of supporting multiple machine vision
analytics tasks with the compressed visual representation, namely, the
information compression problem in analytics taxonomy. By utilizing the
intrinsic transferability among different tasks, our framework successfully
constructs compact and expressive representations at low bit-rates to support a
diversified set of machine vision tasks, including both high-level
semantic-related tasks and mid-level geometry analytic tasks. In order to
impose compactness in the representations, we propose a codebook-based
hyperprior, which helps map the representation into a low-dimensional manifold.
As it well fits the signal structure of the deep visual feature, it facilitates
more accurate entropy estimation, and results in higher compression efficiency.
With the proposed framework and the codebook-based hyperprior, we further
investigate the relationship of different task features owning different levels
of abstraction granularity. Experimental results demonstrate that with the
proposed scheme, a set of diversified tasks can be supported at a significantly
lower bit-rate, compared with existing compression schemes.

negative:
Image instance retrieval is the problem of retrieving images from a database
which contain the same object. Convolutional Neural Network (CNN) based
descriptors are becoming the dominant approach for generating {\it global image
descriptors} for the instance retrieval problem. One major drawback of
CNN-based {\it global descriptors} is that uncompressed deep neural network
models require hundreds of megabytes of storage making them inconvenient to
deploy in mobile applications or in custom hardware. In this work, we study the
problem of neural network model compression focusing on the image instance
retrieval task. We study quantization, coding, pruning and weight sharing
techniques for reducing model size for the instance retrieval problem. We
provide extensive experimental results on the trade-off between retrieval
performance and model size for different types of networks on several data sets
providing the most comprehensive study on this topic. We compress models to the
order of a few MBs: two orders of magnitude smaller than the uncompressed
models while achieving negligible loss in retrieval performance.

negative:
Convolutional Neural Networks (CNNs) have achieved superior performance on
object image retrieval, while Bag-of-Words (BoW) models with handcrafted local
features still dominate the retrieval of overlapping images in 3D
reconstruction. In this paper, we narrow down this gap by presenting an
efficient CNN-based method to retrieve images with overlaps, which we refer to
as the matchable image retrieval problem. Different from previous methods that
generates training data based on sparse reconstruction, we create a large-scale
image database with rich 3D geometrics and exploit information from surface
reconstruction to obtain fine-grained training data. We propose a batched
triplet-based loss function combined with mesh re-projection to effectively
learn the CNN representation. The proposed method significantly accelerates the
image retrieval process in 3D reconstruction and outperforms the
state-of-the-art CNN-based and BoW methods for matchable image retrieval. The
code and data are available at https://github.com/hlzz/mirror.

negative:
Recent works in deep-learning have shown that second-order information is
beneficial in many computer-vision tasks. Second-order information can be
enforced both in the spatial context and the abstract feature dimensions. In
this work, we explore two second-order components. One is focused on
second-order spatial information to increase the performance of image
descriptors, both local and global. It is used to re-weight feature maps, and
thus emphasise salient image locations that are subsequently used for
description. The second component is concerned with a second-order similarity
(SOS) loss, that we extend to global descriptors for image retrieval, and is
used to enhance the triplet loss with hard-negative mining. We validate our
approach on two different tasks and datasets for image retrieval and image
matching. The results show that our two second-order components complement each
other, bringing significant performance improvements in both tasks and lead to
state-of-the-art results across the public benchmarks. Code available at:
http://github.com/tonyngjichun/SOLAR

negative:
The content based image retrieval aims to find the similar images from a
large scale dataset against a query image. Generally, the similarity between
the representative features of the query image and dataset images is used to
rank the images for retrieval. In early days, various hand designed feature
descriptors have been investigated based on the visual cues such as color,
texture, shape, etc. that represent the images. However, the deep learning has
emerged as a dominating alternative of hand-designed feature engineering from a
decade. It learns the features automatically from the data. This paper presents
a comprehensive survey of deep learning based developments in the past decade
for content based image retrieval. The categorization of existing
state-of-the-art methods from different perspectives is also performed for
greater understanding of the progress. The taxonomy used in this survey covers
different supervision, different networks, different descriptor type and
different retrieval type. A performance analysis is also performed using the
state-of-the-art methods. The insights are also presented for the benefit of
the researchers to observe the progress and to make the best choices. The
survey presented in this paper will help in further research progress in image
retrieval using deep learning.

negative:
Convolutional Neural Networks (CNNs) achieve state-of-the-art performance in
many computer vision tasks. However, this achievement is preceded by extreme
manual annotation in order to perform either training from scratch or
fine-tuning for the target task. In this work, we propose to fine-tune CNN for
image retrieval from a large collection of unordered images in a fully
automated manner. We employ state-of-the-art retrieval and
Structure-from-Motion (SfM) methods to obtain 3D models, which are used to
guide the selection of the training data for CNN fine-tuning. We show that both
hard positive and hard negative examples enhance the final performance in
particular object retrieval with compact codes.

negative:
Image representations derived from pre-trained Convolutional Neural Networks
(CNNs) have become the new state of the art in computer vision tasks such as
instance retrieval. This work explores the suitability for instance retrieval
of image- and region-wise representations pooled from an object detection CNN
such as Faster R-CNN. We take advantage of the object proposals learned by a
Region Proposal Network (RPN) and their associated CNN features to build an
instance search pipeline composed of a first filtering stage followed by a
spatial reranking. We further investigate the suitability of Faster R-CNN
features when the network is fine-tuned for the same objects one wants to
retrieve. We assess the performance of our proposed system with the Oxford
Buildings 5k, Paris Buildings 6k and a subset of TRECVid Instance Search 2013,
achieving competitive results.

negative:
This paper presents a simple yet effective supervised deep hash approach that
constructs binary hash codes from labeled data for large-scale image search. We
assume that the semantic labels are governed by several latent attributes with
each attribute on or off, and classification relies on these attributes. Based
on this assumption, our approach, dubbed supervised semantics-preserving deep
hashing (SSDH), constructs hash functions as a latent layer in a deep network
and the binary codes are learned by minimizing an objective function defined
over classification error and other desirable hash codes properties. With this
design, SSDH has a nice characteristic that classification and retrieval are
unified in a single learning model. Moreover, SSDH performs joint learning of
image representations, hash codes, and classification in a point-wised manner,
and thus is scalable to large-scale datasets. SSDH is simple and can be
realized by a slight enhancement of an existing deep architecture for
classification; yet it is effective and outperforms other hashing approaches on
several benchmarks and large datasets. Compared with state-of-the-art
approaches, SSDH achieves higher retrieval accuracy, while the classification
performance is not sacrificed.

negative:
Image retrieval can be formulated as a ranking problem where the goal is to
order database images by decreasing similarity to the query. Recent deep models
for image retrieval have outperformed traditional methods by leveraging
ranking-tailored loss functions, but important theoretical and practical
problems remain. First, rather than directly optimizing the global ranking,
they minimize an upper-bound on the essential loss, which does not necessarily
result in an optimal mean average precision (mAP). Second, these methods
require significant engineering efforts to work well, e.g. special pre-training
and hard-negative mining. In this paper we propose instead to directly optimize
the global mAP by leveraging recent advances in listwise loss formulations.
Using a histogram binning approximation, the AP can be differentiated and thus
employed to end-to-end learning. Compared to existing losses, the proposed
method considers thousands of images simultaneously at each iteration and
eliminates the need for ad hoc tricks. It also establishes a new state of the
art on many standard retrieval benchmarks. Models and evaluation scripts have
been made available at https://europe.naverlabs.com/Deep-Image-Retrieval/

negative:
Image retrieval task consists of finding similar images to a query image from
a set of gallery (database) images. Such systems are used in various
applications e.g. person re-identification (ReID) or visual product search.
Despite active development of retrieval models it still remains a challenging
task mainly due to large intra-class variance caused by changes in view angle,
lighting, background clutter or occlusion, while inter-class variance may be
relatively low. A large portion of current research focuses on creating more
robust features and modifying objective functions, usually based on Triplet
Loss. Some works experiment with using centroid/proxy representation of a class
to alleviate problems with computing speed and hard samples mining used with
Triplet Loss. However, these approaches are used for training alone and
discarded during the retrieval stage. In this paper we propose to use the mean
centroid representation both during training and retrieval. Such an aggregated
representation is more robust to outliers and assures more stable features. As
each class is represented by a single embedding - the class centroid - both
retrieval time and storage requirements are reduced significantly. Aggregating
multiple embeddings results in a significant reduction of the search space due
to lowering the number of candidate target vectors, which makes the method
especially suitable for production deployments. Comprehensive experiments
conducted on two ReID and Fashion Retrieval datasets demonstrate effectiveness
of our method, which outperforms the current state-of-the-art. We propose
centroid training and retrieval as a viable method for both Fashion Retrieval
and ReID applications.

negative:
Content-based image retrieval (CBIR) of medical images is a crucial task that
can contribute to a more reliable diagnosis if applied to big data. Recent
advances in feature extraction and classification have enormously improved CBIR
results for digital images. However, considering the increasing accessibility
of big data in medical imaging, we are still in need of reducing both memory
requirements and computational expenses of image retrieval systems. This work
proposes to exclude the features of image blocks that exhibit a low encoding
error when learned by a $n/p/n$ autoencoder ($p\!<\!n$). We examine the
histogram of autoendcoding errors of image blocks for each image class to
facilitate the decision which image regions, or roughly what percentage of an
image perhaps, shall be declared relevant for the retrieval task. This leads to
reduction of feature dimensionality and speeds up the retrieval process. To
validate the proposed scheme, we employ local binary patterns (LBP) and support
vector machines (SVM) which are both well-established approaches in CBIR
research community. As well, we use IRMA dataset with 14,410 x-ray images as
test data. The results show that the dimensionality of annotated feature
vectors can be reduced by up to 50% resulting in speedups greater than 27% at
expense of less than 1% decrease in the accuracy of retrieval when validating
the precision and recall of the top 20 hits.

negative:
Image features for retrieval-based localization must be invariant to dynamic
objects (e.g. cars) as well as seasonal and daytime changes. Such invariances
are, up to some extent, learnable with existing methods using triplet-like
losses, given a large number of diverse training images. However, due to the
high algorithmic training complexity, there exists insufficient comparison
between different loss functions on large datasets. In this paper, we train and
evaluate several localization methods on three different benchmark datasets,
including Oxford RobotCar with over one million images. This large scale
evaluation yields valuable insights into the generalizability and performance
of retrieval-based localization. Based on our findings, we develop a novel
method for learning more accurate and better generalizing localization
features. It consists of two main contributions: (i) a feature volume-based
loss function, and (ii) hard positive and pairwise negative mining. On the
challenging Oxford RobotCar night condition, our method outperforms the
well-known triplet loss by 24.4% in localization accuracy within 5m.

negative:
The research on hashing techniques for visual data is gaining increased
attention in recent years due to the need for compact representations
supporting efficient search/retrieval in large-scale databases such as online
images. Among many possibilities, Mean Average Precision(mAP) has emerged as
the dominant performance metric for hashing-based retrieval. One glaring
shortcoming of mAP is its inability in balancing retrieval accuracy and
utilization of hash codes: pushing a system to attain higher mAP will
inevitably lead to poorer utilization of the hash codes. Poor utilization of
the hash codes hinders good retrieval because of increased collision of samples
in the hash space. This means that a model giving a higher mAP values does not
necessarily do a better job in retrieval. In this paper, we introduce a new
metric named Mean Local Group Average Precision (mLGAP) for better evaluation
of the performance of hashing-based retrieval. The new metric provides a
retrieval performance measure that also reconciles the utilization of hash
codes, leading to a more practically meaningful performance metric than
conventional ones like mAP. To this end, we start by mathematical analysis of
the deficiencies of mAP for hashing-based retrieval. We then propose mLGAP and
show why it is more appropriate for hashing-based retrieval. Experiments on
image retrieval are used to demonstrate the effectiveness of the proposed
metric.

negative:
Text-to-image retrieval is an essential task in cross-modal information
retrieval, i.e., retrieving relevant images from a large and unlabelled dataset
given textual queries. In this paper, we propose VisualSparta, a novel
(Visual-text Sparse Transformer Matching) model that shows significant
improvement in terms of both accuracy and efficiency. VisualSparta is capable
of outperforming previous state-of-the-art scalable methods in MSCOCO and
Flickr30K. We also show that it achieves substantial retrieving speed
advantages, i.e., for a 1 million image index, VisualSparta using CPU gets
~391X speedup compared to CPU vector search and ~5.4X speedup compared to
vector search with GPU acceleration. Experiments show that this speed advantage
even gets bigger for larger datasets because VisualSparta can be efficiently
implemented as an inverted index. To the best of our knowledge, VisualSparta is
the first transformer-based text-to-image retrieval model that can achieve
real-time searching for large-scale datasets, with significant accuracy
improvement compared to previous state-of-the-art methods.

negative:
Most existing works of adversarial samples focus on attacking image
recognition models, while little attention is paid to the image retrieval task.
In this paper, we identify two inherent challenges in applying prevailing image
recognition attack methods to image retrieval. First, image retrieval demands
discriminative visual features, which is significantly different from the
one-hot class prediction in image recognition. Second, due to the disjoint and
potentially unrelated classes between the training and test set in image
retrieval, predicting the query category from predefined training classes is
not accurate and leads to a sub-optimal adversarial gradient. To address these
limitations, we propose a new white-box attack approach, Opposite-Direction
Feature Attack (ODFA), to generate adversarial queries. Opposite-Direction
Feature Attack (ODFA) effectively exploits feature-level adversarial gradients
and takes advantage of feature distance in the representation space. To our
knowledge, we are among the early attempts to design an attack method
specifically for image retrieval. When we deploy an attacked image as the
query, the true matches are prone to receive low ranks. We demonstrate through
extensive experiments that (1) only crafting adversarial queries is sufficient
to fool the state-of-the-art retrieval systems; (2) the proposed attack method,
ODFA, leads to a higher attack success rate than classification attack methods,
validating the necessity of leveraging characteristics of image retrieval; (3)
the adversarial queries generated by our method have good transferability to
other retrieval models without accessing their parameters, i.e.,the black-box
setting.

negative:
Hash coding has been widely used in approximate nearest neighbor search for
large-scale image retrieval. Given semantic annotations such as class labels
and pairwise similarities of the training data, hashing methods can learn and
generate effective and compact binary codes. While some newly introduced images
may contain undefined semantic labels, which we call unseen images, zeor-shot
hashing techniques have been studied. However, existing zeor-shot hashing
methods focus on the retrieval of single-label images, and cannot handle
multi-label images. In this paper, for the first time, a novel transductive
zero-shot hashing method is proposed for multi-label unseen image retrieval. In
order to predict the labels of the unseen/target data, a visual-semantic bridge
is built via instance-concept coherence ranking on the seen/source data. Then,
pairwise similarity loss and focal quantization loss are constructed for
training a hashing model using both the seen/source and unseen/target data.
Extensive evaluations on three popular multi-label datasets demonstrate that,
the proposed hashing method achieves significantly better results than the
competing methods.

negative:
The analysis of large collections of image data is still a challenging
problem due to the difficulty of capturing the true concepts in visual data.
The similarity between images could be computed using different and possibly
multimodal features such as color or edge information or even text labels. This
motivates the design of image analysis solutions that are able to effectively
integrate the multi-view information provided by different feature sets. We
therefore propose a new image retrieval solution that is able to sort images
through a random walk on a multi-layer graph, where each layer corresponds to a
different type of information about the image data. We study in depth the
design of the image graph and propose in particular an effective method to
select the edge weights for the multi-layer graph, such that the image ranking
scores are optimised. We then provide extensive experiments in different
real-world photo collections, which confirm the high performance of our new
image retrieval algorithm that generally surpasses state-of-the-art solutions
due to a more meaningful image similarity computation.

negative:
This paper describes two approaches for content-based image retrieval and
pattern spotting in document images using deep learning. The first approach
uses a pre-trained CNN model to cope with the lack of training data, which is
fine-tuned to achieve a compact yet discriminant representation of queries and
image candidates. The second approach uses a Siamese Convolution Neural Network
trained on a previously prepared subset of image pairs from the ImageNet
dataset to provide the similarity-based feature maps. In both methods, the
learned representation scheme considers feature maps of different sizes which
are evaluated in terms of retrieval performance. A robust experimental protocol
using two public datasets (Tobacoo-800 and DocExplore) has shown that the
proposed methods compare favorably against state-of-the-art document image
retrieval and pattern spotting methods.

negative:
This paper explores the task of interactive image retrieval using natural
language queries, where a user progressively provides input queries to refine a
set of retrieval results. Moreover, our work explores this problem in the
context of complex image scenes containing multiple objects. We propose
Drill-down, an effective framework for encoding multiple queries with an
efficient compact state representation that significantly extends current
methods for single-round image retrieval. We show that using multiple rounds of
natural language queries as input can be surprisingly effective to find
arbitrarily specific images of complex scenes. Furthermore, we find that
existing image datasets with textual captions can provide a surprisingly
effective form of weak supervision for this task. We compare our method with
existing sequential encoding and embedding networks, demonstrating superior
performance on two proposed benchmarks: automatic image retrieval on a
simulated scenario that uses region captions as queries, and interactive image
retrieval using real queries from human evaluators.

negative:
Supervised hashing aims to map the original features to compact binary codes
that are able to preserve label based similarity in the Hamming space.
Non-linear hash functions have demonstrated the advantage over linear ones due
to their powerful generalization capability. In the literature, kernel
functions are typically used to achieve non-linearity in hashing, which achieve
encouraging retrieval performance at the price of slow evaluation and training
time. Here we propose to use boosted decision trees for achieving non-linearity
in hashing, which are fast to train and evaluate, hence more suitable for
hashing with high dimensional data. In our approach, we first propose
sub-modular formulations for the hashing binary code inference problem and an
efficient GraphCut based block search method for solving large-scale inference.
Then we learn hash functions by training boosted decision trees to fit the
binary codes. Experiments demonstrate that our proposed method significantly
outperforms most state-of-the-art methods in retrieval precision and training
time. Especially for high-dimensional data, our method is orders of magnitude
faster than many methods in terms of training time.

negative:
In this paper, we aim to learn a mapping (or embedding) from images to a
compact binary space in which Hamming distances correspond to a ranking measure
for the image retrieval task.
  We make use of a triplet loss because this has been shown to be most
effective for ranking problems.
  However, training in previous works can be prohibitively expensive due to the
fact that optimization is directly performed on the triplet space, where the
number of possible triplets for training is cubic in the number of training
examples.
  To address this issue, we propose to formulate high-order binary codes
learning as a multi-label classification problem by explicitly separating
learning into two interleaved stages.
  To solve the first stage, we design a large-scale high-order binary codes
inference algorithm to reduce the high-order objective to a standard binary
quadratic problem such that graph cuts can be used to efficiently infer the
binary code which serve as the label of each training datum.
  In the second stage we propose to map the original image to compact binary
codes via carefully designed deep convolutional neural networks (CNNs) and the
hashing function fitting can be solved by training binary CNN classifiers.
  An incremental/interleaved optimization strategy is proffered to ensure that
these two steps are interactive with each other during training for better
accuracy.
  We conduct experiments on several benchmark datasets, which demonstrate both
improved training time (by as much as two orders of magnitude) as well as
producing state-of-the-art hashing for various retrieval tasks.

negative:
Image retrieval utilizes image descriptors to retrieve the most similar
images to a given query image. Convolutional neural network (CNN) is becoming
the dominant approach to extract image descriptors for image retrieval. For
low-power hardware implementation of image retrieval, the drawback of CNN-based
feature descriptor is that it requires hundreds of megabytes of storage. To
address this problem, this paper applies deep model quantization and
compression to CNN in ASIC chip for image retrieval. It is demonstrated that
the CNN-based features descriptor can be extracted using as few as 2-bit
weights quantization to deliver a similar performance as floating-point model
for image retrieval. In addition, to implement CNN in ASIC, especially for
large scale images, the limited buffer size of chips should be considered. To
retrieve large scale images, we propose an improved pooling strategy, region
nested invariance pooling (RNIP), which uses cropped sub-images for CNN.
Testing results on chip show that integrating RNIP with the proposed 2-bit CNN
model compression approach is capable of retrieving large scale images.

negative:
A laser scanner can easily acquire the geometric data of physical
environments in the form of a point cloud. Recognizing objects from a point
cloud is often required for industrial 3D reconstruction, which should include
not only geometry information but also semantic information. However,
recognition process is often a bottleneck in 3D reconstruction because it
requires expertise on domain knowledge and intensive labor. To address this
problem, various methods have been developed to recognize objects by retrieving
the corresponding model in the database from an input geometry query. In recent
years, the technique of converting geometric data into an image and applying
view-based 3D shape retrieval has demonstrated high accuracy. Depth image which
encodes depth value as intensity of pixel is frequently used for view-based 3D
shape retrieval. However, geometric data collected from objects is often
incomplete due to the occlusions and the limit of line of sight. Image
generated by occluded point clouds lowers the performance of view-based 3D
object retrieval due to loss of information. In this paper, we propose a method
of viewpoint and image resolution estimation method for view-based 3D shape
retrieval from point cloud query. Automatic selection of viewpoint and image
resolution by calculating the data acquisition rate and density from the
sampled viewpoints and image resolutions are proposed. The retrieval
performance from the images generated by the proposed method is experimented
and compared for various dataset. Additionally, view-based 3D shape retrieval
performance with deep convolutional neural network has been experimented with
the proposed method.

negative:
The objective of this work is to learn a compact embedding of a set of
descriptors that is suitable for efficient retrieval and ranking, whilst
maintaining discriminability of the individual descriptors. We focus on a
specific example of this general problem -- that of retrieving images
containing multiple faces from a large scale dataset of images. Here the set
consists of the face descriptors in each image, and given a query for multiple
identities, the goal is then to retrieve, in order, images which contain all
the identities, all but one, \etc
  To this end, we make the following contributions: first, we propose a CNN
architecture -- {\em SetNet} -- to achieve the objective: it learns face
descriptors and their aggregation over a set to produce a compact fixed length
descriptor designed for set retrieval, and the score of an image is a count of
the number of identities that match the query; second, we show that this
compact descriptor has minimal loss of discriminability up to two faces per
image, and degrades slowly after that -- far exceeding a number of baselines;
third, we explore the speed vs.\ retrieval quality trade-off for set retrieval
using this compact descriptor; and, finally, we collect and annotate a large
dataset of images containing various number of celebrities, which we use for
evaluation and is publicly released.

negative:
This paper presents a novel approach for image retrieval and pattern spotting
in document image collections. The manual feature engineering is avoided by
learning a similarity-based representation using a Siamese Neural Network
trained on a previously prepared subset of image pairs from the ImageNet
dataset. The learned representation is used to provide the similarity-based
feature maps used to find relevant image candidates in the data collection
given an image query. A robust experimental protocol based on the public
Tobacco800 document image collection shows that the proposed method compares
favorably against state-of-the-art document image retrieval methods, reaching
0.94 and 0.83 of mean average precision (mAP) for retrieval and pattern
spotting (IoU=0.7), respectively. Besides, we have evaluated the proposed
method considering feature maps of different sizes, showing the impact of
reducing the number of features in the retrieval performance and
time-consuming.

negative:
In this work, we evaluate contrastive models for the task of image retrieval.
We hypothesise that models that are learned to encode semantic similarity among
instances via discriminative learning should perform well on the task of image
retrieval, where relevancy is defined in terms of instances of the same object.
Through our extensive evaluation, we find that representations from models
trained using contrastive methods perform on-par with (and outperforms) a
pre-trained supervised baseline trained on the ImageNet labels in retrieval
tasks under various configurations. This is remarkable given that the
contrastive models require no explicit supervision. Thus, we conclude that
these models can be used to bootstrap base models to build more robust image
retrieval engines.

negative:
Face image retrieval, which searches for images of the same identity from the
query input face image, is drawing more attention as the size of the image
database increases rapidly. In order to conduct fast and accurate retrieval, a
compact hash code-based methods have been proposed, and recently, deep face
image hashing methods with supervised classification training have shown
outstanding performance. However, classification-based scheme has a
disadvantage in that it cannot reveal complex similarities between face images
into the hash code learning. In this paper, we attempt to improve the face
image retrieval quality by proposing a Similarity Guided Hashing (SGH) method,
which gently considers self and pairwise-similarity simultaneously. SGH employs
various data augmentations designed to explore elaborate similarities between
face images, solving both intra and inter identity-wise difficulties. Extensive
experimental results on the protocols with existing benchmarks and an
additionally proposed large scale higher resolution face image dataset
demonstrate that our SGH delivers state-of-the-art retrieval performance.

negative:
Image-text retrieval of natural scenes has been a popular research topic.
Since image and text are heterogeneous cross-modal data, one of the key
challenges is how to learn comprehensive yet unified representations to express
the multi-modal data. A natural scene image mainly involves two kinds of visual
concepts, objects and their relationships, which are equally essential to
image-text retrieval. Therefore, a good representation should account for both
of them. In the light of recent success of scene graph in many CV and NLP tasks
for describing complex natural scenes, we propose to represent image and text
with two kinds of scene graphs: visual scene graph (VSG) and textual scene
graph (TSG), each of which is exploited to jointly characterize objects and
relationships in the corresponding modality. The image-text retrieval task is
then naturally formulated as cross-modal scene graph matching. Specifically, we
design two particular scene graph encoders in our model for VSG and TSG, which
can refine the representation of each node on the graph by aggregating
neighborhood information. As a result, both object-level and relationship-level
cross-modal features can be obtained, which favorably enables us to evaluate
the similarity of image and text in the two levels in a more plausible way. We
achieve state-of-the-art results on Flickr30k and MSCOCO, which verifies the
advantages of our graph matching based approach for image-text retrieval.

negative:
This paper considers matching images of low-light scenes, aiming to widen the
frontier of SfM and visual SLAM applications. Recent image sensors can record
the brightness of scenes with more than eight-bit precision, available in their
RAW-format image. We are interested in making full use of such high-precision
information to match extremely low-light scene images that conventional methods
cannot handle. For extreme low-light scenes, even if some of their brightness
information exists in the RAW format images' low bits, the standard raw image
processing on cameras fails to utilize them properly. As was recently shown by
Chen et al., CNNs can learn to produce images with a natural appearance from
such RAW-format images. To consider if and how well we can utilize such
information stored in RAW-format images for image matching, we have created a
new dataset named MID (matching in the dark). Using it, we experimentally
evaluated combinations of eight image-enhancing methods and eleven image
matching methods consisting of classical/neural local descriptors and
classical/neural initial point-matching methods. The results show the advantage
of using the RAW-format images and the strengths and weaknesses of the above
component methods. They also imply there is room for further research.

negative:
We propose an unsupervised hashing method which aims to produce binary codes
that preserve the ranking induced by a real-valued representation. Such compact
hash codes enable the complete elimination of real-valued feature storage and
allow for significant reduction of the computation complexity and storage cost
of large-scale image retrieval applications. Specifically, we learn a neural
network-based model, which transforms the input representation into a binary
representation. We formalize the training objective of the network in an
intuitive and effective way, considering each training sample as a query and
aiming to obtain the same retrieval results using the produced hash codes as
those obtained with the original features. This training formulation directly
optimizes the hashing model for the target usage of the hash codes it produces.
We further explore the addition of a decoder trained to obtain an approximated
reconstruction of the original features. At test time, we retrieved the most
promising database samples with an efficient graph-based search procedure using
only our hash codes and perform re-ranking using the reconstructed features,
thus without needing to access the original features at all. Experiments
conducted on multiple publicly available large-scale datasets show that our
method consistently outperforms all compared state-of-the-art unsupervised
hashing methods and that the reconstruction procedure can effectively boost the
search accuracy with a minimal constant additional cost.

negative:
Image retrieval has been a top topic in the field of both computer vision and
machine learning for a long time. Content based image retrieval, which tries to
retrieve images from a database visually similar to a query image, has
attracted much attention. Two most important issues of image retrieval are the
representation and ranking of the images. Recently, bag-of-words based method
has shown its power as a representation method. Moreover, nonnegative matrix
factorization is also a popular way to represent the data samples. In addition,
contextual similarity learning has also been studied and proven to be an
effective method for the ranking problem. However, these technologies have
never been used together. In this paper, we developed an effective image
retrieval system by representing each image using the bag-of-words method as
histograms, and then apply the nonnegative matrix factorization to factorize
the histograms, and finally learn the ranking score using the contextual
similarity learning method. The proposed novel system is evaluated on a large
scale image database and the effectiveness is shown.

negative:
Learning the similarity between remote sensing (RS) images forms the
foundation for content-based RS image retrieval (CBIR). Recently, deep metric
learning approaches that map the semantic similarity of images into an
embedding (metric) space have been found very popular in RS. A common approach
for learning the metric space relies on the selection of triplets of similar
(positive) and dissimilar (negative) images to a reference image called as an
anchor. Choosing triplets is a difficult task particularly for multi-label RS
CBIR, where each training image is annotated by multiple class labels. To
address this problem, in this paper we propose a novel triplet sampling method
in the framework of deep neural networks (DNNs) defined for multi-label RS CBIR
problems. The proposed method selects a small set of the most representative
and informative triplets based on two main steps. In the first step, a set of
anchors that are diverse to each other in the embedding space is selected from
the current mini-batch using an iterative algorithm. In the second step,
different sets of positive and negative images are chosen for each anchor by
evaluating the relevancy, hardness and diversity of the images among each other
based on a novel strategy. Experimental results obtained on two multi-label
benchmark archives show that the selection of the most informative and
representative triplets in the context of DNNs results in: i) reducing the
computational complexity of the training phase of the DNNs without any
significant loss on the performance; and ii) an increase in learning speed
since informative triplets allow fast convergence. The code of the proposed
method is publicly available at
https://git.tu-berlin.de/rsim/image-retrieval-from-triplets.

negative:
We investigate the gains in precision and speed, that can be obtained by
using Convolutional Networks (ConvNets) for on-the-fly retrieval - where
classifiers are learnt at run time for a textual query from downloaded images,
and used to rank large image or video datasets.
  We make three contributions: (i) we present an evaluation of state-of-the-art
image representations for object category retrieval over standard benchmark
datasets containing 1M+ images; (ii) we show that ConvNets can be used to
obtain features which are incredibly performant, and yet much lower dimensional
than previous state-of-the-art image representations, and that their
dimensionality can be reduced further without loss in performance by
compression using product quantization or binarization. Consequently, features
with the state-of-the-art performance on large-scale datasets of millions of
images can fit in the memory of even a commodity GPU card; (iii) we show that
an SVM classifier can be learnt within a ConvNet framework on a GPU in parallel
with downloading the new training images, allowing for a continuous refinement
of the model as more images become available, and simultaneous training and
ranking. The outcome is an on-the-fly system that significantly outperforms its
predecessors in terms of: precision of retrieval, memory requirements, and
speed, facilitating accurate on-the-fly learning and ranking in under a second
on a single GPU.

negative:
Unsupervised binary representation allows fast data retrieval without any
annotations, enabling practical application like fast person re-identification
and multimedia retrieval. It is argued that conflicts in binary space are one
of the major barriers to high-performance unsupervised hashing as current
methods failed to capture the precise code conflicts in the full domain. A
novel relaxation method called Shuffle and Learn is proposed to tackle code
conflicts in the unsupervised hash. Approximated derivatives for joint
probability and the gradients for the binary layer are introduced to bridge the
update from the hash to the input. Proof on $\epsilon$-Convergence of joint
probability with approximated derivatives is provided to guarantee the
preciseness on update applied on the mutual information. The proposed algorithm
is carried out with iterative global updates to minimize mutual information,
diverging the code before regular unsupervised optimization. Experiments
suggest that the proposed method can relax the code optimization from local
optimum and help to generate binary representations that are more
discriminative and informative without any annotations. Performance benchmarks
on image retrieval with the unsupervised binary code are conducted on three
open datasets, and the model achieves state-of-the-art accuracy on image
retrieval task for all those datasets. Datasets and reproducible code are
provided.

negative:
Document Image Analysis, like any Digital Image Analysis requires
identification and extraction of proper features, which are generally extracted
from uncompressed images, though in reality images are made available in
compressed form for the reasons such as transmission and storage efficiency.
However, this implies that the compressed image should be decompressed, which
indents additional computing resources. This limitation induces the motivation
to research in extracting features directly from the compressed image. In this
research, we propose to extract essential features such as projection profile,
run-histogram and entropy for text document analysis directly from run-length
compressed text-documents. The experimentation illustrates that features are
extracted directly from the compressed image without going through the stage of
decompression, because of which the computing time is reduced. The feature
values so extracted are exactly identical to those extracted from uncompressed
images.

negative:
Convolutional Neural Network (CNN) is a very powerful approach to extract
discriminative local descriptors for effective image search. Recent work adopts
fine-tuned strategies to further improve the discriminative power of the
descriptors. Taking a different approach, in this paper, we propose a novel
framework to achieve competitive retrieval performance. Firstly, we propose
various masking schemes, namely SIFT-mask, SUM-mask, and MAX-mask, to select a
representative subset of local convolutional features and remove a large number
of redundant features. We demonstrate that this can effectively address the
burstiness issue and improve retrieval accuracy. Secondly, we propose to employ
recent embedding and aggregating methods to further enhance feature
discriminability. Extensive experiments demonstrate that our proposed framework
achieves state-of-the-art retrieval accuracy.

negative:
Recently it has become essential to search for and retrieve high-resolution
and efficient images easily due to swift development of digital images, many
present annotation algorithms facing a big challenge which is the variance for
represent the image where high level represent image semantic and low level
illustrate the features, this issue is known as semantic gab. This work has
been used MPEG-7 standard to extract the features from the images, where the
color feature was extracted by using Scalable Color Descriptor (SCD) and Color
Layout Descriptor (CLD), whereas the texture feature was extracted by employing
Edge Histogram Descriptor (EHD), the CLD produced high dimensionality feature
vector therefore it is reduced by Principal Component Analysis (PCA). The
features that have extracted by these three descriptors could be passing to the
classifiers (Naive Bayes and Decision Tree) for training. Finally, they
annotated the query image. In this study TUDarmstadt image bank had been used.
The results of tests and comparative performance evaluation indicated better
precision and executing time of Naive Bayes classification in comparison with
Decision Tree classification.

negative:
With the prolification of multimodal interaction in various domains, recently
there has been much interest in text based image retrieval in the computer
vision community. However most of the state of the art techniques model this
problem in a purely neural way, which makes it difficult to incorporate
pragmatic strategies in searching a large scale catalog especially when the
search requirements are insufficient and the model needs to resort to an
interactive retrieval process through multiple iterations of
question-answering. Motivated by this, we propose a neural-symbolic approach
for a one-shot retrieval of images from a large scale catalog, given the
caption description. To facilitate this, we represent the catalog and caption
as scene-graphs and model the retrieval task as a learnable graph matching
problem, trained end-to-end with a REINFORCE algorithm. Further, we briefly
describe an extension of this pipeline to an iterative retrieval framework,
based on interactive questioning and answering.

negative:
To retrieve images based on their content is one of the most studied topics
in the field of computer vision. Nowadays, this problem can be addressed using
modern techniques such as feature extraction using machine learning, but over
the years different classical methods have been developed. In this paper, we
implement a query by example retrieval system for finding paintings in a museum
image collection using classic computer vision techniques. Specifically, we
study the performance of the color, texture, text and feature descriptors in
datasets with different perturbations in the images: noise, overlapping text
boxes, color corruption and rotation. We evaluate each of the cases using the
Mean Average Precision (MAP) metric, and we obtain results that vary between
0.5 and 1.0 depending on the problem conditions.

negative:
Visual localization algorithms have achieved significant improvements in
performance thanks to recent advances in camera technology and vision-based
techniques. However, there remains one critical caveat: all current approaches
that are based on image retrieval currently scale at best linearly with the
size of the environment with respect to both storage, and consequentially in
most approaches, query time. This limitation severely curtails the capability
of autonomous systems in a wide range of compute, power, storage, size, weight
or cost constrained applications such as drones. In this work, we present a
novel binary tree encoding approach for visual localization which can serve as
an alternative for existing quantization and indexing techniques. The proposed
tree structure allows us to derive a compressed training scheme that achieves
sub-linearity in both required storage and inference time. The encoding memory
can be easily configured to satisfy different storage constraints. Moreover,
our approach is amenable to an optional sequence filtering mechanism to further
improve the localization results, while maintaining the same amount of storage.
Our system is entirely agnostic to the front-end descriptors, allowing it to be
used on top of recent state-of-the-art image representations. Experimental
results show that the proposed method significantly outperforms
state-of-the-art approaches under limited storage constraints.

negative:
With the development of Information technology and communication, a large
part of the databases is dedicated to images and videos. Thus retrieving images
related to a query image from a large database has become an important area of
research in computer vision. Until now, there are various methods of image
retrieval that try to define image contents by texture, color or shape
properties. In this paper, a method is presented for image retrieval based on a
combination of local texture information derived from two different texture
descriptors. First, the color channels of the input image are separated. The
texture information is extracted using two descriptors such as evaluated local
binary patterns and predefined pattern units. After extracting the features,
the similarity matching is done based on distance criteria. The performance of
the proposed method is evaluated in terms of precision and recall on the
Simplicity database. The comparative results showed that the proposed approach
offers higher precision rate than many known methods.

negative:
In this paper, we study the task of image retrieval, where the input query is
specified in the form of an image plus some text that describes desired
modifications to the input image. For example, we may present an image of the
Eiffel tower, and ask the system to find images which are visually similar but
are modified in small ways, such as being taken at nighttime instead of during
the day. To tackle this task, we learn a similarity metric between a target
image and a source image plus source text, an embedding and composing function
such that target image feature is close to the source image plus text
composition feature. We propose a new way to combine image and text using such
function that is designed for the retrieval task. We show this outperforms
existing approaches on 3 different datasets, namely Fashion-200k, MIT-States
and a new synthetic dataset we create based on CLEVR. We also show that our
approach can be used to classify input queries, in addition to image retrieval.

negative:
Image to image translation aims to learn a mapping that transforms an image
from one visual domain to another. Recent works assume that images descriptors
can be disentangled into a domain-invariant content representation and a
domain-specific style representation. Thus, translation models seek to preserve
the content of source images while changing the style to a target visual
domain. However, synthesizing new images is extremely challenging especially in
multi-domain translations, as the network has to compose content and style to
generate reliable and diverse images in multiple domains. In this paper we
propose the use of an image retrieval system to assist the image-to-image
translation task. First, we train an image-to-image translation model to map
images to multiple domains. Then, we train an image retrieval model using real
and generated images to find images similar to a query one in content but in a
different domain. Finally, we exploit the image retrieval system to fine-tune
the image-to-image translation model and generate higher quality images. Our
experiments show the effectiveness of the proposed solution and highlight the
contribution of the retrieval network, which can benefit from additional
unlabeled data and help image-to-image translation models in the presence of
scarce data.

negative:
The main focus of this paper is document image classification and retrieval,
where we analyze and compare different parameters for the RunLeght Histogram
(RL) and Fisher Vector (FV) based image representations. We do an exhaustive
experimental study using different document image datasets, including the MARG
benchmarks, two datasets built on customer data and the images from the Patent
Image Classification task of the Clef-IP 2011. The aim of the study is to give
guidelines on how to best choose the parameters such that the same features
perform well on different tasks. As an example of such need, we describe the
Image-based Patent Retrieval task's of Clef-IP 2011, where we used the same
image representation to predict the image type and retrieve relevant patents.

negative:
In recent years, text-guided image manipulation has gained increasing
attention in the multimedia and computer vision community. The input to
conditional image generation has evolved from image-only to multimodality. In
this paper, we study a setting that allows users to edit an image with multiple
objects using complex text instructions to add, remove, or change the objects.
The inputs of the task are multimodal including (1) a reference image and (2)
an instruction in natural language that describes desired modifications to the
image. We propose a GAN-based method to tackle this problem. The key idea is to
treat text as neural operators to locally modify the image feature. We show
that the proposed model performs favorably against recent strong baselines on
three public datasets. Specifically, it generates images of greater fidelity
and semantic relevance, and when used as a image query, leads to better
retrieval performance.

negative:
Universal adversarial perturbations (UAPs), a.k.a. input-agnostic
perturbations, has been proved to exist and be able to fool cutting-edge deep
learning models on most of the data samples. Existing UAP methods mainly focus
on attacking image classification models. Nevertheless, little attention has
been paid to attacking image retrieval systems. In this paper, we make the
first attempt in attacking image retrieval systems. Concretely, image retrieval
attack is to make the retrieval system return irrelevant images to the query at
the top ranking list. It plays an important role to corrupt the neighbourhood
relationships among features in image retrieval attack. To this end, we propose
a novel method to generate retrieval-against UAP to break the neighbourhood
relationships of image features via degrading the corresponding ranking metric.
To expand the attack method to scenarios with varying input sizes or
untouchable network parameters, a multi-scale random resizing scheme and a
ranking distillation strategy are proposed. We evaluate the proposed method on
four widely-used image retrieval datasets, and report a significant performance
drop in terms of different metrics, such as mAP and mP@10. Finally, we test our
attack methods on the real-world visual search engine, i.e., Google Images,
which demonstrates the practical potentials of our methods.

negative:
Important information that relates to a specific topic in a document is often
organized in tabular format to assist readers with information retrieval and
comparison, which may be difficult to provide in natural language. However,
tabular data in unstructured digital documents, e.g., Portable Document Format
(PDF) and images, are difficult to parse into structured machine-readable
format, due to complexity and diversity in their structure and style. To
facilitate image-based table recognition with deep learning, we develop the
largest publicly available table recognition dataset PubTabNet
(https://github.com/ibm-aur-nlp/PubTabNet), containing 568k table images with
corresponding structured HTML representation. PubTabNet is automatically
generated by matching the XML and PDF representations of the scientific
articles in PubMed Central Open Access Subset (PMCOA). We also propose a novel
attention-based encoder-dual-decoder (EDD) architecture that converts images of
tables into HTML code. The model has a structure decoder which reconstructs the
table structure and helps the cell decoder to recognize cell content. In
addition, we propose a new Tree-Edit-Distance-based Similarity (TEDS) metric
for table recognition, which more appropriately captures multi-hop cell
misalignment and OCR errors than the pre-established metric. The experiments
demonstrate that the EDD model can accurately recognize complex tables solely
relying on the image representation, outperforming the state-of-the-art by 9.7%
absolute TEDS score.

negative:
We introduce the first work to tackle the image retrieval problem as a
continuous operation. While the proposed approaches in the literature can be
roughly categorized into two main groups: category- and instance-based
retrieval, in this work we show that the retrieval task is much richer and more
complex. Image similarity goes beyond this discrete vantage point and spans a
continuous spectrum among the classical operating points of category and
instance similarity. However, current retrieval models are static and incapable
of exploring this rich structure of the retrieval space since they are trained
and evaluated with a single operating point as a target objective. Hence, we
introduce a novel retrieval model that for a given query is capable of
producing a dynamic embedding that can target an arbitrary point along the
continuous retrieval spectrum. Our model disentangles the visual signal of a
query image into its basic components of categorical and attribute information.
Furthermore, using a continuous control parameter our model learns to
reconstruct a dynamic embedding of the query by mixing these components with
different proportions to target a specific point along the retrieval simplex.
We demonstrate our idea in a comprehensive evaluation of the proposed model and
highlight the advantages of our approach against a set of well-established
discrete retrieval models.

negative:
This paper addresses the problem of learning word image representations:
given the cropped image of a word, we are interested in finding a descriptive,
robust, and compact fixed-length representation. Machine learning techniques
can then be supplied with these representations to produce models useful for
word retrieval or recognition tasks. Although many works have focused on the
machine learning aspect once a global representation has been produced, little
work has been devoted to the construction of those base image representations:
most works use standard coding and aggregation techniques directly on top of
standard computer vision features such as SIFT or HOG.
  We propose to learn local mid-level features suitable for building word image
representations. These features are learnt by leveraging character bounding box
annotations on a small set of training images. However, contrary to other
approaches that use character bounding box information, our approach does not
rely on detecting the individual characters explicitly at testing time. Our
local mid-level features can then be aggregated to produce a global word image
signature. When pairing these features with the recent word attributes
framework of Almaz\'an et al., we obtain results comparable with or better than
the state-of-the-art on matching and recognition tasks using global descriptors
of only 96 dimensions.

negative:
Content-based image retrieval (CBIR) is a task of retrieving images from
their contents. Since retrieval process is a time-consuming task in large image
databases, acceleration methods can be very useful. This paper presents a novel
method to speed up CBIR systems. In the proposed method, first Zernike moments
are extracted from query image and an interval is calculated for that query.
Images in database which are out of the interval are ignored in retrieval
process. Therefore, a database reduction occurs before retrieval which leads to
speed up. It is shown that in reduced database, relevant images to query image
are preserved and irrelevant images are throwed away. Therefore, the proposed
method speed up retrieval process and preserve CBIR accuracy, simultaneously.

negative:
Content-based image retrieval has seen astonishing progress over the past
decade, especially for the task of retrieving images of the same object that is
depicted in the query image. This scenario is called instance or object
retrieval and requires matching fine-grained visual patterns between images.
Semantics, however, do not play a crucial role. This brings rise to the
question: Do the recent advances in instance retrieval transfer to more generic
image retrieval scenarios? To answer this question, we first provide a brief
overview of the most relevant milestones of instance retrieval. We then apply
them to a semantic image retrieval task and find that they perform inferior to
much less sophisticated and more generic methods in a setting that requires
image understanding. Following this, we review existing approaches to closing
this so-called semantic gap by integrating prior world knowledge. We conclude
that the key problem for the further advancement of semantic image retrieval
lies in the lack of a standardized task definition and an appropriate benchmark
dataset.

negative:
Hashing produces compact representations for documents, to perform tasks like
classification or retrieval based on these short codes. When hashing is
supervised, the codes are trained using labels on the training data. This paper
first shows that the evaluation protocols used in the literature for supervised
hashing are not satisfactory: we show that a trivial solution that encodes the
output of a classifier significantly outperforms existing supervised or
semi-supervised methods, while using much shorter codes. We then propose two
alternative protocols for supervised hashing: one based on retrieval on a
disjoint set of classes, and another based on transfer learning to new classes.
We provide two baseline methods for image-related tasks to assess the
performance of (semi-)supervised hashing: without coding and with unsupervised
codes. These baselines give a lower- and upper-bound on the performance of a
supervised hashing scheme.

negative:
Suppose we have a black-box function (e.g., deep neural network) that takes
an image as input and outputs a value that indicates preference. How can we
retrieve optimal images with respect to this function from an external database
on the Internet? Standard retrieval problems in the literature (e.g., item
recommendations) assume that an algorithm has full access to the set of items.
In other words, such algorithms are designed for service providers. In this
paper, we consider the retrieval problem under different assumptions.
Specifically, we consider how users with limited access to an image database
can retrieve images using their own black-box functions. This formulation
enables a flexible and finer-grained image search defined by each user. We
assume the user can access the database through a search query with tight API
limits. Therefore, a user needs to efficiently retrieve optimal images in terms
of the number of queries. We propose an efficient retrieval algorithm Tiara for
this problem. In the experiments, we confirm that our proposed method performs
better than several baselines under various settings.

negative:
This paper approaches the image retrieval system on the base of visual
features local region RBIR (region-based image retrieval). First of all, the
paper presents a method for extracting the interest points based on
Harris-Laplace to create the feature region of the image. Next, in order to
reduce the storage space and speed up query image, the paper builds the binary
signature structure to describe the visual content of image. Based on the
image's binary signature, the paper builds the SG (signature graph) to classify
and store image's binary signatures. Since then, the paper builds the image
retrieval algorithm on SG through the similar measure EMD (earth mover's
distance) between the image's binary signatures. Last but not least, the paper
gives an image retrieval model RBIR, experiments and assesses the image
retrieval method on Corel image database over 10,000 images.

negative:
In this work, a region-based Deep Convolutional Neural Network framework is
proposed for document structure learning. The contribution of this work
involves efficient training of region based classifiers and effective
ensembling for document image classification. A primary level of `inter-domain'
transfer learning is used by exporting weights from a pre-trained VGG16
architecture on the ImageNet dataset to train a document classifier on whole
document images. Exploiting the nature of region based influence modelling, a
secondary level of `intra-domain' transfer learning is used for rapid training
of deep learning models for image segments. Finally, stacked generalization
based ensembling is utilized for combining the predictions of the base deep
neural network models. The proposed method achieves state-of-the-art accuracy
of 92.2% on the popular RVL-CDIP document image dataset, exceeding benchmarks
set by existing algorithms.

negative:
With the advances in both stable interest region detectors and robust and
distinctive descriptors, local feature-based image or object retrieval has
become a popular research topic. %All of the local feature-based image
retrieval system involves two important processes: local feature extraction and
image representation. The other key technology for image retrieval systems is
image representation such as the bag-of-visual words (BoVW), Fisher vector, or
Vector of Locally Aggregated Descriptors (VLAD) framework. In this paper, we
review local features and image representations for image retrieval. Because
many and many methods are proposed in this area, these methods are grouped into
several classes and summarized. In addition, recent deep learning-based
approaches for image retrieval are briefly reviewed.

negative:
In this paper, we consider a different data format for images: vector
graphics. In contrast to raster graphics which are widely used in image
recognition, vector graphics can be scaled up or down into any resolution
without aliasing or information loss, due to the analytic representation of the
primitives in the document. Furthermore, vector graphics are able to give extra
structural information on how low-level elements group together to form high
level shapes or structures. These merits of graphic vectors have not been fully
leveraged in existing methods. To explore this data format, we target on the
fundamental recognition tasks: object localization and classification. We
propose an efficient CNN-free pipeline that does not render the graphic into
pixels (i.e. rasterization), and takes textual document of the vector graphics
as input, called YOLaT (You Only Look at Text). YOLaT builds multi-graphs to
model the structural and spatial information in vector graphics, and a
dual-stream graph neural network is proposed to detect objects from the graph.
Our experiments show that by directly operating on vector graphics, YOLaT
out-performs raster-graphic based object detection baselines in terms of both
average precision and efficiency.

negative:
The explosive growth of digital images in video surveillance and social media
has led to the significant need for efficient search of persons of interest in
law enforcement and forensic applications. Despite tremendous progress in
primary biometric traits (e.g., face and fingerprint) based person
identification, a single biometric trait alone cannot meet the desired
recognition accuracy in forensic scenarios. Tattoos, as one of the important
soft biometric traits, have been found to be valuable for assisting in person
identification. However, tattoo search in a large collection of unconstrained
images remains a difficult problem, and existing tattoo search methods mainly
focus on matching cropped tattoos, which is different from real application
scenarios. To close the gap, we propose an efficient tattoo search approach
that is able to learn tattoo detection and compact representation jointly in a
single convolutional neural network (CNN) via multi-task learning. While the
features in the backbone network are shared by both tattoo detection and
compact representation learning, individual latent layers of each sub-network
optimize the shared features toward the detection and feature learning tasks,
respectively. We resolve the small batch size issue inside the joint tattoo
detection and compact representation learning network via random image stitch
and preceding feature buffering. We evaluate the proposed tattoo search system
using multiple public-domain tattoo benchmarks, and a gallery set with about
300K distracter tattoo images compiled from these datasets and images from the
Internet. In addition, we also introduce a tattoo sketch dataset containing 300
tattoos for sketch-based tattoo search. Experimental results show that the
proposed approach has superior performance in tattoo detection and tattoo
search at scale compared to several state-of-the-art tattoo retrieval
algorithms.

negative:
When one wants to train a neural network to perform semantic segmentation,
creating pixel-level annotations for each of the images in the database is a
tedious task. If he works with aerial or satellite images, which are usually
very large, it is even worse. With that in mind, we investigate how to use
image-level annotations in order to perform semantic segmentation. Image-level
annotations are much less expensive to acquire than pixel-level annotations,
but we lose a lot of information for the training of the model. From the
annotations of the images, the model must find by itself how to classify the
different regions of the image. In this work, we use the method proposed by Anh
and Kwak [1] to produce pixel-level annotation from image level annotation. We
compare the overall quality of our generated dataset with the original dataset.
In addition, we propose an adaptation of the AffinityNet that allows us to
directly perform a semantic segmentation. Our results show that the generated
labels lead to the same performances for the training of several segmentation
networks. Also, the quality of semantic segmentation performed directly by the
AffinityNet and the Random Walk is close to the one of the best
fully-supervised approaches.

negative:
Existing deep quantization methods provided an efficient solution for
large-scale image retrieval. However, the significant intra-class variations
like pose, illumination, and expressions in face images, still pose a challenge
for face image retrieval. In light of this, face image retrieval requires
sufficiently powerful learning metrics, which are absent in current deep
quantization works. Moreover, to tackle the growing unseen identities in the
query stage, face image retrieval drives more demands regarding model
generalization and system scalability than general image retrieval tasks. This
paper integrates product quantization with orthonormal constraints into an
end-to-end deep learning framework to effectively retrieve face images.
Specifically, a novel scheme that uses predefined orthonormal vectors as
codewords is proposed to enhance the quantization informativeness and reduce
codewords' redundancy. A tailored loss function maximizes discriminability
among identities in each quantization subspace for both the quantized and
original features. An entropy-based regularization term is imposed to reduce
the quantization error. Experiments are conducted on four commonly-used face
datasets under both seen and unseen identities retrieval settings. Our method
outperforms all the compared deep hashing/quantization state-of-the-arts under
both settings. Results validate the effectiveness of the proposed orthonormal
codewords in improving models' standard retrieval performance and
generalization ability. Combing with further experiments on two general image
datasets, it demonstrates the broad superiority of our method for scalable
image retrieval.

negative:
Recovering the original printed texts from a document with added handwritten
annotations in the marginal area is one of the challenging problems, especially
when the original document is not available. Therefore, this paper aims at
salvaging automatically the original document from the annotated document by
detecting and removing any handwritten annotations that appear in the marginal
area of the document without any loss of information. Here a two stage
algorithm is proposed, where in the first stage due to approximate marginal
boundary detection with horizontal and vertical projection profiles, all of the
marginal annotations along with some part of the original printed text that may
appear very close to the marginal boundary are removed. Therefore as a second
stage, using the connected components, a strategy is applied to bring back the
printed text components cropped during the first stage. The proposed method is
validated using a dataset of 50 documents having complex handwritten
annotations, which gives an overall accuracy of 89.01% in removing the marginal
annotations and 97.74% in case of retrieving the original printed text
document.

negative:
Everyone knows that thousand of words are represented by a single image. As a
result image search has become a very popular mechanism for the Web searchers.
Image search means, the search results are produced by the search engine should
be a set of images along with their Web page Unified Resource Locator. Now Web
searcher can perform two types of image search, they are Text to Image and
Image to Image search. In Text to Image search, search query should be a text.
Based on the input text data system will generate a set of images along with
their Web page URL as an output. On the other hand, in Image to Image search,
search query should be an image and based on this image system will generate a
set of images along with their Web page URL as an output. According to the
current scenarios, Text to Image search mechanism always not returns perfect
result. It matches the text data and then displays the corresponding images as
an output, which is not always perfect. To resolve this problem, Web
researchers have introduced the Image to Image search mechanism. In this paper,
we have also proposed an alternate approach of Image to Image search mechanism
using Histogram.

negative:
Access to online visual search engines implies sharing of private user
content - the query images. We introduce the concept of targeted mismatch
attack for deep learning based retrieval systems to generate an adversarial
image to conceal the query image. The generated image looks nothing like the
user intended query, but leads to identical or very similar retrieval results.
Transferring attacks to fully unseen networks is challenging. We show
successful attacks to partially unknown systems, by designing various loss
functions for the adversarial image construction. These include loss functions,
for example, for unknown global pooling operation or unknown input resolution
by the retrieval system. We evaluate the attacks on standard retrieval
benchmarks and compare the results retrieved with the original and adversarial
image.