query: Our team is developing street views for a mapping application, and we need to censor any potentially sensitive text information within the images we capture. As such, we require a model capable of detecting and interpreting multi-oriented text in various scenes. Traditional methods are proving too slow to process the volume of images we capture for the street view. We think the most efficient solution would be to use two independent models. The first model should be able to extract text regions from the original image, while the second should interpret the text from the region, regardless of its orientation. We are seeking a suitable structure of models that can identify text from images, and ideally, these structures should support parallel training due to the large training set we have for this task. The second model should also be capable of understanding different languages and local slang. Additionally, we would prefer models that come with commercially licensed code.

positive:
Key information extraction from document images is of paramount importance in
office automation. Conventional template matching based approaches fail to
generalize well to document images of unseen templates, and are not robust
against text recognition errors. In this paper, we propose an end-to-end
Spatial Dual-Modality Graph Reasoning method (SDMG-R) to extract key
information from unstructured document images. We model document images as
dual-modality graphs, nodes of which encode both the visual and textual
features of detected text regions, and edges of which represent the spatial
relations between neighboring text regions. The key information extraction is
solved by iteratively propagating messages along graph edges and reasoning the
categories of graph nodes. In order to roundly evaluate our proposed method as
well as boost the future research, we release a new dataset named WildReceipt,
which is collected and annotated tailored for the evaluation of key information
extraction from document images of unseen templates in the wild. It contains 25
key information categories, a total of about 69000 text boxes, and is about 2
times larger than the existing public datasets. Extensive experiments validate
that all information including visual features, textual features and spatial
relations can benefit key information extraction. It has been shown that SDMG-R
can effectively extract key information from document images of unseen
templates, and obtain new state-of-the-art results on the recent popular
benchmark SROIE and our WildReceipt. Our code and dataset will be publicly
released.

positive:
In this work, we propose a new technique that combines appearance and text in
a Convolutional Neural Network (CNN), with the aim of detecting regions of
different textual categories. We define a novel visual representation of the
semantic meaning of text that allows a seamless integration in a standard CNN
architecture. This representation, referred to as text-map, is integrated with
the actual image to provide a much richer input to the network. Text-maps are
colored with different intensities depending on the relevance of the words
recognized over the image. Concretely, these words are previously extracted
using Optical Character Recognition (OCR) and they are colored according to the
probability of belonging to a textual category of interest. In this sense, this
solution is especially relevant in the context of item coding for supermarket
products, where different types of textual categories must be identified, such
as ingredients or nutritional facts. We evaluated our solution in the
proprietary item coding dataset of Nielsen Brandbank, which contains more than
10,000 images for train and 2,000 images for test. The reported results
demonstrate that our approach focused on visual and textual data outperforms
state-of-the-art algorithms only based on appearance, such as standard Faster
R-CNN. These enhancements are reflected in precision and recall, which are
improved in 42 and 33 points respectively.

positive:
We introduce Chinese Text in the Wild, a very large dataset of Chinese text
in street view images. While optical character recognition (OCR) in document
images is well studied and many commercial tools are available, detection and
recognition of text in natural images is still a challenging problem,
especially for more complicated character sets such as Chinese text. Lack of
training data has always been a problem, especially for deep learning methods
which require massive training data.
  In this paper we provide details of a newly created dataset of Chinese text
with about 1 million Chinese characters annotated by experts in over 30
thousand street view images. This is a challenging dataset with good diversity.
It contains planar text, raised text, text in cities, text in rural areas, text
under poor illumination, distant text, partially occluded text, etc. For each
character in the dataset, the annotation includes its underlying character, its
bounding box, and 6 attributes. The attributes indicate whether it has complex
background, whether it is raised, whether it is handwritten or printed, etc.
The large size and diversity of this dataset make it suitable for training
robust neural networks for various tasks, particularly detection and
recognition. We give baseline results using several state-of-the-art networks,
including AlexNet, OverFeat, Google Inception and ResNet for character
recognition, and YOLOv2 for character detection in images. Overall Google
Inception has the best performance on recognition with 80.5% top-1 accuracy,
while YOLOv2 achieves an mAP of 71.0% on detection. Dataset, source code and
trained models will all be publicly available on the website.

positive:
A significant number of images shared on social media platforms such as
Facebook and Instagram contain text in various forms. It's increasingly
becoming commonplace for bad actors to share misinformation, hate speech or
other kinds of harmful content as text overlaid on images on such platforms. A
scene-text understanding system should hence be able to handle text in various
orientations that the adversary might use. Moreover, such a system can be
incorporated into screen readers used to aid the visually impaired. In this
work, we extend the scene-text extraction system at Facebook, Rosetta, to
efficiently handle text in various orientations. Specifically, we incorporate
the Rotation Region Proposal Networks (RRPN) in our text extraction pipeline
and offer practical suggestions for building and deploying a model for
detecting and recognizing text in arbitrary orientations efficiently.
Experimental results show a significant improvement on detecting rotated text.

positive:
Text detection and recognition in natural images have long been considered as
two separate tasks that are processed sequentially. Training of two tasks in a
unified framework is non-trivial due to significant dif- ferences in
optimisation difficulties. In this work, we present a conceptually simple yet
efficient framework that simultaneously processes the two tasks in one shot.
Our main contributions are three-fold: 1) we propose a novel text-alignment
layer that allows it to precisely compute convolutional features of a text
instance in ar- bitrary orientation, which is the key to boost the per-
formance; 2) a character attention mechanism is introduced by using character
spatial information as explicit supervision, leading to large improvements in
recognition; 3) two technologies, together with a new RNN branch for word
recognition, are integrated seamlessly into a single model which is end-to-end
trainable. This allows the two tasks to work collaboratively by shar- ing
convolutional features, which is critical to identify challenging text
instances. Our model achieves impressive results in end-to-end recognition on
the ICDAR2015 dataset, significantly advancing most recent results, with
improvements of F-measure from (0.54, 0.51, 0.47) to (0.82, 0.77, 0.63), by
using a strong, weak and generic lexicon respectively. Thanks to joint
training, our method can also serve as a good detec- tor by achieving a new
state-of-the-art detection performance on two datasets.

positive:
Recent advancements in the area of Computer Vision with state-of-art Neural
Networks has given a boost to Optical Character Recognition (OCR) accuracies.
However, extracting characters/text alone is often insufficient for relevant
information extraction as documents also have a visual structure that is not
captured by OCR. Extracting information from tables, charts, footnotes, boxes,
headings and retrieving the corresponding structured representation for the
document remains a challenge and finds application in a large number of
real-world use cases. In this paper, we propose a novel enterprise based
end-to-end framework called DeepReader which facilitates information extraction
from document images via identification of visual entities and populating a
meta relational model across different entities in the document image. The
model schema allows for an easy to understand abstraction of the entities
detected by the deep vision models and the relationships between them.
DeepReader has a suite of state-of-the-art vision algorithms which are applied
to recognize handwritten and printed text, eliminate noisy effects, identify
the type of documents and detect visual entities like tables, lines and boxes.
Deep Reader maps the extracted entities into a rich relational schema so as to
capture all the relevant relationships between entities (words, textboxes,
lines etc) detected in the document. Relevant information and fields can then
be extracted from the document by writing SQL queries on top of the
relationship tables. A natural language based interface is added on top of the
relationship schema so that a non-technical user, specifying the queries in
natural language, can fetch the information with minimal effort. In this paper,
we also demonstrate many different capabilities of Deep Reader and report
results on a real-world use case.

positive:
We present a novel deep neural model for text detection in document images.
For robust text detection in noisy scanned documents, the advantages of
multi-task learning are adopted by adding an auxiliary task of text
enhancement. Namely, our proposed model is designed to perform noise reduction
and text region enhancement as well as text detection. Moreover, we enrich the
training data for the model with synthesized document images that are fully
labeled for text detection and enhancement, thus overcome the insufficiency of
labeled document image data. For the effective exploitation of the synthetic
and real data, the training process is separated in two phases. The first phase
is training only synthetic data in a fully-supervised manner. Then real data
with only detection labels are added in the second phase. The enhancement task
for the real data is weakly-supervised with information from their detection
labels. Our methods are demonstrated in a real document dataset with
performances exceeding those of other text detection methods. Moreover,
ablations are conducted and the results confirm the effectiveness of the
synthetic data, auxiliary task, and weak-supervision. Whereas the existing text
detection studies mostly focus on the text in scenes, our proposed method is
optimized to the applications for the text in scanned documents.

positive:
Nowadays, as cameras are rapidly adopted in our daily routine, images of
documents are becoming both abundant and prevalent. Unlike natural images that
capture physical objects, document-images contain a significant amount of text
with critical semantics and complicated layouts. In this work, we devise a
generic unsupervised technique to learn multimodal affinities between textual
entities in a document-image, considering their visual style, the content of
their underlying text and their geometric context within the image. We then use
these learned affinities to automatically cluster the textual entities in the
image into different semantic groups. The core of our approach is a deep
optimization scheme dedicated for an image provided by the user that detects
and leverages reliable pairwise connections in the multimodal representation of
the textual elements in order to properly learn the affinities. We show that
our technique can operate on highly varying images spanning a wide range of
documents and demonstrate its applicability for various editing operations
manipulating the content, appearance and geometry of the image.

positive:
Extracting texts of various size and shape from images containing multiple
objects is an important problem in many contexts, especially, in connection to
e-commerce, augmented reality assistance system in natural scene, etc. The
existing works (based on only CNN) often perform sub-optimally when the image
contains regions of high entropy having multiple objects. This paper presents
an end-to-end text detection strategy combining a segmentation algorithm and an
ensemble of multiple text detectors of different types to detect text in every
individual image segments independently. The proposed strategy involves a
super-pixel based image segmenter which splits an image into multiple regions.
A convolutional deep neural architecture is developed which works on each of
the segments and detects texts of multiple shapes, sizes, and structures. It
outperforms the competing methods in terms of coverage in detecting texts in
images especially the ones where the text of various types and sizes are
compacted in a small region along with various other objects. Furthermore, the
proposed text detection method along with a text recognizer outperforms the
existing state-of-the-art approaches in extracting text from high entropy
images. We validate the results on a dataset consisting of product images on an
e-commerce website.

positive:
Text in an image provides vital information for interpreting its contents,
and text in a scene can aide with a variety of tasks from navigation, to
obstacle avoidance, and odometry. Despite its value, however, identifying
general text in images remains a challenging research problem. Motivated by the
need to consider the widely varying forms of natural text, we propose a
bottom-up approach to the problem which reflects the `characterness' of an
image region. In this sense our approach mirrors the move from saliency
detection methods to measures of `objectness'. In order to measure the
characterness we develop three novel cues that are tailored for character
detection, and a Bayesian method for their integration. Because text is made up
of sets of characters, we then design a Markov random field (MRF) model so as
to exploit the inherent dependencies between characters.
  We experimentally demonstrate the effectiveness of our characterness cues as
well as the advantage of Bayesian multi-cue integration. The proposed text
detector outperforms state-of-the-art methods on a few benchmark scene text
detection datasets. We also show that our measurement of `characterness' is
superior than state-of-the-art saliency detection models when applied to the
same task.

positive:
Reading text from images remains challenging due to multi-orientation,
perspective distortion and especially the curved nature of irregular text. Most
of existing approaches attempt to solve the problem in two or multiple stages,
which is considered to be the bottleneck to optimize the overall performance.
To address this issue, we propose an end-to-end trainable network architecture,
named TextNet, which is able to simultaneously localize and recognize irregular
text from images. Specifically, we develop a scale-aware attention mechanism to
learn multi-scale image features as a backbone network, sharing fully
convolutional features and computation for localization and recognition. In
text detection branch, we directly generate text proposals in quadrangles,
covering oriented, perspective and curved text regions. To preserve text
features for recognition, we introduce a perspective RoI transform layer, which
can align quadrangle proposals into small feature maps. Furthermore, in order
to extract effective features for recognition, we propose to encode the aligned
RoI features by RNN into context information, combining spatial attention
mechanism to generate text sequences. This overall pipeline is capable of
handling both regular and irregular cases. Finally, text localization and
recognition tasks can be jointly trained in an end-to-end fashion with designed
multi-task loss. Experiments on standard benchmarks show that the proposed
TextNet can achieve state-of-the-art performance, and outperform existing
approaches on irregular datasets by a large margin.

positive:
Text spotting in natural scene images is of great importance for many image
understanding tasks. It includes two sub-tasks: text detection and recognition.
In this work, we propose a unified network that simultaneously localizes and
recognizes text with a single forward pass, avoiding intermediate processes
such as image cropping and feature re-calculation, word separation, and
character grouping.
  In contrast to existing approaches that consider text detection and
recognition as two distinct tasks and tackle them one by one, the proposed
framework settles these two tasks concurrently. The whole framework can be
trained end-to-end and is able to handle text of arbitrary shapes. The
convolutional features are calculated only once and shared by both detection
and recognition modules. Through multi-task training, the learned features
become more discriminate and improve the overall performance. By employing the
$2$D attention model in word recognition, the irregularity of text can be
robustly addressed. It provides the spatial location for each character, which
not only helps local feature extraction in word recognition, but also indicates
an orientation angle to refine text localization. Our proposed method has
achieved state-of-the-art performance on several standard text spotting
benchmarks, including both regular and irregular ones.

positive:
Scene text detection methods based on neural networks have emerged recently
and have shown promising results. Previous methods trained with rigid
word-level bounding boxes exhibit limitations in representing the text region
in an arbitrary shape. In this paper, we propose a new scene text detection
method to effectively detect text area by exploring each character and affinity
between characters. To overcome the lack of individual character level
annotations, our proposed framework exploits both the given character-level
annotations for synthetic images and the estimated character-level
ground-truths for real images acquired by the learned interim model. In order
to estimate affinity between characters, the network is trained with the newly
proposed representation for affinity. Extensive experiments on six benchmarks,
including the TotalText and CTW-1500 datasets which contain highly curved texts
in natural images, demonstrate that our character-level text detection
significantly outperforms the state-of-the-art detectors. According to the
results, our proposed method guarantees high flexibility in detecting
complicated scene text images, such as arbitrarily-oriented, curved, or
deformed texts.

positive:
Recent deep learning models have demonstrated strong capabilities for
classifying text and non-text components in natural images. They extract a
high-level feature computed globally from a whole image component (patch),
where the cluttered background information may dominate true text features in
the deep representation. This leads to less discriminative power and poorer
robustness. In this work, we present a new system for scene text detection by
proposing a novel Text-Attentional Convolutional Neural Network (Text-CNN) that
particularly focuses on extracting text-related regions and features from the
image components. We develop a new learning mechanism to train the Text-CNN
with multi-level and rich supervised information, including text region mask,
character label, and binary text/nontext information. The rich supervision
information enables the Text-CNN with a strong capability for discriminating
ambiguous texts, and also increases its robustness against complicated
background components. The training process is formulated as a multi-task
learning problem, where low-level supervised information greatly facilitates
main task of text/non-text classification. In addition, a powerful low-level
detector called Contrast- Enhancement Maximally Stable Extremal Regions
(CE-MSERs) is developed, which extends the widely-used MSERs by enhancing
intensity contrast between text patterns and background. This allows it to
detect highly challenging text patterns, resulting in a higher recall. Our
approach achieved promising results on the ICDAR 2013 dataset, with a F-measure
of 0.82, improving the state-of-the-art results substantially.

positive:
Detecting incidental scene text is a challenging task because of
multi-orientation, perspective distortion, and variation of text size, color
and scale. Retrospective research has only focused on using rectangular
bounding box or horizontal sliding window to localize text, which may result in
redundant background noise, unnecessary overlap or even information loss. To
address these issues, we propose a new Convolutional Neural Networks (CNNs)
based method, named Deep Matching Prior Network (DMPNet), to detect text with
tighter quadrangle. First, we use quadrilateral sliding windows in several
specific intermediate convolutional layers to roughly recall the text with
higher overlapping area and then a shared Monte-Carlo method is proposed for
fast and accurate computing of the polygonal areas. After that, we designed a
sequential protocol for relative regression which can exactly predict text with
compact quadrangle. Moreover, a auxiliary smooth Ln loss is also proposed for
further regressing the position of text, which has better overall performance
than L2 loss and smooth L1 loss in terms of robustness and stability. The
effectiveness of our approach is evaluated on a public word-level,
multi-oriented scene text database, ICDAR 2015 Robust Reading Competition
Challenge 4 "Incidental scene text localization". The performance of our method
is evaluated by using F-measure and found to be 70.64%, outperforming the
existing state-of-the-art method with F-measure 63.76%.

positive:
We present a novel single-shot text detector that directly outputs word-level
bounding boxes in a natural image. We propose an attention mechanism which
roughly identifies text regions via an automatically learned attentional map.
This substantially suppresses background interference in the convolutional
features, which is the key to producing accurate inference of words,
particularly at extremely small sizes. This results in a single model that
essentially works in a coarse-to-fine manner. It departs from recent FCN- based
text detectors which cascade multiple FCN models to achieve an accurate
prediction. Furthermore, we develop a hierarchical inception module which
efficiently aggregates multi-scale inception features. This enhances local
details, and also encodes strong context information, allow- ing the detector
to work reliably on multi-scale and multi- orientation text with single-scale
images. Our text detector achieves an F-measure of 77% on the ICDAR 2015 bench-
mark, advancing the state-of-the-art results in [18, 28]. Demo is available at:
http://sstd.whuang.org/.

positive:
In this paper, we develop a novel unified framework called DeepText for text
region proposal generation and text detection in natural images via a fully
convolutional neural network (CNN). First, we propose the inception region
proposal network (Inception-RPN) and design a set of text characteristic prior
bounding boxes to achieve high word recall with only hundred level candidate
proposals. Next, we present a powerful textdetection network that embeds
ambiguous text category (ATC) information and multilevel region-of-interest
pooling (MLRP) for text and non-text classification and accurate localization.
Finally, we apply an iterative bounding box voting scheme to pursue high recall
in a complementary manner and introduce a filtering algorithm to retain the
most suitable bounding box, while removing redundant inner and outer boxes for
each text instance. Our approach achieves an F-measure of 0.83 and 0.85 on the
ICDAR 2011 and 2013 robust text detection benchmarks, outperforming previous
state-of-the-art results.

positive:
Text in natural images is of arbitrary orientations, requiring detection in
terms of oriented bounding boxes. Normally, a multi-oriented text detector
often involves two key tasks: 1) text presence detection, which is a
classification problem disregarding text orientation; 2) oriented bounding box
regression, which concerns about text orientation. Previous methods rely on
shared features for both tasks, resulting in degraded performance due to the
incompatibility of the two tasks. To address this issue, we propose to perform
classification and regression on features of different characteristics,
extracted by two network branches of different designs. Concretely, the
regression branch extracts rotation-sensitive features by actively rotating the
convolutional filters, while the classification branch extracts
rotation-invariant features by pooling the rotation-sensitive features. The
proposed method named Rotation-sensitive Regression Detector (RRD) achieves
state-of-the-art performance on three oriented scene text benchmark datasets,
including ICDAR 2015, MSRA-TD500, RCTW-17 and COCO-Text. Furthermore, RRD
achieves a significant improvement on a ship collection dataset, demonstrating
its generality on oriented object detection.

positive:
Recent state-of-the-art scene text recognition methods have primarily focused
on horizontal text in images. However, in several Asian countries, including
China, large amounts of text in signs, books, and TV commercials are vertically
directed. Because the horizontal and vertical texts exhibit different
characteristics, developing an algorithm that can simultaneously recognize both
types of text in real environments is necessary. To address this problem, we
adopted the direction encoding mask (DEM) and selective attention network (SAN)
methods based on supervised learning. DEM contains directional information to
compensate in cases that lack text direction; therefore, our network is trained
using this information to handle the vertical text. The SAN method is designed
to work individually for both types of text. To train the network to recognize
both types of text and to evaluate the effectiveness of the designed model, we
prepared a new synthetic vertical text dataset and collected an actual vertical
text dataset (VTD142) from the Web. Using these datasets, we proved that our
proposed model can accurately recognize both vertical and horizontal text and
can achieve state-of-the-art results in experiments using benchmark datasets,
including the street view test (SVT), IIIT-5k, and ICDAR. Although our model is
relatively simple as compared to its predecessors, it maintains the accuracy
and is trained in an end-to-end manner.

positive:
Unifying text detection and text recognition in an end-to-end training
fashion has become a new trend for reading text in the wild, as these two tasks
are highly relevant and complementary. In this paper, we investigate the
problem of scene text spotting, which aims at simultaneous text detection and
recognition in natural images. An end-to-end trainable neural network named as
Mask TextSpotter is presented. Different from the previous text spotters that
follow the pipeline consisting of a proposal generation network and a
sequence-to-sequence recognition network, Mask TextSpotter enjoys a simple and
smooth end-to-end learning procedure, in which both detection and recognition
can be achieved directly from two-dimensional space via semantic segmentation.
Further, a spatial attention module is proposed to enhance the performance and
universality. Benefiting from the proposed two-dimensional representation on
both detection and recognition, it easily handles text instances of irregular
shapes, for instance, curved text. We evaluate it on four English datasets and
one multi-language dataset, achieving consistently superior performance over
state-of-the-art methods in both detection and end-to-end text recognition
tasks. Moreover, we further investigate the recognition module of our method
separately, which significantly outperforms state-of-the-art methods on both
regular and irregular text datasets for scene text recognition.

positive:
Texts appearing in daily scenes that can be recognized by OCR (Optical
Character Recognition) tools contain significant information, such as street
name, product brand and prices. Two tasks -- text-based visual question
answering and text-based image captioning, with a text extension from existing
vision-language applications, are catching on rapidly. To address these
problems, many sophisticated multi-modality encoding frameworks (such as
heterogeneous graph structure) are being used. In this paper, we argue that a
simple attention mechanism can do the same or even better job without any bells
and whistles. Under this mechanism, we simply split OCR token features into
separate visual- and linguistic-attention branches, and send them to a popular
Transformer decoder to generate answers or captions. Surprisingly, we find this
simple baseline model is rather strong -- it consistently outperforms
state-of-the-art (SOTA) models on two popular benchmarks, TextVQA and all three
tasks of ST-VQA, although these SOTA models use far more complex encoding
mechanisms. Transferring it to text-based image captioning, we also surpass the
TextCaps Challenge 2020 winner. We wish this work to set the new baseline for
this two OCR text related applications and to inspire new thinking of
multi-modality encoder design. Code is available at
https://github.com/ZephyrZhuQi/ssbaseline

positive:
Recognizing text from natural images is a hot research topic in computer
vision due to its various applications. Despite the enduring research of
several decades on optical character recognition (OCR), recognizing texts from
natural images is still a challenging task. This is because scene texts are
often in irregular (e.g. curved, arbitrarily-oriented or seriously distorted)
arrangements, which have not yet been well addressed in the literature.
Existing methods on text recognition mainly work with regular (horizontal and
frontal) texts and cannot be trivially generalized to handle irregular texts.
In this paper, we develop the arbitrary orientation network (AON) to directly
capture the deep features of irregular texts, which are combined into an
attention-based decoder to generate character sequence. The whole network can
be trained end-to-end by using only images and word-level annotations.
Extensive experiments on various benchmarks, including the CUTE80,
SVT-Perspective, IIIT5k, SVT and ICDAR datasets, show that the proposed
AON-based method achieves the-state-of-the-art performance in irregular
datasets, and is comparable to major existing methods in regular datasets.

negative:
This paper describes a dataset containing small images of text from everyday
scenes. The purpose of the dataset is to support the development of new
automated systems that can detect and analyze text. Although much research has
been devoted to text detection and recognition in scanned documents, relatively
little attention has been given to text detection in other types of images,
such as photographs that are posted on social-media sites. This new dataset,
known as COCO-Text-Patch, contains approximately 354,000 small images that are
each labeled as "text" or "non-text". This dataset particularly addresses the
problem of text verification, which is an essential stage in the end-to-end
text detection and recognition pipeline. In order to evaluate the utility of
this dataset, it has been used to train two deep convolution neural networks to
distinguish text from non-text. One network is inspired by the GoogLeNet
architecture, and the second one is based on CaffeNet. Accuracy levels of 90.2%
and 90.9% were obtained using the two networks, respectively. All of the
images, source code, and deep-learning trained models described in this paper
will be publicly available

negative:
Regional language extraction from a natural scene image is always a
challenging proposition due to its dependence on the text information extracted
from Image. Text Extraction on the other hand varies on different lighting
condition, arbitrary orientation, inadequate text information, heavy background
influence over text and change of text appearance. This paper presents a novel
unified method for tackling the above challenges. The proposed work uses an
image correction and segmentation technique on the existing Text Detection
Pipeline an Efficient and Accurate Scene Text Detector (EAST). EAST uses
standard PVAnet architecture to select features and non maximal suppression to
detect text from image. Text recognition is done using combined architecture of
MaxOut convolution neural network (CNN) and Bidirectional long short term
memory (LSTM) network. After recognizing text using the Deep Learning based
approach, the native Languages are translated to English and tokenized using
standard Text Tokenizers. The tokens that very likely represent a location is
used to find the Global Positioning System (GPS) coordinates of the location
and subsequently the regional languages spoken in that location is extracted.
The proposed method is tested on a self generated dataset collected from
Government of India dataset and experimented on Standard Dataset to evaluate
the performance of the proposed technique. Comparative study with a few
state-of-the-art methods on text detection, recognition and extraction of
regional language from images shows that the proposed method outperforms the
existing methods.

negative:
Many approaches have recently been proposed to detect irregular scene text
and achieved promising results. However, their localization results may not
well satisfy the following text recognition part mainly because of two reasons:
1) recognizing arbitrary shaped text is still a challenging task, and 2)
prevalent non-trainable pipeline strategies between text detection and text
recognition will lead to suboptimal performances. To handle this
incompatibility problem, in this paper we propose an end-to-end trainable text
spotting approach named Text Perceptron. Concretely, Text Perceptron first
employs an efficient segmentation-based text detector that learns the latent
text reading order and boundary information. Then a novel Shape Transform
Module (abbr. STM) is designed to transform the detected feature regions into
regular morphologies without extra parameters. It unites text detection and the
following recognition part into a whole framework, and helps the whole network
achieve global optimization. Experiments show that our method achieves
competitive performance on two standard text benchmarks, i.e., ICDAR 2013 and
ICDAR 2015, and also obviously outperforms existing methods on irregular text
benchmarks SCUT-CTW1500 and Total-Text.

negative:
Recently, scene text detection has become an active research topic in
computer vision and document analysis, because of its great importance and
significant challenge. However, vast majority of the existing methods detect
text within local regions, typically through extracting character, word or line
level candidates followed by candidate aggregation and false positive
elimination, which potentially exclude the effect of wide-scope and long-range
contextual cues in the scene. To take full advantage of the rich information
available in the whole natural image, we propose to localize text in a holistic
manner, by casting scene text detection as a semantic segmentation problem. The
proposed algorithm directly runs on full images and produces global, pixel-wise
prediction maps, in which detections are subsequently formed. To better make
use of the properties of text, three types of information regarding text
region, individual characters and their relationship are estimated, with a
single Fully Convolutional Network (FCN) model. With such predictions of text
properties, the proposed algorithm can simultaneously handle horizontal,
multi-oriented and curved text in real-world natural images. The experiments on
standard benchmarks, including ICDAR 2013, ICDAR 2015 and MSRA-TD500,
demonstrate that the proposed algorithm substantially outperforms previous
state-of-the-art approaches. Moreover, we report the first baseline result on
the recently-released, large-scale dataset COCO-Text.

negative:
Since the low quality of document images will greatly undermine the chances
of success in automatic text recognition and analysis, it is necessary to
assess the quality of document images uploaded in online business process, so
as to reject those images of low quality. In this paper, we attempt to achieve
document image quality assessment and our contributions are twofold. Firstly,
since document image quality assessment is more interested in text, we propose
a text line based framework to estimate document image quality, which is
composed of three stages: text line detection, text line quality prediction,
and overall quality assessment. Text line detection aims to find potential text
lines with a detector. In the text line quality prediction stage, the quality
score is computed for each text line with a CNN-based prediction model. The
overall quality of document images is finally assessed with the ensemble of all
text line quality. Secondly, to train the prediction model, a large-scale
dataset, comprising 52,094 text line images, is synthesized with diverse
attributes. For each text line image, a quality label is computed with a
piece-wise function. To demonstrate the effectiveness of the proposed
framework, comprehensive experiments are evaluated on two popular document
image quality assessment benchmarks. Our framework significantly outperforms
the state-of-the-art methods by large margins on the large and complicated
dataset.

negative:
Conventional approaches to image-text retrieval mainly focus on indexing
visual objects appearing in pictures but ignore the interactions between these
objects. Such objects occurrences and interactions are equivalently useful and
important in this field as they are usually mentioned in the text. Scene graph
presentation is a suitable method for the image-text matching challenge and
obtained good results due to its ability to capture the inter-relationship
information. Both images and text are represented in scene graph levels and
formulate the retrieval challenge as a scene graph matching challenge. In this
paper, we introduce the Local and Global Scene Graph Matching (LGSGM) model
that enhances the state-of-the-art method by integrating an extra graph
convolution network to capture the general information of a graph.
Specifically, for a pair of scene graphs of an image and its caption, two
separate models are used to learn the features of each graph's nodes and edges.
Then a Siamese-structure graph convolution model is employed to embed graphs
into vector forms. We finally combine the graph-level and the vector-level to
calculate the similarity of this image-text pair. The empirical experiments
show that our enhancement with the combination of levels can improve the
performance of the baseline method by increasing the recall by more than 10% on
the Flickr30k dataset.

negative:
The question we answer with this work is: can we convert a text document into
an image to exploit best image classification models to classify documents? To
answer this question we present a novel text classification method which
converts a text document into an encoded image, using word embedding and
capabilities of Convolutional Neural Networks (CNNs), successfully employed in
image classification. We evaluate our approach by obtaining promising results
on some well-known benchmark datasets for text classification. This work allows
the application of many of the advanced CNN architectures developed for
Computer Vision to Natural Language Processing. We test the proposed approach
on a multi-modal dataset, proving that it is possible to use a single deep
model to represent text and image in the same feature space.

negative:
Although many methods have been proposed to deal with nature image
super-resolution (SR) and get impressive performance, the text images SR is not
good due to their ignorance of document images. In this paper, we propose a
matting-based dual generative adversarial network (mdGAN) for document image
SR. Firstly, the input image is decomposed into document text, foreground and
background layers using deep image matting. Then two parallel branches are
constructed to recover text boundary information and color information
respectively. Furthermore, in order to improve the restoration accuracy of
characters in output image, we use the input image's corresponding ground truth
text label as extra supervise information to refine the two-branch networks
during training. Experiments on real text images demonstrate that our method
outperforms several state-of-the-art methods quantitatively and qualitatively.

negative:
A growing demand for natural-scene text detection has been witnessed by the
computer vision community since text information plays a significant role in
scene understanding and image indexing. Deep neural networks are being used due
to their strong capabilities of pixel-wise classification or word localization,
similar to being used in common vision problems. In this paper, we present a
novel two-task network with integrating bottom and top cues. The first task
aims to predict a pixel-by-pixel labeling and based on which, word proposals
are generated with a canonical connected component analysis. The second task
aims to output a bundle of character candidates used later to verify the word
proposals. The two sub-networks share base convolutional features and moreover,
we present a new loss to strengthen the interaction between them. We evaluate
the proposed network on public benchmark datasets and show it can detect
arbitrary-orientation scene text with a finer output boundary. In ICDAR 2013
text localization task, we achieve the state-of-the-art performance with an
F-score of 0.919 and a much better recall of 0.915.

negative:
An unconstrained end-to-end text localization and recognition method is
presented. The method detects initial text hypothesis in a single pass by an
efficient region-based method and subsequently refines the text hypothesis
using a more robust local text model, which deviates from the common assumption
of region-based methods that all characters are detected as connected
components.
  Additionally, a novel feature based on character stroke area estimation is
introduced. The feature is efficiently computed from a region distance map, it
is invariant to scaling and rotations and allows to efficiently detect text
regions regardless of what portion of text they capture.
  The method runs in real time and achieves state-of-the-art text localization
and recognition results on the ICDAR 2013 Robust Reading dataset.

negative:
Daily engagement in life experiences is increasingly interwoven with mobile
device use. Screen capture at the scale of seconds is being used in behavioral
studies and to implement "just-in-time" health interventions. The increasing
psychological breadth of digital information will continue to make the actual
screens that people view a preferred if not required source of data about life
experiences. Effective and efficient Information Extraction and Retrieval from
digital screenshots is a crucial prerequisite to successful use of screen data.
In this paper, we present the experimental workflow we exploited to: (i)
pre-process a unique collection of screen captures, (ii) extract unstructured
text embedded in the images, (iii) organize image text and metadata based on a
structured schema, (iv) index the resulting document collection, and (v) allow
for Image Retrieval through a dedicated vertical search engine application. The
adopted procedure integrates different open source libraries for traditional
image processing, Optical Character Recognition (OCR), and Image Retrieval. Our
aim is to assess whether and how state-of-the-art methodologies can be applied
to this novel data set. We show how combining OpenCV-based pre-processing
modules with a Long short-term memory (LSTM) based release of Tesseract OCR,
without ad hoc training, led to a 74% character-level accuracy of the extracted
text. Further, we used the processed repository as baseline for a dedicated
Image Retrieval system, for the immediate use and application for behavioral
and prevention scientists. We discuss issues of Text Information Extraction and
Retrieval that are particular to the screenshot image case and suggest
important future work.

negative:
Document Layout Analysis is a fundamental step in Handwritten Text Processing
systems, from the extraction of the text lines to the type of zone it belongs
to. We present a system based on artificial neural networks which is able to
determine not only the baselines of text lines present in the document, but
also performs geometric and logic layout analysis of the document. Experiments
in three different datasets demonstrate the potential of the method and show
competitive results with respect to state-of-the-art methods.

negative:
The Know Your Customer (KYC) and Anti Money Laundering (AML) are worldwide
practices to online customer identification based on personal identification
documents, similarity and liveness checking, and proof of address. To answer
the basic regulation question: are you whom you say you are? The customer needs
to upload valid identification documents (ID). This task imposes some
computational challenges since these documents are diverse, may present
different and complex backgrounds, some occlusion, partial rotation, poor
quality, or damage. Advanced text and document segmentation algorithms were
used to process the ID images. In this context, we investigated a method based
on U-Net to detect the document edges and text regions in ID images. Besides
the promising results on image segmentation, the U-Net based approach is
computationally expensive for a real application, since the image segmentation
is a customer device task. We propose a model optimization based on Octave
Convolutions to qualify the method to situations where storage, processing, and
time resources are limited, such as in mobile and robotic applications. We
conducted the evaluation experiments in two new datasets CDPhotoDataset and
DTDDataset, which are composed of real ID images of Brazilian documents. Our
results showed that the proposed models are efficient to document segmentation
tasks and portable.

negative:
Since real-world ubiquitous documents (e.g., invoices, tickets, resumes and
leaflets) contain rich information, automatic document image understanding has
become a hot topic. Most existing works decouple the problem into two separate
tasks, (1) text reading for detecting and recognizing texts in images and (2)
information extraction for analyzing and extracting key elements from
previously extracted plain text. However, they mainly focus on improving
information extraction task, while neglecting the fact that text reading and
information extraction are mutually correlated. In this paper, we propose a
unified end-to-end text reading and information extraction network, where the
two tasks can reinforce each other. Specifically, the multimodal visual and
textual features of text reading are fused for information extraction and in
turn, the semantics in information extraction contribute to the optimization of
text reading. On three real-world datasets with diverse document images (from
fixed layout to variable layout, from structured text to semi-structured text),
our proposed method significantly outperforms the state-of-the-art methods in
both efficiency and accuracy.

negative:
Text detection in natural scene images for content analysis is an interesting
task. The research community has seen some great developments for
English/Mandarin text detection. However, Urdu text extraction in natural scene
images is a task not well addressed. In this work, firstly, a new dataset is
introduced for Urdu text in natural scene images. The dataset comprises of 500
standalone images acquired from real scenes. Secondly, the channel enhanced
Maximally Stable Extremal Region (MSER) method is applied to extract Urdu text
regions as candidates in an image. Two-stage filtering mechanism is applied to
eliminate non-candidate regions. In the first stage, text and noise are
classified based on their geometric properties. In the second stage, a support
vector machine classifier is trained to discard non-text candidate regions.
After this, text candidate regions are linked using centroid-based vertical and
horizontal distances. Text lines are further analyzed by a different classifier
based on HOG features to remove non-text regions. Extensive experimentation is
performed on the locally developed dataset to evaluate the performance. The
experimental results show good performance on test set images. The dataset will
be made available for research use. To the best of our knowledge, the work is
the first of its kind for the Urdu language and would provide a good dataset
for free research use and serve as a baseline performance on the task of Urdu
text extraction.

negative:
Previous deep learning based state-of-the-art scene text detection methods
can be roughly classified into two categories. The first category treats scene
text as a type of general objects and follows general object detection paradigm
to localize scene text by regressing the text box locations, but troubled by
the arbitrary-orientation and large aspect ratios of scene text. The second one
segments text regions directly, but mostly needs complex post processing. In
this paper, we present a method that combines the ideas of the two types of
methods while avoiding their shortcomings. We propose to detect scene text by
localizing corner points of text bounding boxes and segmenting text regions in
relative positions. In inference stage, candidate boxes are generated by
sampling and grouping corner points, which are further scored by segmentation
maps and suppressed by NMS. Compared with previous methods, our method can
handle long oriented text naturally and doesn't need complex post processing.
The experiments on ICDAR2013, ICDAR2015, MSRA-TD500, MLT and COCO-Text
demonstrate that the proposed algorithm achieves better or comparable results
in both accuracy and efficiency. Based on VGG16, it achieves an F-measure of
84.3% on ICDAR2015 and 81.5% on MSRA-TD500.

negative:
Text detection, the key technology for understanding scene text, has become
an attractive research topic. For detecting various scene texts, researchers
propose plenty of detectors with different advantages: detection-based models
enjoy fast detection speed, and segmentation-based algorithms are not limited
by text shapes. However, for most intelligent systems, the detector needs to
detect arbitrary-shaped texts with high speed and accuracy simultaneously.
Thus, in this study, we design an efficient pipeline named as MT, which can
detect adhesive arbitrary-shaped texts with only a single binary mask in the
inference stage. This paper presents the contributions on three aspects: (1) a
light-weight detection framework is designed to speed up the inference process
while keeping high detection accuracy; (2) a multi-perspective feature module
is proposed to learn more discriminative representations to segment the mask
accurately; (3) a multi-factor constraints IoU minimization loss is introduced
for training the proposed model. The effectiveness of MT is evaluated on four
real-world scene text datasets, and it surpasses all the state-of-the-art
competitors to a large extent.

negative:
Image-based sequence recognition has been a long-standing research topic in
computer vision. In this paper, we investigate the problem of scene text
recognition, which is among the most important and challenging tasks in
image-based sequence recognition. A novel neural network architecture, which
integrates feature extraction, sequence modeling and transcription into a
unified framework, is proposed. Compared with previous systems for scene text
recognition, the proposed architecture possesses four distinctive properties:
(1) It is end-to-end trainable, in contrast to most of the existing algorithms
whose components are separately trained and tuned. (2) It naturally handles
sequences in arbitrary lengths, involving no character segmentation or
horizontal scale normalization. (3) It is not confined to any predefined
lexicon and achieves remarkable performances in both lexicon-free and
lexicon-based scene text recognition tasks. (4) It generates an effective yet
much smaller model, which is more practical for real-world application
scenarios. The experiments on standard benchmarks, including the IIIT-5K,
Street View Text and ICDAR datasets, demonstrate the superiority of the
proposed algorithm over the prior arts. Moreover, the proposed algorithm
performs well in the task of image-based music score recognition, which
evidently verifies the generality of it.

negative:
Localizing text instances in natural scenes is regarded as a fundamental
challenge in computer vision. Nevertheless, owing to the extremely varied
aspect ratios and scales of text instances in real scenes, most conventional
text detectors suffer from the sub-text problem that only localizes the
fragments of text instance (i.e., sub-texts). In this work, we quantitatively
analyze the sub-text problem and present a simple yet effective design,
COntrastive RElation (CORE) module, to mitigate that issue. CORE first
leverages a vanilla relation block to model the relations among all text
proposals (sub-texts of multiple text instances) and further enhances
relational reasoning via instance-level sub-text discrimination in a
contrastive manner. Such way naturally learns instance-aware representations of
text proposals and thus facilitates scene text detection. We integrate the CORE
module into a two-stage text detector of Mask R-CNN and devise our text
detector CORE-Text. Extensive experiments on four benchmarks demonstrate the
superiority of CORE-Text. Code is available:
\url{https://github.com/jylins/CORE-Text}.

negative:
Text classification is a fundamental task in NLP applications. Latest
research in this field has largely been divided into two major sub-fields.
Learning representations is one sub-field and learning deeper models, both
sequential and convolutional, which again connects back to the representation
is the other side. We posit the idea that the stronger the representation is,
the simpler classifier models are needed to achieve higher performance. In this
paper we propose a completely novel direction to text classification research,
wherein we convert text to a representation very similar to images, such that
any deep network able to handle images is equally able to handle text. We take
a deeper look at the representation of documents as an image and subsequently
utilize very simple convolution based models taken as is from computer vision
domain. This image can be cropped, re-scaled, re-sampled and augmented just
like any other image to work with most of the state-of-the-art large
convolution based models which have been designed to handle large image
datasets. We show impressive results with some of the latest benchmarks in the
related fields. We perform transfer learning experiments, both from text to
text domain and also from image to text domain. We believe this is a paradigm
shift from the way document understanding and text classification has been
traditionally done, and will drive numerous novel research ideas in the
community.

negative:
Recently, methods based on deep learning have dominated the field of text
recognition. With a large number of training data, most of them can achieve the
state-of-the-art performances. However, it is hard to harvest and label
sufficient text sequence images from the real scenes. To mitigate this issue,
several methods to synthesize text sequence images were proposed, yet they
usually need complicated preceding or follow-up steps. In this work, we present
a method which is able to generate infinite training data without any auxiliary
pre/post-process. We tackle the generation task as an image-to-image
translation one and utilize conditional adversarial networks to produce
realistic text sequence images in the light of the semantic ones. Some
evaluation metrics are involved to assess our method and the results
demonstrate that the caliber of the data is satisfactory. The code and dataset
will be publicly available soon.

negative:
Detection and recognition of text in natural images are two main problems in
the field of computer vision that have a wide variety of applications in
analysis of sports videos, autonomous driving, industrial automation, to name a
few. They face common challenging problems that are factors in how text is
represented and affected by several environmental conditions. The current
state-of-the-art scene text detection and/or recognition methods have exploited
the witnessed advancement in deep learning architectures and reported a
superior accuracy on benchmark datasets when tackling multi-resolution and
multi-oriented text. However, there are still several remaining challenges
affecting text in the wild images that cause existing methods to underperform
due to there models are not able to generalize to unseen data and the
insufficient labeled data. Thus, unlike previous surveys in this field, the
objectives of this survey are as follows: first, offering the reader not only a
review on the recent advancement in scene text detection and recognition, but
also presenting the results of conducting extensive experiments using a unified
evaluation framework that assesses pre-trained models of the selected methods
on challenging cases, and applies the same evaluation criteria on these
techniques. Second, identifying several existing challenges for detecting or
recognizing text in the wild images, namely, in-plane-rotation, multi-oriented
and multi-resolution text, perspective distortion, illumination reflection,
partial occlusion, complex fonts, and special characters. Finally, the paper
also presents insight into the potential research directions in this field to
address some of the mentioned challenges that are still encountering scene text
detection and recognition techniques.

negative:
Document layout comprises both structural and visual (eg. font-sizes)
information that is vital but often ignored by machine learning models. The few
existing models which do use layout information only consider textual contents,
and overlook the existence of contents in other modalities such as images.
Additionally, spatial interactions of presented contents in a layout were never
really fully exploited. To bridge this gap, we parse a document into content
blocks (eg. text, table, image) and propose a novel layout-aware multimodal
hierarchical framework, LAMPreT, to model the blocks and the whole document.
Our LAMPreT encodes each block with a multimodal transformer in the lower-level
and aggregates the block-level representations and connections utilizing a
specifically designed transformer at the higher-level. We design hierarchical
pretraining objectives where the lower-level model is trained similarly to
multimodal grounding models, and the higher-level model is trained with our
proposed novel layout-aware objectives. We evaluate the proposed model on two
layout-aware tasks -- text block filling and image suggestion and show the
effectiveness of our proposed hierarchical architecture as well as pretraining
techniques.

negative:
We introduce a novel type of text representation that preserves the 2D layout
of a document. This is achieved by encoding each document page as a
two-dimensional grid of characters. Based on this representation, we present a
generic document understanding pipeline for structured documents. This pipeline
makes use of a fully convolutional encoder-decoder network that predicts a
segmentation mask and bounding boxes. We demonstrate its capabilities on an
information extraction task from invoices and show that it significantly
outperforms approaches based on sequential text or document images.

negative:
In this paper, we are interested in editing text in natural images, which
aims to replace or modify a word in the source image with another one while
maintaining its realistic look. This task is challenging, as the styles of both
background and text need to be preserved so that the edited image is visually
indistinguishable from the source image. Specifically, we propose an end-to-end
trainable style retention network (SRNet) that consists of three modules: text
conversion module, background inpainting module and fusion module. The text
conversion module changes the text content of the source image into the target
text while keeping the original text style. The background inpainting module
erases the original text, and fills the text region with appropriate texture.
The fusion module combines the information from the two former modules, and
generates the edited text images. To our knowledge, this work is the first
attempt to edit text in natural images at the word level. Both visual effects
and quantitative results on synthetic and real-world dataset (ICDAR 2013) fully
confirm the importance and necessity of modular decomposition. We also conduct
extensive experiments to validate the usefulness of our method in various
real-world applications such as text image synthesis, augmented reality (AR)
translation, information hiding, etc.

negative:
Document image enhancement and binarization methods are often used to improve
the accuracy and efficiency of document image analysis tasks such as text
recognition. Traditional non-machine-learning methods are constructed on
low-level features in an unsupervised manner but have difficulty with
binarization on documents with severely degraded backgrounds. Convolutional
neural network-based methods focus only on grayscale images and on local
textual features. In this paper, we propose a two-stage color document image
enhancement and binarization method using generative adversarial neural
networks. In the first stage, four color-independent adversarial networks are
trained to extract color foreground information from an input image for
document image enhancement. In the second stage, two independent adversarial
networks with global and local features are trained for image binarization of
documents of variable size. For the adversarial neural networks, we formulate
loss functions between a discriminator and generators having an encoder-decoder
structure. Experimental results show that the proposed method achieves better
performance than many classical and state-of-the-art algorithms over the
Document Image Binarization Contest (DIBCO) datasets, the LRDE Document
Binarization Dataset (LRDE DBD), and our shipping label image dataset. We plan
to release the shipping label dataset as well as our implementation code at
github.com/opensuh/DocumentBinarization/.

negative:
Document Image Analysis, like any Digital Image Analysis requires
identification and extraction of proper features, which are generally extracted
from uncompressed images, though in reality images are made available in
compressed form for the reasons such as transmission and storage efficiency.
However, this implies that the compressed image should be decompressed, which
indents additional computing resources. This limitation induces the motivation
to research in extracting features directly from the compressed image. In this
research, we propose to extract essential features such as projection profile,
run-histogram and entropy for text document analysis directly from run-length
compressed text-documents. The experimentation illustrates that features are
extracted directly from the compressed image without going through the stage of
decompression, because of which the computing time is reduced. The feature
values so extracted are exactly identical to those extracted from uncompressed
images.

negative:
This paper introduces a novel rotation-based framework for arbitrary-oriented
text detection in natural scene images. We present the Rotation Region Proposal
Networks (RRPN), which are designed to generate inclined proposals with text
orientation angle information. The angle information is then adapted for
bounding box regression to make the proposals more accurately fit into the text
region in terms of the orientation. The Rotation Region-of-Interest (RRoI)
pooling layer is proposed to project arbitrary-oriented proposals to a feature
map for a text region classifier. The whole framework is built upon a
region-proposal-based architecture, which ensures the computational efficiency
of the arbitrary-oriented text detection compared with previous text detection
systems. We conduct experiments using the rotation-based framework on three
real-world scene text detection datasets and demonstrate its superiority in
terms of effectiveness and efficiency over previous approaches.

negative:
Scene text detection attracts much attention in computer vision, because it
can be widely used in many applications such as real-time text translation,
automatic information entry, blind person assistance, robot sensing and so on.
Though many methods have been proposed for horizontal and oriented texts,
detecting irregular shape texts such as curved texts is still a challenging
problem. To solve the problem, we propose a robust scene text detection method
with adaptive text region representation. Given an input image, a text region
proposal network is first used for extracting text proposals. Then, these
proposals are verified and refined with a refinement network. Here, recurrent
neural network based adaptive text region representation is proposed for text
region refinement, where a pair of boundary points are predicted each time step
until no new points are found. In this way, text regions of arbitrary shapes
are detected and represented with adaptive number of boundary points. This
gives more accurate description of text regions. Experimental results on five
benchmarks, namely, CTW1500, TotalText, ICDAR2013, ICDAR2015 and MSRATD500,
show that the proposed method achieves state-of-the-art in scene text
detection.

negative:
The ubiquity of smartphone cameras has led to more and more documents being
captured by cameras rather than scanned. Unlike flatbed scanners, photographed
documents are often folded and crumpled, resulting in large local variance in
text structure. The problem of document rectification is fundamental to the
Optical Character Recognition (OCR) process on documents, and its ability to
overcome geometric distortions significantly affects recognition accuracy.
Despite the great progress in recent OCR systems, most still rely on a
pre-process that ensures the text lines are straight and axis aligned. Recent
works have tackled the problem of rectifying document images taken in-the-wild
using various supervision signals and alignment means. However, they focused on
global features that can be extracted from the document's boundaries, ignoring
various signals that could be obtained from the document's content.
  We present CREASE: Content Aware Rectification using Angle Supervision, the
first learned method for document rectification that relies on the document's
content, the location of the words and specifically their orientation, as hints
to assist in the rectification process. We utilize a novel pixel-wise angle
regression approach and a curvature estimation side-task for optimizing our
rectification model. Our method surpasses previous approaches in terms of OCR
accuracy, geometric error and visual similarity.

negative:
Structured information extraction from document images usually consists of
three steps: text detection, text recognition, and text field labeling. While
text detection and text recognition have been heavily studied and improved a
lot in literature, text field labeling is less explored and still faces many
challenges. Existing learning based methods for text labeling task usually
require a large amount of labeled examples to train a specific model for each
type of document. However, collecting large amounts of document images and
labeling them is difficult and sometimes impossible due to privacy issues.
Deploying separate models for each type of document also consumes a lot of
resources. Facing these challenges, we explore one-shot learning for the text
field labeling task. Existing one-shot learning methods for the task are mostly
rule-based and have difficulty in labeling fields in crowded regions with few
landmarks and fields consisting of multiple separate text regions. To alleviate
these problems, we proposed a novel deep end-to-end trainable approach for
one-shot text field labeling, which makes use of attention mechanism to
transfer the layout information between document images. We further applied
conditional random field on the transferred layout information for the
refinement of field labeling. We collected and annotated a real-world one-shot
field labeling dataset with a large variety of document types and conducted
extensive experiments to examine the effectiveness of the proposed model. To
stimulate research in this direction, the collected dataset and the one-shot
model will be released1.

negative:
Text in curve orientation, despite being one of the common text orientations
in real world environment, has close to zero existence in well received scene
text datasets such as ICDAR2013 and MSRA-TD500. The main motivation of
Total-Text is to fill this gap and facilitate a new research direction for the
scene text community. On top of the conventional horizontal and multi-oriented
texts, it features curved-oriented text. Total-Text is highly diversified in
orientations, more than half of its images have a combination of more than two
orientations. Recently, a new breed of solutions that casted text detection as
a segmentation problem has demonstrated their effectiveness against
multi-oriented text. In order to evaluate its robustness against curved text,
we fine-tuned DeconvNet and benchmark it on Total-Text. Total-Text with its
annotation is available at https://github.com/cs-chan/Total-Text-Dataset

negative:
India is a multilingual multi-script country. In every state of India there
are two languages one is state local language and the other is English. For
example in Andhra Pradesh, a state in India, the document may contain text
words in English and Telugu script. For Optical Character Recognition (OCR) of
such a bilingual document, it is necessary to identify the script before
feeding the text words to the OCRs of individual scripts. In this paper, we are
introducing a simple and efficient technique of script identification for
Kannada, English and Hindi text words of a printed document. The proposed
approach is based on the horizontal and vertical projection profile for the
discrimination of the three scripts. The feature extraction is done based on
the horizontal projection profile of each text words. We analysed 700 different
words of Kannada, English and Hindi in order to extract the discrimination
features and for the development of knowledge base. We use the horizontal
projection profile of each text word and based on the horizontal projection
profile we extract the appropriate features. The proposed system is tested on
100 different document images containing more than 1000 text words of each
script and a classification rate of 98.25%, 99.25% and 98.87% is achieved for
Kannada, English and Hindi respectively.

negative:
In this paper, we propose a way of synthesizing realistic images directly
with natural language description, which has many useful applications, e.g.
intelligent image manipulation. We attempt to accomplish such synthesis: given
a source image and a target text description, our model synthesizes images to
meet two requirements: 1) being realistic while matching the target text
description; 2) maintaining other image features that are irrelevant to the
text description. The model should be able to disentangle the semantic
information from the two modalities (image and text), and generate new images
from the combined semantics. To achieve this, we proposed an end-to-end neural
architecture that leverages adversarial learning to automatically learn
implicit loss functions, which are optimized to fulfill the aforementioned two
requirements. We have evaluated our model by conducting experiments on
Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated
that our model is capable of synthesizing realistic images that match the given
descriptions, while still maintain other features of original images.

negative:
Previous approaches for scene text detection usually rely on manually defined
sliding windows. This work presents an intuitive two-stage region-based method
to detect multi-oriented text without any prior knowledge regarding the textual
shape. In the first stage, we estimate the possible locations of text instances
by detecting and linking corners instead of shifting a set of default anchors.
The quadrilateral proposals are geometry adaptive, which allows our method to
cope with various text aspect ratios and orientations. In the second stage, we
design a new pooling layer named Dual-RoI Pooling which embeds data
augmentation inside the region-wise subnetwork for more robust classification
and regression over these proposals. Experimental results on public benchmarks
confirm that the proposed method is capable of achieving comparable performance
with state-of-the-art methods. The code is publicly available at
https://github.com/xhzdeng/crpn

negative:
Previous approaches for scene text detection have already achieved promising
performances across various benchmarks. However, they usually fall short when
dealing with challenging scenarios, even when equipped with deep neural network
models, because the overall performance is determined by the interplay of
multiple stages and components in the pipelines. In this work, we propose a
simple yet powerful pipeline that yields fast and accurate text detection in
natural scenes. The pipeline directly predicts words or text lines of arbitrary
orientations and quadrilateral shapes in full images, eliminating unnecessary
intermediate steps (e.g., candidate aggregation and word partitioning), with a
single neural network. The simplicity of our pipeline allows concentrating
efforts on designing loss functions and neural network architecture.
Experiments on standard datasets including ICDAR 2015, COCO-Text and MSRA-TD500
demonstrate that the proposed algorithm significantly outperforms
state-of-the-art methods in terms of both accuracy and efficiency. On the ICDAR
2015 dataset, the proposed algorithm achieves an F-score of 0.7820 at 13.2fps
at 720p resolution.

negative:
The technological advancement and sophistication in cameras and gadgets
prompt researchers to have focus on image analysis and text understanding. The
deep learning techniques demonstrated well to assess the potential for
classifying text from natural scene images as reported in recent years. There
are variety of deep learning approaches that prospects the detection and
recognition of text, effectively from images. In this work, we presented Arabic
scene text recognition using Convolutional Neural Networks (ConvNets) as a deep
learning classifier. As the scene text data is slanted and skewed, thus to deal
with maximum variations, we employ five orientations with respect to single
occurrence of a character. The training is formulated by keeping filter size 3
x 3 and 5 x 5 with stride value as 1 and 2. During text classification phase,
we trained network with distinct learning rates. Our approach reported
encouraging results on recognition of Arabic characters from segmented Arabic
scene images.

negative:
Many historical map sheets are publicly available for studies that require
long-term historical geographic data. The cartographic design of these maps
includes a combination of map symbols and text labels. Automatically reading
text labels from map images could greatly speed up the map interpretation and
helps generate rich metadata describing the map content. Many text detection
algorithms have been proposed to locate text regions in map images
automatically, but most of the algorithms are trained on out-ofdomain datasets
(e.g., scenic images). Training data determines the quality of machine learning
models, and manually annotating text regions in map images is labor-extensive
and time-consuming. On the other hand, existing geographic data sources, such
as Open- StreetMap (OSM), contain machine-readable map layers, which allow us
to separate out the text layer and obtain text label annotations easily.
However, the cartographic styles between OSM map tiles and historical maps are
significantly different. This paper proposes a method to automatically generate
an unlimited amount of annotated historical map images for training text
detection models. We use a style transfer model to convert contemporary map
images into historical style and place text labels upon them. We show that the
state-of-the-art text detection models (e.g., PSENet) can benefit from the
synthetic historical maps and achieve significant improvement for historical
map text detection.

negative:
Many tasks are related to determining if a particular text string exists in
an image. In this work, we propose a new framework that learns this task in an
end-to-end way. The framework takes an image and a text string as input and
then outputs the probability of the text string being present in the image.
This is the first end-to-end framework that learns such relationships between
text and images in scene text area. The framework does not require explicit
scene text detection or recognition and thus no bounding box annotations are
needed for it. It is also the first work in scene text area that tackles suh a
weakly labeled problem. Based on this framework, we developed a model called
Guided Attention. Our designed model achieves much better results than several
state-of-the-art scene text reading based solutions for a challenging Street
View Business Matching task. The task tries to find correct business names for
storefront images and the dataset we collected for it is substantially larger,
and more challenging than existing scene text dataset. This new real-world task
provides a new perspective for studying scene text related problems. We also
demonstrate the uniqueness of our task via a comparison between our problem and
a typical Visual Question Answering problem.

negative:
In this paper, we investigate the problem of retrieving images from a
database based on a multi-modal (image-text) query. Specifically, the query
text prompts some modification in the query image and the task is to retrieve
images with the desired modifications. For instance, a user of an E-Commerce
platform is interested in buying a dress, which should look similar to her
friend's dress, but the dress should be of white color with a ribbon sash. In
this case, we would like the algorithm to retrieve some dresses with desired
modifications in the query dress. We propose an autoencoder based model,
ComposeAE, to learn the composition of image and text query for retrieving
images. We adopt a deep metric learning approach and learn a metric that pushes
composition of source image and text query closer to the target images. We also
propose a rotational symmetry constraint on the optimization problem. Our
approach is able to outperform the state-of-the-art method TIRG \cite{TIRG} on
three benchmark datasets, namely: MIT-States, Fashion200k and Fashion IQ. In
order to ensure fair comparison, we introduce strong baselines by enhancing
TIRG method. To ensure reproducibility of the results, we publish our code
here: \url{https://github.com/ecom-research/ComposeAE}.

negative:
Video text spotting is still an important research topic due to its various
real-applications. Previous approaches usually fall into the four-staged
pipeline: text detection in individual images, framewisely recognizing
localized text regions, tracking text streams and generating final results with
complicated post-processing skills, which might suffer from the huge
computational cost as well as the interferences of low-quality text. In this
paper, we propose a fast and robust video text spotting framework by only
recognizing the localized text one-time instead of frame-wisely recognition.
Specifically, we first obtain text regions in videos with a well-designed
spatial-temporal detector. Then we concentrate on developing a novel text
recommender for selecting the highest-quality text from text streams and only
recognizing the selected ones. Here, the recommender assembles text tracking,
quality scoring and recognition into an end-to-end trainable module, which not
only avoids the interferences from low-quality text but also dramatically
speeds up the video text spotting process. In addition, we collect a larger
scale video text dataset (LSVTD) for promoting the video text spotting
community, which contains 100 text videos from 22 different real-life
scenarios. Extensive experiments on two public benchmarks show that our method
greatly speeds up the recognition process averagely by 71 times compared with
the frame-wise manner, and also achieves the remarkable state-of-the-art.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
In this paper, we study the compositional learning of images and texts for
image retrieval. The query is given in the form of an image and text that
describes the desired modifications to the image; the goal is to retrieve the
target image that satisfies the given modifications and resembles the query by
composing information in both the text and image modalities. To remedy this, we
propose a novel architecture designed for the image-text composition task and
show that the proposed structure can effectively encode the differences between
the source and target images conditioned on the text. Furthermore, we introduce
a new joint training technique based on the graph convolutional network that is
generally applicable for any existing composition methods in a plug-and-play
manner. We found that the proposed technique consistently improves performance
and achieves state-of-the-art scores on various benchmarks. To avoid misleading
experimental results caused by trivial training hyper-parameters, we reproduce
all individual baselines and train models with a unified training environment.
We expect this approach to suppress undesirable effects from irrelevant
components and emphasize the image-text composition module's ability. Also, we
achieve the state-of-the-art score without restricting the training
environment, which implies the superiority of our method considering the gains
from hyper-parameter tuning. The code, including all the baseline methods, are
released https://github.com/nashory/rtic-gcn-pytorch.

negative:
Image foreground extraction is a classical problem in image processing and
vision, with a large range of applications. In this dissertation, we focus on
the extraction of text and graphics in mixed-content images, and design novel
approaches for various aspects of this problem.
  We first propose a sparse decomposition framework, which models the
background by a subspace containing smooth basis vectors, and foreground as a
sparse and connected component. We then formulate an optimization framework to
solve this problem, by adding suitable regularizations to the cost function to
promote the desired characteristics of each component. We present two
techniques to solve the proposed optimization problem, one based on alternating
direction method of multipliers (ADMM), and the other one based on robust
regression. Promising results are obtained for screen content image
segmentation using the proposed algorithm.
  We then propose a robust subspace learning algorithm for the representation
of the background component using training images that could contain both
background and foreground components, as well as noise. With the learnt
subspace for the background, we can further improve the segmentation results,
compared to using a fixed subspace. Lastly, we investigate a different class of
signal/image decomposition problem, where only one signal component is active
at each signal element. In this case, besides estimating each component, we
need to find their supports, which can be specified by a binary mask. We
propose a mixed-integer programming problem, that jointly estimates the two
components and their supports through an alternating optimization scheme. We
show the application of this algorithm on various problems, including image
segmentation, video motion segmentation, and also separation of text from
textured images.

negative:
Scene text is an important feature to be extracted, especially in
vision-based mobile robot navigation as many potential landmarks such as
nameplates and information signs contain text. In this paper, a novel two-step
text localization method for Indoor Mobile Robot Navigation is introduced. This
method is based on morphological operators and machine learning techniques and
can be used in real time environments. The proposed method has two steps. At
First, a new set of morphological operators is applied with a particular
sequence to extract high contrast areas that have high probability of text
existence. Using of morphological operators has many advantages such as: high
computation speed, being invariant to several geometrical transformations like
translation, rotations, and scaling, and being able to extract all areas
containing text. After extracting text candidate regions, a set of nine
features are extracted for accurate detection and deletion of the regions that
don't have text. These features are descriptors for texture properties and are
computed in real time. Then, we use a SVM classifier to detect the existence of
text in the region. Performance of the proposed algorithm is compared against a
number of widely used text localization algorithms and the results show that
this method can quickly and effectively localize and extract text regions from
real scenes and can be used in mobile robot navigation under an indoor
environment to detect text based landmarks.

negative:
Recognizing arbitrary multi-character text in unconstrained natural
photographs is a hard problem. In this paper, we address an equally hard
sub-problem in this domain viz. recognizing arbitrary multi-digit numbers from
Street View imagery. Traditional approaches to solve this problem typically
separate out the localization, segmentation, and recognition steps. In this
paper we propose a unified approach that integrates these three steps via the
use of a deep convolutional neural network that operates directly on the image
pixels. We employ the DistBelief implementation of deep neural networks in
order to train large, distributed neural networks on high quality images. We
find that the performance of this approach increases with the depth of the
convolutional network, with the best performance occurring in the deepest
architecture we trained, with eleven hidden layers. We evaluate this approach
on the publicly available SVHN dataset and achieve over $96\%$ accuracy in
recognizing complete street numbers. We show that on a per-digit recognition
task, we improve upon the state-of-the-art, achieving $97.84\%$ accuracy. We
also evaluate this approach on an even more challenging dataset generated from
Street View imagery containing several tens of millions of street number
annotations and achieve over $90\%$ accuracy. To further explore the
applicability of the proposed system to broader text recognition tasks, we
apply it to synthetic distorted text from reCAPTCHA. reCAPTCHA is one of the
most secure reverse turing tests that uses distorted text to distinguish humans
from bots. We report a $99.8\%$ accuracy on the hardest category of reCAPTCHA.
Our evaluations on both tasks indicate that at specific operating thresholds,
the performance of the proposed system is comparable to, and in some cases
exceeds, that of human operators.

negative:
Handwritten document images can be highly affected by degradation for
different reasons: Paper ageing, daily-life scenarios (wrinkles, dust, etc.),
bad scanning process and so on. These artifacts raise many readability issues
for current Handwritten Text Recognition (HTR) algorithms and severely devalue
their efficiency. In this paper, we propose an end to end architecture based on
Generative Adversarial Networks (GANs) to recover the degraded documents into a
clean and readable form. Unlike the most well-known document binarization
methods, which try to improve the visual quality of the degraded document, the
proposed architecture integrates a handwritten text recognizer that promotes
the generated document image to be more readable. To the best of our knowledge,
this is the first work to use the text information while binarizing handwritten
documents. Extensive experiments conducted on degraded Arabic and Latin
handwritten documents demonstrate the usefulness of integrating the recognizer
within the GAN architecture, which improves both the visual quality and the
readability of the degraded document images. Moreover, we outperform the state
of the art in H-DIBCO challenges, after fine tuning our pre-trained model with
synthetically degraded Latin handwritten images, on this task.

negative:
This paper presents results of Document Visual Question Answering Challenge
organized as part of "Text and Documents in the Deep Learning Era" workshop, in
CVPR 2020. The challenge introduces a new problem - Visual Question Answering
on document images. The challenge comprised two tasks. The first task concerns
with asking questions on a single document image. On the other hand, the second
task is set as a retrieval task where the question is posed over a collection
of images. For the task 1 a new dataset is introduced comprising 50,000
questions-answer(s) pairs defined over 12,767 document images. For task 2
another dataset has been created comprising 20 questions over 14,362 document
images which share the same document template.

negative:
Line separators are used to segregate text-lines from one another in document
image analysis. Finding the separator points at every line terminal in a
document image would enable text-line segmentation. In particular, identifying
the separators in handwritten text could be a thrilling exercise. Obviously it
would be challenging to perform this in the compressed version of a document
image and that is the proposed objective in this research. Such an effort would
prevent the computational burden of decompressing a document for text-line
segmentation. Since document images are generally compressed using run length
encoding (RLE) technique as per the CCITT standards, the first column in the
RLE will be a white column. The value (depth) in the white column is very low
when a particular line is a text line and the depth could be larger at the
point of text line separation. A longer consecutive sequence of such larger
depth should indicate the gap between the text lines, which provides the
separator region. In case of over separation and under separation issues,
corrective actions such as deletion and insertion are suggested respectively.
An extensive experimentation is conducted on the compressed images of the
benchmark datasets of ICDAR13 and Alireza et al [17] to demonstrate the
efficacy.

negative:
We introduce FontCode, an information embedding technique for text documents.
Provided a text document with specific fonts, our method embeds user-specified
information in the text by perturbing the glyphs of text characters while
preserving the text content. We devise an algorithm to chooses unobtrusive yet
machine-recognizable glyph perturbations, leveraging a recently developed
generative model that alters the glyphs of each character continuously on a
font manifold. We then introduce an algorithm that embeds a user-provided
message in the text document and produces an encoded document whose appearance
is minimally perturbed from the original document. We also present a glyph
recognition method that recovers the embedded information from an encoded
document stored as a vector graphic or pixel image, or even on a printed paper.
In addition, we introduce a new error-correction coding scheme that rectifies a
certain number of recognition errors. Lastly, we demonstrate that our technique
enables a wide array of applications, using it as a text document metadata
holder, an unobtrusive optical barcode, a cryptographic message embedding
scheme, and a text document signature.

negative:
Document segmentation is a method of rending the document into distinct
regions. A document is an assortment of information and a standard mode of
conveying information to others. Pursuance of data from documents involves ton
of human effort, time intense and might severely prohibit the usage of data
systems. So, automatic information pursuance from the document has become a big
issue. It is been shown that document segmentation will facilitate to beat such
problems. This paper proposes a new approach to segment and classify the
document regions as text, image, drawings and table. Document image is divided
into blocks using Run length smearing rule and features are extracted from
every blocks. Discipulus tool has been used to construct the Genetic
programming based classifier model and located 97.5% classification accuracy.

negative:
In natural scenes and documents, we can find the correlation between a text
and its color. For instance, the word, "hot", is often printed in red, while
"cold" is often in blue. This correlation can be thought of as a feature that
represents the semantic difference between the words. Based on this
observation, we propose the idea of using text color for word embeddings. While
text-only word embeddings (e.g. word2vec) have been extremely successful, they
often represent antonyms as similar since they are often interchangeable in
sentences. In this paper, we try two tasks to verify the usefulness of text
color in understanding the meanings of words, especially in identifying
synonyms and antonyms. First, we quantify the color distribution of words from
the book cover images and analyze the correlation between the color and meaning
of the word. Second, we try to retrain word embeddings with the color
distribution of words as a constraint. By observing the changes in the word
embeddings of synonyms and antonyms before and after re-training, we aim to
understand the kind of words that have positive or negative effects in their
word embeddings when incorporating text color information.

negative:
This paper describes the COCO-Text dataset. In recent years large-scale
datasets like SUN and Imagenet drove the advancement of scene understanding and
object recognition. The goal of COCO-Text is to advance state-of-the-art in
text detection and recognition in natural images. The dataset is based on the
MS COCO dataset, which contains images of complex everyday scenes. The images
were not collected with text in mind and thus contain a broad variety of text
instances. To reflect the diversity of text in natural scenes, we annotate text
with (a) location in terms of a bounding box, (b) fine-grained classification
into machine printed text and handwritten text, (c) classification into legible
and illegible text, (d) script of the text and (e) transcriptions of legible
text. The dataset contains over 173k text annotations in over 63k images. We
provide a statistical analysis of the accuracy of our annotations. In addition,
we present an analysis of three leading state-of-the-art photo Optical
Character Recognition (OCR) approaches on our dataset. While scene text
detection and recognition enjoys strong advances in recent years, we identify
significant shortcomings motivating future work.

negative:
We present a novel data generation tool for document processing. The tool
focuses on providing a maximal level of visual information in a normal type
document, ranging from character position to paragraph-level position. It also
enables working with a large dataset on low-resource languages as well as
providing a mean of processing thorough full-level information of the
documented text. The data generation tools come with a dataset of 320000
Vietnamese synthetic document images and an instruction to generate a dataset
of similar size in other languages. The repository can be found at:
https://github.com/tson1997/SDL-Document-Image-Generation

negative:
To understand how people look, interact, or perform tasks, we need to quickly
and accurately capture their 3D body, face, and hands together from an RGB
image. Most existing methods focus only on parts of the body. A few recent
approaches reconstruct full expressive 3D humans from images using 3D body
models that include the face and hands. These methods are optimization-based
and thus slow, prone to local optima, and require 2D keypoints as input. We
address these limitations by introducing ExPose (EXpressive POse and Shape
rEgression), which directly regresses the body, face, and hands, in SMPL-X
format, from an RGB image. This is a hard problem due to the high
dimensionality of the body and the lack of expressive training data.
Additionally, hands and faces are much smaller than the body, occupying very
few image pixels. This makes hand and face estimation hard when body images are
downscaled for neural networks. We make three main contributions. First, we
account for the lack of training data by curating a dataset of SMPL-X fits on
in-the-wild images. Second, we observe that body estimation localizes the face
and hands reasonably well. We introduce body-driven attention for face and hand
regions in the original image to extract higher-resolution crops that are fed
to dedicated refinement modules. Third, these modules exploit part-specific
knowledge from existing face- and hand-only datasets. ExPose estimates
expressive 3D humans more accurately than existing optimization methods at a
small fraction of the computational cost. Our data, model and code are
available for research at https://expose.is.tue.mpg.de .

negative:
Most existing text reading benchmarks make it difficult to evaluate the
performance of more advanced deep learning models in large vocabularies due to
the limited amount of training data. To address this issue, we introduce a new
large-scale text reading benchmark dataset named Chinese Street View Text
(C-SVT) with 430,000 street view images, which is at least 14 times as large as
the existing Chinese text reading benchmarks. To recognize Chinese text in the
wild while keeping large-scale datasets labeling cost-effective, we propose to
annotate one part of the CSVT dataset (30,000 images) in locations and text
labels as full annotations and add 400,000 more images, where only the
corresponding text-of-interest in the regions is given as weak annotations. To
exploit the rich information from the weakly annotated data, we design a text
reading network in a partially supervised learning framework, which enables to
localize and recognize text, learn from fully and weakly annotated data
simultaneously. To localize the best matched text proposals from weakly labeled
images, we propose an online proposal matching module incorporated in the whole
model, spotting the keyword regions by sharing parameters for end-to-end
training. Compared with fully supervised training algorithms, this model can
improve the end-to-end recognition performance remarkably by 4.03% in F-score
at the same labeling cost. The proposed model can also achieve state-of-the-art
results on the ICDAR 2017-RCTW dataset, which demonstrates the effectiveness of
the proposed partially supervised learning framework.

negative:
Citizen engagement and technology usage are two emerging trends driven by
smart city initiatives. Governments around the world are adopting technology
for faster resolution of civic issues. Typically, citizens report issues, such
as broken roads, garbage dumps, etc. through web portals and mobile apps, in
order for the government authorities to take appropriate actions. Several
mediums -- text, image, audio, video -- are used to report these issues.
Through a user study with 13 citizens and 3 authorities, we found that image is
the most preferred medium to report civic issues. However, analyzing civic
issue related images is challenging for the authorities as it requires manual
effort. Moreover, previous works have been limited to identifying a specific
set of issues from images. In this work, given an image, we propose to generate
a Civic Issue Graph consisting of a set of objects and the semantic relations
between them, which are representative of the underlying civic issue. We also
release two multi-modal (text and images) datasets, that can help in further
analysis of civic issues from images. We present a novel approach for
adversarial training of existing scene graph models that enables the use of
scene graphs for new applications in the absence of any labelled training data.
We conduct several experiments to analyze the efficacy of our approach, and
using human evaluation, we establish the appropriateness of our model at
representing different civic issues.

negative:
The camera captured images have various aspects to investigate. Generally,
the emphasis of research depends on the interesting regions. Sometimes the
focus could be on color segmentation, object detection or scene text analysis.
The image analysis, visibility and layout analysis are the tasks easier for
humans as suggested by behavioral trait of humans, but in contrast when these
same tasks are supposed to perform by machines then it seems to be challenging.
The learning machines always learn from the properties associated to provided
samples. The numerous approaches are designed in recent years for scene text
extraction and recognition and the efforts are underway to improve the
accuracy. The convolutional approach provided reasonable results on non-cursive
text analysis appeared in natural images. The work presented in this manuscript
exploited the strength of linear pyramids by considering each pyramid as a
feature of the provided sample. Each pyramid image process through various
empirically selected kernels. The performance was investigated by considering
Arabic text on each image pyramid of EASTR-42k dataset. The error rate of 0.17%
was reported on Arabic scene text recognition.

negative:
Neural Networks and Deep Learning have seen an upsurge of research in the
past decade due to the improved results. Generates text from the given image is
a crucial task that requires the combination of both sectors which are computer
vision and natural language processing in order to understand an image and
represent it using a natural language. However existing works have all been
done on a particular lingual domain and on the same set of data. This leads to
the systems being developed to perform poorly on images that belong to specific
locales' geographical context. TextMage is a system that is capable of
understanding visual scenes that belong to the Bangladeshi geographical context
and use its knowledge to represent what it understands in Bengali. Hence, we
have trained a model on our previously developed and published dataset named
BanglaLekhaImageCaptions. This dataset contains 9,154 images along with two
annotations for each image. In order to access performance, the proposed model
has been implemented and evaluated.

negative:
The text-independent approach to writer identification does not require the
writer to write some predetermined text. Previous research on text-independent
writer identification has been based on identifying writer-specific features
designed by experts. However, in the last decade, deep learning methods have
been successfully applied to learn features from data automatically. We propose
here an end-to-end deep-learning method for text-independent writer
identification that does not require prior identification of features. A
Convolutional Neural Network (CNN) is trained initially to extract local
features, which represent characteristics of individual handwriting in the
whole character images and their sub-regions. Randomly sampled tuples of images
from the training set are used to train the CNN and aggregate the extracted
local features of images from the tuples to form global features. For every
training epoch, the process of randomly sampling tuples is repeated, which is
equivalent to a large number of training patterns being prepared for training
the CNN for text-independent writer identification. We conducted experiments on
the JEITA-HP database of offline handwritten Japanese character patterns. With
200 characters, our method achieved an accuracy of 99.97% to classify 100
writers. Even when using 50 characters for 100 writers or 100 characters for
400 writers, our method achieved accuracy levels of 92.80% or 93.82%,
respectively. We conducted further experiments on the Firemaker and IAM
databases of offline handwritten English text. Using only one page per writer
to train, our method achieved over 91.81% accuracy to classify 900 writers.
Overall, we achieved a better performance than the previously published best
result based on handcrafted features and clustering algorithms, which
demonstrates the effectiveness of our method for handwritten English text also.

negative:
Video game genre classification based on its cover and textual description
would be utterly beneficial to many modern identification, collocation, and
retrieval systems. At the same time, it is also an extremely challenging task
due to the following reasons: First, there exists a wide variety of video game
genres, many of which are not concretely defined. Second, video game covers
vary in many different ways such as colors, styles, textual information, etc,
even for games of the same genre. Third, cover designs and textual descriptions
may vary due to many external factors such as country, culture, target reader
populations, etc. With the growing competitiveness in the video game industry,
the cover designers and typographers push the cover designs to its limit in the
hope of attracting sales. The computer-based automatic video game genre
classification systems become a particularly exciting research topic in recent
years. In this paper, we propose a multi-modal deep learning framework to solve
this problem. The contribution of this paper is four-fold. First, we compiles a
large dataset consisting of 50,000 video games from 21 genres made of cover
images, description text, and title text and the genre information. Second,
image-based and text-based, state-of-the-art models are evaluated thoroughly
for the task of genre classification for video games. Third, we developed an
efficient and salable multi-modal framework based on both images and texts.
Fourth, a thorough analysis of the experimental results is given and future
works to improve the performance is suggested. The results show that the
multi-modal framework outperforms the current state-of-the-art image-based or
text-based models. Several challenges are outlined for this task. More efforts
and resources are needed for this classification task in order to reach a
satisfactory level.

negative:
We model the process of human full interpretation of object images, namely
the ability to identify and localize all semantic features and parts that are
recognized by human observers. The task is approached by dividing the
interpretation of the complete object to the interpretation of multiple reduced
but interpretable local regions. We model interpretation by a structured
learning framework, in which there are primitive components and relations that
play a useful role in local interpretation by humans. To identify useful
components and relations used in the interpretation process, we consider the
interpretation of minimal configurations, namely reduced local regions that are
minimal in the sense that further reduction will turn them unrecognizable and
uninterpretable. We show experimental results of our model, and results of
predicting and testing relations that were useful to the model via transformed
minimal images.

negative:
One of the major prerequisites for any deep learning approach is the
availability of large-scale training data. When dealing with scanned document
images in real world scenarios, the principal information of its content is
stored in the layout itself. In this work, we have proposed an automated deep
generative model using Graph Neural Networks (GNNs) to generate synthetic data
with highly variable and plausible document layouts that can be used to train
document interpretation systems, in this case, specially in digital mailroom
applications. It is also the first graph-based approach for document layout
generation task experimented on administrative document images, in this case,
invoices.

negative:
We present in this paper a new approach for the automatic annotation of
medical images, using the approach of "bag-of-words" to represent the visual
content of the medical image combined with text descriptors based approach
tf.idf and reduced by latent semantic to extract the co-occurrence between
terms and visual terms. A medical report is composed of a text describing a
medical image. First, we are interested to index the text and extract all
relevant terms using a thesaurus containing MeSH medical concepts. In a second
phase, the medical image is indexed while recovering areas of interest which
are invariant to change in scale, light and tilt. To annotate a new medical
image, we use the approach of "bagof-words" to recover the feature vector.
Indeed, we use the vector space model to retrieve similar medical image from
the database training. The calculation of the relevance value of an image to
the query image is based on the cosine function. We conclude with an experiment
carried out on five types of radiological imaging to evaluate the performance
of our system of medical annotation. The results showed that our approach works
better with more images from the radiology of the skull.

negative:
In recent scene recognition research images or large image regions are often
represented as disorganized "bags" of features which can then be analyzed using
models originally developed to capture co-variation of word counts in text.
However, image feature counts are likely to be constrained in different ways
than word counts in text. For example, as a camera pans upwards from a building
entrance over its first few floors and then further up into the sky Fig. 1,
some feature counts in the image drop while others rise -- only to drop again
giving way to features found more often at higher elevations. The space of all
possible feature count combinations is constrained both by the properties of
the larger scene and the size and the location of the window into it. To
capture such variation, in this paper we propose the use of the counting grid
model. This generative model is based on a grid of feature counts, considerably
larger than any of the modeled images, and considerably smaller than the real
estate needed to tile the images next to each other tightly. Each modeled image
is assumed to have a representative window in the grid in which the feature
counts mimic the feature distribution in the image. We provide a learning
procedure that jointly maps all images in the training set to the counting grid
and estimates the appropriate local counts in it. Experimentally, we
demonstrate that the resulting representation captures the space of feature
count combinations more accurately than the traditional models, not only when
the input images come from a panning camera, but even when modeling images of
different scenes from the same category.

negative:
Identity documents recognition is an important sub-field of document
analysis, which deals with tasks of robust document detection, type
identification, text fields recognition, as well as identity fraud prevention
and document authenticity validation given photos, scans, or video frames of an
identity document capture. Significant amount of research has been published on
this topic in recent years, however a chief difficulty for such research is
scarcity of datasets, due to the subject matter being protected by security
requirements. A few datasets of identity documents which are available lack
diversity of document types, capturing conditions, or variability of document
field values. In addition, the published datasets were typically designed only
for a subset of document recognition problems, not for a complex identity
document analysis. In this paper, we present a dataset MIDV-2020 which consists
of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock
identity documents, each with unique text field values and unique artificially
generated faces, with rich annotation. For the presented benchmark dataset
baselines are provided for such tasks as document location and identification,
text fields recognition, and face detection. With 72409 annotated images in
total, to the date of publication the proposed dataset is the largest publicly
available identity documents dataset with variable artificially generated data,
and we believe that it will prove invaluable for advancement of the field of
document analysis and recognition. The dataset is available for download at
ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr .

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Traditionally, text generation models take in a sequence of text as input,
and iteratively generate the next most probable word using pre-trained
parameters. In this work, we propose the architecture to use images instead of
text as the input of the text generation model, called StoryGen. In the
architecture, we design a Relational Text Data Generator algorithm that relates
different features from multiple images. The output samples from the model
demonstrate the ability to generate meaningful paragraphs of text containing
the extracted features from the input images. This is an undergraduate project
report. Completed Dec. 2019 at the Cooper Union.

negative:
In this paper we introduce a new method for text detection in natural images.
The method comprises two contributions: First, a fast and scalable engine to
generate synthetic images of text in clutter. This engine overlays synthetic
text to existing background images in a natural way, accounting for the local
3D scene geometry. Second, we use the synthetic images to train a
Fully-Convolutional Regression Network (FCRN) which efficiently performs text
detection and bounding-box regression at all locations and multiple scales in
an image. We discuss the relation of FCRN to the recently-introduced YOLO
detector, as well as other end-to-end object detection systems based on deep
learning. The resulting detection network significantly out performs current
methods for text detection in natural images, achieving an F-measure of 84.2%
on the standard ICDAR 2013 benchmark. Furthermore, it can process 15 images per
second on a GPU.

negative:
The continually increasing number of documents produced each year
necessitates ever improving information processing methods for searching,
retrieving, and organizing text. Central to these information processing
methods is document classification, which has become an important application
for supervised learning. Recently the performance of these traditional
classifiers has degraded as the number of documents has increased. This is
because along with this growth in the number of documents has come an increase
in the number of categories. This paper approaches this problem differently
from current document classification methods that view the problem as
multi-class classification. Instead we perform hierarchical classification
using an approach we call Hierarchical Deep Learning for Text classification
(HDLTex). HDLTex employs stacks of deep learning architectures to provide
specialized understanding at each level of the document hierarchy.

negative:
Digital camera and mobile document image acquisition are new trends arising
in the world of Optical Character Recognition and text detection. In some
cases, such process integrates many distortions and produces poorly scanned
text or text-photo images and natural images, leading to an unreliable OCR
digitization. In this paper, we present a novel nonparametric and unsupervised
method to compensate for undesirable document image distortions aiming to
optimally improve OCR accuracy. Our approach relies on a very efficient stack
of document image enhancing techniques to recover deformation of the entire
document image. First, we propose a local brightness and contrast adjustment
method to effectively handle lighting variations and the irregular distribution
of image illumination. Second, we use an optimized greyscale conversion
algorithm to transform our document image to greyscale level. Third, we sharpen
the useful information in the resulting greyscale image using Un-sharp Masking
method. Finally, an optimal global binarization approach is used to prepare the
final document image to OCR recognition. The proposed approach can
significantly improve text detection rate and optical character recognition
accuracy. To demonstrate the efficiency of our approach, an exhaustive
experimentation on a standard dataset is presented.

negative:
Computer vision systems for wood identification have the potential to empower
both producer and consumer countries to combat illegal logging if they can be
deployed effectively in the field. In this work, carried out as part of an
active international partnership with the support of UNIDO, we constructed and
curated a field-relevant image data set to train a classifier for wood
identification of $15$ commercial Ghanaian woods using the XyloTron system. We
tested model performance in the laboratory, and then collected real-world field
performance data across multiple sites using multiple XyloTron devices. We
present efficacies of the trained model in the laboratory and in the field,
discuss practical implications and challenges of deploying machine learning
wood identification models, and conclude that field testing is a necessary step
- and should be considered the gold-standard - for validating computer vision
wood identification systems.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.

negative:
This work presents a method for visual text recognition without using any
paired supervisory data. We formulate the text recognition task as one of
aligning the conditional distribution of strings predicted from given text
images, with lexically valid strings sampled from target corpora. This enables
fully automated, and unsupervised learning from just line-level text-images,
and unpaired text-string samples, obviating the need for large aligned
datasets. We present detailed analysis for various aspects of the proposed
method, namely - (1) impact of the length of training sequences on convergence,
(2) relation between character frequencies and the order in which they are
learnt, (3) generalisation ability of our recognition network to inputs of
arbitrary lengths, and (4) impact of varying the text corpus on recognition
accuracy. Finally, we demonstrate excellent text recognition accuracy on both
synthetically generated text images, and scanned images of real printed books,
using no labelled training examples.

negative:
In this paper we introduce a script identification method based on
hand-crafted texture features and an artificial neural network. The proposed
pipeline achieves near state-of-the-art performance for script identification
of video-text and state-of-the-art performance on visual language
identification of handwritten text. More than using the deep network as a
classifier, the use of its intermediary activations as a learned metric
demonstrates remarkable results and allows the use of discriminative models on
unknown classes. Comparative experiments in video-text and text in the wild
datasets provide insights on the internals of the proposed deep network.

negative:
Recovering the original printed texts from a document with added handwritten
annotations in the marginal area is one of the challenging problems, especially
when the original document is not available. Therefore, this paper aims at
salvaging automatically the original document from the annotated document by
detecting and removing any handwritten annotations that appear in the marginal
area of the document without any loss of information. Here a two stage
algorithm is proposed, where in the first stage due to approximate marginal
boundary detection with horizontal and vertical projection profiles, all of the
marginal annotations along with some part of the original printed text that may
appear very close to the marginal boundary are removed. Therefore as a second
stage, using the connected components, a strategy is applied to bring back the
printed text components cropped during the first stage. The proposed method is
validated using a dataset of 50 documents having complex handwritten
annotations, which gives an overall accuracy of 89.01% in removing the marginal
annotations and 97.74% in case of retrieving the original printed text
document.

negative:
Everyone knows that thousand of words are represented by a single image. As a
result image search has become a very popular mechanism for the Web searchers.
Image search means, the search results are produced by the search engine should
be a set of images along with their Web page Unified Resource Locator. Now Web
searcher can perform two types of image search, they are Text to Image and
Image to Image search. In Text to Image search, search query should be a text.
Based on the input text data system will generate a set of images along with
their Web page URL as an output. On the other hand, in Image to Image search,
search query should be an image and based on this image system will generate a
set of images along with their Web page URL as an output. According to the
current scenarios, Text to Image search mechanism always not returns perfect
result. It matches the text data and then displays the corresponding images as
an output, which is not always perfect. To resolve this problem, Web
researchers have introduced the Image to Image search mechanism. In this paper,
we have also proposed an alternate approach of Image to Image search mechanism
using Histogram.

negative:
Segmentation of a text-document into lines, words and characters, which is
considered to be the crucial pre-processing stage in Optical Character
Recognition (OCR) is traditionally carried out on uncompressed documents,
although most of the documents in real life are available in compressed form,
for the reasons such as transmission and storage efficiency. However, this
implies that the compressed image should be decompressed, which indents
additional computing resources. This limitation has motivated us to take up
research in document image analysis using compressed documents. In this paper,
we think in a new way to carry out segmentation at line, word and character
level in run-length compressed printed-text-documents. We extract the
horizontal projection profile curve from the compressed file and using the
local minima points perform line segmentation. However, tracing vertical
information which leads to tracking words-characters in a run-length compressed
file is not very straight forward. Therefore, we propose a novel technique for
carrying out simultaneous word and character segmentation by popping out column
runs from each row in an intelligent sequence. The proposed algorithms have
been validated with 1101 text-lines, 1409 words and 7582 characters from a
data-set of 35 noise and skew free compressed documents of Bengali, Kannada and
English Scripts.

negative:
Recognition of document images have important applications in restoring old
and classical texts. The problem involves quality improvement before passing it
to a properly trained OCR to get accurate recognition of the text. The image
enhancement and quality improvement constitute important steps as subsequent
recognition depends upon the quality of the input image. There are scenarios
when high resolution images are not available and our experiments show that the
OCR accuracy reduces significantly with decrease in the spatial resolution of
document images. Thus the only option is to improve the resolution of such
document images. The goal is to construct a high resolution image, given a
single low resolution binary image, which constitutes the problem of single
image super-resolution. Most of the previous work in super-resolution deal with
natural images which have more information-content than the document images.
Here, we use Convolution Neural Network to learn the mapping between low and
the corresponding high resolution images. We experiment with different number
of layers, parameter settings and non-linear functions to build a fast
end-to-end framework for document image super-resolution. Our proposed model
shows a very good PSNR improvement of about 4 dB on 75 dpi Tamil images,
resulting in a 3 % improvement of word level accuracy by the OCR. It takes less
time than the recent sparse based natural image super-resolution technique,
making it useful for real-time document recognition applications.

negative:
Part information has been shown to be resistant to occlusions and viewpoint
changes, which is beneficial for various vision-related tasks. However, we
found very limited work in car pose estimation and reconstruction from street
views leveraging the part information. There are two major contributions in
this paper. Firstly, we make the first attempt to build a framework to
simultaneously estimate shape, translation, orientation, and semantic parts of
cars in 3D space from a single street view. As it is labor-intensive to
annotate semantic parts on real street views, we propose a specific approach to
implicitly transfer part features from synthesized images to real street views.
For pose and shape estimation, we propose a novel network structure that
utilizes both part features and 3D losses. Secondly, we are the first to
construct a high-quality dataset that contains 348 different car models with
physical dimensions and part-level annotations based on global and local
deformations. Given these models, we further generate 60K synthesized images
with randomization of orientation, illumination, occlusion, and texture. Our
results demonstrate that our part segmentation performance is significantly
improved after applying our implicit transfer approach. Our network for pose
and shape estimation achieves the state-of-the-art performance on the
ApolloCar3D dataset and outperforms 3D-RCNN and DeepMANTA by 12.57 and 8.91
percentage points in terms of mean A3DP-Abs.

negative:
Synthesising photo-realistic images from natural language is one of the
challenging problems in computer vision. Over the past decade, a number of
approaches have been proposed, of which the improved Stacked Generative
Adversarial Network (StackGAN-v2) has proven capable of generating high
resolution images that reflect the details specified in the input text
descriptions. In this paper, we aim to assess the robustness and
fault-tolerance capability of the StackGAN-v2 model by introducing variations
in the training data. However, due to the working principle of Generative
Adversarial Network (GAN), it is difficult to predict the output of the model
when the training data are modified. Hence, in this work, we adopt Metamorphic
Testing technique to evaluate the robustness of the model with a variety of
unexpected training dataset. As such, we first implement StackGAN-v2 algorithm
and test the pre-trained model provided by the original authors to establish a
ground truth for our experiments. We then identify a metamorphic relation, from
which test cases are generated. Further, metamorphic relations were derived
successively based on the observations of prior test results. Finally, we
synthesise the results from our experiment of all the metamorphic relations and
found that StackGAN-v2 algorithm is susceptible to input images with obtrusive
objects, even if it overlaps with the main object minimally, which was not
reported by the authors and users of StackGAN-v2 model. The proposed
metamorphic relations can be applied to other text-to-image synthesis models to
not only verify the robustness but also to help researchers understand and
interpret the results made by the machine learning models.

negative:
One of the major challenges in training text-to-image generation models is
the need of a large number of high-quality image-text pairs. While image
samples are often easily accessible, the associated text descriptions typically
require careful human captioning, which is particularly time- and
cost-consuming. In this paper, we propose the first work to train text-to-image
generation models without any text data. Our method leverages the well-aligned
multi-modal semantic space of the powerful pre-trained CLIP model: the
requirement of text-conditioning is seamlessly alleviated via generating text
features from image features. Extensive experiments are conducted to illustrate
the effectiveness of the proposed method. We obtain state-of-the-art results in
the standard text-to-image generation tasks. Importantly, the proposed
language-free model outperforms most existing models trained with full
image-text pairs. Furthermore, our method can be applied in fine-tuning
pre-trained models, which saves both training time and cost in training
text-to-image generation models. Our pre-trained model obtains competitive
results in zero-shot text-to-image generation on the MS-COCO dataset, yet with
around only 1% of the model size and training data size relative to the
recently proposed large DALL-E model.

negative:
Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.

negative:
Robust text reading from street view images provides valuable information for
various applications. Performance improvement of existing methods in such a
challenging scenario heavily relies on the amount of fully annotated training
data, which is costly and in-efficient to obtain. To scale up the amount of
training data while keeping the labeling procedure cost-effective, this
competition introduces a new challenge on Large-scale Street View Text with
Partial Labeling (LSVT), providing 50, 000 and 400, 000 images in full and weak
annotations, respectively. This competition aims to explore the abilities of
state-of-the-art methods to detect and recognize text instances from
large-scale street view images, closing the gap between research benchmarks and
real applications. During the competition period, a total of 41 teams
participated in the two proposed tasks with 132 valid submissions, i.e., text
detection and end-to-end text spotting. This paper includes dataset
descriptions, task definitions, evaluation protocols and results summaries of
the ICDAR 2019-LSVT challenge.

negative:
The StarLight program conceptualizes fast interstellar travel via small wafer
satellites (wafersats) that are propelled by directed energy. This process is
wildly different from traditional space travel and trades large and slow
spacecraft for small, fast, inexpensive, and fragile ones. The main goal of
these wafer satellites is to gather useful images during their deep space
journey. We introduce and solve some of the main problems that accompany this
concept. First, we need an object detection system that can detect planets that
we have never seen before, some containing features that we may not even know
exist in the universe. Second, once we have images of exoplanets, we need a way
to take these images and rank them by importance. Equipment fails and data
rates are slow, thus we need a method to ensure that the most important images
to humankind are the ones that are prioritized for data transfer. Finally, the
energy on board is minimal and must be conserved and used sparingly. No
exoplanet images should be missed, but using energy erroneously would be
detrimental. We introduce simulator-based methods that leverage artificial
intelligence, mostly in the form of computer vision, in order to solve all
three of these issues. Our results confirm that simulators provide an extremely
rich training environment that surpasses that of real images, and can be used
to train models on features that have yet to be observed by humans. We also
show that the immersive and adaptable environment provided by the simulator,
combined with deep learning, lets us navigate and save energy in an otherwise
implausible way.

negative:
The existing lensless compressive camera
($\text{L}^2\text{C}^2$)~\cite{Huang13ICIP} suffers from low capture rates,
resulting in low resolution images when acquired over a short time. In this
work, we propose a new regime to mitigate these drawbacks. We replace the
global-based compressive sensing used in the existing $\text{L}^2\text{C}^2$ by
the local block (patch) based compressive sensing. We use a single sensor for
each block, rather than for the entire image, thus forming a multiple but
spatially parallel sensor $\text{L}^2\text{C}^2$. This new camera retains the
advantages of existing $\text{L}^2\text{C}^2$ while leading to the following
additional benefits: 1) Since each block can be very small, {\em e.g.}$~8\times
8$ pixels, we only need to capture $\sim 10$ measurements to achieve reasonable
reconstruction. Therefore the capture time can be reduced significantly. 2) The
coding patterns used in each block can be the same, therefore the sensing
matrix is only of the block size compared to the entire image size in existing
$\text{L}^2\text{C}^2$. This saves the memory requirement of the sensing matrix
as well as speeds up the reconstruction. 3) Patch based image reconstruction is
fast and since real time stitching algorithms exist, we can perform real time
reconstruction. 4) These small blocks can be integrated to any desirable
number, leading to ultra high resolution images while retaining fast capture
rate and fast reconstruction. We develop multiple geometries of this block-wise
$\text{L}^2\text{C}^2$ in this paper. We have built prototypes of the proposed
block-wise $\text{L}^2\text{C}^2$ and demonstrated excellent results of real
data.

negative:
The goal of cross-view image based geo-localization is to determine the
location of a given street view image by matching it against a collection of
geo-tagged satellite images. This task is notoriously challenging due to the
drastic viewpoint and appearance differences between the two domains. We show
that we can address this discrepancy explicitly by learning to synthesize
realistic street views from satellite inputs. Following this observation, we
propose a novel multi-task architecture in which image synthesis and retrieval
are considered jointly. The rationale behind this is that we can bias our
network to learn latent feature representations that are useful for retrieval
if we utilize them to generate images across the two input domains. To the best
of our knowledge, ours is the first approach that creates realistic street
views from satellite images and localizes the corresponding query street-view
simultaneously in an end-to-end manner. In our experiments, we obtain
state-of-the-art performance on the CVUSA and CVACT benchmarks. Finally, we
show compelling qualitative results for satellite-to-street view synthesis.

negative:
Deep Convolutional Neural Networks (DCNN) have been proven to be effective
for various computer vision problems. In this work, we demonstrate its
effectiveness on a continuous object orientation estimation task, which
requires prediction of 0 to 360 degrees orientation of the objects. We do so by
proposing and comparing three continuous orientation prediction approaches
designed for the DCNNs. The first two approaches work by representing an
orientation as a point on a unit circle and minimizing either L2 loss or
angular difference loss. The third method works by first converting the
continuous orientation estimation task into a set of discrete orientation
estimation tasks and then converting the discrete orientation outputs back to
the continuous orientation using a mean-shift algorithm. By evaluating on a
vehicle orientation estimation task and a pedestrian orientation estimation
task, we demonstrate that the discretization-based approach not only works
better than the other two approaches but also achieves state-of-the-art
performance. We also demonstrate that finding an appropriate feature
representation is critical to achieve a good performance when adapting a DCNN
trained for an image recognition task.