query: I am currently working on a project that involves recognizing text in images, utilizing the attention-based encoder-decoder framework. However, I am encountering difficulties when the system misinterprets words with extra or missing characters in the text, which complicates the training process and reduces accuracy. I am in search of a model capable of assigning probabilities to words within the images, specifically regarding their likelihood of containing missing or superfluous characters. Additionally, I require techniques that can be integrated into the existing framework. Ideally, these techniques would enable the framework to identify potential spelling errors and attempt to restore the original, correct word. I anticipate that the implementation of such a model and techniques will enhance the performance of my text recognition framework. To test this, I am also seeking datasets that contain spelling errors. I would appreciate any guidance on how to effectively train the model with these datasets and how to evaluate the models’ performance in a reliable way.

positive:
Scanned documents in electronic health records (EHR) have been a challenge
for decades, and are expected to stay in the foreseeable future. Current
approaches for processing often include image preprocessing, optical character
recognition (OCR), and text mining. However, there is limited work that
evaluates the choice of image preprocessing methods, the selection of NLP
models, and the role of document layout. The impact of each element remains
unknown. We evaluated this method on a use case of two key indicators for sleep
apnea, Apnea hypopnea index (AHI) and oxygen saturation (SaO2) values, from
scanned sleep study reports. Our data that included 955 manually annotated
reports was secondarily utilized from a previous study in the University of
Texas Medical Branch. We performed image preprocessing: gray-scaling followed
by 1 iteration of dilating and erode, and 20% contrast increasing. The OCR was
implemented with the Tesseract OCR engine. A total of seven Bag-of-Words models
(Logistic Regression, Ridge Regression, Lasso Regression, Support Vector
Machine, k-Nearest Neighbor, Na\"ive Bayes, and Random Forest) and three deep
learning-based models (BiLSTM, BERT, and Clinical BERT) were evaluated. We also
evaluated the combinations of image preprocessing methods (gray-scaling, dilate
& erode, increased contrast by 20%, increased contrast by 60%), and two deep
learning architectures (with and without structured input that provides
document layout information). Our proposed method using Clinical BERT reached
an AUROC of 0.9743 and document accuracy of 94.76% for AHI, and an AUROC of
0.9523, and document accuracy of 91.61% for SaO2. We demonstrated the proper
use of image preprocessing and document layout could be beneficial to scanned
document processing.

negative:
State-of-the-art text spotting systems typically aim to detect isolated words
or word-by-word text in images of natural scenes and ignore the semantic
coherence within a region of text. However, when interpreted together,
seemingly isolated words may be easier to recognize. On this basis, we propose
a novel "semantic-based text recognition" (STR) deep learning model that reads
text in images with the help of understanding context. STR consists of several
modules. We introduce the Text Grouping and Arranging (TGA) algorithm to
connect and order isolated text regions. A text-recognition network interprets
isolated words. Benefiting from semantic information, a sequenceto-sequence
network model efficiently corrects inaccurate and uncertain phrases produced
earlier in the STR pipeline. We present experiments on two new distinct
datasets that contain scanned catalog images of interior designs and
photographs of protesters with hand-written signs, respectively. Our results
show that our STR model outperforms a baseline method that uses
state-of-the-art single-wordrecognition techniques on both datasets. STR yields
a high accuracy rate of 90% on the catalog images and 71% on the more difficult
protest images, suggesting its generality in recognizing text.

negative:
Recent models for cross-modal retrieval have benefited from an increasingly
rich understanding of visual scenes, afforded by scene graphs and object
interactions to mention a few. This has resulted in an improved matching
between the visual representation of an image and the textual representation of
its caption. Yet, current visual representations overlook a key aspect: the
text appearing in images, which may contain crucial information for retrieval.
In this paper, we first propose a new dataset that allows exploration of
cross-modal retrieval where images contain scene-text instances. Then, armed
with this dataset, we describe several approaches which leverage scene text,
including a better scene-text aware cross-modal retrieval method which uses
specialized representations for text from the captions and text from the visual
scene, and reconcile them in a common embedding space. Extensive experiments
confirm that cross-modal retrieval approaches benefit from scene text and
highlight interesting research questions worth exploring further. Dataset and
code are available at http://europe.naverlabs.com/stacmr

negative:
Despite the ubiquity of mobile and wearable text messaging applications, the
problem of keyboard text decoding is not tackled sufficiently in the light of
the enormous success of the deep learning Recurrent Neural Network (RNN) and
Convolutional Neural Networks (CNN) for natural language understanding. In
particular, considering that the keyboard decoders should operate on devices
with memory and processor resource constraints, makes it challenging to deploy
industrial scale deep neural network (DNN) models. This paper proposes a
sequence-to-sequence neural attention network system for automatic text
correction and completion. Given an erroneous sequence, our model encodes
character level hidden representations and then decodes the revised sequence
thus enabling auto-correction and completion. We achieve this by a combination
of character level CNN and gated recurrent unit (GRU) encoder along with and a
word level gated recurrent unit (GRU) attention decoder. Unlike traditional
language models that learn from billions of words, our corpus size is only 12
million words; an order of magnitude smaller. The memory footprint of our
learnt model for inference and prediction is also an order of magnitude smaller
than the conventional language model based text decoders. We report baseline
performance for neural keyboard decoders in such limited domain. Our models
achieve a word level accuracy of $90\%$ and a character error rate CER of
$2.4\%$ over the Twitter typo dataset. We present a novel dataset of noisy to
corrected mappings by inducing the noise distribution from the Twitter data
over the OpenSubtitles 2009 dataset; on which our model predicts with a word
level accuracy of $98\%$ and sequence accuracy of $68.9\%$. In our user study,
our model achieved an average CER of $2.6\%$ with the state-of-the-art
non-neural touch-screen keyboard decoder at CER of $1.6\%$.

negative:
This paper presents a novel training method for end-to-end scene text
recognition. End-to-end scene text recognition offers high recognition
accuracy, especially when using the encoder-decoder model based on Transformer.
To train a highly accurate end-to-end model, we need to prepare a large
image-to-text paired dataset for the target language. However, it is difficult
to collect this data, especially for resource-poor languages. To overcome this
difficulty, our proposed method utilizes well-prepared large datasets in
resource-rich languages such as English, to train the resource-poor
encoder-decoder model. Our key idea is to build a model in which the encoder
reflects knowledge of multiple languages while the decoder specializes in
knowledge of just the resource-poor language. To this end, the proposed method
pre-trains the encoder by using a multilingual dataset that combines the
resource-poor language's dataset and the resource-rich language's dataset to
learn language-invariant knowledge for scene text recognition. The proposed
method also pre-trains the decoder by using the resource-poor language's
dataset to make the decoder better suited to the resource-poor language.
Experiments on Japanese scene text recognition using a small, publicly
available dataset demonstrate the effectiveness of the proposed method.

negative:
Recent advancements in the area of Computer Vision with state-of-art Neural
Networks has given a boost to Optical Character Recognition (OCR) accuracies.
However, extracting characters/text alone is often insufficient for relevant
information extraction as documents also have a visual structure that is not
captured by OCR. Extracting information from tables, charts, footnotes, boxes,
headings and retrieving the corresponding structured representation for the
document remains a challenge and finds application in a large number of
real-world use cases. In this paper, we propose a novel enterprise based
end-to-end framework called DeepReader which facilitates information extraction
from document images via identification of visual entities and populating a
meta relational model across different entities in the document image. The
model schema allows for an easy to understand abstraction of the entities
detected by the deep vision models and the relationships between them.
DeepReader has a suite of state-of-the-art vision algorithms which are applied
to recognize handwritten and printed text, eliminate noisy effects, identify
the type of documents and detect visual entities like tables, lines and boxes.
Deep Reader maps the extracted entities into a rich relational schema so as to
capture all the relevant relationships between entities (words, textboxes,
lines etc) detected in the document. Relevant information and fields can then
be extracted from the document by writing SQL queries on top of the
relationship tables. A natural language based interface is added on top of the
relationship schema so that a non-technical user, specifying the queries in
natural language, can fetch the information with minimal effort. In this paper,
we also demonstrate many different capabilities of Deep Reader and report
results on a real-world use case.

negative:
Scene text recognition is a hot research topic in computer vision. Recently,
many recognition methods based on the encoder-decoder framework have been
proposed, and they can handle scene texts of perspective distortion and curve
shape. Nevertheless, they still face lots of challenges like image blur, uneven
illumination, and incomplete characters. We argue that most encoder-decoder
methods are based on local visual features without explicit global semantic
information. In this work, we propose a semantics enhanced encoder-decoder
framework to robustly recognize low-quality scene texts. The semantic
information is used both in the encoder module for supervision and in the
decoder module for initializing. In particular, the state-of-the art ASTER
method is integrated into the proposed framework as an exemplar. Extensive
experiments demonstrate that the proposed framework is more robust for
low-quality text images, and achieves state-of-the-art results on several
benchmark datasets.

negative:
In this work we present a state-of-the-art approach for unconstrained natural
scene text recognition. We propose a cascade approach that incorporates a
convolutional neural network (CNN) architecture followed by a long short term
memory model (LSTM). The CNN learns visual features for the characters and uses
them with a softmax layer to detect sequence of characters. While the CNN gives
very good recognition results, it does not model relation between characters,
hence gives rise to false positive and false negative cases (confusing
characters due to visual similarities like "g" and "9", or confusing background
patches with characters; either removing existing characters or adding
non-existing ones) To alleviate these problems we leverage recent developments
in LSTM architectures to encode contextual information. We show that the LSTM
can dramatically reduce such errors and achieve state-of-the-art accuracy in
the task of unconstrained natural scene text recognition. Moreover we manually
remove all occurrences of the words that exist in the test set from our
training set to test whether our approach will generalize to unseen data. We
use the ICDAR 13 test set for evaluation and compare the results with the state
of the art approaches [11, 18]. We finally present an application of the work
in the domain of for traffic monitoring.

negative:
Segmentation of additive manufacturing (AM) defects in X-ray Computed
Tomography (XCT) images is challenging, due to the poor contrast, small sizes
and variation in appearance of defects. Automatic segmentation can, however,
provide quality control for additive manufacturing. Over recent years,
three-dimensional convolutional neural networks (3D CNNs) have performed well
in the volumetric segmentation of medical images. In this work, we leverage
techniques from the medical imaging domain and propose training a 3D U-Net
model to automatically segment defects in XCT images of AM samples. This work
not only contributes to the use of machine learning for AM defect detection but
also demonstrates for the first time 3D volumetric segmentation in AM. We train
and test with three variants of the 3D U-Net on an AM dataset, achieving a mean
intersection of union (IOU) value of 88.4%.

negative:
Recent studies on machine reading comprehension have focused on text-level
understanding but have not yet reached the level of human understanding of the
visual layout and content of real-world documents. In this study, we introduce
a new visual machine reading comprehension dataset, named VisualMRC, wherein
given a question and a document image, a machine reads and comprehends texts in
the image to answer the question in natural language. Compared with existing
visual question answering (VQA) datasets that contain texts in images,
VisualMRC focuses more on developing natural language understanding and
generation abilities. It contains 30,000+ pairs of a question and an
abstractive answer for 10,000+ document images sourced from multiple domains of
webpages. We also introduce a new model that extends existing
sequence-to-sequence models, pre-trained with large-scale text corpora, to take
into account the visual layout and content of documents. Experiments with
VisualMRC show that this model outperformed the base sequence-to-sequence
models and a state-of-the-art VQA model. However, its performance is still
below that of humans on most automatic evaluation metrics. The dataset will
facilitate research aimed at connecting vision and language understanding.

negative:
Recognizing irregular text in natural scene images is challenging due to the
large variance in text appearance, such as curvature, orientation and
distortion. Most existing approaches rely heavily on sophisticated model
designs and/or extra fine-grained annotations, which, to some extent, increase
the difficulty in algorithm implementation and data collection. In this work,
we propose an easy-to-implement strong baseline for irregular scene text
recognition, using off-the-shelf neural network components and only word-level
annotations. It is composed of a $31$-layer ResNet, an LSTM-based
encoder-decoder framework and a 2-dimensional attention module. Despite its
simplicity, the proposed method is robust and achieves state-of-the-art
performance on both regular and irregular scene text recognition benchmarks.
Code is available at: https://tinyurl.com/ShowAttendRead

negative:
Text normalization is a ubiquitous process that appears as the first step of
many Natural Language Processing problems. However, previous Deep Learning
approaches have suffered from so-called silly errors, which are undetectable on
unsupervised frameworks, making those models unsuitable for deployment. In this
work, we make use of an attention-based encoder-decoder architecture that
overcomes these undetectable errors by using a fine-grained character-level
approach rather than a word-level one. Furthermore, our new general-purpose
encoder based on causal convolutions, called Causal Feature Extractor (CFE), is
introduced and compared to other common encoders. The experimental results show
the feasibility of this encoder, which leverages the attention mechanisms the
most and obtains better results in terms of accuracy, number of parameters and
convergence time. While our method results in a slightly worse initial accuracy
(92.74%), errors can be automatically detected and, thus, more readily solved,
obtaining a more robust model for deployment. Furthermore, there is still
plenty of room for future improvements that will push even further these
advantages.

negative:
Encoder-decoder models have become an effective approach for sequence
learning tasks like machine translation, image captioning and speech
recognition, but have yet to show competitive results for handwritten text
recognition. To this end, we propose an attention-based sequence-to-sequence
model. It combines a convolutional neural network as a generic feature
extractor with a recurrent neural network to encode both the visual
information, as well as the temporal context between characters in the input
image, and uses a separate recurrent neural network to decode the actual
character sequence. We make experimental comparisons between various attention
mechanisms and positional encodings, in order to find an appropriate alignment
between the input and output sequence. The model can be trained end-to-end and
the optional integration of a hybrid loss allows the encoder to retain an
interpretable and usable output, if desired. We achieve competitive results on
the IAM and ICFHR2016 READ data sets compared to the state-of-the-art without
the use of a language model, and we significantly improve over any recent
sequence-to-sequence approaches.

negative:
The pursuit of high performance on public benchmarks has been the driving
force for research in scene text recognition, and notable progress has been
achieved. However, a close investigation reveals a startling fact that the
state-of-the-art methods perform well on images with words within vocabulary
but generalize poorly to images with words outside vocabulary. We call this
phenomenon "vocabulary reliance". In this paper, we establish an analytical
framework to conduct an in-depth study on the problem of vocabulary reliance in
scene text recognition. Key findings include: (1) Vocabulary reliance is
ubiquitous, i.e., all existing algorithms more or less exhibit such
characteristic; (2) Attention-based decoders prove weak in generalizing to
words outside vocabulary and segmentation-based decoders perform well in
utilizing visual features; (3) Context modeling is highly coupled with the
prediction layers. These findings provide new insights and can benefit future
research in scene text recognition. Furthermore, we propose a simple yet
effective mutual learning strategy to allow models of two families
(attention-based and segmentation-based) to learn collaboratively. This remedy
alleviates the problem of vocabulary reliance and improves the overall scene
text recognition performance.

negative:
In the Gastric Histopathology Image Classification (GHIC) tasks, which are
usually weakly supervised learning missions, there is inevitably redundant
information in the images. Therefore, designing networks that can focus on
effective distinguishing features has become a popular research topic. In this
paper, to accomplish the tasks of GHIC superiorly and to assist pathologists in
clinical diagnosis, an intelligent Hierarchical Conditional Random Field based
Attention Mechanism (HCRF-AM) model is proposed. The HCRF-AM model consists of
an Attention Mechanism (AM) module and an Image Classification (IC) module. In
the AM module, an HCRF model is built to extract attention regions. In the IC
module, a Convolutional Neural Network (CNN) model is trained with the
attention regions selected and then an algorithm called Classification
Probability-based Ensemble Learning is applied to obtain the image-level
results from patch-level output of the CNN. In the experiment, a classification
specificity of 96.67% is achieved on a gastric histopathology dataset with 700
images. Our HCRF-AM model demonstrates high classification performance and
shows its effectiveness and future potential in the GHIC field.

negative:
Key information extraction from document images is of paramount importance in
office automation. Conventional template matching based approaches fail to
generalize well to document images of unseen templates, and are not robust
against text recognition errors. In this paper, we propose an end-to-end
Spatial Dual-Modality Graph Reasoning method (SDMG-R) to extract key
information from unstructured document images. We model document images as
dual-modality graphs, nodes of which encode both the visual and textual
features of detected text regions, and edges of which represent the spatial
relations between neighboring text regions. The key information extraction is
solved by iteratively propagating messages along graph edges and reasoning the
categories of graph nodes. In order to roundly evaluate our proposed method as
well as boost the future research, we release a new dataset named WildReceipt,
which is collected and annotated tailored for the evaluation of key information
extraction from document images of unseen templates in the wild. It contains 25
key information categories, a total of about 69000 text boxes, and is about 2
times larger than the existing public datasets. Extensive experiments validate
that all information including visual features, textual features and spatial
relations can benefit key information extraction. It has been shown that SDMG-R
can effectively extract key information from document images of unseen
templates, and obtain new state-of-the-art results on the recent popular
benchmark SROIE and our WildReceipt. Our code and dataset will be publicly
released.

negative:
Text recognition has attracted considerable research interests because of its
various applications. The cutting-edge text recognition methods are based on
attention mechanisms. However, most of attention methods usually suffer from
serious alignment problem due to its recurrency alignment operation, where the
alignment relies on historical decoding results. To remedy this issue, we
propose a decoupled attention network (DAN), which decouples the alignment
operation from using historical decoding results. DAN is an effective, flexible
and robust end-to-end text recognizer, which consists of three components: 1) a
feature encoder that extracts visual features from the input image; 2) a
convolutional alignment module that performs the alignment operation based on
visual features from the encoder; and 3) a decoupled text decoder that makes
final prediction by jointly using the feature map and attention maps.
Experimental results show that DAN achieves state-of-the-art performance on
multiple text recognition tasks, including offline handwritten text recognition
and regular/irregular scene text recognition.

negative:
Spelling error correction is one of topics which have a long history in
natural language processing. Although previous studies have achieved remarkable
results, challenges still exist. In the Vietnamese language, a state-of-the-art
method for the task infers a syllable's context from its adjacent syllables.
The method's accuracy can be unsatisfactory, however, because the model may
lose the context if two (or more) spelling mistakes stand near each other. In
this paper, we propose a novel method to correct Vietnamese spelling errors. We
tackle the problems of mistyped errors and misspelled errors by using a deep
learning model. The embedding layer, in particular, is powered by the byte pair
encoding technique. The sequence to sequence model based on the Transformer
architecture makes our approach different from the previous works on the same
problem. In the experiment, we train the model with a large synthetic dataset,
which is randomly introduced spelling errors. We test the performance of the
proposed method using a realistic dataset. This dataset contains 11,202
human-made misspellings in 9,341 different Vietnamese sentences. The
experimental results show that our method achieves encouraging performance with
86.8% errors detected and 81.5% errors corrected, which improves the
state-of-the-art approach 5.6% and 2.2%, respectively.

negative:
In this paper, we study the problem of text line recognition. Unlike most
approaches targeting specific domains such as scene-text or handwritten
documents, we investigate the general problem of developing a universal
architecture that can extract text from any image, regardless of source or
input modality. We consider two decoder families (Connectionist Temporal
Classification and Transformer) and three encoder modules (Bidirectional LSTMs,
Self-Attention, and GRCLs), and conduct extensive experiments to compare their
accuracy and performance on widely used public datasets of scene and
handwritten text. We find that a combination that so far has received little
attention in the literature, namely a Self-Attention encoder coupled with the
CTC decoder, when compounded with an external language model and trained on
both public and internal data, outperforms all the others in accuracy and
computational complexity. Unlike the more common Transformer-based models, this
architecture can handle inputs of arbitrary length, a requirement for universal
line recognition. Using an internal dataset collected from multiple sources, we
also expose the limitations of current public datasets in evaluating the
accuracy of line recognizers, as the relatively narrow image width and sequence
length distributions do not allow to observe the quality degradation of the
Transformer approach when applied to the transcription of long lines.

negative:
Texts appearing in daily scenes that can be recognized by OCR (Optical
Character Recognition) tools contain significant information, such as street
name, product brand and prices. Two tasks -- text-based visual question
answering and text-based image captioning, with a text extension from existing
vision-language applications, are catching on rapidly. To address these
problems, many sophisticated multi-modality encoding frameworks (such as
heterogeneous graph structure) are being used. In this paper, we argue that a
simple attention mechanism can do the same or even better job without any bells
and whistles. Under this mechanism, we simply split OCR token features into
separate visual- and linguistic-attention branches, and send them to a popular
Transformer decoder to generate answers or captions. Surprisingly, we find this
simple baseline model is rather strong -- it consistently outperforms
state-of-the-art (SOTA) models on two popular benchmarks, TextVQA and all three
tasks of ST-VQA, although these SOTA models use far more complex encoding
mechanisms. Transferring it to text-based image captioning, we also surpass the
TextCaps Challenge 2020 winner. We wish this work to set the new baseline for
this two OCR text related applications and to inspire new thinking of
multi-modality encoder design. Code is available at
https://github.com/ZephyrZhuQi/ssbaseline

negative:
Structured information extraction from document images usually consists of
three steps: text detection, text recognition, and text field labeling. While
text detection and text recognition have been heavily studied and improved a
lot in literature, text field labeling is less explored and still faces many
challenges. Existing learning based methods for text labeling task usually
require a large amount of labeled examples to train a specific model for each
type of document. However, collecting large amounts of document images and
labeling them is difficult and sometimes impossible due to privacy issues.
Deploying separate models for each type of document also consumes a lot of
resources. Facing these challenges, we explore one-shot learning for the text
field labeling task. Existing one-shot learning methods for the task are mostly
rule-based and have difficulty in labeling fields in crowded regions with few
landmarks and fields consisting of multiple separate text regions. To alleviate
these problems, we proposed a novel deep end-to-end trainable approach for
one-shot text field labeling, which makes use of attention mechanism to
transfer the layout information between document images. We further applied
conditional random field on the transferred layout information for the
refinement of field labeling. We collected and annotated a real-world one-shot
field labeling dataset with a large variety of document types and conducted
extensive experiments to examine the effectiveness of the proposed model. To
stimulate research in this direction, the collected dataset and the one-shot
model will be released1.

negative:
In this work, a region-based Deep Convolutional Neural Network framework is
proposed for document structure learning. The contribution of this work
involves efficient training of region based classifiers and effective
ensembling for document image classification. A primary level of `inter-domain'
transfer learning is used by exporting weights from a pre-trained VGG16
architecture on the ImageNet dataset to train a document classifier on whole
document images. Exploiting the nature of region based influence modelling, a
secondary level of `intra-domain' transfer learning is used for rapid training
of deep learning models for image segments. Finally, stacked generalization
based ensembling is utilized for combining the predictions of the base deep
neural network models. The proposed method achieves state-of-the-art accuracy
of 92.2% on the popular RVL-CDIP document image dataset, exceeding benchmarks
set by existing algorithms.

negative:
Attention-based scene text recognizers have gained huge success, which
leverages a more compact intermediate representation to learn 1d- or 2d-
attention by a RNN-based encoder-decoder architecture. However, such methods
suffer from attention-drift problem because high similarity among encoded
features leads to attention confusion under the RNN-based local attention
mechanism. Moreover, RNN-based methods have low efficiency due to poor
parallelization. To overcome these problems, we propose the MASTER, a
self-attention based scene text recognizer that (1) not only encodes the
input-output attention but also learns self-attention which encodes
feature-feature and target-target relationships inside the encoder and decoder
and (2) learns a more powerful and robust intermediate representation to
spatial distortion, and (3) owns a great training efficiency because of high
training parallelization and a high-speed inference because of an efficient
memory-cache mechanism. Extensive experiments on various benchmarks demonstrate
the superior performance of our MASTER on both regular and irregular scene
text. Pytorch code can be found at https://github.com/wenwenyu/MASTER-pytorch,
and Tensorflow code can be found at https://github.com/jiangxiluning/MASTER-TF.

negative:
Reading irregular scene text of arbitrary shape in natural images is still a
challenging problem, despite the progress made recently. Many existing
approaches incorporate sophisticated network structures to handle various
shapes, use extra annotations for stronger supervision, or employ hard-to-train
recurrent neural networks for sequence modeling. In this work, we propose a
simple yet strong approach for scene text recognition. With no need to convert
input images to sequence representations, we directly connect two-dimensional
CNN features to an attention-based sequence decoder which guided by holistic
representation. The holistic representation can guide the attention-based
decoder focus on more accurate area. As no recurrent module is adopted, our
model can be trained in parallel. It achieves 1.5x to 9.4x acceleration to
backward pass and 1.3x to 7.9x acceleration to forward pass, compared with the
RNN counterparts. The proposed model is trained with only word-level
annotations. With this simple design, our method achieves state-of-the-art or
competitive recognition performance on the evaluated regular and irregular
scene text benchmark datasets.

negative:
Texts from scene images typically consist of several characters and exhibit a
characteristic sequence structure. Existing methods capture the structure with
the sequence-to-sequence models by an encoder to have the visual
representations and then a decoder to translate the features into the label
sequence. In this paper, we study text recognition framework by considering the
long-term temporal dependencies in the encoder stage. We demonstrate that the
proposed Temporal Convolutional Encoder with increased sequential extents
improves the accuracy of text recognition. We also study the impact of
different attention modules in convolutional blocks for learning accurate text
representations. We conduct comparisons on seven datasets and the experiments
demonstrate the effectiveness of our proposed approach.

negative:
Scene text recognition has attracted a great many researches due to its
importance to various applications. Existing methods mainly adopt recurrence or
convolution based networks. Though have obtained good performance, these
methods still suffer from two limitations: slow training speed due to the
internal recurrence of RNNs, and high complexity due to stacked convolutional
layers for long-term feature extraction. This paper, for the first time,
proposes a no-recurrence sequence-to-sequence text recognizer, named NRTR, that
dispenses with recurrences and convolutions entirely. NRTR follows the
encoder-decoder paradigm, where the encoder uses stacked self-attention to
extract image features, and the decoder applies stacked self-attention to
recognize texts based on encoder output. NRTR relies solely on self-attention
mechanism thus could be trained with more parallelization and less complexity.
Considering scene image has large variation in text and background, we further
design a modality-transform block to effectively transform 2D input images to
1D sequences, combined with the encoder to extract more discriminative
features. NRTR achieves state-of-the-art or highly competitive performance on
both regular and irregular benchmarks, while requires only a small fraction of
training time compared to the best model from the literature (at least 8 times
faster).

negative:
Scene text recognition has been a hot research topic in computer vision due
to its various applications. The state of the art is the attention-based
encoder-decoder framework that learns the mapping between input images and
output sequences in a purely data-driven way. However, we observe that existing
attention-based methods perform poorly on complicated and/or low-quality
images. One major reason is that existing methods cannot get accurate
alignments between feature areas and targets for such images. We call this
phenomenon "attention drift". To tackle this problem, in this paper we propose
the FAN (the abbreviation of Focusing Attention Network) method that employs a
focusing attention mechanism to automatically draw back the drifted attention.
FAN consists of two major components: an attention network (AN) that is
responsible for recognizing character targets as in the existing methods, and a
focusing network (FN) that is responsible for adjusting attention by evaluating
whether AN pays attention properly on the target areas in the images.
Furthermore, different from the existing methods, we adopt a ResNet-based
network to enrich deep representations of scene text images. Extensive
experiments on various benchmarks, including the IIIT5k, SVT and ICDAR
datasets, show that the FAN method substantially outperforms the existing
methods.

negative:
Scene Text Recognition (STR), the task of recognizing text against complex
image backgrounds, is an active area of research. Current state-of-the-art
(SOTA) methods still struggle to recognize text written in arbitrary shapes. In
this paper, we introduce a novel architecture for STR, named Selective Context
ATtentional Text Recognizer (SCATTER). SCATTER utilizes a stacked block
architecture with intermediate supervision during training, that paves the way
to successfully train a deep BiLSTM encoder, thus improving the encoding of
contextual dependencies. Decoding is done using a two-step 1D attention
mechanism. The first attention step re-weights visual features from a CNN
backbone together with contextual features computed by a BiLSTM layer. The
second attention step, similar to previous papers, treats the features as a
sequence and attends to the intra-sequence relationships. Experiments show that
the proposed approach surpasses SOTA performance on irregular text recognition
benchmarks by 3.7\% on average.

negative:
The paper proposes a new text recognition network for scene-text images. Many
state-of-the-art methods employ the attention mechanism either in the text
encoder or decoder for the text alignment. Although the encoder-based attention
yields promising results, these schemes inherit noticeable limitations. They
perform the feature extraction (FE) and visual attention (VA) sequentially,
which bounds the attention mechanism to rely only on the FE final single-scale
output. Moreover, the utilization of the attention process is limited by only
applying it directly to the single scale feature-maps. To address these issues,
we propose a new multi-scale and encoder-based attention network for text
recognition that performs the multi-scale FE and VA in parallel. The
multi-scale channels also undergo regular fusion with each other to develop the
coordinated knowledge together. Quantitative evaluation and robustness analysis
on the standard benchmarks demonstrate that the proposed network outperforms
the state-of-the-art in most cases.

negative:
The attention-based encoder-decoder framework has recently achieved
impressive results for scene text recognition, and many variants have emerged
with improvements in recognition quality. However, it performs poorly on
contextless texts (e.g., random character sequences) which is unacceptable in
most of real application scenarios. In this paper, we first deeply investigate
the decoding process of the decoder. We empirically find that a representative
character-level sequence decoder utilizes not only context information but also
positional information. Contextual information, which the existing approaches
heavily rely on, causes the problem of attention drift. To suppress such
side-effect, we propose a novel position enhancement branch, and dynamically
fuse its outputs with those of the decoder attention module for scene text
recognition. Specifically, it contains a position aware module to enable the
encoder to output feature vectors encoding their own spatial positions, and an
attention module to estimate glimpses using the positional clue (i.e., the
current decoding time step) only. The dynamic fusion is conducted for more
robust feature via an element-wise gate mechanism. Theoretically, our proposed
method, dubbed \emph{RobustScanner}, decodes individual characters with dynamic
ratio between context and positional clues, and utilizes more positional ones
when the decoding sequences with scarce context, and thus is robust and
practical. Empirically, it has achieved new state-of-the-art results on popular
regular and irregular text recognition benchmarks while without much
performance drop on contextless benchmarks, validating its robustness in both
contextual and contextless application scenarios.

negative:
Scene text recognition has attracted particular research interest because it
is a very challenging problem and has various applications. The most
cutting-edge methods are attentional encoder-decoder frameworks that learn the
alignment between the input image and output sequences. In particular, the
decoder recurrently outputs predictions, using the prediction of the previous
step as a guidance for every time step. In this study, we point out that the
inappropriate use of previous predictions in existing attention mechanisms
restricts the recognition performance and brings instability. To handle this
problem, we propose a novel module, namely adaptive embedding gate(AEG). The
proposed AEG focuses on introducing high-order character language models to
attention mechanism by controlling the information transmission between
adjacent characters. AEG is a flexible module and can be easily integrated into
the state-of-the-art attentional methods. We evaluate its effectiveness as well
as robustness on a number of standard benchmarks, including the IIIT$5$K, SVT,
SVT-P, CUTE$80$, and ICDAR datasets. Experimental results demonstrate that AEG
can significantly boost recognition performance and bring better robustness.

negative:
In this paper, we propose an end-to-end trainable framework for restoring
historical documents content that follows the correct reading order. In this
framework, two branches named character branch and layout branch are added
behind the feature extraction network. The character branch localizes
individual characters in a document image and recognizes them simultaneously.
Then we adopt a post-processing method to group them into text lines. The
layout branch based on fully convolutional network outputs a binary mask. We
then use Hough transform for line detection on the binary mask and combine
character results with the layout information to restore document content.
These two branches can be trained in parallel and are easy to train.
Furthermore, we propose a re-score mechanism to minimize recognition error.
Experiment results on the extended Chinese historical document MTHv2 dataset
demonstrate the effectiveness of the proposed framework.

negative:
In this work, we adhere to explore a Multi-Tasking learning (MTL) based
network to perform document attribute classification such as the font type,
font size, font emphasis and scanning resolution classification of a document
image. To accomplish these tasks, we operate on either segmented word level or
on uniformed size patches randomly cropped out of the document. Furthermore, a
hybrid convolution neural network (CNN) architecture "MTL+MI", which is based
on the combination of MTL and Multi-Instance (MI) of patch and word is used to
accomplish joint learning for the classification of the same document
attributes. The contribution of this paper are three fold: firstly, based on
segmented word images and patches, we present a MTL based network for the
classification of a full document image. Secondly, we propose a MTL and MI
(using segmented words and patches) based combined CNN architecture ("MTL+MI")
for the classification of same document attributes. Thirdly, based on the
multi-tasking classifications of the words and/or patches, we propose an
intelligent voting system which is based on the posterior probabilities of each
words and/or patches to perform the classification of document's attributes of
complete document image.

negative:
Recognizing scene text is a challenging problem, even more so than the
recognition of scanned documents. This problem has gained significant attention
from the computer vision community in recent years, and several methods based
on energy minimization frameworks and deep learning approaches have been
proposed. In this work, we focus on the energy minimization framework and
propose a model that exploits both bottom-up and top-down cues for recognizing
cropped words extracted from street images. The bottom-up cues are derived from
individual character detections from an image. We build a conditional random
field model on these detections to jointly model the strength of the detections
and the interactions between them. These interactions are top-down cues
obtained from a lexicon-based prior, i.e., language statistics. The optimal
word represented by the text image is obtained by minimizing the energy
function corresponding to the random field model. We evaluate our proposed
algorithm extensively on a number of cropped scene text benchmark datasets,
namely Street View Text, ICDAR 2003, 2011 and 2013 datasets, and IIIT 5K-word,
and show better performance than comparable methods. We perform a rigorous
analysis of all the steps in our approach and analyze the results. We also show
that state-of-the-art convolutional neural network features can be integrated
in our framework to further improve the recognition performance.

negative:
Given image labels as the only supervisory signal, we focus on harvesting, or
mining, thoracic disease localizations from chest X-ray images. Harvesting such
localizations from existing datasets allows for the creation of improved data
sources for computer-aided diagnosis and retrospective analyses. We train a
convolutional neural network (CNN) for image classification and propose an
attention mining (AM) strategy to improve the model's sensitivity or saliency
to disease patterns. The intuition of AM is that once the most salient disease
area is blocked or hidden from the CNN model, it will pay attention to
alternative image regions, while still attempting to make correct predictions.
However, the model requires to be properly constrained during AM, otherwise, it
may overfit to uncorrelated image parts and forget the valuable knowledge that
it has learned from the original image classification task. To alleviate such
side effects, we then design a knowledge preservation (KP) loss, which
minimizes the discrepancy between responses for X-ray images from the original
and the updated networks. Furthermore, we modify the CNN model to include
multi-scale aggregation (MSA), improving its localization ability on
small-scale disease findings, e.g., lung nodules. We experimentally validate
our method on the publicly-available ChestX-ray14 dataset, outperforming a
class activation map (CAM)-based approach, and demonstrating the value of our
novel framework for mining disease locations.

negative:
From the literature, it is demonstrated that performing text-line
segmentation directly in the run-length compressed handwritten document images
significantly reduces the computational time and memory space. In this paper,
we investigate the issues of word and character segmentation directly on the
run-length compressed document images. Primarily, the spreads of the characters
are intelligently extracted from the foreground runs of the compressed data and
subsequently connected components are established. The spacing between the
connected components would be larger between the adjacent words when compared
to that of intra-words. With this knowledge, a threshold is empirically chosen
for inter-word separation. Every connected component within a word is further
analysed for character segmentation. Here, min-cut graph concept is used for
separating the touching characters. Over-segmentation and under-segmentation
issues are addressed by insertion and deletion operations respectively. The
approach has been developed particularly for compressed handwritten English
document images. However, the model has been tested on non-English document
images.

negative:
The paper approaches the task of handwritten text recognition (HTR) with
attentional encoder-decoder networks trained on sequences of characters, rather
than words. We experiment on lines of text from popular handwriting datasets
and compare different activation functions for the attention mechanism used for
aligning image pixels and target characters. We find that softmax attention
focuses heavily on individual characters, while sigmoid attention focuses on
multiple characters at each step of the decoding. When the sequence alignment
is one-to-one, softmax attention is able to learn a more precise alignment at
each step of the decoding, whereas the alignment generated by sigmoid attention
is much less precise. When a linear function is used to obtain attention
weights, the model predicts a character by looking at the entire sequence of
characters and performs poorly because it lacks a precise alignment between the
source and target. Future research may explore HTR in natural scene images,
since the model is capable of transcribing handwritten text without the need
for producing segmentations or bounding boxes of text in images.

negative:
The recently proposed Multilinear Compressive Learning (MCL) framework
combines Multilinear Compressive Sensing and Machine Learning into an
end-to-end system that takes into account the multidimensional structure of the
signals when designing the sensing and feature synthesis components. The key
idea behind MCL is the assumption of the existence of a tensor subspace which
can capture the essential features from the signal for the downstream learning
task. Thus, the ability to find such a discriminative tensor subspace and
optimize the system to project the signals onto that data manifold plays an
important role in Multilinear Compressive Learning. In this paper, we propose a
novel solution to address both of the aforementioned requirements, i.e., How to
find those tensor subspaces in which the signals of interest are highly
separable? and How to optimize the sensing and feature synthesis components to
transform the original signals to the data manifold found in the first
question? In our proposal, the discovery of a high-quality data manifold is
conducted by training a nonlinear compressive learning system on the inference
task. Its knowledge of the data manifold of interest is then progressively
transferred to the MCL components via multi-stage supervised training with the
supervisory information encoding how the compressed measurements, the
synthesized features, and the predictions should be like. The proposed
knowledge transfer algorithm also comes with a semi-supervised adaption that
enables compressive learning models to utilize unlabeled data effectively.
Extensive experiments demonstrate that the proposed knowledge transfer method
can effectively train MCL models to compressively sense and synthesize better
features for the learning tasks with improved performances, especially when the
complexity of the learning task increases.

negative:
Attention-based encoder-decoder framework is widely used in the scene text
recognition task. However, for the current state-of-the-art(SOTA) methods,
there is room for improvement in terms of the efficient usage of local visual
and global context information of the input text image, as well as the robust
correlation between the scene processing module(encoder) and the text
processing module(decoder). In this paper, we propose a Representation and
Correlation Enhanced Encoder-Decoder Framework(RCEED) to address these
deficiencies and break performance bottleneck. In the encoder module, local
visual feature, global context feature, and position information are aligned
and fused to generate a small-size comprehensive feature map. In the decoder
module, two methods are utilized to enhance the correlation between scene and
text feature space. 1) The decoder initialization is guided by the holistic
feature and global glimpse vector exported from the encoder. 2) The feature
enriched glimpse vector produced by the Multi-Head General Attention is used to
assist the RNN iteration and the character prediction at each time step.
Meanwhile, we also design a Layernorm-Dropout LSTM cell to improve model's
generalization towards changeable texts. Extensive experiments on the
benchmarks demonstrate the advantageous performance of RCEED in scene text
recognition tasks, especially the irregular ones.

negative:
Here we address a challenging problem: recognizing multiple text sequences
from an image by pure end-to-end learning. It is twofold: 1) Multiple text
sequences recognition. Each image may contain multiple text sequences of
different content, location and orientation, and we try to recognize all the
text sequences contained in the image. 2) Pure end-to-end (PEE) learning.We
solve the problem in a pure end-to-end learning way where each training image
is labeled by only text transcripts of all contained sequences, without any
geometric annotations. Most existing works recognize multiple text sequences
from an image in a non-end-to-end (NEE) or quasi-end-to-end (QEE) way, in which
each image is trained with both text transcripts and text locations.Only
recently, a PEE method was proposed to recognize text sequences from an image
where the text sequence was split to several lines in the image. However, it
cannot be directly applied to recognizing multiple text sequences from an
image. So in this paper, we propose a pure end-to-end learning method to
recognize multiple text sequences from an image. Our method directly learns
multiple sequences of probability distribution conditioned on each input image,
and outputs multiple text transcripts with a well-designed decoding strategy.To
evaluate the proposed method, we constructed several datasets mainly based on
an existing public dataset andtwo real application scenarios. Experimental
results show that the proposed method can effectively recognize multiple text
sequences from images, and outperforms CTC-based and attention-based baseline
methods.

negative:
In this paper, we address the problem of having characters with different
scales in scene text recognition. We propose a novel scale aware feature
encoder (SAFE) that is designed specifically for encoding characters with
different scales. SAFE is composed of a multi-scale convolutional encoder and a
scale attention network. The multi-scale convolutional encoder targets at
extracting character features under multiple scales, and the scale attention
network is responsible for selecting features from the most relevant scale(s).
SAFE has two main advantages over the traditional single-CNN encoder used in
current state-of-the-art text recognizers. First, it explicitly tackles the
scale problem by extracting scale-invariant features from the characters. This
allows the recognizer to put more effort in handling other challenges in scene
text recognition, like those caused by view distortion and poor image quality.
Second, it can transfer the learning of feature encoding across different
character scales. This is particularly important when the training set has a
very unbalanced distribution of character scales, as training with such a
dataset will make the encoder biased towards extracting features from the
predominant scale. To evaluate the effectiveness of SAFE, we design a simple
text recognizer named scale-spatial attention network (S-SAN) that employs SAFE
as its feature encoder, and carry out experiments on six public benchmarks.
Experimental results demonstrate that S-SAN can achieve state-of-the-art (or,
in some cases, extremely competitive) performance without any post-processing.

negative:
Scene text recognition is a challenging task due to diverse variations of
text instances in natural scene images. Conventional methods based on
CNN-RNN-CTC or encoder-decoder with attention mechanism may not fully
investigate stable and efficient feature representations for multi-oriented
scene texts. In this paper, we propose a primitive representation learning
method that aims to exploit intrinsic representations of scene text images. We
model elements in feature maps as the nodes of an undirected graph. A pooling
aggregator and a weighted aggregator are proposed to learn primitive
representations, which are transformed into high-level visual text
representations by graph convolutional networks. A Primitive REpresentation
learning Network (PREN) is constructed to use the visual text representations
for parallel decoding. Furthermore, by integrating visual text representations
into an encoder-decoder model with the 2D attention mechanism, we propose a
framework called PREN2D to alleviate the misalignment problem in
attention-based methods. Experimental results on both English and Chinese scene
text recognition tasks demonstrate that PREN keeps a balance between accuracy
and efficiency, while PREN2D achieves state-of-the-art performance.

negative:
Inspired by speech recognition, recent state-of-the-art algorithms mostly
consider scene text recognition as a sequence prediction problem. Though
achieving excellent performance, these methods usually neglect an important
fact that text in images are actually distributed in two-dimensional space. It
is a nature quite different from that of speech, which is essentially a
one-dimensional signal. In principle, directly compressing features of text
into a one-dimensional form may lose useful information and introduce extra
noise. In this paper, we approach scene text recognition from a two-dimensional
perspective. A simple yet effective model, called Character Attention Fully
Convolutional Network (CA-FCN), is devised for recognizing the text of
arbitrary shapes. Scene text recognition is realized with a semantic
segmentation network, where an attention mechanism for characters is adopted.
Combined with a word formation module, CA-FCN can simultaneously recognize the
script and predict the position of each character. Experiments demonstrate that
the proposed algorithm outperforms previous methods on both regular and
irregular text datasets. Moreover, it is proven to be more robust to imprecise
localizations in the text detection phase, which are very common in practice.

negative:
This paper investigates how to correct Chinese text errors with types of
mistaken, missing and redundant characters, which is common for Chinese native
speakers. Most existing models based on detect-correct framework can correct
mistaken characters errors, but they cannot deal with missing or redundant
characters. The reason is that lengths of sentences before and after correction
are not the same, leading to the inconsistence between model inputs and
outputs. Although the Seq2Seq-based or sequence tagging methods provide
solutions to the problem and achieved relatively good results on English
context, but they do not perform well in Chinese context according to our
experimental results. In our work, we propose a novel detect-correct framework
which is alignment-agnostic, meaning that it can handle both text aligned and
non-aligned occasions, and it can also serve as a cold start model when there
are no annotated data provided. Experimental results on three datasets
demonstrate that our method is effective and achieves the best performance
among existing published models.

negative:
Offline handwritten text recognition from images is an important problem for
enterprises attempting to digitize large volumes of handmarked scanned
documents/reports. Deep recurrent models such as Multi-dimensional LSTMs have
been shown to yield superior performance over traditional Hidden Markov Model
based approaches that suffer from the Markov assumption and therefore lack the
representational power of RNNs. In this paper we introduce a novel approach
that combines a deep convolutional network with a recurrent Encoder-Decoder
network to map an image to a sequence of characters corresponding to the text
present in the image. The entire model is trained end-to-end using Focal Loss,
an improvement over the standard Cross-Entropy loss that addresses the class
imbalance problem, inherent to text recognition. To enhance the decoding
capacity of the model, Beam Search algorithm is employed which searches for the
best sequence out of a set of hypotheses based on a joint distribution of
individual characters. Our model takes as input a downsampled version of the
original image thereby making it both computationally and memory efficient. The
experimental results were benchmarked against two publicly available datasets,
IAM and RIMES. We surpass the state-of-the-art word level accuracy on the
evaluation set of both datasets by 3.5% & 1.1%, respectively.

negative:
Natural language correction has the potential to help language learners
improve their writing skills. While approaches with separate classifiers for
different error types have high precision, they do not flexibly handle errors
such as redundancy or non-idiomatic phrasing. On the other hand, word and
phrase-based machine translation methods are not designed to cope with
orthographic errors, and have recently been outpaced by neural models.
Motivated by these issues, we present a neural network-based approach to
language correction. The core component of our method is an encoder-decoder
recurrent neural network with an attention mechanism. By operating at the
character level, the network avoids the problem of out-of-vocabulary words. We
illustrate the flexibility of our approach on dataset of noisy, user-generated
text collected from an English learner forum. When combined with a language
model, our method achieves a state-of-the-art $F_{0.5}$-score on the CoNLL 2014
Shared Task. We further demonstrate that training the network on additional
data with synthesized errors can improve performance.

negative:
Generating textual descriptions for images has been an attractive problem for
the computer vision and natural language processing researchers in recent
years. Dozens of models based on deep learning have been proposed to solve this
problem. The existing approaches are based on neural encoder-decoder structures
equipped with the attention mechanism. These methods strive to train decoders
to minimize the log likelihood of the next word in a sentence given the
previous ones, which results in the sparsity of the output space. In this work,
we propose a new approach to train decoders to regress the word embedding of
the next word with respect to the previous ones instead of minimizing the log
likelihood. The proposed method is able to learn and extract long-term
information and can generate longer fine-grained captions without introducing
any external memory cell. Furthermore, decoders trained by the proposed
technique can take the importance of the generated words into consideration
while generating captions. In addition, a novel semantic attention mechanism is
proposed that guides attention points through the image, taking the meaning of
the previously generated word into account. We evaluate the proposed approach
with the MS-COCO dataset. The proposed model outperformed the state of the art
models especially in generating longer captions. It achieved a CIDEr score
equal to 125.0 and a BLEU-4 score equal to 50.5, while the best scores of the
state of the art models are 117.1 and 48.0, respectively.

negative:
Attention-based encoder-decoder model has achieved impressive results for
both automatic speech recognition (ASR) and text-to-speech (TTS) tasks. This
approach takes advantage of the memorization capacity of neural networks to
learn the mapping from the input sequence to the output sequence from scratch,
without the assumption of prior knowledge such as the alignments. However, this
model is prone to overfitting, especially when the amount of training data is
limited. Inspired by SpecAugment and BERT, in this paper, we propose a semantic
mask based regularization for training such kind of end-to-end (E2E) model. The
idea is to mask the input features corresponding to a particular output token,
e.g., a word or a word-piece, in order to encourage the model to fill the token
based on the contextual information. While this approach is applicable to the
encoder-decoder framework with any type of neural network architecture, we
study the transformer-based model for ASR in this work. We perform experiments
on Librispeech 960h and TedLium2 data sets, and achieve the state-of-the-art
performance on the test set in the scope of E2E models.

negative:
In this technical report I present my method for automatic synthetic dataset
generation for object detection and demonstrate it on the video game League of
Legends. This report furthermore serves as a handbook on how to automatically
generate datasets and as an introduction on the dataset generation part of the
LeagueAI framework. The LeagueAI framework is a software framework that
provides detailed information about the game League of Legends based on the
same input a human player would have, namely vision. The framework allows
researchers and enthusiasts to develop their own intelligent agents or to
extract detailed information about the state of the game. A big problem of
machine vision applications usually is the laborious work of gathering large
amounts of hand labeled data. Thus, a crucial part of the vision pipeline of
the LeagueAI framework, the dataset generation, is presented in this report.
The method involves extracting image raw data from the game's 3D models and
combining them with the game background to create game-like synthetic images
and to generate the corresponding labels automatically. In an experiment I
compared a model trained on synthetic data to a model trained on hand labeled
data and a model trained on a combined dataset. The model trained on the
synthetic data showed higher detection precision on more classes and more
reliable tracking performance of the player character. The model trained on the
combined dataset did not perform better because of the different formats of the
older hand labeled dataset and the synthetic data.

negative:
Detection and recognition of text in natural images are two main problems in
the field of computer vision that have a wide variety of applications in
analysis of sports videos, autonomous driving, industrial automation, to name a
few. They face common challenging problems that are factors in how text is
represented and affected by several environmental conditions. The current
state-of-the-art scene text detection and/or recognition methods have exploited
the witnessed advancement in deep learning architectures and reported a
superior accuracy on benchmark datasets when tackling multi-resolution and
multi-oriented text. However, there are still several remaining challenges
affecting text in the wild images that cause existing methods to underperform
due to there models are not able to generalize to unseen data and the
insufficient labeled data. Thus, unlike previous surveys in this field, the
objectives of this survey are as follows: first, offering the reader not only a
review on the recent advancement in scene text detection and recognition, but
also presenting the results of conducting extensive experiments using a unified
evaluation framework that assesses pre-trained models of the selected methods
on challenging cases, and applies the same evaluation criteria on these
techniques. Second, identifying several existing challenges for detecting or
recognizing text in the wild images, namely, in-plane-rotation, multi-oriented
and multi-resolution text, perspective distortion, illumination reflection,
partial occlusion, complex fonts, and special characters. Finally, the paper
also presents insight into the potential research directions in this field to
address some of the mentioned challenges that are still encountering scene text
detection and recognition techniques.

negative:
In the big data era, the impetus to digitize the vast reservoirs of data
trapped in unstructured scanned documents such as invoices, bank documents and
courier receipts has gained fresh momentum. The scanning process often results
in the introduction of artifacts such as background noise, blur due to camera
motion, watermarkings, coffee stains, or faded text. These artifacts pose many
readability challenges to current text recognition algorithms and significantly
degrade their performance. Existing learning based denoising techniques require
a dataset comprising of noisy documents paired with cleaned versions. In such
scenarios, a model can be trained to generate clean documents from noisy
versions. However, very often in the real world such a paired dataset is not
available, and all we have for training our denoising model are unpaired sets
of noisy and clean images. This paper explores the use of GANs to generate
denoised versions of the noisy documents. In particular, where paired
information is available, we formulate the problem as an image-to-image
translation task i.e, translating a document from noisy domain ( i.e.,
background noise, blurred, faded, watermarked ) to a target clean document
using Generative Adversarial Networks (GAN). However, in the absence of paired
images for training, we employed CycleGAN which is known to learn a mapping
between the distributions of the noisy images to the denoised images using
unpaired data to achieve image-to-image translation for cleaning the noisy
documents. We compare the performance of CycleGAN for document cleaning tasks
using unpaired images with a Conditional GAN trained on paired data from the
same dataset. Experiments were performed on a public document dataset on which
different types of noise were artificially induced, results demonstrate that
CycleGAN learns a more robust mapping from the space of noisy to clean
documents.

negative:
Since the low quality of document images will greatly undermine the chances
of success in automatic text recognition and analysis, it is necessary to
assess the quality of document images uploaded in online business process, so
as to reject those images of low quality. In this paper, we attempt to achieve
document image quality assessment and our contributions are twofold. Firstly,
since document image quality assessment is more interested in text, we propose
a text line based framework to estimate document image quality, which is
composed of three stages: text line detection, text line quality prediction,
and overall quality assessment. Text line detection aims to find potential text
lines with a detector. In the text line quality prediction stage, the quality
score is computed for each text line with a CNN-based prediction model. The
overall quality of document images is finally assessed with the ensemble of all
text line quality. Secondly, to train the prediction model, a large-scale
dataset, comprising 52,094 text line images, is synthesized with diverse
attributes. For each text line image, a quality label is computed with a
piece-wise function. To demonstrate the effectiveness of the proposed
framework, comprehensive experiments are evaluated on two popular document
image quality assessment benchmarks. Our framework significantly outperforms
the state-of-the-art methods by large margins on the large and complicated
dataset.

negative:
The demand for on-device document recognition systems increases in
conjunction with the emergence of more strict privacy and security
requirements. In such systems, there is no data transfer from the end device to
a third-party information processing servers. The response time is vital to the
user experience of on-device document recognition. Combined with the
unavailability of discrete GPUs, powerful CPUs, or a large RAM capacity on
consumer-grade end devices such as smartphones, the time limitations put
significant constraints on the computational complexity of the applied
algorithms for on-device execution.
  In this work, we consider document location in an image without prior
knowledge of the document content or its internal structure. In accordance with
the published works, at least 5 systems offer solutions for on-device document
location. All these systems use a location method which can be considered
Hough-based. The precision of such systems seems to be lower than that of the
state-of-the-art solutions which were not designed to account for the limited
computational resources.
  We propose an advanced Hough-based method. In contrast with other approaches,
it accounts for the geometric invariants of the central projection model and
combines both edge and color features for document boundary detection. The
proposed method allowed for the second best result for SmartDoc dataset in
terms of precision, surpassed by U-net like neural network. When evaluated on a
more challenging MIDV-500 dataset, the proposed algorithm guaranteed the best
precision compared to published methods. Our method retained the applicability
to on-device computations.

negative:
In the Bag-of-Words (BoW) model based image retrieval task, the precision of
visual matching plays a critical role in improving retrieval performance.
Conventionally, local cues of a keypoint are employed. However, such strategy
does not consider the contextual evidences of a keypoint, a problem which would
lead to the prevalence of false matches. To address this problem, this paper
defines "true match" as a pair of keypoints which are similar on three levels,
i.e., local, regional, and global. Then, a principled probabilistic framework
is established, which is capable of implicitly integrating discriminative cues
from all these feature levels.
  Specifically, the Convolutional Neural Network (CNN) is employed to extract
features from regional and global patches, leading to the so-called "Deep
Embedding" framework. CNN has been shown to produce excellent performance on a
dozen computer vision tasks such as image classification and detection, but few
works have been done on BoW based image retrieval. In this paper, firstly we
show that proper pre-processing techniques are necessary for effective usage of
CNN feature. Then, in the attempt to fit it into our model, a novel indexing
structure called "Deep Indexing" is introduced, which dramatically reduces
memory usage.
  Extensive experiments on three benchmark datasets demonstrate that, the
proposed Deep Embedding method greatly promotes the retrieval accuracy when CNN
feature is integrated. We show that our method is efficient in terms of both
memory and time cost, and compares favorably with the state-of-the-art methods.

negative:
Synthesizing high-quality images from text descriptions is a challenging
problem in computer vision and has many practical applications. Samples
generated by existing text-to-image approaches can roughly reflect the meaning
of the given descriptions, but they fail to contain necessary details and vivid
object parts. In this paper, we propose Stacked Generative Adversarial Networks
(StackGAN) to generate 256x256 photo-realistic images conditioned on text
descriptions. We decompose the hard problem into more manageable sub-problems
through a sketch-refinement process. The Stage-I GAN sketches the primitive
shape and colors of the object based on the given text description, yielding
Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text
descriptions as inputs, and generates high-resolution images with
photo-realistic details. It is able to rectify defects in Stage-I results and
add compelling details with the refinement process. To improve the diversity of
the synthesized images and stabilize the training of the conditional-GAN, we
introduce a novel Conditioning Augmentation technique that encourages
smoothness in the latent conditioning manifold. Extensive experiments and
comparisons with state-of-the-arts on benchmark datasets demonstrate that the
proposed method achieves significant improvements on generating photo-realistic
images conditioned on text descriptions.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Recently, artificial intelligence technologies and algorithms have become a
major focus for advancements in treatment planning for radiation therapy. As
these are starting to become incorporated into the clinical workflow, a major
concern from clinicians is not whether the model is accurate, but whether the
model can express to a human operator when it does not know if its answer is
correct. We propose to use Monte Carlo dropout (MCDO) and the bootstrap
aggregation (bagging) technique on deep learning models to produce uncertainty
estimations for radiation therapy dose prediction. We show that both models are
capable of generating a reasonable uncertainty map, and, with our proposed
scaling technique, creating interpretable uncertainties and bounds on the
prediction and any relevant metrics. Performance-wise, bagging provides
statistically significant reduced loss value and errors in most of the metrics
investigated in this study. The addition of bagging was able to further reduce
errors by another 0.34% for Dmean and 0.19% for Dmax, on average, when compared
to the baseline framework. Overall, the bagging framework provided
significantly lower MAE of 2.62, as opposed to the baseline framework's MAE of
2.87. The usefulness of bagging, from solely a performance standpoint, does
highly depend on the problem and the acceptable predictive error, and its high
upfront computational cost during training should be factored in to deciding
whether it is advantageous to use it. In terms of deployment with uncertainty
estimations turned on, both frameworks offer the same performance time of about
12 seconds. As an ensemble-based metaheuristic, bagging can be used with
existing machine learning architectures to improve stability and performance,
and MCDO can be applied to any deep learning models that have dropout as part
of their architecture.

negative:
Despite recent advances in the field of supervised deep learning for text
line segmentation, unsupervised deep learning solutions are beginning to gain
popularity. In this paper, we present an unsupervised deep learning method that
embeds document image patches to a compact Euclidean space where distances
correspond to a coarse text line pattern similarity. Once this space has been
produced, text line segmentation can be easily implemented using standard
techniques with the embedded feature vectors. To train the model, we extract
random pairs of document image patches with the assumption that neighbour
patches contain a similar coarse trend of text lines, whereas if one of them is
rotated, they contain different coarse trends of text lines. Doing well on this
task requires the model to learn to recognize the text lines and their salient
parts. The benefit of our approach is zero manual labelling effort. We evaluate
the method qualitatively and quantitatively on several variants of text line
segmentation datasets to demonstrate its effectivity.

negative:
Additive Manufacturing (AM) is a crucial component of the smart industry. In
this paper, we propose an automated quality grading system for the AM process
using a deep convolutional neural network (CNN) model. The CNN model is trained
offline using the images of the internal and surface defects in the
layer-by-layer deposition of materials and tested online by studying the
performance of detecting and classifying the failure in AM process at different
extruder speeds and temperatures. The model demonstrates the accuracy of 94%
and specificity of 96%, as well as above 75% in three classifier measures of
the Fscore, the sensitivity, and precision for classifying the quality of the
printing process in five grades in real-time. The proposed online model adds an
automated, consistent, and non-contact quality control signal to the AM process
that eliminates the manual inspection of parts after they are entirely built.
The quality monitoring signal can also be used by the machine to suggest
remedial actions by adjusting the parameters in real-time. The proposed quality
predictive model serves as a proof-of-concept for any type of AM machines to
produce reliable parts with fewer quality hiccups while limiting the waste of
both time and materials.

negative:
Annotating words in a historical document image archive for word image
recognition purpose demands time and skilled human resource (like historians,
paleographers). In a real-life scenario, obtaining sample images for all
possible words is also not feasible. However, Zero-shot learning methods could
aptly be used to recognize unseen/out-of-lexicon words in such historical
document images. Based on previous state-of-the-art method for zero-shot word
recognition Pho(SC)Net, we propose a hybrid model based on the CTC framework
(Pho(SC)-CTC) that takes advantage of the rich features learned by Pho(SC)Net
followed by a connectionist temporal classification (CTC) framework to perform
the final classification. Encouraging results were obtained on two publicly
available historical document datasets and one synthetic handwritten dataset,
which justifies the efficacy of Pho(SC)-CTC and Pho(SC)Net.

negative:
Recognizing text in natural images is a challenging task with many unsolved
problems. Different from those in documents, words in natural images often
possess irregular shapes, which are caused by perspective distortion, curved
character placement, etc. We propose RARE (Robust text recognizer with
Automatic REctification), a recognition model that is robust to irregular text.
RARE is a specially-designed deep neural network, which consists of a Spatial
Transformer Network (STN) and a Sequence Recognition Network (SRN). In testing,
an image is firstly rectified via a predicted Thin-Plate-Spline (TPS)
transformation, into a more "readable" image for the following SRN, which
recognizes text through a sequence recognition approach. We show that the model
is able to recognize several types of irregular text, including perspective
text and curved text. RARE is end-to-end trainable, requiring only images and
associated text labels, making it convenient to train and deploy the model in
practical systems. State-of-the-art or highly-competitive performance achieved
on several benchmarks well demonstrates the effectiveness of the proposed
model.

negative:
Visual text recognition is undoubtedly one of the most extensively researched
topics in computer vision. Great progress have been made to date, with the
latest models starting to focus on the more practical "in-the-wild" setting.
However, a salient problem still hinders practical deployment -- prior arts
mostly struggle with recognising unseen (or rarely seen) character sequences.
In this paper, we put forward a novel framework to specifically tackle this
"unseen" problem. Our framework is iterative in nature, in that it utilises
predicted knowledge of character sequences from a previous iteration, to
augment the main network in improving the next prediction. Key to our success
is a unique cross-modal variational autoencoder to act as a feedback module,
which is trained with the presence of textual error distribution data. This
module importantly translate a discrete predicted character space, to a
continuous affine transformation parameter space used to condition the visual
feature map at next iteration. Experiments on common datasets have shown
competitive performance over state-of-the-arts under the conventional setting.
Most importantly, under the new disjoint setup where train-test labels are
mutually exclusive, ours offers the best performance thus showcasing the
capability of generalising onto unseen words.

negative:
Recently, there has been a growing interest in end-to-end speech recognition
that directly transcribes speech to text without any predefined alignments. In
this paper, we explore the use of attention-based encoder-decoder model for
Mandarin speech recognition on a voice search task. Previous attempts have
shown that applying attention-based encoder-decoder to Mandarin speech
recognition was quite difficult due to the logographic orthography of Mandarin,
the large vocabulary and the conditional dependency of the attention model. In
this paper, we use character embedding to deal with the large vocabulary.
Several tricks are used for effective model training, including L2
regularization, Gaussian weight noise and frame skipping. We compare two
attention mechanisms and use attention smoothing to cover long context in the
attention model. Taken together, these tricks allow us to finally achieve a
character error rate (CER) of 3.58% and a sentence error rate (SER) of 7.43% on
the MiTV voice search dataset. While together with a trigram language model,
CER and SER reach 2.81% and 5.77%, respectively.

negative:
Recently, there has been an increasing interest in end-to-end speech
recognition that directly transcribes speech to text without any predefined
alignments. One approach is the attention-based encoder-decoder framework that
learns a mapping between variable-length input and output sequences in one step
using a purely data-driven method. The attention model has often been shown to
improve the performance over another end-to-end approach, the Connectionist
Temporal Classification (CTC), mainly because it explicitly uses the history of
the target character without any conditional independence assumptions. However,
we observed that the performance of the attention has shown poor results in
noisy condition and is hard to learn in the initial training stage with long
input sequences. This is because the attention model is too flexible to predict
proper alignments in such cases due to the lack of left-to-right constraints as
used in CTC. This paper presents a novel method for end-to-end speech
recognition to improve robustness and achieve fast convergence by using a joint
CTC-attention model within the multi-task learning framework, thereby
mitigating the alignment issue. An experiment on the WSJ and CHiME-4 tasks
demonstrates its advantages over both the CTC and attention-based
encoder-decoder baselines, showing 5.4-14.6% relative improvements in Character
Error Rate (CER).

negative:
Many tasks are related to determining if a particular text string exists in
an image. In this work, we propose a new framework that learns this task in an
end-to-end way. The framework takes an image and a text string as input and
then outputs the probability of the text string being present in the image.
This is the first end-to-end framework that learns such relationships between
text and images in scene text area. The framework does not require explicit
scene text detection or recognition and thus no bounding box annotations are
needed for it. It is also the first work in scene text area that tackles suh a
weakly labeled problem. Based on this framework, we developed a model called
Guided Attention. Our designed model achieves much better results than several
state-of-the-art scene text reading based solutions for a challenging Street
View Business Matching task. The task tries to find correct business names for
storefront images and the dataset we collected for it is substantially larger,
and more challenging than existing scene text dataset. This new real-world task
provides a new perspective for studying scene text related problems. We also
demonstrate the uniqueness of our task via a comparison between our problem and
a typical Visual Question Answering problem.

negative:
Arabic text recognition is a challenging task because of the cursive nature
of Arabic writing system, its joint writing scheme, the large number of
ligatures and many other challenges. Deep Learning DL models achieved
significant progress in numerous domains including computer vision and sequence
modelling. This paper presents a model that can recognize Arabic text that was
printed using multiple font types including fonts that mimic Arabic handwritten
scripts. The proposed model employs a hybrid DL network that can recognize
Arabic printed text without the need for character segmentation. The model was
tested on a custom dataset comprised of over two million word samples that were
generated using 18 different Arabic font types. The objective of the testing
process was to assess the model capability in recognizing a diverse set of
Arabic fonts representing a varied cursive styles. The model achieved good
results in recognizing characters and words and it also achieved promising
results in recognizing characters when it was tested on unseen data. The
prepared model, the custom datasets and the toolkit for generating similar
datasets are made publicly available, these tools can be used to prepare models
for recognizing other font types as well as to further extend and enhance the
performance of the proposed model.

negative:
"If I provide you a face image of mine (without telling you the actual age
when I took the picture) and a large amount of face images that I crawled
(containing labeled faces of different ages but not necessarily paired), can
you show me what I would look like when I am 80 or what I was like when I was
5?" The answer is probably a "No." Most existing face aging works attempt to
learn the transformation between age groups and thus would require the paired
samples as well as the labeled query image. In this paper, we look at the
problem from a generative modeling perspective such that no paired samples is
required. In addition, given an unlabeled image, the generative model can
directly produce the image with desired age attribute. We propose a conditional
adversarial autoencoder (CAAE) that learns a face manifold, traversing on which
smooth age progression and regression can be realized simultaneously. In CAAE,
the face is first mapped to a latent vector through a convolutional encoder,
and then the vector is projected to the face manifold conditional on age
through a deconvolutional generator. The latent vector preserves personalized
face features (i.e., personality) and the age condition controls progression
vs. regression. Two adversarial networks are imposed on the encoder and
generator, respectively, forcing to generate more photo-realistic faces.
Experimental results demonstrate the appealing performance and flexibility of
the proposed framework by comparing with the state-of-the-art and ground truth.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
We introduce a novel type of text representation that preserves the 2D layout
of a document. This is achieved by encoding each document page as a
two-dimensional grid of characters. Based on this representation, we present a
generic document understanding pipeline for structured documents. This pipeline
makes use of a fully convolutional encoder-decoder network that predicts a
segmentation mask and bounding boxes. We demonstrate its capabilities on an
information extraction task from invoices and show that it significantly
outperforms approaches based on sequential text or document images.

negative:
Document Layout Analysis is a fundamental step in Handwritten Text Processing
systems, from the extraction of the text lines to the type of zone it belongs
to. We present a system based on artificial neural networks which is able to
determine not only the baselines of text lines present in the document, but
also performs geometric and logic layout analysis of the document. Experiments
in three different datasets demonstrate the potential of the method and show
competitive results with respect to state-of-the-art methods.

negative:
Image denoising is always a challenging task in the field of computer vision
and image processing. In this paper, we have proposed an encoder-decoder model
with direct attention, which is capable of denoising and reconstruct highly
corrupted images. Our model consists of an encoder and a decoder, where the
encoder is a convolutional neural network and decoder is a multilayer Long
Short-Term memory network. In the proposed model, the encoder reads an image
and catches the abstraction of that image in a vector, where decoder takes that
vector as well as the corrupted image to reconstruct a clean image. We have
trained our model on MNIST handwritten digit database after making lower half
of every image as black as well as adding noise top of that. After a massive
destruction of the images where it is hard for a human to understand the
content of those images, our model can retrieve that image with minimal error.
Our proposed model has been compared with convolutional encoder-decoder, where
our model has performed better at generating missing part of the images than
convolutional autoencoder.

negative:
Handwritten document image binarization is challenging due to high
variability in the written content and complex background attributes such as
page style, paper quality, stains, shadow gradients, and non-uniform
illumination. While the traditional thresholding methods do not effectively
generalize on such challenging real-world scenarios, deep learning-based
methods have performed relatively well when provided with sufficient training
data. However, the existing datasets are limited in size and diversity. This
work proposes LS-HDIB - a large-scale handwritten document image binarization
dataset containing over a million document images that span numerous real-world
scenarios. Additionally, we introduce a novel technique that uses a combination
of adaptive thresholding and seamless cloning methods to create the dataset
with accurate ground truths. Through an extensive quantitative and qualitative
evaluation over eight different deep learning based models, we demonstrate the
enhancement in the performance of these models when trained on the LS-HDIB
dataset and tested on unseen images.

negative:
Identity documents recognition is an important sub-field of document
analysis, which deals with tasks of robust document detection, type
identification, text fields recognition, as well as identity fraud prevention
and document authenticity validation given photos, scans, or video frames of an
identity document capture. Significant amount of research has been published on
this topic in recent years, however a chief difficulty for such research is
scarcity of datasets, due to the subject matter being protected by security
requirements. A few datasets of identity documents which are available lack
diversity of document types, capturing conditions, or variability of document
field values. In addition, the published datasets were typically designed only
for a subset of document recognition problems, not for a complex identity
document analysis. In this paper, we present a dataset MIDV-2020 which consists
of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock
identity documents, each with unique text field values and unique artificially
generated faces, with rich annotation. For the presented benchmark dataset
baselines are provided for such tasks as document location and identification,
text fields recognition, and face detection. With 72409 annotated images in
total, to the date of publication the proposed dataset is the largest publicly
available identity documents dataset with variable artificially generated data,
and we believe that it will prove invaluable for advancement of the field of
document analysis and recognition. The dataset is available for download at
ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr .

negative:
Despite significant progress on current state-of-the-art image generation
models, synthesis of document images containing multiple and complex object
layouts is a challenging task. This paper presents a novel approach, called
DocSynth, to automatically synthesize document images based on a given layout.
In this work, given a spatial layout (bounding boxes with object categories) as
a reference by the user, our proposed DocSynth model learns to generate a set
of realistic document images consistent with the defined layout. Also, this
framework has been adapted to this work as a superior baseline model for
creating synthetic document image datasets for augmenting real data during
training for document layout analysis tasks. Different sets of learning
objectives have been also used to improve the model performance.
Quantitatively, we also compare the generated results of our model with real
data using standard evaluation metrics. The results highlight that our model
can successfully generate realistic and diverse document images with multiple
objects. We also present a comprehensive qualitative analysis summary of the
different scopes of synthetic image generation tasks. Lastly, to our knowledge
this is the first work of its kind.

negative:
Class labels have been empirically shown useful in improving the sample
quality of generative adversarial nets (GANs). In this paper, we mathematically
study the properties of the current variants of GANs that make use of class
label information. With class aware gradient and cross-entropy decomposition,
we reveal how class labels and associated losses influence GAN's training.
Based on that, we propose Activation Maximization Generative Adversarial
Networks (AM-GAN) as an advanced solution. Comprehensive experiments have been
conducted to validate our analysis and evaluate the effectiveness of our
solution, where AM-GAN outperforms other strong baselines and achieves
state-of-the-art Inception Score (8.91) on CIFAR-10. In addition, we
demonstrate that, with the Inception ImageNet classifier, Inception Score
mainly tracks the diversity of the generator, and there is, however, no
reliable evidence that it can reflect the true sample quality. We thus propose
a new metric, called AM Score, to provide a more accurate estimation of the
sample quality. Our proposed model also outperforms the baseline methods in the
new metric.

negative:
We study how to train a student deep neural network for visual recognition by
distilling knowledge from a blackbox teacher model in a data-efficient manner.
Progress on this problem can significantly reduce the dependence on large-scale
datasets for learning high-performing visual recognition models. There are two
major challenges. One is that the number of queries into the teacher model
should be minimized to save computational and/or financial costs. The other is
that the number of images used for the knowledge distillation should be small;
otherwise, it violates our expectation of reducing the dependence on
large-scale datasets. To tackle these challenges, we propose an approach that
blends mixup and active learning. The former effectively augments the few
unlabeled images by a big pool of synthetic images sampled from the convex hull
of the original images, and the latter actively chooses from the pool hard
examples for the student neural network and query their labels from the teacher
model. We validate our approach with extensive experiments.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
This work analyses the impact of self-supervised pre-training on document
images in the context of document image classification. While previous
approaches explore the effect of self-supervision on natural images, we show
that patch-based pre-training performs poorly on document images because of
their different structural properties and poor intra-sample semantic
information. We propose two context-aware alternatives to improve performance
on the Tobacco-3482 image classification task. We also propose a novel method
for self-supervision, which makes use of the inherent multi-modality of
documents (image and text), which performs better than other popular
self-supervised methods, including supervised ImageNet pre-training, on
document image classification scenarios with a limited amount of data.

negative:
Generative Adversarial Networks have been crucial in the developments made in
unsupervised learning in recent times. Exemplars of image synthesis from text
or other images, these networks have shown remarkable improvements over
conventional methods in terms of performance. Trained on the adversarial
training philosophy, these networks aim to estimate the potential distribution
from the real data and then use this as input to generate the synthetic data.
Based on this fundamental principle, several frameworks can be generated that
are paragon implementations in several real-life applications such as art
synthesis, generation of high resolution outputs and synthesis of images from
human drawn sketches, to name a few. While theoretically GANs present better
results and prove to be an improvement over conventional methods in many
factors, the implementation of these frameworks for dedicated applications
remains a challenge. This study explores and presents a taxonomy of these
frameworks and their use in various image to image synthesis and text to image
synthesis applications. The basic GANs, as well as a variety of different niche
frameworks, are critically analyzed. The advantages of GANs for image
generation over conventional methods as well their disadvantages amongst other
frameworks are presented. The future applications of GANs in industries such as
healthcare, art and entertainment are also discussed.

negative:
Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.

negative:
Scene text recognition has been a hot topic in computer vision. Recent
methods adopt the attention mechanism for sequence prediction which achieve
convincing results. However, we argue that the existing attention mechanism
faces the problem of attention diffusion, in which the model may not focus on a
certain character area. In this paper, we propose Gaussian Constrained
Attention Network to deal with this problem. It is a 2D attention-based method
integrated with a novel Gaussian Constrained Refinement Module, which predicts
an additional Gaussian mask to refine the attention weights. Different from
adopting an additional supervision on the attention weights simply, our
proposed method introduces an explicit refinement. In this way, the attention
weights will be more concentrated and the attention-based recognition network
achieves better performance. The proposed Gaussian Constrained Refinement
Module is flexible and can be applied to existing attention-based methods
directly. The experiments on several benchmark datasets demonstrate the
effectiveness of our proposed method. Our code has been available at
https://github.com/Pay20Y/GCAN.

negative:
Automated recognition of texts in scenes has been a research challenge for
years, largely due to the arbitrary variation of text appearances in
perspective distortion, text line curvature, text styles and different types of
imaging artifacts. The recent deep networks are capable of learning robust
representations with respect to imaging artifacts and text style changes, but
still face various problems while dealing with scene texts with perspective and
curvature distortions. This paper presents an end-to-end trainable scene text
recognition system (ESIR) that iteratively removes perspective distortion and
text line curvature as driven by better scene text recognition performance. An
innovative rectification network is developed which employs a novel
line-fitting transformation to estimate the pose of text lines in scenes. In
addition, an iterative rectification pipeline is developed where scene text
distortions are corrected iteratively towards a fronto-parallel view. The ESIR
is also robust to parameter initialization and the training needs only scene
text images and word-level annotations as required by most scene text
recognition systems. Extensive experiments over a number of public datasets
show that the proposed ESIR is capable of rectifying scene text distortions
accurately, achieving superior recognition performance for both normal scene
text images and those suffering from perspective and curvature distortions.

negative:
Recent progress has been made on developing a unified framework for joint
text detection and recognition in natural images, but existing joint models
were mostly built on two-stage framework by involving ROI pooling, which can
degrade the performance on recognition task. In this work, we propose
convolutional character networks, referred as CharNet, which is an one-stage
model that can process two tasks simultaneously in one pass. CharNet directly
outputs bounding boxes of words and characters, with corresponding character
labels. We utilize character as basic element, allowing us to overcome the main
difficulty of existing approaches that attempted to optimize text detection
jointly with a RNN-based recognition branch. In addition, we develop an
iterative character detection approach able to transform the ability of
character detection learned from synthetic data to real-world images. These
technical improvements result in a simple, compact, yet powerful one-stage
model that works reliably on multi-orientation and curved text. We evaluate
CharNet on three standard benchmarks, where it consistently outperforms the
state-of-the-art approaches [25, 24] by a large margin, e.g., with improvements
of 65.33%->71.08% (with generic lexicon) on ICDAR 2015, and 54.0%->69.23% on
Total-Text, on end-to-end text recognition. Code is available at:
https://github.com/MalongTech/research-charnet.

negative:
State-of-the-art approaches for image captioning require supervised training
data consisting of captions with paired image data. These methods are typically
unable to use unsupervised data such as textual data with no corresponding
images, which is a much more abundant commodity. We here propose a novel way of
using such textual data by artificially generating missing visual information.
We evaluate this learning approach on a newly designed model that detects
visual concepts present in an image and feed them to a reviewer-decoder
architecture with an attention mechanism. Unlike previous approaches that
encode visual concepts using word embeddings, we instead suggest using regional
image features which capture more intrinsic information. The main benefit of
this architecture is that it synthesizes meaningful thought vectors that
capture salient image properties and then applies a soft attentive decoder to
decode the thought vectors and generate image captions. We evaluate our model
on both Microsoft COCO and Flickr30K datasets and demonstrate that this model
combined with our semi-supervised learning method can largely improve
performance and help the model to generate more accurate and diverse captions.

negative:
Spelling normalization for low resource languages is a challenging task
because the patterns are hard to predict and large corpora are usually required
to collect enough examples. This work shows a comparison of a neural model and
character language models with varying amounts on target language data. Our
usage scenario is interactive correction with nearly zero amounts of training
examples, improving models as more data is collected, for example within a chat
app. Such models are designed to be incrementally improved as feedback is given
from users. In this work, we design a knowledge-base and prediction model
embedded system for spelling correction in low-resource languages. Experimental
results on multiple languages show that the model could become effective with a
small amount of data. We perform experiments on both natural and synthetic
data, as well as on data from two endangered languages (Ainu and Griko). Last,
we built a prototype system that was used for a small case study on Hinglish,
which further demonstrated the suitability of our approach in real world
scenarios.

negative:
This work addresses the problem of Question Answering (QA) on handwritten
document collections. Unlike typical QA and Visual Question Answering (VQA)
formulations where the answer is a short text, we aim to locate a document
snippet where the answer lies. The proposed approach works without recognizing
the text in the documents. We argue that the recognition-free approach is
suitable for handwritten documents and historical collections where robust text
recognition is often difficult. At the same time, for human users, document
image snippets containing answers act as a valid alternative to textual
answers. The proposed approach uses an off-the-shelf deep embedding network
which can project both textual words and word images into a common sub-space.
This embedding bridges the textual and visual domains and helps us retrieve
document snippets that potentially answer a question. We evaluate results of
the proposed approach on two new datasets: (i) HW-SQuAD: a synthetic,
handwritten document image counterpart of SQuAD1.0 dataset and (ii) BenthamQA:
a smaller set of QA pairs defined on documents from the popular Bentham
manuscripts collection. We also present a thorough analysis of the proposed
recognition-free approach compared to a recognition-based approach which uses
text recognized from the images using an OCR. Datasets presented in this work
are available to download at docvqa.org

negative:
Cloud computing offers the potential to help scientists to process massive
number of computing resources often required in machine learning application
such as computer vision problems. This proposal would like to show that which
benefits can be obtained from cloud in order to help medical image analysis
users (including scientists, clinicians, and research institutes). As security
and privacy of algorithms are important for most of algorithms inventors, these
algorithms can be hidden in a cloud to allow the users to use the algorithms as
a package without any access to see/change their inside. In another word, in
the user part, users send their images to the cloud and configure the algorithm
via an interface. In the cloud part, the algorithms are applied to this image
and the results are returned back to the user. My proposal has two parts: (1)
investigate the potential of cloud computing for computer vision problems and
(2) study the components of a proposed cloud-based framework for medical image
analysis application and develop them (depending on the length of the
internship). The investigation part will involve a study on several aspects of
the problem including security, usability (for medical end users of the
service), appropriate programming abstractions for vision problems, scalability
and resource requirements. In the second part of this proposal I am going to
thoroughly study of the proposed framework components and their relations and
develop them. The proposed cloud-based framework includes an integrated
environment to enable scientists and clinicians to access to the previous and
current medical image analysis algorithms using a handful user interface
without any access to the algorithm codes and procedures.

negative:
One of the major prerequisites for any deep learning approach is the
availability of large-scale training data. When dealing with scanned document
images in real world scenarios, the principal information of its content is
stored in the layout itself. In this work, we have proposed an automated deep
generative model using Graph Neural Networks (GNNs) to generate synthetic data
with highly variable and plausible document layouts that can be used to train
document interpretation systems, in this case, specially in digital mailroom
applications. It is also the first graph-based approach for document layout
generation task experimented on administrative document images, in this case,
invoices.

negative:
A lot of research has been devoted to identity documents analysis and
recognition on mobile devices. However, no publicly available datasets designed
for this particular problem currently exist. There are a few datasets which are
useful for associated subtasks but in order to facilitate a more comprehensive
scientific and technical approach to identity document recognition more
specialized datasets are required. In this paper we present a Mobile Identity
Document Video dataset (MIDV-500) consisting of 500 video clips for 50
different identity document types with ground truth which allows to perform
research in a wide scope of document analysis problems. The paper presents
characteristics of the dataset and evaluation results for existing methods of
face detection, text line recognition, and document fields data extraction.
Since an important feature of identity documents is their sensitiveness as they
contain personal data, all source document images used in MIDV-500 are either
in public domain or distributed under public copyright licenses.
  The main goal of this paper is to present a dataset. However, in addition and
as a baseline, we present evaluation results for existing methods for face
detection, text line recognition, and document data extraction, using the
presented dataset.
  (The dataset is available for download at ftp://smartengines.com/midv-500/.)

negative:
Graphical elements: particularly tables and figures contain a visual summary
of the most valuable information contained in a document. Therefore,
localization of such graphical objects in the document images is the initial
step to understand the content of such graphical objects or document images. In
this paper, we present a novel end-to-end trainable deep learning based
framework to localize graphical objects in the document images called as
Graphical Object Detection (GOD). Our framework is data-driven and does not
require any heuristics or meta-data to locate graphical objects in the document
images. The GOD explores the concept of transfer learning and domain adaptation
to handle scarcity of labeled training images for graphical object detection
task in the document images. Performance analysis carried out on the various
public benchmark data sets: ICDAR-2013, ICDAR-POD2017,and UNLV shows that our
model yields promising results as compared to state-of-the-art techniques.

negative:
When digitizing a document into an image, it is common to include a
surrounding border region to visually indicate that the entire document is
present in the image. However, this border should be removed prior to automated
processing. In this work, we present a deep learning based system, PageNet,
which identifies the main page region in an image in order to segment content
from both textual and non-textual border noise. In PageNet, a Fully
Convolutional Network obtains a pixel-wise segmentation which is post-processed
into the output quadrilateral region. We evaluate PageNet on 4 collections of
historical handwritten documents and obtain over 94% mean intersection over
union on all datasets and approach human performance on 2 of these collections.
Additionally, we show that PageNet can segment documents that are overlayed on
top of other documents.

negative:
There is a large collection of Handwritten English paper documents of
Historical and Scientific importance. But paper documents are not recognized
directly by computer. Hence the closest way of indexing these documents is by
storing their document digital image. Hence a large database of document images
can replace the paper documents. But the document and data corresponding to
each image cannot be directly recognized by the computer.
  This paper applies the technique of word spotting using Modified Character
Shape Code to Handwritten English document images for quick and efficient query
search of words on a database of document images. It is different from other
Word Spotting techniques as it implements two level of selection for word
segments to match search query. First based on word size and then based on
character shape code of query. It makes the process faster and more efficient
and reduces the need of multiple pre-processing.

negative:
This paper describes the short-term competition on the Components
Segmentation Task of Document Photos that was prepared in the context of the
16th International Conference on Document Analysis and Recognition (ICDAR
2021). This competition aims to bring together researchers working in the field
of identification document image processing and provides them a suitable
benchmark to compare their techniques on the component segmentation task of
document images. Three challenge tasks were proposed entailing different
segmentation assignments to be performed on a provided dataset. The collected
data are from several types of Brazilian ID documents, whose personal
information was conveniently replaced. There were 16 participants whose results
obtained for some or all the three tasks show different rates for the adopted
metrics, like Dice Similarity Coefficient ranging from 0.06 to 0.99. Different
Deep Learning models were applied by the entrants with diverse strategies to
achieve the best results in each of the tasks. Obtained results show that the
currently applied methods for solving one of the proposed tasks (document
boundary detection) are already well established. However, for the other two
challenge tasks (text zone and handwritten sign detection) research and
development of more robust approaches are still required to achieve acceptable
results.

negative:
Documents often exhibit various forms of degradation, which make it hard to
be read and substantially deteriorate the performance of an OCR system. In this
paper, we propose an effective end-to-end framework named Document Enhancement
Generative Adversarial Networks (DE-GAN) that uses the conditional GANs (cGANs)
to restore severely degraded document images. To the best of our knowledge,
this practice has not been studied within the context of generative adversarial
deep networks. We demonstrate that, in different tasks (document clean up,
binarization, deblurring and watermark removal), DE-GAN can produce an enhanced
version of the degraded document with a high quality. In addition, our approach
provides consistent improvements compared to state-of-the-art methods over the
widely used DIBCO 2013, DIBCO 2017 and H-DIBCO 2018 datasets, proving its
ability to restore a degraded document image to its ideal condition. The
obtained results on a wide variety of degradation reveal the flexibility of the
proposed model to be exploited in other document enhancement problems.

negative:
The antinuclear antibody detection with human epithelial cells is a popular
approach for autoimmune diseases diagnosis. The manual evaluation demands time,
effort and capital, and automation in screening can greatly aid the physicians
in these respects. In this work, we employ simple, efficient and visually more
interpretable, class-specific features which defined based on the visual
characteristics of each class. We believe that defining features with a good
visual interpretation, is indeed important in a scenario, where such an
approach is used in an interactive CAD system for pathologists. Considering
that problem consists of few classes, and our rather simplistic feature
definitions, frameworks can be structured as hierarchies of various binary
classifiers. These variants include frameworks which are earlier explored and
some which are not explored for this task. We perform various experiments which
include traditional texture features and demonstrate the effectiveness of
class-specific features in various frameworks. We make insightful comparisons
between different types of classification frameworks given their silent aspects
and pros and cons over each other. We also demonstrate an experiment with only
intermediates samples for testing. The proposed work yields encouraging results
with respect to the state-of-the-art and highlights the role of class-specific
features in different classification frameworks.

negative:
Video text spotting is still an important research topic due to its various
real-applications. Previous approaches usually fall into the four-staged
pipeline: text detection in individual images, framewisely recognizing
localized text regions, tracking text streams and generating final results with
complicated post-processing skills, which might suffer from the huge
computational cost as well as the interferences of low-quality text. In this
paper, we propose a fast and robust video text spotting framework by only
recognizing the localized text one-time instead of frame-wisely recognition.
Specifically, we first obtain text regions in videos with a well-designed
spatial-temporal detector. Then we concentrate on developing a novel text
recommender for selecting the highest-quality text from text streams and only
recognizing the selected ones. Here, the recommender assembles text tracking,
quality scoring and recognition into an end-to-end trainable module, which not
only avoids the interferences from low-quality text but also dramatically
speeds up the video text spotting process. In addition, we collect a larger
scale video text dataset (LSVTD) for promoting the video text spotting
community, which contains 100 text videos from 22 different real-life
scenarios. Extensive experiments on two public benchmarks show that our method
greatly speeds up the recognition process averagely by 71 times compared with
the frame-wise manner, and also achieves the remarkable state-of-the-art.

negative:
Machine learning techniques have excelled in the automatic semantic analysis
of images, reaching human-level performances on challenging benchmarks. Yet,
the semantic analysis of videos remains challenging due to the significantly
higher dimensionality of the input data, respectively, the significantly higher
need for annotated training examples. By studying the automatic recognition of
German sign language videos, we demonstrate that on the relatively scarce
training data of 2.800 videos, modern deep learning architectures for video
analysis (such as ResNeXt) along with transfer learning on large gesture
recognition tasks, can achieve about 75% character accuracy. Considering that
this leaves us with a probability of under 25% that a 5 letter word is spelled
correctly, spell-correction systems are crucial for producing readable outputs.
The contribution of this paper is to propose a convolutional neural network for
spell-correction that expects the softmax outputs of the character recognition
network (instead of a misspelled word) as an input. We demonstrate that purely
learning on softmax inputs in combination with scarce training data yields
overfitting as the network learns the inputs by heart. In contrast, training
the network on several variants of the logits of the classification output i.e.
scaling by a constant factor, adding of random noise, mixing of softmax and
hardmax inputs or purely training on hardmax inputs, leads to better
generalization while benefitting from the significant information hidden in
these outputs (that have 98% top-5 accuracy), yielding a readable text despite
the comparably low character accuracy.

negative:
Automatic document content processing is affected by artifacts caused by the
shape of the paper, non-uniform and diverse color of lighting conditions.
Fully-supervised methods on real data are impossible due to the large amount of
data needed. Hence, the current state of the art deep learning models are
trained on fully or partially synthetic images. However, document shadow or
shading removal results still suffer because: (a) prior methods rely on
uniformity of local color statistics, which limit their application on
real-scenarios with complex document shapes and textures and; (b) synthetic or
hybrid datasets with non-realistic, simulated lighting conditions are used to
train the models. In this paper we tackle these problems with our two main
contributions. First, a physically constrained learning-based method that
directly estimates document reflectance based on intrinsic image formation
which generalizes to challenging illumination conditions. Second, a new dataset
that clearly improves previous synthetic ones, by adding a large range of
realistic shading and diverse multi-illuminant conditions, uniquely customized
to deal with documents in-the-wild. The proposed architecture works in a
self-supervised manner where only the synthetic texture is used as a weak
training signal (obviating the need for very costly ground truth with
disentangled versions of shading and reflectance). The proposed approach leads
to a significant generalization of document reflectance estimation in real
scenes with challenging illumination. We extensively evaluate on the real
benchmark datasets available for intrinsic image decomposition and document
shadow removal tasks. Our reflectance estimation scheme, when used as a
pre-processing step of an OCR pipeline, shows a 26% improvement of character
error rate (CER), thus, proving the practical applicability.

negative:
Document image binarization is the initial step and a crucial in many
document analysis and recognition scheme. In fact, it is still a relevant
research subject and a fundamental challenge due to its importance and
influence. This paper provides an original multi-phases system that hybridizes
various efficient image thresholding methods in order to get the best
binarization output. First, to improve contrast in particularly defective
images, the application of CLAHE algorithm is suggested and justified. We then
use a cooperative technique to segment image into two separated classes. At the
end, a special transformation is applied for the purpose of removing scattered
noise and of correcting characters forms. Experimentations demonstrate the
precision and the robustness of our framework applied on historical degraded
documents images within three benchmarks compared to other noted methods.

negative:
The digital camera captured document images may often be warped and distorted
due to different camera angles or document surfaces. A robust technique is
needed to solve this kind of distortion. The research on dewarping of the
document suffers due to the limited availability of benchmark public dataset.
In recent times, deep learning based approaches are used to solve the problems
accurately. To train most of the deep neural networks a large number of
document images is required and generating such a large volume of document
images manually is difficult. In this paper, we propose a technique to generate
a synthetic warped image from a flat-bedded scanned document image. It is done
by calculating warping factors for each pixel position using two warping
position parameters (WPP) and eight warping control parameters (WCP). These
parameters can be specified as needed depending upon the desired warping. The
results are compared with similar real captured images both qualitative and
quantitative way.

negative:
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.

negative:
We present a novel data generation tool for document processing. The tool
focuses on providing a maximal level of visual information in a normal type
document, ranging from character position to paragraph-level position. It also
enables working with a large dataset on low-resource languages as well as
providing a mean of processing thorough full-level information of the
documented text. The data generation tools come with a dataset of 320000
Vietnamese synthetic document images and an instruction to generate a dataset
of similar size in other languages. The repository can be found at:
https://github.com/tson1997/SDL-Document-Image-Generation

negative:
Textual information in a captured scene plays an important role in scene
interpretation and decision making. Though there exist methods that can
successfully detect and interpret complex text regions present in a scene, to
the best of our knowledge, there is no significant prior work that aims to
modify the textual information in an image. The ability to edit text directly
on images has several advantages including error correction, text restoration
and image reusability. In this paper, we propose a method to modify text in an
image at character-level. We approach the problem in two stages. At first, the
unobserved character (target) is generated from an observed character (source)
being modified. We propose two different neural network architectures - (a)
FANnet to achieve structural consistency with source font and (b) Colornet to
preserve source color. Next, we replace the source character with the generated
character maintaining both geometric and visual consistency with neighboring
characters. Our method works as a unified platform for modifying text in
images. We present the effectiveness of our method on COCO-Text and ICDAR
datasets both qualitatively and quantitatively.

negative:
This paper presents results of Document Visual Question Answering Challenge
organized as part of "Text and Documents in the Deep Learning Era" workshop, in
CVPR 2020. The challenge introduces a new problem - Visual Question Answering
on document images. The challenge comprised two tasks. The first task concerns
with asking questions on a single document image. On the other hand, the second
task is set as a retrieval task where the question is posed over a collection
of images. For the task 1 a new dataset is introduced comprising 50,000
questions-answer(s) pairs defined over 12,767 document images. For task 2
another dataset has been created comprising 20 questions over 14,362 document
images which share the same document template.

negative:
Segmentation of a text-document into lines, words and characters, which is
considered to be the crucial pre-processing stage in Optical Character
Recognition (OCR) is traditionally carried out on uncompressed documents,
although most of the documents in real life are available in compressed form,
for the reasons such as transmission and storage efficiency. However, this
implies that the compressed image should be decompressed, which indents
additional computing resources. This limitation has motivated us to take up
research in document image analysis using compressed documents. In this paper,
we think in a new way to carry out segmentation at line, word and character
level in run-length compressed printed-text-documents. We extract the
horizontal projection profile curve from the compressed file and using the
local minima points perform line segmentation. However, tracing vertical
information which leads to tracking words-characters in a run-length compressed
file is not very straight forward. Therefore, we propose a novel technique for
carrying out simultaneous word and character segmentation by popping out column
runs from each row in an intelligent sequence. The proposed algorithms have
been validated with 1101 text-lines, 1409 words and 7582 characters from a
data-set of 35 noise and skew free compressed documents of Bengali, Kannada and
English Scripts.

negative:
Numerous activities in our daily life, including transactions, access to
services and transportation, require us to verify who we are by showing our ID
documents containing face images, e.g. passports and driver licenses. An
automatic system for matching ID document photos to live face images in real
time with high accuracy would speedup the verification process and remove the
burden on human operators. In this paper, by employing the transfer learning
technique, we propose a new method, DocFace, to train a domain-specific network
for ID document photo matching without a large dataset. Compared with the
baseline of applying existing methods for general face recognition to this
problem, our method achieves considerable improvement. A cross validation on an
ID-Selfie dataset shows that DocFace improves the TAR from 61.14% to 92.77% at
FAR=0.1%. Experimental results also indicate that given more training data, a
viable system for automatic ID document photo matching can be developed and
deployed.

negative:
Multimodal prediction results are essential for trajectory prediction task as
there is no single correct answer for the future. Previous frameworks can be
divided into three categories: regression, generation and classification
frameworks. However, these frameworks have weaknesses in different aspects so
that they cannot model the multimodal prediction task comprehensively. In this
paper, we present a novel insight along with a brand-new prediction framework
by formulating multimodal prediction into three steps: modality clustering,
classification and synthesis, and address the shortcomings of earlier
frameworks. Exhaustive experiments on popular benchmarks have demonstrated that
our proposed method surpasses state-of-the-art works even without introducing
social and map information. Specifically, we achieve 19.2% and 20.8%
improvement on ADE and FDE respectively on ETH/UCY dataset. Our code will be
made publicly availabe.

negative:
In Convolutional Neural Network (CNN) based image processing, most studies
propose networks that are optimized to single-level (or single-objective);
thus, they underperform on other levels and must be retrained for delivery of
optimal performance. Using multiple models to cover multiple levels involves
very high computational costs. To solve these problems, recent approaches train
networks on two different levels and propose their own interpolation methods to
enable arbitrary intermediate levels. However, many of them fail to generalize
or have certain side effects in practical usage. In this paper, we define these
frameworks as network tuning and interpolation and propose a novel module for
continuous-level learning, called Filter Transition Network (FTN). This module
is a structurally smoother module than existing ones. Therefore, the frameworks
with FTN generalize well across various tasks and networks and cause fewer
undesirable side effects. For stable learning of FTN, we additionally propose a
method to initialize non-linear neural network layers with identity mappings.
Extensive results for various image processing tasks indicate that the
performance of FTN is comparable in multiple continuous levels, and is
significantly smoother and lighter than that of other frameworks.

negative:
One major drawback of state of the art Neural Networks (NN)-based approaches
for document classification purposes is the large number of training samples
required to obtain an efficient classification. The minimum required number is
around one thousand annotated documents for each class. In many cases it is
very difficult, if not impossible, to gather this number of samples in real
industrial processes. In this paper, we analyse the efficiency of NN-based
document classification systems in a sub-optimal training case, based on the
situation of a company document stream. We evaluated three different
approaches, one based on image content and two on textual content. The
evaluation was divided into four parts: a reference case, to assess the
performance of the system in the lab; two cases that each simulate a specific
difficulty linked to document stream processing; and a realistic case that
combined all of these difficulties. The realistic case highlighted the fact
that there is a significant drop in the efficiency of NN-Based document
classification systems. Although they remain efficient for well represented
classes (with an over-fitting of the system for those classes), it is
impossible for them to handle appropriately less well represented classes.
NN-Based document classification systems need to be adapted to resolve these
two problems before they can be considered for use in a company document
stream.

negative:
Everyone knows that thousand of words are represented by a single image. As a
result image search has become a very popular mechanism for the Web searchers.
Image search means, the search results are produced by the search engine should
be a set of images along with their Web page Unified Resource Locator. Now Web
searcher can perform two types of image search, they are Text to Image and
Image to Image search. In Text to Image search, search query should be a text.
Based on the input text data system will generate a set of images along with
their Web page URL as an output. On the other hand, in Image to Image search,
search query should be an image and based on this image system will generate a
set of images along with their Web page URL as an output. According to the
current scenarios, Text to Image search mechanism always not returns perfect
result. It matches the text data and then displays the corresponding images as
an output, which is not always perfect. To resolve this problem, Web
researchers have introduced the Image to Image search mechanism. In this paper,
we have also proposed an alternate approach of Image to Image search mechanism
using Histogram.

negative:
Document segmentation is a method of rending the document into distinct
regions. A document is an assortment of information and a standard mode of
conveying information to others. Pursuance of data from documents involves ton
of human effort, time intense and might severely prohibit the usage of data
systems. So, automatic information pursuance from the document has become a big
issue. It is been shown that document segmentation will facilitate to beat such
problems. This paper proposes a new approach to segment and classify the
document regions as text, image, drawings and table. Document image is divided
into blocks using Run length smearing rule and features are extracted from
every blocks. Discipulus tool has been used to construct the Genetic
programming based classifier model and located 97.5% classification accuracy.

negative:
The image processing community has witnessed remarkable advances in enhancing
and restoring images. Nevertheless, restoring the visual quality of underwater
images remains a great challenge. End-to-end frameworks might fail to enhance
the visual quality of underwater images since in several scenarios it is not
feasible to provide the ground truth of the scene radiance. In this work, we
propose a CNN-based approach that does not require ground truth data since it
uses a set of image quality metrics to guide the restoration learning process.
The experiments showed that our method improved the visual quality of
underwater images preserving their edges and also performed well considering
the UCIQE metric.

negative:
Digital camera and mobile document image acquisition are new trends arising
in the world of Optical Character Recognition and text detection. In some
cases, such process integrates many distortions and produces poorly scanned
text or text-photo images and natural images, leading to an unreliable OCR
digitization. In this paper, we present a novel nonparametric and unsupervised
method to compensate for undesirable document image distortions aiming to
optimally improve OCR accuracy. Our approach relies on a very efficient stack
of document image enhancing techniques to recover deformation of the entire
document image. First, we propose a local brightness and contrast adjustment
method to effectively handle lighting variations and the irregular distribution
of image illumination. Second, we use an optimized greyscale conversion
algorithm to transform our document image to greyscale level. Third, we sharpen
the useful information in the resulting greyscale image using Un-sharp Masking
method. Finally, an optimal global binarization approach is used to prepare the
final document image to OCR recognition. The proposed approach can
significantly improve text detection rate and optical character recognition
accuracy. To demonstrate the efficiency of our approach, an exhaustive
experimentation on a standard dataset is presented.

negative:
Line separators are used to segregate text-lines from one another in document
image analysis. Finding the separator points at every line terminal in a
document image would enable text-line segmentation. In particular, identifying
the separators in handwritten text could be a thrilling exercise. Obviously it
would be challenging to perform this in the compressed version of a document
image and that is the proposed objective in this research. Such an effort would
prevent the computational burden of decompressing a document for text-line
segmentation. Since document images are generally compressed using run length
encoding (RLE) technique as per the CCITT standards, the first column in the
RLE will be a white column. The value (depth) in the white column is very low
when a particular line is a text line and the depth could be larger at the
point of text line separation. A longer consecutive sequence of such larger
depth should indicate the gap between the text lines, which provides the
separator region. In case of over separation and under separation issues,
corrective actions such as deletion and insertion are suggested respectively.
An extensive experimentation is conducted on the compressed images of the
benchmark datasets of ICDAR13 and Alireza et al [17] to demonstrate the
efficacy.

negative:
In this work\footnote {This work was supported in part by the National
Science Foundation under grant IIS-1212948.}, we present a method to represent
a video with a sequence of words, and learn the temporal sequencing of such
words as the key information for predicting and recognizing human actions. We
leverage core concepts from the Natural Language Processing (NLP) literature
used in sentence classification to solve the problems of action prediction and
action recognition. Each frame is converted into a word that is represented as
a vector using the Bag of Visual Words (BoW) encoding method. The words are
then combined into a sentence to represent the video, as a sentence. The
sequence of words in different actions are learned with a simple but effective
Temporal Convolutional Neural Network (T-CNN) that captures the temporal
sequencing of information in a video sentence. We demonstrate that a key
characteristic of the proposed method is its low-latency, i.e. its ability to
predict an action accurately with a partial sequence (sentence). Experiments on
two datasets, \textit{UCF101} and \textit{HMDB51} show that the method on
average reaches 95\% of its accuracy within half the video frames. Results,
also demonstrate that our method achieves compatible state-of-the-art
performance in action recognition (i.e. at the completion of the sentence) in
addition to action prediction.

negative:
We introduce FontCode, an information embedding technique for text documents.
Provided a text document with specific fonts, our method embeds user-specified
information in the text by perturbing the glyphs of text characters while
preserving the text content. We devise an algorithm to chooses unobtrusive yet
machine-recognizable glyph perturbations, leveraging a recently developed
generative model that alters the glyphs of each character continuously on a
font manifold. We then introduce an algorithm that embeds a user-provided
message in the text document and produces an encoded document whose appearance
is minimally perturbed from the original document. We also present a glyph
recognition method that recovers the embedded information from an encoded
document stored as a vector graphic or pixel image, or even on a printed paper.
In addition, we introduce a new error-correction coding scheme that rectifies a
certain number of recognition errors. Lastly, we demonstrate that our technique
enables a wide array of applications, using it as a text document metadata
holder, an unobtrusive optical barcode, a cryptographic message embedding
scheme, and a text document signature.

negative:
Imagery texts are usually organized as a hierarchy of several visual
elements, i.e. characters, words, text lines and text blocks. Among these
elements, character is the most basic one for various languages such as
Western, Chinese, Japanese, mathematical expression and etc. It is natural and
convenient to construct a common text detection engine based on character
detectors. However, training character detectors requires a vast of location
annotated characters, which are expensive to obtain. Actually, the existing
real text datasets are mostly annotated in word or line level. To remedy this
dilemma, we propose a weakly supervised framework that can utilize word
annotations, either in tight quadrangles or the more loose bounding boxes, for
character detector training. When applied in scene text detection, we are thus
able to train a robust character detector by exploiting word annotations in the
rich large-scale real scene text datasets, e.g. ICDAR15 and COCO-text. The
character detector acts as a key role in the pipeline of our text detection
engine. It achieves the state-of-the-art performance on several challenging
scene text detection benchmarks. We also demonstrate the flexibility of our
pipeline by various scenarios, including deformed text detection and math
expression recognition.

negative:
We address the problem of predicting similarity between a pair of handwritten
document images written by different individuals. This has applications related
to matching and mining in image collections containing handwritten content. A
similarity score is computed by detecting patterns of text re-usages between
document images irrespective of the minor variations in word morphology, word
ordering, layout and paraphrasing of the content. Our method does not depend on
an accurate segmentation of words and lines. We formulate the document matching
problem as a structured comparison of the word distributions across two
document images. To match two word images, we propose a convolutional neural
network (CNN) based feature descriptor. Performance of this representation
surpasses the state-of-the-art on handwritten word spotting. Finally, we
demonstrate the applicability of our method on a practical problem of matching
handwritten assignments.

negative:
Recognition of document images have important applications in restoring old
and classical texts. The problem involves quality improvement before passing it
to a properly trained OCR to get accurate recognition of the text. The image
enhancement and quality improvement constitute important steps as subsequent
recognition depends upon the quality of the input image. There are scenarios
when high resolution images are not available and our experiments show that the
OCR accuracy reduces significantly with decrease in the spatial resolution of
document images. Thus the only option is to improve the resolution of such
document images. The goal is to construct a high resolution image, given a
single low resolution binary image, which constitutes the problem of single
image super-resolution. Most of the previous work in super-resolution deal with
natural images which have more information-content than the document images.
Here, we use Convolution Neural Network to learn the mapping between low and
the corresponding high resolution images. We experiment with different number
of layers, parameter settings and non-linear functions to build a fast
end-to-end framework for document image super-resolution. Our proposed model
shows a very good PSNR improvement of about 4 dB on 75 dpi Tamil images,
resulting in a 3 % improvement of word level accuracy by the OCR. It takes less
time than the recent sparse based natural image super-resolution technique,
making it useful for real-time document recognition applications.

negative:
Smartphones have enabled effortless capturing and sharing of documents in
digital form. The documents, however, often undergo various types of
degradation due to aging, stains, or shortcoming of capturing environment such
as shadow, non-uniform lighting, etc., which reduces the comprehensibility of
the document images. In this work, we consider the problem of document image
cleanup on embedded applications such as smartphone apps, which usually have
memory, energy, and latency limitations due to the device and/or for best human
user experience. We propose a light-weight encoder decoder based convolutional
neural network architecture for removing the noisy elements from document
images. To compensate for generalization performance with a low network
capacity, we incorporate the perceptual loss for knowledge transfer from
pre-trained deep CNN network in our loss function. In terms of the number of
parameters and product-sum operations, our models are 65-1030 and 3-27 times,
respectively, smaller than existing state-of-the-art document enhancement
models. Overall, the proposed models offer a favorable resource versus accuracy
trade-off and we empirically illustrate the efficacy of our approach on several
real-world benchmark datasets.