query: I am working on a natural language processing project focused on entity recognition, which necessitates the comprehension and processing of vast amounts of text data. My goal is to integrate real-world knowledge during the training phase to bolster the model's performance across various tasks, thus bridging the gap between raw text and meaningful entities. Therefore, I am in search of models that can generate and learn embeddings for words and entities by leveraging extensive real-world knowledge bases. Naturally, the model needs to be able to process knowledge and concept in some differentiable way. Ideally, the trained model should excel in performance on entity-centric datasets and produce competitive results on other standard benchmark datasets.

positive:
The inception of modeling contextual information using models such as BERT,
ELMo, and Flair has significantly improved representation learning for words.
It has also given SOTA results in almost every NLP task - Machine Translation,
Text Summarization and Named Entity Recognition, to name a few. In this work,
in addition to using these dominant context-aware representations, we propose a
Knowledge Aware Representation Learning (KARL) Network for Named Entity
Recognition (NER). We discuss the challenges of using existing methods in
incorporating world knowledge for NER and show how our proposed methods could
be leveraged to overcome those challenges. KARL is based on a Transformer
Encoder that utilizes large knowledge bases represented as fact triplets,
converts them to a graph context, and extracts essential entity information
residing inside to generate contextualized triplet representation for feature
augmentation. Experimental results show that the augmentation done using KARL
can considerably boost the performance of our NER system and achieve
significantly better results than existing approaches in the literature on
three publicly available NER datasets, namely CoNLL 2003, CoNLL++, and
OntoNotes v5. We also observe better generalization and application to a
real-world setting from KARL on unseen entities.

positive:
Contextual word representations, typically trained on unstructured, unlabeled
text, do not contain any explicit grounding to real world entities and are
often unable to remember facts about those entities. We propose a general
method to embed multiple knowledge bases (KBs) into large scale models, and
thereby enhance their representations with structured, human-curated knowledge.
For each KB, we first use an integrated entity linker to retrieve relevant
entity embeddings, then update contextual word representations via a form of
word-to-entity attention. In contrast to previous approaches, the entity
linkers and self-supervised language modeling objective are jointly trained
end-to-end in a multitask setting that combines a small amount of entity
linking supervision with a large amount of raw text. After integrating WordNet
and a subset of Wikipedia into BERT, the knowledge enhanced BERT (KnowBert)
demonstrates improved perplexity, ability to recall facts as measured in a
probing task and downstream performance on relationship extraction, entity
typing, and word sense disambiguation. KnowBert's runtime is comparable to
BERT's and it scales to large KBs.

positive:
Pretrained language models have shown success in many natural language
processing tasks. Many works explore incorporating knowledge into language
models. In the biomedical domain, experts have taken decades of effort on
building large-scale knowledge bases. For example, the Unified Medical Language
System (UMLS) contains millions of entities with their synonyms and defines
hundreds of relations among entities. Leveraging this knowledge can benefit a
variety of downstream tasks such as named entity recognition and relation
extraction. To this end, we propose KeBioLM, a biomedical pretrained language
model that explicitly leverages knowledge from the UMLS knowledge bases.
Specifically, we extract entities from PubMed abstracts and link them to UMLS.
We then train a knowledge-aware language model that firstly applies a text-only
encoding layer to learn entity representation and applies a text-entity fusion
encoding to aggregate entity representation. Besides, we add two training
objectives as entity detection and entity linking. Experiments on the named
entity recognition and relation extraction from the BLURB benchmark demonstrate
the effectiveness of our approach. Further analysis on a collected probing
dataset shows that our model has better ability to model medical knowledge.

positive:
Learned vector representations of words are useful tools for many information
retrieval and natural language processing tasks due to their ability to capture
lexical semantics. However, while many such tasks involve or even rely on named
entities as central components, popular word embedding models have so far
failed to include entities as first-class citizens. While it seems intuitive
that annotating named entities in the training corpus should result in more
intelligent word features for downstream tasks, performance issues arise when
popular embedding approaches are naively applied to entity annotated corpora.
Not only are the resulting entity embeddings less useful than expected, but one
also finds that the performance of the non-entity word embeddings degrades in
comparison to those trained on the raw, unannotated corpus. In this paper, we
investigate approaches to jointly train word and entity embeddings on a large
corpus with automatically annotated and linked entities. We discuss two
distinct approaches to the generation of such embeddings, namely the training
of state-of-the-art embeddings on raw-text and annotated versions of the
corpus, as well as node embeddings of a co-occurrence graph representation of
the annotated corpus. We compare the performance of annotated embeddings and
classical word embeddings on a variety of word similarity, analogy, and
clustering evaluation tasks, and investigate their performance in
entity-specific tasks. Our findings show that it takes more than training
popular word embedding models on an annotated corpus to create entity
embeddings with acceptable performance on common test cases. Based on these
results, we discuss how and when node embeddings of the co-occurrence graph
representation of the text can restore the performance.

positive:
This paper addresses the problem of mapping natural language text to
knowledge base entities. The mapping process is approached as a composition of
a phrase or a sentence into a point in a multi-dimensional entity space
obtained from a knowledge graph. The compositional model is an LSTM equipped
with a dynamic disambiguation mechanism on the input word embeddings (a
Multi-Sense LSTM), addressing polysemy issues. Further, the knowledge base
space is prepared by collecting random walks from a graph enhanced with textual
features, which act as a set of semantic bridges between text and knowledge
base entities. The ideas of this work are demonstrated on large-scale
text-to-entity mapping and entity classification tasks, with state of the art
results.

positive:
Existing state of the art neural entity linking models employ attention-based
bag-of-words context model and pre-trained entity embeddings bootstrapped from
word embeddings to assess topic level context compatibility. However, the
latent entity type information in the immediate context of the mention is
neglected, which causes the models often link mentions to incorrect entities
with incorrect type. To tackle this problem, we propose to inject latent entity
type information into the entity embeddings based on pre-trained BERT. In
addition, we integrate a BERT-based entity similarity score into the local
context model of a state-of-the-art model to better capture latent entity type
information. Our model significantly outperforms the state-of-the-art entity
linking models on standard benchmark (AIDA-CoNLL). Detailed experiment analysis
demonstrates that our model corrects most of the type errors produced by the
direct baseline.

positive:
Knowledge graphs (KGs) contain rich information about world knowledge,
entities and relations. Thus, they can be great supplements to existing
pre-trained language models. However, it remains a challenge to efficiently
integrate information from KG into language modeling. And the understanding of
a knowledge graph requires related context. We propose a novel joint
pre-training framework, JAKET, to model both the knowledge graph and language.
The knowledge module and language module provide essential information to
mutually assist each other: the knowledge module produces embeddings for
entities in text while the language module generates context-aware initial
embeddings for entities and relations in the graph. Our design enables the
pre-trained model to easily adapt to unseen knowledge graphs in new domains.
Experimental results on several knowledge-aware NLP tasks show that our
proposed framework achieves superior performance by effectively leveraging
knowledge in language understanding.

positive:
Due to the lack of structured knowledge applied in learning distributed
representation of cate- gories, existing work cannot incorporate category
hierarchies into entity information. We propose a framework that embeds
entities and categories into a semantic space by integrating structured
knowledge and taxonomy hierarchy from large knowledge bases. The framework
allows to com- pute meaningful semantic relatedness between entities and
categories. Our framework can han- dle both single-word concepts and
multiple-word concepts with superior performance on concept categorization and
yield state of the art results on dataless hierarchical classification.

positive:
We describe a neural network model that jointly learns distributed
representations of texts and knowledge base (KB) entities. Given a text in the
KB, we train our proposed model to predict entities that are relevant to the
text. Our model is designed to be generic with the ability to address various
NLP tasks with ease. We train the model using a large corpus of texts and their
entity annotations extracted from Wikipedia. We evaluated the model on three
important NLP tasks (i.e., sentence textual similarity, entity linking, and
factoid question answering) involving both unsupervised and supervised
settings. As a result, we achieved state-of-the-art results on all three of
these tasks. Our code and trained models are publicly available for further
academic research.

positive:
Information Extraction (IE) from text refers to the task of extracting
structured knowledge from unstructured text. The task typically consists of a
series of sub-tasks such as Named Entity Recognition and Relation Extraction.
Sourcing entity and relation type specific training data is a major bottleneck
in domains with limited resources such as biomedicine. In this work we present
a slot filling approach to the task of biomedical IE, effectively replacing the
need for entity and relation-specific training data, allowing us to deal with
zero-shot settings. We follow the recently proposed paradigm of coupling a
Tranformer-based bi-encoder, Dense Passage Retrieval, with a Transformer-based
reading comprehension model to extract relations from biomedical text. We
assemble a biomedical slot filling dataset for both retrieval and reading
comprehension and conduct a series of experiments demonstrating that our
approach outperforms a number of simpler baselines. We also evaluate our
approach end-to-end for standard as well as zero-shot settings. Our work
provides a fresh perspective on how to solve biomedical IE tasks, in the
absence of relevant training data. Our code, models and datasets are available
at https://github.com/ypapanik/biomedical-slot-filling.

positive:
The challenge of recognizing named entities in a given text has been a very
dynamic field in recent years. This is due to the advances in neural network
architectures, increase of computing power and the availability of diverse
labeled datasets, which deliver pre-trained, highly accurate models. These
tasks are generally focused on tagging common entities, but domain-specific
use-cases require tagging custom entities which are not part of the pre-trained
models. This can be solved by either fine-tuning the pre-trained models, or by
training custom models. The main challenge lies in obtaining reliable labeled
training and test datasets, and manual labeling would be a highly tedious task.
  In this paper we present PharmKE, a text analysis platform focused on the
pharmaceutical domain, which applies deep learning through several stages for
thorough semantic analysis of pharmaceutical articles. It performs text
classification using state-of-the-art transfer learning models, and thoroughly
integrates the results obtained through a proposed methodology. The methodology
is used to create accurately labeled training and test datasets, which are then
used to train models for custom entity labeling tasks, centered on the
pharmaceutical domain. The obtained results are compared to the fine-tuned BERT
and BioBERT models trained on the same dataset. Additionally, the PharmKE
platform integrates the results obtained from named entity recognition tasks to
resolve co-references of entities and analyze the semantic relations in every
sentence, thus setting up a baseline for additional text analysis tasks, such
as question answering and fact extraction. The recognized entities are also
used to expand the knowledge graph generated by DBpedia Spotlight for a given
pharmaceutical text.

positive:
Background: Finding biomedical named entities is one of the most essential
tasks in biomedical text mining. Recently, deep learning-based approaches have
been applied to biomedical named entity recognition (BioNER) and showed
promising results. However, as deep learning approaches need an abundant amount
of training data, a lack of data can hinder performance. BioNER datasets are
scarce resources and each dataset covers only a small subset of entity types.
Furthermore, many bio entities are polysemous, which is one of the major
obstacles in named entity recognition. Results: To address the lack of data and
the entity type misclassification problem, we propose CollaboNet which utilizes
a combination of multiple NER models. In CollaboNet, models trained on a
different dataset are connected to each other so that a target model obtains
information from other collaborator models to reduce false positives. Every
model is an expert on their target entity type and takes turns serving as a
target and a collaborator model during training time. The experimental results
show that CollaboNet can be used to greatly reduce the number of false
positives and misclassified entities including polysemous words. CollaboNet
achieved state-of-the-art performance in terms of precision, recall and F1
score. Conclusions: We demonstrated the benefits of combining multiple models
for BioNER. Our model has successfully reduced the number of misclassified
entities and improved the performance by leveraging multiple datasets annotated
for different entity types. Given the state-of-the-art performance of our
model, we believe that CollaboNet can improve the accuracy of downstream
biomedical text mining applications such as bio-entity relation extraction.

positive:
This study proposes a Neural Attentive Bag-of-Entities model, which is a
neural network model that performs text classification using entities in a
knowledge base. Entities provide unambiguous and relevant semantic signals that
are beneficial for capturing semantics in texts. We combine simple high-recall
entity detection based on a dictionary, to detect entities in a document, with
a novel neural attention mechanism that enables the model to focus on a small
number of unambiguous and relevant entities. We tested the effectiveness of our
model using two standard text classification datasets (i.e., the 20 Newsgroups
and R8 datasets) and a popular factoid question answering dataset based on a
trivia quiz game. As a result, our model achieved state-of-the-art results on
all datasets. The source code of the proposed model is available online at
https://github.com/wikipedia2vec/wikipedia2vec.

positive:
Text representations using neural word embeddings have proven effective in
many NLP applications. Recent researches adapt the traditional word embedding
models to learn vectors of multiword expressions (concepts/entities). However,
these methods are limited to textual knowledge bases (e.g., Wikipedia). In this
paper, we propose a novel and simple technique for integrating the knowledge
about concepts from two large scale knowledge bases of different structure
(Wikipedia and Probase) in order to learn concept representations. We adapt the
efficient skip-gram model to seamlessly learn from the knowledge in Wikipedia
text and Probase concept graph. We evaluate our concept embedding models on two
tasks: (1) analogical reasoning, where we achieve a state-of-the-art
performance of 91% on semantic analogies, (2) concept categorization, where we
achieve a state-of-the-art performance on two benchmark datasets achieving
categorization accuracy of 100% on one and 98% on the other. Additionally, we
present a case study to evaluate our model on unsupervised argument type
identification for neural semantic parsing. We demonstrate the competitive
accuracy of our unsupervised method and its ability to better generalize to out
of vocabulary entity mentions compared to the tedious and error prone methods
which depend on gazetteers and regular expressions.

positive:
Joint representation learning of text and knowledge within a unified semantic
space enables us to perform knowledge graph completion more accurately. In this
work, we propose a novel framework to embed words, entities and relations into
the same continuous vector space. In this model, both entity and relation
embeddings are learned by taking knowledge graph and plain text into
consideration. In experiments, we evaluate the joint learning model on three
tasks including entity prediction, relation prediction and relation
classification from text. The experiment results show that our model can
significantly and consistently improve the performance on the three tasks as
compared with other baselines.

positive:
Extracting entities and relations for types of interest from text is
important for understanding massive text corpora. Traditionally, systems of
entity relation extraction have relied on human-annotated corpora for training
and adopted an incremental pipeline. Such systems require additional human
expertise to be ported to a new domain, and are vulnerable to errors cascading
down the pipeline. In this paper, we investigate joint extraction of typed
entities and relations with labeled data heuristically obtained from knowledge
bases (i.e., distant supervision). As our algorithm for type labeling via
distant supervision is context-agnostic, noisy training data poses unique
challenges for the task. We propose a novel domain-independent framework,
called CoType, that runs a data-driven text segmentation algorithm to extract
entity mentions, and jointly embeds entity mentions, relation mentions, text
features and type labels into two low-dimensional spaces (for entity and
relation mentions respectively), where, in each space, objects whose types are
close will also have similar representations. CoType, then using these learned
embeddings, estimates the types of test (unlinkable) mentions. We formulate a
joint optimization problem to learn embeddings from text corpora and knowledge
bases, adopting a novel partial-label loss function for noisy labeled data and
introducing an object "translation" function to capture the cross-constraints
of entities and relations on each other. Experiments on three public datasets
demonstrate the effectiveness of CoType across different domains (e.g., news,
biomedical), with an average of 25% improvement in F1 score compared to the
next best method.

positive:
In standard methodology for natural language processing, entities in text are
typically embedded in dense vector spaces with pre-trained models. The
embeddings produced this way are effective when fed into downstream models, but
they require end-task fine-tuning and are fundamentally difficult to interpret.
In this paper, we present an approach to creating entity representations that
are human readable and achieve high performance on entity-related tasks out of
the box. Our representations are vectors whose values correspond to posterior
probabilities over fine-grained entity types, indicating the confidence of a
typing model's decision that the entity belongs to the corresponding type. We
obtain these representations using a fine-grained entity typing model, trained
either on supervised ultra-fine entity typing data (Choi et al. 2018) or
distantly-supervised examples from Wikipedia. On entity probing tasks involving
recognizing entity identity, our embeddings used in parameter-free downstream
models achieve competitive performance with ELMo- and BERT-based embeddings in
trained models. We also show that it is possible to reduce the size of our type
set in a learning-based way for particular domains. Finally, we show that these
embeddings can be post-hoc modified through a small number of rules to
incorporate domain knowledge and improve performance.

positive:
Named entity recognition (NER) is one of the tasks in natural language
processing that can greatly benefit from the use of external knowledge sources.
We propose a named entity recognition framework composed of knowledge-based
feature extractors and a deep learning model including contextual word
embeddings, long short-term memory (LSTM) layers and conditional random fields
(CRF) inference layer. We use an entity linking module to integrate our system
with Wikipedia. The combination of effective neural architecture and external
resources allows us to obtain state-of-the-art results on recognition of Polish
proper names. We evaluate our model on data from PolEval 2018 NER challenge on
which it outperforms other methods, reducing the error rate by 22.4% compared
to the winning solution. Our work shows that combining neural NER model and
entity linking model with a knowledge base is more effective in recognizing
named entities than using NER model alone.

positive:
Entities are at the center of how we represent and aggregate knowledge. For
instance, Encyclopedias such as Wikipedia are structured by entities (e.g., one
per Wikipedia article). The ability to retrieve such entities given a query is
fundamental for knowledge-intensive tasks such as entity linking and
open-domain question answering. Current approaches can be understood as
classifiers among atomic labels, one for each entity. Their weight vectors are
dense entity representations produced by encoding entity meta information such
as their descriptions. This approach has several shortcomings: (i) context and
entity affinity is mainly captured through a vector dot product, potentially
missing fine-grained interactions; (ii) a large memory footprint is needed to
store dense representations when considering large entity sets; (iii) an
appropriately hard set of negative data has to be subsampled at training time.
In this work, we propose GENRE, the first system that retrieves entities by
generating their unique names, left to right, token-by-token in an
autoregressive fashion. This mitigates the aforementioned technical issues
since: (i) the autoregressive formulation directly captures relations between
context and entity name, effectively cross encoding both; (ii) the memory
footprint is greatly reduced because the parameters of our encoder-decoder
architecture scale with vocabulary size, not entity count; (iii) the softmax
loss is computed without subsampling negative data. We experiment with more
than 20 datasets on entity disambiguation, end-to-end entity linking and
document retrieval tasks, achieving new state-of-the-art or very competitive
results while using a tiny fraction of the memory footprint of competing
systems. Finally, we demonstrate that new entities can be added by simply
specifying their names. Code and pre-trained models at
https://github.com/facebookresearch/GENRE.

positive:
Deep neural network models have helped named entity (NE) recognition achieve
amazing performance without handcrafting features. However, existing systems
require large amounts of human annotated training data. Efforts have been made
to replace human annotations with external knowledge (e.g., NE dictionary,
part-of-speech tags), while it is another challenge to obtain such effective
resources. In this work, we propose a fully unsupervised NE recognition model
which only needs to take informative clues from pre-trained word embeddings. We
first apply Gaussian Hidden Markov Model and Deep Autoencoding Gaussian Mixture
Model on word embeddings for entity span detection and type prediction, and
then further design an instance selector based on reinforcement learning to
distinguish positive sentences from noisy sentences and refine these
coarse-grained annotations through neural networks. Extensive experiments on
CoNLL benchmark datasets demonstrate that our proposed light NE recognition
model achieves remarkable performance without using any annotated lexicon or
corpus.

positive:
Much of biomedical and healthcare data is encoded in discrete, symbolic form
such as text and medical codes. There is a wealth of expert-curated biomedical
domain knowledge stored in knowledge bases and ontologies, but the lack of
reliable methods for learning knowledge representation has limited their
usefulness in machine learning applications. While text-based representation
learning has significantly improved in recent years through advances in natural
language processing, attempts to learn biomedical concept embeddings so far
have been lacking. A recent family of models called knowledge graph embeddings
have shown promising results on general domain knowledge graphs, and we explore
their capabilities in the biomedical domain. We train several state-of-the-art
knowledge graph embedding models on the SNOMED-CT knowledge graph, provide a
benchmark with comparison to existing methods and in-depth discussion on best
practices, and make a case for the importance of leveraging the
multi-relational nature of knowledge graphs for learning biomedical knowledge
representation. The embeddings, code, and materials will be made available to
the communitY.

positive:
Knowledge bases (KBs) and text often contain complementary knowledge: KBs
store structured knowledge that can support long range reasoning, while text
stores more comprehensive and timely knowledge in an unstructured way.
Separately embedding the individual knowledge sources into vector spaces has
demonstrated tremendous successes in encoding the respective knowledge, but how
to jointly embed and reason with both knowledge sources to fully leverage the
complementary information is still largely an open problem. We conduct a
large-scale, systematic investigation of aligning KB and text embeddings for
joint reasoning. We set up a novel evaluation framework with two evaluation
tasks, few-shot link prediction and analogical reasoning, and evaluate an array
of KB-text embedding alignment methods. We also demonstrate how such alignment
can infuse textual information into KB embeddings for more accurate link
prediction on emerging entities and events, using COVID-19 as a case study.

positive:
A typical architecture for end-to-end entity linking systems consists of
three steps: mention detection, candidate generation and entity disambiguation.
In this study we investigate the following questions: (a) Can all those steps
be learned jointly with a model for contextualized text-representations, i.e.
BERT (Devlin et al., 2019)? (b) How much entity knowledge is already contained
in pretrained BERT? (c) Does additional entity knowledge improve BERT's
performance in downstream tasks? To this end, we propose an extreme
simplification of the entity linking setup that works surprisingly well: simply
cast it as a per token classification over the entire entity vocabulary (over
700K classes in our case). We show on an entity linking benchmark that (i) this
model improves the entity representations over plain BERT, (ii) that it
outperforms entity linking architectures that optimize the tasks separately and
(iii) that it only comes second to the current state-of-the-art that does
mention detection and entity disambiguation jointly. Additionally, we
investigate the usefulness of entity-aware token-representations in the
text-understanding benchmark GLUE, as well as the question answering benchmarks
SQUAD V2 and SWAG and also the EN-DE WMT14 machine translation benchmark. To
our surprise, we find that most of those benchmarks do not benefit from
additional entity knowledge, except for a task with very small training data,
the RTE task in GLUE, which improves by 2%.

positive:
Entity synonyms discovery is crucial for entity-leveraging applications.
However, existing studies suffer from several critical issues: (1) the input
mentions may be out-of-vocabulary (OOV) and may come from a different semantic
space of the entities; (2) the connection between mentions and entities may be
hidden and cannot be established by surface matching; and (3) some entities
rarely appear due to the long-tail effect. To tackle these challenges, we
facilitate knowledge graphs and propose a novel entity synonyms discovery
framework, named \emph{KGSynNet}. Specifically, we pre-train subword embeddings
for mentions and entities using a large-scale domain-specific corpus while
learning the knowledge embeddings of entities via a joint TransC-TransE model.
More importantly, to obtain a comprehensive representation of entities, we
employ a specifically designed \emph{fusion gate} to adaptively absorb the
entities' knowledge information into their semantic features. We conduct
extensive experiments to demonstrate the effectiveness of our \emph{KGSynNet}
in leveraging the knowledge graph. The experimental results show that the
\emph{KGSynNet} improves the state-of-the-art methods by 14.7\% in terms of
hits@3 in the offline evaluation and outperforms the BERT model by 8.3\% in the
positive feedback rate of an online A/B test on the entity linking module of a
question answering system.

positive:
Traditional language models are unable to efficiently model entity names
observed in text. All but the most popular named entities appear infrequently
in text providing insufficient context. Recent efforts have recognized that
context can be generalized between entity names that share the same type (e.g.,
\emph{person} or \emph{location}) and have equipped language models with access
to an external knowledge base (KB). Our Knowledge-Augmented Language Model
(KALM) continues this line of work by augmenting a traditional model with a KB.
Unlike previous methods, however, we train with an end-to-end predictive
objective optimizing the perplexity of text. We do not require any additional
information such as named entity tags. In addition to improving language
modeling performance, KALM learns to recognize named entities in an entirely
unsupervised way by using entity type information latent in the model. On a
Named Entity Recognition (NER) task, KALM achieves performance comparable with
state-of-the-art supervised models. Our work demonstrates that named entities
(and possibly other types of world knowledge) can be modeled successfully using
predictive learning and training on large corpora of text without any
additional information.

positive:
In this paper, we describe TextEnt, a neural network model that learns
distributed representations of entities and documents directly from a knowledge
base (KB). Given a document in a KB consisting of words and entity annotations,
we train our model to predict the entity that the document describes and map
the document and its target entity close to each other in a continuous vector
space. Our model is trained using a large number of documents extracted from
Wikipedia. The performance of the proposed model is evaluated using two tasks,
namely fine-grained entity typing and multiclass text classification. The
results demonstrate that our model achieves state-of-the-art performance on
both tasks. The code and the trained representations are made available online
for further academic research.

positive:
We propose a novel deep learning model for joint document-level entity
disambiguation, which leverages learned neural representations. Key components
are entity embeddings, a neural attention mechanism over local context windows,
and a differentiable joint inference stage for disambiguation. Our approach
thereby combines benefits of deep learning with more traditional approaches
such as graphical models and probabilistic mention-entity maps. Extensive
experiments show that we are able to obtain competitive or state-of-the-art
accuracy at moderate computational costs.

positive:
Neural language models are a powerful tool to embed words into semantic
vector spaces. However, learning such models generally relies on the
availability of abundant and diverse training examples. In highly specialised
domains this requirement may not be met due to difficulties in obtaining a
large corpus, or the limited range of expression in average use. Such domains
may encode prior knowledge about entities in a knowledge base or ontology. We
propose a generative model which integrates evidence from diverse data sources,
enabling the sharing of semantic information. We achieve this by generalising
the concept of co-occurrence from distributional semantics to include other
relationships between entities or words, which we model as affine
transformations on the embedding space. We demonstrate the effectiveness of
this approach by outperforming recent models on a link prediction task and
demonstrating its ability to profit from partially or fully unobserved data
training labels. We further demonstrate the usefulness of learning from
different data sources with overlapping vocabularies.

positive:
Information extraction is an important task in NLP, enabling the automatic
extraction of data for relational database filling. Historically, research and
data was produced for English text, followed in subsequent years by datasets in
Arabic, Chinese (ACE/OntoNotes), Dutch, Spanish, German (CoNLL evaluations),
and many others. The natural tendency has been to treat each language as a
different dataset and build optimized models for each. In this paper we
investigate a single Named Entity Recognition model, based on a multilingual
BERT, that is trained jointly on many languages simultaneously, and is able to
decode these languages with better accuracy than models trained only on one
language. To improve the initial model, we study the use of regularization
strategies such as multitask learning and partial gradient updates. In addition
to being a single model that can tackle multiple languages (including code
switch), the model could be used to make zero-shot predictions on a new
language, even ones for which training data is not available, out of the box.
The results show that this model not only performs competitively with
monolingual models, but it also achieves state-of-the-art results on the
CoNLL02 Dutch and Spanish datasets, OntoNotes Arabic and Chinese datasets.
Moreover, it performs reasonably well on unseen languages, achieving
state-of-the-art for zero-shot on three CoNLL languages.

positive:
Named entity recognition (NER) and relation extraction (RE) are two important
tasks in information extraction and retrieval (IE \& IR). Recent work has
demonstrated that it is beneficial to learn these tasks jointly, which avoids
the propagation of error inherent in pipeline-based systems and improves
performance. However, state-of-the-art joint models typically rely on external
natural language processing (NLP) tools, such as dependency parsers, limiting
their usefulness to domains (e.g. news) where those tools perform well. The few
neural, end-to-end models that have been proposed are trained almost completely
from scratch. In this paper, we propose a neural, end-to-end model for jointly
extracting entities and their relations which does not rely on external NLP
tools and which integrates a large, pre-trained language model. Because the
bulk of our model's parameters are pre-trained and we eschew recurrence for
self-attention, our model is fast to train. On 5 datasets across 3 domains, our
model matches or exceeds state-of-the-art performance, sometimes by a large
margin.

positive:
An image caption should fluently present the essential information in a given
image, including informative, fine-grained entity mentions and the manner in
which these entities interact. However, current captioning models are usually
trained to generate captions that only contain common object names, thus
falling short on an important "informativeness" dimension. We present a
mechanism for integrating image information together with fine-grained labels
(assumed to be generated by some upstream models) into a caption that describes
the image in a fluent and informative manner. We introduce a multimodal,
multi-encoder model based on Transformer that ingests both image features and
multiple sources of entity labels. We demonstrate that we can learn to control
the appearance of these entity labels in the output, resulting in captions that
are both fluent and informative.

positive:
The amount of archaeological literature is growing rapidly. Until recently,
these data were only accessible through metadata search. We implemented a text
retrieval engine for a large archaeological text collection ($\sim 658$ Million
words). In archaeological IR, domain-specific entities such as locations, time
periods, and artefacts, play a central role. This motivated the development of
a named entity recognition (NER) model to annotate the full collection with
archaeological named entities. In this paper, we present ArcheoBERTje, a BERT
model pre-trained on Dutch archaeological texts. We compare the model's quality
and output on a Named Entity Recognition task to a generic multilingual model
and a generic Dutch model. We also investigate ensemble methods for combining
multiple BERT models, and combining the best BERT model with a domain thesaurus
using Conditional Random Fields (CRF). We find that ArcheoBERTje outperforms
both the multilingual and Dutch model significantly with a smaller standard
deviation between runs, reaching an average F1 score of 0.735. The model also
outperforms ensemble methods combining the three models. Combining ArcheoBERTje
predictions and explicit domain knowledge from the thesaurus did not increase
the F1 score. We quantitatively and qualitatively analyse the differences
between the vocabulary and output of the BERT models on the full collection and
provide some valuable insights in the effect of fine-tuning for specific
domains. Our results indicate that for a highly specific text domain such as
archaeology, further pre-training on domain-specific data increases the model's
quality on NER by a much larger margin than shown for other domains in the
literature, and that domain-specific pre-training makes the addition of domain
knowledge from a thesaurus unnecessary.

positive:
Most benchmark datasets targeting commonsense reasoning focus on everyday
scenarios: physical knowledge like knowing that you could fill a cup under a
waterfall [Talmor et al., 2019], social knowledge like bumping into someone is
awkward [Sap et al., 2019], and other generic situations. However, there is a
rich space of commonsense inferences anchored to knowledge about specific
entities: for example, deciding the truthfulness of a claim "Harry Potter can
teach classes on how to fly on a broomstick." Can models learn to combine
entity knowledge with commonsense reasoning in this fashion? We introduce
CREAK, a testbed for commonsense reasoning about entity knowledge, bridging
fact-checking about entities (Harry Potter is a wizard and is skilled at riding
a broomstick) with commonsense inferences (if you're good at a skill you can
teach others how to do it). Our dataset consists of 13k human-authored English
claims about entities that are either true or false, in addition to a small
contrast set. Crowdworkers can easily come up with these statements and human
performance on the dataset is high (high 90s); we argue that models should be
able to blend entity knowledge and commonsense reasoning to do well here. In
our experiments, we focus on the closed-book setting and observe that a
baseline model finetuned on existing fact verification benchmark struggles on
CREAK. Training a model on CREAK improves accuracy by a substantial margin, but
still falls short of human performance. Our benchmark provides a unique probe
into natural language understanding models, testing both its ability to
retrieve facts (e.g., who teaches at the University of Chicago?) and unstated
commonsense knowledge (e.g., butlers do not yell at guests).

positive:
Entity Linking (EL) seeks to align entity mentions in text to entries in a
knowledge-base and is usually comprised of two phases: candidate generation and
candidate ranking. While most methods focus on the latter, it is the candidate
generation phase that sets an upper bound to both time and accuracy performance
of the overall EL system. This work's contribution is a significant improvement
in candidate generation which thus raises the performance threshold for EL, by
generating candidates that include the gold entity in the least candidate set
(top-K). We propose a simple approach that efficiently embeds mention-entity
pairs in dense space through a BERT-based bi-encoder. Specifically, we extend
(Wu et al., 2020) by introducing a new pooling function and incorporating
entity type side-information. We achieve a new state-of-the-art 84.28% accuracy
on top-50 candidates on the Zeshel dataset, compared to the previous 82.06% on
the top-64 of (Wu et al., 2020). We report the results from extensive
experimentation using our proposed model on both seen and unseen entity
datasets. Our results suggest that our method could be a useful complement to
existing EL approaches.

positive:
Entity embeddings, which represent different aspects of each entity with a
single vector like word embeddings, are a key component of neural entity
linking models. Existing entity embeddings are learned from canonical Wikipedia
articles and local contexts surrounding target entities. Such entity embeddings
are effective, but too distinctive for linking models to learn contextual
commonality. We propose a simple yet effective method, FGS2EE, to inject
fine-grained semantic information into entity embeddings to reduce the
distinctiveness and facilitate the learning of contextual commonality. FGS2EE
first uses the embeddings of semantic type words to generate semantic
embeddings, and then combines them with existing entity embeddings through
linear aggregation. Extensive experiments show the effectiveness of such
embeddings. Based on our entity embeddings, we achieved new sate-of-the-art
performance on entity linking.

positive:
Knowledge is captured in the form of entities and their relationships and
stored in knowledge graphs. Knowledge graphs enhance the capabilities of
applications in many different areas including Web search, recommendation, and
natural language understanding. This is mainly because, entities enable
machines to understand things that go beyond simple tokens. Many modern
algorithms use learned entity embeddings from these structured representations.
However, building a knowledge graph takes time and effort, hence very costly
and nontrivial. On the other hand, many Web sources describe entities in some
structured format and therefore, finding ways to get them into useful entity
knowledge is advantageous. We propose an approach that processes entity centric
textual knowledge sources to learn entity embeddings and in turn avoids the
need for a traditional knowledge graph. We first extract triples into the new
representation format that does not use traditional complex triple extraction
methods defined by pre-determined relationship labels. Then we learn entity
embeddings through this new type of triples. We show that the embeddings
learned from our approach are: (i) high quality and comparable to a known
knowledge graph-based embeddings and can be used to improve them further, (ii)
better than a contextual language model-based entity embeddings, and (iii) easy
to compute and versatile in domain-specific applications where a knowledge
graph is not readily available

positive:
A growing number of applications users daily interact with have to operate in
(near) real-time: chatbots, digital companions, knowledge work support systems
-- just to name a few. To perform the services desired by the user, these
systems have to analyze user activity logs or explicit user input extremely
fast. In particular, text content (e.g. in form of text snippets) needs to be
processed in an information extraction task. Regarding the aforementioned
temporal requirements, this has to be accomplished in just a few milliseconds,
which limits the number of methods that can be applied. Practically, only very
fast methods remain, which on the other hand deliver worse results than slower
but more sophisticated Natural Language Processing (NLP) pipelines. In this
paper, we investigate and propose methods for real-time capable Named Entity
Recognition (NER). As a first improvement step we address are word variations
induced by inflection, for example present in the German language. Our approach
is ontology-based and makes use of several language information sources like
Wiktionary. We evaluated it using the German Wikipedia (about 9.4B characters),
for which the whole NER process took considerably less than an hour. Since
precision and recall are higher than with comparably fast methods, we conclude
that the quality gap between high speed methods and sophisticated NLP pipelines
can be narrowed a bit more without losing too much runtime performance.

negative:
Over the years, Twitter has become one of the largest communication platforms
providing key data to various applications such as brand monitoring, trend
detection, among others. Entity linking is one of the major tasks in natural
language understanding from tweets and it associates entity mentions in text to
corresponding entries in knowledge bases in order to provide unambiguous
interpretation and additional con- text. State-of-the-art techniques have
focused on linking explicitly mentioned entities in tweets with reasonable
success. However, we argue that in addition to explicit mentions i.e. The movie
Gravity was more ex- pensive than the mars orbiter mission entities (movie
Gravity) can also be mentioned implicitly i.e. This new space movie is crazy.
you must watch it!. This paper introduces the problem of implicit entity
linking in tweets. We propose an approach that models the entities by
exploiting their factual and contextual knowledge. We demonstrate how to use
these models to perform implicit entity linking on a ground truth dataset with
397 tweets from two domains, namely, Movie and Book. Specifically, we show: 1)
the importance of linking implicit entities and its value addition to the
standard entity linking task, and 2) the importance of exploiting contextual
knowledge associated with an entity for linking their implicit mentions. We
also make the ground truth dataset publicly available to foster the research in
this new research area.

negative:
Named Entity Recognition (NER) is a fundamental task in Natural Language
Processing, concerned with identifying spans of text expressing references to
entities. NER research is often focused on flat entities only (flat NER),
ignoring the fact that entity references can be nested, as in [Bank of [China]]
(Finkel and Manning, 2009). In this paper, we use ideas from graph-based
dependency parsing to provide our model a global view on the input via a
biaffine model (Dozat and Manning, 2017). The biaffine model scores pairs of
start and end tokens in a sentence which we use to explore all spans, so that
the model is able to predict named entities accurately. We show that the model
works well for both nested and flat NER through evaluation on 8 corpora and
achieving SoTA performance on all of them, with accuracy gains of up to 2.2
percentage points.

negative:
As a natural language generation task, it is challenging to generate
informative and coherent review text. In order to enhance the informativeness
of the generated text, existing solutions typically learn to copy entities or
triples from knowledge graphs (KGs). However, they lack overall consideration
to select and arrange the incorporated knowledge, which tends to cause text
incoherence.
  To address the above issue, we focus on improving entity-centric coherence of
the generated reviews by leveraging the semantic structure of KGs. In this
paper, we propose a novel Coherence Enhanced Text Planning model (CETP) based
on knowledge graphs (KGs) to improve both global and local coherence for review
generation. The proposed model learns a two-level text plan for generating a
document: (1) the document plan is modeled as a sequence of sentence plans in
order, and (2) the sentence plan is modeled as an entity-based subgraph from
KG. Local coherence can be naturally enforced by KG subgraphs through
intra-sentence correlations between entities. For global coherence, we design a
hierarchical self-attentive architecture with both subgraph- and node-level
attention to enhance the correlations between subgraphs. To our knowledge, we
are the first to utilize a KG-based text planning model to enhance text
coherence for review generation. Extensive experiments on three datasets
confirm the effectiveness of our model on improving the content coherence of
generated texts.

negative:
In this work, we aim at equipping pre-trained language models with structured
knowledge. We present two self-supervised tasks learning over raw text with the
guidance from knowledge graphs. Building upon entity-level masked language
models, our first contribution is an entity masking scheme that exploits
relational knowledge underlying the text. This is fulfilled by using a linked
knowledge graph to select informative entities and then masking their mentions.
In addition we use knowledge graphs to obtain distractors for the masked
entities, and propose a novel distractor-suppressed ranking objective which is
optimized jointly with masked language model. In contrast to existing
paradigms, our approach uses knowledge graphs implicitly, only during
pre-training, to inject language models with structured knowledge via learning
from raw text. It is more efficient than retrieval-based methods that perform
entity linking and integration during finetuning and inference, and generalizes
more effectively than the methods that directly learn from concatenated graph
triples. Experiments show that our proposed model achieves improved performance
on five benchmark datasets, including question answering and knowledge base
completion tasks.

negative:
Much research effort has been put to multilingual knowledge graph (KG)
embedding methods to address the entity alignment task, which seeks to match
entities in different languagespecific KGs that refer to the same real-world
object. Such methods are often hindered by the insufficiency of seed alignment
provided between KGs. Therefore, we propose an incidentally supervised model,
JEANS , which jointly represents multilingual KGs and text corpora in a shared
embedding scheme, and seeks to improve entity alignment with incidental
supervision signals from text. JEANS first deploys an entity grounding process
to combine each KG with the monolingual text corpus. Then, two learning
processes are conducted: (i) an embedding learning process to encode the KG and
text of each language in one embedding space, and (ii) a selflearning based
alignment learning process to iteratively induce the matching of entities and
that of lexemes between embeddings. Experiments on benchmark datasets show that
JEANS leads to promising improvement on entity alignment with incidental
supervision, and significantly outperforms state-of-the-art methods that solely
rely on internal information of KGs.

negative:
In this paper, we propose a new strategy for the task of named entity
recognition (NER). We cast the task as a query-based machine reading
comprehension task: e.g., the task of extracting entities with PER is
formalized as answering the question of "which person is mentioned in the text
?". Such a strategy comes with the advantage that it solves the long-standing
issue of handling overlapping or nested entities (the same token that
participates in more than one entity categories) with sequence-labeling
techniques for NER. Additionally, since the query encodes informative prior
knowledge, this strategy facilitates the process of entity extraction, leading
to better performances. We experiment the proposed model on five widely used
NER datasets on English and Chinese, including MSRA, Resume, OntoNotes, ACE04
and ACE05. The proposed model sets new SOTA results on all of these datasets.

negative:
Recently, end-to-end (E2E) trained models for question answering over
knowledge graphs (KGQA) have delivered promising results using only a weakly
supervised dataset. However, these models are trained and evaluated in a
setting where hand-annotated question entities are supplied to the model,
leaving the important and non-trivial task of entity resolution (ER) outside
the scope of E2E learning. In this work, we extend the boundaries of E2E
learning for KGQA to include the training of an ER component. Our model only
needs the question text and the answer entities to train, and delivers a
stand-alone QA model that does not require an additional ER component to be
supplied during runtime. Our approach is fully differentiable, thanks to its
reliance on a recent method for building differentiable KGs (Cohen et al.,
2020). We evaluate our E2E trained model on two public datasets and show that
it comes close to baseline models that use hand-annotated entities.

negative:
Knowledge graph learning plays a critical role in integrating domain specific
knowledge bases when deploying machine learning and data mining models in
practice. Existing methods on knowledge graph learning primarily focus on
modeling the relations among entities as translations among the relations and
entities, and many of these methods are not able to handle zero-shot problems,
when new entities emerge. In this paper, we present a new convolutional neural
network (CNN)-based dual-chain model. Different from translation based methods,
in our model, interactions among relations and entities are directly captured
via CNN over their embeddings. Moreover, a secondary chain of learning is
conducted simultaneously to incorporate additional information and to enable
better performance. We also present an extension of this model, which
incorporates descriptions of entities and learns a second set of entity
embeddings from the descriptions. As a result, the extended model is able to
effectively handle zero-shot problems. We conducted comprehensive experiments,
comparing our methods with 15 methods on 8 benchmark datasets. Extensive
experimental results demonstrate that our proposed methods achieve or
outperform the state-of-the-art results on knowledge graph learning, and
outperform other methods on zero-shot problems. In addition, our methods
applied to real-world biomedical data are able to produce results that conform
to expert domain knowledge.

negative:
We present a multilingual bag-of-entities model that effectively boosts the
performance of zero-shot cross-lingual text classification by extending a
multilingual pre-trained language model (e.g., M-BERT). It leverages the
multilingual nature of Wikidata: entities in multiple languages representing
the same concept are defined with a unique identifier. This enables entities
described in multiple languages to be represented using shared embeddings. A
model trained on entity features in a resource-rich language can thus be
directly applied to other languages. Our experimental results on cross-lingual
topic classification (using the MLDoc and TED-CLDC datasets) and entity typing
(using the SHINRA2020-ML dataset) show that the proposed model consistently
outperforms state-of-the-art models.

negative:
The state-of-the-art named entity recognition (NER) systems are statistical
machine learning models that have strong generalization capability (i.e., can
recognize unseen entities that do not appear in training data) based on lexical
and contextual information. However, such a model could still make mistakes if
its features favor a wrong entity type. In this paper, we utilize Wikipedia as
an open knowledge base to improve multilingual NER systems. Central to our
approach is the construction of high-accuracy, high-coverage multilingual
Wikipedia entity type mappings. These mappings are built from weakly annotated
data and can be extended to new languages with no human annotation or
language-dependent knowledge involved. Based on these mappings, we develop
several approaches to improve an NER system. We evaluate the performance of the
approaches via experiments on NER systems trained for 6 languages. Experimental
results show that the proposed approaches are effective in improving the
accuracy of such systems on unseen entities, especially when a system is
applied to a new domain or it is trained with little training data (up to 18.3
F1 score improvement).

negative:
We address the problem of calibrating prediction confidence for output
entities of interest in natural language processing (NLP) applications. It is
important that NLP applications such as named entity recognition and question
answering produce calibrated confidence scores for their predictions,
especially if the system is to be deployed in a safety-critical domain such as
healthcare. However, the output space of such structured prediction models is
often too large to adapt binary or multi-class calibration methods directly. In
this study, we propose a general calibration scheme for output entities of
interest in neural-network based structured prediction models. Our proposed
method can be used with any binary class calibration scheme and a neural
network model. Additionally, we show that our calibration method can also be
used as an uncertainty-aware, entity-specific decoding step to improve the
performance of the underlying model at no additional training cost or data
requirements. We show that our method outperforms current calibration
techniques for named-entity-recognition, part-of-speech and question answering.
We also improve our model's performance from our decoding step across several
tasks and benchmark datasets. Our method improves the calibration and model
performance on out-of-domain test scenarios as well.

negative:
The first stage of every knowledge base question answering approach is to
link entities in the input question. We investigate entity linking in the
context of a question answering task and present a jointly optimized neural
architecture for entity mention detection and entity disambiguation that models
the surrounding context on different levels of granularity. We use the Wikidata
knowledge base and available question answering datasets to create benchmarks
for entity linking on question answering data. Our approach outperforms the
previous state-of-the-art system on this data, resulting in an average 8%
improvement of the final score. We further demonstrate that our model delivers
a strong performance across different entity categories.

negative:
Natural language understanding involves reading between the lines with
implicit background knowledge. Current systems either rely on pre-trained
language models as the sole implicit source of world knowledge, or resort to
external knowledge bases (KBs) to incorporate additional relevant knowledge. We
propose an unsupervised framework based on self-talk as a novel alternative to
multiple-choice commonsense tasks. Inspired by inquiry-based discovery learning
(Bruner, 1961), our approach inquires language models with a number of
information seeking questions such as "$\textit{what is the definition of
...}$" to discover additional background knowledge. Empirical results
demonstrate that the self-talk procedure substantially improves the performance
of zero-shot language model baselines on four out of six commonsense
benchmarks, and competes with models that obtain knowledge from external KBs.
While our approach improves performance on several benchmarks, the self-talk
induced knowledge even when leading to correct answers is not always seen as
useful by human judges, raising interesting questions about the inner-workings
of pre-trained language models for commonsense reasoning.

negative:
We focus on the problem of learning distributed representations for entity
search queries, named entities, and their short descriptions. With our
representation learning models, the entity search query, named entity and
description can be represented as low-dimensional vectors. Our goal is to
develop a simple but effective model that can make the distributed
representations of query related entities similar to the query in the vector
space. Hence, we propose three kinds of learning strategies, and the difference
between them mainly lies in how to deal with the relationship between an entity
and its description. We analyze the strengths and weaknesses of each learning
strategy and validate our methods on public datasets which contain four kinds
of named entities, i.e., movies, TV shows, restaurants and celebrities. The
experimental results indicate that our proposed methods can adapt to different
types of entity search queries, and outperform the current state-of-the-art
methods based on keyword matching and vanilla word2vec models. Besides, the
proposed methods can be trained fast and be easily extended to other similar
tasks.

negative:
Current systems of fine-grained entity typing use distant supervision in
conjunction with existing knowledge bases to assign categories (type labels) to
entity mentions. However, the type labels so obtained from knowledge bases are
often noisy (i.e., incorrect for the entity mention's local context). We define
a new task, Label Noise Reduction in Entity Typing (LNR), to be the automatic
identification of correct type labels (type-paths) for training examples, given
the set of candidate type labels obtained by distant supervision with a given
type hierarchy. The unknown type labels for individual entity mentions and the
semantic similarity between entity types pose unique challenges for solving the
LNR task. We propose a general framework, called PLE, to jointly embed entity
mentions, text features and entity types into the same low-dimensional space
where, in that space, objects whose types are semantically close have similar
representations. Then we estimate the type-path for each training example in a
top-down manner using the learned embeddings. We formulate a global objective
for learning the embeddings from text corpora and knowledge bases, which adopts
a novel margin-based loss that is robust to noisy labels and faithfully models
type correlation derived from knowledge bases. Our experiments on three public
typing datasets demonstrate the effectiveness and robustness of PLE, with an
average of 25% improvement in accuracy compared to next best method.

negative:
Entity representations are useful in natural language tasks involving
entities. In this paper, we propose new pretrained contextualized
representations of words and entities based on the bidirectional transformer.
The proposed model treats words and entities in a given text as independent
tokens, and outputs contextualized representations of them. Our model is
trained using a new pretraining task based on the masked language model of
BERT. The task involves predicting randomly masked words and entities in a
large entity-annotated corpus retrieved from Wikipedia. We also propose an
entity-aware self-attention mechanism that is an extension of the
self-attention mechanism of the transformer, and considers the types of tokens
(words or entities) when computing attention scores. The proposed model
achieves impressive empirical performance on a wide range of entity-related
tasks. In particular, it obtains state-of-the-art results on five well-known
datasets: Open Entity (entity typing), TACRED (relation classification),
CoNLL-2003 (named entity recognition), ReCoRD (cloze-style question answering),
and SQuAD 1.1 (extractive question answering). Our source code and pretrained
representations are available at https://github.com/studio-ousia/luke.

negative:
This thesis presents new methods for unsupervised learning of distributed
representations of words and entities from text and knowledge bases. The first
algorithm presented in the thesis is a multi-view algorithm for learning
representations of words called Multiview Latent Semantic Analysis (MVLSA). By
incorporating up to 46 different types of co-occurrence statistics for the same
vocabulary of english words, I show that MVLSA outperforms other
state-of-the-art word embedding models. Next, I focus on learning entity
representations for search and recommendation and present the second method of
this thesis, Neural Variational Set Expansion (NVSE). NVSE is also an
unsupervised learning method, but it is based on the Variational Autoencoder
framework. Evaluations with human annotators show that NVSE can facilitate
better search and recommendation of information gathered from noisy, automatic
annotation of unstructured natural language corpora. Finally, I move from
unstructured data and focus on structured knowledge graphs. I present novel
approaches for learning embeddings of vertices and edges in a knowledge graph
that obey logical constraints.

negative:
In biomedical literature, it is common for entity boundaries to not align
with word boundaries. Therefore, effective identification of entity spans
requires approaches capable of considering tokens that are smaller than words.
We introduce a novel, subword approach for named entity recognition (NER) that
uses byte-pair encodings (BPE) in combination with convolutional and recurrent
neural networks to produce byte-level tags of entities. We present experimental
results on several standard biomedical datasets, namely the BioCreative VI
Bio-ID, JNLPBA, and GENETAG datasets. We demonstrate competitive performance
while bypassing the specialized domain expertise needed to create biomedical
text tokenization rules.

negative:
Rich entity representations are useful for a wide class of problems involving
entities. Despite their importance, there is no standardized benchmark that
evaluates the overall quality of entity representations. In this work, we
propose EntEval: a test suite of diverse tasks that require nontrivial
understanding of entities including entity typing, entity similarity, entity
relation prediction, and entity disambiguation. In addition, we develop
training techniques for learning better entity representations by using natural
hyperlink annotations in Wikipedia. We identify effective objectives for
incorporating the contextual information in hyperlinks into state-of-the-art
pretrained language models and show that they improve strong baselines on
multiple EntEval tasks.

negative:
In this work, we formulate the NER task as a multi-answer knowledge guided QA
task (KGQA) which helps to predict entities only by assigning B, I and O tags
without associating entity types with the tags. We provide different knowledge
contexts, such as, entity types, questions, definitions and examples along with
the text and train on a combined dataset of 18 biomedical corpora. This
formulation (a) enables systems to jointly learn NER specific features from
varied NER datasets, (b) can use knowledge-text attention to identify words
having higher similarity to provided knowledge, improving performance, (c)
reduces system confusion by reducing the prediction classes to B, I, O only,
and (d) makes detection of nested entities easier. We perform extensive
experiments of this KGQA formulation on 18 biomedical NER datasets, and through
experiments we note that knowledge helps in achieving better performance. Our
problem formulation is able to achieve state-of-the-art results in 12 datasets.

negative:
Learning representations for knowledge base entities and concepts is becoming
increasingly important for NLP applications. However, recent entity embedding
methods have relied on structured resources that are expensive to create for
new domains and corpora. We present a distantly-supervised method for jointly
learning embeddings of entities and text from an unnanotated corpus, using only
a list of mappings between entities and surface forms. We learn embeddings from
open-domain and biomedical corpora, and compare against prior methods that rely
on human-annotated text or large knowledge graph structure. Our embeddings
capture entity similarity and relatedness better than prior work, both in
existing biomedical datasets and a new Wikipedia-based dataset that we release
to the community. Results on analogy completion and entity sense disambiguation
indicate that entities and words capture complementary information that can be
effectively combined for downstream use.

negative:
``Classical'' word embeddings, such as Word2Vec, have been shown to capture
the semantics of words based on their distributional properties. However, their
ability to represent the different meanings that a word may have is limited.
Such approaches also do not explicitly encode relations between entities, as
denoted by words. Embeddings of knowledge bases (KB) capture the explicit
relations between entities denoted by words, but are not able to directly
capture the syntagmatic properties of these words. To our knowledge, recent
research have focused on representation learning that augment the strengths of
one with the other. In this work, we begin exploring another approach using
contextual and KB embeddings jointly at the same level and propose two tasks --
an entity typing and a relation typing task -- that evaluate the performance of
contextual and KB embeddings. We also evaluated a concatenated model of
contextual and KB embeddings with these two tasks, and obtain conclusive
results on the first task. We hope our work may contribute as a basis for
models and datasets that develop in the direction of this approach.

negative:
Named Entity Disambiguation (NED) refers to the task of resolving multiple
named entity mentions in a document to their correct references in a knowledge
base (KB) (e.g., Wikipedia). In this paper, we propose a novel embedding method
specifically designed for NED. The proposed method jointly maps words and
entities into the same continuous vector space. We extend the skip-gram model
by using two models. The KB graph model learns the relatedness of entities
using the link structure of the KB, whereas the anchor context model aims to
align vectors such that similar words and entities occur close to one another
in the vector space by leveraging KB anchors and their context words. By
combining contexts based on the proposed embedding with standard NED features,
we achieved state-of-the-art accuracy of 93.1% on the standard CoNLL dataset
and 85.2% on the TAC 2010 dataset.

negative:
Named entity recognition is one of the core tasks in NLP. Although many
improvements have been made on this task during the last years, the
state-of-the-art systems do not explicitly take into account the recursive
nature of language. Instead of only treating the text as a plain sequence of
words, we incorporate a linguistically-inspired way to recognize entities based
on syntax and tree structures. Our model exploits syntactic relationships among
words using a Tree-LSTM guided by dependency trees. Then, we enhance these
features by applying relative and global attention mechanisms. On the one hand,
the relative attention detects the most informative words in the sentence with
respect to the word being evaluated. On the other hand, the global attention
spots the most relevant words in the sequence. Lastly, we linearly project the
weighted vectors into the tagging space so that a conditional random field
classifier predicts the entity labels. Our findings show that the model detects
words that disclose the entity types based on their syntactic roles in a
sentence (e.g., verbs such as speak and write are attended when the entity type
is PERSON, whereas meet and travel strongly relate to LOCATION). We confirm our
findings and establish a new state of the art on two datasets.

negative:
The lack of labeled training data has limited the development of natural
language processing tools, such as named entity recognition, for many languages
spoken in developing countries. Techniques such as distant and weak supervision
can be used to create labeled data in a (semi-) automatic way. Additionally, to
alleviate some of the negative effects of the errors in automatic annotation,
noise-handling methods can be integrated. Pretrained word embeddings are
another key component of most neural named entity classifiers. With the advent
of more complex contextual word embeddings, an interesting trade-off between
model size and performance arises. While these techniques have been shown to
work well in high-resource settings, we want to study how they perform in
low-resource scenarios. In this work, we perform named entity recognition for
Hausa and Yor\`ub\'a, two languages that are widely spoken in several
developing countries. We evaluate different embedding approaches and show that
distant supervision can be successfully leveraged in a realistic low-resource
scenario where it can more than double a classifier's performance.

negative:
Previous models for learning entity and relationship embeddings of knowledge
graphs such as TransE, TransH, and TransR aim to explore new links based on
learned representations. However, these models interpret relationships as
simple translations on entity embeddings. In this paper, we try to learn more
complex connections between entities and relationships. In particular, we use a
Convolutional Neural Network (CNN) to learn entity and relationship
representations in knowledge graphs. In our model, we treat entities and
relationships as one-dimensional numerical sequences with the same length.
After that, we combine each triplet of head, relationship, and tail together as
a matrix with height 3. CNN is applied to the triplets to get confidence
scores. Positive and manually corrupted negative triplets are used to train the
embeddings and the CNN model simultaneously. Experimental results on public
benchmark datasets show that the proposed model outperforms state-of-the-art
models on exploring unseen relationships, which proves that CNN is effective to
learn complex interactive patterns between entities and relationships.

negative:
Universal schema jointly embeds knowledge bases and textual patterns to
reason about entities and relations for automatic knowledge base construction
and information extraction. In the past, entity pairs and relations were
represented as learned vectors with compatibility determined by a scoring
function, limiting generalization to unseen text patterns and entities.
Recently, 'column-less' versions of Universal Schema have used compositional
pattern encoders to generalize to all text patterns. In this work we take the
next step and propose a 'row-less' model of universal schema, removing explicit
entity pair representations. Instead of learning vector representations for
each entity pair in our training set, we treat an entity pair as a function of
its relation types. In experimental results on the FB15k-237 benchmark we
demonstrate that we can match the performance of a comparable model with
explicit entity pair representations using a model of attention over relation
types. We further demonstrate that the model per- forms with nearly the same
accuracy on entity pairs never seen during training.

negative:
Fine-grained Named Entity Recognition is a task whereby we detect and
classify entity mentions to a large set of types. These types can span diverse
domains such as finance, healthcare, and politics. We observe that when the
type set spans several domains the accuracy of the entity detection becomes a
limitation for supervised learning models. The primary reason being the lack of
datasets where entity boundaries are properly annotated, whilst covering a
large spectrum of entity types. Furthermore, many named entity systems suffer
when considering the categorization of fine grained entity types. Our work
attempts to address these issues, in part, by combining state-of-the-art deep
learning models (ELMo) with an expansive knowledge base (Wikidata). Using our
framework, we cross-validate our model on the 112 fine-grained entity types
based on the hierarchy given from the Wiki(gold) dataset.

negative:
We study the problem of training named entity recognition (NER) models using
only distantly-labeled data, which can be automatically obtained by matching
entity mentions in the raw text with entity types in a knowledge base. The
biggest challenge of distantly-supervised NER is that the distant supervision
may induce incomplete and noisy labels, rendering the straightforward
application of supervised learning ineffective. In this paper, we propose (1) a
noise-robust learning scheme comprised of a new loss function and a noisy label
removal step, for training NER models on distantly-labeled data, and (2) a
self-training method that uses contextualized augmentations created by
pre-trained language models to improve the generalization ability of the NER
model. On three benchmark datasets, our method achieves superior performance,
outperforming existing distantly-supervised NER models by significant margins.

negative:
Fine-grained entity typing is a challenging problem since it usually involves
a relatively large tag set and may require to understand the context of the
entity mention. In this paper, we use entity linking to help with the
fine-grained entity type classification process. We propose a deep neural model
that makes predictions based on both the context and the information obtained
from entity linking results. Experimental results on two commonly used datasets
demonstrates the effectiveness of our approach. On both datasets, it achieves
more than 5\% absolute strict accuracy improvement over the state of the art.

negative:
This paper explores the problem of matching entities across different
knowledge graphs. Given a query entity in one knowledge graph, we wish to find
the corresponding real-world entity in another knowledge graph. We formalize
this problem and present two large-scale datasets for this task based on
exiting cross-ontology links between DBpedia and Wikidata, focused on several
hundred thousand ambiguous entities. Using a classification-based approach, we
find that a simple multi-layered perceptron based on representations derived
from RDF2Vec graph embeddings of entities in each knowledge graph is sufficient
to achieve high accuracy, with only small amounts of training data. The
contributions of our work are datasets for examining this problem and strong
baselines on which future work can be based.

negative:
Entity retrieval, which aims at disambiguating mentions to canonical entities
from massive KBs, is essential for many tasks in natural language processing.
Recent progress in entity retrieval shows that the dual-encoder structure is a
powerful and efficient framework to nominate candidates if entities are only
identified by descriptions. However, they ignore the property that meanings of
entity mentions diverge in different contexts and are related to various
portions of descriptions, which are treated equally in previous works. In this
work, we propose Multi-View Entity Representations (MuVER), a novel approach
for entity retrieval that constructs multi-view representations for entity
descriptions and approximates the optimal view for mentions via a heuristic
searching method. Our method achieves the state-of-the-art performance on
ZESHEL and improves the quality of candidates on three standard Entity Linking
datasets

negative:
This paper introduces SocialVec, a general framework for eliciting social
world knowledge from social networks, and applies this framework to Twitter.
SocialVec learns low-dimensional embeddings of popular accounts, which
represent entities of general interest, based on their co-occurrences patterns
within the accounts followed by individual users, thus modeling entity
similarity in socio-demographic terms. Similar to word embeddings, which
facilitate tasks that involve text processing, we expect social entity
embeddings to benefit tasks of social flavor. We have learned social embeddings
for roughly 200,000 popular accounts from a sample of the Twitter network that
includes more than 1.3 million users and the accounts that they follow, and
evaluate the resulting embeddings on two different tasks. The first task
involves the automatic inference of personal traits of users from their social
media profiles. In another study, we exploit SocialVec embeddings for gauging
the political bias of news sources in Twitter. In both cases, we prove
SocialVec embeddings to be advantageous compared with existing entity embedding
schemes. We will make the SocialVec entity embeddings publicly available to
support further exploration of social world knowledge as reflected in Twitter.

negative:
Question-answering systems and voice assistants are becoming major part of
client service departments of many organizations, helping them to reduce the
labor costs of staff. In many such systems, there is always natural language
understanding module that solves intent classification task. This task is
complicated because of its case-dependency - every subject area has its own
semantic kernel. The state of art approaches for intent classification are
different machine learning and deep learning methods that use text vector
representations as input. The basic vector representation models such as Bag of
words and TF-IDF generate sparse matrixes, which are becoming very big as the
amount of input data grows. Modern methods such as word2vec and FastText use
neural networks to evaluate word embeddings with fixed dimension size. As we
are developing a question-answering system for students and enrollees of the
Perm National Research Polytechnic University, we have faced the problem of
user's intent detection. The subject area of our system is very specific, that
is why there is a lack of training data. This aspect makes intent
classification task more challenging for using state of the art deep learning
methods. In this paper, we propose an approach of the questions embeddings
representation based on calculation of Shannon entropy.The goal of the approach
is to produce low dimensional question vectors as neural approaches do and to
outperform related methods, described above in condition of small dataset. We
evaluate and compare our model with existing ones using logistic regression and
dataset that contains questions asked by students and enrollees. The data is
labeled into six classes. Experimental comparison of proposed approach and
other models revealed that proposed model performed better in the given task.

negative:
Knowledge graphs (KGs) consisting of a large number of triples have become
widespread recently, and many knowledge graph embedding (KGE) methods are
proposed to embed entities and relations of a KG into continuous vector spaces.
Such embedding methods simplify the operations of conducting various in-KG
tasks (e.g., link prediction) and out-of-KG tasks (e.g., question answering).
They can be viewed as general solutions for representing KGs. However, existing
KGE methods are not applicable to inductive settings, where a model trained on
source KGs will be tested on target KGs with entities unseen during model
training. Existing works focusing on KGs in inductive settings can only solve
the inductive relation prediction task. They can not handle other out-of-KG
tasks as general as KGE methods since they don't produce embeddings for
entities. In this paper, to achieve inductive knowledge graph embedding, we
propose a model MorsE, which does not learn embeddings for entities but learns
transferable meta-knowledge that can be used to produce entity embeddings. Such
meta-knowledge is modeled by entity-independent modules and learned by
meta-learning. Experimental results show that our model significantly
outperforms corresponding baselines for in-KG and out-of-KG tasks in inductive
settings.

negative:
The Biocreative VII Track-2 challenge consists of named entity recognition,
entity-linking (or entity-normalization), and topic indexing tasks -- with
entities and topics limited to chemicals for this challenge. Named entity
recognition is a well-established problem and we achieve our best performance
with BERT-based BioMegatron models. We extend our BERT-based approach to the
entity linking task. After the second stage of pretraining BioBERT with a
metric-learning loss strategy called self-alignment pretraining (SAP), we link
entities based on the cosine similarity between their SAP-BioBERT word
embeddings. Despite the success of our named entity recognition experiments, we
find the chemical indexing task generally more challenging.
  In addition to conventional NER methods, we attempt both named entity
recognition and entity linking with a novel text-to-text or "prompt" based
method that uses generative language models such as T5 and GPT. We achieve
encouraging results with this new approach.

negative:
Much of human knowledge is encoded in text, available in scientific
publications, books, and the web. Given the rapid growth of these resources, we
need automated methods to extract such knowledge into machine-processable
structures, such as knowledge graphs. An important task in this process is
entity normalization, which consists of mapping noisy entity mentions in text
to canonical entities in well-known reference sets. However, entity
normalization is a challenging problem; there often are many textual forms for
a canonical entity that may not be captured in the reference set, and entities
mentioned in text may include many syntactic variations, or errors. The problem
is particularly acute in scientific domains, such as biology. To address this
problem, we have developed a general, scalable solution based on a deep Siamese
neural network model to embed the semantic information about the entities, as
well as their syntactic variations. We use these embeddings for fast mapping of
new entities to large reference sets, and empirically show the effectiveness of
our framework in challenging bio-entity normalization datasets.

negative:
Entity alignment (EA) is the task to discover entities referring to the same
real-world object from different knowledge graphs (KGs), which is the most
crucial step in integrating multi-source KGs. The majority of the existing
embeddings-based entity alignment methods embed entities and relations into a
vector space based on relation triples of KGs for local alignment. As these
methods insufficiently consider the multiple relations between entities, the
structure information of KGs has not been fully leveraged. In this paper, we
propose a novel framework based on Relation-aware Graph Attention Networks to
capture the interactions between entities and relations. Our framework adopts
the self-attention mechanism to spread entity information to the relations and
then aggregate relation information back to entities. Furthermore, we propose a
global alignment algorithm to make one-to-one entity alignments with a
fine-grained similarity matrix. Experiments on three real-world cross-lingual
datasets show that our framework outperforms the state-of-the-art methods.

negative:
Named entity recognition is a natural language processing task to recognize
and extract spans of text associated with named entities and classify them in
semantic Categories.
  Google BERT is a deep bidirectional language model, pre-trained on large
corpora that can be fine-tuned to solve many NLP tasks such as question
answering, named entity recognition, part of speech tagging and etc. In this
paper, we use the pre-trained deep bidirectional network, BERT, to make a model
for named entity recognition in Persian.
  We also compare the results of our model with the previous state of the art
results achieved on Persian NER. Our evaluation metric is CONLL 2003 score in
two levels of word and phrase. This model achieved second place in NSURL-2019
task 7 competition which associated with NER for the Persian language. our
results in this competition are 83.5 and 88.4 f1 CONLL score respectively in
phrase and word level evaluation.

negative:
The objective of knowledge graph embedding is to encode both entities and
relations of knowledge graphs into continuous low-dimensional vector spaces.
Previously, most works focused on symbolic representation of knowledge graph
with structure information, which can not handle new entities or entities with
few facts well. In this paper, we propose a novel deep architecture to utilize
both structural and textual information of entities. Specifically, we introduce
three neural models to encode the valuable information from text description of
entity, among which an attentive model can select related information as
needed. Then, a gating mechanism is applied to integrate representations of
structure and text into a unified architecture. Experiments show that our
models outperform baseline by margin on link prediction and triplet
classification tasks. Source codes of this paper will be available on Github.

negative:
Reading comprehension is a challenging task in natural language processing
and requires a set of skills to be solved. While current approaches focus on
solving the task as a whole, in this paper, we propose to use a neural network
`skill' transfer approach. We transfer knowledge from several lower-level
language tasks (skills) including textual entailment, named entity recognition,
paraphrase detection and question type classification into the reading
comprehension model.
  We conduct an empirical evaluation and show that transferring language skill
knowledge leads to significant improvements for the task with much fewer steps
compared to the baseline model. We also show that the skill transfer approach
is effective even with small amounts of training data. Another finding of this
work is that using token-wise deep label supervision for text classification
improves the performance of transfer learning.

negative:
Deep learning currently dominates the benchmarks for various NLP tasks and,
at the basis of such systems, words are frequently represented as embeddings
--vectors in a low dimensional space-- learned from large text corpora and
various algorithms have been proposed to learn both word and concept
embeddings. One of the claimed benefits of such embeddings is that they capture
knowledge about semantic relations. Such embeddings are most often evaluated
through tasks such as predicting human-rated similarity and analogy which only
test a few, often ill-defined, relations. In this paper, we propose a method
for (i) reliably generating word and concept pair datasets for a wide number of
relations by using a knowledge graph and (ii) evaluating to what extent
pre-trained embeddings capture those relations. We evaluate the approach
against a proprietary and a public knowledge graph and analyze the results,
showing which lexico-semantic relational knowledge is captured by current
embedding learning approaches.

negative:
Named entity recognition (NER) is a fundamental component in many
applications, such as Web Search and Voice Assistants. Although deep neural
networks greatly improve the performance of NER, due to the requirement of
large amounts of training data, deep neural networks can hardly scale out to
many languages in an industry setting. To tackle this challenge, cross-lingual
NER transfers knowledge from a rich-resource language to languages with low
resources through pre-trained multilingual language models. Instead of using
training data in target languages, cross-lingual NER has to rely on only
training data in source languages, and optionally adds the translated training
data derived from source languages. However, the existing cross-lingual NER
methods do not make good use of rich unlabeled data in target languages, which
is relatively easy to collect in industry applications. To address the
opportunities and challenges, in this paper we describe our novel practice in
Microsoft to leverage such large amounts of unlabeled data in target languages
in real production settings. To effectively extract weak supervision signals
from the unlabeled data, we develop a novel approach based on the ideas of
semi-supervised learning and reinforcement learning. The empirical study on
three benchmark data sets verifies that our approach establishes the new
state-of-the-art performance with clear edges. Now, the NER techniques reported
in this paper are on their way to become a fundamental component for Web
ranking, Entity Pane, Answers Triggering, and Question Answering in the
Microsoft Bing search engine. Moreover, our techniques will also serve as part
of the Spoken Language Understanding module for a commercial voice assistant.
We plan to open source the code of the prototype framework after deployment.

negative:
Named entity recognition (NER) is a widely applicable natural language
processing task and building block of question answering, topic modeling,
information retrieval, etc. In the medical domain, NER plays a crucial role by
extracting meaningful chunks from clinical notes and reports, which are then
fed to downstream tasks like assertion status detection, entity resolution,
relation extraction, and de-identification. Reimplementing a Bi-LSTM-CNN-Char
deep learning architecture on top of Apache Spark, we present a single
trainable NER model that obtains new state-of-the-art results on seven public
biomedical benchmarks without using heavy contextual embeddings like BERT. This
includes improving BC4CHEMD to 93.72% (4.1% gain), Species800 to 80.91% (4.6%
gain), and JNLPBA to 81.29% (5.2% gain). In addition, this model is freely
available within a production-grade code base as part of the open-source Spark
NLP library; can scale up for training and inference in any Spark cluster; has
GPU support and libraries for popular programming languages such as Python, R,
Scala and Java; and can be extended to support other human languages with no
code changes.

negative:
Most state-of-the-art models in natural language processing (NLP) are neural
models built on top of large, pre-trained, contextual language models that
generate representations of words in context and are fine-tuned for the task at
hand. The improvements afforded by these "contextual embeddings" come with a
high computational cost. In this work, we explore a simple technique that
substantially and consistently improves performance over a strong baseline with
negligible increase in run time. We concatenate multiple pre-trained embeddings
to strengthen our representation of words. We show that this concatenation
technique works across many tasks, datasets, and model types. We analyze
aspects of pre-trained embedding similarity and vocabulary coverage and find
that the representational diversity between different pre-trained embeddings is
the driving force of why this technique works. We provide open source
implementations of our models in both TensorFlow and PyTorch.

negative:
This paper addresses the problem of corpus-level entity typing, i.e.,
inferring from a large corpus that an entity is a member of a class such as
"food" or "artist". The application of entity typing we are interested in is
knowledge base completion, specifically, to learn which classes an entity is a
member of. We propose FIGMENT to tackle this problem. FIGMENT is embedding-
based and combines (i) a global model that scores based on aggregated
contextual information of an entity and (ii) a context model that first scores
the individual occurrences of an entity and then aggregates the scores. Each of
the two proposed models has some specific properties. For the global model,
learning high quality entity representations is crucial because it is the only
source used for the predictions. Therefore, we introduce representations using
name and contexts of entities on the three levels of entity, word, and
character. We show each has complementary information and a multi-level
representation is the best. For the context model, we need to use distant
supervision since the context-level labels are not available for entities.
Distant supervised labels are noisy and this harms the performance of models.
Therefore, we introduce and apply new algorithms for noise mitigation using
multi-instance learning. We show the effectiveness of our models in a large
entity typing dataset, built from Freebase.

negative:
In many scenarios, named entity recognition (NER) models severely suffer from
unlabeled entity problem, where the entities of a sentence may not be fully
annotated. Through empirical studies performed on synthetic datasets, we find
two causes of performance degradation. One is the reduction of annotated
entities and the other is treating unlabeled entities as negative instances.
The first cause has less impact than the second one and can be mitigated by
adopting pretraining language models. The second cause seriously misguides a
model in training and greatly affects its performances. Based on the above
observations, we propose a general approach, which can almost eliminate the
misguidance brought by unlabeled entities. The key idea is to use negative
sampling that, to a large extent, avoids training NER models with unlabeled
entities. Experiments on synthetic datasets and real-world datasets show that
our model is robust to unlabeled entity problem and surpasses prior baselines.
On well-annotated datasets, our model is competitive with the state-of-the-art
method.

negative:
Semantic annotation, the process of identifying key-phrases in texts and
linking them to concepts in a knowledge base, is an important basis for
semantic information retrieval and the Semantic Web uptake. Despite the
emergence of semantic annotation systems, very few comparative studies have
been published on their performance. In this paper, we provide an evaluation of
the performance of existing systems over three tasks: full semantic annotation,
named entity recognition, and keyword detection. More specifically, the
spotting capability (recognition of relevant surface forms in text) is
evaluated for all three tasks, whereas the disambiguation (correctly
associating an entity from Wikipedia or DBpedia to the spotted surface forms)
is evaluated only for the first two tasks. Our evaluation is twofold: First, we
compute standard precision and recall on the output of semantic annotators on
diverse datasets, each best suited for one of the identified tasks. Second, we
build a statistical model using logistic regression to identify significant
performance differences. Our results show that systems that provide full
annotation perform better than named entities annotators and keyword
extractors, for all three tasks. However, there is still much room for
improvement for the identification of the most relevant entities described in a
text.

negative:
Machine understanding of user utterances in conversational systems is of
utmost importance for enabling engaging and meaningful conversations with
users. Entity Linking (EL) is one of the means of text understanding, with
proven efficacy for various downstream tasks in information retrieval. In this
paper, we study entity linking for conversational systems. To develop a better
understanding of what EL in a conversational setting entails, we analyze a
large number of dialogues from existing conversational datasets and annotate
references to concepts, named entities, and personal entities using
crowdsourcing. Based on the annotated dialogues, we identify the main
characteristics of conversational entity linking. Further, we report on the
performance of traditional EL systems on our Conversational Entity Linking
dataset, ConEL, and present an extension to these methods to better fit the
conversational setting. The resources released with this paper include
annotated datasets, detailed descriptions of crowdsourcing setups, as well as
the annotations produced by various EL systems. These new resources allow for
an investigation of how the role of entities in conversations is different from
that in documents or isolated short text utterances like queries and tweets,
and complement existing conversational datasets.

negative:
Named entity recognition (NER) is a vital task in spoken language
understanding, which aims to identify mentions of named entities in text e.g.,
from transcribed speech. Existing neural models for NER rely mostly on
dedicated word-level representations, which suffer from two main shortcomings.
First, the vocabulary size is large, yielding large memory requirements and
training time. Second, these models are not able to learn morphological or
phonological representations. To remedy the above shortcomings, we adopt a
neural solution based on bidirectional LSTMs and conditional random fields,
where we rely on subword units, namely characters, phonemes, and bytes. For
each word in an utterance, our model learns a representation from each of the
subword units. We conducted experiments in a real-world large-scale setting for
the use case of a voice-controlled device covering four languages with up to
5.5M utterances per language. Our experiments show that (1) with increasing
training data, performance of models trained solely on subword units becomes
closer to that of models with dedicated word-level embeddings (91.35 vs 93.92
F1 for English), while using a much smaller vocabulary size (332 vs 74K), (2)
subword units enhance models with dedicated word-level embeddings, and (3)
combining different subword units improves performance.

negative:
The field of natural language understanding has experienced exponential
progress in the last few years, with impressive results in several tasks. This
success has motivated researchers to study the underlying knowledge encoded by
these models. Despite this, attempts to understand their semantic capabilities
have not been successful, often leading to non-conclusive, or contradictory
conclusions among different works. Via a probing classifier, we extract the
underlying knowledge graph of nine of the most influential language models of
the last years, including word embeddings, text generators, and context
encoders. This probe is based on concept relatedness, grounded on WordNet. Our
results reveal that all the models encode this knowledge, but suffer from
several inaccuracies. Furthermore, we show that the different architectures and
training strategies lead to different model biases. We conduct a systematic
evaluation to discover specific factors that explain why some concepts are
challenging. We hope our insights will motivate the development of models that
capture concepts more precisely.

negative:
This paper deals with the entity extraction task (named entity recognition)
of a text mining process that aims at unveiling non-trivial semantic
structures, such as relationships and interaction between entities or
communities. In this paper we present a simple and efficient named entity
extraction algorithm. The method, named PAMPO (PAttern Matching and POs tagging
based algorithm for NER), relies on flexible pattern matching, part-of-speech
tagging and lexical-based rules. It was developed to process texts written in
Portuguese, however it is potentially applicable to other languages as well.
  We compare our approach with current alternatives that support Named Entity
Recognition (NER) for content written in Portuguese. These are Alchemy, Zemanta
and Rembrandt. Evaluation of the efficacy of the entity extraction method on
several texts written in Portuguese indicates a considerable improvement on
$recall$ and $F_1$ measures.

negative:
Humans use language to refer to entities in the external world. Motivated by
this, in recent years several models that incorporate a bias towards learning
entity representations have been proposed. Such entity-centric models have
shown empirical success, but we still know little about why. In this paper we
analyze the behavior of two recently proposed entity-centric models in a
referential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4).
We show that these models outperform the state of the art on this task, and
that they do better on lower frequency entities than a counterpart model that
is not entity-centric, with the same model size. We argue that making models
entity-centric naturally fosters good architectural decisions. However, we also
show that these models do not really build entity representations and that they
make poor use of linguistic context. These negative results underscore the need
for model analysis, to test whether the motivations for particular
architectures are borne out in how models behave when deployed.

negative:
In this research, we improve upon the current state of the art in entity
retrieval by re-ranking the result list using graph embeddings. The paper shows
that graph embeddings are useful for entity-oriented search tasks. We
demonstrate empirically that encoding information from the knowledge graph into
(graph) embeddings contributes to a higher increase in effectiveness of entity
retrieval results than using plain word embeddings. We analyze the impact of
the accuracy of the entity linker on the overall retrieval effectiveness. Our
analysis further deploys the cluster hypothesis to explain the observed
advantages of graph embeddings over the more widely used word embeddings, for
user tasks involving ranking entities.

negative:
We introduce EventNarrative, a knowledge graph-to-text dataset from publicly
available open-world knowledge graphs. Given the recent advances in
event-driven Information Extraction (IE), and that prior research on
graph-to-text only focused on entity-driven KGs, this paper focuses on
event-centric data. However, our data generation system can still be adapted to
other other types of KG data. Existing large-scale datasets in the
graph-to-text area are non-parallel, meaning there is a large disconnect
between the KGs and text. The datasets that have a paired KG and text, are
small scale and manually generated or generated without a rich ontology, making
the corresponding graphs sparse. Furthermore, these datasets contain many
unlinked entities between their KG and text pairs. EventNarrative consists of
approximately 230,000 graphs and their corresponding natural language text, 6
times larger than the current largest parallel dataset. It makes use of a rich
ontology, all of the KGs entities are linked to the text, and our manual
annotations confirm a high data quality. Our aim is two-fold: help break new
ground in event-centric research where data is lacking, and to give researchers
a well-defined, large-scale dataset in order to better evaluate existing and
future knowledge graph-to-text models. We also evaluate two types of baseline
on EventNarrative: a graph-to-text specific model and two state-of-the-art
language models, which previous work has shown to be adaptable to the knowledge
graph-to-text domain.

negative:
One of the key obstacles in making learning protocols realistic in
applications is the need to supervise them, a costly process that often
requires hiring domain experts. We consider the framework to use the world
knowledge as indirect supervision. World knowledge is general-purpose
knowledge, which is not designed for any specific domain. Then the key
challenges are how to adapt the world knowledge to domains and how to represent
it for learning. In this paper, we provide an example of using world knowledge
for domain dependent document clustering. We provide three ways to specify the
world knowledge to domains by resolving the ambiguity of the entities and their
types, and represent the data with world knowledge as a heterogeneous
information network. Then we propose a clustering algorithm that can cluster
multiple types and incorporate the sub-type information as constraints. In the
experiments, we use two existing knowledge bases as our sources of world
knowledge. One is Freebase, which is collaboratively collected knowledge about
entities and their organizations. The other is YAGO2, a knowledge base
automatically extracted from Wikipedia and maps knowledge to the linguistic
knowledge base, WordNet. Experimental results on two text benchmark datasets
(20newsgroups and RCV1) show that incorporating world knowledge as indirect
supervision can significantly outperform the state-of-the-art clustering
algorithms as well as clustering algorithms enhanced with world knowledge
features.

negative:
Knowledge bases of real-world facts about entities and their relationships
are useful resources for a variety of natural language processing tasks.
However, because knowledge bases are typically incomplete, it is useful to be
able to perform link prediction or knowledge base completion, i.e., predict
whether a relationship not in the knowledge base is likely to be true. This
paper combines insights from several previous link prediction models into a new
embedding model STransE that represents each entity as a low-dimensional
vector, and each relation by two matrices and a translation vector. STransE is
a simple combination of the SE and TransE models, but it obtains better link
prediction performance on two benchmark datasets than previous embedding
models. Thus, STransE can serve as a new baseline for the more complex models
in the link prediction task.

negative:
Pre-trained word embeddings learned from unlabeled text have become a
standard component of neural network architectures for NLP tasks. However, in
most cases, the recurrent network that operates on word-level representations
to produce context sensitive representations is trained on relatively little
labeled data. In this paper, we demonstrate a general semi-supervised approach
for adding pre- trained context embeddings from bidirectional language models
to NLP systems and apply it to sequence labeling tasks. We evaluate our model
on two standard datasets for named entity recognition (NER) and chunking, and
in both cases achieve state of the art results, surpassing previous systems
that use other forms of transfer or joint learning with additional labeled data
and task specific gazetteers.

negative:
Answering complex questions over knowledge bases (KB-QA) faces huge input
data with billions of facts, involving millions of entities and thousands of
predicates. For efficiency, QA systems first reduce the answer search space by
identifying a set of facts that is likely to contain all answers and relevant
cues. The most common technique for doing this is to apply named entity
disambiguation (NED) systems to the question, and retrieve KB facts for the
disambiguated entities. This work presents CLOCQ, an efficient method that
prunes irrelevant parts of the search space using KB-aware signals. CLOCQ uses
a top-k query processor over score-ordered lists of KB items that combine
signals about lexical matching, relevance to the question, coherence among
candidate items, and connectivity in the KB graph. Experiments with two recent
QA benchmarks for complex questions demonstrate the superiority of CLOCQ over
state-of-the-art baselines with respect to answer presence, size of the search
space, and runtimes.

negative:
Effectively modeling text-rich fresh content such as news articles at
document-level is a challenging problem. To ensure a content-based model
generalize well to a broad range of applications, it is critical to have a
training dataset that is large beyond the scale of human labels while achieving
desired quality. In this work, we address those two challenges by proposing a
novel approach to mine semantically-relevant fresh documents, and their topic
labels, with little human supervision. Meanwhile, we design a multitask model
called NewsEmbed that alternatively trains a contrastive learning with a
multi-label classification to derive a universal document encoder. We show that
the proposed approach can provide billions of high quality organic training
examples and can be naturally extended to multilingual setting where texts in
different languages are encoded in the same semantic space. We experimentally
demonstrate NewsEmbed's competitive performance across multiple natural
language understanding tasks, both supervised and unsupervised.

negative:
Using deep learning for different machine learning tasks such as image
classification and word embedding has recently gained many attentions. Its
appealing performance reported across specific Natural Language Processing
(NLP) tasks in comparison with other approaches is the reason for its
popularity. Word embedding is the task of mapping words or phrases to a low
dimensional numerical vector. In this paper, we use deep learning to embed
Wikipedia Concepts and Entities. The English version of Wikipedia contains more
than five million pages, which suggest its capability to cover many English
Entities, Phrases, and Concepts. Each Wikipedia page is considered as a
concept. Some concepts correspond to entities, such as a person's name, an
organization or a place. Contrary to word embedding, Wikipedia Concepts
Embedding is not ambiguous, so there are different vectors for concepts with
similar surface form but different mentions. We proposed several approaches and
evaluated their performance based on Concept Analogy and Concept Similarity
tasks. The results show that proposed approaches have the performance
comparable and in some cases even higher than the state-of-the-art methods.

negative:
Named Entity Recognition (NER) has been mostly studied in the context of
written text. Specifically, NER is an important step in de-identification
(de-ID) of medical records, many of which are recorded conversations between a
patient and a doctor. In such recordings, audio spans with personal information
should be redacted, similar to the redaction of sensitive character spans in
de-ID for written text. The application of NER in the context of audio
de-identification has yet to be fully investigated. To this end, we define the
task of audio de-ID, in which audio spans with entity mentions should be
detected. We then present our pipeline for this task, which involves Automatic
Speech Recognition (ASR), NER on the transcript text, and text-to-audio
alignment. Finally, we introduce a novel metric for audio de-ID and a new
evaluation benchmark consisting of a large labeled segment of the Switchboard
and Fisher audio datasets and detail our pipeline's results on it.

negative:
Given that in practice training data is scarce for all but a small set of
problems, a core question is how to incorporate prior knowledge into a model.
In this paper, we consider the case of prior procedural knowledge for neural
networks, such as knowing how a program should traverse a sequence, but not
what local actions should be performed at each step. To this end, we present an
end-to-end differentiable interpreter for the programming language Forth which
enables programmers to write program sketches with slots that can be filled
with behaviour trained from program input-output data. We can optimise this
behaviour directly through gradient descent techniques on user-specified
objectives, and also integrate the program into any larger neural computation
graph. We show empirically that our interpreter is able to effectively leverage
different levels of prior program structure and learn complex behaviours such
as sequence sorting and addition. When connected to outputs of an LSTM and
trained jointly, our interpreter achieves state-of-the-art accuracy for
end-to-end reasoning about quantities expressed in natural language stories.

negative:
This paper addresses the problem of corpus-level entity typing, i.e.,
inferring from a large corpus that an entity is a member of a class such as
"food" or "artist". The application of entity typing we are interested in is
knowledge base completion, specifically, to learn which classes an entity is a
member of. We propose FIGMENT to tackle this problem. FIGMENT is
embedding-based and combines (i) a global model that scores based on aggregated
contextual information of an entity and (ii) a context model that first scores
the individual occurrences of an entity and then aggregates the scores. In our
evaluation, FIGMENT strongly outperforms an approach to entity typing that
relies on relations obtained by an open information extraction system.

negative:
This survey presents a comprehensive description of recent neural entity
linking (EL) systems developed since 2015 as a result of the "deep learning
revolution" in natural language processing. Its goal is to systemize design
features of neural entity linking systems and compare their performance to the
remarkable classic methods on common benchmarks. This work distills a generic
architecture of a neural EL system and discusses its components, such as
candidate generation, mention-context encoding, and entity ranking, summarizing
prominent methods for each of them. The vast variety of modifications of this
general architecture are grouped by several common themes: joint entity mention
detection and disambiguation, models for global linking, domain-independent
techniques including zero-shot and distant supervision methods, and
cross-lingual approaches. Since many neural models take advantage of entity and
mention/context embeddings to represent their meaning, this work also overviews
prominent entity embedding techniques. Finally, the survey touches on
applications of entity linking, focusing on the recently emerged use-case of
enhancing deep pre-trained masked language models based on the Transformer
architecture.

negative:
We address the task of Named Entity Disambiguation (NED) for noisy text. We
present WikilinksNED, a large-scale NED dataset of text fragments from the web,
which is significantly noisier and more challenging than existing news-based
datasets. To capture the limited and noisy local context surrounding each
mention, we design a neural model and train it with a novel method for sampling
informative negative examples. We also describe a new way of initializing word
and entity embeddings that significantly improves performance. Our model
significantly outperforms existing state-of-the-art methods on WikilinksNED
while achieving comparable performance on a smaller newswire dataset.

negative:
Named entity recognition is an important task in natural language processing.
It is very well studied for rich language, but still under explored for
low-resource languages. The main reason is that the existing techniques
required a lot of annotated data to reach good performance. Recently, a new
distributional representation of words has been proposed to project named
entities from a rich language to a low-resource one. This representation has
been coupled to a neural network in order to project named entities from
English to Ewondo, a Bantu language spoken in Cameroon. Although the proposed
method reached appreciable results, the size of the used neural network was too
large compared to the size of the dataset. Furthermore the impact of the model
parameters has not been studied. In this paper, we show experimentally that the
same results can be obtained using a smaller neural network. We also emphasize
the parameters that are highly correlated to the network performance. This work
is a step forward to build a reliable and robust network architecture for named
entity projection in low resource languages.

negative:
Named Entity Recognition (NER) is a fundamental task in the fields of natural
language processing and information extraction. NER has been widely used as a
standalone tool or an essential component in a variety of applications such as
question answering, dialogue assistants and knowledge graphs development.
However, training reliable NER models requires a large amount of labelled data
which is expensive to obtain, particularly in specialized domains. This paper
describes a method to learn a domain-specific NER model for an arbitrary set of
named entities when domain-specific supervision is not available. We assume
that the supervision can be obtained with no human effort, and neural models
can learn from each other. The code, data and models are publicly available.

negative:
This paper presents a discovery that the length of the entities in various
datasets follows a family of scale-free power law distributions. The concept of
entity here broadly includes the named entity, entity mention, time expression,
aspect term, and domain-specific entity that are well investigated in natural
language processing and related areas. The entity length denotes the number of
words in an entity. The power law distributions in entity length possess the
scale-free property and have well-defined means and finite variances. We
explain the phenomenon of power laws in entity length by the principle of least
effort in communication and the preferential mechanism.

negative:
Constraint-based clustering algorithms exploit background knowledge to
construct clusterings that are aligned with the interests of a particular user.
This background knowledge is often obtained by allowing the clustering system
to pose pairwise queries to the user: should these two elements be in the same
cluster or not? Active clustering methods aim to minimize the number of queries
needed to obtain a good clustering by querying the most informative pairs
first. Ideally, a user should be able to answer a couple of these queries,
inspect the resulting clustering, and repeat these two steps until a
satisfactory result is obtained. We present COBRAS, an approach to active
clustering with pairwise constraints that is suited for such an interactive
clustering process. A core concept in COBRAS is that of a super-instance: a
local region in the data in which all instances are assumed to belong to the
same cluster. COBRAS constructs such super-instances in a top-down manner to
produce high-quality results early on in the clustering process, and keeps
refining these super-instances as more pairwise queries are given to get more
detailed clusterings later on. We experimentally demonstrate that COBRAS
produces good clusterings at fast run times, making it an excellent candidate
for the iterative clustering scenario outlined above.

negative:
Entity relatedness has emerged as an important feature in a plethora of
applications such as information retrieval, entity recommendation and entity
linking. Given an entity, for instance a person or an organization, entity
relatedness measures can be exploited for generating a list of highly-related
entities. However, the relation of an entity to some other entity depends on
several factors, with time and context being two of the most important ones
(where, in our case, context is determined by a particular corpus). For
example, the entities related to the International Monetary Fund are different
now compared to some years ago, while these entities also may highly differ in
the context of a USA news portal compared to a Greek news portal. In this
paper, we propose a simple but flexible model for entity relatedness which
considers time and entity aware word embeddings by exploiting the underlying
corpus. The proposed model does not require external knowledge and is language
independent, which makes it widely useful in a variety of applications.

negative:
Continual learning is the problem of sequentially learning new tasks or
knowledge while protecting previously acquired knowledge. However, catastrophic
forgetting poses a grand challenge for neural networks performing such learning
process. Thus, neural networks that are deployed in the real world often
struggle in scenarios where the data distribution is non-stationary (concept
drift), imbalanced, or not always fully available, i.e., rare edge cases. We
propose a Differentiable Hebbian Consolidation model which is composed of a
Differentiable Hebbian Plasticity (DHP) Softmax layer that adds a rapid
learning plastic component (compressed episodic memory) to the fixed (slow
changing) parameters of the softmax output layer; enabling learned
representations to be retained for a longer timescale. We demonstrate the
flexibility of our method by integrating well-known task-specific synaptic
consolidation methods to penalize changes in the slow weights that are
important for each target task. We evaluate our approach on the Permuted MNIST,
Split MNIST and Vision Datasets Mixture benchmarks, and introduce an imbalanced
variant of Permuted MNIST -- a dataset that combines the challenges of class
imbalance and concept drift. Our proposed model requires no additional
hyperparameters and outperforms comparable baselines by reducing forgetting.

negative:
Is pushing numbers on a single benchmark valuable in automatic speech
recognition? Research results in acoustic modeling are typically evaluated
based on performance on a single dataset. While the research community has
coalesced around various benchmarks, we set out to understand generalization
performance in acoustic modeling across datasets - in particular, if models
trained on a single dataset transfer to other (possibly out-of-domain)
datasets. We show that, in general, reverberative and additive noise
augmentation improves generalization performance across domains. Further, we
demonstrate that when a large enough set of benchmarks is used, average word
error rate (WER) performance over them provides a good proxy for performance on
real-world noisy data. Finally, we show that training a single acoustic model
on the most widely-used datasets - combined - reaches competitive performance
on both research and real-world benchmarks.

negative:
Recent advances in natural language processing (NLP) have the ability to
transform how classroom learning takes place. Combined with the increasing
integration of technology in today's classrooms, NLP systems leveraging
question answering and dialog processing techniques can serve as private tutors
or participants in classroom discussions to increase student engagement and
learning. To progress towards this goal, we use the classroom discourse
framework of academically productive talk (APT) to learn strategies that make
for the best learning experience. In this paper, we introduce a new task,
called future talk move prediction (FTMP): it consists of predicting the next
talk move -- an utterance strategy from APT -- given a conversation history
with its corresponding talk moves. We further introduce a neural network model
for this task, which outperforms multiple baselines by a large margin. Finally,
we compare our model's performance on FTMP to human performance and show
several similarities between the two.

negative:
In this work, we analyze how human gaze during reading comprehension is
conditioned on the given reading comprehension question, and whether this
signal can be beneficial for machine reading comprehension. To this end, we
collect a new eye-tracking dataset with a large number of participants engaging
in a multiple choice reading comprehension task. Our analysis of this data
reveals increased fixation times over parts of the text that are most relevant
for answering the question. Motivated by this finding, we propose making
automated reading comprehension more human-like by mimicking human
information-seeking reading behavior during reading comprehension. We
demonstrate that this approach leads to performance gains on multiple choice
question answering in English for a state-of-the-art reading comprehension
model.

negative:
Data processing is an important step in various natural language processing
tasks. As the commonly used datasets in named entity recognition contain only a
limited number of samples, it is important to obtain additional labeled data in
an efficient and reliable manner. A common practice is to utilize large
monolingual unlabeled corpora. Another popular technique is to create synthetic
data from the original labeled data (data augmentation). In this work, we
investigate the impact of these two methods on the performance of three
different named entity recognition tasks.

negative:
Additive Manufacturing (AM) is a crucial component of the smart industry. In
this paper, we propose an automated quality grading system for the AM process
using a deep convolutional neural network (CNN) model. The CNN model is trained
offline using the images of the internal and surface defects in the
layer-by-layer deposition of materials and tested online by studying the
performance of detecting and classifying the failure in AM process at different
extruder speeds and temperatures. The model demonstrates the accuracy of 94%
and specificity of 96%, as well as above 75% in three classifier measures of
the Fscore, the sensitivity, and precision for classifying the quality of the
printing process in five grades in real-time. The proposed online model adds an
automated, consistent, and non-contact quality control signal to the AM process
that eliminates the manual inspection of parts after they are entirely built.
The quality monitoring signal can also be used by the machine to suggest
remedial actions by adjusting the parameters in real-time. The proposed quality
predictive model serves as a proof-of-concept for any type of AM machines to
produce reliable parts with fewer quality hiccups while limiting the waste of
both time and materials.

negative:
In this paper, we pose the question: do people talk about women and men in
different ways? We introduce two datasets and a novel integration of approaches
for automatically inferring gender associations from language, discovering
coherent word clusters, and labeling the clusters for the semantic concepts
they represent. The datasets allow us to compare how people write about women
and men in two different settings - one set draws from celebrity news and the
other from student reviews of computer science professors. We demonstrate that
there are large-scale differences in the ways that people talk about women and
men and that these differences vary across domains. Human evaluations show that
our methods significantly outperform strong baselines.

negative:
We address the problem of solving mixed random linear equations. We have
unlabeled observations coming from multiple linear regressions, and each
observation corresponds to exactly one of the regression models. The goal is to
learn the linear regressors from the observations. Classically, Alternating
Minimization (AM) (which is a variant of Expectation Maximization (EM)) is used
to solve this problem. AM iteratively alternates between the estimation of
labels and solving the regression problems with the estimated labels.
Empirically, it is observed that, for a large variety of non-convex problems
including mixed linear regression, AM converges at a much faster rate compared
to gradient based algorithms. However, the existing theory suggests similar
rate of convergence for AM and gradient based methods, failing to capture this
empirical behavior. In this paper, we close this gap between theory and
practice for the special case of a mixture of $2$ linear regressions. We show
that, provided initialized properly, AM enjoys a \emph{super-linear} rate of
convergence in certain parameter regimes. To the best of our knowledge, this is
the first work that theoretically establishes such rate for AM. Hence, if we
want to recover the unknown regressors upto an error (in $\ell_2$ norm) of
$\epsilon$, AM only takes $\mathcal{O}(\log \log (1/\epsilon))$ iterations.
Furthermore, we compare AM with a gradient based heuristic algorithm
empirically and show that AM dominates in iteration complexity as well as
wall-clock time.