query: My goal is to develop a learning model that can handle multiple tasks simultaneously. This proposed learning model will function as a sub-model selector, meaning that when presented with a new task, it will determine the most suitable sub-model to learn this task. Additionally, my learning model could also operate as a sub-model constructor. If it determines that no existing sub-model can learn the task, it will generate innovative model architectures to construct a new sub-model suitable for the task. At present, I am considering the application of a combination of reinforcement learning and model architecture search algorithms to achieve this. To effectively evaluate the performance, I require a comprehensive benchmark that includes a variety of datasets that can be used as different sub-tasks. My aim is for my learning model to achieve state-of-the-art performance on this benchmark. In general, I am open to any methodologies that enhance the performance and adaptability of my learning model across different problem domains.

positive:
Multi-task learning, as it is understood nowadays, consists of using one
single model to carry out several similar tasks. From classifying hand-written
characters of different alphabets to figuring out how to play several Atari
games using reinforcement learning, multi-task models have been able to widen
their performance range across different tasks, although these tasks are
usually of a similar nature. In this work, we attempt to widen this range even
further, by including heterogeneous tasks in a single learning procedure. To do
so, we firstly formally define a multi-network model, identifying the necessary
components and characteristics to allow different adaptations of said model
depending on the tasks it is required to fulfill. Secondly, employing the
formal definition as a starting point, we develop an illustrative model example
consisting of three different tasks (classification, regression and data
sampling). The performance of this model implementation is then analyzed,
showing its capabilities. Motivated by the results of the analysis, we
enumerate a set of open challenges and future research lines over which the
full potential of the proposed model definition can be exploited.

positive:
The usefulness and value of Multi-step Machine Learning (ML), where a task is
organized into connected sub-tasks with known intermediate inference goals, as
opposed to a single large model learned end-to-end without intermediate
sub-tasks, is presented. Pre-optimized ML models are connected and better
performance is obtained by re-optimizing the connected one. The selection of an
ML model from several small ML model candidates for each sub-task has been
performed by using the idea based on Neural Architecture Search (NAS). In this
paper, Differentiable Architecture Search (DARTS) and Single Path One-Shot NAS
(SPOS-NAS) are tested, where the construction of loss functions is improved to
keep all ML models smoothly learning. Using DARTS and SPOS-NAS as an
optimization and selection as well as the connections for multi-step machine
learning systems, we find that (1) such a system can quickly and successfully
select highly performant model combinations, and (2) the selected models are
consistent with baseline algorithms, such as grid search, and their outputs are
well controlled.

positive:
For most deep learning practitioners, sequence modeling is synonymous with
recurrent networks. Yet recent results indicate that convolutional
architectures can outperform recurrent networks on tasks such as audio
synthesis and machine translation. Given a new sequence modeling task or
dataset, which architecture should one use? We conduct a systematic evaluation
of generic convolutional and recurrent architectures for sequence modeling. The
models are evaluated across a broad range of standard tasks that are commonly
used to benchmark recurrent networks. Our results indicate that a simple
convolutional architecture outperforms canonical recurrent networks such as
LSTMs across a diverse range of tasks and datasets, while demonstrating longer
effective memory. We conclude that the common association between sequence
modeling and recurrent networks should be reconsidered, and convolutional
networks should be regarded as a natural starting point for sequence modeling
tasks. To assist related work, we have made code available at
http://github.com/locuslab/TCN .

positive:
Deep learning models require extensive architecture design exploration and
hyperparameter optimization to perform well on a given task. The exploration of
the model design space is often made by a human expert, and optimized using a
combination of grid search and search heuristics over a large space of possible
choices. Neural Architecture Search (NAS) is a Reinforcement Learning approach
that has been proposed to automate architecture design. NAS has been
successfully applied to generate Neural Networks that rival the best
human-designed architectures. However, NAS requires sampling, constructing, and
training hundreds to thousands of models to achieve well-performing
architectures. This procedure needs to be executed from scratch for each new
task. The application of NAS to a wide set of tasks currently lacks a way to
transfer generalizable knowledge across tasks. In this paper, we present the
Multitask Neural Model Search (MNMS) controller. Our goal is to learn a
generalizable framework that can condition model construction on successful
model searches for previously seen tasks, thus significantly speeding up the
search for new tasks. We demonstrate that MNMS can conduct an automated
architecture search for multiple tasks simultaneously while still learning
well-performing, specialized models for each task. We then show that
pre-trained MNMS controllers can transfer learning to new tasks. By leveraging
knowledge from previous searches, we find that pre-trained MNMS models start
from a better location in the search space and reduce search time on unseen
tasks, while still discovering models that outperform published human-designed
models.

positive:
Approaches to continual learning aim to successfully learn a set of related
tasks that arrive in an online manner. Recently, several frameworks have been
developed which enable deep learning to be deployed in this learning scenario.
A key modelling decision is to what extent the architecture should be shared
across tasks. On the one hand, separately modelling each task avoids
catastrophic forgetting but it does not support transfer learning and leads to
large models. On the other hand, rigidly specifying a shared component and a
task-specific part enables task transfer and limits the model size, but it is
vulnerable to catastrophic forgetting and restricts the form of task-transfer
that can occur. Ideally, the network should adaptively identify which parts of
the network to share in a data driven way. Here we introduce such an approach
called Continual Learning with Adaptive Weights (CLAW), which is based on
probabilistic modelling and variational inference. Experiments show that CLAW
achieves state-of-the-art performance on six benchmarks in terms of overall
continual learning performance, as measured by classification accuracy, and in
terms of addressing catastrophic forgetting.

positive:
As deep learning applications continue to become more diverse, an interesting
question arises: Can general problem solving arise from jointly learning
several such diverse tasks? To approach this question, deep multi-task learning
is extended in this paper to the setting where there is no obvious overlap
between task architectures. The idea is that any set of (architecture,task)
pairs can be decomposed into a set of potentially related subproblems, whose
sharing is optimized by an efficient stochastic algorithm. The approach is
first validated in a classic synthetic multi-task learning benchmark, and then
applied to sharing across disparate architectures for vision, NLP, and genomics
tasks. It discovers regularities across these domains, encodes them into
sharable modules, and combines these modules systematically to improve
performance in the individual tasks. The results confirm that sharing learned
functionality across diverse domains and architectures is indeed beneficial,
thus establishing a key ingredient for general problem solving in the future.

positive:
Lifelong learning remains an open problem. One of its main difficulties is
catastrophic forgetting. Many dynamic expansion approaches have been proposed
to address this problem, but they all use homogeneous models of predefined
structure for all tasks. The common original model and expansion structures
ignore the requirement of different model structures on different tasks, which
leads to a less compact model for multiple tasks and causes the model size to
increase rapidly as the number of tasks increases. Moreover, they can not
perform best on all tasks. To solve those problems, in this paper, we propose a
new lifelong learning framework named Searchable Extension Units (SEU) by
introducing Neural Architecture Search into lifelong learning, which breaks
down the need for a predefined original model and searches for specific
extension units for different tasks, without compromising the performance of
the model on different tasks. Our approach can obtain a much more compact model
without catastrophic forgetting. The experimental results on the PMNIST, the
split CIFAR10 dataset, the split CIFAR100 dataset, and the Mixture dataset
empirically prove that our method can achieve higher accuracy with much smaller
model, whose size is about 25-33 percentage of that of the state-of-the-art
methods.

positive:
Can deep learning solve multiple tasks simultaneously, even when they are
unrelated and very different? We investigate how the representations of the
underlying tasks affect the ability of a single neural network to learn them
jointly. We present theoretical and empirical findings that a single neural
network is capable of simultaneously learning multiple tasks from a combined
data set, for a variety of methods for representing tasks -- for example, when
the distinct tasks are encoded by well-separated clusters or decision trees
over certain task-code attributes. More concretely, we present a novel analysis
that shows that families of simple programming-like constructs for the codes
encoding the tasks are learnable by two-layer neural networks with standard
training. We study more generally how the complexity of learning such combined
tasks grows with the complexity of the task codes; we find that combining many
tasks may incur a sample complexity penalty, even though the individual tasks
are easy to learn. We provide empirical support for the usefulness of the
learning bounds by training networks on clusters, decision trees, and SQL-style
aggregation.

positive:
Continual learning with neural networks is an important learning framework in
AI that aims to learn a sequence of tasks well. However, it is often confronted
with three challenges: (1) overcome the catastrophic forgetting problem, (2)
adapt the current network to new tasks, and meanwhile (3) control its model
complexity. To reach these goals, we propose a novel approach named as
Continual Learning with Efficient Architecture Search, or CLEAS in short. CLEAS
works closely with neural architecture search (NAS) which leverages
reinforcement learning techniques to search for the best neural architecture
that fits a new task. In particular, we design a neuron-level NAS controller
that decides which old neurons from previous tasks should be reused (knowledge
transfer), and which new neurons should be added (to learn new knowledge). Such
a fine-grained controller allows one to find a very concise architecture that
can fit each new task well. Meanwhile, since we do not alter the weights of the
reused neurons, we perfectly memorize the knowledge learned from previous
tasks. We evaluate CLEAS on numerous sequential classification tasks, and the
results demonstrate that CLEAS outperforms other state-of-the-art alternative
methods, achieving higher classification accuracy while using simpler neural
architectures.

positive:
Many advanced Learning from Demonstration (LfD) methods consider the
decomposition of complex, real-world tasks into simpler sub-tasks. By reusing
the corresponding sub-policies within and between tasks, they provide training
data for each policy from different high-level tasks and compose them to
perform novel ones. Existing approaches to modular LfD focus either on learning
a single high-level task or depend on domain knowledge and temporal
segmentation. In contrast, we propose a weakly supervised, domain-agnostic
approach based on task sketches, which include only the sequence of sub-tasks
performed in each demonstration. Our approach simultaneously aligns the
sketches with the observed demonstrations and learns the required sub-policies.
This improves generalisation in comparison to separate optimisation procedures.
We evaluate the approach on multiple domains, including a simulated 3D robot
arm control task using purely image-based observations. The results show that
our approach performs commensurately with fully supervised approaches, while
requiring significantly less annotation effort.

positive:
Domain similarity measures can be used to gauge adaptability and select
suitable data for transfer learning, but existing approaches define ad hoc
measures that are deemed suitable for respective tasks. Inspired by work on
curriculum learning, we propose to \emph{learn} data selection measures using
Bayesian Optimization and evaluate them across models, domains and tasks. Our
learned measures outperform existing domain similarity measures significantly
on three tasks: sentiment analysis, part-of-speech tagging, and parsing. We
show the importance of complementing similarity with diversity, and that
learned measures are -- to some degree -- transferable across models, domains,
and even tasks.

positive:
Model-based reinforcement learning (MBRL) methods have shown strong sample
efficiency and performance across a variety of tasks, including when faced with
high-dimensional visual observations. These methods learn to predict the
environment dynamics and expected reward from interaction and use this
predictive model to plan and perform the task. However, MBRL methods vary in
their fundamental design choices, and there is no strong consensus in the
literature on how these design decisions affect performance. In this paper, we
study a number of design decisions for the predictive model in visual MBRL
algorithms, focusing specifically on methods that use a predictive model for
planning. We find that a range of design decisions that are often considered
crucial, such as the use of latent spaces, have little effect on task
performance. A big exception to this finding is that predicting future
observations (i.e., images) leads to significant task performance improvement
compared to only predicting rewards. We also empirically find that image
prediction accuracy, somewhat surprisingly, correlates more strongly with
downstream task performance than reward prediction accuracy. We show how this
phenomenon is related to exploration and how some of the lower-scoring models
on standard benchmarks (that require exploration) will perform the same as the
best-performing models when trained on the same training data. Simultaneously,
in the absence of exploration, models that fit the data better usually perform
better on the downstream task as well, but surprisingly, these are often not
the same models that perform the best when learning and exploring from scratch.
These findings suggest that performance and exploration place important and
potentially contradictory requirements on the model.

positive:
We propose an algorithm for meta-learning that is model-agnostic, in the
sense that it is compatible with any model trained with gradient descent and
applicable to a variety of different learning problems, including
classification, regression, and reinforcement learning. The goal of
meta-learning is to train a model on a variety of learning tasks, such that it
can solve new learning tasks using only a small number of training samples. In
our approach, the parameters of the model are explicitly trained such that a
small number of gradient steps with a small amount of training data from a new
task will produce good generalization performance on that task. In effect, our
method trains the model to be easy to fine-tune. We demonstrate that this
approach leads to state-of-the-art performance on two few-shot image
classification benchmarks, produces good results on few-shot regression, and
accelerates fine-tuning for policy gradient reinforcement learning with neural
network policies.

positive:
Multi-task learning in Convolutional Networks has displayed remarkable
success in the field of recognition. This success can be largely attributed to
learning shared representations from multiple supervisory tasks. However,
existing multi-task approaches rely on enumerating multiple network
architectures specific to the tasks at hand, that do not generalize. In this
paper, we propose a principled approach to learn shared representations in
ConvNets using multi-task learning. Specifically, we propose a new sharing
unit: "cross-stitch" unit. These units combine the activations from multiple
networks and can be trained end-to-end. A network with cross-stitch units can
learn an optimal combination of shared and task-specific representations. Our
proposed method generalizes across multiple tasks and shows dramatically
improved performance over baseline methods for categories with few training
examples.

positive:
A core aspect of human intelligence is the ability to learn new tasks quickly
and switch between them flexibly. Here, we describe a modular continual
reinforcement learning paradigm inspired by these abilities. We first introduce
a visual interaction environment that allows many types of tasks to be unified
in a single framework. We then describe a reward map prediction scheme that
learns new tasks robustly in the very large state and action spaces required by
such an environment. We investigate how properties of module architecture
influence efficiency of task learning, showing that a module motif
incorporating specific design principles (e.g. early bottlenecks, low-order
polynomial nonlinearities, and symmetry) significantly outperforms more
standard neural network motifs, needing fewer training examples and fewer
neurons to achieve high levels of performance. Finally, we present a
meta-controller architecture for task switching based on a dynamic neural
voting scheme, which allows new modules to use information learned from
previously-seen tasks to substantially improve their own learning efficiency.

positive:
The growing interest in both the automation of machine learning and deep
learning has inevitably led to the development of a wide variety of automated
methods for neural architecture search. The choice of the network architecture
has proven to be critical, and many advances in deep learning spring from its
immediate improvements. However, deep learning techniques are computationally
intensive and their application requires a high level of domain knowledge.
Therefore, even partial automation of this process helps to make deep learning
more accessible to both researchers and practitioners. With this survey, we
provide a formalism which unifies and categorizes the landscape of existing
methods along with a detailed analysis that compares and contrasts the
different approaches. We achieve this via a comprehensive discussion of the
commonly adopted architecture search spaces and architecture optimization
algorithms based on principles of reinforcement learning and evolutionary
algorithms along with approaches that incorporate surrogate and one-shot
models. Additionally, we address the new research directions which include
constrained and multi-objective architecture search as well as automated data
augmentation, optimizer and activation function search.

positive:
The reinforcement learning community has made great strides in designing
algorithms capable of exceeding human performance on specific tasks. These
algorithms are mostly trained one task at the time, each new task requiring to
train a brand new agent instance. This means the learning algorithm is general,
but each solution is not; each agent can only solve the one task it was trained
on. In this work, we study the problem of learning to master not one but
multiple sequential-decision tasks at once. A general issue in multi-task
learning is that a balance must be found between the needs of multiple tasks
competing for the limited resources of a single learning system. Many learning
algorithms can get distracted by certain tasks in the set of tasks to solve.
Such tasks appear more salient to the learning process, for instance because of
the density or magnitude of the in-task rewards. This causes the algorithm to
focus on those salient tasks at the expense of generality. We propose to
automatically adapt the contribution of each task to the agent's updates, so
that all tasks have a similar impact on the learning dynamics. This resulted in
state of the art performance on learning to play all games in a set of 57
diverse Atari games. Excitingly, our method learned a single trained policy -
with a single set of weights - that exceeds median human performance. To our
knowledge, this was the first time a single agent surpassed human-level
performance on this multi-task domain. The same approach also demonstrated
state of the art performance on a set of 30 tasks in the 3D reinforcement
learning platform DeepMind Lab.

positive:
Deep reinforcement learning (DRL) has recently shown its success in tackling
complex combinatorial optimization problems. When these problems are extended
to multiobjective ones, it becomes difficult for the existing DRL approaches to
flexibly and efficiently deal with multiple subproblems determined by weight
decomposition of objectives. This paper proposes a concise meta-learning-based
DRL approach. It first trains a meta-model by meta-learning. The meta-model is
fine-tuned with a few update steps to derive submodels for the corresponding
subproblems. The Pareto front is then built accordingly. Compared with other
learning-based methods, our method can greatly shorten the training time of
multiple submodels. Due to the rapid and excellent adaptability of the
meta-model, more submodels can be derived so as to increase the quality and
diversity of the found solutions. The computational experiments on
multiobjective traveling salesman problems and multiobjective vehicle routing
problem with time windows demonstrate the superiority of our method over most
of learning-based and iteration-based approaches.

positive:
The application of reinforcement learning algorithms onto real life problems
always bears the challenge of filtering the environmental state out of raw
sensor readings. While most approaches use heuristics, biology suggests that
there must exist an unsupervised method to construct such filters
automatically. Besides the extraction of environmental states, the filters have
to represent them in a fashion that support modern reinforcement algorithms.
Many popular algorithms use a linear architecture, so one should aim at filters
that have good approximation properties in combination with linear functions.
This thesis wants to propose the unsupervised method slow feature analysis
(SFA) for this task. Presented with a random sequence of sensor readings, SFA
learns a set of filters. With growing model complexity and training examples,
the filters converge against trigonometric polynomial functions. These are
known to possess excellent approximation capabilities and should therfore
support the reinforcement algorithms well. We evaluate this claim on a robot.
The task is to learn a navigational control in a simple environment using the
least square policy iteration (LSPI) algorithm. The only accessible sensor is a
head mounted video camera, but without meaningful filtering, video images are
not suited as LSPI input. We will show that filters learned by SFA, based on a
random walk video of the robot, allow the learned control to navigate
successfully in ca. 80% of the test trials.

positive:
Traditional model-based reinforcement learning approaches learn a model of
the environment dynamics without explicitly considering how it will be used by
the agent. In the presence of misspecified model classes, this can lead to poor
estimates, as some relevant available information is ignored. In this paper, we
introduce a novel model-based policy search approach that exploits the
knowledge of the current agent policy to learn an approximate transition model,
focusing on the portions of the environment that are most relevant for policy
improvement. We leverage a weighting scheme, derived from the minimization of
the error on the model-based policy gradient estimator, in order to define a
suitable objective function that is optimized for learning the approximate
transition model. Then, we integrate this procedure into a batch policy
improvement algorithm, named Gradient-Aware Model-based Policy Search (GAMPS),
which iteratively learns a transition model and uses it, together with the
collected trajectories, to compute the new policy parameters. Finally, we
empirically validate GAMPS on benchmark domains analyzing and discussing its
properties.

positive:
Meta-learning approaches have shown great success in vision and language
domains. However, few studies discuss the practice of meta-learning for
large-scale industrial applications. Although e-commerce companies have spent
many efforts on learning representations to provide a better user experience,
we argue that such efforts cannot be stopped at this step. In addition to
learning a strong profile, the challenging question about how to effectively
transfer the learned representation is raised simultaneously. This paper
introduces the contributions that we made to address these challenges from
three aspects. 1) Meta-learning model: In the context of representation
learning with e-commerce user behavior data, we propose a meta-learning
framework called the Meta-Profile Network, which extends the ideas of matching
network and relation network for knowledge transfer and fast adaptation; 2)
Encoding strategy: To keep high fidelity of large-scale long-term sequential
behavior data, we propose a time-heatmap encoding strategy that allows the
model to encode data effectively; 3) Deep network architecture: A multi-modal
model combined with multi-task learning architecture is utilized to address the
cross-domain knowledge learning and insufficient label problems. Moreover, we
argue that an industrial model should not only have good performance in terms
of accuracy, but also have better robustness and uncertainty performance under
extreme conditions. We evaluate the performance of our model with extensive
control experiments in various extreme scenarios, i.e. out-of-distribution
detection, data insufficiency and class imbalance scenarios. The Meta-Profile
Network shows significant improvement in the model performance when compared to
baseline models.

positive:
We consider problems of making sequences of decisions to accomplish tasks,
interacting via the medium of language. These problems are often tackled with
reinforcement learning approaches. We find that these models do not generalize
well when applied to novel task domains. However, the large amount of
computation necessary to adequately train and explore the search space of
sequential decision making, under a reinforcement learning paradigm, precludes
the inclusion of large contextualized language models, which might otherwise
enable the desired generalization ability. We introduce a teacher-student
imitation learning methodology and a means of converting a reinforcement
learning model into a natural language understanding model. Together, these
methodologies enable the introduction of contextualized language models into
the sequential decision making problem space. We show that models can learn
faster and generalize more, leveraging both the imitation learning and the
reformulation. Our models exceed teacher performance on various held-out
decision problems, by up to 7% on in-domain problems and 24% on out-of-domain
problems.

positive:
In order to efficiently learn with small amount of data on new tasks,
meta-learning transfers knowledge learned from previous tasks to the new ones.
However, a critical challenge in meta-learning is the task heterogeneity which
cannot be well handled by traditional globally shared meta-learning methods. In
addition, current task-specific meta-learning methods may either suffer from
hand-crafted structure design or lack the capability to capture complex
relations between tasks. In this paper, motivated by the way of knowledge
organization in knowledge bases, we propose an automated relational
meta-learning (ARML) framework that automatically extracts the cross-task
relations and constructs the meta-knowledge graph. When a new task arrives, it
can quickly find the most relevant structure and tailor the learned structure
knowledge to the meta-learner. As a result, the proposed framework not only
addresses the challenge of task heterogeneity by a learned meta-knowledge
graph, but also increases the model interpretability. We conduct extensive
experiments on 2D toy regression and few-shot image classification and the
results demonstrate the superiority of ARML over state-of-the-art baselines.

positive:
Through many recent successes in simulation, model-free reinforcement
learning has emerged as a promising approach to solving continuous control
robotic tasks. The research community is now able to reproduce, analyze and
build quickly on these results due to open source implementations of learning
algorithms and simulated benchmark tasks. To carry forward these successes to
real-world applications, it is crucial to withhold utilizing the unique
advantages of simulations that do not transfer to the real world and experiment
directly with physical robots. However, reinforcement learning research with
physical robots faces substantial resistance due to the lack of benchmark tasks
and supporting source code. In this work, we introduce several reinforcement
learning tasks with multiple commercially available robots that present varying
levels of learning difficulty, setup, and repeatability. On these tasks, we
test the learning performance of off-the-shelf implementations of four
reinforcement learning algorithms and analyze sensitivity to their
hyper-parameters to determine their readiness for applications in various
real-world tasks. Our results show that with a careful setup of the task
interface and computations, some of these implementations can be readily
applicable to physical robots. We find that state-of-the-art learning
algorithms are highly sensitive to their hyper-parameters and their relative
ordering does not transfer across tasks, indicating the necessity of re-tuning
them for each task for best performance. On the other hand, the best
hyper-parameter configuration from one task may often result in effective
learning on held-out tasks even with different robots, providing a reasonable
default. We make the benchmark tasks publicly available to enhance
reproducibility in real-world reinforcement learning.

positive:
With the explosive growth of online products and content, recommendation
techniques have been considered as an effective tool to overcome information
overload, improve user experience, and boost business revenue. In recent years,
we have observed a new desideratum of considering long-term rewards of multiple
related recommendation tasks simultaneously. The consideration of long-term
rewards is strongly tied to business revenue and growth. Learning multiple
tasks simultaneously could generally improve the performance of individual task
due to knowledge sharing in multi-task learning. While a few existing works
have studied long-term rewards in recommendations, they mainly focus on a
single recommendation task. In this paper, we propose {\it PoDiRe}: a
\underline{po}licy \underline{di}stilled \underline{re}commender that can
address long-term rewards of recommendations and simultaneously handle multiple
recommendation tasks. This novel recommendation solution is based on a marriage
of deep reinforcement learning and knowledge distillation techniques, which is
able to establish knowledge sharing among different tasks and reduce the size
of a learning model. The resulting model is expected to attain better
performance and lower response latency for real-time recommendation services.
In collaboration with Samsung Game Launcher, one of the world's largest
commercial mobile game platforms, we conduct a comprehensive experimental study
on large-scale real data with hundreds of millions of events and show that our
solution outperforms many state-of-the-art methods in terms of several standard
evaluation metrics.

positive:
Reinforcement learning can train policies that effectively perform complex
tasks. However for long-horizon tasks, the performance of these methods
degrades with horizon, often necessitating reasoning over and chaining
lower-level skills. Hierarchical reinforcement learning aims to enable this by
providing a bank of low-level skills as action abstractions. Hierarchies can
further improve on this by abstracting the space states as well. We posit that
a suitable state abstraction should depend on the capabilities of the available
lower-level policies. We propose Value Function Spaces: a simple approach that
produces such a representation by using the value functions corresponding to
each lower-level skill. These value functions capture the affordances of the
scene, thus forming a representation that compactly abstracts task relevant
information and robustly ignores distractors. Empirical evaluations for
maze-solving and robotic manipulation tasks demonstrate that our approach
improves long-horizon performance and enables better zero-shot generalization
than alternative model-free and model-based methods.

positive:
This study investigates the use of reinforcement learning to guide a general
purpose cache manager decisions. Cache managers directly impact the overall
performance of computer systems. They govern decisions about which objects
should be cached, the duration they should be cached for, and decides on which
objects to evict from the cache if it is full. These three decisions impact
both the cache hit rate and size of the storage that is needed to achieve that
cache hit rate. An optimal cache manager will avoid unnecessary operations,
maximise the cache hit rate which results in fewer round trips to a slower
backend storage system, and minimise the size of storage needed to achieve a
high hit-rate.
  This project investigates using reinforcement learning in cache management by
designing three separate agents for each of the cache manager tasks.
Furthermore, the project investigates two advanced reinforcement learning
architectures for multi-decision problems: a single multi-task agent and a
multi-agent. We also introduce a framework to simplify the modelling of
computer systems problems as a reinforcement learning task. The framework
abstracts delayed experiences observations and reward assignment in computer
systems while providing a flexible way to scale to multiple agents.
  Simulation results based on an established database benchmark system show
that reinforcement learning agents can achieve a higher cache hit rate over
heuristic driven algorithms while minimising the needed space. They are also
able to adapt to a changing workload and dynamically adjust their caching
strategy accordingly. The proposed cache manager model is generic and
applicable to other types of caches, such as file system caches. This project
is the first, to our knowledge, to model cache manager decisions as a
multi-task control problem.

positive:
We present here a model to take advantage of the multi-task nature of complex
datasets by learning to separate tasks and subtasks in and end to end manner by
biasing competitive interactions in the network. This method does not require
additional labelling or reformatting of data in a dataset. We propose an
alternate view to the monolithic one-task-fits-all learning of multi-task
problems, and describe a model based on a theory of neuronal attention from
neuroscience, proposed by Desimone. We create and exhibit a new toy dataset,
based on the MNIST dataset, which we call MNIST-QA, for testing Visual Question
Answering architectures in a low-dimensional environment while preserving the
more difficult components of the Visual Question Answering task, and
demonstrate the proposed network architecture on this new dataset, as well as
on COCO-QA and DAQUAR-FULL. We then demonstrate that this model eliminates
catastrophic interference between tasks on a newly created toy dataset and
provides competitive results in the Visual Question Answering space. We provide
further evidence that Visual Question Answering can be approached as a
multi-task problem, and demonstrate that this new architecture based on the
Biased Competition model is capable of learning to separate and learn the tasks
in an end-to-end fashion without the need for task labels.

positive:
We describe a framework for multitask deep reinforcement learning guided by
policy sketches. Sketches annotate tasks with sequences of named subtasks,
providing information about high-level structural relationships among tasks but
not how to implement them---specifically not providing the detailed guidance
used by much previous work on learning policy abstractions for RL (e.g.
intermediate rewards, subtask completion signals, or intrinsic motivations). To
learn from sketches, we present a model that associates every subtask with a
modular subpolicy, and jointly maximizes reward over full task-specific
policies by tying parameters across shared subpolicies. Optimization is
accomplished via a decoupled actor--critic training objective that facilitates
learning common behaviors from multiple dissimilar reward functions. We
evaluate the effectiveness of our approach in three environments featuring both
discrete and continuous control, and with sparse rewards that can be obtained
only after completing a number of high-level subgoals. Experiments show that
using our approach to learn policies guided by sketches gives better
performance than existing techniques for learning task-specific or shared
policies, while naturally inducing a library of interpretable primitive
behaviors that can be recombined to rapidly adapt to new tasks.

positive:
We are interested in learning models of non-stationary environments, which
can be framed as a multi-task learning problem. Model-free reinforcement
learning algorithms can achieve good asymptotic performance in multi-task
learning at a cost of extensive sampling, due to their approach, which requires
learning from scratch. While model-based approaches are among the most data
efficient learning algorithms, they still struggle with complex tasks and model
uncertainties. Meta-reinforcement learning addresses the efficiency and
generalization challenges on multi task learning by quickly leveraging the
meta-prior policy for a new task. In this paper, we propose a
meta-reinforcement learning approach to learn the dynamic model of a
non-stationary environment to be used for meta-policy optimization later. Due
to the sample efficiency of model-based learning methods, we are able to
simultaneously train both the meta-model of the non-stationary environment and
the meta-policy until dynamic model convergence. Then, the meta-learned dynamic
model of the environment will generate simulated data for meta-policy
optimization. Our experiment demonstrates that our proposed method can
meta-learn the policy in a non-stationary environment with the data efficiency
of model-based learning approaches while achieving the high asymptotic
performance of model-free meta-reinforcement learning.

positive:
Meta-reinforcement learning (meta-RL) aims to learn from multiple training
tasks the ability to adapt efficiently to unseen test tasks. Despite the
success, existing meta-RL algorithms are known to be sensitive to the task
distribution shift. When the test task distribution is different from the
training task distribution, the performance may degrade significantly. To
address this issue, this paper proposes Model-based Adversarial
Meta-Reinforcement Learning (AdMRL), where we aim to minimize the worst-case
sub-optimality gap -- the difference between the optimal return and the return
that the algorithm achieves after adaptation -- across all tasks in a family of
tasks, with a model-based approach. We propose a minimax objective and optimize
it by alternating between learning the dynamics model on a fixed task and
finding the adversarial task for the current model -- the task for which the
policy induced by the model is maximally suboptimal. Assuming the family of
tasks is parameterized, we derive a formula for the gradient of the
suboptimality with respect to the task parameters via the implicit function
theorem, and show how the gradient estimator can be efficiently implemented by
the conjugate gradient method and a novel use of the REINFORCE estimator. We
evaluate our approach on several continuous control benchmarks and demonstrate
its efficacy in the worst-case performance over all tasks, the generalization
power to out-of-distribution tasks, and in training and test time sample
efficiency, over existing state-of-the-art meta-RL algorithms.

positive:
Federated learning is promising for its capability to collaboratively train
models with multiple clients without accessing their data, but vulnerable when
clients' data distributions diverge from each other. This divergence further
leads to a dilemma: "Should we prioritize the learned model's generic
performance (for future use at the server) or its personalized performance (for
each client)?" These two, seemingly competing goals have divided the community
to focus on one or the other, yet in this paper we show that it is possible to
approach both at the same time. Concretely, we propose a novel federated
learning framework that explicitly decouples a model's dual duties with two
prediction tasks. On the one hand, we introduce a family of losses that are
robust to non-identical class distributions, enabling clients to train a
generic predictor with a consistent objective across them. On the other hand,
we formulate the personalized predictor as a lightweight adaptive module that
is learned to minimize each client's empirical risk on top of the generic
predictor. With this two-loss, two-predictor framework which we name Federated
Robust Decoupling (Fed-RoD), the learned model can simultaneously achieve
state-of-the-art generic and personalized performance, essentially bridging the
two tasks.

positive:
The aim of multi-task reinforcement learning is two-fold: (1) efficiently
learn by training against multiple tasks and (2) quickly adapt, using limited
samples, to a variety of new tasks. In this work, the tasks correspond to
reward functions for environments with the same (or similar) dynamical models.
We propose to learn a dynamical model during the training process and use this
model to perform sample-efficient adaptation to new tasks at test time. We use
significantly fewer samples by performing policy optimization only in a
"virtual" environment whose transitions are given by our learned dynamical
model. Our algorithm sequentially trains against several tasks. Upon
encountering a new task, we first warm-up a policy on our learned dynamical
model, which requires no new samples from the environment. We then adapt the
dynamical model with samples from this policy in the real environment. We
evaluate our approach on several continuous control benchmarks and demonstrate
its efficacy over MAML, a state-of-the-art meta-learning algorithm, on these
tasks.

positive:
We propose a framework for verifiable and compositional reinforcement
learning (RL) in which a collection of RL subsystems, each of which learns to
accomplish a separate subtask, are composed to achieve an overall task. The
framework consists of a high-level model, represented as a parametric Markov
decision process (pMDP) which is used to plan and to analyze compositions of
subsystems, and of the collection of low-level subsystems themselves. By
defining interfaces between the subsystems, the framework enables automatic
decompositions of task specifications, e.g., reach a target set of states with
a probability of at least 0.95, into individual subtask specifications, i.e.
achieve the subsystem's exit conditions with at least some minimum probability,
given that its entry conditions are met. This in turn allows for the
independent training and testing of the subsystems; if they each learn a policy
satisfying the appropriate subtask specification, then their composition is
guaranteed to satisfy the overall task specification. Conversely, if the
subtask specifications cannot all be satisfied by the learned policies, we
present a method, formulated as the problem of finding an optimal set of
parameters in the pMDP, to automatically update the subtask specifications to
account for the observed shortcomings. The result is an iterative procedure for
defining subtask specifications, and for training the subsystems to meet them.
As an additional benefit, this procedure allows for particularly challenging or
important components of an overall task to be determined automatically, and
focused on, during training. Experimental results demonstrate the presented
framework's novel capabilities.

positive:
Temporal information is essential to learning effective policies with
Reinforcement Learning (RL). However, current state-of-the-art RL algorithms
either assume that such information is given as part of the state space or,
when learning from pixels, use the simple heuristic of frame-stacking to
implicitly capture temporal information present in the image observations. This
heuristic is in contrast to the current paradigm in video classification
architectures, which utilize explicit encodings of temporal information through
methods such as optical flow and two-stream architectures to achieve
state-of-the-art performance. Inspired by leading video classification
architectures, we introduce the Flow of Latents for Reinforcement Learning
(Flare), a network architecture for RL that explicitly encodes temporal
information through latent vector differences. We show that Flare (i) recovers
optimal performance in state-based RL without explicit access to the state
velocity, solely with positional state information, (ii) achieves
state-of-the-art performance on pixel-based challenging continuous control
tasks within the DeepMind control benchmark suite, namely quadruped walk,
hopper hop, finger turn hard, pendulum swing, and walker run, and is the most
sample efficient model-free pixel-based RL algorithm, outperforming the prior
model-free state-of-the-art by 1.9X and 1.5X on the 500k and 1M step
benchmarks, respectively, and (iv), when augmented over rainbow DQN,
outperforms this state-of-the-art level baseline on 5 of 8 challenging Atari
games at 100M time step benchmark.

positive:
In this work, we consider the problem of instance-wise dynamic network model
selection for multi-task learning. To this end, we propose an efficient
approach to exploit a compact but accurate model in a backbone architecture for
each instance of all tasks. The proposed method consists of an estimator and a
selector. The estimator is based on a backbone architecture and structured
hierarchically. It can produce multiple different network models of different
configurations in a hierarchical structure. The selector chooses a model
dynamically from a pool of candidate models given an input instance. The
selector is a relatively small-size network consisting of a few layers, which
estimates a probability distribution over the candidate models when an input
instance of a task is given. Both estimator and selector are jointly trained in
a unified learning framework in conjunction with a sampling-based learning
strategy, without additional computation steps. We demonstrate the proposed
approach for several image classification tasks compared to existing approaches
performing model selection or learning multiple tasks. Experimental results
show that our approach gives not only outstanding performance compared to other
competitors but also the versatility to perform instance-wise model selection
for multiple tasks.

positive:
This paper considers the problem of learning a model in model-based
reinforcement learning (MBRL). We examine how the planning module of an MBRL
algorithm uses the model, and propose that the model learning module should
incorporate the way the planner is going to use the model. This is in contrast
to conventional model learning approaches, such as those based on maximum
likelihood estimate, that learn a predictive model of the environment without
explicitly considering the interaction of the model and the planner. We focus
on policy gradient type of planning algorithms and derive new loss functions
for model learning that incorporate how the planner uses the model. We call
this approach Policy-Aware Model Learning (PAML). We theoretically analyze a
generic model-based policy gradient algorithm and provide a convergence
guarantee for the optimized policy. We also empirically evaluate PAML on some
benchmark problems, showing promising results.

positive:
The use of imitation learning to learn a single policy for a complex task
that has multiple modes or hierarchical structure can be challenging. In fact,
previous work has shown that when the modes are known, learning separate
policies for each mode or sub-task can greatly improve the performance of
imitation learning. In this work, we discover the interaction between sub-tasks
from their resulting state-action trajectory sequences using a directed
graphical model. We propose a new algorithm based on the generative adversarial
imitation learning framework which automatically learns sub-task policies from
unsegmented demonstrations. Our approach maximizes the directed information
flow in the graphical model between sub-task latent variables and their
generated trajectories. We also show how our approach connects with the
existing Options framework, which is commonly used to learn hierarchical
policies.

positive:
Hierarchical reinforcement learning deals with the problem of breaking down
large tasks into meaningful sub-tasks. Autonomous discovery of these sub-tasks
has remained a challenging problem. We propose a novel method of learning
sub-tasks by combining paradigms of routing in computer networks and graph
based skill discovery within the options framework to define meaningful
sub-goals. We apply the recent advancements of learning embeddings using
Riemannian optimisation in the hyperbolic space to embed the state set into the
hyperbolic space and create a model of the environment. In doing so we enforce
a global topology on the states and are able to exploit this topology to learn
meaningful sub-tasks. We demonstrate empirically, both in discrete and
continuous domains, how these embeddings can improve the learning of meaningful
sub-tasks.

positive:
Graph Neural Networks (GNNs) are a framework for graph representation
learning, where a model learns to generate low dimensional node embeddings that
encapsulate structural and feature-related information. GNNs are usually
trained in an end-to-end fashion, leading to highly specialized node
embeddings. However, generating node embeddings that can be used to perform
multiple tasks (with performance comparable to single-task models) is an open
problem. We propose a novel meta-learning strategy capable of producing
multi-task node embeddings. Our method avoids the difficulties arising when
learning to perform multiple tasks concurrently by, instead, learning to
quickly (i.e. with a few steps of gradient descent) adapt to multiple tasks
singularly. We show that the embeddings produced by our method can be used to
perform multiple tasks with comparable or higher performance than classically
trained models. Our method is model-agnostic and task-agnostic, thus applicable
to a wide variety of multi-task domains.

positive:
When faced with learning challenging new tasks, humans often follow sequences
of steps that allow them to incrementally build up the necessary skills for
performing these new tasks. However, in machine learning, models are most often
trained to solve the target tasks directly.Inspired by human learning, we
propose a novel curriculum learning approach which decomposes challenging tasks
into sequences of easier intermediate goals that are used to pre-train a model
before tackling the target task. We focus on classification tasks, and design
the intermediate tasks using an automatically constructed label hierarchy. We
train the model at each level of the hierarchy, from coarse labels to fine
labels, transferring acquired knowledge across these levels. For instance, the
model will first learn to distinguish animals from objects, and then use this
acquired knowledge when learning to classify among more fine-grained classes
such as cat, dog, car, and truck. Most existing curriculum learning algorithms
for supervised learning consist of scheduling the order in which the training
examples are presented to the model. In contrast, our approach focuses on the
output space of the model. We evaluate our method on several established
datasets and show significant performance gains especially on classification
problems with many labels. We also evaluate on a new synthetic dataset which
allows us to study multiple aspects of our method.

positive:
Recent research has proposed the lottery ticket hypothesis, suggesting that
for a deep neural network, there exist trainable sub-networks performing
equally or better than the original model with commensurate training steps.
While this discovery is insightful, finding proper sub-networks requires
iterative training and pruning. The high cost incurred limits the applications
of the lottery ticket hypothesis. We show there exists a subset of the
aforementioned sub-networks that converge significantly faster during the
training process and thus can mitigate the cost issue. We conduct extensive
experiments to show such sub-networks consistently exist across various model
structures for a restrictive setting of hyperparameters ($e.g.$, carefully
selected learning rate, pruning ratio, and model capacity). As a practical
application of our findings, we demonstrate that such sub-networks can help in
cutting down the total time of adversarial training, a standard approach to
improve robustness, by up to 49\% on CIFAR-10 to achieve the state-of-the-art
robustness.

positive:
Imitation learning has been commonly applied to solve different tasks in
isolation. This usually requires either careful feature engineering, or a
significant number of samples. This is far from what we desire: ideally, robots
should be able to learn from very few demonstrations of any given task, and
instantly generalize to new situations of the same task, without requiring
task-specific engineering. In this paper, we propose a meta-learning framework
for achieving such capability, which we call one-shot imitation learning.
  Specifically, we consider the setting where there is a very large set of
tasks, and each task has many instantiations. For example, a task could be to
stack all blocks on a table into a single tower, another task could be to place
all blocks on a table into two-block towers, etc. In each case, different
instances of the task would consist of different sets of blocks with different
initial states. At training time, our algorithm is presented with pairs of
demonstrations for a subset of all tasks. A neural net is trained that takes as
input one demonstration and the current state (which initially is the initial
state of the other demonstration of the pair), and outputs an action with the
goal that the resulting sequence of states and actions matches as closely as
possible with the second demonstration. At test time, a demonstration of a
single instance of a new task is presented, and the neural net is expected to
perform well on new instances of this new task. The use of soft attention
allows the model to generalize to conditions and tasks unseen in the training
data. We anticipate that by training this model on a much greater variety of
tasks and settings, we will obtain a general system that can turn any
demonstrations into robust policies that can accomplish an overwhelming variety
of tasks.
  Videos available at https://bit.ly/nips2017-oneshot .

positive:
Hybrid ensemble, an essential branch of ensembles, has flourished in the
regression field, with studies confirming diversity's importance. However,
previous ensembles consider diversity in the sub-model training stage, with
limited improvement compared to single models. In contrast, this study
automatically selects and weights sub-models from a heterogeneous model pool.
It solves an optimization problem using an interior-point filtering
linear-search algorithm. The objective function innovatively incorporates
negative correlation learning as a penalty term, with which a diverse model
subset can be selected. The best sub-models from each model class are selected
to build the NCL ensemble, which performance is better than the simple average
and other state-of-the-art weighting methods. It is also possible to improve
the NCL ensemble with a regularization term in the objective function. In
practice, it is difficult to conclude the optimal sub-model for a dataset prior
due to the model uncertainty. Regardless, our method would achieve comparable
accuracy as the potential optimal sub-models. In conclusion, the value of this
study lies in its ease of use and effectiveness, allowing the hybrid ensemble
to embrace diversity and accuracy.

positive:
This paper presents a novel framework for automatic learning of complex
strategies in human decision making. The task that we are interested in is to
better facilitate long term planning for complex, multi-step events. We observe
temporal relationships at the subtask level of expert demonstrations, and
determine the different strategies employed in order to successfully complete a
task. To capture the relationship between the subtasks and the overall goal, we
utilise two external memory modules, one for capturing dependencies within a
single expert demonstration, such as the sequential relationship among
different sub tasks, and a global memory module for modelling task level
characteristics such as best practice employed by different humans based on
their domain expertise. Furthermore, we demonstrate how the hidden state
representation of the memory can be used as a reward signal to smooth the state
transitions, eradicating subtle changes. We evaluate the effectiveness of the
proposed model for an autonomous highway driving application, where we
demonstrate its capability to learn different expert policies and outperform
state-of-the-art methods. The scope in industrial applications extends to any
robotics and automation application which requires learning from complex
demonstrations containing series of subtasks.

negative:
In this work, we study the problem of learning a single model for multiple
domains. Unlike the conventional machine learning scenario where each domain
can have the corresponding model, multiple domains (i.e., applications/users)
may share the same machine learning model due to maintenance loads in cloud
computing services. For example, a digit-recognition model should be applicable
to hand-written digits, house numbers, car plates, etc. Therefore, an ideal
model for cloud computing has to perform well at each applicable domain. To
address this new challenge from cloud computing, we develop a framework of
robust optimization over multiple domains. In lieu of minimizing the empirical
risk, we aim to learn a model optimized to the adversarial distribution over
multiple domains. Hence, we propose to learn the model and the adversarial
distribution simultaneously with the stochastic algorithm for efficiency.
Theoretically, we analyze the convergence rate for convex and non-convex
models. To our best knowledge, we first study the convergence rate of learning
a robust non-convex model with a practical algorithm. Furthermore, we
demonstrate that the robustness of the framework and the convergence rate can
be further enhanced by appropriate regularizers over the adversarial
distribution. The empirical study on real-world fine-grained visual
categorization and digits recognition tasks verifies the effectiveness and
efficiency of the proposed framework.

negative:
In recent years, model-free methods that use deep learning have achieved
great success in many different reinforcement learning environments. Most
successful approaches focus on solving a single task, while multi-task
reinforcement learning remains an open problem. In this paper, we present a
model based approach to deep reinforcement learning which we use to solve
different tasks simultaneously. We show that our approach not only does not
degrade but actually benefits from learning multiple tasks. For our model, we
also present a new kind of recurrent neural network inspired by residual
networks that decouples memory from computation allowing to model complex
environments that do not require lots of memory.

negative:
We introduce network with sub-networks, a neural network which its weight
layers could be detached into sub-neural networks during inference. To develop
weights and biases which could be inserted in both base and sub-neural
networks, firstly, the parameters are copied from sub-model to base-model. Each
model is forward-propagated separately. Gradients from a pair of networks are
averaged and, used to update both networks. Our base model achieves the
test-accuracy which is comparable to the regularly trained models, while the
model maintains the ability to detach weight layers.

negative:
When a neural language model (LM) is adapted to perform a new task, what
aspects of the task predict the eventual performance of the model? In NLP,
systematic features of LM generalization to individual examples are well
characterized, but systematic aspects of LM adaptability to new tasks are not
nearly as well understood. We present a large-scale empirical study of the
features and limits of LM adaptability using a new benchmark, TaskBench500,
built from 500 procedurally generated sequence modeling tasks. These tasks
combine core aspects of language processing, including lexical semantics,
sequence processing, memorization, logical reasoning, and world knowledge.
Using TaskBench500, we evaluate three facets of adaptability, finding that: (1)
adaptation procedures differ dramatically in their ability to memorize small
datasets; (2) within a subset of task types, adaptation procedures exhibit
compositional adaptability to complex tasks; and (3) failure to match training
label distributions is explained by mismatches in the intrinsic difficulty of
predicting individual labels. Our experiments show that adaptability to new
tasks, like generalization to new examples, can be systematically described and
understood, and we conclude with a discussion of additional aspects of
adaptability that could be studied using the new benchmark.

negative:
Continual learning is the problem of learning new tasks or knowledge while
protecting old knowledge and ideally generalizing from old experience to learn
new tasks faster. Neural networks trained by stochastic gradient descent often
degrade on old tasks when trained successively on new tasks with different data
distributions. This phenomenon, referred to as catastrophic forgetting, is
considered a major hurdle to learning with non-stationary data or sequences of
new tasks, and prevents networks from continually accumulating knowledge and
skills. We examine this issue in the context of reinforcement learning, in a
setting where an agent is exposed to tasks in a sequence. Unlike most other
work, we do not provide an explicit indication to the model of task boundaries,
which is the most general circumstance for a learning agent exposed to
continuous experience. While various methods to counteract catastrophic
forgetting have recently been proposed, we explore a straightforward, general,
and seemingly overlooked solution - that of using experience replay buffers for
all past events - with a mixture of on- and off-policy learning, leveraging
behavioral cloning. We show that this strategy can still learn new tasks
quickly yet can substantially reduce catastrophic forgetting in both Atari and
DMLab domains, even matching the performance of methods that require task
identities. When buffer storage is constrained, we confirm that a simple
mechanism for randomly discarding data allows a limited size buffer to perform
almost as well as an unbounded one.

negative:
Fictitious play with reinforcement learning is a general and effective
framework for zero-sum games. However, using the current deep neural network
models, the implementation of fictitious play faces crucial challenges. Neural
network model training employs gradient descent approaches to update all
connection weights, and thus is easy to forget the old opponents after training
to beat the new opponents. Existing approaches often maintain a pool of
historical policy models to avoid the forgetting. However, learning to beat a
pool in stochastic games, i.e., a wide distribution over policy models, is
either sample-consuming or insufficient to exploit all models with limited
amount of samples. In this paper, we propose a learning process with neural
fictitious play to alleviate the above issues. We train a single model as our
policy model, which consists of sub-models and a selector. Everytime facing a
new opponent, the model is expanded by adding a new sub-model, where only the
new sub-model is updated instead of the whole model. At the same time, the
selector is also updated to mix up the new sub-model with the previous ones at
the state-level, so that the model is maintained as a behavior strategy instead
of a wide distribution over policy models. Experiments on Kuhn poker, a
grid-world Treasure Hunting game, and Mini-RTS environments show that the
proposed approach alleviates the forgetting problem, and consequently improves
the learning efficiency and the robustness of neural fictitious play.

negative:
An important long-term goal in machine learning systems is to build learning
agents that, like humans, can learn many tasks over their lifetime, and
moreover use information from these tasks to improve their ability to do so
efficiently. In this work, our goal is to provide new theoretical insights into
the potential of this paradigm. In particular, we propose a lifelong learning
framework that adheres to a novel notion of resource efficiency that is
critical in many real-world domains where feature evaluations are costly. That
is, our learner aims to reuse information from previously learned related tasks
to learn future tasks in a feature-efficient manner. Furthermore, we consider
novel combinatorial ways in which learning tasks can relate. Specifically, we
design lifelong learning algorithms for two structurally different and widely
used families of target functions: decision trees/lists and
monomials/polynomials. We also provide strong feature-efficiency guarantees for
these algorithms; in fact, we show that in order to learn future targets, we
need only slightly more feature evaluations per training example than what is
needed to predict on an arbitrary example using those targets. We also provide
algorithms with guarantees in an agnostic model where not all the targets are
related to each other. Finally, we also provide lower bounds on the performance
of a lifelong learner in these models, which are in fact tight under some
conditions.

negative:
The motivation of this study is to leverage recent breakthroughs in
artificial intelligence research to unlock novel solutions to important
scientific problems encountered in computational science. To address the human
intelligence limitations in discovering reduced-order models, we propose to
supplement human thinking with artificial intelligence. Our three-pronged
strategy consists of learning (i) models expressed in analytical form, (ii)
which are evaluated a posteriori, and iii) using exclusively integral
quantities from the reference solution as prior knowledge. In point (i), we
pursue interpretable models expressed symbolically as opposed to black-box
neural networks, the latter only being used during learning to efficiently
parameterize the large search space of possible models. In point (ii), learned
models are dynamically evaluated a posteriori in the computational solver
instead of based on a priori information from preprocessed high-fidelity data,
thereby accounting for the specificity of the solver at hand such as its
numerics. Finally in point (iii), the exploration of new models is solely
guided by predefined integral quantities, e.g., averaged quantities of
engineering interest in Reynolds-averaged or large-eddy simulations (LES). We
use a coupled deep reinforcement learning framework and computational solver to
concurrently achieve these objectives. The combination of reinforcement
learning with objectives (i), (ii) and (iii) differentiate our work from
previous modeling attempts based on machine learning. In this report, we
provide a high-level description of the model discovery framework with
reinforcement learning. The method is detailed for the application of
discovering missing terms in differential equations. An elementary
instantiation of the method is described that discovers missing terms in the
Burgers' equation.

negative:
Deep learning models are considered to be state-of-the-art in many offline
machine learning tasks. However, many of the techniques developed are not
suitable for online learning tasks. The problem of using deep learning models
with sequential data becomes even harder when several loss functions need to be
considered simultaneously, as in many real-world applications. In this paper,
we, therefore, propose a novel online deep learning training procedure which
can be used regardless of the neural network's architecture, aiming to deal
with the multiple objectives case. We demonstrate and show the effectiveness of
our algorithm on the Neyman-Pearson classification problem on several benchmark
datasets.

negative:
Many NLP learning tasks can be decomposed into several distinct sub-tasks,
each associated with a partial label. In this paper we focus on a popular class
of learning problems, sequence prediction applied to several sentiment analysis
tasks, and suggest a modular learning approach in which different sub-tasks are
learned using separate functional modules, combined to perform the final task
while sharing information. Our experiments show this approach helps constrain
the learning process and can alleviate some of the supervision efforts.

negative:
Few-Shot Learning is the challenge of training a model with only a small
amount of data. Many solutions to this problem use meta-learning algorithms,
i.e. algorithms that learn to learn. By sampling few-shot tasks from a larger
dataset, we can teach these algorithms to solve new, unseen tasks. This
document reports my work on meta-learning algorithms for Few-Shot Computer
Vision. This work was done during my internship at Sicara, a French company
building image recognition solutions for businesses. It contains: 1. an
extensive review of the state-of-the-art in few-shot computer vision; 2. a
benchmark of meta-learning algorithms for few-shot image classification; 3. the
introduction to a novel meta-learning algorithm for few-shot object detection,
which is still in development.

negative:
In the artificial intelligence field, learning often corresponds to changing
the parameters of a parameterized function. A learning rule is an algorithm or
mathematical expression that specifies precisely how the parameters should be
changed. When creating an artificial intelligence system, we must make two
decisions: what representation should be used (i.e., what parameterized
function should be used) and what learning rule should be used to search
through the resulting set of representable functions. Using most learning
rules, these two decisions are coupled in a subtle (and often unintentional)
way. That is, using the same learning rule with two different representations
that can represent the same sets of functions can result in two different
outcomes. After arguing that this coupling is undesirable, particularly when
using artificial neural networks, we present a method for partially decoupling
these two decisions for a broad class of learning rules that span unsupervised
learning, reinforcement learning, and supervised learning.

negative:
Labeling data (e.g., labeling the people, objects, actions and scene in
images) comprehensively and efficiently is a widely needed but challenging
task. Numerous models were proposed to label various data and many approaches
were designed to enhance the ability of deep learning models or accelerate
them. Unfortunately, a single machine-learning model is not powerful enough to
extract various semantic information from data. Given certain applications,
such as image retrieval platforms and photo album management apps, it is often
required to execute a collection of models to obtain sufficient labels. With
limited computing resources and stringent delay, given a data stream and a
collection of applicable resource-hungry deep-learning models, we design a
novel approach to adaptively schedule a subset of these models to execute on
each data item, aiming to maximize the value of the model output (e.g., the
number of high-confidence labels). Achieving this lofty goal is nontrivial
since a model's output on any data item is content-dependent and unknown until
we execute it. To tackle this, we propose an Adaptive Model Scheduling
framework, consisting of 1) a deep reinforcement learning-based approach to
predict the value of unexecuted models by mining semantic relationship among
diverse models, and 2) two heuristic algorithms to adaptively schedule the
model execution order under a deadline or deadline-memory constraints
respectively. The proposed framework doesn't require any prior knowledge of the
data, which works as a powerful complement to existing model optimization
technologies. We conduct extensive evaluations on five diverse image datasets
and 30 popular image labeling models to demonstrate the effectiveness of our
design: our design could save around 53\% execution time without loss of any
valuable labels.

negative:
The objective of a reinforcement learning agent is to behave so as to
maximise the sum of a suitable scalar function of state: the reward. These
rewards are typically given and immutable. In this paper, we instead consider
the proposition that the reward function itself can be a good locus of learned
knowledge. To investigate this, we propose a scalable meta-gradient framework
for learning useful intrinsic reward functions across multiple lifetimes of
experience. Through several proof-of-concept experiments, we show that it is
feasible to learn and capture knowledge about long-term exploration and
exploitation into a reward function. Furthermore, we show that unlike policy
transfer methods that capture "how" the agent should behave, the learned reward
functions can generalise to other kinds of agents and to changes in the
dynamics of the environment by capturing "what" the agent should strive to do.

negative:
Interpretability has become a necessary feature for machine learning models
deployed in critical scenarios, e.g. legal system, healthcare. In these
situations, algorithmic decisions may have (potentially negative) long-lasting
effects on the end-user affected by the decision. In many cases, the
representational power of deep learning models is not needed, therefore simple
and interpretable models (e.g. linear models) should be preferred. However, in
high-dimensional and/or complex domains (e.g. computer vision), the universal
approximation capabilities of neural networks are required. Inspired by linear
models and the Kolmogorov-Arnold representation theorem, we propose a novel
class of structurally-constrained neural networks, which we call FLANs
(Feature-wise Latent Additive Networks). Crucially, FLANs process each input
feature separately, computing for each of them a representation in a common
latent space. These feature-wise latent representations are then simply summed,
and the aggregated representation is used for prediction. These constraints
(which are at the core of the interpretability of linear models) allow a user
to estimate the effect of each individual feature independently from the
others, enhancing interpretability. In a set of experiments across different
domains, we show how without compromising excessively the test performance, the
structural constraints proposed in FLANs indeed facilitates the
interpretability of deep learning models. We quantitatively compare FLANs
interpretability to post-hoc methods using recently introduced metrics,
discussing the advantages of natively interpretable models over a post-hoc
analysis.

negative:
This paper presents a meta-learning framework for few-shots One-Class
Classification (OCC) at test-time, a setting where labeled examples are only
available for the positive class, and no supervision is given for the negative
example. We consider that we have a set of `one-class classification'
objective-tasks with only a small set of positive examples available for each
task, and a set of training tasks with full supervision (i.e. highly imbalanced
classification). We propose an approach using order-equivariant networks to
learn a 'meta' binary-classifier. The model will take as input an example to
classify from a given task, as well as the corresponding supervised set of
positive examples for this OCC task. Thus, the output of the model will be
'conditioned' on the available positive example of a given task, allowing to
predict on new tasks and new examples without labeled negative examples. In
this paper, we are motivated by an astronomy application. Our goal is to
identify if stars belong to a specific stellar group (the 'one-class' for a
given task), called \textit{stellar streams}, where each stellar stream is a
different OCC-task. We show that our method transfers well on unseen (test)
synthetic streams, and outperforms the baselines even though it is not
retrained and accesses a much smaller part of the data per task to predict
(only positive supervision). We see however that it doesn't transfer as well on
the real stream GD-1. This could come from intrinsic differences from the
synthetic and real stream, highlighting the need for consistency in the
'nature' of the task for this method. However, light fine-tuning improve
performances and outperform our baselines. Our experiments show encouraging
results to further explore meta-learning methods for OCC tasks.

negative:
Curriculum learning in reinforcement learning is a training methodology that
seeks to speed up learning of a difficult target task, by first training on a
series of simpler tasks and transferring the knowledge acquired to the target
task. Automatically choosing a sequence of such tasks (i.e. a curriculum) is an
open problem that has been the subject of much recent work in this area. In
this paper, we build upon a recent method for curriculum design, which
formulates the curriculum sequencing problem as a Markov Decision Process. We
extend this model to handle multiple transfer learning algorithms, and show for
the first time that a curriculum policy over this MDP can be learned from
experience. We explore various representations that make this possible, and
evaluate our approach by learning curriculum policies for multiple agents in
two different domains. The results show that our method produces curricula that
can train agents to perform on a target task as fast or faster than existing
methods.

negative:
Continual learning is a concept of online learning with multiple sequential
tasks. One of the critical barriers of continual learning is that a network
should learn a new task keeping the knowledge of old tasks without access to
any data of the old tasks. In this paper, we propose a neuron activation
importance-based regularization method for stable continual learning regardless
of the order of tasks. We conduct comprehensive experiments on existing
benchmark data sets to evaluate not just the stability and plasticity of our
method with improved classification accuracy also the robustness of the
performance along the changes of task order.

negative:
The work of McCloskey and Cohen popularized the concept of catastrophic
interference. They used a neural network that tried to learn addition using two
groups of examples as two different tasks. In their case, learning the second
task rapidly deteriorated the acquired knowledge about the previous one. We
hypothesize that this could be a symptom of a fundamental problem: addition is
an algorithmic task that should not be learned through pattern recognition.
Therefore, other model architectures better suited for this task would avoid
catastrophic forgetting. We use a neural network with a different architecture
that can be trained to recover the correct algorithm for the addition of binary
numbers. This neural network includes conditional clauses that are naturally
treated within the back-propagation algorithm. We test it in the setting
proposed by McCloskey and Cohen and training on random additions one by one.
The neural network not only does not suffer from catastrophic forgetting but it
improves its predictive power on unseen pairs of numbers as training
progresses. We also show that this is a robust effect, also present when
averaging many simulations. This work emphasizes the importance that neural
network architecture has for the emergence of catastrophic forgetting and
introduces a neural network that is able to learn an algorithm.

negative:
Machine understanding of complex images is a key goal of artificial
intelligence. One challenge underlying this task is that visual scenes contain
multiple inter-related objects, and that global context plays an important role
in interpreting the scene. A natural modeling framework for capturing such
effects is structured prediction, which optimizes over complex labels, while
modeling within-label interactions. However, it is unclear what principles
should guide the design of a structured prediction model that utilizes the
power of deep learning components. Here we propose a design principle for such
architectures that follows from a natural requirement of permutation
invariance. We prove a necessary and sufficient characterization for
architectures that follow this invariance, and discuss its implication on model
design. Finally, we show that the resulting model achieves new state of the art
results on the Visual Genome scene graph labeling benchmark, outperforming all
recent approaches.

negative:
Multi-task learning aims to improve generalization performance of multiple
prediction tasks by appropriately sharing relevant information across them. In
the context of deep neural networks, this idea is often realized by
hand-designed network architectures with layers that are shared across tasks
and branches that encode task-specific features. However, the space of possible
multi-task deep architectures is combinatorially large and often the final
architecture is arrived at by manual exploration of this space subject to
designer's bias, which can be both error-prone and tedious. In this work, we
propose a principled approach for designing compact multi-task deep learning
architectures. Our approach starts with a thin network and dynamically widens
it in a greedy manner during training using a novel criterion that promotes
grouping of similar tasks together. Our Extensive evaluation on person
attributes classification tasks involving facial and clothing attributes
suggests that the models produced by the proposed method are fast, compact and
can closely match or exceed the state-of-the-art accuracy from strong baselines
by much more expensive models.

negative:
It has been a long-standing goal in machine learning, as well as in AI more
generally, to develop life-long learning systems that learn many different
tasks over time, and reuse insights from tasks learned, "learning to learn" as
they do so. In this work we pose and provide efficient algorithms for several
natural theoretical formulations of this goal. Specifically, we consider the
problem of learning many different target functions over time, that share
certain commonalities that are initially unknown to the learning algorithm. Our
aim is to learn new internal representations as the algorithm learns new target
functions, that capture this commonality and allow subsequent learning tasks to
be solved more efficiently and from less data. We develop efficient algorithms
for two very different kinds of commonalities that target functions might
share: one based on learning common low-dimensional and unions of
low-dimensional subspaces and one based on learning nonlinear Boolean
combinations of features. Our algorithms for learning Boolean feature
combinations additionally have a dual interpretation, and can be viewed as
giving an efficient procedure for constructing near-optimal sparse Boolean
autoencoders under a natural "anchor-set" assumption.

negative:
One of the questions that arises when designing models that learn to solve
multiple tasks simultaneously is how much of the available training budget
should be devoted to each individual task. We refer to any formalized approach
to addressing this problem (learned or otherwise) as a task selection policy.
In this work we provide an empirical evaluation of the performance of some
common task selection policies in a synthetic bandit-style setting, as well as
on the GLUE benchmark for natural language understanding. We connect task
selection policy learning to existing work on automated curriculum learning and
off-policy evaluation, and suggest a method based on counterfactual estimation
that leads to improved model performance in our experimental settings.

negative:
In this paper, we compare the three most popular algorithms for
hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm)
and attempt to use them for neural architecture search (NAS). We use these
algorithms for building a convolutional neural network (search architecture).
Experimental results on CIFAR-10 dataset further demonstrate the performance
difference between compared algorithms. The comparison results are based on the
execution time of the above algorithms and accuracy of the proposed models.

negative:
In open set learning, a model must be able to generalize to novel classes
when it encounters a sample that does not belong to any of the classes it has
seen before. Open set learning poses a realistic learning scenario that is
receiving growing attention. Existing studies on open set learning mainly
focused on detecting novel classes, but few studies tried to model them for
differentiating novel classes. In this paper, we recognize that novel classes
should be different from each other, and propose distribution networks for open
set learning that can model different novel classes based on probability
distributions. We hypothesize that, through a certain mapping, samples from
different classes with the same classification criterion should follow
different probability distributions from the same distribution family. A deep
neural network is learned to map the samples in the original feature space to a
latent space where the distributions of known classes can be jointly learned
with the network. We additionally propose a distribution parameter transfer and
updating strategy for novel class modeling when a novel class is detected in
the latent space. By novel class modeling, the detected novel classes can serve
as known classes to the subsequent classification. Our experimental results on
image datasets MNIST and CIFAR10 show that the distribution networks can detect
novel classes accurately, and model them well for the subsequent classification
tasks.

negative:
Sharing information between multiple tasks enables algorithms to achieve good
generalization performance even from small amounts of training data. However,
in a realistic scenario of multi-task learning not all tasks are equally
related to each other, hence it could be advantageous to transfer information
only between the most related tasks. In this work we propose an approach that
processes multiple tasks in a sequence with sharing between subsequent tasks
instead of solving all tasks jointly. Subsequently, we address the question of
curriculum learning of tasks, i.e. finding the best order of tasks to be
learned. Our approach is based on a generalization bound criterion for choosing
the task order that optimizes the average expected classification performance
over all tasks. Our experimental results show that learning multiple related
tasks sequentially can be more effective than learning them jointly, the order
in which tasks are being solved affects the overall performance, and that our
model is able to automatically discover the favourable order of tasks.

negative:
Medical image segmentation has significantly benefitted thanks to deep
learning architectures. Furthermore, semi-supervised learning (SSL) has
recently been a growing trend for improving a model's overall performance by
leveraging abundant unlabeled data. Moreover, learning multiple tasks within
the same model further improves model generalizability. To generate smoother
and accurate segmentation masks from 3D cardiac MR images, we present a
Multi-task Cross-task learning consistency approach to enforce the correlation
between the pixel-level (segmentation) and the geometric-level (distance map)
tasks. Our extensive experimentation with varied quantities of labeled data in
the training sets justifies the effectiveness of our model for the segmentation
of the left atrial cavity from Gadolinium-enhanced magnetic resonance (GE-MR)
images. With the incorporation of uncertainty estimates to detect failures in
the segmentation masks generated by CNNs, our study further showcases the
potential of our model to flag low-quality segmentation from a given model.

negative:
Prior work on training generative Visual Dialog models with reinforcement
learning(Das et al.) has explored a Qbot-Abot image-guessing game and shown
that this 'self-talk' approach can lead to improved performance at the
downstream dialog-conditioned image-guessing task. However, this improvement
saturates and starts degrading after a few rounds of interaction, and does not
lead to a better Visual Dialog model. We find that this is due in part to
repeated interactions between Qbot and Abot during self-talk, which are not
informative with respect to the image. To improve this, we devise a simple
auxiliary objective that incentivizes Qbot to ask diverse questions, thus
reducing repetitions and in turn enabling Abot to explore a larger state space
during RL ie. be exposed to more visual concepts to talk about, and varied
questions to answer. We evaluate our approach via a host of automatic metrics
and human studies, and demonstrate that it leads to better dialog, ie. dialog
that is more diverse (ie. less repetitive), consistent (ie. has fewer
conflicting exchanges), fluent (ie. more human-like),and detailed, while still
being comparably image-relevant as prior work and ablations.

negative:
Despite its potential to improve sample complexity versus model-free
approaches, model-based reinforcement learning can fail catastrophically if the
model is inaccurate. An algorithm should ideally be able to trust an imperfect
model over a reasonably long planning horizon, and only rely on model-free
updates when the model errors get infeasibly large. In this paper, we
investigate techniques for choosing the planning horizon on a state-dependent
basis, where a state's planning horizon is determined by the maximum cumulative
model error around that state. We demonstrate that these state-dependent model
errors can be learned with Temporal Difference methods, based on a novel
approach of temporally decomposing the cumulative model errors. Experimental
results show that the proposed method can successfully adapt the planning
horizon to account for state-dependent model accuracy, significantly improving
the efficiency of policy learning compared to model-based and model-free
baselines.

negative:
Machine learning algorithms have enabled computers to predict things by
learning from previous data. The data storage and processing power are
increasing rapidly, thus increasing machine learning and Artificial
intelligence applications. Much of the work is done to improve the accuracy of
the models built in the past, with little research done to determine the
computational costs of machine learning acquisitions. In this paper, I will
proceed with this later research work and will make a performance comparison of
multi-threaded machine learning clustering algorithms. I will be working on
Linear Regression, Random Forest, and K-Nearest Neighbors to determine the
performance characteristics of the algorithms as well as the computation costs
to the obtained results. I will be benchmarking system hardware performance by
running these multi-threaded algorithms to train and test the models on a
dataset to note the differences in performance matrices of the algorithms. In
the end, I will state the best performing algorithms concerning the performance
efficiency of these algorithms on my system.

negative:
Model complexity is a fundamental problem in deep learning. In this paper we
conduct a systematic overview of the latest studies on model complexity in deep
learning. Model complexity of deep learning can be categorized into expressive
capacity and effective model complexity. We review the existing studies on
those two categories along four important factors, including model framework,
model size, optimization process and data complexity. We also discuss the
applications of deep learning model complexity including understanding model
generalization, model optimization, and model selection and design. We conclude
by proposing several interesting future directions.

negative:
During sleep and awake rest, the hippocampus replays sequences of place cells
that have been activated during prior experiences. These have been interpreted
as a memory consolidation process, but recent results suggest a possible
interpretation in terms of reinforcement learning. The Dyna reinforcement
learning algorithms use off-line replays to improve learning. Under limited
replay budget, a prioritized sweeping approach, which requires a model of the
transitions to the predecessors, can be used to improve performance. We
investigate whether such algorithms can explain the experimentally observed
replays. We propose a neural network version of prioritized sweeping
Q-learning, for which we developed a growing multiple expert algorithm, able to
cope with multiple predecessors. The resulting architecture is able to improve
the learning of simulated agents confronted to a navigation task. We predict
that, in animals, learning the world model should occur during rest periods,
and that the corresponding replays should be shuffled.

negative:
To solve the problem of redundant information and overlapping relations of
the entity and relation extraction model, we propose a joint extraction model.
This model can directly extract multiple pairs of related entities without
generating unrelated redundant information. We also propose a recurrent neural
network named Encoder-LSTM that enhances the ability of recurrent units to
model sentences. Specifically, the joint model includes three sub-modules: the
Named Entity Recognition sub-module consisted of a pre-trained language model
and an LSTM decoder layer, the Entity Pair Extraction sub-module which uses
Encoder-LSTM network to model the order relationship between related entity
pairs, and the Relation Classification sub-module including Attention
mechanism. We conducted experiments on the public datasets ADE and CoNLL04 to
evaluate the effectiveness of our model. The results show that the proposed
model achieves good performance in the task of entity and relation extraction
and can greatly reduce the amount of redundant information.

negative:
All famous machine learning algorithms that comprise both supervised and
semi-supervised learning work well only under a common assumption: the training
and test data follow the same distribution. When the distribution changes, most
statistical models must be reconstructed from newly collected data, which for
some applications can be costly or impossible to obtain. Therefore, it has
become necessary to develop approaches that reduce the need and the effort to
obtain new labeled samples by exploiting data that are available in related
areas, and using these further across similar fields. This has given rise to a
new machine learning framework known as transfer learning: a learning setting
inspired by the capability of a human being to extrapolate knowledge across
tasks to learn more efficiently. Despite a large amount of different transfer
learning scenarios, the main objective of this survey is to provide an overview
of the state-of-the-art theoretical results in a specific, and arguably the
most popular, sub-field of transfer learning, called domain adaptation. In this
sub-field, the data distribution is assumed to change across the training and
the test data, while the learning task remains the same. We provide a first
up-to-date description of existing results related to domain adaptation problem
that cover learning bounds based on different statistical learning frameworks.

negative:
Entity Matching (EM) is a core data cleaning task, aiming to identify
different mentions of the same real-world entity. Active learning is one way to
address the challenge of scarce labeled data in practice, by dynamically
collecting the necessary examples to be labeled by an Oracle and refining the
learned model (classifier) upon them. In this paper, we build a unified active
learning benchmark framework for EM that allows users to easily combine
different learning algorithms with applicable example selection algorithms. The
goal of the framework is to enable concrete guidelines for practitioners as to
what active learning combinations will work well for EM. Towards this, we
perform comprehensive experiments on publicly available EM datasets from
product and publication domains to evaluate active learning methods, using a
variety of metrics including EM quality, #labels and example selection
latencies. Our most surprising result finds that active learning with fewer
labels can learn a classifier of comparable quality as supervised learning. In
fact, for several of the datasets, we show that there is an active learning
combination that beats the state-of-the-art supervised learning result. Our
framework also includes novel optimizations that improve the quality of the
learned model by roughly 9% in terms of F1-score and reduce example selection
latencies by up to 10x without affecting the quality of the model.

negative:
The idea of using multi-task learning approaches to address the joint
extraction of entity and relation is motivated by the relatedness between the
entity recognition task and the relation classification task. Existing methods
using multi-task learning techniques to address the problem learn interactions
among the two tasks through a shared network, where the shared information is
passed into the task-specific networks for prediction. However, such an
approach hinders the model from learning explicit interactions between the two
tasks to improve the performance on the individual tasks. As a solution, we
design a multi-task learning model which we refer to as recurrent interaction
network which allows the learning of interactions dynamically, to effectively
model task-specific features for classification. Empirical studies on two
real-world datasets confirm the superiority of the proposed model.

negative:
Given the fact of a case, Legal Judgment Prediction (LJP) involves a series
of sub-tasks such as predicting violated law articles, charges and term of
penalty. We propose leveraging a unified text-to-text Transformer for LJP,
where the dependencies among sub-tasks can be naturally established within the
auto-regressive decoder. Compared with previous works, it has three advantages:
(1) it fits in the pretraining pattern of masked language models, and thereby
can benefit from the semantic prompts of each sub-task rather than treating
them as atomic labels, (2) it utilizes a single unified architecture, enabling
full parameter sharing across all sub-tasks, and (3) it can incorporate both
classification and generative sub-tasks. We show that this unified transformer,
albeit pretrained on general-domain text, outperforms pretrained models
tailored specifically for the legal domain. Through an extensive set of
experiments, we find that the best order to capture dependencies is different
from human intuitions, and the most reasonable logical order for humans can be
sub-optimal for the model. We further include two more auxiliary tasks: court
view generation and article content prediction, showing they can not only
improve the prediction accuracy, but also provide interpretable explanations
for model outputs even when an error is made. With the best configuration, our
model outperforms both previous SOTA and a single-tasked version of the unified
transformer by a large margin.

negative:
Currently the only techniques for sharing governance of a deep learning model
are homomorphic encryption and secure multiparty computation. Unfortunately,
neither of these techniques is applicable to the training of large neural
networks due to their large computational and communication overheads. As a
scalable technique for shared model governance, we propose splitting deep
learning model between multiple parties. This paper empirically investigates
the security guarantee of this technique, which is introduced as the problem of
model completion: Given the entire training data set or an environment
simulator, and a subset of the parameters of a trained deep learning model, how
much training is required to recover the model's original performance? We
define a metric for evaluating the hardness of the model completion problem and
study it empirically in both supervised learning on ImageNet and reinforcement
learning on Atari and DeepMind~Lab. Our experiments show that (1) the model
completion problem is harder in reinforcement learning than in supervised
learning because of the unavailability of the trained agent's trajectories, and
(2) its hardness depends not primarily on the number of parameters of the
missing part, but more so on their type and location. Our results suggest that
model splitting might be a feasible technique for shared model governance in
some settings where training is very expensive.

negative:
Identifying relationships between items is a key task of an online
recommender system, in order to help users discover items that are functionally
complementary or visually compatible. In domains like clothing recommendation,
this task is particularly challenging since a successful system should be
capable of handling a large corpus of items, a huge amount of relationships
among them, as well as the high-dimensional and semantically complicated
features involved. Furthermore, the human notion of "compatibility" to capture
goes beyond mere similarity: For two items to be compatible---whether jeans and
a t-shirt, or a laptop and a charger---they should be similar in some ways, but
systematically different in others.
  In this paper we propose a novel method, Monomer, to learn complicated and
heterogeneous relationships between items in product recommendation settings.
Recently, scalable methods have been developed that address this task by
learning similarity metrics on top of the content of the products involved.
Here our method relaxes the metricity assumption inherent in previous work and
models multiple localized notions of 'relatedness,' so as to uncover ways in
which related items should be systematically similar, and systematically
different. Quantitatively, we show that our system achieves state-of-the-art
performance on large-scale compatibility prediction tasks, especially in cases
where there is substantial heterogeneity between related items. Qualitatively,
we demonstrate that richer notions of compatibility can be learned that go
beyond similarity, and that our model can make effective recommendations of
heterogeneous content.

negative:
To act and plan in complex environments, we posit that agents should have a
mental simulator of the world with three characteristics: (a) it should build
an abstract state representing the condition of the world; (b) it should form a
belief which represents uncertainty on the world; (c) it should go beyond
simple step-by-step simulation, and exhibit temporal abstraction. Motivated by
the absence of a model satisfying all these requirements, we propose TD-VAE, a
generative sequence model that learns representations containing explicit
beliefs about states several steps into the future, and that can be rolled out
directly without single-step transitions. TD-VAE is trained on pairs of
temporally separated time points, using an analogue of temporal difference
learning used in reinforcement learning.

negative:
This paper presents a new deep regression model, which we call DeepDistance,
for cell detection in images acquired with inverted microscopy. This model
considers cell detection as a task of finding most probable locations that
suggest cell centers in an image. It represents this main task with a
regression task of learning an inner distance metric. However, different than
the previously reported regression based methods, the DeepDistance model
proposes to approach its learning as a multi-task regression problem where
multiple tasks are learned by using shared feature representations. To this
end, it defines a secondary metric, normalized outer distance, to represent a
different aspect of the problem and proposes to define its learning as
complementary to the main cell detection task. In order to learn these two
complementary tasks more effectively, the DeepDistance model designs a fully
convolutional network (FCN) with a shared encoder path and end-to-end trains
this FCN to concurrently learn the tasks in parallel. DeepDistance uses the
inner distances estimated by this FCN in a detection algorithm to locate
individual cells in a given image. For further performance improvement on the
main task, this paper also presents an extended version of the DeepDistance
model. This extended model includes an auxiliary classification task and learns
it in parallel to the two regression tasks by sharing feature representations
with them. Our experiments on three different human cell lines reveal that the
proposed multi-task learning models, the DeepDistance model and its extended
version, successfully identify cell locations, even for the cell line that was
not used in training, and improve the results of the previous deep learning
methods.

negative:
We propose an adversarial training procedure for learning a causal implicit
generative model for a given causal graph. We show that adversarial training
can be used to learn a generative model with true observational and
interventional distributions if the generator architecture is consistent with
the given causal graph. We consider the application of generating faces based
on given binary labels where the dependency structure between the labels is
preserved with a causal graph. This problem can be seen as learning a causal
implicit generative model for the image and labels. We devise a two-stage
procedure for this problem. First we train a causal implicit generative model
over binary labels using a neural network consistent with a causal graph as the
generator. We empirically show that WassersteinGAN can be used to output
discrete labels. Later, we propose two new conditional GAN architectures, which
we call CausalGAN and CausalBEGAN. We show that the optimal generator of the
CausalGAN, given the labels, samples from the image distributions conditioned
on these labels. The conditional GAN combined with a trained causal implicit
generative model for the labels is then a causal implicit generative model over
the labels and the generated image. We show that the proposed architectures can
be used to sample from observational and interventional image distributions,
even for interventions which do not naturally occur in the dataset.

negative:
This doctoral thesis covers some of my advances in electron microscopy with
deep learning. Highlights include a comprehensive review of deep learning in
electron microscopy; large new electron microscopy datasets for machine
learning, dataset search engines based on variational autoencoders, and
automatic data clustering by t-distributed stochastic neighbour embedding;
adaptive learning rate clipping to stabilize learning; generative adversarial
networks for compressed sensing with spiral, uniformly spaced and other fixed
sparse scan paths; recurrent neural networks trained to piecewise adapt sparse
scan paths to specimens by reinforcement learning; improving signal-to-noise;
and conditional generative adversarial networks for exit wavefunction
reconstruction from single transmission electron micrographs. This thesis adds
to my publications by presenting their relationships, reflections, and holistic
conclusions. This version of my thesis is typeset for online dissemination to
improve readability, whereas the thesis submitted to the University of Warwick
in support of my application for the degree of Doctor of Philosophy in Physics
is typeset for physical printing and binding.

negative:
This paper makes a first step towards a logic of learning from experiments.
For this, we investigate formal frameworks for modeling the interaction of
causal and (qualitative) epistemic reasoning. Crucial for our approach is the
idea that the notion of an intervention can be used as a formal expression of a
(real or hypothetical) experiment. In a first step we extend the well-known
causal models with a simple Hintikka-style representation of the epistemic
state of an agent. In the resulting setting, one can talk not only about the
knowledge of an agent about the values of variables and how interventions
affect them, but also about knowledge update. The resulting logic can model
reasoning about thought experiments. However, it is unable to account for
learning from experiments, which is clearly brought out by the fact that it
validates the no learning principle for interventions. Therefore, in a second
step, we implement a more complex notion of knowledge that allows an agent to
observe (measure) certain variables when an experiment is carried out. This
extended system does allow for learning from experiments. For all the proposed
logical systems, we provide a sound and complete axiomatization.

negative:
When data is unlabelled and the target task is not known a priori, divergent
search offers a strategy for learning a wide range of skills. Having such a
repertoire allows a system to adapt to new, unforeseen tasks. Unlabelled image
data is plentiful, but it is not always known which features will be required
for downstream tasks. We propose a method for divergent search in the few-shot
image classification setting and evaluate with Omniglot and Mini-ImageNet. This
high-dimensional behavior space includes all possible ways of partitioning the
data. To manage divergent search in this space, we rely on a meta-learning
framework to integrate useful features from diverse tasks into a single model.
The final layer of this model is used as an index into the `archive' of all
past behaviors. We search for regions in the behavior space that the current
archive cannot reach. As expected, divergent search is outperformed by models
with a strong bias toward the evaluation tasks. But it is able to match and
sometimes exceed the performance of models that have a weak bias toward the
target task or none at all. This demonstrates that divergent search is a viable
approach, even in high-dimensional behavior spaces.

negative:
Despite the potential of active inference for visual-based control, learning
the model and the preferences (priors) while interacting with the environment
is challenging. Here, we study the performance of a deep active inference
(dAIF) agent on OpenAI's car racing benchmark, where there is no access to the
car's state. The agent learns to encode the world's state from high-dimensional
input through unsupervised representation learning. State inference and control
are learned end-to-end by optimizing the expected free energy. Results show
that our model achieves comparable performance to deep Q-learning. However,
vanilla dAIF does not reach state-of-the-art performance compared to other
world model approaches. Hence, we discuss the current model implementation's
limitations and potential architectures to overcome them.

negative:
Out-of-distribution (OOD) detection has received much attention lately due to
its practical importance in enhancing the safe deployment of neural networks.
One of the primary challenges is that models often produce highly confident
predictions on OOD data, which undermines the driving principle in OOD
detection that the model should only be confident about in-distribution
samples. In this work, we propose ReAct--a simple and effective technique for
reducing model overconfidence on OOD data. Our method is motivated by novel
analysis on internal activations of neural networks, which displays highly
distinctive signature patterns for OOD distributions. Our method can generalize
effectively to different network architectures and different OOD detection
scores. We empirically demonstrate that ReAct achieves competitive detection
performance on a comprehensive suite of benchmark datasets, and give
theoretical explication for our method's efficacy. On the ImageNet benchmark,
ReAct reduces the false positive rate (FPR95) by 25.05% compared to the
previous best method.

negative:
This paper describes an application of reinforcement learning to the mention
detection task. We define a novel action-based formulation for the mention
detection task, in which a model can flexibly revise past labeling decisions by
grouping together tokens and assigning partial mention labels. We devise a
method to create mention-level episodes and we train a model by rewarding
correctly labeled complete mentions, irrespective of the inner structure
created. The model yields results which are on par with a competitive
supervised counterpart while being more flexible in terms of achieving targeted
behavior through reward modeling and generating internal mention structure,
especially on longer mentions.

negative:
Deep neural networks have achieved impressive results on a wide variety of
tasks. However, quantifying uncertainty in the network's output is a
challenging task. Bayesian models offer a mathematical framework to reason
about model uncertainty. Variational methods have been used for approximating
intractable integrals that arise in Bayesian inference for neural networks. In
this report, we review the major variational inference concepts pertinent to
Bayesian neural networks and compare various approximation methods used in
literature. We also talk about the applications of variational bayes in
Reinforcement learning and continual learning.

negative:
In model-based reinforcement learning it is typical to decouple the problems
of learning the dynamics model and learning the reward function. However, when
the dynamics model is flawed, it may generate erroneous states that would never
occur in the true environment. It is not clear a priori what value the reward
function should assign to such states. This paper presents a novel error bound
that accounts for the reward model's behavior in states sampled from the model.
This bound is used to extend the existing Hallucinated DAgger-MC algorithm,
which offers theoretical performance guarantees in deterministic MDPs that do
not assume a perfect model can be learned. Empirically, this approach to reward
learning can yield dramatic improvements in control performance when the
dynamics model is flawed.

negative:
Objects in the world usually appear in context, participating in spatial
relationships and interactions that are predictable and expected. Knowledge of
these contexts can be used in the task of using a mobile camera to search for a
specified object in a room. We call this the object search task. This paper is
concerned with representing this knowledge in a manner facilitating its
application to object search while at the same time lending itself to
autonomous learning by a robot. The ability for the robot to learn such
knowledge without supervision is crucial due to the vast number of possible
relationships that can exist for any given set of objects. Moreover, since a
robot will not have an infinite amount of time to learn, it must be able to
determine an order in which to look for possible relationships so as to
maximize the rate at which new knowledge is gained. In effect, there must be a
"focus of interest" operator that allows the robot to choose which examples are
likely to convey the most new information and should be examined first. This
paper demonstrates how a representation based on statistical confidence
intervals allows the construction of a system that achieves the above goals. An
algorithm, based on the Highest Impact First heuristic, is presented as a means
for providing a "focus of interest" with which to control the learning process,
and examples are given.

negative:
In this article I describe a research agenda for securing machine learning
models against adversarial inputs at test time. This article does not present
results but instead shares some of my thoughts about where I think that the
field needs to go. Modern machine learning works very well on I.I.D. data: data
for which each example is drawn {\em independently} and for which the
distribution generating each example is {\em identical}. When these assumptions
are relaxed, modern machine learning can perform very poorly. When machine
learning is used in contexts where security is a concern, it is desirable to
design models that perform well even when the input is designed by a malicious
adversary. So far most research in this direction has focused on an adversary
who violates the {\em identical} assumption, and imposes some kind of
restricted worst-case distribution shift. I argue that machine learning
security researchers should also address the problem of relaxing the {\em
independence} assumption and that current strategies designed for robustness to
distribution shift will not do so. I recommend {\em dynamic models} that change
each time they are run as a potential solution path to this problem, and show
an example of a simple attack using correlated data that can be mitigated by a
simple dynamic defense. This is not intended as a real-world security measure,
but as a recommendation to explore this research direction and develop more
realistic defenses.

negative:
Recently, there has been great interest in learning how to best represent
proteins, specifically with fixed-length embeddings. Deep learning has become a
popular tool for protein representation learning as a model's hidden layers
produce potentially useful vector embeddings. TAPE introduced a number of
benchmark tasks and showed that semi-supervised learning, via pretraining
language models on a large protein corpus, improved performance on downstream
tasks. Two of the tasks (fluorescence prediction and stability prediction)
involve learning fitness landscapes. In this paper, we show that CNN models
trained solely using supervised learning both compete with and sometimes
outperform the best models from TAPE that leverage expensive pretraining on
large protein datasets. These CNN models are sufficiently simple and small that
they can be trained using a Google Colab notebook. We also find for the
fluorescence task that linear regression outperforms our models and the TAPE
models. The benchmarking tasks proposed by TAPE are excellent measures of a
model's ability to predict protein function and should be used going forward.
However, we believe it is important to add baselines from simple models to put
the performance of the semi-supervised models that have been reported so far
into perspective.

negative:
We consider a many-to-one wireless architecture for federated learning at the
network edge, where multiple edge devices collaboratively train a model using
local data. The unreliable nature of wireless connectivity, together with
constraints in computing resources at edge devices, dictates that the local
updates at edge devices should be carefully crafted and compressed to match the
wireless communication resources available and should work in concert with the
receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent
algorithms for such settings. Specifically, for the wireless edge employing
over-the-air computing, a common subset of k-coordinates of the gradient
updates across edge devices are selected by the receiver in each iteration, and
then transmitted simultaneously over k sub-carriers, each experiencing
time-varying channel conditions. We characterize the impact of communication
error and compression, in terms of the resulting gradient bias and mean squared
error, on the convergence of the proposed algorithms. We then study
learning-driven communication error minimization via joint optimization of
power allocation and learning rates. Our findings reveal that optimal power
allocation across different sub-carriers should take into account both the
gradient values and channel conditions, thus generalizing the widely used
water-filling policy. We also develop sub-optimal distributed solutions
amenable to implementation.

negative:
We emphasize the importance of asking the right question when interpreting
the decisions of a learning model. We discuss a natural extension of the
theoretical machinery from Janzing et. al. 2020, which answers the question
"Why did my model predict a person has cancer?" for answering a more involved
question, "What caused my model to predict a person has cancer?" While the
former quantifies the direct effects of variables on the model, the latter also
accounts for indirect effects, thereby providing meaningful insights wherever
human beings can reason in terms of cause and effect. We propose three broad
categories for interpretations: observational, model-specific and causal each
of which are significant in their own right. Furthermore, this paper quantifies
feature relevance by weaving different natures of interpretations together with
different measures as characteristic functions for Shapley symmetrization.
Besides the widely used expected value of the model, we also discuss measures
of statistical uncertainty and dispersion as informative candidates, and their
merits in generating explanations for each data point, some of which are used
in this context for the first time. These measures are not only useful for
studying the influence of variables on the model output, but also on the
predictive performance of the model, and for that we propose relevant
characteristic functions that are also used for the first time.

negative:
Model interpretation, or explanation of a machine learning classifier, aims
to extract generalizable knowledge from a trained classifier into a
human-understandable format, for various purposes such as model assessment,
debugging and trust. From a computaional viewpoint, it is formulated as
approximating the target classifier using a simpler interpretable model, such
as rule models like a decision set/list/tree. Often, this approximation is
handled as standard supervised learning and the only difference is that the
labels are provided by the target classifier instead of ground truth. This
paradigm is particularly popular because there exists a variety of well-studied
supervised algorithms for learning an interpretable classifier. However, we
argue that this paradigm is suboptimal for it does not utilize the unique
property of the model interpretation problem, that is, the ability to generate
synthetic instances and query the target classifier for their labels. We call
this the active-query property, suggesting that we should consider model
interpretation from an active learning perspective. Following this insight, we
argue that the active-query property should be employed when designing a model
interpretation algorithm, and that the generation of synthetic instances should
be integrated seamlessly with the algorithm that learns the model
interpretation. In this paper, we demonstrate that by doing so, it is possible
to achieve more faithful interpretation with simpler model complexity. As a
technical contribution, we present an active algorithm Active Decision Set
Induction (ADS) to learn a decision set, a set of if-else rules, for model
interpretation. ADS performs a local search over the space of all decision
sets. In every iteration, ADS computes confidence intervals for the value of
the objective function of all local actions and utilizes active-query to
determine the best one.

negative:
Entity resolution (ER) refers to the problem of matching records in one or
more relations that refer to the same real-world entity. While supervised
machine learning (ML) approaches achieve the state-of-the-art results, they
require a large amount of labeled examples that are expensive to obtain and
often times infeasible. We investigate an important problem that vexes
practitioners: is it possible to design an effective algorithm for ER that
requires Zero labeled examples, yet can achieve performance comparable to
supervised approaches? In this paper, we answer in the affirmative through our
proposed approach dubbed ZeroER. Our approach is based on a simple observation
-- the similarity vectors for matches should look different from that of
unmatches. Operationalizing this insight requires a number of technical
innovations. First, we propose a simple yet powerful generative model based on
Gaussian Mixture Models for learning the match and unmatch distributions.
Second, we propose an adaptive regularization technique customized for ER that
ameliorates the issue of feature overfitting. Finally, we incorporate the
transitivity property into the generative model in a novel way resulting in
improved accuracy. On five benchmark ER datasets, we show that ZeroER greatly
outperforms existing unsupervised approaches and achieves comparable
performance to supervised approaches.

negative:
Neural network-based methods for abstractive summarization produce outputs
that are more fluent than other techniques, but which can be poor at content
selection. This work proposes a simple technique for addressing this issue: use
a data-efficient content selector to over-determine phrases in a source
document that should be part of the summary. We use this selector as a
bottom-up attention step to constrain the model to likely phrases. We show that
this approach improves the ability to compress text, while still generating
fluent summaries. This two-step process is both simpler and higher performing
than other end-to-end content selection models, leading to significant
improvements on ROUGE for both the CNN-DM and NYT corpus. Furthermore, the
content selector can be trained with as little as 1,000 sentences, making it
easy to transfer a trained summarizer to a new domain.

negative:
In this paper, I will discuss the work I am currently doing as a Ph.D.
student at the University of Potsdam, under the tutoring of T. Schaub. I'm
currently looking into action description in ASP. More precisely, my goal is to
explore how to represent actions with durations in ASP, in different contexts.
Right now, I'm focused on Multi-Agent Path Finding (MAPF), looking at how to
represent speeds for different agents and contexts.
  Before tackling duration, I wanted to explore and compare different
representations of action taking in ASP. For this, I started comparing
different simple encodings tackling the MAPF problem. Even in simple code,
choices and assumptions have been made in their creations. The objective of my
work is to present the consequences of those design decisions in terms of
performance and knowledge representation. As far as I know, there is no current
research on this topic.
  Besides that, I'm also exploring different ways to represent duration and to
solve related problems. I planed to compare them the same way I described
before. I also want this to help me find innovative and effective ways to solve
problems with duration.

negative:
My research lies in the intersection of security and machine learning. This
overview summarizes one component of my research: combining computer vision
with malware exploit detection for enhanced security solutions. I will present
the perspectives of efficacy, reliability and resiliency to formulate threat
detection as computer vision problems and develop state-of-the-art image-based
malware classification. Representing malware binary as images provides a direct
visualization of data samples, reduces the efforts for feature extraction, and
consumes the whole binary for holistic structural analysis. Employing transfer
learning of deep neural networks effective for large scale image classification
to malware classification demonstrates superior classification efficacy
compared with classical machine learning algorithms. To enhance reliability of
these vision-based malware detectors, interpretation frameworks can be
constructed on the malware visual representations and useful for extracting
faithful explanation, so that security practitioners have confidence in the
model before deployment. In cyber-security applications, we should always
assume that a malware writer constantly modifies code to bypass detection.
Addressing the resiliency of the malware detectors is equivalently important as
efficacy and reliability. Via understanding the attack surfaces of machine
learning models used for malware detection, we can greatly improve the
robustness of the algorithms to combat malware adversaries in the wild. Finally
I will discuss future research directions worth pursuing in this research
community.

negative:
Recently we see a rising number of methods in the field of eXplainable
Artificial Intelligence. To our surprise, their development is driven by model
developers rather than a study of needs for human end users. The analysis of
needs, if done, takes the form of an A/B test rather than a study of open
questions. To answer the question "What would a human operator like to ask the
ML model?" we propose a conversational system explaining decisions of the
predictive model. In this experiment, we developed a chatbot called dr_ant to
talk about machine learning model trained to predict survival odds on Titanic.
People can talk with dr_ant about different aspects of the model to understand
the rationale behind its predictions. Having collected a corpus of 1000+
dialogues, we analyse the most common types of questions that users would like
to ask. To our knowledge, it is the first study which uses a conversational
system to collect the needs of human operators from the interactive and
iterative dialogue explorations of a predictive model.

negative:
Deciding what and when to observe is critical when making observations is
costly. In a medical setting where observations can be made sequentially,
making these observations (or not) should be an active choice. We refer to this
as the active sensing problem. In this paper, we propose a novel deep learning
framework, which we call ASAC (Active Sensing using Actor-Critic models) to
address this problem. ASAC consists of two networks: a selector network and a
predictor network. The selector network uses previously selected observations
to determine what should be observed in the future. The predictor network uses
the observations selected by the selector network to predict a label, providing
feedback to the selector network (well-selected variables should be predictive
of the label). The goal of the selector network is then to select variables
that balance the cost of observing the selected variables with their predictive
power; we wish to preserve the conditional label distribution. During training,
we use the actor-critic models to allow the loss of the selector to be
"back-propagated" through the sampling process. The selector network "acts" by
selecting future observations to make. The predictor network acts as a "critic"
by feeding predictive errors for the selected variables back to the selector
network. In our experiments, we show that ASAC significantly outperforms
state-of-the-arts in two real-world medical datasets.

negative:
Named entity linking is to map an ambiguous mention in documents to an entity
in a knowledge base. The named entity linking is challenging, given the fact
that there are multiple candidate entities for a mention in a document. It is
difficult to link a mention when it appears multiple times in a document, since
there are conflicts by the contexts around the appearances of the mention. In
addition, it is difficult since the given training dataset is small due to the
reason that it is done manually to link a mention to its mapping entity. In the
literature, there are many reported studies among which the recent embedding
methods learn vectors of entities from the training dataset at document level.
To address these issues, we focus on how to link entity for mentions at a
sentence level, which reduces the noises introduced by different appearances of
the same mention in a document at the expense of insufficient information to be
used. We propose a new unified embedding method by maximizing the relationships
learned from knowledge graphs. We confirm the effectiveness of our method in
our experimental studies.

negative:
We introduce a new task of rephrasing for a more natural virtual assistant.
Currently, virtual assistants work in the paradigm of intent slot tagging and
the slot values are directly passed as-is to the execution engine. However,
this setup fails in some scenarios such as messaging when the query given by
the user needs to be changed before repeating it or sending it to another user.
For example, for queries like 'ask my wife if she can pick up the kids' or
'remind me to take my pills', we need to rephrase the content to 'can you pick
up the kids' and 'take your pills' In this paper, we study the problem of
rephrasing with messaging as a use case and release a dataset of 3000 pairs of
original query and rephrased query. We show that BART, a pre-trained
transformers-based masked language model with auto-regressive decoding, is a
strong baseline for the task, and show improvements by adding a copy-pointer
and copy loss to it. We analyze different tradeoffs of BART-based and
LSTM-based seq2seq models, and propose a distilled LSTM-based seq2seq as the
best practical model.

negative:
We introduce the task of changing the narrative point of view, where
characters are assigned a narrative perspective that is different from the one
originally used by the writer. The resulting shift in the narrative point of
view alters the reading experience and can be used as a tool in fiction writing
or to generate types of text ranging from educational to self-help and
self-diagnosis. We introduce a benchmark dataset containing a wide range of
types of narratives annotated with changes in point of view from deictic (first
or second person) to anaphoric (third person) and describe a pipeline for
processing raw text that relies on a neural architecture for mention selection.
Evaluations on the new benchmark dataset show that the proposed architecture
substantially outperforms the baselines by generating mentions that are less
ambiguous and more natural.

negative:
Two interpretations about syllogistic statements are described in this paper.
One is the so-called set-based interpretation, which assumes that quantified
statements and syllogisms talk about quantity-relationships between sets. The
other one, the so-called conditional interpretation, assumes that quantified
propositions talk about conditional propositions and how strong are the links
between the antecedent and the consequent. Both interpretations are compared
attending to three different questions (existential import, singular statements
and non-proportional quantifiers) from the point of view of their impact on the
further development of this type of reasoning.

negative:
There is knowledge. There is belief. And there is tacit agreement.' 'We may
talk about objects. We may talk about attributes of the objects. Or we may talk
both about objects and their attributes.' This work inspects tacit agreements
on assumptions about the relation between objects and their attributes, and
studies a way of expressing them, presenting as the result what we term gradual
logic in which the sense of truth gradually shifts. It extends classical logic
instances with a new logical connective capturing the object-attribute
relation. A formal semantics is presented. Decidability is proved. Para-
consistent/epistemic/conditional/intensional/description/combined logics are
compared.

negative:
Probability theory, epistemically interpreted, provides an excellent, if not
the best available account of inductive reasoning. This is so because there are
general and definite rules for the change of subjective probabilities through
information or experience; induction and belief change are one and same topic,
after all. The most basic of these rules is simply to conditionalize with
respect to the information received; and there are similar and more general
rules. 1 Hence, a fundamental reason for the epistemological success of
probability theory is that there at all exists a well-behaved concept of
conditional probability. Still, people have, and have reasons for, various
concerns over probability theory. One of these is my starting point:
Intuitively, we have the notion of plain belief; we believe propositions2 to be
true (or to be false or neither). Probability theory, however, offers no formal
counterpart to this notion. Believing A is not the same as having probability 1
for A, because probability 1 is incorrigible3; but plain belief is clearly
corrigible. And believing A is not the same as giving A a probability larger
than some 1 - c, because believing A and believing B is usually taken to be
equivalent to believing A & B.4 Thus, it seems that the formal representation
of plain belief has to take a non-probabilistic route. Indeed, representing
plain belief seems easy enough: simply represent an epistemic state by the set
of all propositions believed true in it or, since I make the common assumption
that plain belief is deductively closed, by the conjunction of all propositions
believed true in it. But this does not yet provide a theory of induction, i.e.
an answer to the question how epistemic states so represented are changed
tbrough information or experience. There is a convincing partial answer: if the
new information is compatible with the old epistemic state, then the new
epistemic state is simply represented by the conjunction of the new information
and the old beliefs. This answer is partial because it does not cover the quite
common case where the new information is incompatible with the old beliefs. It
is, however, important to complete the answer and to cover this case, too;
otherwise, we would not represent plain belief as conigible. The crucial
problem is that there is no good completion. When epistemic states are
represented simply by the conjunction of all propositions believed true in it,
the answer cannot be completed; and though there is a lot of fruitful work, no
other representation of epistemic states has been proposed, as far as I know,
which provides a complete solution to this problem. In this paper, I want to
suggest such a solution. In [4], I have more fully argued that this is the only
solution, if certain plausible desiderata are to be satisfied. Here, in section
2, I will be content with formally defining and intuitively explaining my
proposal. I will compare my proposal with probability theory in section 3. It
will turn out that the theory I am proposing is structurally homomorphic to
probability theory in important respects and that it is thus equally easily
implementable, but moreover computationally simpler. Section 4 contains a very
brief comparison with various kinds of logics, in particular conditional logic,
with Shackle's functions of potential surprise and related theories, and with
the Dempster - Shafer theory of belief functions.

negative:
I am Joachim Jansen and this is my research summary, part of my application
to the Doctoral Consortium at ICLP'14. I am a PhD student in the Knowledge
Representation and Reasoning (KRR) research group, a subgroup of the
Declarative Languages and Artificial Intelligence (DTAI) group at the
department of Computer Science at KU Leuven. I started my PhD in September
2012. My promotor is prof. dr. ir. Gerda Janssens and my co-promotor is prof.
dr. Marc Denecker. I can be contacted at joachim.jansen@cs.kuleuven.be or at:
Room 01.167 Celestijnenlaan 200A 3001 Heverlee Belgium An extended abstract /
full version of a paper accepted to be presented at the Doctoral Consortium of
the 30th International Conference on Logic Programming (ICLP 2014), July 19-22,
Vienna, Austria

negative:
"Information Processing" is a recently launched buzzword whose meaning is
vague and obscure even for the majority of its users. The reason for this is
the lack of a suitable definition for the term "information". In my attempt to
amend this bizarre situation, I have realized that, following the insights of
Kolmogorov's Complexity theory, information can be defined as a description of
structures observable in a given data set. Two types of structures could be
easily distinguished in every data set - in this regard, two types of
information (information descriptions) should be designated: physical
information and semantic information. Kolmogorov's theory also posits that the
information descriptions should be provided as a linguistic text structure.
This inevitably leads us to an assertion that information processing has to be
seen as a kind of text processing. The idea is not new - inspired by the
observation that human information processing is deeply rooted in natural
language handling customs, Lotfi Zadeh and his followers have introduced the
so-called "Computing With Words" paradigm. Despite of promotional efforts, the
idea is not taking off yet. The reason - a lack of a coherent understanding of
what should be called "information", and, as a result, misleading research
roadmaps and objectives. I hope my humble attempt to clarify these issues would
be helpful in avoiding common traps and pitfalls.

negative:
We argue that logical semantics might have faltered due to its failure in
distinguishing between two fundamentally very different types of concepts:
ontological concepts, that should be types in a strongly-typed ontology, and
logical concepts, that are predicates corresponding to properties of and
relations between objects of various ontological types. We will then show that
accounting for these differences amounts to the integration of lexical and
compositional semantics in one coherent framework, and to an embedding in our
logical semantics of a strongly-typed ontology that reflects our commonsense
view of the world and the way we talk about it in ordinary language. We will
show that in such a framework a number of challenges in natural language
semantics can be adequately and systematically treated.