query: I am currently engaged in a project that involves identifying and isolating concealed objects in videos, with a particular emphasis on tracking their movements. I am encountering difficulties in consistently outlining the edges of camouflaged objects, especially when certain parts of the video exhibit no movement. I require a system capable of aligning video frames based on the background while simultaneously tracking target objects and ensuring consistent tracking. The intended application of this model is in nature reserves, where it will be used to monitor the living conditions of endangered species. Therefore, I am seeking a model that can effectively handle various animal types and accurately label them during video storage. Ideally, the model should be trained on a comprehensive dataset encompassing a broad spectrum of animal species. Furthermore, I am interested in understanding how to apply supervised training models to unsupervised tasks, given that some of the animals may be too rare to be included in the training dataset.

negative:
Object tracking is one of the most important problems in computer vision. The
aim of video tracking is to extract the trajectories of a target or object of
interest, i.e. accurately locate a moving target in a video sequence and
discriminate target from non-targets in the feature space of the sequence. So,
feature descriptors can have significant effects on such discrimination. In
this paper, we use the basic idea of many trackers which consists of three main
components of the reference model, i.e., object modeling, object detection and
localization, and model updating. However, there are major improvements in our
system. Our forth component, occlusion handling, utilizes the r-spatiogram to
detect the best target candidate. While spatiogram contains some moments upon
the coordinates of the pixels, r-spatiogram computes region-based compactness
on the distribution of the given feature in the image that captures richer
features to represent the objects. The proposed research develops an efficient
and robust way to keep tracking the object throughout video sequences in the
presence of significant appearance variations and severe occlusions. The
proposed method is evaluated on the Princeton RGBD tracking dataset considering
sequences with different challenges and the obtained results demonstrate the
effectiveness of the proposed method.

negative:
Tracking is one of the most important but still difficult tasks in computer
vision and pattern recognition. The main difficulties in the tracking field are
appearance variation and occlusion. Most traditional tracking methods set the
parameters or templates to track target objects in advance and should be
modified accordingly. Thus, we propose a new and robust tracking method using a
Fully Convolutional Network (FCN) to obtain an object probability map and
Dynamic Programming (DP) to seek the globally optimal path through all frames
of video. Our proposed method solves the object appearance variation problem
with the use of a FCN and deals with occlusion by DP. We show that our method
is effective in tracking various single objects through video frames.

negative:
For many years, multi-object tracking benchmarks have focused on a handful of
categories. Motivated primarily by surveillance and self-driving applications,
these datasets provide tracks for people, vehicles, and animals, ignoring the
vast majority of objects in the world. By contrast, in the related field of
object detection, the introduction of large-scale, diverse datasets (e.g.,
COCO) have fostered significant progress in developing highly robust solutions.
To bridge this gap, we introduce a similarly diverse dataset for Tracking Any
Object (TAO). It consists of 2,907 high resolution videos, captured in diverse
environments, which are half a minute long on average. Importantly, we adopt a
bottom-up approach for discovering a large vocabulary of 833 categories, an
order of magnitude more than prior tracking benchmarks. To this end, we ask
annotators to label objects that move at any point in the video, and give names
to them post factum. Our vocabulary is both significantly larger and
qualitatively different from existing tracking datasets. To ensure scalability
of annotation, we employ a federated approach that focuses manual effort on
labeling tracks for those relevant objects in a video (e.g., those that move).
We perform an extensive evaluation of state-of-the-art trackers and make a
number of important discoveries regarding large-vocabulary tracking in an
open-world. In particular, we show that existing single- and multi-object
trackers struggle when applied to this scenario in the wild, and that
detection-based, multi-object trackers are in fact competitive with
user-initialized ones. We hope that our dataset and analysis will boost further
progress in the tracking community.

negative:
Current state-of-the-art trackers only rely on a target appearance model in
order to localize the object in each frame. Such approaches are however prone
to fail in case of e.g. fast appearance changes or presence of distractor
objects, where a target appearance model alone is insufficient for robust
tracking. Having the knowledge about the presence and locations of other
objects in the surrounding scene can be highly beneficial in such cases. This
scene information can be propagated through the sequence and used to, for
instance, explicitly avoid distractor objects and eliminate target candidate
regions.
  In this work, we propose a novel tracking architecture which can utilize
scene information for tracking. Our tracker represents such information as
dense localized state vectors, which can encode, for example, if the local
region is target, background, or distractor. These state vectors are propagated
through the sequence and combined with the appearance model output to localize
the target. Our network is learned to effectively utilize the scene information
by directly maximizing tracking performance on video segments. The proposed
approach sets a new state-of-the-art on 3 tracking benchmarks, achieving an AO
score of 63.6% on the recent GOT-10k dataset.

negative:
Robots navigating autonomously need to perceive and track the motion of
objects and other agents in its surroundings. This information enables planning
and executing robust and safe trajectories. To facilitate these processes, the
motion should be perceived in 3D Cartesian space. However, most recent
multi-object tracking (MOT) research has focused on tracking people and moving
objects in 2D RGB video sequences. In this work we present JRMOT, a novel 3D
MOT system that integrates information from RGB images and 3D point clouds to
achieve real-time, state-of-the-art tracking performance. Our system is built
with recent neural networks for re-identification, 2D and 3D detection and
track description, combined into a joint probabilistic data-association
framework within a multi-modal recursive Kalman architecture. As part of our
work, we release the JRDB dataset, a novel large scale 2D+3D dataset and
benchmark, annotated with over 2 million boxes and 3500 time consistent 2D+3D
trajectories across 54 indoor and outdoor scenes. JRDB contains over 60 minutes
of data including 360 degree cylindrical RGB video and 3D pointclouds in social
settings that we use to develop, train and evaluate JRMOT. The presented 3D MOT
system demonstrates state-of-the-art performance against competing methods on
the popular 2D tracking KITTI benchmark and serves as first 3D tracking
solution for our benchmark. Real-robot tests on our social robot JackRabbot
indicate that the system is capable of tracking multiple pedestrians fast and
reliably. We provide the ROS code of our tracker at
https://sites.google.com/view/jrmot.

negative:
Object tracking is an essential task in computer vision that has been studied
since the early days of the field. Being able to follow objects that undergo
different transformations in the video sequence, including changes in scale,
illumination, shape and occlusions, makes the problem extremely difficult. One
of the real challenges is to keep track of the changes in objects appearance
and not drift towards the background clutter. Different from previous
approaches, we obtain robustness against background with a tracker model that
is composed of many different parts. They are classifiers that respond at
different scales and locations. The tracker system functions as a society of
parts, each having its own role and level of credibility. Reliable classifiers
decide the tracker's next move, while newcomers are first monitored before
gaining the necessary level of reliability to participate in the decision
process. Some parts that loose their consistency are rejected, while others
that show consistency for a sufficiently long time are promoted to permanent
roles. The tracker system, as a whole, could also go through different phases,
from the usual, normal functioning to states of weak agreement and even crisis.
The tracker system has different governing rules in each state. What truly
distinguishes our work from others is not necessarily the strength of
individual tracking parts, but the way in which they work together and build a
strong and robust organization. We also propose an efficient way to learn
simultaneously many tracking parts, with a single closed-form formulation. We
obtain a fast and robust tracker with state of the art performance on the
challenging OTB50 dataset.

negative:
Visual object tracking is an important function in many real-time video
surveillance applications, such as localization and spatio-temporal recognition
of persons. In real-world applications, an object detector and tracker must
interact on a periodic basis to discover new objects, and thereby to initiate
tracks. Periodic interactions with the detector can also allow the tracker to
validate and/or update its object template with new bounding boxes. However,
bounding boxes provided by a state-of-the-art detector are noisy, due to
changes in appearance, background and occlusion, which can cause the tracker to
drift. Moreover, CNN-based detectors can provide a high level of accuracy at
the expense of computational complexity, so interactions should be minimized
for real-time applications.
  In this paper, a new approach is proposed to manage detector-tracker
interactions for trackers from the Siamese-FC family. By integrating a change
detection mechanism into a deep Siamese-FC tracker, its template can be adapted
in response to changes in a target's appearance that lead to drifts during
tracking. An abrupt change detection triggers an update of tracker template
using the bounding box produced by the detector, while in the case of a gradual
change, the detector is used to update an evolving set of templates for robust
matching.
  Experiments were performed using state-of-the-art Siamese-FC trackers and the
YOLOv3 detector on a subset of videos from the OTB-100 dataset that mimic video
surveillance scenarios. Results highlight the importance for reliable VOT of
using accurate detectors. They also indicate that our adaptive Siamese trackers
are robust to noisy object detections, and can significantly improve the
performance of Siamese-FC tracking.

negative:
Traditionally, object tracking and segmentation are treated as two separate
problems and solved independently. However, in this paper, we argue that
tracking and segmentation are actually closely related and solving one should
help the other. On one hand, the object track, which is a set of bounding boxes
with one bounding box in every frame, would provide strong high-level guidance
for the target/background segmentation task. On the other hand, the object
segmentation would separate object from other objects and background, which
will be useful for determining track locations in every frame. We propose a
novel framework which combines online multiple target tracking and segmentation
in a video. In our approach, the tracking and segmentation problems are coupled
by Lagrange dual decomposition, which leads to more accurate segmentation
results and also \emph{helps resolve typical difficulties in multiple target
tracking, such as occlusion handling, ID-switch and track drifting}. To track
targets, an individual appearance model is learned for each target via
structured learning and network flow is employed to generate tracks from
densely sampled candidates. For segmentation, multi-label Conditional Random
Field (CRF) is applied to a superpixel based spatio-temporal graph in a segment
of video to assign background or target labels to every superpixel. The
experiments on diverse sequences show that our method outperforms
state-of-the-art approaches for multiple target tracking as well as
segmentation.

negative:
We address Unsupervised Video Object Segmentation (UVOS), the task of
automatically generating accurate pixel masks for salient objects in a video
sequence and of tracking these objects consistently through time, without any
input about which objects should be tracked. Towards solving this task, we
present UnOVOST (Unsupervised Offline Video Object Segmentation and Tracking)
as a simple and generic algorithm which is able to track and segment a large
variety of objects. This algorithm builds up tracks in a number stages, first
grouping segments into short tracklets that are spatio-temporally consistent,
before merging these tracklets into long-term consistent object tracks based on
their visual similarity. In order to achieve this we introduce a novel
tracklet-based Forest Path Cutting data association algorithm which builds up a
decision forest of track hypotheses before cutting this forest into paths that
form long-term consistent object tracks. When evaluating our approach on the
DAVIS 2017 Unsupervised dataset we obtain state-of-the-art performance with a
mean J &F score of 67.9% on the val, 58% on the test-dev and 56.4% on the
test-challenge benchmarks, obtaining first place in the DAVIS 2019 Unsupervised
Video Object Segmentation Challenge. UnOVOST even performs competitively with
many semi-supervised video object segmentation algorithms even though it is not
given any input as to which objects should be tracked and segmented.

negative:
Animals have been a common sighting on roads in India which leads to several
accidents between them and vehicles every year. This makes it vital to develop
a support system for driverless vehicles that assists in preventing these forms
of accidents. In this paper, we propose a neoteric framework for avoiding
vehicle-to-animal collisions by developing an efficient approach for the
detection of animals on highways using deep learning and computer vision
techniques on dashcam video. Our approach leverages the Mask R-CNN model for
detecting and identifying various commonly found animals. Then, we perform lane
detection to deduce whether a detected animal is on the vehicle's lane or not
and track its location and direction of movement using a centroid based object
tracking algorithm. This approach ensures that the framework is effective at
determining whether an animal is obstructing the path or not of an autonomous
vehicle in addition to predicting its movement and giving feedback accordingly.
This system was tested under various lighting and weather conditions and was
observed to perform relatively well, which leads the way for prominent
driverless vehicle's support systems for avoiding vehicular collisions with
animals on Indian roads in real-time.

negative:
The problem of video object segmentation can become extremely challenging
when multiple instances co-exist. While each instance may exhibit large scale
and pose variations, the problem is compounded when instances occlude each
other causing failures in tracking. In this study, we formulate a deep
recurrent network that is capable of segmenting and tracking objects in video
simultaneously by their temporal continuity, yet able to re-identify them when
they re-appear after a prolonged occlusion. We combine both temporal
propagation and re-identification functionalities into a single framework that
can be trained end-to-end. In particular, we present a re-identification module
with template expansion to retrieve missing objects despite their large
appearance changes. In addition, we contribute a new attention-based recurrent
mask propagation approach that is robust to distractors not belonging to the
target segment. Our approach achieves a new state-of-the-art global mean
(Region Jaccard and Boundary F measure) of 68.2 on the challenging DAVIS 2017
benchmark (test-dev set), outperforming the winning solution which achieves a
global mean of 66.1 on the same partition.

negative:
End-to-end production of object tracklets from high resolution video in
real-time and with high accuracy remains a challenging problem due to the cost
of object detection on each frame. In this work we present Localization-based
Tracking (LBT), an extension to any tracker that follows the tracking by
detection or joint detection and tracking paradigms. Localization-based
Tracking focuses only on regions likely to contain objects to boost detection
speed and avoid matching errors. We evaluate LBT as an extension to two example
trackers (KIOU and SORT) on the UA-DETRAC and MOT20 datasets. LBT-extended
trackers outperform all other reported algorithms in terms of PR-MOTA, PR-MOTP,
and mostly tracked objects on the UA-DETRAC benchmark, establishing a new
state-of-the art. relative to tracking by detection with KIOU, LBT-extended
KIOU achieves a 25% higher frame-rate and is 1.1% more accurate in terms of
PR-MOTA on the UA-DETRAC dataset. LBT-extended SORT achieves a 62% speedup and
a 3.2% increase in PR-MOTA on the UA-DETRAC dataset. On MOT20, LBT-extended
KIOU has a 50% higher frame-rate than tracking by detection and is 0.4% more
accurate in terms of MOTA. As of submission time, our LBT-extended KIOU tracker
places 10th overall on the MOT20 benchmark.

negative:
This paper presents a method, called AOGTracker, for simultaneously tracking,
learning and parsing (TLP) of unknown objects in video sequences with a
hierarchical and compositional And-Or graph (AOG) representation. %The AOG
captures both structural and appearance variations of a target object in a
principled way. The TLP method is formulated in the Bayesian framework with a
spatial and a temporal dynamic programming (DP) algorithms inferring object
bounding boxes on-the-fly. During online learning, the AOG is discriminatively
learned using latent SVM to account for appearance (e.g., lighting and partial
occlusion) and structural (e.g., different poses and viewpoints) variations of
a tracked object, as well as distractors (e.g., similar objects) in background.
Three key issues in online inference and learning are addressed: (i)
maintaining purity of positive and negative examples collected online, (ii)
controling model complexity in latent structure learning, and (iii) identifying
critical moments to re-learn the structure of AOG based on its intrackability.
The intrackability measures uncertainty of an AOG based on its score maps in a
frame. In experiments, our AOGTracker is tested on two popular tracking
benchmarks with the same parameter setting: the TB-100/50/CVPR2013 benchmarks,
and the VOT benchmarks --- VOT 2013, 2014, 2015 and TIR2015 (thermal imagery
tracking). In the former, our AOGTracker outperforms state-of-the-art tracking
algorithms including two trackers based on deep convolutional network. In the
latter, our AOGTracker outperforms all other trackers in VOT2013 and is
comparable to the state-of-the-art methods in VOT2014, 2015 and TIR2015.

negative:
In this paper, Object Detection and Tracking System (ODTS) in combination
with a well-known deep learning network, Faster Regional Convolution Neural
Network (Faster R-CNN), for Object Detection and Conventional Object Tracking
algorithm will be introduced and applied for automatic detection and monitoring
of unexpected events on CCTVs in tunnels, which are likely to (1) Wrong-Way
Driving (WWD), (2) Stop, (3) Person out of vehicle in tunnel (4) Fire. ODTS
accepts a video frame in time as an input to obtain Bounding Box (BBox) results
by Object Detection and compares the BBoxs of the current and previous video
frames to assign a unique ID number to each moving and detected object. This
system makes it possible to track a moving object in time, which is not usual
to be achieved in conventional object detection frameworks. A deep learning
model in ODTS was trained with a dataset of event images in tunnels to Average
Precision (AP) values of 0.8479, 0.7161 and 0.9085 for target objects: Car,
Person, and Fire, respectively. Then, based on trained deep learning model, the
ODTS based Tunnel CCTV Accident Detection System was tested using four accident
videos which including each accident. As a result, the system can detect all
accidents within 10 seconds. The more important point is that the detection
capacity of ODTS could be enhanced automatically without any changes in the
program codes as the training dataset becomes rich.

negative:
In this paper, we address the basic problem of recognizing moving objects in
video images using Visual Vocabulary model and Bag of Words and track our
object of interest in the subsequent video frames using species inspired PSO.
Initially, the shadow free images are obtained by background modelling followed
by foreground modeling to extract the blobs of our object of interest.
Subsequently, we train a cubic SVM with human body datasets in accordance with
our domain of interest for recognition and tracking. During training, using the
principle of Bag of Words we extract necessary features of certain domains and
objects for classification. Subsequently, matching these feature sets with
those of the extracted object blobs that are obtained by subtracting the shadow
free background from the foreground, we detect successfully our object of
interest from the test domain. The performance of the classification by cubic
SVM is satisfactorily represented by confusion matrix and ROC curve reflecting
the accuracy of each module. After classification, our object of interest is
tracked in the test domain using species inspired PSO. By combining the
adaptive learning tools with the efficient classification of description, we
achieve optimum accuracy in recognition of the moving objects. We evaluate our
algorithm benchmark datasets: iLIDS, VIVID, Walking2, Woman. Comparative
analysis of our algorithm against the existing state-of-the-art trackers shows
very satisfactory and competitive results.

negative:
We propose an automatic system for organizing the content of a collection of
unstructured videos of an articulated object class (e.g. tiger, horse). By
exploiting the recurring motion patterns of the class across videos, our
system: 1) identifies its characteristic behaviors; and 2) recovers
pixel-to-pixel alignments across different instances. Our system can be useful
for organizing video collections for indexing and retrieval. Moreover, it can
be a platform for learning the appearance or behaviors of object classes from
Internet video. Traditional supervised techniques cannot exploit this wealth of
data directly, as they require a large amount of time-consuming manual
annotations.
  The behavior discovery stage generates temporal video intervals, each
automatically trimmed to one instance of the discovered behavior, clustered by
type. It relies on our novel motion representation for articulated motion based
on the displacement of ordered pairs of trajectories (PoTs). The alignment
stage aligns hundreds of instances of the class to a great accuracy despite
considerable appearance variations (e.g. an adult tiger and a cub). It uses a
flexible Thin Plate Spline deformation model that can vary through time. We
carefully evaluate each step of our system on a new, fully annotated dataset.
On behavior discovery, we outperform the state-of-the-art Improved DTF
descriptor. On spatial alignment, we outperform the popular SIFT Flow
algorithm.

negative:
Having accurate, detailed, and up-to-date information about the location and
behavior of animals in the wild would revolutionize our ability to study and
conserve ecosystems. We investigate the ability to automatically, accurately,
and inexpensively collect such data, which could transform many fields of
biology, ecology, and zoology into "big data" sciences. Motion sensor "camera
traps" enable collecting wildlife pictures inexpensively, unobtrusively, and
frequently. However, extracting information from these pictures remains an
expensive, time-consuming, manual task. We demonstrate that such information
can be automatically extracted by deep learning, a cutting-edge type of
artificial intelligence. We train deep convolutional neural networks to
identify, count, and describe the behaviors of 48 species in the
3.2-million-image Snapshot Serengeti dataset. Our deep neural networks
automatically identify animals with over 93.8% accuracy, and we expect that
number to improve rapidly in years to come. More importantly, if our system
classifies only images it is confident about, our system can automate animal
identification for 99.3% of the data while still performing at the same 96.6%
accuracy as that of crowdsourced teams of human volunteers, saving more than
8.4 years (at 40 hours per week) of human labeling effort (i.e. over 17,000
hours) on this 3.2-million-image dataset. Those efficiency gains immediately
highlight the importance of using deep neural networks to automate data
extraction from camera-trap images. Our results suggest that this technology
could enable the inexpensive, unobtrusive, high-volume, and even real-time
collection of a wealth of information about vast numbers of animals in the
wild.

negative:
Predicting on-road abnormalities such as road accidents or traffic violations
is a challenging task in traffic surveillance. If such predictions can be done
in advance, many damages can be controlled. Here in our wok, we tried to
formulate a solution for automated collision prediction in traffic surveillance
videos with computer vision and deep networks. It involves object detection,
tracking, trajectory estimation, and collision prediction. We propose an
end-to-end collision prediction system, named as COLLIDE-PRED, that
intelligently integrates the information of past and future trajectories of
moving objects to predict collisions in videos. It is a pipeline that starts
with object detection, which is used for object tracking, and then trajectory
prediction is performed which concludes by collision detection. The probable
place of collision, and the objects those may cause the collision, both can be
identified correctly with COLLIDE-PRED. The proposed method is experimentally
validated with a number of different videos and proves to be effective in
identifying accident in advance.

negative:
In this paper, we propose a novel approach for exploiting structural
relations to track multiple objects that may undergo long-term occlusion and
abrupt motion. We use a model-free approach that relies only on annotations
given in the first frame of the video to track all the objects online, i.e.
without knowledge from future frames. We initialize a probabilistic Attributed
Relational Graph (ARG) from the first frame, which is incrementally updated
along the video. Instead of using the structural information only to evaluate
the scene, the proposed approach considers it to generate new tracking
hypotheses. In this way, our method is capable of generating relevant object
candidates that are used to improve or recover the track of lost objects. The
proposed method is evaluated on several videos of table tennis, volleyball, and
on the ACASVA dataset. The results show that our approach is very robust,
flexible and able to outperform other state-of-the-art methods in sports videos
that present structural patterns.

negative:
The Discriminative Correlation Filter (CF) uses a circulant convolution
operation to provide several training samples for the design of a classifier
that can distinguish the target from the background. The filter design may be
interfered by objects close to the target during the tracking process,
resulting in tracking failure. This paper proposes an adaptive distraction
context aware tracking algorithm to solve this problem. In the response map
obtained for the previous frame by the CF algorithm, we adaptively find the
image blocks that are similar to the target and use them as negative samples.
This diminishes the influence of similar image blocks on the classifier in the
tracking process and its accuracy is improved. The tracking results on video
sequences show that the algorithm can cope with rapid changes such as occlusion
and rotation, and can adaptively use the distractive objects around the target
as negative samples to improve the accuracy of target tracking.

negative:
This paper presents the development of a real time tracking algorithm that
runs on a 1.2 GHz PC/104 computer on-board a small UAV. The algorithm uses zero
mean normalized cross correlation to detect and locate an object in the image.
A kalman filter is used to make the tracking algorithm computationally
efficient. Object position in an image frame is predicted using the motion
model and a search window, centered at the predicted position is generated.
Object position is updated with the measurement from object detection. The
detected position is sent to the motion controller to move the gimbal so that
the object stays at the center of the image frame. Detection and tracking is
autonomously carried out on the payload computer and the system is able to work
in two different methods. The first method starts detecting and tracking using
a stored image patch. The second method allows the operator on the ground to
select the interest object for the UAV to track. The system is capable of
re-detecting an object, in the event of tracking failure. Performance of the
tracking system was verified both in the lab and on the field by mounting the
payload on a vehicle and simulating a flight. Tests show that the system can
detect and track a diverse set of objects in real time. Flight testing of the
system will be conducted at the next available opportunity.

negative:
This paper proposes a novel approach to create an automated visual
surveillance system which is very efficient in detecting and tracking moving
objects in a video captured by moving camera without any apriori information
about the captured scene. Separating foreground from the background is
challenging job in videos captured by moving camera as both foreground and
background information change in every consecutive frames of the image
sequence; thus a pseudo-motion is perceptive in background. In the proposed
algorithm, the pseudo-motion in background is estimated and compensated using
phase correlation of consecutive frames based on the principle of Fourier shift
theorem. Then a method is proposed to model an acting background from recent
history of commonality of the current frame and the foreground is detected by
the differences between the background model and the current frame. Further
exploiting the recent history of dissimilarities of the current frame, actual
moving objects are detected in the foreground. Next, a two-stepped
morphological operation is proposed to refine the object region for an optimum
object size. Each object is attributed by its centroid, dimension and three
highest peaks of its gray value histogram. Finally, each object is tracked
using Kalman filter based on its attributes. The major advantage of this
algorithm over most of the existing object detection and tracking algorithms is
that, it does not require initialization of object position in the first frame
or training on sample data to perform. Performance of the algorithm is tested
on benchmark videos containing variable background and very satisfiable results
is achieved. The performance of the algorithm is also comparable with some of
the state-of-the-art algorithms for object detection and tracking.

negative:
In this paper we propose a novel approach for detecting and tracking objects
in videos with variable background i.e. videos captured by moving cameras
without any additional sensor. In a video captured by a moving camera, both the
background and foreground are changing in each frame of the image sequence. So
for these videos, modeling a single background with traditional background
modeling methods is infeasible and thus the detection of actual moving object
in a variable background is a challenging task. To detect actual moving object
in this work, spatio-temporal blobs have been generated in each frame by
spatio-temporal analysis of the image sequence using a three-dimensional Gabor
filter. Then individual blobs, which are parts of one object are merged using
Minimum Spanning Tree to form the moving object in the variable background. The
height, width and four-bin gray-value histogram of the object are calculated as
its features and an object is tracked in each frame using these features to
generate the trajectories of the object through the video sequence. In this
work, problem of data association during tracking is solved by Linear
Assignment Problem and occlusion is handled by the application of kalman
filter. The major advantage of our method over most of the existing tracking
algorithms is that, the proposed method does not require initialization in the
first frame or training on sample data to perform. Performance of the algorithm
has been tested on benchmark videos and very satisfactory result has been
achieved. The performance of the algorithm is also comparable and superior with
respect to some benchmark algorithms.

negative:
We propose an unsupervised visual tracking method in this paper. Different
from existing approaches using extensive annotated data for supervised
learning, our CNN model is trained on large-scale unlabeled videos in an
unsupervised manner. Our motivation is that a robust tracker should be
effective in both the forward and backward predictions (i.e., the tracker can
forward localize the target object in successive frames and backtrace to its
initial position in the first frame). We build our framework on a Siamese
correlation filter network, which is trained using unlabeled raw videos.
Meanwhile, we propose a multiple-frame validation method and a cost-sensitive
loss to facilitate unsupervised learning. Without bells and whistles, the
proposed unsupervised tracker achieves the baseline accuracy of fully
supervised trackers, which require complete and accurate labels during
training. Furthermore, unsupervised framework exhibits a potential in
leveraging unlabeled or weakly labeled data to further improve the tracking
accuracy.

negative:
The study of animal behavior increasingly relies on (semi-) automatic methods
for the extraction of relevant behavioral features from video or picture data.
To date, several specialized software products exist to detect and track
animals' positions in simple (laboratory) environments. Tracking animals in
their natural environments, however, often requires substantial customization
of the image processing algorithms to the problem-specific image
characteristics. Here we introduce BioTracker, an open-source computer vision
framework, that provides programmers with core functionalities that are
essential parts of a tracking software, such as video I/O, graphics overlays
and mouse and keyboard interfaces. BioTracker additionally provides a number of
different tracking algorithms suitable for a variety of image recording
conditions. The main feature of BioTracker is however the straightforward
implementation of new problem-specific tracking modules and vision algorithms
that can build upon BioTracker's core functionalities. With this open-source
framework the scientific community can accelerate their research and focus on
the development of new vision algorithms.

negative:
Wildlife monitoring is crucial to nature conservation and has been done by
manual observations from motion-triggered camera traps deployed in the field.
Widespread adoption of such in-situ sensors has resulted in unprecedented data
volumes being collected over the last decade. A significant challenge exists to
process and reliably identify what is in these images efficiently. Advances in
computer vision are poised to provide effective solutions with custom AI models
built to automatically identify images of interest and label the species in
them. Here we outline the data unification effort for the Wildlife Insights
platform from various conservation partners, and the challenges involved. Then
we present an initial deep convolutional neural network model, trained on 2.9M
images across 465 fine-grained species, with a goal to reduce the load on human
experts to classify species in images manually. The long-term goal is to enable
scientists to make conservation recommendations from near real-time analysis of
species abundance and population health.

negative:
A robust visual tracking system requires an object appearance model that is
able to handle occlusion, pose, and illumination variations in the video
stream. This can be difficult to accomplish when the model is trained using
only a single image. In this paper, we first propose a tracking approach based
on affine subspaces (constructed from several images) which are able to
accommodate the abovementioned variations. We use affine subspaces not only to
represent the object, but also the candidate areas that the object may occupy.
We furthermore propose a novel approach to measure affine subspace-to-subspace
distance via the use of non-Euclidean geometry of Grassmann manifolds. The
tracking problem is then considered as an inference task in a Markov Chain
Monte Carlo framework via particle filtering. Quantitative evaluation on
challenging video sequences indicates that the proposed approach obtains
considerably better performance than several recent state-of-the-art methods
such as Tracking-Learning-Detection and MILtrack.

negative:
Multi-object tracking from RGB-D video sequences is a challenging problem due
to the combination of changing viewpoints, motion, and occlusions over time. We
observe that having the complete geometry of objects aids in their tracking,
and thus propose to jointly infer the complete geometry of objects as well as
track them, for rigidly moving objects over time. Our key insight is that
inferring the complete geometry of the objects significantly helps in tracking.
By hallucinating unseen regions of objects, we can obtain additional
correspondences between the same instance, thus providing robust tracking even
under strong change of appearance. From a sequence of RGB-D frames, we detect
objects in each frame and learn to predict their complete object geometry as
well as a dense correspondence mapping into a canonical space. This allows us
to derive 6DoF poses for the objects in each frame, along with their
correspondence between frames, providing robust object tracking across the
RGB-D sequence. Experiments on both synthetic and real-world RGB-D data
demonstrate that we achieve state-of-the-art performance on dynamic object
tracking. Furthermore, we show that our object completion significantly helps
tracking, providing an improvement of $6.5\%$ in mean MOTA.

negative:
This paper presents a review of human activity recognition and behaviour
understanding in video sequence. The key objective of this paper is to provide
a general review on the overall process of a surveillance system used in the
current trend. Visual surveillance system is directed on automatic
identification of events of interest, especially on tracking and classification
of moving objects. The processing step of the video surveillance system
includes the following stages: Surrounding model, object representation, object
tracking, activity recognition and behaviour understanding. It describes
techniques that use to define a general set of activities that are applicable
to a wide range of scenes and environments in video sequence.

negative:
The ability to detect and track objects in the visual world is a crucial
skill for any intelligent agent, as it is a necessary precursor to any
object-level reasoning process. Moreover, it is important that agents learn to
track objects without supervision (i.e. without access to annotated training
videos) since this will allow agents to begin operating in new environments
with minimal human assistance. The task of learning to discover and track
objects in videos, which we call \textit{unsupervised object tracking}, has
grown in prominence in recent years; however, most architectures that address
it still struggle to deal with large scenes containing many objects. In the
current work, we propose an architecture that scales well to the large-scene,
many-object setting by employing spatially invariant computations (convolutions
and spatial attention) and representations (a spatially local object
specification scheme). In a series of experiments, we demonstrate a number of
attractive features of our architecture; most notably, that it outperforms
competing methods at tracking objects in cluttered scenes with many objects,
and that it can generalize well to videos that are larger and/or contain more
objects than videos encountered during training.

negative:
Tracking by detection, the dominant approach for online multi-object
tracking, alternates between localization and association steps. As a result,
it strongly depends on the quality of instantaneous observations, often failing
when objects are not fully visible. In contrast, tracking in humans is
underlined by the notion of object permanence: once an object is recognized, we
are aware of its physical existence and can approximately localize it even
under full occlusions. In this work, we introduce an end-to-end trainable
approach for joint object detection and tracking that is capable of such
reasoning. We build on top of the recent CenterTrack architecture, which takes
pairs of frames as input, and extend it to videos of arbitrary length. To this
end, we augment the model with a spatio-temporal, recurrent memory module,
allowing it to reason about object locations and identities in the current
frame using all the previous history. It is, however, not obvious how to train
such an approach. We study this question on a new, large-scale, synthetic
dataset for multi-object tracking, which provides ground truth annotations for
invisible objects, and propose several approaches for supervising tracking
behind occlusions. Our model, trained jointly on synthetic and real data,
outperforms the state of the art on KITTI and MOT17 datasets thanks to its
robustness to occlusions.

negative:
Understanding visual reality involves acquiring common-sense knowledge about
countless regularities in the visual world, e.g., how illumination alters the
appearance of objects in a scene, and how motion changes their apparent spatial
relationship. These regularities are hard to label for training supervised
machine learning algorithms; consequently, algorithms need to learn these
regularities from the real world in an unsupervised way. We present a novel
network meta-architecture that can learn world dynamics from raw, continuous
video. The components of this network can be implemented using any algorithm
that possesses three key capabilities: prediction of a signal over time,
reduction of signal dimensionality (compression), and the ability to use
supplementary contextual information to inform the prediction. The presented
architecture is highly-parallelized and scalable, and is implemented using
localized connectivity, processing, and learning. We demonstrate an
implementation of this architecture where the components are built from
multi-layer perceptrons. We apply the implementation to create a system capable
of stable and robust visual tracking of objects as seen by a moving camera.
Results show performance on par with or exceeding state-of-the-art tracking
algorithms. The tracker can be trained in either fully supervised or
unsupervised-then-briefly-supervised regimes. Success of the briefly-supervised
regime suggests that the unsupervised portion of the model extracts useful
information about visual reality. The results suggest a new class of AI
algorithms that uniquely combine prediction and scalability in a way that makes
them suitable for learning from and --- and eventually acting within --- the
real world.

negative:
The advancement of visual tracking has continuously been brought by deep
learning models. Typically, supervised learning is employed to train these
models with expensive labeled data. In order to reduce the workload of manual
annotations and learn to track arbitrary objects, we propose an unsupervised
learning method for visual tracking. The motivation of our unsupervised
learning is that a robust tracker should be effective in bidirectional
tracking. Specifically, the tracker is able to forward localize a target object
in successive frames and backtrace to its initial position in the first frame.
Based on such a motivation, in the training process, we measure the consistency
between forward and backward trajectories to learn a robust tracker from
scratch merely using unlabeled videos. We build our framework on a Siamese
correlation filter network, and propose a multi-frame validation scheme and a
cost-sensitive loss to facilitate unsupervised learning. Without bells and
whistles, the proposed unsupervised tracker achieves the baseline accuracy as
classic fully supervised trackers while achieving a real-time speed.
Furthermore, our unsupervised framework exhibits a potential in leveraging more
unlabeled or weakly labeled data to further improve the tracking accuracy.

negative:
Recently, the Kernelized Correlation Filters tracker (KCF) achieved
competitive performance and robustness in visual object tracking. On the other
hand, visual trackers are not typically used in multiple object tracking. In
this paper, we investigate how a robust visual tracker like KCF can improve
multiple object tracking. Since KCF is a fast tracker, many can be used in
parallel and still result in fast tracking. We build a multiple object tracking
system based on KCF and background subtraction. Background subtraction is
applied to extract moving objects and get their scale and size in combination
with KCF outputs, while KCF is used for data association and to handle
fragmentation and occlusion problems. As a result, KCF and background
subtraction help each other to take tracking decision at every frame. Sometimes
KCF outputs are the most trustworthy (e.g. during occlusion), while in some
other case, it is the background subtraction outputs. To validate the
effectiveness of our system, the algorithm is demonstrated on four urban video
recordings from a standard dataset. Results show that our method is competitive
with state-of-the-art trackers even if we use a much simpler data association
step.

negative:
Recent interest in self-supervised dense tracking has yielded rapid progress,
but performance still remains far from supervised methods. We propose a dense
tracking model trained on videos without any annotations that surpasses
previous self-supervised methods on existing benchmarks by a significant margin
(+15%), and achieves performance comparable to supervised methods. In this
paper, we first reassess the traditional choices used for self-supervised
training and reconstruction loss by conducting thorough experiments that
finally elucidate the optimal choices. Second, we further improve on existing
methods by augmenting our architecture with a crucial memory component. Third,
we benchmark on large-scale semi-supervised video object segmentation(aka.
dense tracking), and propose a new metric: generalizability. Our first two
contributions yield a self-supervised network that for the first time is
competitive with supervised methods on standard evaluation metrics of dense
tracking. When measuring generalizability, we show self-supervised approaches
are actually superior to the majority of supervised methods. We believe this
new generalizability metric can better capture the real-world use-cases for
dense tracking, and will spur new interest in this research direction.

negative:
Visual animal biometrics is rapidly gaining popularity as it enables a
non-invasive and cost-effective approach for wildlife monitoring applications.
Widespread usage of camera traps has led to large volumes of collected images,
making manual processing of visual content hard to manage. In this work, we
develop a framework for automatic detection and recognition of individuals in
different patterned species like tigers, zebras and jaguars. Most existing
systems primarily rely on manual input for localizing the animal, which does
not scale well to large datasets. In order to automate the detection process
while retaining robustness to blur, partial occlusion, illumination and pose
variations, we use the recently proposed Faster-RCNN object detection framework
to efficiently detect animals in images. We further extract features from
AlexNet of the animal's flank and train a logistic regression (or Linear SVM)
classifier to recognize the individuals. We primarily test and evaluate our
framework on a camera trap tiger image dataset that contains images that vary
in overall image quality, animal pose, scale and lighting. We also evaluate our
recognition system on zebra and jaguar images to show generalization to other
patterned species. Our framework gives perfect detection results in camera
trapped tiger images and a similar or better individual recognition performance
when compared with state-of-the-art recognition techniques.

negative:
Anomaly detection in surveillance videos is currently a challenge because of
the diversity of possible events. We propose a deep convolutional neural
network (CNN) that addresses this problem by learning a correspondence between
common object appearances (e.g. pedestrian, background, tree, etc.) and their
associated motions. Our model is designed as a combination of a reconstruction
network and an image translation model that share the same encoder. The former
sub-network determines the most significant structures that appear in video
frames and the latter one attempts to associate motion templates to such
structures. The training stage is performed using only videos of normal events
and the model is then capable to estimate frame-level scores for an unknown
input. The experiments on 6 benchmark datasets demonstrate the competitive
performance of the proposed approach with respect to state-of-the-art methods.

negative:
Quantifying behavior is crucial for many applications in neuroscience.
Videography provides easy methods for the observation and recording of animal
behavior in diverse settings, yet extracting particular aspects of a behavior
for further analysis can be highly time consuming. In motor control studies,
humans or other animals are often marked with reflective markers to assist with
computer-based tracking, yet markers are intrusive (especially for smaller
animals), and the number and location of the markers must be determined a
priori. Here, we present a highly efficient method for markerless tracking
based on transfer learning with deep neural networks that achieves excellent
results with minimal training data. We demonstrate the versatility of this
framework by tracking various body parts in a broad collection of experimental
settings: mice odor trail-tracking, egg-laying behavior in drosophila, and
mouse hand articulation in a skilled forelimb task. For example, during the
skilled reaching behavior, individual joints can be automatically tracked (and
a confidence score is reported). Remarkably, even when a small number of frames
are labeled ($\approx 200$), the algorithm achieves excellent tracking
performance on test frames that is comparable to human accuracy.

negative:
Egocentric vision is an emerging field of computer vision that is
characterized by the acquisition of images and video from the first person
perspective. In this paper we address the challenge of egocentric human action
recognition by utilizing the presence and position of detected regions of
interest in the scene explicitly, without further use of visual features.
  Initially, we recognize that human hands are essential in the execution of
actions and focus on obtaining their movements as the principal cues that
define actions. We employ object detection and region tracking techniques to
locate hands and capture their movements. Prior knowledge about egocentric
views facilitates hand identification between left and right. With regard to
detection and tracking, we contribute a pipeline that successfully operates on
unseen egocentric videos to find the camera wearer's hands and associate them
through time. Moreover, we emphasize on the value of scene information for
action recognition. We acknowledge that the presence of objects is significant
for the execution of actions by humans and in general for the description of a
scene. To acquire this information, we utilize object detection for specific
classes that are relevant to the actions we want to recognize.
  Our experiments are targeted on videos of kitchen activities from the
Epic-Kitchens dataset. We model action recognition as a sequence learning
problem of the detected spatial positions in the frames. Our results show that
explicit hand and object detections with no other visual information can be
relied upon to classify hand-related human actions. Testing against methods
fully dependent on visual features, signals that for actions where hand motions
are conceptually important, a region-of-interest-based description of a video
contains equally expressive information with comparable classification
performance.

negative:
Visual tracking addresses the problem of identifying and localizing an
unknown target in a video given the target specified by a bounding box in the
first frame. In this paper, we propose a dual network to better utilize
features among layers for visual tracking. It is observed that features in
higher layers encode semantic context while its counterparts in lower layers
are sensitive to discriminative appearance. Thus we exploit the hierarchical
features in different layers of a deep model and design a dual structure to
obtain better feature representation from various streams, which is rarely
investigated in previous work. To highlight geometric contours of the target,
we integrate the hierarchical feature maps with an edge detector as the coarse
prior maps to further embed local details around the target. To leverage the
robustness of our dual network, we train it with random patches measuring the
similarities between the network activation and target appearance, which serves
as a regularization to enforce the dual network to focus on target object. The
proposed dual network is updated online in a unique manner based on the
observation that the target being tracked in consecutive frames should share
more similar feature representations than those in the surrounding background.
It is also found that for a target object, the prior maps can help further
enhance performance by passing message into the output maps of the dual
network. Therefore, an independent component analysis with reference algorithm
(ICA-R) is employed to extract target context using prior maps as guidance.
Online tracking is conducted by maximizing the posterior estimate on the final
maps with stochastic and periodic update. Quantitative and qualitative
evaluations on two large-scale benchmark data sets show that the proposed
algorithm performs favourably against the state-of-the-arts.

negative:
In this paper, we present a new method for detecting road users in an urban
environment which leads to an improvement in multiple object tracking. Our
method takes as an input a foreground image and improves the object detection
and segmentation. This new image can be used as an input to trackers that use
foreground blobs from background subtraction. The first step is to create
foreground images for all the frames in an urban video. Then, starting from the
original blobs of the foreground image, we merge the blobs that are close to
one another and that have similar optical flow. The next step is extracting the
edges of the different objects to detect multiple objects that might be very
close (and be merged in the same blob) and to adjust the size of the original
blobs. At the same time, we use the optical flow to detect occlusion of objects
that are moving in opposite directions. Finally, we make a decision on which
information we keep in order to construct a new foreground image with blobs
that can be used for tracking. The system is validated on four videos of an
urban traffic dataset. Our method improves the recall and precision metrics for
the object detection task compared to the vanilla background subtraction method
and improves the CLEAR MOT metrics in the tracking tasks for most videos.

negative:
In this paper, we propose and study a novel visual object tracking approach
based on convolutional networks and recurrent networks. The proposed approach
is distinct from the existing approaches to visual object tracking, such as
filtering-based ones and tracking-by-detection ones, in the sense that the
tracking system is explicitly trained off-line to track anonymous objects in a
noisy environment. The proposed visual tracking model is end-to-end trainable,
minimizing any adversarial effect from mismatches in object representation and
between the true underlying dynamics and learning dynamics. We empirically show
that the proposed tracking approach works well in various scenarios by
generating artificial video sequences with varying conditions; the number of
objects, amount of noise and the match between the training shapes and test
shapes.

negative:
In this paper, the task of video panoptic segmentation is studied and two
different methods to solve the task will be proposed. Video panoptic
segmentation (VPS) is a recently introduced computer vision task that requires
classifying and tracking every pixel in a given video. The nature of this task
makes the cost of annotating datasets for it prohibiting. To understand video
panoptic segmentation, first, earlier introduced constituent tasks that focus
on semantics and tracking separately will be researched. Thereafter, two
data-driven approaches which do not require training on a tailored VPS dataset
will be selected to solve it. The first approach will show how a model for
video panoptic segmentation can be built by heuristically fusing the outputs of
a pre-trained semantic segmentation model and a pre-trained multi-object
tracking model. This can be desired if one wants to easily extend the
capabilities of either model. The second approach will counter some of the
shortcomings of the first approach by building on top of a shared neural
network backbone with task-specific heads. This network is designed for
panoptic segmentation and will be extended by a mask propagation module to link
instance masks across time, yielding the video panoptic segmentation format.

negative:
We propose an end-to-end learning framework for segmenting generic objects in
videos. Our method learns to combine appearance and motion information to
produce pixel level segmentation masks for all prominent objects in videos. We
formulate this task as a structured prediction problem and design a two-stream
fully convolutional neural network which fuses together motion and appearance
in a unified framework. Since large-scale video datasets with pixel level
segmentations are problematic, we show how to bootstrap weakly annotated videos
together with existing image recognition datasets for training. Through
experiments on three challenging video segmentation benchmarks, our method
substantially improves the state-of-the-art for segmenting generic (unseen)
objects. Code and pre-trained models are available on the project website.

negative:
We present the first systematic study on concealed object detection (COD),
which aims to identify objects that are "perfectly" embedded in their
background. The high intrinsic similarities between the concealed objects and
their background make COD far more challenging than traditional object
detection/segmentation. To better understand this task, we collect a
large-scale dataset, called COD10K, which consists of 10,000 images covering
concealed objects in diverse real-world scenarios from 78 object categories.
Further, we provide rich annotations including object categories, object
boundaries, challenging attributes, object-level labels, and instance-level
annotations. Our COD10K is the largest COD dataset to date, with the richest
annotations, which enables comprehensive concealed object understanding and can
even be used to help progress several other vision tasks, such as detection,
segmentation, classification, etc. Motivated by how animals hunt in the wild,
we also design a simple but strong baseline for COD, termed the Search
Identification Network (SINet). Without any bells and whistles, SINet
outperforms 12 cutting-edge baselines on all datasets tested, making them
robust, general architectures that could serve as catalysts for future research
in COD. Finally, we provide some interesting findings and highlight several
potential applications and future directions. To spark research in this new
field, our code, dataset, and online demo are available on our project page:
http://mmcheng.net/cod.

negative:
The world is composed of objects, the ground, and the sky. Visual perception
of objects requires solving two fundamental challenges: segmenting visual input
into discrete units, and tracking identities of these units despite appearance
changes due to object deformation, changing perspective, and dynamic occlusion.
Current computer vision approaches to segmentation and tracking that approach
human performance all require learning, raising the question: can objects be
segmented and tracked without learning? Here, we show that the mathematical
structure of light rays reflected from environment surfaces yields a natural
representation of persistent surfaces, and this surface representation provides
a solution to both the segmentation and tracking problems. We describe how to
generate this surface representation from continuous visual input, and
demonstrate that our approach can segment and invariantly track objects in
cluttered synthetic video despite severe appearance changes, without requiring
learning.

negative:
This paper introduces an automated vision system for animal detection in
trail-camera images taken from a field under the administration of the Texas
Parks and Wildlife Department. As traditional wildlife counting techniques are
intrusive and labor intensive to conduct, trail-camera imaging is a
comparatively non-intrusive method for capturing wildlife activity. However,
given the large volume of images produced from trail-cameras, manual analysis
of the images remains time-consuming and inefficient. We implemented a
two-stage deep convolutional neural network pipeline to find animal-containing
images in the first stage and then process these images to detect birds in the
second stage. The animal classification system classifies animal images with
overall 93% sensitivity and 96% specificity. The bird detection system achieves
better than 93% sensitivity, 92% specificity, and 68% average
Intersection-over-Union rate. The entire pipeline processes an image in less
than 0.5 seconds as opposed to an average 30 seconds for a human labeler. We
also addressed post-deployment issues related to data drift for the animal
classification system as image features vary with seasonal changes. This system
utilizes an automatic retraining algorithm to detect data drift and update the
system. We introduce a novel technique for detecting drifted images and
triggering the retraining procedure. Two statistical experiments are also
presented to explain the prediction behavior of the animal classification
system. These experiments investigate the cues that steers the system towards a
particular decision. Statistical hypothesis testing demonstrates that the
presence of an animal in the input image significantly contributes to the
system's decisions.

negative:
This paper addresses the problem of tracking moving objects of variable
appearance in challenging scenes rich with features and texture. Reliable
tracking is of pivotal importance in surveillance applications. It is made
particularly difficult by the nature of objects encountered in such scenes:
these too change in appearance and scale, and are often articulated (e.g.
humans). We propose a method which uses fast motion detection and segmentation
as a constraint for both building appearance models and their robust
propagation (matching) in time. The appearance model is based on sets of local
appearances automatically clustered using spatio-kinetic similarity, and is
updated with each new appearance seen. This integration of all seen appearances
of a tracked object makes it extremely resilient to errors caused by occlusion
and the lack of permanence of due to low data quality, appearance change or
background clutter. These theoretical strengths of our algorithm are
empirically demonstrated on two hour long video footage of a busy city
marketplace.

negative:
From video streaming to security and surveillance applications, video data
play an important role in our daily living today. However, managing a large
amount of video data and retrieving the most useful information for the user
remain a challenging task. In this paper, we propose a novel video
classification system that would benefit the scene understanding task. We
define our classification problem as classifying background and foreground
motions using the same feature representation for outdoor scenes. This means
that the feature representation needs to be robust enough and adaptable to
different classification tasks. We propose a lightweight Loss Switching Fusion
Network (LSFNet) for the fusion of spatiotemporal descriptors and a similarity
search scheme with soft voting to boost the classification performance. The
proposed system has a variety of potential applications such as content-based
video clustering, video filtering, etc. Evaluation results on two private
industry datasets show that our system is robust in both classifying different
background motions and detecting human motions from these background motions.

negative:
In this paper, an online adaptive model-free tracker is proposed to track
single objects in video sequences to deal with real-world tracking challenges
like low-resolution, object deformation, occlusion and motion blur. The novelty
lies in the construction of a strong appearance model that captures features
from the initialized bounding box and then are assembled into anchor-point
features. These features memorize the global pattern of the object and have an
internal star graph-like structure. These features are unique and flexible and
helps tracking generic and deformable objects with no limitation on specific
objects. In addition, the relevance of each feature is evaluated online using
short-term consistency and long-term consistency. These parameters are adapted
to retain consistent features that vote for the object location and that deal
with outliers for long-term tracking scenarios. Additionally, voting in a
Gaussian manner helps in tackling inherent noise of the tracking system and in
accurate object localization. Furthermore, the proposed tracker uses pairwise
distance measure to cope with scale variations and combines pixel-level binary
features and global weighted color features for model update. Finally,
experimental results on a visual tracking benchmark dataset are presented to
demonstrate the effectiveness and competitiveness of the proposed tracker.

negative:
In this paper we propose a system capable of tracking multiple soccer players
in different types of video quality. The main goal, in contrast to most
state-of-art soccer player tracking systems, is the ability of execute
effectively tracking in videos of low-quality. We adapted a state-of-art
Multiple Object Tracking to the task. In order to do that adaptation, we
created a Detection and a Tracking Dataset for 3 different qualities of video.
The results of our system are conclusive of its high performance.

negative:
Tracking moving objects from a video sequence requires segmentation of these
objects from the background image. However, getting the actual background image
automatically without object detection and using only the video is difficult.
In this paper, we describe a novel algorithm that generates background from
real world images without foreground detection. The algorithm assumes that the
background image is shown in the majority of the video. Given this simple
assumption, the method described in this paper is able to accurately generate,
with high probability, the background image from a video using only a small
number of binary operations.

negative:
Class-agnostic object tracking is particularly difficult in cluttered
environments as target specific discriminative models cannot be learned a
priori. Inspired by how the human visual cortex employs spatial attention and
separate "where" and "what" processing pathways to actively suppress irrelevant
visual features, this work develops a hierarchical attentive recurrent model
for single object tracking in videos. The first layer of attention discards the
majority of background by selecting a region containing the object of interest,
while the subsequent layers tune in on visual features particular to the
tracked object. This framework is fully differentiable and can be trained in a
purely data driven fashion by gradient methods. To improve training
convergence, we augment the loss function with terms for a number of auxiliary
tasks relevant for tracking. Evaluation of the proposed model is performed on
two datasets: pedestrian tracking on the KTH activity recognition dataset and
the more difficult KITTI object tracking dataset.

negative:
Being heavily reliant on animals, it is our ethical obligation to improve
their well-being by understanding their needs. Several studies show that animal
needs are often expressed through their faces. Though remarkable progress has
been made towards the automatic understanding of human faces, this has
regrettably not been the case with animal faces. There exists significant room
and appropriate need to develop automatic systems capable of interpreting
animal faces. Among many transformative impacts, such a technology will foster
better and cheaper animal healthcare, and further advance animal psychology
understanding.
  We believe the underlying research progress is mainly obstructed by the lack
of an adequately annotated dataset of animal faces, covering a wide spectrum of
animal species. To this end, we introduce a large-scale, hierarchical annotated
dataset of animal faces, featuring 21.9K faces from 334 diverse species and 21
animal orders across biological taxonomy. These faces are captured
`in-the-wild' conditions and are consistently annotated with 9 landmarks on key
facial features. The proposed dataset is structured and scalable by design; its
development underwent four systematic stages involving rigorous, manual
annotation effort of over 6K man-hours. We benchmark it for face alignment
using the existing art under novel problem settings. Results showcase its
challenging nature, unique attributes and present definite prospects for novel,
adaptive, and generalized face-oriented CV algorithms. We further benchmark the
dataset for face detection and fine-grained recognition tasks, to demonstrate
multi-task applications and room for improvement. Experiments indicate that
this dataset will push the algorithmic advancements across many related CV
tasks and encourage the development of novel systems for animal facial
behaviour monitoring. We will make the dataset publicly available.

negative:
Photographs of wild animals in their natural habitats can be recorded
unobtrusively via cameras that are triggered by motion nearby. The installation
of such camera traps is becoming increasingly common across the world. Although
this is a convenient source of invaluable data for biologists, ecologists and
conservationists, the arduous task of poring through potentially millions of
pictures each season introduces prohibitive costs and frustrating delays. We
develop automatic algorithms that are able to detect animals, identify the
species of animals and to recognize individual animals for two species. we
propose the first fully-automatic tool that can recognize specific individuals
of leopard and tiger due to their characteristic body markings. We adopt a
class of supervised learning approach of machine learning where a Deep
Convolutional Neural Network (DCNN) is trained using several instances of
manually-labelled images for each of the three classification tasks. We
demonstrate the effectiveness of our approach on a data set of camera-trap
images recorded in the jungles of Southern India.

negative:
We propose the first multi-frame video object detection framework trained to
detect great apes. It is applicable to challenging camera trap footage in
complex jungle environments and extends a traditional feature pyramid
architecture by adding self-attention driven feature blending in both the
spatial as well as the temporal domain. We demonstrate that this extension can
detect distinctive species appearance and motion signatures despite significant
partial occlusion. We evaluate the framework using 500 camera trap videos of
great apes from the Pan African Programme containing 180K frames, which we
manually annotated with accurate per-frame animal bounding boxes. These clips
contain significant partial occlusions, challenging lighting, dynamic
backgrounds, and natural camouflage effects. We show that our approach performs
highly robustly and significantly outperforms frame-based detectors. We also
perform detailed ablation studies and validation on the full ILSVRC 2015 VID
data corpus to demonstrate wider applicability at adequate performance levels.
We conclude that the framework is ready to assist human camera trap inspection
efforts. We publish code, weights, and ground truth annotations with this
paper.

negative:
Animal pose estimation has recently come into the limelight due to its
application in biology, zoology, and aquaculture. Deep learning methods have
effectively been applied to human pose estimation. However, the major
bottleneck to the application of these methods to animal pose estimation is the
unavailability of sufficient quantities of labeled data. Though there are ample
quantities of unlabelled data publicly available, it is economically
impractical to label large quantities of data for each animal. In addition, due
to the wide variety of body shapes in the animal kingdom, the transfer of
knowledge across domains is ineffective. Given the fact that the human brain is
able to recognize animal pose without requiring large amounts of labeled data,
it is only reasonable that we exploit unsupervised learning to tackle the
problem of animal pose recognition from the available, unlabelled data. In this
paper, we introduce a novel architecture that is able to recognize the pose of
multiple animals fromunlabelled data. We do this by (1) removing background
information from each image and employing an edge detection algorithm on the
body of the animal, (2) Tracking motion of the edge pixels and performing
agglomerative clustering to segment body parts, (3) employing contrastive
learning to discourage grouping of distant body parts together. Hence we are
able to distinguish between body parts of the animal, based on their visual
behavior, instead of the underlying anatomy. Thus, we are able to achieve a
more effective classification of the data than their human-labeled
counterparts. We test our model on the TigDog and WLD (WildLife Documentary)
datasets, where we outperform state-of-the-art approaches by a significant
margin. We also study the performance of our model on other public data to
demonstrate the generalization ability of our model.

negative:
Satellite video cameras can provide continuous observation for a large-scale
area, which is important for many remote sensing applications. However,
achieving moving object detection and tracking in satellite videos remains
challenging due to the insufficient appearance information of objects and lack
of high-quality datasets. In this paper, we first build a large-scale satellite
video dataset with rich annotations for the task of moving object detection and
tracking. This dataset is collected by the Jilin-1 satellite constellation and
composed of 47 high-quality videos with 1,646,038 instances of interest for
object detection and 3,711 trajectories for object tracking. We then introduce
a motion modeling baseline to improve the detection rate and reduce false
alarms based on accumulative multi-frame differencing and robust matrix
completion. Finally, we establish the first public benchmark for moving object
detection and tracking in satellite videos, and extensively evaluate the
performance of several representative approaches on our dataset. Comprehensive
experimental analyses and insightful conclusions are also provided. The dataset
is available at https://github.com/QingyongHu/VISO.

negative:
The problem of tracking multiple objects in a video sequence poses several
challenging tasks. For tracking-by-detection, these include object
re-identification, motion prediction and dealing with occlusions. We present a
tracker (without bells and whistles) that accomplishes tracking without
specifically targeting any of these tasks, in particular, we perform no
training or optimization on tracking data. To this end, we exploit the bounding
box regression of an object detector to predict the position of an object in
the next frame, thereby converting a detector into a Tracktor. We demonstrate
the potential of Tracktor and provide a new state-of-the-art on three
multi-object tracking benchmarks by extending it with a straightforward
re-identification and camera motion compensation. We then perform an analysis
on the performance and failure cases of several state-of-the-art tracking
methods in comparison to our Tracktor. Surprisingly, none of the dedicated
tracking methods are considerably better in dealing with complex tracking
scenarios, namely, small and occluded objects or missing detections. However,
our approach tackles most of the easy tracking scenarios. Therefore, we
motivate our approach as a new tracking paradigm and point out promising future
research directions. Overall, Tracktor yields superior tracking performance
than any current tracking method and our analysis exposes remaining and
unsolved tracking challenges to inspire future research directions.

negative:
Target tracking in hyperspectral videos is a new research topic. In this
paper, a novel method based on convolutional network and Kernelized Correlation
Filter (KCF) framework is presented for tracking objects of interest in
hyperspectral videos. We extract a set of normalized three-dimensional cubes
from the target region as fixed convolution filters which contain spectral
information surrounding a target. The feature maps generated by convolutional
operations are combined to form a three-dimensional representation of an
object, thereby providing effective encoding of local spectral-spatial
information. We show that a simple two-layer convolutional networks is
sufficient to learn robust representations without the need of offline training
with a large dataset. In the tracking step, KCF is adopted to distinguish
targets from neighboring environment. Experimental results demonstrate that the
proposed method performs well on sample hyperspectral videos, and outperforms
several state-of-the-art methods tested on grayscale and color videos in the
same scene.

negative:
Understanding the abundance of a species is the first step towards
understanding both its long-term sustainability and the impact that we may be
having upon it. Ecologists use camera traps to remotely survey for the presence
of specific animal species. Previous studies have shown that deep learning
models can be trained to automatically detect and classify animals within
camera trap imagery with high levels of confidence. However, the ability to
train these models is reliant upon having enough high-quality training data.
What happens when the animal is rare or the data sets are non-existent? This
research proposes an approach of using images of rare animals in captivity
(focusing on the Scottish wildcat) to generate the training dataset. We explore
the challenges associated with generalising a model trained on captivity data
when applied to data collected in the wild. The research is contextualised by
the needs of ecologists in planning/engineering. Following precedents from
other research, this project establishes an ensemble for object detection,
image segmentation and image classification models which are then tested using
different image manipulation and class structuring techniques to encourage
model generalisation. The research concludes, in the context of Scottish
wildcat, that models trained on captivity imagery cannot be generalised to wild
camera trap imagery using existing techniques. However, final model
performances based on a two-class model Wildcat vs Not Wildcat achieved an
overall accuracy score of 81.6% and Wildcat accuracy score of 54.8% on a test
set in which only 1% of images contained a wildcat. This suggests using
captivity images is feasible with further research. This is the first research
which attempts to generate a training set based on captivity data and the first
to explore the development of such models in the context of ecologists in
planning/engineering.

negative:
By assuming that the most frequently occuring color in a video or a region of
a video I propose a new algorithm for detecting foreground objects in a video.
The process of detecting the foreground objects is complicated because of the
fact that there may be swaying trees, objects of the background being moved
around or lighting changes in the video. To deal with such complexities many
have come up with solutions which heavily rely on expensive floating point
operations. In this paper I used a data structure called Octree which is
implemented only using binary operations. Traditionally octrees were used for
color quantization but here in this paper I used it as a data structure to
store the most frequently occuring colors in a video as well. For each of the
starting few video frames, I constructed a Octree using all the colors of that
frame. Next I pruned all the trees by removing nodes below a certain height and
gave the leaf nodes a color which is dependant on the topological path from
that node to its parent. Hence any two leaf nodes in two different octrees with
the same topological path from themselves to the root will represent the same
color. Next I merged all these individual trees into a single tree retaining
only those nodes whose topological path to itself from the root is most common
among all the trees. The colors represented by the leaf nodes in the resultant
tree will be the most frequently occuring colors in the starting video frames
of the video. Hence any color of an incomming frame that is not close to any of
the colors represented by the leaf node of the merged tree can be regarded as
belonging to a foreground object.
  As an Octree is constructed using only binary operations, it is very fast
compared to other leading algorithms.

negative:
Motion models play a great role in visual tracking applications for
predicting the possible locations of objects in the next frame. Unlike target
tracking in radar or aerospace domain which considers only points, object
tracking in computer vision involves sizes of objects. Constant velocity motion
model is the most widely used motion model for visual tracking, however, there
is no clear and understandable derivation involving sizes of objects specially
for new researchers joining this research field. In this document, we derive
the constant velocity motion model that incorporates sizes of objects that, we
think, can help the new researchers to adapt to it very quickly.

negative:
Object tracking is one of the most challenging task and has secured
significant attention of computer vision researchers in the past two decades.
Recent deep learning based trackers have shown good performance on various
tracking challenges. A tracking method should track objects in sequential
frames accurately in challenges such as deformation, low resolution, occlusion,
scale and light variations. Most trackers achieve good performance on specific
challenges instead of all tracking problems, hence there is a lack of general
purpose tracking algorithms that can perform well in all conditions. Moreover,
performance of tracking techniques has not been evaluated in noisy
environments. Visual object tracking has real world applications and there is
good chance that noise may get added during image acquisition in surveillance
cameras. We aim to study the robustness of two state of the art trackers in the
presence of noise including Efficient Convolutional Operators (ECO) and
Correlation Filter Network (CFNet). Our study demonstrates that the performance
of these trackers degrades as the noise level increases, which demonstrate the
need to design more robust tracking algorithms.

negative:
This paper investigates long-term face tracking of a specific person given
his/her face image in a single frame as a query in a video stream. Through
taking advantage of pre-trained deep learning models on big data, a novel
system is developed for accurate video face tracking in the unconstrained
environments depicting various people and objects moving in and out of the
frame. In the proposed system, we present a detection-verification-tracking
method (dubbed as 'DVT') which accomplishes the long-term face tracking task
through the collaboration of face detection, face verification, and
(short-term) face tracking. An offline trained detector based on cascaded
convolutional neural networks localizes all faces appeared in the frames, and
an offline trained face verifier based on deep convolutional neural networks
and similarity metric learning decides if any face or which face corresponds to
the queried person. An online trained tracker follows the face from frame to
frame. When validated on a sitcom episode and a TV show, the DVT method
outperforms tracking-learning-detection (TLD) and face-TLD in terms of recall
and precision. The proposed system is also tested on many other types of videos
and shows very promising results.

negative:
For multi-target tracking, target representation plays a crucial rule in
performance. State-of-the-art approaches rely on the deep learning-based visual
representation that gives an optimal performance at the cost of high
computational complexity. In this paper, we come up with a simple yet effective
target representation for human tracking. Our inspiration comes from the fact
that the human body goes through severe deformation and inter/intra occlusion
over the passage of time. So, instead of tracking the whole body part, a
relative rigid organ tracking is selected for tracking the human over an
extended period of time. Hence, we followed the tracking-by-detection paradigm
and generated the target hypothesis of only the spatial locations of heads in
every frame. After the localization of head location, a Kalman filter with a
constant velocity motion model is instantiated for each target that follows the
temporal evolution of the targets in the scene. For associating the targets in
the consecutive frames, combinatorial optimization is used that associates the
corresponding targets in a greedy fashion. Qualitative results are evaluated on
four challenging video surveillance dataset and promising results has been
achieved.

negative:
The primary aim of this project is to build a contextual Question-Answering
model for videos. The current methodologies provide a robust model for image
based Question-Answering, but we are aim to generalize this approach to be
videos. We propose a graphical representation of video which is able to handle
several types of queries across the whole video. For example, if a frame has an
image of a man and a cat sitting, it should be able to handle queries like,
where is the cat sitting with respect to the man? or ,what is the man holding
in his hand?. It should be able to answer queries relating to temporal
relationships also.

negative:
In the field of bioimaging, an important part of analyzing the motion of
objects is tracking. We propose a method that applies the Sinkhorn distance for
solving the optimal transport problem to track objects. The advantage of this
method is that it can flexibly incorporate various assumptions in tracking as a
cost matrix. First, we extend the Sinkhorn distance from two dimensions to
three dimensions. Using this three-dimensional distance, we compare the
performance of two types of tracking technique, namely tracking that associates
objects that are close to each other, which conventionally uses the
nearest-neighbor method, and tracking that assumes that the object is moving at
constant velocity, using three types of simulation data. The results suggest
that when tracking objects moving at constant velocity, our method is superior
to conventional nearest-neighbor tracking as long as the added noise is not
excessively large. We show that the Sinkhorn method can be applied effectively
to object tracking. Our simulation data analysis suggests that when objects are
moving at constant velocity, our method, which sets acceleration as a cost,
outperforms the traditional nearest-neighbor method in terms of tracking
objects. To apply the proposed method to real bioimaging data, it is necessary
to set an appropriate cost indicator based on the movement features.

negative:
Camera trapping is increasingly used to monitor wildlife, but this technology
typically requires extensive data annotation. Recently, deep learning has
significantly advanced automatic wildlife recognition. However, current methods
are hampered by a dependence on large static data sets when wildlife data is
intrinsically dynamic and involves long-tailed distributions. These two
drawbacks can be overcome through a hybrid combination of machine learning and
humans in the loop. Our proposed iterative human and automated identification
approach is capable of learning from wildlife imagery data with a long-tailed
distribution. Additionally, it includes self-updating learning that facilitates
capturing the community dynamics of rapidly changing natural systems. Extensive
experiments show that our approach can achieve a ~90% accuracy employing only
~20% of the human annotations of existing approaches. Our synergistic
collaboration of humans and machines transforms deep learning from a relatively
inefficient post-annotation tool to a collaborative on-going annotation tool
that vastly relieves the burden of human annotation and enables efficient and
constant model updates.

negative:
Moving object detection and tracking have various applications, including
surveillance, anomaly detection, vehicle navigation, etc. The literature on
object detection and tracking is rich enough, and several essential survey
papers exist. However, the research on camouflage object detection and tracking
limited due to the complexity of the problem. Existing work on this problem has
been done based on either biological characteristics of the camouflaged objects
or computer vision techniques. In this article, we review the existing
camouflaged object detection and tracking techniques using computer vision
algorithms from the theoretical point of view. This article also addresses
several issues of interest as well as future research direction on this area.
We hope this review will help the reader to learn the recent advances in
camouflaged object detection and tracking.

negative:
Animals have evolved highly functional visual systems to understand motion,
assisting perception even under complex environments. In this paper, we work
towards developing a computer vision system able to segment objects by
exploiting motion cues, i.e. motion segmentation. We make the following
contributions: First, we introduce a simple variant of the Transformer to
segment optical flow frames into primary objects and the background. Second, we
train the architecture in a self-supervised manner, i.e. without using any
manual annotations. Third, we analyze several critical components of our method
and conduct thorough ablation studies to validate their necessity. Fourth, we
evaluate the proposed architecture on public benchmarks (DAVIS2016, SegTrackv2,
and FBMS59). Despite using only optical flow as input, our approach achieves
superior or comparable results to previous state-of-the-art self-supervised
methods, while being an order of magnitude faster. We additionally evaluate on
a challenging camouflage dataset (MoCA), significantly outperforming the other
self-supervised approaches, and comparing favourably to the top supervised
approach, highlighting the importance of motion cues, and the potential bias
towards visual appearance in existing video segmentation models.

negative:
The robust detection of small targets against cluttered background is
important for future artificial visual systems in searching and tracking
applications. The insects' visual systems have demonstrated excellent ability
to avoid predators, find prey or identify conspecifics - which always appear as
small dim speckles in the visual field. Build a computational model of the
insects' visual pathways could provide effective solutions to detect small
moving targets. Although a few visual system models have been proposed, they
only make use of small-field visual features for motion detection and their
detection results often contain a number of false positives. To address this
issue, we develop a new visual system model for small target motion detection
against cluttered moving backgrounds. Compared to the existing models, the
small-field and wide-field visual features are separately extracted by two
motion-sensitive neurons to detect small target motion and background motion.
These two types of motion information are further integrated to filter out
false positives. Extensive experiments showed that the proposed model can
outperform the existing models in terms of detection rates.

negative:
We present the first method to perform automatic 3D pose, shape and texture
capture of animals from images acquired in-the-wild. In particular, we focus on
the problem of capturing 3D information about Grevy's zebras from a collection
of images. The Grevy's zebra is one of the most endangered species in Africa,
with only a few thousand individuals left. Capturing the shape and pose of
these animals can provide biologists and conservationists with information
about animal health and behavior. In contrast to research on human pose, shape
and texture estimation, training data for endangered species is limited, the
animals are in complex natural scenes with occlusion, they are naturally
camouflaged, travel in herds, and look similar to each other. To overcome these
challenges, we integrate the recent SMAL animal model into a network-based
regression pipeline, which we train end-to-end on synthetically generated
images with pose, shape, and background variation. Going beyond
state-of-the-art methods for human shape and pose estimation, our method learns
a shape space for zebras during training. Learning such a shape space from
images using only a photometric loss is novel, and the approach can be used to
learn shape in other settings with limited 3D supervision. Moreover, we couple
3D pose and shape prediction with the task of texture synthesis, obtaining a
full texture map of the animal from a single image. We show that the predicted
texture map allows a novel per-instance unsupervised optimization over the
network features. This method, SMALST (SMAL with learned Shape and Texture)
goes beyond previous work, which assumed manual keypoints and/or segmentation,
to regress directly from pixels to 3D animal shape, pose and texture. Code and
data are available at https://github.com/silviazuffi/smalst.

negative:
In this paper, we are interested in pose estimation of animals. Animals
usually exhibit a wide range of variations on poses and there is no available
animal pose dataset for training and testing. To address this problem, we build
an animal pose dataset to facilitate training and evaluation. Considering the
heavy labor needed to label dataset and it is impossible to label data for all
concerned animal species, we, therefore, proposed a novel cross-domain
adaptation method to transform the animal pose knowledge from labeled animal
classes to unlabeled animal classes. We use the modest animal pose dataset to
adapt learned knowledge to multiple animals species. Moreover, humans also
share skeleton similarities with some animals (especially four-footed mammals).
Therefore, the easily available human pose dataset, which is of a much larger
scale than our labeled animal dataset, provides important prior knowledge to
boost up the performance on animal pose estimation. Experiments show that our
proposed method leverages these pieces of prior knowledge well and achieves
convincing results on animal pose estimation.

negative:
Self-driving vehicle vision systems must deal with an extremely broad and
challenging set of scenes. They can potentially exploit an enormous amount of
training data collected from vehicles in the field, but the volumes are too
large to train offline naively. Not all training instances are equally valuable
though, and importance sampling can be used to prioritize which training images
to collect. This approach assumes that objects in images are labeled with high
accuracy. To generate accurate labels in the field, we exploit the
spatio-temporal coherence of vehicle video. We use a near-to-far labeling
strategy by first labeling large, close objects in the video, and tracking them
back in time to induce labels on small distant presentations of those objects.
In this paper we demonstrate the feasibility of this approach in several steps.
First, we note that an optimal subset (relative to all the objects encountered
and labeled) of labeled objects in images can be obtained by importance
sampling using gradients of the recognition network. Next we show that these
gradients can be approximated with very low error using the loss function,
which is already available when the CNN is running inference. Then, we
generalize these results to objects in a larger scene using an object detection
system. Finally, we describe a self-labeling scheme using object tracking.
Objects are tracked back in time (near-to-far) and labels of near objects are
used to check accuracy of those objects in the far field. We then evaluate the
accuracy of models trained on importance sampled data vs models trained on
complete data.

negative:
The demand for on-device document recognition systems increases in
conjunction with the emergence of more strict privacy and security
requirements. In such systems, there is no data transfer from the end device to
a third-party information processing servers. The response time is vital to the
user experience of on-device document recognition. Combined with the
unavailability of discrete GPUs, powerful CPUs, or a large RAM capacity on
consumer-grade end devices such as smartphones, the time limitations put
significant constraints on the computational complexity of the applied
algorithms for on-device execution.
  In this work, we consider document location in an image without prior
knowledge of the document content or its internal structure. In accordance with
the published works, at least 5 systems offer solutions for on-device document
location. All these systems use a location method which can be considered
Hough-based. The precision of such systems seems to be lower than that of the
state-of-the-art solutions which were not designed to account for the limited
computational resources.
  We propose an advanced Hough-based method. In contrast with other approaches,
it accounts for the geometric invariants of the central projection model and
combines both edge and color features for document boundary detection. The
proposed method allowed for the second best result for SmartDoc dataset in
terms of precision, surpassed by U-net like neural network. When evaluated on a
more challenging MIDV-500 dataset, the proposed algorithm guaranteed the best
precision compared to published methods. Our method retained the applicability
to on-device computations.

negative:
We introduce a novel type of text representation that preserves the 2D layout
of a document. This is achieved by encoding each document page as a
two-dimensional grid of characters. Based on this representation, we present a
generic document understanding pipeline for structured documents. This pipeline
makes use of a fully convolutional encoder-decoder network that predicts a
segmentation mask and bounding boxes. We demonstrate its capabilities on an
information extraction task from invoices and show that it significantly
outperforms approaches based on sequential text or document images.

negative:
In this paper, we present supervision-by-registration, an unsupervised
approach to improve the precision of facial landmark detectors on both images
and video. Our key observation is that the detections of the same landmark in
adjacent frames should be coherent with registration, i.e., optical flow.
Interestingly, the coherency of optical flow is a source of supervision that
does not require manual labeling, and can be leveraged during detector
training. For example, we can enforce in the training loss function that a
detected landmark at frame$_{t-1}$ followed by optical flow tracking from
frame$_{t-1}$ to frame$_t$ should coincide with the location of the detection
at frame$_t$. Essentially, supervision-by-registration augments the training
loss function with a registration loss, thus training the detector to have
output that is not only close to the annotations in labeled images, but also
consistent with registration on large amounts of unlabeled videos. End-to-end
training with the registration loss is made possible by a differentiable
Lucas-Kanade operation, which computes optical flow registration in the forward
pass, and back-propagates gradients that encourage temporal coherency in the
detector. The output of our method is a more precise image-based facial
landmark detector, which can be applied to single images or video. With
supervision-by-registration, we demonstrate (1) improvements in facial landmark
detection on both images (300W, ALFW) and video (300VW, Youtube-Celebrities),
and (2) significant reduction of jittering in video detections.

negative:
Despite significant progress on current state-of-the-art image generation
models, synthesis of document images containing multiple and complex object
layouts is a challenging task. This paper presents a novel approach, called
DocSynth, to automatically synthesize document images based on a given layout.
In this work, given a spatial layout (bounding boxes with object categories) as
a reference by the user, our proposed DocSynth model learns to generate a set
of realistic document images consistent with the defined layout. Also, this
framework has been adapted to this work as a superior baseline model for
creating synthetic document image datasets for augmenting real data during
training for document layout analysis tasks. Different sets of learning
objectives have been also used to improve the model performance.
Quantitatively, we also compare the generated results of our model with real
data using standard evaluation metrics. The results highlight that our model
can successfully generate realistic and diverse document images with multiple
objects. We also present a comprehensive qualitative analysis summary of the
different scopes of synthetic image generation tasks. Lastly, to our knowledge
this is the first work of its kind.

negative:
Segmentation of additive manufacturing (AM) defects in X-ray Computed
Tomography (XCT) images is challenging, due to the poor contrast, small sizes
and variation in appearance of defects. Automatic segmentation can, however,
provide quality control for additive manufacturing. Over recent years,
three-dimensional convolutional neural networks (3D CNNs) have performed well
in the volumetric segmentation of medical images. In this work, we leverage
techniques from the medical imaging domain and propose training a 3D U-Net
model to automatically segment defects in XCT images of AM samples. This work
not only contributes to the use of machine learning for AM defect detection but
also demonstrates for the first time 3D volumetric segmentation in AM. We train
and test with three variants of the 3D U-Net on an AM dataset, achieving a mean
intersection of union (IOU) value of 88.4%.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
A lot of research has been devoted to identity documents analysis and
recognition on mobile devices. However, no publicly available datasets designed
for this particular problem currently exist. There are a few datasets which are
useful for associated subtasks but in order to facilitate a more comprehensive
scientific and technical approach to identity document recognition more
specialized datasets are required. In this paper we present a Mobile Identity
Document Video dataset (MIDV-500) consisting of 500 video clips for 50
different identity document types with ground truth which allows to perform
research in a wide scope of document analysis problems. The paper presents
characteristics of the dataset and evaluation results for existing methods of
face detection, text line recognition, and document fields data extraction.
Since an important feature of identity documents is their sensitiveness as they
contain personal data, all source document images used in MIDV-500 are either
in public domain or distributed under public copyright licenses.
  The main goal of this paper is to present a dataset. However, in addition and
as a baseline, we present evaluation results for existing methods for face
detection, text line recognition, and document data extraction, using the
presented dataset.
  (The dataset is available for download at ftp://smartengines.com/midv-500/.)

negative:
We study the problem of segmenting moving objects in unconstrained videos.
Given a video, the task is to segment all the objects that exhibit independent
motion in at least one frame. We formulate this as a learning problem and
design our framework with three cues: (i) independent object motion between a
pair of frames, which complements object recognition, (ii) object appearance,
which helps to correct errors in motion estimation, and (iii) temporal
consistency, which imposes additional constraints on the segmentation. The
framework is a two-stream neural network with an explicit memory module. The
two streams encode appearance and motion cues in a video sequence respectively,
while the memory module captures the evolution of objects over time, exploiting
the temporal consistency. The motion stream is a convolutional neural network
trained on synthetic videos to segment independently moving objects in the
optical flow field. The module to build a 'visual memory' in video, i.e., a
joint representation of all the video frames, is realized with a convolutional
recurrent unit learned from a small number of training video sequences.
  For every pixel in a frame of a test video, our approach assigns an object or
background label based on the learned spatio-temporal features as well as the
'visual memory' specific to the video. We evaluate our method extensively on
three benchmarks, DAVIS, Freiburg-Berkeley motion segmentation dataset and
SegTrack. In addition, we provide an extensive ablation study to investigate
both the choice of the training data and the influence of each component in the
proposed framework.

negative:
The digital camera captured document images may often be warped and distorted
due to different camera angles or document surfaces. A robust technique is
needed to solve this kind of distortion. The research on dewarping of the
document suffers due to the limited availability of benchmark public dataset.
In recent times, deep learning based approaches are used to solve the problems
accurately. To train most of the deep neural networks a large number of
document images is required and generating such a large volume of document
images manually is difficult. In this paper, we propose a technique to generate
a synthetic warped image from a flat-bedded scanned document image. It is done
by calculating warping factors for each pixel position using two warping
position parameters (WPP) and eight warping control parameters (WCP). These
parameters can be specified as needed depending upon the desired warping. The
results are compared with similar real captured images both qualitative and
quantitative way.

negative:
Additive Manufacturing (AM) is a crucial component of the smart industry. In
this paper, we propose an automated quality grading system for the AM process
using a deep convolutional neural network (CNN) model. The CNN model is trained
offline using the images of the internal and surface defects in the
layer-by-layer deposition of materials and tested online by studying the
performance of detecting and classifying the failure in AM process at different
extruder speeds and temperatures. The model demonstrates the accuracy of 94%
and specificity of 96%, as well as above 75% in three classifier measures of
the Fscore, the sensitivity, and precision for classifying the quality of the
printing process in five grades in real-time. The proposed online model adds an
automated, consistent, and non-contact quality control signal to the AM process
that eliminates the manual inspection of parts after they are entirely built.
The quality monitoring signal can also be used by the machine to suggest
remedial actions by adjusting the parameters in real-time. The proposed quality
predictive model serves as a proof-of-concept for any type of AM machines to
produce reliable parts with fewer quality hiccups while limiting the waste of
both time and materials.

negative:
Computation of document image quality metrics often depends upon the
availability of a ground truth image corresponding to the document. This limits
the applicability of quality metrics in applications such as hyperparameter
optimization of image processing algorithms that operate on-the-fly on unseen
documents. This work proposes the use of surrogate models to learn the behavior
of a given document quality metric on existing datasets where ground truth
images are available. The trained surrogate model can later be used to predict
the metric value on previously unseen document images without requiring access
to ground truth images. The surrogate model is empirically evaluated on the
Document Image Binarization Competition (DIBCO) and the Handwritten Document
Image Binarization Competition (H-DIBCO) datasets.

negative:
This paper presents results of Document Visual Question Answering Challenge
organized as part of "Text and Documents in the Deep Learning Era" workshop, in
CVPR 2020. The challenge introduces a new problem - Visual Question Answering
on document images. The challenge comprised two tasks. The first task concerns
with asking questions on a single document image. On the other hand, the second
task is set as a retrieval task where the question is posed over a collection
of images. For the task 1 a new dataset is introduced comprising 50,000
questions-answer(s) pairs defined over 12,767 document images. For task 2
another dataset has been created comprising 20 questions over 14,362 document
images which share the same document template.

negative:
Convolutional neural networks perform well on object recognition because of a
number of recent advances: rectified linear units (ReLUs), data augmentation,
dropout, and large labelled datasets. Unsupervised data has been proposed as
another way to improve performance. Unfortunately, unsupervised pre-training is
not used by state-of-the-art methods leading to the following question: Is
unsupervised pre-training still useful given recent advances? If so, when? We
answer this in three parts: we 1) develop an unsupervised method that
incorporates ReLUs and recent unsupervised regularization techniques, 2)
analyze the benefits of unsupervised pre-training compared to data augmentation
and dropout on CIFAR-10 while varying the ratio of unsupervised to supervised
samples, 3) verify our findings on STL-10. We discover unsupervised
pre-training, as expected, helps when the ratio of unsupervised to supervised
samples is high, and surprisingly, hurts when the ratio is low. We also use
unsupervised pre-training with additional color augmentation to achieve near
state-of-the-art performance on STL-10.

negative:
Disentanglement is at the forefront of unsupervised learning, as disentangled
representations of data improve generalization, interpretability, and
performance in downstream tasks. Current unsupervised approaches remain
inapplicable for real-world datasets since they are highly variable in their
performance and fail to reach levels of disentanglement of (semi-)supervised
approaches. We introduce population-based training (PBT) for improving
consistency in training variational autoencoders (VAEs) and demonstrate the
validity of this approach in a supervised setting (PBT-VAE). We then use
Unsupervised Disentanglement Ranking (UDR) as an unsupervised heuristic to
score models in our PBT-VAE training and show how models trained this way tend
to consistently disentangle only a subset of the generative factors. Building
on top of this observation we introduce the recursive rPU-VAE approach. We
train the model until convergence, remove the learned factors from the dataset
and reiterate. In doing so, we can label subsets of the dataset with the
learned factors and consecutively use these labels to train one model that
fully disentangles the whole dataset. With this approach, we show striking
improvement in state-of-the-art unsupervised disentanglement performance and
robustness across multiple datasets and metrics.

negative:
Imagine trying to track one particular fruitfly in a swarm of hundreds.
Higher biological visual systems have evolved to track moving objects by
relying on both appearance and motion features. We investigate if
state-of-the-art deep neural networks for visual tracking are capable of the
same. For this, we introduce PathTracker, a synthetic visual challenge that
asks human observers and machines to track a target object in the midst of
identical-looking "distractor" objects. While humans effortlessly learn
PathTracker and generalize to systematic variations in task design,
state-of-the-art deep networks struggle. To address this limitation, we
identify and model circuit mechanisms in biological brains that are implicated
in tracking objects based on motion cues. When instantiated as a recurrent
network, our circuit model learns to solve PathTracker with a robust visual
strategy that rivals human performance and explains a significant proportion of
their decision-making on the challenge. We also show that the success of this
circuit model extends to object tracking in natural videos. Adding it to a
transformer-based architecture for object tracking builds tolerance to visual
nuisances that affect object appearance, resulting in a new state-of-the-art
performance on the large-scale TrackingNet object tracking challenge. Our work
highlights the importance of building artificial vision models that can help us
better understand human vision and improve computer vision.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Class labels have been empirically shown useful in improving the sample
quality of generative adversarial nets (GANs). In this paper, we mathematically
study the properties of the current variants of GANs that make use of class
label information. With class aware gradient and cross-entropy decomposition,
we reveal how class labels and associated losses influence GAN's training.
Based on that, we propose Activation Maximization Generative Adversarial
Networks (AM-GAN) as an advanced solution. Comprehensive experiments have been
conducted to validate our analysis and evaluate the effectiveness of our
solution, where AM-GAN outperforms other strong baselines and achieves
state-of-the-art Inception Score (8.91) on CIFAR-10. In addition, we
demonstrate that, with the Inception ImageNet classifier, Inception Score
mainly tracks the diversity of the generator, and there is, however, no
reliable evidence that it can reflect the true sample quality. We thus propose
a new metric, called AM Score, to provide a more accurate estimation of the
sample quality. Our proposed model also outperforms the baseline methods in the
new metric.

negative:
In the Gastric Histopathology Image Classification (GHIC) tasks, which are
usually weakly supervised learning missions, there is inevitably redundant
information in the images. Therefore, designing networks that can focus on
effective distinguishing features has become a popular research topic. In this
paper, to accomplish the tasks of GHIC superiorly and to assist pathologists in
clinical diagnosis, an intelligent Hierarchical Conditional Random Field based
Attention Mechanism (HCRF-AM) model is proposed. The HCRF-AM model consists of
an Attention Mechanism (AM) module and an Image Classification (IC) module. In
the AM module, an HCRF model is built to extract attention regions. In the IC
module, a Convolutional Neural Network (CNN) model is trained with the
attention regions selected and then an algorithm called Classification
Probability-based Ensemble Learning is applied to obtain the image-level
results from patch-level output of the CNN. In the experiment, a classification
specificity of 96.67% is achieved on a gastric histopathology dataset with 700
images. Our HCRF-AM model demonstrates high classification performance and
shows its effectiveness and future potential in the GHIC field.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
Document segmentation is a method of rending the document into distinct
regions. A document is an assortment of information and a standard mode of
conveying information to others. Pursuance of data from documents involves ton
of human effort, time intense and might severely prohibit the usage of data
systems. So, automatic information pursuance from the document has become a big
issue. It is been shown that document segmentation will facilitate to beat such
problems. This paper proposes a new approach to segment and classify the
document regions as text, image, drawings and table. Document image is divided
into blocks using Run length smearing rule and features are extracted from
every blocks. Discipulus tool has been used to construct the Genetic
programming based classifier model and located 97.5% classification accuracy.

negative:
What happens if one pushes a cup sitting on a table toward the edge of the
table? How about pushing a desk against a wall? In this paper, we study the
problem of understanding the movements of objects as a result of applying
external forces to them. For a given force vector applied to a specific
location in an image, our goal is to predict long-term sequential movements
caused by that force. Doing so entails reasoning about scene geometry, objects,
their attributes, and the physical rules that govern the movements of objects.
We design a deep neural network model that learns long-term sequential
dependencies of object movements while taking into account the geometry and
appearance of the scene by combining Convolutional and Recurrent Neural
Networks. Training our model requires a large-scale dataset of object movements
caused by external forces. To build a dataset of forces in scenes, we
reconstructed all images in SUN RGB-D dataset in a physics simulator to
estimate the physical movements of objects caused by external forces applied to
them. Our Forces in Scenes (ForScene) dataset contains 10,335 images in which a
variety of external forces are applied to different types of objects resulting
in more than 65,000 object movements represented in 3D. Our experimental
evaluations show that the challenging task of predicting long-term movements of
objects as their reaction to external forces is possible from a single image.

negative:
One of the major prerequisites for any deep learning approach is the
availability of large-scale training data. When dealing with scanned document
images in real world scenarios, the principal information of its content is
stored in the layout itself. In this work, we have proposed an automated deep
generative model using Graph Neural Networks (GNNs) to generate synthetic data
with highly variable and plausible document layouts that can be used to train
document interpretation systems, in this case, specially in digital mailroom
applications. It is also the first graph-based approach for document layout
generation task experimented on administrative document images, in this case,
invoices.

negative:
Video understanding is one of the most challenging topics in computer vision.
In this paper, a four-stage video understanding pipeline is presented to
simultaneously recognize all atomic actions and the single on-going activity in
a video. This pipeline uses objects and motions from the video and a
graph-based knowledge representation network as prior reference. Two deep
networks are trained to identify objects and motions in each video sequence
associated with an action. Low Level image features are then used to identify
objects of interest in that video sequence. Confidence scores are assigned to
objects of interest based on their involvement in the action and to motion
classes based on results from a deep neural network that classifies the
on-going action in video into motion classes. Confidence scores are computed
for each candidate functional unit associated with an action using a knowledge
representation network, object confidences, and motion confidences. Each action
is therefore associated with a functional unit and the sequence of actions is
further evaluated to identify the single on-going activity in the video. The
knowledge representation used in the pipeline is called the functional
object-oriented network which is a graph-based network useful for encoding
knowledge about manipulation tasks. Experiments are performed on a dataset of
cooking videos to test the proposed algorithm with action inference and
activity classification. Experiments show that using functional object oriented
network improves video understanding significantly.

negative:
Graphical elements: particularly tables and figures contain a visual summary
of the most valuable information contained in a document. Therefore,
localization of such graphical objects in the document images is the initial
step to understand the content of such graphical objects or document images. In
this paper, we present a novel end-to-end trainable deep learning based
framework to localize graphical objects in the document images called as
Graphical Object Detection (GOD). Our framework is data-driven and does not
require any heuristics or meta-data to locate graphical objects in the document
images. The GOD explores the concept of transfer learning and domain adaptation
to handle scarcity of labeled training images for graphical object detection
task in the document images. Performance analysis carried out on the various
public benchmark data sets: ICDAR-2013, ICDAR-POD2017,and UNLV shows that our
model yields promising results as compared to state-of-the-art techniques.

negative:
In this work, a region-based Deep Convolutional Neural Network framework is
proposed for document structure learning. The contribution of this work
involves efficient training of region based classifiers and effective
ensembling for document image classification. A primary level of `inter-domain'
transfer learning is used by exporting weights from a pre-trained VGG16
architecture on the ImageNet dataset to train a document classifier on whole
document images. Exploiting the nature of region based influence modelling, a
secondary level of `intra-domain' transfer learning is used for rapid training
of deep learning models for image segments. Finally, stacked generalization
based ensembling is utilized for combining the predictions of the base deep
neural network models. The proposed method achieves state-of-the-art accuracy
of 92.2% on the popular RVL-CDIP document image dataset, exceeding benchmarks
set by existing algorithms.

negative:
Identity documents recognition is an important sub-field of document
analysis, which deals with tasks of robust document detection, type
identification, text fields recognition, as well as identity fraud prevention
and document authenticity validation given photos, scans, or video frames of an
identity document capture. Significant amount of research has been published on
this topic in recent years, however a chief difficulty for such research is
scarcity of datasets, due to the subject matter being protected by security
requirements. A few datasets of identity documents which are available lack
diversity of document types, capturing conditions, or variability of document
field values. In addition, the published datasets were typically designed only
for a subset of document recognition problems, not for a complex identity
document analysis. In this paper, we present a dataset MIDV-2020 which consists
of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock
identity documents, each with unique text field values and unique artificially
generated faces, with rich annotation. For the presented benchmark dataset
baselines are provided for such tasks as document location and identification,
text fields recognition, and face detection. With 72409 annotated images in
total, to the date of publication the proposed dataset is the largest publicly
available identity documents dataset with variable artificially generated data,
and we believe that it will prove invaluable for advancement of the field of
document analysis and recognition. The dataset is available for download at
ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr .

negative:
One major drawback of state of the art Neural Networks (NN)-based approaches
for document classification purposes is the large number of training samples
required to obtain an efficient classification. The minimum required number is
around one thousand annotated documents for each class. In many cases it is
very difficult, if not impossible, to gather this number of samples in real
industrial processes. In this paper, we analyse the efficiency of NN-based
document classification systems in a sub-optimal training case, based on the
situation of a company document stream. We evaluated three different
approaches, one based on image content and two on textual content. The
evaluation was divided into four parts: a reference case, to assess the
performance of the system in the lab; two cases that each simulate a specific
difficulty linked to document stream processing; and a realistic case that
combined all of these difficulties. The realistic case highlighted the fact
that there is a significant drop in the efficiency of NN-Based document
classification systems. Although they remain efficient for well represented
classes (with an over-fitting of the system for those classes), it is
impossible for them to handle appropriately less well represented classes.
NN-Based document classification systems need to be adapted to resolve these
two problems before they can be considered for use in a company document
stream.

negative:
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.

negative:
The unsupervised training of GANs and VAEs has enabled them to generate
realistic images mimicking real-world distributions and perform image-based
unsupervised clustering or semi-supervised classification. Combining the power
of these two generative models, we introduce Multi-Adversarial Variational
autoEncoder Networks (MAVENs), a novel network architecture that incorporates
an ensemble of discriminators in a VAE-GAN network, with simultaneous
adversarial learning and variational inference. We apply MAVENs to the
generation of synthetic images and propose a new distribution measure to
quantify the quality of the generated images. Our experimental results using
datasets from the computer vision and medical imaging domains---Street View
House Numbers, CIFAR-10, and Chest X-Ray datasets---demonstrate competitive
performance against state-of-the-art semi-supervised models both in image
generation and classification tasks.

negative:
With the advent of mobile and hand-held cameras, document images have found
their way into almost every domain. Dewarping of these images for the removal
of perspective distortions and folds is essential so that they can be
understood by document recognition algorithms. For this, we propose an
end-to-end CNN architecture that can produce distortion free document images
from warped documents it takes as input. We train this model on warped document
images simulated synthetically to compensate for lack of enough natural data.
Our method is novel in the use of a bifurcated decoder with shared weights to
prevent intermingling of grid coordinates, in the use of residual networks in
the U-Net skip connections to allow flow of data from different receptive
fields in the model, and in the use of a gated network to help the model focus
on structure and line level detail of the document image. We evaluate our
method on the DocUNet dataset, a benchmark in this domain, and obtain results
comparable to state-of-the-art methods.