query: I am currently involved in a project focused on developing a night-mode facial recognition application. This technology is primarily used in security systems and driver assistance technologies that operate in dark environments. Our project requires a system capable of efficiently processing and analyzing human facial features, head orientations, and eye gaze under near-infrared illumination. To initiate this project, we are in search of a comprehensive dataset containing near-infrared videos of various subjects exhibiting different facial expressions. This dataset should be meticulously labeled to allow us to validate and evaluate algorithms for face detection, eye detection, head pose, and eye gazing tasks in dim environments.  Additionally, it would be beneficial if the dataset included a diverse range of subjects to ensure the robustness and generalizability of the algorithms we develop based on it.  Moreover, we are also seeking recommendations on human-machine interaction in dim environments since the model needs to provide further instructions to the user.

positive:
Commercial application of facial recognition demands robustness to a variety
of challenges such as illumination, occlusion, spoofing, disguise, etc.
Disguised face recognition is one of the emerging issues for access control
systems, such as security checkpoints at the borders. However, the lack of
availability of face databases with a variety of disguise addons limits the
development of academic research in the area. In this paper, we present a
multimodal disguised face dataset to facilitate the disguised face recognition
research. The presented database contains 8 facial add-ons and 7 additional
combinations of these add-ons to create a variety of disguised face images.
Each facial image is captured in visible, visible plus infrared, infrared, and
thermal spectra. Specifically, the database contains 100 subjects divided into
subset-A (30 subjects, 1 image per modality) and subset-B (70 subjects, 5 plus
images per modality). We also present baseline face detection results performed
on the proposed database to provide reference results and compare the
performance in different modalities. Qualitative and quantitative analysis is
performed to evaluate the challenging nature of disguise addons. The dataset
will be publicly available with the acceptance of the research article. The
database is available at: https://github.com/usmancheema89/SejongFaceDatabase.

positive:
Face recognition in real life situations like low illumination condition is
still an open challenge in biometric security. It is well established that the
state-of-the-art methods in face recognition provide low accuracy in the case
of poor illumination. In this work, we propose an algorithm for a more robust
illumination invariant face recognition using a multi-modal approach. We
propose a new dataset consisting of aligned faces of thermal and visual images
of a hundred subjects. We then apply face detection on thermal images using the
biggest blob extraction method and apply them for fusing images of different
modalities for the purpose of face recognition. An algorithm is proposed to
implement fusion of thermal and visual images. We reason for why relying on
only one modality can give erroneous results. We use a lighter and faster CNN
model called MobileNet for the purpose of face recognition with faster
inferencing and to be able to be use it in real time biometric systems. We test
our proposed method on our own created dataset to show that real-time face
recognition on fused images shows far better results than using visual or
thermal images separately.

positive:
Facial action unit recognition has many applications from market research to
psychotherapy and from image captioning to entertainment. Despite its recent
progress, deployment of these models has been impeded due to their limited
generalization to unseen people and demographics. This work conducts an
in-depth analysis of performance across several dimensions: individuals(40
subjects), genders (male and female), skin types (darker and lighter), and
databases (BP4D and DISFA). To help suppress the variance in data, we use the
notion of self-supervised denoising autoencoders to design a method for deep
face normalization(DeepFN) that transfers facial expressions of different
people onto a common facial template which is then used to train and evaluate
facial action recognition models. We show that person-independent models yield
significantly lower performance (55% average F1 and accuracy across 40
subjects) than person-dependent models (60.3%), leading to a generalization gap
of 5.3%. However, normalizing the data with the newly introduced DeepFN
significantly increased the performance of person-independent models (59.6%),
effectively reducing the gap. Similarly, we observed generalization gaps when
considering gender (2.4%), skin type (5.3%), and dataset (9.4%), which were
significantly reduced with the use of DeepFN. These findings represent an
important step towards the creation of more generalizable facial action unit
recognition systems.

positive:
Automatic recognition of spontaneous facial expressions is a major challenge
in the field of affective computing. Head rotation, face pose, illumination
variation, occlusion etc. are the attributes that increase the complexity of
recognition of spontaneous expressions in practical applications. Effective
recognition of expressions depends significantly on the quality of the database
used. Most well-known facial expression databases consist of posed expressions.
However, currently there is a huge demand for spontaneous expression databases
for the pragmatic implementation of the facial expression recognition
algorithms. In this paper, we propose and establish a new facial expression
database containing spontaneous expressions of both male and female
participants of Indian origin. The database consists of 428 segmented video
clips of the spontaneous facial expressions of 50 participants. In our
experiment, emotions were induced among the participants by using emotional
videos and simultaneously their self-ratings were collected for each
experienced emotion. Facial expression clips were annotated carefully by four
trained decoders, which were further validated by the nature of stimuli used
and self-report of emotions. An extensive analysis was carried out on the
database using several machine learning algorithms and the results are provided
for future reference. Such a spontaneous database will help in the development
and validation of algorithms for recognition of spontaneous expressions.

positive:
In our everyday lives and social interactions we often try to perceive the
emotional states of people. There has been a lot of research in providing
machines with a similar capacity of recognizing emotions. From a computer
vision perspective, most of the previous efforts have been focusing in
analyzing the facial expressions and, in some cases, also the body pose. Some
of these methods work remarkably well in specific settings. However, their
performance is limited in natural, unconstrained environments. Psychological
studies show that the scene context, in addition to facial expression and body
pose, provides important information to our perception of people's emotions.
However, the processing of the context for automatic emotion recognition has
not been explored in depth, partly due to the lack of proper data. In this
paper we present EMOTIC, a dataset of images of people in a diverse set of
natural situations, annotated with their apparent emotion. The EMOTIC dataset
combines two different types of emotion representation: (1) a set of 26
discrete categories, and (2) the continuous dimensions Valence, Arousal, and
Dominance. We also present a detailed statistical and algorithmic analysis of
the dataset along with annotators' agreement analysis. Using the EMOTIC dataset
we train different CNN models for emotion recognition, combining the
information of the bounding box containing the person with the contextual
information extracted from the scene. Our results show how scene context
provides important information to automatically recognize emotional states and
motivate further research in this direction. Dataset and code is open-sourced
and available at: https://github.com/rkosti/emotic and link for the
peer-reviewed published article: https://ieeexplore.ieee.org/document/8713881

positive:
Face identification/recognition has significantly advanced over the past
years. However, most of the proposed approaches rely on static RGB frames and
on neutral facial expressions. This has two disadvantages. First, important
facial shape cues are ignored. Second, facial deformations due to expressions
can have an impact on the performance of such a method. In this paper, we
propose a novel framework for dynamic 3D face identification/recognition based
on facial keypoints. Each dynamic sequence of facial expressions is represented
as a spatio-temporal graph, which is constructed using 3D facial landmarks.
Each graph node contains local shape and texture features that are extracted
from its neighborhood. For the classification/identification of faces, a
Spatio-temporal Graph Convolutional Network (ST-GCN) is used. Finally, we
evaluate our approach on a challenging dynamic 3D facial expression dataset.

negative:
A key step to driver safety is to observe the driver's activities with the
face being a key step in this process to extracting information such as head
pose, blink rate, yawns, talking to passenger which can then help derive higher
level information such as distraction, drowsiness, intent, and where they are
looking. In the context of driving safety, it is important for the system
perform robust estimation under harsh lighting and occlusion but also be able
to detect when the occlusion occurs so that information predicted from occluded
parts of the face can be taken into account properly. This paper introduces the
Occluded Stacked Hourglass, based on the work of original Stacked Hourglass
network for body pose joint estimation, which is retrained to process a
detected face window and output 68 occlusion heat maps, each corresponding to a
facial landmark. Landmark location, occlusion levels and a refined face
detection score, to reject false positives, are extracted from these heat maps.
Using the facial landmark locations, features such as head pose and eye/mouth
openness can be extracted to derive driver attention and activity. The system
is evaluated for face detection, head pose, and occlusion estimation on various
datasets in the wild, both quantitatively and qualitatively, and shows
state-of-the-art results.

negative:
The DeepFakes, which are the facial manipulation techniques, is the emerging
threat to digital society. Various DeepFake detection methods and datasets are
proposed for detecting such data, especially for face-swapping. However, recent
researches less consider facial animation, which is also important in the
DeepFake attack side. It tries to animate a face image with actions provided by
a driving video, which also leads to a concern about the security of recent
payment systems that reply on liveness detection to authenticate real users via
recognising a sequence of user facial actions. However, our experiments show
that the existed datasets are not sufficient to develop reliable detection
methods. While the current liveness detector cannot defend such videos as the
attack. As a response, we propose a new human face animation dataset, called
DeepFake MNIST+, generated by a SOTA image animation generator. It includes
10,000 facial animation videos in ten different actions, which can spoof the
recent liveness detectors. A baseline detection method and a comprehensive
analysis of the method is also included in this paper. In addition, we analyze
the proposed dataset's properties and reveal the difficulty and importance of
detecting animation datasets under different types of motion and compression
quality.

negative:
Learning-based methods are believed to work well for unconstrained gaze
estimation, i.e. gaze estimation from a monocular RGB camera without
assumptions regarding user, environment, or camera. However, current gaze
datasets were collected under laboratory conditions and methods were not
evaluated across multiple datasets. Our work makes three contributions towards
addressing these limitations. First, we present the MPIIGaze that contains
213,659 full face images and corresponding ground-truth gaze positions
collected from 15 users during everyday laptop use over several months. An
experience sampling approach ensured continuous gaze and head poses and
realistic variation in eye appearance and illumination. To facilitate
cross-dataset evaluations, 37,667 images were manually annotated with eye
corners, mouth corners, and pupil centres. Second, we present an extensive
evaluation of state-of-the-art gaze estimation methods on three current
datasets, including MPIIGaze. We study key challenges including target gaze
range, illumination conditions, and facial appearance variation. We show that
image resolution and the use of both eyes affect gaze estimation performance
while head pose and pupil centre information are less informative. Finally, we
propose GazeNet, the first deep appearance-based gaze estimation method.
GazeNet improves the state of the art by 22% percent (from a mean error of 13.9
degrees to 10.8 degrees) for the most challenging cross-dataset evaluation.

negative:
Facial expressions are one of the most powerful, natural and immediate means
for human being to communicate their emotions and intensions. Recognition of
facial expression has many applications including human-computer interaction,
cognitive science, human emotion analysis, personality development etc. In this
paper, we propose a new method for the recognition of facial expressions from
single image frame that uses combination of appearance and geometric features
with support vector machines classification. In general, appearance features
for the recognition of facial expressions are computed by dividing face region
into regular grid (holistic representation). But, in this paper we extracted
region specific appearance features by dividing the whole face region into
domain specific local regions. Geometric features are also extracted from
corresponding domain specific regions. In addition, important local regions are
determined by using incremental search approach which results in the reduction
of feature dimension and improvement in recognition accuracy. The results of
facial expressions recognition using features from domain specific regions are
also compared with the results obtained using holistic representation. The
performance of the proposed facial expression recognition system has been
validated on publicly available extended Cohn-Kanade (CK+) facial expression
data sets.

negative:
Face recognition is a long standing challenge in the field of Artificial
Intelligence (AI). The goal is to create systems that accurately detect,
recognize, verify, and understand human faces. There are significant technical
hurdles in making these systems accurate, particularly in unconstrained
settings due to confounding factors related to pose, resolution, illumination,
occlusion, and viewpoint. However, with recent advances in neural networks,
face recognition has achieved unprecedented accuracy, largely built on
data-driven deep learning methods. While this is encouraging, a critical aspect
that is limiting facial recognition accuracy and fairness is inherent facial
diversity. Every face is different. Every face reflects something unique about
us. Aspects of our heritage - including race, ethnicity, culture, geography -
and our individual identify - age, gender, and other visible manifestations of
self-expression, are reflected in our faces. We expect face recognition to work
equally accurately for every face. Face recognition needs to be fair. As we
rely on data-driven methods to create face recognition technology, we need to
ensure necessary balance and coverage in training data. However, there are
still scientific questions about how to represent and extract pertinent facial
features and quantitatively measure facial diversity. Towards this goal,
Diversity in Faces (DiF) provides a data set of one million annotated human
face images for advancing the study of facial diversity. The annotations are
generated using ten well-established facial coding schemes from the scientific
literature. The facial coding schemes provide human-interpretable quantitative
measures of facial features. We believe that by making the extracted coding
schemes available on a large set of faces, we can accelerate research and
development towards creating more fair and accurate facial recognition systems.

negative:
Affect recognition based on subjects' facial expressions has been a topic of
major research in the attempt to generate machines that can understand the way
subjects feel, act and react. In the past, due to the unavailability of large
amounts of data captured in real-life situations, research has mainly focused
on controlled environments. However, recently, social media and platforms have
been widely used. Moreover, deep learning has emerged as a means to solve
visual analysis and recognition problems. This paper exploits these advances
and presents significant contributions for affect analysis and recognition
in-the-wild. Affect analysis and recognition can be seen as a dual knowledge
generation problem, involving: i) creation of new, large and rich in-the-wild
databases and ii) design and training of novel deep neural architectures that
are able to analyse affect over these databases and to successfully generalise
their performance on other datasets. The paper focuses on large in-the-wild
databases, i.e., Aff-Wild and Aff-Wild2 and presents the design of two classes
of deep neural networks trained with these databases. The first class refers to
uni-task affect recognition, focusing on prediction of the valence and arousal
dimensional variables. The second class refers to estimation of all main
behavior tasks, i.e. valence-arousal prediction; categorical emotion
classification in seven basic facial expressions; facial Action Unit detection.
A novel multi-task and holistic framework is presented which is able to jointly
learn and effectively generalize and perform affect recognition over all
existing in-the-wild databases. Large experimental studies illustrate the
achieved performance improvement over the existing state-of-the-art in affect
recognition.

negative:
Automatic face recognition is an area with immense practical potential which
includes a wide range of commercial and law enforcement applications. Hence it
is unsurprising that it continues to be one of the most active research areas
of computer vision. Even after over three decades of intense research, the
state-of-the-art in face recognition continues to improve, benefitting from
advances in a range of different research fields such as image processing,
pattern recognition, computer graphics, and physiology. Systems based on
visible spectrum images, the most researched face recognition modality, have
reached a significant level of maturity with some practical success. However,
they continue to face challenges in the presence of illumination, pose and
expression changes, as well as facial disguises, all of which can significantly
decrease recognition accuracy. Amongst various approaches which have been
proposed in an attempt to overcome these limitations, the use of infrared (IR)
imaging has emerged as a particularly promising research direction. This paper
presents a comprehensive and timely review of the literature on this subject.
Our key contributions are: (i) a summary of the inherent properties of infrared
imaging which makes this modality promising in the context of face recognition,
(ii) a systematic review of the most influential approaches, with a focus on
emerging common trends as well as key differences between alternative
methodologies, (iii) a description of the main databases of infrared facial
images available to the researcher, and lastly (iv) a discussion of the most
promising avenues for future research.

negative:
In the face recognition application scenario, we need to process facial
images captured in various conditions, such as at night by near-infrared (NIR)
surveillance cameras. The illumination difference between NIR and visible-light
(VIS) causes a domain gap between facial images, and the variations in pose and
emotion also make facial matching more difficult. Heterogeneous face
recognition (HFR) has difficulties in domain discrepancy, and many studies have
focused on extracting domain-invariant features, such as facial part relational
information. However, when pose variation occurs, the facial component position
changes, and a different part relation is extracted. In this paper, we propose
a part relation attention module that crops facial parts obtained through a
semantic mask and performs relational modeling using each of these
representative features. Furthermore, we suggest component adaptive triplet
loss function using adaptive weights for each part to reduce the intra-class
identity regardless of the domain as well as pose. Finally, our method exhibits
a performance improvement in the CASIA NIR-VIS 2.0 and achieves superior result
in the BUAA-VisNir with large pose and emotion variations.

negative:
Human emotions analysis has been the focus of many studies, especially in the
field of Affective Computing, and is important for many applications, e.g.
human-computer intelligent interaction, stress analysis, interactive games,
animations, etc. Solutions for automatic emotion analysis have also benefited
from the development of deep learning approaches and the availability of vast
amount of visual facial data on the internet. This paper proposes a novel
method for human emotion recognition from a single RGB image. We construct a
large-scale dataset of facial videos (\textbf{FaceVid}), rich in facial
dynamics, identities, expressions, appearance and 3D pose variations. We use
this dataset to train a deep Convolutional Neural Network for estimating
expression parameters of a 3D Morphable Model and combine it with an effective
back-end emotion classifier. Our proposed framework runs at 50 frames per
second and is capable of robustly estimating parameters of 3D expression
variation and accurately recognizing facial expressions from in-the-wild
images. We present extensive experimental evaluation that shows that the
proposed method outperforms the compared techniques in estimating the 3D
expression parameters and achieves state-of-the-art performance in recognising
the basic emotions from facial images, as well as recognising stress from
facial videos. %compared to the current state of the art in emotion recognition
from facial images.

negative:
Face recognition is a popular and well-studied area with wide applications in
our society. However, racial bias had been proven to be inherent in most State
Of The Art (SOTA) face recognition systems. Many investigative studies on face
recognition algorithms have reported higher false positive rates of African
subjects cohorts than the other cohorts. Lack of large-scale African face image
databases in public domain is one of the main restrictions in studying the
racial bias problem of face recognition. To this end, we collect a face image
database namely CASIA-Face-Africa which contains 38,546 images of 1,183 African
subjects. Multi-spectral cameras are utilized to capture the face images under
various illumination settings. Demographic attributes and facial expressions of
the subjects are also carefully recorded. For landmark detection, each face
image in the database is manually labeled with 68 facial keypoints. A group of
evaluation protocols are constructed according to different applications,
tasks, partitions and scenarios. The performances of SOTA face recognition
algorithms without re-training are reported as baselines. The proposed database
along with its face landmark annotations, evaluation protocols and preliminary
results form a good benchmark to study the essential aspects of face biometrics
for African subjects, especially face image preprocessing, face feature
analysis and matching, facial expression recognition, sex/age estimation,
ethnic classification, face image generation, etc. The database can be
downloaded from our http://www.cripacsir.cn/dataset/

negative:
Patients with schizophrenia often display impairments in the expression of
emotion and speech and those are observed in their facial behaviour. Automatic
analysis of patients' facial expressions that is aimed at estimating symptoms
of schizophrenia has received attention recently. However, the datasets that
are typically used for training and evaluating the developed methods, contain
only a small number of patients (4-34) and are recorded while the subjects were
performing controlled tasks such as listening to life vignettes, or answering
emotional questions. In this paper, we use videos of professional-patient
interviews, in which symptoms were assessed in a standardised way as they
should/may be assessed in practice, and which were recorded in realistic
conditions (i.e. varying illumination levels and camera viewpoints) at the
patients' homes or at mental health services. We automatically analyse the
facial behaviour of 91 out-patients - this is almost 3 times the number of
patients in other studies - and propose SchiNet, a novel neural network
architecture that estimates expression-related symptoms in two different
assessment interviews. We evaluate the proposed SchiNet for patient-independent
prediction of symptoms of schizophrenia. Experimental results show that some
automatically detected facial expressions are significantly correlated to
symptoms of schizophrenia, and that the proposed network for estimating symptom
severity delivers promising results.

negative:
A driver's gaze is critical for determining their attention, state,
situational awareness, and readiness to take over control from partially
automated vehicles. Estimating the gaze direction is the most obvious way to
gauge a driver's state under ideal conditions when limited to using
non-intrusive imaging sensors. Unfortunately, the vehicular environment
introduces a variety of challenges that are usually unaccounted for - harsh
illumination, nighttime conditions, and reflective eyeglasses. Relying on head
pose alone under such conditions can prove to be unreliable and erroneous. In
this study, we offer solutions to address these problems encountered in the
real world. To solve issues with lighting, we demonstrate that using an
infrared camera with suitable equalization and normalization suffices. To
handle eyeglasses and their corresponding artifacts, we adopt image-to-image
translation using generative adversarial networks to pre-process images prior
to gaze estimation. Our proposed Gaze Preserving CycleGAN (GPCycleGAN) is
trained to preserve the driver's gaze while removing potential eyeglasses from
face images. GPCycleGAN is based on the well-known CycleGAN approach - with the
addition of a gaze classifier and a gaze consistency loss for additional
supervision. Our approach exhibits improved performance, interpretability,
robustness and superior qualitative results on challenging real-world datasets.

negative:
To detect bias in face recognition networks, it can be useful to probe a
network under test using samples in which only specific attributes vary in some
controlled way. However, capturing a sufficiently large dataset with specific
control over the attributes of interest is difficult. In this work, we describe
a simulator that applies specific head pose and facial expression adjustments
to images of previously unseen people. The simulator first fits a 3D morphable
model to a provided image, applies the desired head pose and facial expression
controls, then renders the model into an image. Next, a conditional Generative
Adversarial Network (GAN) conditioned on the original image and the rendered
morphable model is used to produce the image of the original person with the
new facial expression and head pose. We call this conditional GAN -- MorphGAN.
Images generated using MorphGAN conserve the identity of the person in the
original image, and the provided control over head pose and facial expression
allows test sets to be created to identify robustness issues of a facial
recognition deep network with respect to pose and expression. Images generated
by MorphGAN can also serve as data augmentation when training data are scarce.
We show that by augmenting small datasets of faces with new poses and
expressions improves the recognition performance by up to 9% depending on the
augmentation and data scarcity.

negative:
Automated facial identification and facial expression recognition have been
topics of active research over the past few decades. Facial and expression
recognition find applications in human-computer interfaces, subject tracking,
real-time security surveillance systems and social networking. Several holistic
and geometric methods have been developed to identify faces and expressions
using public and local facial image databases. In this work we present the
evolution in facial image data sets and the methodologies for facial
identification and recognition of expressions such as anger, sadness,
happiness, disgust, fear and surprise. We observe that most of the earlier
methods for facial and expression recognition aimed at improving the
recognition rates for facial feature-based methods using static images.
However, the recent methodologies have shifted focus towards robust
implementation of facial/expression recognition from large image databases that
vary with space (gathered from the internet) and time (video recordings). The
evolution trends in databases and methodologies for facial and expression
recognition can be useful for assessing the next-generation topics that may
have applications in security systems or personal identification systems that
involve "Quantitative face" assessments.

negative:
Automatic affect recognition using visual cues is an important task towards a
complete interaction between humans and machines. Applications can be found in
tutoring systems and human computer interaction. A critical step towards that
direction is facial feature extraction. In this paper, we propose a facial
feature extractor model trained on an in-the-wild and massively collected video
dataset provided by the RealEyes company. The dataset consists of a million
labelled frames and 2,616 thousand subjects. As temporal information is
important to the emotion recognition domain, we utilise LSTM cells to capture
the temporal dynamics in the data. To show the favourable properties of our
pre-trained model on modelling facial affect, we use the RECOLA database, and
compare with the current state-of-the-art approach. Our model provides the best
results in terms of concordance correlation coefficient.

negative:
Facial features deformed according to the intended facial expression.
Specific facial features are associated with specific facial expression, i.e.
happy means the deformation of mouth. This paper presents the study of facial
feature deformation for each facial expression by using an optical flow
algorithm and segmented into three different regions of interest. The
deformation of facial features shows the relation between facial the and facial
expression. Based on the experiments, the deformations of eye and mouth are
significant in all expressions except happy. For happy expression, cheeks and
mouths are the significant regions. This work also suggests that different
facial features' intensity varies in the way that they contribute to the
recognition of the different facial expression intensity. The maximum magnitude
across all expressions is shown by the mouth for surprise expression which is
9x10-4. While the minimum magnitude is shown by the mouth for angry expression
which is 0.4x10-4.

negative:
Facial expression recognition is a challenging task when neural network is
applied to pattern recognition. Most of the current recognition research is
based on single source facial data, which generally has the disadvantages of
low accuracy and low robustness. In this paper, a neural network algorithm of
facial expression recognition based on multimodal data fusion is proposed. The
algorithm is based on the multimodal data, and it takes the facial image, the
histogram of oriented gradient of the image and the facial landmarks as the
input, and establishes CNN, LNN and HNN three sub neural networks to extract
data features, using multimodal data feature fusion mechanism to improve the
accuracy of facial expression recognition. Experimental results show that,
benefiting by the complementarity of multimodal data, the algorithm has a great
improvement in accuracy, robustness and detection speed compared with the
traditional facial expression recognition algorithm. Especially in the case of
partial occlusion, illumination and head posture transformation, the algorithm
also shows a high confidence.

negative:
Head detection and localization is a demanding task and a key element for
many computer vision applications, like video surveillance, Human Computer
Interaction and face analysis. The stunning amount of work done for detecting
faces on RGB images, together with the availability of huge face datasets,
allowed to setup very effective systems on that domain. However, due to
illumination issues, infrared or depth cameras may be required in real
applications. In this paper, we introduce a novel method for head detection on
depth images that exploits the classification ability of deep learning
approaches. In addition to reduce the dependency on the external illumination,
depth images implicitly embed useful information to deal with the scale of the
target objects. Two public datasets have been exploited: the first one, called
Pandora, is used to train a deep binary classifier with face and non-face
images. The second one, collected by Cornell University, is used to perform a
cross-dataset test during daily activities in unconstrained environments.
Experimental results show that the proposed method overcomes the performance of
state-of-art methods working on depth images.

negative:
Differently from computer vision systems which require explicit supervision,
humans can learn facial expressions by observing people in their environment.
In this paper, we look at how similar capabilities could be developed in
machine vision. As a starting point, we consider the problem of relating facial
expressions to objectively measurable events occurring in videos. In
particular, we consider a gameshow in which contestants play to win significant
sums of money. We extract events affecting the game and corresponding facial
expressions objectively and automatically from the videos, obtaining large
quantities of labelled data for our study. We also develop, using benchmarks
such as FER and SFEW 2.0, state-of-the-art deep neural networks for facial
expression recognition, showing that pre-training on face verification data can
be highly beneficial for this task. Then, we extend these models to use facial
expressions to predict events in videos and learn nameable expressions from
them. The dataset and emotion recognition models are available at
http://www.robots.ox.ac.uk/~vgg/data/facevalue

negative:
Face detection in low light scenarios is challenging but vital to many
practical applications, e.g., surveillance video, autonomous driving at night.
Most existing face detectors heavily rely on extensive annotations, while
collecting data is time-consuming and laborious. To reduce the burden of
building new datasets for low light conditions, we make full use of existing
normal light data and explore how to adapt face detectors from normal light to
low light. The challenge of this task is that the gap between normal and low
light is too huge and complex for both pixel-level and object-level. Therefore,
most existing low-light enhancement and adaptation methods do not achieve
desirable performance. To address the issue, we propose a joint High-Low
Adaptation (HLA) framework. Through a bidirectional low-level adaptation and
multi-task high-level adaptation scheme, our HLA-Face outperforms
state-of-the-art methods even without using dark face labels for training. Our
project is publicly available at https://daooshee.github.io/HLA-Face-Website/

negative:
Teleconference or telepresence based on virtual reality (VR) headmount
display (HMD) device is a very interesting and promising application since HMD
can provide immersive feelings for users. However, in order to facilitate
face-to-face communications for HMD users, real-time 3D facial performance
capture of a person wearing HMD is needed, which is a very challenging task due
to the large occlusion caused by HMD. The existing limited solutions are very
complex either in setting or in approach as well as lacking the performance
capture of 3D eye gaze movement. In this paper, we propose a convolutional
neural network (CNN) based solution for real-time 3D face-eye performance
capture of HMD users without complex modification to devices. To address the
issue of lacking training data, we generate massive pairs of HMD face-label
dataset by data synthesis as well as collecting VR-IR eye dataset from multiple
subjects. Then, we train a dense-fitting network for facial region and an eye
gaze network to regress 3D eye model parameters. Extensive experimental results
demonstrate that our system can efficiently and effectively produce in real
time a vivid personalized 3D avatar with the correct identity, pose, expression
and eye motion corresponding to the HMD user.

negative:
In this paper, we focus on the facial expression translation task and propose
a novel Expression Conditional GAN (ECGAN) which can learn the mapping from one
image domain to another one based on an additional expression attribute. The
proposed ECGAN is a generic framework and is applicable to different expression
generation tasks where specific facial expression can be easily controlled by
the conditional attribute label. Besides, we introduce a novel face mask loss
to reduce the influence of background changing. Moreover, we propose an entire
framework for facial expression generation and recognition in the wild, which
consists of two modules, i.e., generation and recognition. Finally, we evaluate
our framework on several public face datasets in which the subjects have
different races, illumination, occlusion, pose, color, content and background
conditions. Even though these datasets are very diverse, both the qualitative
and quantitative results demonstrate that our approach is able to generate
facial expressions accurately and robustly.

negative:
Automated estimation of the allocation of a driver's visual attention may be
a critical component of future Advanced Driver Assistance Systems. In theory,
vision-based tracking of the eye can provide a good estimate of gaze location.
In practice, eye tracking from video is challenging because of sunglasses,
eyeglass reflections, lighting conditions, occlusions, motion blur, and other
factors. Estimation of head pose, on the other hand, is robust to many of these
effects, but cannot provide as fine-grained of a resolution in localizing the
gaze. However, for the purpose of keeping the driver safe, it is sufficient to
partition gaze into regions. In this effort, we propose a system that extracts
facial features and classifies their spatial configuration into six regions in
real-time. Our proposed method achieves an average accuracy of 91.4% at an
average decision rate of 11 Hz on a dataset of 50 drivers from an on-road
study.

negative:
Accurate, robust, inexpensive gaze tracking in the car can help keep a driver
safe by facilitating the more effective study of how to improve (1) vehicle
interfaces and (2) the design of future Advanced Driver Assistance Systems. In
this paper, we estimate head pose and eye pose from monocular video using
methods developed extensively in prior work and ask two new interesting
questions. First, how much better can we classify driver gaze using head and
eye pose versus just using head pose? Second, are there individual-specific
gaze strategies that strongly correlate with how much gaze classification
improves with the addition of eye pose information? We answer these questions
by evaluating data drawn from an on-road study of 40 drivers. The main insight
of the paper is conveyed through the analogy of an "owl" and "lizard" which
describes the degree to which the eyes and the head move when shifting gaze.
When the head moves a lot ("owl"), not much classification improvement is
attained by estimating eye pose on top of head pose. On the other hand, when
the head stays still and only the eyes move ("lizard"), classification accuracy
increases significantly from adding in eye pose. We characterize how that
accuracy varies between people, gaze strategies, and gaze regions.

negative:
The task of action recognition in dark videos is useful in various scenarios,
e.g., night surveillance and self-driving at night. Though progress has been
made in the action recognition task for videos in normal illumination, few have
studied action recognition in the dark. This is partly due to the lack of
sufficient datasets for such a task. In this paper, we explored the task of
action recognition in dark videos. We bridge the gap of the lack of data for
this task by collecting a new dataset: the Action Recognition in the Dark
(ARID) dataset. It consists of over 3,780 video clips with 11 action
categories. To the best of our knowledge, it is the first dataset focused on
human actions in dark videos. To gain further understandings of our ARID
dataset, we analyze the ARID dataset in detail and exhibited its necessity over
synthetic dark videos. Additionally, we benchmarked the performance of several
current action recognition models on our dataset and explored potential methods
for increasing their performances. Our results show that current action
recognition models and frame enhancement methods may not be effective solutions
for the task of action recognition in dark videos.

negative:
With the growth of popularity of facial micro-expressions in recent years,
the demand for long videos with micro- and macro-expressions remains high.
Extended from SAMM, a micro-expressions dataset released in 2016, this paper
presents SAMM Long Videos dataset for spontaneous micro- and macro-expressions
recognition and spotting. SAMM Long Videos dataset consists of 147 long videos
with 343 macro-expressions and 159 micro-expressions. The dataset is FACS-coded
with detailed Action Units (AUs). We compare our dataset with Chinese Academy
of Sciences Macro-Expressions and Micro-Expressions (CAS(ME)2) dataset, which
is the only available fully annotated dataset with micro- and
macro-expressions. Furthermore, we preprocess the long videos using OpenFace,
which includes face alignment and detection of facial AUs. We conduct facial
expression spotting using this dataset and compare it with the baseline of MEGC
III. Our spotting method outperformed the baseline result with F1-score of
0.3299.

negative:
Much of the work on automatic facial expression recognition relies on
databases containing a certain number of emotion classes and their exaggerated
facial configurations (generally six prototypical facial expressions), based on
Ekman's Basic Emotion Theory. However, recent studies have revealed that facial
expressions in our human life can be blended with multiple basic emotions. And
the emotion labels for these in-the-wild facial expressions cannot easily be
annotated solely on pre-defined AU patterns. How to analyze the action units
for such complex expressions is still an open question. To address this issue,
we develop a RAF-AU database that employs a sign-based (i.e., AUs) and
judgement-based (i.e., perceived emotion) approach to annotating blended facial
expressions in the wild. We first reviewed the annotation methods in existing
databases and identified crowdsourcing as a promising strategy for labeling
in-the-wild facial expressions. Then, RAF-AU was finely annotated by
experienced coders, on which we also conducted a preliminary investigation of
which key AUs contribute most to a perceived emotion, and the relationship
between AUs and facial expressions. Finally, we provided a baseline for AU
recognition in RAF-AU using popular features and multi-label learning methods.

negative:
Face recognition performance has improved remarkably in the last decade. Much
of this success can be attributed to the development of deep learning
techniques such as convolutional neural networks (CNNs). While CNNs have pushed
the state-of-the-art forward, their training process requires a large amount of
clean and correctly labelled training data. If a CNN is intended to tolerate
facial pose, then we face an important question: should this training data be
diverse in its pose distribution, or should face images be normalized to a
single pose in a pre-processing step? To address this question, we evaluate a
number of popular facial landmarking and pose correction algorithms to
understand their effect on facial recognition performance. Additionally, we
introduce a new, automatic, single-image frontalization scheme that exceeds the
performance of current algorithms. CNNs trained using sets of different
pre-processing methods are used to extract features from the Point and Shoot
Challenge (PaSC) and CMU Multi-PIE datasets. We assert that the subsequent
verification and recognition performance serves to quantify the effectiveness
of each pose correction scheme.

negative:
This paper presents a database of human faces for persons wearing spectacles.
The database consists of images of faces having significant variations with
respect to illumination, head pose, skin color, facial expressions and sizes,
and nature of spectacles. The database contains data of 60 subjects. This
database is expected to be a precious resource for the development and
evaluation of algorithms for face detection, eye detection, head tracking, eye
gaze tracking, etc., for subjects wearing spectacles. As such, this can be a
valuable contribution to the computer vision community.

negative:
One of the key research areas in computer vision addressed by a vast number
of publications is the processing and understanding of images containing human
faces. The most often addressed tasks include face detection, facial landmark
localization, face recognition and facial expression analysis. Other, more
specialized tasks such as affective computing, the extraction of vital signs
from videos or analysis of social interaction usually require one or several of
the aforementioned tasks that have to be performed. In our work, we analyze
that a large number of tasks for facial image processing in thermal infrared
images that are currently solved using specialized rule-based methods or not
solved at all can be addressed with modern learning-based approaches. We have
used USTC-NVIE database for training of a number of machine learning algorithms
for facial landmark localization.

negative:
Deep Neural Networks (DNNs) have shown to outperform traditional methods in
various visual recognition tasks including Facial Expression Recognition (FER).
In spite of efforts made to improve the accuracy of FER systems using DNN,
existing methods still are not generalizable enough in practical applications.
This paper proposes a 3D Convolutional Neural Network method for FER in videos.
This new network architecture consists of 3D Inception-ResNet layers followed
by an LSTM unit that together extracts the spatial relations within facial
images as well as the temporal relations between different frames in the video.
Facial landmark points are also used as inputs to our network which emphasize
on the importance of facial components rather than the facial regions that may
not contribute significantly to generating facial expressions. Our proposed
method is evaluated using four publicly available databases in
subject-independent and cross-database tasks and outperforms state-of-the-art
methods.

negative:
Concern regarding the wide-spread use of fraudulent images/videos in social
media necessitates precise detection of such fraud. The importance of facial
expressions in communication is widely known, and adversarial attacks often
focus on manipulating the expression related features. Thus, it is important to
develop methods that can detect manipulations in facial expressions, and
localize the manipulated regions. To address this problem, we propose a
framework that is able to detect manipulations in facial expression using a
close combination of facial expression recognition and image manipulation
methods. With the addition of feature maps extracted from the facial expression
recognition framework, our manipulation detector is able to localize the
manipulated region. We show that, on the Face2Face dataset, where there is
abundant expression manipulation, our method achieves over 3% higher accuracy
for both classification and localization of manipulations compared to
state-of-the-art methods. In addition, results on the NeuralTextures dataset
where the facial expressions corresponding to the mouth regions have been
modified, show 2% higher accuracy in both classification and localization of
manipulation. We demonstrate that the method performs at-par with the
state-of-the-art methods in cases where the expression is not manipulated, but
rather the identity is changed, thus ensuring generalizability of the approach.

negative:
Gaze and face tracking algorithms have traditionally battled a compromise
between computational complexity and accuracy; the most accurate neural net
algorithms cannot be implemented in real time, but less complex real-time
algorithms suffer from higher error. This project seeks to better bridge that
gap by improving on real-time eye and facial recognition algorithms in order to
develop accurate, real-time gaze estimation with an emphasis on minimizing
training data and computational complexity. Our goal is to use eye and facial
recognition techniques to enable users to perform limited tasks based on gaze
and facial input using only a standard, low-quality web cam found in most
modern laptops and smart phones and the limited computational power and
training data typical of those scenarios. We therefore identified seven
promising, fundamentally different algorithms based on different user features
and developed one outstanding, one workable, and one honorable mention gaze
tracking pipelines that match the performance of modern gaze trackers while
using no training data.

negative:
Automatic facial expression recognition is an important research area in the
emotion recognition and computer vision. Applications can be found in several
domains such as medical treatment, driver fatigue surveillance, sociable
robotics, and several other human-computer interaction systems. Therefore, it
is crucial that the machine should be able to recognize the emotional state of
the user with high accuracy. In recent years, deep neural networks have been
used with great success in recognizing emotions. In this paper, we present a
new model for continuous emotion recognition based on facial expression
recognition by using an unsupervised learning approach based on transfer
learning and autoencoders. The proposed approach also includes preprocessing
and post-processing techniques which contribute favorably to improving the
performance of predicting the concordance correlation coefficient for arousal
and valence dimensions. Experimental results for predicting spontaneous and
natural emotions on the RECOLA 2016 dataset have shown that the proposed
approach based on visual information can achieve CCCs of 0.516 and 0.264 for
valence and arousal, respectively.

negative:
Detecting facial action units (AU) is one of the fundamental steps in
automatic recognition of facial expression of emotions and cognitive states.
Though there have been a variety of approaches proposed for this task, most of
these models are trained only for the specific target AUs, and as such they
fail to easily adapt to the task of recognition of new AUs (i.e., those not
initially used to train the target models). In this paper, we propose a deep
learning approach for facial AU detection that can easily and in a fast manner
adapt to a new AU or target subject by leveraging only a few labeled samples
from the new task (either an AU or subject). To this end, we propose a modeling
approach based on the notion of the model-agnostic meta-learning, originally
proposed for the general image recognition/detection tasks (e.g., the character
recognition from the Omniglot dataset). Specifically, each subject and/or AU is
treated as a new learning task and the model learns to adapt based on the
knowledge of the previous tasks (the AUs and subjects used to pre-train the
target models). Thus, given a new subject or AU, this meta-knowledge (that is
shared among training and test tasks) is used to adapt the model to the new
task using the notion of deep learning and model-agnostic meta-learning. We
show on two benchmark datasets (BP4D and DISFA) for facial AU detection that
the proposed approach can be easily adapted to new tasks (AUs/subjects). Using
only a few labeled examples from these tasks, the model achieves large
improvements over the baselines (i.e., non-adapted models).

negative:
For real-world driver drowsiness detection from videos, the variation of head
pose is so large that the existing methods on global face is not capable of
extracting effective features, such as looking aside and lowering head.
Temporal dependencies with variable length are also rarely considered by the
previous approaches, e.g., yawning and speaking. In this paper, we propose a
Long-term Multi-granularity Deep Framework to detect driver drowsiness in
driving videos containing the frontal faces. The framework includes two key
components: (1) Multi-granularity Convolutional Neural Network (MCNN), a novel
network utilizes a group of parallel CNN extractors on well-aligned facial
patches of different granularities, and extracts facial representations
effectively for large variation of head pose, furthermore, it can flexibly fuse
both detailed appearance clues of the main parts and local to global spatial
constraints; (2) a deep Long Short Term Memory network is applied on facial
representations to explore long-term relationships with variable length over
sequential frames, which is capable to distinguish the states with temporal
dependencies, such as blinking and closing eyes. Our approach achieves 90.05%
accuracy and about 37 fps speed on the evaluation set of the public NTHU-DDD
dataset, which is the state-of-the-art method on driver drowsiness detection.
Moreover, we build a new dataset named FI-DDD, which is of higher precision of
drowsy locations in temporal dimension.

negative:
In this paper, we propose a Geometry-Contrastive Generative Adversarial
Network (GC-GAN) for transferring continuous emotions across different
subjects. Given an input face with certain emotion and a target facial
expression from another subject, GC-GAN can generate an identity-preserving
face with the target expression. Geometry information is introduced into cGANs
as continuous conditions to guide the generation of facial expressions. In
order to handle the misalignment across different subjects or emotions,
contrastive learning is used to transform geometry manifold into an embedded
semantic manifold of facial expressions. Therefore, the embedded geometry is
injected into the latent space of GANs and control the emotion generation
effectively. Experimental results demonstrate that our proposed method can be
applied in facial expression transfer even there exist big differences in
facial shapes and expressions between different subjects.

negative:
This study demonstrates how facial biometrics, acquired using multi-spectral
sensors, such as RGB, depth, and infrared, assist the data accumulation in the
process of authorizing users of automated and semi-automated access systems.
This data serves the purposes of person authentication, as well as facial
temperature estimation. We utilize depth data taken using an inexpensive RGB-D
sensor to find the head pose of a subject. This allows the selection of video
frames containing a frontal-view head pose for face recognition and face
temperature reading. Usage of the frontal-view frames improves the efficiency
of face recognition while the corresponding synchronized IR video frames allow
for more efficient temperature estimation for facial regions of interest. In
addition, this study reports emerging applications of biometrics in biomedical
and health care solutions. Including surveys of recent pilot projects,
involving new sensors of biometric data and new applications of human
physiological and behavioral biometrics. It also shows the new and promising
horizons of using biometrics in natural and contactless control interfaces for
surgical control, rehabilitation and accessibility.

negative:
Mobile eye-tracking systems have been available for about a decade now and
are becoming increasingly popular in different fields of application, including
marketing, sociology, usability studies and linguistics. While the
user-friendliness and ergonomics of the hardware are developing at a rapid
pace, the software for the analysis of mobile eye-tracking data in some points
still lacks robustness and functionality. With this paper, we investigate which
state-of-the-art computer vision algorithms may be used to automate the
post-analysis of mobile eye-tracking data. For the case study in this paper, we
focus on mobile eye-tracker recordings made during human-human face-to-face
interactions. We compared two recent publicly available frameworks (YOLOv2 and
OpenPose) to relate the gaze location generated by the eye-tracker to the head
and hands visible in the scene camera data. In this paper we will show that the
use of this single-pipeline framework provides robust results, which are both
more accurate and faster than previous work in the field. Moreover, our
approach does not rely on manual interventions during this process.

negative:
Explainable face recognition is the problem of explaining why a facial
matcher matches faces. In this paper, we provide the first comprehensive
benchmark and baseline evaluation for explainable face recognition. We define a
new evaluation protocol called the ``inpainting game'', which is a curated set
of 3648 triplets (probe, mate, nonmate) of 95 subjects, which differ by
synthetically inpainting a chosen facial characteristic like the nose, eyebrows
or mouth creating an inpainted nonmate. An explainable face matcher is tasked
with generating a network attention map which best explains which regions in a
probe image match with a mated image, and not with an inpainted nonmate for
each triplet. This provides ground truth for quantifying what image regions
contribute to face matching. Furthermore, we provide a comprehensive benchmark
on this dataset comparing five state of the art methods for network attention
in face recognition on three facial matchers. This benchmark includes two new
algorithms for network attention called subtree EBP and Density-based Input
Sampling for Explanation (DISE) which outperform the state of the art by a wide
margin. Finally, we show qualitative visualization of these network attention
techniques on novel images, and explore how these explainable face recognition
models can improve transparency and trust for facial matchers.

negative:
We aim to construct a system that captures real-world facial images through
the front camera on a laptop. The system is capable of processing/recognizing
the captured image and predict a result in real-time. In this system, we
exploit the power of deep learning technique to learn a facial emotion
recognition (FER) model based on a set of labeled facial images. Finally,
experiments are conducted to evaluate our model using largely used public
database.

negative:
Facial expressions are important cues to observe human emotions. Facial
expression recognition has attracted many researchers for years, but it is
still a challenging topic since expression features vary greatly with the head
poses, environments, and variations in the different persons involved. In this
work, three major steps are involved to improve the performance of micro-facial
expression recognition. First, an Adaptive Homomorphic Filtering is used for
face detection and rotation rectification processes. Secondly, Micro-facial
features were used to extract the appearance variations of a testing
image-spatial analysis. The features of motion information are used for
expression recognition in a sequence of facial images. An effective
Micro-Facial Expression Based Deep-Rooted Learning (MFEDRL) classifier is
proposed in this paper to better recognize spontaneous micro-expressions by
learning parameters on the optimal features. This proposed method includes two
loss functions such as cross entropy loss function and centre loss function.
Then the performance of the algorithm will be evaluated using recognition rate
and false measures. Simulation results show that the predictive performance of
the proposed method outperforms that of the existing classifiers such as
Convolutional Neural Network (CNN), Deep Neural Network (DNN), Artificial
Neural Network (ANN), Support Vector Machine (SVM), and k-Nearest Neighbours
(KNN) in terms of accuracy and Mean Absolute Error (MAE).

negative:
Current state-of-the-art models for automatic Facial Expression Recognition
(FER) are based on very deep neural networks that are effective but rather
expensive to train. Given the dynamic conditions of FER, this characteristic
hinders such models of been used as a general affect recognition. In this
paper, we address this problem by formalizing the FaceChannel, a light-weight
neural network that has much fewer parameters than common deep neural networks.
We introduce an inhibitory layer that helps to shape the learning of facial
features in the last layer of the network and thus improving performance while
reducing the number of trainable parameters. To evaluate our model, we perform
a series of experiments on different benchmark datasets and demonstrate how the
FaceChannel achieves a comparable, if not better, performance to the current
state-of-the-art in FER. Our experiments include cross-dataset analysis, to
estimate how our model behaves on different affective recognition conditions.
We conclude our paper with an analysis of how FaceChannel learns and adapt the
learned facial features towards the different datasets.

negative:
Recent advances in deep learning have significantly pushed the
state-of-the-art in photorealistic video animation given a single image. In
this paper, we extrapolate those advances to the 3D domain, by studying 3D
image-to-video translation with a particular focus on 4D facial expressions.
Although 3D facial generative models have been widely explored during the past
years, 4D animation remains relatively unexplored. To this end, in this study
we employ a deep mesh encoder-decoder like architecture to synthesize realistic
high resolution facial expressions by using a single neutral frame along with
an expression identification. In addition, processing 3D meshes remains a
non-trivial task compared to data that live on grid-like structures, such as
images. Given the recent progress in mesh processing with graph convolutions,
we make use of a recently introduced learnable operator which acts directly on
the mesh structure by taking advantage of local vertex orderings. In order to
generalize to 4D facial expressions across subjects, we trained our model using
a high resolution dataset with 4D scans of six facial expressions from 180
subjects. Experimental results demonstrate that our approach preserves the
subject's identity information even for unseen subjects and generates high
quality expressions. To the best of our knowledge, this is the first study
tackling the problem of 4D facial expression synthesis.

negative:
The majority of learning-based semantic segmentation methods are optimized
for daytime scenarios and favorable lighting conditions. Real-world driving
scenarios, however, entail adverse environmental conditions such as nighttime
illumination or glare which remain a challenge for existing approaches. In this
work, we propose a multimodal semantic segmentation model that can be applied
during daytime and nighttime. To this end, besides RGB images, we leverage
thermal images, making our network significantly more robust. We avoid the
expensive annotation of nighttime images by leveraging an existing daytime
RGB-dataset and propose a teacher-student training approach that transfers the
dataset's knowledge to the nighttime domain. We further employ a domain
adaptation method to align the learned feature spaces across the domains and
propose a novel two-stage training scheme. Furthermore, due to a lack of
thermal data for autonomous driving, we present a new dataset comprising over
20,000 time-synchronized and aligned RGB-thermal image pairs. In this context,
we also present a novel target-less calibration method that allows for
automatic robust extrinsic and intrinsic thermal camera calibration. Among
others, we employ our new dataset to show state-of-the-art results for
nighttime semantic segmentation.

negative:
Determining the direction in which a person is looking is an important
problem in a wide range of HCI applications. In this paper we describe a highly
accurate algorithm that performs gaze estimation using an affordable and widely
available device such as Kinect. The method we propose starts by performing
accurate head pose estimation achieved by fitting a person specific morphable
model of the face to depth data. The ordinarily competing requirements of high
accuracy and high speed are met concurrently by formulating the fitting
objective function as a combination of terms which excel either in accurate or
fast fitting, and then by adaptively adjusting their relative contributions
throughout fitting. Following pose estimation, pose normalization is done by
re-rendering the fitted model as a frontal face. Finally gaze estimates are
obtained through regression from the appearance of the eyes in synthetic,
normalized images. Using EYEDIAP, the standard public dataset for the
evaluation of gaze estimation algorithms from RGB-D data, we demonstrate that
our method greatly outperforms the state of the art.

negative:
Gaze-tracking is a novel way of interacting with computers which allows new
scenarios, such as enabling people with motor-neuron disabilities to control
their computers or doctors to interact with patient information without
touching screen or keyboard. Further, there are emerging applications of
gaze-tracking in interactive gaming, user experience research, human attention
analysis and behavioral studies. Accurate estimation of the gaze may involve
accounting for head-pose, head-position, eye rotation, distance from the object
as well as operating conditions such as illumination, occlusion, background
noise and various biological aspects of the user. Commercially available
gaze-trackers utilize specialized sensor assemblies that usually consist of an
infrared light source and camera. There are several challenges in the universal
proliferation of gaze-tracking as accessibility technologies, specifically its
affordability, reliability, and ease-of-use. In this paper, we try to address
these challenges through the development of a hardware-agnostic gaze-tracker.
We present a deep neural network architecture as an appearance-based method for
constrained gaze-tracking that utilizes facial imagery captured on an ordinary
RGB camera ubiquitous in all modern computing devices. Our system achieved an
error of 1.8073cm on GazeCapture dataset without any calibration or device
specific fine-tuning. This research shows promise that one day soon any
computer, tablet, or phone will be controllable using just your eyes due to the
prediction capabilities of deep neutral networks.

negative:
Gaze estimation reveals where a person is looking. It is an important clue
for understanding human intention. The recent development of deep learning has
revolutionized many computer vision tasks, the appearance-based gaze estimation
is no exception. However, it lacks a guideline for designing deep learning
algorithms for gaze estimation tasks. In this paper, we present a comprehensive
review of the appearance-based gaze estimation methods with deep learning. We
summarize the processing pipeline and discuss these methods from four
perspectives: deep feature extraction, deep neural network architecture design,
personal calibration as well as device and platform. Since the data
pre-processing and post-processing methods are crucial for gaze estimation, we
also survey face/eye detection method, data rectification method, 2D/3D gaze
conversion method, and gaze origin conversion method. To fairly compare the
performance of various gaze estimation approaches, we characterize all the
publicly available gaze estimation datasets and collect the code of typical
gaze estimation algorithms. We implement these codes and set up a benchmark of
converting the results of different methods into the same evaluation metrics.
This paper not only serves as a reference to develop deep learning-based gaze
estimation methods but also a guideline for future gaze estimation research.
Implemented methods and data processing codes are available at
http://phi-ai.org/GazeHub.

negative:
With the emergence of Virtual and Mixed Reality (XR) devices, eye tracking
has received significant attention in the computer vision community. Eye gaze
estimation is a crucial component in XR -- enabling energy efficient rendering,
multi-focal displays, and effective interaction with content. In head-mounted
XR devices, the eyes are imaged off-axis to avoid blocking the field of view.
This leads to increased challenges in inferring eye related quantities and
simultaneously provides an opportunity to develop accurate and robust learning
based approaches. To this end, we present MagicEyes, the first large scale eye
dataset collected using real MR devices with comprehensive ground truth
labeling. MagicEyes includes $587$ subjects with $80,000$ images of
human-labeled ground truth and over $800,000$ images with gaze target labels.
We evaluate several state-of-the-art methods on MagicEyes and also propose a
new multi-task EyeNet model designed for detecting the cornea, glints and pupil
along with eye segmentation in a single forward pass.

negative:
Facial landmark detection, head pose estimation, and facial deformation
analysis are typical facial behavior analysis tasks in computer vision. The
existing methods usually perform each task independently and sequentially,
ignoring their interactions. To tackle this problem, we propose a unified
framework for simultaneous facial landmark detection, head pose estimation, and
facial deformation analysis, and the proposed model is robust to facial
occlusion. Following a cascade procedure augmented with model-based head pose
estimation, we iteratively update the facial landmark locations, facial
occlusion, head pose and facial de- formation until convergence. The
experimental results on benchmark databases demonstrate the effectiveness of
the proposed method for simultaneous facial landmark detection, head pose and
facial deformation estimation, even if the images are under facial occlusion.

negative:
A head-mounted display (HMD) could be an important component of augmented
reality system. However, as the upper face region is seriously occluded by the
device, the user experience could be affected in applications such as
telecommunication and multi-player video games. In this paper, we first present
a novel experimental setup that consists of two near-infrared (NIR) cameras to
point to the eye regions and one visible-light RGB camera to capture the
visible face region. The main purpose of this paper is to synthesize realistic
face images without occlusions based on the images captured by these cameras.
To this end, we propose a novel synthesis framework that contains four modules:
3D head reconstruction, face alignment and tracking, face synthesis, and eye
synthesis. In face synthesis, we propose a novel algorithm that can robustly
align and track a personalized 3D head model given a face that is severely
occluded by the HMD. In eye synthesis, in order to generate accurate eye
movements and dynamic wrinkle variations around eye regions, we propose another
novel algorithm to colorize the NIR eye images and further remove the "red eye"
effects caused by the colorization. Results show that both hardware setup and
system framework are robust to synthesize realistic face images in video
sequences.

negative:
Human eye gaze detection plays an important role in various fields, including
human-computer interaction, virtual reality and cognitive science. Although
different relatively accurate systems of eye tracking and gaze detection exist,
they are usually either too expensive to be bought for low cost applications or
too complex to be implemented easily. In this article, we propose a
non-wearable system for eye tracking and gaze detection with low complexity and
cost. The proposed system provides a medium accuracy which makes it suitable
for general applications in which low cost and easy implementation is more
important than achieving very precise gaze detection. The proposed method
includes pupil and marker detection using infrared image processing, and gaze
evaluation using an interpolation-based strategy. The interpolation-based
strategy exploits the positions of the detected pupils and markers in a target
captured image and also in some previously captured training images for
estimating the position of a point that the user is gazing at. The proposed
system has been evaluated by three users in two different lighting conditions.
The experimental results show that the accuracy of this low cost system can be
between 90% and 100% for finding major gazing directions.

negative:
We present a novel multistream network that learns robust eye representations
for gaze estimation. We first create a synthetic dataset containing eye region
masks detailing the visible eyeball and iris using a simulator. We then perform
eye region segmentation with a U-Net type model which we later use to generate
eye region masks for real-world eye images. Next, we pretrain an eye image
encoder in the real domain with self-supervised contrastive learning to learn
generalized eye representations. Finally, this pretrained eye encoder, along
with two additional encoders for visible eyeball region and iris, are used in
parallel in our multistream framework to extract salient features for gaze
estimation from real-world images. We demonstrate the performance of our method
on the EYEDIAP dataset in two different evaluation settings and achieve
state-of-the-art results, outperforming all the existing benchmarks on this
dataset. We also conduct additional experiments to validate the robustness of
our self-supervised network with respect to different amounts of labeled data
used for training.

negative:
Previous approaches to model and analyze facial expression analysis use three
different techniques: facial action units, geometric features and graph based
modelling. However, previous approaches have treated these technique
separately. There is an interrelationship between these techniques. The facial
expression analysis is significantly improved by utilizing these mappings
between major geometric features involved in facial expressions and the subset
of facial action units whose presence or absence are unique to a facial
expression. This paper combines dimension reduction techniques and image
classification with search space pruning achieved by this unique subset of
facial action units to significantly prune the search space. The performance
results on the publicly facial expression database shows an improvement in
performance by 70% over time while maintaining the emotion recognition
correctness.

negative:
Surveillance based on Computer Vision has become a major necessity in current
era. Most of the surveillance systems operate on visible light imaging, but
performance based on visible light imaging is limited due to some factors like
variation in light intensity during the daytime. The matter of concern lies in
the need for processing images in low light, such as in the need of nighttime
surveillance. In this paper, we have proposed a novel approach for human
detection using FLIR(Forward Looking Infrared) camera. As the principle
involves sensing based on thermal radiation in the Near IR Region, it is
possible to detect Humans from an image captured using a FLIR camera even in
low light. The proposed method for human detection involves processing of
Thermal images by using HOG (Histogram of Oriented Gradients) feature
extraction technique along with some enhancements. The principle of the
proposed technique lies in an adaptive background subtraction algorithm, which
works in association with the HOG technique. By means of this method, we are
able to reduce execution time, precision and some other parameters, which
result in improvement of overall accuracy of the human detection system.

negative:
In many visual systems, visual tracking often bases on RGB image sequences,
in which some targets are invalid in low-light conditions, and tracking
performance is thus affected significantly. Introducing other modalities such
as depth and infrared data is an effective way to handle imaging limitations of
individual sources, but multi-modal imaging platforms usually require elaborate
designs and cannot be applied in many real-world applications at present.
Near-infrared (NIR) imaging becomes an essential part of many surveillance
cameras, whose imaging is switchable between RGB and NIR based on the light
intensity. These two modalities are heterogeneous with very different visual
properties and thus bring big challenges for visual tracking. However, existing
works have not studied this challenging problem. In this work, we address the
cross-modal object tracking problem and contribute a new video dataset,
including 654 cross-modal image sequences with over 481K frames in total, and
the average video length is more than 735 frames. To promote the research and
development of cross-modal object tracking, we propose a new algorithm, which
learns the modality-aware target representation to mitigate the appearance gap
between RGB and NIR modalities in the tracking process. It is plug-and-play and
could thus be flexibly embedded into different tracking frameworks. Extensive
experiments on the dataset are conducted, and we demonstrate the effectiveness
of the proposed algorithm in two representative tracking frameworks against 17
state-of-the-art tracking methods. We will release the dataset for free
academic usage, dataset download link and code will be released soon.

negative:
Recognizing human affect and emotions is a problem that has a wide range of
applications within both academia and industry. Affect and emotion recognition
within computer vision primarily relies on images of faces. With the prevalence
of portable devices (e.g. smart phones and/or smart glasses),acquiring user
facial images requires focus, time, and precision. While existing systems work
great for full frontal faces, they tend to not work so well with partial faces
like those of the operator of the device when under use. Due to this, we
propose a methodology in which we can accurately infer the overall affect of a
person by looking at the ocular region of an individual.

negative:
We present labelled pupils in the wild (LPW), a novel dataset of 66
high-quality, high-speed eye region videos for the development and evaluation
of pupil detection algorithms. The videos in our dataset were recorded from 22
participants in everyday locations at about 95 FPS using a state-of-the-art
dark-pupil head-mounted eye tracker. They cover people with different
ethnicities, a diverse set of everyday indoor and outdoor illumination
environments, as well as natural gaze direction distributions. The dataset also
includes participants wearing glasses, contact lenses, as well as make-up. We
benchmark five state-of-the-art pupil detection algorithms on our dataset with
respect to robustness and accuracy. We further study the influence of image
resolution, vision aids, as well as recording location (indoor, outdoor) on
pupil detection performance. Our evaluations provide valuable insights into the
general pupil detection problem and allow us to identify key challenges for
robust pupil detection on head-mounted eye trackers.

negative:
Estimation eye gaze direction is useful in various human-computer interaction
tasks. Knowledge of gaze direction can give valuable information regarding
users point of attention. Certain patterns of eye movements known as eye
accessing cues are reported to be related to the cognitive processes in the
human brain. We propose a real-time framework for the classification of eye
gaze direction and estimation of eye accessing cues. In the first stage, the
algorithm detects faces using a modified version of the Viola-Jones algorithm.
A rough eye region is obtained using geometric relations and facial landmarks.
The eye region obtained is used in the subsequent stage to classify the eye
gaze direction. A convolutional neural network is employed in this work for the
classification of eye gaze direction. The proposed algorithm was tested on Eye
Chimera database and found to outperform state of the art methods. The
computational complexity of the algorithm is very less in the testing phase.
The algorithm achieved an average frame rate of 24 fps in the desktop
environment.

negative:
Despite significant advances in improving the gaze tracking accuracy under
controlled conditions, the tracking robustness under real-world conditions,
such as large head pose and movements, use of eyeglasses, illumination and eye
type variations, remains a major challenge in eye tracking. In this paper, we
revisit this challenge and introduce a real-time multi-camera eye tracking
framework to improve the tracking robustness. First, differently from previous
work, we design a multi-view tracking setup that allows for acquiring multiple
eye appearances simultaneously. Leveraging multi-view appearances enables to
more reliably detect gaze features under challenging conditions, particularly
when they are obstructed in conventional single-view appearance due to large
head movements or eyewear effects. The features extracted on various
appearances are then used for estimating multiple gaze outputs. Second, we
propose to combine estimated gaze outputs through an adaptive fusion mechanism
to compute user's overall point of regard. The proposed mechanism firstly
determines the estimation reliability of each gaze output according to user's
momentary head pose and predicted gazing behavior, and then performs a
reliability-based weighted fusion. We demonstrate the efficacy of our framework
with extensive simulations and user experiments on a collected dataset
featuring 20 subjects. Our results show that in comparison with
state-of-the-art eye trackers, the proposed framework provides not only a
significant enhancement in accuracy but also a notable robustness. Our
prototype system runs at 30 frames-per-second (fps) and achieves 1 degree
accuracy under challenging experimental scenarios, which makes it suitable for
applications demanding high accuracy and robustness.

negative:
Multi-task learning is an effective learning strategy for deep-learning-based
facial expression recognition tasks. However, most existing methods take into
limited consideration the feature selection, when transferring information
between different tasks, which may lead to task interference when training the
multi-task networks. To address this problem, we propose a novel selective
feature-sharing method, and establish a multi-task network for facial
expression recognition and facial expression synthesis. The proposed method can
effectively transfer beneficial features between different tasks, while
filtering out useless and harmful information. Moreover, we employ the facial
expression synthesis task to enlarge and balance the training dataset to
further enhance the generalization ability of the proposed method. Experimental
results show that the proposed method achieves state-of-the-art performance on
those commonly used facial expression recognition benchmarks, which makes it a
potential solution to real-world facial expression recognition problems.

negative:
Recent advances in facial expression synthesis have shown promising results
using diverse expression representations including facial action units. Facial
action units for an elaborate facial expression synthesis need to be
intuitively represented for human comprehension, not a numeric categorization
of facial action units. To address this issue, we utilize human-friendly
approach: use of natural language where language helps human grasp conceptual
contexts. In this paper, therefore, we propose a new facial expression
synthesis model from language-based facial expression description. Our method
can synthesize the facial image with detailed expressions. In addition,
effectively embedding language features on facial features, our method can
control individual word to handle each part of facial movement. Extensive
qualitative and quantitative evaluations were conducted to verify the
effectiveness of the natural language.

negative:
Image-based age estimation aims to predict a person's age from facial images.
It is used in a variety of real-world applications. Although end-to-end deep
models have achieved impressive results for age estimation on benchmark
datasets, their performance in-the-wild still leaves much room for improvement
due to the challenges caused by large variations in head pose, facial
expressions, and occlusions. To address this issue, we propose a simple yet
effective method to explicitly incorporate facial semantics into age
estimation, so that the model would learn to correctly focus on the most
informative facial components from unaligned facial images regardless of head
pose and non-rigid deformation. To this end, we design a face parsing-based
network to learn semantic information at different scales and a novel face
parsing attention module to leverage these semantic features for age
estimation. To evaluate our method on in-the-wild data, we also introduce a new
challenging large-scale benchmark called IMDB-Clean. This dataset is created by
semi-automatically cleaning the noisy IMDB-WIKI dataset using a constrained
clustering method. Through comprehensive experiment on IMDB-Clean and other
benchmark datasets, under both intra-dataset and cross-dataset evaluation
protocols, we show that our method consistently outperforms all existing age
estimation methods and achieves a new state-of-the-art performance. To the best
of our knowledge, our work presents the first attempt of leveraging face
parsing attention to achieve semantic-aware age estimation, which may be
inspiring to other high level facial analysis tasks. Code and data are
available on \url{https://github.com/ibug-group/fpage}.

negative:
Images of the eye are key in several computer vision problems, such as shape
registration and gaze estimation. Recent large-scale supervised methods for
these problems require time-consuming data collection and manual annotation,
which can be unreliable. We propose synthesizing perfectly labelled
photo-realistic training data in a fraction of the time. We used computer
graphics techniques to build a collection of dynamic eye-region models from
head scan geometry. These were randomly posed to synthesize close-up eye images
for a wide range of head poses, gaze directions, and illumination conditions.
We used our model's controllability to verify the importance of realistic
illumination and shape variations in eye-region training data. Finally, we
demonstrate the benefits of our synthesized training data (SynthesEyes) by
out-performing state-of-the-art methods for eye-shape registration as well as
cross-dataset appearance-based gaze estimation in the wild.

negative:
The interaction between the vestibular and ocular system has primarily been
studied in controlled environments. Consequently, off-the shelf tools for
categorization of gaze events (e.g. fixations, pursuits, saccade) fail when
head movements are allowed. Our approach was to collect a novel, naturalistic,
and multimodal dataset of eye+head movements when subjects performed everyday
tasks while wearing a mobile eye tracker equipped with an inertial measurement
unit and a 3D stereo camera. This Gaze-in-the-Wild dataset (GW) includes
eye+head rotational velocities (deg/s), infrared eye images and scene imagery
(RGB+D). A portion was labelled by coders into gaze motion events with a mutual
agreement of 0.72 sample based Cohen's $\kappa$. This labelled data was used to
train and evaluate two machine learning algorithms, Random Forest and a
Recurrent Neural Network model, for gaze event classification. Assessment
involved the application of established and novel event based performance
metrics. Classifiers achieve $\sim$90$\%$ human performance in detecting
fixations and saccades but fall short (60$\%$) on detecting pursuit movements.
Moreover, pursuit classification is far worse in the absence of head movement
information. A subsequent analysis of feature significance in our
best-performing model revealed a reliance upon absolute eye and head velocity,
indicating that classification does not require spatial alignment of the head
and eye tracking coordinate systems. The GW dataset, trained classifiers and
evaluation metrics will be made publicly available with the intention of
facilitating growth in the emerging area of head-free gaze event
classification.

negative:
Eye-based information channels include the pupils, gaze, saccades, fixational
movements, and numerous forms of eye opening and closure. Pupil size variation
indicates cognitive load and emotion, while a person's gaze direction is said
to be congruent with the motivation to approach or avoid stimuli. The eyelids
are involved in facial expressions that can encode basic emotions.
Additionally, eye-based cues can have implications for human annotators of
emotions or feelings. Despite these facts, the use of eye-based cues in
affective computing is in its infancy, however, and this work is intended to
start to address this. Eye-based feature sets, incorporating data from all of
the aforementioned information channels, that can be estimated from video are
proposed. Feature set refinement is provided by way of continuous arousal and
valence learning and prediction experiments on the RECOLA validation set. The
eye-based features are then combined with a speech feature set to provide
confirmation of their usefulness and assess affect prediction performance
compared with group-of-humans-level performance on the RECOLA test set. The
core contribution of this paper, a refined eye-based feature set, is shown to
provide benefits for affect prediction. It is hoped that this work stimulates
further research into eye-based affective computing.

negative:
In this paper, we propose an automatic approach for localizing the inner eye
canthus in thermal face images. We first coarsely detect 5 facial keypoints
corresponding to the center of the eyes, the nosetip and the ears. Then we
compute a sparse 2D-3D points correspondence using a 3D Morphable Face Model
(3DMM). This correspondence is used to project the entire 3D face onto the
image, and subsequently locate the inner eye canthus. Detecting this location
allows to obtain the most precise body temperature measurement for a person
using a thermal camera. We evaluated the approach on a thermal face dataset
provided with manually annotated landmarks. However, such manual annotations
are normally conceived to identify facial parts such as eyes, nose and mouth,
and are not specifically tailored for localizing the eye canthus region. As
additional contribution, we enrich the original dataset by using the annotated
landmarks to deform and project the 3DMM onto the images. Then, by manually
selecting a small region corresponding to the eye canthus, we enrich the
dataset with additional annotations. By using the manual landmarks, we ensure
the correctness of the 3DMM projection, which can be used as ground-truth for
future evaluations. Moreover, we supply the dataset with the 3D head poses and
per-point visibility masks for detecting self-occlusions. The data will be
publicly released.

negative:
Estimating eye-gaze from images alone is a challenging task, in large parts
due to un-observable person-specific factors. Achieving high accuracy typically
requires labeled data from test users which may not be attainable in real
applications. We observe that there exists a strong relationship between what
users are looking at and the appearance of the user's eyes. In response to this
understanding, we propose a novel dataset and accompanying method which aims to
explicitly learn these semantic and temporal relationships. Our video dataset
consists of time-synchronized screen recordings, user-facing camera views, and
eye gaze data, which allows for new benchmarks in temporal gaze tracking as
well as label-free refinement of gaze. Importantly, we demonstrate that the
fusion of information from visual stimuli as well as eye images can lead
towards achieving performance similar to literature-reported figures acquired
through supervised personalization. Our final method yields significant
performance improvements on our proposed EVE dataset, with up to a 28 percent
improvement in Point-of-Gaze estimates (resulting in 2.49 degrees in angular
error), paving the path towards high-accuracy screen-based eye tracking purely
from webcam sensors. The dataset and reference source code are available at
https://ait.ethz.ch/projects/2020/EVE

negative:
Onfocus detection aims at identifying whether the focus of the individual
captured by a camera is on the camera or not. Based on the behavioral research,
the focus of an individual during face-to-camera communication leads to a
special type of eye contact, i.e., the individual-camera eye contact, which is
a powerful signal in social communication and plays a crucial role in
recognizing irregular individual status (e.g., lying or suffering mental
disease) and special purposes (e.g., seeking help or attracting fans). Thus,
developing effective onfocus detection algorithms is of significance for
assisting the criminal investigation, disease discovery, and social behavior
analysis. However, the review of the literature shows that very few efforts
have been made toward the development of onfocus detector due to the lack of
large-scale public available datasets as well as the challenging nature of this
task. To this end, this paper engages in the onfocus detection research by
addressing the above two issues. Firstly, we build a large-scale onfocus
detection dataset, named as the OnFocus Detection In the Wild (OFDIW). It
consists of 20,623 images in unconstrained capture conditions (thus called ``in
the wild'') and contains individuals with diverse emotions, ages, facial
characteristics, and rich interactions with surrounding objects and background
scenes. On top of that, we propose a novel end-to-end deep model, i.e., the
eye-context interaction inferring network (ECIIN), for onfocus detection, which
explores eye-context interaction via dynamic capsule routing. Finally,
comprehensive experiments are conducted on the proposed OFDIW dataset to
benchmark the existing learning models and demonstrate the effectiveness of the
proposed ECIIN. The project (containing both datasets and codes) is at
https://github.com/wintercho/focus.

negative:
We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.

negative:
Driving support systems, such as car navigation systems are becoming common
and they support driver in several aspects. Non-intrusive method of detecting
Fatigue and drowsiness based on eye-blink count and eye directed instruction
controlhelps the driver to prevent from collision caused by drowsy driving. Eye
detection and tracking under various conditions such as illumination,
background, face alignment and facial expression makes the problem
complex.Neural Network based algorithm is proposed in this paper to detect the
eyes efficiently. In the proposed algorithm, first the neural Network is
trained to reject the non-eye regionbased on images with features of eyes and
the images with features of non-eye using Gabor filter and Support Vector
Machines to reduce the dimension and classify efficiently. In the algorithm,
first the face is segmented using L*a*btransform color space, then eyes are
detected using HSV and Neural Network approach. The algorithm is tested on
nearly 100 images of different persons under different conditions and the
results are satisfactory with success rate of 98%.The Neural Network is trained
with 50 non-eye images and 50 eye images with different angles using Gabor
filter. This paper is a part of research work on "Development of Non-Intrusive
system for real-time Monitoring and Prediction of Driver Fatigue and
drowsiness" project sponsored by Department of Science & Technology, Govt. of
India, New Delhi at Vignan Institute of Technology and Sciences, Vignan Hills,
Hyderabad.

negative:
Human face analysis is an important task in computer vision. According to
cognitive-psychological studies, facial dynamics could provide crucial cues for
face analysis. The motion of a facial local region in facial expression is
related to the motion of other facial local regions. In this paper, a novel
deep learning approach, named facial dynamics interpreter network, has been
proposed to interpret the important relations between local dynamics for
estimating facial traits from expression sequence. The facial dynamics
interpreter network is designed to be able to encode a relational importance,
which is used for interpreting the relation between facial local dynamics and
estimating facial traits. By comparative experiments, the effectiveness of the
proposed method has been verified. The important relations between facial local
dynamics are investigated by the proposed facial dynamics interpreter network
in gender classification and age estimation. Moreover, experimental results
show that the proposed method outperforms the state-of-the-art methods in
gender classification and age estimation.

negative:
Video-to-video synthesis is a challenging problem aiming at learning a
translation function between a sequence of semantic maps and a photo-realistic
video depicting the characteristics of a driving video. We propose a
head-to-head system of our own implementation capable of fully transferring the
human head 3D pose, facial expressions and eye gaze from a source to a target
actor, while preserving the identity of the target actor. Our system produces
high-fidelity, temporally-smooth and photo-realistic synthetic videos
faithfully transferring the human time-varying head attributes from the source
to the target actor. Our proposed implementation: 1) works in real time ($\sim
20$ fps), 2) runs on a commodity laptop with a webcam as the only input, 3) is
interactive, allowing the participant to drive a target person, e.g. a
celebrity, politician, etc, instantly by varying their expressions, head pose,
and eye gaze, and visualising the synthesised video concurrently.

negative:
To date, most facial expression analysis have been based on posed image
databases and is carried out without being able to protect the identity of the
subjects whose expressions are being recognised. In this paper, we propose and
implement a system for classifying facial expressions of images in the
encrypted domain based on a Paillier cryptosystem implementation of Fisher
Linear Discriminant Analysis and k-nearest neighbour (FLDA + kNN). We present
results of experiments carried out on a recently developed natural visible and
infrared facial expression (NVIE) database of spontaneous images. To the best
of our knowledge, this is the first system that will allow the recog-nition of
encrypted spontaneous facial expressions by a remote server on behalf of a
client.

negative:
In this work, we propose a novel approach for generating videos of the six
basic facial expressions given a neutral face image. We propose to exploit the
face geometry by modeling the facial landmarks motion as curves encoded as
points on a hypersphere. By proposing a conditional version of manifold-valued
Wasserstein generative adversarial network (GAN) for motion generation on the
hypersphere, we learn the distribution of facial expression dynamics of
different classes, from which we synthesize new facial expression motions. The
resulting motions can be transformed to sequences of landmarks and then to
images sequences by editing the texture information using another conditional
Generative Adversarial Network. To the best of our knowledge, this is the first
work that explores manifold-valued representations with GAN to address the
problem of dynamic facial expression generation. We evaluate our proposed
approach both quantitatively and qualitatively on two public datasets;
Oulu-CASIA and MUG Facial Expression. Our experimental results demonstrate the
effectiveness of our approach in generating realistic videos with continuous
motion, realistic appearance and identity preservation. We also show the
efficiency of our framework for dynamic facial expressions generation, dynamic
facial expression transfer and data augmentation for training improved emotion
recognition models.

negative:
Facial feature detection from facial images has attracted great attention in
the field of computer vision. It is a nontrivial task since the appearance and
shape of the face tend to change under different conditions. In this paper, we
propose a hierarchical probabilistic model that could infer the true locations
of facial features given the image measurements even if the face is with
significant facial expression and pose. The hierarchical model implicitly
captures the lower level shape variations of facial components using the
mixture model. Furthermore, in the higher level, it also learns the joint
relationship among facial components, the facial expression, and the pose
information through automatic structure learning and parameter estimation of
the probabilistic model. Experimental results on benchmark databases
demonstrate the effectiveness of the proposed hierarchical probabilistic model.

negative:
Eye movements play a vital role in perceiving the world. Eye gaze can give a
direct indication of the users point of attention, which can be useful in
improving human-computer interaction. Gaze estimation in a non-intrusive manner
can make human-computer interaction more natural. Eye tracking can be used for
several applications such as fatigue detection, biometric authentication,
disease diagnosis, activity recognition, alertness level estimation,
gaze-contingent display, human-computer interaction, etc. Even though
eye-tracking technology has been around for many decades, it has not found much
use in consumer applications. The main reasons are the high cost of eye
tracking hardware and lack of consumer level applications. In this work, we
attempt to address these two issues. In the first part of this work,
image-based algorithms are developed for gaze tracking which includes a new
two-stage iris center localization algorithm. We have developed a new algorithm
which works in challenging conditions such as motion blur, glint, and varying
illumination levels. A person independent gaze direction classification
framework using a convolutional neural network is also developed which
eliminates the requirement of user-specific calibration.
  In the second part of this work, we have developed two applications which can
benefit from eye tracking data. A new framework for biometric identification
based on eye movement parameters is developed. A framework for activity
recognition, using gaze data from a head-mounted eye tracker is also developed.
The information from gaze data, ego-motion, and visual features are integrated
to classify the activities.

negative:
Photorealistic facial expression synthesis from single face image can be
widely applied to face recognition, data augmentation for emotion recognition
or entertainment. This problem is challenging, in part due to a paucity of
labeled facial expression data, making it difficult for algorithms to
disambiguate changes due to identity and changes due to expression. In this
paper, we propose the conditional difference adversarial autoencoder, CDAAE,
for facial expression synthesis. The CDAAE takes a facial image of a previously
unseen person and generates an image of that human face with a target emotion
or facial action unit label. The CDAAE adds a feedforward path to an
autoencoder structure connecting low level features at the encoder to features
at the corresponding level at the decoder. It handles the problem of
disambiguating changes due to identity and changes due to facial expression by
learning to generate the difference between low-level features of images of the
same person but with different facial expressions. The CDAAE structure can be
used to generate novel expressions by combining and interpolating between
facial expressions/action units within the training set. Our experimental
results demonstrate that the CDAAE can preserve identity information when
generating facial expression for unseen subjects more faithfully than previous
approaches. This is especially advantageous when training with small databases.

negative:
Recent research has demonstrated the ability to estimate gaze on mobile
devices by performing inference on the image from the phone's front-facing
camera, and without requiring specialized hardware. While this offers wide
potential applications such as in human-computer interaction, medical diagnosis
and accessibility (e.g., hands free gaze as input for patients with motor
disorders), current methods are limited as they rely on collecting data from
real users, which is a tedious and expensive process that is hard to scale
across devices. There have been some attempts to synthesize eye region data
using 3D models that can simulate various head poses and camera settings,
however these lack in realism.
  In this paper, we improve upon a recently suggested method, and propose a
generative adversarial framework to generate a large dataset of high resolution
colorful images with high diversity (e.g., in subjects, head pose, camera
settings) and realism, while simultaneously preserving the accuracy of gaze
labels. The proposed approach operates on extended regions of the eye, and even
completes missing parts of the image. Using this rich synthesized dataset, and
without using any additional training data from real users, we demonstrate
improvements over state-of-the-art for estimating 2D gaze position on mobile
devices. We further demonstrate cross-device generalization of model
performance, as well as improved robustness to diverse head pose, blur and
distance.

negative:
Facial expressions are widely used in the behavioral interpretation of
emotions, cognitive science, and social interactions. In this paper, we present
a novel method for fully automatic facial expression recognition in facial
image sequences. As the facial expression evolves over time facial landmarks
are automatically tracked in consecutive video frames, using displacements
based on elastic bunch graph matching displacement estimation. Feature vectors
from individual landmarks, as well as pairs of landmarks tracking results are
extracted, and normalized, with respect to the first frame in the sequence. The
prototypical expression sequence for each class of facial expression is formed,
by taking the median of the landmark tracking results from the training facial
expression sequences. Multi-class AdaBoost with dynamic time warping similarity
distance between the feature vector of input facial expression and prototypical
facial expression, is used as a weak classifier to select the subset of
discriminative feature vectors. Finally, two methods for facial expression
recognition are presented, either by using multi-class AdaBoost with dynamic
time warping, or by using support vector machine on the boosted feature
vectors. The results on the Cohn-Kanade (CK+) facial expression database show a
recognition accuracy of 95.17% and 97.35% using multi-class AdaBoost and
support vector machines, respectively.

negative:
Recognition of human emotions from the imaging templates is useful in a wide
variety of human-computer interaction and intelligent systems applications.
However, the automatic recognition of facial expressions using image template
matching techniques suffer from the natural variability with facial features
and recording conditions. In spite of the progress achieved in facial emotion
recognition in recent years, the effective and computationally simple feature
selection and classification technique for emotion recognition is still an open
problem. In this paper, we propose an efficient and straightforward facial
emotion recognition algorithm to reduce the problem of inter-class pixel
mismatch during classification. The proposed method includes the application of
pixel normalization to remove intensity offsets followed-up with a Min-Max
metric in a nearest neighbor classifier that is capable of suppressing feature
outliers. The results indicate an improvement of recognition performance from
92.85% to 98.57% for the proposed Min-Max classification method when tested on
JAFFE database. The proposed emotion recognition technique outperforms the
existing template matching methods.

negative:
Facial recognition is changing the way we live in and interact with our
society. Here we discuss the two sides of facial recognition, summarizing
potential risks and current concerns. We introduce current policies and
regulations in different countries. Very importantly, we point out that the
risks and concerns are not only from facial recognition, but also realistically
very similar to other biometric recognition technology, including but not
limited to gait recognition, iris recognition, fingerprint recognition, voice
recognition, etc. To create a responsible future, we discuss possible
technological moves and efforts that should be made to keep facial recognition
(and biometric recognition in general) developing for social good.

negative:
The randomness and uniqueness of human eye patterns is a major breakthrough
in the search for quicker, easier and highly reliable forms of automatic human
identification. It is being used extensively in security solutions. This
includes access control to physical facilities, security systems and
information databases, Suspect tracking, surveillance and intrusion detection
and by various Intelligence agencies through out the world. We use the
advantage of human eye uniqueness to identify people and approve its validity
as a biometric. . Eye detection involves first extracting the eye from a
digital face image, and then encoding the unique patterns of the eye in such a
way that they can be compared with pre-registered eye patterns. The eye
detection system consists of an automatic segmentation system that is based on
the wavelet transform, and then the Wavelet analysis is used as a pre-processor
for a back propagation neural network with conjugate gradient learning. The
inputs to the neural network are the wavelet maxima neighborhood coefficients
of face images at a particular scale. The output of the neural network is the
classification of the input into an eye or non-eye region. An accuracy of 90%
is observed for identifying test images under different conditions included in
training stage.

negative:
Gaze correction aims to redirect the person's gaze into the camera by
manipulating the eye region, and it can be considered as a specific image
resynthesis problem. Gaze correction has a wide range of applications in real
life, such as taking a picture with staring at the camera. In this paper, we
propose a novel method that is based on the inpainting model to learn from the
face image to fill in the missing eye regions with new contents representing
corrected eye gaze. Moreover, our model does not require the training dataset
labeled with the specific head pose and eye angle information, thus, the
training data is easy to collect. To retain the identity information of the eye
region in the original input, we propose a self-guided pretrained model to
learn the angle-invariance feature. Experiments show our model achieves very
compelling gaze-corrected results in the wild dataset which is collected from
the website and will be introduced in details. Code is available at
https://github.com/zhangqianhui/GazeCorrection.

negative:
Model-based eye tracking has been a dominant approach for eye gaze tracking
because of its ability to generalize to different subjects, without the need of
any training data and eye gaze annotations. Model-based eye tracking, however,
is susceptible to eye feature detection errors, in particular for eye tracking
in the wild. To address this issue, we propose a Bayesian framework for
model-based eye tracking. The proposed system consists of a cascade-Bayesian
Convolutional Neural Network (c-BCNN) to capture the probabilistic
relationships between eye appearance and its landmarks, and a geometric eye
model to estimate eye gaze from the eye landmarks. Given a testing eye image,
the Bayesian framework can generate, through Bayesian inference, the eye gaze
distribution without explicit landmark detection and model training, based on
which it not only estimates the most likely eye gaze but also its uncertainty.
Furthermore, with Bayesian inference instead of point-based inference, our
model can not only generalize better to different sub-jects, head poses, and
environments but also is robust to image noise and landmark detection errors.
Finally, with the estimated gaze uncertainty, we can construct a cascade
architecture that allows us to progressively improve gaze estimation accuracy.
Compared to state-of-the-art model-based and learning-based methods, the
proposed Bayesian framework demonstrates significant improvement in
generalization capability across several benchmark datasets and in accuracy and
robustness under challenging real-world conditions.

negative:
Facial expression recognition is a challenging task due to two major
problems: the presence of inter-subject variations in facial expression
recognition dataset and impure expressions posed by human subjects. In this
paper we present a novel Human-to-Animation conditional Generative Adversarial
Network (HA-GAN) to overcome these two problems by using many (human faces) to
one (animated face) mapping. Specifically, for any given input human expression
image, our HA-GAN transfers the expression information from the input image to
a fixed animated identity. Stylized animated characters from the Facial
Expression Research Group-Database (FERGDB) are used for the generation of
fixed identity. By learning this many-to-one identity mapping function using
our proposed HA-GAN, the effect of inter-subject variations can be reduced in
Facial Expression Recognition(FER). We also argue that the expressions in the
generated animated images are pure expressions and since FER is performed on
these generated images, the performance of facial expression recognition is
improved. Our initial experimental results on the state-of-the-art datasets
show that facial expression recognition carried out on the generated animated
images using our HA-GAN framework outperforms the baseline deep neural network
and produces comparable or even better results than the state-of-the-art
methods for facial expression recognition.

negative:
The infrared small-dim target detection is one of the key techniques in the
infrared search and tracking system. Since the local regions similar to
infrared small-dim targets spread over the whole background, exploring the
interaction information amongst image features in large-range dependencies to
mine the difference between the target and background is crucial for robust
detection. However, existing deep learning-based methods are limited by the
locality of convolutional neural networks, which impairs the ability to capture
large-range dependencies. Additionally, the small-dim appearance of the
infrared target makes the detection model highly possible to miss detection. To
this end, we propose a robust and general infrared small-dim target detection
method with the transformer. We adopt the self-attention mechanism of the
transformer to learn the interaction information of image features in a larger
range. Moreover, we design a feature enhancement module to learn discriminative
features of small-dim targets to avoid miss detection. After that, to avoid the
loss of the target information, we adopt a decoder with the U-Net-like skip
connection operation to contain more information of small-dim targets. Finally,
we get the detection result by a segmentation head. Extensive experiments on
two public datasets show the obvious superiority of the proposed method over
state-of-the-art methods and the proposed method has stronger cross-scene
generalization and anti-noise performance.

negative:
The muscular activities caused the activation of certain AUs for every facial
expression at the certain duration of time throughout the facial expression.
This paper presents the methods to recognise facial Action Unit (AU) using
facial distance of the facial features which activates the muscles. The seven
facial action units involved are AU1, AU4, AU6, AU12, AU15, AU17 and AU25 that
characterises happy and sad expression. The recognition is performed on each AU
according to rules defined based on the distance of each facial points. The
facial distances chosen are extracted from twelve facial features. Then the
facial distances are trained using Support Vector Machine (SVM) and Neural
Network (NN). Classification result using SVM is presented with several
different SVM kernels while result using NN is presented for each training,
validation and testing phase.

negative:
Facial expression is the most natural means for human beings to communicate
their emotions. Most facial expression analysis studies consider the case of
acted expressions. Spontaneous facial expression recognition is significantly
more challenging since each person has a different way to react to a given
emotion. We consider the problem of recognizing spontaneous facial expression
by learning discriminative dictionaries for sparse representation. Facial
images are represented as a sparse linear combination of prototype atoms via
Orthogonal Matching Pursuit algorithm. Sparse codes are then used to train an
SVM classifier dedicated to the recognition task. The dictionary that
sparsifies the facial images (feature points with the same class labels should
have similar sparse codes) is crucial for robust classification. Learning
sparsifying dictionaries heavily relies on the initialization process of the
dictionary. To improve the performance of dictionaries, a random face feature
descriptor based on the Random Projection concept is developed. The
effectiveness of the proposed method is evaluated through several experiments
on the spontaneous facial expressions DynEmo database. It is also estimated on
the well-known acted facial expressions JAFFE database for a purpose of
comparison with state-of-the-art methods.

negative:
This thesis describes the development of fast algorithms for the computation
of PERcentage CLOSure of eyes (PERCLOS) and Saccadic Ratio (SR). PERCLOS and SR
are two ocular parameters reported to be measures of alertness levels in human
beings. PERCLOS is the percentage of time in which at least 80% of the eyelid
remains closed over the pupil. Saccades are fast and simultaneous movement of
both the eyes in the same direction. SR is the ratio of peak saccadic velocity
to the saccadic duration. This thesis addresses the issues of image based
estimation of PERCLOS and SR, prevailing in the literature such as illumination
variation, poor illumination conditions, head rotations etc. In this work,
algorithms for real-time PERCLOS computation has been developed and implemented
on an embedded platform. The platform has been used as a case study for
assessment of loss of attention in automotive drivers. The SR estimation has
been carried out offline as real-time implementation requires high frame rates
of processing which is difficult to achieve due to hardware limitations. The
accuracy in estimation of the loss of attention using PERCLOS and SR has been
validated using brain signals, which are reported to be an authentic cue for
estimating the state of alertness in human beings. The major contributions of
this thesis include database creation, design and implementation of fast
algorithms for estimating PERCLOS and SR on embedded computing platforms.

negative:
A user's eyes provide means for Human Computer Interaction (HCI) research as
an important modal. The time to time scientific explorations of the eye has
already seen an upsurge of the benefits in HCI applications from gaze
estimation to the measure of attentiveness of a user looking at a screen for a
given time period. The eye tracking system as an assisting, interactive tool
can be incorporated by physically disabled individuals, fitted best for those
who have eyes as only a limited set of communication. The threefold objective
of this paper is - 1. To introduce a neural network based architecture to
predict users' gaze at 9 positions displayed in the 11.31{\deg} visual range on
the screen, through a low resolution based system such as a webcam in real time
by learning various aspects of eyes as an ocular feature set. 2.A collection of
coarsely supervised feature set obtained in real time which is also validated
through the user case study presented in the paper for 21 individuals ( 17 men
and 4 women ) from whom a 35k set of instances was derived with an accuracy
score of 82.36% and f1_score of 82.2% and 3.A detailed study over applicability
and underlying challenges of such systems. The experimental results verify the
feasibility and validity of the proposed eye gaze tracking model.

negative:
The ability to edit facial expressions has a wide range of applications in
computer graphics. The ideal facial expression editing algorithm needs to
satisfy two important criteria. First, it should allow precise and targeted
editing of individual facial actions. Second, it should generate high fidelity
outputs without artifacts. We build a solution based on StyleGAN, which has
been used extensively for semantic manipulation of faces. As we do so, we add
to our understanding of how various semantic attributes are encoded in
StyleGAN. In particular, we show that a naive strategy to perform editing in
the latent space results in undesired coupling between certain action units,
even if they are conceptually distinct. For example, although brow lowerer and
lip tightener are distinct action units, they appear correlated in the training
data. Hence, StyleGAN has difficulty in disentangling them. We allow
disentangled editing of such action units by computing detached regions of
influence for each action unit, and restrict editing to these regions. We
validate the effectiveness of our local editing method through perception
experiments conducted with 23 subjects. The results show that our method
provides higher control over local editing and produces images with superior
fidelity compared to the state-of-the-art methods.

negative:
The locations of the fiducial facial landmark points around facial components
and facial contour capture the rigid and non-rigid facial deformations due to
head movements and facial expressions. They are hence important for various
facial analysis tasks. Many facial landmark detection algorithms have been
developed to automatically detect those key points over the years, and in this
paper, we perform an extensive review of them. We classify the facial landmark
detection algorithms into three major categories: holistic methods, Constrained
Local Model (CLM) methods, and the regression-based methods. They differ in the
ways to utilize the facial appearance and shape information. The holistic
methods explicitly build models to represent the global facial appearance and
shape information. The CLMs explicitly leverage the global shape model but
build the local appearance models. The regression-based methods implicitly
capture facial shape and appearance information. For algorithms within each
category, we discuss their underlying theories as well as their differences. We
also compare their performances on both controlled and in the wild benchmark
datasets, under varying facial expressions, head poses, and occlusion. Based on
the evaluations, we point out their respective strengths and weaknesses. There
is also a separate section to review the latest deep learning-based algorithms.
  The survey also includes a listing of the benchmark databases and existing
software. Finally, we identify future research directions, including combining
methods in different categories to leverage their respective strengths to solve
landmark detection "in-the-wild".

negative:
Deep neural networks have been widely used for feature learning in facial
expression recognition systems. However, small datasets and large intra-class
variability can lead to overfitting. In this paper, we propose a method which
learns an optimized compact network topology for real-time facial expression
recognition utilizing localized facial landmark features. Our method employs a
spatio-temporal bilinear layer as backbone to capture the motion of facial
landmarks during the execution of a facial expression effectively. Besides, it
takes advantage of Monte Carlo Dropout to capture the model's uncertainty which
is of great importance to analyze and treat uncertain cases. The performance of
our method is evaluated on three widely used datasets and it is comparable to
that of video-based state-of-the-art methods while it has much less complexity.

negative:
Near-infrared gray images captured together with corresponding visible color
images have recently proven useful for image restoration and classification.
This paper introduces a new coloring method to add colors to near-infrared gray
images based on a contrast-preserving mapping model. A naive coloring method
directly adds the colors from the visible color image to the near-infrared gray
image; however, this method results in an unrealistic image because of the
discrepancies in brightness and image structure between the captured
near-infrared gray image and the visible color image. To solve the discrepancy
problem, first we present a new contrast-preserving mapping model to create a
new near-infrared gray image with a similar appearance in the luminance plane
to the visible color image, while preserving the contrast and details of the
captured near-infrared gray image. Then based on the proposed
contrast-preserving mapping model, we develop a method to derive realistic
colors that can be added to the newly created near-infrared gray image.
Experimental results show that the proposed method can not only preserve the
local contrasts and details of the captured near-infrared gray image, but
transfers the realistic colors from the visible color image to the newly
created near-infrared gray image. Experimental results also show that the
proposed approach can be applied to near-infrared denoising.

negative:
In this paper, we introduce a novel deep learning method for photo-realistic
manipulation of the emotional state of actors in "in-the-wild" videos. The
proposed method is based on a parametric 3D face representation of the actor in
the input scene that offers a reliable disentanglement of the facial identity
from the head pose and facial expressions. It then uses a novel deep domain
translation framework that alters the facial expressions in a consistent and
plausible manner, taking into account their dynamics. Finally, the altered
facial expressions are used to photo-realistically manipulate the facial region
in the input scene based on an especially-designed neural face renderer. To the
best of our knowledge, our method is the first to be capable of controlling the
actor's facial expressions by even using as a sole input the semantic labels of
the manipulated emotions, while at the same time preserving the speech-related
lip movements. We conduct extensive qualitative and quantitative evaluations
and comparisons, which demonstrate the effectiveness of our approach and the
especially promising results that we obtain. Our method opens a plethora of new
possibilities for useful applications of neural rendering technologies, ranging
from movie post-production and video games to photo-realistic affective
avatars.

negative:
Analyzing the gaze accuracy characteristics of an eye tracker is a critical
task as its gaze data is frequently affected by non-ideal operating conditions
in various consumer eye tracking applications. In this study, gaze error
patterns produced by a commercial eye tracking device were studied with the
help of machine learning algorithms, such as classifiers and regression models.
Gaze data were collected from a group of participants under multiple conditions
that commonly affect eye trackers operating on desktop and handheld platforms.
These conditions (referred here as error sources) include user distance, head
pose, and eye-tracker pose variations, and the collected gaze data were used to
train the classifier and regression models. It was seen that while the impact
of the different error sources on gaze data characteristics were nearly
impossible to distinguish by visual inspection or from data statistics, machine
learning models were successful in identifying the impact of the different
error sources and predicting the variability in gaze error levels due to these
conditions. The objective of this study was to investigate the efficacy of
machine learning methods towards the detection and prediction of gaze error
patterns, which would enable an in-depth understanding of the data quality and
reliability of eye trackers under unconstrained operating conditions. Coding
resources for all the machine learning methods adopted in this study were
included in an open repository named MLGaze to allow researchers to replicate
the principles presented here using data from their own eye trackers.

negative:
This report explores the latest advances in the field of digital document
recognition. With the focus on printed document imagery, we discuss the major
developments in optical character recognition (OCR) and document image
enhancement/restoration in application to Latin and non-Latin scripts. In
addition, we review and discuss the available technologies for hand-written
document recognition. In this report, we also provide some company-accumulated
benchmark results on available OCR engines.

negative:
In this paper, a modified algorithm for the detection of nasal and temporal
eye corners is presented. The algorithm is a modification of the Santos and
Proenka Method. In the first step, we detect the face and the eyes using
classifiers based on Haar-like features. We then segment out the sclera, from
the detected eye region. From the segmented sclera, we segment out an
approximate eyelid contour. Eye corner candidates are obtained using Harris and
Stephens corner detector. We introduce a post-pruning of the Eye corner
candidates to locate the eye corners, finally. The algorithm has been tested on
Yale, JAFFE databases as well as our created database.

negative:
Despite its great success, deep learning severely suffers from robustness;
that is, deep neural networks are very vulnerable to adversarial attacks, even
the simplest ones. Inspired by recent advances in brain science, we propose the
Denoised Internal Models (DIM), a novel generative autoencoder-based model to
tackle this challenge. Simulating the pipeline in the human brain for visual
signal processing, DIM adopts a two-stage approach. In the first stage, DIM
uses a denoiser to reduce the noise and the dimensions of inputs, reflecting
the information pre-processing in the thalamus. Inspired from the sparse coding
of memory-related traces in the primary visual cortex, the second stage
produces a set of internal models, one for each category. We evaluate DIM over
42 adversarial attacks, showing that DIM effectively defenses against all the
attacks and outperforms the SOTA on the overall robustness.

negative:
This paper presents a new state-of-the-art for document image classification
and retrieval, using features learned by deep convolutional neural networks
(CNNs). In object and scene analysis, deep neural nets are capable of learning
a hierarchical chain of abstraction from pixel inputs to concise and
descriptive representations. The current work explores this capacity in the
realm of document analysis, and confirms that this representation strategy is
superior to a variety of popular hand-crafted alternatives. Experiments also
show that (i) features extracted from CNNs are robust to compression, (ii) CNNs
trained on non-document images transfer well to document analysis tasks, and
(iii) enforcing region-specific feature-learning is unnecessary given
sufficient training data. This work also makes available a new labelled subset
of the IIT-CDIP collection, containing 400,000 document images across 16
categories, useful for training new CNNs for document analysis.

negative:
Most existing person re-identification (ReID) methods have good feature
representations to distinguish pedestrians with deep convolutional neural
network (CNN) and metric learning methods. However, these works concentrate on
the similarity between encoder output and ground-truth, ignoring the
correlation between input and encoder output, which affects the performance of
identifying different pedestrians. To address this limitation, We design a Deep
InfoMax (DIM) network to maximize the mutual information (MI) between the input
image and encoder output, which doesn't need any auxiliary labels. To evaluate
the effectiveness of the DIM network, we propose end-to-end Global-DIM and
Local-DIM models. Additionally, the DIM network provides a new solution for
cross-dataset unsupervised ReID issue as it needs no extra labels. The
experiments prove the superiority of MI theory on the ReID issue, which
achieves the state-of-the-art results.

negative:
This paper addresses the following questions pertaining to the intrinsic
dimensionality of any given image representation: (i) estimate its intrinsic
dimensionality, (ii) develop a deep neural network based non-linear mapping,
dubbed DeepMDS, that transforms the ambient representation to the minimal
intrinsic space, and (iii) validate the veracity of the mapping through image
matching in the intrinsic space. Experiments on benchmark image datasets (LFW,
IJB-C and ImageNet-100) reveal that the intrinsic dimensionality of deep neural
network representations is significantly lower than the dimensionality of the
ambient features. For instance, SphereFace's 512-dim face representation and
ResNet's 512-dim image representation have an intrinsic dimensionality of 16
and 19 respectively. Further, the DeepMDS mapping is able to obtain a
representation of significantly lower dimensionality while maintaining
discriminative ability to a large extent, 59.75% TAR @ 0.1% FAR in 16-dim vs
71.26% TAR in 512-dim on IJB-C and a Top-1 accuracy of 77.0% at 19-dim vs 83.4%
at 512-dim on ImageNet-100.

negative:
In this paper, we focus on the problem of unsupervised image-sentence
matching. Existing research explores to utilize document-level structural
information to sample positive and negative instances for model training.
Although the approach achieves positive results, it introduces a sampling bias
and fails to distinguish instances with high semantic similarity. To alleviate
the bias, we propose a new sampling strategy to select additional
intra-document image-sentence pairs as positive or negative samples.
Furthermore, to recognize the complex pattern in intra-document samples, we
propose a Transformer based model to capture fine-grained features and
implicitly construct a graph for each document, where concepts in a document
are introduced to bridge the representation learning of images and sentences in
the context of a document. Experimental results show the effectiveness of our
approach to alleviate the bias and learn well-aligned multimodal
representations.

negative:
Rapid increase of digitized document give birth to high demand of document
image retrieval. While conventional document image retrieval approaches depend
on complex OCR-based text recognition and text similarity detection, this paper
proposes a new content-based approach, in which more attention is paid to
features extraction and fusion. In the proposed approach, multiple features of
document images are extracted by different CNN models. After that, the
extracted CNN features are reduced and fused into weighted average feature.
Finally, the document images are ranked based on feature similarity to a
provided query image. Experimental procedure is performed on a group of
document images that transformed from academic papers, which contain both
English and Chinese document, the results show that the proposed approach has
good ability to retrieve document images with similar text content, and the
fusion of CNN features can effectively improve the retrieval accuracy.

negative:
Document segmentation is a method of rending the document into distinct
regions. A document is an assortment of information and a standard mode of
conveying information to others. Pursuance of data from documents involves ton
of human effort, time intense and might severely prohibit the usage of data
systems. So, automatic information pursuance from the document has become a big
issue. It is been shown that document segmentation will facilitate to beat such
problems. This paper proposes a new approach to segment and classify the
document regions as text, image, drawings and table. Document image is divided
into blocks using Run length smearing rule and features are extracted from
every blocks. Discipulus tool has been used to construct the Genetic
programming based classifier model and located 97.5% classification accuracy.

negative:
Recently, face super-resolution (FSR) methods either feed whole face image
into convolutional neural networks (CNNs) or utilize extra facial priors (e.g.,
facial parsing maps, facial landmarks) to focus on facial structure, thereby
maintaining the consistency of the facial structure while restoring facial
details. However, the limited receptive fields of CNNs and inaccurate facial
priors will reduce the naturalness and fidelity of the reconstructed face. In
this paper, we propose a novel paradigm based on the self-attention mechanism
(i.e., the core of Transformer) to fully explore the representation capacity of
the facial structure feature. Specifically, we design a Transformer-CNN
aggregation network (TANet) consisting of two paths, in which one path uses
CNNs responsible for restoring fine-grained facial details while the other
utilizes a resource-friendly Transformer to capture global information by
exploiting the long-distance visual relation modeling. By aggregating the
features from the above two paths, the consistency of global facial structure
and fidelity of local facial detail restoration are strengthened
simultaneously. Experimental results of face reconstruction and recognition
verify that the proposed method can significantly outperform the
state-of-the-art methods.