Tracking Emotions using an Evolutionary Model of Mental State Transitions: Introducing a New Paradigm

Intelligent Computing (A Science Partner Journal)


A Multi-attention and Depthwise Separable Convolution Network for Medical Image Segmentation


Dual-TBNet: Improving the Robustness of Speech Features via Dual-Transformer-BiLSTM for Speech Emotion Recognition

IEEE/ACM Transactions on Audio, Speech, and Language Processing


Neuro or Symbolic? Fine-tuned Transformer with Unsupervised LDA Topic Clustering for Text Sentiment Analysis

IEEE Transactions on Affective Computing


Prompt Consistency for Multi-label Textual Emotion Detection


Active learning with complementary sampling for instructing class-biased multi-label text emotion classification


An efficient framework for constructing speech emotion corpus based on integrated active learning strategies


An Effective Surface Defect Classification Method Based on RepVGG with CBAM Attention Mechanism (RepVGG-CBAM) for Aluminum Profiles


Emotion Analysis and Dialogue Breakdown Detection in Dialogue of Chat Systems Based on Deep Neural Networks

In dialogues between robots or computers and humans, dialogue breakdown analysis is an important tool for achieving better chat dialogues. Conventional dialogue breakdown detection methods focus on semantic variance. Although these methods can detect dialogue breakdowns based on semantic gaps, they cannot always detect emotional breakdowns in dialogues. In chat dialogue systems, emotions are sometimes included in the utterances of the system when responding to the speaker. In this study, we detect emotions from utterances, analyze emotional changes, and use them as the dialogue breakdown feature. The proposed method estimates emotions by utterance unit and generates features by calculating the similarity of the emotions of the utterance and the emotions that have appeared in prior utterances. We employ deep neural networks using sentence distributed representation vectors as the feature. In an evaluation of experimental results, the proposed method achieved a higher dialogue breakdown detection rate when compared to the method using a sentence distributed representation vectors.


Double bayesian pairwise learning for one-class collaborative filtering

Recommender systems have become an indispensable tool for real-world applications. Only one-class feedback can be obtained in many applications. Therefore, the one-class recommendation problem has attracted much attention. Pairwise ranking methods are popular for dealing with the one-class problem. Bayesian Personalized Ranking (BPR) is one of the most popular pairwise methods, assuming users prefer the observed item to the unobserved item. The parameters in BPR are learned based on stochastic gradient descent (SGD). However, the previous work has shown that existing the vanishing gradient problem in the learning process when the preference difference between the observed item and the unobserved item is very large. In this paper, we propose a novel algorithm called Double Bayesian Pairwise Learning (DBPL). In the learning process of DBPL, the preference difference between the observed item and the unobserved item can be reduced by fusing a relatively smaller preference difference between another pair of items. Moreover, we calculate potential preference scores between users and items based on user-item interactions to measure preference differences between unobserved items of each user. Experimental results on three real-world datasets show the effectiveness of the DBPL algorithm.


Intent-Enhanced Attentive Bert Capsule Network for Zero-shot Intention Detection

Spoken language understanding (SLU) plays an indispensable role in the dialogue system. The traditional intention detection task is regarded as a classification problem where utterances are associated with pre-defined intents. However, the various expressions of user's intents and constantly emerging novel intents make the annotating time-consuming and labor-intensive, building massive obstacles for extending the model to new tasks. Identifying unexpected user intention and achieving the user's desire goal is a challenging task. Therefore, we conduct zero-shot intention detection based on a transformation-based learning manner. In this paper, we propose an intent-enhanced attentive capsule network (IE-BertCapsNet) further guides the aggregation process of the capsule network and generalizable useful features that can be adapted to emerging intentions. Coupling with the large margin cosine loss function, the proposed model can identify discriminative features by forcing the whole network to minimize inter-class distance and minimize intra-class distance. Finally, we leverage the IE-BertCapsNet's feature extraction ability and knowledge transferring capability to conduct zero-shot intent detection and generalized zero-shot intent detection. Extensive experiments on five benchmark task-oriented datasets in four languages demonstrate that the proposed model can achieve competitive performance that can better discriminate known intents and detect unknown intents.


A Two-Stage Spatiotemporal Attention Convolution Network for Continuous Dimensional Emotion Recognition from Facial Video

Continuous dimensional emotion recognition for facial video sequence is a crucial and challenging task in Affective Computing and Human-Computer Intelligent Interaction. The key of this task is to effectively extract and discriminate spatialtemporal features in a more fine-grained way. In this paper, a Two-Stage Spatiotemporal Attention Temporal Convolution Network (TS-SATCN) is designed for continuous dimensional emotion recognition of facial videos. The first stage generates an initial recognition result that is later fed into the second for correction. In each stage, the introduced spatiotemporal attention branch helps the network learn different attention levels and focuses on the informative spatial-temporal features adaptively. The network is trained by a proposed smooth loss function which can further improve the predictions quality. Extensive experiments are performed on two datasets, RECOLA and AFEW-VA, which shows that the proposed method achieves significant improvement over state-of-the-art methods.


WiONE: One-Shot Learning for Environment-Robust Device-Free User Authentication via Commodity Wi-Fi in Man-Machine System

User authentication is the first and most critical step in protecting a man-machine system from a malicious spoofer. However, security and privacy are just like the two sides of one coin, hard to see both at the same time, especially by the current mainstream credential- and biometric-based approaches. To this end, we propose WiONE, a safe and privacy-preserving user authentication system leveraging the ubiquitous Wi-Fi infrastructure by exploring ``how you behave'' rather than ``who you are''. The key idea is to apply deep learning to user physical behavior captured by Wi-Fi channel state information (CSI) to identify legitimate users while rejecting spoofers. The design of WiONE faces two challenges, namely, how to capture the subtle behavior, such as a keystroke on CSI, and how to mitigate the heavy environment-specific training required by deep learning. For the former, we design a behavior enhancement model based on the Rician fading to highlight the behavior-induced information by suppressing the behavior-unrelated information on channel response. For the latter, we develop a behavior characterization method tailored for the prototypical networks to facilitate the extraction of the domain-independent behavioral features and enable one-shot recognition of a new user in a new environment. Numerous experiments are conducted in several real-world environments, and the results show that WiONE outperforms its state-of-the-art rivals in authentication performance with much less training effort.

IEEE Transactions on Computational Social Systems


Robust Heart Rate Estimation with Spatial-Temporal Attention Network from Facial Videos

In order to solve the problems of highly redundant spatial information and motion noise in the heart rate (HR) estimation from facial videos based on remote Photoplethysmography(rPPG), this paper proposes a novel HR estimation method based on spatial-temporal attention model. Firstly, to reduce the redundant information and strengthen the association relationships of long-range videos, the spatial-temporal facial features are extracted by the 2D convolutional neural network (2DCNN) and 3D convolutional neural network (3DCNN), respectively. The aggregation function is adopted to incorporate feature maps into short segment spatial-temporal feature maps. Secondly, the spatial-temporal strip pooling is designed in the spatial-temporal attention module to reduce head movement noises. Then, via the two-part loss function, the model can focus more on the rPPG signal rather than the interference. We conduct extensive experiments on two public datasets to verify the effectiveness of our model. The experimental results show that the proposed method achieves significantly better performances than the state-of-the-art baselines: The mean absolute error could be reduced by 11% on the PURE dataset, and by 25% on the COHFACE dataset.

IEEE Transactions on Cognitive and Developmental Systems


Prior-based bayesian pairwise ranking for one-class collaborative filtering

In many real-world applications, only user-item interactions (one-class feedback) can be observed. The recommendation methods have been studied for personalized ranking with one-class feedback in recent years. Pairwise ranking methods have been widely used for dealing with the one-class problem with the assumption that users prefer their observed items over unobserved items. However, existing some items that users have not seen yet. It is unsuitable for treating all unobserved items of the user as negative feedback. In this paper, we propose a Prior-based Bayesian Pairwise Ranking (PBPR) model, which relaxes the simple pairwise preference assumption in previous works by further considering the pairwise preference between two unobserved items. Moreover, we calculate users' potential preference scores on unobserved items, i.e., prior information, based on historical interactions. The prior information can be used to measure the fine-grained preference difference between any two unobserved items of each user. Through extensive experiments on real-world datasets, we demonstrate the effectiveness of our proposed recommendation method.


A Survey of Textual Emotion Recognition and Its Challenges

Textual language is the most natural carrier of human emotion. In natural language processing, textual emotion recognition (TER) has become an important topic due to its significant academic and commercial potential. With the advanced development of deep learning technologies, TER has attracted growing attention and has significantly been promoted in recent years. This paper provides a systematic survey of latest TER advances, focusing on approaches using deep neural networks. According to how deep learning works at each stage, TER approaches are reviewed on word embedding, architecture, and training levels, respectively. We discussed the remaining challenges and opportunities from four aspects: the shortage of large-scale and high-quality dataset, fuzzy emotional boundaries, incomplete extractable emotional information in texts, and TER in dialogue. This paper creates a systematic and in-depth overview of deep TER technologies. It provides the necessary knowledge and new insights for relevant researchers to understand better the research state, remaining challenges, and future directions in this field.

IEEE Transactions on Affective Computing


Diagnostic Evaluation of Policy-Gradient-Based Ranking

Learning-to-rank has been intensively studied and has shown significantly increasing values in a wide range of domains, such as web search, recommender systems, dialogue systems, machine translation, and even computational biology, to name a few. In light of recent advances in neural networks, there has been a strong and continuing interest in exploring how to deploy popular techniques, such as reinforcement learning and adversarial learning, to solve ranking problems. However, armed with the aforesaid popular techniques, most studies tend to show how effective a new method is. A comprehensive comparison between techniques and an in-depth analysis of their deficiencies are somehow overlooked. This paper is motivated by the observation that recent ranking methods based on either reinforcement learning or adversarial learning boil down to policy-gradient-based optimization. Based on the widely used benchmark collections with complete information (where relevance labels are known for all items), such as MSLRWEB30K and Yahoo-Set1, we thoroughly investigate the extent to which policy-gradient-based ranking methods are effective. On one hand, we analytically identify the pitfalls of policy-gradient-based ranking. On the other hand, we experimentally compare a wide range of representative methods. The experimental results echo our analysis and show that policy-gradient-based ranking methods are, by a large margin, inferior to many conventional ranking methods. Regardless of whether we use reinforcement learning or adversarial learning, the failures are largely attributable to the gradient estimation based on sampled rankings, which significantly diverge from ideal rankings. In particular, the larger the number of documents per query and the more fine-grained the ground-truth labels, the greater the impact policy-gradient-based ranking suffers. Careful examination of this weakness is highly recommended for developing enhanced methods based on policy gradient.


rPPG-Based Heart Rate Estimation using Spatial-Temporal Attention Network

Remote photoplethysmography (rPPG) based on computer vision technology is widely used to calculate the heart rate (HR) from facial videos. Existing rPPG techniques have been subject to some limitations (e.g., highly redundant spatial information, head movement noise and region of interest (ROI) selection). To address these limitations, this paper introduces an effective spatial-temporal attention network. A temporal fusion module is firstly proposed to fully exploit the time-domain information, aiming to reduce the redundant video information and strengthen the association relationships of long-range videos. Furthermore, a spatial attention mechanism is designed in the backbone net to precisely target the skin ROIs. Finally, to assist the network in learning the weights between channels, we project the RGB images using plane orthogonal to skin (POS) algorithm and add motion representation to complement physiological signals' extraction. Extensive experiments on the public PURE, MMSE-HR, and UBFC-rPPG datasets demonstrate that our model achieve competitive results compared with other methods.


Blind Quality Evaluator of Tone-Mapped HDR and Multi-Exposure Fused Images for Electronic Display

The problem of reproducing high dynamic range (HDR) images on electronic display and photography with restricted dynamic range has gained a lot of interest in the consumer electronics community. There exist various approaches to this issue, e.g., tone mapping operators (TMOs) and multi-exposure fusion algorithms (MEFs). Many existing image quality assessment (IQA) methods have been proposed to compare images of quality degradation generated by TMOs/MEFs. Although promising performances have been achieved, they seldom consider local specific artifacts difference (i.e., abnormal exposure and color cast) related with the TMOs/MEFs. To address this limitation, this paper proposes a Blind Quality Evaluator of Tone-Mapped HDR and Multi-Exposure Fused Images (BQE-TM/MEFI). First, two purpose-designed segment models are utilized to distinguish well-exposedness dense patches (WEDPes) and non-WEDPes, color cast patches (CCPes) and non-CCPes respectively. Second, multiple quality-perception features are extracted to measure local artifacts: 1) structure and sharpness features from WEDPes, 2) saturation features from non-CCPes, and 3) edge structure features. Then, three new low-complexity regional features (over-exposure ratio, entropy and color confidence index) are calculated based on over-exposure segmentation model. Finally, all extracted features are aggregated into a machine-learning regression model to pool a quality score. The simplicity and good performance of the proposed method makes it suitable for electronic displays and other consumer electronics.

IEEE Transactions on Consumer Electronics


3D Facial Expression Recognition via Attention-based Multi-channel Data Fusion Network

Facial expression has long been recognized as containing meaningful non-verbal affective cues for decoding human emotions. Recently, multi-modal 2D+3D fusion method has shown significant potential in facial expression recognition (FER) due to its fine-grained face descriptions in various spatial channels. However, current work mainly relies on feature-level or even score-level fusion to find emotion cues spread in different channels, and may miss key information due to lack of focus. To this end, we propose an attention-based multi-channel data fusion network (AMDFN) to better preserve and find such key facial cues. More specifically, we first map a 3D face scan into multi-channel images, and then fuse them in a ResNet18 backbone to get layered emotion features. Secondly, we leverage a layer attention model to explore the dependencies between features of different layers to learn discriminative affective cues for effective emotion recognition. Our comprehensive experiments on two widely-used datasets (i.e., Facescape and Bosphorus) have verified the performance of our approach compared to several state-of-the-art rivals.

IEEE Transactions on Instrumentation and Measurement


Utilizing External Knowledge to Enhance Semantics in Emotion Detection in Conversation

Enabling machines to emotion recognition in conversation is challenging, mainly because information in human dialogue innately conveys emotions by long-term experience, abundant knowledge, context, and the intricate patterns between the affective states. We address the task of emotion recognition in conversations using external knowledge to enhance semantics. We propose KES model, a new framework that incorporates different elements of external knowledge and conversational semantic role labeling, where build upon them to learn interactions between interlocutors participating in a conversation. We design a self-attention layer specialized for enhanced semantic text features with external commonsense knowledge. Then, two different networks composed of LSTM are responsible for tracking individual internal state and context external state. In addition, the proposed model has experimented on three datasets in emotion detection in conversation. The experimental results show that our model outperforms the state-of-the-art approaches on most of the tested datasets.


An Effective Clustering Optimization Method for Unsupervised Linear Discriminant Analysis

The recent work Unsupervised Linear Discriminant Analysis (Un-LDA) completes its clustering process during the alternating optimization by converting equivalently the objective and finally using the K-means algorithm. However, the K-means algorithm has its inherent drawbacks. It is hard for the K-means algorithm to deal well with some complex clustering cases where there are too many real clusters or non-convex clusters. In this paper, a novel clustering optimization method is presented to accomplish the clustering process in Un-LDA and the resulting method can be named Un-LDA(CD). Specifically, instead of the K-means algorithm, an elaborately designed coordinate descent algorithm is adopted to obtain the clusters after the objective function goes through a series of simple but deft equivalent conversions. Extensive experiments have demonstrated that the coordinate descent clustering solution for Un-LDA can outperform the original K-means based solution on the tested data sets especially those complex data sets with a pretty large number of real clusters.

IEEE Transactions on Knowledge and Data Engineering


Enhance the Language Ability of Humanoid Robot NAO through Deep Learning to Interact with Autistic Children

Autism spectrum disorder (ASD) is a life-long neurological disability, and a cure has not yet been found. ASD begins early in childhood and lasts throughout a person's life. Through early intervention, many actions can be taken to improve the quality of life of children. Robots are one of the best choices for accompanying children with autism. However, for most robots, the dialogue system uses traditional techniques to produce responses. Robots cannot produce meaningful answers when the conversations have not been recorded in a database. The main contribution of our work is the incorporation of a conversation model into an actual robot system for supporting children with autism. We present the use a neural network model as the generative conversational agent, which aimed at generating meaningful and coherent dialogue responses given the dialogue history. The proposed model shares an embedding layer between the encoding and decoding processes through adoption. The model is different from the canonical Seq2Seq model in which the encoder output is used only to set-up the initial state of the decoder to avoid favoring short and unconditional responses with high prior probability. In order to improve the sensitivity to context, we changed the input method of the model to better adapt to the utterances of children with autism. We adopted transfer learning to make the proposed model learn the characteristics of dialogue with autistic children and to solve the problem of the insufficient corpus of dialogue. Experiments showed that the proposed method was superior to the canonical Seq2sSeq model and the GAN-based dialogue model in both automatic evaluation indicators and human evaluation, including pushing the BLEU precision to 0.23, the greedy matching score to 0.69, the embedding average score to 0.82, the vector extrema score to 0.55, the skip-thought score to 0.65, the KL divergence score to 5.73, and the EMD score to 12.21.


Deep Multibranch Fusion Residual Network for Insect Pest Recognition

Earlier insect pest recognition is one of the critical factors for agricultural yield. Thus, an effective method to recognize the category of insect pests has become significant issues in the agricultural field. In this article, we proposed a new residual block to learn multiscale representation. In each block, it contains three branches: one is parameter-free, and the others contain several successive convolution layers. Moreover, we proposed a module and embedded it into the new residual block to recalibrate the channelwise feature response and to model the relationship of the three branches. By stacking this kind of block, we constructed the deep multibranch fusion residual network (DMF-ResNet). For evaluating the model performance, we first test our model on CIFAR-10 and CIFAR-100 benchmark data sets. The experimental results show that DMF-ResNet outperforms the baseline models significantly. Then, we construct DMF-ResNet with different depths for high-resolution image classification tasks and apply it to recognize insect pests. We evaluate the model performance on the IP102 data set, and the experimental results show that DMF-ResNet could achieve the best accuracy performance than the baseline models and other stateof- the-art methods. Based on these empirical experiments, we demonstrate the effectiveness of our approach.

IEEE Transactions on Cognitive and Developmental Systems


IEEE Transactions on Emerging Topics in Computational Intelligence


Emotion recognition based on fusion of long short-term memory networks and SVMs

This paper proposes a multimodal fusion emotion recognition method based on Dempster-Shafer evidence theory, which includes electroencephalogram (EEG) and electrocardiogram (ECG). For EEG, we use the SVM classifier to classify features, and for ECG, we establish the corresponding Bi-directional Long Short-Term Memory network emotion recognition structure, which is fused with EEG classification results through the evidence theory. We selected 25 video clips with five emotions (happy, relaxed, angry, sad, and disgusted), and a total of 20 subjects participated in our emotional experiment. The experimental results prove that the performance of the multi-modal fusion model proposed in this paper is superior to the single-modal emotion recognition model. In the Arousal and Valance dimensions, the average accuracy is improved by 2.64% and 2.75% compared with the EEG signal-based emotion recognition model. Compared with the emotion recognition model based on the ECG signal, the accuracy is improved by 7.37% and 8.73%.


Data-Driven Channel Pruning towards Local Binary Convolution Inverse Bottleneck Network Based on Squeeze-and-Excitation Optimization Weights

This paper proposed a model pruning method based on local binary convolution (LBC) and squeeze-and-excitation (SE) optimization weights. We first proposed an efficient deep separation convolution model based on the LBC kernel. By expanding the number of LBC kernels in the model, we have trained a larger model with better results, but more parameters and slower calculation speed. Then, we extract the SE optimization weight value of each SE module according to the data samples and score the LBC kernel accordingly. Based on the score of each LBC kernel corresponding to the convolution channel, we performed channel-based model pruning, which greatly reduced the number of model parameters and accelerated the calculation speed. The model pruning method proposed in this paper is verified in the image classification database. Experiments show that, in the model using the LBC kernel, as the number of LBC kernels increases, the recognition accuracy will increase. At the same time, the experiment also proved that the recognition accuracy is maintained at a similar level in the small parameter model after channel-based model pruning by the SE optimization weight value.


A spatio-temporal integrated model based on local and global features for video expression recognition


Multi-Task and Attention Collaborative Network for Facial Emotion Recognition

Facial expression is one of the most direct and effective ways to recognize emotions, widely used in human-computer interaction, affective computing, and other research fields. Expression recognition can be divided into discrete expression classification and continuous dimensional emotion recognition. Most of the existing multi-dimensional emotional estimation only considers the data under laboratory conditions. In this paper, facial emotion estimation is performed based on real-world images and combined with the advantages of multi-task learning and attention mechanism. We improve the multi-task attention network (MTAN) from two aspects: task and feature. At the aspect of the task, the multi-task collaborative attention network (MTCAN), which is based on task correlation, is proposed to solve task deviation in multi-task learning. At the aspect of the feature, based on MTCAN, we came up with MTACN, which used the self-attention mechanism to measure the importance of each attention module for each specific task. Then, we can capture the local-to-global connection in one step and fully exploit the feature within different levels of each task. Experimental results on the AffectNet dataset show that the performance of the model is significantly better than the original network, and the Root-mean-square error and consistency correlation coefficient results are superior to other existing models.

IEEJ Transactions on Electrical and Electronic Engineering (TEEE)


Human Psychological Burden and Thinking Process while Operating a Humanoid Robot Conversation App Program

International Journal of Advanced Intelligence (IJAI)


The AI Development Through Transforming Tacit Knowledge to Explicit Knowledge of Nurses' Dialogue for Patients with Dementia


Developmental Issues of Communication for Robot Applications in Older People Care: An Integrative Review


Lock Isolation Security Architecture for Reconfigurable Scanning Networks

To protect the reconfigurable scanning network from the influence of unauthorized access, tampering of transmitted data by malicious instruments and sniffing, a lock isolation security structure is proposed. Firstly, the presented structure divides the instruments without security threat to each other into one group, and the groups can be accessed one by one via the isolated signals, which can prevent malicious instruments tampering and sniffing transmitted data. Secondly, the key instruments are protected by the lock segment insertion bit. The lock segment insertion bit can only be opened when multiple key values at specific positions are set to specific values (0, 1 sequence), which can increase the difficulty of unauthorized access. Besides, to reduce the hardware overhead, and the difficulty of place and route caused by excessive instrument groups, instrument grouping algorithm is proposed. The proposed algorithm firstly constructs an undirected graph based on the security relationship among instruments. Then the maximal independent sets of the undirected are obtained. In this way, the number of instrument groups is reduced greatly. Compared to other similar methodologies, the experimental results conducted on ITC 02 benchmark circuits show that the time consumption for opening the locking segment insertion bit of the proposed security structure increased by 7 times, and the average percentage reduction of area, power consumption and wire length is 3.81%,

Journal of Computer-Aided Design & Computer Graphics


ETA-rPPGNet: Effective Time-Domain Attention Network for Remote Heart Rate Measurement

IEEE Transactions on Instrumentation and Measurement


Hierarchical Network with Label Embedding for Contextual Emotion Recognition

Emotion recognition has been used widely in various applications such as mental health monitoring and emotional management. Usually, emotion recognition is regarded as a text classification task. Emotion recognition is a more complex problem, and the relations of emotions expressed in a text are nonnegligible. In this paper, a hierarchical model with label embedding is proposed for contextual emotion recognition. Especially, a hierarchical model is utilized to learn the emotional representation of a given sentence based on its contextual information. To give emotion correlation-based recognition, a label embedding matrix is trained by joint learning, which contributes to the final prediction. Comparison experiments are conducted on Chinese emotional corpus RenCECps, and the experimental results indicate that our approach has a satisfying performance in textual emotion recognition task.


WRGAN: Improvement of RelGAN with Wasserstein Loss for Text Generation

Generative adversarial networks (GANs) were rst proposed in 2014, and have been widely used in computer vision, such as for image generation and other tasks. However, the GANs used for text generation have made slow progress. One of the reasons is that the discriminator's guidance for the generator is too weak, which means that the generator can only get a ``true or false'' probability in return. Compared with the current loss function, the Wasserstein distance can provide more information to the generator, but RelGAN does not work well with Wasserstein distance in experiments. In this paper, we propose an improved neural network based on RelGAN and Wasserstein loss named WRGAN. Differently from RelGAN, we modi ed the discriminator network structure with 1D convolution of multiple different kernel sizes. Correspondingly, we also changed the loss function of the network with a gradient penalty Wasserstein loss. Our experiments on multiplepublicdatasetsshowthatWRGANoutperformsmostoftheexistingstate-of-the-artmethods, and the Bilingual Evaluation Understudy(BLEU) scores are improved with our novel method.


Coarse-to-Fine 3D Road Model Registration for Traffic Video Augmentation


A Review on Human-Computer Interaction and Intelligent Robots

In the field of artificial intelligence, human computer interaction (HCI) technology and its related intelligent robot technologies are essential and interesting contents of research. From the perspective of software algorithm and hardware system, these abovementioned technologies study and try to build a natural HCI environment. The purpose of this research is to provide an overview of HCI and intelligent robots. This research highlights the existing technologies of listening, speaking, reading, writing, and other senses, which are widely used in human interaction. Based on these same technologies, this research introduces some intelligent robot systems and platforms. This paper also forecasts some vital challenges of researching HCI and intelligent robots. The authors hope that this work will help researchers in the field to acquire the necessary information and technologies to further conduct more advanced research.

International Journal of Information Technology & Decision Making


A linear multivariate binary decision tree classifier based on K-means splitting

A novel linear multivariate decision tree classifier, Binary Decision Tree based on K-means Splitting (BDTKS), is presented in this paper. The unsupervised K-means clustering is recursively integrated into the binary tree, building a hierarchical classifier. The introduction of the unsupervised K-means clustering provides the powerful generalization ability for the resulting BDTKS model. Then, the good generalization ability of BDTKS ensures the classification performance. A novel non-split condition with an easy-setting hyperparameter which focuses more on minority classes of the current node is proposed and applied in the BDTKS model, avoiding ignoring the minority classes in the class imbalance cases. Furthermore, the K-means centroid based BDTKS model is converted into the hyperplane based decision tree, speeding up the process of classification. Extensive experiments on the publicly available data sets have demonstrated that the proposed BDTKS matches or outperforms the previous decision trees.


DFF-ResNet: An Insect Pest Recognition Model based on Residual Networks


Multi-label Emotion Detection via Emotion-Specified Feature Extraction and Emotion Correlation Learning

Textual emotion detection is an attractive task while previous studies mainly focused on polarity or single-emotion classification. However, human expressions are complex, and multiple emotions often co-occur with non-negligible emotion correlations. In this paper, a Multi-label Emotion Detection Architecture (MEDA) is proposed to detect all associated emotions expressed in a given piece of text. MEDA is mainly composed of two modules: Multi-Channel Emotion-Specified Feature Extractor (MC-ESFE) and Emotion Correlation Learner (ECorL). MEDA captures underlying emotion-specified features through MC-ESFE module, which is composed of multiple channel-wise ESFE networks. Each channel in MC-ESFE is devoted to the feature extraction of a specified emotion from sentence-level to context-level through a hierarchical structure. With underlying features, emotion correlation learning is implemented through an emotion sequence predictor in ECorL. Furthermore, we define a new loss function: multi-label focal loss. With this loss function, the model can focus more on misclassified positive-negative emotion pairs and improve the overall performance by balancing the prediction of positive and negative emotions. The evaluation of proposed MEDA architecture is carried out on emotional corpus: RenCECps and NLPCC2018 datasets. The experimental results indicate that the proposed method can achieve better performance than state-of-the-art methods in this task.

IEEE Transactions on Affective Computing


Multi-reservoirs EEG signal feature sensing and recognition method based on generative adversarial networks

EEG emotion recognition is one of the interesting and challenging tasks in the research based emotion human computer interface system. In this paper, a multi-reservoirs feature coding continuous label fusion semi-supervised Generative Adversarial Networks (MCLFS-GAN) is proposed by using permutation phase transfer entropy as the EEG signal feature. Firstly, the obtained features are encapsulated in time series, and then the features are sent into multi-reservoirs according to the division of brain intra, brain interval or frequency band. After convolution optimization, the feature expression with time sequence relationship is obtained. The generic representation between the features and pseudo effective feature expression are iteratively learned in encoder E and generator G in the generative adversarial way. In addition, the continuous fusion for class intra tags can help to form continuous differences between classes. The experimental results show that the accuracy for the four classification is 81.32% and 54.87% respectively by using SAP and LOSO in DEAP database. Compared with other models, this algorithm can effectively improve the recognition performance.


Facial Expression Imitation Method for Humanoid Robot Based on Smooth-Constraint Reversed Mechanical Model (SRMM)

To improve the space time similarity and motion smoothness of facial expression imitation (FEI), a real-time FEI method for a humanoid robot is proposed based on smooth-constraint reversed mechanical model (SRMM) by combining a sequence-to-sequence deep learning model and a motion-smoothing constraint. First, on the basis of facial data from a Kinect capture device, a facial feature vector is characterized based on 3 head postures, 17 facial animation units, and facial geometric deformation cascaded by Laplace coordinates. Second, a reversed mechanical model is constructed via a multilayer long short-term memory neural network to accomplish direct mapping from facial feature sequences to motor position sequences. Additionally, to overcome the motor chattering phenomenon during real-time FEI, a high-order polynomial is constructed to fit the position sequence of motors, and an SRMM is proposed and designed based on the deviation of position, velocity, and acceleration. Finally, aiming to imitate the real-time facial feature sequences of a performer captured from Kinect, the optimal position sequences generated based on the SRMM is sent to the hardware system to keep the space time characteristics consistent with those of the performer. The experimental results demonstrate that the motor position deviation of the SRMM is less than 8%. The space time similarity between the robot and the performer is greater than 85%, and the motion smoothness of the online FEI exceeded 90%. Compared with other related methods, the proposed method achieves a remarkable improvement in motor position deviation, space time similarity, and motion smoothness.

IEEE Transactions on Human-Machine Systems


Emotion recognition using pupil position


EEG emotion recognition model based on the LIBSVM classifier

This paper proposes an electroencephalogram(EEG) emotion recognition method based on the LIBSVM classifier. EEG features are calculated to represent the characterisitics associated with emotion states. First calculating the Lempel Ziv complexity and wavelet detail coefficients for the pre-processed EEG signals; then obtaining the co-integration relationship that reflecting the relationship between channels according to the cointegration test; next performing Empirical Mode Decomposition(EMD) on the pre-processed EEG signals; finally calculating the average approximate entropy of the first four Intrinsic Mode Functions(IMFs). The calculated four features are input into the LIBSVM classifier to realize the sentiment classification of each channel data, and then the classification results of each channel are fused by the Takagi Sugeno fuzzy model to achieve the final emotion classification. The experimental results show that when the two-category classifications are performed on Arousal and Valance, the average sentiment recognition rates are 74.88% and 82.63%, respectively.


Sentence-Embedding and Similarity via Hybrid Bidirectional-LSTM and CNN Utilizing Weighted-Pooling Attention

Neural networks have received considerable attention in sentence similarity measuring systems due to their efficiency in dealing with semantic composition. However, existing neural network methods are not sufficiently effective in capturing the most significant semantic information buried in an input. To address this problem, a novel weighted-pooling attention layer is proposed to retain the most remarkable attention vector. It has already been established that long short-term memory and a convolution neural network have a strong ability to accumulate enriched patterns of whole sentence semantic representation. First, a sentence representation is generated by employing a siamese structure based on bidirectional long short-term memory and a convolutional neural network. Subsequently, a weighted-pooling attention layer is applied to obtain an attention vector. Finally, the attention vector pair information is leveraged to calculate the score of sentence similarity. An amalgamation of both, bidirectional long short-term memory and a convolutional neural network has resulted in a model that enhances information extracting and learning capacity. Investigations show that the proposed method outperforms the state-of-the-art approaches to datasets for two tasks, namely semantic relatedness and Microsoft research paraphrase identification. The new model improves the learning capability and also boosts the similarity accuracy as well.

IEICE Transactions on Information and Systems


Surface defect detection of steel strips based on classification priority YOLOv3-dense network

The steel strip is an essential raw material in the machinery industry. Besides, the surface defects of the steel strip directly determine its performance. To achieve rapid and effective detection of the defects, a CP-YOLOv3-dense (classification priority YOLOv3 DenseNet) neural network was proposed in the present study. The model used YOLOv3 as basic network, implemented priority classification on the images, and then replaced the two residual network modules with two dense network modules. Therefore, the network can receive the multi-layer convolution features output by the dense connection block before making predictions, consequently enhancing the reuse and fusion of features. Finally, the six kinds of surface defects were detected by the improved network. According to the results, the detection precision of the CP-YOLOv3-dense network is 85.7%, the recall rate is 82.3%, the mean average precision is 82.73%, and the detection time of each image is 9.68 ms.


LAUN Improved StarGAN for Facial Emotion Recognition

In the eld of facial expression recognition, deep learning is extensively used. However, insuf cient and unbalanced facial training data in available public databases is a major challenge for improving the expression recognition rate. Generative Adversarial Networks (GANs) can produce more one-to-onefaceswithdifferentexpressions,whichcanbeusedtoenhancedatabases.StarGANcanperform one-to-many translations for multiple expressions. Compared with original GANs, StarGAN can increase the ef ciency of sample generation. Nevertheless, there are some defects in essential areas of the generated face,suchasthemouthandthefuzzysidefaceimagegeneration.Toaddresstheselimitations,weimproved StarGAN to alleviate the defects of images generation by modifying the reconstruction loss and adding the Contextual loss. Meanwhile, we added the Attention U-Net to StarGAN's generator, replacing StarGAN's original generator. Therefore, we proposed the Contextual loss and Attention U-Net (LAUN) improved StarGAN.TheU-shapestructureandskipconnectioninAttentionU-Netcaneffectivelyintegratethedetails and semantic features of images. The network's attention structure can pay attention to the essential areas of the human face. The experimental results demonstrate that the improved model can alleviate some aws in the face generated by the original StarGAN. Therefore, it can generate person images with better quality withdifferentposesandexpressions.TheexperimentswereconductedontheKarolinskaDirectedEmotional Facesdatabase,andtheaccuracyoffacialexpressionrecognitionis95.97%,2.19%higherthanthatbyusing StarGAN. Meanwhile, the experiments were carried out on the MMI Facial Expression Database, and the accuracy of expression is 98.30%, 1.21% higher than that by using StarGAN. Moreover, experiment results have better performance based on the LAUN improved StarGAN enhanced databases than those without enhancement.

表情認識の分野では，深層学習が広く使用されています．ただし，利用可能な公開データベースの不十分で不均衡な顔面トレーニングデータは，表現認識率を向上させるための主要な課題です． Generative Adversarial Networks(GAN)は，データベースを拡張するために使用できる，異なる表現を持つ1対1の面をさらに生成できます．StarGANは，複数の表現に対して1対多の変換を実行できます．オリジナルのGANと比較して，StarGANはサンプル生成の効率を高めることができます．それにもかかわらず，生成された顔の本質的な領域には，口やファジーサイド画像の生成など，いくつかの欠陥があります．これらの制限に対処するために，StarGANを改良して，再構成損失を変更し，コンテキスト損失を追加することにより，画像生成の欠陥を軽減しました．一方，StarGANの元のジェネレーターを置き換えて，注意U-NetをStarGANのジェネレーターに追加しました．したがって，我々はコンテキスト損失と注意U-Net(LAUN)が改善されたStarGANを提案しました．


CERG: Chinese Emotional Response Generator with Retrieval Method

The dialogue system has always been one of the important topics in the domain of artificial intelligence. So far, most of the mature dialogue systems are task-oriented based, while non-task-oriented dialogue systems still have a lot of room for improvement. We propose a data-driven non-task-oriented dialogue generator ``CERG'' based on neural networks. This model has the emotion recognition capability and can generate corresponding responses. The data set we adopt comes from the NTCIR-14 STC-3 CECG subtask, which contains more than 1.7 million Chinese Weibo post-response pairs and 6 emotion categories. We try to concatenate the post and the response with the emotion, then mask the response part of the input text character by character to emulate the encoder-decoder framework. We use the improved transformer blocks as the core to build the model and add regularization methods to alleviate the problems of overcorrection and exposure bias. We introduce the retrieval method to the inference process to improve the semantic relevance of generated responses. The results of the manual evaluation show that our proposed model can make different responses to different emotions to improve the human-computer interaction experience. This model can be applied to lots of domains, such as automatic reply robots of social application.


Test Strategy for 2.5D ICs Using Auxiliary Interposer and E-fuses


Vowel priority lip matching scheme and similarity evaluation model based on humanoid robot Ren-Xin

Journal of Ambient Intelligence and Humanized Computing


BeAware: Convolutional neural network(CNN) based user behavior understanding through WiFi channel state information

In modern informatics society, human beings are becoming more and more attached to the computer. Therefore, understanding user behavior is critical to various application elds like sedentary analysis, human-computer interaction, and affective computing. Current sensor-based and vision-based user behavior understanding approaches are either contact or obtrusive to users, jeopardizing their availability and practicality. To this end, we present BeAware, a contactless Radio Frequency (RF) based user behavior understanding system leveraging the WiFi Channel State Information (CSI). The key idea is to visualize the channel data affected by human movements into time-series heat-map images, which are processed by a Convolutional Neural Network (CNN) to understand the corresponding user behaviors. We prototype BeAware on commodity low-cost WiFi devices and evaluate its performance in real-world environments. Experimental results have veri ed its effectiveness in recognizing user behaviors.

996/5000 現代の情報社会では，人間はますますコンピュータに愛着を帯びてきています． したがって，座りがちな分析，人間とコンピュータの相互作用，感情に訴えるコンピューティングなど，さまざまなアプリケーション分野でユーザーの行動を理解することが重要です． 現在のセンサーベースおよびビジョンベースのユーザーの行動を理解するアプローチは，ユーザーへの接触または邪魔になり，その可用性と実用性を危険にさらしています． この目的のために，WiFiチャネル状態情報(CSI)を利用した非接触無線周波数(RF)ベースのユーザー行動理解システムであるBeAwareを紹介します． 重要なアイデアは，人間の動きによって影響を受けるチャネルデータを時系列のヒートマップイメージに視覚化し，対応するユーザーの行動を理解するために畳み込みニューラルネットワーク(CNN)で処理することです． BeAwareを市販の低コストWiFiデバイスでプロトタイプ化し，実際の環境でのパフォーマンスを評価します． 実験結果により，ユーザーの行動の認識におけるその有効性が検証されました．


EmoSense: Computational Intelligence Driven Emotion Sensing via Wireless Channel Data

Emotion is well recognized as a distinguished symbol ofhumanbeings,anditplaysacrucialroleinourdailylives.Existingvision-basedorsensor-basedsolutionsareeitherobstructiveto use or rely on specialized hardware, hindering their applicability. This paper introduces EmoSense, a rst-of-its-kind wireless emotionsensingsystemdrivenbycomputationalintelligence.Thebasic methodologyistoexplorethephysicalexpressionofemotionsfrom wireless channel response via data mining. The design and implementation of EmoSense faces two major challenges extracting physical expression from wireless channel data and recovering emotion from the corresponding physical expression. For the former, we present a Fresnel zone-based theoretical model depicting the ngerprint of the physical expression on channel response. For the latter, we design an ef cient computational intelligence driven mechanism to recognize emotion from the corresponding ngerprints.WeprototypedEmoSenseonthecommodityWiFiinfrastructure and compared it with mainstream sensor-based and vision-basedapproachesinthereal-worldscenario.Thenumerical study over 3360 cases con rms that EmoSense achieves a comparable performance to the vision-based and sensor-based rivals under different scenarios. EmoSense only leverages the low-cost and prevalent WiFi infrastructures and thus, constitutes a tempting solution for emotion sensing.

IEEE Transactions on Emerging Topics in Computational Intelligence


Sleepy: Wireless Channel Data Driven Sleep Monitoring via Commodity WiFi Devices

Sleep is a major event of our daily lives. Its quality constitutes a critical indicator of people's health conditions, both mentally and physically. Existing sensor-based or vision-based sleep monitoring systems either are obstructive to use or fail to provide adequate coverage. With the fast expansion of wireless infrastructures nowadays, channel data, which is pervasive and transparent, emerges as another alternative. To this end, we propose Sleepy, a wireless channel data driven sleep monitoring system leveraging commercial WiFi devices. The key idea of Sleepy is that the energy feature of the wireless channel follows a Gaussian Mixture Model (GMM) derived from the accumulated channel data over a long period. Therefore, a GMM based foreground extraction method has been designed to adaptively distinguish motions like rollovers (foreground) from background (stationary postures), leading to certain major merits, e.g., no calibrations or target-dependent training needed. We prototype Sleepy and evaluate it in two real environments. In the short-term controlled experiments, Sleepy achieves 95.04% detection accuracy and 4.07% false negative rate. In the 60-minute real case studies, Sleepy demonstrates strong stability. Considering that Sleepy is compatible with existing WiFi infrastructure, it constitutes a low-cost yet promising solution for sleep monitoring.

睡眠は私たちの日常生活の主要な出来事である．その品質は，精神的にも肉体的にも，人々の健康状態の重要な指標となる．既存のセンサベースまたは視覚ベースの睡眠監視システムは，使用するのに邪魔になるか，または十分な範囲を提供することができない．最近のワイヤレスインフラストラクチャの急速な拡大に伴い，広く普及しているチャネルデータが別の選択肢として登場している．我々はSleepyを提案する．これは，市販のWiFi装置を利用する無線チャネルデータ駆動型睡眠監視システムである．スリーピーの重要な考え方は，無線チャネルのエネルギー特性が，長期間にわたって累積チャネルデータから導出されたガウス混合モデル(GMM)に従うということである．したがって，GMMベースの前景抽出方法は，ロールオーバー(前景)のような動きを背景(静止姿勢)から適応的に区別するように設計されており，較正やターゲット依存のトレーニングが不要などの大きな利点をもたらす．私たちはSleepyをプロトタイプ化し，それを2つの実環境で評価した．短期間の対照実験では，Sleepyは95.04%の検出精度を達成した． 60分の実際のケーススタディで，Sleepyは強い安定性を示している．


Intention Detection Based on Siamese Neural Network With Triplet Loss

Understanding the user's intention is an essential task for the spoken language understanding (SLU) module in the dialogue system, which further illustrates vital information for managing and generating future action and response. In this paper, we propose a triplet training framework based on the multiclassclassi cationapproachtoconductthetrainingfortheintentiondetectiontask.Precisely,weutilize aSiameseneuralnetworkarchitecturewithmetriclearningtoconstructarobustanddiscriminativeutterance feature embedding model. We modi ed the RMCNN model and ne-tuned BERT model as Siamese encoderstotrainutterancetripletsfromdifferentsemanticaspects.Thetripletlosscaneffectivelydistinguish the details of two input data by learning a mapping from sequence utterances to a compact Euclidean space. After generating the mapping, the intention detection task can be easily implemented using standard techniques with pre-trained embeddings as feature vectors. Besides, we use the fusion strategy to enhance utterance feature representation in the downstream of intention detection task. We conduct experiments on several benchmark datasets of intention detection task: Snips dataset, ATIS dataset, Facebook multilingual task-oriented datasets, Daily Dialogue dataset, and MRDA dataset. The results illustrate that the proposed methodcaneffectivelyimprovetherecognitionperformanceofthesedatasetsandachievesnewstate-of-theart results on single-turn task-oriented datasets (Snips dataset, Facebook dataset), and a multi-turn dataset (Daily Dialogue dataset).

ユーザーの意図を理解することは，対話システムの音声言語理解(SLU)モジュールの重要なタスクです．これは，将来のアクションと応答を管理および生成するための重要な情報をさらに示します．このホワイトペーパーでは，意図検出タスクのトレーニングを実施するためのマルチクラス分類アプローチに基づいたトリプレットトレーニングフレームワークを提案します．正確には，Siameseneuralnetworkアーキテクチャでメトリック学習を使用して，ロバストで弁別的発話の特徴埋め込みモデルを構築します．シーケンス発話からコンパクトなユークリッド空間へのマッピングを学習することで，2つの入力データの詳細を効果的に区別できます．マッピングの生成後，事前検出された埋め込みを特徴ベクトルとして使用する標準的な手法を使用して，意図検出タスクを簡単に実装できます．さらに，融合戦略を使用して，意図検出タスクの下流で発話特徴表現を強化します．意図検出タスクのいくつかのベンチマークデータセットで実験を行います:スニップデータセット，ATISデータセット，Facebook多言語タスク指向データセット，毎日の対話データセット，およびMRDAデータセット．結果は，提案された方法がこれらのデータセットの認識パフォーマンスを効果的に改善し，シングルターンのタスク指向データセット(Snipsデータセット，Facebookデータセット)，およびマルチターンデータセット(Daily Dialogueデータセット)で最新の結果を達成できることを示しています．


Knowledge Graph based Question and Answer System for Cosmetic Domain

International Journal of Advanced Intelligence (IJAI)


An Emotion Expression Extraction Method for Chinese Microblog Sentences

With the rapid spread of Chinese microblog, a large number of microblog topics are being generated in real-time. More and more users pay attention to emotion expressions of these opinionated sentences in different topics. It is challenging to label the emotion expressions of opinionated sentences manually. For this endeavor, an emotion expression extraction method is proposed to process millions of user-generated opinionated sentences automatically in this paper. Specifically, the proposed method mainly contains two tasks: emotion classification and opinion target extraction. We first use a lexicon-based emotion classification method to compute different emotion values in emotion label vectors of opinionated sentences. Then emotion label vectors of opinionated sentences are revised by an unsupervised emotion label propagation algorithm. After extracting candidate opinion targets of opinionated sentences, the opinion target extraction task is performed on a random walk-based ranking algorithm, which considers the connection between candidate opinion targets and the textual similarity between opinionated sentences, ranks candidate opinion targets of opinionated sentences. Experimental results demonstrate the effectiveness of algorithms in the proposed method.

中国語マイクロブログの急速な普及に伴い，多数のマイクロブログトピックがリアルタイムで生成されています．ますます多くのユーザーが，さまざまなトピックでこれらの独断的な文章の感情表現に注意を払っています．有罪判決を受けた文章の感情表現に手動でラベルを付けることは困難です．この取り組みのために，本論文では，ユーザーが生成した何百万もの意見のある文章を自動的に処理する感情表現抽出方法を提案します．具体的には，提案された方法は主に2つのタスクを含みます:感情分類とオピニオンターゲット抽出．まず，レキシコンベースの感情分類方法を使用して，意欲的な文の感情ラベルベクトルのさまざまな感情値を計算します．次に，意見のある文の感情ラベルベクトルが，教師なしの感情ラベル伝播アルゴリズムによって修正されます．意見文の候補意見対象を抽出した後，意見対象抽出タスクは，ランダムウォークベースのランキングアルゴリズムで実行されます．このアルゴリズムは，意見対象候補間の関係と意見対象文章間のテキストの類似性を考慮し，意見文の候補意見対象をランク付けします．実験結果は，提案された方法におけるアルゴリズムの有効性を示しています．


Knowledge graph entity typing via learning connecting embeddings

Knowledge graph (KG) entity typing aims at inferring possible missing entity type instances in KG, which is a very significant but still under-explored subtask of knowledge graph completion. In this paper, we propose a novel approach for KG entity typing which is trained by jointly utilizing local typing knowledge from existing entity type assertions and global triple knowledge in KGs. Specifically, we present two distinct knowledge-driven effective mechanisms of entity type inference. Accordingly, we build two novel embedding models to realize the mechanisms. Afterward, a joint model via connecting them is used to infer missing entity type instances, which favors inferences that agree with both entity type instances and triple knowledge in KGs. Experimental results on two real-world datasets (Freebase and YAGO) demonstrate the effectiveness of our proposed mechanisms and models for improving KG entity typing.

ナレッジグラフ(KG)エンティティタイピングは，KGで欠落している可能性のあるエンティティタイプのインスタンスを推測することを目的としています．これは，ナレッジグラフ完了の非常に重要ですが，まだ十分に調査されていないサブタスクです． このホワイトペーパーでは，既存のエンティティタイプアサーションからのローカルタイピング知識とKGのグローバルトリプルナレッジを併用することでトレーニングされるKGエンティティタイピングの新しいアプローチを提案します． 具体的には，エンティティタイプの推論に関する2つの異なる知識主導の効果的なメカニズムを示します． したがって，メカニズムを実現するために2つの新しい埋め込みモデルを構築します． その後，それらを接続することによる結合モデルを使用して，欠落しているエンティティタイプのインスタンスを推論します．これにより，エンティティタイプのインスタンスとKGのトリプルナレッジの両方に一致する推論が優先されます． 2つの実世界のデータセット(FreebaseとYAGO)の実験結果は，KGエンティティのタイピングを改善するための提案されたメカニズムとモデルの有効性を示しています．


CGMVQA: A New Classification and Generative Model for Medical Visual Question Answering

Medical images are playing an important role in the medical domain. A mature medical visual question answering system can aid diagnosis, but there is no satisfactory method to solve this comprehensive problem so far. Considering that there are many different types of questions, we propose a model called CGMVQA, including classification and answer generation capabilities to turn this complex problem into multiple simple problems in this paper. We adopt data augmentation on images and tokenization on texts. We use pre-trained ResNet152 to extract image features and add three kinds of embeddings together to deal with texts. We reduce the parameters of the multi-head self-attention transformer to cut the computational cost down. We adjust the masking and output layers to change the functions of the model. This model establishes new state-of-the-art results: 0.640 of classification accuracy, 0.659 of word matching and 0.678 of semantic similarity in ImageCLEF 2019 VQA-Med data set. It suggests that the CGMVQA is effective in medical visual question answering and can better assist doctors in clinical analysis and diagnosis.

医療画像は，医療分野で重要な役割を果たしています．成熟した医療用視覚的質問応答システムは診断に役立ちますが，この包括的な問題を解決する満足のいく方法は今のところありません．このペーパーでは，さまざまな種類の質問があることを考慮して，この複雑な問題を複数の単純な問題に変換する分類および回答生成機能を含む，CGMVQAと呼ばれるモデルを提案します．画像のデータ拡張とテキストのトークン化を採用しています．事前トレーニング済みのResNet152を使用して画像の特徴を抽出し，3種類の埋め込みを一緒に追加してテキストを処理します．マルチヘッド自己注意トランスフォーマーのパラメーターを減らして，計算コストを削減します．マスキングレイヤーと出力レイヤーを調整して，モデルの機能を変更します．このモデルは，ImageCLEF 2019 VQA-Medデータセットでの分類精度0.640，単語マッチング0.659，意味的類似性0.678の新しい最先端の結果を確立します．これは，CGMVQAが医学的視覚的質問応答に効果的であり，臨床分析と診断において医師をよりよく支援できることを示唆しています．


TSVs crosstalk fault grouping test and diagnostic strategy


Improved run-length codes compression method based on tri-state signal


Energy Optimized Congestion Control Based Temperature Aware Routing Algorithm For Software Defined Wireless Body Area Networks

Wireless Body Area Network(WBAN) is a promising cost-effective technology for the privacy con ned military applications and healthcare applications like remote health monitoring, telemedicine, and e-health services. The use of a Software-De ned Network (SDN) approach improves the control and management processes of the complex structured WBANs and also provides higher exibility and dynamic network structure. To seamless routing performance in SDN-based WBAN, the energy-ef ciency problems must be tackled effectively. The main contribution of this paper is to develop a novel Energy Optimized Congestion Control based on Temperature Aware Routing Algorithm (EOCC-TARA) using Enhanced Multi-objective Spider Monkey Optimization (EMSMO) for SDN-based WBAN. This algorithm overcomes the vital challenges, namely energy-ef ciency, congestion-free communication, and reducing adverse thermal effectsin WBAN routing. First,the proposed EOCC-TARA routing algorithm considers the effects of temperature due to the thermal dissipation of sensor nodes and formulates a strategy to adaptively select the forwarding nodes based on temperature and energy. Then the congestion avoidance concept is added with the energy-ef ciency, link reliability, and path loss for modeling the cost function based on which the EMSMO provides the optimal routing. Simulations were performed, and the evaluation results showed that the proposed EOCC-TARA routing algorithm has superior performance than the traditional routing approaches in terms of energy consumption, network lifetime, throughput, temperature control, congestion overhead, delay, and successful transmission rate.

ワイヤレスボディエリアネットワーク(WBAN)は，プライバシーが制限された軍事アプリケーションや，リモートヘルスモニタリング，遠隔医療，e-ヘルスサービスなどのヘルスケアアプリケーション向けの有望な費用対効果の高いテクノロジーです．ソフトウェア定義ネットワーク(SDN)アプローチを使用すると，複雑な構造のWBANの制御および管理プロセスが改善され，柔軟性と動的なネットワーク構造が向上します． SDNベースのWBANでシームレスなルーティングパフォーマンスを実現するには，エネルギー効率の問題に効果的に取り組む必要があります．この論文の主な貢献は，SDNベースのWBAN向けに強化された多目的スパイダーモンキー最適化(EMSMO)を使用して，温度認識ルーティングアルゴリズム(EOCC-TARA)に基づく新しいエネルギー最適化輻輳制御を開発することです．このアルゴリズムは，エネルギー効率，輻輳のない通信，およびWBANルーティングでの熱の悪影響を減らすという重要な課題を克服します．まず，提案されたEOCC-TARAルーティングアルゴリズムは，センサーノードの熱散逸による温度の影響を考慮し，温度とエネルギーに基づいて転送ノードを適応的に選択する戦略を策定します．次に，輻輳回避の概念にエネルギー効率，リンクの信頼性，およびEMSMOが最適なルーティングを提供するコスト関数をモデル化するためのパス損失を追加します．シミュレーションが実行され，評価結果は，提案されたEOCC-TARAルーティングアルゴリズムが，エネルギー消費，ネットワークライフタイム，スループット，温度制御，輻輳オーバーヘッド，遅延，および正常な伝送速度に関して，従来のルーティングアプローチよりも優れたパフォーマンスを発揮することを示しました．


Video Emotion Recognition Based on Hierarchical Attention Model

LSTM network is widely used in facial expression recognition of video sequences. In view of the limited representation ability of single-layer LSTM and the limitation of its generalization ability when solving complex problems, a hierarchical attention model is proposed. Hierarchical representation of time series data is learned by stacking LSTM, self-attention mechanism is used to construct differentiated hierarchical relationships, and a penalty term is constructed and further combined with the loss function to optimize the network performance. Experiments on CK+ and MMI datasets, demonstrate that due to the construction of good hierarchical features, each step in time series can select information from the more interesting feature hierarchy. Compared with ordinary single-layer LSTM, hierarchical attention model can express the emotional information of video sequences more effectively.

Journal of Computer-Aided Design & Computer Graphics


Homography-based traffic sign localisation and pose estimation from image sequence

This study proposes a vision-based method for traffic sign attribute estimation, i.e. 3D position and pose, from image sequences by binocular or monocular cameras. The method starts with acquiring robust feature correspondences based on homography constraints from image pairs. Then the objective function is designed to integrate the feature correspondences to optimise the parameters of the traffic sign plane in the 3D coordinate. Finally, the sign plane is utilised for attribute estimation. In addition, the authors provide an extension for the raw KITTI dataset, which can be utilised for 3D tasks of traffic sign localisation and pose estimation. In the experiments, three popular methods are employed for comparisons based on the publicly available BelgiumTS and KITTI datasets. The results show that the authors' method based on SIFT and SURF features can locate the traffic signs with a mean error of ∼0.44 and 0.51 m in the BelgiumTS and KITTI datasets, respectively, and estimate the pose with a mean error of ∼14.45° in the KITTI dataset.

この研究は，両眼カメラまたは単眼カメラによる画像シーケンスからの交通標識属性推定，つまり3D位置とポーズのためのビジョンベースの方法を提案します．この方法は，画像ペアからのホモグラフィ制約に基づいてロバストな特徴の対応を取得することから始まります．次に，目的関数は，3D座標で交通標識平面のパラメーターを最適化するために機能の対応を統合するように設計されています．最後に，記号平面は属性推定に使用されます．さらに，著者は未加工のKITTIデータセットの拡張機能を提供します．これは，交通標識の位置特定と姿勢推定の3Dタスクに利用できます．実験では，一般に利用可能なベルギーのTSおよびKITTIのデータセットに基づく比較のために，3つの一般的な方法が採用されています．結果は，SIFTおよびSURF機能に基づく著者の方法が，ベルギーTSおよびKITTIデータセットでそれぞれ平均誤差約0.44および0.51 mの交通標識を特定し，平均誤差約14.45°で姿勢を推定できることを示しています．


Two-Level Attention with Multi-task Learning for Facial Emotion Estimation


Design on the Winter Jujubes Harvesting and Sorting Device

According to the existing problems of winter jujube harvesting, such as the intensive labor of manual picking, damage to the surface of winter jujubes, a winter jujube harvesting and sorting device was developed. This device consisted of vibration mechanism, collection mechanism, and sorting mechanism. The eccentric vibration mechanism made the winter jujubes fall, and the umbrella collecting mechanism can collect winter jujube and avoid the impact of winter jujube on the ground, and the sorting mechanism removed jujube leaves and divided the jujube into two types, and the automatic leveling mechanism made the device run smoothly in the field. Through finite element analysis and BP (Back Propagation) neural network analysis, the results show that: The vibration displacement of jujube tree is related to the trunk diameter and vibration position; the impact force of winter jujubes falling is related to the elastic modulus of umbrella material; the collecting area can be increased four times for each additional step of the collection mechanism; jujube leaves can be effectively removed when blower wind speed reaches 45.64 m/s. According to the evaluation standard grades of the jujubes harvesting and sorting, the device has good effects and the excellent rate up to 90%, which has good practicability and economy.

冬のナツメの収穫の既存の問題によると，冬のナツメの収穫，および選別装置の開発のために，手摘みの集中的な労働などのように，冬のナツメの表面に損傷を与えました．このデバイスは，振動メカニズム，コレクションメカニズム，ソートメカニズムで構成されています．冬のナツメのは，冬のナツメのに落ちて，冬のナツメのの影響を避けるために，冬のナツメのとの種の分類を削除することができます．デバイスをフィールドでスムーズに実行します．有限要素解析とBP(Back Propagation)ニューラルネットワーク解析により，結果は次のことを示しています．ナツメの木の振動変位は，トランクの直径と振動位置に関連しています．冬のナツメの落下力の影響は，傘の材料の弾性係数に関係しています．収集メカニズムの各追加ステップごとに，収集領域を4倍に増やすことができます．ナツメの葉は，送風機の風速が45.64m / sに達すると効果的に除去できます．ナツメの収穫と選別の評価基準に応じて，デバイスは優れた効果を発揮し，優れたレートは90%まで向上し，実用性と経済性に優れています．


A variable-length compatible compression scheme based on tristate signals

The test data volume for the manufacturing test of chips is increasing rapidly. This is due to the fact that the complexity of these chips is increasing rapidly and the transistor count is increasing exponentially. Aimed at solving the problems of large test data volume and long test time in chips test, this paper presents a deterministic test-based compression method that uses tristate signals encoding and compatible test cube merging. Firstly, the partial input reduction is carried out for the original test set. The proportion of ``don't care'' bits in the original test set is increased by merging compatible or inversely compatible inputs in the test cubes, so the compatibility among the test cubes is improved. Then, the test set is compressed and encoded by using tristate signal. Each test cube is divided into several data segments, which are encoded by tristate signals and multiple compatibility rules. This scheme can improve the compression ratio of the test set. The experimental results show that the proposed scheme achieves a good compression ratio, without excessive test power and area overhead. The average test compression ratio can reach 82.15%.

チップの製造テストのテストデータ量は急速に増加しています．これは，これらのチップの複雑さが急速に増加しており，トランジスタ数が指数関数的に増加しているという事実によるものです．チップテストでの大量のテストデータ量と長いテスト時間の問題を解決することを目的として，このホワイトペーパーでは，トライステート信号エンコーディングと互換性のあるテストキューブマージを使用する決定論的なテストベースの圧縮方法を紹介します．まず，元のテストセットに対して部分的な入力削減が実行されます．元のテストセットの「ドントケア」ビットの割合は，テストキューブの互換性のある入力または逆互換性のある入力をマージすることで増加し，テストキューブ間の互換性が向上します．次に，トライステート信号を使用してテストセットを圧縮およびエンコードします．各テストキューブは複数のデータセグメントに分割され，それらはトライステート信号と複数の互換性ルールによってエンコードされます．このスキームにより，テストセットの圧縮率を改善できます．実験結果は，提案された方式が，過剰なテスト電力と面積オーバーヘッドなしで，良好な圧縮率を達成することを示します．平均テスト圧縮率は82.15%に達する可能性があります．


Diagonal-based for TSV Fault Tolerance Design

Three dimensional integrated circuits, using through-silicon vias(TSVs) as the communication link between vertical chip, with the advantages of high density, high bandwidth and low power consumption. As the TSV during processing and using process may be failure, will lead to the entire three-dimensional chip failure. In order to improve the yield of 3D chips, the yield of TSV must be raised as much as possible. This paper presents a novel diagonal-based redundant TSV repair scheme and proposes a repair algorithm based on maximum flow algorithm to tolerance the faulty TSV with redundant TSVs in the diagonal to improve the yield of the entire three-dimensional chip. Experimental results show that the yield of the proposed repair strategy is 98.38% to 98.96% and the hardware overhead can be reduced by up to 70% compared to routerbased technique.

縦型チップ間の通信リンクとしてスルーシリコンビア(TSV)を使用した3次元集積回路．高密度，広帯域幅，低消費電力という利点があります． 処理中および使用中のTSVが故障する可能性があるため，全体の3次元チップの故障につながります． 3Dチップの歩留まりを向上させるためには，TSVの歩留まりをできるだけ上げる必要があります． 本論文では，新しい対角ベースの冗長TSV修復方式を提示し，全体の3次元チップの歩留まりを改善するために対角線内の冗長TSVを持つ故障TSVを許容する最大フローアルゴリズムに基づく修復アルゴリズムを提案した． 実験結果は，提案した修理戦略の収率は98.38%から98.96%であり，ハードウェアオーバーヘッドはルータベースの技術と比較して最大70%削減できることを示した．


Multi-Level Attention Based BLSTM Neural Network for Biomedical Event Extraction

Biomedical event extraction is an important and challenging task in Information Extraction, which plays a key role for medicine research and disease prevention. Most of the existing event detection methods are based on shallow machine learning methods which mainly rely on domain knowledge and elaborately designed features. Another challenge is that some crucial information as well as the interactions among words or arguments may be ignored since most works treat words and sentences equally. Therefore, we employ a Bidirectional Long Short Term Memory (BLSTM) neural network for event extraction, which can skip handcrafted complex feature extraction. Furthermore, we propose a multi-level attention mechanism, including word level attention which determines the importance of words in a sentence, and the sentence level attention which determines the importance of relevant arguments. Finally, we train dependency word embeddings and add sentence vectors to enrich semantic information. The experimental results show that our model achieves an F-score of 59.61% on the commonly used dataset (MLEE) of biomedical event extraction, which outperforms other state-of-the-art methods.

生物医学的イベント抽出は，情報の抽出において重要かつ挑戦的なタスクであり，医学研究と疾病予防に重要な役割を果たします．既存のイベント検出方法のほとんどは，主にドメインの知識と精巧に設計された機能に依存する浅い機械学習方法に基づいています．別の課題は，ほとんどの作品が単語と文を等しく扱うため，いくつかの重要な情報と単語または引数間の相互作用が無視される可能性があることです．したがって，イベント抽出には双方向のLong Short Term Memory(BLSTM)ニューラルネットワークを使用します．これにより，手作りの複雑な特徴抽出をスキップできます．さらに，文中の単語の重要度を決定する単語レベルの注意と，関連する引数の重要度を決定する文レベルの注意を含む，マルチレベルの注意メカニズムを提案します．最後に，依存語の埋め込みをトレーニングし，意味情報を充実させるために文ベクトルを追加します．実験結果は，我々のモデルが生物医学的イベント抽出の一般的に使用されるデータセット(MLEE)で59.61%のFスコアを達成していることを示しています．

IEICE Transactions on Information and Systems


Feature Reuse Residual Networks for Insect Pest Recognition

Insect pests are one of the main threats to the commercially important crops. An effective insect pest recognition method can avoid economic losses. In this paper, we proposed a new and simple structure based on the original residual block and named as feature reuse residual block which combines feature from the input signal of a residual block with the residual signal. In each feature reuse residual block, it enhances the capacity of representation by learning half and reuse half feature. By stacking the feature reuse residual block, we obtained the feature reuse residual network (FR-ResNet) and evaluated the performance on IP102 benchmark dataset. The experimental results showed that FR-ResNet can achieve significant performance improvement in terms of insect pest classification. Moreover, to demonstrate the adaptive of our approach, we applied it to various kinds of residual networks, including ResNet, Pre-ResNet, and WRN, and we tested the performance on a series of benchmark datasets: CIFAR-10, CIFAR-100, and SVHN. The experimental results showed that the performance can be improved obviously than original networks. Based on these experiments on CIFAR-10, CIFAR-100, SVHN, and IP102 benchmark datasets, it demonstrates the effectiveness of our approach.

害虫は，商業的に重要な作物に対する主要な脅威の1つです．効果的な害虫認識方法は，経済的損失を回避できます．本稿では，元の残差ブロックに基づいた新しい単純な構造を提案し，残差ブロックの入力信号からの特徴を残差信号と結合する特徴再利用残差ブロックと名付けた．各特徴の再利用残差ブロックでは，半分の特徴を学習し，半分の特徴を再利用することで表現の能力を高めます．特徴再利用残差ブロックを積み重ねることにより，特徴再利用残差ネットワーク(FR-ResNet)を取得し，IP102ベンチマークデータセットのパフォーマンスを評価しました．実験結果は，FR-ResNetが害虫の分類に関して大幅な性能改善を達成できることを示しました．さらに，アプローチの適応性を実証するために，ResNet，Pre-ResNet，WRNなどのさまざまな種類の残余ネットワークに適用し，一連のベンチマークデータセット(CIFAR-10，CIFAR-100，およびSVHN．実験結果は，元のネットワークよりも明らかにパフォーマンスを改善できることを示しました． CIFAR-10，CIFAR-100，SVHN，およびIP102ベンチマークデータセットに関するこれらの実験に基づいて，このアプローチの有効性が実証されています．


Facial Expression Recognition Based on Fusion Features of Center-Symmetric Local Signal Magnitude Pattern

Local feature descriptors play a fundamental and important role in facial expression recognition. This paper presents a new descriptor, Center-Symmetric Local Signal Magnitude Pattern (CS-LSMP), which is used for extracting texture features from facial images. CS-LSMP operator takes signal and magnitude information of local regions into account compared to conventional LBP-based operators. Additionally, due to the limitation of single feature extraction method and in order to make full advantagesofdifferentfeatures,thispaperemploysCS-LSMPoperatortoextractfeaturesfromOrientational Magnitude Feature Maps (OMFMs), Positive-and-Negative Magnitude Feature Maps (PNMFMs), Gabor FeatureMaps(GFMs)andfacialpatches(eyebrows-eyes,mouths)forobtainingfusedfeatures.UnlikeHOG, which only retains horizontal and vertical magnitudes, our work generates Orientational Magnitude Feature Maps (OMFMs) by expanding multi-orientations. This paper build two distinct feature maps by dividing local magnitudes into two groups, i.e., positive and negative magnitude feature maps. The generated Gabor FeatureMaps(GFMs)arealsogroupedtoreducethecomputationalcomplexity.ExperimentsontheJAFFE andCK+facialexpressiondatasetsshowedthattheproposedframeworkachievedsigni cantimprovement and outperformed some state-of-the-art methods.


Facial Expression Recognition Using Fusion Features Based on Center-Symmetric Local Octonary Pattern

A local feature descriptor has gained a lot of interest in many applications, such as image retrieval, texture classi cation, and face recognition. This paper proposes a novel local feature descriptor: center-symmetric local octonary pattern (CS-LOP) for facial expression recognition. A CS-LOP operator not only considers the difference of the gray value between central pixels and neighboring pixels in all eight directions but also compares the gray value of four pairs of center-symmetric pixels. Besides, this paper used the CS-LOP to extract diverse features from the preprocessed facial image, the feature map of gradient magnitude, and the feature map of Gabor, and also to make extracted features more abundant and detailed. To evaluate the performance of the proposed method, experiments on JAFFE and CK facial expression datasets demonstrate that the proposed method outperforms the method using the individual descriptor. Compared with other state-of-the-art methods, our approach improves the overall recognition accuracy.


BeSense: Leveraging WiFi Channel Data and Computational Intelligence for Behavior Analysis

The ever-evolving informatics technology has gradually bounded human and computer in a compact way. Understanding user behavior becomes a key enabler in many fields such as sedentary-related healthcare, human-computer interaction and affective computing. Traditional sensor-based and vision-based user behavior analysis approaches are obtrusive in general, hindering their usage in real-world. Therefore, in this article, we first introduce the WiFi signal as a new source instead of sensor and vision for unobtrusive user behaviors analysis. Then we design BeSense, a contactless behavior analysis system leveraging signal processing and computational intelligence over WiFi channel state information. We prototype BeSense on commodity low-cost WiFi devices and evaluate its performance in real-world environments. Experimental results have verified its effectiveness in recognizing user behaviors.

IEEE Computational Intelligence Magazine


Drug-Drug Interaction Extraction Based on Transfer Weight Matrix and Memory Network

Extracting drug drug interaction (DDI) in the text is the process of identifying how two target drugs in a given sentence interact. Previous methods, which were limited to conventional machine learning techniques, we are susceptible to issues such as ``vocabulary gap'' and unattainable automation processes in feature extraction. Inspired by deep learning in natural language preprocessing, we addressed theaforementionedproblemsbasedondynamictransfermatrixandmemorynetworks.ATM-RNNmethod is proposed by adding the transfer weight matrix in multilayer bidirectional LSTM to improve robustness andintroduceamemorynetworkforfeaturefusion.WeevaluatedtheTM-RNNmodelontheDDIExtraction 2013Task.TheproposedmodelachievedanoverallF-scoreof72.43,whichoutperformsthelatestmethods based on support vector machine and other neural networks. Meanwhile, the experimental results also indicated that the proposed model is more stable and less affected by negative samples.

テキスト内の薬物間相互作用(DDI)を抽出することは，特定の文の2つのターゲット薬物が相互作用する方法を識別するプロセスです． 従来の機械学習手法に限定されていた以前の方法では，「語彙ギャップ」や特徴抽出における達成不可能な自動化プロセスなどの問題の影響を受けやすくなっています． 自然言語の前処理における深層学習に着想を得て，前述の動的転送行列およびメモリネットワークに基づく問題に対処しました． およびその他のニューラルネットワーク． 一方，実験結果は，提案されたモデルがより安定しており，負のサンプルによる影響が少ないことも示しました．


Test data compatible compression method based on tri-state signal


Two-level attention with two-stage multi-task learning for facial emotion recognition

Compared with facial emotion recognition on categorical model, the dimensional emotion recognition can describe numerous emotions of the real world more accurately. Most prior works of dimensional emotion estimation only considered laboratory data and used video, speech or other multi-modal features. The effect of these methods applied on static images in the real world is unknown. In this paper, a two-level attention with two-stage multi-task learning (2Att-2Mt) framework is proposed for facial emotion estimation on only static images. Firstly, the features of corresponding region(position-level features) are extracted and enhanced automatically by first-level attention mechanism. In the following, we utilize Bi-directional Recurrent Neural Network(Bi-RNN) with self-attention(second-level attention) to make full use of the relationship features of different layers(layer-level features) adaptively. Owing to the inherent complexity of dimensional emotion recognition, we propose a two-stage multi-task learning structure to exploited categorical representations to ameliorate the dimensional representations and estimate valence and arousal simultaneously in view of the correlation of the two targets. The quantitative results conducted on AffectNet dataset show significant advancement on Concordance Correlation Coefficient(CCC) and Root Mean Square Error(RMSE), illustrating the superiority of the proposed framework. Besides, extensive comparative experiments have also fully demonstrated the effectiveness of different components.

Journal of Visual Communication and Image Representation


Emotion Recognition Method Based on Instantaneous Energy of Electroencephalography


Hybrid Spatiotemporal Models for Sentiment Classification Via Galvanic Skin Response

Sentiment plays an important role in cognition, creativity, attention, and decision making in people's daily lives. Researchers have made great progress in sentiment recognition through images and speech. In this paper, a multimodal dataset is proposed for sentiment classi cation (MDSTC 1 ) a multimodal dataset col lected with multimodal channels by customized physiological sensors and manually labelled for human sentiment analysis. MDSTC contains galvanic skin response (GSR), pulse, speech, and facial expression data of 100 volunteers labelled with timeline, emotional self-ratings, and personal information such as age, sex and Big Five personality scores, which were collected while watching video clips to stimulate emotions. Using the MDSTC dataset, the GSR channel with six types of emotion labels is used to perform human sentiment analysis in this paper. The GSR signal is converted into a spectrogram to adopt image related methods and deep learning models. A convolutional neural network, long short-term memory, and self-attention mechanism are adopted and combined, and spatiotemporal hybrid models are pro posed to perform human sentiment analysis. Comparisons are performed between the proposed model and state-of-the-art models in related works. The experimental results show that with the proposed spa tiotemporal hybrid models, better results are obtained with respect to precision, recall, and F1-score. The experimental results also show that with the proposed spatiotemporal hybrid models working with GSR, people's emotional changes can be obtained in real time with high precision.


Toward action comprehension for searching: Mining actionable intents in query entities

Journal of the Association for Information Science and Technology


Recognition of Collocation Frames from Sentences

Collocation is a ubiquitous phenomenon in languages and accurate collocation recognition and extraction is of great signi cance to many natural language processing tasks. Collocations can be di erentiated from simple bigram collocations to collocation frames(referring to distant multi-gram collocations). So far little focus is put on collocation frames. Oriented to translation and parsing, this study aims to recognize and extract the longest possible collocation frames from given sentences. We rst extract bigram collocations with distributional semantics based method by introducing collocation patterns and integrating some state-of-the-art association measures. Based on bigram collocations extracted by the proposed method, we get the longest collocation frames according to recursive nature and linguistic rules of collocations. Compared with the baseline systems, the proposed method performs signi cantly better in bigram collocation extraction both inprecision and recall. And in extracting collocation frames, the proposed method performs even better with the precision similar to its bigram collocation extraction results.

コロケーションは，言語における遍在する現象であり，正確なコロケーションの認識および抽出は，多くの自然言語処理タスクにとって非常に重要である．コロケーションは，単純なバイグラムコロケーションからコロケーションフレームへと区別することができます(離れたマルチグラムコロケーションを参照)．これまでのところ，コロケーションフレームにはほとんど焦点を当てていません．翻訳と構文解析を目的としたこの研究は，与えられた文から最長のコロケーションフレームを認識し抽出することを目的としています．まず，コロケーションパターンを導入し，いくつかの最先端の関連尺度を統合することにより，分散セマンティクスベースの方法でバイグラムコロケーションを抽出します．提案した方法によって抽出されたバイグラムコロケーションに基づいて，再帰的性質およびコロケーションの言語学的規則に従って最長のコロケーションフレームを得る．ベースラインシステムと比較して，提案された方法は，不正確さおよび想起の両方において，バイグラムコロケーション抽出において著しく良好に機能する．そしてコロケーションフレームを抽出する際に，提案された方法は，そのバイグラムコロケーション抽出結果と同様の精度でさらに良好に機能する．

IEICE Transactions on Information and Systems


A Robust Parameter-Free Thresholding Method for Image Segmentation

Inthispaper,wepresentedanewparameter-freethresholdingmethodforimagesegmentation. In separating an image into two classes, the method employs an objective function that not only maximizes the between-class variance but also the distance between the mean of each class and the global mean of the image. The design of the objective function aims to circumvent the challenge that many existing techniques encounter when the underlying two classes have very different sizes or variances. The advantages of the new method are twofold. First, it is parameter-free, meaning that it can generate consistent results. Second, the new method has a simple form that makes it easy to adapt to different applications. We tested and compared the new method with the standard Otsu method, the maximum entropy method, and the 2D Otsu method on the simulated and real biomedical and photographic images and found that the new method can achieve a more accurate and robust performance.

この論文では，パラメータ - フリースレッシュホールド法を用いて画像分割を行った．画像を2つのクラスに分離する際に，この方法は，クラス間分散を最大化するだけでなく，各クラスの平均と画像の全体的平均との間の距離も最大化する目的関数を使用する．目的関数の設計は，基礎となる2つのクラスのサイズや分散が非常に異なる場合に既存の多くの技法が遭遇する課題を回避することを目的としている．新しい方法の利点としては，パラメータがないため，一貫した結果が得られること，異なるアプリケーションへの適応を容易にする簡単な形式をしている，ことがあげられる．実際の生物医学的および写真の画像について，新しい方法を標準のOtsu法，最大エントロピー法，および2D Otsu法と比較し，新しい方法がより正確で堅牢な性能を達成できることを確かめた．


Slang Feature Extraction by Analyzing Topic Change on Social Media

Recently, we often see words such as youth slang, neologism and Internet slang on social networking sites (SNSs) that are not registered on dictionaries. Because the documents posted to SNSs include a lot of fresh information, they are thought to be useful for collecting information. It is important to analyze these words (hereinafter referred to as slang) and capture their features for the improvement of the accuracy of automatic information collection. This paper aims to analyze what features can be observed in slang by focusing on the topic. We construct topic models from document groups including target slang on Twitter by Latent Dirichlet Allocation (LDA). With the models, we chronologically analyze change of topics during a certain period of time to find out the difference in the features between slang and general words. Then, we propose a slang classification method based on the change of features.

CAAI Transactions on Intelligence Technology


Background Knowledge Based Multi-Stream Neural Network for Text Classification

As a foundation and typical task in natural language processing, text classification has been widely applied in many fields. However, as the basis of text classification, most existing corpus are imbalanced and often result in the classifier tending its performance to those categories with more texts. In this paper, we propose a background knowledge based multi-stream neural network to make up for the imbalance or insufficient information caused by the limitations of training corpus. The multi-stream network mainly consists of the basal stream, which retains original sequence information, and background knowledge-based streams. Background knowledge is composed of keywords and co-occurred words which are extracted from external corpus. Background knowledge-based streams are devoted to realizing supplemental information and reinforce the basal stream. To better fuse the features extracted from different streams, early-fusion and two after-fusion strategies are employed. According to the results obtained from both Chinese corpus and English corpus, it is demonstrated that the proposed background knowledge based multi-stream neural network performs well in classification tasks.

自然言語処理の基礎となる典型的なタスクとして，テキスト分類は多くの分野で広く適用されてきた．しかし，テキスト分類の基礎として，既存のコーパスのほとんどが不均衡であり，しばしば分類子がより多くのテキストを有するカテゴリにその性能を向ける結果となる．本論文では，トレーニングコーパスの限界に起因する不均衡や不十分な情報を補うための背景知識ベースのマルチストリームニューラルネットワークを提案する．マルチストリームネットワークは，元のシーケンス情報を保持する基底ストリームと背景知識ベースのストリームから主に構成される．背景知識は，外部コーパスから抽出されたキーワードと共起語からなる．背景知識ベースのストリームは，補足情報を実現し，基礎ストリームを強化することに専念している．異なるストリームから抽出された特徴をよりよく融合させるために，初期融合および2つのアフター融合戦略が用いられる．実験により，提案された背景知識ベースのマルチストリームニューラルネットワークが分類作業において良好に機能することが実証された．


Sentiment Analysis Method Based on an Improved Modifying-MatrixLanguage Model

In this paper, a sentence-level sentiment analysis method is proposed to deal with sentiment measurement and classificationproblems. It is developed from a model called the synthetic and computational language model (SCLM), which representsmodifying and modified information, respectively, using matrices and vectors. In the proposed method, a global modifyingmatrix of a sentence is constructed, the determinant value of this matrix is calculated and adjusted, and then the final valueis used as the sentiment value of the sentence. Regression experiment shows that the deviation between the output sentimentand the target sentiment does not exceed a class distance of five classes. The classification experiment shows that the proposedmethod has improved most of the performance compared to the simplified SCLM and in some cases, such as in `very positive'class and `very negative' class, reaches higher precision performance than the baseline method.

本論文では，センチメント測定と分類問題を扱うために文レベルのセンチメント分析法を提案した．それは，合成および計算言語モデル(SCLM)と呼ばれるモデルから開発され，それはそれぞれ行列およびベクトルを使用して修正および修正された情報を表す．提案された方法では，文の大域的修正行列が構築され，この行列の行列式の値が計算され調整され，そして最終値が文の感情値として使用される．回帰実験は，出力感情と目標感情の間の偏差が5クラスのクラス距離を超えないことを示しています．分類実験は，提案された方法が単純化されたSCLMと比較して性能の大部分を改善し，場合によっては「非常にポジティブ」クラスおよび「非常にネガティブ」クラスのようにベースライン方法よりも高い精度性能に達することを示す．

IEEJ Transactions on Electrical and Electronic Engineering (TEEE)


Emotion recognition using empirical mode decomposition and approximation entropy

Automatic human emotion recognition is a key technology for human-machine interac tion. In this paper, we propose an electroencephalogram (EEG) feature extraction method that leverages empirical mode decomposition and Approximation Entropy. In our proposed method, Empirical Mode Decomposition (EMD) is used to process EEG signals after data processing and obtains several intrinsic eigenmode functions. The Approximation Entropy (ApEn) of the rst four Intrinsic Mode Functions (IMFs) is computed, which is used as the features from EEG signals for learning and recognition. An integration of Deep Belief Net work and Support Vector Machine is devised for classi cation, which takes the eigenvec tors from the extracted feature to identify four principal human emotions, namely happy, calm, sad, and fear. Experiments are conducted with EEG data acquired with a 16-lead device. Our experimental results demonstrate that the proposed method achieves an im proved accuracy that is highly competitive to the state-of-the-art methods. The average accuracy is 83.34%, and the best accuracy reaches 87.32%

感情認識は人間と機械の相互作用のための重要な技術である．本論文では，経験的モード分解と近似エントロピーを利用した脳波(EEG)特徴抽出法を提案する．提案手法では，データ処理後の脳波信号を処理するためにEMD(Empirical Mode Decomposition)が用いられ，いくつかの固有固有モード関数が得られる．最初の4つの固有モード関数(IMF)の近似エントロピー(ApEn)が計算され，学習と認識のためのEEG信号のフィーチャとして使用される． Deep Belief NetとSupport Vector Machineの統合は，抽出された特徴から固有ベクトルをとり，4つの主要な人間の感情，すなわち幸せ，穏やか，悲しみ，恐怖を識別する分類のために考案される． 16誘導装置で得られた脳波データを用いて実験を行う．我々の実験結果は，平均精度は83.34%であり，最高精度は87.32%である．これは提案された方法の有効性を確かめることができた．


Emotion recognition based on physiological signals using brain asymmetry index and echo state network

This paper proposes a method to evaluate the degree of emotion being motivated in continuous music videos based on asymmetry index (AsI). By collecting two groups of electroencephalogram (EEG) signals from 6 channels (Fp1, Fp2, Fz and AF3, AF4, Fz) in the left and right hemispheres, multidimensional directed information is used to measure the mutual information shared between two frontal lobes, and then, we get AsI to estimate the degree of emotional induction. In order to evaluate the effect of AsI processing on physiological emotion recognition, 32-channel EEG signals, 2-channel EEG signals and 2-channel EMG signals are selected for each subject from the DEAP dataset, and different sub-bands are extracted using wavelet packet transform. k-means algorithm is used to cluster the wavelet packet coef cients of each subband, and the probability distribution of the coef cients under each cluster is calculated. Finally, the probability distribution value of each sample is sent as the original features into echo state network for unsupervised intrinsic plasticity training; the reservoir state nodes are selected as the nal feature vector and fed into the support vector machine. The experimental results show that the proposed algorithm can achieve an average recognition rate of 70.5% when the subjects are independent. Compared with the case without AsI, the recognition rate is increased by 8.73%. On the other hand, the ESN is adopted for the original physiological feature re nement which can signi cantly reduce feature dimensions and be more bene cial to the emotion classi cation. Therefore, this study can effectively improve the performance of human machine interface systems based on emotion recognition.

本論文では，アシンメトリー指数(AsI)に基づいた連続的な音楽ビデオの動機づけの程度を評価する方法を提案する．左右半球の6つのチャネルから2つの脳波(EEG)信号群を収集することにより，2つの前頭葉間で共有される相互情報を多次元指向情報で測定し，次に，AsIに感情誘導の程度を推定させる．生理的感情認識へのAsI処理の効果を評価するために，DEAPデータセットから32チャンネルEEG信号，2チャンネルEEG信号，2チャンネルEMG信号を選択し，ウェーブレットを用いて異なるサブバンドを抽出する．各サブバンドのウェーブレットパケット係数をクラスタリングするためにk-meansアルゴリズムを使用し，各クラスタ下の係数の確率分布を計算する．最後に，各サンプルの確率分布値は，監督されていない内因性可塑性訓練のために元の特徴としてエコー状態ネットワークに送られる．実験結果は，被験者が独立している場合，提案アルゴリズムが平均認識率70.5%を達成できることを示している． AsIを用いない場合と比較して，認識率は8.73%向上した．一方，ESNは，元の生理学的特徴のために採用されており，特徴の大きさを著しく減少させることができ，感情の分類に対してより有益である．したがって，感情認識に基づくヒューマンマシンインタフェースシステムの性能を効果的に向上させることができる．


Occluded Facial Expression Recognition Based on Asymmetric Region Weber Local Descriptor and Block Similarity Weighting

非対称領域局所記述子とブロック類似性重み付けに基づく隠れた表情認識


Video facial expression recognition combined with sliding window dynamic time warping and CNN

スライディングウィンドウ動的タイムワープとCNNを組み合わせたビデオ表情認識


A Novel Two-Layered Reinforcement Learning for Task Of oading with Tradeoff between Physical Machine Utilization Rate and Delay

Mobile devices could augment their ability via cloud resources in mobile cloud computing environments. This paper developed a novel two-layered reinforcement learning (TLRL) algorithm toconsider taskof oading forresource-constrained mobiledevices. Asopposed toexisting literature, the utilization rate of the physical machine and the delay for of oaded tasks are taken into account simultaneously by introducing a weighted reward. The high dimensionality of the state space and action space might affect the speed of convergence. Therefore, a novel reinforcement learning algorithm with a two-layered structure is presented to address this problem. First, k clusters of the physical machines are generated based on the k-nearest neighbors algorithm (k-NN). The rst layer of TLRL is implemented by a deep reinforcement learning to determine the cluster to be assigned for the of oaded tasks. On this basis, the second layer intends to further specify a physical machine for task execution. Finally, simulation examples are carried out to verify that the proposed TLRL algorithm is able to speed up the optimal policy learning and can deal with the tradeoff between physical machine utilization rate and delay.


Dual-Modal Emotion Recognition Based on Facial Expression and Body Posture in Video Sequences

ビデオシーケンスにおける顔の表情と体位に基づく二重モード感情認識


Rough Set Based Knowledge Predicate Analysis of Chinese Knowledge Based Question Answering

In knowledge based question answering system, the performance of knowledge predicate analysis can affect the overall match result of knowledge triple The knowledge predicate analysis of Chinese short questionis difficult because of the uncertainty of Chinese knowledge predicate representation Based on the rough set theory, a new definition of knowledge predicate analysis of knowledge based questions answering was given, and a new method was proposed to analyze the knowledge predicate of question It can reduce the words which are weakly related with the knowledge predicate, and then the words which are more related with knowledge predicate representation will be used to match the knowledge triples to improve the overall performance of system The experiment results verify the validity of the method


Parallel test scheduling optimization method for three-dimensional chip with multi-core and multi-layer

マルチコアとマルチレイヤを持つ三次元チップのための並列テストスケジューリング最適化法

In order to solve the problem of high cost of chip testing in the process of Three-Dimensional ( 3D) chip manufacturing a new scheduling method based on Time Division Multiplexing ( TDM) was proposed to optimize the testing resources between layers layer and core cooperatively Firstly the shift registers were arranged on each layer of 3D chip and the testing frequency was divided properly between the layers and cores of the same layer under the control of shift register group on input data so that the cores in different locations could be tested in parallel Secondly greedy algorithm was used to optimize the allocation of registers for reducing the free test cycles of core parallel test Finally Discrete Binary Particle Swarm Optimization ( DBPSO) algorithm was used to find out the best 3D stack layout so that the transmission potential of the Through Silicon Via ( TSV) could be adequately used to improve the parallel testing efficiency and reduce the testing time The experimental results show that under the power constraints the utilization rate of the optimized whole Test Access Mechanism ( TAM) is increased by an average of 16 28% and the testing time of the optimized 3D stack is reduced by an average of 13 98% The proposed method can decrease the time and reduce the cost of testing

三次元(3D)チップ製造の過程における高コストのチップ試験の問題を解決するために，時分割多重化(TDM)に基づく新しいスケジューリング法を提案し，層間，層とコア間の試験資源を最適化した．協力的に．第一に，シフトレジスタを3Dチップの各層に配置し，テスト周波数を入力データのシフトレジスタ群の制御下で同じ層の層とコアの間で適切に分割した．並行してテスト．第二に，欲張りアルゴリズムを用いてコア並列テストのフリーテストサイクルを減らすためにレジスタの割り当てを最適化した．最後に，離散二成分粒子群最適化(DBPSO)アルゴリズムを用いて最良の3Dスタックレイアウトを見つけたので，スルーシリコンビア(TSV)の伝送能力を適切に用いて並列試験効率を改善し試験時間を短縮することができた． ．実験結果は，電力制約の下で，最適化された全体テストアクセスメカニズム(TAM)の利用率が平均16だけ増加することを示した． 28%，最適化された3Dスタックのテスト時間は平均13分短縮された． 98%．提案した方法は時間を短縮し試験のコストを削減することができる．