Emotional Intelligence is the Future of Artificial Intelligence

Source: http://www.zdnet.com/article/emotional-intelligence-is-the-future-of-artificial-intelligence-fjord/ — Fjord

Prologue:

By examining these different forms of mind activity, Minsky says, we can explain why our thought sometimes takes the form of carefully reasoned analysis and at other times turns to emotion. He shows how our minds progress from simple, instinctive kinds of thought to more complex forms, such as consciousness or self-awareness. And he argues that because we tend to see our thinking as fragmented, we fail to appreciate what powerful thinkers we really are. Indeed, says Minsky, if thinking can be understood as the step-by-step process that it is, then we can build machines — artificial intelligences — that not only can assist with our thinking by thinking as we do but have the potential to be as conscious as we are [¹].

Figure 1: The EMOTION MACHINE.

Abstract of the Interview:

The most successful artificial intelligence (AI) systems will be those comprising an emotional intelligence almost indistinguishable from human-to-human interaction, according to Bronwyn van der Merwe, group director at Fjord Australia and New Zealand — Accenture Interactive’s design and innovation arm.

While the concept of AI is not new, in 2017 van der Merwe expects emotional intelligence to emerge as the driving force behind what she called the next generation in AI, as humans will be drawn to human-like interaction.

Van der Merwe explained that building on the first phase of AI technology, emotional intelligence enhances AI’s ability to understand emotional input, and continually adapt to and learn from information to provide human-like responses in real time.

With consumer appetite for AI expected to continue to grow at a rapid pace, van der Merwe predicts emotional intelligence will be the critical differentiator separating the great from the good in AI products, especially given that by 2020 she expects the average person to have more conversations with chat bots than with human staff.

While the debate over machines displacing workers has been discussed at length, van der Merwe is certain that AI won’t ever completely replace human beings.

“As human beings, we have contextual understanding and we have empathy, and right now there isn’t a lot of that built into AI. We do believe that in the future, the companies that are going to succeed will be those that can build into their technology that kind of understanding,” she said.

Technical Review:

Definition of Emotional Intelligence

Emotional intelligence (EI) or emotional quotient (EQ) is the ability of individuals to recognize their own and other people’s emotions, to discriminate between different feelings and label them appropriately, and to use emotional information to guide thinking and behavior. The term gained prominence in the 1995 book by that title, written by the author, psychologist, and science journalist Daniel Goleman.

Emotional intelligence consists of the ability to recognize, express, and have emotions, coupled with the ability to regulate these emotions, harness them for constructive purposes, and skillfully handle the emotions of others. The skills of emotional intelligence have been argued to be a better predictor than IQ for measuring aspects of success in life [²].

Necessity of Emotional Intelligence

Machines may never need all of the emotional skills that people need; however, there is evidence that machines will require at least some of these skills to appear intelligent when interacting with people. A relevant theory is that of Reeves and Nass at Stanford: Human-computer interaction is inherently natural and social, following the basics of human-human interaction [³].

Stanford University’s leading AI scientist Fei-Fei Li mentioned that the next step in the development of artificial intelligence is to strengthen the understanding of emotion from not only brain science, but also cognitive science. Because we currently have very little understanding of human emotions, which is very important for artificial intelligence.

Figure 2: Prof. Fei-Fei Li was giving a talk.

Emotion Recognition — Computer Science of Emotional Intelligence

The ability to recognize emotion is one of the hallmarks of emotional intelligence, an aspect of human intelligence that has been argued to be even more important than mathematical and verbal intelligences. Researchers believe that the first step of achieving emotion intelligence of machine is emotion recognition [4]. Emotion can be detected from several sources, such as facial expression, speech, text, etc.

• Emotion Recognition from Facial Expression

There are many ways for humans display their emotions. The most natural way to display emotions is using facial expressions. In the last 20 years, there has been a great deal of research on recognizing emotion through facial expressions. This research was pioneered by Ekman and Friesen who started their work from psychology perspectives.

In general, emotion recognition from facial expression is a two-steps procedure which involves extraction of significant features and classification. Feature extraction determines a set of independent attributes, which together can portray an expression of facial emotion. For classification, the features are mapped into various emotion classes like anger, happy, sad, disgust, surprise, etc. To calculate the effectiveness of a facial expression identification model, both the group of feature attributes which have been taken for feature extraction, and the classifier that is responsible classification are equivalently significant. In some cases, for a poorly chosen collection of feature attributes, even a smart classification mechanism will not be able to produce an ideal outcome. Thus, selecting superior features plays a major role in achieving high classification accuracy and qualitative outcome[5, 6].

Figure 3: Different emotions are recognized from facial expression.

Emotion recognition from facial expression has achieved great success in recent years, thanks to the development of neural network and GPU. However, most reported facial emotion recognition systems are not fully considered subject-independent dynamic features, so they are not robust enough for real life recognition tasks with subject (human face) variation, head movement and illumination change.

• Emotion Recognition from Speech

Speech signal is one of the most natural methods of communication between humans. The basic steps of emotion recognition from speech is the same as that from facial expression, which are feature extraction and classification.

Figure 4: The architecture of emotion recognition from speech.

The two major types of models used to describe emotion are discrete models and dimensional models. Discrete models are established based on the Darwinian theories, which insist on the existence of a small number of basic or fundamental emotions. The best known discrete model is the “Big Six”: joy, sadness, anger, fear, disgust and surprise. Dimensional models map emotional states in a low-dimensional space (with two or three dimensions). The two widely accepted dimensions are valence and arousal. The third one often refers to dominance.

After a long time in research, speech emotion recognition has been used in call center applications, mobile communication, and disease detection. However, for some reasons, the task of accurate recognition is still very challenging. Researchers are still debating what features influence the recognition of emotion in speech. There is also considerable uncertainty as to the best algorithm for classifying emotion, and which emotions to class together [7].

• Emotion Recognition from Text

Emotion recognition from text is a recent field of research that is closely related to sentiment analysis. Sentiment analysis aims to detect positive, neutral, or negative feelings from text, whereas emotion analysis aims to detect and recognize types of feelings through the expression of texts [8].

Figure 5: The steps of emotion recognition from text.

One of the biggest challenges in determining emotion is the context-dependence of emotions within text. A phrase can have element of anger without using the word “anger” or any of its synonyms, an example would be “Shut up!”. Another challenge is the difficulty that other components of NLP are facing, such as word-sense disambiguation and co-reference resolution. It is difficult to anticipate the success rate of machine learning approaches without first trying. Another challenge in emotion detection is the lack of a labelled emotion database to enable active innovation. Currently, few publicly accessible databases are available.

Interdisciplinary Approach of Emotional Intelligence

In computer science, emotions are only recognized from external information based on data and algorithm. But why do people change their facial expression? Why do they use different voices? To answer these questions, cognitive science, neuroscience, along with psychology are aimed at figuring out the internal workings.

• Cognitive Science of Emotional Intelligence

Cognitive approaches offer clear links between how emotions are thought about in everyday life and how they are investigated psychologically. Cognitive researchers have focused on how emotions are affected by events or other people, and on how emotions influence processes such as reasoning, memory, and attention. There are three representative cognitive theories of emotion continuously being developed: the action-readiness theory, the core-affect theory, and the communicative theory.

The action-readiness theory is the longest standing cognitive theory of emotions, which is based on evidence that different emotions relate to both different appraisals and different states of readiness for kinds of action. It has led to computational models, analyses of Chinese poetics, and cross-cultural studies.

The core-affect theory was proposed by Russel that underlying any emotion is core affect, a state with two dimensions: level of arousal and pleasure versus displeasure. The intuition behind the theory of core affect is that although people talk of emotions such as anger and fear, such states are not distinct and they are not evolutionary universals. As prototypes, they overlap.

The communicative theory postulates that emotions are communications within the brain and among individuals. Rapid appraisals of situations in relation to current goals fall into a small number of generic events, such as trains of action going well, losses, frustrations, and dangers. Appraisals are cognitive, although not necessarily conscious. They are signals that set body and mind into modes that have been shaped by evolution and individual experience, to prompt a person toward certain kinds of action appropriate to the generic event and to impart urgency to these actions [9].

Figure 6: According to the communicative theory, emotions can be free floating although some must have known objects, along with antecedents and consequences for individual and relational action.

• Neuroscience of Emotional Intelligence

The growth of EI has been driven by some significant steps in our understanding of how the brain works. To understand and develop EI, we must understand the neuroscience behind it.

Early models in psychology described human behavior in terms of stimulus and response. However, advancements in psychology and neuroscience have shown that several stages fall in-between stimulus and response. That is, information is initially filtered through our attitudes before being processed as feelings, emotions and thoughts. The response to this is our behavior, from which there is an outcome.

Figure 7: Several stages fall in-between stimulus and response.

Science also shows that different regions of the brain facilitate EI. These regions are broadly represented below.

Figure 8: Different regions of the brain facilitate emotional intelligence.

These insights into the workings of the brain have profound implications on how to develop EI. For example, people often know what they should do but do not put this into practice. One reason is that knowing about something lives in a different part of the brain (neocortex) from doing something (limbic region). The emotional or limbic brain learns through doing. Therefore, in order to turn good intentions into habits of behavior an individual needs to put them into practice through rehearsal and physical experience [10].

Conclusion:

The way human experience emotions evolves dramatically from birth to adulthood. The expression of one’s emotions is heavily regulated by culture and taboos. Machines don’t mature emotionally, do not go through puberty, do not have hormonal cycles, and do not undergo hormonal change based on their age, diet and environment. Artificial intelligence could learn from experience and mature intellectually, but not mature emotionally like a child becoming an adult. This is the vital difference that shouldn’t be underestimated. Marvin Minsky argues persuasively that emotions, intuitions, and feelings are not distinct things, but different ways of thinking. Although emotion has been researched for decades, it’s still far away from being accurately recognized or expressed by machines. Computer scientists are using deep learning to utilize data, but cognitive scientist and neuroscientists haven’t found a common and useful model to to deal with emotion. There is still a long way to go to achieve emotional intelligence before we figure out what emotion is.

Epilogue:

“It’s about thinking. The main theory is that emotions are nothing special. Each emotional state is a different style of thinking. So it’s not a general theory of emotions, because the main idea is that each of the major emotions is quite different. They have different management organizations for how you are thinking you will proceed.”

“Because the main point of the book [The Emotion Machine] is that it’s trying to make theories of how thinking works. Our traditional idea is that there is something called ‘thinking’ and that it is contaminated, modulated or affected by emotions. What I am saying is that emotions aren’t separate.”

-Marvin Minsky – interview with Tom Steinert-Threlkeld, ZDNet/Interactive Week (February 25, 2001)

Figure 9: In honor of Marvin Minsky.

References:

1. Minsky M. The emotion machine[J]. New York: Pantheon, 2006, 56.
2. Picard R W, Vyzas E, Healey J. Toward machine emotional intelligence: Analysis of affective physiological state[J]. IEEE transactions on pattern analysis and machine intelligence, 2001, 23(10): 1175-1191.
3. D. Goleman,Emotional Intelligence.New York: Bantam Books,1995.
4. B. Reeves and C. Nass,The Media Equation.Cambridge Univ.Press, Center for the Study of Language and Information, 1996.
5. Elfenbein H A, Marsh A A, Ambady N. Emotional intelligence and the recognition of emotion from facial expressions[J]. The wisdom in feeling: Psychological processes in emotional intelligence, 2002: 37-59.
6. Dagar D, Hudait A, Tripathy H K, et al. Automatic emotion detection model from facial expression[C]//Advanced Communication Control and Computing Technologies (ICACCCT), 2016 International Conference on. IEEE, 2016: 77-85.
7. El Ayadi M, Kamel M S, Karray F. Survey on speech emotion recognition: Features, classification schemes, and databases[J]. Pattern Recognition, 2011, 44(3): 572-587.
8. Shivhare S N, Khethawat S. Emotion detection from text[J]. arXiv preprint arXiv:1205.4944, 2012.
9. Oatley K, Johnson-Laird P N. Cognitive approaches to emotions[J]. Trends in cognitive sciences, 2014, 18(3): 134-140.
10. Emotional Intelligence @ Work: How to make change stick

Followed up questions:

1. As Minsky said, emotion is about thinking. When can we solve the question that how human think to achieve EI? External information (facial expression, speech, text, gesture, body movement, etc.) and internal information (the change of psychology, the activity of cerebral cortex, etc.) seem enough, in the end what else is missing?
2. Current robots are perceived as cold machines, but many scientists hold the opinion that AI might threaten our life. If robots obtain EI in the future, will people be satisfied or scared? If a robot can read your psychological state, will you communicate with it?

Analyst: Oscar Li| Localized by Synced Global Team : Xiang Chen