With the expansion of the new colon virus infection, the mask wearing is normal, and the face authentication of the smartphone cannot be used, it is difficult to read the other person's expression during the conversation, or it feels inconvenient.
A new technology adapted to such a new daily life has been developed by the KDDI Research Institute.It is a "expression recognition AI technology" that can recognize facial expressions even when wearing a mask.
In the first place, facial expression AI technology is a technology that uses AI to recognize, analyze facial expressions, and estimate the state of the person.
KDDIの表情認識AI技術In familiar places, it has been practical as an automatic shooting function by digital camera smiling detection, and many people may feel the convenience.In addition, in the advertising and television industries, it is being used in various scenes, such as audience reaction surveys, and corporate meetings, such as analyzing discussions of discussions.
However, most of the people's faces have been covered by the daily masking of the corona.This has stopped the conventional method.
The KDDI Research Institute has developed a technique called "Facial Area -adapted expression AI" to solve issues facing the conventional expression AI technology due to Corona.
What is the technology that can recognize facial expressions even when wearing a mask?I asked the person in charge of the KDDI Research Institute developed.
KDDI総合研究所の研究員。左から、楊 博、呉 剣明、服部 元―― Can you tell me the technical mechanism that enables facial expressions even if you wear a mask?
Yang "If you wear a mask, about 70 % of the face area will be covered. In this state, the conventional expression recognition AI technology cannot be fully recognized. Lips, mouth angles, cheeks, etc. below the nose.Because movement is an important factor in recognizing facial expressions. This is the same as when humans detect the other person's expression. "
What I thought about was an analysis after dividing the "area where the face is exposed" and the "area covered with a mask" separately.
「顔露出領域」と「マスク着用領域」を別々に分析して表情を認識KDDI総合研究所 イノベーションセンター マルチモーダルコミュニケーショングループ 研究主査 楊 博Yang: In the area where the face is exposed, the area where the muscles are exposed, such as the nose and cheeks, in addition to the surrounding eyes, such as wrinkles between the eyebrows that are likely to appear in the expression, are used as much as possible.In the area in which the muscles are changed when the expression changes, the muscles move faintly, and the mask itself occurs, so the characteristics are also learned and quantified.By introducing, it has become possible to recognize and analyze the expression of people with high accuracy even when wearing a mask. "
――Why did you have any particular difficulties in developing this technology?
Kure "There are various shapes of masks in a bite, and some have colors and patterns. There are more than 50 types of masks, and hundreds of masks wearing images of AI.Incorporated into the data. At the beginning of the development of this technology, the accuracy of recognition was about 60 %, but the expansion of learning data increased to more than 90 %. "
KDDI総合研究所 イノベーションセンター マルチモーダルコミュニケーショングループ 研究マネージャー 呉 剣明Can you really recognize your expression even when you wear a mask?I actually tried it.
First, test without wearing a mask.The top of the screen is a video of the built -in camera of the laptop PC, and the detected face is shown in red.The lower left of the screen indicates whether the mask is worn, and the graph at the bottom right of the screen represents the state of the person's emotion.The graph expresses three emotions: "POSITIVE", which corresponds to a smile, "Negative", which is a crying face and an angry face, and "Neutral", which represents expressionlessness and normal mind, and the numbers are total.1.00 is the maximum value.In the photo above, the presence or absence of the mask is "No Mask (no mask)", and the graph that expresses emotion is "neutral"..00 is displayed.
Let's put on a mask and try it again.
Then, at the bottom left of the screen, "Masked" was displayed.The graph that expresses emotions remains "neutral".
Next, I wrinkled between the eyebrows and made an angry expression.
Then, the display of the graph that represents emotions changes instantly, and "negative" is 1.To 00.It was confirmed that the expression was recognized even when wearing a mask.
Next, I made a smile.It's hard to understand in the picture, but the under the mask is a full smile.
Then, the graph that expresses emotion is "positive" 1.To 00.At first glance, it seems that it is not very different from the first "neutral", but AI judged "positive".
In addition, there may be multiple graphs that express emotions.
For example, here.The graph "Positive" that expresses emotions is 0.48, "Neutral" 0.52.It is a subtle expression that can be taken as "positive" or "neutral".
Is it recognized properly except for general white non -woven masks?Or what if you wear glasses?I actually tried it.
Black masks, pattern masks, and glasses were recognized without any problems.By incorporating many masks worn images into AI learning data, it seems that it can support various scenes that can be assumed on a daily basis.
The expression recognition AI technology of the KDDI Research Institute that enables recognition of facial expressions even when wearing a mask.What kind of services are expected through this technology?I asked the person in charge.
――What kind of scenes do you use in what kind of scenes that can be recognized even when wearing a mask?
Hattori "We are aiming to realize new services in various situations, such as companies, educational institutions, and public facilities. For example, in zoos, aquariums, amusement parks, event venues, etc., it is used as a tool to confirm the reaction of visitors.It is expected. "
KDDI総合研究所 イノベーションセンター マルチモーダルコミュニケーショングループ グループリーダー 服部 元Yang: "I want to use it to verify whether I can serve customers with a smile."If the expression is positive, the voice will be brighter, and a comfortable response will be possible.As a result, it leads to improved user satisfaction.By the way, since 2018, the expression recognition AI technology has been introduced to the call center since 2018, and it has been useful to raise the awareness of the staff when responding to the telephone."
※写真はイメージですKure: "Recently, telework has increased due to the catastrophe, but companies want to measure the concentration of work at home and remote offices, so that the reaction of participants wearing a mask at the online conference can be understood.We have the needs such as "I want to do it, and we have inquiries."
Kure: "Facial expression AI also has a group function, so you can check the changes in everyday emotions. If you take statistics, for example, there are many" positive "times on Friday.There are many holidays the next day, and my feelings will be positive (laughs)."
Hattori "KDDI's expression recognition AI technology has been attracting a lot of views in academic societies and has been attracting attention worldwide in recent years.I think. I would like to continue conducting demonstration experiments and conducting research for further practical use. "
KDDI's initiatives related to the new colon virus are not limited to the expression recognition AI technology when wearing a mask.For example, visualization of behavior patterns of people in Tokyo based on the location information collected from au smartphones, and 80 % of the living behavior can fit in a small community (as of November 2020).As a result of estimating with AI along with the infection, the fact that "stay -in community" is important to keep living behavior within a small community to prevent the spread of infection and the economy.I found that there were characteristics in high places.Through research and development of advanced technology, KDDI will contribute to the development of a new relationship between humans and technology, the realization of a society that can live with peace of mind and sustainable development.
文：TIME&SPACE編集部
* Information on KDDI's products and services published is as of the date of publication.Please note that information such as product / service fee, service content, specifications, etc. may be changed without notice.