The nursing care industry has a higher fluidity of human resources than in other industries and suffers from a shortage of human resources.In order for nursing care staff to provide optimal care for residents, it is necessary to always know how each resident and what care is needed.Nevertheless, the number of staff members tends to increase, and there is a limit to providing optimal services individually due to the shortage of human resources.
In order to solve the social issues in the nursing care industry, KDDI is developing a new nursing care work support system that utilizes AR technology along with the Social Welfare Corporation Zenkoikai, which operates nursing care facilities.
The KDDI Research Institute and the Social Welfare Corporation Zenkoikai developed a “hands -free nursing care support system” that displays residents' nursing care related information through AR glasses and reads out by voice for the purpose of realizing detailed care. did.
The "Hands -Free Nursing Care Support System" combines facial recognition technology and speech synthesis technology developed by the KDDI Research Institute with the smart nursing care platform "SCOP" developed by Zenkokai, and lenses with AR glasses with cameras.It is displayed in and read out by voice.
"SCOP" is centrally managed healthcare data such as the body temperature and blood pressure of the resident, and recognizes the resident's face through AR glasses based on photos taken from the front or diagonal in advance.Read the resident's healthcare data from "SCOP" through the Internet line and projected on the AR glasses lens.At the same time, read out the name of the resident with voice.By the way, the content of the audio can be set in various ways depending on the situation, but this time it is set to read the name.
Originally, in nursing homes operated by Zenkokai, staff could use smartphones and tablets to examine residents from "SCOP".However, in the nursing care site, there are many tasks using both hands, such as bathing assistance, and in some cases, such as when the residents have stumbled, it is necessary to have a sudden response.I was lacking.
In that regard, in the case of the "hands -free nursing care work support system", there is no wiring such as an adapter or LAN cable, and only the AR glasses are installed, so both hands can be used freely.There is no worry that staff and residents will be caught by the wiring, as well as the response to the residents.Being able to use both hands and no wiring is very important in the nursing care field.
Let's actually look at the demonstration.The right of the photo is the staff, and the AR glasses and bone conduction earphones are set.The left is the setting of the resident.
If you look at the resident's face with AR glasses, the resident's healthcare data will be displayed in the right eye lens section.
The displayed data is "Name", "Need for nursing care", "hospital / hospital room", "blood pressure", "body temperature", "state of meals", "Moving", "presence or absence of excretion".In addition, the name of the resident flows by voice from the earphone.
For example, if you know the blood pressure of the resident, you will judge that you will have a high blood pressure today, so let's shorten the bathing time.You can also encourage you.In addition, even if the staff, such as new residents, has not yet grasped detailed information, it will be easier for staff to take appropriate response because the information of the residents and the physical condition on the day.
From October 13, 2020 to December 18, 2020, a demonstration experiment was conducted at the Hands -Free Nursing Care Support System at a nursing home at Zenkokai.
As a result of the use of six staff members, "I am grateful that there is no need to search for information with a smartphone."There were voices such as.The point of concern is that the AR glasses were heavy and the appearance looked like sunglasses.
It is easier for staff to concentrate on nursing care by quickly obtaining information from residents, which leads to good services for residents, but there is still room for improvement.
The “Hands -free Nursing Care Support System” has a mechanism to display the resident's information immediately by adopting a new face recognition technology.We asked Tomohiro Ohara, the Innovation Center Innovation Cooperation Group, KDDI Research Institute Institute for Innovation Center, and Multi -Modal Commodal Communication Group, Kure Kure Fengaki.
左／KDDI総合研究所 イノベーションセンター・マルチモーダルコミュニケーショングループ 呉 剣明 右／イノベーションセンター・イノベーション協創グループ 小原朋広"The KDDI Research Institute is conducting research using robots and sensors, but in listening to the voices of various industries in the field of various industries, the nursing care site uses sensors to improve nursing care.I knew. And while I learned about the Zenkokai, which uses advanced technology, and exchanging opinions, I found that there was a need to "want to be in contact with a new resident like an acquaintance."I thought that I could use the face recognition technology of the KDDI Research Institute to respond to the request. "(Tomohiro Ohara).
The "Hands -Free Nursing Care Support System" has newly adopted two -stage face authentication technology.It was Kure Fengaki who developed it.
"In the general face recognition method, AI searches for data (face photos, etc.) that match only the registered image. If you register a diagonal face image, you will find it, but then you will recognize your face.Even if you try to register many images, such as right, left, right side, left side, you need to register. Based on a small amount of information, it is very efficient to look for all huge data and match.It may take some time to recognize or do not respond.
Therefore, the hands -free nursing care work support system adopts two -stage face recognition.In the first stage, the area of the face is detected, and the correction process is performed when the direction of the diagonal or the like is performed in the second stage, and the face is recognized.As a result, you get an ID that represents someone, access the SCOP server based on that ID and get the resident data.By conducting face recognition in two stages, we realized a high recognition accuracy regardless of the direction of the face without increasing the calculation amount "(Kure Fengaki).
The important thing is "lighter" such as AI movements and data.
"When you actually communicate with the resident, you need to know the data of the resident immediately, and the speed is life. In the" Hands -Free Nursing Care Support System ", the residents are recognized with AR glasses, and the Internet line.Through the resident's healthcare data, access the server that manages the resident, projects the caregiver information on the AR glasses and instantly reads the audio.
In order to search for valid information from a huge amount of data in a short time, it was important to reduce the operation and data of AI.For example, the AI learns the data with a small feature when removing a face photo, etc., and 10,000 face photo data in 1 second, 100 people are 0..You can identify it in 1 second.In addition, in speech synthesis technology, we adopt a Japanese audio reading technology that works lightly with a small memory "(Kure Kenmei).
What kind of field will the technology cultivated in the development of this “Hands -Free Nursing Care Support System” will be used in the future?
"We believe that the technology of the" Hands -free Nursing Care Support System "can be used not only in nursing care but also in another field. For example, in nursery schools, the information of each child and the body temperature measured in the morning can be understood immediately.It may be more efficient to share information and understand the health condition of children in between. Hands -free, which can be used by both hands, is effective as children can hug or fall.I think it can be used to understand the names of parents. "
The "Hands -free Nursing Care Support System" was born from the needs of "even new residents want to be treated like an acquaintance."It is likely that the technology can be applied not only in nursing care but also in other fields.KDDI will continue to utilize the power of communication, with partners, to conduct research that will solve social issues solving and aim for sustainable growth and development of society.