While viewing the exhibits in the museum, a virtual character will appear in front of you and begin to provide commentary.
KDDI established "au VISION STUDIO" to promote social implementation of advanced technologies such as 5G and XR. In collaboration with KDDI Research Institute and the National Museum of Emerging Science and Innovation, we have started a demonstration experiment on virtual humans using AR.
The demonstration experiment conducted at the National Museum of Emerging Science and Innovation is a virtual exhibition entitled "HYPER LANDSCAPE." You can enjoy an AR exhibition using smart glasses "Nreal light".
The guide for the exhibition is a 3D character (virtual human) “coh”. Cloud rendering using 5G MEC with characters made with CG realized smooth movement even on small terminals such as smartphones.
At the HYPER LANDSCAPE exhibition, you will be given a 5G smartphone and smart glasses at the entrance. When you stand in front of the Earth-shaped display "Geo-Cosmos", which is also the symbol of the museum, coh appears and introduces himself.
Encounter with coh The Geo-Cosmos that you can see with the naked eye that appears at a certain point and talks to youThe exhibition experience is a tour that goes around the aisles surrounding Geo-Cosmos. As you follow the guidance of coh, you can enjoy how the passages and Geo-Cosmos transform into shapes that cannot be experienced in reality. Information such as the status of satellites flying over the earth and the status of immigration restrictions in countries around the world due to the new coronavirus are displayed.
Mr. Shinto Seguchi, manager of the exhibition planning and development section of the Miraikan Business Department, talks about the need to adapt to the changing environment due to the corona crisis. According to Mr. Seguchi, although there are signs of recovery, the number of visitors to museums, including the National Museum of Emerging Science and Innovation, is sluggish due to the corona crisis. At this timing, KDDI had new technologies such as 5G and VPS, so it became a three-party joint effort.
Mr. Seguchi, National Museum of Emerging Science and InnovationMr. Seguchi said, "The content of the current exhibition cannot be changed to suit the viewer.In the future, the content will also change to suit the individual, and the knowledge and information we want to deliver We have to deliver it. I want to change that from now on."
Related articleAn experiment to digitize the exhibition space at the National Museum of Emerging Science and Innovation Begins Event
Mr. Seguchi said, "This demonstration experiment is the first step (for change). Miraikan will change more and more from now on, so I hope you can feel that kind of 'premonition'." Told.
Experiences are being held until March 14 (15:00-17:00). First-come, first-served basis. Also, for those who were unable to visit the museum, there are plans to release an app that allows visitors to experience the exhibition on March 19.
au VISION STUDIO was established with the aim of promoting the social implementation of advanced technologies such as 5G and XR technology. At the same time, we are also considering building an ecosystem to continue the technologies and experiences created in collaboration with external creators and partner companies.
Left = Mr. Kozuki from KDDI Right = Mr. Mizuta from KDDI We will do it ethically," he said.Related articleKDDI launches creative team "au VISION STUDIO" to create new experiences with 5G and XR
Efforts include “unlimited viewing experience”, “zero distance world”, “UX that expands the five senses”, “shopping that is friendly to people and the earth”, etc. coh is one of them “virtual human It is aimed at “everyday life”.
coh enables natural operation even on mobile devices such as smartphones and smart glasses by performing processing on the MEC side instead of the terminal side. We are working on development and business jointly with four partners for social implementation.
"By moving freely, virtual humans can become a new user interface," said Mr. Kozuki. She says it will allow people to have conversations between people. He wants to incorporate non-verbal communication such as gestures and facial expressions into coh. It is said that it can demonstrate its power as a new era UI in the era of smart glasses, which is likely to spread in the future, and in the new normal era where contactlessness is emphasized.
In addition to the aforementioned National Museum of Emerging Science and Innovation, in addition to being an attendant at KDDI ART GALLERY, it has been decided to collaborate with Kanebo's brand "I HOPE" as a single model.
The three-party initiative started in November 2020. "By creating an exhibition experience that utilizes 5G and indoor VPS, we can create a new exhibition style by providing something that was not possible in the past," says KDDI Personal Business Headquarters Service Management Headquarters 5G/xR Service Planning and Development Department Service and Product Planning Mr. Osamu Mizuta, the group leader, said.
In addition, as the impact of the new coronavirus spread during the planning process, it was said that the project proceeded with a view to the future, asking ``what kind of exhibition will be held after the corona?'' This is how "HYPER LANDSCAPE" was born.
In the experience, not only coh appears, but also the content is displayed so that it sticks to the walls of the facility and the handrails of the passages, giving you the experience of being in a virtual space. This was achieved with a VPS (Visual Positioning System). Kenshi Komorita, Group Leader of the Media Recognition Group, Media ICT Division, KDDI Research Institute, said, "Humans know where they are if they know where they are, but VPS can recognize them." It's done by machines," he explains.
In the demonstration, the VPS around Geo-Cosmos was photographed using a spherical camera to create the VPS map. It uses parallax between images to recognize three-dimensional structures. It is said that it recognizes the place by comparing the map and the photograph made in this way.
According to Mr. Komorita, the VPS verification at the National Museum of Emerging Science and Innovation was "a very challenging environment." Due to the nature of VPS, which relies on information from photographs, it is said that it is extremely difficult due to the brightness and darkness of the light, the transmission and reflection of the glass near the handrail, and the same scenery continuing due to the corridor.
KDDI Research Institute created a VPS map using a method that can handle such a harsh environment. Even if the same photo is compared between daytime and nighttime, the existing method has 17 points to compare the features of the object, while the method developed by the company has 229 points.
With these techniques, the average positional error is about 35 cm. The average angular error is now less than 1 degree each, allowing you to enjoy accurate content display.
This time, the original character coh has appeared, but the character can be replaced depending on the situation, and it is said that flexible operation can be expected. In this demonstration experiment, we verified the operation of the VPS and the acceptability of the virtual human. In addition to working with the National Museum of Emerging Science and Innovation, we will consider developing AR technology as a tool in parallel.
Related Links