CEDEC 2019, Japan's largest conference for computer entertainment developers, was held from September 4th to 6th, 2019.
Tetsuya Mizuguchi, who is known for "Rez Infinite" and "TETRIS EFFECT", gave a keynote speech titled "Going beyond the game - Towards the creation of new experiences", which was held at the opening of the first day. is on stage. In this article, we will report on the pattern of this lecture.
In 2014, he started Enhance in the United States, and in 2016, he launched "Rez Infinite," and in 2018, "Rez Infinite." Mr. Mizuguchi who announced "TETRIS EFFECT". His origins in game production date back to 1990 when he joined Sega. In 1990, he was shocked by the experience-based housing “R-360” announced by Sega, and joined Sega as a new graduate.
(Sega R-360, a sensory arcade game cabinet that rotates 360 degrees. Seeing this overwhelming experience that transcends language, he felt the potential of the game industry.)
Mr.was also strongly impressed by the VR system announced by NASA in 1988, and he seems to have had a high interest in advanced technologies like VR from that time.
The Virtual Interface Environment Workstation (VIEW), 1990 (Partnership with VPL Research, Inc.)
The 1980s and 1990s was the so-called first VR boom. It was the year that VR technology, which was born in the laboratory in the 1960s, began to enter the industry. At that time, the term “virtual reality” was not yet familiar. It seems that he emphasized VR even at the job interview, but it seems that the executives at the time did not understand it very much.
Reference articles on the history of VR:
[CEDEC2016] Future Prospects Unraveled from the History of VR | Mogura VR
MoguraVR
How has VR changed in 30 years? ``VR Birthday Festival'' report that looks back on history and envisions the future | Mogura VR
MoguraVR
In the midst of such a VR boom, the first job that Mr. Mizuguchi did "without permission" in his first year at Sega was the prototype development of an AR system.
(The device at the time was made by combining a half-mirror and Styrofoam with the portable game machine "Game Gear". Because 3DCG technology was not advanced at the time, it was a system that superimposed pixel art. It is reminiscent of the world's first VR/AR headset invented by Ivan Sutherland in the 1960s.)
"From that time on, I was a believer in VR. Or rather, I always wanted to create an experience beyond that," said Mizuguchi. However, when he brought this AR system to Sega's board meeting, he said, 'It's interesting, but it's still early,' he said.
After that, he worked on the research and development of VR arcade games for about two years. By 1994, technology for handling 3DCG had gradually developed, and it became possible to apply CG to textures in real time. At this time, Mr. Mizuguchi was involved in the racing game "Sega Rally Championship".
("Sega Rally Championship". Later, it was also exhibited in conjunction with the actual car (right photo))
By the early 2000s, Mr. Mizuguchi had begun developing console games for Dreamcast and PlayStation.
The works that I worked on there are conspicuously related to music. He said, "I have always loved music, and if I hadn't developed games, I would have become a music video director. The idea of the fusion of music and game experience has been inherited in recent VR works without change.
("Space Channel 5". A music game with elements of shooting and action, where the story progresses in a musical style)
("Rez" (announced in 2001), which can be said to be the origin of the PSVR version of "Rez Infinite". We also developed an additional vibrator in pursuit of a rich tactile experience that goes beyond the vibration of the controller (right photo). Arcade. You can get a glimpse of Mr. Mizuguchi's background, who came from the game.)
From around this time, Mr. Mizuguchi was already pursuing experiences that combine sight, hearing, and even touch through the fusion of music and games. However, while the space he created and the experience he wanted to present had a three-dimensional expanse, the media (display hardware) of the time could only convey a two-dimensional experience. It seems that he also felt a dilemma. This hunger for expression led to "Rez Infinite" later.
On the other hand, in 2003 Sony announced the PlayStation Portable (PSP). Also known as an "interactive walkman," this hardware was a portable game console with a headphone jack, which was rare at the time.
At the same time, in 2003, Mr. Mizuguchi went independent and established Q Entertainment. There, he will develop Lumines, an experience that fuses music and games for the PSP. At the time, Mr. Mizuguchi was moved by the fact that he was able to create a music game experience that could be carried anywhere.
Furthermore, in 2011, Mr. Mizuguchi released "Child of Eden," which combines a sensor "Kinect" that can sense the movement of the human body.
https://www.youtube.com/watch?v=RzoxiZeGGO8
Mr. Mizuguchi has always created challenging works by combining the two elements of music and games, but after the release of "Child of Eden" he said, "I'm tired." .
No matter how hard you try to fully mobilize advanced devices such as Kinect, the media that conveys the experience is only a narrow frame of a square flat monitor. Furthermore, there is no tactile feedback, and it is far from the ideal experience that is three-dimensional and full of five-sensory information.
"I felt that no matter how hard I tried, I couldn't create a new experience," says Mr. Mizuguchi. From there, he took a break from game development for about three years.
Oculus Rift appeared on Kickstarter, a US crowdfunding platform, in 2012. From there, inexpensive and high-quality VR headsets such as HTC VIVE and PlayStation VR (Project Morpheus at that time) were announced one after another, and in 2016, those products were put on the market all at once.
Mr. Mizuguchi, who quickly sensed the wave of the "second VR boom", founded Enhance in the United States in 2014, saying, "A device that can realize what I've always wanted to do has finally appeared." In conjunction with the release of PlayStation VR in 2016, we will announce "Rez Infinite", which is an improved experience for PlayStation VR.
https://www.youtube.com/watch?v=1G8kzgCpdd0
(Trailer of new stage "Area X" made for VR experience)
As a notable point of the experience, he mentioned the keyword "Synesthesia" many times during the lecture.
Synesthesia literally means the phenomenon of simultaneously perceiving other sensory stimuli B and C, which are normally not felt, in response to a certain sensory stimulus A. For example, it has been reported that when we see letters, we perceive colors, when we hear sounds, we perceive specific colors, and when we perceive shapes.
Mr. Mizuguchi, however, describes a multimodal experience in which the visual, auditory, and tactile senses are linked and integrated as a “synesthetic experience.” It is not just a random display of light and sound, but rather a convincing integration of light, sound, and vibration, all brought together as a single experience.
("Rez Infinite" for PlayStation VR)
Related articles:
![The first major revolution in 600 years is happening— — Tetsuya Mizuguchi talks about the future of games and “experiences” [CEDEC 2019] | Mogura VR](https://website-google-hk.oss-cn-hongkong.aliyuncs.com/drawing/article_results_9/2022/3/23/95a43e28511a1d74302580f34e1ee528_1.jpeg "The Great Revolution for the first time in 600 years — Tetsuya Mizuguchi talks about the future of games and")
[PSVR] Tetsuya Mizuguchi opens up a new frontier for "only in VR" "Rez Infinite" new stage Area X | Mogura VR
MoguraVR
Of course, even in video games so far, visual (television), auditory (stereo speakers), and tactile (controller vibrations, etc.) should have all been integrated as a production experience. For example, when an enemy hits you, your controller may shake with the painful sound of being hit, and your monitor may glow with a reddish effect to indicate a warning.
However, in VR, the field of view is wide open, and the sound with three-dimensional localization surrounds the surroundings. The persuasive power of these experiences and the magnitude of the emotions that such experiences create are overwhelmingly greater than when displays were flat.
After experiencing "Rez Infinite", everyone was at a loss for words, or excited, or could only cry. Such a scene that could not be realized in video games so far is probably due to the overwhelming "synesthesia" of the experience that the player gets.
"Rez Infinite" won the "Best VR Game" newly established in 2016 at the US game award "The Game Awards". Mr. Mizuguchi responded, "This might be the happiest moment of my life. I was able to wipe away the frustration that I felt when I was told 'It's still too early' when I made 'Rez' a long time ago. I'm happy to prove that the experience I wanted to express in 2001 with 'Rez' was still true, timeless and essential."
After releasing "Rez Infinite", Mr. Mizuguchi's next work was "Tetris".
https://www.youtube.com/watch?v=Mr8fVT_Ds4Q
("TETRIS EFFECT" for PSVR released in 2018. Pre-production started around 2013, and after that, it seems that it was continued for a long time in parallel with "Rez Infinite")
Tetris is a game that is already "finished" as everyone in the world knows. how to redesign it? From an engineer's point of view, it seems that there is nothing more to do, but from an artist's perspective, it makes me want to think about the question, 'How can I make people cry with Tetris?'
Even after two years of pre-production, the development team didn't get bored of the game.
Related articles:
[PSVR] Tetsuya Mizuguchi's ``new comfort'' in Tetris that can be played in VR | Mogura VR
MoguraVR
When I told people outside the company that I was going to develop content based on Tetris, they always didn't understand at first, he says. However, after showing concept art and demo videos, he was convinced.
Mr. Mizuguchi pointed out the challenges that the game industry will face in the future, saying, "It's getting harder to explain the fun of experiences in the same way as before, such as words."
Mr.
(Tactile suit "Synastasia Suit" corresponding to "Rez Infinite". The vibration elements placed throughout the body vibrate in conjunction with the visual and auditory experience of "Rez Infinite", making it even richer " achieve a synesthesia-like experience)
Related articles:
[Experience report] "Rez Infinite" evolves with Synesthesia suit 2.0X Particles of light penetrate the body! | Mogura VR
MoguraVR
PSVR "Rez Infinite" experience for 100 people at the same time! Experimental Event 2nd | Mogura VR
MoguraVR
("Synesthesia X1 – 2.44", a chair with 2 sound speakers and 44 vibrating elements. It was created in collaboration with Synesthesia Lab (*) and sound artist evala. , can only be described as an experiential work,” explains Mizuguchi.)
* Synesthesia Lab is an alliance-type laboratory established to pursue experiments and research on synesthesia and sensory complex experiences, as well as the design of the associated experiences. Enhance, Rhizomatiks, Resonair, Embodied Media (Keio Media Design), and other artists, developers, and companies are collaborating according to the theme.
Looking back at history, we can see that until now, the "media" on which content has been posted has been rich in information, including the five senses, such as television (mainly visual and auditory) and records (mainly auditory). It has developed in a way that specializes only in a specific part of the senses.
However, in "Synesthesia Suit" and "Synesthesia X1 – 2.44", for example, it is possible to feel the sound not only with the ears but also with the body. Mizuguchi says that through this, he realized that there was a lot of missing information that could not be expressed by the method of separating individual senses.
https://photos.app.goo.gl/1Tfy5J81dmRNRSoF8 (“Synesthesia X1 – 2.44” exhibited at the technology art event “Media Ambition Tokyo 2019”)
As an example of the problem of integration of modalities (senses), Mr. Mizuguchi gave ``the feeling when coming up with an idea''. When people come up with an idea, they get mixed up with various modalities in a complicated way and get a sense of "messiness". For example, people use media such as words and pictures to describe their flashing sensations, such as ``My head is clear,'' ``My heart is pounding with excitement,'' and ``Itching to realize an idea.'' Become. However, they cannot explain 100% of the "messy", and they always cut out a part, that is, lack of information.
VR has the potential to solve this problem. An experiential media that excels in naturally integrating modalities such as visual, auditory, and tactile sensations, just like in reality. Mr. Mizuguchi says that as the resolution and interactivity of the experience increases in the future, the excitement that can be obtained will continue to grow.
Mr. Mizuguchi talked about the future of games at the end of his lecture.
5G, Apple Arcade, Google Stadia, Microsoft Scorpio, etc. have appeared, and new devices such as cloud technology and PlayStation 5 are also expected today. Mr. Mizuguchi is particularly looking forward to AR/MR technology centered on glass-type devices. He expects a future in which glass-type devices, which are lighter and have higher resolution, will permeate our lives while being connected to other technologies such as IoT and AI.
https://www.youtube.com/watch?v=AQEizp-VrVU
(“Minecraft Earth,” which is set in the real world, is currently in beta testing for smartphones.)
https://www.youtube.com/watch?v=YeopYh0IrOU
(Title "Angry Birds FPS: First Person Slingshot" for Magic Leap One, an MR device)
With current VR technology, it is possible to intuitively operate an avatar in the same way as a real body. is still higher in the real environment. AR/MR, which fuses the virtual with the real environment, makes it easier to create human-to-human communication than VR, and this will be important for the future of games.
Following this, Mr. Mizuguchi made several predictions under the title of "Things to happen slowly from now on."
First, the transition from the information age to the experience age. The background to this was the possibility of real-time transmission of experiences, the end of the two-dimensional + square frame era, and the fusion of the real and the virtual.
"Transmission of experiences" means that when conveying one's own experience to the other party, it is a live experience that contains rich information such as five senses without using "media that cuts out a part of the experience" such as words and images. It means being able to share it. As a pioneer, we may be able to share tactile sensations as data in the future, just as there are SNS and video posting sites for sharing visual and auditory information.
Related articles:
The past and future of "tactile" design, which is also important in VR [CEDEC2018] | Mogura VR
MoguraVR
The era of experience, that is, the era of spatial experience design and spatial computing, will bring about an increase in "synesthetic" experiences, says Mizuguchi. In such an era, it is conceivable that the boundaries between the first and third person, the active and the passive will melt, and experiences that are a mixture of movies and games will emerge.
Furthermore, it is predicted that the evolution of visual resolution presented by devices will reach a critical point in the next 15 years. Since it is extremely difficult for the human eye to distinguish resolutions of 8K or higher, he says, "the quality of the experience should advance from there."
Mr. Mizuguchi said that the changes that will occur from here on will be the first major revolution in 600 years since the invention of the printing press. What happens next in the game? In the age of three-dimensional, spatial, synesthetic experiences, why not think about what we can do, not only with games themselves, but also with something new that starts with games, says Mr. Mizuguchi. I finished my lecture.