Sep 04, 2019

Solutions for a Better Future / Feature Story

Products & Solutions

Adroit Visuals That Track an Athlete's Movements - A New Age in Experience Value Engineered by Sports x Technology

With the Olympic and Paralympic Games Tokyo 2020 just a year away, a "1 Year to Go!" ceremony was held on July 24, 2019. It opened with a stunning performance that was creatively engineered and rendered with Panasonic's real-time tracking and projection mapping technology and the company's high luminance projectors. The innovative production gave a captivated audience a taste of what might be expected by "fusing sports and technology." This article delves into how that was done.

Summary of this article:
- The opening performance at the "1 Year to Go!" ceremony for the Olympic Games Tokyo 2020 was engineered with real-time tracking and projection mapping technology using projectors.
- The system was developed by Panasonic, an Official Worldwide Partner of the Olympic Games Tokyo 2020, and opens the door to new performances choreographed on "speed."
- Those involved shared their thoughts on how the interaction between imagery and human motion is accelerating the "fusion of sports and technology."

A spectacular performance opens the ceremony

With the Olympic Games Tokyo 2020 finally just a year away, an absolutely spectacular "1 Year to Go!" ceremony was held on July 24, 2019.

Already before anything was happening, the hall was teeming with enthusiasm. Some 4,000 people between guests and those tied to the event in one way or another packed Hall A of the Tokyo International Forum, waiting anxiously on the edges of their seats for the show to begin. The junior high and high school kids seated in the front rows could be seen larking about posing for commemorative photos. Bearers of the next generation, these boys and girls will experience the Olympic Games Tokyo 2020 during their most impressionable years of their lives. So, what's not to get excited about?

Zap! Suddenly, the stage went dark and suspense filled the air. All breaths held, the audience looked on as the Yoshida Brothers from the Tsugaru region in northern Japan raised the curtain with an energizing duet on the shamisen. Their opening instrumental done, the brothers were swiftly whisked to opposites sides of the stage to make way for a mesh screen that floated in center-front. Then ...

Behind the screen, rhythmic gymnast Honami Tsuboi began dancing across the stage like a nimble fairy to the musical accompaniment of the Yoshida Brothers. The tip of the wand she held in her right hand painted a glowing trail of light that seemed alive as it followed her unbridled fluid motions. The dynamic and overtly "Japanese" imagery was fused into the dance using a technology known as "projection mapping." Very high-definition and 3D, it looked like Tsuboi would jump out of the screen at any moment. All eyes were glued to the stage in awe by this never-seen-before performance. The loud applause at the end told just how thrilled and amazed the audience was.

(Video) Opening Performance at the "1 Year to Go!" Ceremony for the Olympic Games Tokyo 2020

Because it makes perfectly good sense, most people probably thought that Tsuboi "danced beautifully in tune with the images." Yet, it was actually the opposite: the images followed Tsuboi's motions in real-time. The technology that does that was developed by Panasonic, an Official Worldwide Partner of the Olympic and Paralympic Games Tokyo 2020. This article goes behind the scenes - so to speak - to discover how real-time tracking and projection mapping was born and honed into the artistic tool it proved to be that day.

Technology has the power to change sports

This was not the first time that Panasonic showcased its real-time tracking and projection mapping technology and high luminance projectors. Just in relation to the Olympic Games, the new technology had already been announced in March 2019 at the "TOKYO VISION: 500 Days to Go! Night," a series of events planned 500 days out from the Olympic and Paralympic Games Tokyo 2020.

photo: Projection mapping attraction presented to the public as parts of the

Projection mapping attraction presented to the public as parts of the "500 Days to Go!" events

Hiroyuki Kobayashi of the Tokyo Organising Committee of the Olympic and Paralympic Games described the opening performance for the "1 Year to Go!" ceremony as follows.

"As is laid out in our vision of the Olympic and Paralympic Games Tokyo 2020, we want to be the most innovative in the history of the Games and bring about positive change in this world. By that, I mean our goal is to create new values beyond just the sports component, by adding technology and culture into the mix."

"When we thought about how to fuse sports, technology and culture, we decided to try something new using real-time tracking and projection mapping compatible projectors."

photo: Hiroyuki Kobayashi, Director, Communications and Engagement Bureau, The Tokyo Organising Committee of the Olympic and Paralympic Games

Hiroyuki Kobayashi
Director, Communications and Engagement Bureau
The Tokyo Organising Committee of the Olympic and Paralympic Games

Having actually witnessed the performance, Mr. Kobayashi rated it as "powerful on a whole nother level in terms of the imagery, acoustics and movement." The impression he walked away with was that new content had been created by fusing together the distinctively separate elements of the Tsugaru shamisen, rhythmic gymnastics and the latest in imaging technology. He added, "This opening act was designed to energize those watching."

"Technological advances are creating opportunities for people who are not that familiar with sports to appreciate them more. That goes not just for playing but spectating as well. It is also an important role of ours to stir latent interests by conveying more and more the fun and attraction of sports through the Olympic and Paralympic Games Tokyo 2020."

"We see sports as capable of changing the future and technology as capable of changing sports. So, we expect something unimaginable to come from combining the two."

Real-time tracking at less than 2/1000 sec in latency

The system was developed by the Media Entertainment Business Division at Panasonic's Connected Solutions Company. The division makes projectors, flat panels, broadcasting equipment, sound equipment, conference systems and other BtoB solutions.

Kenji Fujiune, who hails from that division, recounted, "We started developing the system in 2015, bearing in mind that it was a very new challenge in the imaging industry and that we would be pushing speeds as far as they could go. This is meant to enhance the user's experience."

photo: Kenji Fujiune, Manager, Agile Development, Business Solutions Center, Media Entertainment Business Division, Connected Solutions Company, Panasonic Corporation

Kenji Fujiune
Manager, Agile Development, Business Solutions Center
Media Entertainment Business Division
Connected Solutions Company
Panasonic Corporation

Projectors have been improved along the tracks of both picture quality and brightness. In terms of picture quality, there are SD, HD, Full HD and 4K products out there, while some high-end commercial projectors have reached 30,000 lm in brightness. Panasonic's focus on "speed," as opposed to the industry's standard approach to development, was because it wanted to go with something different.

Hirotatsu Asai proudly explained, "With projection mapping everywhere these days, people have gotten tired of looking at just images projected on the wall of a building. Real-time tracking and projection mapping integrates performances and imagery into something life-like."

photo: Hirotatsu Asai, Project Manager, Project Promotion Department, Business Solutions Center
Media Entertainment Business Division, Connected Solutions Company, Panasonic Corporation

Hirotatsu Asai
Project Manager, Project Promotion Department, Business Solutions Center
Media Entertainment Business Division
Connected Solutions Company
Panasonic Corporation

The system consists of a projector for projecting the images, an infrared light for sensing and a high-speed camera. The infrared light is radiated on the marker at the tip of the performer's wand and picked up by the camera. The location of the marker in space is then pinpointed within a virtually negligible time lag by feeding the captured image into a processor built into the projector. Based on this positional information, the projector instantly projects the images at 1,920 fps (frames per second). This configuration allows real-time tracking with just a lag of 0.0016 sec from when positional information is detected until the content is projected.

Real-time tracking with a time lag of less than 2/1000 sec from the detection of positional information to content projection is rendered with a high-speed projector that processes information without going through a PC.

photo: Four high-speed projectors were used for the opening performance at the

Four high-speed projectors were used for the opening performance at the "1 Year to Go!" ceremony.

According to Mr. Fujiune, most people who see the performance for the first time are likely to say something like, "The dancer went all out to keep up with the images." Mr. Asai noted, "You cannot detect the time lag with your eyes. Since a typical projector is 60 fps, the smoothness of the images deserves particular mention."

"At the 'TOKYO VISION: 500 Days to Go! Night' event, dancer Kento Mori performed 3 days in a row. But, what struck me was when he said it was a lot of fun to change up his dancing. Since the images track his movements, I gather he slightly changed his routine," added Mr. Fujiune.

(Video) "Ready for 2020" Performance by Kento Mori

Though the same technology as the "TOKYO VISION: 500 Days to Go! Night" event was used for the opening performance of the "1 Year to Go!" ceremony, the screen for this latter was four times larger. And, a total four projectors (including 1 backup) were on-site to enhance the brightness. As Mr. Fujiune put it, "The system blurs the boundaries between the real and virtual worlds. It augments the 3D effect and blending sensation by tracking high-definition, high-brightness images as an added component, whereby enabling a higher level of performance. The opening performance of the '1 Year to Go!' ceremony perfectly demonstrated that power of expression."

Looking to the future, Mr. Asai said, "If further evolved, the system could possibly project images on the walls and floors in someone's home so that they could feel like they are inside the games they play. Moreover, advances in technology may spawn sports that don't yet exist. This system takes us one step closer to realizing those kinds of dreams."

With virtual reality, one immerses him/herself alone because goggles are required, but with real-time tracking and projection mapping, multiple persons can share the "experience." Already, a positive response has been obtained from users who tried using the tracking feature with ping-pong, soccer and other sports. So, the fusion of sports and technology has already begun via the interaction between high-speed tracking and human motions.

Events will pick up even more speed next year and, before you know it, the Olympic Games will be upon us.

The "1 Year to Go!" ceremony was a great success, but it was just one stop along the way to the big show. Next year is the Olympic Year and come April the Olympic Torch Relay will make its way to Japan and the "TOKYO 2020 NIPPON FESTIVAL" will start.

As Mr. Kobayashi explained, "The TOKYO 2020 NIPPON FESTIVAL will be a compilation of all of the participatory events promoted to date. The main purpose is to acquaint people with the Olympic and Paralympic Games Tokyo 2020 by getting more of them to participate. With the Tokyo 2020 Games, we want to get as many people as possible interested in sports."

Official Worldwide Partner Panasonic thinks likewise. As Mr. Fujiune put it, "The Olympic Games are one opportunity for us to continue developing sports and entertainment. We can firmly support those foundations with our technology." Mr. Asai noted, "These will be the first Olympic Games since Panasonic feted its 100th anniversary. On the technology front, we want to make a bigger impact than all of the Games before these."

New events like karate, skateboarding and surfing have been added to the Olympic Games Tokyo 2020. Who knows? Maybe, "new technology-derived sports" will be adopted as new events two or three Games from now. Just the thought thereof conjures up exciting images of the future Olympic Games.

photo:
photo:
photo:

Real-time tracking and projection mapping make the viewer think that the dancer is moving in sync with the images.

A capable creative team to support the content

Moment Factory was in charge of the image content for the opening performance at the "1 Year to Go!" ceremony. Founded in 2001 as a digital art group with a core focus on visual imagery, the company is headquartered in Montreal, Canada and has an office in Shibuya, Tokyo.

Moment Factory pursues an organic "reality" where large groups of people can share the excitement, rather the virtual realm of games or VR. The company has been exchanging and polishing ideas with the development team from Panasonic through a collaborative project that began in 2017. That all came together as the content for the "TOKYO VISION: 500 Days to Go! Night" and "1 Year to Go!" ceremony.

Dominic Audet, Co-Founding Partner, Chief Innovation Officer, Moment Factory

Dominic Audet
Co-Founding Partner, Chief Innovation Officer
Moment Factory

Moment Factory Co-Founder Dominic Audet commented, "The concept behind the content of the opening performance at the '1 Year to Go!' ceremony was to strike a collaboration between tradition and state-of-the-art technology, on a pillar of Panasonic's high-tech. We reached new ground where athletes aren't bound by limitations." Looking back, Mr. Fujiune recounted, "In working with their creators, we discussed all sorts of themes and topics like what everyone wanted to create, what people would enjoy and what would be best as a performance. It was a very motivating experience."

Moment Factory has its sights on pioneering augmented sports in the future. As Mr. Audet explained it, "The big goal is to fuse technology and actual sports like was done with ping-pong and soccer at the 'TOKYO VISION: 500 Days to Go! Night.'" Panasonic's Mr. Asai is looking forward to playing tag more with his cohorts from Moment Factory because of the confidence boost it gives him, as he described it, "In this kind of back and forth collaboration, both sides stimulate each other, which ultimately should lead to new modes of expression."

- Reproduced from the website "Mirai-kotohajime," by courtesy of Nikkei Business Publications, Inc.

# # #

Disclaimer:
We would like to note that Panasonic Newsroom is not a place to address personal Customer Service issues. Even though this is not the forum, Panasonic is always eager to resolve your concerns. Our local customer services contacts can be found at Global Support or you can see our list of Social Media Accounts to find the right channel for your queries and concerns.

Related Videos

The content in this website is accurate at the time of publication but may be subject to change without notice.
Please note therefore that these documents may not always contain the most up-to-date information.
Please note that German, French and Chinese versions are machine translations, so the quality and accuracy may vary.

Articles within series