Mar 15, 2019

Solutions for a Better Future / Feature Story

Products & Solutions

What Is the AI Supporting the First Step? The Reasons for the Birth of the Passenger Boarding Bridge Automated Docking System (Part 2 of 2)

While inbound demand continues to increase each year in Japan, the airports, which serve as the gateways to the skies, are suffering from a lack of human resources. One solution is the development of "Smart Airports" using AI and IoT. ShinMaywa Industries has completed an automated docking system for PBB that connect airport buildings with the aircraft, and successfully put it into practical use for the first time in the world at Tokushima Awaodori Airport. Panasonic's AI played a vital role in achieving the automation of the PBB docking system. How was this innovative mechanism, which will also be introduced at Narita International Airport and Singapore Changi Airport, created? The people involved were interviewed about their thoughts on this project.

A summary of this article:
- There is currently a shortage of airport workers in Japan due to the ageing population and a high turnover rate.
- The automated docking system for the Passenger Boarding Bridge (PBB) developed by ShinMaywa Industries has been equipped with Panasonic's AI technology.
- Panasonic has developed image recognition and image solutions that have been acquired over time.

The AI-mounted PBB is the "first step" towards the future of Smart Airports

The people behind the scenes who support airport operations are broadly divided into the ground staff who work at the check-in counter and carry out gate tasks, and the ground handling staff who work outdoors near the aircraft. In order to ensure the safety and peace of mind of scheduled flights and passengers, smooth-flowing operation is required in the work carried out by these airport staff. The various services received at the airport, which are usually taken for granted, are brought about by the sincere attitude of these on-site professionals.

The AI-equipped PBB that will be introduced here is the "first step" towards the future of Smart Airports. With the touch of a button, the PBB, which weighs close to 30 tons, automatically approaches to within 10 cm of the aircraft door. The standardization of this task has been successfully achieved by automating this operation, which is conventionally carried out while making fine adjustments by experienced staff members. This has made it possible for less experienced staff members to bring the PBB closer to the aircraft door in the same position and amount of time. This is truly a solution that combines labor saving and efficiency.

photo: A Passenger Boarding Bridge equipped with an automated docking system at Tokushima Awaodori Airport

A Passenger Boarding Bridge equipped with an automated docking system (Tokushima Awaodori Airport)

Leading PBB manufacturer ShinMaywa Industries and Panasonic Solution Technologies (Panasonic) were the key players in the development of this automated docking system. ShinMaywa Industries' conventional PBBs possess a patented technology (preset run) that is superior to their competitors by enabling them to automatically approach up to within 1 meter of the aircraft. Panasonic's PLC (control device) is used to control the automatic operation, laying the groundwork that connects the cooperative relationship between the companies for this latest challenge.

The PBB serves as a memorable bridge for passengers

"It all began after we were consulted by an airline company to help engage in the improved efficiency and automation of airports. We had already delivered PBBs to airports in various regions, so we thought about whether we could contribute to improve efficiency by further evolving them," recalled Akihiko Kitano of ShinMaywa Industries.

photo: Akihiko Kitano, Chief Engineer, Systems Department, Parking System Division, ShinMaywa Industries, Ltd.

Akihiko Kitano, Chief Engineer, Systems Department, Parking System Division, ShinMaywa Industries, Ltd.

The preset Drive function, an original technology of ShinMaywa Industries, which automatically positions the PBB within 1 meter of the aircraft door, had already been useful in reducing the work time from before. The question of how to automatically dock the PBB still remained for the development of a fully automated system. It was then that the president of ShinMaywa Industries offered a proposal to "further evolve the PBBs by equipping them with image recognition technology." Experiments with image recognition using AI began in 2016. Yoshihiro Sonoda, who was the company's on-site supervisor at the time, had the following to say.

"We installed cameras in the airport and continued to record image data near the aircraft door from morning to night. We then handed over the recorded data to Panasonic, where it was analyzed using deep learning. They then returned the analyzed data, and we repeated automated docking experiments at night using parked aircrafts and PBBs." (Mr. Sonoda)

In order to increase the accuracy of the image recognition, a massive amount of actual image data recorded on site is required. Therefore, all types of image data must be prepared, including unique situations such as bad weather and exposure to strong afternoon sunlight. The AI technology used this time requires people to be teachers to develop it, and cannot learn on its own. Learning by the use of human hands made it possible to accurately recognize the position of the aircraft door when the airplane was not parked precisely in a predetermined location, or when several airplanes had different appearances because they were different models or had various painting designs.

photo: Yoshihiro Sonoda, Senior Staff Engineer, System Design Specialist, Airport Equipment Section, Systems Department, Parking System Division, ShinMaywa Industries, Ltd.

Yoshihiro Sonoda, Senior Staff Engineer, System Design Specialist, Airport Equipment Section, Systems Department, Parking System Division, ShinMaywa Industries, Ltd.

As a result, it led to the full operation of the AI-equipped PBBs at Tokushima Awaodori Airport starting in 2018. There are no other examples of the actual operation of PBBs to automatically come within 10 cm of the aircraft door. "Both our young and veteran staff members are delighted, because the final adjustments that require extreme precision were no longer necessary," added Mr. Sonoda. This achievement was recognized, and in 2019, it was decided that the system would be introduced at Narita International Airport and Singapore Changi Airport. The process of collecting the image data has already been initiated at both airports.

"Detailed adjustments require the "right touch" added with human knowledge, but the engineers we have in charge have the flexibility to meet these requirements, and it was a success in the sense that they were very positive about it. The timing just so happened to overlap with the case study for implementing AI, so in terms of timing it was perfect," commented Mr. Sonoda, regarding Panasonic.

photo: Yoshihiro Sonoda, Senior Staff Engineer, System Design Specialist, Airport Equipment Section, Systems Department, Parking System Division, ShinMaywa Industries, Ltd.

Speaking about the vision for the future, Mr. Kitano added, "We want to use this technology and ultimately develop it into a fully automated docking system. There are still issues regarding safety, but we are continuing to hold discussions with authorities and aim to complete development by 2020." Mr. Kitano described their future vision, saying, "There is also a plan to conduct remote monitoring of the PBB to forecast malfunctions." Mr. Sonoda explained, "This is the start of the introduction of IoT in airports. The installation of the system at Narita International Airport will lay the foundations for future development. Airports with automated facilities that link and work together are the vision of the future for the airport industry." Mr. Sonoda expressed it as follows.

photo: Akihiko Kitano, Chief Engineer, Systems Department, Parking System Division, ShinMaywa Industries, Ltd.

"For airplane passengers, the PBB is the first and last place they walk through. By equipping it with AI, that first step has become much smoother. In that sense, the PBB is a product with dreams, a bridge of memories, and I am very proud to have the opportunity to be involved in this work." (Mr. Sonoda)

The spread of AI systems in everyday life is the ideal

Senior engineer Shigeru Shimazaki was in charge of AI at Panasonic. Since he joined the company, Mr. Shimazaki has been involved in the research and development of OCR (optical character recognition), and is currently engaged in the development of AI platforms using his acquired knowledge.

photo: Shigeru Shimazaki, Senior Engineer, AI Platform Division, AI & Analytics Department, Panasonic Solution Technologies Co., Ltd.

Shigeru Shimazaki, Senior Engineer, AI Platform Division, AI & Analytics Department, Panasonic Solution Technologies Co., Ltd.

Overseeing the development of this project was very rewarding, but "The key point is not whether simply having AI will resolve everything, but rather proposing how to use what type of AI to reach the goal that our customers expect. There are things that even AI cannot do, and it is certainly not perfect, so taking the right measures is very important. This latest PBB case is the same, and there were many areas that we were unsure of until actually testing it out, so we proceeded by getting together and holding discussions." (Mr. Shimazaki) In other words, both companies sharing what they accomplished using AI was very important.

The ability to work flexibly according to the customer's case is a result of Panasonic's extensive experience in industrial and infrastructure systems that has been acquired over the years. Taking advantage of this background, Panasonic has defined new solution groups with ICT infrastructure technology and IoT technology at their core as a priority area of the next-generation business. Of them, the data analysis and AI business is achieving improved results as a promising sector.

"There are members at the company who have been involved in image analysis at the factory production lines, so it is possible to absorb their industry-related knowledge. In addition, image and video solutions are central businesses of the Panasonic Group, so there are many areas where we can cooperate across departments. In this case, figuring out how to develop software as user-friendly as possible was very important. I believe that the ability to totally complete everything is one of Panasonic's strengths." (Mr. Shimazaki)

Outline of the PBB Automated docking system (AI docking system)

Of course, this technology is a long-term solution with an eye on horizontal expansion. "From the first stage of development, both companies shared the same idea of expanding this solution. ShinMaywa Industries insists on extremely high precision, so we worked hard to meet their expectations and were able to win their trust. Based on the results achieved at Tokushima Awaodori Airport, ShinMaywa Industries' automated docking system was selected to be installed at Narita International Airport and Singapore Changi Airport, and I realized that Panasonic's technologies were recognized globally." (Mr. Shimazaki)

"It would be ideal for the AI system that we developed to spread to peoples' everyday lives," added Mr. Shimazaki. This case study brilliantly proved the hypothesis that AI is not something that will take jobs from humans, but rather a technology for further drawing out their performance.

*This PBB can only be provided in certain countries and regions due to the required construction.

- Reproduced from the website "Mirai-kotohajime", by courtesy of Nikkei Business Publications, Inc.

# # #

Disclaimer:
We would like to note that Panasonic Newsroom is not a place to address personal Customer Service issues. Even though this is not the forum, Panasonic is always eager to resolve your concerns. Our local customer services contacts can be found at Global Support or you can see our list of Social Media Accounts to find the right channel for your queries and concerns.

The content in this website is accurate at the time of publication but may be subject to change without notice.
Please note therefore that these documents may not always contain the most up-to-date information.
Please note that German, French and Chinese versions are machine translations, so the quality and accuracy may vary.

Articles within series