What Is the AI Supporting the First Step? The Reasons for the Birth of the Passenger Boarding Bridge Automated Docking System (Part 2 of 2)

The AI-mounted PBB is the "first step" towards the future of Smart Airports

The people behind the scenes who support airport operations are broadly divided into the ground staff who work at the check-in counter and carry out gate tasks, and the ground handling staff who work outdoors near the aircraft. In order to ensure the safety and peace of mind of scheduled flights and passengers, smooth-flowing operation is required in the work carried out by these airport staff. The various services received at the airport, which are usually taken for granted, are brought about by the sincere attitude of these on-site professionals.

The AI-equipped PBB that will be introduced here is the "first step" towards the future of Smart Airports. With the touch of a button, the PBB, which weighs close to 30 tons, automatically approaches to within 10 cm of the aircraft door. The standardization of this task has been successfully achieved by automating this operation, which is conventionally carried out while making fine adjustments by experienced staff members. This has made it possible for less experienced staff members to bring the PBB closer to the aircraft door in the same position and amount of time. This is truly a solution that combines labor saving and efficiency.

Leading PBB manufacturer ShinMaywa Industries and Panasonic Solution Technologies (Panasonic) were the key players in the development of this automated docking system. ShinMaywa Industries' conventional PBBs possess a patented technology (preset run) that is superior to their competitors by enabling them to automatically approach up to within 1 meter of the aircraft. Panasonic's PLC (control device) is used to control the automatic operation, laying the groundwork that connects the cooperative relationship between the companies for this latest challenge.

The PBB serves as a memorable bridge for passengers

"It all began after we were consulted by an airline company to help engage in the improved efficiency and automation of airports. We had already delivered PBBs to airports in various regions, so we thought about whether we could contribute to improve efficiency by further evolving them," recalled Akihiko Kitano of ShinMaywa Industries.

The preset Drive function, an original technology of ShinMaywa Industries, which automatically positions the PBB within 1 meter of the aircraft door, had already been useful in reducing the work time from before. The question of how to automatically dock the PBB still remained for the development of a fully automated system. It was then that the president of ShinMaywa Industries offered a proposal to "further evolve the PBBs by equipping them with image recognition technology." Experiments with image recognition using AI began in 2016. Yoshihiro Sonoda, who was the company's on-site supervisor at the time, had the following to say.

"We installed cameras in the airport and continued to record image data near the aircraft door from morning to night. We then handed over the recorded data to Panasonic, where it was analyzed using deep learning. They then returned the analyzed data, and we repeated automated docking experiments at night using parked aircrafts and PBBs." (Mr. Sonoda)

In order to increase the accuracy of the image recognition, a massive amount of actual image data recorded on site is required. Therefore, all types of image data must be prepared, including unique situations such as bad weather and exposure to strong afternoon sunlight. The AI technology used this time requires people to be teachers to develop it, and cannot learn on its own. Learning by the use of human hands made it possible to accurately recognize the position of the aircraft door when the airplane was not parked precisely in a predetermined location, or when several airplanes had different appearances because they were different models or had various painting designs.

As a result, it led to the full operation of the AI-equipped PBBs at Tokushima Awaodori Airport starting in 2018. There are no other examples of the actual operation of PBBs to automatically come within 10 cm of the aircraft door. "Both our young and veteran staff members are delighted, because the final adjustments that require extreme precision were no longer necessary," added Mr. Sonoda. This achievement was recognized, and in 2019, it was decided that the system would be introduced at Narita International Airport and Singapore Changi Airport. The process of collecting the image data has already been initiated at both airports.

"Detailed adjustments require the "right touch" added with human knowledge, but the engineers we have in charge have the flexibility to meet these requirements, and it was a success in the sense that they were very positive about it. The timing just so happened to overlap with the case study for implementing AI, so in terms of timing it was perfect," commented Mr. Sonoda, regarding Panasonic.

Speaking about the vision for the future, Mr. Kitano added, "We want to use this technology and ultimately develop it into a fully automated docking system. There are still issues regarding safety, but we are continuing to hold discussions with authorities and aim to complete development by 2020." Mr. Kitano described their future vision, saying, "There is also a plan to conduct remote monitoring of the PBB to forecast malfunctions." Mr. Sonoda explained, "This is the start of the introduction of IoT in airports. The installation of the system at Narita International Airport will lay the foundations for future development. Airports with automated facilities that link and work together are the vision of the future for the airport industry." Mr. Sonoda expressed it as follows.

"For airplane passengers, the PBB is the first and last place they walk through. By equipping it with AI, that first step has become much smoother. In that sense, the PBB is a product with dreams, a bridge of memories, and I am very proud to have the opportunity to be involved in this work." (Mr. Sonoda)

The spread of AI systems in everyday life is the ideal

Senior engineer Shigeru Shimazaki was in charge of AI at Panasonic. Since he joined the company, Mr. Shimazaki has been involved in the research and development of OCR (optical character recognition), and is currently engaged in the development of AI platforms using his acquired knowledge.

Overseeing the development of this project was very rewarding, but "The key point is not whether simply having AI will resolve everything, but rather proposing how to use what type of AI to reach the goal that our customers expect. There are things that even AI cannot do, and it is certainly not perfect, so taking the right measures is very important. This latest PBB case is the same, and there were many areas that we were unsure of until actually testing it out, so we proceeded by getting together and holding discussions." (Mr. Shimazaki) In other words, both companies sharing what they accomplished using AI was very important.

The ability to work flexibly according to the customer's case is a result of Panasonic's extensive experience in industrial and infrastructure systems that has been acquired over the years. Taking advantage of this background, Panasonic has defined new solution groups with ICT infrastructure technology and IoT technology at their core as a priority area of the next-generation business. Of them, the data analysis and AI business is achieving improved results as a promising sector.

"There are members at the company who have been involved in image analysis at the factory production lines, so it is possible to absorb their industry-related knowledge. In addition, image and video solutions are central businesses of the Panasonic Group, so there are many areas where we can cooperate across departments. In this case, figuring out how to develop software as user-friendly as possible was very important. I believe that the ability to totally complete everything is one of Panasonic's strengths." (Mr. Shimazaki)

Of course, this technology is a long-term solution with an eye on horizontal expansion. "From the first stage of development, both companies shared the same idea of expanding this solution. ShinMaywa Industries insists on extremely high precision, so we worked hard to meet their expectations and were able to win their trust. Based on the results achieved at Tokushima Awaodori Airport, ShinMaywa Industries' automated docking system was selected to be installed at Narita International Airport and Singapore Changi Airport, and I realized that Panasonic's technologies were recognized globally." (Mr. Shimazaki)

"It would be ideal for the AI system that we developed to spread to peoples' everyday lives," added Mr. Shimazaki. This case study brilliantly proved the hypothesis that AI is not something that will take jobs from humans, but rather a technology for further drawing out their performance.

*This PBB can only be provided in certain countries and regions due to the required construction.

- Reproduced from the website "Mirai-kotohajime", by courtesy of Nikkei Business Publications, Inc.

# # #