The side and tail numbers that adorn every aircraft on an aircraft carrier flight deck are central to orchestrating their movements and flight operations. Accurately reading them via cameras that use advanced algorithms with Optical Character Recognition (OCR) could help automate flight deck management and improve safety, the Navy says.
Since carrier aviation began, aircraft side and tail numbers have been read by deck crew and relayed to flight deck managers. But mistakes happen.
Aircraft numbers are sometimes misread owing to poor visibility, visual obstructions, or simply due to task saturation. Relaying the wrong side or tail number of an aircraft to the carrier’s Aircraft Handling Officer (“Handler”) who controls all flight deck movements in coordination with the Air Boss can throw a wrench into the highly complex and very fluid metal ‘ballet’ they manage.
A team from the Naval Air Warfare Center Aircraft Division (NAWCAD) at Joint Base McGuire-Dix-Lakehurst, N.J. is researching and developing a Night Operations Character Transcription & Recognition for Naval Aircraft Logistics (NOCTRNAL) algorithm to recognize aircraft side numbers.
NOCTRNAL actually fits into a broader research effort at Lakehurst to automate aircraft tracking on the flight deck. A project called PATRIOT (Panoramic Tracking of Real-time Information for the Ouija Tabletop) uses cameras and AI to auto-populate the real-time picture of aircraft locations and movements on the flight deck used by the Handler and Air Boss.
Today, as in the past, aircraft identities, locations, and movements are represented by flat aircraft silhouettes manually moved around a tabletop like pieces on a chess or Ouija board, which gave the big master deck control table its nickname.The Handler continually references the Ouija Board to maintain awareness and manage movement.
NOCTRNAL would take over providing side number identification of individual aircraft from humans for the virtual PATRIOT display.
“The goal of the PATRIOT system is to increase and automate situational awareness of all the activity occurring on flight decks,” Dr. Todd Morehouse, NOCTRNAL project lead said in a press release.
“The capability being researched as part of NOCTRNAL fits perfectly within that goal by providing that fine-grain tail number information. With that information, the overall accuracy of the system improves, and consequently its usefulness to the Sailors.”
Dr. Morehouse expanded on the research in written responses to questions from The War Zone including the question of whether existing tail and side numbers as traditionally applied to carrier aircraft have to be altered for NOCTRNAL to work.
“NOCTRNAL does read existing markings,” Morehouse confirmed, “aiming to be as non-intrusive as possible. The goal is to develop an algorithm that is capable of recognizing the existing markings used to identify aircraft that a Sailor would traditionally use.”
The non-intrusiveness of NOCTRNAL lies partly in its use of the same flight deck cameras that PATRIOT would use, requiring as little modification to on-deck sensors as possible. The system will not require any specific camera type according to Dr. Morehouse. This suggests that both the existing CCTV (closed circuit television) camera systems on Nimitz-class aircraft carriers and the fully stitched panoptic-like system used on the USS Ford and future Ford-class carriers could both integrate NOCTRNAL.
Intriguingly, the OCR capability that the NOCTRNAL team is developing is based on state-of-the-art machine learning algorithms which could be applied elsewhere he says.
“In our research we adapted and tuned these models to address the unique challenges on an aircraft carrier’s flight deck. However, these types of algorithms have been trained on rich and diverse datasets allowing them to operate in many environments. While NOCTRNAL does focus on side number recognition, further research could allow it to be used for identifying other features or markings on an aircraft.”
This logically raises the question of whether NOCTRNAL will be applied on other non-aircraft carrier flight decks. This could include airports large and small.
“The NOCTRNAL algorithm can serve as a useful baseline in automated text recognition cases where visibility is low or obscured,” Morehouse replied. “Since NOCTRNAL has been developed and tuned for carrier flight decks, it could be adapted to other non-aircraft carrier flight decks with similar visual environments.”
The system might also be useful in reading critical aircraft launch weights, a task traditionally done by a Weight Board operator on deck who relays aircraft takeoff weight to the catapult officer (shooter) via a digital board and confirms the weight with the pilot just before launch.
Mass Communication Specialist 3rd Class Joshua L. Leonard
Launch-weight boards/data are not part of the current intention or focus of NOCTURNAL Morehouse said but he agreed that automating launch weight data via the system is “something that could be considered in future research”.
The NOCTURNAL team didn’t offer a timeline for testing and fielding the system, citing “security reasons” However, Morehouse did allow that NOCTRNAL research is being applied to both simulation and real aircraft. “In its current stage, we are testing on both of these datasets, evaluating the performance, and making improvements to the machine learning algorithms.”
How well is it working in challenging environmental conditions? Is there an ID failure rate associated with NOCTRNAL? The system “is currently in the early stages of testing where we are learning how it performs under various conditions,” was the extent of Dr. Morehouse’s reply.
NAWCAD characterizes NOCTRNAL as an automated tool that frees up sailors working on deck so they can focus on other tasks. The flip side is that it might potentially decrease flight deck situational awareness for plane directors, aircraft handling officers, or others who will no longer pay attention to or communicate tail/side numbers.
The PATRIOT project including NOCTRNAL “will take a look at the overall picture and impact during testing in partnership with all stakeholders to provide wider benefits to the flight deck,” Morehouse says, indicating that there’s still a lot to learn.
One of the biggest issues facing future carrier operations is how multiple uncrewed aircraft will move around the tight confines of the flight deck safely. The carrier air wing will be dominated by these types in the future, according to the Navy’s plans. Given the possibilities, NOCTRNAL/PATRIOT seems like it could be beneficial to those systems, potentially interfacing with platforms like the Navy’s forthcoming MQ-25 Stingray unmanned aerial refueling tanker.
“We are still evaluating where and how the technology will make the greatest impact,” Morehouse acknowledges. “Investigating how specific platforms might interface with this information in the future is beyond the current scope of the work.”
The NOCTRNAL team is currently looking at the best way to integrate the data needed to make the system work into one package that can be used for future testing.
Regardless of its greater future potential, NOCTRNAL is yet another example of how machine learning and automation could impact the future of air combat, including century-old art of keeping tabs of aircraft moving around the deck.
Contact the editor: tyler@twz.com