The U.S. Army recently awarded a massive contract, which could ultimately be worth close to $22 billion over the next decade, to Microsoft for the production of new augmented reality vision systems. These helmet-mounted systems feature advanced sensor fusion capabilities, similar to those found on the latest generations of night vision and other image-enhancing optics. In addition, they will have the ability to project a wide array of data piped in from various sources, including off-board platforms, providing improved situational awareness and other potentially game-changing benefits to troops in the field.
The Army announced the deal for production of what it calls the Integrated Visual Augmentation System (IVAS) on March 31, 2021, but said that it had actually been signed five days earlier. The contract includes “a five year Indefinite Delivery/Indefinite Quantity (IDIQ)-like ordering period” and “five one-year options for CLS [contractor logistics support],” with “a potential agreement ceiling of $21.88B,” the service told Breaking Defense.
It’s not clear how many IVAS units the Army might eventually buy, but it has talked about purchases of as many as 40,000 of these systems in the past. The U.S. Marine Corps and U.S. special operations forces might also piggyback onto those orders in the future.
“The IVAS aggregates multiple technologies into an architecture that allows the Soldier to Fight, Rehearse, and Train using a single platform,” the Army said in its March 31 press release. “The suite of capabilities leverages existing high-resolution night, thermal, and Soldier-borne sensors integrated into a unified Heads Up Display to provide the improved situational awareness, target engagement, and informed decision-making necessary to achieve overmatch against current and future adversaries.”
IVAS is a militarized derivative of Microsoft’s commercial HoloLens augmented reality system. The company began selling the first generation of Hololens in 2016.
The initial IVAS prototype variants, also known as Capability Set 1 and 2, or CS1 and CS2, were effectively HoloLenses with some modifications to meet the Army’s requirements. Those systems proved to not be robust enough for military use, notably being unable to work in the rain. Since 2019, the current version, known as Capability Set 3 (CS3), is much more rugged and was just recently put through cold weather testing in Alaska.
As already noted, the goal of IVAS is to provide individual soldiers with a wealth of capabilities and important data in a similar fashion, in broad strokes, to what one might expect to see from the kinds of helmet-mounted displays used by the pilots of combat aircraft. Projecting all that information on a helmet-mounted visor is very much akin, in concept, to the advanced helmet used by pilots flying variants of the F-35 Joint Strike Fighter.
IVAS has night vision and thermal video cameras, which allow individuals to see at night or through smoke, dust, and other obscurants, much like more traditional night vision or thermal optics. It will be able to fuse those feeds together to maximize the fidelity and other benefits that these different kinds of imagery offer in different environments.
The system might eventually be able to automatically spot and mark objects of interest for the user and IVAS reportedly already has some level of facial recognition capability, which could assist in positively identifying specific individuals during raids. In the future, artificial intelligence-driven systems could further help speed up the process of identifying potential threats that might not be immediately obvious, especially in an actual firefight where things can easily be quite chaotic. Those kinds of capabilities are already being integrated into larger fire control systems on vehicles, as seen in the video below.
IVAS can also pipe in a video feed from a suitable optic mounted on a rifle, carbine, or a machine gun, giving personnel a way to peer around corners or into other hard-to-reach areas without first having to expose themselves to any significant degree to possible enemy fire. In the same way, sensor feeds from off-board platforms, including video cameras mounted on manned or unmanned aircraft, including tiny drones that individual soldiers can deploy themselves, as well as ground vehicles, can be projected right onto the visor.
“In the past, as the soldier in the back who’s going to actually be dismounting [from a vehicle] on the objective you may have a single screen to look at that can maybe toggle between the driver’s view or the commander’s view, or the gunners view, or perhaps you’re looking through periscope blocks or asking the crew themselves what is actually happening around you,” Army Sergeant First Class Joshua Braly said after taking part in a test of the IVAS together with new remote vision systems installed on M2 Bradley Infantry Fighting Vehicles and a Stryker wheeled armored vehicles earlier this year. “But overall when you are buttoned up in the back of a platform you have very limited situational awareness to what you’re walking into.”
“This changes how we operate honestly,” Army Sergeant Philip Bartel, a member of the 1st Brigade, 2nd Infantry Division, who also took part in that test, added. “Now guys aren’t hanging out of vehicles in dangerous situations trying to get views on what’s going on.”
In this particular test, the combination of IVAS with the cameras mounted on the vehicles allowed troops in the rear compartment to “see” through the hull to was happening outside. This is similar to how the Distributed Aperture System (DAS) works on the F-35. Various branches of the U.S. military, as well as other militaries around the world and private companies, have already been exploring how to apply similar technology to other aircraft, as well as ground vehicles and even ships.
Many of these vision-enhancing capabilities are already offered by advanced night vision optics, such as the Army’s own Enhanced Night Vision Goggle-Binocular (ENVG-B). You can read about those goggles in more detail here.
The big additional capability that IVAS brings to the table is its augmented reality functionality. This means additional data can be projected onto the visor, such as map information and waypoints, locations of potential enemy forces or innocent bystanders, and more. The way IVAS is designed, that kind of symbology could be presented in a way that appears more directly overlaid on the object in question, such as a building or other terrain feature, rather than simply an icon on a flat screen.
“The system also leverages augmented reality and machine learning to enable a life-like mixed reality training environment so the CCF can rehearse before engaging any adversaries,” the Army noted in its press release. What the service is talking about here is developing capabilities so that simulated opponents and other objects, including vehicles or buildings, can be virtually inserted into actual live training exercises. This is a capability that a company called Red 6 is also developing for aerial combat training, as you can read about more in these past stories.
Augmented reality training holds the promise of being able to conduct larger-scale exercises involving a wider array of enemy capabilities without having to physically simulate every aspect of an exercise. This, in turn, presents potential cost savings and opens up the possibility of using less robust facilities for more complicated training.
This could also help simplify more complex training exercises involving personnel on the ground, as well as in the air. Red 6’s Dan Robinson explained this potential use of distributed augmented reality to help with training, as well as in operational contexts, to The War Zone during an interview last year, saying:
Just let me give you an example, if there’s a special forces guy on the ground calling in an attack on, let’s say, a tank, or a farmhouse, or something like that, and it’s a training scenario, well, wouldn’t it be great if that tank was synthetically generated and the guy on the ground is in our AR system, talking to the individual up in his airplane that’s about to drop a smart bomb on that tank or something like that? Wouldn’t it be great if they could see the same common picture within the augmented world? And we’re calling that the joint augmented battlespace because you’re exactly right. We’re solving, probably the most complex challenge of all, because airplanes flying around at multiple hundreds of miles an hour in three dimensions, well, if we’ve solved it for that we’ve certainly solved it for a tank, a boat, whatever it is on the ground. The ability to connect those individuals together is something that represents a truly exciting and massive market opportunity. And it doesn’t stop there. I touched upon it a little earlier because I think whilst there are massive training applications for this, there are also massive operational applications through this technology.
Let me give you an example, guys and girls on the ground fighting for their lives trying to call in close air support. For the individual in the cockpit that’s about to press that button, that’s a tremendous amount of stress, especially if they’re in a sort of danger-close scenario, right where there are troops in contact. You want to make sure that you get that right. So, we all just used the traditional ‘nine line’ to call in close air support, and stuff like that. Well, what if we were connected in an augmented world? And, we could paint targets on the ground, bad guys and good guys clearly delineated through whatever visuals you wish to use. It would take away ambiguity, it would shorten the kill chain massively, and it would absolutely save lives, and there are multiple applications that we could think of and discuss, but I think there are huge operational aspects for this tech as well.
It’s not immediately clear when the Army, as well as the U.S. Marine Corps and elements of U.S. Special Operations Command, might expects to reach an initial operational capability with the IVAS system. The service says that there is still further test and evaluation to be done even after signing this new contract. Future budgets could have an impact on the project, as well. Congress notably trimmed $230 million from IVAS’s research and development in the National Defense Authorize Act (NDAA), which was passed and signed into law earlier this year.
It’s also worth noting that Microsoft has faced some controversy for this work in the past. In 2019, employees sent a letter to company executives demanding a halt to work on the project over criticisms about helping to turn warfare into a “video game” and helping the U.S. military “increase lethality.”
Still, the Army’s ability to move in the space of around two years from the CS1 version of the system to the latest, much more capable model it has been using in experiments for months already shows that considerable progress has already been made toward making this potentially revolutionary capability a reality.
Contact the author: joe@thedrive.com