If today’s Pentagon leaders get their way, the next generation of fighter jets, body armor, computer systems, and weapons will understand more about the pilots, soldiers, and analysts using them than those operators understand about the machines they are using. The very experience of flying the plane, analyzing satellite images, even firing a gun could change depending on what the weapon, vehicle, or software detects about the person to whom the weapon is bound. To make this dream real, Pentagon-backed researchers are designing an entirely new generation of wearable health monitors that make Silicon Valley’s best consumer fitness gear look quaint. They’re discovering how to detect incredibly slight changes in focus, alertness, health, and stress—and to convey those signals to machines. Design the boots well enough and the super soldier will arrive to fill them.
Army Research Laboratory researchers already monitor individual subjects from six months to two years. Brooks wants to expand that to other military training environments, such as the U.S. Military Academy at West Point, N.Y., and then to more than a dozen universities. He hopes the data will reveal how people of varied size, weight, height, health, level of alertness, etc., differ in terms of the signals they send out—hence the name “human variability.” That, in turn, will help researchers gather much more precise information on how different people interact with their environment. The ultimate goal is sensors that can tell the Pentagon how each human soldier performs, or could perform, to their best ability, from battlefield to home front.
“It’s not just while they’re at work, but also when they go on leave,” says Brooks. “This is continuous, with the highest practical resolution that we can obtain for a long period of time. Hopefully, we would see information going into many programs” to build future gear. “A greater understanding of natural human variability would then feed pretty much any system that adapts to the person.”
It’s an ambitious undertaking, considering the current limitations of body-worn sensors. Over the past two years, the military bought more than $2 million worth of Fitbits and other biomedical tracking devices. But it turns out that off-the-shelf consumer devices aren’t good enough for the military’s biotracking ambitions. So researchers are creating a new class of wearables, based on new research into embedding electronic components into fabric. If the electrodes are too small, the signal is worthless; too big, and they feel like an artificial electric shell separating the wearer from the real world. The connection between the environment and the human must remain seamless.
One application for such sensors is helmets that record brain activity while their wearers do their jobs. An ARL team is preparing for continuous electroencephalography, or EEG, by using 3-D printing to create helmets that fit perfectly to each individual soldier’s head. But the military is not eager to embed wires and metal into gear that’s meant to protect a soldier during a massive blast. So the lab is constantly looking at new materials, solutions, and tradeoffs, inching toward sensors that collect information without getting in the way of soldiering. Lab technicians showed me one experimental electrode that they were making that was so small and soft to the touch it seemed to have no metal in it at all (they are in fact constructed of nanofibers that conduct electricity, encased in silicon).
* * *
The Air Force, as well, needs a next generation of wearables to help tomorrow’s combat aircraft understand their pilots. Modern fighter jets expose human bodies to physical forces that are still not entirely understood. In 2010, multiple F-22 pilots reported in-flight episodes of confusion, shortness of breath, and skin-color changes—all symptoms of hypoxia, or decreased oxygen in the blood. The reason was speed.
“I pull a G in the airplane, blood has a tendency to collect in some of those dependent areas of the body, like the arms and legs and that,” said Lloyd Tripp, a program manager for aerospace physiology and toxicology at the Air Force Research Laboratory’s 711th Human Performance Wing. Two years later, the Air Force began to affix sensors inside the helmets of F-22 pilots to read the blood-oxygen level of their temporal artery.
Around the same time, the Russian military was also seeing confusion and skin-color changes among their pilots who pulled high G-forces, Tripp said. Lacking the same sensor technology, Russian commanders began to give pilots blood transfusions before their flights. It didn’t work. Russian pilots flying at supersonic speeds suffered hypoxia at greater rates. “They didn’t actually admit that for quite a few years,” he said. Correct diagnoses enabled the U.S. Air Force to read the problem and improve performance.
Beyond helmets, Air Force researchers are working on what they call a comprehensive cognitive monitoring system. This means exploring what sensor technologies work well for what purposes, and what signals can be detected without interfering with or disturbing the pilot—who is, after all, supposed to be flying a combat mission. Depending on what you seek to measure, they found, you may no longer need a physical sensor on the body. You can now collect incredibly intimate and important internal health data with cameras.
Take cerebral oxygenation, the amount of oxygen in the tissue of specific portions of a pilot’s brain. You can measure this key biophysical signal by shining infrared light on the forehead because the blood in front of the skull is about as oxygenated as the brain tissue behind the skull wall. “If I’m shining that infrared light through the skin, I can see the amount of oxygen within the blood in that tissue. As I increase G-force, I’m decreasing the amount of oxygen that I have here and that decrease in oxygen is directly correlated back to decreases in cognitive function,” said James Christensen, a portfolio manager with the 711th Human Performance Wing.
Another research project configured simple laptop-camera lenses to detect whether a person’s hemoglobin is oxygenated, which makes blood shows up slightly redder, or de-oxygenated, which is slightly bluer. Essentially, this lets you read a person’s heart rate from a distance.
Even your breath says something about your physical state. “The ratio between oxygen and carbon dioxide will change as I become more and more fatigued. That’s important because as I’m fatigued, it takes about 24 hours for me to actually recover 100 percent,” Christensen said. “That fatigue is important because my muscles can’t strain to push the blood back to my head and so the probability of me losing consciousness increases significantly.”
Good sensors can even detect changes in metabolism that indicate weariness and stress before the person notices. When you’re stressed, you exhale fat—or rather, water-soluble molecules called ketones that your liver produces from fat. Stress is detectable by the molecular content of your breath.
“We’re working with some folks over at our materials lab and they have a couple of companies that are looking at sensors that are going to be placed in the [pilot’s oxygen] mask that’ll look at those types of fatigue-related volatile organic compounds,” says Christensen.
Your eyes, too, give you away. “Imagine eye-tracking cameras,” Christensen said. “If those can collect not just the motion data and the eye-motion data, but those are also getting heart rate and respiration, then we can have no hardware on you at all and still get all the same physiological metrics ... A certain amount of cognitive workload tends to correlate pretty highly with stress generically. You can combine heart rate with several other measures to get at workload stress; vigilance, even.”
“We are comparing it, just for reference, with wet medical electrodes on the chest. Under most conditions, you can do about as well as wet electrodes,” he said. The lab is “testing the limits of how far away can you get and still get a reliable signal. It turns out, it’s mostly an optics problem.” That means cameras and lenses alone can detect those subtle changes in stress and attention. It’s just a matter of figuring out which ones.
There are privacy ramifications to collecting so much information. A simple camera can gather enough biometric data on an individual to understand how small changes in heart rate can be a sign of stress. For a fighter pilot, an analyst, or a soldier, this might help warn of decreased cognitive ability. But among the general population, stress can also be a signal of deception, depending on the context in which that stress expresses itself, such as an interview at a checkpoint. Today’s military-funded biophysical research shows that it’s possible to detect that stress response from 100 meters away, and perhaps even at longer distances. In theory, if you could create a lens that could capture infrared data at sufficient resolution (currently, only a theoretical possibility), you could measure brain tissue oxygenation from low-earth orbit. You could see stress from space.
When performed without a subject’s awareness or permission, biophysical monitoring can be a violation of privacy. But conducted as part of an experiment with knowing volunteers, like elite soldiers eager to understand their bodies and improve their own performance, it becomes a powerful tool. One former special operations training psychologist, who currently works for a major league baseball team, said the elite soldiers he had served with were eager to improve their performance through data. In the Air Force, pilots want to improve how they fly, complete their missions, interact with their equipment, etc.
Bit by bit, this science is making its way into actual gear and weapons. In the year 2020, Navy SEAL teams and Army Rangers could take down high-value targets while wearing an exoskeleton that’s earned the nickname ‘Iron Man.’ Biophysical sensors will play a big role in the way the suit functions.
In February and March, the Air Force successfully tested a new helmet with “physiological monitoring capabilities,” as Tripp put it. Its heads-up display shows different information based on how the pilot is feeling and other factors. The goal is to give every pilot a slightly different experience based on their unique physical and mental strengths and weaknesses, as well as their physical condition at the moment. Lab researchers and contractors anticipate it will guide the design of the next U.S. fighter jet, to be launched between 2025 and 2030.
“I may do a really, really good job on a spatial cognitive task where I’m looking at a radar warning display, and maybe James doesn’t,” Tripp said. “The thought, down the road, is to quantify my performance in these decreased physiological conditions from a cognitive perspective, and then use the changes in physiology to make the airplane smart about what kind of help I need.”