2019/02/14 Teaching AI Systems to Adapt to Dynamic Environments

Current AI systems excel at tasks defined by rigid rules – such as mastering the board games Go and chess with proficiency surpassing world-class human players. However, AI systems aren’t very good at adapting to constantly changing conditions commonly faced by troops in the real world – from reacting to an adversary’s surprise actions, to fluctuating weather, to operating in unfamiliar terrain. For AI systems to effectively partner with humans across a spectrum of military applications, intelligent machines need to graduate from closed-world problem solving within confined boundaries to open-world challenges characterized by fluid and novel situations.

To attempt this leap, DARPA today announced the Science of Artificial Intelligence and Learning for Open-world Novelty (SAIL-ON) program. SAIL-ON intends to research and develop the underlying scientific principles and general engineering techniques and algorithms needed to create AI systems that act appropriately and effectively in novel situations that occur in open worlds. The program’s goals are to develop scientific principles to quantify and characterize novelty in open-world domains, create AI systems that react to novelty in those domains, and to demonstrate and evaluate these systems in a selected DoD domain. A Proposers Day for interested proposers is scheduled for March 5, 2019, in Arlington, Virginia.

“Imagine if the rules for chess were changed mid-game,” said Ted Senator, program manager in DARPA’s Defense Sciences Office. “How would an AI system know if the board had become larger, or if the object of the game was no longer to checkmate your opponent’s king but to capture all his pawns? Or what if rooks could now move like bishops? Would the AI be able to figure out what had changed and be able to adapt to it?”

Existing AI systems become ineffective and are unable to adapt when something significant and unexpected occurs. Unlike people, who recognize new experiences and adjust their behavior accordingly, machines continue to apply outmoded techniques until they are retrained.

Given enough data, machines are able to do statistical reasoning well, such as classifying images for face-recognition, Senator said. Another example is DARPA’s AI push in self-driving cars in the early 2000s, which led to the current revolution in autonomous vehicles. Thanks to massive amounts of data that include rare-event experiences collected from tens of millions of autonomous miles, self-driving technology is coming into its own. But the available data is specific to generally well-defined environments with known rules of the road.

“It wouldn’t be practical to try to generate a similar data set of millions of self-driving miles for military ground systems that travel off-road, in hostile environments and constantly face novel conditions with high stakes, let alone for autonomous military systems operating in the air and on sea,” Senator said.

If successful, SAIL-ON would teach an AI system how to learn and react appropriately without needing to be retrained on a large data set. The program seeks to lay the technical foundation that would empower machines, regardless of the domain, to go through the military OODA loop process themselves – observe the situation, orient to what they observe, decide the best course of action, and then act.

“The first thing an AI system has to do is recognize the world has changed. The second thing it needs to do is characterize how the world changed. The third thing it needs to do is adapt its response appropriately,” Senator said. “The fourth thing, once it learns to adapt, is for it to update its model of the world.”

SAIL-ON will require performers and teams to characterize and quantify types and degrees of novelty in open worlds, to construct software that generates novel situations at distinct levels of a novelty hierarchy in selected domains, and to develop algorithms and systems that are capable of identifying and responding to novelty in multiple open-world domains.

SAIL-ON seeks expertise in multiple subfields of AI, including machine learning, plan recognition, knowledge representation, anomaly detection, fault diagnosis and recovery, probabilistic programming, and others.