What’s New in Abmarl

Abmarl version 0.2.8 features a new use case, showcasing Abmarl’s usage as a proxy simulator to inform reinforcement learning training in an external simulator; a refactored cli scripting interface, allowing each Abmarl command (debug, train, visualize, an analyze) to be issued in a python script in addition to being run from the terminal; and an OrientationAgent and DriftMoveActor support moving an agent through the grid by drifting it in the direction it is facing.

Using Abmarl as a Proxy Simulation

Abmarl’s GridWorld Simulation Framework has shown promise as a proxy simulation to iterate the training experience in a reinforcement learning experiment using an external simulator. In this experiment, researchers used Abmarl’s simulation interface to connect a C++ based conflict simulation JCATS to reinforcement learning algorithms in order to train an agent to navigate to a waypoint. All state updates are controlled by the JCATS simulation itself. Positional observations are reported to the RL policy, which in turn issues movement commands to the the simulator. Researchers leveraged Abmarl as a proxy simulation to rapidly find a warm start configuration. Training is performed on a cluster of 4 nodes utilizing RLlib’s client-server architecture. They successfully generated 136 million training steps and trained the agent to navigate the scenario. See the featured description for more information.

Abmarl Commands in a Python Script

All of Abmarl’s CLI commands can be used directly in a python script instead of relying on the CLI by importing those moodules and running them with the experiment configuration. See Python Scripts and a full workflow example for more details.

Drifting Agents

The combination of the OrientationAgent and DriftMoveActor allows us to move an agent through the grid by drifting it in the direction it is facing. An agent can attempt to change its direction, and the Actor will attempt to move it in whatever direction it is facing.

Miscellaneous

Interface changes

  • Exploration is off by default during visualization and can be turned on with the --explore flag.

  • External wrappers now support the unwrapped property, which returns the underlying Simulation Manager object.

Other Features

  • Support for installing Abmarl with various extras: core, develop, workflow, and all.

  • Ranomize placement order parameter for randomly iterating through dictionary of agents when initially placing them in the grid.

  • Gridworld agents have customizable render_size, which defaults to 200.

  • Traffic corridor simulates agents navigating a tight corridor. They must cooperate by taking turns in order for all of them to make it through.

  • Pacman variation simulates the Pacman arcade game, with support for training the “baddie” agents too.

Bug fixes