Policies over Poses: Reinforcement Learning based Distributed Pose-Graph Optimization for Multi-Robot SLAM

This repository implements GraphAware-Soft Actor–Critic (GA-SAC) with a central critic and decentralized actors over a graph‑neural‑network encoder to solve multi‑robot pose‑graph optimization (PGO). Environments are built on PettingZoo/Gymnasium and graphs are represented with PyTorch Geometric (PyG). The following figure shows the conceptual overview of proposed framework:

Experimental Video and Demo

MRS25_0029_VI_fi.mp4

Citation

If you publish work using this codebase, please cite our project appropriately (https://arxiv.org/abs/2510.22740)

    @inproceedings{Ghanta2025PoliciesOP,
      title={Policies over Poses: Reinforcement Learning based Distributed Pose-Graph Optimization for Multi-Robot SLAM},
      author={Sai Krishna Ghanta and Ramviyas Parasuraman},
      year={2025},
      url={https://api.semanticscholar.org/CorpusID:282389056}
    }

Repository layout

Github/
├── eval.py                          # Evaluate trained actors vs. classic solvers; uses GTSAM for GN/LM
├── train_sac.py                     # Main training loop (SAC with central critic)
├── env/
│   └── pgo_env.py                   # ParallelEnv for pose-graph RL (PettingZoo)
├── models/
│   ├── actor.py                     # Actor network(s)
│   ├── critic.py                    # Central critic
│   ├── edge_net.py                  # Edge MLP used by GNN
│   └── gnn_encoder.py               # Edge-conditioned GNN encoder (PyG)
├── partition/
│   └── Graph_partition.py           # Graph partitioning (METIS / NetworkX)
└── pg_generator/
    └── generate_pose_graphs_realistic_with_timestamps.py  # Synthetic graph generator

---

Requirements

• Python 3.9–3.11 recommended
• CUDA‑enabled PyTorch (optional but recommended)
• See requirements.txt in the project root for pip packages

PyTorch Geometric note: PyG has platform‑specific wheels. If pip install -r requirements.txt fails on PyG, follow the official instructions to match your Torch/CUDA version, then install torch_geometric (and companions like torch_scatter, torch_sparse) from wheels.

---

Quick start

1. Create and activate a virtual environment (optional but recommended).

2. Install dependencies:

   pip install -r requirements.txt

3. Train:

   python Github/train_sac.py --preset n2 --k 2 --seed 0 --log_dir runs/exp1

4. Evaluate (after training; requires GTSAM & matplotlib):

   python Github/eval.py --ckpt_dir runs/exp1

---

Configuration & tips

• Environment: env/pgo_env.py registers a parallel PettingZoo environment (pose_graph_pgo_v3). The action is (edge_selector, Δx, Δy, Δθ); observations are PyG Data graphs per agent.
• Models: models/ contains the GNN encoder (NNConv‑based), actor(s), and a central critic. Hidden sizes and recurrence options are defined in gnn_encoder.py.
• Partitioning: partition/Graph_partition.py uses METIS via NetworkX to split the pose graph across agents.
• Graph generation: pg_generator/generate_pose_graphs_realistic_with_timestamps.py synthesizes multi‑robot graphs used in training.
• Logging & checkpoints: train_sac.py saves actor/critic weights under the specified --log_dir.

---

Contributions

• Sai Krishna Ghanta - PhD Candidate
• Dr. Ramviyas Parasuraman - Lab Director

Heterogeneous Robotics Lab

School of Computing, University of Georgia.

For further information, please contact Dr. Ramviyas Parasuraman at ramviyas@uga.edu.