Intelligent Driving System

Hierarchical DRL–A*–MPC framework for autonomous navigation,
featuring PPO decision-making, global A* planning, and MPC tracking solved via CVXPY + OSQP, with reproducible simulation demos.

Features

• High-level decision — PPO agent (Stable-Baselines3) with clipped objective & GAE for macro-action selection
• Global planning — 8-neighbor A* with Euclidean heuristic and obstacle avoidance
• Low-level control — Linear time-varying MPC (LT-MPC) using a kinematic bicycle model, formulated in CVXPY and solved with OSQP
• Modular interfaces — each layer is swappable without breaking the pipeline
• Reproducible experiments — fixed seeds, pinned dependencies, logged artifacts
• Simulation demos — 4×4 empty, 4×4 with obstacles, and 8×8 zero-shot

---

Project Structure

• A Planning*: Astar.py, astar_agent.py
• High-Level Decision (PPO): ppo_agent.py, env_gridworld.py
• Low-Level Control (MPC): mpc_agent.py, mpc_module.py
• Visualization & Utilities: visualization.py, dialog_parser.py
• • Simulation: simulation.py

Train PPO

python ppo_agent.py

hierarchical(Simulation Demo I — 4×4 Grid (No Obstacles))

[DEMO_HIER] Goal reached! [METRICS] Success Rate = 100.00% [METRICS] Optimality Ratio = 1.000 [METRICS] MPC RMSE = 0.271 [METRICS] Total Moves = 3 [METRICS] Avg Plan Time = 0.05 ms [METRICS] Avg Control Time = 3785.71 ms

obstacle(Simulation Demo II — 4×4 Grid (With Obstacles))

[METRICS] Success Rate = 100.00% [METRICS] Optimality Ratio = 1.000 [METRICS] MPC RMSE = 0.271 [METRICS] Avg Plan Time = 4.43 ms [METRICS] Avg Control Time = 3475.94 ms

larger(Zero-Shot Generalization (8×8))

[METRICS] Success Rate = 0.00% [METRICS] Optimality Ratio = 1.000 [METRICS] MPC RMSE = 0.643 [METRICS] Avg Plan Time = 4.86 ms [METRICS] Avg Control Time = 4636.28 ms

Quantitative Results & Comparisons

<img width="859" height="424" alt="image" src="https://github.com/user-attachments/assets/3cfef225-c70f-4435-b160-7e9a610d7680" />


Generalization: It performs well in environments of the same size but without obstacles (Obstacle), but fails in zero-shot scenarios on larger grids (Larger).
Performance balance: Optimal paths are achieved in both default and obstacle scenarios, and MPC RMSE and control latency remain within reasonable ranges.
Latency bottleneck: MPC's runtime is much greater than A* and increases with scene complexity (map size, number of obstacles), suggesting that more efficient tracking algorithms or parallelization solutions can be explored in the future.

Limitations & Future Work

Multi-scenario joint training and fine-tuning: Joint training or domain adaptation is performed in grid environments of different sizes (e.g., 8×8, 16×16) and with various obstacle densities to improve model generalization.
Control layer optimization: Explore parallel MPC or adopt learning-based control (such as deep reinforcement learning controllers and lightweight PID/LQR hybrid strategies) to significantly reduce tracking latency.
Continuous Space and Dynamic Environment Extension: Migrating the grid world framework to continuous space and adding dynamic obstacles to achieve navigation and obstacle avoidance that is closer to real-world scenarios.
Multi-agent collaboration: Research on the collaborative planning and control of multiple agents in a hierarchical architecture to improve the efficiency and robustness of cluster task execution.