Studying and comparing approaches for inhibiting selfish behaviour of RL-based agents, in environments which encourage it.
This codebase was used for my final year dissertation, as part of my BSc CompSci degree in 2022/23.
Four multi-agent environments are evaluated, where two agents with not-relating or opposing goals exist.
The optimal policy for agent A is not aligned with agent B's goals.
This project compares three approaches for inhibiting selfish behaviour:
- Add B's reward function to A's reward function, with some multiplier
- Devise custom, per-environment, heuristics for reducing selfish behaviour
- Use Inverse Reinforcement Learning to approximate B's goals, then add them to A's reward function
If you want to learn more, you can check out this poster, or presentation (Apple Keynotes), presentation (PDF, but without videos), or just read the unnecessarily long dissertation I wrote about it.
At the time of creation, these commands worked for setting up all dependencies. With SB3 moving to gymnasium & imitation planning to later do the same, this is unlikely to work anymore.
pip install opencv-python matplotlib pandas seaborn tensorboard numpy imitation jupyter pip install git+https://github.com/carlosluis/stable-baselines3@fix_tests
imitation=1.1.1