Stanford NLP Python library for understanding and improving PyTorch models via interventions

This repository collects all relevant resources about interpretability in LLMs

Decomposing and Editing Predictions by Modeling Model Computation

For OpenMOSS Mechanistic Interpretability Team's Sparse Autoencoder (SAE) research.

Steering vectors for transformer language models in Pytorch / Huggingface

Stanford NLP Python library for benchmarking the utility of LLM interpretability methods

Mechanistically interpretable neurosymbolic AI (Nature Comput Sci 2024): losslessly compressing NNs to computer code and discovering new algorithms which generalize out-of-distribution and outperform human-designed algorithms

Interpreting how transformers simulate agents performing RL tasks

Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".

A curated collection of resources focused on the Mechanistic Interpretability (MI) of Large Multimodal Models (LMMs). This repository aggregates surveys, blog posts, and research papers that explore how LMMs represent, transform, and align multimodal information internally.

🧠 Starter templates for doing interpretability research

Sparse and discrete interpretability tool for neural networks

Sparse probing paper full code.

[ICLR 23 spotlight] An automatic and efficient tool to describe functionalities of individual neurons in DNNs

Generating and validating natural-language explanations for the brain.

CausalGym: Benchmarking causal interpretability methods on linguistic tasks

Arrakis is a library to conduct, track and visualize mechanistic interpretability experiments.

Universal Neurons in GPT2 Language Models

This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking".

Multi-Layer Sparse Autoencoders (ICLR 2025)