Stanford NLP Python library for understanding and improving PyTorch models via interventions

This repository collects all relevant resources about interpretability in LLMs

Decomposing and Editing Predictions by Modeling Model Computation

For OpenMOSS Mechanistic Interpretability Team's Sparse Autoencoder (SAE) research.

Steering vectors for transformer language models in Pytorch / Huggingface

Stanford NLP Python library for benchmarking the utility of LLM interpretability methods

Mechanistically interpretable neurosymbolic AI (Nature Comput Sci 2024): losslessly compressing NNs to computer code and discovering new algorithms which generalize out-of-distribution and outperform human-designed algorithms

Interpreting how transformers simulate agents performing RL tasks

Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".

🧠 Starter templates for doing interpretability research

Sparse and discrete interpretability tool for neural networks

Sparse probing paper full code.

Generating and validating natural-language explanations for the brain.

[ICLR 23 spotlight] An automatic and efficient tool to describe functionalities of individual neurons in DNNs

CausalGym: Benchmarking causal interpretability methods on linguistic tasks

Arrakis is a library to conduct, track and visualize mechanistic interpretability experiments.

Universal Neurons in GPT2 Language Models

This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking".

Multi-Layer Sparse Autoencoders (ICLR 2025)

A curated reading list of research in Sparse Autoencoders, Feature Extraction and related topics in Mechanistic Interpretability