Skip to content

Prototype type checking w/ pyrefly#501

Draft
yangdanny97 wants to merge 6 commits into
pydantic:mainfrom
yangdanny97:pyrefly
Draft

Prototype type checking w/ pyrefly#501
yangdanny97 wants to merge 6 commits into
pydantic:mainfrom
yangdanny97:pyrefly

Conversation

@yangdanny97

@yangdanny97 yangdanny97 commented Jun 18, 2026

Copy link
Copy Markdown

This PR prototypes the ability to swap between using ty and Pyrefly as the type checker for Monty, via the MONTY_TYPE_CHECKER env var right now.

The actual diff is small - Pyrefly exposes a minimal API that is easy to call, most of the remaining logic is refactoring to support 2 separate backends. There's just a lot of Cargo.lock churn from pulling in Pyrefly as a dependency.

For context, I believe @samuelcolvin spoke with @migeed-z at PyCon about this, and was open to us experimenting in this direction

It can't be merged in this state, because:

  • This points at my own fork of Pyrefly, which prototypes the new API that Monty uses. I will upstream it once we're on the same page.
  • Monty depends on a fork of ruff, while Pyrefly depends on a different version of ruff, so right now we're pulling in 2 copies of ruff, and we'll need to decide on a way to reconcile the versions and handle upgrades.
  • Needs some profiling and optimization: ATM bench_warm_trivial/bench_warm_builtin are pretty close to ty's performance, but repl-based benchmarks are 3-5x slower (tho admittedly I didn't do much optimization)

I'm putting this PR up just to get some early feedback. Let me know what you guys think!

@codspeed-hq

codspeed-hq Bot commented Jun 18, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 15 skipped benchmarks1


Comparing yangdanny97:pyrefly (c2c0390) with main (592d757)

Open in CodSpeed

Footnotes

  1. 15 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 28.84615% with 74 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/monty-type-checking/src/pyrefly_check.rs 0.00% 62 Missing ⚠️
crates/monty-type-checking/src/type_check.rs 71.42% 11 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@yangdanny97

Copy link
Copy Markdown
Author

Ah, the numbers in the original post were outdated, I measured the wrong commit.

The actual speed difference in repl is more like 3x; tho it varies based on the size of the thread pool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant