Skip to content

Cache the dependent/suggester counts in Redis - #1836

Open
Seldaek wants to merge 1 commit into
mainfrom
perf/cache-dependent-counts
Open

Cache the dependent/suggester counts in Redis#1836
Seldaek wants to merge 1 commit into
mainfrom
perf/cache-dependent-counts

Conversation

@Seldaek

@Seldaek Seldaek commented Sep 7, 2026

Copy link
Copy Markdown
Member

getDependentCount() and getSuggestCount() run a COUNT(*) over the dependent/suggester tables on every package page view (PackageController:685-686), again in the package JSON API (:572-573), and again on the dependents/suggesters pages. For a widely-required package like psr/log that is a several-hundred-thousand-row index scan — and the package page alone is ~4.5M requests per APM period, so this is the largest single source of avoidable MySQL load on the site.

by_type (packageName, type) and all_suggesters (packageName) already cover these queries, so there is nothing left to optimise in the query itself. The fix is to stop running it.

  • Counts are cached for an hour with a random variance, to spread out the refresh of the most-requested packages.
  • TTL-only, no explicit invalidation: DependentRepository::updateDependentSuggesters() writes rows keyed by the required package name while operating on the requiring package, so precise invalidation would mean a key deletion per required name on every package update. A count badge being an hour out of date is harmless.
  • Keys are lowercased, because the packageName columns use a case-insensitive collation and differently-cased requests must not get separate entries.

Also drops a now-stale phpstan-baseline.neon entry: getSuggestCount() never applied max(0, …) despite declaring int<0, max>, and the shared helper does.

Test isolation

The test env now points the cache client at its own Redis DB (REDIS_CACHE_URL) and IntegrationTestCase flushes it per test. The DB is rolled back between tests but Redis is not, so counts keyed by package name would otherwise leak into later tests that reuse a name — latent flakiness for this and any future cache.

Verification

composer phpstan clean, full suite green (794 tests). New tests cover per-type keying, cache reads, case-insensitivity, and that a zero count is cached too (so unknown names don't re-query on every page view).

getDependentCount() and getSuggestCount() run a COUNT(*) over the dependent/suggester
tables on every package page view, again in the package JSON API, and again on the
dependents/suggesters pages. For a widely-required package like psr/log that is a
several-hundred-thousand-row index scan, and the package page alone accounts for 4.5M
requests per APM period.

The existing indexes already cover the queries, so there is nothing left to optimise in
the query itself - the fix is to stop running it. Cache both counts for an hour with a
random variance to spread out the refresh of the most-requested packages. A count badge
being an hour out of date is harmless, so this is TTL-only: dependent rows are written
keyed by the required package name while the writer operates on the requiring package,
so precise invalidation would mean a key deletion per required name on every update.

Keys are lowercased because the packageName columns use a case-insensitive collation and
differently-cased requests must not get separate entries.

The test env now points the cache client at its own Redis DB, and IntegrationTestCase
flushes it per test: the DB is rolled back between tests but Redis is not, so counts
keyed by package name would otherwise leak into later tests that reuse a name.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants