chore(queries): organize and test #366

avivkeller · 2025-07-21T20:09:00Z

The queries and RegExp's we use should be extensively tested to prevent anything slipping through. This PR moves all of our RegExps and queries into their own file, and adds unit tests. This way, we can re-use the same testing structure across our RegExps, to prevent de-syncs.

vercel · 2025-07-21T20:09:05Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
api-docs-tooling	✅ Ready (Inspect)	Visit Preview	Jul 31, 2025 7:18pm

Copilot

Pull Request Overview

This PR addresses issue #356 by refactoring and organizing regex and Unist query utilities into separate, focused modules. The changes improve code organization by extracting regex patterns into a reusable regex module and creating dedicated Unist query functions.

Key Changes

Extracted all regex patterns from queries/index.mjs into a new queries/regex.mjs module for better reusability
Created a new queries/unist.mjs module containing node type checking functions that were previously inline
Updated all import statements across the codebase to reference the new modular structure

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/utils/queries/regex.mjs`	New module containing all regex patterns with comprehensive documentation
`src/utils/queries/unist.mjs`	New module containing Unist node type checking functions
`src/utils/queries/index.mjs`	Refactored to import and use patterns from new modules, removed inline definitions
`src/utils/parser/index.mjs`	Updated import to use `YAML_INNER_CONTENT` from regex module
`src/utils/parser/constants.mjs`	Updated to import heading regex patterns from regex module
Multiple generator and linter files	Updated imports to use functions from the new unist module

src/generators/metadata/utils/parse.mjs

src/utils/queries/regex.mjs

AugustinMauroy

LGTM ! but could you provide a before after for the issue

avivkeller · 2025-07-21T20:17:02Z

After

Before

AugustinMauroy · 2025-07-21T20:35:14Z

Thanks 👍

avivkeller · 2025-07-24T00:37:36Z

Bump @nodejs/web-infra, this is a bug fix

ovflowd

Bump @nodejs/web-infra, this is a bug fix

How is such a huge PR a bug fix? This seems a refactor per PR-description. What are you fixing?

ovflowd · 2025-07-24T10:37:21Z

src/utils/queries/index.mjs

@@ -13,6 +13,13 @@ import {
 } from '../parser/index.mjs';
 import { getRemark } from '../remark.mjs';
 import { transformNodesToString } from '../unist.mjs';
+import {


I'd rather a prefixed import

Cleaner in this situation, these are a bunch on non prefixed constants, doing a wildcard import here better explains where they belong to.

ovflowd · 2025-07-24T10:38:41Z

What I fail to understand is why you arbitrarily decided to redesign the structure of the whole queries. Please revert these changes and only include your bug fix.

If you want to separate certain things, that's fine, but at least give an explanation why. Arbitrary refactors aren't something I support.

avivkeller · 2025-07-25T19:21:53Z

How is such a huge PR a bug fix?

This PR refactors our RegExp queries, and, in the process, fixes #356, I should not have classified this as a "bug fix", it's a change that also fixes a bug.

What I fail to understand is why you arbitrarily decided to redesign the structure of the whole queries

I figured, while I'm adding a RegExp file, why not move the query RegExps there as well, and at that point, also move the Unist functions. But I'm happy to revert, since that was quite the tangent.

ovflowd · 2025-07-25T20:33:23Z

How is such a huge PR a bug fix?

This PR refactors our RegExp queries, and, in the process, fixes #356, I should not have classified this as a "bug fix", it's a change that also fixes a bug.

What I fail to understand is why you arbitrarily decided to redesign the structure of the whole queries

I figured, while I'm adding a RegExp file, why not move the query RegExps there as well, and at that point, also move the Unist functions. But I'm happy to revert, since that was quite the tangent.

I do insist. Not against the refactor, but make a dedicated PR just for the refactor, without the bug fix.

This allows us to discuss the bug fix on this PR and the refactor on another :)

avivkeller · 2025-07-27T14:17:09Z

Blocked by #379.

TODO Add test cases

codecov · 2025-07-28T18:25:56Z

Codecov Report

❌ Patch coverage is 96.92172% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.27%. Comparing base (53b457d) to head (ccdd432).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/utils/queries/unist.mjs	87.37%	13 Missing ⚠️
src/generators/metadata/utils/parse.mjs	47.36%	10 Missing ⚠️
src/generators/legacy-html/utils/buildContent.mjs	60.00%	4 Missing ⚠️
src/generators/jsx-ast/utils/buildContent.mjs	62.50%	3 Missing ⚠️
src/generators/json-simple/index.mjs	50.00%	1 Missing ⚠️
...rc/generators/jsx-ast/utils/buildPropertyTable.mjs	75.00%	1 Missing ⚠️
src/generators/jsx-ast/utils/buildSignature.mjs	50.00%	1 Missing ⚠️
src/generators/legacy-json/utils/parseList.mjs	85.71%	1 Missing ⚠️
src/utils/queries/index.mjs	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #366      +/-   ##
==========================================
+ Coverage   74.27%   76.27%   +1.99%     
==========================================
  Files         118      120       +2     
  Lines       11041    11906     +865     
  Branches      695      718      +23     
==========================================
+ Hits         8201     9081     +880     
+ Misses       2837     2822      -15     
  Partials        3        3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

avivkeller · 2025-07-28T18:29:09Z

The RegExps have 100% test coverage, with around 600 lines of tests (but I'm open to adding more). Ideally, these RegExps will almost never change, lest we break something

ovflowd · 2025-07-31T19:15:01Z

src/generators/api-links/utils/extractExports.mjs

@@ -99,7 +97,7 @@ function handleExpression(node, basename, nameToLineNumberMap) {
            case 'Identifier': {
              exports.identifiers.push(value.name);

-              if (CONSTRUCTOR_EXPRESSION.test(value.name[0])) {
+              if (/^[A-Z]/.test(value.name)) {


Let's keep regexes in constants for easily finding regexes in one place

Even for these, single-character ones? Or, are you referring to all the RegExps?

Copilot AI review requested due to automatic review settings July 21, 2025 20:09

avivkeller requested a review from a team as a code owner July 21, 2025 20:09

Copilot AI reviewed Jul 21, 2025

View reviewed changes

src/generators/metadata/utils/parse.mjs Outdated Show resolved Hide resolved

src/utils/queries/regex.mjs Outdated Show resolved Hide resolved

vercel bot deployed to Preview July 21, 2025 20:10 View deployment

avivkeller force-pushed the organize-regex branch from 2693be0 to 6bc3f1e Compare July 21, 2025 20:10

vercel bot deployed to Preview July 21, 2025 20:12 View deployment

AugustinMauroy approved these changes Jul 21, 2025

View reviewed changes

vercel bot deployed to Preview July 21, 2025 20:16 View deployment

araujogui approved these changes Jul 23, 2025

View reviewed changes

ovflowd requested changes Jul 24, 2025

View reviewed changes

ovflowd reviewed Jul 24, 2025

View reviewed changes

avivkeller changed the title ~~fix(regexp): organize~~ fix(queries): organize Jul 25, 2025

avivkeller mentioned this pull request Jul 25, 2025

fix(regexp): re-use common patterns #376

Merged

avivkeller marked this pull request as draft July 27, 2025 14:16

avivkeller mentioned this pull request Jul 27, 2025

fix(types,css): fix type parsing, and css headers #379

Merged

avivkeller force-pushed the organize-regex branch from ee35904 to 67904ed Compare July 27, 2025 15:16

vercel bot deployed to Preview July 27, 2025 15:18 View deployment

chore(queries): organize and test

2c75bcf

avivkeller changed the title ~~fix(queries): organize~~ chore(queries): organize and test Jul 28, 2025

avivkeller force-pushed the organize-regex branch from 67904ed to 2c75bcf Compare July 28, 2025 18:25

avivkeller marked this pull request as ready for review July 28, 2025 18:25

fixup!

140a438

vercel bot deployed to Preview July 28, 2025 18:29 View deployment

avivkeller added 2 commits July 28, 2025 14:43

fixup!

055219c

fixup!

f8d5c7d

vercel bot deployed to Preview July 28, 2025 18:47 View deployment

Merge branch 'main' into organize-regex

ccdd432

ovflowd reviewed Jul 31, 2025

View reviewed changes

vercel bot deployed to Preview July 31, 2025 19:18 View deployment

chore(queries): organize and test #366

Are you sure you want to change the base?

chore(queries): organize and test #366

Conversation

avivkeller commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

AugustinMauroy left a comment

Choose a reason for hiding this comment

Uh oh!

avivkeller commented Jul 21, 2025

After

Before

Uh oh!

AugustinMauroy commented Jul 21, 2025

Uh oh!

avivkeller commented Jul 24, 2025

Uh oh!

ovflowd left a comment

Choose a reason for hiding this comment

Uh oh!

ovflowd Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

avivkeller Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ovflowd Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

ovflowd commented Jul 24, 2025

Uh oh!

avivkeller commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ovflowd commented Jul 25, 2025

Uh oh!

avivkeller commented Jul 27, 2025

Uh oh!

codecov bot commented Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

avivkeller commented Jul 28, 2025

Uh oh!

ovflowd Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

avivkeller Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

avivkeller commented Jul 21, 2025 •

edited

Loading

vercel bot commented Jul 21, 2025 •

edited

Loading

avivkeller commented Jul 25, 2025 •

edited

Loading

codecov bot commented Jul 28, 2025 •

edited

Loading

avivkeller Jul 31, 2025 •

edited

Loading