Skip to content

Conversation

CharlieFRuan
Copy link
Member

@CharlieFRuan CharlieFRuan commented Apr 27, 2025

This PR updates the web-xgrammar TypeScript package, mainly adding support for Structural Tag. There are two components to this PR:

  1. Address trivial changes to the APIs, including adding any_whitespace and ebnfToGrammarNoNormalization(), and fixing any failed unit tests due to changes in the C++ source (see first commit 6bccf6d)
  2. Add structural tag support. Specifically, we export StructuralTagItem, fromStructuralTag(), and compileStructuralTag(). In addition, we add isGrammarAcceptString() for testing. Necessary changes are made to xgrammar_binding.cc

No change is needed outside of the web/ folder.

The structural tag feature is tested with TypeScript unit tests equivalent to test_grammar_matcher_structural_tag.py, and an E2E example in web/example/.

We subsequently publish 0.1.18 npm package https://www.npmjs.com/package/@mlc-ai/web-xgrammar

@CharlieFRuan CharlieFRuan requested a review from Copilot April 27, 2025 02:09
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for structural tag grammars in the web‑xgrammar package while updating JSON schema conversion and compilation functionality in line with Python improvements.

  • Introduces a new StructuralTagItem class and updates tests to validate structural tag grammar handling.
  • Adds new C++ binding functions and corresponding TypeScript wrappers for structural tag support.
  • Updates JSON schema to EBNF conversion and compilation functions to include an "any_whitespace" parameter.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
web/tests/grammar.test.ts Updated test cases to include structural tag testing and JSON schema adjustments
web/src/xgrammar_binding.cc Added new binding functions for structural tag support
web/src/xgrammar.ts Added StructuralTagItem and updated grammar functions for new parameters and structural tag grammar
web/src/index.ts Updated exports to include StructuralTagItem
web/example/src/example.ts Added an example demonstrating structural tag grammar usage
Comments suppressed due to low confidence (1)

web/src/xgrammar.ts:292

  • There's a spelling error in the comment: 'corrrespondence' should be 'correspondence'.
 *   *   The corrrespondence of tags and triggers is automatically determined: then if the trigger is encountered, the following output must match one

@CharlieFRuan CharlieFRuan changed the title [Web] Support structural tag for web-xgrammar package, make web up-to-date with Python [Web] Support structural tag for web-xgrammar package, update web to 0.1.18 Apr 27, 2025
@CharlieFRuan CharlieFRuan marked this pull request as ready for review April 27, 2025 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant