Skip to content

Add integration test for PayloadFactory $n-in-variable-values bug (Issue #4623)#4789

Open
SasinduDilshara wants to merge 1 commit intowso2:masterfrom
SasinduDilshara:fix-4623
Open

Add integration test for PayloadFactory $n-in-variable-values bug (Issue #4623)#4789
SasinduDilshara wants to merge 1 commit intowso2:masterfrom
SasinduDilshara:fix-4623

Conversation

@SasinduDilshara
Copy link
Copy Markdown

@SasinduDilshara SasinduDilshara commented Apr 5, 2026

Summary

  • Adds an integration test (testPayloadFactoryWithDollarSignInVariableValues) to PayloadFactorySynapseExpressionTestCase that reproduces the bug reported in PayloadFactory default template throws java.lang.IllegalArgumentException when there's a $ #4623.
  • Adds a matching Synapse API resource (/dollar-in-vars) in synapse_expressions_payload_factory.xml that sets variables whose values contain $ followed by digits (simulating Java stack traces).
  • The test verifies that the PayloadFactory mediator with template-type="default" correctly embeds literal $n strings without throwing IllegalArgumentException.

Depends on fix PR: wso2/wso2-synapse#2527
Fixes: #4623

Issue Analysis

Issue Analysis — [Issue #4623]: PayloadFactory default template throws java.lang.IllegalArgumentException when there's a $

Classification

  • Type: Bug
  • Severity Assessment: High — affects any user whose variable values contain $ followed by a digit (e.g., stack traces, Java class names like NativeWorkerPool$1, ThreadPoolExecutor$Worker).
  • Affected Component(s): wso2-synapseRegexTemplateProcessor (org.apache.synapse.mediators.transform.pfutils.RegexTemplateProcessor)
  • Affected Feature(s): PayloadFactory Mediator with template-type="default" (Synapse Expression / ${vars.*} syntax)

Reproducibility

  • Reproducible: Yes
  • Steps Executed:
    1. Deployed API with <payloadFactory media-type="json" template-type="default"> referencing variables whose values contain NativeWorkerPool$1.run, ThreadPoolExecutor$Worker.run.
    2. Sent GET /test-payload.
  • Actual Behavior: Server returned 202 Accepted with empty body; IllegalArgumentException: Illegal group reference in logs from RegexTemplateProcessor.replace().

Root Cause Analysis

Matcher.appendReplacement() interprets $n in replacement strings as capture-group back-references. Fix (wso2/wso2-synapse#2527): wrap all replacement values with Matcher.quoteReplacement().

Test plan

  • New resource /dollar-in-vars added to synapse_expressions_payload_factory.xml
  • testPayloadFactoryWithDollarSignInVariableValues() added to PayloadFactorySynapseExpressionTestCase
  • Verified manually against patched server — HTTP 200 with correct literal $1 strings in response

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Added test coverage for payload factory with special characters in variable values
    • Added test endpoint to verify correct handling of special characters in variable interpolation

…e values)

Add a new test resource and test method to PayloadFactorySynapseExpressionTestCase that covers
the case where variable values passed to the default-template PayloadFactory contain $ followed
by a digit (e.g. NativeWorkerPool$1, ThreadPoolExecutor$Worker). Before the fix these caused
an IllegalArgumentException in RegexTemplateProcessor.

Fixes: wso2#4623

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment on lines +186 to +187
public void testPayloadFactoryWithDollarSignInVariableValues() throws IOException {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log Improvement Suggestion No: 1

Suggested change
public void testPayloadFactoryWithDollarSignInVariableValues() throws IOException {
public void testPayloadFactoryWithDollarSignInVariableValues() throws IOException {
log.info("Testing PayloadFactory with dollar sign in variable values for Issue #4623");
String expectedResponse = "{" +

Comment on lines +196 to +197
String serviceURL = getMainSequenceURL() + "synapseExpressionPayload/dollar-in-vars";
HttpResponse httpResponse = httpClient.doGet(serviceURL, null);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log Improvement Suggestion No: 2

Suggested change
String serviceURL = getMainSequenceURL() + "synapseExpressionPayload/dollar-in-vars";
HttpResponse httpResponse = httpClient.doGet(serviceURL, null);
String serviceURL = getMainSequenceURL() + "synapseExpressionPayload/dollar-in-vars";
if (log.isDebugEnabled()) {
log.debug("Sending GET request to URL: " + serviceURL);
}
HttpResponse httpResponse = httpClient.doGet(serviceURL, null);

Copy link
Copy Markdown
Contributor

@wso2-engineering wso2-engineering bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Agent Log Improvement Checklist

⚠️ Warning: AI-Generated Review Comments

  • The log-related comments and suggestions in this review were generated by an AI tool to assist with identifying potential improvements. Purpose of reviewing the code for log improvements is to improve the troubleshooting capabilities of our products.
  • Please make sure to manually review and validate all suggestions before applying any changes. Not every code suggestion would make sense or add value to our purpose. Therefore, you have the freedom to decide which of the suggestions are helpful.

✅ Before merging this pull request:

  • Review all AI-generated comments for accuracy and relevance.
  • Complete and verify the table below. We need your feedback to measure the accuracy of these suggestions and the value they add. If you are rejecting a certain code suggestion, please mention the reason briefly in the suggestion for us to capture it.
Comment Accepted (Y/N) Reason
#### Log Improvement Suggestion No: 1
#### Log Improvement Suggestion No: 2

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 5, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ab47cbb3-5a3d-4764-9dc4-6c4dbb19b516

📥 Commits

Reviewing files that changed from the base of the PR and between ba3568b and b86487c.

📒 Files selected for processing (2)
  • integration/mediation-tests/tests-mediator-1/src/test/java/org/wso2/carbon/esb/mediator/test/payload/factory/PayloadFactorySynapseExpressionTestCase.java
  • integration/mediation-tests/tests-mediator-1/src/test/resources/artifacts/ESB/server/repository/deployment/server/synapse-configs/default/api/synapse_expressions_payload_factory.xml

Walkthrough

This pull request adds a new test case and corresponding Synapse API configuration to validate handling of dollar signs in payload factory variable values. The test method performs an HTTP GET request and asserts the response contains expected JSON with dollar-sign patterns in class names.

Changes

Cohort / File(s) Summary
Test Case Addition
integration/mediation-tests/tests-mediator-1/src/test/java/.../PayloadFactorySynapseExpressionTestCase.java
Added new TestNG test method testPayloadFactoryWithDollarSignInVariableValues() that invokes the dollar-in-vars endpoint and validates the response payload contains dollar-sign patterns (e.g., $Worker, $1.run).
Synapse API Configuration
integration/mediation-tests/tests-mediator-1/src/test/resources/artifacts/ESB/server/repository/.../synapse_expressions_payload_factory.xml
Added new GET API resource at uri-template="/dollar-in-vars" that defines runtime variables and uses payloadFactory with ${vars.*} expressions to construct an error-shaped JSON response.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A test hops in to check dollar signs,
The API responds with JSON designs,
Variables dance through the payload so neat,
This feature's addition makes testing complete! ✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description provides a comprehensive summary, issue analysis, reproducibility steps, root cause, and test plan, but does not follow the repository's required template structure with all standard sections. Restructure the description to match the required template, including explicit sections for Purpose, Goals, Approach, Release note, Documentation, Security checks, and other applicable sections.
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically summarizes the main change: adding an integration test for a PayloadFactory bug related to dollar-sign handling in variable values, with explicit issue reference.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@SasinduDilshara
Copy link
Copy Markdown
Author

Claude Issue Analysis — [Issue #4623]: PayloadFactory default template throws java.lang.IllegalArgumentException when there's a $

Classification

  • Type: Bug
  • Severity Assessment: High — affects any user whose variable values contain $ followed by a digit (e.g., stack traces, Java class names like NativeWorkerPool$1, ThreadPoolExecutor$Worker).
  • Affected Component(s): wso2-synapseRegexTemplateProcessor (org.apache.synapse.mediators.transform.pfutils.RegexTemplateProcessor)
  • Affected Feature(s): PayloadFactory Mediator with template-type="default" (Synapse Expression / ${vars.*} syntax)

Reproducibility

  • Reproducible: Yes
  • Environment:
    • Product: WSO2 Micro Integrator 4.6.0-SNAPSHOT
    • OS: macOS Darwin 24.0.0 (arm64)
    • Java: openjdk bundled with pack
    • Branch: master
  • Steps Executed:
    1. Extracted wso2mi-4.6.0-SNAPSHOT.zip.
    2. Deployed API TestPayloadFactoryAPI (context /test-payload) with a <payloadFactory media-type="json" template-type="default"> block that references five variables: ERR_CLASS, ERR_CODE, ERR_MESSAGE, ERR_DETAIL, ERR_EXCEPTION.
    3. Set variables ERR_DETAIL and ERR_EXCEPTION to strings containing $ followed by digits, simulating Java stack traces (e.g., NativeWorkerPool$1.run, ThreadPoolExecutor$Worker.run).
    4. Started the server and sent GET http://localhost:8290/test-payload.
  • Expected Behavior: PayloadFactory resolves the variable values and returns a valid JSON payload with the stack-trace strings embedded.
  • Actual Behavior: The server returned 202 Accepted with an empty body, and the following error was logged:
ERROR {org.apache.synapse.mediators.base.SequenceMediator} - {api:TestPayloadFactoryAPI}
Illegal group reference java.lang.IllegalArgumentException: Illegal group reference
    at org.apache.synapse.mediators.transform.pfutils.RegexTemplateProcessor.replace(RegexTemplateProcessor.java:130)
    at org.apache.synapse.mediators.transform.pfutils.RegexTemplateProcessor.processTemplate(RegexTemplateProcessor.java:64)
    at org.apache.synapse.mediators.transform.PayloadFactoryMediator.processTemplate(PayloadFactoryMediator.java:196)
    at org.apache.synapse.mediators.transform.PayloadFactoryMediator.transform(PayloadFactoryMediator.java:189)
    at org.apache.synapse.mediators.transform.PayloadFactoryMediator.mediate(PayloadFactoryMediator.java:113)
    at org.apache.synapse.mediators.transform.PayloadFactoryMediator.mediate(PayloadFactoryMediator.java:102)
  • Logs/Evidence: See above stack trace; confirmed via wso2carbon.log on every request to /test-payload.

Root Cause Analysis

The defect lives in RegexTemplateProcessor (in the wso2-synapse repository, org.apache.synapse.mediators.transform.pfutils.RegexTemplateProcessor).

RegexTemplateProcessor.replace() (around line 130) uses Java's Matcher.appendReplacement(StringBuffer, String). This method interprets the replacement string using the same mini-language as String.replaceAll(): $n is a back-reference to capture group n, and \ is an escape character. When a variable value contains a literal $1, $2, etc. (common in Java stack traces with anonymous/inner classes such as NativeWorkerPool$1 or ThreadPoolExecutor$Worker), the regex engine tries to resolve the non-existent capture group and throws IllegalArgumentException: Illegal group reference.

The fix requires escaping the replacement string before passing it to appendReplacement. The standard Java idiom is Matcher.quoteReplacement(value), which escapes all $ and \ characters so they are treated as literals.

A similar bug was previously fixed for a different code path (issue #2543), but RegexTemplateProcessor was apparently not updated at the same time (or the fix was not applied to this class).

Test Coverage Assessment

  • Existing tests covering this path:
    • integration/mediation-tests/tests-mediator-1/src/test/java/org/wso2/carbon/esb/mediator/test/payload/factory/PayloadFactoryXMLJSONTestCase.java — includes testJsonToXMLWithDollarMarkAndLiteralTrue(), but that test uses literal="true" on the argument, which bypasses this code path.
    • integration/mediation-tests/tests-mediator-1/src/test/java/org/wso2/carbon/esb/mediator/test/payload/factory/PayloadFactorySynapseExpressionTestCase.java — covers template-type="default" but does not test variable values containing $.
    • integration/mediation-tests/tests-mediator-1/src/test/java/org/wso2/carbon/esb/mediator/test/payload/factory/PayloadFactoryIntegrationSpecialCharactersAtPayloadFactoryTestCase.java — tests special characters but likely not $n in replacement values.
  • Coverage gaps identified:
    • No test exercises template-type="default" with a variable whose runtime value contains $ followed by a digit.
    • No negative/edge test for backslash (\) in variable values (related quoteReplacement concern).
  • Proposed test plan:
    • Unit test (RegexTemplateProcessorTest): Call RegexTemplateProcessor.replace() / processTemplate() directly with a template containing a ${vars.X} placeholder and a replacement value of "NativeWorkerPool$1.run". Assert no exception and verify the literal string is preserved in the output.
    • Integration test (extend PayloadFactorySynapseExpressionTestCase or add new class): Deploy an API with template-type="default" that reads a variable whose value includes $1, $2, and \. Send a request and assert the response JSON contains the exact literal string.
    • Negative/edge cases:
      • Value = "$" alone (no digit after).
      • Value = "$$" (double dollar).
      • Value = "\n" (backslash sequence).
      • Value containing multiple $n patterns (e.g., Foo$1$2).
      • Combination of $ in both the template format string and in the variable value.

@SasinduDilshara
Copy link
Copy Markdown
Author

Claude Fix Verification Report

Issue: #4623
Verdict: FIXED

Reproduction Steps Executed

  1. Built synapse-core-4.1.0-wso2v34.jar from branch fix-issue-4623 (off tag v4.1.0-wso2v34) in wso2-synapse.
  2. Patched a fresh wso2mi-4.6.0-SNAPSHOT pack by placing the JAR in patches/patch9999/synapse-core_4.1.0.wso2v34.jar.
  3. Started the server (PID 60832) — confirmed Patch changes detected in the log.
  4. Deployed TestPayloadFactoryAPI (context /test-payload) using:
    • <payloadFactory media-type="json" template-type="default"> referencing 5 variables
    • Variables ERR_DETAIL and ERR_EXCEPTION set to Java stack trace strings containing $1 (e.g., NativeWorkerPool$1.run, ThreadPoolExecutor$Worker.run)
  5. Sent GET http://localhost:8290/test-payload.

Result

The server responded with HTTP 200 OK and a valid JSON body containing the literal $1 strings embedded correctly. No IllegalArgumentException: Illegal group reference was thrown, and no errors appeared in wso2carbon.log.

Evidence

HTTP Response (HTTP 200 OK)

{
    "errorClass": "WSO2-INTERNAL",
    "errorCode": "0",
    "errorMessage": "com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog",
    "errorDetail": "org.apache.synapse.SynapseException: test\n\tat org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)",
    "errorException": "org.apache.synapse.SynapseException: test\n\tat NativeWorkerPool$1.run(NativeWorkerPool.java:172)"
}

Patch Log Entries

[2026-04-06 04:33:40,083] INFO {PatchInstaller perform} - Patch changes detected
[2026-04-06 04:33:40,789] INFO {PatchUtils applyServicepacksAndPatches} - Backed up plugins to patch0000
[2026-04-06 04:33:40,801] INFO {PatchUtils checkMD5Checksum} - Patch verification started
[2026-04-06 04:33:40,814] INFO {PatchUtils checkMD5Checksum} - Patch verification successfully completed

Error Log Check

No IllegalArgumentException, Illegal group reference, or RegexTemplateProcessor errors found in wso2carbon.log after the request.

Fix Summary

The fix applies Matcher.quoteReplacement() to variable replacement values in RegexTemplateProcessor.replace(), preventing Java's regex engine from interpreting $n patterns in variable values as capture-group back-references. This makes the PayloadFactory mediator correctly handle variable values containing $ followed by digits (common in Java stack traces with anonymous inner classes like NativeWorkerPool$1).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PayloadFactory default template throws java.lang.IllegalArgumentException when there's a $

1 participant