build: pin file.encoding so the Maven build works on Windows

[knutwannheden]

mvn verify cannot build the parser on Windows, which is why windows-latest sits commented out of the maven_verify matrix. JJTree reads the .jjt under GRAMMAR_ENCODING but writes the intermediate .jj with the JVM default charset, so a legacy code page mangles the grammar's Unicode character classes and JavaCC rejects what it reads back:

Error: Line 2331, Column 294: Right end of character range '?' has a lower ordinal value than the left end of character range '?'.
[ERROR] Failed to execute goal org.javacc.plugin:javacc-maven-plugin:3.8.0:jjtree-javacc (javacc) on project jsqlparser

The asymmetry sits in org.javacc.jjtree.IO — GRAMMAR_ENCODING governs reading only:

setInput:   in  = new BufferedReader(new InputStreamReader(new FileInputStream(ifn), Options.getGrammarEncoding()));
setOutput:  out = new PrintWriter(new FileWriter(ofile));

Why only the Maven build

gradle.properties has pinned -Dfile.encoding=UTF-8 since af7bc1c (2021-11-29). Take that away and Gradle fails the same way:

$ ./gradlew --rerun-tasks --no-build-cache -Dorg.gradle.jvmargs="-Dfile.encoding=windows-1252" compileJavacc
> Unable to compile 'net/sf/jsqlparser/parser/JSqlParserCC.jj'
Error: Line 2331, Column 38: String in character list may contain only one character.

The two pins

.mvn/jvm.config covers the Maven JVM, where the JavaCC plugin runs the generator in-process. Surefire then forks its test JVMs, and those inherit neither MAVEN_OPTS nor jvm.config, so ParserKeywordsUtilsTest — which regenerates the .jj in-process and reads it back — still failed on the first Windows run:

Tests run: 5390, Failures: 0, Errors: 2, Skipped: 26
ParserKeywordsUtilsTest.getAllKeywordsUsingJavaCC  <<< ERROR!  org.javacc.parser.MetaParseException

Hence the second pin in the surefire argLine. Gradle needs no equivalent: its test workers take their encoding from the daemon.

With both pins, all six jobs pass — including Maven Verify (windows-latest): https://github.com/knutwannheden/JSqlParser/actions/runs/34449425105

Consider JDK 18+ instead

JEP 400 makes UTF-8 the default charset regardless of platform, so a newer JDK removes the whole class of problem with nothing to remember:

java=17  defaultCharset=US-ASCII  native.encoding=US-ASCII
java=21  defaultCharset=UTF-8     native.encoding=US-ASCII

Two pins were needed here, and any future forked JVM would need a third. The exposure is wider than the generator, too: SelectTest reads a fixture with Charset.defaultCharset(), TestUtils and SpecialOracleTest write with bare FileWriter, and non-ASCII fixtures already exist (large-sql-issue-923.txt, interval01.sql, explain01.sql). maven.compiler.release is 11, so the toolchain bump is independent of the target; mvn verify passes on 21 locally.

I kept the pins because they also fix the JDK 17 the workflow currently pins. Happy to switch this PR to the JDK bump, or to both, if you prefer.

Or fix JJTree

One line in the javacc-8 fork this build depends on, using the call setInput makes a few lines above — cherry-pickable independently of this PR:

-out = new PrintWriter(new FileWriter(ofile));
+out = new PrintWriter(new OutputStreamWriter(new FileOutputStream(ofile), Options.getGrammarEncoding()));

getGrammarEncoding() falls back to file.encoding when unset, so this is backward compatible.

It also covers a silent variant the other options miss. <#CJK> is pure ASCII in the grammar, yet JJTree decodes the escapes and emits literals:

grammar .jjt:   | <#CJK: ["\uAC00"-"\uD7A3", "\u4E00"-"\u9FFF"]>
generated .jj:  | <#CJK: ["가"-"힣", "一"-"鿿"]>

A code page representing neither endpoint turns that into ["?"-"?"] — a valid single-character range, so the build succeeds and quietly ships a parser that drops CJK identifiers. JavaCC 7 escaped everything it wrote; JavaCC 8 does not.

[manticore-projects]

Awesome! We were searching for this since months! Thank you much for figuring this out, we will test and get back asap!
I am actually going to fix this in JJTree.

[manticore-projects]

Sorry, one more question: why only on Windows? Why is Linux/Maven and also MacOs/Maven working fine? (As well as Gradle on all platforms)?

[knutwannheden]

why only on Windows?

Good question. I was assuming this was already working well on the other platforms, as I remember seeing reported that this was a problem on Windows only.

AFAIK, depending on the locale, UTF-8 is the default encoding on Linux and also macOS, whereas on Windows it is always cp-1252.

Also note that as of JDK 18 UTF-8 is used as the default encoding on all platforms, so switching the build to run on e.g. JDK 21 would be another solution.

[knutwannheden]

Thanks for merging this. Does this mean that a new release could be out soon?

[manticore-projects]

Thanks for merging this. Does this mean that a new release could be out soon?

Yes, because you just solved the only showstopper!