Automatically setting required "add-opens" and "add-exports" to improve user experience on Java 25+

[Sanne]

Problem statement

Quarkus might need to have some –add-opens and –add-exports being set to function correctly; in some cases, we have fallback capabilities, so while there might not be a risk to expected application semantics, there could still be an impact on efficiency and performance. Finally, sometimes we want to avoid triggering particular warnings, as we consider this an aspect that the framework and library maintainers should track; we shouldn't cause concern to users while these aspects are being worked on, especially when such warnings are not actionable from their side.

As an example, JBoss Threads (a core library of Quarkus) requires the JVM flag --add-opens java.base/java.lang=ALL-UNNAMED to enable some safeguards against potential thread-local leaks. When the flag is not set, the application will still boot correctly, but the safeguard will not be functional. This feature is just a precaution, so not having the flag isn't likely to cause any problem; however, the library will issue a warning when this capability can't be enabled. Clearly, the optimal solution is to specify the JVM argument, and ideally, the various Quarkus launch modes should handle this automatically.

JBoss Threads is a core dependency and therefore any Quarkus library would need that particular module opened; this might lead to the conclusion that a static set of parameters could be hard-coded, however a standard set of JVM flags isn't very flexible: we support thousands of extensions and several of them might require additional settings, which are only necessary when those particular extensions are being used. We'd prefer, therefore, to resolve the general problem of allowing any extension to specify which "add-opens" and "add-exports" are required/recommended for each extension, and apply those automatically and consistently across all launch modes.

Quarkus can be started in several ways. We need to resolve the problem consistently across these launch modes; it would be a significant problem if something worked in a particular mode (e.g., development or testing) but failed in another (e.g., production). For obvious reasons, this would not be acceptable; the good news is that libraries are typically very aggressive with integrity validations, so this will help identify a comprehensive solution which we can implement iteratively, breaking down the problem into multiple improvements, and we can be reassured that there are plenty of safeguards in place. This proposal outlines a coordinated approach to multiple improvements across various areas to achieve consistency.

So we have various run modes, and we need to investigate entry point feasibility for each of them:

• Integration tests
  • run by Gradle
  • run by Maven (surefire)
  • continuous testing, run by dev-mode
• Development mode
  • initiated my Maven
  • initiated by Gradle
  • initiated by CLI
• Production mode
  • JVM
    • fast-jar
    • uber-jar
  • GraalVM native image

While this gives the high-level overview, some of these options should actually be split up further into specific options depending on other settings - we'll discuss details in further sections.

Existing related capabilities

Extensions metadata

The extensions metadata provides an option for extensions to specify JVM parameters to the dev-mode process. This is a useful starting point however is currently addressing dev-mode exclusively - we might want to extend this approach.

Manifest entries

(To not be confused with SBOM manifests: in this context, we discuss the META-INF/MANIFEST.MF file included in the jar archives).
The Java Manifest format allows specifying "add-opens" and "add-export" instructions that need to be set when the jar is executed.
We're already hardcoding a single specific "add-opens" instruction in the Manifest of the jar file of the production mode launcher. This seems like a great solution for production mode, but we'll need more flexibility than a single hard-coded entry. We plan to list additional entries via a build-item, so as to combine the needs of all extensions.

JPMSExportBuildItem

An existing build item JPMSExportBuildItem already existed, which relates to the same field; however, this one was designed specifically for GraalVM native-image and only has an impact on the build of native images. This one focuses only on add-exports - an equivalent build-item for add-opens doesn't currently exist.

Implementation plan

To configure the module system, the most obvious solution is to launch the JVM with the right parameters. Clearly, this is the desired outcome we want to aim for across the various production modes; however, Quarkus doesn't currently generate a Java launcher script. Historically, we always built a simple JAR file and allowed people to launch the built applications however they preferred.

For other modes, we have some additional options. The most complex requirements are driven by live-reload; since extensions and other conditions can change, it's clear we need a way to reconfigure modules dynamically. Luckily, this is possible, and there are several options to do so. None of these options is straightforward, so we'll look at them in more detail.

Dynamic module reconfiguration

Before getting into the details of how we might resolve the problem for each launch mode, let me clarify that we have a POC for various strategies that allow dynamic reconfiguration of the module system. This is usually not possible with "regular" Java code, and for good reasons, but there are some "advanced options" available as the JDK team recognized the need for advanced tools. Not all of the identified options are fully endorsed by the OpenJDK team, but at least two of these are, and use stable, official APIs designed for this very purpose, so this should be reassuring enough for us to know that even if some of our preferred strategies were to be "shut down", we can maintain this capability by switching to some of the more endorsed strategies.
At a high level, these are the options:

• To use a self-attaching Agent. There is an API on Instrumentation meant to reconfigure the module system, and it is fully supported for this purpose; however, we'll need to self-attach to the running JVM, and this is frowned upon: a warning gets triggered, and this capability is likely to get blocked in the future - unless a flag is set to allow for this explicitly. But this approach works just fine today.
• To use an explicit Agent; in fact, we already have an agent attached in specific modes, so we could piggy-back on it - but its presence is not guaranteed: it depends on the launch mode, and it can be disabled.
• To use the internal API "jdk.internal.module.Modules” ; this requires BOTH COMPILING AND RUNNING with --add-exports=java.base/jdk.internal.module=ALL-UNNAMED but therwise works great and is very straightforward.
• To use another sealed API , it needs to be launched with --add-opens=java.base/java.lang.invoke=ALL-UNNAMED to get access to the super-privileged MethodHandles.Lookup, and via that hook get access to restricted APIs.

Via any of these strategies, we can fully reconfigure any other module at any point in time. They all have a common requirement, though: needing some JVM flasgs to be set. Do we have a chicken-and-egg problem? Not quite, as we achieved a fundamental simplification in requiring a single, static JVM argument rather than an unknown, possibly complex set; we could document a single requirement and be done with it, but we aim to fully automate this. 

Plan for Production mode

As suggested above, in production mode we'll want to use the most robust solution, which means following the recommendations of the OpenJDK team. We'll compute the required  "add-opens" and "add-export" expressions and encode these in the Manifest of the built fast-jar and uber-jar formats, or pass the matching flags to the native-image compiler for the GraalVM case.
There's also a need to set --enable-native-access (e.g. the Netty team recommends setting --enable-native-access=io.netty.common in version 4.2, and this is most likely going to be a requirement of future versions as well). We're not using Netty 4.2+ yet, but this is certainly the plan. Luckily this one can also be set in the Jar manifest.
This should solve the immediate problem and has no known drawbacks; however, we need to discuss future-proofing.

Future proofing for production mode: introduce generation of launcher scripts

At this point, given a problem scope limited to set add-opens and add-export instructions, we can get away without generating recommended launch scripts.
However I'm worried that we won't always be able to rely on the Manifest attributes, and there certainly are many more JVM properties which we might need being able to specify (or at least recommend), to have the ability to apply improvements automatically, or address specific issues at minimal user disruption.
We should prepare for this situation in which additional flags are required, by introducing launcher scripts for the various operating systems, and transition our user base to use such launch scripts. Ideally, the same build items being developed in the scope of this proposal will then be treated as input to the process, generating such launch scripts dynamically.
We don't necessarily need to mandate the use of scripts, but it would be beneficial to control - or highly encourage - such flags, while allowing our build system and extension model to define which flags exactly should be set. While it is a natural choice to generate scripts, we can avoid them in certain build targets; for example, when generating container images, or when using jlink, the JVM flags could be set in different ways; it would be nice to be able to avoid relying on script interpreters in constrained targets.
We also want to evaluate if we want to validate on bootstrap that certain essential flags have been set; this particular point suggests that it is useful to differentiate between recommended flags and required flags in preparation for such likely evolutions.

Plan for integration tests

With both Gradle and Maven, it's possible to pass JVM parameters to the JVM that executes the tests.
This still doesn't resolve the problem elegantly, as:

1. we'd need to trust users to have defined the expected configuration settings
2. it's not very helpful to document that some settings will be needed, when such settings are dynamic and complex.
   We need to do better, either by providing excellent guidance in the form of errors/warnings or, even better, by automating it all. We will aim here for full, transparent automation.

Integration tests: Gradle

This is TBD - we'll aim to resolve the Maven case first, as I expect that if it can be done with Maven, it can be done in Gradle.

Integration tests: Maven

There is a notable complexity, namely Surefire will fork the JVM, launching it with surefire-defined parameters, well before any Quarkus code is executed. If the user, or Surefire, haven't set the JVM parameters which we need, we can't do better than complain (log warnings and expected actions). There's also the possibility that the user might have disabled forking of the JVM; in this case also JVM parameters would not be effective.
We can however, get a hook into early Surefire processing and augment & validate the Surefire configuration, when true is enabled for the Quarkus plugin.
Luckily, our code templates already set this property, so this should be fairly common - but we can't currently rely on this.

So, plan:

• Make this property mandatory / highly recommended. PR#50915 modifies the Quarkus plugin to detect if the extensions have been disabled, and provides a helpful log message about it.
• Leverage the extensions lifecycle to enable the (static) JVM option required to enable our dynamic module reconfigurer capabilities (See section "Dynamic module reconfiguration" which lists options and which JVM arguments each strategy requires).
• Ensure the dynamic module reconfigurer is invoked before tests are executed.

Live-reload Mode

Live reload is handled consistently by Gradle and Maven, they both bootstrap the DevMode JVM by invoking DevModeCommandLineBuilder.
So the plan is straight forward:

• Enforce the JVM options required for dynamic module reconfigurer in DevModeCommandLineBuilder.
• Ensure the modules are reconfigured after each augmentation process, before the app is re-loaded, with any needs expressed by the build-items for add-opens and add-exports.

Which strategy to pick for Dynamic module reconfiguration?

The option using jdk.internal.module.Modules is relatively straightforward, seems supported (at least for the time being) and doesn't require an agent, so it is a strong contender for being "the" strategy in the long term.
However, it does require a JVM flag; this shouldn't be a problem - as we enumerated above all (?) entry points, we should be in a good position to ensure such flags are being set.
The self-attach agent mode, on the other hand, doesn't require a flag; a flag is recommended to avoid a warning, but with Java 25 the flag isn't strictly required. Additionally, there is the case where we already have our tooling explicitly attach our own agent, which we could piggyback on.

So my proposal is to combine multiple such strategies, depending on what's available, to ensure the experience is good, even though we might have forgotten, or not yet patched, some entry point, and allow us to make iterative progress while getting early feedback. Essentially this implies a preference to try using the opened modules, but a fallback on using an existing (already attached) agent instance, or ultimately to self-attach an agent when nothing else worked.
The fact that self-attachment of an agent will possibly be removed is not concerning me, as the usefulness of this particular strategy is in the ability to transition to this new system while the preferred strategies haven't been fully activated on all entry points. While we make progress, build tools and launchers get adapted - as soon as we have higher confidence of the transition being complete, we can simplify our strategies and perhaps use a single, straightforward invocation into jdk.internal.module.Modules, and soon remove the not-quite-recommended approaches, and remove the need for the self-attaching agent well before the time in which such options are no longer allowed by a new Java spec revision.

Follow-up tasks to polish:

• deprecate JPMSExportBuildItem and transition usage to the new system
• eventually simplify our strategies to reconfigure the modules
• better integrate with the static extension metadata (today it only expressed dev-mode jvm properties, might be useful to express more needs in such a static, readable format)
• verify CLI actions have been covered too

[dmlloyd]

Regarding "Future proofing for production mode: introduce generation of launcher scripts" - once you have access to jdk.internal.module.Modules, you can add any flags at run time (including but not limited to enable-native-access), so you still only need the one flag to be passed to the launcher itself.

We're not using Netty 4.2+ yet, but this is certainly the plan, and we won't be able to configure --enable-native-access in the same way as we do with add-opens and add-export, as the Jar Manifest doesn't allow for this aspect to be handled

As a point of fact, the JAR file format does indeed allow you to specify enable-native-access in a way that is similar to add-exports or add-opens:

Enable-Native-Access: Enables native access for all code on the classpath (including the code in the executable JAR itself). The only supported value is ALL-UNNAMED; no other module name can be given. It is equivalent to running with --enable-native-access ALL-UNNAMED.

But again if we have Modules then we can just set it ourselves.

So my proposal is to combine multiple such strategies, depending on what's available, to ensure the experience is good, even though we might have forgotten, or not yet patched, some entry point, and allow us to make iterative progress while getting early feedback.

I think it makes sense to use fallback strategies to make the behavior more ergonomic. By providing one option to the launcher, the use case would be solved; however, in the case where it wasn't done or cannot be done yet, falling back to self-attach for now (maybe with a log message) seems reasonable to me.

Let's be sure that we are not effectively making this into a public runtime API to crack all module protections though. At the very least, this would become very brittle if and when we modularize the runtime (since everyone's module identities would change). And of course there are some security benefits that we'd be losing in this case as well.

[Sanne]

As a point of fact, the JAR file format does indeed allow you to specify enable-native-access in a way that is similar to add-exports or add-opens:

Awesome, thanks for correcting me - that's great news! I'll amend the proposal above.

Let's be sure that we are not effectively making this into a public runtime API to crack all module protections though. At the very least, this would become very brittle if and when we modularize the runtime (since everyone's module identities would change). And of course there are some security benefits that we'd be losing in this case as well.

For these reasons I'm specifically suggesting that we should NOT use such capabilities in any production mode; and that's why I'm suggesting we should rather move to generate launch scripts (soon - not necessarily right now) - to address such modularity needs and related like enable-native-access, but also any other needs we might have in the future. Just as a wild and unlikely example, I remember at some point one our libraries needed a compiler directive to be set.. not hoping to see such a problem again, but I think it would be good for our extensions to have such capabilities, in case.

I'll add to my checklist to make sure that all my new code is affecting -deployment modules only.

[dmlloyd]

I think generalizing argument handling as a whole is a significantly larger scope than only narrowing the problem to add-exports, add-opens, and enable-native-access. And I am certain that we will in the future have a (non-public) runtime API to dynamically manage these three things as well, so don't bake in the idea that this will not happen.

[aloubyansky]

AFAIU, test and dev modes would be supported in the same way, correct? Basically, enabling runtime module reconfiguration on the launch and then actually applying extension-specific add-opens and add-exports in-process?

Also minor remark on "tests" vs "integration tests". Surefire is about @QuarkusTest kinds of tests. @QuarkusIntegrationTest is for launching a testing an already built prod app.

[Sanne]

AFAIU, test and dev modes would be supported in the same way, correct? Basically, enabling runtime module reconfiguration on the launch and then actually applying extension-specific add-opens and add-exports in-process?

Yes the are very similar because both would use the same approach to apply runtime reconfiguration. We still need to look at them separately though to ensure the hooks are triggered in the right place, and the JVM startup for both of them has the required JVM parameters set.

Also minor remark on "tests" vs "integration tests". Surefire is about @QuarkusTest kinds of tests. @QuarkusIntegrationTest is for launching a testing an already built prod app.

+1. This reminds me I have forgotten to look into IDE-initiated launching of tests - thanks!

[roamingthings]

As I pointed out in #50422 I want to mention that from my current understanding AWS Lambda Java 25 runtime does not support the --add-opens option. I'm not sure if using the enable-native-access option works in this case.

[Sanne]

@patriot1burke would you have suggestions about the use of JAVA_TOOL_OPTIONS to specify modules options on AWS Lambda? Did you see any problems with it - or what would you suggest us doing?

[roamingthings]

After further investigation using --add-opens=java.base/java.lang=ALL-UNNAMED (with = after --add-opens) seems to work.

[Sanne]

Excellent, thanks for confirming! Yes using the = is the recommended format, it's less so prone to confuse intermediate tools about different arguments.

[Sanne]

And just to be clear - when you say that it seems to work, you mean you have it set as part of JAVA_TOOL_OPTIONS, right?

[roamingthings]

Yes. I did set it yesterday and did not see any immediate error. The Lambda function started without throwing an error as before. However, when things like this happen I'm a bit cautious with final statements. Initially I've just copied the argument (without =) because I've seen it in various locations. Back than I did not no about the = variant and also didn't find this option when searching the web. It has been the AWS support that pointed me into this direction.

[gsmet]

The native access worries me a bit. From what I can see, the only option if we define it in the Manifest is to allow all modules to have native access. Isn't it dangerous long term?

Granted, we are not modularized yet but when we are, I'm not sure having native access enabled for all modules is a good thing for us.

[dmlloyd]

When we are modularized, we will not be using manifest properties for native access; for that kind of packaging, we will already have to use some kind of launcher script or similar, and then we can be surgical about how we grant access.

[jerboaa]

The native access worries me a bit. From what I can see, the only option if we define it in the Manifest is to allow all modules to have native access. Isn't it dangerous long term?

Agreed. This is true for all of them (being dangerous long-term). --add-opens, --add-exports, --enable-native-access. Creating a module-shim becomes increasingly attractive.

[Sanne]

I can't deny that all these options are inherently potentially "dangerous", but in most cases libraries are using them as a transitory mechanism as modularization takes place.
On the other hand, I don't think we can just sit back and wait for the transition to be over, as it's going to take years more, and is probably always going to be somewhat useful for us to be able to control such things.

Let's remember that yes it's potentially dangerous, but it would be our choice (or the extension maintainer's choice) to decide to use such a flag. I think we can all agree that such flags should not be set lightly.

Even the JVM developers took this pragmatic choice - I'm confident that they don't want people to abuse them, but the options have been made available for exceptional circumstances. What we're doing here, is to make it possible to use such capabilities, should such exceptional circumstances materialize with our supported extensions.