Support Bzlmod and add rules_scala to bazel-central-registry

[sfc-gh-pbennes]

Greetings!

It looks like rules_scala isn't part of the Bzlmod effort or added to bazel-central-registry yet. I've opened an issue both here and in https://github.com/bazelbuild/bazel-central-registry to request it: bazelbuild/bazel-central-registry#522

[chrislovecnm]

Just a note to anyone that starts the work native.register_toolchains is not supported when using bzlmod.

[sluongng]

@chrislovecnm it's still available here no? https://bazel.build/rules/lib/globals/module#register_toolchains

[chrislovecnm]

@sluongng When I last tried it, it did not work when using it inside an extension. In rules_python we are skipping the native call:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/python/repositories.bzl#L569

And build the toolchains here:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/python/extensions/private/pythons_hub.bzl#L56

We build the toolchains and use the following:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/MODULE.bazel#L49

[sluongng]

Yeah you would only want to prepare the toolchain (as a new repository) inside the extension I think. Registering it must be called from the MODULE.bazel file.

So native.register_toolchains call from the extension's starlark does not work, but having the extension create @pythons_hub//:all and then MODULE.bazel calling register_toolchains("@pythons_hub//:all") worked. 👍

[mateuszkuta256]

I prepared a minimal bzlmod support here and gonna split it into a few PRs
TODOs after merging it:

• write documentation on how to use rules_scala with bzlmod (some properties like SCALA_VERSION will probably be configurable via repo-env)
• prepare release notes
• contribute repo to the Bazel Central Registry
• add tests that the project builds using both WORKSPACE and MODULE.bazel
• migrate test/external workspaces to blzmod too

[pcj]

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

[mateuszkuta256]

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

hi, unfortunately I'm working on other things now and won't continue with this PR in the foreseeable future
I remember this migration was on hold because interfered with 'cross compilation support'
It looks like great progress was done it this area, so maybe someone can already take over the PR

[mbland]

I'd like to take on the task of Bzlmodifying this repo through a series of pull requests.

I've already created a (mostly) working branch in my own fork. Though I saw a couple draft pull requests here, I ended up taking a different approach and got it mostly working. In fact, I did exactly what @sluongng suggested in #1482 (comment). (I didn't notice this comment before just now—I might've read it, but not understood it at the time—but I did study rules_python and rules_go, and ended up doing exactly that.)

There are still issues I need help to address (recorded in some of the commit messages), I didn't strictly maintain WORKSPACE compatibility (fixable), and I only tested against Scala 2.13. But I can start teasing chunks out of it in a methodical fashion, to ensure that I maintain WORKSPACE and cross-version compatibility.

For an example of what it looks like from a client perspective, here's what the MODULE.bazel stanza from my local EngFlow/example repo looks like (as opposed to the current stanza, explanatory comment notwithstanding):

bazel_dep(name = "rules_scala", repo_name = "io_bazel_rules_scala")
local_path_override(
    module_name = "rules_scala",
    path = "../../bazelbuild/rules_scala"
)   

scala_dev_deps = use_extension(
    "@io_bazel_rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
    dev_dependency = True,
)   
scala_dev_deps.toolchains(
    scalatest = True,
)

So if folks are game for me to do this, I'll start carving off pieces as separate pull requests, and we can resolve any outstanding problems in the process.

cc: @BillyAutrey @jayconrod @benjaminp @TheGrizzlyDev

[mbland]

A quick update for visibility: I'm very close to having the Bzlmodified rules_scala passing 100% of the tests, down to the last couple of failures:

• I need to replace the bind() calls from twitter_scrooge with alias() or some such to get test_version.sh to pass.
• I need to fix the Scala 3.1.2 and Scala 3.3.3 failures for the Scalafmt targets from test/shell/test_cross_build.sh (corresponding to the test3 and library3 test cases in test_scalafmt()).

I also need to get test_lint.sh working again, and update the dt_patches repos for Bzlmod. But the following all pass:

• test_rules_scala.sh
• test_reproducibility.sh
• test_examples.sh
• test_coverage.sh

Hopefully I can get these fixed up today, and I'll start peeling off the next pull request or two. And thanks to @simuons and @liucijus for reviewing and merging #1619 and #1620 already.

[mbland]

D'oh! Didn't mean for #1722 to close this yet. I guess my commit/PR message must've triggered that.

@simuons @liucijus Do you want to reopen this until we publish the module, or open a new issue?

[mbland]

Weekly update!

Overview

#1722 is in, and MODULE.bazel exists on master. Praise Kier!

No, actually: Praise @simuons and @liucijus! 🙂 Not just for landing #1722, but for following up so quickly on all the refinements I've sent after that.

Now we're down to just three pull requests, all orthogonal to one another, and no pending branches! As noted below:

• Add toolchain options API to WORKSPACE and Bzlmod #1730 contains a minor API change affecting both WORKSPACE and Bzlmod.
• Use the Publish to BCR reusable GitHub workflow #1731 adds the Publish to BCR workflow, now that it's ready for prime time.

Once those two land (plus maybe #1729), we're ready to release and publish this thing and close out here.

Merged

• Enable Bzlmod #1722
• Add src/... to test_rules_scala, fix Windows break #1724
• Replace @scalafmt_default with toolchains target #1725
• Establish minimum compatible Bazel, dep versions #1726
• Bump Bazel, rules_*, Guava, Go, golang tools, etc. #1727
• Remove Label wrappers for target string literals #1728

Open

• Add test_dependency_versions, update test_runner #1729
• Add toolchain options API to WORKSPACE and Bzlmod #1730
• Use the Publish to BCR reusable GitHub workflow #1731

Pending branches

None! That's it! No more!

Preview branches

As always, if you want to try everything out before the next release, please only use either my bzlmod or bzlmod-bazel-8 branches. Both contain all the changes from the open pull requests.

Updated scala_toolchains() and scala_deps module extension APIs

I've added what I call "toolchains options support" in #1730. This means that scala_toolchains() accepts either boolean or dict arguments for specific toolchains, and the scala_deps module extension generates them from tag classes. This yields similar changes in both the WORKSPACE and Bzlmod APIs, specifically:

• Add toolchain options API to WORKSPACE and Bzlmod #1730 adds toolchain options support to the scala_toolchains() parameters scalafmt, scala_proto, and twitter_scrooge, and to the scalafmt tag class.

• Add toolchain options API to WORKSPACE and Bzlmod #1730 eliminates the scalafmt_default_config, scala_proto_options, and twitter_scrooge_deps option parameters from scala_toolchains().

• The scala_proto and twitter_scrooge tag class APIs haven't changed, but now they and the scalafmt tag class share a common implementation.

• All these toolchains options dicts are aligned with the underlying toolchain macro arguments, making the relationship between them clear.

• Thanks to the shared implementation, it will be easy to add options support to additional toolchains over time. These updates will require only a minor release, not a major one.

For example, under WORKSPACE, enabling scalafmt and setting a custom config changed like this:

# Before
scala_toolchains(
    # Other toolchains settings...
    scalafmt = True,
    scalafmt_default_config = "//path/to/my/custom:scalafmt.conf",
)

# After
scala_toolchains(
    # Other toolchains settings...
    scalafmt = {"default_config": "//path/to/my/custom:scalafmt.conf"},
)

scalafmt = True will still work if you want to use the default options, but it's implied if you supply your own options object.

Under Bzlmod, the scala_proto tag class changed thus to align the interface with setup_scala_proto_toolchains from //scala_proto:toolchains.bzl:

# Before
scala_deps.scala_proto(
    options = ["grpc"],
)

# After
scala_deps.scala_proto(
    default_gen_opts = ["grpc"],
)

scala_deps.scala_proto() will work in this case, since under the hood, {"default_gen_opts": ["grpc"]} is the default.

Please see the description of #1730 for further details on all the good things.

This came about after @simuons asked in #1722 (comment) to replace the boolean flags in scala_deps.toolchains() with individual tag classes. The thought being, each toolchain can potentially define its own set of attributes via its own tag class. (I mentioned all this in last week's update.)

In the same comment, he also asked whether we could replace toolchain macros (invoked by users in BUILD files) entirely with a module extension. Though I noted this won't be trivial to do, and shouldn't hold up the release, I later experimented with the toolchain macro replacement idea. I'm not yet near achieving that goal, but my experimentation yielded the immediate improvements described above and in #1730.