[RFC] Proposal: Parametrized tasks

[lefou]

Proposal for "Parametrized Tasks"

I propose a new type of tasks with the following requirements:

• Behave as normal named tasks
• can be called from CLI
• are cached
• can be voluntarily persistent
• Have a parameter list (like anonymous tasks)
• Task.dest is derived from the actual parameters (like cross-values are handled)

There should be no need to provide the actual parameter values at planning time. It should be allowed to use task results as parameters.

Use cases

• runMain for ScalaNativeModule runs a JVM launcher #5461 - In Scala Native for a very long time we didn't have a runMain due to the fact that for each selected main, we need to re-run the linker with different settings. To run a given class Foo, we need to run the linker with Foo. But to know if Foo is a valid and linkable main class, we need to run compile. Currently, we don't know the value for valid main classes beforehand, hence we can't properly cache specific results. With cross-tasks, we could make the mainclass parameter of runMain a cross-value, but the valid set of cross-values would need to come from a task. (There is now a partly working solution, but without caching.)

• Caching specific tests. In current Mill, we already split test suites and run tests in multiple threads. If we could map the current model to parametrized tasks accepting the test name as parameter, we could have a task for each test class. Since those tasks are cached, we would get almost everything we currently do manually for free. Work distribution to threads, caching and invalidation of tests, reporting of progress. But since we don't know the valid test classes beforehand, the provider of those parameter values needs to be a task.

Notes

Since I had this idea of parametrized tasks, I've came across many situations, where these would be a great tool. Hence, I think this is valuable for other users too.

I proposed this idea before (here and here) but as it was piggy-backing on another proposal, I never got any feedback.

Implementation wise, parametrized tasks are a mixture of Task and Command (or Task.Anon). The most prominent change is the fact, that the parameter take part of the storage location of the cached state and the destination directory.

// Not sure we need the `.Parametrized`, since it's implicit via the param-list
def foo(version: String): T[PathRef] = Task.Parametrized {
  os.write(Task.dest / "foo.txt", version)
  PathRef(Task.dest)
}

Given the parameter value 1.0, the out layout may look like this:

out/
  foo.param/
    1.0.json
    1.0.dest/
      foo.txt

Usage examples:

> mill show foo 1.0
1] show
"ref:v0:.../out/foo.param/1.0.dest/foo.txt"

> mill showNamed foo 1.0 # using the classic []-notation
1] showNamed
"foo[1.0]": [
  "ref:v0:.../out/foo.param/1.0.dest/foo.txt"
]

> mill showNamed foo 1.0 # using the dot-notation since Mill 1.1.0-RC4
1] showNamed
"foo.1_0": [
  "ref:v0:.../out/foo.param/1.0.dest/foo.txt"
]

> mill show foo[1.0] # Shall we allow this?
1] show
"ref:v0:.../out/foo.param/1.0.dest/foo.txt"

The question is also, if we want to allow non-parameterized tasks with the same name? If we do, it could be used to run a parametrized task with a predefined set of values like follows:

def foo(version: String): T[PathRef] = Task.Parametrized {
  os.write(Task.dest / "foo.txt", version)
  PathRef(Task.dest)
}

def foo: T[Seq[PathRef]] = Task {
  Task.traverse("1.0", "2.0", "3.0")(foo(_))()
}

Allowing this would mean, we need a way to disambiguate the parametrized and non-parametrized tasks on the CLI, otherwise the call mill foo would be ambiguous. Once way to solve this would be to use the []-notation to pass the parameters to the task: mill foo[1.0] like we currently do for cross-modules.

Closed set of parameter values

A very important property is the dynamic nature of the parameters. That makes is easy to have cache results without knowing any possible value beforehand. But being dynamic also means, we can't compute a total set of possible values, so we also can't use placeholders to refer to all values.

Changelog

• 2025-12-19: Initial proposal
• 2025-12-19: Update the out-layout (append a .param) to avoid collisions with sub-modules

[vaslabs]

if I understand this correctly, this will simplify the build variants on android too, for every task that depends on android debug, if it can be passed as a parameter + default the value of androidIsDebug (for the example of debug, but we can rework the whole thing for any user defined variants)

For example one would be able to do ./mill app.androidInstall release or ./mill app.androidInstall debug or any other profile they can think of naming.

About the caching/task directory structure:

is there a chance foo/version with a task foo and any string that's allowed to be passed can conflict with

object foo { def version: T[]] } , since the parameter of the function would break the guarantee of unique class/method paths?

[vaslabs]

maybe instead of 1.0.dest it's something like 1.0.paramdest

[lefou]

Good catch. To really avoid any possible collision, we need to have a dedicated name for <name>.json and <name>.dest paths, or just a common dir. Maybe, <name>.param in the spirit of <name>.super.

So we have:

out/
  foo.param/
    1.0.json
    1.0.dest/
      foo.txt

[lefou]

Update the proposal.