Support coverage from "dart test" in lcov format to match "flutter test", and make the output locations more consistent

[DanTup]

VS Code has APIs for providing code coverage during test runs (which will then be shown in the editor). I've a long-standing request to implement this in Dart-Code (Dart-Code/Dart-Code#2860).

I started looking at this today but found that dart test --coverage and flutter test --coverage are very different. Flutter takes a single filename (--coverage-path) and produces data in the lcov format, but Dart (or pkg:test?) outputs a JSON format (which I'm not sure about - maybe is VM-specific?).

The docs for pkg:test say that the coverage can be converted with pkg:coverage, but for Dart-Code this makes things much more complicated (I try to avoid just pub global activateing things on a users machine, and it adds another level of potential versioning that is different to the SDK).

It would be really nice if we could pass similar flags to dart test and flutter test that results in them producing the same format of output, in the same kind of location (eg. we can provide a filename or a folder to each).

[DanTup]

FWIW, for Dart-Code, the actual format probably isn't that important - I think I'm going to need to parse it into VS Code data structures anyway - but I would like to only need to parse one format.

If flutter test is just using package:coverage to convert to lcov, then having it just have an option to dump the original format (matching dart) would also work for me - as long as that format is stable - I can't use package:coverage to parse it as I need to read it in typescript from Dart-Code.

[bkonyi]

@liamappelbe can you look into this and coordinate with the owners of package:test to figure out if this is something we can do?

[liamappelbe]

Both flutter test and dart test use package:coverage under the hood. So it should be fairly simple to add a flag to dart test to get it to output an lcov file. I'll look into it.

@DanTup Another option you could try in the meantime is package:coverage's test_with_coverage tool. It runs tests and outputs an lcov file: dart pub global run coverage:test_with_coverage -- <args to pass to dart test command>. That's the tool I use for coverage in all my github workflows. I've had a long standing note on my to-do list to unify the behavior of this and dart test --coverage.

[bwilkerson]

+1 to unifying the two. It will be more discoverable for users if the functionality is available from dart test.

[DanTup]

@liamappelbe

Another option you could try in the meantime is package:coverage's test_with_coverage tool.

I presume that either this requires coverage has been pub global activated, or that it would trigger a download of the package from Pub? Neither of these is ideal to code into Dart-Code because having Dart-Code activate packages is a bit weird (we have done it in the past for devtools and webdev, but I don't like it because it adds delays and more paths for errors), or it may introduce delays and rely on a network connection.

If we think we might get Dart/Flutter to be able to produce the same output, I'd prefer to wait for that than write code that will be removed. However, if that's complicated (or just not a priority so won't be done in the short/medium-term) then perhaps it's something to explore.