Metrics v4 draft overview

[lowlighter]

Metrics v4 - Draft

See #1542

metrics will undergo significant changes in next release in order to be more performant, portable, ergonomic, powerful and maintainable.

It will switch to deno runtime and will be rewritten in TypeScript, and the way to pass options will change entirely.

New concepts and usage

metrics will still be based on plugins, but will also introduce actions and transformers. Additionally, templates will now be merged with plugins, and the presets will offer more possibilities

The action will require a config file instead of accepting directly inputs.

Presets

# Presets will now be "contexts" that define the default variables
presets:
  # The default presets is the one used by all plugins/transformers when none is specified
  # It's mostly options that are not plugin-specific, such as token, handle (login), timezone, etc. 
  default:
    plugins:
      token: ${{ secrets.token }}
      handle: octocat
      timezone: Europe/Paris
      logs: info
  # You can create additional presets like below, for instance one for your work account
  work:
    plugins:
       api: https://github.my-company.com
       token: ${{ secrets.token_company }}
       handle: octodog
  # Or for different repositories
  my-repo:
     plugins:
       handle: octocat/hello-world
       entity: repo
       template: repository
  # Or even use remote ones
  community: https://raw.githubusercontent.com/lowlighter/metrics/presets/community/preset.yml

Plugins, actions and transformers

# Plugins are now ordered directly according to the order you define them
plugins:
  # To call a plugin, pass its "name" and configure it with "args"
  # Unless overriden, context is inherited from "presets.default", so the target user here is "octocat" for example
  - name: isocalendar
    args: 
      range: current-year
      colors: winter
    # Transforms will change the output, such as rendering GitHub emojis and formatting ths SVG XML
    transforms:
      - name: render.gemojis
      - name: format.xml
  # A same plugin can now be executed multiple times, and with different contexts
  # Here we could use the "presets.work" that is configured with a different account and token
  # Like explained before, the context is inherited, so here the target user would be "octodog"
  - name: isocalendar
    args: 
      range: current-year
      colors: winter
      preset: work
      timezone: Europe/London
  # You will also be able to use custom plugins, or even specific versions of a plugin
  - name: https://raw.githubusercontent.com/lowlighter/metrics/v4.0/source/plugins/isocalendar/mod.ts

  # ... actions

Actions

  # plugins ... 

  # Actions are "special plugins" (denoted with a "@") that have side effects
  # For example, @render will merge the output of previous plugins into a single SVG
  - name: @render
    args:
       format: svg
    transforms:
     - name: svg.resize
  # And @publish.commit will commit the output on the repository
  - name: @publish.commit
    args:
      filename: metrics.svg
      branch: metrics 
  # It'll also be possible to reuse the same data within the same flow to publish it into differents forms, 
  # without making further API requests
  - name: @render
    args:
       format: json
  - name: @publish.commit
    args:
      filename: metrics.json
      branch: metrics 

Web server

The web server will behave similarly, except that presets cannot be defined on the fly (they need to be defined by the administrator).
The syntax will ba a lax-YAML one:

https://metrics.dev/octocat?plugins=[{name:isocalendar, args:{range:full-year}}, {name:@render}]

Privacy and security

The new version will retest all plugins with GitHub fine-grained token, in order to make sure they're compatible (and are able to properly handle missing scopes without dying) to give users more control on how their data are accessed and processed.

As it'll be based on deno, security flags will superseed extras.features option and be even more powerful.
For example you will be able to run metrics with --allow-net=api.github.com --allow-write=/tmp --allow-run=git,licensed to ensure your data are not leaded anywhere.
You'll be able to run external plugins (and even official ones) with a peaceful mind.

Portability

The new engine is designed to be extremely versatile and can be run more easily as standalone.
It'll be able to be run as a GitHub action and server (like currently), and a proper CLI will be offered later one (maybe even a single binary thanks to deno compile, but to be seen)

The new engine is expected to uncouple the dependencies on base/core plugins to not only make rendering faster, but also to make it possible to be able to run metrics on serverless services - such as Vercel or Deno deploy - for most plugins (some time/cpu intensive ones, or binaries-based one probably won't make the cut though).

Setting up metrics will also be more easy, as the only requirement will be to install the deno runtime (single executable).
As plugin dependencies are URL based, native tree-shaking will also users to not download all dependencies requiring minimal setup and configuration.

Reusability

metrics will also be designed so that third-parties app and scripts can plug to it directly:

import isocalendar from "https://raw.githubusercontent.com/lowlighter/metrics/v4.0/source/plugins/isocalendar/mod.ts"

const data = await isocalendar.run({token:Deno.env.get("GITHUB_TOKEN")})

This will (hopefully) increase contributors engagement and expand the metrics ecosystem to make it more accessible and maintainable, in the hope that this project can live on even with me not being around

Also, thanks to decentralized dependencies, plugins authors will now be able to load any external dependency (or even load a different version of a dependency).
Since deno only load dependency at upon usage, they won't impact performance and size of the app.

For example the RSS plugin:

// Imports
import { is, Plugin} from "@plugin"
import RSS from "y/[email protected]"

/** Plugin */
export default class extends Plugin {

  /** Name */
  readonly name = "🗼 Rss feed"

}

Stability and new features

Plugins will be retested as they're being migrated, and they'll be simplified.
Templates will also be simplified/normalized in order to be both lightweight and easily editable using injected CSS / JS (similarly to extras.css and extras.js options)

New options and features will also be progressively added during this refactor (see migration guide below)

Inputs and outputs validation

Data validation will be delegated to Zod, which is able to both validate content at runtime but also infer typing during developpment. In addition, metrics will start normalizing and validating outputs, which mean that plugins results can be safely used in other applications.

metadata.yml will be removed, but documentation will still be generated (probably using zod-to-json-schema)

Below is an example of data validation looks like:

/** Plugin */
export default class extends Plugin {

  /** Name */
  readonly name = "📅 Isometric commit calendar"
 
  /** Inputs */
  readonly inputs = is.object({
    range:is.union([is.literal("half-year"), is.literal("full-year"), is.literal("current-year"), is.coerce.number().int().min(1970)]).default("half-year").describe('Time range. Use `"half-year"` to display last 180 days, `"full-year"` to display last year, `"current-year"` to display this year, or use a `number` to display a specific year.'),
    colors:is.enum(["auto", "halloween", "winter"]).default("auto").describe("Color scheme"),
  })

  /** Outputs */
  readonly outputs = is.object({
    colors:is.enum(["default", "halloween", "winter"]).describe("Color scheme"),
    average:is.number().min(0).describe("Average number of commits per day"),
    max:is.number().int().min(0).describe("Highest number of commits in a single day"),
    streak:is.object({
      max:is.number().int().min(0).describe("Highest number of consecutive days with commits"),
      current:is.number().int().min(0).describe("Current number of consecutive days with commits"),
    }).describe("Streak statistics"),
    range:is.object({
      start:is.date().describe("Start date"),
      end:is.date().describe("End date"),
    }).describe("Displayed range"),
    weeks:is.array(is.object({days:is.array(is.object({count:is.number().int().min(0), level:is.number().min(0).max(4)}).nullable())})).describe("List of weeks"),
  })

  /** Action */
  async action({handle}:Plugin["context"]) {
    const {range, ...inputs} = await this.inputs.parseAsync(this.context.args)
    /* ... */
    return result
  }

}

Improved testing and coverage

Currently, testing metrics is pretty painful as we need to rebuild the docker container, spawn a fake web instance and simulate different environments against it. It's hyper slow (~30 min) which is also frustrating.

Thanks to the engine rework, it's now insanely faster, and we're also able to provide code coverage.
Data mocking has also been reworked to be way less hacky than currently, making it also more robust for testing:

TypeScript and linting will prevent basic errors.

PRs will be integrated better with serverless application to offer proper previewing.

What for now ?

There is no ETA for this, but a pre-release will eventually come out as I'm rewritting the engine.
v3.35 will probably just be replaced by v4

Once the engine is rewritten with a few plugins, the rest of features will be progressively migrated until v4 is on par with v3.
I'll accept PRs for plugins migrations (and if you're interested to eventually be a maintainer in the future, let me know)

metrics.lecoq.io will probably be migrated to either Deno deploy or Vercel, and even if @main/@master/@latest are bumped to v4, a "configuration transpiler" will be implemented (which will also be offered to users to convert to the future new config format).

I'm still experimenting with what can be done, but I'm open to feedback.

The migration guide below will also act as a progress list

Thanks for reading up until there 💕 !

[JohnyP36]

The migration guide below will also act as a progress list

I don't see one. Or am I blind?

a "configuration transpiler" will be implemented (which will also be offered to users to convert to the future new config format).

As long as you are clear how I have to change my workflow file (.yml) I am happy with the changes.

[lowlighter]

The migration guide below will also act as a progress list

I don't see one. Or am I blind?

Yeah I was still working on it, it's now available below 😅

As long as you are clear how I have to change my workflow file (.yml) I am happy with the changes.

Even if it'll be a major release, it is planned to somewhat be compatible with all current workflows (the configuration transpiler will probably be run on the fly before running the engine) so it should be transparent for most action users.

The only things expected to break are external plugins/templates, and server setups/users.
Actions users probably won't be impacted that much apart from maybe a deprecation message and a lot of warnings

[Andre601]

It probs is worth for me to put the mastodon plugin on hold until a first working version of v4 is usable, so that I can move things over...
And maybe by that time mastodon will have added some qol features beneficial for this plugin (Like returning the untransformed source string)

[lowlighter]

Yeah probably, normally it'll be a lot easier to develop since it won't be necessary to install all dependencies, nothing to config as this time everthing setting will have proper default, and the ci/cd will be a lot faster and better

[gherbetto]

Sounds great! I hope I don't break my head figuring out the TS. I'm still a newbie.

[lowlighter]

It probably won't, usually I like writing "implicit TypeScript" (i.e. letting TS infering the typing) so the syntax is usually really similar to JavaScript 🙂

This is not finalized, but for example the RSS plugin look like below currently:

// Imports
import { is, Plugin } from "@plugin"
import RSS from "y/[email protected]"

/** Plugin */
export default class extends Plugin {

  /** Import meta */
  static readonly meta = import.meta

  /** Name */
  readonly name = "🗼 Rss feed"

  /** Category */
  readonly category = "social"

  /** Supported entities */
  readonly supports = ["user", "organization", "repository"]

  /** Description */
  readonly description = "Displays entries from a RSS feed".trim()

  /** Inputs */
  readonly inputs = is.object({
    feed: is.string().url().describe("RSS feed (e.g. `https://news.ycombinator.com/rss`)"),
    limit: is.number().int().min(1).nullable().default(4).describe("Display limit (use `null` to disable)"),
  })

  /** Outputs */
  readonly outputs = is.object({
    name: is.string().describe("Feed name"),
    description: is.string().describe("Feed description"),
    entries: is.array(is.object({
      title: is.string().describe("Entry title"),
      link: is.string().url().describe("Entry link"),
      date: is.date().nullable().describe("Entry date"),
    })).describe("Feed entries"),
  })

  /** Action */
  async action({args}:Plugin["context"]) {
    const {feed, limit} = await this.inputs.parseAsync(args)
    const content = await this.fetch(feed, {type:"text"})
    const {title:name, description, items} = await (new RSS()).parseString(content)
    let entries = items.map(({title, link, isoDate:date}) => ({title, link, date:date ? new Date(date) : null}))
    if (limit)
      entries = entries.slice(0, limit)
    return {name, description, entries}
  }

}

So apart from a few keywords like readonly and the Plugin["context"] typing, you can see it's mostly similar to regular ESM JavaScript (like when using .mjs with NodeJS). There's an import map which map some path to make it easier (like y/ redirects towards https://esm.sh, std/ to deno standard library and @* for internal paths for example)

What makes this new engine really nice to work on is the Zod support (renamed is to make it more natural), as it acts as both a type inferer, validator and schema builder (which can be later on converted to documentation). For example in:

const {feed, limit} = await this.inputs.parseAsync(args)

TypeScript is capable of automatically infering that feed is a string (in addition to be a validated URL) and limit a number | null

So all in all, I think it'll be a lot easier to developp (and maintain) since you don't really need to focus on user validation (although the old metadata.yml was doing the job ok, but wouldn't allow more complex typing like union or objects, and even for third-parties the output will now be guaranteed to match specified schema), and you'll only have to implement within the action() method, as everything else will be handled by the metrics engine

[RisingSunLight42]

Seems really dope ! If you need any help to migrate plugins, don't hesite to ping me or something. To be honest, the current architecture was scaring me a bit, but seeing TypeScript coming to the scene makes me wanting to contribute a lot to metrics and become a maintainer maybe ? but I'm not sure if I'm qualified enough ^^'.
I'll keep an eye everything !