server, agent: github app installation clone support

[benbroadaway]

Minimal support for cloning repos from GitHub app installation credentials (use private key to generate JWT -> access token).

System-provided auth is defined in server and agent configs in the github.appInstallation section. See default conf for description.

User-provided app installation info (e.g. client id, private key) must be in a single-value secret containing JSON-formatted data like:

{
  "githubAppInstallation": {
    "apiUrl": "https://api.github.com",
    "clientId": "the-client-id",
    "privateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowI..."
    "username": "x-access-token",
    "urlPattern": "(?<baseUrl>github.com)/owner/.*"
  }
}

apiUrl and username are optional and default to https://api.github.com and x-access-token, respectively.

Caching

In order to keep traffic to the GItHub API down, tokens are cache (Default 50 minutes) in memory for re-use on subsequent clones/fetches for the same repo. While clones don't count against the rate limit, there's benefits to caching the tokens. It may take 250-500ms typically to generate a token before cloning which adds up in a busy environment, and cached tokens take <1ms to access.

sequenceDiagram
    participant GitHub
    participant concord-server
    participant server-token-cache
    participant concord-agent
    participant agent-token-cache

    concord-server->>+server-token-cache: get token for repo
    server-token-cache-->>server-token-cache: cached installation token for repo
    server-token-cache-->>server-token-cache: validate token expiration
    server-token-cache-->>+GitHub: or Generate new token
    GitHub-->>-server-token-cache: new 1-hour installation client token
    server-token-cache->>-concord-server: installation token
    concord-server->>+concord-server: clone repo
    concord-server->>+concord-agent: dispatch workflow

    concord-agent->>+agent-token-cache: cached installation token for repo
    agent-token-cache-->>agent-token-cache: validate token expiration
    agent-token-cache-->>+GitHub: or Generate new token
    GitHub-->>-agent-token-cache: new 1-hour installation client token
    agent-token-cache->>-concord-agent: installation token
    concord-agent->>+concord-agent: clone repo
    concord-agent->>+concord-agent: start workflow runner

Loading