Skip to content

Conversation

ptgott
Copy link
Contributor

@ptgott ptgott commented Feb 10, 2025

Closes #5044
Closes #10350

One source of Teleport audit event data is the fixtures file we use for testing the audit event view within the Web UI. We usually update our test fixtures when we add an audit event, since we need to display new audit events as expected within the Web UI. This change adds a generator for the audit event reference documentation based on the test fixtures.

When we add a new event, we need to edit the formatters in makeEvent.ts so the event appears in the Web UI. For any event formatter with no test fixture, include the event, type, code, and description in the reference guide using data from the formatter, but don't include an example. Also remove any test fixtures from the guide that don't correspond to a formatter.

Also log events with no examples when running the generator so we can add test fixtures later on.

Implementation details:

  • Use vite to transpile the test fixture into a library that we can import into a short script.
  • Alphabetize reference sections by event type.
  • Ignore "Unknown" event descriptions, leaving these blank.
  • If one event type includes multiple possible codes, document each code in an H3 section.
  • Ignore instances of the same code beyond the first occurrence.

@public-teleport-github-review-bot

@ptgott - this PR will require admin approval to merge due to its size. Consider breaking it up into a series smaller changes.

Copy link

github-actions bot commented Feb 10, 2025

Amplify deployment status

Branch Commit Job ID Status Preview Updated (UTC)
paul.gottschling/5044-audit-ts d41791f 35 ❌FAILED paul-gottschling-5044-audit-ts 2025-10-10 12:22:27

@zmb3
Copy link
Collaborator

zmb3 commented Feb 10, 2025

The most complete, consistent source of Teleport audit event data is the fixture we use for testing the audit event view within the Web UI. We update our test fixtures every time we add an audit event, since we need to display new audit events as expected within the Web UI.

I'd argue that the makeEvent implementation is the most complete, consistent source. We can't render an event in the UI without updating makeEvent, but we can forget to update the fixtures.

In fact, here's some recent commits that did update makeEvent but failed to update the fixture:

@ptgott
Copy link
Contributor Author

ptgott commented Feb 11, 2025

The most complete, consistent source of Teleport audit event data is the fixture we use for testing the audit event view within the Web UI. We update our test fixtures every time we add an audit event, since we need to display new audit events as expected within the Web UI.

I'd argue that the makeEvent implementation is the most complete, consistent source. We can't render an event in the UI without updating makeEvent, but we can forget to update the fixtures.

In fact, here's some recent commits that did update makeEvent but failed to update the fixture:

Thanks! I am planning to:

  • Use the fixtures where they exist in order to include example JSON objects in the guide
  • If there's no fixture for an event code, use the type and desc fields of the corresponding formatters entry to create a partial section with no example

@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 36c791c to f14b702 Compare February 12, 2025 16:25
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 86ec6f6 to 01a930b Compare February 12, 2025 16:52
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch 2 times, most recently from 91c53fa to 9e8f842 Compare February 12, 2025 17:32
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 9e8f842 to eb38037 Compare February 19, 2025 15:40
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from eb38037 to b590de1 Compare February 24, 2025 18:01
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 7b5d43a to 58bc991 Compare April 22, 2025 15:17
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 58bc991 to 18dc744 Compare April 25, 2025 13:26
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 18dc744 to e72be95 Compare April 29, 2025 17:34
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from e72be95 to a0d8880 Compare April 30, 2025 20:21
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from a0d8880 to 581b047 Compare May 6, 2025 18:40
@ptgott ptgott temporarily deployed to docs-amplify May 6, 2025 18:40 — with GitHub Actions Inactive
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 581b047 to 7223dd9 Compare May 7, 2025 18:21
@ptgott ptgott temporarily deployed to docs-amplify May 7, 2025 18:21 — with GitHub Actions Inactive
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 7223dd9 to 85b69ad Compare May 13, 2025 20:17
@ptgott ptgott temporarily deployed to docs-amplify May 13, 2025 20:17 — with GitHub Actions Inactive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about using TypeScript in this package? It should make maintenance way easier. I have a WIP on r7s/gen-event-ts, I can spend some more time on it tomorrow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so r7s/gen-event-ts has TypeScript working, but at the end I realized there's little value in it since the stuff in the dist directory is not typed. We could copy the types by hand based on what's in web/packages/teleport/src/Audit/fixtures/index.ts and web/packages/teleport/src/services/audit/makeEvent.ts.

However, the overall question is: how do we make sure that a person working on product code in web/packages/teleport is not going to break docs/gen-event-reference?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the script was a part of the teleport package, perhaps we could skip the Vite step altogether and instead import makeEvent.ts and the other file directly?

Then TypeScript should surface compilation errors if someone changes makeEvent.ts in a way that's incompatible with the generator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good to me! I've started reorganizing the code in this branch, which I've opened off of paul.gottschling/5044-audit-ts. Would it make sense to change the new files to .ts and execute the gen-event-reference.ts entrypoint with tsx to generate the reference?

@zmb3 I know you've had some concerns about locating docs generator code next to production code in relation to another project—do those concerns apply here as well?

Copy link
Member

@ravicious ravicious May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to change the new files to .ts

Yeah! I added some more types and commented on what still needs to be done in #55002.

…and execute the gen-event-reference.ts entrypoint with tsx to generate the reference?

This is more tricky. I think there's no contraindications for having the docs gen code next to production code as long as the docs gen code doesn't bring new dependencies. 😏

So tsx and ts-node kind of out of the question. I tried to use native node --experimental-transform-types, but the problem is that we use custom paths in tsconfig.json which the native TypeScript support in Node.js doesn't understand.

Then I tried the tsconfig-paths package which we already use indirectly through Storybook. However, I wasn't able to get it to work.

I'll give Vite a try, this time it'd be used just to strip types from that index.ts file.

Edit: I managed to do this with Vite, more details in my draft PR (#55002).

Closes #5044
Closes #10350

One source of Teleport audit event data is the fixtures file we use for
testing the audit event view within the Web UI. We usually update our
test fixtures when we add an audit event, since we need to display new
audit events as expected within the Web UI. This change adds a generator
for the audit event reference documentation based on the test fixtures.

Previous attempts to generate an audit event reference drew from the
Teleport source, rather than test fixtures, using `AuditEvent`
declarations (#13615) and naming conventions (#38344), but
inconsistencies within the source meant that the resulting generator was
inadequate.

Implementation details:

- Use vite to transpile the test fixture into a library that we can
  import into a short script.
- Alphabetize reference sections by event type.
- Ignore "Unknown" event descriptions, leaving these blank.
- If one event type includes multiple possible codes, document each code
  in an H3 section.
- Ignore instances of the same code beyond the first occurrence.
Responds to zmb3 feedback.

When we add a new event, we need to edit the formatters in
`makeEvent.ts` so the event appears in the Web UI. For any event
formatter with no test fixture, include the event, type, code, and
description in the reference guide using data from the formatter, but
don't include an example. Also remove any test fixtures from the guide
that don't correspond to a formatter.

Also log events with no examples when running the generator so we can
add test fixtures later on.
@ptgott ptgott force-pushed the paul.gottschling/5044-audit-ts branch from 85b69ad to d41791f Compare October 10, 2025 12:17
Copy link

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​jest@​29.7.01001006896100
Addednpm/​@​babel/​plugin-syntax-typescript@​7.25.91001007089100
Addednpm/​@​babel/​parser@​7.27.01001008193100

View full report

@ptgott ptgott closed this Oct 15, 2025
@ptgott ptgott deleted the paul.gottschling/5044-audit-ts branch October 15, 2025 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: list of audit events is incomplete Document event codes and types. Produce JSON schemas for all events.

4 participants