docs: clarify token analytics install workflow (#50)

Co-authored-by: Rick <rick@tevpro.com>

Author: newbish

docs/hermes-token-analytics-install-runbook.md

@@ -43,6 +43,25 @@ Before you start, confirm:
 
 ## Step 1. Install the plugin into the Hermes checkout
 
+> [!WARNING]
+> This repository is a monorepo. The Hermes plugin is **not** at the repo root.
+> Do **not** install the repo root directly with `hermes plugins install <repo-url>`.
+> The actual plugin source lives at `plugins/hermes-token-analytics/`.
+
+### Install target options
+
+There are two valid install targets. They are not the same thing.
+
+1. **Bundled checkout plugin path**, for development inside a Hermes repo checkout
+   - target root: `~/.hermes/hermes-agent`
+   - resulting plugin path: `~/.hermes/hermes-agent/plugins/hermes-token-analytics/`
+2. **User plugin path**, preferred for a normal operator install
+   - resulting plugin path: `~/.hermes/plugins/hermes-token-analytics/`
+
+Preferred default: use the **user plugin path** unless you are intentionally working inside a bundled Hermes checkout.
+
+### Bundled checkout install
+
 From this repo root, copy the plugin into the target Hermes checkout:
 
 ```bash
@@ -67,6 +86,22 @@ What the helper does:
 - writes a compatibility shim at `plugins/observability/token_analytics/`
 - lets older `plugins.enabled` entries keep working during migration
 
+### User plugin install
+
+If you are doing a normal user-level install instead of editing a Hermes checkout in place, copy this repo subdirectory into:
+
+```text
+~/.hermes/plugins/hermes-token-analytics
+```
+
+Use the contents of:
+
+```text
+plugins/hermes-token-analytics/
+```
+
+The main operational point is simple: copy the plugin directory, not the monorepo root.
+
 ## Step 2. Enable the plugin in Hermes
 
 Enable the plugin by its path-derived key:
@@ -75,6 +110,12 @@ Enable the plugin by its path-derived key:
 hermes plugins enable hermes-token-analytics
 ```
 
+Then restart the gateway so Hermes reloads the plugin command surface:
+
+```bash
+hermes gateway restart
+```
+
 Then verify Hermes sees it:
 
 ```bash
@@ -84,6 +125,7 @@ hermes plugins list
 What you want to see:
 
 - `hermes-token-analytics` listed as enabled
+- the plugin commands available after the restart
 
 If the target install still references the old compatibility path, the shim also supports the legacy key:
 
@@ -156,6 +198,10 @@ What success looks like:
 - `show-config` shows the resolved endpoint, workspace fields, and redacted shared secret
 - `sync` posts successfully to `/api/ingest/hermes-usage`
 
+Important: a successful `hermes token-analytics sync` only proves the plugin works manually.
+It does **not** mean continuous reporting is active.
+Continuous reporting requires a cron job.
+
 If `sync` fails, stop here and fix config before adding cron.
 
 ## Step 5. Create the scheduled job
@@ -214,6 +260,19 @@ Confirm:
 - the schedule is correct
 - the cron-triggered run behaves the same as the manual `sync`
 
+## Production readiness checklist
+
+- [ ] plugin copied from `plugins/hermes-token-analytics/` into the intended install target
+- [ ] plugin enabled
+- [ ] gateway restarted
+- [ ] `hermes token-analytics doctor` passes
+- [ ] `hermes token-analytics show-config` is correct
+- [ ] `hermes token-analytics sync` succeeds
+- [ ] `hermes token-analytics install-cron-wrapper` run
+- [ ] cron job created
+- [ ] `hermes cron list --all` shows `token-analytics-sync`
+- [ ] one cron-triggered run verified
+
 ## Normal operating procedure
 
 Use this sequence for changes or repairs:

docs/hermes-token-analytics-plugin.md

@@ -4,20 +4,27 @@
 
 This integration is now a **Hermes-native Python plugin**, not a Node or bash sidecar.
 
+> [!WARNING]
+> This repository is a monorepo. The Hermes plugin is **not** at the repo root.
+> Do **not** install the repo root directly with `hermes plugins install <repo-url>`.
+> The actual plugin source lives at `plugins/hermes-token-analytics/`.
+
 Operator setup is:
 
-1. Set the Cloudflare Worker secret `HERMES_TOKEN_ANALYTICS_SHARED_SECRET`.
-2. Export the plugin env vars shown below.
-3. Validate the install with:
+1. Copy `plugins/hermes-token-analytics/` into the correct install target.
+2. Enable the plugin and restart the Hermes gateway.
+3. Set the Cloudflare Worker secret `HERMES_TOKEN_ANALYTICS_SHARED_SECRET`.
+4. Export the plugin env vars shown below.
+5. Validate the install with:
    ```bash
    hermes token-analytics doctor
    hermes token-analytics show-config
    ```
-4. Push one manual sync:
+6. Push one manual sync:
    ```bash
    hermes token-analytics sync
    ```
-5. Let **Hermes cron own the schedule** by creating a single cron job that runs the sync command on a cadence such as every 15 minutes. That one job covers both rollups and heartbeat freshness.
+7. Let **Hermes cron own the schedule** by creating a single cron job that runs the sync command on a cadence such as every 15 minutes. That one job covers both rollups and heartbeat freshness.
 
 If you just need the exact install sequence, use:
 
@@ -46,6 +53,18 @@ This repo is the dashboard and ingest target. The operator-facing runtime is the
 
 For local Hermes checkout installs, the helper now copies the plugin into `plugins/hermes-token-analytics/` and also writes a legacy compatibility shim at `plugins/observability/token_analytics/` so existing enabled-plugin configs do not break mid-upgrade.
 
+## Common operational failure modes
+
+When operators report that token analytics is "not working," the failure is often in the workflow, not the plugin logic.
+
+The main causes are:
+
+- using the monorepo root instead of `plugins/hermes-token-analytics/`
+- enabling the plugin without restarting the Hermes gateway
+- proving only that a manual sync works, without creating a cron job for continuous reporting
+
+Manual sync working is useful, but it is not the same thing as production readiness.
+
 ## Architecture and ownership
 
 ### Hermes-native plugin, not a sidecar
@@ -72,6 +91,28 @@ UI note: the dashboard may refer to these imported workspaces as **Agents** in u
 If a sync is running at the wrong time, fix Hermes cron.
 If a sync is reading the wrong DB or posting to the wrong workspace, fix plugin config.
 
+### Install target choices
+
+There are two supported install targets:
+
+1. **Bundled checkout path**
+   - use this when you are editing a Hermes repo checkout directly
+   - plugin lives under `~/.hermes/hermes-agent/plugins/hermes-token-analytics/`
+2. **User plugin path**, preferred for normal operator installs
+   - plugin lives under `~/.hermes/plugins/hermes-token-analytics/`
+
+If both are technically available, prefer the **user plugin path** unless you specifically need the plugin to live inside a Hermes checkout for development.
+
+### Gateway reload requirement
+
+After `hermes plugins enable hermes-token-analytics`, run:
+
+```bash
+hermes gateway restart
+```
+
+Without the restart, Hermes may still be running without the newly enabled plugin command surface.
+
 ## Required Worker-side configuration
 
 The Worker route rejects unauthenticated syncs.
@@ -206,6 +247,10 @@ A successful run should:
 4. receive a successful response from the ingest route
 5. make the imported workspace visible in the dashboard
 
+A successful manual `hermes token-analytics sync` only proves that the plugin works in the foreground.
+It does **not** prove that continuous reporting is active.
+Continuous reporting requires a Hermes cron job.
+
 ### `hermes token-analytics install-cron-wrapper`
 
 Install a thin shell wrapper for Hermes cron `no_agent` jobs.

plugins/hermes-token-analytics/README.md

@@ -2,6 +2,11 @@
 
 This directory is the source-of-truth copy of the Hermes token analytics plugin inside the monorepo.
 
+> [!WARNING]
+> This repository is a monorepo. The Hermes plugin is **not** at the repo root.
+> Do **not** install the repo root directly with `hermes plugins install <repo-url>`.
+> The actual plugin source lives at `plugins/hermes-token-analytics/`.
+
 ## What lives here
 
 - `plugins/hermes-token-analytics/` — the plugin files as they should exist inside a Hermes checkout
@@ -30,6 +35,13 @@ python3 -m pytest plugins/hermes-token-analytics/tests -q
 
 ## Local install helper
 
+There are two install patterns:
+
+1. **Bundled checkout install**, useful when you are modifying a Hermes repo checkout directly
+2. **User plugin install**, preferred for normal operator usage under `~/.hermes/plugins/hermes-token-analytics`
+
+Bundled checkout helper:
+
 ```bash
 plugins/hermes-token-analytics/scripts/install-local-plugin.sh /path/to/hermes-agent
 ```
@@ -39,3 +51,12 @@ Default target if no argument is passed:
 - `~/.hermes/hermes-agent`
 
 The install helper also writes a small compatibility shim at `plugins/observability/token_analytics/` inside the target Hermes checkout so older `plugins.enabled` entries keep working during migration.
+
+After enabling the plugin, restart the gateway:
+
+```bash
+hermes plugins enable hermes-token-analytics
+hermes gateway restart
+```
+
+A successful manual `hermes token-analytics sync` proves the plugin works manually. It does **not** prove continuous reporting is active. For that, create the cron job described in `docs/hermes-token-analytics-install-runbook.md`.