Skip to content

This PR wires Mastra tracing to persistent PostgreSQL storage for local development #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MabudAlam wants to merge 1 commit into
base: main
Choose a base branch
from
+42 −9
Open

This PR wires Mastra tracing to persistent PostgreSQL storage for local development #73

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions backend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
"@clerk/backend": "^3.4.11",
"@fastify/cors": "^11.0.0",
"@mastra/core": "^1.36.0",
"@mastra/observability": "^1.13.0",
"@mastra/pg": "^1.11.1",
"@openrouter/ai-sdk-provider": "^2.9.0",
"ai": "^6.0.0",
"convex": "^1.39.1",
Expand Down
11 changes: 7 additions & 4 deletions backend/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import { env } from "./env.js";
import clerkAuthPlugin, { requireAuth } from "./clerk-auth.js";
import { inferSchema } from "./pipeline/schema-inference.js";
import { datasetContextSchema } from "./pipeline/populate.js";
import { populateWorkflow } from "./mastra/workflows/populate.js";
import { convex, api } from "./convex.js";
import { mastra } from "./mastra/index.js";
import { convex, api, internal } from "./convex.js";

const fastify = Fastify({ logger: true });

Expand Down Expand Up @@ -61,16 +61,18 @@ await fastify.register(async (instance) => {
}

try {
const dataset = await convex.query(api.datasets.get, { id: parsed.data.datasetId });
const dataset = await convex.query(internal.datasets.getInternal, { id: parsed.data.datasetId });
if (!dataset) {
return reply.code(404).send({ error: "Dataset not found" });
}
if (dataset.ownerId !== req.auth.userId) {
return reply.code(403).send({ error: "Not authorized to populate this dataset" });
}

const run = await populateWorkflow.createRun();
const workflow = mastra.getWorkflow("populateWorkflow");
const run = await workflow.createRun();
const result = await run.start({ inputData: parsed.data });
await mastra.observability.getDefaultInstance()?.flush();

req.log.info({ workflowStatus: result.status, steps: JSON.stringify(result.steps).slice(0, 2000) }, "Populate workflow completed");

Expand All @@ -80,6 +82,7 @@ await fastify.register(async (instance) => {

return { success: true, result: result.result };
} catch (err) {
await mastra.observability.getDefaultInstance()?.flush();
const msg = err instanceof Error ? err.message : String(err);
if (msg.includes("validator") || msg.includes("Invalid")) {
return reply.code(400).send({ error: "Invalid datasetId" });
Expand Down
34 changes: 29 additions & 5 deletions backend/src/mastra/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,33 @@
import { Mastra } from "@mastra/core/mastra";
import { inferSchemaWorkflow } from "./workflows/infer-schema.js";
import { populateWorkflow } from "./workflows/populate.js";
import { populateAgent } from "./agents/populate.js";
import { Mastra } from '@mastra/core/mastra'
import { PostgresStore } from '@mastra/pg'
import {
Observability,
MastraStorageExporter,
SensitiveDataFilter,
} from '@mastra/observability'
import { inferSchemaWorkflow } from "./workflows/infer-schema.js"
import { populateWorkflow } from "./workflows/populate.js"
import { populateAgent } from "./agents/populate.js"

const storage = new PostgresStore({
id: 'mastra-pg-storage',
connectionString: process.env.MASTRA_DATABASE_URL!,
})
Comment on lines +12 to +15
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Missing validation for MASTRA_DATABASE_URL environment variable.

The non-null assertion (!) on process.env.MASTRA_DATABASE_URL will pass undefined to PostgresStore if the variable is missing, likely causing a cryptic connection error at runtime rather than a clear startup failure.

Consider validating this in env.ts alongside other required variables, or add a guard here:

Proposed fix 
+const mastraDbUrl = process.env.MASTRA_DATABASE_URL;
+if (!mastraDbUrl) {
+  throw new Error('MASTRA_DATABASE_URL environment variable is required');
+}
+
 const storage = new PostgresStore({
   id: 'mastra-pg-storage',
-  connectionString: process.env.MASTRA_DATABASE_URL!,
+  connectionString: mastraDbUrl,
 })
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const storage = new PostgresStore({
id: 'mastra-pg-storage',
connectionString: process.env.MASTRA_DATABASE_URL!,
})
const mastraDbUrl = process.env.MASTRA_DATABASE_URL;
if (!mastraDbUrl) {
throw new Error('MASTRA_DATABASE_URL environment variable is required');
}
const storage = new PostgresStore({
id: 'mastra-pg-storage',
connectionString: mastraDbUrl,
})
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend/src/mastra/index.ts` around lines 12 - 15, The code uses a non-null
assertion for process.env.MASTRA_DATABASE_URL when constructing PostgresStore
(storage), which can pass undefined and produce cryptic runtime errors; replace
that with an explicit validation: read and validate MASTRA_DATABASE_URL (either
in env.ts alongside other vars or at the top of this module), throw a clear
startup Error if it's missing, and then pass the validated string into new
PostgresStore({ id: 'mastra-pg-storage', connectionString:
validatedMastraDatabaseUrl }); ensure you remove the trailing "!" and reference
the validated variable when creating storage.


export const mastra = new Mastra({
agents: { populateAgent },
workflows: { inferSchemaWorkflow, populateWorkflow },
});
storage,
observability: new Observability({

configs: {
default: {
serviceName: 'bigset',
exporters: [new MastraStorageExporter({ strategy: 'auto' })],
spanOutputProcessors: [
new SensitiveDataFilter(),
],
},
},
}),
})
4 changes: 4 additions & 0 deletions docker-compose.dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ services:
- "3501:3501"
volumes:
- ./backend/src:/app/src
- ./backend/.mastra:/app/.mastra
environment:
CLIENT_ORIGIN: http://localhost:3500
CONVEX_URL: http://convex:3210
Expand All @@ -33,6 +34,7 @@ services:
CLERK_PUBLISHABLE_KEY: ${NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY:-}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
TINYFISH_API_KEY: ${TINYFISH_API_KEY:-}
MASTRA_DATABASE_URL: postgresql://bigset:bigset@db:5432/bigset_internal
depends_on:
convex:
condition: service_healthy
Expand All @@ -45,13 +47,15 @@ services:
- "4111:4111"
volumes:
- ./backend/src:/app/src
- ./backend/.mastra:/app/.mastra
environment:
HOST: 0.0.0.0
PORT: 4111
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
CONVEX_URL: http://convex:3210
CONVEX_SELF_HOSTED_ADMIN_KEY: ${CONVEX_SELF_HOSTED_ADMIN_KEY:-}
TINYFISH_API_KEY: ${TINYFISH_API_KEY:-}
MASTRA_DATABASE_URL: postgresql://bigset:bigset@db:5432/bigset_internal
depends_on:
convex:
condition: service_healthy
Comment on lines +58 to 61
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

mastra service should depend on db for PostgreSQL connectivity.

The mastra service uses MASTRA_DATABASE_URL pointing to the db service, but only declares a dependency on convex. If mastra starts before db is healthy, the PostgreSQL connection will fail.

Proposed fix 
     depends_on:
+      db:
+        condition: service_healthy
       convex:
         condition: service_healthy
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
MASTRA_DATABASE_URL: postgresql://bigset:bigset@db:5432/bigset_internal
depends_on:
convex:
condition: service_healthy
MASTRA_DATABASE_URL: postgresql://bigset:bigset@db:5432/bigset_internal
depends_on:
db:
condition: service_healthy
convex:
condition: service_healthy
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker-compose.dev.yml` around lines 58 - 61, The mastra service is missing a
dependency on the db service even though MASTRA_DATABASE_URL points at
PostgreSQL; update the mastra service's depends_on block to include db with
condition: service_healthy (in addition to the existing convex dependency) so
Docker Compose waits for the db to be healthy before starting mastra.

Expand Down