PromptQL Conference Call Note-taker

A focused PromptQL agent that joins a known telephone conference, takes durable notes, and optionally answers questions when someone says a configured wake phrase.

PromptQL invokes the note-taker through a custom OpenAPI integration. Twilio places the PSTN call and enters a known DTMF sequence. Twilio then bridges the conference to OpenAI Realtime over SIP, while this service keeps a sideband WebSocket for transcription, wake-phrase control, and PromptQL MCP tools.

PromptQL thread --custom integration--> this service --Twilio--> conference
       ^                                  |                 |
       |                                  |                 +--> OpenAI Realtime SIP audio
       +--MCP lifecycle/notes/transcript--+
       +--MCP wake-question answers-------+

This repository is deliberately one agent with one job. It does not accept incoming calls, make general-purpose or booking calls, interpret dynamic IVRs, scrape meeting links, or identify individual conference speakers. It only joins a dial-in number and predetermined digit sequence supplied by PromptQL.

What it does

Starts one outbound note-taker per PromptQL thread, with idempotent creation.
Joins Zoom, Google Meet, or another telephone bridge using known DTMF.
Introduces itself once in a warm, concise way after joining, then stays out of the conversation unless invoked.
Records a dual-channel WAV and persists an ordered remote/agent transcript.
Runs silently when wake_phrase is omitted.
With a wake phrase, responds concisely to the turn containing that phrase and requires it again for every later response.
Uses PromptQL MCP to answer wake-phrase questions that need project context.
Posts lifecycle updates and final Markdown notes back to the originating PromptQL thread, attaching transcript.md when it fits the 5 MiB text-attachment limit.

The notes always contain Overview, Decisions, Action items, and Open questions sections. All conference participants are one mixed remote speaker; names are recorded only when spoken, not inferred through diarization.

End-to-end setup

You need Node.js 22+, PostgreSQL, a public HTTPS URL, a voice-capable Twilio number, an OpenAI API project with Realtime access, and a PromptQL project where an administrator can create a bot and custom integration.

The installation order is:

Configure Twilio.
Configure OpenAI and its webhook.
Create the PromptQL bot, its MCP PAT, and room membership.
Create and assign the PromptQL custom claim.
Start this service behind a public HTTPS URL.
Add the OpenAPI integration and wiki to PromptQL.
Run a direct API check, then launch it from a PromptQL thread.

Fast path: ask PromptQL to bootstrap the PromptQL side

Instead of manually performing steps 3, 4, and 6, use a dedicated PromptQL thread as the operator interface. Keep the repository bundle as the source of truth so the setup is repeatable and auditable.

Copy promptql/bootstrap.example.yaml, fill in the project UUID, administrator user UUID, allowed room IDs, project name, and deployed PUBLIC_BASE_URL, and keep all secrets out of it.
In a PromptQL room where you have project-administrator authority, create a dedicated bootstrap thread.
Attach the completed manifest and these bundle files:

Send this prompt:

Bootstrap the Conference Call Note-taker from the attached manifest and bundle.

First inspect existing bot users, profile wikis, wiki scopes, claims, room
memberships, providers, and project integrations. Reuse or update resources
with matching stable names; do not create duplicates. Show the proposed plan
and request approval before mutations.

Reconcile the bot user and profile wiki, exact-scope MCP PAT requirement,
allowed room memberships, conference-call-agent claim and assignment, API-key
provider, OpenAPI operations, project integration and access rule, and the
operational integration wiki. Require approval for POST /v1/meetings.

Never place a PAT or service bearer token in this thread, an artifact, a wiki,
the OpenAPI document, or a URL. Use a secure token handoff if available;
otherwise stop and give me the exact one-time manual token step. Present the
service credential only through the integration connect flow.

Finish with read-only verification: report every stable resource ID, confirm
the bot is active and belongs to each allowed room, confirm claim mappings and
the integration base URL, and call only GET /healthz and GET /readyz. Do not
place a test call.

Approve the proposed administrative changes in PromptQL. Enter CONFERENCE_AGENT_API_KEY only in the integration connect card. Store the one-time MCP-scoped bot PAT directly as PROMPTQL_MCP_PAT in the service secret store or local .env; if PromptQL cannot hand it off securely, use the manual token operation in section 3 below.
Save PromptQL's final resource-ID report with the deployment. Re-run the same bootstrap thread and manifest to reconcile future changes rather than asking it to create everything again.

This fast path automates only the PromptQL control plane. Twilio credentials and number configuration, the OpenAI project key and webhook, the service deployment, and external secret storage still use the corresponding manual sections below. The thread is the operator UI and audit trail; the manifest and attached files remain authoritative.

1. Twilio credentials and number

Sign in to the Twilio Console.
Copy the Account SID (starts with AC) to TWILIO_ACCOUNT_SID and reveal the account Auth Token for TWILIO_AUTH_TOKEN. This service uses that Auth Token both for REST calls and webhook signature verification.
Under Phone Numbers → Manage → Buy a number, buy a voice-capable number and put its E.164 value in TWILIO_FROM_NUMBER, for example +14155550100.
Under Voice → Settings → Geo Permissions, enable only the countries the note-taker needs to dial.
Trial accounts may require the conference destination to be verified. Check the account's calling and spending restrictions before a real test.

Do not configure an incoming-call webhook on the Twilio number for this agent. Each outbound call supplies its own TwiML, status, dial, and recording callback URLs based on PUBLIC_BASE_URL.

Twilio validates callback URLs exactly. If TLS terminates at a tunnel or reverse proxy, PUBLIC_BASE_URL must be the externally visible HTTPS origin with no trailing slash.

2. OpenAI project, key, and webhook

Sign in to the OpenAI API platform and select or create a project with billing and Realtime API access.
Create a project API key and copy it once to OPENAI_API_KEY.
Under Project → General, copy the project ID beginning with proj_ to OPENAI_PROJECT_ID. This is not an organization ID.

Under Project → Webhooks, create a webhook for:

https://YOUR_PUBLIC_HOST/providers/openai/webhook

Subscribe it to realtime.call.incoming, copy its signing secret to OPENAI_WEBHOOK_SECRET, and restart the service whenever that secret changes.

Twilio ultimately dials sip:<OPENAI_PROJECT_ID>@sip.api.openai.com;transport=tls. OpenAI sends the signed incoming-call event to this service, which verifies both the OpenAI signature and an expiring HMAC correlation header before accepting the call. The implementation follows OpenAI's official Realtime SIP, server-side controls, and VAD guidance.

The default models are gpt-realtime-2.1, gpt-live-transcribe, and gpt-5.6-luna. If model access differs for your project, change the corresponding environment variables to supported equivalents.

3. Create the PromptQL bot and MCP PAT

Run the operations in promptql/bot/create-bot.graphql through the PromptQL project's admin GraphQL console as an app-admin or supergraph-admin:

Run CreateConferenceCallBot and save its promptql_user_id.
Pass that ID as botUserId to CreateConferenceCallBotTokens.
Save the returned token immediately; PAT values are shown only once.
For every PromptQL room from which this integration may be launched, run AddConferenceCallBotToRoom with the room ID and bot user ID.

The bot uses one PAT with the exact MCP scope:

Environment variable	Exact PAT scope	Used for
`PROMPTQL_MCP_PAT`	`[{"type":"mcp"}]`	Questions and response waiting, lifecycle messages, final notes and transcript attachments, and correlated cancellation

The bot must be an active participant in the room containing an originating thread. A PAT alone does not grant access to that thread. The MCP transport uses the lowercase header form Authorization: pat <token>.

4. Create and attach the PromptQL access claim

The MCP PAT scope authorizes this service to call PromptQL. A custom claim is a different control: it decides which PromptQL principals may invoke the conference integration.

Run promptql/bot/assign-claim.graphql as an administrator:

Run CreateConferenceCallAgentClaim with the project UUID and your admin PromptQL user UUID.
Copy the returned claim-key ID.
Run AssignConferenceCallAgentClaim for the bot user with that ID. This assigns conference-call-agent=true.
Repeat the assignment for any human or run-as principals that should invoke the integration directly. Do not grant it project-wide by default.

The project integration is configured with this access rule:

{
  "claim_key": "conference-call-agent",
  "claim_value": "true"
}

At runtime PromptQL materializes the claim as x-hasura-custom-claim-conference-call-agent. A principal must satisfy the integration's access scope and, independently, have access to the originating room/thread.

5. Configure and run the service

Copy the example environment and generate two unrelated local secrets:

cp .env.example .env
openssl rand -base64 32  # CONFERENCE_AGENT_API_KEY
openssl rand -base64 32  # SIP_CORRELATION_SECRET

Fill every non-comment value in .env. Important distinctions:

CONFERENCE_AGENT_API_KEY is the bearer credential PromptQL sends to this service. It is not a PromptQL PAT.
PROMPTQL_MCP_URL is the Streamable HTTP MCP endpoint, normally ending in /mcp-server/mcp.
PROMPTQL_MCP_PAT is the exact-scope bot token created in step 3.
PROMPTQL_PROJECT_NAME is the project name/slug accepted by that MCP endpoint.
PUBLIC_BASE_URL is the public HTTPS origin for this service, without a trailing slash.

To run the whole stack in Docker:

docker compose up --build -d
docker compose logs -f conference-agent

PostgreSQL is exposed on host port 55432 by default and the agent listens on 0.0.0.0:3100. Override those with POSTGRES_HOST_PORT and PORT if necessary.

For local application development with only PostgreSQL in Docker:

docker compose up -d postgres
# Use postgres://conference:conference@localhost:55432/conference in local .env
npm ci
npm run db:migrate
npm run dev

Expose port 3100 through your HTTPS tunnel, set PUBLIC_BASE_URL to that origin, and restart the service. Then update the OpenAI webhook from step 2 to the same public host.

Check readiness and generated API docs:

curl --fail https://YOUR_PUBLIC_HOST/healthz
curl --fail https://YOUR_PUBLIC_HOST/readyz

Swagger UI is at https://YOUR_PUBLIC_HOST/docs.

6. Add the PromptQL custom integration

The promptql/ directory is the deployable PromptQL-side bundle:

openapi.yaml describes the API PromptQL may call.
provider.example.json contains provider metadata and bearer-header configuration.
project-integration.example.json contains the base URL and claim restriction.
wiki/conference-call-note-taker.md tells PromptQL when and how to use it.

In the PromptQL custom-integration administration workflow:

Create an API-key provider with ID conference-call-note-taker using provider.example.json as the field-by-field template.
Upload openapi.yaml, replacing https://conference-agent.example.com with PUBLIC_BASE_URL if the workflow does not override its server URL.
Store CONFERENCE_AGENT_API_KEY as the provider credential. Configure the header name as Authorization and prefix as Bearer . Never put this secret in the OpenAPI document or wiki.
Connect the provider to the project with project-integration.example.json, set its base_url, and retain the conference-call-agent=true access scope.
Require user approval for POST /v1/meetings; it places a real PSTN call and incurs Twilio/OpenAI usage. Read and cancel operations can follow your normal project policy.
Create a wiki page from wiki/conference-call-note-taker.md in a scope visible to the intended room and principals. You can paste it in the console or use PromptQL MCP's wiki-page creation tool.

The wiki directs PromptQL to call:

run_http(integration="conference-call-note-taker")

It also requires explicit confirmation of the phone number and DTMF, stable idempotency keys, and no guessing of meeting credentials.

7. Verify end to end

First test the service directly. This creates a real call:

export AGENT_URL=https://YOUR_PUBLIC_HOST
export THREAD_ID=00000000-0000-0000-0000-000000000000

curl --fail-with-body \
  -X POST "$AGENT_URL/v1/meetings" \
  -H "Authorization: Bearer $CONFERENCE_AGENT_API_KEY" \
  -H "Idempotency-Key: conference-demo-001" \
  -H "Content-Type: application/json" \
  --data "{\"thread_id\":\"$THREAD_ID\",\"to\":\"+16505551234\",\"objective\":\"Capture decisions, owners, deadlines, and open questions.\",\"dial_sequence\":\"wwww123456789#W#\",\"wake_phrase\":\"Hey Project Notes\"}"

Use the actual dial-in number and invitation-provided digit sequence. w waits 0.5 seconds and W waits 1 second. Typical shapes are wwww<zoom-meeting-id>#W# or wwww<google-meet-pin>#, but invitations vary.

Then ask PromptQL in a room containing the bot, for example:

Join the weekly sync as a silent note-taker. Dial +1… and enter wwww…#. Capture decisions, owners, deadlines, and open questions. Use this thread for the results.

Or with interactive mode:

Join the call with wake phrase “Hey Project Notes”. Only answer after that exact phrase, and require it again for every question.

PromptQL should request approval before placing the call, post lifecycle updates into the same thread, and eventually post notes plus transcript.md. Saying the wake phrase during the conference should produce one concise response; speech without it should remain unanswered but still be transcribed.

8. Automate it from a calendar

The simplest day-to-day experience is to ask PromptQL once to monitor a connected calendar and prepare each eligible conference automatically. Keep two kinds of threads separate:

one long-lived calendar watcher thread owns the recurring calendar check and the one-time launch triggers;
one fresh meeting thread owns exactly one calendar occurrence and receives every call update, question, note, and transcript.

PromptQL scheduled triggers are attached to an existing thread; there is no separate future “create a thread” object. To get one thread per call, the watcher schedules a one-time trigger on itself for shortly before an event. When that trigger fires, PromptQL refreshes the event, starts a fresh meeting thread, and launches the integration from that new thread. The watcher must pass the new meeting thread's UUID as thread_id; it must never pass its own UUID.

Before asking PromptQL to set this up:

Connect the user's calendar to PromptQL with the least read access needed to see event times and invitation details.
Ensure the user or run-as principal may create threads and scheduled triggers in the destination room, can invoke the conference-call-note-taker integration, and can read the integration wiki.
Add the note-taker bot as an active participant in that room so its MCP PAT can post back to each meeting thread.
Decide which calendar, event filter, destination room, timezone, lead time, objective, and optional wake phrase are authorized. Only events containing an authoritative PSTN number and predetermined DTMF sequence are eligible.

Create a dedicated watcher thread and send a request like this:

Set up calendar-driven conference note-taking for me.

Use this thread only as the calendar watcher and control plane. Every 30
minutes, read <calendar name> in <IANA timezone> and reconcile eligible events
in the next 48 hours. Only manage events where <filter, for example: I am the
organizer and the title contains “[note-taker]”>. Never infer or scrape a dial-in
number, meeting ID, PIN, DTMF timing, or wake phrase. Skip the event and tell me
what is missing unless those values are authoritative calendar fields or invite
details.

For each calendar occurrence, deduplicate by calendar provider + calendar ID +
provider event ID + recurrence occurrence ID. Keep an index in this watcher
thread containing that key, the event revision, scheduled-trigger ID, fire time,
meeting-thread ID when created, and status. Do not create duplicate triggers or
threads. If an event is moved, delete and recreate its pending trigger; if it is
cancelled or I decline it, delete the trigger.

Schedule a one-time trigger for <5 minutes> before each eligible event. When it
fires, refresh the calendar event first. If it is cancelled, declined, moved,
already handled, or missing confirmed telephone/DTMF details, do not dial. If it
is still eligible, start one fresh PromptQL thread in <room> titled
“Meeting notes — <event title> — <start time>”. Seed that thread with the event
key, current event snapshot, dial-in number, exact DTMF sequence, objective,
timezone, duration, authorized wake phrase or notes-only mode, and the
instruction to use the Conference Call Note-taker integration immediately.

The new meeting thread must call POST /v1/meetings using its own current
thread_id and a stable Idempotency-Key derived from the calendar occurrence key.
Never launch the call from this watcher thread. Keep lifecycle updates,
PromptQL lookups, notes, and transcript in the meeting thread; keep only the
deduplication index and links to meeting threads here.

Show me the proposed recurring trigger, filters, timezone, lead time, room, and
approval behavior before creating anything. Preserve per-call approval for the
real outbound POST unless an administrator has explicitly installed a narrower
pre-approved policy for this exact automation.

Thirty minutes is the minimum supported recurring-trigger interval in the current PromptQL scheduler. A five-minute launch lead is a useful starting point, but use a longer lead if a human must approve every outbound call. The initial request delegates calendar monitoring and trigger maintenance; it does not silently override the approval policy on POST /v1/meetings.

For recurring meetings, create a separate meeting thread and idempotency key for every occurrence. At trigger time, always refresh the event before dialing so a stale invite cannot launch a cancelled or rescheduled call. If an event is first discovered after its intended launch time, PromptQL should ask whether to join late rather than placing the call automatically.

API summary

All /v1 endpoints require Authorization: Bearer <CONFERENCE_AGENT_API_KEY>.

Method	Path	Purpose
`POST`	`/v1/meetings`	Start a note-taker; requires `Idempotency-Key`
`GET`	`/v1/meetings/{id}`	State, provider IDs, notes, and artifact links
`POST`	`/v1/meetings/{id}/cancel`	Idempotently end it; active calls still generate notes
`GET`	`/v1/meetings/{id}/transcript`	Ordered mixed-speaker transcript
`GET`	`/v1/meetings/{id}/recording`	Download the local dual-channel WAV when ready

POST /v1/meetings requires thread_id, E.164 to, and objective. dial_sequence and wake_phrase are optional. Duration defaults to 7,200 seconds and is capped at 14,400 seconds. Reusing an idempotency key returns the original meeting; a different active meeting on the same PromptQL thread returns HTTP 409.

States are queued, dialing, answered, connecting_agent, active, waiting_on_promptql, completing, and terminal completed, failed, cancelled, busy, or no_answer.

PromptQL MCP tools

All communication back to PromptQL uses the MCP connection and the bot's MCP PAT:

ask_promptql posts lifecycle updates and final notes to the originating thread_id, attaches transcript.md through its files argument, and starts correlated wake-phrase questions.
get_latest_promptql_thread_response waits for its completed answer.
cancel_promptql_thread_interaction cancels only the exact outstanding interaction identified by its thread_event_id.

The service retains the supplied thread_id for every MCP call, so status messages, questions, notes, and transcript files remain in the thread that launched the meeting. The configured PromptQL MCP server must expose these three tools; cancel_promptql_thread_interaction must accept both thread_id and thread_event_id. The service deliberately has no REST fallback.

The service permits one in-flight PromptQL question per meeting and waits at most 120 seconds. Response watching uses bounded attempts so a dropped long-poll cannot consume the entire deadline after PromptQL has already answered. It never auto-approves PromptQL approval requests.

For a wake-phrase lookup, the note-taker starts PromptQL immediately, then acknowledges the request by voice while that lookup runs and invites the room to keep talking. It gives varied progress updates only when useful, defers them while participants are speaking, and coalesces queued updates so they do not repeat after a long uninterrupted turn. When the answer arrives, it completes the already-authorized response without asking for the wake phrase again. Ordinary interjections do not interrupt its speech; only a recognized wake phrase cancels the current response and opens a new turn. If a PromptQL MCP deployment rejects transcript file upload, final notes are retried without the attachment instead of being lost.

Operations

Logs are structured and redact configured authorization/signature headers:

docker compose logs --tail=200 conference-agent
docker compose logs -f conference-agent

For local development, start the watcher with npm run dev 2>&1 | tee -a .local-dev.log. Follow the same output from another terminal with tail -n 100 -f .local-dev.log. The repository ignores *.log files.

On process restart, the service recovers non-terminal meetings, reconnects active OpenAI sideband sessions when possible, and never creates another PSTN call once a Twilio Call SID has been persisted.

Recordings are stored under RECORDINGS_DIR/{meeting_id}.wav; Compose uses its recordings volume. PostgreSQL stores durable meeting state, append-only events, transcript turns, pending PromptQL requests, webhook deduplication, and recording metadata.

Development and tests

npm ci
npm run typecheck
npm test
npm run build

The opt-in smoke test requires real credentials and creates a chargeable call:

CONFERENCE_AGENT_BASE_URL=https://YOUR_PUBLIC_HOST \
SMOKE_THREAD_ID=00000000-0000-0000-0000-000000000000 \
SMOKE_TO_NUMBER=+16505551234 \
npm run smoke

Security and consent

This is a demonstration harness, not a production compliance layer. It retains transcripts and recordings indefinitely and does not implement recording consent, encryption/KMS, redaction, retention/deletion jobs, participant identity, regional routing, or regulated-workload controls. Inform participants that an automated note-taker is present, obtain legally required consent, restrict phone destinations and PromptQL claims, use least-privilege bot PATs, rotate secrets, and define retention before real organizational use.

Never commit .env, PATs, provider credentials, call recordings, or transcript exports.

Troubleshooting

OpenAI webhook never arrives: confirm Twilio reached this service's TwiML endpoint, OPENAI_PROJECT_ID begins with proj_, and the project webhook subscribes to realtime.call.incoming.
Twilio callback is rejected: make PUBLIC_BASE_URL exactly match the tunnel/proxy URL Twilio called; do not include a trailing slash.
PromptQL returns 401 TOKEN_INVALID: check the MCP PAT value, project, bot activity, and lowercase pat scheme on the MCP transport.
PromptQL returns a scope error: create the bot PAT with exactly [{"type":"mcp"}].
PromptQL cannot write to a thread: add the bot as an active participant in the room that contains it.
PromptQL cannot invoke the integration: assign conference-call-agent=true to the actual invoking/run-as principal and retain the same access scope on the project integration.
The bot answers ordinary meeting speech: omit wake_phrase for silent mode, or verify the configured phrase is sufficiently specific. A phrase match intentionally treats all mixed participants as authorized speakers.
No recording yet: Twilio finalizes and downloads it asynchronously after the conference leg ends; check the recording-status callback logs.

Name		Name	Last commit message	Last commit date
drizzle		drizzle
promptql		promptql
scripts		scripts
src		src
test		test
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
compose.yaml		compose.yaml
package-lock.json		package-lock.json
package.json		package.json
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json

Navigation Menu

Uh oh!

Repository files navigation

PromptQL Conference Call Note-taker

What it does

End-to-end setup

Fast path: ask PromptQL to bootstrap the PromptQL side

1. Twilio credentials and number

2. OpenAI project, key, and webhook

3. Create the PromptQL bot and MCP PAT

4. Create and attach the PromptQL access claim

5. Configure and run the service

6. Add the PromptQL custom integration

7. Verify end to end

8. Automate it from a calendar

API summary

PromptQL MCP tools

Operations

Development and tests

Security and consent

Troubleshooting

About

Releases

Packages

Used by

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

PromptQL Conference Call Note-taker

What it does

End-to-end setup

Fast path: ask PromptQL to bootstrap the PromptQL side

1. Twilio credentials and number

2. OpenAI project, key, and webhook

3. Create the PromptQL bot and MCP PAT

4. Create and attach the PromptQL access claim

5. Configure and run the service

6. Add the PromptQL custom integration

7. Verify end to end

8. Automate it from a calendar

API summary

PromptQL MCP tools

Operations

Development and tests

Security and consent

Troubleshooting

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages