diff --git a/docs/architecture docs/adrs/ADR 210 - Index-type registry.md b/docs/architecture docs/adrs/ADR 210 - Index-type registry.md new file mode 100644 index 0000000000..3f68884235 --- /dev/null +++ b/docs/architecture docs/adrs/ADR 210 - Index-type registry.md @@ -0,0 +1,225 @@ +# ADR 210 — Index-type registry + +> **Decision (in one sentence):** Index types live in a per-contract registry assembled from the contract's extension packs; each entry pairs a `type` literal with an `arktype` validator for its `options`, and that pair is what the authoring DSL narrows against, what the lowering validates against, and what the framework-owned Postgres renderer reads from when emitting `CREATE INDEX … USING WITH (…)`. + +## A grounding example + +Postgres has more than one kind of index. The default is a B-tree, but there's also `gin` (good for full-text search and array containment), `gist` (geometric and range types), `hash`, `brin`, `spgist`. Extensions add more — for example, the `paradedb` extension contributes a `bm25` index built on top of Tantivy for ranked text search. + +Each index method takes its own set of *storage parameters* — passed as `WITH (key = value, …)` in DDL. `gin` accepts `fastupdate` (a boolean). `bm25` accepts `key_field` (a column name) and a few other knobs. Treating "what storage parameters does this method accept?" as a property of the method itself — and asking each method to describe that shape once — is what this ADR is about. + +A pack contributes an index type by declaring it once, alongside an `arktype` validator describing its `options`: + +```ts +// packages/3-extensions/paradedb/src/types/index-types.ts +import { defineIndexTypes } from '@prisma-next/sql-contract/index-types'; +import { type } from 'arktype'; + +export const paradedbIndexTypes = defineIndexTypes().add('bm25', { + options: type({ + '+': 'reject', // reject any extra option keys (registrant opt-in) + key_field: 'string', + }), +}); +``` + +A pack publishes that registration on its descriptor under `indexTypes:`. A contract that attaches the pack can then author indexes against `bm25` and the authoring surface narrows on it: + +```ts +// In a contract that has paradedbPack attached as an extension pack: +const Item = model('Item', { + fields: { id: field.column(int4Column).id(), body: field.column(textColumn) }, +}).sql(({ cols, constraints }) => ({ + table: 'item', + indexes: [ + constraints.index([cols.body], { + type: 'bm25', + options: { key_field: 'id' }, // arktype-validated against bm25's shape + }), + ], +})); +``` + +The same authoring surface in PSL: + +```prisma +model Item { + id Int @id + body String + @@index([body], type: "bm25", options: { key_field: "id" }) +} +``` + +Three classes of mistake are caught at the boundary closest to the author: + +```ts +// Compile error: 'made-up' is not a key in the merged registry +constraints.index([cols.body], { type: 'made-up', options: {} }); + +// Compile error (from arktype's TS narrowing): 'tokenizer' isn't in bm25's shape; +// at runtime, validation also fails because the registrant opted into '+': 'reject' +constraints.index([cols.body], { type: 'bm25', options: { tokenizer: 'std' } }); + +// Runtime error at lowering: missing required key_field +constraints.index([cols.body], { type: 'bm25', options: {} }); +``` + +The rest of this document is the vocabulary, the lifecycle, and the layering rules behind that example. + +## What this is solving + +Without a central concept of *what index types exist and what their options look like*, three things happen: + +- **Authors put any string in the type slot and any object in the options slot** with no feedback until DDL apply time, where Postgres returns an opaque error or — worse — silently accepts an unknown storage parameter that does nothing. +- **Extension authors duplicate type-and-validation work per index type**, and have no shared discipline about how strictness, dialect-neutral naming, and renderer safety interact. +- **There is no extension point for end users** to add their own index types without forking the schema validator. + +The registry resolves all three by giving the system one place that knows which `type` values are legal in a contract, what each type's `options` shape is, and how to render the result safely. Once that single source of truth exists, the authoring DSL, the lowering, the migration planner, the schema verifier, and the DDL renderer all consult it instead of each carrying their own lookup. + +## The registry primitive + +An entry is a pair: a `type` literal and a validator describing the entry's `options`. + +```ts +type IndexTypeEntry = { + readonly type: string; + readonly options: arktype.Type; +}; +``` + +Entries are produced by a small fluent builder. The builder is the only way an entry comes into existence; there is no other constructor: + +```ts +defineIndexTypes() + .add('bm25', { options: type({ '+': 'reject', key_field: 'string' }) }) + .add('vector', { options: type({ '+': 'reject', m: 'number', ef_construction: 'number' }) }); +``` + +`defineIndexTypes()` returns a value carrying both the runtime entry list and a TypeScript-only phantom map of `type` literal → `options` shape. The same value is what a pack stores on its descriptor; both halves stay in lockstep automatically because both come from the same builder call. Drift between the runtime shape and the TS shape becomes a TypeScript error at the `.add(…)` call site, not a runtime surprise downstream. + +Calling `.add(name, …)` twice with the same `name` is a builder-time error naming the duplicate. The builder is immutable — every `.add(…)` returns a new builder — so the resulting registration is safe to share across contracts that attach the same pack. + +## How packs and contracts compose + +Index-type composition is **per-contract**, not workspace-global. Two contracts in the same workspace that attach different packs see different valid `type` sets. This is intentional: a contract's vocabulary is a function of its own pack list and nothing else. + +A pack publishes its registration on its descriptor under a single field (`indexTypes`). The pack stores the value verbatim — there is no copy, no transformation. The contract-definition pipeline reads each pack's `indexTypes`, intersects the per-pack maps at the type level, and builds a fresh per-contract registry at the runtime level: + +| Layer | What composes | +| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Type** | The contract's authoring DSL accepts `IndexTypes = TargetIndexTypes & PackAIndexTypes & PackBIndexTypes & …`. `constraints.index(cols, { type, options })` discriminates on `type`; `options` narrows to the entry's shape; an unknown `type` is a compile error. | +| **Runtime** | `assertStorageSemantics` (called from the contract lowering) instantiates `createIndexTypeRegistry()` and walks the same pack list, registering every entry. Two packs registering the same `type` literal surface as a registration-time error naming the conflict. | + +No `declare module` augmentation is used. A global module augmentation would mean every contract in the workspace saw every loaded pack's index types — wrong by composition: a contract that didn't attach `paradedb` shouldn't see `bm25`. The per-pack registration value avoids this entirely. + +## Where validation runs + +The validation seam is the lowering function that turns a `ContractDefinition` (the in-memory IR produced by either authoring chain) into a final `Contract`. Both authoring surfaces converge on this seam: + +- **TS chain**: `defineContract({…})` → `buildContractFromDsl` → `buildSqlContractFromDefinition`. +- **PSL chain**: `interpretPslDocumentToSqlContract` constructs a `ContractDefinition` from the PSL AST and calls the same lowering. + +At the seam, the lowering builds the per-contract registry from the definition's pack list, walks every index in the storage IR, and rejects: + +- Unregistered `type` literals. +- `options` that fail the registered validator (missing required keys, wrong types, extra keys *if* the registrant opted into `'+': 'reject'`). +- `options` set without `type`. + +Errors fire at authoring time — at the line that wrote the offending model — not when a downstream consumer loads the emitted `contract.json`. + +### Index-type validation is authoring-time, not JSON-loading-time + +`validateContract` (called by runtime drivers when loading `contract.json`) does **structural and referential** validation only: it checks shape, internal references between named objects, codec default decoding — things that depend only on the loaded JSON. It deliberately does **not** consult an index-type registry, because the registry isn't part of the JSON. The registry is a function of the *packs attached to the contract definition*, which exist at design time and are gone by the time the JSON is loaded. + +This split is load-bearing: + +- The runtime path stays simple — `validateContract` doesn't need a registry, and a driver loading a contract doesn't need to know about packs. +- A contract that reaches a driver has, by construction, already been validated against its pack-derived registry at the lowering seam. +- A contract authored by a tool that bypasses the lowering will not be checked against any registry — that is the explicit trade-off the layering makes. The expected fix for such a tool is to *use* the lowering, not to teach `validateContract` about packs. + +The validator that runs at the lowering seam lives in `packages/2-sql/1-core/contract/src/index-type-validation.ts`, separate from the JSON-internal-consistency validators in `validators.ts`. The file boundary signals the layering. + +## Strictness is a registrant choice + +`arktype` is loose-by-default: an `options` object with extra keys passes validation unless the validator explicitly rejects them. The framework does not impose a strictness policy on top. + +A registrant opts into strict-key rejection by including `'+': 'reject'` in their option shape (as `bm25` does in the grounding example). The recommendation is to do so: an entry's option shape is a contract between the registrant and the renderer, and an unrecognised key is far more likely to be a typo than a genuine extension point. Silently dropping it at validate time would mask it from authors and produce surprising DDL. + +The choice belongs to the registrant because the registrant is the one who knows whether their option set is closed (everything that's accepted is enumerated) or genuinely open (e.g. forward-compat with new method-specific knobs). + +## Rendering: framework-owned, single path + +The Postgres adapter's `createIndex` reads `type` and `options` directly from the validated IR and renders: + +```sql +CREATE INDEX ON USING () WITH ( = , …) +``` + +There is **no per-entry rendering hook**. A single universal renderer formats `options` as `key = literal, …`, using the adapter's existing scalar quoting and escaping helpers for strings, numbers, and booleans. `null` and `NaN` are rejected at the renderer. + +Two consequences are worth naming: + +- **The universal renderer is sufficient because validators constrain leaves to scalars.** There is no entry whose options need bespoke rendering, because no entry can declare an options shape with non-scalar leaves. +- **SQL-injection risk is bounded to framework-owned helpers.** An extension author cannot accidentally introduce an unsafe rendering path; the only path that produces SQL string fragments from extension data is the one the framework controls and tests. + +## Index identity and migration semantics + +The schema verifier treats `(columns, type, options)` as the identity of an index. A contract index whose `type` differs from the live database's index — or whose `options` differ — is a real mismatch and is reported as one. Option comparison is *loose* (string-coerced both sides) to absorb the fact that `pg_class.reloptions` stores values as text regardless of the original literal type, so a contract `fillfactor: 70` matches a Postgres `'70'`. + +Any change to `columns`, `type`, or `options` is rendered by the migration planner as `DROP INDEX` followed by `CREATE INDEX`. Postgres has no `ALTER INDEX … SET METHOD` for changing the index method, and option changes are inconsistent across `WITH` keys, so `ALTER` is the wrong primitive for these fields uniformly. The DROP+CREATE shape is a property of how Postgres handles index method and storage-parameter changes, not a choice this design imposes. + +Postgres introspection populates `SqlIndexIR.type` from `pg_am.amname` and `SqlIndexIR.options` from `pg_class.reloptions`, with one asymmetry: when `pg_am.amname` is `'btree'` (the Postgres default), the introspected `type` is dropped to `undefined`. Without that, a contract index without an explicit `type` would never match a default-method index in the live database, and every plan against an unchanged DB would force DROP+CREATE. + +## Authoring surfaces: TS and PSL + +Both surfaces reach the same lowering and therefore the same registry. They differ in what they accept syntactically: + +- **TS authoring** is unconstrained by the validator's expressiveness. `arktype` validators can describe any leaf type, so TS callers can write `options: { fastupdate: false }` against a hypothetical `gin` registration that accepts boolean leaves. +- **PSL grammar** carries `options` as an object literal whose values are string leaves. PSL authors can write `options: { key_field: "id" }`, but `options: { fastupdate: false }` does not parse — the diagnostic explicitly says so and points at the TS surface for non-string options. This is a property of the PSL grammar, not the registry. + +A contract authored half-and-half (some models in PSL, some in TS, against the same pack list) is consistent because both surfaces flow through the same `assertStorageSemantics` call. + +## Consequences + +### Positive + +- **Adding an index type is a single declaration.** A pack writes one `defineIndexTypes()….add(…)` call and stores the value on its descriptor. Authoring narrowing, runtime validation, and DDL rendering all light up without touching framework code. +- **Errors fire at the call site.** Unknown types and bad option shapes are compile errors at the line that wrote them, or runtime errors at the lowering with the model name attached. Neither manifests as surprise DDL. +- **The IR vocabulary is dialect-neutral.** `type` and `options` are free of Postgres-specific keywords (`USING`, `WITH`). A contract is portable across SQL adapters even though only the Postgres renderer exists. +- **Composition is per-contract.** Two contracts that attach different packs see different valid `type` sets. The vocabulary follows the pack list; nothing leaks across contracts. + +### Negative + +- **The renderer is Postgres-shaped.** Other SQL adapters that want to read `type`/`options` need their own rendering path. The IR vocabulary is neutral; the rendering is per-adapter by design, but the cost of a new adapter going through these fields is real. +- **PSL accepts only string-leaf options.** Until the PSL grammar is extended for non-string leaves, registrants whose options shape needs booleans or numbers must document that PSL authoring is not supported and direct authors at the TS surface. +- **Tools that bypass the lowering bypass the registry check.** A tool that produces `contract.json` directly — without going through `defineContract` or PSL interpretation — does not get index-type validation. This is consistent with the layering (the registry isn't part of the JSON), but it is a real limit on JSON-as-an-API-surface. + +## Non-goals + +- **`ALTER INDEX` rendering paths for `type`/`options` changes.** Always `DROP` + `CREATE`. Postgres has no clean `ALTER` primitive for index method or for the heterogeneous space of `WITH` keys. +- **Per-column index options.** `options` is a single record on the index node, not per-column. Per-column operator classes (e.g. for `gist` and `gin`) live in their own design space and would attach to the column reference, not the index. +- **Capability gating per index type.** Capabilities describe the runtime environment (is this server version, this connection, this extension installed). The registry is the *design-time* vocabulary — it answers "can this contract name `bm25`?", not "does the database have ParadeDB installed?". The latter surfaces as a Postgres DDL error at apply time, which is the correct boundary. +- **Built-in registry entries seeded by the framework.** The registry is open by design; there is no fixed list of "official" types. Default-method (B-tree) indexes are expressed by *omitting* `type` entirely, which is consistent with the introspection rule that drops `'btree'` to `undefined`. + +## Alternatives considered + +### Per-entry rendering hooks + +Let each registered entry carry a function that turns its `options` into a SQL fragment. Rejected on uniformity and security grounds. The framework already exposes safe scalar quoting helpers; an extension authoring its own renderer would either duplicate them or, worse, build SQL by string concatenation. The universal renderer is sufficient because validators constrain leaves to scalars. + +### `declare module` augmentation for index types + +A common TypeScript pattern: each pack augments a global type to add its entries. Rejected because it does not compose with per-contract pack lists — every contract in the workspace would see the union of all packs ever loaded, not just its own. Storing the registration on the pack value keeps the merged set scoped to each contract's `defineContract` call. + +### Capability gating per index type + +The capability system ([ADR 117](ADR%20117%20-%20Extension%20capability%20keys.md)) negotiates runtime environment features (server version, installed extensions). It is not the right vocabulary for a *design-time* decision about whether a contract can name a given `type` value. A registered entry asserts a vocabulary, not a runtime property. If the database lacks the underlying server-side extension, Postgres surfaces that as a DDL error at apply time — the correct layer for a runtime failure. + +### Closed-set identifier syntax in PSL (`type: BTree`) + +Prisma's stable PSL uses identifier values for `@@index(type:)`. Rejected because the registry is open-ended by design: extension packs contribute new types, and a closed-set grammar would either need to be regenerated per workspace or fall through to a string-typed argument anyway. PSL accepts a string-quoted `type` value, validated downstream against the merged registry exactly the same way the TS surface is. + +## References + +- [ADR 117 — Extension capability keys](ADR%20117%20-%20Extension%20capability%20keys.md). The orthogonal mechanism that index types are *not*: capabilities describe the runtime environment, the registry describes the design-time vocabulary. +- [ADR 161 — Explicit foreign key constraint and index configuration](ADR%20161%20-%20Explicit%20foreign%20key%20constraint%20and%20index%20configuration.md). Neighbouring decision in the index/constraint area; same model of explicit, contract-visible configuration with per-node fields. diff --git a/packages/1-framework/2-authoring/contract/src/descriptors.ts b/packages/1-framework/2-authoring/contract/src/descriptors.ts index 9f7d624484..e5b3d708d8 100644 --- a/packages/1-framework/2-authoring/contract/src/descriptors.ts +++ b/packages/1-framework/2-authoring/contract/src/descriptors.ts @@ -8,8 +8,8 @@ export type ColumnTypeDescriptor = { export interface IndexDef { readonly columns: readonly string[]; readonly name?: string; - readonly using?: string; - readonly config?: Record; + readonly type?: string; + readonly options?: Record; } export interface ForeignKeyDefaultsState { diff --git a/packages/1-framework/2-authoring/contract/test/descriptors.test.ts b/packages/1-framework/2-authoring/contract/test/descriptors.test.ts index 80a181d265..f76f3f4278 100644 --- a/packages/1-framework/2-authoring/contract/test/descriptors.test.ts +++ b/packages/1-framework/2-authoring/contract/test/descriptors.test.ts @@ -30,15 +30,15 @@ describe('descriptor exports', () => { const index: IndexDef = { columns: ['email'], name: 'user_email_idx', - using: 'btree', - config: { fillfactor: 90 }, + type: 'btree', + options: { fillfactor: 90 }, }; expect(index).toEqual({ columns: ['email'], name: 'user_email_idx', - using: 'btree', - config: { fillfactor: 90 }, + type: 'btree', + options: { fillfactor: 90 }, }); }); }); diff --git a/packages/2-sql/1-core/contract/package.json b/packages/2-sql/1-core/contract/package.json index ef8b84c9aa..ebf9beefb3 100644 --- a/packages/2-sql/1-core/contract/package.json +++ b/packages/2-sql/1-core/contract/package.json @@ -33,6 +33,8 @@ ], "exports": { "./factories": "./dist/factories.mjs", + "./index-type-validation": "./dist/index-type-validation.mjs", + "./index-types": "./dist/index-types.mjs", "./pack-types": "./dist/pack-types.mjs", "./types": "./dist/types.mjs", "./validate": "./dist/validate.mjs", diff --git a/packages/2-sql/1-core/contract/src/exports/index-type-validation.ts b/packages/2-sql/1-core/contract/src/exports/index-type-validation.ts new file mode 100644 index 0000000000..1b49ee3075 --- /dev/null +++ b/packages/2-sql/1-core/contract/src/exports/index-type-validation.ts @@ -0,0 +1 @@ +export * from '../index-type-validation'; diff --git a/packages/2-sql/1-core/contract/src/exports/index-types.ts b/packages/2-sql/1-core/contract/src/exports/index-types.ts new file mode 100644 index 0000000000..5157bc9021 --- /dev/null +++ b/packages/2-sql/1-core/contract/src/exports/index-types.ts @@ -0,0 +1,9 @@ +export { + createIndexTypeRegistry, + defineIndexTypes, + type IndexTypeBuilder, + type IndexTypeEntry, + type IndexTypeMap, + type IndexTypeRegistration, + type IndexTypeRegistry, +} from '../index-types'; diff --git a/packages/2-sql/1-core/contract/src/index-type-validation.ts b/packages/2-sql/1-core/contract/src/index-type-validation.ts new file mode 100644 index 0000000000..f934b20f67 --- /dev/null +++ b/packages/2-sql/1-core/contract/src/index-type-validation.ts @@ -0,0 +1,37 @@ +import type { Contract } from '@prisma-next/contract/types'; +import { ContractValidationError } from '@prisma-next/contract/validate-contract'; +import { type } from 'arktype'; +import type { IndexTypeRegistry } from './index-types'; +import type { SqlStorage } from './types'; + +export function validateIndexTypes( + contract: Contract, + indexTypeRegistry: IndexTypeRegistry, +): void { + for (const [tableName, table] of Object.entries(contract.storage.tables)) { + for (const index of table.indexes) { + if (index.type === undefined && index.options !== undefined) { + throw new ContractValidationError( + `Table "${tableName}" index on columns [${index.columns.join(', ')}] has options without a type`, + 'storage', + ); + } + if (index.type === undefined) continue; + const entry = indexTypeRegistry.get(index.type); + if (entry === undefined) { + throw new ContractValidationError( + `Table "${tableName}" index on columns [${index.columns.join(', ')}] uses unregistered index type "${index.type}"`, + 'storage', + ); + } + const optionsValue = index.options ?? {}; + const result = entry.options(optionsValue); + if (result instanceof type.errors) { + throw new ContractValidationError( + `Table "${tableName}" index on columns [${index.columns.join(', ')}] has invalid options for type "${index.type}": ${result.summary}`, + 'storage', + ); + } + } + } +} diff --git a/packages/2-sql/1-core/contract/src/index-types.ts b/packages/2-sql/1-core/contract/src/index-types.ts new file mode 100644 index 0000000000..20b3030d65 --- /dev/null +++ b/packages/2-sql/1-core/contract/src/index-types.ts @@ -0,0 +1,77 @@ +import type { Type } from 'arktype'; + +export interface IndexTypeEntry { + readonly type: string; + readonly options: Type; +} + +export type IndexTypeMap = { readonly [K in string]: { readonly options: unknown } }; + +export interface IndexTypeRegistration> { + readonly IndexTypes: TMap; + readonly entries: ReadonlyArray; +} + +export interface IndexTypeBuilder> + extends IndexTypeRegistration { + add( + typeLiteral: TLit, + entry: { readonly options: Type }, + ): IndexTypeBuilder>; +} + +class IndexTypeBuilderImpl implements IndexTypeBuilder { + readonly entries: ReadonlyArray; + readonly IndexTypes: TMap; + + constructor(entries: ReadonlyArray) { + this.entries = entries; + this.IndexTypes = {} as TMap; + } + + add( + typeLiteral: TLit, + entry: { readonly options: Type }, + ): IndexTypeBuilder> { + if (this.entries.some((e) => e.type === typeLiteral)) { + throw new Error(`Index type "${typeLiteral}" is already declared in this builder`); + } + return new IndexTypeBuilderImpl>([ + ...this.entries, + { type: typeLiteral, options: entry.options as Type }, + ]); + } +} + +export function defineIndexTypes(): IndexTypeBuilder> { + return new IndexTypeBuilderImpl([]); +} + +export interface IndexTypeRegistry { + register(entry: IndexTypeEntry): void; + get(typeLiteral: string): IndexTypeEntry | undefined; + has(typeLiteral: string): boolean; +} + +class IndexTypeRegistryImpl implements IndexTypeRegistry { + private readonly entries = new Map(); + + register(entry: IndexTypeEntry): void { + if (this.entries.has(entry.type)) { + throw new Error(`Index type "${entry.type}" is already registered`); + } + this.entries.set(entry.type, entry); + } + + get(typeLiteral: string): IndexTypeEntry | undefined { + return this.entries.get(typeLiteral); + } + + has(typeLiteral: string): boolean { + return this.entries.has(typeLiteral); + } +} + +export function createIndexTypeRegistry(): IndexTypeRegistry { + return new IndexTypeRegistryImpl(); +} diff --git a/packages/2-sql/1-core/contract/src/index.ts b/packages/2-sql/1-core/contract/src/index.ts index a9ab36142b..f50d8e143d 100644 --- a/packages/2-sql/1-core/contract/src/index.ts +++ b/packages/2-sql/1-core/contract/src/index.ts @@ -1,4 +1,6 @@ export * from './exports/factories'; +export * from './exports/index-type-validation'; +export * from './exports/index-types'; export * from './exports/types'; export * from './exports/validate'; export * from './exports/validators'; diff --git a/packages/2-sql/1-core/contract/src/types.ts b/packages/2-sql/1-core/contract/src/types.ts index 27d80bcc6a..eb3229cc7b 100644 --- a/packages/2-sql/1-core/contract/src/types.ts +++ b/packages/2-sql/1-core/contract/src/types.ts @@ -43,16 +43,8 @@ export type UniqueConstraint = { export type Index = { readonly columns: readonly string[]; readonly name?: string; - /** - * Optional access method identifier. - * Extension-specific methods are represented as strings and interpreted - * by the owning extension package. - */ - readonly using?: string; - /** - * Optional extension-owned index configuration payload. - */ - readonly config?: Record; + readonly type?: string; + readonly options?: Record; }; export type ForeignKeyReferences = { diff --git a/packages/2-sql/1-core/contract/src/validators.ts b/packages/2-sql/1-core/contract/src/validators.ts index 02787daac1..15ed484c1a 100644 --- a/packages/2-sql/1-core/contract/src/validators.ts +++ b/packages/2-sql/1-core/contract/src/validators.ts @@ -96,8 +96,8 @@ const UniqueConstraintSchema = type.declare().type({ export const IndexSchema = type({ columns: type.string.array().readonly(), 'name?': 'string', - 'using?': 'string', - 'config?': 'Record', + 'type?': 'string', + 'options?': 'Record', }); export const ForeignKeyReferencesSchema = type.declare().type({ @@ -375,6 +375,9 @@ export function validateStorageSemantics(storage: SqlStorage): string[] { seenUniqueDefinitions.add(signature); } + const sortOptions = (o: Record | undefined): Record | null => + o ? Object.fromEntries(Object.entries(o).sort(([a], [b]) => a.localeCompare(b))) : null; + const seenIndexDefinitions = new Set(); for (const index of table.indexes) { const duplicateColumn = findDuplicateValue(index.columns); @@ -384,8 +387,8 @@ export function validateStorageSemantics(storage: SqlStorage): string[] { const signature = JSON.stringify({ columns: index.columns, - using: index.using ?? null, - config: index.config ?? null, + type: index.type ?? null, + options: sortOptions(index.options), }); if (seenIndexDefinitions.has(signature)) { errors.push( diff --git a/packages/2-sql/1-core/contract/test/index-types.test.ts b/packages/2-sql/1-core/contract/test/index-types.test.ts new file mode 100644 index 0000000000..f756bfc4d5 --- /dev/null +++ b/packages/2-sql/1-core/contract/test/index-types.test.ts @@ -0,0 +1,83 @@ +import { type } from 'arktype'; +import { describe, expect, it } from 'vitest'; +import { createIndexTypeRegistry, defineIndexTypes } from '../src/index-types'; + +describe('defineIndexTypes builder', () => { + it('starts empty', () => { + const builder = defineIndexTypes(); + expect(builder.entries).toEqual([]); + }); + + it('add() yields a new builder with the entry appended', () => { + const optionsValidator = type({ key_field: 'string' }); + const builder = defineIndexTypes().add('bm25', { options: optionsValidator }); + expect(builder.entries).toHaveLength(1); + expect(builder.entries[0]?.type).toBe('bm25'); + expect(builder.entries[0]?.options).toBe(optionsValidator); + }); + + it('add() composes multiple distinct entries in order', () => { + const a = type({ a: 'string' }); + const b = type({ b: 'string' }); + const builder = defineIndexTypes().add('alpha', { options: a }).add('beta', { options: b }); + expect(builder.entries.map((e) => e.type)).toEqual(['alpha', 'beta']); + }); + + it('add() does not mutate the prior builder', () => { + const opts = type({ x: 'string' }); + const a = defineIndexTypes(); + const b = a.add('alpha', { options: opts }); + expect(a.entries).toEqual([]); + expect(b.entries).toHaveLength(1); + }); + + it('add() throws on duplicate type literal in the same builder', () => { + const opts = type({ x: 'string' }); + const builder = defineIndexTypes().add('dup', { options: opts }); + expect(() => builder.add('dup', { options: opts })).toThrow(/already declared/); + }); +}); + +describe('createIndexTypeRegistry', () => { + it('register stores an entry; get returns it', () => { + const registry = createIndexTypeRegistry(); + const entry = { type: 'demo', options: type({ fillfactor: 'number' }) }; + registry.register(entry); + expect(registry.get('demo')).toBe(entry); + }); + + it('has reports presence', () => { + const registry = createIndexTypeRegistry(); + expect(registry.has('absent')).toBe(false); + registry.register({ type: 'present', options: type({ k: 'string' }) }); + expect(registry.has('present')).toBe(true); + }); + + it('get returns undefined for unknown types', () => { + const registry = createIndexTypeRegistry(); + expect(registry.get('nonesuch')).toBeUndefined(); + }); + + it('register throws on duplicate type', () => { + const registry = createIndexTypeRegistry(); + const opts = type({ key: 'string' }); + registry.register({ type: 'gin', options: opts }); + expect(() => registry.register({ type: 'gin', options: opts })).toThrow(/already registered/); + }); + + it('error message names the offending type', () => { + const registry = createIndexTypeRegistry(); + registry.register({ type: 'gist', options: type({ k: 'string' }) }); + expect(() => registry.register({ type: 'gist', options: type({ k: 'string' }) })).toThrow( + /gist/, + ); + }); + + it('two registries are independent', () => { + const a = createIndexTypeRegistry(); + const b = createIndexTypeRegistry(); + a.register({ type: 'shared', options: type({ k: 'string' }) }); + expect(a.has('shared')).toBe(true); + expect(b.has('shared')).toBe(false); + }); +}); diff --git a/packages/2-sql/1-core/contract/test/validate.test.ts b/packages/2-sql/1-core/contract/test/validate.test.ts index 4ec840ec0f..8b2e4c068b 100644 --- a/packages/2-sql/1-core/contract/test/validate.test.ts +++ b/packages/2-sql/1-core/contract/test/validate.test.ts @@ -1,7 +1,10 @@ import type { Contract } from '@prisma-next/contract/types'; import { ContractValidationError } from '@prisma-next/contract/validate-contract'; import { emptyCodecLookup } from '@prisma-next/framework-components/codec'; +import { type } from 'arktype'; import { describe, expect, expectTypeOf, it } from 'vitest'; +import { validateIndexTypes } from '../src/index-type-validation'; +import { createIndexTypeRegistry } from '../src/index-types'; import type { SqlStorage } from '../src/types'; import { validateContract } from '../src/validate'; @@ -892,4 +895,92 @@ describe('validateContract', () => { >(); }); }); + + describe('validateIndexTypes', () => { + function makeContractWithIndex(index: Record) { + return makeContract({ + User: { + columns: { + id: { codecId: 'pg/text@1', nativeType: 'text', nullable: false }, + body: { codecId: 'pg/text@1', nativeType: 'text', nullable: false }, + }, + primaryKey: { columns: ['id'] }, + uniques: [], + indexes: [index], + foreignKeys: [], + }, + }) as Contract; + } + + it('accepts a contract whose index references a registered type with valid options', () => { + const registry = createIndexTypeRegistry(); + registry.register({ + type: 'demo', + options: type({ fillfactor: 'number' }), + }); + const contract = makeContractWithIndex({ + columns: ['body'], + type: 'demo', + options: { fillfactor: 70 }, + }); + expect(() => validateIndexTypes(contract, registry)).not.toThrow(); + }); + + it('rejects an index whose type is not registered', () => { + const registry = createIndexTypeRegistry(); + const contract = makeContractWithIndex({ columns: ['body'], type: 'made-up' }); + expect(() => validateIndexTypes(contract, registry)).toThrow(/made-up/); + }); + + it('rejects an index whose options fail the registered validator', () => { + const registry = createIndexTypeRegistry(); + registry.register({ + type: 'demo', + options: type({ fillfactor: 'number' }), + }); + const contract = makeContractWithIndex({ + columns: ['body'], + type: 'demo', + options: { fillfactor: 'not-a-number' }, + }); + expect(() => validateIndexTypes(contract, registry)).toThrow(/fillfactor/); + }); + + it('rejects extra option keys in strict mode', () => { + const registry = createIndexTypeRegistry(); + registry.register({ + type: 'demo', + options: type({ '+': 'reject', fillfactor: 'number' }), + }); + const contract = makeContractWithIndex({ + columns: ['body'], + type: 'demo', + options: { fillfactor: 70, unknown: 1 }, + }); + expect(() => validateIndexTypes(contract, registry)).toThrow(/unknown/); + }); + + it('rejects an index that has options without a type', () => { + const registry = createIndexTypeRegistry(); + const contract = makeContractWithIndex({ + columns: ['body'], + options: { fillfactor: 70 }, + }); + expect(() => validateIndexTypes(contract, registry)).toThrow(/options/); + }); + + it('accepts an index without type or options', () => { + const registry = createIndexTypeRegistry(); + const contract = makeContractWithIndex({ columns: ['body'] }); + expect(() => validateIndexTypes(contract, registry)).not.toThrow(); + }); + + it('rejects a typed index against an empty registry', () => { + const registry = createIndexTypeRegistry(); + const contract = makeContractWithIndex({ columns: ['body'], type: 'made-up' }); + expect(() => validateIndexTypes(contract, registry)).toThrow( + /unregistered index type "made-up"/, + ); + }); + }); }); diff --git a/packages/2-sql/1-core/contract/test/validators.test.ts b/packages/2-sql/1-core/contract/test/validators.test.ts index 88235f1581..67cb89109c 100644 --- a/packages/2-sql/1-core/contract/test/validators.test.ts +++ b/packages/2-sql/1-core/contract/test/validators.test.ts @@ -732,6 +732,31 @@ describe('SQL contract validators', () => { expect(errors[0]).toContain('NOT NULL'); }); + it('detects duplicate index definitions whose options differ only in key order', () => { + const s = createContract({ + storage: { + tables: { + user: table( + { + id: col('int4', 'pg/int4@1'), + email: col('text', 'pg/text@1'), + }, + { + indexes: [ + { columns: ['email'], type: 'gin', options: { a: '1', b: '2' } }, + { columns: ['email'], type: 'gin', options: { b: '2', a: '1' } }, + ], + }, + ), + }, + }, + }).storage; + + const errors = validateStorageSemantics(s); + expect(errors).toHaveLength(1); + expect(errors[0]).toContain('duplicate index definition'); + }); + it('rejects duplicate foreign key definitions within the same table', () => { const s = createContract({ storage: { diff --git a/packages/2-sql/1-core/contract/tsdown.config.ts b/packages/2-sql/1-core/contract/tsdown.config.ts index 859a81b68b..a38e34e25b 100644 --- a/packages/2-sql/1-core/contract/tsdown.config.ts +++ b/packages/2-sql/1-core/contract/tsdown.config.ts @@ -7,5 +7,7 @@ export default defineConfig({ 'src/exports/validate.ts', 'src/exports/factories.ts', 'src/exports/pack-types.ts', + 'src/exports/index-types.ts', + 'src/exports/index-type-validation.ts', ], }); diff --git a/packages/2-sql/1-core/schema-ir/src/types.ts b/packages/2-sql/1-core/schema-ir/src/types.ts index 4b754a60f8..85c522cfe0 100644 --- a/packages/2-sql/1-core/schema-ir/src/types.ts +++ b/packages/2-sql/1-core/schema-ir/src/types.ts @@ -68,6 +68,8 @@ export type SqlIndexIR = { readonly columns: readonly string[]; readonly name?: string; readonly unique: boolean; + readonly type?: string; + readonly options?: Record; readonly annotations?: SqlAnnotations; }; diff --git a/packages/2-sql/2-authoring/contract-psl/package.json b/packages/2-sql/2-authoring/contract-psl/package.json index 58e44f2227..bcdd7065d9 100644 --- a/packages/2-sql/2-authoring/contract-psl/package.json +++ b/packages/2-sql/2-authoring/contract-psl/package.json @@ -29,6 +29,7 @@ "@prisma-next/test-utils": "workspace:*", "@prisma-next/tsconfig": "workspace:*", "@prisma-next/tsdown": "workspace:*", + "arktype": "^2.1.25", "tsdown": "catalog:", "typescript": "catalog:", "vitest": "catalog:" diff --git a/packages/2-sql/2-authoring/contract-psl/src/interpreter.ts b/packages/2-sql/2-authoring/contract-psl/src/interpreter.ts index a82eda1984..83f8ce07c9 100644 --- a/packages/2-sql/2-authoring/contract-psl/src/interpreter.ts +++ b/packages/2-sql/2-authoring/contract-psl/src/interpreter.ts @@ -43,11 +43,13 @@ import { notOk, ok, type Result } from '@prisma-next/utils/result'; import { findDuplicateFieldName, getAttribute, + getNamedArgument, getPositionalArgument, mapFieldNamesToColumns, parseAttributeFieldList, parseConstraintMapArgument, parseMapName, + parseObjectLiteralStringMap, parseQuotedStringLiteral, } from './psl-attribute-parsing'; import type { ColumnDescriptor } from './psl-column-resolution'; @@ -697,9 +699,51 @@ function buildModelNodeFromPsl(input: BuildModelNodeInput): BuildModelNodeResult ...ifDefined('name', constraintName), }); } else { + const indexEntityLabel = `Model "${model.name}" @@index`; + const rawTypeArg = getNamedArgument(modelAttribute, 'type'); + let indexType: string | undefined; + if (rawTypeArg !== undefined) { + const parsed = parseQuotedStringLiteral(rawTypeArg); + if (parsed === undefined) { + diagnostics.push({ + code: 'PSL_INVALID_ATTRIBUTE_ARGUMENT', + message: `${indexEntityLabel} type argument must be a quoted string literal`, + sourceId, + span: modelAttribute.span, + }); + continue; + } + indexType = parsed; + } + const rawOptionsArg = getNamedArgument(modelAttribute, 'options'); + let indexOptions: Record | undefined; + if (rawOptionsArg !== undefined) { + if (indexType === undefined) { + diagnostics.push({ + code: 'PSL_INVALID_ATTRIBUTE_ARGUMENT', + message: `${indexEntityLabel} options argument requires a type argument`, + sourceId, + span: modelAttribute.span, + }); + continue; + } + const parsed = parseObjectLiteralStringMap({ + raw: rawOptionsArg, + diagnostics, + sourceId, + span: modelAttribute.span, + entityLabel: indexEntityLabel, + }); + if (parsed === undefined) { + continue; + } + indexOptions = parsed; + } indexNodes.push({ columns: columnNames, ...ifDefined('name', constraintName), + ...ifDefined('type', indexType), + ...ifDefined('options', indexOptions), }); } continue; diff --git a/packages/2-sql/2-authoring/contract-psl/src/psl-attribute-parsing.ts b/packages/2-sql/2-authoring/contract-psl/src/psl-attribute-parsing.ts index 69dd905a52..7e958e335f 100644 --- a/packages/2-sql/2-authoring/contract-psl/src/psl-attribute-parsing.ts +++ b/packages/2-sql/2-authoring/contract-psl/src/psl-attribute-parsing.ts @@ -133,6 +133,132 @@ export function getPositionalArguments(attribute: PslAttribute): readonly string .map((arg) => (arg.kind === 'positional' ? arg.value : '')); } +/** + * Parses a PSL object-literal attribute argument value of the form + * `{ key1: "value1", key2: "value2" }` into a `Record`. + * + * V1 admits string literals only as leaf values. Boolean and number + * literals are rejected. Trailing commas are allowed. + * + * Returns the parsed record, or pushes a diagnostic and returns undefined + * on malformed input or non-string leaves. + */ +export function parseObjectLiteralStringMap(input: { + readonly raw: string; + readonly diagnostics: ContractSourceDiagnostic[]; + readonly sourceId: string; + readonly span: PslSpan; + readonly entityLabel: string; +}): Record | undefined { + const trimmed = input.raw.trim(); + if (!trimmed.startsWith('{') || !trimmed.endsWith('}')) { + return pushInvalidAttributeArgument({ + diagnostics: input.diagnostics, + sourceId: input.sourceId, + span: input.span, + message: `${input.entityLabel} expected an object literal value of the form { key: "value", ... }`, + }); + } + const body = trimmed.slice(1, -1).trim(); + if (body.length === 0) { + return {}; + } + const result: Record = {}; + for (const part of splitObjectLiteralEntries(body)) { + const colonAt = findTopLevelColon(part); + if (colonAt === -1) { + return pushInvalidAttributeArgument({ + diagnostics: input.diagnostics, + sourceId: input.sourceId, + span: input.span, + message: `${input.entityLabel} object-literal entry "${part}" is missing a "key: value" colon`, + }); + } + const key = part.slice(0, colonAt).trim(); + const rawValue = part.slice(colonAt + 1).trim(); + if (key.length === 0 || !/^[A-Za-z_][A-Za-z0-9_]*$/.test(key)) { + return pushInvalidAttributeArgument({ + diagnostics: input.diagnostics, + sourceId: input.sourceId, + span: input.span, + message: `${input.entityLabel} object-literal key "${key}" must be a bare identifier`, + }); + } + const parsedString = parseQuotedStringLiteral(rawValue); + if (parsedString === undefined) { + return pushInvalidAttributeArgument({ + diagnostics: input.diagnostics, + sourceId: input.sourceId, + span: input.span, + message: `${input.entityLabel} object-literal value for "${key}" must be a quoted string literal (V1 PSL @@index options support string leaves only; use the TS authoring surface for non-string options)`, + }); + } + if (Object.hasOwn(result, key)) { + return pushInvalidAttributeArgument({ + diagnostics: input.diagnostics, + sourceId: input.sourceId, + span: input.span, + message: `${input.entityLabel} object-literal key "${key}" appears more than once`, + }); + } + result[key] = parsedString; + } + return result; +} + +function splitObjectLiteralEntries(body: string): readonly string[] { + const parts: string[] = []; + let depthBrace = 0; + let depthBracket = 0; + let depthParen = 0; + let quote: '"' | "'" | null = null; + let start = 0; + for (let index = 0; index < body.length; index += 1) { + const ch = body[index] ?? ''; + if (quote) { + if (ch === quote && body[index - 1] !== '\\') { + quote = null; + } + continue; + } + if (ch === '"' || ch === "'") { + quote = ch; + continue; + } + if (ch === '{') depthBrace += 1; + else if (ch === '}') depthBrace = Math.max(0, depthBrace - 1); + else if (ch === '[') depthBracket += 1; + else if (ch === ']') depthBracket = Math.max(0, depthBracket - 1); + else if (ch === '(') depthParen += 1; + else if (ch === ')') depthParen = Math.max(0, depthParen - 1); + else if (ch === ',' && depthBrace === 0 && depthBracket === 0 && depthParen === 0) { + const segment = body.slice(start, index).trim(); + if (segment.length > 0) parts.push(segment); + start = index + 1; + } + } + const tail = body.slice(start).trim(); + if (tail.length > 0) parts.push(tail); + return parts; +} + +function findTopLevelColon(entry: string): number { + let quote: '"' | "'" | null = null; + for (let index = 0; index < entry.length; index += 1) { + const ch = entry[index] ?? ''; + if (quote) { + if (ch === quote && entry[index - 1] !== '\\') quote = null; + continue; + } + if (ch === '"' || ch === "'") { + quote = ch; + continue; + } + if (ch === ':') return index; + } + return -1; +} + export function pushInvalidAttributeArgument(input: { readonly diagnostics: ContractSourceDiagnostic[]; readonly sourceId: string; diff --git a/packages/2-sql/2-authoring/contract-psl/test/interpreter.test.ts b/packages/2-sql/2-authoring/contract-psl/test/interpreter.test.ts index 45d9369aeb..295eed53b3 100644 --- a/packages/2-sql/2-authoring/contract-psl/test/interpreter.test.ts +++ b/packages/2-sql/2-authoring/contract-psl/test/interpreter.test.ts @@ -1,5 +1,7 @@ import { parsePslDocument } from '@prisma-next/psl-parser'; +import { defineIndexTypes } from '@prisma-next/sql-contract/index-types'; import type { SqlStorage } from '@prisma-next/sql-contract/types'; +import { type } from 'arktype'; import { describe, expect, it } from 'vitest'; import { type InterpretPslDocumentToSqlContractInput, @@ -11,6 +13,15 @@ import { postgresTarget, } from './fixtures'; +const testIndexPack = { + kind: 'extension', + id: 'test-index-pack', + familyId: 'sql', + targetId: 'postgres', + version: '0.0.1', + indexTypes: defineIndexTypes().add('bm25', { options: type('object') }), +} as const; + describe('interpretPslDocumentToSqlContract', () => { const builtinControlMutationDefaults = createBuiltinLikeControlMutationDefaults(); const interpretPslDocumentToSqlContract = ( @@ -556,4 +567,176 @@ model OrderItem { }, }); }); + + describe('@@index type and options', () => { + it('lowers @@index([body], type: "bm25", options: { key_field: "id" }) to an IR index node with type and options', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { key_field: "id" }, map: "doc_body_bm25_idx") +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + composedExtensionPacks: [testIndexPack.id], + composedExtensionPackRefs: [testIndexPack], + }); + + expect(result.ok).toBe(true); + if (!result.ok) return; + + expect(result.value.storage).toMatchObject({ + tables: { + doc: { + indexes: [ + { + columns: ['body'], + name: 'doc_body_bm25_idx', + type: 'bm25', + options: { key_field: 'id' }, + }, + ], + }, + }, + }); + }); + + it('accepts a multi-key options object with string-literal leaves', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { key_field: "id", language: "en" }) +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + composedExtensionPacks: [testIndexPack.id], + composedExtensionPackRefs: [testIndexPack], + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.storage).toMatchObject({ + tables: { + doc: { + indexes: [{ type: 'bm25', options: { key_field: 'id', language: 'en' } }], + }, + }, + }); + }); + + it('rejects a non-string-literal leaf in options (boolean)', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { key_field: "id", fastupdate: false }) +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect( + result.failure.diagnostics.some((d) => /must be a quoted string literal/.test(d.message)), + ).toBe(true); + }); + + it('rejects a non-string-literal leaf in options (number)', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { fillfactor: 70 }) +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect( + result.failure.diagnostics.some((d) => /must be a quoted string literal/.test(d.message)), + ).toBe(true); + }); + + it('rejects an options argument with no surrounding type argument', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body], options: { key_field: "id" }) +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect( + result.failure.diagnostics.some((d) => + /options argument requires a type argument/.test(d.message), + ), + ).toBe(true); + }); + + it('rejects a malformed options object literal', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { not_an_assignment }) +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect( + result.failure.diagnostics.some((d) => /missing a "key: value" colon/.test(d.message)), + ).toBe(true); + }); + + it('accepts @@index without type or options (existing behaviour unchanged)', () => { + const document = parsePslDocument({ + schema: `model Doc { + id Int @id + body String + @@index([body]) +}`, + sourceId: 'schema.prisma', + }); + + const result = interpretPslDocumentToSqlContract({ + document, + controlMutationDefaults: builtinControlMutationDefaults, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.storage).toMatchObject({ + tables: { doc: { indexes: [{ columns: ['body'] }] } }, + }); + }); + }); }); diff --git a/packages/2-sql/2-authoring/contract-psl/test/ts-psl-parity.test.ts b/packages/2-sql/2-authoring/contract-psl/test/ts-psl-parity.test.ts index a1d730ff07..7044f2280e 100644 --- a/packages/2-sql/2-authoring/contract-psl/test/ts-psl-parity.test.ts +++ b/packages/2-sql/2-authoring/contract-psl/test/ts-psl-parity.test.ts @@ -293,7 +293,7 @@ const representativeTsAuthoring = `defineContract( relations: { author: rel.belongsTo(User, { from: 'authorId', to: 'id' }) }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.authorId, { name: 'post_author_id_idx' })], + indexes: [constraints.index([cols.authorId], { name: 'post_author_id_idx' })], foreignKeys: [constraints.foreignKey(cols.authorId, User.refs.id, { name: 'post_author_id_fkey', onDelete: 'cascade' })], })); return { types, models: { User, Post } }; @@ -334,7 +334,7 @@ function buildTsContract() { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.authorId, { name: 'post_author_id_idx' })], + indexes: [constraints.index([cols.authorId], { name: 'post_author_id_idx' })], foreignKeys: [ constraints.foreignKey(cols.authorId, UserBase.refs.id, { name: 'post_author_id_fkey', diff --git a/packages/2-sql/2-authoring/contract-ts/src/build-contract.ts b/packages/2-sql/2-authoring/contract-ts/src/build-contract.ts index 9d8e7adb05..de1d281017 100644 --- a/packages/2-sql/2-authoring/contract-ts/src/build-contract.ts +++ b/packages/2-sql/2-authoring/contract-ts/src/build-contract.ts @@ -17,6 +17,12 @@ import { type StorageHashBase, } from '@prisma-next/contract/types'; import type { CodecLookup } from '@prisma-next/framework-components/codec'; +import { validateIndexTypes } from '@prisma-next/sql-contract/index-type-validation'; +import { + createIndexTypeRegistry, + type IndexTypeMap, + type IndexTypeRegistration, +} from '@prisma-next/sql-contract/index-types'; import { applyFkDefaults, type SqlStorage, @@ -63,11 +69,37 @@ function encodeColumnDefault( }; } -function assertStorageSemantics(storage: SqlStorage): void { - const semanticErrors = validateStorageSemantics(storage); +function assertStorageSemantics( + definition: ContractDefinition, + contract: Contract, +): void { + const semanticErrors = validateStorageSemantics(contract.storage); if (semanticErrors.length > 0) { throw new Error(`Contract semantic validation failed: ${semanticErrors.join('; ')}`); } + + const indexTypeRegistry = createIndexTypeRegistry(); + const packsToRegister: ReadonlyArray<{ readonly id?: string; readonly indexTypes?: unknown }> = [ + definition.target, + ...Object.values(definition.extensionPacks ?? {}), + ]; + for (const pack of packsToRegister) { + const registration = pack.indexTypes; + if (registration === undefined) continue; + if ( + typeof registration !== 'object' || + registration === null || + !Array.isArray((registration as { entries?: unknown }).entries) + ) { + throw new Error( + `Pack "${pack.id ?? ''}" declares "indexTypes" but its value is not an IndexTypeRegistration (expected an object with an "entries" array; got ${typeof registration}).`, + ); + } + for (const entry of (registration as IndexTypeRegistration).entries) { + indexTypeRegistry.register(entry); + } + } + validateIndexTypes(contract, indexTypeRegistry); } function assertKnownTargetModel( @@ -277,8 +309,8 @@ export function buildSqlContractFromDefinition( indexes: (semanticModel.indexes ?? []).map((i) => ({ columns: i.columns, ...ifDefined('name', i.name), - ...ifDefined('using', i.using), - ...ifDefined('config', i.config), + ...ifDefined('type', i.type), + ...ifDefined('options', i.options), })), foreignKeys, ...(semanticModel.id @@ -447,7 +479,7 @@ export function buildSqlContractFromDefinition( meta: {}, }; - assertStorageSemantics(contract.storage); + assertStorageSemantics(definition, contract); return contract; } diff --git a/packages/2-sql/2-authoring/contract-ts/src/composed-authoring-helpers.ts b/packages/2-sql/2-authoring/contract-ts/src/composed-authoring-helpers.ts index 7373fe6a2f..34082127e9 100644 --- a/packages/2-sql/2-authoring/contract-ts/src/composed-authoring-helpers.ts +++ b/packages/2-sql/2-authoring/contract-ts/src/composed-authoring-helpers.ts @@ -26,7 +26,14 @@ import type { TupleFromArgumentDescriptors, UnionToIntersection, } from './authoring-type-utils'; +import type { + AnyRelationBuilder, + ContractModelBuilder, + IndexTypeMap, + ScalarFieldBuilder, +} from './contract-dsl'; import { buildFieldPreset, field, model, rel } from './contract-dsl'; +import type { MergeExtensionIndexTypes } from './contract-types'; type ExtractTypeNamespaceFromPack = Pack extends { readonly authoring?: { readonly type?: infer Namespace extends AuthoringTypeNamespace }; @@ -93,6 +100,33 @@ type TypeHelpersFromNamespace = { type CoreFieldHelpers = Pick; +type MergeAllPackIndexTypes = MergeExtensionIndexTypes< + { readonly __family: Family; readonly __target: Target } & (ExtensionPacks extends Record< + string, + unknown + > + ? ExtensionPacks + : Record) +>; + +type PackAwareModel = { + < + const ModelName extends string, + Fields extends Record, + Relations extends Record = Record, + >( + modelName: ModelName, + input: { readonly fields: Fields; readonly relations?: Relations }, + ): ContractModelBuilder; + < + Fields extends Record, + Relations extends Record = Record, + >(input: { + readonly fields: Fields; + readonly relations?: Relations; + }): ContractModelBuilder; +}; + export type ComposedAuthoringHelpers< Family extends FamilyPackRef, Target extends TargetPackRef<'sql', string>, @@ -104,7 +138,7 @@ export type ComposedAuthoringHelpers< ExtractFieldNamespaceFromPack & MergeExtensionFieldNamespaces >; - readonly model: typeof model; + readonly model: PackAwareModel>; readonly rel: typeof rel; readonly type: TypeHelpersFromNamespace< ExtractTypeNamespaceFromPack & diff --git a/packages/2-sql/2-authoring/contract-ts/src/contract-definition.ts b/packages/2-sql/2-authoring/contract-ts/src/contract-definition.ts index 49f2daca36..3daefcdd40 100644 --- a/packages/2-sql/2-authoring/contract-ts/src/contract-definition.ts +++ b/packages/2-sql/2-authoring/contract-ts/src/contract-definition.ts @@ -31,8 +31,8 @@ export interface UniqueConstraintNode { export interface IndexNode { readonly columns: readonly string[]; readonly name?: string; - readonly using?: string; - readonly config?: Record; + readonly type?: string; + readonly options?: Record; } export interface ForeignKeyNode { diff --git a/packages/2-sql/2-authoring/contract-ts/src/contract-dsl.ts b/packages/2-sql/2-authoring/contract-ts/src/contract-dsl.ts index 00a402e089..d0b8819c75 100644 --- a/packages/2-sql/2-authoring/contract-ts/src/contract-dsl.ts +++ b/packages/2-sql/2-authoring/contract-ts/src/contract-dsl.ts @@ -471,7 +471,7 @@ export type RelationState = | ManyToManyRelation; type AnyRelationState = RelationState; -type AnyRelationBuilder = RelationBuilder; +export type AnyRelationBuilder = RelationBuilder; type ApplyBelongsToRelationSqlSpec< State extends RelationState, @@ -543,11 +543,21 @@ type ConstraintOptions = { readonly name?: Name; }; -type IndexOptions = - ConstraintOptions & { - readonly using?: string; - readonly config?: Record; - }; +export type IndexTypeMap = Record; + +type IndexInput< + Name extends string | undefined, + IndexTypes extends IndexTypeMap, +> = keyof IndexTypes extends never + ? ConstraintOptions + : + | (ConstraintOptions & { readonly type?: never; readonly options?: never }) + | { + readonly [K in keyof IndexTypes & string]: ConstraintOptions & { + readonly type: K; + readonly options: IndexTypes[K]['options']; + }; + }[keyof IndexTypes & string]; type ForeignKeyOptions = ConstraintOptions & { @@ -583,8 +593,8 @@ export type IndexConstraint< readonly kind: 'index'; readonly fields: FieldNames; readonly name?: Name; - readonly using?: string; - readonly config?: Record; + readonly type?: string; + readonly options?: Record; }; export type ForeignKeyConstraint< @@ -629,7 +639,7 @@ function normalizeTargetFieldRefInput(input: TargetFieldRef | readonly TargetFie }; } -function createConstraintsDsl() { +function createConstraintsDsl>() { function ref( modelName: ModelName, fieldName: FieldName, @@ -680,24 +690,26 @@ function createConstraintsDsl() { }; } - function index( - field: ColumnRef, - options?: IndexOptions, - ): IndexConstraint; function index( fields: { readonly [K in keyof FieldNames]: ColumnRef }, - options?: IndexOptions, + options?: IndexInput, ): IndexConstraint; function index( - fieldOrFields: ColumnRef | readonly ColumnRef[], - options?: IndexOptions, + fields: readonly ColumnRef[], + options?: { + readonly name?: string; + readonly type?: string; + readonly options?: unknown; + }, ): IndexConstraint { return { kind: 'index', - fields: normalizeFieldRefInput(fieldOrFields), - ...(options?.name ? { name: options.name } : {}), - ...(options?.using ? { using: options.using } : {}), - ...(options?.config ? { config: options.config } : {}), + fields: normalizeFieldRefInput(fields), + ...(options?.name !== undefined ? { name: options.name } : {}), + ...(options?.type !== undefined ? { type: options.type } : {}), + ...(options?.options !== undefined + ? { options: options.options as Record } + : {}), }; } @@ -782,9 +794,26 @@ type AttributeContext> = { readonly constraints: Pick; }; -type SqlContext> = { +type PackAwareIndex = < + FieldNames extends readonly string[], + Name extends string | undefined = undefined, +>( + fields: { readonly [K in keyof FieldNames]: ColumnRef }, + options?: IndexInput, +) => IndexConstraint; + +type PackAwareSqlConstraints = { + readonly foreignKey: ConstraintsDsl['foreignKey']; + readonly ref: ConstraintsDsl['ref']; + readonly index: PackAwareIndex; +}; + +export type SqlContext< + Fields extends Record, + IndexTypes extends IndexTypeMap = Record, +> = { readonly cols: FieldRefs; - readonly constraints: Pick; + readonly constraints: PackAwareSqlConstraints; }; function createFieldRefs>( @@ -835,8 +864,10 @@ function createAttributeConstraintsDsl(): AttributeContext< }; } -function createSqlConstraintsDsl(): SqlContext>['constraints'] { - const constraints = createConstraintsDsl(); +function createSqlConstraintsDsl< + IndexTypes extends IndexTypeMap = Record, +>(): SqlContext, IndexTypes>['constraints'] { + const constraints = createConstraintsDsl(); return { index: constraints.index, foreignKey: constraints.foreignKey, @@ -952,12 +983,14 @@ export class ContractModelBuilder< Relations extends Record = Record, AttributesSpec extends ModelAttributesSpec | undefined = undefined, SqlSpec extends SqlStageSpec | undefined = undefined, + IndexTypes extends IndexTypeMap = Record, > { declare readonly __name: ModelName; declare readonly __fields: Fields; declare readonly __relations: Relations; declare readonly __attributes: AttributesSpec; declare readonly __sql: SqlSpec; + declare readonly __indexTypes: IndexTypes; readonly refs: ModelName extends string ? ModelTokenRefs : never; constructor( @@ -967,7 +1000,7 @@ export class ContractModelBuilder< readonly relations: Relations; }, readonly attributesFactory?: StageInput, AttributesSpec>, - readonly sqlFactory?: StageInput, SqlSpec>, + readonly sqlFactory?: StageInput, SqlSpec>, ) { this.refs = ( stageOne.modelName ? createModelTokenRefs(stageOne.modelName, stageOne.fields) : undefined @@ -976,7 +1009,7 @@ export class ContractModelBuilder< ref( this: ModelName extends string - ? ContractModelBuilder + ? ContractModelBuilder : never, fieldName: FieldName, ): TargetFieldRef { @@ -995,7 +1028,14 @@ export class ContractModelBuilder< relations>( relations: NextRelations, - ): ContractModelBuilder { + ): ContractModelBuilder< + ModelName, + Fields, + Relations & NextRelations, + AttributesSpec, + SqlSpec, + IndexTypes + > { const duplicateRelationName = findDuplicateRelationName(this.stageOne.relations, relations); if (duplicateRelationName) { throw new Error( @@ -1021,15 +1061,15 @@ export class ContractModelBuilder< AttributeContext, ValidateAttributesStageSpec >, - ): ContractModelBuilder { + ): ContractModelBuilder { return new ContractModelBuilder(this.stageOne, specOrFactory, this.sqlFactory); } sql( - specOrFactory: StageInput, NextSqlSpec>, + specOrFactory: StageInput, NextSqlSpec>, ): [ValidateSqlStageSpec] extends [never] - ? ContractModelBuilder - : ContractModelBuilder { + ? ContractModelBuilder + : ContractModelBuilder { // Conditional return type cannot be verified by the implementation; the // runtime value is always a valid ContractModelBuilder regardless of the // validation outcome (validation is type-level only). @@ -1053,7 +1093,7 @@ export class ContractModelBuilder< } return buildStageSpec(this.sqlFactory, { cols: createColumnRefs(this.stageOne.fields), - constraints: createSqlConstraintsDsl() as SqlContext['constraints'], + constraints: createSqlConstraintsDsl(), }); } } diff --git a/packages/2-sql/2-authoring/contract-ts/src/contract-lowering.ts b/packages/2-sql/2-authoring/contract-ts/src/contract-lowering.ts index 31f419ac78..713a931298 100644 --- a/packages/2-sql/2-authoring/contract-ts/src/contract-lowering.ts +++ b/packages/2-sql/2-authoring/contract-ts/src/contract-lowering.ts @@ -1,5 +1,6 @@ import type { ColumnTypeDescriptor } from '@prisma-next/contract-authoring'; import type { StorageTypeInstance } from '@prisma-next/sql-contract/types'; +import { ifDefined } from '@prisma-next/utils/defined'; import type { ContractDefinition, FieldNode, @@ -573,9 +574,9 @@ function resolveModelNode( })) satisfies readonly UniqueConstraintNode[]; const indexes = (spec.sqlSpec?.indexes ?? []).map((index) => ({ columns: mapFieldNamesToColumnNames(spec.modelName, index.fields, spec.fieldToColumn), - ...(index.name ? { name: index.name } : {}), - ...(index.using ? { using: index.using } : {}), - ...(index.config ? { config: index.config } : {}), + ...ifDefined('name', index.name), + ...ifDefined('type', index.type), + ...ifDefined('options', index.options), })) satisfies readonly IndexNode[]; const foreignKeys = resolveForeignKeyNodes(spec, allSpecs); const relations = Object.entries(spec.relations).map(([relationName, relationBuilder]) => diff --git a/packages/2-sql/2-authoring/contract-ts/src/contract-types.ts b/packages/2-sql/2-authoring/contract-ts/src/contract-types.ts index 0043f06bfc..ff4415f7af 100644 --- a/packages/2-sql/2-authoring/contract-ts/src/contract-types.ts +++ b/packages/2-sql/2-authoring/contract-ts/src/contract-types.ts @@ -5,6 +5,7 @@ import type { StorageHashBase, } from '@prisma-next/contract/types'; import type { ExtensionPackRef, TargetPackRef } from '@prisma-next/framework-components/components'; +import type { IndexTypeRegistration } from '@prisma-next/sql-contract/index-types'; import type { ContractWithTypeMaps, Index, @@ -34,6 +35,28 @@ type MergeExtensionCodecTypesSafe = : MergeExtensionCodecTypes : Record; +export type ExtractIndexTypesFromPack

= P extends { + readonly indexTypes: IndexTypeRegistration; +} + ? M + : Record; + +type AllIndexTypeLiterals = + Packs extends Record + ? { [K in keyof Packs]: keyof ExtractIndexTypesFromPack }[keyof Packs] & string + : never; + +export type MergeExtensionIndexTypes> = { + readonly [Lit in AllIndexTypeLiterals]: Extract< + { + [K in keyof Packs]: Lit extends keyof ExtractIndexTypesFromPack + ? ExtractIndexTypesFromPack[Lit] + : never; + }[keyof Packs], + { readonly options: unknown } + >; +}; + export type MergeExtensionPackRefs< Existing extends Record | undefined, Added extends Record>, @@ -64,6 +87,16 @@ type CodecTypesFromDefinition = ExtractCodecTypesFromPack< > & MergeExtensionCodecTypesSafe>; +type DefinitionTarget = Definition extends { readonly target: infer Target } + ? Target + : never; + +type AllPacks = DefinitionExtensionPacks & { + readonly __target: DefinitionTarget; +}; + +export type IndexTypesFromDefinition = MergeExtensionIndexTypes>; + type DefinitionModels = Definition extends { readonly models?: unknown; } diff --git a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.constraints.test.ts b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.constraints.test.ts index 3842c30833..1f8cb48662 100644 --- a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.constraints.test.ts +++ b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.constraints.test.ts @@ -2,6 +2,7 @@ import type { FamilyPackRef, TargetPackRef } from '@prisma-next/framework-compon import { describe, expect, it } from 'vitest'; import { type ContractInput, defineContract, field, model, rel } from '../src/contract-builder'; import { columnDescriptor } from './helpers/column-descriptor'; +import { testIndexPack } from './helpers/test-index-pack'; const int4Column = columnDescriptor('pg/int4@1'); const textColumn = columnDescriptor('pg/text@1'); @@ -122,7 +123,7 @@ describe('contract definition constraint support', () => { }, }).sql(({ cols, constraints }) => ({ table: 'user', - indexes: [constraints.index(cols.email)], + indexes: [constraints.index([cols.email])], })), }, }); @@ -141,7 +142,7 @@ describe('contract definition constraint support', () => { }, }).sql(({ cols, constraints }) => ({ table: 'user', - indexes: [constraints.index(cols.email, { name: 'user_email_idx' })], + indexes: [constraints.index([cols.email], { name: 'user_email_idx' })], })), }, }); @@ -230,13 +231,72 @@ describe('contract definition constraint support', () => { }, }).sql(({ cols, constraints }) => ({ table: 'user', - indexes: [constraints.index(cols.id, { name: 'user_pkey' })], + indexes: [constraints.index([cols.id], { name: 'user_pkey' })], })), }, }), ).toThrow(/Contract semantic validation failed:.*user_pkey/); }); + it('throws a contextual error when an extension pack declares a malformed indexTypes value', () => { + const malformedIndexPack = { + kind: 'extension', + id: 'malformed-pack', + familyId: 'sql', + targetId: 'postgres', + version: '0.0.1', + indexTypes: 'oops', + } as const; + + expect(() => + defineContract({ + family: bareFamilyPack, + target: postgresTargetPack, + // The pack is intentionally malformed for this test; the runtime + // shape check is what we want to exercise. + extensionPacks: { malformed: malformedIndexPack as unknown as typeof testIndexPack }, + models: { + Doc: model('Doc', { + fields: { + id: field.column(int4Column).id(), + }, + }).sql({ table: 'doc' }), + }, + }), + ).toThrow(/malformed-pack/); + }); + + it('throws at authoring time when an index uses an unregistered type', () => { + expect(() => + defineContract( + { + family: bareFamilyPack, + target: postgresTargetPack, + extensionPacks: { testIndexes: testIndexPack }, + }, + ({ model: helperModel, field: helperField }) => ({ + models: { + Doc: helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + constraints.index([cols.body], { + // @ts-expect-error - exercise the authoring-time runtime validator on an unregistered type literal. + type: 'made-up', + options: {}, + }), + ], + })), + }, + }), + ), + ).toThrow(/unregistered index type "made-up"/); + }); + it('supports multiple constraints on the same table', () => { const contract = defineTestContract({ models: { @@ -248,7 +308,7 @@ describe('contract definition constraint support', () => { }, }).sql(({ cols, constraints }) => ({ table: 'user', - indexes: [constraints.index(cols.email), constraints.index(cols.username)], + indexes: [constraints.index([cols.email]), constraints.index([cols.username])], })), }, }); diff --git a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.dsl.test.ts b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.dsl.test.ts index a57817bf54..a6ef025bb9 100644 --- a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.dsl.test.ts +++ b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.dsl.test.ts @@ -123,7 +123,7 @@ describe('contract DSL authoring surface', () => { }, }).sql(({ cols, constraints }) => ({ table: 'blog_post', - indexes: [constraints.index(cols.userId, { name: 'blog_post_user_id_idx' })], + indexes: [constraints.index([cols.userId], { name: 'blog_post_user_id_idx' })], foreignKeys: [ constraints.foreignKey([cols.userId], [UserBase.refs['id']!], { name: 'blog_post_user_id_fkey', @@ -396,7 +396,7 @@ describe('contract DSL authoring surface', () => { }, }).sql(({ cols, constraints }) => ({ table: 'app_user', - indexes: [constraints.index(cols.id, { name: 'app_user_pkey' })], + indexes: [constraints.index([cols.id], { name: 'app_user_pkey' })], })); expect(() => @@ -560,7 +560,7 @@ describe('contract DSL authoring surface', () => { authorId: field.column(textColumn).column('author_identifier'), }, }).sql(({ cols, constraints }) => ({ - indexes: [constraints.index(cols.authorId, { name: 'blog_post_author_identifier_idx' })], + indexes: [constraints.index([cols.authorId], { name: 'blog_post_author_identifier_idx' })], })); const contract = defineTestContract({ @@ -799,7 +799,7 @@ describe('contract DSL authoring surface', () => { User.ref('posts'); return { - indexes: [constraints.index(cols.email)], + indexes: [constraints.index([cols.email])], }; }); diff --git a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.index-types.test.ts b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.index-types.test.ts new file mode 100644 index 0000000000..7d829c86e1 --- /dev/null +++ b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.index-types.test.ts @@ -0,0 +1,68 @@ +import type { ExtensionPackRef, TargetPackRef } from '@prisma-next/framework-components/components'; +import type { IndexTypeRegistration } from '@prisma-next/sql-contract/index-types'; +import { describe, expectTypeOf, it } from 'vitest'; +import type { + ExtractIndexTypesFromPack, + IndexTypesFromDefinition, + MergeExtensionIndexTypes, +} from '../src/contract-types'; + +type DemoIndexTypes = { + readonly demo: { readonly options: { readonly fillfactor: number } }; +}; + +type AnalyticsIndexTypes = { + readonly analytics: { readonly options: { readonly bucket: string } }; +}; + +type DemoPack = ExtensionPackRef<'sql', 'postgres'> & { + readonly indexTypes: IndexTypeRegistration; +}; + +type AnalyticsPack = ExtensionPackRef<'sql', 'postgres'> & { + readonly indexTypes: IndexTypeRegistration; +}; + +describe('index-type pack threading', () => { + it("ExtractIndexTypesFromPack pulls the registration's IndexTypes off a pack", () => { + expectTypeOf>().toEqualTypeOf(); + }); + + it('ExtractIndexTypesFromPack returns an empty record for packs without indexTypes', () => { + type PlainPack = ExtensionPackRef<'sql', 'postgres'>; + expectTypeOf>().toEqualTypeOf>(); + }); + + it('MergeExtensionIndexTypes intersects across multiple packs', () => { + type ExtractedDemo = ExtractIndexTypesFromPack; + type ExtractedAnalytics = ExtractIndexTypesFromPack; + expectTypeOf().toEqualTypeOf(); + expectTypeOf().toEqualTypeOf(); + type Merged = MergeExtensionIndexTypes<{ + demo: DemoPack; + analytics: AnalyticsPack; + }>; + type DemoOptions = Merged['demo']['options']; + type AnalyticsOptions = Merged['analytics']['options']; + expectTypeOf().toEqualTypeOf<{ readonly fillfactor: number }>(); + expectTypeOf().toEqualTypeOf<{ readonly bucket: string }>(); + }); + + it('IndexTypesFromDefinition merges target + extension packs', () => { + type Definition = { + readonly target: TargetPackRef<'sql', 'postgres'>; + readonly extensionPacks: { readonly demo: DemoPack; readonly analytics: AnalyticsPack }; + }; + type Resolved = IndexTypesFromDefinition; + type DemoOptions = Resolved['demo']['options']; + type AnalyticsOptions = Resolved['analytics']['options']; + expectTypeOf().toEqualTypeOf<{ readonly fillfactor: number }>(); + expectTypeOf().toEqualTypeOf<{ readonly bucket: string }>(); + }); + + it('IndexTypesFromDefinition is an empty record when no packs contribute', () => { + type Definition = { readonly target: TargetPackRef<'sql', 'postgres'> }; + type Resolved = IndexTypesFromDefinition; + expectTypeOf().toEqualTypeOf>(); + }); +}); diff --git a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.normalization.test.ts b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.normalization.test.ts index cc1f74b0c4..063e57a0b5 100644 --- a/packages/2-sql/2-authoring/contract-ts/test/contract-builder.normalization.test.ts +++ b/packages/2-sql/2-authoring/contract-ts/test/contract-builder.normalization.test.ts @@ -2,6 +2,7 @@ import type { FamilyPackRef, TargetPackRef } from '@prisma-next/framework-compon import { describe, expect, it } from 'vitest'; import { defineContract, field, model } from '../src/contract-builder'; import { columnDescriptor } from './helpers/column-descriptor'; +import { testIndexPack } from './helpers/test-index-pack'; const int4Column = columnDescriptor('pg/int4@1'); const textColumn = columnDescriptor('pg/text@1'); @@ -166,83 +167,41 @@ describe('contract builder normalization', () => { expect(contract.storage.tables.user.columns.email.nullable).toBe(false); }); - it('passes through extension-owned index fields (using, config)', () => { - const contract = defineContract({ - family: bareFamilyPack, - target: postgresTargetPack, - models: { - Item: model('Item', { - fields: { - id: field.column(int4Column).id(), - description: field.column(textColumn), - }, - }).sql(({ cols, constraints }) => ({ - table: 'items', - indexes: [ - constraints.index(cols.description, { - name: 'search_idx', - using: 'bm25', - config: { - keyField: 'id', - fields: [{ column: 'description', tokenizer: 'simple' }], - }, - }), - ], - })), + it('passes type and options on indexes through to storage IR', () => { + const contract = defineContract( + { + family: bareFamilyPack, + target: postgresTargetPack, + extensionPacks: { testIndexes: testIndexPack }, }, - }); + ({ model, field }) => ({ + models: { + Item: model('Item', { + fields: { + id: field.column(int4Column).id(), + description: field.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'items', + indexes: [ + constraints.index([cols.description], { + name: 'search_idx', + type: 'bm25', + options: { key_field: 'id' }, + }), + ], + })), + }, + }), + ); const indexes = contract.storage.tables.items.indexes; expect(indexes).toHaveLength(1); - expect(indexes[0]).toMatchObject({ + expect(indexes[0]).toEqual({ columns: ['description'], - using: 'bm25', + type: 'bm25', name: 'search_idx', - config: { - keyField: 'id', - fields: [{ column: 'description', tokenizer: 'simple' }], - }, - }); - }); - - it('passes through extension index config with expression fields', () => { - const contract = defineContract({ - family: bareFamilyPack, - target: postgresTargetPack, - models: { - Item: model('Item', { - fields: { - id: field.column(int4Column).id(), - description: field.column(textColumn), - }, - }).sql(({ cols, constraints }) => ({ - table: 'items', - indexes: [ - constraints.index(cols.description, { - using: 'bm25', - config: { - keyField: 'id', - fields: [ - { column: 'description' }, - { - expression: "description || ' ' || category", - alias: 'concat', - tokenizer: 'simple', - }, - ], - }, - }), - ], - })), - }, - }); - - const idx = contract.storage.tables.items.indexes[0]!; - expect(idx.using).toBe('bm25'); - expect((idx.config as { fields: readonly unknown[] }).fields).toHaveLength(2); - expect((idx.config as { fields: readonly unknown[] }).fields[1]).toMatchObject({ - expression: "description || ' ' || category", - alias: 'concat', + options: { key_field: 'id' }, }); }); @@ -258,7 +217,7 @@ describe('contract builder normalization', () => { }, }).sql(({ cols, constraints }) => ({ table: 'user', - indexes: [constraints.index(cols.email)], + indexes: [constraints.index([cols.email])], })), }, }); diff --git a/packages/2-sql/2-authoring/contract-ts/test/contract-dsl.runtime.test.ts b/packages/2-sql/2-authoring/contract-ts/test/contract-dsl.runtime.test.ts index 53e58adb3c..d66a48abac 100644 --- a/packages/2-sql/2-authoring/contract-ts/test/contract-dsl.runtime.test.ts +++ b/packages/2-sql/2-authoring/contract-ts/test/contract-dsl.runtime.test.ts @@ -1,16 +1,24 @@ -import type { TargetPackRef } from '@prisma-next/framework-components/components'; +import type { FamilyPackRef, TargetPackRef } from '@prisma-next/framework-components/components'; import { describe, expect, it } from 'vitest'; +import { createComposedAuthoringHelpers } from '../src/composed-authoring-helpers'; import { applyNaming, field, isContractInput, - model, normalizeRelationFieldNames, rel, resolveRelationModelName, type TargetFieldRef, } from '../src/contract-dsl'; import { columnDescriptor } from './helpers/column-descriptor'; +import { testIndexPack } from './helpers/test-index-pack'; + +const bareFamilyPack: FamilyPackRef<'sql'> = { + kind: 'family', + id: 'sql', + familyId: 'sql', + version: '0.0.1', +}; const postgresTargetPack: TargetPackRef<'sql', 'postgres'> = { kind: 'target', @@ -20,6 +28,12 @@ const postgresTargetPack: TargetPackRef<'sql', 'postgres'> = { version: '0.0.1', }; +const { model } = createComposedAuthoringHelpers({ + family: bareFamilyPack, + target: postgresTargetPack, + extensionPacks: { testIndexes: testIndexPack }, +}); + const int4Column = columnDescriptor('pg/int4@1'); const textColumn = columnDescriptor('pg/text@1'); const charColumn = columnDescriptor('sql/char@1', 'character'); @@ -123,10 +137,10 @@ describe('contract DSL runtime helpers', () => { }, }).sql(({ cols, constraints }) => ({ indexes: [ - constraints.index(cols.teamId, { + constraints.index([cols.teamId], { name: 'audit_log_team_id_idx', - using: 'hash', - config: { fillfactor: 70 }, + type: 'hash', + options: { fillfactor: 70 }, }), ], foreignKeys: [ @@ -146,8 +160,8 @@ describe('contract DSL runtime helpers', () => { kind: 'index', fields: ['teamId'], name: 'audit_log_team_id_idx', - using: 'hash', - config: { fillfactor: 70 }, + type: 'hash', + options: { fillfactor: 70 }, }, ], foreignKeys: [ diff --git a/packages/2-sql/2-authoring/contract-ts/test/contract-lowering.runtime.test.ts b/packages/2-sql/2-authoring/contract-ts/test/contract-lowering.runtime.test.ts index f3bcfb55c3..47d691d8fe 100644 --- a/packages/2-sql/2-authoring/contract-ts/test/contract-lowering.runtime.test.ts +++ b/packages/2-sql/2-authoring/contract-ts/test/contract-lowering.runtime.test.ts @@ -1,9 +1,11 @@ import type { FamilyPackRef, TargetPackRef } from '@prisma-next/framework-components/components'; import { describe, expect, it } from 'vitest'; -import { field, model, rel } from '../src/contract-builder'; +import { createComposedAuthoringHelpers } from '../src/composed-authoring-helpers'; +import { field, rel } from '../src/contract-builder'; import { ContractModelBuilder, ScalarFieldBuilder } from '../src/contract-dsl'; import { buildContractDefinition } from '../src/contract-lowering'; import { columnDescriptor } from './helpers/column-descriptor'; +import { testIndexPack } from './helpers/test-index-pack'; const bareFamilyPack: FamilyPackRef<'sql'> = { kind: 'family', @@ -23,6 +25,12 @@ const postgresTargetPack: TargetPackRef<'sql', 'postgres'> = { const int4Column = columnDescriptor('pg/int4@1'); const textColumn = columnDescriptor('pg/text@1'); +const { model } = createComposedAuthoringHelpers({ + family: bareFamilyPack, + target: postgresTargetPack, + extensionPacks: { testIndexes: testIndexPack }, +}); + function buildDefinition( definition: Omit[0], 'target' | 'family'>, ) { @@ -409,10 +417,10 @@ describe('contract definition lowering runtime checks', () => { }, }).sql(({ cols, constraints }) => ({ indexes: [ - constraints.index(cols.userId, { + constraints.index([cols.userId], { name: 'membership_user_id_idx', - using: 'hash', - config: { fillfactor: 70 }, + type: 'hash', + options: { fillfactor: 70 }, }), ], })); @@ -432,8 +440,8 @@ describe('contract definition lowering runtime checks', () => { { columns: ['userId'], name: 'membership_user_id_idx', - using: 'hash', - config: { fillfactor: 70 }, + type: 'hash', + options: { fillfactor: 70 }, }, ]); expect(membership?.foreignKeys).toEqual([ diff --git a/packages/2-sql/2-authoring/contract-ts/test/helpers/test-index-pack.ts b/packages/2-sql/2-authoring/contract-ts/test/helpers/test-index-pack.ts new file mode 100644 index 0000000000..ff13b10e05 --- /dev/null +++ b/packages/2-sql/2-authoring/contract-ts/test/helpers/test-index-pack.ts @@ -0,0 +1,15 @@ +import { defineIndexTypes } from '@prisma-next/sql-contract/index-types'; +import { type } from 'arktype'; + +const testIndexTypes = defineIndexTypes() + .add('bm25', { options: type('object') }) + .add('hash', { options: type('object') }); + +export const testIndexPack = { + kind: 'extension', + id: 'test-index-pack', + familyId: 'sql', + targetId: 'postgres', + version: '0.0.1', + indexTypes: testIndexTypes, +} as const; diff --git a/packages/2-sql/3-tooling/emitter/src/index.ts b/packages/2-sql/3-tooling/emitter/src/index.ts index 653251e959..0330b4cdbc 100644 --- a/packages/2-sql/3-tooling/emitter/src/index.ts +++ b/packages/2-sql/3-tooling/emitter/src/index.ts @@ -244,11 +244,12 @@ export const sqlEmission = { const indexes = table.indexes .map((i) => { const cols = i.columns.map((c: string) => serializeValue(c)).join(', '); - const name = i.name ? `; readonly name: ${serializeValue(i.name)}` : ''; - const using = i.using !== undefined ? `; readonly using: ${serializeValue(i.using)}` : ''; - const config = - i.config !== undefined ? `; readonly config: ${serializeValue(i.config)}` : ''; - return `{ readonly columns: readonly [${cols}]${name}${using}${config} }`; + const name = i.name !== undefined ? `; readonly name: ${serializeValue(i.name)}` : ''; + const indexType = + i.type !== undefined ? `; readonly type: ${serializeValue(i.type)}` : ''; + const indexOptions = + i.options !== undefined ? `; readonly options: ${serializeValue(i.options)}` : ''; + return `{ readonly columns: readonly [${cols}]${name}${indexType}${indexOptions} }`; }) .join(', '); tableParts.push(`indexes: readonly [${indexes}]`); diff --git a/packages/2-sql/3-tooling/emitter/test/emitter-hook.generation.basic.test.ts b/packages/2-sql/3-tooling/emitter/test/emitter-hook.generation.basic.test.ts index 7c729723f3..9f4c03bc3b 100644 --- a/packages/2-sql/3-tooling/emitter/test/emitter-hook.generation.basic.test.ts +++ b/packages/2-sql/3-tooling/emitter/test/emitter-hook.generation.basic.test.ts @@ -873,9 +873,9 @@ describe('sql-target-family-hook', () => { indexes: [ { columns: ['description'], - using: 'bm25', + type: 'bm25', name: 'search_idx', - config: { + options: { keyField: 'id', fields: [ { @@ -894,8 +894,8 @@ describe('sql-target-family-hook', () => { }); const types = generateContractDts(ir, sqlEmission, [], [], testHashes); - expect(types).toContain("readonly using: 'bm25'"); - expect(types).toContain("readonly config: { readonly keyField: 'id'"); + expect(types).toContain("readonly type: 'bm25'"); + expect(types).toContain("readonly options: { readonly keyField: 'id'"); expect(types).toContain("readonly name: 'search_idx'"); expect(types).toContain("readonly column: 'description'"); expect(types).toContain("readonly tokenizer: 'simple'"); @@ -918,8 +918,8 @@ describe('sql-target-family-hook', () => { indexes: [ { columns: ['description'], - using: 'bm25', - config: { + type: 'bm25', + options: { keyField: 'id', fields: [ { @@ -958,8 +958,8 @@ describe('sql-target-family-hook', () => { indexes: [ { columns: ['description'], - using: 'bm25', - config: { + type: 'bm25', + options: { keyField: 'id', 'min-token-size': 2, fields: [{ column: 'description', tokenizerParams: { 'max-ngram': 5 } }], diff --git a/packages/2-sql/3-tooling/emitter/test/emitter-hook.structure.test.ts b/packages/2-sql/3-tooling/emitter/test/emitter-hook.structure.test.ts index 479c649662..5ab7492794 100644 --- a/packages/2-sql/3-tooling/emitter/test/emitter-hook.structure.test.ts +++ b/packages/2-sql/3-tooling/emitter/test/emitter-hook.structure.test.ts @@ -821,8 +821,8 @@ describe('sql-target-family-hook', () => { indexes: [ { columns: ['description'], - using: 'bm25', - config: { + type: 'bm25', + options: { keyField: 'id', fields: [{ column: 'description', tokenizer: 'simple' }], }, @@ -839,7 +839,7 @@ describe('sql-target-family-hook', () => { }).not.toThrow(); }); - it('still validates index column references independent of extension config', () => { + it('still validates index column references independent of extension options', () => { const ir = createContract({ storage: { tables: { @@ -853,8 +853,8 @@ describe('sql-target-family-hook', () => { indexes: [ { columns: ['nonexistent'], - using: 'bm25', - config: { + type: 'bm25', + options: { keyField: 'id', fields: [{ expression: "description || ' ' || category", tokenizer: 'simple' }], }, diff --git a/packages/2-sql/9-family/src/core/schema-verify/verify-helpers.ts b/packages/2-sql/9-family/src/core/schema-verify/verify-helpers.ts index 8ef469b890..a7071e357f 100644 --- a/packages/2-sql/9-family/src/core/schema-verify/verify-helpers.ts +++ b/packages/2-sql/9-family/src/core/schema-verify/verify-helpers.ts @@ -21,6 +21,39 @@ import type { } from '@prisma-next/sql-schema-ir/types'; import type { ComponentDatabaseDependency } from '../migrations/types'; +function indexOptionsLooselyEqual( + a: Record | undefined, + b: Record | undefined, +): boolean { + const aKeys = a ? Object.keys(a).sort() : []; + const bKeys = b ? Object.keys(b).sort() : []; + if (aKeys.length !== bKeys.length) return false; + for (let i = 0; i < aKeys.length; i += 1) { + if (aKeys[i] !== bKeys[i]) return false; + } + if (aKeys.length === 0) return true; + for (const key of aKeys) { + // Postgres introspection returns reloptions values as raw strings (e.g. + // `'70'`, `'false'`), while contract option leaves are typed (number, + // boolean, string). Compare via String() so a contract `fillfactor: 70` + // matches an introspected `fillfactor: '70'` without a spurious mismatch. + if ( + String((a as Record)[key]) !== String((b as Record)[key]) + ) { + return false; + } + } + return true; +} + +function indexExtrasMatch( + contractIndex: Index, + schemaIndex: { readonly type?: string; readonly options?: Record }, +): boolean { + if ((contractIndex.type ?? null) !== (schemaIndex.type ?? null)) return false; + return indexOptionsLooselyEqual(contractIndex.options, schemaIndex.options); +} + /** * Compares two arrays of strings for equality (order-sensitive). */ @@ -391,13 +424,19 @@ export function verifyIndexes( // Check for any matching index (unique or non-unique) // A unique index can satisfy a non-unique index requirement (stronger satisfies weaker) - const matchingIndex = schemaIndexes.find((idx) => - arraysEqual(idx.columns, contractIndex.columns), + const matchingIndex = schemaIndexes.find( + (idx) => + arraysEqual(idx.columns, contractIndex.columns) && indexExtrasMatch(contractIndex, idx), ); - // Also check if a unique constraint satisfies the index requirement + // Also check if a unique constraint satisfies the index requirement. + // Unique constraints carry no type/options of their own, so they can only + // satisfy a contract index that doesn't request a specific type/options. const matchingUniqueConstraint = - !matchingIndex && schemaUniques.find((u) => arraysEqual(u.columns, contractIndex.columns)); + !matchingIndex && + contractIndex.type === undefined && + contractIndex.options === undefined && + schemaUniques.find((u) => arraysEqual(u.columns, contractIndex.columns)); if (!matchingIndex && !matchingUniqueConstraint) { issues.push({ @@ -442,8 +481,9 @@ export function verifyIndexes( continue; } - const matchingIndex = contractIndexes.find((idx) => - arraysEqual(idx.columns, schemaIndex.columns), + const matchingIndex = contractIndexes.find( + (idx) => + arraysEqual(idx.columns, schemaIndex.columns) && indexExtrasMatch(idx, schemaIndex), ); if (!matchingIndex) { diff --git a/packages/2-sql/9-family/test/schema-verify.helpers.ts b/packages/2-sql/9-family/test/schema-verify.helpers.ts index 6c8c4e236b..dc403bf322 100644 --- a/packages/2-sql/9-family/test/schema-verify.helpers.ts +++ b/packages/2-sql/9-family/test/schema-verify.helpers.ts @@ -84,7 +84,12 @@ export function createContractTable( index?: boolean; }>; uniques?: ReadonlyArray<{ columns: readonly string[]; name?: string }>; - indexes?: ReadonlyArray<{ columns: readonly string[]; name?: string }>; + indexes?: ReadonlyArray<{ + columns: readonly string[]; + name?: string; + type?: string; + options?: Record; + }>; }, ): StorageTable { const result: StorageTable = { @@ -130,7 +135,13 @@ export function createSchemaTable( onUpdate?: SqlReferentialAction; }>; uniques?: ReadonlyArray<{ columns: readonly string[]; name?: string }>; - indexes?: ReadonlyArray<{ columns: readonly string[]; unique: boolean; name?: string }>; + indexes?: ReadonlyArray<{ + columns: readonly string[]; + unique: boolean; + name?: string; + type?: string; + options?: Record; + }>; }, ): SqlTableIR { const result: SqlTableIR = { diff --git a/packages/2-sql/9-family/test/schema-verify.semantic-satisfaction.test.ts b/packages/2-sql/9-family/test/schema-verify.semantic-satisfaction.test.ts index df79f400f7..3932ac6ab4 100644 --- a/packages/2-sql/9-family/test/schema-verify.semantic-satisfaction.test.ts +++ b/packages/2-sql/9-family/test/schema-verify.semantic-satisfaction.test.ts @@ -334,6 +334,142 @@ describe('verifySqlSchema - semantic satisfaction', () => { }), ); }); + + it('fails when contract index type differs from schema index type (same columns)', () => { + const contract = createTestContract({ + doc: createContractTable( + { + id: { nativeType: 'int4', nullable: false }, + body: { nativeType: 'text', nullable: false }, + }, + { indexes: [{ columns: ['body'], type: 'gin' }] }, + ), + }); + + const schema = createTestSchemaIR({ + doc: createSchemaTable( + 'doc', + { + id: { nativeType: 'int4', nullable: false }, + body: { nativeType: 'text', nullable: false }, + }, + { indexes: [{ columns: ['body'], unique: false, name: 'doc_body_idx' }] }, + ), + }); + + const result = verifySqlSchema({ + contract, + schema, + strict: true, + typeMetadataRegistry: emptyTypeMetadataRegistry, + frameworkComponents: [], + }); + + expect(result.ok).toBe(false); + expect(result.schema.issues).toContainEqual( + expect.objectContaining({ kind: 'index_mismatch', table: 'doc' }), + ); + expect(result.schema.issues).toContainEqual( + expect.objectContaining({ kind: 'extra_index', table: 'doc' }), + ); + }); + + it('fails when contract index options differ from schema index options', () => { + const contract = createTestContract({ + doc: createContractTable( + { + id: { nativeType: 'int4', nullable: false }, + body: { nativeType: 'text', nullable: false }, + }, + { + indexes: [{ columns: ['body'], type: 'gin', options: { fastupdate: false } }], + }, + ), + }); + + const schema = createTestSchemaIR({ + doc: createSchemaTable( + 'doc', + { + id: { nativeType: 'int4', nullable: false }, + body: { nativeType: 'text', nullable: false }, + }, + { + indexes: [ + { + columns: ['body'], + unique: false, + name: 'doc_body_idx', + type: 'gin', + options: { fastupdate: true }, + }, + ], + }, + ), + }); + + const result = verifySqlSchema({ + contract, + schema, + strict: true, + typeMetadataRegistry: emptyTypeMetadataRegistry, + frameworkComponents: [], + }); + + expect(result.ok).toBe(false); + expect(result.schema.issues).toContainEqual( + expect.objectContaining({ kind: 'index_mismatch', table: 'doc' }), + ); + expect(result.schema.issues).toContainEqual( + expect.objectContaining({ kind: 'extra_index', table: 'doc' }), + ); + }); + + it('passes when contract type/options match schema type/options exactly', () => { + const contract = createTestContract({ + doc: createContractTable( + { + id: { nativeType: 'int4', nullable: false }, + body: { nativeType: 'text', nullable: false }, + }, + { + indexes: [{ columns: ['body'], type: 'gin', options: { fastupdate: false } }], + }, + ), + }); + + const schema = createTestSchemaIR({ + doc: createSchemaTable( + 'doc', + { + id: { nativeType: 'int4', nullable: false }, + body: { nativeType: 'text', nullable: false }, + }, + { + indexes: [ + { + columns: ['body'], + unique: false, + name: 'doc_body_idx', + type: 'gin', + options: { fastupdate: false }, + }, + ], + }, + ), + }); + + const result = verifySqlSchema({ + contract, + schema, + strict: true, + typeMetadataRegistry: emptyTypeMetadataRegistry, + frameworkComponents: [], + }); + + expect(result.ok).toBe(true); + expect(result.schema.issues).toHaveLength(0); + }); }); describe('foreign key', () => { diff --git a/packages/3-extensions/paradedb/README.md b/packages/3-extensions/paradedb/README.md index fadfcb2c1b..08fa20d6b5 100644 --- a/packages/3-extensions/paradedb/README.md +++ b/packages/3-extensions/paradedb/README.md @@ -4,21 +4,21 @@ ParadeDB full-text search extension pack for Prisma Next. ## Overview -This extension pack adds support for [ParadeDB](https://docs.paradedb.com/) BM25 full-text search indexes in the contract authoring layer. It keeps BM25 metadata inside extension-owned index `config` payloads, so `contract.json` and `contract.d.ts` carry the full search schema without hard-coding ParadeDB types into core SQL index IR. +This extension pack registers a `'bm25'` index type with the SQL family's index-type registry, so contracts can author BM25 full-text search indexes via the standard `constraints.index(...)` surface and the Postgres adapter emits `CREATE INDEX ... USING bm25 WITH (...)` DDL. -This is the **contract-only foundation**. Query-plane support (`@@@` operator, `pdb.*` functions) and migration-plane support (`CREATE INDEX ... USING bm25` DDL generation) are planned as follow-up work. +The v1 surface covers the `key_field` storage parameter only. Per-field tokenizer and column configuration is deferred to expression-index support. ## Responsibilities -- **BM25 Index Authoring**: Typed `bm25.*` field builders for defining BM25 indexes in `contract.ts` -- **Tokenizer Catalog**: `TokenizerId` type union covering all 12 built-in ParadeDB tokenizers -- **Extension Descriptor**: Declares `paradedb/bm25` capability for contract-level feature detection -- **Pack Ref Export**: Ships a pure `/pack` entrypoint for TypeScript contract authoring +- **bm25 index registration**: declares a `'bm25'` entry via `defineIndexTypes()` carrying an arktype validator for the bm25 options shape +- **Extension descriptor**: declares the `paradedb/bm25` capability for contract-level feature detection +- **Pack ref export**: ships a pure `/pack` entrypoint for TypeScript contract authoring ## Dependencies -- **`@prisma-next/contract`**: Core contract types -- **`@prisma-next/contract-authoring`**: Column type descriptor interface +- **`@prisma-next/sql-contract`**: index-type registry primitive +- **`@prisma-next/contract`** / **`@prisma-next/contract-authoring`**: core contract types +- **`arktype`**: option-shape validation ## Installation @@ -28,15 +28,14 @@ pnpm add @prisma-next/extension-paradedb ## Usage -### Contract Definition +### Contract definition -Define BM25 indexes on your tables using `bm25` field builders plus `bm25Index()`: +Author bm25 indexes via the standard `constraints.index(...)` surface; the registered `'bm25'` entry narrows `options` per-`type`: ```typescript -import { int4Column, textColumn, jsonbColumn } from '@prisma-next/adapter-postgres/column-types'; +import { int4Column, textColumn } from '@prisma-next/adapter-postgres/column-types'; import sqlFamily from '@prisma-next/family-sql/pack'; import { defineContract, field, model } from '@prisma-next/sql-contract-ts/contract-builder'; -import { bm25, bm25Index } from '@prisma-next/extension-paradedb/index-types'; import paradedb from '@prisma-next/extension-paradedb/pack'; import postgres from '@prisma-next/target-postgres/pack'; @@ -48,121 +47,40 @@ export const contract = defineContract({ Item: model('Item', { fields: { id: field.column(int4Column).id(), - description: field.column(textColumn), - category: field.column(textColumn), - rating: field.column(int4Column), - metadata: field.column(jsonbColumn), + body: field.column(textColumn), }, - }).sql({ + }).sql(({ cols, constraints }) => ({ table: 'items', indexes: [ - bm25Index({ - keyField: 'id', - fields: [ - bm25.text('description', { tokenizer: 'simple', stemmer: 'english' }), - bm25.text('category'), - bm25.numeric('rating'), - bm25.json('metadata', { tokenizer: 'ngram', min: 2, max: 3 }), - ], - name: 'search_idx', + constraints.index([cols.body], { + name: 'item_body_bm25_idx', + type: 'bm25', + options: { key_field: 'id' }, }), ], - }), + })), }, }); ``` -### Field Builders +### key_field -The `bm25` namespace provides typed field builders that produce `Bm25FieldConfig` objects: - -| Builder | Description | Tokenizer support | -|---------|-------------|-------------------| -| `bm25.text(column, opts?)` | Text field (`text`, `varchar`) | Yes — any tokenizer + stemmer, remove_emojis | -| `bm25.numeric(column)` | Numeric field (filterable, sortable) | No | -| `bm25.boolean(column)` | Boolean field | No | -| `bm25.json(column, opts?)` | JSON/JSONB field | Yes — tokenizer + ngram params | -| `bm25.datetime(column)` | Timestamp/date field | No | -| `bm25.range(column)` | Range field | No | -| `bm25.expression(sql, opts)` | Raw SQL expression | Yes — `alias` required | - -### Expression-Based Fields - -For computed or JSON sub-field indexing, use `bm25.expression()` with a raw SQL string: - -```typescript -.index( - bm25Index({ - keyField: 'id', - fields: [ - bm25.text('description'), - bm25.expression("description || ' ' || category", { - alias: 'concat', - tokenizer: 'simple', - }), - bm25.expression("(metadata->>'color')", { - alias: 'meta_color', - tokenizer: 'ngram', - min: 2, - max: 3, - }), - ], - }), -) -``` - -### keyField Behavior - -ParadeDB BM25 indexes require a `key_field` — a unique column that identifies each document: - -- **Required**: Set `keyField` explicitly in `bm25Index(...)`. -- **Recommended**: Use the table primary key in most cases. -- **Override**: You can choose another unique column when needed. - -```typescript -.index(bm25Index({ keyField: 'id', fields: [bm25.text('body')] })) -.index(bm25Index({ keyField: 'uuid', fields: [bm25.text('body')] })) -``` - -## Tokenizers - -All 12 built-in ParadeDB tokenizers are available via the `TokenizerId` type: - -| Tokenizer | Description | -|-----------|-------------| -| `unicode_words` | Default. Unicode word boundaries (UAX #29). Lowercases. | -| `simple` | Splits on non-alphanumeric. Lowercases. | -| `ngram` | Character n-grams of configurable length. | -| `icu` | ICU Unicode standard segmentation. Multilingual. | -| `regex_pattern` | Regex-based tokenization. | -| `source_code` | camelCase / snake_case splitting. | -| `literal` | No splitting. Exact match, sort, aggregation. | -| `literal_normalized` | Literal + lowercase + token filters. | -| `whitespace` | Whitespace splitting + lowercase. | -| `chinese_compatible` | CJK-aware word segmentation. | -| `jieba` | Chinese segmentation via Jieba. | -| `lindera` | Japanese/Korean/Chinese via Lindera. | +ParadeDB BM25 indexes require a `key_field` — a unique column that identifies each document. It is required, must be a string, and is typically (but not always) the table's primary key. ## Capabilities -The extension declares: - -- `paradedb/bm25`: Indicates support for BM25 full-text search indexes - -## Not Yet Implemented +- `paradedb/bm25` — indicates support for BM25 full-text search indexes -The following are planned for follow-up work: +## Not yet implemented -- **Query plane**: `@@@` operator support in the sql-orm query builder -- **Query plane**: `pdb.*` query builder functions (match, term, phrase, fuzzy, etc.) -- **Migration plane**: `CREATE INDEX ... USING bm25` DDL generation from contract diffs -- **Runtime**: Scoring, aggregation, and highlight functions -- **Database dependencies**: `CREATE EXTENSION pg_search` via migration planner +- Per-column / per-expression tokenizer configuration (deferred to expression-index support) +- `@@@` operator and `pdb.*` query builder functions +- `CREATE EXTENSION pg_search` via migration planner +- Scoring, aggregation, and highlight functions ## References - [ParadeDB documentation](https://docs.paradedb.com/) - [ParadeDB CREATE INDEX](https://docs.paradedb.com/documentation/indexing/create-index) -- [ParadeDB Tokenizers](https://docs.paradedb.com/documentation/tokenizers/overview) -- [pg_search source](https://github.com/paradedb/paradedb/tree/main/pg_search) +- [ADR 210 — Index-type registry](../../../docs/architecture%20docs/adrs/ADR%20210%20-%20Index-type%20registry.md) - [Prisma Next Architecture Overview](../../../docs/Architecture%20Overview.md) diff --git a/packages/3-extensions/paradedb/package.json b/packages/3-extensions/paradedb/package.json index 7d94a3d4e6..41db0b106e 100644 --- a/packages/3-extensions/paradedb/package.json +++ b/packages/3-extensions/paradedb/package.json @@ -15,7 +15,9 @@ }, "dependencies": { "@prisma-next/contract": "workspace:*", - "@prisma-next/contract-authoring": "workspace:*" + "@prisma-next/contract-authoring": "workspace:*", + "@prisma-next/sql-contract": "workspace:*", + "arktype": "^2.1.25" }, "devDependencies": { "@prisma-next/tsconfig": "workspace:*", diff --git a/packages/3-extensions/paradedb/src/core/constants.ts b/packages/3-extensions/paradedb/src/core/constants.ts index d79d481b25..0ac60122d3 100644 --- a/packages/3-extensions/paradedb/src/core/constants.ts +++ b/packages/3-extensions/paradedb/src/core/constants.ts @@ -2,21 +2,3 @@ * Extension ID for ParadeDB pg_search. */ export const PARADEDB_EXTENSION_ID = 'paradedb' as const; - -/** - * Built-in ParadeDB tokenizer IDs. - * These correspond to the `pdb.*` casting syntax in `CREATE INDEX ... USING bm25`. - */ -export type TokenizerId = - | 'unicode_words' - | 'simple' - | 'ngram' - | 'icu' - | 'regex_pattern' - | 'source_code' - | 'literal' - | 'literal_normalized' - | 'whitespace' - | 'chinese_compatible' - | 'jieba' - | 'lindera'; diff --git a/packages/3-extensions/paradedb/src/core/descriptor-meta.ts b/packages/3-extensions/paradedb/src/core/descriptor-meta.ts index c4fc21ba58..cc5ebf11c0 100644 --- a/packages/3-extensions/paradedb/src/core/descriptor-meta.ts +++ b/packages/3-extensions/paradedb/src/core/descriptor-meta.ts @@ -1,3 +1,4 @@ +import { paradedbIndexTypes } from '../types/index-types'; import { PARADEDB_EXTENSION_ID } from './constants'; export const paradedbPackMeta = { @@ -11,4 +12,5 @@ export const paradedbPackMeta = { 'paradedb/bm25': true, }, }, + indexTypes: paradedbIndexTypes, } as const; diff --git a/packages/3-extensions/paradedb/src/exports/index-types.ts b/packages/3-extensions/paradedb/src/exports/index-types.ts index a4aa25bf68..30fa5ea3fa 100644 --- a/packages/3-extensions/paradedb/src/exports/index-types.ts +++ b/packages/3-extensions/paradedb/src/exports/index-types.ts @@ -1,12 +1,2 @@ -export type { TokenizerId } from '../core/constants'; -export type { - Bm25ColumnFieldConfig, - Bm25ExpressionFieldConfig, - Bm25ExpressionFieldOptions, - Bm25FieldConfig, - Bm25IndexConfig, - Bm25IndexOptions, - Bm25JsonFieldOptions, - Bm25TextFieldOptions, -} from '../types/index-types'; -export { bm25, bm25Index } from '../types/index-types'; +export type { Bm25IndexOptions, IndexTypes } from '../types/index-types'; +export { paradedbIndexTypes } from '../types/index-types'; diff --git a/packages/3-extensions/paradedb/src/types/index-types.ts b/packages/3-extensions/paradedb/src/types/index-types.ts index dd7c064bad..f1405dfd16 100644 --- a/packages/3-extensions/paradedb/src/types/index-types.ts +++ b/packages/3-extensions/paradedb/src/types/index-types.ts @@ -1,179 +1,12 @@ -import type { IndexDef } from '@prisma-next/contract-authoring'; -import type { TokenizerId } from '../core/constants'; - -/** - * BM25 field config for a table column. - */ -export type Bm25ColumnFieldConfig = { - readonly column: string; - readonly expression?: never; - readonly tokenizer?: string; - readonly tokenizerParams?: Record; - readonly alias?: string; -}; - -/** - * BM25 field config for a SQL expression. - */ -export type Bm25ExpressionFieldConfig = { - readonly expression: string; - readonly column?: never; - readonly alias: string; - readonly tokenizer?: string; - readonly tokenizerParams?: Record; -}; - -/** - * BM25 field config union. - */ -export type Bm25FieldConfig = Bm25ColumnFieldConfig | Bm25ExpressionFieldConfig; - -/** - * BM25 index configuration payload stored in `IndexDef.config`. - */ -export type Bm25IndexConfig = { - readonly keyField: string; - readonly fields: readonly Bm25FieldConfig[]; -}; - -/** - * Options for a BM25 text field (text, varchar columns). - */ -export type Bm25TextFieldOptions = { - readonly tokenizer?: TokenizerId | (string & {}); - readonly stemmer?: string; - readonly alias?: string; - readonly remove_emojis?: boolean; -}; - -/** - * Options for a BM25 JSON field (json, jsonb columns). - */ -export type Bm25JsonFieldOptions = { - readonly tokenizer?: TokenizerId | (string & {}); - readonly alias?: string; - /** Ngram-specific params when tokenizer is 'ngram'. */ - readonly min?: number; - readonly max?: number; -}; - -/** - * Options for a BM25 expression-based field. - */ -export type Bm25ExpressionFieldOptions = { - readonly alias: string; - readonly tokenizer?: TokenizerId | (string & {}); - readonly min?: number; - readonly max?: number; - readonly stemmer?: string; - readonly pattern?: string; -}; - -type TokenizerConfig = { - readonly tokenizer?: string; - readonly tokenizerParams?: Record; -}; - -/** - * Options for constructing a BM25 index definition. - */ -export type Bm25IndexOptions = { - readonly keyField: string; - readonly fields: readonly Bm25FieldConfig[]; - readonly name?: string; -}; - -/** - * Typed BM25 field builders. - * These produce `Bm25FieldConfig` objects for use in `bm25Index()`. - */ -export const bm25 = { - /** Text field with optional tokenizer config. */ - text(column: string, opts?: Bm25TextFieldOptions): Bm25FieldConfig { - return { - column, - ...buildTokenizerConfig(opts?.tokenizer, { - stemmer: opts?.stemmer, - remove_emojis: opts?.remove_emojis, - }), - ...(opts?.alias !== undefined && { alias: opts.alias }), - }; - }, - - /** Numeric field (filterable, sortable in BM25). */ - numeric(column: string): Bm25FieldConfig { - return { column }; - }, - - /** Boolean field. */ - boolean(column: string): Bm25FieldConfig { - return { column }; - }, - - /** JSON/JSONB field with optional tokenizer config. */ - json(column: string, opts?: Bm25JsonFieldOptions): Bm25FieldConfig { - return { - column, - ...buildTokenizerConfig(opts?.tokenizer, { min: opts?.min, max: opts?.max }), - ...(opts?.alias !== undefined && { alias: opts.alias }), - }; - }, - - /** Datetime (timestamp/date) field. */ - datetime(column: string): Bm25FieldConfig { - return { column }; - }, - - /** Range field. */ - range(column: string): Bm25FieldConfig { - return { column }; - }, - - /** Raw SQL expression field. `alias` is required. */ - expression(sql: string, opts: Bm25ExpressionFieldOptions): Bm25FieldConfig { - return { - expression: sql, - alias: opts.alias, - ...buildTokenizerConfig(opts.tokenizer, { - min: opts.min, - max: opts.max, - stemmer: opts.stemmer, - pattern: opts.pattern, - }), - }; - }, -} as const; - -/** - * Creates a generic index definition with a ParadeDB BM25 payload. - * - * `columns` only includes real table columns so core index validation remains - * target-agnostic. Expression fields stay in extension-owned `config.fields`. - */ -export function bm25Index(opts: Bm25IndexOptions): IndexDef { - return { - columns: opts.fields.flatMap((field) => ('column' in field ? [field.column] : [])), - ...(opts.name !== undefined && { name: opts.name }), - using: 'bm25', - config: { - keyField: opts.keyField, - fields: opts.fields, - } satisfies Bm25IndexConfig, - }; -} - -/** - * Builds `{ tokenizer, tokenizerParams? }` from a tokenizer ID and a bag of params. - * Filters out undefined values and omits `tokenizerParams` when empty. - */ -function buildTokenizerConfig( - tokenizer: string | undefined, - params: Record, -): TokenizerConfig { - if (!tokenizer) return {}; - const filtered = Object.fromEntries(Object.entries(params).filter(([, v]) => v !== undefined)); - return { - tokenizer, - ...(Object.keys(filtered).length > 0 && { tokenizerParams: filtered }), - }; -} +import { defineIndexTypes } from '@prisma-next/sql-contract/index-types'; +import { type } from 'arktype'; + +export const paradedbIndexTypes = defineIndexTypes().add('bm25', { + options: type({ + '+': 'reject', + key_field: 'string', + }), +}); + +export type IndexTypes = typeof paradedbIndexTypes.IndexTypes; +export type Bm25IndexOptions = IndexTypes['bm25']['options']; diff --git a/packages/3-extensions/paradedb/test/index-types.test.ts b/packages/3-extensions/paradedb/test/index-types.test.ts index ff2dc59f44..22b38f5c5e 100644 --- a/packages/3-extensions/paradedb/test/index-types.test.ts +++ b/packages/3-extensions/paradedb/test/index-types.test.ts @@ -1,6 +1,7 @@ +import { type } from 'arktype'; import { describe, expect, it } from 'vitest'; import { paradedbPackMeta } from '../src/core/descriptor-meta'; -import { bm25, bm25Index } from '../src/types/index-types'; +import { paradedbIndexTypes } from '../src/types/index-types'; describe('ParadeDB extension', () => { describe('paradedbPackMeta', () => { @@ -16,163 +17,44 @@ describe('ParadeDB extension', () => { postgres: { 'paradedb/bm25': true }, }); }); - }); - - describe('bm25 field builders', () => { - describe('bm25.text', () => { - it('creates a text field with defaults', () => { - expect(bm25.text('description')).toEqual({ column: 'description' }); - }); - - it('creates a text field with tokenizer', () => { - expect(bm25.text('description', { tokenizer: 'simple' })).toEqual({ - column: 'description', - tokenizer: 'simple', - }); - }); - - it('creates a text field with stemmer', () => { - expect(bm25.text('description', { tokenizer: 'simple', stemmer: 'english' })).toEqual({ - column: 'description', - tokenizer: 'simple', - tokenizerParams: { stemmer: 'english' }, - }); - }); - - it('creates a text field with remove_emojis', () => { - expect( - bm25.text('description', { tokenizer: 'unicode_words', remove_emojis: true }), - ).toEqual({ - column: 'description', - tokenizer: 'unicode_words', - tokenizerParams: { remove_emojis: true }, - }); - }); - - it('creates a text field with alias for multi-tokenizer', () => { - expect( - bm25.text('description', { tokenizer: 'simple', alias: 'description_simple' }), - ).toEqual({ - column: 'description', - tokenizer: 'simple', - alias: 'description_simple', - }); - }); - }); - - describe('bm25.numeric', () => { - it('creates a numeric field', () => { - expect(bm25.numeric('rating')).toEqual({ column: 'rating' }); - }); - }); - - describe('bm25.boolean', () => { - it('creates a boolean field', () => { - expect(bm25.boolean('active')).toEqual({ column: 'active' }); - }); - }); - describe('bm25.json', () => { - it('creates a json field with defaults', () => { - expect(bm25.json('metadata')).toEqual({ column: 'metadata' }); - }); - - it('creates a json field with ngram tokenizer', () => { - expect(bm25.json('metadata', { tokenizer: 'ngram', min: 2, max: 3 })).toEqual({ - column: 'metadata', - tokenizer: 'ngram', - tokenizerParams: { min: 2, max: 3 }, - }); - }); + it('exposes the bm25 entry in indexTypes', () => { + expect(paradedbPackMeta.indexTypes.entries).toHaveLength(1); + expect(paradedbPackMeta.indexTypes.entries[0]?.type).toBe('bm25'); }); + }); - describe('bm25.datetime', () => { - it('creates a datetime field', () => { - expect(bm25.datetime('created_at')).toEqual({ column: 'created_at' }); - }); + describe('paradedbIndexTypes', () => { + it('declares a single bm25 entry', () => { + expect(paradedbIndexTypes.entries.map((e) => e.type)).toEqual(['bm25']); }); - describe('bm25.range', () => { - it('creates a range field', () => { - expect(bm25.range('price_range')).toEqual({ column: 'price_range' }); - }); + it('validates bm25 options with a key_field string', () => { + const entry = paradedbIndexTypes.entries[0]; + if (!entry) throw new Error('expected bm25 entry'); + const result = entry.options({ key_field: 'id' }); + expect(result instanceof type.errors).toBe(false); }); - describe('bm25.expression', () => { - it('creates an expression field with alias', () => { - expect( - bm25.expression("description || ' ' || category", { - alias: 'concat', - tokenizer: 'simple', - }), - ).toEqual({ - expression: "description || ' ' || category", - alias: 'concat', - tokenizer: 'simple', - }); - }); - - it('creates expression field with tokenizer params', () => { - expect( - bm25.expression("(metadata->>'color')", { - alias: 'meta_color', - tokenizer: 'ngram', - min: 2, - max: 3, - }), - ).toEqual({ - expression: "(metadata->>'color')", - alias: 'meta_color', - tokenizer: 'ngram', - tokenizerParams: { min: 2, max: 3 }, - }); - }); - - it('creates expression field without tokenizer', () => { - expect(bm25.expression('rating + 1', { alias: 'rating_plus' })).toEqual({ - expression: 'rating + 1', - alias: 'rating_plus', - }); - }); + it('rejects bm25 options without key_field', () => { + const entry = paradedbIndexTypes.entries[0]; + if (!entry) throw new Error('expected bm25 entry'); + const result = entry.options({}); + expect(result instanceof type.errors).toBe(true); }); - }); - - describe('bm25Index', () => { - it('creates index definition with extension config payload', () => { - const indexDef = bm25Index({ - keyField: 'id', - name: 'search_idx', - fields: [bm25.text('description', { tokenizer: 'simple' })], - }); - expect(indexDef).toEqual({ - columns: ['description'], - name: 'search_idx', - using: 'bm25', - config: { - keyField: 'id', - fields: [{ column: 'description', tokenizer: 'simple' }], - }, - }); + it('rejects bm25 options with extra unknown keys', () => { + const entry = paradedbIndexTypes.entries[0]; + if (!entry) throw new Error('expected bm25 entry'); + const result = entry.options({ key_field: 'id', extra: 'nope' }); + expect(result instanceof type.errors).toBe(true); }); - it('keeps expression fields in config while preserving core-safe columns', () => { - const indexDef = bm25Index({ - keyField: 'id', - fields: [ - bm25.text('description'), - bm25.expression("description || ' ' || category", { alias: 'concat' }), - ], - }); - - expect(indexDef.columns).toEqual(['description']); - expect(indexDef.config).toEqual({ - keyField: 'id', - fields: [ - { column: 'description' }, - { expression: "description || ' ' || category", alias: 'concat' }, - ], - }); + it('rejects bm25 options where key_field is not a string', () => { + const entry = paradedbIndexTypes.entries[0]; + if (!entry) throw new Error('expected bm25 entry'); + const result = entry.options({ key_field: 42 }); + expect(result instanceof type.errors).toBe(true); }); }); }); diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts index 9c2e1c632b..c4cd630f8e 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts @@ -15,6 +15,7 @@ import type { SqlPlannerConflict, SqlPlannerConflictLocation, } from '@prisma-next/family-sql/control'; +import { arraysEqual } from '@prisma-next/family-sql/schema-verify'; import type { TargetBoundComponentDescriptor } from '@prisma-next/framework-components/components'; import type { SchemaIssue } from '@prisma-next/framework-components/control'; import type { @@ -211,7 +212,12 @@ function mapIssueToCall( ]; for (const index of contractTable.indexes) { const indexName = index.name ?? `${issue.table}_${index.columns.join('_')}_idx`; - calls.push(new CreateIndexCall(schemaName, issue.table, indexName, [...index.columns])); + const extras: { type?: string; options?: Record } = {}; + if (index.type !== undefined) extras.type = index.type; + if (index.options !== undefined) extras.options = index.options; + calls.push( + new CreateIndexCall(schemaName, issue.table, indexName, [...index.columns], extras), + ); } const explicitIndexColumnSets = new Set( contractTable.indexes.map((idx) => idx.columns.join(',')), @@ -446,8 +452,14 @@ function mapIssueToCall( return notOk(issueConflict('indexIncompatible', 'Index issue has no table name')); if (isMissing(issue) && issue.expected) { const columns = issue.expected.split(', '); - const indexName = `${issue.table}_${columns.join('_')}_idx`; - return ok([new CreateIndexCall(schemaName, issue.table, indexName, columns)]); + const contractIndex = ctx.toContract.storage.tables[issue.table]?.indexes.find((idx) => + arraysEqual(idx.columns, columns), + ); + const indexName = contractIndex?.name ?? `${issue.table}_${columns.join('_')}_idx`; + const extras: { type?: string; options?: Record } = {}; + if (contractIndex?.type !== undefined) extras.type = contractIndex.type; + if (contractIndex?.options !== undefined) extras.options = contractIndex.options; + return ok([new CreateIndexCall(schemaName, issue.table, indexName, columns, extras)]); } return notOk( issueConflict( diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts index 1a0c533009..31089dcbe3 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts @@ -506,6 +506,10 @@ export class CreateIndexCall extends PostgresOpFactoryCallNode { readonly tableName: string; readonly indexName: string; readonly columns: readonly string[]; + // Named indexType (not typeName) to avoid collision with CreateEnumTypeCall.typeName, + // which identifies a CREATE TYPE target and is read by `locationForCall` in issue-planner.ts. + readonly indexType: string | undefined; + readonly options: Record | undefined; readonly label: string; constructor( @@ -513,22 +517,40 @@ export class CreateIndexCall extends PostgresOpFactoryCallNode { tableName: string, indexName: string, columns: readonly string[], + extras?: { readonly type?: string; readonly options?: Record }, ) { super(); this.schemaName = schemaName; this.tableName = tableName; this.indexName = indexName; this.columns = columns; + this.indexType = extras?.type; + this.options = extras?.options; this.label = `Create index "${indexName}" on "${tableName}"`; this.freeze(); } toOp(): Op { - return createIndex(this.schemaName, this.tableName, this.indexName, this.columns); + const extras: { type?: string; options?: Record } = {}; + if (this.indexType !== undefined) extras.type = this.indexType; + if (this.options !== undefined) extras.options = this.options; + return createIndex(this.schemaName, this.tableName, this.indexName, this.columns, extras); } renderTypeScript(): string { - return `createIndex(${jsonToTsSource(this.schemaName)}, ${jsonToTsSource(this.tableName)}, ${jsonToTsSource(this.indexName)}, ${jsonToTsSource(this.columns)})`; + const args = [ + jsonToTsSource(this.schemaName), + jsonToTsSource(this.tableName), + jsonToTsSource(this.indexName), + jsonToTsSource(this.columns), + ]; + if (this.indexType !== undefined || this.options !== undefined) { + const extrasParts: string[] = []; + if (this.indexType !== undefined) extrasParts.push(`type: ${jsonToTsSource(this.indexType)}`); + if (this.options !== undefined) extrasParts.push(`options: ${jsonToTsSource(this.options)}`); + args.push(`{ ${extrasParts.join(', ')} }`); + } + return `createIndex(${args.join(', ')})`; } } diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts index 4fce8a9f3b..71b9130f41 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts @@ -1,15 +1,40 @@ -import { quoteIdentifier } from '../../sql-utils'; +import { escapeLiteral, quoteIdentifier } from '../../sql-utils'; import { qualifyTableName, toRegclassLiteral } from '../planner-sql-checks'; import { type Op, step, targetDetails } from './shared'; +export interface CreateIndexExtras { + readonly type?: string; + readonly options?: Record; +} + +function renderIndexOptionValue(key: string, value: unknown): string { + if (typeof value === 'string') return `'${escapeLiteral(value)}'`; + if (typeof value === 'number' && Number.isFinite(value)) return String(value); + if (typeof value === 'boolean') return value ? 'true' : 'false'; + throw new Error( + `Index option "${key}" must be a string, finite number, or boolean; got ${typeof value}`, + ); +} + +function renderIndexOptions(options: Record): string { + return Object.entries(options) + .map(([key, value]) => `${quoteIdentifier(key)} = ${renderIndexOptionValue(key, value)}`) + .join(', '); +} + export function createIndex( schemaName: string, tableName: string, indexName: string, columns: readonly string[], + extras?: CreateIndexExtras, ): Op { const qualified = qualifyTableName(schemaName, tableName); const columnList = columns.map(quoteIdentifier).join(', '); + const using = extras?.type ? ` USING ${quoteIdentifier(extras.type)}` : ''; + const options = extras?.options; + const withClause = + options && Object.keys(options).length > 0 ? ` WITH (${renderIndexOptions(options)})` : ''; return { id: `index.${tableName}.${indexName}`, label: `Create index "${indexName}" on "${tableName}"`, @@ -24,7 +49,7 @@ export function createIndex( execute: [ step( `create index "${indexName}"`, - `CREATE INDEX ${quoteIdentifier(indexName)} ON ${qualified} (${columnList})`, + `CREATE INDEX ${quoteIdentifier(indexName)} ON ${qualified}${using} (${columnList})${withClause}`, ), ], postcheck: [ diff --git a/packages/3-targets/3-targets/postgres/test/migrations/index-ddl.test.ts b/packages/3-targets/3-targets/postgres/test/migrations/index-ddl.test.ts new file mode 100644 index 0000000000..b26aad7923 --- /dev/null +++ b/packages/3-targets/3-targets/postgres/test/migrations/index-ddl.test.ts @@ -0,0 +1,78 @@ +import { describe, expect, it } from 'vitest'; +import { createIndex } from '../../src/core/migrations/operations/indexes'; + +function executeSql(op: ReturnType): string { + const stmt = op.execute[0]; + if (!stmt) throw new Error('createIndex op has no execute step'); + return stmt.sql; +} + +describe('createIndex DDL emission', () => { + it('emits a plain CREATE INDEX when no extras are supplied', () => { + const op = createIndex('public', 'user', 'user_email_idx', ['email']); + expect(executeSql(op)).toBe('CREATE INDEX "user_email_idx" ON "public"."user" ("email")'); + }); + + it('emits USING when type is supplied', () => { + const op = createIndex('public', 'doc', 'doc_body_idx', ['body'], { type: 'gin' }); + expect(executeSql(op)).toBe( + 'CREATE INDEX "doc_body_idx" ON "public"."doc" USING "gin" ("body")', + ); + }); + + it('emits WITH (...) when options are supplied', () => { + const op = createIndex('public', 'doc', 'doc_body_idx', ['body'], { + type: 'gin', + options: { fastupdate: false }, + }); + expect(executeSql(op)).toBe( + 'CREATE INDEX "doc_body_idx" ON "public"."doc" USING "gin" ("body") WITH ("fastupdate" = false)', + ); + }); + + it('omits WITH when options is an empty object', () => { + const op = createIndex('public', 'doc', 'doc_body_idx', ['body'], { + type: 'gin', + options: {}, + }); + expect(executeSql(op)).toBe( + 'CREATE INDEX "doc_body_idx" ON "public"."doc" USING "gin" ("body")', + ); + }); + + it('renders number, boolean, and string option leaves correctly', () => { + const op = createIndex('public', 'doc', 'doc_body_idx', ['body'], { + type: 'demo', + options: { fillfactor: 70, fastupdate: false, pdb_locale: 'en-US' }, + }); + expect(executeSql(op)).toBe( + `CREATE INDEX "doc_body_idx" ON "public"."doc" USING "demo" ("body") WITH ("fillfactor" = 70, "fastupdate" = false, "pdb_locale" = 'en-US')`, + ); + }); + + it('escapes single quotes in string option values', () => { + const op = createIndex('public', 'doc', 'doc_body_idx', ['body'], { + type: 'demo', + options: { needle: "with'quote" }, + }); + expect(executeSql(op)).toContain(`"needle" = 'with''quote'`); + }); + + it('rejects null option values', () => { + expect(() => + createIndex('public', 'doc', 'doc_body_idx', ['body'], { + type: 'demo', + options: { weird: null }, + }), + ).toThrow(/Index option/); + }); + + it('rejects non-finite numeric option values', () => { + expect(() => + createIndex('public', 'doc', 'doc_body_idx', ['body'], { + type: 'demo', + options: { weird: Number.NaN }, + }), + ).toThrow(/Index option/); + }); +}); diff --git a/packages/3-targets/3-targets/postgres/test/migrations/issue-planner.test.ts b/packages/3-targets/3-targets/postgres/test/migrations/issue-planner.test.ts index 6e8dc4f449..16b9b6b99e 100644 --- a/packages/3-targets/3-targets/postgres/test/migrations/issue-planner.test.ts +++ b/packages/3-targets/3-targets/postgres/test/migrations/issue-planner.test.ts @@ -452,6 +452,144 @@ describe('planIssues', () => { }); }); + describe('index_mismatch', () => { + it('threads contract index type and options into CreateIndexCall when the index is missing', () => { + const toContract = makeContract({ + tables: { + doc: { + columns: { + id: { nativeType: 'uuid', codecId: 'pg/uuid@1', nullable: false }, + body: { nativeType: 'text', codecId: 'pg/text@1', nullable: false }, + }, + primaryKey: { columns: ['id'] }, + uniques: [], + indexes: [{ columns: ['body'], type: 'gin', options: { fastupdate: false } }], + foreignKeys: [], + }, + }, + }); + const issues: SchemaIssue[] = [ + { + kind: 'index_mismatch', + table: 'doc', + expected: 'body', + message: 'Table "doc" is missing index: body', + }, + ]; + + const result = planIssues({ + ...defaultCtx, + issues, + toContract, + fromContract: null, + storageTypes: toContract.storage.types ?? {}, + }); + + expect(result.ok).toBe(true); + if (!result.ok) throw new Error('expected ok'); + expect(result.value.calls).toHaveLength(1); + expect(result.value.calls[0]).toMatchObject({ + factoryName: 'createIndex', + tableName: 'doc', + indexType: 'gin', + options: { fastupdate: false }, + }); + }); + + it('uses the contract index name when set', () => { + const toContract = makeContract({ + tables: { + doc: { + columns: { + id: { nativeType: 'uuid', codecId: 'pg/uuid@1', nullable: false }, + body: { nativeType: 'text', codecId: 'pg/text@1', nullable: false }, + }, + primaryKey: { columns: ['id'] }, + uniques: [], + indexes: [ + { + columns: ['body'], + name: 'doc_body_bm25_idx', + type: 'bm25', + options: { key_field: 'id' }, + }, + ], + foreignKeys: [], + }, + }, + }); + const issues: SchemaIssue[] = [ + { + kind: 'index_mismatch', + table: 'doc', + expected: 'body', + message: 'Table "doc" is missing index: body', + }, + ]; + + const result = planIssues({ + ...defaultCtx, + issues, + toContract, + fromContract: null, + storageTypes: toContract.storage.types ?? {}, + }); + + expect(result.ok).toBe(true); + if (!result.ok) throw new Error('expected ok'); + expect(result.value.calls[0]).toMatchObject({ + factoryName: 'createIndex', + tableName: 'doc', + indexName: 'doc_body_bm25_idx', + indexType: 'bm25', + options: { key_field: 'id' }, + }); + }); + + it('falls back to a default index name when the contract index has no name', () => { + const toContract = makeContract({ + tables: { + doc: { + columns: { + id: { nativeType: 'uuid', codecId: 'pg/uuid@1', nullable: false }, + body: { nativeType: 'text', codecId: 'pg/text@1', nullable: false }, + }, + primaryKey: { columns: ['id'] }, + uniques: [], + indexes: [{ columns: ['body'] }], + foreignKeys: [], + }, + }, + }); + const issues: SchemaIssue[] = [ + { + kind: 'index_mismatch', + table: 'doc', + expected: 'body', + message: 'Table "doc" is missing index: body', + }, + ]; + + const result = planIssues({ + ...defaultCtx, + issues, + toContract, + fromContract: null, + storageTypes: toContract.storage.types ?? {}, + }); + + expect(result.ok).toBe(true); + if (!result.ok) throw new Error('expected ok'); + expect(result.value.calls[0]).toMatchObject({ + factoryName: 'createIndex', + tableName: 'doc', + indexName: 'doc_body_idx', + indexType: undefined, + options: undefined, + }); + }); + }); + describe('foreign_key_mismatch', () => { it('returns foreignKeyConflict when the destination contract lacks a matching FK entry', () => { const toContract = makeContract({ diff --git a/packages/3-targets/6-adapters/postgres/src/core/control-adapter.ts b/packages/3-targets/6-adapters/postgres/src/core/control-adapter.ts index daa3fb8dc0..afca657c3e 100644 --- a/packages/3-targets/6-adapters/postgres/src/core/control-adapter.ts +++ b/packages/3-targets/6-adapters/postgres/src/core/control-adapter.ts @@ -301,17 +301,22 @@ export class PostgresControlAdapter implements SqlControlAdapter<'postgres'> { indisunique: boolean; attname: string; attnum: number; + amname: string; + reloptions: readonly string[] | null; }>( `SELECT i.tablename, i.indexname, ix.indisunique, a.attname, - a.attnum + a.attnum, + am.amname, + ic.reloptions FROM pg_indexes i JOIN pg_class ic ON ic.relname = i.indexname JOIN pg_namespace ins ON ins.oid = ic.relnamespace AND ins.nspname = $1 JOIN pg_index ix ON ix.indexrelid = ic.oid + JOIN pg_am am ON am.oid = ic.relam JOIN pg_class t ON t.oid = ix.indrelid JOIN pg_namespace tn ON tn.oid = t.relnamespace AND tn.nspname = $1 LEFT JOIN pg_attribute a ON a.attrelid = t.oid AND a.attnum = ANY(ix.indkey) AND a.attnum > 0 @@ -469,7 +474,16 @@ export class PostgresControlAdapter implements SqlControlAdapter<'postgres'> { })); // Process indexes - const indexesMap = new Map(); + const indexesMap = new Map< + string, + { + columns: string[]; + name: string; + unique: boolean; + type: string | undefined; + options: Record | undefined; + } + >(); for (const idxRow of indexesByTable.get(tableName) ?? []) { if (!idxRow.attname) { continue; @@ -478,10 +492,17 @@ export class PostgresControlAdapter implements SqlControlAdapter<'postgres'> { if (existing) { existing.columns.push(idxRow.attname); } else { + // Drop btree (the Postgres default) so a contract index without an + // explicit type matches a default-method introspected index without + // forcing DROP+CREATE on every plan. + const indexType = idxRow.amname === 'btree' ? undefined : idxRow.amname; + const indexOptions = parsePgReloptions(idxRow.reloptions, idxRow.indexname); indexesMap.set(idxRow.indexname, { columns: [idxRow.attname], name: idxRow.indexname, unique: idxRow.indisunique, + type: indexType, + options: indexOptions, }); } } @@ -489,6 +510,8 @@ export class PostgresControlAdapter implements SqlControlAdapter<'postgres'> { columns: Object.freeze([...idx.columns]) as readonly string[], name: idx.name, unique: idx.unique, + ...(idx.type !== undefined && { type: idx.type }), + ...(idx.options !== undefined && { options: idx.options }), })); tables[tableName] = { @@ -622,6 +645,39 @@ function mapReferentialAction(rule: string): SqlReferentialAction | undefined { * Groups an array of objects by a specified key. * Returns a Map for O(1) lookup by group key. */ +/** + * Parses a `pg_class.reloptions` array into a `Record`. + * + * Postgres returns reloptions as a `text[]` whose entries are `key=value` + * strings; the value side is always a string regardless of the underlying + * scalar type. The verifier compares contract options to introspected + * options after coercing both sides to strings, so keeping the raw text + * here is correct. + * + * Returns `undefined` when the input is null/empty (no WITH clause). + */ +export function parsePgReloptions( + reloptions: readonly string[] | null, + indexName: string, +): Record | undefined { + if (!reloptions || reloptions.length === 0) { + return undefined; + } + const result: Record = {}; + for (const entry of reloptions) { + const eq = entry.indexOf('='); + if (eq === -1) { + throw new Error( + `Postgres introspection: malformed reloption entry "${entry}" on index "${indexName}" (expected "key=value")`, + ); + } + const key = entry.slice(0, eq); + const value = entry.slice(eq + 1); + result[key] = value; + } + return Object.keys(result).length > 0 ? result : undefined; +} + function groupBy(items: readonly T[], key: K): Map { const map = new Map(); for (const item of items) { diff --git a/packages/3-targets/6-adapters/postgres/test/control-adapter.test.ts b/packages/3-targets/6-adapters/postgres/test/control-adapter.test.ts index 2650114c53..47e71a0077 100644 --- a/packages/3-targets/6-adapters/postgres/test/control-adapter.test.ts +++ b/packages/3-targets/6-adapters/postgres/test/control-adapter.test.ts @@ -2,7 +2,7 @@ import type { ControlDriverInstance } from '@prisma-next/framework-components/co import { normalizeSchemaNativeType } from '@prisma-next/target-postgres/native-type-normalizer'; import { timeouts } from '@prisma-next/test-utils'; import { describe, expect, it } from 'vitest'; -import { PostgresControlAdapter } from '../src/core/control-adapter'; +import { PostgresControlAdapter, parsePgReloptions } from '../src/core/control-adapter'; type QueryHandler = { readonly match: (sql: string) => boolean; @@ -1650,4 +1650,24 @@ describe('PostgresControlAdapter', () => { expect(normalizeSchemaNativeType(input)).toBe(expected); }); }); + + describe('parsePgReloptions', () => { + it('throws when a reloption entry has no "=" separator', () => { + expect(() => parsePgReloptions(['no_eq_sign'], 'item_body_idx')).toThrow( + /malformed reloption entry "no_eq_sign" on index "item_body_idx"/, + ); + }); + + it('parses well-formed key=value entries into a record', () => { + expect(parsePgReloptions(['fillfactor=70', 'fastupdate=true'], 'item_body_idx')).toEqual({ + fillfactor: '70', + fastupdate: 'true', + }); + }); + + it('returns undefined for a null or empty input', () => { + expect(parsePgReloptions(null, 'item_body_idx')).toBeUndefined(); + expect(parsePgReloptions([], 'item_body_idx')).toBeUndefined(); + }); + }); }); diff --git a/packages/3-targets/6-adapters/postgres/test/migrations/index-introspection.integration.test.ts b/packages/3-targets/6-adapters/postgres/test/migrations/index-introspection.integration.test.ts new file mode 100644 index 0000000000..a65498845e --- /dev/null +++ b/packages/3-targets/6-adapters/postgres/test/migrations/index-introspection.integration.test.ts @@ -0,0 +1,105 @@ +/** + * Postgres index introspection populates `SqlIndexIR.type` and + * `SqlIndexIR.options` from `pg_am.amname` and `pg_class.reloptions`. + * + * Without these fields the migration planner would treat any contract + * index whose `type` is set as different from any introspected index on + * the same columns — forcing a spurious DROP+CREATE on every plan even + * when the live index already matches the contract. + */ +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; +import { + createDriver, + createTestDatabase, + familyInstance, + type PostgresControlDriver, + resetDatabase, + testTimeout, +} from './fixtures/runner-fixtures'; + +describe.sequential('Postgres index introspection — type and options', () => { + let database: Awaited>; + let driver: PostgresControlDriver | undefined; + + beforeAll(async () => { + database = await createTestDatabase(); + }, testTimeout); + + afterAll(async () => { + if (database) await database.close(); + }, testTimeout); + + beforeEach(async () => { + driver = await createDriver(database.connectionString); + await resetDatabase(driver); + }, testTimeout); + + afterEach(async () => { + if (driver) { + await driver.close(); + driver = undefined; + } + }); + + it( + 'leaves type and options unset on a default btree index', + { timeout: testTimeout }, + async () => { + await driver!.query('CREATE TABLE doc (id int PRIMARY KEY, body text NOT NULL)'); + await driver!.query('CREATE INDEX doc_body_idx ON doc (body)'); + + const schema = await familyInstance.introspect({ driver: driver! }); + const indexes = schema.tables['doc']?.indexes ?? []; + const idx = indexes.find((i) => i.name === 'doc_body_idx'); + expect(idx).toBeDefined(); + expect(idx?.type).toBeUndefined(); + expect(idx?.options).toBeUndefined(); + }, + ); + + it('populates type for non-default index methods (gin)', { timeout: testTimeout }, async () => { + await driver!.query('CREATE TABLE doc (id int PRIMARY KEY, tags jsonb NOT NULL)'); + await driver!.query('CREATE INDEX doc_tags_gin_idx ON doc USING gin (tags)'); + + const schema = await familyInstance.introspect({ driver: driver! }); + const idx = schema.tables['doc']?.indexes.find((i) => i.name === 'doc_tags_gin_idx'); + expect(idx).toBeDefined(); + expect(idx?.type).toBe('gin'); + expect(idx?.options).toBeUndefined(); + }); + + it( + 'populates options from reloptions when WITH parameters are set', + { timeout: testTimeout }, + async () => { + await driver!.query('CREATE TABLE doc (id int PRIMARY KEY, body text NOT NULL)'); + await driver!.query('CREATE INDEX doc_body_idx ON doc (body) WITH (fillfactor = 70)'); + + const schema = await familyInstance.introspect({ driver: driver! }); + const idx = schema.tables['doc']?.indexes.find((i) => i.name === 'doc_body_idx'); + expect(idx).toBeDefined(); + // btree is the Postgres default → type is dropped to undefined + expect(idx?.type).toBeUndefined(); + // reloptions are returned as raw text; the family verifier compares + // contract values to introspected strings via String() coercion. + expect(idx?.options).toEqual({ fillfactor: '70' }); + }, + ); + + it( + 'populates both type and options together (gin with fastupdate)', + { timeout: testTimeout }, + async () => { + await driver!.query('CREATE TABLE doc (id int PRIMARY KEY, tags jsonb NOT NULL)'); + await driver!.query( + 'CREATE INDEX doc_tags_gin_idx ON doc USING gin (tags) WITH (fastupdate = false)', + ); + + const schema = await familyInstance.introspect({ driver: driver! }); + const idx = schema.tables['doc']?.indexes.find((i) => i.name === 'doc_tags_gin_idx'); + expect(idx).toBeDefined(); + expect(idx?.type).toBe('gin'); + expect(idx?.options).toEqual({ fastupdate: 'false' }); + }, + ); +}); diff --git a/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.rendering.test.ts b/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.rendering.test.ts index 73c0307438..5d0c4659e4 100644 --- a/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.rendering.test.ts +++ b/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.rendering.test.ts @@ -235,6 +235,16 @@ describe('Postgres call classes - per-class renderTypeScript coverage', () => { expectFactoryImport(di, 'dropIndex'); }); + it('CreateIndexCall renders type and options when they are provided', () => { + const ci = new CreateIndexCall('public', 'doc', 'doc_body_idx', ['body'], { + type: 'gin', + options: { fastupdate: false }, + }); + expect(ci.renderTypeScript()).toBe( + 'createIndex("public", "doc", "doc_body_idx", ["body"], { type: "gin", options: { fastupdate: false } })', + ); + }); + it('CreateEnumTypeCall emits the enum values as an array literal', () => { const call = new CreateEnumTypeCall('public', 'status', ['active', 'archived']); expect(call.renderTypeScript()).toBe( diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 5fd51c3740..c613e64e4e 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -2040,6 +2040,9 @@ importers: '@prisma-next/tsdown': specifier: workspace:* version: link:../../../0-config/tsdown + arktype: + specifier: ^2.1.25 + version: 2.1.29 tsdown: specifier: 'catalog:' version: 0.18.4(@emnapi/core@1.9.2)(@emnapi/runtime@1.9.2)(typescript@5.9.3) @@ -2550,6 +2553,12 @@ importers: '@prisma-next/contract-authoring': specifier: workspace:* version: link:../../1-framework/2-authoring/contract + '@prisma-next/sql-contract': + specifier: workspace:* + version: link:../../2-sql/1-core/contract + arktype: + specifier: ^2.1.25 + version: 2.1.29 devDependencies: '@prisma-next/tsconfig': specifier: workspace:* @@ -3546,6 +3555,9 @@ importers: '@prisma-next/extension-arktype-json': specifier: workspace:* version: link:../../packages/3-extensions/arktype-json + '@prisma-next/extension-paradedb': + specifier: workspace:* + version: link:../../packages/3-extensions/paradedb '@prisma-next/extension-pgvector': specifier: workspace:* version: link:../../packages/3-extensions/pgvector diff --git a/test/e2e/framework/test/sqlite/migrations/additive.test.ts b/test/e2e/framework/test/sqlite/migrations/additive.test.ts index 0cab99ea3d..b20c3fdf76 100644 --- a/test/e2e/framework/test/sqlite/migrations/additive.test.ts +++ b/test/e2e/framework/test/sqlite/migrations/additive.test.ts @@ -164,7 +164,7 @@ describe('SQLite Migration E2E - From empty schema', () => { fields: { id: int.id(), name: text, date: text, location: text.optional() }, }).sql((ctx) => ({ indexes: [ - ctx.constraints.index(ctx.cols.date, { name: 'idx_events_date' }), + ctx.constraints.index([ctx.cols.date], { name: 'idx_events_date' }), ctx.constraints.index([ctx.cols.name, ctx.cols.date], { name: 'idx_events_name_date', }), @@ -264,7 +264,7 @@ describe('SQLite Migration E2E - Schema evolution', () => { ...pack, models: { User: model('User', { fields: { id: int.id(), email: text } }).sql((ctx) => ({ - indexes: [ctx.constraints.index(ctx.cols.email, { name: 'idx_users_email' })], + indexes: [ctx.constraints.index([ctx.cols.email], { name: 'idx_users_email' })], })), }, }), @@ -293,7 +293,7 @@ describe('SQLite Migration E2E - Schema evolution', () => { status: text.default('active'), }, }).sql((ctx) => ({ - indexes: [ctx.constraints.index(ctx.cols.email, { name: 'idx_users_email' })], + indexes: [ctx.constraints.index([ctx.cols.email], { name: 'idx_users_email' })], })), Post: model('Post', { fields: { id: int.id(), title: text, userId: int.column('user_id') }, diff --git a/test/e2e/framework/test/sqlite/migrations/destructive.test.ts b/test/e2e/framework/test/sqlite/migrations/destructive.test.ts index a43be2b787..796290799e 100644 --- a/test/e2e/framework/test/sqlite/migrations/destructive.test.ts +++ b/test/e2e/framework/test/sqlite/migrations/destructive.test.ts @@ -39,7 +39,7 @@ describe('SQLite Migration E2E - Destructive operations', () => { ...pack, models: { User: model('User', { fields: { id: int.id(), email: text } }).sql((ctx) => ({ - indexes: [ctx.constraints.index(ctx.cols.email, { name: 'idx_users_email' })], + indexes: [ctx.constraints.index([ctx.cols.email], { name: 'idx_users_email' })], })), }, }), @@ -63,7 +63,7 @@ describe('SQLite Migration E2E - Destructive operations', () => { models: { User: model('User', { fields: { id: int.id(), email: text, name: text } }).sql( (ctx) => ({ - indexes: [ctx.constraints.index(ctx.cols.email, { name: 'idx_email' })], + indexes: [ctx.constraints.index([ctx.cols.email], { name: 'idx_email' })], }), ), }, @@ -72,7 +72,9 @@ describe('SQLite Migration E2E - Destructive operations', () => { ...pack, models: { User: model('User', { fields: { id: int.id(), email: text, name: text } }).sql( - (ctx) => ({ indexes: [ctx.constraints.index(ctx.cols.name, { name: 'idx_name' })] }), + (ctx) => ({ + indexes: [ctx.constraints.index([ctx.cols.name], { name: 'idx_name' })], + }), ), }, }), diff --git a/test/e2e/framework/test/sqlite/migrations/fk-preservation.test.ts b/test/e2e/framework/test/sqlite/migrations/fk-preservation.test.ts index 44470e13d0..0f295e7655 100644 --- a/test/e2e/framework/test/sqlite/migrations/fk-preservation.test.ts +++ b/test/e2e/framework/test/sqlite/migrations/fk-preservation.test.ts @@ -147,7 +147,7 @@ describe('SQLite Migration E2E - FK preservation through recreate-table', () => fields: { id: int.id(), email: text, name: text }, }).sql((ctx) => ({ indexes: [ - ctx.constraints.index(ctx.cols.email, { name: 'idx_users_email' }), + ctx.constraints.index([ctx.cols.email], { name: 'idx_users_email' }), ctx.constraints.index([ctx.cols.name, ctx.cols.email], { name: 'idx_users_name_email' }), ], })); @@ -155,7 +155,7 @@ describe('SQLite Migration E2E - FK preservation through recreate-table', () => fields: { id: int.id(), email: text, name: text.optional() }, }).sql((ctx) => ({ indexes: [ - ctx.constraints.index(ctx.cols.email, { name: 'idx_users_email' }), + ctx.constraints.index([ctx.cols.email], { name: 'idx_users_email' }), ctx.constraints.index([ctx.cols.name, ctx.cols.email], { name: 'idx_users_name_email' }), ], })); diff --git a/test/integration/package.json b/test/integration/package.json index 2cb9e8a692..7774126040 100644 --- a/test/integration/package.json +++ b/test/integration/package.json @@ -25,6 +25,7 @@ "@prisma-next/driver-postgres": "workspace:*", "@prisma-next/emitter": "workspace:*", "@prisma-next/extension-arktype-json": "workspace:*", + "@prisma-next/extension-paradedb": "workspace:*", "@prisma-next/extension-pgvector": "workspace:*", "@prisma-next/family-sql": "workspace:*", "@prisma-next/ids": "workspace:*", diff --git a/test/integration/test/authoring/paradedb-bm25-narrowing.test.ts b/test/integration/test/authoring/paradedb-bm25-narrowing.test.ts new file mode 100644 index 0000000000..f6d914dd59 --- /dev/null +++ b/test/integration/test/authoring/paradedb-bm25-narrowing.test.ts @@ -0,0 +1,242 @@ +/** + * End-to-end TS narrowing for the paradedb bm25 index type. + * + * Verifies that when a contract attaches `paradedbPack` via the + * `defineContract({...}, ({ model }) => ...)` factory form, the + * `constraints.index({ type: 'bm25', options: ... })` call site narrows + * `options` against the registered shape and rejects unregistered types + * and bad option shapes at compile time. + */ +import { int4Column, textColumn } from '@prisma-next/adapter-postgres/column-types'; +import paradedbPack from '@prisma-next/extension-paradedb/pack'; +import sqlFamily from '@prisma-next/family-sql/pack'; +import { defineContract, field, model } from '@prisma-next/sql-contract-ts/contract-builder'; +import postgresPack from '@prisma-next/target-postgres/pack'; +import { describe, expect, expectTypeOf, it } from 'vitest'; + +describe('paradedb bm25 narrowing in TS authoring DSL', () => { + it('typechecks and accepts a well-formed bm25 index via the helpers factory', () => { + const contract = defineContract( + { + family: sqlFamily, + target: postgresPack, + extensionPacks: { paradedb: paradedbPack }, + }, + ({ model: helperModel, field: helperField }) => { + const Doc = helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + constraints.index([cols.body], { + type: 'bm25', + options: { key_field: 'id' }, + name: 'doc_body_bm25_idx', + }), + ], + })); + return { models: { Doc } }; + }, + ); + + const indexes = contract.storage.tables.doc.indexes; + expect(indexes).toHaveLength(1); + expect(indexes[0]).toMatchObject({ + columns: ['body'], + name: 'doc_body_bm25_idx', + type: 'bm25', + options: { key_field: 'id' }, + }); + }); + + it('rejects a bm25 index with an unknown options key at compile time', () => { + expect(() => + defineContract( + { + family: sqlFamily, + target: postgresPack, + extensionPacks: { paradedb: paradedbPack }, + }, + ({ model: helperModel, field: helperField }) => { + const Doc = helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + constraints.index([cols.body], { + type: 'bm25', + // @ts-expect-error — bm25 options is { key_field: string } in strict mode; unknown_key is rejected + options: { key_field: 'id', unknown_key: 'x' }, + }), + ], + })); + return { models: { Doc } }; + }, + ), + ).toThrow(/unknown_key/); + }); + + it('rejects a bm25 index missing the required key_field at compile time', () => { + expect(() => + defineContract( + { + family: sqlFamily, + target: postgresPack, + extensionPacks: { paradedb: paradedbPack }, + }, + ({ model: helperModel, field: helperField }) => { + const Doc = helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + constraints.index([cols.body], { + type: 'bm25', + // @ts-expect-error — bm25 options requires key_field + options: {}, + }), + ], + })); + return { models: { Doc } }; + }, + ), + ).toThrow(/key_field/); + }); + + it('rejects an unregistered index type at compile time', () => { + expect(() => + defineContract( + { + family: sqlFamily, + target: postgresPack, + extensionPacks: { paradedb: paradedbPack }, + }, + ({ model: helperModel, field: helperField }) => { + const Doc = helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + constraints.index([cols.body], { + // @ts-expect-error — only 'bm25' is registered when paradedb is attached; 'made-up' is not + type: 'made-up', + options: { key_field: 'id' }, + }), + ], + })); + return { models: { Doc } }; + }, + ), + ).toThrow(/unregistered index type "made-up"/); + }); + + it('rejects options without a type at compile time', () => { + expect(() => + defineContract( + { + family: sqlFamily, + target: postgresPack, + extensionPacks: { paradedb: paradedbPack }, + }, + ({ model: helperModel, field: helperField }) => { + const Doc = helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + // @ts-expect-error — providing options without a type is a compile error when packs contribute index types + constraints.index([cols.body], { + options: { key_field: 'id' }, + }), + ], + })); + return { models: { Doc } }; + }, + ), + ).toThrow(/options without a type/); + }); + + it('imported bare model() rejects any type/options — strict by default', () => { + const Doc = model('Doc', { + fields: { + id: field.column(int4Column).id(), + body: field.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [ + // @ts-expect-error - bare model() has no attached packs, so no index + // type literals are registered; type/options aren't allowed at all. + constraints.index([cols.body], { type: 'made-up', options: {} }), + ], + })); + + expect(() => + defineContract({ + family: sqlFamily, + target: postgresPack, + models: { Doc }, + }), + ).toThrow(/unregistered index type "made-up"/); + }); + + it('imported bare model() still accepts a default index with no type/options', () => { + const Doc = model('Doc', { + fields: { + id: field.column(int4Column).id(), + body: field.column(textColumn), + }, + }).sql(({ cols, constraints }) => ({ + table: 'doc', + indexes: [constraints.index([cols.body])], + })); + expectTypeOf().toEqualTypeOf>(); + + defineContract({ + family: sqlFamily, + target: postgresPack, + models: { Doc }, + }); + }); + + it("helpers-bound model() carries the merged packs' index-type map", () => { + defineContract( + { + family: sqlFamily, + target: postgresPack, + extensionPacks: { paradedb: paradedbPack }, + }, + ({ model: helperModel, field: helperField }) => { + const Doc = helperModel('Doc', { + fields: { + id: helperField.column(int4Column).id(), + body: helperField.column(textColumn), + }, + }); + expectTypeOf().toMatchTypeOf<{ + readonly bm25: { readonly options: { readonly key_field: string } }; + }>(); + return { + models: { + Doc: Doc.sql({ table: 'doc' }), + }, + }; + }, + ); + }); +}); diff --git a/test/integration/test/authoring/parity/core-surface/contract.ts b/test/integration/test/authoring/parity/core-surface/contract.ts index 1b6d3fe2ed..db9b5b8a01 100644 --- a/test/integration/test/authoring/parity/core-surface/contract.ts +++ b/test/integration/test/authoring/parity/core-surface/contract.ts @@ -49,7 +49,7 @@ const Post = model('Post', { })) .sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], })); export const contract = defineContract({ diff --git a/test/integration/test/authoring/parity/map-attributes/contract.ts b/test/integration/test/authoring/parity/map-attributes/contract.ts index 7c684d7fec..82b4de0a95 100644 --- a/test/integration/test/authoring/parity/map-attributes/contract.ts +++ b/test/integration/test/authoring/parity/map-attributes/contract.ts @@ -29,7 +29,7 @@ const Member = model('Member', { })) .sql(({ cols, constraints }) => ({ table: 'team_member', - indexes: [constraints.index(cols.teamId)], + indexes: [constraints.index([cols.teamId])], })); export const contract = defineContract({ diff --git a/test/integration/test/authoring/psl-index-type-options.integration.test.ts b/test/integration/test/authoring/psl-index-type-options.integration.test.ts new file mode 100644 index 0000000000..aa2b9c6bbf --- /dev/null +++ b/test/integration/test/authoring/psl-index-type-options.integration.test.ts @@ -0,0 +1,85 @@ +import { ContractValidationError } from '@prisma-next/contract/validate-contract'; +import paradedbPack from '@prisma-next/extension-paradedb/pack'; +import { parsePslDocument } from '@prisma-next/psl-parser'; +import { interpretPslDocumentToSqlContract } from '@prisma-next/sql-contract-psl'; +import postgresPack from '@prisma-next/target-postgres/pack'; +import { describe, expect, it } from 'vitest'; + +const scalarTypeDescriptors = new Map([ + ['Int', { codecId: 'pg/int4@1', nativeType: 'int4' }], + ['String', { codecId: 'pg/text@1', nativeType: 'text' }], +]); + +function interpret(schema: string) { + return interpretPslDocumentToSqlContract({ + document: parsePslDocument({ schema, sourceId: 'schema.prisma' }), + target: postgresPack, + scalarTypeDescriptors, + composedExtensionPacks: [paradedbPack.id], + composedExtensionPackRefs: [paradedbPack], + }); +} + +describe('PSL @@index type and options — integration with real paradedb pack', () => { + it('lowers the documented example to a Contract IR index node carrying type, options, and name', () => { + const result = interpret(`model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { key_field: "id" }, map: "doc_body_bm25_idx") +}`); + + expect(result.ok).toBe(true); + if (!result.ok) return; + expect(result.value.storage).toMatchObject({ + tables: { + doc: { + indexes: [ + { + columns: ['body'], + name: 'doc_body_bm25_idx', + type: 'bm25', + options: { key_field: 'id' }, + }, + ], + }, + }, + }); + }); + + it('the interpreter rejects a PSL-authored bm25 index whose options miss key_field', () => { + let thrown: unknown; + try { + interpret(`model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: { wrong_field: "x" }) +}`); + } catch (e) { + thrown = e; + } + expect(thrown).toBeInstanceOf(ContractValidationError); + const message = (thrown as ContractValidationError).message; + expect(message).toContain('bm25'); + expect(message).toContain('key_field'); + }); + + it('the interpreter rejects a PSL-authored index whose type is not registered', () => { + expect(() => + interpret(`model Doc { + id Int @id + body String + @@index([body], type: "made-up") +}`), + ).toThrow(/unregistered index type "made-up"/); + }); + + it('the interpreter rejects an empty options literal for bm25 (missing key_field)', () => { + expect(() => + interpret(`model Doc { + id Int @id + body String + @@index([body], type: "bm25", options: {}) +}`), + ).toThrow(/key_field/); + }); +}); diff --git a/test/integration/test/family.schema-verify.basic.integration.test.ts b/test/integration/test/family.schema-verify.basic.integration.test.ts index 998d8f63a6..e5f801c8db 100644 --- a/test/integration/test/family.schema-verify.basic.integration.test.ts +++ b/test/integration/test/family.schema-verify.basic.integration.test.ts @@ -75,7 +75,7 @@ describe('family instance schemaVerify', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [constraints.foreignKey(cols.userId, User.refs.id)], })); diff --git a/test/integration/test/family.schema-verify.basic.test.ts b/test/integration/test/family.schema-verify.basic.test.ts index d0577e0975..404c3ef6a6 100644 --- a/test/integration/test/family.schema-verify.basic.test.ts +++ b/test/integration/test/family.schema-verify.basic.test.ts @@ -66,7 +66,7 @@ describe('family instance schemaVerify - basic', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], })); const contract = defineContract({ diff --git a/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract-add-project-slug.ts b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract-add-project-slug.ts index 1e022f5208..2dd04255a7 100644 --- a/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract-add-project-slug.ts +++ b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract-add-project-slug.ts @@ -46,7 +46,7 @@ const Project = model('Project', { }, }).sql(({ cols, constraints }) => ({ table: 'project', - indexes: [constraints.index(cols.accountId)], + indexes: [constraints.index([cols.accountId])], foreignKeys: [constraints.foreignKey(cols.accountId, Account.refs.id)], })); diff --git a/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract.ts b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract.ts index 4445de642d..a5c1d2a91b 100644 --- a/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract.ts +++ b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/db-update-scenarios/contract.ts @@ -45,7 +45,7 @@ const Project = model('Project', { }, }).sql(({ cols, constraints }) => ({ table: 'project', - indexes: [constraints.index(cols.accountId)], + indexes: [constraints.index([cols.accountId])], foreignKeys: [constraints.foreignKey(cols.accountId, Account.refs.id)], })); diff --git a/test/integration/test/fixtures/cli/cli-integration-test-app/fixtures/emit-command/contract.parity.ts b/test/integration/test/fixtures/cli/cli-integration-test-app/fixtures/emit-command/contract.parity.ts index 459712ac04..9d008ca7af 100644 --- a/test/integration/test/fixtures/cli/cli-integration-test-app/fixtures/emit-command/contract.parity.ts +++ b/test/integration/test/fixtures/cli/cli-integration-test-app/fixtures/emit-command/contract.parity.ts @@ -44,7 +44,7 @@ const Post = model('Post', { })) .sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [ constraints.foreignKey(cols.userId, User.refs.id, { onDelete: 'cascade', diff --git a/test/integration/test/referential-actions.integration.test.ts b/test/integration/test/referential-actions.integration.test.ts index 3da3c801a3..6559739912 100644 --- a/test/integration/test/referential-actions.integration.test.ts +++ b/test/integration/test/referential-actions.integration.test.ts @@ -227,7 +227,7 @@ describe('referential actions integration', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [ constraints.foreignKey(cols.userId, User.refs.id, { onDelete: 'cascade', @@ -323,7 +323,7 @@ describe('referential actions integration', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [ constraints.foreignKey(cols.userId, User.refs.id, { onDelete: 'cascade', @@ -397,7 +397,7 @@ describe('referential actions integration', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [ constraints.foreignKey(cols.userId, User.refs.id, { onDelete: 'cascade', @@ -498,7 +498,7 @@ describe('referential actions integration', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [constraints.foreignKey(cols.userId, User.refs.id)], })); @@ -574,7 +574,7 @@ describe('referential actions integration', () => { }, }).sql(({ cols, constraints }) => ({ table: 'post', - indexes: [constraints.index(cols.userId)], + indexes: [constraints.index([cols.userId])], foreignKeys: [ constraints.foreignKey(cols.userId, User.refs.id, { onDelete: 'cascade',