Skip to content

Commit 8b7f3a3

Browse files
committed
docs: add AI comparison docs
PACT-4279
1 parent dfd1a7e commit 8b7f3a3

File tree

1 file changed

+43
-1
lines changed

1 file changed

+43
-1
lines changed

website/docs/docs/ai/using.md

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -414,4 +414,46 @@ In AI, context is key. The more relevant information you can provide the more ta
414414

415415
### 3. Use Test Templates
416416

417-
Test Templates reduce the amount of refactoring required after generation. If you start to see a common generation pattern you wish to avoid, update the Test Template to help PactFlow AI generate improved output.
417+
Test Templates reduce the amount of refactoring required after generation. If you start to see a common generation pattern you wish to avoid, update the Test Template to help PactFlow AI generate improved output.
418+
419+
420+
421+
## Comparing PactFlow AI with Other AI Tools
422+
423+
PactFlow AI is purpose-built for contract testing workflows. While general-purpose AI tools like ChatGPT, GitHub Copilot, and IDE-native assistants (e.g., Windsurf) offer broad utility, they are not optimized for *specific* domains and use cases like contract testing with Pact.
424+
425+
The table below outlines key differences between these tools to help you understand how PactFlow AI can be incorporated into your development process.
426+
427+
### Feature Comparison
428+
429+
| Feature | **PactFlow AI** | **ChatGPT / Claude** | **GitHub Copilot** | **IDE-native AI (e.g., Windsurf)** |
430+
|--------|------------------|----------------------|--------------------|-------------------------------|
431+
| **Primary Use Case** | Contract test generation, review, and maintenance | General knowledge, Q&A, prototyping | Inline code suggestions | IDE-embedded assistant for development tasks |
432+
| **Breadth of Application** | Focused on Pact | Wide range of dev and non-dev tasks | General-purpose coding | Full-stack code, terminal, file editing |
433+
| **Strength in General Coding** | ❌ Not built for general-purpose coding | ✅ Excellent at most languages and use cases | ✅ Great for scaffolding and boilerplate | ✅ Helps with app logic, CLI, tests, refactors |
434+
| **Integration Surface** | CLI (now), IDE via MCP (coming), UI (planned) | Chat/web/app with integrations | Deep IDE integration (VS Code, JetBrains), Web/UI | Native to Cursor, Cline, etc. |
435+
| **Contract Testing Support** | ✅ Pact-specific knowledge, updated DSLs, test structure and matcher guidance | ❌ Generic or outdated knowledge | ❌ May suggest invalid or old Pact patterns | ❌ No contract testing awareness |
436+
| **Contract Testing Maintenance** | ✅ Review, update, and fix existing Pact tests | ❌ Prompt-based only | ❌ Suggests code, not test upkeep | ❌ Not designed for test maintenance |
437+
| **TDD Alignment** | ✅ Encourages scoped, purposeful test creation | ❌ May reinforce ad hoc development | ✅ Scaffolds tests, but not goal-oriented | ⚠️ Speed-optimized, not test-structured |
438+
| **Human-in-the-loop Design** |AI suggestions are reviewable and optional | ⚠️ User must prompt and validate output | ✅ Autocomplete with human review | ⚠️ High-speed, low-friction suggestions |
439+
| **Governance & Control** |RBAC-aware, CLI and platform-controlled | ⚠️ Black-box models, limited traceability | ⚠️ Limited observability | ⚠️ Tool-dependent privacy and trust model |
440+
441+
### When to Use Each Tool
442+
443+
| Use Case | Recommended Tool |
444+
|----------|------------------|
445+
| Creating and maintaining Pact tests | **PactFlow AI** |
446+
| Prototyping or researching APIs, code patterns | **ChatGPT / Claude** |
447+
| Writing app logic, scaffolding components or full-stack development in IDE | **GitHub Copilot**, **IDE-native AI tools** |
448+
449+
### Summary
450+
451+
PactFlow AI is not designed to replace general-purpose AI tools. Instead, it complements them by offering deep, current knowledge of contract testing—particularly Pact.
452+
453+
While other tools can help with coding in general, they do not provide guidance on:
454+
- Pact test quality
455+
- Over-specification and drift detection
456+
- DSL correctness and best practices
457+
- And other contract-testing specific capabilities
458+
459+
For teams working with Pact, PactFlow AI acts as a domain expert—reviewing, generating, and maintaining tests with precision and context.

0 commit comments

Comments
 (0)