🚀 Prompt-to-Pipeline Generator

Transform natural language prompts into sophisticated multi-step AI workflows with a single command!

This innovative system demonstrates the power of prompt engineering by automatically decomposing complex requests into executable pipelines, showcasing how AI can act as a workflow orchestrator rather than just a chatbot.

🌟 Features

• 🧠 Intelligent Task Decomposition: Automatically breaks down complex prompts into structured, executable tasks
• 🔗 Dynamic Prompt Chaining: Chains multiple AI operations together for sophisticated workflows
• 📊 Multi-Format Output: Generate text summaries, quizzes, presentations, code, and more
• ⚡ Real-Time Execution: Watch your pipeline execute in real-time with live status updates
• 🎨 Beautiful UI: Modern, responsive interface with smooth animations
• 🔌 Extensible Architecture: Easy to add custom modules and task types
• 🚦 Smart Dependencies: Automatic dependency resolution between tasks
• 📈 Performance Monitoring: Track execution times and optimize workflows

🎯 Use Cases

• 📚 Research Assistant: PDF → Summary → Key Points → Quiz → Presentation
• ✍️ Content Creator: Topic → Research → Outline → Article → Social Media Posts
• 📊 Data Analyst: CSV → Analysis → Visualization → Report → Dashboard
• 🎓 Education: Lecture Notes → Study Guide → Flashcards → Practice Tests
• 💼 Business: Meeting Notes → Action Items → Task Assignments → Follow-up Emails

🏗️ Architecture

User Input → Intent Parser → Task Decomposer → Pipeline Builder → Task Executor → Output Aggregator
     ↓            ↓               ↓                  ↓                ↓              ↓
   Prompt    Identify Tasks   Create Graph    Order Tasks    Run Modules   Combine Results

🎬 Demo

🚀 Quick Start

Prerequisites

• Python 3.9+
• Node.js 16+
• Docker & Docker Compose (optional)
• OpenAI API Key or Anthropic Claude API Key

Installation

1. Clone the repository

git clone https://github.com/SteveTM-git/prompt-to-pipeline.git
cd prompt-to-pipeline

2. Set up the backend

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

3. Set up environment variables

cp .env.example .env
# Edit .env and add your API keys

4. Run the backend server

uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

5. Set up the frontend (in a new terminal)

cd frontend
npm install
npm run dev

6. Open your browser Navigate to http://localhost:3000

🐳 Docker Setup (Alternative)

# Build and run all services
docker-compose up --build

# Access the application
# Frontend: http://localhost:3000
# Backend API: http://localhost:8000
# API Docs: http://localhost:8000/docs

📖 API Documentation

Create Pipeline

POST /pipeline/create
Content-Type: application/json

{
  "prompt": "Summarize this document and create a quiz",
  "input_data": {"text": "Your content here..."},
  "params": {"quiz_questions": 5}
}

Execute Pipeline

POST /pipeline/{pipeline_id}/execute

Get Pipeline Status

GET /pipeline/{pipeline_id}/status

WebSocket for Real-time Updates

const ws = new WebSocket('ws://localhost:8000/ws/{pipeline_id}');
ws.onmessage = (event) => {
  const update = JSON.parse(event.data);
  console.log('Pipeline update:', update);
};

🔧 Extending the System

Adding a New Task Module

1. Create a new module in backend/app/modules/:

# backend/app/modules/custom_module.py
from core.pipeline_system import TaskModule

class CustomTaskModule(TaskModule):
    async def execute(self, input_data, params):
        # Your custom logic here
        result = process_data(input_data)
        return result

2. Register the module in the executor:

# backend/app/core/pipeline_system.py
self.modules[TaskType.CUSTOM] = CustomTaskModule()

3. Add intent patterns for automatic detection:

# backend/app/core/pipeline_system.py
TaskType.CUSTOM: [r'\bcustom\b', r'\bspecial\b']

📊 Performance Benchmarks

Operation	Average Time	Token Usage	Success Rate
Simple Pipeline (3 tasks)	4.2s	~2,500	98%
Complex Pipeline (7 tasks)	12.8s	~8,000	95%
Parallel Execution	6.5s	~8,000	94%

🧪 Testing

# Run backend tests
cd backend
pytest tests/ -v --cov=app

# Run frontend tests
cd frontend
npm test

# Run integration tests
docker-compose -f docker-compose.test.yml up --abort-on-container-exit

📁 Project Structure

prompt-to-pipeline/
├── backend/
│   ├── app/
│   │   ├── main.py              # FastAPI application
│   │   ├── core/
│   │   │   ├── pipeline_system.py
│   │   │   ├── intent_parser.py
│   │   │   └── task_decomposer.py
│   │   ├── modules/             # Task implementation modules
│   │   ├── prompts/             # Prompt templates
│   │   └── utils/
│   ├── tests/
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── components/          # React components
│   │   ├── pages/               # Next.js pages
│   │   └── services/            # API clients
│   ├── public/
│   └── package.json
├── docker-compose.yml
├── .env.example
└── README.md

🚀 GitHub Setup & Deployment

Preparing for GitHub

1. Initialize Git repository

git init
git add .
git commit -m "Initial commit: Prompt-to-Pipeline Generator"

2. Create GitHub repository

# Using GitHub CLI
gh repo create prompt-to-pipeline --public --description "Transform natural language into AI workflows"

# Or manually create on GitHub and add remote
git remote add origin https://github.com/SteveTM-git/prompt-to-pipeline.git

3. Set up GitHub Actions for CI/CD Create .github/workflows/ci.yml:

name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test-backend:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
      - name: Install dependencies
        run: |
          cd backend
          pip install -r requirements.txt
      - name: Run tests
        run: |
          cd backend
          pytest tests/

  test-frontend:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: '18'
      - name: Install and test
        run: |
          cd frontend
          npm ci
          npm test

4. Add GitHub Secrets Go to Settings → Secrets and add:

• OPENAI_API_KEY
• ANTHROPIC_API_KEY
• DATABASE_URL

5. Push to GitHub

git branch -M main
git push -u origin main

Deployment Options

Vercel (Frontend)

# Install Vercel CLI
npm i -g vercel

# Deploy frontend
cd frontend
vercel --prod

Railway/Render (Backend)

1. Connect your GitHub repo
2. Set environment variables
3. Deploy with one click

AWS/GCP/Azure

Use provided Docker setup for container deployment

🤝 Contributing

We welcome contributions!Here's how:

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

🙏 Acknowledgments

• OpenAI & Anthropic for LLM APIs
• FastAPI for the amazing web framework
• React & Next.js communities
• All contributors and supporters

📧 Contact

• Author: STEVE THOMAS MULAMOOTTIL
• Email: st816043@gmail.com
• LinkedIn: https://www.linkedin.com/in/steve-thomas-mulamoottil

---

Made with ❤️ by students passionate about AI and automation