Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 

Repository files navigation

DeepFabric Dataset Generator: Claude Code Skill

A Claude Code skill that turns a single sentence into a fine-tuning-ready synthetic dataset. You describe your topic, and Claude handles the config, generation, and training script.

Note: This skill currently only works with the terminal version of Claude Code (CLI). It is not supported in the VS Code extension or other GUI clients.

What is a Claude Code Skill?

Claude Code skills are structured instruction sets that extend Claude Code's capabilities for specific workflows. When installed, the DeepFabric generator skill guides you through the entire dataset creation process, from gathering requirements to running deepfabric generate and producing a ready-to-use fine-tuning script, all through an interactive conversation.

Setup

Video guide: https://youtu.be/WDt71V2tuZw
A quick walkthrough showing how to install and run the skill.

1. Install the skill

Clone the repo directly into your Claude Code skills directory:

git clone https://github.com/always-further/deepfabric-skill.git ~/.claude/skills/deepfabric-generator

Or for a project-scoped install (available only in the current repo):

git clone https://github.com/always-further/deepfabric-skill.git .claude/skills/deepfabric-generator

To update the skill later, just pull:

git -C ~/.claude/skills/deepfabric-generator pull

Your directory should look like this:

~/.claude/skills/deepfabric-generator/
├── SKILL.md
├── README.md
└── assets/
    ├── config_template.yaml
    └── template_train.py

The skill is automatically available the next time you start Claude Code.

2. Install DeepFabric

pip3 install deepfabric

3. Set your API key

Export your API key in the terminal before launching Claude Code. The key must be present in the environment when generation runs.

# Gemini (recommended, has a free tier)
export GEMINI_API_KEY="your-key-here"

# Anthropic
export ANTHROPIC_API_KEY="your-key-here"

To avoid setting this every session, add the export to your shell profile (~/.zshrc or ~/.bashrc).

VS Code users: The integrated terminal may not inherit your shell environment. Either add the export to your shell profile or run it manually in the VS Code terminal before generating.

4. Mock files setup (optional)

If you want your dataset to include mock files (simulated project files the AI can read/write during training conversations), you need a running Spin server. The skill will ask which method you prefer:

Option A: Docker (no extra toolchain needed)

cd /path/to/deepfabric/tools-sdk && docker compose up -d

Runs on port 3030.

Option B: Local Spin

Requires Spin CLI (v2.0+) and Rust with the wasm32-wasip1 target:

cd /path/to/deepfabric/tools-sdk && spin build && spin up

Runs on port 3000.

For non-coding topics (recipes, history, math concepts), mock files don't add value, so skip this entirely.

Usage

Tell Claude Code you want to generate a dataset:

"Generate a dataset about building REST APIs in Python"

The skill walks you through setup interactively.

Step 1: Knowledge graph

Before asking for settings, the skill explains how DeepFabric structures generation using a knowledge graph, using your topic as the example:

REST APIs in Python
├── Authentication     → JWT, OAuth2, API Keys
├── Request Handling   → Routing, Validation, Middleware
└── Data Layer         → ORM, Migrations, Query Optimization

Depth = how many levels deep the graph goes. Degree = how many branches at each node. Together they determine the number of unique subtopics and the diversity of your dataset.

You pick both from an interactive menu, and the skill shows you exactly how many topics each combination produces.

Step 2: Provider and customization

The skill asks which API provider to use (Gemini or Anthropic), then offers two paths:

  • Pick the best defaults for me: Claude analyzes your topic and selects conversation type, reasoning style, sample count, and mock file settings. It explains the choices before anything is created so you can object if something feels off.
  • Customize further: walks you through each option manually: conversation type (Q&A, instruction-following, chat), reasoning style (chain-of-thought, direct, step-by-step), number of samples, and mock files.

Step 3: Config and generation

The choices feed into a YAML config. For example, a REST API dataset with mock files:

topics:
  prompt: "Building REST APIs in Python"
  mode: graph
  depth: 3
  degree: 3
  save_as: "topics.jsonl"
  llm:
    provider: "gemini"
    model: "gemini-2.5-flash"

generation:
  system_prompt: "You are an expert Python backend developer..."
  conversation:
    type: cot
    reasoning_style: agent
  tools:
    spin_endpoint: "http://localhost:3030"
    components:
      builtin: [read_file, write_file, list_files]
    scenario_seed:
      files:
        "main.py": |
          from fastapi import FastAPI
          app = FastAPI()
          # TODO: add your routes here
    max_per_query: 3
    max_agent_steps: 5
  llm:
    provider: "gemini"
    model: "gemini-2.5-flash"

output:
  num_samples: 27
  save_as: "dataset.jsonl"

Then runs:

cd rest_apis_python && deepfabric generate rest_apis_python_config.yaml --tui simple

Generation typically takes 25–30 minutes for a balanced depth/degree config. Once done, the skill verifies the output file exists, is non-empty, and has the expected sample count.

Step 4: Training script

The skill creates a fine-tuning script using TRL's SFTTrainer. The default model is Qwen/Qwen3-0.6B, small enough to train locally and capable enough to validate dataset quality. You can swap it out before the script is finalized.

Output

Each run produces a self-contained project folder:

File Description
<topic>_config.yaml DeepFabric configuration
topics.jsonl Generated knowledge graph
dataset.jsonl Synthetic training dataset
<topic>_train.py Fine-tuning script

Requirements

  • Python 3.8+
  • deepfabric package (pip3 install deepfabric)
  • trl and datasets packages (for training)
  • A valid Gemini or Anthropic API key
  • (Optional) Docker or Spin CLI for mock file support

Resources

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages