FAST_TRAINING_GUIDE.md

Fast Single-Machine Path (CPU + GPU)

This document is a practical route to reach strong task-level quality quickly on one machine.

Important scope:

• Do NOT train a GPT-3.5-scale model from scratch.
• Use parameter-efficient fine-tuning (LoRA/QLoRA) on an open base model.
• Target: approach GPT-3.5-level quality on a bounded benchmark suite.

1) Objective Definition

Define a constrained target benchmark:

• Domain QA (your domain)
• Structured instruction following
• Chinese/English bilingual tasks (if needed)
• Hallucination-sensitive prompts

Primary metrics:

• task accuracy/pass rate
• hallucination rate
• latency and memory footprint

2) Recommended Training Strategy

1. Start from an open base model (7B/8B class)
2. Use QLoRA (4-bit loading + LoRA adapters)
3. Curate small high-quality supervised dataset first
4. Run short training cycles with frequent eval checkpoints
5. Stop early once validation plateau appears

3) Hardware-Aware Rules

• CPU role: data pipeline, tokenization, checkpoint I/O
• GPU role: forward/backward and optimizer math
• Use bf16/fp16 when available
• Keep batch size small and use grad accumulation

4) Minimal Workflow

1. Prepare data in instruction format JSONL
2. Run baseline eval on base model
3. Fine-tune with QLoRA
4. Re-evaluate on same benchmark
5. Compare against baseline and target model outputs

5) What Counts as Success

• Significant gain over base model on your benchmark
• Reproducible across at least 3 seeds
• Acceptable latency on your single machine

6) Reporting Template

• Base model:
• Train method: QLoRA
• Train time:
• Peak VRAM:
• Benchmark score before:
• Benchmark score after:
• Relative gain:

7) Next Step in This Repo

Use the checklist in FAST_TRAINING_CHECKLIST.md and log runs in output/reports/fast_training_runs.csv.