v1.8.0 - Quantized 8-Bit Model & Dynamic 8-Bit Quantization

[Enemyx-net]

🎉 Quantized 8-Bit Model Release

• New Model: VibeVoice-Large-Q8 - Perfect quality with 40% less VRAM
• Dynamic 8-Bit: Added 8-bit option for on-the-fly quantization

📦 VibeVoice-Large-Q8

Download

• HuggingFace: FabioSarracino/VibeVoice-Large-Q8
• Size: 11.6GB (38% smaller than full precision)
• VRAM: ~12GB (40% reduction from ~20GB)

Key Innovation

• Perfect Audio Quality: Identical to full precision - zero degradation
• Selective Quantization: Audio-critical components remain at full precision
  • Diffusion head ✓ Full precision
  • VAE ✓ Full precision
  • Connectors ✓ Full precision
  • LLM ✓ 8-bit quantized

🔧 Dynamic 8-Bit Quantization

Enhanced quantize_llm Parameter

Now supports three options:

• full precision (default) - Original quality
• 8bit (NEW) - Balanced quality/VRAM
• 4bit - Maximum VRAM savings

Smart Component Selection

8-bit mode intelligently skips:

• Diffusion head
• Acoustic/semantic connectors
• Tokenizers
• All audio-processing components

💾 VRAM Comparison

Model/Mode	VRAM Usage	Quality
VibeVoice-Large	~20GB	Perfect
VibeVoice-Large-Q8	~12GB	Perfect
Dynamic 8-bit	~13-14GB	Excellent
Dynamic 4-bit	~10-11GB	Very Good
VibeVoice-Large-Q4	~8GB	Good

🎯 Perfect For

• RTX 3060 (12GB): Run VibeVoice-Large-Q8 with perfect quality
• RTX 4070 Ti (12GB): Full model without compromises
• RTX 3050 (8GB): Use VibeVoice-Large-Q4 or dynamic 4-bit
• Production: Best quality-to-VRAM ratio with Q8
• Development: Fast iteration with dynamic quantization

⚙️ Requirements

For dynamic quantization:

• NVIDIA CUDA GPU
• bitsandbytes library
• Falls back to full precision on CPU/MPS

💾 Installation

Install via ComfyUI Manager or manually:

git clone https://github.com/Enemyx-net/VibeVoice-ComfyUI

---

This discussion was created from the release v1.8.0 - Quantized 8-Bit Model & Dynamic 8-Bit Quantization.