Skip to content
#

8gb-vram

Here are 14 public repositories matching this topic...

Run a 35B LLM on a GTX 1070 with 8 GB VRAM: custom CUDA inference engine for Ornith-1.5-35B-A3B (MoE, NVFP4) on Windows, ~25-30 tok/s. Expert streaming, fused NVFP4 kernels, prefix cache, OpenAI-, Anthropic- and Ollama-compatible API for local AI agents.

  • Updated Nov 14, 2026
  • Python

Everything I made run fast on a 2016 GPU with 8 GB: llama.cpp kernels, CUDA graphs, speculative decoding, ternary models, LoRA training, video diffusion on a GTX 1080 (sm_61). Measured, with sources.

  • Updated Nov 3, 2026
  • Python

Add this topic to your repo

To associate your repository with the 8gb-vram topic, visit your repo's landing page and select "manage topics."

Learn more