Where model memory goes
Qwen3.8-Flash-Next · GSQ-RCO · IQ2_XS / ISTA-DASLab / 68.03 GB files (63.35 GiB)
Which weights work?
An expert is an FFN weight block inside a layer, not a separate model. All 48 layers are still traversed. Chunks and experts are different units; this is not a model-size comparison. Outlines mark recurring toy selections.
Where our memory goes
Each cell is 1 GiB; partial cells show exact extent.
KV capacity was preallocated; changing logical use does not change allocated memory.
Beyond VRAM
The selected model’s 33 GiB expert weights alone exceed the 16 GiB GPU. They cannot all fit there.
Offloading can use RAM and SSD to extend capacity, with CPU, PCIe and storage costs. Live KV moves between RAM and GPU, not SSD.
Ollama documents CPU/GPU splits; SGLang documents CPU offload. Support depends on model and backend. These are options, not a matched speed comparison, and not every engine runs this GGUF.