Can I run Qwen3.8 27B locally?
Qwen3.8 27B uses a hybrid architecture: only 16 of its 64 layers use full attention with a KV cache, while the other 48 use linear attention with a fixed-size state. That is why its memory grows slowly with context compared with a classic 27B–32B model, and why calculators that assume every layer has a KV cache overestimate it.
Weights below are real GGUF file sizes from Hugging Face. For some levels only Unsloth “dynamic” (UD) files exist; the calculator labels those.
Estimates with default assumptions: 10% runtime overhead, ~90% usable VRAM (~72% on Macs), FP16 KV cache, one sequence. Model data synced from Hugging Face on 2026-10-18. Methodology.
Qwen3.8 27B: memory by quantization
27.3B parameters, 64 layers. Weights are real GGUF file sizes from unsloth/Qwen3.8-27B-GGUF.
| Quantization | Weights | Total at 8K context | Total at 32K context |
|---|---|---|---|
| F16 | 50.9 GB | 56.6 GB | 58.1 GB |
| Q8_0 | 27.1 GB | 30.4 GB | 31.9 GB |
| Q6_KUnsloth dynamic file (UD-Q6_K) | 20.5 GB | 23.2 GB | 24.7 GB |
| Q5_K_MUnsloth dynamic file (UD-Q5_K_M) | 18.4 GB | 20.9 GB | 22.4 GB |
| Q4_K_MUnsloth dynamic file (UD-Q4_K_M) | 15.3 GB | 17.5 GB | 19.0 GB |
| IQ4_XSUnsloth dynamic file (UD-IQ4_XS) | 13.3 GB | 15.2 GB | 16.7 GB |
Which hardware runs Qwen3.8 27B?
Highest-quality quantization that fits fully. “Offload” means it runs only with part of the model in system RAM (64 GB assumed).
| Hardware | 8K context | 32K context |
|---|---|---|
| NVIDIA GeForce RTX 4060 Ti 16GB | Offload IQ4_XS (~60 layers on GPU) | Offload IQ4_XS (~55 layers on GPU) |
| NVIDIA GeForce RTX 3090 | Q5_K_M (20.9 GB) | Q4_K_M (19.0 GB) |
| NVIDIA GeForce RTX 5090 | Q6_K (23.2 GB) | Q6_K (24.7 GB) |
| Mac with M4 Pro (24 GB) | IQ4_XS (15.2 GB) | IQ4_XS (16.7 GB) |
| Mac with M4 Max (36 GB) | Q6_K (23.2 GB) | Q6_K (24.7 GB) |
| Mac with M4 Pro (48 GB) | Q8_0 (30.4 GB) | Q8_0 (31.9 GB) |
| NVIDIA L40S (48 GB) | Q8_0 (30.4 GB) | Q8_0 (31.9 GB) |
Try your own setup
Result
Fits fully
Qwen3.8 27B · Q4_K_M · 32K → NVIDIA GeForce RTX 4090. 2.59 GB of headroom.
| Component | Memory |
|---|---|
| Weightsactual GGUF file: UD-Q4_K_M | 15.3 GB |
| KV cache32K tokens × 1 seq, F16 | 2.00 GB |
| Recurrent statelinear-attention layers, fixed size | 0.15 GB |
| Runtime overhead10% of weights | 1.53 GB |
| Total needed | 19.0 GB |
| Usable on your hardware90% of 24.0 GB | 21.6 GB |
- Only an Unsloth “dynamic” file exists at this level (UD-Q4_K_M); its size is used.
Run it
Starting point, not a guarantee. Downloads the GGUF from Hugging Face.
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M -c 32768 -ngl 999OLLAMA_CONTEXT_LENGTH=32768 ollama serve
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_MModel details
- Parameters
- 27.3BDerived from the BF16 GGUF size (54657735616 bytes / 2 bytes per weight), text model only
- Layers
- 64
- Attention
- Hybrid: linear attention + full attention48× linear · 16× full (4 KV × 256)
- Max context
- 256K
- License
- apache-2.0
- Config source
- Qwen/Qwen3.8-27Bsynced 2026-10-18
Questions
Does Qwen3.8 27B fit on a 16 GB GPU?
Not fully at 4-bit. The Q4_K_M file is about 15 GB and even IQ4_XS needs about 15 GB with KV cache and overhead at 8K, just over the ~14.4 GB usable on a 16 GB card, so it runs with partial CPU offload. A 24 GB card fits Q5_K_M at 8K and Q4_K_M at 32K.
Why is the KV cache so small for Qwen3.8?
Only the full-attention layers (16 of 64, with 4 KV heads of dimension 256) store keys and values per token. The 48 linear-attention layers keep a fixed recurrent state of about 3 MB each per sequence, independent of context length.
Does the calculator include the vision encoder?
No. Qwen3.8 is a vision-language model, but the GGUF text files exclude the vision projector (mmproj), which is a separate download. If you load the projector too, add its file size.