RECORD / 62C3B8 / REVIEWED

AirLLM

AirLLM lowers peak accelerator memory by loading model layers sequentially instead of keeping the full checkpoint resident during inference.

State: available
Layer: infrastructure
Reviewed: 2026-10-17

DECISION / USE

Why it matters

Use it to test models that exceed available VRAM when slow disk-backed generation is acceptable.

CAUTION / LIMITS

What to check first

The memory reduction shifts pressure to storage and I/O, so latency can be high; model compatibility and required disk space must be checked for each checkpoint.

REL / KNOWLEDGE

1 related record

01KV Cache Transform Coding →KV Cache Transform Coding applies learned decorrelation, adaptive quantization, and entropy coding to reduce storage and transfer costs for reusable language-model KV caches.research paper