Inference and serving

AirLLM lowers peak accelerator memory by loading model layers sequentially instead of keeping the full checkpoint resident during inference.

Cache-to-Cache learns projections that fuse a source model's KV cache into a receiver, letting paired language models exchange latent state without generating intermediate text.

KV Cache Transform Coding applies learned decorrelation, adaptive quantization, and entropy coding to reduce storage and transfer costs for reusable language-model KV caches.