AirLLM

Overview

An inference approach that enables 70B-class models on a single 4GB GPU and sparse MoE models on roughly 12GB by streaming one expert at a time. It is useful for teams experimenting with local inference, constrained hardware, or cost-sensitive deployment.