sgl-project/sglang

Overview

SGLang is a high-performance serving framework for large language and multimodal models, combining an optimized inference backend with a domain-specific language for prompt programming and workflow composition.[3] It targets production deployments where throughput and latency matter, and gives developers structured ways to define complex chains, tools, and agents. Developers should care because it reduces infra overhead while giving a more rigorous abstraction than ad hoc prompt strings.