Millwright

Overview

Millwright is an open-source, self-hosted LLM router written in Rust, exposing a CLI and server for routing requests among multiple providers.[4] It supports OpenAI-compatible APIs, Anthropic, and Amazon Bedrock, letting developers define routing rules based on model, cost, or latency. With a focus on performance and observability, it is ideal for teams that want vendor-agnostic LLM orchestration without adopting large frameworks. Developers should care because it centralizes multi-provider management and can cut costs by smartly routing workloads.