jundot/omlx

Overview

omlx is an LLM inference server optimized for Apple Silicon, featuring continuous batching and SSD-based caching, and is managed from the macOS menu bar.[12] It targets local, high-throughput inference while keeping latency low, giving developers a first-class local dev environment for LLM apps on macOS. Developers should care because it turns Apple laptops into efficient local inference nodes without complex manual setup.