oMLX

oMLX is an innovative solution that transforms your Mac into a powerful full LLM inference server, accessible directly from the menu bar. It supports a range of models including text, vision, OCR, embeddings, and rerankers, all optimized with continuous batching for high efficiency. Its unique RAM+SSD tiered key-value cache ensures persistent speed improvements even after restarts, dramatically reducing response times from around 90 seconds to just 5 seconds for popular models like Claude Code and Cursor. Built with native Swift rather than Electron, oMLX offers a lightweight, seamless experience for developers and AI practitioners looking to deploy and experiment with large language models locally. Compatibility with OpenAI and Anthropic APIs means it integrates smoothly into existing workflows, making it ideal for those seeking faster inference on their Mac hardware. Being open source under Apache 2.0, it also invites customization and community collaboration, positioning itself as a compelling tool in the AI developer ecosystem.