YC startup Magnitude launches self-tuning inference engine for AI agents

Magnitude, a Y Combinator-backed startup, has launched an inference engine designed specifically for running AI agents locally rather than in data centers. Founders Anders and Tom previously built an open-source browser agent with thousands of GitHub stars before deciding existing inference tools didn't fit their needs.

The engine automatically compiles and tunes its kernels based on a user's specific hardware, aiming to match the performance of specialized hardware setups while still working across Mac, Linux and Windows. It also uses dynamic memory allocation, reserving only what's needed for model weights and expanding as agent sessions run, so a computer remains usable for other tasks.

The founders say Magnitude focuses on popular open-weight model architectures and borrows techniques from engines like SGLang to support multiple concurrent agent sessions, positioning it as faster than general-purpose tools like llama.cpp or Ollama for everyday local AI agent use.