INFERENCE

C++ inference engine for LLMs. GGUF format, quantization, CPU and Metal/CUDA support. The foundation most local tools build on.

High-throughput inference server for LLMs. PagedAttention memory management. The go-to for serious local or self-hosted serving.

Apple's array framework for machine learning on Apple Silicon. Native Metal support, unified memory, first-class LLM inference.

Run 50,000+ ML models with a simple API. No infrastructure management. Pay-per-second billing. Deploy custom models with Cog. Popular for image generation and audio.

Fastest inference for open-source models. 200+ models via unified API. Ranks #1 on speed benchmarks for DeepSeek, Qwen, Kimi, and Llama. Serverless pay-per-token pricing.

LPU-powered inference delivering 500-1,000+ tokens/sec. Purpose-built chip with on-chip SRAM instead of HBM. 5-10x faster than GPU providers. Free tier available.

Wafer-scale AI inference at 3,000+ tokens/sec. The WSE-3 chip has 4 trillion transistors and 900K AI cores. 20x faster than GPU providers. OpenAI partnership for inference.