INFERENCE

Google Trends put CBRS stock on the board after Cerebras' first public-company earnings. The developer takeaway is not a trade. It is that AI inference demand is now being priced, questioned, and audited in public.

Google released DiffusionGemma today, a 26B MoE open model that generates entire 256-token blocks in parallel instead of one token at a time. Here is what that means for latency, local inference, and the post-autoregressive landscape.

How KV caching speeds up LLM inference - the math, the code, the memory tradeoffs, and when it stops helping. Every dev running local models hits this wall.

C++ inference engine for LLMs. GGUF format, quantization, CPU and Metal/CUDA support. The foundation most local tools build on.

High-throughput inference server for LLMs. PagedAttention memory management. The go-to for serious local or self-hosted serving.

Apple's array framework for machine learning on Apple Silicon. Native Metal support, unified memory, first-class LLM inference.

Run 50,000+ ML models with a simple API. No infrastructure management. Pay-per-second billing. Deploy custom models with Cog. Popular for image generation and audio.

Fastest inference for open-source models. 200+ models via unified API. Ranks #1 on speed benchmarks for DeepSeek, Qwen, Kimi, and Llama. Serverless pay-per-token pricing.

LPU-powered inference delivering 500-1,000+ tokens/sec. Purpose-built chip with on-chip SRAM instead of HBM. 5-10x faster than GPU providers. Free tier available.

Wafer-scale AI inference at 3,000+ tokens/sec. The WSE-3 chip has 4 trillion transistors and 900K AI cores. 20x faster than GPU providers. OpenAI partnership for inference.