Every major inference framework shipped prefill-decode disaggregation this year. NVIDIA built it into Dynamo. SGLang made it the default for large-scale deployments. vLLM added a KV connector API to support it natively. The consensus is forming fast: split your prefill and decode onto separate GPU pools, and throughput improves. The consensus is wrong for most …

Disaggregation Is a Thousand-GPU Problem Read More »

We ran an experiment: three small datasets, one AI model, and the questions a business team asks in a normal week — what’s our average delivery time, which region is our best performer, how many athletes are in this file. Then we added a review pass. We handed the model its own answer back and …

I Asked ChatGPT to Analyze 3 Datasets. It Made the Same Mistakes Every Time Read More »

Today, we’re launching agentic video understanding across our latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis. Similar to agentic vision, which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools …

Introducing Agentic Video in Gemini Read More »

As the old adage goes, a wise man once said nothing at all. Unfortunately, the same reverence is rarely extended to missing data. A row with a blank cell is often treated as a problem to be solved before analysis can begin: drop it, fill it with an average, do whatever is easiest and move …

What We Miss About Missing Values Read More »

There are many ways to get more from the models and GPU infrastructure you already have. Quantization, optimized kernels, and better inference engines can all help, but speculative decoding is especially useful because it can increase generation speed without simply adding more GPUs. There are now several approaches to speculative decoding. Traditional methods use a …

Speed Up LLM Inference with DSpark Speculative Decoding Read More »