By AI Platform’s Model Runtime team and Inference team Introduction Most organizations consume LLMs through hosted APIs. Netflix went further — we run the full stack ourselves, from model deployment through inference, inside our existing production environment rather than a separate ML silo. Some of those decisions weren’t obvious, and a few revealed their trade-offs only under production load. This…
#llm
20 posts
17 Jul
Building Swiggy’s AI onboarding companion for delivery executives, where accuracy alone wasn’t enough and mentorship had to scale. For a new Delivery Executive (DE) joining Swiggy, the experience is often overwhelming, throwing a world of SOPs, incentive structures, zone logistics, and app workflows at them all at once. This friction is evident in our metrics, but the drop-off isn’t due…
14 Jul
Training an LLM is the easy part. The hard part is designing experiments and evaluations that you can trust enough to know whether the new model is actually an improvement. By : Baharak Saberidokht Introduction Shipping a production LLM system means iterating fast on improvements to something that is, by construction, non-deterministic. Models drift, judges disagree with themselves, references regenerate…
8 Jul
I went into a recent project excited. The newer LLM tools had crossed a threshold for me. They felt like something I could finally lean on hard for a tricky visual-design problem: cross-domain research, concept generation, and prototyping all running in one loop. I was genuinely confident going in. And it delivered. The LLM helped […] The post Where the…
30 Jun
Expedia Group Technology — Innovation Using large language models to reveal bottlenecks in Spark SQL execution plans Photo by Luis del Río If you’ve ever stared at a 300-plus-node physical plan at 2 a.m. trying to spot a missing broadcast or one cursed skewed partition, this is for you. Spark makes it deceptively easy to write complex SQL that looks…
25 Jun
We used DSPy to improve LLM judges and optimize our chat experience, creating an evaluation-driven feedback loop that produced better outputs.
28 May
Authors: Ahmed Ben Yahmed , Antoine Schnepf , Karim Kassab , and Mélissa Tamine . The 14th International Conference on Learning Representations ( ICLR 2026 ) was held from April 23 to 27, 2026, at the Riocentro Convention and Event Center in Rio de Janeiro, Brazil. It was the first time the conference made its way to South America. As…
18 May
In recent weeks, we pointed Mythos and other security-focused LLMs at live code across critical parts of our infrastructure. We share what we observed, the models’ strengths and weaknesses, and what the work around them needs to look like before any of it can scale.
16 Apr
Author: Fabian Höring Agentic systems powered by LLMs can be incredibly impressive in demos. With a few well-crafted prompts, they can demonstrate reasoning, calling tools, and solving complex tasks [1]. Demos are effective at showcasing what’s possible. Production environments, however, are where those capabilities are tested at scale and under real-world conditions. The same agent that performs perfectly on curated…
24 Mar
This tutorial is for developers, designers who code, or anyone new to AI who wants a hands-on introduction to building a custom AI chatbot that can search and answer questions using your own data. I wanted to build an AI-powered tool for our team, but I had zero experience building AI applications—so I decided to […] The post How to…
17 Mar
We used DSPy to turn prompt engineering for our relevance judge into a measurable, automated optimization loop, improving task performance, cost, and how reliably it works in production.
3 Mar
A Story from the Frontlines of Threat Intelligence The identification of cyber threats in e-commerce is challenging because discussions on topics like gift cards and discounts, when found on underground forums, can signal fraud yet closely resemble normal customer behavior. For security teams, distinguishing a genuine threat from routine chatter is extremely difficult, making risk identification feel like searching for…
26 Feb
How we train Dash's search ranking models with a mix of human and LLM-assisted labeling.
28 Jan
Engineering VP Josh Clemm deep-dives into how we think about knowledge graphs, indexes, MCP, and prompt optimization using tools like DSPy.
Learn how Microsoft.Extensions.AI provides a unified API for building intelligent .NET applications with any LLM provider - one interface, endless possibilities, with built-in middleware, telemetry, structured outputs and more. The post .NET AI Essentials – The Core Building Blocks Explained appeared first on .NET Blog.
15 Jan
We use GenAI in every facet of technology now - internal knowledge bases, customer support systems, and code review bots, to name just a few use cases. And in nearly every one of these, someone eventually asks: What stops the model from returning something the user shouldn’t see?” This is a roadblock that companies building...
18 Dec 2025
The feature store is a critical part of how we rank and retrieve the right context across your work.
12 Nov 2025
Imagine a scenario where you want to translate a sentence from English to another language. Without additional logic or processing, a simple phrase like “the cat is sitting on the mat” in English might translate to “Le chat est séance sur le tapis.” The actual translation should be “Le chat est assis sur le tapis,” ...
3 Jul 2025
Over the past 15 months I have been actively working with LLM’s in my job as a software engineer. I have had many moments where I thought singularity was here, but then almost immediately after I would see the agent fall flat on its face. I have become interested in figuring out when and why these new tools fail, in…
25 Nov 2024
How we improved Canva’s private design search while respecting the privacy of our community.