Heroku is introducing significant updates to Managed Inference and Agents. These changes focus on reducing developer friction, expanding model catalogue, and streamlining deployment workflows. More flexibility with the new standard plan Until now, Heroku’s model-based plans required developers to provision a specific add-on for a specific model. This created significant operational overhead. If you wanted […] The post Whats New…
#heroku ai
23 posts
19 Feb
17 Feb
Large language models are good at writing code. Data from Anthropic shows that allowing Claude to execute scripts, rather than relying on sequential tool calls, reduces token consumption by an average of 37%, with some use cases seeing reductions as high as 98%. Untrusted code needs a secure and isolated place to execute. We solved […] The post Code Execution…
29 Jan
If you’ve built a RAG (Retrieval Augmented Generation) system, you’ve probably hit this wall: your vector search returns 20 documents that are semantically similar to the query, but half of them don’t actually answer it. A user asks “how do I handle authentication errors?” and gets back documentation about authentication, errors, and error handling in […] The post Building AI…
15 Jan
Today, we are announcing the general availability of reranking models on Heroku Managed Inference and Agents, featuring support for Cohere Rerank 3.5 and Amazon Rerank 1.0. Semantic reranking models score documents based on their relevance to a specific query. Unlike keyword search or vector similarity, rerank models understand nuanced semantic relationships to identify the most […] The post Optimize Search…
18 Dec 2025
Heroku AI: Accelerating AI Development With New Models, Performance Improvements, and Messages API
HerokuThis month marks significant expansion for Heroku Managed Inference and Agents, directly accelerating our AI PaaS framework. We’re announcing a substantial addition to our model catalog, providing access to leading proprietary AI models such as Claude Opus 4.5, Nova 2, and open-weight models such as Kimi K2 thinking, MiniMax M2, and Qwen3. These resources are […] The post Heroku AI:…
4 Dec 2025
Heroku is launching automatic prompt caching starting December 18, 2025. Prompt caching delivers a notable, zero-effort performance increase for Heroku Managed Inference and Agents. Enabled by default, this feature is designed to deliver significantly faster responses for common workloads. We have taken a pragmatic approach and currently only enabled this to cache system prompts and […] The post Faster Agents…
14 Oct 2025
Today’s businesses face a tremendous amount of complexity in tools, data silos, and systems that teams need to navigate to deliver unique and engaging experiences to their customers. Meanwhile developers are only able to spend a fraction of their time coding due to the cognitive load of technology complexity, constant context switching, and figuring out how to adopt AI effectively…
8 Oct 2025
Introducing the pilot of Heroku Vibes, your collaborative agent for turning ideas into running apps. For those who have been with us on this journey for a while, the name “Heroku Garden” might stir up a bit of nostalgia. It was the web experience that enabled developers to become immediately productive in creating and deploying […] The post Welcome to…
17 Sept 2025
Ever found yourself in the endless loop of tweaking a prompt, running your code, and waiting to see if you finally got the output you wanted? That slow, frustrating feedback cycle is a common headache for AI developers. What if you could speed that up and get back to what you do best? Let’s focus on building amazing applications. We're…
4 Sept 2025
The AI revolution presents a critical challenge: moving from experimentation to production. This year, Heroku has evolved beyond a traditional PaaS to become an AI PaaS, a fully managed platform designed to solve this problem and accelerate the delivery of AI-powered apps. With new capabilities like AppLink, Managed Inference and Agents, and MCP on Heroku, Heroku now provides a fully…
26 Aug 2025
Building intelligent applications requires powerful, cost-effective AI. Today, we’re simplifying that process by making Amazon’s cutting-edge Nova models directly available via Heroku Managed Inference and Agents. Provisioning these models is as simple as attaching the add-on to your Heroku application, providing a direct, managed path for developers and businesses to leverage a new class of […] The post Amazon Nova…
20 Aug 2025
Start building with OpenAI’s new open-weight model, gpt-oss-120b, now available on Heroku Managed Inference and Agents. This gives developers a powerful, transparent, and flexible way to build and deploy AI applications on the platform they already trust. Access gpt-oss-120b with our OpenAI-compatible chat completions API, which you can drop into any OpenAI-compatible SDK or framework. […] The post Heroku AI…
19 Aug 2025
Building AI applications that can interact with private data is a common goal for many organizations. The challenge often lies in connecting large language models (LLMs) with proprietary datasets. A combination of Heroku Managed Inference and Agents and LlamaIndex provides an elegant stack for this purpose. This post explores how to use these tools to […] The post Building Data-Aware…
8 Aug 2025
Building production-grade AI applications can be complex, but with Heroku and Pydantic AI, developers gain a powerful and reliable solution for integrating advanced AI capabilities. Heroku makes it easy to integrate AI into your applications with Heroku Managed Inference and Agents. With a single click, you can attach powerful Large Language Models like Anthropic’s Claude […] The post Building Agents…
26 Jun 2025
Today, we are pleased to announce the evolution of Heroku to an AI Platform as a Service (AI PaaS). GenAI has changed how we build software (vibe coding), the kind of technology we use (Cursor, LLMs), the type of software we make (agents), and redefined what it means to be a developer in this new […] The post Introducing the…
24 Jun 2025
Today, we’re thrilled to announce a new way in which agents can access the Heroku platform using the Heroku Remote MCP Server, now available at https://mcp.heroku.com/mcp. This new remote server is an expansion of our earlier stdio-based MCP server and comes with secure OAuth authentication. It’s designed to provide a secure, scalable, and incredibly simple […] The post Heroku AI:…
29 May 2025
Anthropic’s Claude 4 Sonnet, part of the next generation of Claude models, is now available on Heroku Managed Inference and Agents. This gives developers immediate access to a model designed for coding, advanced reasoning, and the support of capable AI agents. Heroku Managed Inference and Agents expands your AI choices, offering the freedom to build […] The post Heroku AI:…
15 May 2025
Many of the most exciting experiences we’re beginning to rely on every day are powered by AI; whether it’s conversational assistants, personalized recommendations or code generation, these experiences are powered by inference systems and intelligent agents. Behind the scenes, developers offload complex decisions, automate tasks, and compose intelligent applications using large language models and tool […] The post Heroku AI:…
5 May 2025
Generative AI has been one incredible tool to improve my productivity not only for work but for personal projects too. I use it every day, from generating stories and images for my online role playing games to solving code and engineering problems and building awesome demos. Lately I’ve leaned into Cursor as my go‑to AI […] The post How I…
10 Apr 2025
We’re excited to announce the launch of the Heroku MCP Server, designed to bridge the gap between agent-driven development and Heroku’s AI PaaS. Having defined the platform experience for apps in the cloud, Heroku extends our developer and operator experience to AI capabilities. With the Heroku MCP Server, you can now expose Heroku’s robust platform […] The post Introducing the…
2 Dec 2024
Over the last couple of years, we’ve repeatedly heard the question “who will build the Heroku of AI?”. The answer to that question is that Heroku will, of course. We are excited to bring AI to the Heroku platform with the pilot of Managed Inference and Agents, delivered with the graceful developer and operational experience […] The post Heroku AI…
17 May 2024
Today, we’re announcing the integration of the Heroku CLI with Amazon Q Developer. This integration, a result of our expanded Salesforce/AWS partnership, enables Amazon Q Developer command line suggestions of Heroku commands. This integration empowers Heroku users to auto-complete commands, thereby saving time and eliminating error-prone manual configurations of apps. Developers configure and manage their […] The post Heroku Integration…
28 Mar 2024
How to connect your GPT on OpenAI to a backend Node.js app Late in 2023, OpenAI introduced GPTs, a way for developers to build customized versions of ChatGPT that can bundle in specialized knowledge, follow preset instructions, or perform actions like reaching out to external APIs. As more and more businesses and individuals use ChatGPT, […] The post Building a…