No comments yet.
Wanted to show you something we've been working on: Durable Endpoints. It brings durability and steps directly into your APIs. Each step persists the state in the background. The happy path means no additional latency for your users, and you get tracing & metrics out of the box, and failures resume from the last successful step checkpoint rather than restarting from scratch.
The problem we kept hitting: durable execution systems (Temporal, our own workflows, etc.) are designed for async background work. But with AI workloads, you often need durability and low latency in a synchronous request: a user is waiting on a chat response that involves multiple LLM calls, any of which can fail or timeout.
The typical solution is splitting into two systems: a streaming frontend for responsiveness, plus a background job system for reliability. Durable Endpoints let you keep it as one cohesive implementation (here's a Next example):
import { Inngest } from "inngest";
import { endpointAdapter } from "inngest/next";
const inngest = new Inngest({ id: "my-app", endpointAdapter });
export const handler = inngest.endpoint(async (req: Request) => {
const { input } = parseParams(req);
// Each step.run() checkpoints on completion
const processed = await step.run("process-input", async () => {
return await expensiveOperation(input);
});
// If this fails, we resume here — not from the beginning
const result = await step.run("generate-output", async () => {
return await anotherExpensiveOperation(processed);
});
return Response.json({ result });
});
- Steps are identified by their string ID, not execution order. This makes the system tolerant to code changes... you can refactor, add/remove steps, and existing in-flight executions continue correctly.
- We use checkpointing (https://www.inngest.com/blog/introducing-checkpointing) to async checkpoint steps. State is persisted to our execution store rather than held in memory.
- If endpoints turn async (step errors or waitForEvent), we redirect the user to a new URL which uses keepalives and waits for the API result. Inngest re-enters the endpoint and picks up where it left off, then sends the result to the redirected user.
- Similarly, on failure, our executor re-invokes the endpoint. Completed steps are memoized from the checkpoint... their functions don't re-run, we just return the stored result.
- Works with streaming responses via ReadableStream.
The same step primitives we use in durable workflows (`step.run`, `step.sleep`, `step.waitForEvent`) work here, so you get retries, timeouts, and human-in-the-loop patterns in synchronous handlers.
Beyond AI, this is useful anywhere failure mid-request is expensive: payment flows, webhook processing, third-party API orchestration.
It's in public beta. Docs: https://www.inngest.com/docs/learn/durable-endpoints. Code is in our TypeScript SDK.
Happy to dig into the execution model, how we handle determinism, or the checkpointing implementation.