Prompt caching

The SDK caches resolved prompts in memory for fifteen seconds by default.

const prompt = await anpord.prompts.get({ id: "support-reply" });

console.log(
  prompt.anpord.freshness,
  prompt.anpord.ageMs,
  prompt.anpord.reason
);

Freshness	Meaning
`fresh`	The API returned the value on this call
`cached`	Memory returned the value, including during background refresh
`stale`	An awaited refresh failed, so the SDK returned the last good value
`fallback`	No cached value existed, so the SDK returned caller-supplied content

Configure the cache

const anpord = new Anpord({
  cache: { ttlMs: 60_000 },
});

Option	Default	Purpose
`ttlMs`	`15_000`	Age that starts a background refresh
`maxStaleMs`	`86_400_000`	Age that makes the next caller wait for refresh
`capacity`	`1000`	Maximum cached selectors
`maxConcurrentRefresh`	`8`	Maximum refreshes in flight

Set cache: false to disable caching. Pinned versions have no TTL, but capacity eviction and same-client writes can remove them. Channel and default selectors use stale-while-revalidate. After ttlMs, the next call returns the cached value while a refresh runs. After maxStaleMs, the next call waits for refresh. If that request fails, the SDK may still return the cached value and mark it stale. Writes through the same client clear its cached selectors for that prompt.

Add a fallback

const prompt = await anpord.prompts.get({
  id: "support-reply",
  fallback: "You are a support agent. Answer briefly.",
});

Fallbacks apply only when the API is unavailable and no cached value exists. They may be a content string or an object with content and config. Missing prompts, invalid keys, forbidden requests, and malformed selectors still throw. Keep a short fallback for prompts on a request path. Pin a version and fail the job for reproducible evals.

Getting started

Evals

Guides

Prompts

Tools

Help

Prompt caching

Configure the cache

Add a fallback

​Configure the cache

​Add a fallback

Configure the cache

Add a fallback