Cache for efficient API calls

A connector that calls the same upstream endpoint five times in a row for the same id is a connector that should cache. Caching does not change topology behaviour — the message flow, branching, and outputs stay identical — it only stops you from paying for work you have already done. Less upstream load, less latency, less rate-limit pressure, and often a meaningfully smaller bill at the end of the month.

The trade-off is staleness. A cache always returns a slightly older answer; the question is only how old is acceptable for a given lookup. The patterns below walk through three tiers of cache, each with a different staleness budget and a different operational cost.

When to cache #

Cache when all of the following are true:

• The upstream response is the same for the same input within a known time window. A customer's billing address won't change in the next 60 seconds; a real-time stock level might.
• The cost of the upstream call is non-trivial — measured in latency, rate-limit budget, or money.
• A stale answer is acceptable for the configured TTL. If freshness is non-negotiable, don't cache; fix the upstream pattern instead.

Don't cache:

• Mutations. Cache reads, never writes.
• Per-request authentication tokens. The SDK already manages those; layering your own cache on top is a great way to ship credentials to the wrong tenant.
• Anything where freshness is the entire point — live stock, payment status, OTP codes, fraud-check decisions. These need a fresh round-trip every time, and a cache is actively dangerous.

Three tiers, three time scales #

Tier 1: in-process memoization #

Inside a single processAction() call, just keep a Map. Useful when a custom node iterates over many records that share a foreign key — orders that point at the same owner, line items that point at the same product — and would otherwise call the same upstream lookup many times.

public async processAction(dto: ProcessDto): Promise<ProcessDto> {
    const items = dto.getJsonData().items as Array<{ ownerId: string }>;
    const ownerCache = new Map<string, Owner>();

    for (const item of items) {
        if (!ownerCache.has(item.ownerId)) {
            ownerCache.set(item.ownerId, await this.fetchOwner(item.ownerId));
        }
        // use ownerCache.get(item.ownerId) ...
    }
    return dto;
}

Lifetime: one node call. No infrastructure, no risk, no staleness window — the cache is gone the moment the function returns. This is the safest tier and the one to reach for first.

Tier 2: short-TTL shared cache #

For lookups that recur across messages and processes, use a shared cache (Redis, Memcached) with a short TTL — minutes, sometimes seconds. Wrap the upstream call in a "get from cache, fall back to upstream, write to cache" helper so the rest of the connector code stays unaware of the cache.

The right TTL is the longest staleness your business can tolerate. For "what's the customer's email" inside a sync that runs every 5 minutes, a 5-minute TTL is usually fine. For "what's the current shipping rate", probably not.

Tier 3: ID resolution cache #

A special case of tier 2. When you constantly translate "the upstream identifier for our internal customer X", that mapping rarely changes and is exactly the ID mapping pattern. Treat it as a permanent cache that you maintain explicitly through resolve / persist nodes — not as a TTL'd one. TTLs make sense for data; identifiers should be authoritative.

Invalidate on write #

The moment you mutate something upstream, drop the matching cache key in the same node. Otherwise the next read returns the stale answer for the rest of the TTL — and you will spend a debugging afternoon explaining why a customer's updated address keeps reverting.

await this.updateContact(id, payload);
await this.cache.delete(`contact:${id}`);

Invalidate after the upstream write succeeds, not before. If you delete the key first and the upstream call fails, the next read will repopulate the cache with the unchanged value and hide the failure.

Operational notes #

• User in the key. Always include the user identifier (or Application install id) in the cache key. Otherwise a tier-2 cache shared across tenants will happily serve one customer's data to another. This is the single most common cache bug in multi-tenant integrations.
• Auth tokens in the key, never in the value. Don't accidentally store credentials in cache values; key them out and let the SDK refresh them on its own schedule.
• Observability. Log a hit / miss counter from every cache lookup. The hit rate is the single number that tells you whether the cache is paying for itself. A 5% hit rate is a cache that is costing more than it saves; a 95% hit rate is a cache earning its keep.
• Cold starts. A freshly deployed worker has an empty cache and the first wave of requests will all miss. Plan deploys in low-traffic windows or warm the cache explicitly for known hot keys.

Related #

• Patterns: API caching (docs) — the reference page that backs this guide.
• ID mapping guide — the explicit, non-TTL'd cache for identifiers.
• Handling large datasets with Orchesty — fan-out patterns where tier 1 memoization is most effective.