Why public-data APIs need a unified response envelope

Why public-data APIs need a unified response envelope
The UnifAPI team
The UnifAPI team

Each upstream platform ships a different response shape. We standardized 14 platforms behind one envelope — here's the schema and why it matters for AI agents.

Every public-data upstream returns a different response shape. TikTok wraps results in { aweme_list, has_more, max_cursor }. Reddit uses { data: { children: [...], after } }. YouTube returns { items, nextPageToken }. LinkedIn paginates by skip-and-limit. If your agent talks to all of them, it inherits all of that variation — pagination logic forks 14 ways, retry policies fork, error parsing forks. We solved this by ramming every upstream through one envelope. This is the schema and why it matters.

The problem

Upstream variation is the silent tax on multi-platform agents. The agent's tool-use loop has to handle 14 different shapes, and that complexity shows up in three places that are expensive to maintain: the prompt (the model has to be taught each shape), the retry policy (different upstreams page differently after failure), and the error envelope (a Reddit 401 looks nothing like a TikTok 401).

We watched our own demo agents spend most of their tokens on shape-juggling instead of reasoning, and decided the right fix was at the gateway, not the agent.

The UnifAPI envelope

Every endpoint we expose, across every platform, returns the same outer shape: { data: T[], pagination: { cursor: string | null, has_more: boolean }, meta: { records: number, latency_ms: number, platform: string } }. The fields inside T are platform-specific — a TikTok video has hashtags, a Reddit post has subreddit — but everything around it is identical.

Errors collapse into a four-category set: rate_limited, not_found, upstream_unavailable, bad_request. Each carries a human-readable message and a machine-readable code. The model can branch on the code without having to recognize platform-specific text.

Reshape patterns we use

Standardizing 14 upstreams behind one envelope means our reshape layer is a few hundred lines per platform. Three patterns dominate.

Cursor unification — every upstream's pagination token (max_cursor, after, nextPageToken, end_cursor, page) becomes pagination.cursor. We compute has_more from whatever signal the upstream provides (length === limit, an explicit has_more flag, a null next-cursor).

Field flattening — when an upstream nests data three levels deep (data.data.children.data, looking at you Reddit), we lift it to top-level data[]. The model never sees the intermediate envelopes; it sees the records.

Defensive parsing — upstream platforms change field types without notice (a count becomes a string, an array becomes a single item, a timestamp becomes null). The reshape layer coerces each field to its declared type and drops the record only if the required fields are missing. That single discipline catches roughly 80% of the silent failures we saw in raw passthrough.

What breaks if you skip this

Without an envelope, two things go wrong at scale. First, your agent prompt grows linearly with platform count — every new platform adds a paragraph explaining its response shape. By platform 10, the prompt is the slowest part of the call.

Second, your retry and pagination code grows multiplicatively. Each upstream-specific quirk interacts with each other quirk, and the matrix of "does retry-after-cursor work on platform X under condition Y" blows up. Teams that don't normalize early end up writing a normalization layer late, after the bugs.

What's next

The current envelope covers list-style endpoints cleanly. The next iteration we're working on covers streaming and webhook-shaped upstreams — places where the response isn't a one-shot { data: [...] } but a series of events over time. We're shaping it to feel like an async iterator from the agent's side so the prompt change is minimal.

If you're building a multi-platform agent today and want to see how the envelope plays out in practice, the OpenAPI spec at unifapi.com/openapi.json is the source of truth — every endpoint, every field, one shape.