Designing an API for LLMs, Not Humans

Most APIs are designed for humans clicking through docs and reading responses in dev tools. Agents interact with APIs differently, and a lot of the REST conventions we’ve relied on for years quietly work against them. We started noticing this when we pointed Claude at our own API and watched what happened.

We asked it to write a data-driven report about US healthcare costs using our Research framework (currently in Alpha). It dispatched 4 parallel research agents, made 72 API calls, and spent ~4 minutes on the research phase before a single word of the report was written.

The report turned out great: eight charts, five narrative sections covering spending trends, life expectancy, drug prices, insurance coverage, and medical inflation all correlated and backed by the real data. The kind of thing that would take a human researcher multiple days to assemble, but 72 API calls to fetch data from 13 datasets across 7 providers felt like a lot. Most of those calls were the agents inspecting schemas, retrying queries that failed on quoting, and re-discovering datasets that another agent had already found.

The API worked, it just wasn’t designed well for the thing using it.

The Feedback Loop

We didn’t guess what agents need. We ran the task, then ran a post-mortem asking the agent what went wrong. The feedback is structured and specific. Not “the API is slow” but:

SQL string quoting causes ~10-15% of queries to fail on first try because of triple-nested escaping in shell/JSON/SQL.
Suggested fix: parameterized queries with ? placeholders and a params array.

We ran three rounds of this in one afternoon: ship the fix, re-run the exact same research task, measure the difference (API calls, tokens consumed, payload sizes), and see what the agent complains about next.

The 72-Call Baseline

The post-mortem identified six problems. Each one matters specifically because the API consumer is an LLM, not a human.

1. Discovery redundancy

Four parallel agents each called /v1/discover independently with overlapping queries, and three datasets showed up in multiple agents’ results. There was no way to batch or deduplicate, so the API needed to handle that server-side.

2. Schema inspection round-trips

For every dataset, the agent called /columns to check types and units, then /views to check for curated views. Two extra calls per dataset before writing a single query. Across 13 datasets: ~21 calls just to understand schemas.

Humans skim a docs page and infer what columns mean. Agents need explicit metadata or they probe the data to figure it out, and probing means more API calls.

3. The string escaping problem

Agents construct SQL inside JSON inside shell commands. A country name like “United States” requires escaping at three levels. ~10-15% of SQL queries failed on the first try due to quoting errors, requiring retries.

Humans use SQL clients with proper escaping. Agents are constructing curl commands with string interpolation, and the '"United States"' pattern breaks more often than it works.

# What the agent actually tried to construct:
curl -s -X POST "https://api.tryopendata.ai/v1/datasets/owid/healthcare-spending/query" \
  -H "Content-Type: application/json" \
  -d '{"sql": "SELECT country, year, spending_per_capita FROM data WHERE country IN ('"'"'United States'"'"', '"'"'Japan'"'"', '"'"'Germany'"'"') AND year >= 2020 ORDER BY year"}'

# First attempt often failed. Agent retried with different escaping.
# Sometimes took 2-3 attempts to get the quoting right.

4. Cross-dataset join anxiety

Our SQL endpoint supports cross-dataset joins, but agents couldn’t tell whether country_code in one dataset matched country_code in another without querying both first. So instead of writing a single join query, they’d fetch two full data payloads separately and assemble the results themselves, sometimes by writing a custom Python join script. That’s two large responses worth of tokens plus the reasoning to combine them, when a single server-side query would have returned exactly the joined rows they needed.

5. Discover-then-query round-trips

Even for simple questions (“what’s US healthcare spending?”), agents had to discover the dataset, inspect the schema, then query. There was no way to get column metadata from the discover response itself.

6. No category browsing

Agents couldn’t narrow searches by domain without knowing the taxonomy. Every search was keyword-based.

Here’s what a single dataset workflow looked like in Round 1:

1. GET  /v1/discover?q=healthcare+spending              → find datasets
2. GET  /v1/datasets/owid/healthcare-spending/columns    → inspect schema
3. GET  /v1/datasets/owid/healthcare-spending/views      → check for views
4. POST /v1/datasets/owid/healthcare-spending/query      → probe: SELECT DISTINCT country LIMIT 20
5. POST /v1/datasets/owid/healthcare-spending/query      → probe: SELECT MIN(year), MAX(year) FROM data
6. POST /v1/datasets/owid/healthcare-spending/query      → real query (fails on quoting)
7. POST /v1/datasets/owid/healthcare-spending/query      → retry with fixed quoting
= 7 calls for one dataset. Some discover calls were shared, but across 13 datasets
  this pattern added up to ~72 total calls.

What We Shipped

Three changes, all driven directly by the agent’s feedback.

Batch discover

POST /v1/discover/batch takes multiple queries in one call and deduplicates server-side. The response separates query_results (per-query relevance scores) from datasets (the unique set of matching datasets with full metadata).

curl -s -X POST "https://api.tryopendata.ai/v1/discover/batch" \
  -H "Authorization: Bearer $OPENDATA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": [
      "healthcare spending per capita by country",
      "life expectancy and health outcomes by country",
      "drug prices and pharmaceutical spending",
      "health insurance coverage and medical inflation US"
    ],
    "limit_per_query": 5,
    "deduplicate": true
  }'
# 1 call. 17 unique datasets. 3 overlapping datasets deduplicated.

Four discover calls became one. The three datasets that appeared across multiple agents’ queries were correctly deduplicated in the response.

Self-describing responses

21 calls just to understand schemas. The fix wasn’t making those inspection calls faster, it was making them unnecessary. We made the discover response self-describing: everything an agent needs to write a query is in the response that told it the dataset exists.

{
  "columns": [
    {
      "name": "spending_per_capita",
      "type": "float",
      "display_name": "Spending per Capita (USD PPP)",
      "value_range": { "min": 10.53, "max": 12009.9 },
      "sample_values": ["85.86", "118.41", "146.84", "261.57", "461.18"],
      "distinct_count": 996
    },
    {
      "name": "country_code",
      "type": "string",
      "display_name": "Country Code",
      "sample_values": ["AFG", "ALB", "DZA", "AGO", "ARG"],
      "distinct_count": 206
    }
  ],
  "available_views": [
    { "name": "latest", "description": "Most recent data by country" },
    { "name": "us-vs-peers", "description": "US compared to peer nations" }
  ]
}

Value ranges, sample values, display names, available views, all inline. The agent reads one response and has enough context to write a targeted SQL query immediately. That’s the core idea of an LLM-first API: the response itself is the documentation.

Parameterized queries

Agents shouldn’t have to embed values in query strings at all. The SQL goes in one field with ? placeholders, and the values go in a separate params array. The API binds them server-side:

curl -s -X POST "https://api.tryopendata.ai/v1/datasets/owid/healthcare-spending/query" \
  -H "Content-Type: application/json" \
  -d '{
    "sql": "SELECT country, year, spending_per_capita FROM data WHERE country IN (?, ?, ?) AND year >= ? ORDER BY year",
    "params": ["United States", "Japan", "Germany", 2020]
  }'

No escaping. The string "United States" stays a plain JSON string, never touches SQL quoting. If the agent passes the wrong number of parameters, it gets a structured error (PARAM_COUNT_MISMATCH) with a hint instead of a cryptic SQL parse failure. This works on both single-dataset and cross-dataset SQL endpoints.

The results

We estimated a 30-40% improvement. Actual was 89%. The savings compound because entire categories of round-trips disappeared rather than each one getting a little faster. The remaining polish (dedicated unit fields so agents stop guessing whether 0.125 is 12.5% or a raw proportion, semantic column labels for confident cross-dataset joins, and a ?category=health filter to narrow discovery) shaved off one more call.

Here’s the final workflow:

1. POST /v1/discover/batch                                  → all 4 queries, deduplicated
2. POST /v1/datasets/owid/healthcare-spending/query          → parameterized SQL
3. POST /v1/datasets/owid/life-expectancy/query              → parameterized SQL
4. POST /v1/datasets/oecd/pharma-spending/query              → parameterized SQL
5. POST /v1/datasets/fred/cpi-medical/query                  → parameterized SQL
6. POST /v1/datasets/census/health-insurance-coverage/query  → parameterized SQL
7. POST /v1/datasets/cms/part-d-spending/query               → parameterized SQL
8. POST /v1/query                                            → cross-dataset join
= 8 calls total for 6 datasets + 1 cross-dataset join. All responses columnar.

This is the API that now powers the OpenData MCP server, which means any agent with MCP support gets these same optimizations out of the box.

The Columnar Format Decision

After the round-trip count was already optimized, we looked at what was left: the response payloads themselves. Is the format wasting tokens?

Standard JSON API responses return rows as arrays of objects:

{
  "data": [
    {
      "country": "United States",
      "country_code": "USA",
      "spending_per_capita": 12586.196,
      "year": 2022
    },
    {
      "country": "United States",
      "country_code": "USA",
      "spending_per_capita": 13473.193,
      "year": 2023
    },
    {
      "country": "Japan",
      "country_code": "JPN",
      "spending_per_capita": 5845.99,
      "year": 2022
    },
    {
      "country": "Japan",
      "country_code": "JPN",
      "spending_per_capita": 6102.44,
      "year": 2023
    },
    {
      "country": "Germany",
      "country_code": "DEU",
      "spending_per_capita": 8306.01,
      "year": 2022
    },
    {
      "country": "Germany",
      "country_code": "DEU",
      "spending_per_capita": 8733.18,
      "year": 2023
    }
  ]
}

Every key name repeated on every row. For 6 rows with 4 columns, that’s 24 redundant key strings. At 73 rows: ~292 redundant keys. This is the JSON convention. Every API does it. Nobody questions it because humans read one object and skip the rest.

But LLMs tokenize the entire response. Every repeated "country": costs tokens. Tokens cost money and consume context window.

Columnar format:

{
  "columns": ["country", "country_code", "spending_per_capita", "year"],
  "types": ["string", "string", "float", "integer"],
  "rows": [
    ["United States", "USA", 12586.196, 2022],
    ["United States", "USA", 13473.193, 2023],
    ["Japan", "JPN", 5845.99, 2022],
    ["Japan", "JPN", 6102.44, 2023],
    ["Germany", "DEU", 8306.01, 2022],
    ["Germany", "DEU", 8733.18, 2023]
  ]
}

Same data, ~45% fewer tokens. The 73-row healthcare spending response went from ~2,200 tokens to ~1,200 tokens. Across a full research session with 6 queries, that’s 3,000-5,000 tokens saved on response payloads alone.

We made columnar the default and kept ?response_format=objects as an opt-in for anyone who needs it.

What We’d Tell Other API Teams

None of this is speculative: every row in this table came from watching agents fail and fixing the specific thing that tripped them up.

Most of these fixes are straightforward. The hard part is knowing which ones matter, and the fastest way to find out is to ask the thing using your API.