Reconcile an AI Vendor Bill Against Your Usage Meter

This is a hands-on kata, not a think-piece. The goal: take the invoice your model vendor just sent - Anthropic, OpenAI, whoever - and reconcile it against your own meter line by line, so you know exactly where the gap between what you paid and what you charged customers comes from. By the end you will have your metered total per model, a self-check that proves your own numbers before you blame anyone, a per-model gap against the vendor invoice, a localized cause, and a correcting record that closes the loop without rewriting history. All over the real metering API, no spreadsheet archaeology.

If you have done Kata #1 (meter an AI usage event to an invoice line), you already have records flowing. This kata is the month-end question that follows: the vendor bill says one thing, your invoices to customers say another, and someone has to explain the delta before margin quietly erodes.

Step 0: the model split has to exist before you need it

A UsageBox record carries exactly five fields - subscription, product_item, meter, value, timestamp. There is no free-form dimension bag you can slice open afterwards. The meter is the dimension. So if you want to reconcile per model, each model needs its own meter (claude-opus-tokens, claude-sonnet-tokens, claude-haiku-tokens) from the day you start ingesting. A split you did not create at ingest is a split you cannot make at month end - that is the one design decision this whole kata rests on.

Step 1: pull what you recorded, per model

Start with your own truth. Ask the meter for last month's rollups; you get one aggregated row per meter:

curl "https://api.usagebox.com/api/v1/usage/rollups?month=2026-05" \
  -H "x-api-key: $USAGEBOX_KEY"

{
  "month": "2026-05",
  "rollups": [
    { "month": "2026-05", "meter_key": "claude-opus-tokens",   "aggregation": "sum", "value": 42180000 },
    { "month": "2026-05", "meter_key": "claude-sonnet-tokens", "aggregation": "sum", "value": 191400000 },
    { "month": "2026-05", "meter_key": "claude-haiku-tokens",  "aggregation": "sum", "value": 88060000 }
  ]
}

That is your billable token quantity per model for May, already aggregated - no scan over raw records, cheap even over a full month. Keep these three numbers; they are the left-hand column of the reconciliation.

Step 2: check your own numbers first (the step everyone skips)

Before you accuse the vendor of overbilling, ask whether your own collector actually recorded everything it thought it did. The failure mode is specific and quiet: POST /api/v1/usage returns 200 with processed: 0 when a batch is well-formed but carries an ingest key that does not resolve on your account. Nothing is metered, nothing errors, and a month later the vendor bill is bigger than your meter for reasons that have nothing to do with the vendor.

So the self-check is on your side of the wire: grep your collector's logs for any response where processed was less than records, and read the skipped_details it returned. Then spot-check the store itself - the most recent raw records read back newest first:

curl "https://api.usagebox.com/api/v1/usage?limit=200" \
  -H "x-api-key: $USAGEBOX_KEY"

Each row carries the identifiers you sent it under, so you can confirm the meter keys landed the way you meant. Be honest about what this proves: limit caps at 200, so this is a spot check, not a full-month recount. The rollups are computed inline from the same records as they arrive, so the interesting question is never "did the rollup drift" - it is "did the record arrive at all".

Step 3: lay your total next to the vendor invoice

Now put the two columns side by side. Your meter says 321.64M tokens for May. The vendor invoice (converted from dollars back to token quantity at the rate card you are on, per model) says something a little higher. Compute the gap per model:

model                  metered (you)   vendor invoice   gap
claude-opus-tokens        42,180,000      42,180,000        0
claude-sonnet-tokens     191,400,000     197,920,000   +6,520,000
claude-haiku-tokens       88,060,000      88,060,000        0
TOTAL                    321,640,000     328,160,000   +6,520,000

The gap is not spread evenly - it is concentrated entirely in Sonnet, about 3.4% over what you metered. A flat percentage across every model would suggest a rate-card or currency mismatch; a single-model spike says something specific happened on Sonnet traffic. That localization is the whole game.

Step 4: localize it

Scope the same read down until the gap has a name. By meter, then by customer:

curl "https://api.usagebox.com/api/v1/usage/rollups?month=2026-05&meter=claude-sonnet-tokens" \
  -H "x-api-key: $USAGEBOX_KEY"

curl "https://api.usagebox.com/api/v1/usage/rollups?month=2026-05&meter=claude-sonnet-tokens&subscription=acme-prod" \
  -H "x-api-key: $USAGEBOX_KEY"

Loop your subscriptions and you have the Sonnet total per customer, which tells you whether the gap is one account or all of them. What you will not find is a breakdown of the 6.52M you never recorded - because it was never sent. That absence is itself the finding. The usual suspects are cached reads, retries that hit the vendor but never produced a customer-facing result, and system-prompt tokens you treated as fixed overhead. If you want those visible next month, they each need a meter.

Step 5: decide which kind of gap this is

Every reconciliation gap is one of two things, and the fix is completely different for each:

• Metering gap. You actually served the usage and should have charged for it, but your collector failed to record it. The vendor billed you, the customer used it, and you ate the cost. Fix: instrument the missing path so future records get sent, and decide whether to bill the customer retroactively.
• Pass-through gap. The vendor charged you for overhead you legitimately should not pass to the customer - speculative retries, internal evals, your own system prompt that every request shares. This is real cost-of-goods, not under-billing. Fix: price it into your margin, do not invoice it as customer usage.

In our case, suppose 6.0M of the 6.52M was prompt caching on a shared system prompt (overhead you chose not to bill) and the remaining 0.52M was a retried agent loop on one account that genuinely served the customer but never emitted a record. So: 6.0M is a pass-through gap to absorb, 0.52M is a metering gap to correct. You now know the number and the cause for both, and you can defend either one in a review.

Step 6: close the loop with a correcting record

For the 0.52M you under-metered, you do not edit history - records already written are not mutable through the API. You send the adjustment as its own record, timestamped into the period it belongs to. On a sum meter a negative value nets against the total, and a positive one adds to it:

curl -X POST https://api.usagebox.com/api/v1/usage \
  -H "x-api-key: $USAGEBOX_KEY" \
  -H "Idempotency-Key: recon-2026-05-anthropic-acme-sonnet" \
  -H "Content-Type: application/json" \
  -d '[{
    "subscription": "acme-prod",
    "product_item": "assistant-seats",
    "meter": "claude-sonnet-tokens",
    "value": 520000,
    "timestamp": "2026-05-31T23:59:59Z"
  }]'

The Idempotency-Key matters more here than anywhere else: a reconciliation is the one batch most likely to be run twice by a nervous human, and the key is what stops the second run from double-correcting. If you would rather keep adjustments visibly separate from organic traffic, give them their own meter (claude-sonnet-tokens-adjustments) and read both rows at invoice time - the rollup returns one row per meter, so the split costs you nothing and the correction never hides inside the original figure.

Production notes before you ship it

• Check processed, not the status code. A 200 with processed: 0 is the most common cause of a gap that looks like the vendor's fault and is not.
• Meter the same way the vendor bills. If your vendor prices cached reads, retries and system-prompt tokens separately, give each one a meter at ingest, or reconciliation stays a forensic exercise.
• Corrections are records, not edits. Timestamp them into the period they belong to and always send an Idempotency-Key.
• Reconcile monthly, in the month. Rollups bucket by UTC calendar month, so a correction timestamped 2026-05-31T23:59:59Z lands in May and one timestamped an hour later does not.

Kata variations to try

• Per-customer attribution. Loop your subscriptions through the Step 4 read to find which customer's traffic drove the metering gap, then bill or absorb per account.
• Adjustment meters. Mirror every billable meter with an -adjustments twin so corrections are always visible as their own line.
• Month-over-month drift. Run Step 1 for month=2026-04 and month=2026-05 and compare the ratio of your metered total to the vendor invoice; a moving ratio is a collector that is degrading, not a one-off.
• Overhead meter. Start sending the system-prompt and retry tokens to a non-billable meter, so next month the pass-through gap is a number you can read instead of a number you infer.

Kata FAQ

The vendor invoice is in dollars and my meter is in tokens - how do I compare? Convert the invoice back to quantity per model using your rate card, then compare quantity to quantity. Reconciling on tokens (the thing both sides actually counted) isolates volume gaps from pricing gaps, which is what you want.

Why not just trust whichever total is bigger? Because the two gaps need opposite responses. A metering gap means you under-billed a customer; a pass-through gap means you should absorb vendor overhead into margin. Treating them the same either eats your profit or overcharges a customer.

Can I split one meter by model after the fact? No. Records carry subscription, product_item, meter, value and timestamp - there is no dimension bag to group on later. Create the meter you will want to read.

Do corrections change the records I already sent? No. The originals are untouched; a correction is an additional record that nets into the same monthly rollup. Put it on its own meter if you want both figures visible side by side.

What you just avoided building

In six steps you reconciled an external vendor bill against your own meter: a per-model metered total, a self-check that finds the silent ingest failure before you blame anyone, a per-model gap, a drill-down that named the cause, a clean split between metering gaps and pass-through overhead, and a correcting record that lands in the right period exactly once. Built in-house, that is an aggregation path to keep consistent with your raw records, idempotent ingest that survives a nervous re-run, and monthly buckets that do not shift under you - a real metering database, not a reconciliation spreadsheet. That is why the gap between AI list price and real cost is so easy to lose track of, and why metering became strategic enough to drive the 2026 acquisition wave.

Keep reading: Kata #1 (meter a usage event to an invoice line), Kata #2 (live spend caps on real-time usage), and Kata #4 (per-customer, per-model cost with dimensions) - plus the usage-based billing guide for the bigger picture.