Backtesting Arena

Backtesting Arena

Back to blog

Agent Payments: Why No Standard Checks Whether the Work Was Any Good

Agent payments are standardised: authorisation, identity, mandate, settlement. Whether the work was any good, nobody checks. Why that is a boundary and not a gap.

Backtesting Arena·August 15, 2026·13 min read·1 views
Agent Payments: Why No Standard Checks Whether the Work Was Any Good

Since July 2026 there has been a standards body for agent payments, with Visa, Mastercard, American Express, Stripe, Google and AWS on board. It governs how one AI pays another. What it does not govern is whether the thing paid for was any good.

That sounds like a gap someone will close shortly. It is not a gap. It is a boundary, and the difference decides which products in this field will work and which will not.

Four Problems Are Solved, and Quality Is Not One of Them

Today's agent payment standards address four things: authorisation (may this agent pay at all), identity (who is it), mandate (for what exactly, up to what amount) and settlement (how the money actually arrives).

All four are serious problems and all four are being solved seriously. Google's AP2, the Agent Payments Protocol, shipped in September 2025 and covers authorisation and mandates. The Agentic Commerce Protocol from OpenAI and Stripe landed the same month. x402 supplies settlement. Visa runs Intelligent Commerce and the Trusted Agent Protocol; Mastercard is folding its work into Agent Pay for Machines.

None of them answers whether the delivered work was acceptable. That question requires something that judges: an oracle. And an oracle for service quality is categorically different from an oracle for an exchange rate. One reads a number. The other forms an opinion.

At a Tenth of a Cent, Quality Regulates Itself

When an API call costs a tenth of a cent, no verification is needed. If the first call is bad there is no second one. Maximum loss is one unit, reversal would cost more than the damage, and repetition supplies the signal that verification was supposed to supply.

This is why the demos in this field look so convincing. They show almost exclusively the case in which the problem does not exist.

It appears when three conditions coincide. The amount is large enough that a total loss hurts. The work is indivisible, or delivered as a package. And buyer and provider may only ever meet once, so repetition is unavailable as a disciplining mechanism.

That constellation is the interesting part of the agent market: research, data delivery, analysis, commissioned work. Not the tenth-of-a-cent call.

Two Routes, and the Choice Decides Who Owns the Disputes

The market has split, but the split is not really blockchain against card. It is about whether a reversal path exists.

Card-Anchored

The classic stack of issuer, acquirer and network stays. What is new is a layer above it: short-lived authorisations bound to a merchant and an amount. Stripe issues Shared Payment Tokens that specify exactly which merchant an agent may pay, for how much, and within what window; since March 2026 they also cover Mastercard Agent Pay, Visa Intelligent Commerce and buy-now-pay-later methods.

The decisive side effect: the grown chargeback and liability rules survive. You inherit a dispute resolution system instead of building one.

Chain-Settled

Here there is no operator in the classic sense, because settlement is the consensus of the chain itself.

x402 originated at Coinbase and was open-sourced in May 2025. The Linux Foundation announced its intent to take the protocol in April 2026 and completed the contribution on 14 July 2026, the day the x402 Foundation went operational with 40 members, 17 of them premier: Visa, Mastercard, American Express, Stripe, Adyen, Fiserv, Google, AWS, Cloudflare, Coinbase, Circle, Shopify, Ripple, the Solana Foundation, the Stellar Development Foundation, the Monad Foundation and MoonPay.

Technically x402 revives the HTTP status code 402 Payment Required, dormant since the earliest days of the web. An agent hits a paid API, receives a 402, settles the charge and retries, with no account, no subscription and no card entry. Google's documentation describes x402 as the stablecoin settlement extension to AP2.

After finality the money is gone. There is no chargeback, because no entity exists that could execute one.

The Split Is Untidy, and That Belongs in the Picture

Stripe and Tempo launched the Machine Payments Protocol on 18 March 2026, on the same HTTP 402 mechanism, deliberately rail-agnostic, settling on the Tempo chain but with card and fiat methods alongside stablecoins.

The clean two-camp picture is therefore a simplification. What is not a simplification is the reversal path. That is what to choose on, not the label.

32 Cents Per Transaction Is the Number Behind the Standard

In the 30 days before the Foundation launch, the x402 network recorded 75.41 million transactions carrying 24.24 million dollars, across roughly 94,060 buyers and 22,000 sellers. That works out to an average ticket of about 32 cents. CoinDesk reported on 16 July 2026 that analysts estimate a substantial share of that is not genuine demand but test traffic.

The average is the point. At 32 cents the verification problem does not exist. Which means the volume the standard is promoted with consists almost entirely of the case the standard already handles.

Who Answers for an Unauthorised Agent Payment Is Unsettled

If an autonomous agent triggers an unauthorised stablecoin payment, US law does not conclusively settle who answers for it. Recent stablecoin legislation addresses issuance, not dispute rights for machine-initiated payments. Europe differs, but is likewise not written with agents in mind.

Five Approaches, Ranked by How Well They Hold

1. Split the amount instead of securing it.

The strongest approach is the least spectacular: turn one large payment into a stream of small ones. Per call, per token, per milestone. That scales exactly the logic that already works at a tenth of a cent — abort after the first bad unit, and one unit is all you can lose.

The objection arrives reliably from the provider: the work is indivisible. That is true far less often than it is claimed. Research splits into interim states, data delivery into batches, analysis into partial results. Indivisibility is usually a preference for prepayment rather than a technical circumstance.

2. Escrow with a machine-checkable acceptance criterion.

This works where quality is deterministically testable: schema conformance, latency commitments, test suite green, hash matches, result reproducible. The criterion is fixed before payment, hashed, and signed by both sides. A checker runs, and its verdict releases the funds. If the deadline passes without a verdict the money returns automatically, otherwise the provider has an interest in letting the check hang.

For "is the analysis clever" none of this works, and none of it ever will.

3. A provider bond.

The provider posts security that is forfeited on a lost dispute. Economically this often dominates: instead of checking every job, check five percent and penalise hard. Deterrence is comparable and costs one twentieth. Verification is expensive; deterrence is cheap.

The bond also solves cold start, because it buys reputation the provider does not yet have. Xiong and co-authors recommend precisely this shape for the next revision of ERC-8004: feedback should carry a cost, and the cost should rise with its influence.

4. Choose the route deliberately.

For larger amounts the card-anchored route is not the slower option, it is the only one with a functioning reversal path. That is an uncomfortable thing to say in a field that defines itself through programmability, but a grown dispute system is harder to rebuild than a settlement layer.

5. Arbitration.

A model as a cheap pre-filter against the pre-agreed specification, then escalation to humans. A model inside the payment path is manipulable and does not always return the same answer. It is usable as a reputation signal, not as a trigger for moving money.

Goodhart: The Acceptance Criterion Becomes the Product Definition

The moment an acceptance criterion is fixed, the provider optimises exactly for it. Five thousand rows of garbage pass a row-count check.

The partial escape is a change of perspective: do not check whether the goal was reached, check whether it was demonstrably missed. No nulls in required fields, no duplicates, timestamps monotonic, distribution not degenerate. Such criteria are harder to game because they define an exclusion zone rather than a score to optimise toward.

This solves nothing completely. From the moment of fixing, the criteria become the actual product definition. They should be written as though someone will read them adversarially, because someone will.

Measured: ERC-8004 Reputation Is Mostly Sybil Noise

This is where theory and measurement diverge most sharply.

The obvious approach has existed as an open standard on Ethereum mainnet since 29 January 2026. ERC-8004 defines three registries. The Identity Registry assigns portable ERC-721-based identities. The Reputation Registry records publicly readable feedback. The Validation Registry stores independent attestations of agent performance. Together they are meant to let an agent evaluate an unknown counterpart before committing value.

The architecture is well considered. What is actually in the registries is not.

Xihan Xiong, Zelin Li, Wei Wei, Qin Wang, William Knottenbelt and Zhipeng Wang crawled the real registrations on Ethereum, BNB Smart Chain and Base through 13 May 2026 and published the result as "Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem" (arXiv:2606.26028, version v2 of 8 July 2026). Across the three chains:

  • Only 3%, 4% and 15% of registrations (Ethereum / BSC / Base) exposed a valid registration file with at least one live service endpoint. The remainder were placeholders, not active agents.
  • 73.5%, 59.2% and 90.6% of reviewers shared funding provenance, a pattern consistent with coordinated Sybil activity.
  • Removing that feedback leaves 15.8%, 77.9% and 86.8% of rated agents with no valid feedback at all.
  • Between 98.7% and 100% of all feedback records carried neither a payment proof nor a link to a concrete task.

The authors put it plainly: the Reputation Registry as currently deployed cannot function as a trust signal. Values are not commensurable, feedback records are rarely grounded in verifiable interactions, and reputation can be manipulated at minimal cost.

That "minimal cost" has a price tag. The registry aggregates by arithmetic mean, and an entry's value is capped only at 10^38. A single record therefore moves an agent's score to any target value, no matter how many honest ratings already exist. On every one of the three chains, the median agent crosses a trust threshold of 90 on one rating. On Base that single rating costs 0.0027 dollars. And the median agent there guards a payment volume 259 times larger than the attack that flips it.

The authors are careful with their own numbers, which belongs in the account. Where a payment could not be attributed unambiguously to an agent, they dropped the case rather than guessing, and reported the resulting 93.8% of Base reviewers with no payment history explicitly as a conservative lower bound rather than a point estimate.

One detail gets lost in the coverage: on the three chains studied, the Validation Registry had no confirmed mainnet deployment during the entire window. The component meant to carry evidence-backed attestations, the part that would actually address the problem, was nowhere in use.

The lesson is not that the standard is bad. It is that feedback not bound to a proven, paid transaction is free to produce, and agents produce free things very quickly.

That also makes the analogy to review platforms like Trustpilot misleading. What transfers is the idea of public history. What does not transfer: voluntary reviews, star averages, a central operator able to curate entries. What would carry is something else: signed verification results from actual transactions, in an append-only log the provider does not write and therefore cannot curate. Not rating, but an observation record. The study's own recommendations point the same way, requiring each feedback record to reference a settled x402 payment or an attested task.

Collusion and Cold Start Point the Same Direction

Two agents commissioning each other and waving each other through build history without work. Real transaction costs and graph analysis dampen this; they do not solve it.

Conversely, a new provider has no history and receives no jobs with which to build one. Both point the same direction: reputation alone does not carry. It needs something that costs money.

Subjective Quality Has No Deterministic Criterion

The hardest point comes last, because it cannot be engineered away.

For "was the analysis good", "was the text persuasive", "was the recommendation sound", no fixed criterion exists. Any procedure claiming to check this has merely moved the trust, usually to a place where it is no longer visible.

What remains is unglamorous: many rounds with the same counterparty, or a human in the approval path. Both scale badly, and both are the only answers that hold up today.

Who Is Claiming Which Role

As of August 2026, four camps are distinguishable.

Open standard: ERC-8004, with integration commitments from ENS, EigenLayer, The Graph, Taiko and the Ethereum Foundation's dAI team.

Identity and payment: Skyfire, contributing the open KYAPay protocol for packaging and exchanging agent-related information across platforms.

The credit bureaus: Experian announced Agent Trust on 30 April 2026, a Know Your Agent framework developed with Visa, Cloudflare and Skyfire as ecosystem collaborators. At its core is Human-to-Agent Binding, a persistent link between a verified consumer, their device and the agent acting for them, issuing a real-time Agent Trust Token. An accompanying Agent Registry tracks behavioural signals over time and converts them into dynamic trust scores. What is notable is not the technology but the starting position: scoring is already the business model there and only needs porting.

Network and merchant layer: Visa's Trusted Agent Protocol for merchant-side verification, and Cloudflare enforcing the trust layer at the network edge. Alongside these run further consortia: the FIDO Alliance announced an Agentic Authentication Working Group in April 2026, and Mastercard is developing a Verifiable Intent framework with Google.

The overlaps are considerable, interoperability is unresolved, and the major card networks are members almost everywhere at once. That hedge says one thing clearly: nobody knows which standard wins.

What This Means for Products

Promising: metering and billing — boring, paid immediately, no chicken-and-egg problem. Verification in narrow, genuinely testable domains, sold as quality control rather than as a trust layer. And compliance, as soon as legislators define dispute rights for machine payments; that is the strongest conceivable driver because it creates a mandatory market.

Structurally weak: the reputation registry as a standalone product. It carries every classic killer at once — chicken-and-egg, unclear willingness to pay (the buyer wants it, the provider would have to fund it), and Sybil exposure. And whoever operates settlement sees the transactions anyway and can hand trust out as a feature rather than sell it as a business model.

What Remains

None of these approaches solves the problem. They move it, from "was the work good" to "was it demonstrably bad", and accept that mediocre work gets paid.

That is not a failure of the design. It is its boundary. A protocol promising otherwise has relocated the trust rather than removed it.

Anyone building here should answer two questions early. Does the product work without a finished ecosystem? And is the acceptance criterion written so that it survives a hostile reading?

Two yeses and you are building something durable. Waiting on the ecosystem is a bet.

Frequently Asked Questions

What do agent payment standards actually govern? Four things: whether an agent may pay, who it is, for what and up to what amount, and how the money arrives. Whether the work paid for was any good is governed by none of them.

What is x402? An open payment protocol built on the HTTP status code 402 Payment Required. An agent calls a paid API, receives a 402, settles the charge and retries, with no account and no subscription. Coinbase built it; since 14 July 2026 it sits with the x402 Foundation under the Linux Foundation.

Why is a reputation registry not enough? Because feedback not bound to a paid, verifiable transaction costs nothing to produce. The study by Xiong and co-authors shows this on ERC-8004: on Base, 90.6% of reviewers shared funding provenance, and a single rating costing 0.0027 dollars is enough to lift the median agent across the trust threshold.

Can quality be checked by machine at all? Where it is deterministically testable, yes: schema, latency, hashes, reproducibility. For questions like "was the analysis good", no. No fixed criterion exists, and none will.


This is not investment advice, not a recommendation and not a forecast. This text describes mechanisms and their sources of risk. Companies and protocols named appear as examples of a mechanism, never as a recommendation. All figures carry the stated date; membership rosters, volumes and the legal position change.

Sources

As of 15 August 2026.

Try it yourself

Run the backtest with your own parameters and time ranges.

Run backtest →
📬

Don't miss new blog posts

One short email per new post — strategies, backtests, market analysis. No spam, unsubscribe with one click anytime.

By subscribing you accept our privacy policy. We use Resend for delivery. Double opt-in confirmation required.

Comments (0)

Join free to post comments.

Sign up →

No comments yet. Be the first!