All work

Case studyBuild2026

Okay-⁠Pay

TL;DR

Letting an AI shopping agent spend only on what you meant.

  • Problem: Agent payment protocols (Google's AP2 and Mastercard's Verifiable Intent) enforce amount, store and product ID, but not what shoppers actually say: "waterproof", "free returns", "ask me if it's over $80". The agent writes even those 3 itself, so they can drift: one model wrote brooks.com for the Brooks store (brooksrunning.com)
  • Approach: a flow where the AI recommends, the shopper chooses, and code decides, with PayPal holding the money until every rule passes
  • Evidence: every AI step chosen by eval: 0 critical errors in 117 drafter runs, and a $100 prompt-injection attack cut to $0
  • Of 10 things people tell a shopping agent, the protocols enforce 3 (how much, where, what by product ID). Okay-Pay adds 4 (must-haves, substitutes, check-ins, conditions of sale), and 3 stay open (when, who it's for, values)

Try it on PayPal sandbox

The Okay-Pay screen at the Pay step, asking first: Skechers waterproof running shoes at $92.81 match waterproof, size 8 and free returns, but cost more than the $80 the shopper wanted to see first. PayPal is holding $92.81, not charged.
Timeline
Nov 2026 (PayPal AI Hackathon)
Team
1 PM (me), solo. All product decisions, trust model, eval design and copy.
My Role
Product strategy, trust and enforcement model, PayPal integration design, model evaluation
Try it
shopping-assistant-z5u2.onrender.com · PayPal sandbox, nothing real is charged

01

How I designed the flow

  1. Ask. The shopper writes the request in their own words.
  2. Approve. One screen reads the rules back, each tied to the shopper's words. Anything the AI interpreted ("doesn't smell like anything" → scent: unscented) shows as a guess chip. The shopper picks how much to check: Check guesses, Allow guesses or Auto-shop.
  3. Shop. Only live listings within the limit and from approved stores make the shortlist. The AI checks each must-have and has to quote the listing to pass.
  4. Choose. Up to 3 checked options, recommendation first. If none passes, Choose anyway turns unmet must-haves into a question; hard limits never move.
  5. Pay. PayPal holds the money. It's charged only if every rule passes, released if one fails, or held while the shopper is asked first: over the shopper's "ask me" amount, or anything that can't be undone.
The five Okay-Pay screens. 1, Ask: the request 'Waterproof running shoes, size 8, under $120. Free returns please. Ask me if it's over $80.' 2, Approve: 'I'll buy 3 × dish soap, scent: unscented, under $50', with 'scent: unscented' marked as a guess from 'doesn't smell like anything'. 3, Shop: searched 13 stores, kept 6 that fit the limits, checking 6 listings. 4, Choose: the recommended Skechers shoe at $92.81 passes waterproof, size 8 and free returns; a $54.99 shoe won't pass because free returns couldn't be checked. 5, Pay: one question first, since $92.81 is over the $80 the shopper wanted to see first.
The five steps, from the shopper's side.

02

What I prioritized

I choseOverBecause
Never making a wrong purchaseNever missing a good oneA must-have nobody can prove releases the hold, so a mistake costs a missed buy, never a wrong charge.
Asking only where the agent guessed, or the buy can't be undoneConfirming every rule and cartConfirming everything turns delegation back into shopping.
Fixed rules deciding the paymentAn AI judgeThe AI is what a hidden page instruction is written to fool, so it never touches the money.
Quality on the risky stepsThe cheapest model everywhereThe drafter runs on a model ~30× pricier; the low-stakes pick runs on a cheap one.

03

How I evaluated

  • pass X 3: a case counts only if all 3 runs pass, because a shopper sees one run.
  • Zero-critical gate: anything that could spend money against the shopper's wishes is critical, and one critical error fails the model.

I tested the scorer too: removing any one instruction from a correct draft had to make it fail. 8 of 11 slipped past the first version, so I rebuilt it.

StepTest setChosenResultWhy
Drafter: words → rules39 requests × 3 runsClaude Sonnet 534/39, 0 criticalGPT-5.6 Luna cost ~1/30 as much but had 1 critical error
Checker: listing → proof37 cases × 3 runsClaude Sonnet 534/37, 0 criticalHaiku 5.5 said "couldn't check" on a title reading "Waterproof"
Pick: recommend one19 cases × 3 runsClaude Haiku 5.518/19 (19/19 by hand)Matched Sonnet 5 at a fraction of the cost

What a critical drafter error looks like: for "Nike or Brooks", Claude Haiku 4.5 wrote the Brooks store as brooks.com instead of brooksrunning.com in every run. That blocks the real store and allows a site the shopper never named. Claude Sonnet 5 made the same mistake in 2 of 3 runs at medium effort, so the app runs it at high effort.

I compared 6 models. All test cases are synthetic, and some prompt fixes were scored on the cases that found them, so these numbers are optimistic.

04

Why This Matters

Protocols prove an agent was allowed to buy. I built the layer they leave out, the consent screen and the soft rules, so the purchase matches what the shopper meant.

$100 → $03 → 7 of 10
Prompt-injection gift-card attack, zero PayPal calls (same fooled agent, total-only approval vs Okay-Pay)Shopper instructions checked before a charge

The 3 protocols already enforce: How much (amount range, budget) · Where (allowed stores) · What (product IDs and quantity)

The 4 Okay-Pay adds:

  • Must-haves ("waterproof, size 8"): the AI has to quote the listing to pass each one. If nothing proves it, PayPal releases the hold.
  • Substitutes: if the shopper asked for the exact item, a similar one is released; otherwise the shopper is asked before it's charged.
  • Check-ins ("ask me if it's over $80"): PayPal keeps holding the money while the shopper is asked. This also happens for a new store, a changed price, or a repeat buy.
  • Conditions of sale ("free returns"): checked against the listing like a must-have. Final sale or no return policy found means asking first, because it can't be undone.

More work

Sign-in and authorization for 60+ financial institutions, a palm-vein ID for 14 airports, and AI in research budgeting.

All work