Why My AMM Math Was Lying to Me

The code is correct. Every formula checks out. I've verified the constant product derivation, triple-checked the fee subtraction logic, confirmed the reserve lookups are pulling the right accounts. The math matches what's in every DeFi textbook, every Medium article, every YouTube tutorial. And yet, when I compare my predicted swap outputs against what actually happens on-chain, the numbers don't match. Not by a little. By 47%.

I'm staring at 275 consecutive simulation failures. Every single one comes back with the same error code: InsufficientProfit(6000). My screener evaluates a three-hop cycle, predicts 0.19% profit after fees, decides it's worth executing, builds the transaction, and then — the on-chain program checks the actual output and says no, the real number is below 0.10%. Not even close. The screener is telling me there's meat on the bone, and the chain is telling me the bone is bare.

Fifty-two Jito bundles. All returned INVALID. Fifty-two times I've paid the computational cost of building, signing, and submitting a bundle that was dead on arrival. It's like filing your tax return with a math error in every line — the IRS doesn't care that your calculator app works fine on other problems. Your numbers are wrong on this return, and they're sending it back.

Something is systematically inflating my predicted outputs. My code is correct in the way a recipe can be correct — every step makes sense in isolation — but the cake isn't rising. The formula is right. The implementation is right. But somewhere between the textbook and the blockchain, reality diverges, and I need to find exactly where.

The Clean World of x * y = k

Let me set the stage by recalling what the textbook says. I implemented this formula weeks ago. It's the heart of constant product AMMs:

output = (amount_in * reserve_b) / (reserve_a + amount_in)

Clean. Elegant. You put tokens in, the curve slides, tokens come out. To handle fees, you subtract the fee percentage from your input first:

effective_in = amount_in * (1 - fee_rate)
output = (effective_in * reserve_b) / (reserve_a + effective_in)

Still clean. This is the version I implemented, and in a perfect world — a world of infinite precision floating-point numbers and uniform fee structures — it works beautifully. I can compute the output for any input amount across any constant product pool, chain the outputs through a three-hop cycle, and determine whether the cycle is profitable.

The problem is that I don't live in that world. I live in the world of Solana programs compiled to BPF bytecode, operating on unsigned 64-bit integers, where every division truncates and every fee rounds in a specific direction. The textbook gave me a map. But the map doesn't match the territory.

It takes me four phases of debugging to understand just how far off the map is.

Phase 1: The Offsets Are Wrong

The first crack appears when I start reading on-chain source code for each DEX protocol, comparing their actual fee structures against what my screener assumes. I'm not looking for conceptual errors yet — I'm just checking whether I'm reading the right bytes from the right accounts.

I find three bugs in the first afternoon.

Bug one: a phantom byte. My CPMM pool parser reads the AmmConfig account starting at byte offset 9. The comment in my code says "skip 8-byte discriminator plus 1-byte bump field." Reasonable. Except when I pull up the actual CPMM PoolState struct definition, there is no bump field after the discriminator. The struct goes straight from the 8-byte Anchor discriminator to the 32-byte amm_config pubkey. My offset is off by one byte. I'm reading data[9:41] when I should be reading data[8:40]. Every field after that is shifted. The fee rate I'm extracting isn't the fee rate — it's whatever happens to live one byte to the right of the fee rate.

This is like misreading your W-2 because you assumed there was a blank column between the employer ID and the wages field. Every number downstream is wrong, and you can't tell by looking at the formula — the formula is fine. The input is garbage.

Bug two: numerator and denominator, swapped. AMM V4 stores its swap fee as a fraction — fee_numerator over fee_denominator. The values are 25 and 10,000, giving a 0.25% fee rate. My parser reads them in the wrong order. I'm computing 10,000/25 = 400 instead of 25/10,000 = 0.0025. Since I then subtract this "fee rate" from the input amount, a fee rate of 400 means I'm computing a negative effective input, which gets clamped to zero, which means my predicted output is... well, it depends on how the downstream math handles it, but it's certainly not reflecting reality.

I actually feel nauseous when I find this one. It's the kind of bug that makes you question every other field parse in the codebase. If I got the fee numerator and denominator backwards, what else is backwards?

Bug three: the invisible fee. Meteora DLMM pools have a volatility accumulator that contributes to a variable fee component. The total fee isn't just the base fee — it's base_fee + variable_fee, where the variable fee is derived from (volatility_accumulator * bin_step)^2. My parser ignores the volatility accumulator entirely. I'm computing fees using only the base rate, missing an entire fee component that can be substantial during periods of high trading activity.

Three bugs. Three different DEX protocols. Three different ways to misread on-chain data. I fix all three and rerun my simulations.

The error drops. But it doesn't disappear. My predictions are still systematically too optimistic. The overestimation is smaller — maybe 15-20% instead of 47% — but it's still there, still consistent, still biased in the same direction. I'm closer to reality, but I haven't found reality yet.

Phase 2: The Lamport That Changed Everything

This is the most subtle bug I've ever encountered, and I almost miss it entirely because the numbers are so close.

Here's the thing about AMM math on a blockchain: there are no floating-point numbers. There are no decimals. Everything is an unsigned 64-bit integer. When the on-chain program computes a fee, it doesn't get 2500.0000 — it gets 2500 or 2501, nothing in between. And the direction it rounds matters.

My Python implementation uses arbitrary-precision decimal arithmetic. When I compute a fee, I get the exact mathematical result. The on-chain Rust program can't do that. It uses integer division, and it follows two rules that are invisible in the textbook formula but absolutely critical in practice:

Fees round up (ceiling). When the program computes the fee to deduct from your input, it rounds up to the nearest integer. If the exact fee is 2500.0001, the program charges 2501. The protocol always takes at least as much as the mathematical fee — never less.

Outputs round down (floor). When the program computes how many tokens you receive, it rounds down to the nearest integer. If the exact output is 14999.9999, you get 14999. You always receive at most the mathematical output — never more.

The formula in pseudocode:

// On-chain fee calculation (ceiling division)
fee = (amount * fee_numerator + fee_denominator - 1) / fee_denominator

// On-chain output calculation (floor division)
output = (effective_in * reserve_out) / (reserve_in + effective_in)

That + fee_denominator - 1 in the fee calculation is the ceiling trick for integer division. It ensures the result always rounds up. And the output calculation uses plain integer division, which in every language I know of truncates toward zero — effectively rounding down for positive numbers.

Let me make this concrete. Input amount: 1,000,000 lamports. Fee rate: 2500/1,000,000.

My Python code computes: fee = 1,000,000 * 2500 / 1,000,000 = 2500.000000. Exact. Clean.

The on-chain program computes: fee = (1,000,000 * 2500 + 1,000,000 - 1) / 1,000,000 = (2,500,000,000 + 999,999) / 1,000,000 = 2,500,999,999 / 1,000,000 = 2500. Wait — same answer? Yes, because in this case the division is exact. No remainder.

But change the input to 1,000,001. Now: fee = (1,000,001 * 2500 + 999,999) / 1,000,000 = 2,501,002,499 / 1,000,000 = 2501. Python gives 2500.0025. The on-chain fee is 2501. One lamport difference.

One lamport. Who cares about one lamport? A lamport is one-billionth of a SOL. At current prices, it's worth a fraction of a fraction of a cent. It's the penny you find between couch cushions. It's the rounding error on your grocery store receipt that you never notice because it's below the threshold of caring.

Except in a three-hop arbitrage cycle, it's not one lamport. It's one lamport in the first hop's fee calculation. Then the slightly-smaller-than-expected output feeds into the second hop, where the rounding goes the wrong way again. Then the third hop. And at each step, there's rounding on both the fee (up) and the output (down). In a three-hop cycle, you have six rounding events — three fee calculations and three output calculations — and every single one of them moves the actual result in the same direction: against you.

It's like the difference between computing your taxes with a calculator that rounds to the nearest cent and using the IRS's own tables, which always round in the government's favor. Over a simple return with two line items, the difference might be a dollar. But if your return has dozens of itemized deductions, each one rounded against you, the cumulative impact starts to matter. And when you're operating on profit margins of 0.15%, every basis point of systematic error eats into a margin that was already razor-thin.

I rewrite my swap output functions to use integer arithmetic with the same ceiling/floor rounding as the on-chain programs. No more Decimal. No more floating-point. Pure integer math, matching the chain operation for operation.

The error drops again. But I'm still not at zero. Two specific protocols — Meteora DLMM and Whirlpool — still show discrepancies.

Phase 3: Meteora's Layered Toll Booths

Meteora DLMM isn't a constant product AMM. It's a concentrated liquidity protocol that divides the price range into discrete bins, each containing its own liquidity. When you swap through a Meteora pool, your input amount doesn't interact with a single pair of reserves. It traverses bins, consuming liquidity from each one until the input is fully spent or there are no more bins with liquidity.

I know this in theory. What I didn't know — what cost me another day of debugging — is how fees work across bin traversals.

My initial implementation applies the fee once, at the beginning. Take the total input, subtract the total fee, then traverse bins with the fee-adjusted amount. It's the same approach that works for constant product AMMs: compute the effective input, then compute the output.

This is wrong for Meteora. Dead wrong.

In reality, Meteora applies the fee at every bin. Each bin traversal is its own fee event. If your swap crosses three bins, you pay three separate fees, each computed independently on the amount consumed in that bin.

And the fee calculation itself has a wrinkle. Whether a bin is partially filled or fully drained determines the fee formula:

For a partial fill — where the bin has enough liquidity to absorb your remaining input:

fee = ceiling(consumed_amount * fee_rate / PRECISION)

For a full drain — where you exhaust all liquidity in the bin and move to the next:

fee = ceiling(net_amount * fee_rate / (PRECISION - fee_rate))

These formulas give different results for the same underlying amounts because of how the fee is factored. The partial fill formula computes the fee as a fraction of what you're putting in. The full drain formula works backwards from the net amount the bin provides, solving for the fee that would make the gross input equal to the bin's total available liquidity.

On top of that, the fee rate itself isn't static. It has two components: a base fee and a variable fee driven by recent volatility. The variable fee increases when the pool has experienced rapid price movement — essentially, the pool charges more when the market is volatile, penalizing trades that might be exploiting short-term mispricings. Which, of course, is exactly what arbitrage trades are doing.

So my Meteora math is wrong in three compounding ways: I'm applying fees once instead of per-bin, I'm using the wrong fee formula for drained bins, and I'm understating the fee rate by ignoring the volatility component.

Think of it like driving on a toll road where you assumed there's one toll booth at the entrance. You budget for a single $5 toll. But actually, there's a toll booth every two miles, each charging $1.50, and the toll increases during rush hour. You planned for $5 and you're paying $12. That's what's happening to my predicted swap outputs.

I rewrite the Meteora swap calculation to simulate the actual bin traversal. For each bin: compute the available liquidity, determine whether it's a partial fill or full drain, apply the appropriate fee formula using the composite fee rate (base + variable), compute the output from that bin, and move to the next. It's significantly more complex than the single-formula approach, but it's what the on-chain program actually does.

After this fix, Meteora DLMM predictions drop to 0.0001% error — one raw unit of rounding difference. I can live with that. That's the unavoidable residual of integer arithmetic: sometimes the last rounding goes one way, sometimes the other. The systematic bias is gone.

Phase 4: Where the Liquidity Changes Under Your Feet

Whirlpool — Orca's concentrated liquidity protocol — presents its own version of this problem, and it's the one that takes me the longest to even conceptualize.

In a constant product AMM, liquidity is uniform. The entire curve has the same depth everywhere. A dollar of input at price 150 moves the price the same amount as a dollar of input at price 200. The formula handles this seamlessly because there's one set of reserves, one curve, one computation.

Concentrated liquidity breaks this model. Liquidity providers can deposit their tokens into specific price ranges — tick ranges, in Whirlpool's terminology. The pool doesn't have uniform reserves. It has different amounts of liquidity at different prices. When your swap moves the price through a tick boundary, the available liquidity changes discontinuously. One tick range might have $500,000 of liquidity. The next might have $50,000. Or $5,000,000.

My initial Whirlpool implementation treats the swap as a single computation using the pool's current liquidity value. This works fine when the swap is small enough to stay within one tick range. But when the swap crosses a tick boundary, the liquidity changes, and my single-liquidity-value computation gives the wrong answer.

How wrong? Near tick boundaries, where liquidity can change dramatically from one tick range to the next, my linear approximation overestimates the output by up to 15%. Fifteen percent. In a business where the total profit margin is measured in tenths of a percent, a 15% overestimation in one hop of a three-hop cycle is catastrophic. It's like a baseball scout whose radar gun reads 15% high — suddenly every college sophomore looks like they're throwing 95, and you're drafting a roster full of disappointments.

The fix requires implementing tick-by-tick swap simulation:

1. Start at the current sqrt_price (Whirlpool stores prices as square roots in Q64.64 fixed-point format — another detail the textbook glosses over).
2. Determine the current tick range and its liquidity.
3. Compute the maximum input amount that can be consumed within the current tick range before hitting the boundary.
4. If my remaining input exceeds that maximum, consume up to the boundary, compute the output for that portion, cross the tick, update the liquidity, and continue.
5. If my remaining input fits within the current tick range, compute the output and stop.

Each of these steps involves its own integer arithmetic with specific rounding — sqrt_price calculations round toward zero when moving left (price decreasing) and away from zero when moving right (price increasing). The token amounts derived from sqrt_price differences use different rounding for token_a versus token_b. Every conversion has a direction, and every direction has consequences.

I implement the tick-traversal simulation, matching the on-chain Whirlpool program's logic step by step. I add the sqrt_price-to-tick conversion, the tick-to-sqrt_price conversion, the maximum-input-per-tick calculation, the liquidity-dependent output formula, and all the directional rounding.

I run the comparison tests.

Whirlpool: 0.0000% error. Bit-perfect match.

I actually sit back in my chair and stare at the screen for a minute. After days of chasing rounding errors and off-by-one byte offsets and per-bin fee applications and tick boundary liquidity changes, seeing four zeros after the decimal point feels like a physical relief. It's the feeling of your tax return coming back with a zero balance due — every number matches, every deduction checks out, every form is correct. The IRS has nothing to say. You and the government agree on the math.

The Compound Effect

Let me put the full picture together, because the individual bugs don't tell the whole story. The story is about how they compound.

In a three-hop arbitrage cycle, the output of each hop becomes the input of the next. A 2% overestimation in hop one doesn't just add to a 2% overestimation in hop two. It feeds forward. The inflated output from hop one makes the input to hop two larger than it should be, which (on the curved AMM surface) produces an even more inflated output for hop two. By hop three, the predicted profit can be wildly disconnected from reality.

My 47% overestimation wasn't one 47% error. It was several categories of errors — byte offsets, reversed fields, missing fee components, wrong rounding direction, single-tick approximation, per-start-instead-of-per-bin fees — each contributing a few percent here, a few percent there, compounding through three hops until the final predicted output was in a different universe from the actual on-chain result.

And all of these errors pointed in the same direction: too optimistic. Every one of them made my predicted output higher than the real output. This isn't a coincidence. It's structural. When you don't ceiling your fees, you undercount the fee. When you don't floor your outputs, you overcount the output. When you ignore per-bin fees, you undercount total fees. When you use uniform liquidity near a tick boundary, you overestimate the output from the higher-liquidity tick range and underestimate the impact of the lower-liquidity one — and because of the curve's shape, the overestimation dominates.

The result is a screener that sees profit everywhere. A screener that fires 275 simulations and 52 bundles at opportunities that don't exist. A screener that burns compute and wastes Jito submission slots on phantoms. Not because the formulas are wrong — the formulas are in every textbook — but because the textbook versions are approximations of what the on-chain programs actually do, and in the thin-margin world of arbitrage, approximations kill you.

What the Textbook Doesn't Teach

Here's what no DeFi math tutorial I've ever read covers:

One. On-chain AMMs don't use floating-point math. They use unsigned 64-bit integer arithmetic. Every division truncates. Every fee rounds up. Every output rounds down. These are not implementation details you can abstract away — they are the math. If your off-chain model doesn't match this arithmetic exactly, your model is wrong.

Two. Fee structures vary wildly between protocols. Constant product AMMs apply a single fee to the input. Concentrated liquidity AMMs apply fees differently at each tick crossing. Meteora applies fees at each bin traversal. Some protocols have multi-component fees that change based on market conditions. There is no "just subtract the fee" — you need to know exactly how each protocol computes its fee, on which amount, at which point in the swap, with which rounding.

Three. Concentrated liquidity is not "regular AMM with position ranges." It is fundamentally different math. The swap computation is iterative, not formulaic. You can't derive a closed-form expression for the output of a swap that crosses multiple tick ranges, because the liquidity at each range is different. You have to simulate the traversal, tick by tick, bin by bin, with full integer arithmetic at each step.

Four. The errors compound through multi-hop cycles. A tiny per-hop overestimation becomes a large cycle-level overestimation because each hop's error feeds into the next hop's input. Three-hop and four-hop cycles are where this hurts the most, because there are more stages for errors to amplify.

Five. Every error is in the same direction. Approximations in AMM math virtually always overestimate the output. Fee rounding goes against the trader. Output rounding goes against the trader. Ignoring fee components goes against the trader. Approximating non-uniform liquidity as uniform goes against the trader. There is no cancellation of errors — they stack.

After the Fix

After implementing all four phases of corrections, I run the full comparison suite — my off-chain predictions versus on-chain simulation outputs across hundreds of pools and input amounts.

CPMM: integer math match confirmed.
AMM V4: integer math match confirmed.
Meteora DLMM: 0.0001% error — one raw unit, one time in a thousand.
Whirlpool: 0.0000% error. Bit-perfect.

These numbers are no longer lying to me. When my screener says a cycle produces 0.15% profit, it actually produces 0.15% profit. When it says a cycle isn't profitable, it isn't profitable. The phantoms are gone.

But here's the thing that keeps nagging at me as I close the lid on this debugging marathon. I spent — I don't even want to count the hours — days chasing these discrepancies. Days learning that data[9:41] should be data[8:40]. Days learning that integer division rounds toward zero. Days learning that Meteora charges fees per bin, not per swap. Days learning that liquidity isn't uniform across tick ranges.

None of this is secret knowledge. It's all in the source code. It's all in the protocol documentation, if you know where to look and you read carefully enough. The on-chain programs are open source. The Rust code is right there. But no textbook, no tutorial, no "how to build a DeFi bot" guide I've ever found covers the gap between the clean formula and the messy reality.

The textbook teaches you x * y = k. It doesn't teach you that k lives in a u64 and the division truncates and the fee rounds up and the volatility accumulator adds a variable component and the concentrated liquidity changes at tick boundaries and all of these tiny imprecisions compound through your multi-hop cycle until your predicted profit is fiction.

I've fixed the math. My screener and the chain finally agree.

But agreement on the math is just the beginning. Knowing the exact output of a swap is necessary but nowhere near sufficient. There are layers beyond the formula — layers I can sense but haven't yet mapped. The formula is one piece. How many more pieces are there?

I fixed the math. I still don't know if it's enough.