0% Comprehensive Diagnosis — What Is the Root Cause

I have spent the last several articles cutting open the bot's performance and examining individual organs. Log analysis showed me the raw numbers. WebSocket feeds exposed the data pipeline's fragility. InsufficientProfit revealed the prediction-reality gap. Bundle status codes taught me the vocabulary of failure. The leader schedule explained why timing matters. And the FAILED black box demonstrated how much remains unknowable even after everything else is understood.

Each investigation answered a question. Each question led to another question. Each answer was partial, illuminating one corner of the problem while leaving the rest in shadow. Now I am standing back and looking at all the findings together, because the question I need to answer is no longer "what went wrong here" but "what is actually going on."

The landing rate is zero percent. Not approximately zero. Not trending toward zero. Zero. Hundreds of attempts, zero successes. The bot detects opportunities, constructs transactions, submits bundles, and none of them land. Every single one fails. The machinery runs. Nothing works. And after examining every piece of the system individually, I am beginning to understand that the answer is not going to be a single broken component. It is going to be something worse.

The Annual Physical Results

There is a particular experience in American medicine that captures what I am going through. You go in for an annual physical — the full workup. Blood panel, cholesterol, blood pressure, metabolic screen, the whole thing. You have been feeling off. Not sick exactly, but not right. Something is wrong, and you want the test to find it.

The results come back, and no single number is catastrophic. Your cholesterol is elevated but not dangerous. Your blood pressure is a little high. Your blood sugar is in the pre-diabetic range but not diabetic. Your vitamin D is low. Your triglycerides are above normal. Individually, none of these numbers would alarm your doctor. None of them would trigger an emergency. Each one, in isolation, gets the same advice: watch it, maybe adjust your diet, come back in six months.

But the doctor does not look at each number in isolation. The doctor looks at the panel as a whole. And what the panel shows is a pattern. The cholesterol, the blood sugar, the blood pressure, the triglycerides — they are not five independent problems. They are five symptoms of one underlying condition. Metabolic syndrome. A systemic issue that manifests through multiple measurements, none of which is individually alarming but all of which, taken together, tell a story that is very different from "basically fine."

The bot's zero percent landing rate is not caused by one thing. It is metabolic syndrome. Multiple factors, each contributing, each reinforcing the others, each individually insufficient to explain the total failure but collectively producing a system that cannot succeed. Finding the answer requires looking at the whole panel, not individual readings.

The Punch List

When a building is constructed, there is a moment near the end of the process called the punch list walkthrough. The general contractor, the architect, and the owner walk through every room, every corridor, every system, and document everything that is not right. A light switch that does not work. A door that sticks. A paint drip on a baseboard. A HVAC vent that rattles. A tile that is cracked. A cabinet door that does not close flush.

No single item on the punch list prevents the building from standing. The foundation is solid. The walls are up. The roof does not leak. The structure is sound. But the building is not habitable until the punch list is resolved, because the sum total of small problems creates a space that does not function as intended.

I am doing the punch list walkthrough on my arbitrage bot. And the list is long.

Cause One: The Data Is Stale

This is the largest item on the list. It is not just a punch list item — it is a load-bearing wall that was poured with the wrong concrete mix.

The bot reads pool state from the blockchain. It uses that state to calculate whether an arbitrage cycle is profitable. It builds a transaction based on that calculation. It submits the transaction to the network. The transaction executes against the pool's state at the moment of execution — not the state the bot read, but the current state. If the state changed between reading and executing, the calculation is wrong. The opportunity that existed when the bot looked at the data may not exist when the transaction runs.

This is not a subtle problem. It is the fundamental constraint of the entire system. The bot is making decisions based on information that is, by definition, already in the past. The question is how far in the past. If the data is 50 milliseconds old, the chance that the relevant pools have changed is relatively small. If the data is 500 milliseconds old, the chance is significant. If the data is several seconds old — which happens more often than I want to admit, due to WebSocket connection instability, polling delays, and processing latency — the chance approaches certainty for any pool that sees meaningful trading volume.

The analogy is a stock trader making decisions based on a price ticker that runs on a fifteen-minute delay while competing against traders who see prices in real time. The delayed trader identifies an opportunity that existed fifteen minutes ago. By the time the trade executes, the price has moved. The opportunity is gone. It does not matter how sophisticated the trading strategy is. It does not matter how accurate the calculations are. The data is old, and in a market where other participants have fresher data, old data is worse than no data at all — because no data at least prevents you from acting on false confidence.

The stale data problem permeates every other failure mode. It makes the AMM calculations wrong even when the math is correct, because the math is being applied to the wrong inputs. It makes competition losses inevitable, because competitors with fresher data see opportunities earlier and act on more accurate information. It makes timing mismatches worse, because the delay between observation and action is compounded by the delay between reality and observation. It is not one problem on the list. It is the problem that makes every other problem worse.

Cause Two: The Competition Wins

Even when the data is fresh — even in the moments where the bot reads a pool state that is genuinely current and identifies a genuinely profitable opportunity — someone else gets there first.

This is not a surprise. It is, in some sense, the expected outcome. The MEV ecosystem on Solana is populated by teams that have been doing this for months or years. They have invested in infrastructure that I have not built. They run nodes closer to the validators. They have optimized their transaction construction to shave microseconds. They have relationships with block engine operators. They operate at a level of sophistication that my bot — a solo project, running on a single server, built from scratch — does not approach.

The business analogy is opening a sandwich shop on the same block as a Subway, a Jersey Mike's, and a local favorite that has been there for twenty years. Your sandwiches might be good. Your ingredients might be fresh. Your pricing might be competitive. But the other shops have supply chains, brand recognition, loyal customer bases, and operational efficiencies that you, on day one with a single employee, simply do not have. You are not competing on the quality of your product. You are competing on infrastructure, and infrastructure takes time and capital to build.

The competition problem is structural. It is not a bug I can fix. It is not a configuration I can tune. The other bots are faster because they have faster data sources, faster hardware, and faster network paths. The only way to compete is to acquire similar advantages, and that is not a software change — it is an infrastructure investment that changes the fundamental architecture of how the bot receives information and submits transactions.

Cause Three: The AMM Math Is Wrong

The bot's off-chain simulation of AMM behavior does not perfectly replicate the on-chain execution. This is the InsufficientProfit problem examined in detail earlier, but it deserves its place on the comprehensive list because its contribution to the zero percent rate is real and distinct from the other causes.

The bot calculates what it thinks the output of a swap will be. The on-chain program calculates the actual output. These two numbers should be the same, but they are not, because the off-chain math makes simplifications and approximations that the on-chain code does not. Rounding behavior differs. Fee calculations may not match precisely. State transitions — the way a pool's internal state updates during a swap — are modeled but not perfectly replicated. The accumulated error across a multi-hop cycle can be enough to turn a predicted profit into an actual loss.

This is like using a map app that calculates your drive time at 35 minutes, but every time you make the drive it takes 42 minutes. The app's algorithm is not fundamentally broken. It accounts for distance, speed limits, traffic patterns. But it rounds optimistically on every turn, underestimates the delay at every traffic light, and ignores the two school zones that add three minutes each. Each individual error is small. The accumulated error is significant enough to make you late every time.

The good news about this cause is that it is fixable. The math can be corrected. The rounding behavior can be matched to the on-chain implementation. The fee calculations can be made precise. I have already fixed several of these discrepancies, and the prediction accuracy has improved. This is the punch list item where the contractor says "yeah, we can fix that Tuesday." It is real work, but it is known work with a clear path to resolution.

Cause Four: Fee Underestimation

The bot's cost model does not capture every fee. The obvious fees are accounted for — the AMM swap fees, the Jito tip, the base transaction fee. But there are smaller costs that accumulate. Network priority fees in congested conditions. Token account rent for accounts that need to be created. Computational overhead that pushes the transaction closer to compute limits. Each of these is individually small, but when the predicted profit margin is itself small — which it usually is, because large margins are captured by faster bots before I even see them — the small fees become proportionally significant.

This is the difference between calculating whether you can afford a house by looking at the mortgage payment versus the total cost of ownership. The mortgage is the obvious number. But property taxes, homeowner's insurance, HOA fees, maintenance reserves, and utilities add a substantial layer of cost that the mortgage number alone does not capture. A house that looks affordable based on the monthly payment might be stretching the budget when every cost is included. The bot's profit calculations look viable before every fee is included. After every fee is included, the margins often vanish.

This cause is partially fixable. The known fees can be modeled more accurately. But some fee components are variable and unpredictable — they depend on network conditions at the moment of execution, which the bot cannot know in advance. There is a floor below which improvement becomes impossible without structural changes to how the bot estimates costs.

Cause Five: Timing Mismatch

The bot submits bundles without precise awareness of the Jito auction cycle or the leader schedule dynamics. This means bundles sometimes arrive at suboptimal moments — after the current auction window has closed, before the next Jito-connected leader's slot begins, or during a period when the block engine is processing a backlog of higher-priority bundles. The bundle is valid. The opportunity is real. The timing is off.

This is like arriving at the post office at 5:01 PM. The letter is correctly addressed. The postage is correct. The envelope is sealed. Everything about the package is right. But the window closed at 5:00 PM, and the clerk is already processing the last batch. Your letter will go out tomorrow, and by tomorrow the opportunity the letter represents — whatever it is — might be gone.

The timing problem is partially solvable through better integration with the leader schedule and auction timing. But it is also partially a consequence of the stale data problem: if the bot's data is already behind, the timing of its submission is starting from a deficit. Faster data would naturally improve timing because the entire pipeline — observe, calculate, build, submit — would complete closer to the moment of relevance.

The Interaction Effects

The most important insight from this comprehensive review is that these causes do not operate independently. They interact. They compound. They create failure scenarios that no single cause could produce on its own.

Stale data makes the AMM math wrong even when the math itself is correct, because the math runs on outdated inputs. The AMM math error then overstates the expected profit, which means the bot pursues opportunities that were never actually profitable, which means those submissions are guaranteed failures regardless of competition or timing. Meanwhile, stale data makes competition losses worse because the bot is identifying opportunities later than competitors who have fresher data, which means the bot is always starting the race behind.

Timing mismatches compound competition losses because a bundle that arrives at a suboptimal moment is more likely to be outbid by a competitor whose bundle arrived at the optimal moment. Fee underestimation compounds AMM math errors because both push the prediction in the same direction — toward overestimating profitability. The bot thinks the opportunity is more profitable than it is (AMM error) and thinks the costs are lower than they are (fee underestimation), and the combination produces a confidence in opportunities that do not exist.

This is the metabolic syndrome parallel. High blood sugar alone is manageable. High blood pressure alone is manageable. High cholesterol alone is manageable. But high blood sugar plus high blood pressure plus high cholesterol creates cardiovascular risk that is not the arithmetic sum of the three individual risks — it is multiplicatively worse. The conditions feed each other. Insulin resistance contributes to high triglycerides. High blood pressure damages arterial walls already weakened by cholesterol deposits. The system-level failure is greater than the component-level failures would suggest.

The bot's zero percent landing rate is not the result of being slightly wrong about five things. It is the result of being slightly wrong about five things that all make each other worse.

The SAT Score Breakdown

When a student gets their SAT score back, the total number is the headline. But the score report breaks it down by section — reading, writing, math — and within each section, by skill area. A student who scores 1200 total might have a 650 in reading and a 550 in math, or a 550 in reading and a 650 in math. Same total. Very different study plan.

More importantly, the score report shows which question types were missed. Did the student struggle with algebra or geometry? Reading comprehension or vocabulary in context? Grammar rules or rhetorical effectiveness? The breakdown reveals which skills are weak and which are strong, and that information is what makes improvement possible. Without it, "study harder" is the only advice. With it, "focus on quadratic equations and passage-based inference questions" becomes the prescription.

My bot's score breakdown looks like this: the largest contributor to failure is stale data. The second largest is competition. The third is AMM prediction error. Fee underestimation and timing mismatch are smaller but real contributors. Each has a different character, a different degree of fixability, and a different implication for what I should do next.

Fixable Versus Structural

Not all items on the punch list are equal. Some are paint drips — cosmetic, easily corrected, requiring only attention and time. Some are structural — foundation issues that cannot be fixed with touch-up work, that require rethinking the building's design.

AMM math errors are paint drips. The math can be corrected. The rounding can be matched. The fee calculations can be made precise. I have already started this work, and the results are measurable. Each correction brings the off-chain prediction closer to the on-chain reality. This is known work with known solutions. It requires effort but not invention.

Fee underestimation is partially a paint drip and partially structural. The known fees can be modeled better. But the variable fees that depend on real-time network conditions require infrastructure changes to predict accurately.

Stale data is structural. The current architecture — polling pools via RPC calls, receiving updates via WebSocket subscriptions that drop and reconnect, processing state changes through a pipeline that adds latency at every stage — has a floor on how fresh the data can be. I can optimize the pipeline. I can reduce the processing overhead. I can improve the WebSocket reconnection logic. But these optimizations operate within the current architecture's constraints. There is a limit to how much improvement is possible without changing the fundamental way the bot receives information.

Competition is structural. The other bots are faster because they have faster infrastructure. Optimizing my code, tuning my parameters, and fixing my bugs will help, but it will not close the gap created by a fundamentally different tier of data access. This is the sandwich shop problem: you can make better sandwiches, but you cannot out-infrastructure a chain with a thousand locations by improving your recipe.

Timing mismatch is partially fixable within the current architecture and partially dependent on solving the stale data problem, which is structural.

The pattern that emerges from this classification is uncomfortable but clear. The fixable problems are the smaller contributors. The larger contributors are structural. Fixing the paint drips will improve the system, but the system will still fail because the foundation issues remain.

The Post-Mortem

In business, when a venture fails, the partners sit down for a post-mortem. Not to assign blame — that is a different kind of meeting — but to understand what happened. What did the market look like? What assumptions were made? Which assumptions proved wrong? What would have to be different for the outcome to change?

A good post-mortem distinguishes between tactical errors and strategic errors. A tactical error is pricing the product wrong. It is hiring the wrong vendor. It is launching in the wrong season. Tactical errors are correctable within the existing strategy. You fix the price. You switch vendors. You relaunch in the spring. The strategy survives.

A strategic error is entering a market that does not exist. It is building a product that solves a problem no one has. It is competing on a dimension where you have a structural disadvantage that no amount of tactical excellence can overcome. Strategic errors require a strategy change, not better execution of the current strategy.

My post-mortem is pointing at a strategic error.

The current strategy is what I think of as trigger-only architecture. The bot polls pools. It reads their state. It identifies discrepancies. It builds transactions. It submits bundles. The entire pipeline is triggered by the bot's own observation cycle — it looks at the world, decides something interesting is happening, and acts. The bot is the initiator. The blockchain is the target. The bot scans, detects, acts.

This architecture has a fundamental problem: it is always behind. The bot reads state that is already committed to the blockchain. By the time the state is readable, it is already public. Every other bot with access to the same RPC endpoints can read the same state at approximately the same time. The competition is not about who sees the opportunity first — it is about who acts on it fastest after everyone sees it simultaneously. And in that race, infrastructure wins. Not algorithms. Not math. Not cleverness. Infrastructure.

The alternative is event-driven architecture. Instead of polling the world and reacting to what it sees, the bot listens for events — on-chain state changes, transaction confirmations, program invocations — and reacts to them as they happen. The data comes to the bot instead of the bot going to the data. The pipeline is triggered by the blockchain, not by the bot's polling cycle. The latency between event and reaction shrinks because the observation step is eliminated — there is nothing to observe because the event itself is the observation.

This is a fundamental strategy change. It is not an optimization of the current approach. It is a different approach. A different architecture. A different way of thinking about when and how the bot should act.

Where I Am

I started this series at zero percent. After months of building — learning Solana's account model, mastering AMM math, integrating multiple DEXes, building a custom on-chain program, solving the transaction size problem with address lookup tables, implementing structured logging, debugging WebSocket connections, decoding bundle status codes, understanding leader schedules, and confronting the black box of FAILED bundles — I am still at zero percent.

The number has not changed. But I have.

At the beginning, zero percent was a mystery. I did not know why the bot was failing because I did not know enough to ask the right questions. I did not understand what a bundle status meant. I did not know how leaders were scheduled. I did not appreciate the difference between reading state and acting on state. I did not understand the depth of the competition. I did not know how AMM math could be right and wrong simultaneously. Zero percent was a wall with no doors.

Now, zero percent is a diagnosis. I know the five causes. I know their relative contributions. I know which are fixable and which are structural. I know that the fixable ones are the smaller contributors and the structural ones are the larger contributors. I know that optimizing within the current architecture will improve things but will not reach viability. I know that a strategy change is needed — not a tweak, not a parameter adjustment, not a bug fix, but a fundamental rethinking of how the bot observes and acts.

Zero percent with no understanding is despair. Zero percent with complete understanding is a starting point. The same number means entirely different things depending on what you know about why it is that number.

Understanding Is the Beginning of Change

There is an old principle in engineering that you cannot improve what you cannot measure, and you cannot fix what you do not understand. I can now measure everything. I understand everything — or at least enough of everything to know what I do not understand and why it matters.

The comprehensive diagnosis is complete. The panel of test results is in. No single reading is the answer. The answer is the pattern across all the readings: a system that is structurally positioned to fail because its fundamental architecture puts it at a disadvantage that tactical improvements cannot overcome.

The strategy must change.

That is not a defeat. That is a diagnosis. The difference between a doctor who says "there is nothing wrong" and a doctor who says "here is exactly what is wrong and here is what we need to change" is the difference between helplessness and a plan. The first doctor sends you home with the same symptoms. The second doctor gives you a path forward.

I have the path. The bot needs to stop scanning the world and start listening to it. The architecture needs to change from trigger-only to event-driven. The data pipeline needs to be rebuilt around real-time event streams instead of polling cycles. The entire timing model needs to be restructured around reacting to changes as they happen instead of discovering them after the fact.

I do not know yet whether this will work. I do not know whether event-driven architecture will close the gap enough to turn zero percent into something above zero. I do not know whether the competition's infrastructure advantage extends to the event-driven domain as thoroughly as it dominates the polling domain. I do not know the answer.

But I know the question. And after months of not even knowing the question, that is progress.

The landing rate is still zero. But for the first time, I understand exactly why.