Five Wrong Assumptions That Keep Tripping Me Up on Solana

When the Same Mistake Keeps Putting on a New Costume

I keep a running log of every transaction that fails on me — every AccountDidNotDeserialize, every ConstraintSeeds, every silent drop where the bot thinks it submitted something and the chain shrugs and forgets it ever existed. After enough entries, I am starting to notice something uncomfortable. The bugs look different on the surface, but underneath, the assumption that broke is almost always the same one wearing a new costume.

This is the realization I want to sit with today. Not a list of specific incidents — I have written episodes about those — but the shape of the wrong assumption itself. Five shapes, actually. Five patterns that keep reappearing no matter how many times I think I have learned my lesson on each one individually.

Why a Pattern View Beats an Incident View

When something breaks, the natural reflex is to fix the specific thing. The discriminator was wrong, so I add a discriminator check. The seed order was reversed, so I write down the correct order in a comment. The blockhash expired, so I refresh it. Patch, patch, patch.

The problem is that patches do not generalize. A month later, a new bug shows up that has nothing to do with discriminators or seed orders or blockhashes, and yet it feels eerily familiar. That familiarity is the signal. Underneath the costume, it is the same assumption wearing different clothes. If I can identify the shape, I can guard against the class of mistake, not just the instance. This article is my attempt to do that for the five shapes I keep meeting.

Pattern One: "I Read the Code, Therefore I Know the Layout"

The first trap is the one I fell into hardest. I would find a protocol's open-source library, locate the struct definition for whatever account I wanted to read, and assume the bytes on chain match that struct exactly. After all, the source code is right there in front of me. What could be more authoritative than the source code?

A lot of things, it turns out. A developer resource notes: "Account and transaction data are often encoded, which is good for efficiency but bad for developer sanity." The struct I am reading is a description of intent. The bytes on chain are the fact. Those two are usually aligned, but "usually" is doing a tremendous amount of work in that sentence.

A framework like Anchor, for example, silently prepends an 8-byte discriminator to every account it creates. If I size my account by adding up the fields in the struct — say 32 + 1 for a pubkey plus a flag — I have just under-allocated by eight bytes. The error I get back will not say "you forgot the discriminator." It will say AccountDidNotDeserialize, which sounds like a data corruption problem and sends me down a rabbit hole that has nothing to do with the actual cause.

This is the part that gets me: the source code did not lie. The discriminator behavior is documented. But I did not observe what was actually on chain. I observed my mental model of what should be on chain, derived from the struct, and acted on that. The cure, which I am still internalizing, is to verify the byte layout directly before trusting it — fetch the raw account, look at the first eight bytes, confirm the discriminator matches what I expect, and only then proceed to decode the rest. A QuickNode walkthrough on account deserialization is a useful concrete reference here: it lays out the byte-offset structure for an account — discriminator first, then each declared field at its serialization-specific offset — so you can reconcile the on-chain bytes against the layout in a methodical way rather than guessing. The source code is a map. The chain is the territory. Treating the map as the territory is the original sin of this whole category.

There is a quieter version of this trap that hurts even more, which is reading a struct in a third-party library and assuming the order and size of its fields is what is actually serialized on chain. Languages have padding rules. Serialization frameworks have their own conventions. The on-chain bytes are not the Rust struct in memory; they are the result of a specific encoder run with specific assumptions. Two encoders looking at the same struct can produce different bytes. The way out is to find an example transaction on a block explorer, pull the raw bytes, and reconcile them against the layout I think I have. If the reconciliation fails, I know my map is wrong before I commit to using it.

Pattern Two: "It Worked Locally, So It Works"

The second trap is the one I am still falling into the most often. Tests pass locally. The transaction lands in simulation. I push to production and watch it fail in ways the local environment never reproduced.

The failure modes here are sneaky because they are silent or near-silent. A few from my recent collection:

A transaction can quietly exceed the compute unit budget under real network load even though it ran fine in tests with synthetic data. Heavy compute paths — CPIs, large loops, and cryptographic proof verification — are the most common reasons a transaction exceeds its budget. The same developer guide names the resulting failure as ComputeBudgetExceeded. What makes it nasty in production is the gap between simulation and live network load: the error can appear with no change to code that worked fine in tests, and the error itself does not name which instruction blew the budget. Separately, transactions can also disappear with no on-chain failure at all when the validator queue is overloaded — a different class of silent drop with a different root cause, but the same outcome from the submitter's seat: a transaction that was supposed to land and did not.

A blockhash can expire between the moment I sign and the moment I submit, especially if there is any queueing or retry logic in between. Locally, the gap between sign and submit is microseconds. In production, it can be seconds, and seconds are enough to invalidate the blockhash. Same code, different timing, completely different outcome.

Accounts created with insufficient lamports for rent exemption fail mysteriously — sometimes appearing to succeed from the submitter's side while the account does not actually persist. Migrations leave behind state drift where two versions of an account coexist on chain because the migration script ran on a subset of accounts before something interrupted it.

The meta-mistake is treating "no error" as equivalent to "success." Solana's failure spectrum includes silent discards, vague error messages, and environment gaps that local testing does not exercise. A developer writeup captures the experience: "You're staring at a red 'Transaction failed' message with no clue whether you just shipped a logic bug, hit a rate limit, or accidentally tried to write to a read-only account." When the error message is opaque or the failure is silent, every assumption I made along the way becomes a suspect, and there is no easy way to narrow the suspect list.

The practical antidote, which I am building into my own workflow piece by piece, is to make production-realistic conditions part of the local loop. Track compute usage in tests and fail CI if it regresses. Fetch fresh blockhashes just before signing rather than reusing one from earlier. Reproduce the failure with the smallest possible script that does not depend on the full application — strip the variables down until what remains is unmistakably the cause.

There is also a category of "works locally" mistake that has nothing to do with the chain and everything to do with my own test fixtures. The test environment has a specific account population: the accounts I created during setup. Production has every account that ever existed, in every possible permutation of state. A code path that handles "the account I just created in my test" perfectly well can fail on the account that has been around since 2022 and has accumulated three years of edge-case state. The fixture is a small, polite sample of the world. The world is not small or polite.

Pattern Three: "My Environment Is Configured the Way I Think It Is"

The third trap looks the most boring on paper but has cost me more debugging hours than any other. It is the assumption that my local development environment is set up the way I believe it is.

A canonical example comes from a developer guide on macOS toolchains: the default tar on macOS is BSD tar, while Solana's build tooling expects GNU tar. The error you get back is not "wrong tar." It is some opaque archive failure that points nowhere in particular. The system's default tool is silently wrong, and there is no warning to tell me about it.

The Borsh serialization library has a similar trap. Different crates in the dependency tree can pull in different Borsh versions, and the compiler returns a type-constraint error that points at Pubkey rather than at the version conflict that is actually the cause. A developer guide notes that mismatched Borsh versions across crates can create incompatibilities without obvious warnings.

Rust toolchain conflicts are the same shape. I have a global Rust version. Solana sometimes uses a forked compiler with its own version requirements. Dependencies further down the tree may require a newer or older rustc. The PATH variable decides which one actually runs, and PATH is influenced by shell init files I last edited months ago and have completely forgotten. The error message — something like "package solana-program v1.16.3 cannot be built because it requires rustc 1.68.0" — tells me the symptom but not the cause. I assumed rustup default was the source of truth. The actual source of truth was a stale shim earlier in PATH.

This category is so insidious because the environment looks fine. Commands run. Tools respond. Outputs come back. They just happen to be wrong. The shape of the assumption here is treating the appearance of a working toolchain as evidence that the toolchain is working. It is the local-equivalent of confusing the absence of an error message with the presence of correctness.

The remedy is to be skeptical specifically about the things that are easy to take for granted. Pin dependency versions explicitly. Audit which rustc is actually being invoked, not which one I think is being invoked. When something fails in an unexpected way, check the environment before checking the code. The bug is more often in the floor than in the ceiling.

There is a corollary that I keep relearning: build pipelines can also silently rewrite my code into broken code. One developer reference flags a case where the build process converts ES2020+ BigInt syntax into incompatible Math.pow() format. I wrote correct code. The bundler shipped wrong code. Nothing in my source tree explains the bug because the bug is not in my source tree. It is in the transformation that happens between source and runtime. That whole layer — the build chain, the bundler, the transpilers — is part of my environment, and it lies about its own behavior all the time.

Pattern Four: "The Runtime Will Figure It Out"

The fourth trap is the assumption that the program-account relationship is flexible — that as long as I name things plausibly and pass them in roughly the right order, the runtime will sort it out. It will not.

A few sub-traps within this category, each of which I have personally walked into:

The SPL Token program and Token-2022 are not interchangeable. They look similar at the API level, they handle similar concepts, and they are different programs with different program IDs. Calling Token-2022 instructions while passing the classic Token program ID, or vice versa, fails in ways that do not immediately announce themselves as a program-ID confusion.

Program-derived addresses do not sign transactions on their own. If a PDA is the authority for an action, the calling program has to explicitly pass the signer seeds in the cross-program invocation context. The assumption that the runtime infers signing authority from account ownership is wrong. Authority has to be asserted at the moment of the call, not inferred from the account graph.

Seed order is rigid. A developer guide tells a story I recognize too well: a developer swapped the order of seeds in one place from ['vault', market] to [market, 'vault']. Locally everything looked fine because the inputs happened to align in the test fixture. In production, every cross-program invocation failed. The seeds derive a different address. The runtime does not autocorrect.

Delegate state on token accounts is another version of this. A cross-program flow worked fine in every test environment, then failed in the wild only when the user happened to have a delegate set on their associated token account. The assumption was that no delegate existed; the runtime did not enforce or surface that assumption; the world produced a counterexample.

The common shape is assuming that account-to-program relationships have implicit slack. That if I get the rough structure right, the runtime will fill in the gaps. It does not fill in the gaps. Every relationship is explicit. Every signer must be declared. Every seed must be in the right order. Every program ID must be the one you actually mean. The runtime is not a forgiving collaborator that nudges you toward correctness. It is more like a TSA checkpoint: every field on the form has to be exactly right, and if it is not, the answer is no, with minimal explanation.

What makes this category extra painful is the opacity of CPI failures. When an instruction inside a cross-program invocation fails, the error log often shows only the failed instruction, not the path that led to it. I do not get a stack trace in any familiar sense. I get the last frame. Reconstructing what called what, and why, is on me. That reconstruction work is most of the cost of debugging in this category, and there is no good way around it short of logging more aggressively up front than feels reasonable in the moment.

Pattern Five: "The Docs and the Chain Are in Sync"

The fifth trap is the most philosophical and the hardest to defend against. It is the assumption that the documentation, the example code, the library version on my machine, and the actual deployed protocol are all in agreement with each other.

They often are. Sometimes they are not. A developer guide flags a specific instance: a stale IDL (interface description) on the client causes wrong enum variants to be serialized, and the chain rejects the instruction with Invalid Instruction Data. The client code was correct. The IDL it was built against was outdated. The fix — regenerate types from current IDL and pin SDK versions — is straightforward once you know to look. The problem is knowing to look. From the developer's seat, the error message does not point at the IDL. It points at the instruction. So I waste an hour reading my instruction logic before remembering to check the layer underneath.

There's a related version of this trap at a higher level. New approaches to Solana often carry a REST or SQL mental model — expecting the chain to hand back "a user's tokens" or "all swap activity" as if it kept product-level objects ready to return. It does not. The mental model imported from REST APIs and SQL databases is fundamentally incompatible with how Solana data is actually structured. You start with accounts, then add decoding, indexing, and protocol-specific parsing on top. The documentation describes the account model accurately. The developer's intuition, formed elsewhere, fills in the gaps with assumptions that are not supported by what is actually there.

Commitment levels are the version of this trap that is most relevant to anyone doing latency-sensitive work. The wrong commitment level means I am acting on stale data — possibly seconds old — while believing I am acting on the current chain state. For arbitrage and liquidation logic, seconds is forever. The documentation explains commitment levels clearly. The mistake is not in the documentation; it is in the developer's assumption that the default is the right default for their use case.

The meta-pattern across all of these is treating documentation as a perfect mirror of on-chain reality. Documentation describes intent and design. The chain at this specific moment, under these specific conditions, with this specific account's history, can diverge. As the protocol evolves — for instance, with changes to consensus mechanics — assumptions that were correct a year ago can quietly become wrong. The same roadmap notes that Alpenglow's design moves consensus finality from probabilistic toward near-deterministic — a meaningful shift for any strategy that previously had to hedge against finality uncertainty. The architectural assumption about how to handle finality is itself something that gets updated as the protocol changes underneath you.

The Thread That Connects All Five

When I lay these five patterns next to each other, I can see what they share. Every single one of them is a moment where I trusted a representation of the system — the source code, the local environment, the test result, the documentation, the absence of an error message — instead of directly observing what the actual system was actually doing under actual conditions.

This sounds obvious when stated bluntly, but it is exactly the kind of obvious thing that is easy to violate in practice. Direct observation is expensive. It takes time. It requires writing scripts, fetching real data, decoding bytes by hand, comparing what you expected to what is actually there. The representations are right there, free, in front of me. The struct definition. The README. The test logs. The green checkmark in CI. Of course I trust them. They are not wrong, exactly. They are just incomplete in ways I do not always know in advance.

A developer writeup states it more bluntly: "Most Solana 'bugs' aren't actually bugs but rather mismatched accounts, wrong commitment levels, compute limits, stale RPC data, or a transaction that looked fine until it hit a different validator." None of those are bugs in the program logic. They are all mismatches between what the developer assumed and what the actual system was doing. The fix in each case is not to write better logic. It is to align the assumption with reality.

There is also a broader industry context that I find worth holding in mind. Industry surveys put the failure rate for blockchain initiatives at around seventy percent, with rigid architectures and usability gaps as recurring culprits. The pattern is not Solana-specific; it is a general fact about systems where the developer's mental model and the deployed reality diverge faster than the documentation can catch up. One 2025 retrospective on Web3 development tracks a more recent contraction in the active developer base — roughly from twelve thousand four hundred weekly active open-source contributors down to around seven thousand six hundred, with weekly commits collapsing from a peak of around one hundred seventy-six thousand to under one hundred thousand by mid-year. The same author's framing has stuck with me: "The gap between technology working and people actually wanting to use it remains much wider than the industry has been willing to admit." That gap is, in a smaller way, what each of my five patterns is also about. The technology works in the sense that the documentation is correct and the code compiles. It just does not work in the sense that what I built matches what the chain actually does, on a Tuesday afternoon, with a real user's account state.

What Is Solved Versus What Still Hurts

Not every pattern in this list is equally tractable. Some have become much easier to handle as the ecosystem matures:

• Seed-order confusion is essentially solved by centralizing PDA derivation in one helper and importing it everywhere. If derivation logic lives in one file and one file only, the seed order cannot drift between call sites.
• Discriminator mistakes go away with an explicit discriminator check before deserialization.
• Blockhash expiration is handled by fetching just-in-time.
• Compute unit overflow is catchable in CI if you track compute usage as part of your test suite.
• Version conflicts are tractable with explicit pinning and disciplined toolchain management.

These are the wins. The categories where, if I just adopt the practice, the mistake mostly stops happening.

Other categories remain genuinely hard. A development overview is candid: "Debugging is still one of the weakest links." Anchor macro errors can be opaque and hard to trace. There is still no visual debugger or instruction-level trace explorer for the kind of cross-program flows that make up the bulk of any non-trivial application. State drift during incomplete migrations remains genuinely hard to detect in advance. Mainnet-only intermittent failures — the ones that never reproduce in any test environment — are still where most of my bad debugging hours go.

It would be tidy to say the ecosystem will solve these next, or that I have a plan for handling each one. Honestly, I do not. Some of these are open problems that the broader community is also still working out. The best I can do for now is to know which ones are which: when I hit a problem that falls in the "largely solved" category, I know there is a known practice and I should adopt it. When I hit one in the "still hard" category, I know I am at the frontier and I should budget more time, log more state, and not expect a clean answer.

What This Changes About How I Work

The practical effect of seeing these five patterns is a small but real shift in default behavior. Before I act on a struct definition, I check what is actually on chain. Before I push a passing test, I think about which production conditions the test did not exercise. Before I blame my code for a strange error, I check whether my environment is the one I think it is. Before I assume a relationship between program and account is going to work, I make every part of that relationship explicit. Before I trust the docs, I check the version of the docs against the version of the chain.

None of these are heroic measures. They are small, almost boring discipline moves. The reason I keep needing to remind myself of them is that, in the moment, the cheap thing feels safe. The struct looks right. The test passes. The environment seems fine. The runtime should handle it. The docs say. Each of these is a small invitation to skip the verification step. The cost of skipping is zero today and unbounded next week.

The lesson I am drawing — not finished, still working through — is something close to: representations of a system are useful but never authoritative. The only authoritative thing is the system itself, observed under the conditions that actually matter. Everything else is a hypothesis. The mistake is treating hypotheses as facts.

I suspect, too, that part of why this lesson is so hard to keep is that it is uncomfortable. Treating my own reading of a struct as a hypothesis means I do not really know what the chain holds until I check. Treating my passing test suite as a hypothesis means I do not really know my code works until production tells me. There is a small, constant erosion of confidence required to live this way, and confidence is what makes me ship things. So the practice is not to abandon confidence but to put it in the right place: confident in my process for checking, less confident in any specific representation. That is a slightly different shape of self-trust than the one I started with.

Key Takeaways

• The same assumption error wears five different costumes. Layout assumptions, environment assumptions, runtime assumptions, documentation assumptions, and "it worked locally" assumptions are different surfaces of one underlying mistake: trusting a representation instead of observing the system.
• Silent failures are the real enemy. A wrong assumption that produces a clear error is a lucky outcome. The dangerous ones produce no error, vague errors, or errors that point in the wrong direction.
• Some traps are solved practices away from disappearing. Centralized PDA derivation, just-in-time blockhash fetching, CI compute budget tracking, pinned dependency versions, and explicit discriminator checks each eliminate a whole category of mistake.
• Some traps remain genuinely hard. CPI-heavy debugging, mainnet-only intermittent failures, and migration state drift do not yet have clean playbooks. Budget time accordingly, log more state than feels necessary, and do not expect tidy answers at the frontier.
• The fix is not better code; it is closer observation. Most "Solana bugs" turn out to be mismatched accounts, wrong commitment levels, stale data, or assumptions that quietly stopped holding. The remedy is reaching past the representation to the thing being represented.