Solana came within roughly 20 million SOL of losing transaction finality after a routing failure at Teraswitch’s Miami facility disconnected almost 29% of the network’s staked tokens, according to Marinade Finance. The incident lasted about 33 minutes and did not become a blockchain-wide freeze, but it revealed how a network can appear widely distributed at the validator level while remaining vulnerable to a small number of internet infrastructure providers.
The failure began with a bad route originating at Teraswitch’s Miami data center early Wednesday, August 12, Marinade said. Connectivity problems then spread across facilities in London, Amsterdam, Frankfurt, Singapore and Tokyo. North American operations remained online, while approximately 90 validators were affected. Many did not reconnect for the duration of the incident because backup systems failed to take over automatically.
The routing problem itself was resolved in roughly 10 minutes, with traffic restored by 4:16 a.m. UTC. But the disruption continued to affect validators for longer, producing a 33-minute period in which Solana’s finality margin became uncomfortably narrow.
That margin is one of the most important safety properties in a proof-of-stake network. Solana stops finalizing transactions when more than one-third of the stake participating in consensus becomes delinquent. Marinade estimated that the network came within roughly 20 million SOL of that threshold.
The incident therefore was not simply another temporary connectivity problem. It was a demonstration of how physical hosting locations, cloud providers, internet service providers, autonomous systems and routing policies can create correlated points of failure beneath a decentralized validator set.
For Solana, the question is no longer only how many validators secure the chain. It is whether those validators can continue communicating when one company, one route, one region or one operational dependency fails.
A chain can remain online without remaining final
The distinction between availability and finality is central to understanding the incident.
A blockchain may continue producing some blocks while still failing to finalize transactions. In practical terms, that means users can see activity on the network, wallets can display pending transactions and applications can remain connected to RPC endpoints, but the chain may be unable to provide the economic certainty that makes a transaction irreversible.
Finality matters because businesses and protocols use it to determine when an action is complete. A decentralized exchange may wait for finality before releasing an asset. A lending protocol may treat a liquidation as settled only after the underlying transaction is finalized. A payment processor may not credit a merchant until the transaction can no longer be rolled back. An institutional trading system may suspend withdrawals when the network cannot provide reliable settlement.
Solana’s consensus design relies on supermajority agreement among validators weighted by their staked SOL. The one-third threshold is not a general measure of whether one-third of machines are offline. It is a measure of how much stake is unavailable or unable to participate in the consensus process.
That distinction makes concentration especially significant. If 90 relatively small validators disappear, the impact may be limited. If those validators collectively represent nearly 29% of staked SOL because they share a connectivity provider, the same physical event becomes a systemic risk.
Marinade said one network operator identified as AS2032 accounted for more than a quarter of the SOL securing the network. That exceeded Solana’s own safety limit for a single connectivity provider. When the operator’s dependent validators went offline together, the network’s remaining resilience buffer became very small. Other providers lost a further 14 million SOL of stake during the same window.
The affected validators reportedly lost 333 SOL in rewards. Marinade said those losses would be covered through validator bonds, a mechanism intended to compensate operators for missed income. That remedy is useful for the validators themselves, but it does not address the larger consequences that would have followed a failure of finality.
A validator can be reimbursed for lost rewards. A trading firm cannot be easily reimbursed for a missed liquidation, a delayed payment or a forced position unwind caused by a frozen settlement layer.
The hidden geography of decentralization
Crypto networks often describe decentralization using visible metrics: the number of validators, the distribution of stake, the share controlled by the largest operators and the number of entities participating in governance.
Those measures matter, but they do not tell the whole story.
A validator is not just a cryptographic identity. It is a computer connected to the internet, usually through a data center, cloud provider or specialized hosting company. It depends on power, cooling, physical security, upstream transit, routers, software, monitoring, domain name systems and technicians who can respond when something goes wrong.
Two validators may be operated by separate legal entities and use different identities on-chain while relying on the same hosting company. Ten validators may run in different cities but share an upstream network provider. A global operator may appear geographically distributed while maintaining the same configuration, backup policy and failover assumptions across every location.
This creates a difference between logical decentralization and operational decentralization.
Logical decentralization describes what can be observed on-chain: who controls stake, how many validators are active and how consensus voting power is distributed. Operational decentralization asks a different set of questions:
- How many validators use the same data-center provider?
- How many depend on the same autonomous system?
- How many share a route to the public internet?
- How many rely on the same cloud region or backup service?
- How many can fail over without manual intervention?
- How many have tested that failover under real network conditions?
- How many are operated by teams with genuinely separate infrastructure?
The Teraswitch incident showed why these questions cannot remain secondary. A validator set can look dispersed in a dashboard while being concentrated in the facilities and networks that allow those validators to communicate.
The physical internet is full of shared dependencies. Providers buy transit from one another. Data centers host multiple “independent” operators. Internet routes can change rapidly in response to outages, configuration errors or commercial decisions. A problem at one facility can affect customers in several countries if those customers use the same network architecture.
The consequence is a form of correlated failure. Instead of validators failing independently, they fail together.
That is a more serious risk than ordinary uptime statistics suggest. If a validator has a 99.9% individual availability rate but shares its critical infrastructure with 20% of the network, the relevant question is not only whether that validator is reliable. It is whether the entire group can disappear at the same time.
Why backup systems did not protect the network
The most consequential detail in the episode may be that many affected validators remained disconnected because backup systems did not take over.
Redundancy is often presented as a simple solution: run a second server in another location, maintain a backup internet connection or keep a standby validator ready to assume operations. In practice, failover is a system of engineering decisions, and each one can fail.
A standby machine may not have the latest ledger data. It may lack the correct private voting key or require a manual approval process before activation. A secondary site may depend on the same provider as the primary site. Two facilities may use separate local connections but converge on the same upstream carrier. A backup validator may be intentionally kept offline to avoid double voting or conflicting state.
There are also security trade-offs. Validator operators must protect consensus keys. An automatically activated backup can improve availability, but poorly designed automation can create signing risks. If two machines become active at once, an operator could produce conflicting votes or trigger penalties. If the backup process requires a person to confirm each step, it may be too slow for a rapidly developing outage.
Reliable failover therefore requires more than purchasing a second server. It requires synchronized data, tested key management, independent connectivity, monitoring, clear activation rules and repeated exercises.
The industry has often measured validator resilience through uptime and missed votes. Those metrics are useful, but they may overlook the difference between independent failures and shared failures. An operator that recovers quickly from a local machine crash may still be exposed to a provider-wide routing problem if every validator uses the same network.
A stronger standard would examine recovery under a defined failure scenario. For example: if a hosting provider disappears for 30 minutes, how many validators can resume voting through a separate provider, and how quickly? If the primary route is withdrawn, does traffic switch automatically? If a data center loses connectivity but remains powered, can the operator reach the validator through an out-of-band management channel?
These are traditional reliability questions, but they have unusual importance in a public blockchain. The failure of one customer’s server is an operational inconvenience. The correlated failure of a large share of voting stake can threaten the settlement of an entire financial ecosystem.
The difference between a validator and its dependencies
A validator is commonly treated as the relevant unit of decentralization because it is the entity that votes in consensus. But a validator is better understood as a stack of dependencies.
At the top is the validator software and its operator. Beneath that are the machine, storage and operating system. Beneath those are the data center, power supply and physical network. The data center connects to local providers and upstream transit networks. Those networks interact through the Border Gateway Protocol, which determines how traffic moves between autonomous systems. The validator may also depend on cloud services, monitoring systems, time synchronization and remote key management.
A failure at any layer can impair consensus participation.
This creates several forms of concentration:
Hosting concentration: Many validators may lease servers from the same facility or company.
Network concentration: Validators may use different data centers but rely on the same autonomous system or transit provider.
Geographic concentration: Operators may distribute machines across cities but cluster them in regions exposed to the same weather, power grid or regulatory event.
Software concentration: A large share of validators may run the same client version or configuration, making a software defect more likely to affect them together.
Operational concentration: Different validator identities may be controlled by the same company, team or automation system.
Key-management concentration: Multiple sites may depend on a common signing or authorization process that prevents independent failover.
The Teraswitch event primarily exposed hosting and routing concentration, but the framework applies to every layer. A network that diversifies data centers but not software clients remains exposed to a client bug. A network with multiple hosting providers but one dominant cloud region remains exposed to a regional outage. A network with independent validators but a small number of RPC providers may remain usable for some applications yet appear unavailable to ordinary users.
This is why decentralization cannot be reduced to a single number. It is a multidimensional property that must be evaluated across control, capital, geography, software and connectivity.
What a finality halt would mean for users
A breach of the one-third threshold would not automatically mean that funds disappear or that the ledger is permanently damaged. It would mean that the network could no longer reach the supermajority needed to finalize new transactions.
The immediate experience would vary by application.
A wallet might submit a transfer that remains pending. An exchange could pause deposits and withdrawals because it cannot determine whether the transaction has settled. A decentralized exchange could continue showing quotes while trades wait for confirmation. A lending protocol might face uncertainty over whether collateral movements, repayments or liquidations are complete.
Automated systems would be forced to choose between two undesirable options: accept transactions without finality or stop processing altogether. Accepting them could expose the system to reorganization or inconsistent state. Stopping them could freeze users and create a backlog that must be processed after the network recovers.
The effects could also be asymmetric. A market maker with access to multiple venues might hedge elsewhere, while a retail user’s transaction remains stuck. A sophisticated protocol could pause new activity, but a less prepared application might continue accepting deposits or issuing credits against transactions that have not settled.
The economic damage would not necessarily be visible in a blockchain explorer. It could appear as wider spreads, missed arbitrage, delayed payroll, failed liquidations, collateral shortfalls and emergency risk controls.
Marinade compared the threat to Solana’s roughly five-hour outage in February 2024. The comparison is useful because it illustrates how quickly a technical interruption can become a business event. The longer a high-throughput chain remains unavailable or uncertain, the more applications need to coordinate their response. Some must preserve solvency, some must manage user expectations and some must prevent transactions from being interpreted differently by different systems.
Solana’s approximately $4.3 billion in decentralized finance value locked, as cited in the reporting around the incident, raises the stakes. DeFi protocols are not merely applications waiting for a website to reload. They are interconnected financial systems whose contracts can trigger liquidations, collateral transfers, trades and repayments automatically.
A finality halt would pause the economic clock for those systems, but it would not necessarily pause external markets. Prices could continue moving on centralized exchanges or other chains. When Solana resumed, protocols and traders might face a sudden wave of catch-up activity, stale assumptions and competing attempts to execute transactions.
Why the risk is larger than Solana
Solana’s architecture makes the issue particularly visible because the network is optimized for high transaction throughput and fast confirmation. That performance supports trading, payments, consumer applications and other use cases that depend on rapid settlement.
But the infrastructure problem is not unique to Solana.
Every proof-of-stake network depends on validators communicating through physical infrastructure. Ethereum, Avalanche, Cosmos-based chains and many other networks have their own forms of hosting, cloud and geographic concentration. A network may have thousands of validators and still rely heavily on a handful of providers.
The specific failure threshold differs across protocols. Some chains may continue processing with a smaller active set. Others may stop finalizing when a particular portion of voting power becomes unavailable. The terminology also varies: a network might remain live but lose finality, or it might experience a halt in block production, depending on its consensus mechanism.
The underlying lesson is consistent. Stake distribution and infrastructure distribution are different datasets.
A delegator can move stake away from a large validator, but that may not reduce physical concentration if the alternative validator uses the same hosting provider. A foundation can encourage more operators, but adding identities without adding independent infrastructure may produce only cosmetic decentralization. A network can boast global geographic coverage while remaining dependent on one routing architecture.
This matters for investors and institutions evaluating blockchain settlement. Risk teams typically examine smart contracts, bridge exposure, governance and custody. Infrastructure correlation deserves a similar place in the framework.
If a protocol settles billions of dollars on a network, it should know not only how many validators exist, but how many independent infrastructure clusters support the voting power behind them.
What delegators can, and cannot, see today
Delegators are often asked to help improve decentralization by choosing smaller validators. The advice is directionally correct but incomplete.
On-chain data can reveal a validator’s stake, vote performance, commission rate and sometimes its operator identity. Public dashboards may show data-center location or autonomous system information. But those disclosures are not always standardized, current or independently verified.
A validator may list a city that identifies its legal or operating headquarters rather than the location of its actual machine. An operator may use multiple providers without documenting them. Two apparently separate validators may share an owner or infrastructure. Conversely, a single operator may run genuinely independent deployments across multiple networks and regions.
Delegators also face a practical trade-off. The most established operators may offer stronger security practices, better monitoring and more reliable performance. Smaller operators may improve stake distribution but lack the resources for robust failover. The goal is not to choose the least professional operator; it is to reward professional operators that also diversify critical dependencies.
A useful delegation policy would consider at least four categories:
- Performance: Does the validator consistently vote and remain online?
- Operator independence: Is it controlled by a distinct organization rather than a cluster of nominally separate identities?
- Infrastructure diversity: Does it avoid the same provider, region and network path used by a large share of stake?
- Recovery capability: Has it documented and tested failover?
The last category is the hardest to evaluate. Operators rarely publish detailed architecture for security reasons, and public declarations can become outdated. Yet without some form of transparency, delegators are forced to optimize using incomplete information.
That creates an opportunity for ecosystem tooling. A validator-risk dashboard could map stake not only by identity, but by hosting provider, autonomous system, geography, software client and operator group. It could show how much stake would become unavailable if a particular provider or region failed.
Such a dashboard would not eliminate risk. It would make risk legible.
The case for infrastructure-aware incentives
Solana’s ecosystem can respond through incentives as well as rules.
Delegation programs, liquid staking protocols and institutional allocators could incorporate infrastructure diversity into their policies. Rather than distributing stake only according to validator performance or commission, they could assign a premium to operators that use independent providers and maintain tested recovery procedures.
Liquid staking protocols are particularly important because they aggregate large amounts of stake and influence where that stake is delegated. Marinade’s role in identifying and communicating the concentration risk illustrates how such protocols can become infrastructure stewards, not merely yield products.
A liquid staking protocol could set concentration limits for hosting providers, require standardized infrastructure disclosures and periodically rebalance delegations when one provider becomes too dominant. It could also distinguish between true diversity and superficial distribution, for example, separate validators in different facilities that still depend on the same transit network.
There are risks to formalizing these incentives. If operators are required to reveal too much information, they may expose security-sensitive details. If delegators move stake too aggressively in response to a dashboard, they could create new concentration elsewhere. If compliance becomes expensive, smaller operators might be pushed out, reducing competition.
The solution is likely to be graduated disclosure. Operators could report provider categories, regions and redundancy status without publishing sensitive network diagrams. Independent auditors or ecosystem organizations could verify claims. Metrics could be expressed in ranges rather than precise locations where necessary.
The most important principle is that infrastructure diversity should be treated as a measurable contribution to network security.
Should Solana impose hard limits?
The incident raises a difficult governance question: should Solana enforce a hard cap on the amount of stake associated with one hosting or connectivity provider?
A cap could directly address the danger highlighted by the event. If no provider were allowed to support more than a specified percentage of voting stake, a single outage would have less chance of threatening finality. Protocol-level or ecosystem-level limits could also make concentration visible before it reaches a crisis point.
But enforcement is complicated.
The network may not be able to determine reliably whether two validators share a provider. Operators could conceal infrastructure relationships, use resellers or change routes dynamically. A provider might serve as a backup for one validator and a primary host for another, creating ambiguity about how its risk should be counted. Strict limits could also penalize operators that have built high-quality infrastructure and force stake toward less reliable alternatives.
There is a difference between a protocol-enforced rule and an operational safety limit. The first can affect validator eligibility or consensus. The second can be maintained by delegators, staking protocols, foundations and monitoring organizations.
Solana may not need a single permanent cap if it can create a transparent warning system. If any infrastructure cluster approaches a defined percentage of stake, delegators and operators could receive alerts. If the cluster exceeds the threshold, liquid staking protocols could automatically reduce exposure or require a mitigation plan.
The precise mechanism should be developed with operators and security researchers, but the principle is clear: a stated safety limit is useful only if the ecosystem can measure compliance and act before the limit becomes a live threat.
The institutional implications
For institutional users, the incident changes the meaning of operational due diligence.
A company integrating Solana payments or settlement may previously have asked whether the network had enough validators and whether its average confirmation time met business requirements. It now has reason to ask how the validator set is connected.
Financial institutions may need provider-diversity policies for blockchain exposure similar to those used for cloud computing. A bank rarely wants every critical system in one cloud region. A market infrastructure firm may require independent connectivity, redundant data centers and tested disaster recovery. Blockchain integrations will increasingly face the same expectations.
This could influence how institutions choose networks. Performance remains important, but performance without resilience can produce a poor risk-adjusted outcome. A fast chain that occasionally loses finality may be less useful for a payment or trading product than a slower chain with stronger operational independence.
The business opportunity is substantial. Infrastructure providers can differentiate by offering multi-provider validator deployments, independent routing, automated failover and audited recovery procedures. Cloud companies may develop specialized blockchain hosting products that reduce correlated risk rather than simply renting compute.
Monitoring firms can build early-warning systems that combine on-chain voting data with internet routing information. Insurance providers could price validator and protocol coverage using correlated infrastructure exposure. Custodians and exchanges could include network-health conditions in their transaction policies.
In this sense, the near-miss may accelerate a new layer of crypto infrastructure: not just tools for running validators, but tools for measuring the dependencies that connect them.
A roadmap for stronger resilience
The next step should be a systematic infrastructure audit rather than a one-time redistribution of stake.
Solana ecosystem participants could begin by constructing a confidential but independently verifiable map of validator dependencies. The map should include hosting providers, autonomous systems, geographic regions, software clients and operator relationships. It should calculate the amount of stake exposed to each failure domain.
Operators should then be encouraged, or required by major delegators, to document failover procedures. Documentation should answer practical questions: How long does recovery take? Is the backup synchronized? Is the signing process safe? Does the backup use a separate provider and route? Has the procedure been tested recently?
Testing matters because theoretical redundancy often fails under pressure. An operator might discover that the standby machine has stale data, that credentials have expired or that the backup route is not actually independent. Controlled exercises can expose these weaknesses without waiting for a live outage.
The ecosystem should also improve public reporting. A dashboard could show:
- Stake concentration by hosting and network provider.
- The estimated stake exposed to a regional or provider outage.
- Validator recovery times after past incidents.
- The percentage of validators with verified multi-provider failover.
- Software-client and operating-system concentration.
- The number of stake clusters controlled by the same operator.
These metrics should be interpreted carefully. Publishing a list of providers does not guarantee that their failures are independent. But transparent measurement would allow the network to identify dangerous trends earlier.
Finally, the industry needs to treat recovery as a shared responsibility. Validator operators control their machines, but delegators influence stake. Liquid staking protocols influence allocation. Foundations shape standards. Infrastructure providers control routes and facilities. Application developers decide how systems behave when finality is delayed.
No single participant can solve correlated risk alone.
From near miss to design principle
Solana avoided a network-wide finality halt because the routing problem was corrected before the affected stake crossed the critical threshold. That outcome should not be mistaken for proof that the existing architecture is sufficient.
The incident provided a warning at a useful moment: the network remained operational, the failure was limited in duration and the ecosystem could study the exposure before a more severe event. The cost was measured in lost validator rewards rather than frozen markets. But the narrow margin demonstrated that resilience can change quickly when a common infrastructure dependency fails.
The broader lesson reaches beyond Solana. Decentralization is not finished when stake is spread across many validator identities. It must extend into the data centers, network paths, software clients and recovery systems that allow those identities to participate in consensus.
For a blockchain supporting billions of dollars in financial activity, infrastructure redundancy is not a technical luxury. It is part of the settlement guarantee.
The next generation of blockchain competition will therefore be judged on more than throughput, fees and confirmation times. Networks will also need to show that their performance survives the failure of the companies and connections beneath them.
Solana’s near-miss makes that challenge visible. The opportunity now is to turn the lesson into better measurement, stronger incentives and infrastructure that is as decentralized as the ledger it supports.