It doesn’t matter how fast or cheap a blockchain is: these attributes mean nothing if it isn’t also decentralized. This, after all, the entire value proposition of blockchain: an independent record that no entity can falsify, censor, or opaquely amend. It’s no coincidence that the world’s most valuable blockchain networks – Bitcoin, Ethereum, and Solana – are also among its most decentralized.
Base layer decentralization is relatively easy to measure using benchmarks such as number of nodes, geographical distribution, governance framework, and the diversity of developer commits. But things get more complicated once you attempt to gauge the decentralization of the applications operating on these networks.
Because dapps are required to connect to off-chain and third-party infrastructure, including oracles, conventional databases, and indexing services, there’s a risk of the properties that make decentralized apps so valuable being eroded. Which is why decentralized storage is so valuable but so hard to provision at scale due to the constraints of blockchain architecture. But recent innovations across the multichain landscape have raised hopes that decentralized data storage may soon be cracked, solving one of the last key problems to delivering everything onchain without relying on centralized gatekeepers.
Decentralized storage is essential for blockchain applications because it reinforces decentralization, security, censorship resistance, and user control – values that are central to blockchain. There’s little point in using a decentralized network to route data if the data itself is stored on centralized servers that could be shut down at the flick of a switch.
Despite this, a large number of blockchain projects are still reliant wholly or partially on centralized cloud storage due to its performance and scalability. It’s used because it’s typically a cheaper and faster option than decentralized solutions and because most blockchain networks severely limit the amount of data that can be stored onchain.
The solution to cloud storage is to assign data to decentralized storage networks such as IPFS (InterPlanetary File System), Filecoin, Storj, and Arweave. Files are split into smaller chunks and distributed across multiple nodes in the network. Each node stores only a portion of the data, and no node has the complete dataset – a unique cryptographic hash attached to each piece of data that ensures its integrity. For instance, if a file’s content changes, the hash will change, meaning data can’t be altered without detection. When users need to access the data, the chunks are retrieved from various nodes, verified, and reassembled.
So-called first generation solutions such as IPFS score highly for decentralization but fare less well when graded for capacity and speed. They’re fine for dapps looking to store and retrieve a 10,000 PFP collection, but struggle when being asked to spit out huge databases of information on a dime: think AI training data or generative images and video.
The challenge for the next generation of decentralized storage solutions is to deliver virtually unlimited data on demand and at low cost. It’s taken time for blockchain developers to solve this problem, but they’ve gotten there through a combination of perseverance and ingenuity. Or to put it more bluntly, trial and error.
While data delivery networks are free to make their layers as fast and inexpensive as their tech will allow (without compromising on the all-important decentralization), they must grapple with the limitations of the blockchains they connect to. We can call this “the last mile problem.” As a result, the most promising dapp data solutions tend to be optimized for a specific blockchain or blockchain language in use on a series of interconnected chains.
On Solana, for example, the project looking likeliest to succeed with cracking this challenge is Xandeum, whose scalable storage layer for Solana aims to form the decentralized hard drive its high volume dapps demand. Xandeum Buckets is the name given to its decentralized file system that has the potential to hold exabytes of storage – multiples more than any blockchain dapp is ever likely to need.
Unlike conventional decentralized storage networks, Xandeum Buckets aren't stored on every validator node. Instead, storage is assigned to a separate network of provider nodes, tasked with ensuring data integrity without overburdening the network. It’s a clever way of increasing data responsiveness while still retaining a fully decentralized network.
Other data networks have taken alternative routes to scaling while provisioning dapps with the data they need to power onchain services. SQD, for example, is on a mission to make blockchain data from multiple networks available through a single web3 network. For enterprises who simply want blockchain data and are less concerned with maximizing decentralization, SQD Cloud provides a fast indexing solution with sub-second latency. While fine for feeding data to ccentralized platforms, this solution is less suited to dapps for reasons outlined above.
As blockchains become faster and the number of dapps and use cases they support expands, the demands placed on data delivery networks will only intensify. As a result, it’s not simply enough for data layers to be capable of meeting the needs of today’s dapps: they need to be able to accommodate the vast amounts of data that the next wave of data-hungry, AI-powered dapps will require.
What’s clear is that general purpose L1s and L2s are not the solution for storing exabytes of data: it needs to be hosted on dedicated networks that have been optimized for speed and throughput. The trick for data networks is to find the sweet spot in which data is replicated across multiple nodes, ensuring high availability and redundancy, but whereby it can be delivered with sub-second latency – wherever and whatever the request.
If there’s one thing the history of home computing has taught us, it’s that storage that is deemed acceptable today will be unacceptable in the near future. The race is thus on for data networks to stay ahead of the blockchain dapps and their insatiable demands for more data.