Decentralized blockchain data growth challenge

Now, that could well be the most sexy title ever, right? Nah, probably not, but it is about a subject that is difficult to sell in the nude. Raw vast amounts of data, how to store them on a decentralized trustless network effectively...

9 years ago, comments: 3, votes: 38, reward: $0.38

Now, that could well be the most sexy title ever, right? Nah, probably not, but it is about a subject that is difficult to sell in the nude. Raw vast amounts of data, how to store them on a decentralized trustless network effectively.

It needs to be fast and secure at the same time. Every amount of data added is a transaction, that needs to be stored in a block. Where it finally gets locked into a constantly growing blockchain. There is the first challenge, where nature tends to use cycles, blockchains want to grow endless.

Purge data

Now this could be solved by purging data. Like you throw away financial paperwork every five year, or so. If we have a limited amount of space available we will compromise on what to hold on to.

So, from my point of view, a blockchain that is able to purge itself, trustless fast and safe, will have a future headstart. Maybe this could even lead to a Proof Of Purge {POP} block reward. At a certain point I would only care for the amount of value I have in a certain address. And I would know, like the rest of the network, that there is a purge block to proof it. Going a long way back proved by many confirmations.

This is free thinking theory, so don't ask ready steady cook solutions in this post. And while these thoughts lingered on in my mind for a while I had an epiphany. Well, it felt like that...

Huge amounts of data to be handled

A lot of challenge comes from data growth and keeping up putting it all into blocks. Steem ON Graphene {SONG} can handle lots if transactions every 3 seconds. SteemIt dot com counts on that the decentralized network can keep up with the posts, replies, votes and so on. For you as an active Steemer it would be best if you even did not notice the technology behind it all. Because if you did start to notice time-outs too often that could mean the blockchain got choked. It is okay if it would now and then, there are many reasons why hick-ups in transactions may occur.

Graphene 2.0, the blockchain motor driving Steem and Bitshares, will be able to address this challenge more efficient. In a sense this will be done by spliting up the total amount of data to be handled in memory at once. This is a good step forward. Still, it could limit social media on a blockchain, in dynamics, on the long term.

Sidechains enter the field

At SteemIt dot com it the data created by posts gets stored in the Steem on Graphene blockchain. To put it simple, there will be text stored mainly. With some tricks even binary data could be stored. And don't worry won't bother you with the technical details.

Still, I don't think Steemers would be happy users if megabytes of data got stored on the blockchain every post. That is why images, video's and audio are linked to an external source. But text is still stored on the Steem On Graphene blockchain.

And that has it's limits. Voting, replying and so on. Only two possible payout rounds, while a post could have real valuable information for years to come. Like this one, hahaha. ;-)

In come the sidechains. And maybe in a way you would not have thought about it. The theory would be that there is one mainchain. This mainly handles small data transactions. Able to create big blocks with many transactions. At say hundreds of thousands every 3 second cycle. Let's asume this will be Graphene 3.0, the parallel blockchain revolution.

The mainchain would handle the quantum scale of transactions. It would be very fast and slick in it's operstion. It's real strength would not only come from purge blocks, but also from the sidechains.

Sidechains would define the platform

If Bitcoin were to be used, that would be the mainchain. But, as far as I can see, it was not build to function with sidechains. The NXT 2.0 blockchain, known as Ardor, might be fit for the job. But, as I'm a bit biased towards Graphene, that one might have the best change of solving a lot of future blockchain challenges. Hopefully completely free and open source. Using sidechains as a parallel data handling solution.

The way this could work is that the information that needs to be stored gets a sidechain. Meaning text gets one, images, video, sound, binary, all different sidechains. A pure text based micro-blogging system could be build quite easy that way.

Transactions of value still would be handled by the mainchain. Also creating and destroying value. And things like Proof Of Purge would keep a kind of natural proces going. Nothing needs to be recorded for ever. If so, then better cast it in stone.

Pay for keeps

Imagine you would want secure something in a blockchain 3.0 cloud. At first it could maybe generate value. If publicly available and valued. A sidechains could keep track of that. Every cycle, as set by the algorythm for the specific plaform. It could be done logarithmicly or by fabonacci math. What I mean is that it could work like 1, 2, 3, 5, 8 cycles of potential payout, times 30 days. But no matter how, that would be done on a unique sidechain. The text itself on a sidechain of it's own.

Pure text data would not add up to extreme numbers fast. Not even when used for complete books. But it is a different story for binary, image, sound and video storage. The amount of data growth caused by these types of information could bring a network to it's knees fast. Unless...

If you want to keep it, add value to it

Some things in life you want to keep as long as you live. Because it has value to you, not specific meaning 'money' related. But maybe you would not keep those kind of things on a public ledger im the first place.

Maybe you can imagine that there are things you wanted to keep in the blockchain 3.0 cloud. As a backup maybe secured by the trustless Graphene 3.0 technology. And you might want to keep it there longer then a purge cycle. Then you'd simply pay the network for keeping it another purge cycle, or maybe more rounds.

Software that would let you use this would keep track of the next purge cycle. And alert you, or maybe evrn auto-backup the data automaticly on a local storage.

Something like SteemIt might be used to create value, earned by drawing in attention. Where the earned payout can be used to keep the post available for more curration rounds. While 'burning' some value in the proces. Or it can be used to pay for data storage on a connected sidechain. Securing it until the next purge cycle. And the price to be payed would depand on demand and the reputation of the network.

Are you still with me? ;-)

Parallel blockchaining could it be?

Time to wrap this sharing of ideas that were lingering on in my head for a while. It is about solving some challenges any current blockchain will have. Simply stating that the growing internet bandwidth, datastorage and computing speed will solve it seems very unrealistic to me. So I looked at nature and came to the conclusion that cycles are key in everlasting movement.

In this post I thought outloud about how cycles could be put to good use in blockchain technology. Not a complete solid solution, but an inventation to let it wander through your mind.

But I'm almost pretty certain that sidechains will have a key role in the future blockchain 3.0 technology.

And my bet is on Graphene.


photo cc-by-sa @oaldamster

This is a 50-50 payout post