“Consortium blockchains” (e.g. DPoS & Tendermint) can’t Internet scale

A recent comparison¹ of Tendermint versus EOS/Steem’s delegated proof-of-stake (DPoS)² doesn’t elucidate the most critical flaw that is shared by all these “consortium blockchains” (including Tendermint, IOHK’s Ouroboros³, Red Belly Blockchain, and Byteball), which delegate consensus to a bounded, permissioned set of validating block (or stability point⁴) producers (aka delegates or witnesses).

Consortium blockchains are any attempted (and the impossibility of a perfect) solution to the Byzantine Generals Problem (aka Byzantine agreement) which has bounded synchronous finality of irreversibility⁵; and thus mathematically must have a bounded+permissioned delegate set subject to a ¹/₃ liveness threshold with ²/₃ safety margin. Whereas, Satoshi’s proof-of-work has only probabilistic (never absolutely final) irreversibility but allows unbounded asynchrony, unbounded+permissionless set of block producers, has no liveness threshold⁶, and yet with only ¹/₂ (aka “50% attack”) safety margin.

---

¹ _{The linked blog is the perspective of the Tendermint project, and presumably originates from a Github issues discussion which I participated in and which Dan blogged about. And note my refutation of one of the DPoS-shill comments there, which contains significant additional points to supplement this blog.}

² _{DPoS was originally invented by Daniel Larimer in April 2014 for the Graphene blockchain of Bitshares. I participated in the community discussions of the design at its inception. DPoS is the blockchain consensus technology adopted by many altcoins, including Bitshares, Steem, EOS, Lisk, Ark, etc..}

³ _{Although Ouroboros’ proof-of-stake has delegation as an optional configuration of their technology, the claimed “provably secure” aspect requires a majority of the stake to be online always; and thus must realistically be delegated. Additionally, consistently high transaction throughput volume and low confirmation latency requires delegation to validating block producers with high performance servers and network connectivity.}

⁴ _{Byteball’s permissioned set of validators form consensus about “stability points” in the DAG.}

⁵ _{Irreversibility means for example inability to be double-spend or more generally the inability of a block to be orphaned by a subsequent fork.}

⁶ _{A 50+% attack can orphan minority hashrate blocks and censor any or all transactions; thus simulating some effects of a liveness threshold attack. Except that the majority has to sustain the expenditure on its hashrate in order to sustain the attack, so in theory the liveness could be recover in the future, unlike for consortium blockchains which require a community hardfork process for voting out non-responsive delegates to recover from a liveness threshold attack. The Math of safety & liveness section explains why it’s unsafe to replace non-responsive delegates without a (potentially contentious) community hardfork process. Also there’s the possibility of egregiously reduced rate of block production due to PoW difficulty adjustment attacks.}

Permissioned liveness = overlords

Consortium blockchains have a liveness threshold which is at most ¹/₃ of the delegates— a minimum safety requirement which apparently Daniel Larimer still doesn’t understand⁷ even after I explained it to him. Thus (_{^{being worse than ephemeral liveness attacks such as a DDoS attack on 33% of the delegates}}), by electing non-responsive delegates which exceed the liveness threshold, a colluding malevolent 33% of the stake can permanently and irreparably shutdown the blockchain⁷ (_{^{pending a potentially contentious community hardfork process to replace non-responding delegates}}) unless there exists is a coordinated 50+% majority of the stake opposing in every stake election. Some colluding whales with 33% of the stake could enter short positions betting the exchange price of the ledger’s token will decline, shutdown the blockchain for a period of time sufficient to close their short positions with profits, spend the profits to buy more tokens extremely cheaply on exchanges, then stop the attack and profit on the rebound in the price of the tokens. Thus over time the colluding whales will own eventually the majority of the tokens and they will be forever unopposed in stake elections, i.e. constituting oligarchy control.

As overlording whales they can extract higher and higher rents such as by voting for higher and higher compensation for themselves as the delegates, extracting the maximum that the market will bear. Or if that’s a bit too blatantly obvious, they can extract rents in obfuscated ways, such as front-running their yo-yo manipulation of the token’s exchange price by pretending they’ve been DDoS attacked. Or they can devise a scheme such as Steem’s voting paradigm, for minting money supply that appears to be for a noble cause but mathematically can only end up aggregated in the wallets of the sockpuppets of the whales. Or a highly obfuscated Rube Goldberg scheme for further concentrating the premeditated sneakyfastmine to the whales.

Wait. This consortium blockchain scam is even worse when one digs down the rabbit hole.

Actually the real-world case is the blatant liveness attack isn’t likely ever necessary because the overlords must take control in any sufficiently obfuscated way.

Even if there is an honest majority of the stake, it is a well known flaw of voting⁸ that it is impossible to coordinate an intelligent vote result from the tyranny of the masses, unless the majority of the stake is held by a small number of highly informed whales. But if a small number of whales are in control, then as sure as flies-to-honey, they have an incentive to maximize their extraction of the aforementioned parasitic (overlording-aggregates-all) rents.⁸ And even at the inception as has been recently exposed for EOS⁹ when they’re also pocketing profits from selling FOMO (_{^{fear-of-missing-out, left holding the}}) bags to greater fools.

Even if we assume the majority of the stake is honest and not colluding, unless they’re colluding such that they control the delegates they elect, the honest voters have no way to objectively determine which of the stake is voting for delegates which are attacking the liveness, because the attacking whales can offer sockpuppet delegate candidates into every election. Thus for consortium blockchains to not be evil overlords, not only would we have to presume that the inviolable facts about political-economics⁸ are not inviolable, but the honest majority of the stake would also all have to agree and coordinate on the reputations of delegates which each voter subjectively presumes aren’t sockpuppets of the attacking whales. Nope. This subjective relativity of reputation is what makes voting not free and a winner-take-all for the whales which can extract the most rents (as predicted by the theory⁸), which is what I explained to Daniel Larimer in April 2014 at the inception of his invention of DPoS. Winner-takes-all political economics means that those whales who try to not extract the maximum possible rents will lose to those who do.

Thus, these “marvelous” consortium blockchains re-accomplish the evil of the governments we already have, coupled with Visa-like servers run by the obfuscated overlording whales.

And what if the governments rubber hose the overlords. Bigger fish eat smaller fish (^{_{even Satoshi politely slapped Dan back down in his deserved subservient role}}) in this centralized control outcome of political-economic game theory⁸ of elections and governance. Governance = top-down centralization = more Visa/Windoze, inept monopolistic, rent extracting failure = inability to scale decentralized.

Well Satoshi’s proof-of-work has an analogous design weakness in that as significant revenue for miners comes from transactions fees as the protocol block reward declines and transaction volume increases, the incentive for the consensus to converge is lost and requires a colluding oligarchy, which of course will also extract maximum rents in either blatant ways such as constrained block size causing higher and higher fees, or obfuscated trickery for extracting rents.

---

⁷ _{DPoS is designed to side-step the liveness threshold, but the Math of safety & liveness section explains why it’s unsafe to replace non-responsive delegates without a (potentially contentious) community hardfork process.}

⁸ _{See the Pitfalls of Proof-of-Stake section, Paul Sztorc’s explanation of voting political economics, and Mancur Olsen’s The Logic Of Collective Action.}

⁹ _{Such as the apparently intentionally designed gaming of the EOS token sale and the premeditated sneakyfastmine of Steem both of which presumably put or will put 50+% of the Steem and EOS tokens in the wallets of a few whales.}

Overlording doesn’t Internet scale

The Internet grew so fast because no party could extract the maximum rents that the market could bear. This was in large part of benefit of the Internet’s decentralized protocols, such as BGP routing and TCP/IP. Open source prospered because it enabled companies to invest in mutual code development without fear of closed source extortion of maximum rents and failures/negligence of the centralized parties in control.

The third party ecosystems will be justifiably wary about investing in consortium blockchains; because they must become overlorded. Viral adoption will be underwhelming as is evident by Steem versus early stage Facebook growth, given much of the vaunted recent growth in traffic may be misleading.

Math of safety & liveness

Daniel Larimer asserts that DPoS creates irreversible blocks when ²/₃ (^{_{actually plus one or rounded to a whole number}}) of the delegates have acknowledged the block in the lineage chain of blocks. In DPoS, delegates take turns producing blocks in round-robin order. Yet he also asserts that — instead of the network halting requiring a (potentially contentious) community hardfork to safely recover, as must be the case for all consortium blockchains which desire to have safety — DPoS can side-step the ¹/₃ liveness threshold, elect new delegates, and recover without the blockchain shutting down.

Notwithstanding the discussion between Dan and the Cosmos/Tendermint Github community that all¹⁰ consortium blockchains are subject to loss of finality (i.e. ambiguity about forks) when the network is partitioned such that the bounded synchrony requirement¹¹ is subverted, then as correctly pointed out in the aforementioned comparison blog, by not enforcing the liveness threshold then DPoS enables Byzantine unsafety such that quorums for conflicting blocks are possible enabling an ambiguous forkathon in the case where the blockchain should have been halted due to a liveness threshold violation.

The naive reader (and the neophyte Dan Larimer!) will be confused as to why the non-responding delegates can’t be replaced by a stake election. There are three reasons why such a naive assumption is mathematically impossible:

• If the liveness threshold has been exceeded, then any blocks which confirm the result of a stake election aren’t objectively final due to the math of Byzantine fault tolerant agreement (consensus) as detailed below, i.e. an unbounded number of conflicting forks can be created.

  If instead record the results of the election offchain, then we’re executing a (potentially contentious) hardfork, thus not avoiding what Dan alleged that DPoS avoids! I sort of expect readers to work this out in their head, but I guess I realize I need to be more explicit in addressing all the various incorrect logic that naive minds can wander to.

• Who is voting over and over for non-responding delegates (unbounded into the future) may not be objectively knowable nor defeatable as explained in the Permissioned liveness = overlords section of this blog.

• If more than ²/₃ of the delegates are colluding, it’s mathematically impossible to determine which delegates are non-responding, because they can produce an unbounded number of forks wherein in each fork a different set of them are non-responding, yet all of the forks exceed the liveness threshold. Thus it is ambiguous which are non-responding, because they all are! Yet in any given fork, only some are; thus, it’s not objectively provable which are non-responding. The naive non-mathematical mind can’t grok that.

What we realize above is an underlying generative essence which is that Byzantine fault tolerance is a relativity problem and when the mathematical thresholds are exceeded, then all objectivity is lost. Stake voting can’t overcome that mathematical generative essence.

I explained the math quoted as follows from the yet unpublished rough draft for the white paper of the decentralized ledger technology I designed for an altcoin project I’m currently developing:

First considering the scenario which doesn’t require Byzantine fault tolerance such that validators of blocks never vote for a conflicting transaction such as a double-spend nor does any other type of Byzantine fault occur, then given two attempted elections for a pair of consecutive blocks of transactions, the minimum number of voters which mathematically must commit to both elections is fₛ = 2T - N - 1, where T is the minimum quorum size for each election and N is the size of the set of validators. Two instances of quorums of size T (one for each block election) minus N - 1 available voting validators (from the perspective of any validator thus - 1 for excluding self given a validator can trust himself), yielding the number fₛ which must be duplicated in both instances. Thus the minimum quorum size T ≥ (N + 1)/2 where 2T - N - 1 ≥ 0.

Whereas, in the Byzantine fault tolerant case wherein malevolent and/or asynchronous (i.e. caused by random, ambiguous propagation ordering) validators can vote for blocks that contain conflicting transactions, the minimum number of voters which commit to both elections fₛ = 2T - N - 1 must be greater than the number of excess voters not needed to form a quorum fₗ = N - T, so that a quorum for a conflicting pair of blocks can’t exist because there aren’t enough uncommitted voters to vote for it, i.e. T ≥ N - fₛ. Refer also to §Byzantine Consensus Algorithm: Proof of Safety of the Tendermint Github project. Thus T ≥ (2N + 1)/3 where 2T - N - 1 ≥ N - T, fₛ is the safety margin, and fₗ is liveness. This notation is taken from §5.4.3. Comparison to centralized voting on page 15 of the Stellar white paper. This result is mathematically equivalent to N = 3f + 1 where f = fₛ = fₗ, is analogous to the N = 3m + 1 generals for m traitors result for the Byzantine Generals Problem.

Liveness fₗ in this context is the maximum number of unresponsive nodes allowed for the system to not stall all quorums. Safety margin fₛ is the minimum number of validators which must commit to (i.e. vote for) a pair of blocks to prevent ambiguous ordering of blocks. Even if only a single election would ever be held for a set of signing keys which represent the set of voting validators, the Byzantine fault tolerance applies when there isn’t trust of a pre-designated centralized tally entity, because voters can irrefutably claim that an epoch (i.e. synchronous bound) for a quorum instance expired and thus they signed a new vote for a new epoch. It is even irrefutable that a trusted centralized tally did not receive a vote. C.f. §2.2 Proofs and verifiable evidence and what I explained to Dan in 2013 about unprovable accountability for network communication.

Some systems may opt for more safety margin at the detriment of reduced liveness by choosing a larger minimum quorum size T.

Dan’s DPoS design thus accepts arbitrary behavior (if the system were not coordinated by overlording whales¹⁰) instead of halting the blockchain.

Note however that research published in 2007 reveals it’s possible to enable some limited forms of non-arbitrary behavior with a liveness threshold greater than ¹/₃. Yet afaik consortium blockchains don’t avail of this research insight. It may not be immediately obvious how apply this research insight to the double-spend or other general capabilities enabled by a chronological order preserving ledger, but apparently in Q4 2016 before I became aware of the research, I independently invented a facet of that “fork* consistency” principle for my decentralized ledger design.

Btw, I had elaborated with an example to help visualize the aforementioned mathematical derivation (^{_{employing the + and - prefix and postfix notation from the Tendermint Github white paper}}):

Marbles in Jars Example

The proof in the Byzantine agreement case is easy to visualize with 3 voters who can each vote with marbles of one color representing their signing key: red, white, and blue. Given 3 jars representing ballot boxes for 3 elections on the consensus ordering of any two of them (wherein the jars could represent blockchain blocks), two marbles can be placed in each jar without any color voting more than twice― i.e. no voter has cheated yet the consensus is ambiguous. Declaring any of the jars as the first, results in two choices (aka “forks”) for which jar is the next. One jar has red and white, another red and blue, and another white and blue. This is why the proof requires that the excess of the quorum be less than ¹/₃ (aka “-¹/₃”). So by adding a voter with green marbles so that smallest minority reduces from ¹/₃ to ¹/₄ which is thus -¹/₃, the ambiguity is resolved because the green marble can only be placed in 2 of the 3 jars― again presuming no voter cheated to vote more than twice. Thus +²/₃ instead of ¹/₂+ is required for quorums in the presence of possible malevolence (and/or honest ambiguity due to random propagation delays) given an asynchronous network, otherwise the ordering of the quorums can’t be proven.

Note if any 3 of the 4 marbles are colluding, i.e. ²/₃ or more (aka “+²/₃”) of the voters are malevolent, they can vote for as many conflicting quorums (forks) as they wish and it will be ambiguous which of the 4 marbles is cheating, because given 3 sets of 3 jars, a different one of the 3 cheaters can vote in each set. And the honest voter has to vote on the fork which the quorum is extending, because otherwise not voting is indistinguishable from unresponsive. It can’t be irrefutably proven that the 3 malevolent marbles were cheating and colluding because they can each claim that other one became unresponsive; thus voting for more than two quorums became necessary. And the honest marble voted more than twice also.

---

¹⁰ _{And Dan’s afaics utter nonsense about the purpose of ²/₃ safety margin for bounded synchronous finality, because he’s obviously ignorant of the math of safety & liveness given he was evidently only analyzing the risk of forks due to networking partitioning (i.e. the mathematical point ostensibly never even entered his consciousness) and not in the case of the math of accepting ambiguous quorums for replacing non-responsive delegates in the case where the liveness threshold is exceeded. Hence his construction of a false/irrelevant dichotomy strawman. And the idiot community that praises Dan apparently bought into the thus irrelevant implication of some 99.999% relationship to the attributes of proof-of-work. As correctly pointed out in the comparison blog, Dan doesn’t seem to understand that 1000s of uncoordinated validators can’t correct for the mathematical errors in design of what is supposed to a Byzantine fault tolerant coordination algorithm. Because. Duh. They’re uncoordinated. Dan’s faith in his design is bolstered by the fact that his DPoS systems are run by highly coordinated overlording whales and thus Byzantine fault tolerance issues are more or less limited to bounded network propagation hiccups, so thus we cycle back to the title of this blog as the actually realized flaw.}

¹¹ _{And Dan doesn’t even understand that his DPoS system relies on bounded synchrony. How the heck does Dan think his round-robin rotation of block producers functions if there is not a time-out on how long to wait for the prior block producer to propagate a block before the next one produces a block.}

DPoS’ Rube Goldberg transaction fees

The zero transaction fee gimmick of DPoS creates centralization and overlording around the necessary decentralization of funding which can for example lead to a DDoS attack on the forward progress of ledger.