On customised distributed consensus - the "firmware" of blockchains

While working on something (a project named Equinox, in fact. Stay tuned.), I needed to re-think why a blockchain solution is needed at all, and to figure out how to actively use the strengths of a blockchain in a project without it being just a bolt-on addition, and unnecessary at that, as it is in many projects. This line of thinking got me to consider the role and use of consensus in blockchains.

Consensus is the somewhat loose and colloquial name for "integrity rules" as applied to blockchains. SQL databases have ACID, and within the "I" (for Integrity) part there are several mechanisms which can be used to check and enforce data integrity, such as primary keys, foreign keys, unique indexes, check constraints and the like. Consensus rules are like that, only used within the concept of the blockchain. They can be rather simple, like the UTXO rule in Bitcoin, or almost arbitrarily complex, such as with smart contracts in Ethereum. The latter case is worth a more detailed look. 

By having "live", Turing-complete code optionally attached to each transaction, there are in effect two layers of consensus code: the "hard" one which governs the structure of the Ethereum blockchain: the rules about how transactions look like, when they are valid, and how state is managed, and the "soft" one which is the somewhat emergent property of smart contract code. Note that Bitcoin only has the "hard" layer.

What I'd like to advocate here is the introduction of a middle layer, the "firm" consensus layer. Of course this is by analogy with "hardware <-> firmware <-> software", and the purpose of such a layer would be to introduce consensus rules which would be changeable, not a part of the platform code (i.e. not a part of the C++/Java/Go... code in which the blockchain itself is written, as is the case with "hard" consensus rules), but also more efficient and, more importantly, cheaper per transaction than doing everything in smart contract code.

Here's a description of the use case that sparked this idea: if we want to use a blockchain as a means of storing validated data, literally, I mean any kind of non-trivial validation, for example examining the data structure attached to transactions, comparing it to some business rules, etc., the only way to do that currently is to implement a system where there is a smart contract sitting on an address, and all data is sent to this contract via transactions, and this contract validates all the data as it seems fit. If the validation process is expensive in terms of CPU time, memory resources, etc., then - tough luck. Since this is the only way to do it, just bite the bullet and prepare to issue some expensive transactions. There is literally no way to "tag" arbitrary transactions (with arbitrary senders and receivers) as belonging to a certain type which enforces certain specific transaction validity calculations without the transaction being sent to a smart contract. 

Let's say I want to have a system which decides what the transaction fee is, based on the data attached to the transaction. Not by the size of the data (as is common), but the actual content of the data. Maybe I want this data to be in JSON format and I'm looking for the deepest nesting level, or I am just mean and want to set the transaction fee as a percentage of the value transferred instead of the transaction size in bytes. Those are just examples, of course. In such a system, I'd like to implement a special type of smart contract which would handle (or pre-process) all transactions which are tagged in a certain way, and I'd like this special smart contract (the "firmware") to be especially fast, always loaded in memory, JIT compiled, etc., so it is optimally efficient in what it does. This special smart contract could (and probably should) be expensive to store in the blockchain, but it probably shouldn't automatically affect the fees required for those transactions (though this point is debatable).

Such a thing would literally sit just on top of the "hard" consensus rules, so every transaction with an appropriate tag is first processed by the "hard" rules, then by the "firm" rules, then the smart contract code in the transaction itself is executed (if present) as the final, "soft" layer.

The main benefit of such a system is that it would enable the use of blockchains for complex business-oriented purposes where transactions (and data attached to them) need to conform to certain, relatively stable and in some cases immutable rules, but it would be either too expensive to do everything with ordinary smart contracts, or it would be undesirable to send all such transactions to the smart contract address for validation.

Private blockchains can of course do all of that in the "hard" rules since they have complete control of the platform, so this idea is mostly applicable to public ones or to blockchain frameworks.

What do you think?

(why yes, I was a database researcher before becoming a blockchain researcher, and my hobby is electronics - does it show? :D )