How Engine Works
Many of you reading the announcement post wanted to know how Engine works, so I’m going to explain the high level concepts here. This is also very similar to how Steem Monsters, steemsmartcontracts, and steem-state work.
If you haven’t read it yet, here’s the Engine announcement post.
The best way to show how Engine and soft-consensus in general works (I think) is with an example. Alice wants to transfer 10 ENGN to Bob. She creates a transaction to the Steem blockchain saying ‘transfer bob 10’. A witness then includes that in the next block. That block is broadcasted to all Steem nodes and those Steem nodes send it to the Engine nodes. These Engine nodes see that Alice transferred Bob 10 ENGN so they subtract 10 from Alice’s balance and add 10 to Bob’s balance, actually completing the transaction. One could see how this could easily be expanded to include more features than just a token, e.g. SteemMonsters.
That is, Engine transactions are not verified or executed by Steem nodes, only by Engine nodes, which the next example demonstrates. This time Mallory wants to attack the network and send 1000 ENGN to her friend Lewis, even though she only owns 10 ENGN. Mallory creates a transaction to the Steem blockchain like Alice did, saying ‘transfer lewis 1000’. The Steem blockchain doesn’t verify this transaction and includes it in the next block, even though it is invalid (Mallory is trying to transfer ENGN that she doesn’t have). This is broadcasted to all Steem nodes, and finally reaches the Engine nodes. But these Engine nodes know that the transaction is invalid (they look at their internal ledger of balances and see that Mallory doesn’t own enough Engine), so they simply ignore it.
Invalid Engine transactions can be placed on the blockchain, but they will simply be ignored. This demonstrates that only Engine nodes verify and execute the transactions, but Steem nodes place them on blocks, deal with fees, maintain consensus, etc. The Steem nodes only need enough programming to be able to handle consensus and they can leave all the unrecognized transactions to be handled by other types of nodes such as Engine or Steem Monsters nodes.
Benefits
Scalability
Having seperate node types perform seperate operations (e.g. Engine nodes, SteemMonsters nodes, DLUX nodes) using Steem as a base consensus layer could be called a form of sharding or a form of sidechain, usually being called the latter. This helps Steem scale, effectively splitting consensus and features into seperate nodes and computers.
Security
If the Engine nodes have a bug or a problem, only Engine features are affected; the Steem blockchain cannot be harmed by an Engine hack. Same goes for Steem Monsters, steemsmartcontracts, DLUX, etc.
Agility
Adding new features to Engine does not require any change to the Steem source code. That is, there is no need to cooperate with Steemit Inc to add features and improve the blockchain, and there is no need to have an understanding of C++ and knowledge of Steem’s source code to contribute. That is, adding new features to Engine is much quicker to do than adding new features to Steem.
Community-Driven
Engine is a fully community driven project (I’m actually already asking for community feedback) because of no reliance on Steemit Inc as described above and a more accessible, smaller, and easier to learn codebase written in Javascript, one of the most-used languages in the world and definitely the most used by Steem’s developer community.
Other Projects
Steem Monsters
Steem Momsters runs on a very similar concept to Engine, except for one key difference — the node is not open-source. That is, only @aggroed, @yabapmatt, and the rest of the team are able to run a node. This means that Steem Monsters is not fully decentralized like Engine is.
steem-state
steem-state makes it easier to make DApps running on Steem using the methods described above. Engine uses steem-state, as well as DLUX, Chibera, and Hashkings.
steemsmartcontracts
steemsmartcontracts, developed by @harpagon uses similar concepts to Engine to implement smart contracts on Steem. Smart contracts are working, but the project is vulnerable to a few major problems due to the lack of fees for computation, ask me if you are interested in a more detailed explanation.
Why the ENGN Token Exists
Some of you may be wondering — why create a seperate token and not be loyal to Steem? The answer is that I would love to use Steem rather than the Engine token, but here is where Engine (and soft-consensus in general) has its limitations. The Engine DApp does not have the ability to manipulate Steem directly — no way to own or force transfer of Steem, but does have the ability to manipulate the ENGN token directly because it is part of the ‘shard’ or ‘sidechain’ that is Engine. ENGN is not intended to be a security to be bought and held, only a utility to act as monetary transfer in Engine. Because of this, we will only be holding a very tiny giveaway at the network launch to kickstart the token.
The ENGN token is meant to act as an extension to Steem, not as a competitor or scammy ICO coin. Plus, Engine will almost directly add value to Steem because transactions cost Resource Credits, which can only be acquired by holding Steem.
We would very much like to open source the steem monsters node software, and it will definitely happen at some point, but there are still some issues that we know of, and probably some more that we don't, that could cause multiple nodes to get out of sync / fork in certain situations.
We have limited resources and an unbelievably large list of changes and features, most of which are more important to keeping the business up and running than open sourcing the software, so we have to prioritize.
We are in the process of trying to raise funding now (something which is also quite time consuming) which should help us work on some of these items.
Also, please feel free to get in touch with me any time. I prefer Discord but anything works. I would also be interested in talking about your project and hearing your concerns with harpagon's project which we plan to use for some of our stuff.
Thanks for the info about SM open-source!
If I'm missing something, please tell me because I haven't done much research on the steemsmartcontracts project and might be missing something :shrug:.
My biggest concern with @harpagon's project is the lack of computational fees or computational limits per transaction. So someone could create a smart contract that on every transaction triggers a while(true) loop. Of course, there would likely be a limit put in place to stop that e.g. a timeout for 100 milliseconds for the transaction's computation, then all would be fine. But attacking this feature someone could create a more sinister contract, such as on a transaction something like this (obviously actually using the db api as in the code, this is just a proof of concept):
What this would do is break consensus among different nodes. Node A runs this for 100 milliseconds, at which point the loop had performed 10,000 iterations, leaving counter.storage at 10,000, while Node B, with a slower computer, only manages to reach 5,000 iterations, leaving its copy of counter.storage at 5,000. This would then break consensus between the nodes A and B. This wouldn't practically have any problems, because no smart contract wanting to be secure would use this, but it does mean that contracts have to be designed to maintain consensus.
Plus, with no computational fees, while(true) loop spamming would be easy because it would cost almost no RC, the transaction is nearly empty--which makes it theoretically less secure, but again, probably not practically. The only way to fix this problem would be to lower the threshold of compute time to something like 10 milliseconds so that the network could handle at maximum 30 transactions at maximum capacity per block, increasing the cost of attack. But low limits could introduce more problems.
What if there needs to be a 1,000 iteration loop in a smart contract? Node B is slower but Node A is faster, meaning that Node B halts transaction execution (it reaches the limit) and Node A completes the transaction because it has a faster computer, meaning that consensus is lost again, and this is probably the most practical and least theoretical flaw.
In all, I think steemsmartcontracts has a bit of theoretical flaws related to computation fees, but in general it's fine :).
thanks for your comments, these are indeed problems that we are aware of and that's why the mainnet will first not allow the deployment of smart contracts by the users (as the nodes are only protected against infinite loop for now), we will most likely open this feature once we are done testing the fees on the testnet.
For now, we are focusing on the contracts that will be shipped with the core code ;)
Hmm... any ideas on how fees could be done? Deterministic fees obviously couldn't work without a custom VM like the EVM, but maybe you could do some sort of compute time oracle?
Nice to see that that's something that you are aware of!
I read your most recent post, I'm interested in the proposals; I believe it's similar to how my project Engine will work.
But I'm wondering -- why the need to have seperate blocks and block producers, isn't that all handled by the Steem blockchain? e.g. couldn't you just stream transactions instead of having to seperate them into blocks?
I don't think that a fee on the computation time is necessary, at least if you have other forms of fees in place, the goal is to find a way to reach a consensus across the nodes and there are different ways to do that. (and that's why I built a blockchain/sidechain, these blocks will be used and verified by the block producers). I'm focusing on the core contracts now but once this is done, I'll go back to the fees/consensus implementation.
Regarding the proposal system, for now it's only accumulating tokens, I haven't yet decided on how they will be distributed but it will for sure use a stake based voting system.
I see with the fees and proposal system, but why do you need a way to reach consensus considering that Steem already does that for you? I’m just trying to understand — thank you for your time.
Basically, the Steem blockchain provides the consensus for the inputs but you need to have a way to reach a consensus on the outputs, If I take your example with the counters, if one node is somehow not able to reach 1000 loops, it will then provide invalid data, so you need to have a way to indicate that the transactions were (not) processed as they should have. Then you compare this output across the other nodes and try to reach a consensus. (but that's theory... for now I'm just thinking about a tool that will get the blocks from the nodes, compare them and expose the results of the comparison, that will easily show which nodes are not "doing their job" correctly so that you can vote them out if necessary. (as the nodes that are in the top 21 are getting tokens every time a block is produced)
Ok great; Stratos is doing a similar thing but I just don't call them 'blocks', I call them 'consensus checks'. Every 5 minutes or so every node (who opts in) posts a hash of their current state to the blockchain and compares that to everyone else's. So then you can audit whether the nodes are in consensus easily as well as tell if there is a problem with consensus while in testnet.
LOl did all of this become Steem-engine ? @shredz7 did ENGN become ENG ?
can you come talk to fyrtsikkena nd us on @steemspeak https://steemspeak.com about your non EOS STeemDAC ?
Very excited. Resteemed and hoping some more whales and devs see this! ;)
Posted using Partiko Android
Okay, this is very interesting, because some of my ideas basically relied on creating this exact thing. It's good to know that we won't all need to reinvent the wheel.
I thought this would be solved by having to spend resource credits.
But I guess not...
I also think it's strange that Steem Monsters created a centralized system. That's the main reason why I haven't really bought into it.
I've contacted the developers of SM, and it seems that they do want to open-source the node and that they do have it on their roadmap after tournaments, but the node is not designed well to be able to be open-sourced. They could design it so that it works without having trusted nodes, but it isn't designed so that it can at the moment. Here's part of a comment in response to this post by Yabapmatt:
You also said:
Because the computation itself is not done on the Steem nodes, you only have to pay in RC for the storage space that it takes up, not the computational power that it uses. All the Steem nodes are doing is storing the transactions on the blockchain, not computing them. Engine/steemsmartcontracts nodes are computing the transactions -- but they don't have the authority to charge transaction fees for computation.
Sorry if I didn't explain this well; tell me if you have any more questions.
No I think you explained it quite well I just don't really see the vulnerability. The client/server/node ignores invalid transactions and you can't spam invalid transactions forever because of RC costs... not to mention the stake it would require to do such a thing and the flags and reputation loss that would rain down on you for such an action.
No, but the difference is that in steemsmartcontracts you can define your own operations/contract functions, and things to do based on those functions. So you could define an function that, when you create the cooresponding operation, causes all steemsmartcontract nodes to go into an infinite loop or something similar.
Look at [this comment] for more details about how this vulnerability can be extended further as long as there are no computational fees.
You could use an account that has never posted -- even created through anonsteem because these transactions cost very little RC to avoid flags.
Ah nice thanks for clearing that up. My ignorance on the subject is great.
Not really, though. You understand a lot more than most people, and I just wasn’t explaining well.
This post has been included in today's SOS Daily News - a digest of all you need to know about the State of Steem.
Editor of the The State of Steem SoS Daily News.
Promoter of The State of Steem SoS Weekly Forums.
Editor of the weekly listing of steem radio shows, podcasts & social broadcasts.
Founder of the A Dollar A Day charitable giving project.
Congratulations! Your post has been selected as a daily Steemit truffle! It is listed on rank 7 of all contributions awarded today. You can find the TOP DAILY TRUFFLE PICKS HERE.
I upvoted your contribution because to my mind your post is at least 6 SBD worth and should receive 187 votes. It's now up to the lovely Steemit community to make this come true.
I am
TrufflePig, an Artificial Intelligence Bot that helps minnows and content curators using Machine Learning. If you are curious how I select content, you can find an explanation here!Have a nice day and sincerely yours,
TrufflePigInteresting and valuable work! Thanks for your efforts as they are the thoughts that need to proliferate in the ecosystem to become truly decentralized.
Posted using Partiko iOS