RE: About Hivemind, the recent Steemit outage, and "Fake Nodes"
Thank you very much for these insightful explanation.
A particular transaction, malformed in a very specific way, appeared on the chain in a new block. Hivemind didn't know how to parse it correctly, and crashed.
This sounds a bit like an attack scenario that definitely needs to be analysed. I hope it will be solved in such a way that attackers will no longer be able to exploit this "issue" and crash Steemit.
Other independent nodes would have needed to apply the same fix, available from the Hivemind github repo, in order to bring up their own Hiveminds again.
Since it affects all independent condenser installations, a patch is absolutely necessary here so that all other nodes can also be updated. However, I have not seen that the repo on github has been updated accordingly.
There's no facility for the end user to change that without deploying their own version Condenser (the Steemit.com front end codebase).
I ran the condenser locally in developer mode during the downtime of Steemit. I assume that the API requests are directed to https://api.steemitdev.com. Obviously the requests could be replied to there anyway or there is also a dev-hivemind version.
I'd also like to hear more about this malformed transaction, what it was, how it got onto the chain, and whether this points out a vulnerability that needs to be fixed.
While I strongly agree that there's no security through obscurity, and even though @ety001 has already patched the issue, I'd rather not elaborate on the particular transaction here until I've had time to audit the codebase a little deeper. I hope you understand!
Just to add that this tx was completely valid from the chain perspective -- no issue with steemd.
I think it probably has been fixed. I noticed this in github the other day: remove special char from hive role title.
Thanks. I overlooked the commit. Probably because it has not yet been pulled into the branch.
That is the fix, yes :)
The pull request on the repo is still pending. I believe it won't be approved until it meets the Continuous Integration checks.
However, I miss-spoke regarding operators needing to update from Github. Rather, most are running Hivemind in production using this docker container.