Gridcoin Superblock Dev Update
This will just be a quick followup to my previous superblock issues post. A further explanation, an apology and a way forward.
Magnitude Boost
In Gridcoin version 3.5.1.7 business logic was added to superblock verifications to ensure that the superblocks were valid, presumably by avoiding large project dropouts. This meant that if the average magnitude of all the reaserchers in the block was less than 10 the block would be ignored. This worked fine for a very long time, but as the network grew so did the competition. More crunchers meant less magnitude per cruncher which, after 1.5 years, brought the average magnitude down below 10 and superblocks could no longer be created. As a workaround to this every researcher's magnitude was increased by 35%. This is referred to as the magnitude boost. Everybody is getting overpayed, but at least everyone is getting paid paid and the magnitude unit will make sure nobody is getting overpaid (correction by @dutch).
The workaround, or the problem, came at a very inconvenient time. You see, the real fix is to stop checking these thresholds at all and let the network reach a consensus on its own. There are already project count limits in place so if too many projects fall out then no superblock will be created. However, in order to remove those checks we need to release a mandatory upgrade, something which requires a 2 weeks notice to the exchanges. Since we already had an upcoming mandatory in form of the V8 stake engine it didn't make sense to release two mandatories when V8 was "so close".
We failed to properly communicate all of the above which caused a lot of confusion and frustration. I apologize on behalf of all of the devs (I think) for that. I think we can do much better, see Lessons Learned.
The proper fix for the issue has been added to the upcoming Gridcoin release. When the network switches to V8 blocks superblock average limits will no longer have an effect. The boost will be removed along with this and everyone's mag will be back to normal.
Lessons Learned
What we have taken with us from this incident is that we, the developers, need to be much more communicative when something like this happens. We are actively working on issues as they arise but we tend to assume that others will seek us up for information, and that's something that should not be necessary. The next time something like this happens I will, if I remember, try to make sure that either me or someone else quickly updates the community with information as it arrives.
- An initial post describing the current event. "We have noticed that peoples' beacons are disappearing and are currently investigating the issues and possible solutions", or something similar.
- A more thorough explanation. Root causes,discovered workarounds and a solution plan.
- A post-mortem with historical explanations, a detailed description of the problem and its fix, lessons learned (similar to this post), how to prevent this in the future etc.
With that approach it will be much easier for the community members to get a feel for what's going on and it's easier for them to get involved themselves. Note that due to security issues it might not be possible to delve deep into the issue in the "second" post but we'll try to explain as good as we can.
@ravonn, thank you for the update. I am consistently impressed by your willingness to step up and cater to the requests of the community.
I am not a code developer, but I have found myself on the development side of many entrepreneurial projects I've lead up in my lifetime. Often a community struggles to understand how difficult it is to maintain development of a project while also serving as a public relations spokesman. Indeed, they are very different skill-sets and often they are conflicting interests. It's easy to assume the community is just not interested in the "nuts and bolts" of the project.
You are doing a fine job. If ever this gets to be too burdensome for you and there is a need for a specified public relations person, please feel free to contact me. I am fairly good at putting dev-speak into layman's terms and as you know (I am ZenMercenary on Slack) I am fairly well plugged-in. Feel free to ask if you need help.
That is actually something we need. You follow the discussions on Slack so you probably see the days of text that goes into a "Yeah, we have an issue" post. I think we should have at least one or to persons being able to distill what we discuss and present it to the public. If you're up for joining I'm all ears :)
Not having a Mag component for PoS sure does help to simplify the explaining of stake weight. It would be nice to have some modifier based on work though, even if it exponentially degrades as Mag increases to prevent someone throwing loads of compute at the coin and mining all te blocks out.
ravonn, thank you for the update it's really appreciated by the community.
I do however think that you shouldn't have to apologise for previous misdemeanors in which you were not involved.
The dev or devs previously responsible for these should make it their responsibility and own up and apologise, especially if they are still in the development team and still responsible for any development matters.
If they are not members of steemit and an apology has been issued elsewhere, then this should be posted or linked to in here by some one who has a steemit account on their behalf.
Thank you for all your work and I would like to reiterate that everyone in the dev team should be paid for the time they spend code cleaning in order to ensure that we don't have unnecessary dead code or code which may be a security risk in the future.
Nah, I'm not responsible for the original code but I am a part of the dev team. Since people rely on what we do we must become much better at information forwarding. That we can do, we just have to get into the grooves and make it a habit. The current coin contribution structure is actually quite new so the routines aren't there yet. Anyway, my apology was for us sucking donkey porn at providing information, but that's about to change.
Man ravonn, I know you have broad shoulder and you are bearing the burden but my previous remarks are still valid.
It is unreasonable of those involved to abrogate responsibility.
Referencing the Gridcoin licensing on GitHub:
I have seen a few people asking about this in IRC and Slack. Thought it was worth clarifying that while we have higher mag, the dynamic nature of the mag unit caused that to drop so the payout remained level with what was the intended daily mint.
Oh right, I forgot about that.
Good to see one of the lead devs communicating with the community. Thank you!
If only an explanation/summary like this was included both in the communications to stakeholders, but also the documentation of the upgrade itself.
I dont want to speak for others, but personally, I know not having this information and transparency has made my experience far more frustrating to have to deal with (particularly given the work required to compile and upgrade systems), generated more mystery surrounding my understanding of the technologies in use currently but more importantly any temporary provisions being released until a broader solution is/can be implemented.
As someone who has been dabbling with boinc for a few years, and made an initial start with gridcoin about a year ago, my first attempts didnt keep me engaged because cryptocurrencies were a foreign concept, and then the patience required to put the code in place and join the blockchain didn't seem like it would be worth the trouble nor did I get a sense of direction to entice me to participate.
In hopes of not coming off too vague in my criticism and experiences, and to try to summarize: It was a few months of nearly daily engagement, often in small 15-30 min/day tasks, from reading up on the developer/github code conversations, forums, steemit, or external blogs/news/youtube/media, that I was starting to feel like I was starting to get a sense of what I missed (before joining the project), the current state of affairs, and somewhat of a vague sense of where things might be heading. The investment in time and effort felt worth the hoops and frustrations of unfamiliar financial/commerce and technology, but highly beneficial and practical in both geeky/code interests, but also real world global market trends and inevitable disruptive technologies.
Some of the temporary solutions being implemented (ie: reporting snafus of projects and inconsistencies in cross-resource reporting (when comparing for example gridcoinstats to dc, etc.) but while this article outlines why it was necessary to implement the temporary magnitude boost, it lacked providing any rationale, explanation (and "in plain english" summary of what it means), which started to make me lose confidence in the competency of those implementing these changes, but more importantly, i was losing confidence in the integrity of the system as a whole because now the apples of yesterday could no longer be compared to oranges of today, let alone the bananas just planted.
These last few upgrades to the wallet were also. a bit more painful than upgrades past because I began keeping two staking wallets intentionally... one with the current version, and a secondary with the upgrades/releases I'd jump to as quickly as possible. Unfortunately, the new releases generally were less successfully in that post compilation and attempts to rejoin the block and staking/mining efforts failed, and I was glad I was then using dual-wallets to feel confident I'd be both maintaining progress, but not losing out from unintentional (user-error included) or network wide issues that are to an extent to be expected as growing pains.
Hope my thoughts arent too vague but make sense to some of you who may have gone through or are going through similar experiences.
While this was a dev update and I tried to make it as straight forward as I could, I'm more than happy to clarify it as best as I can.
The boost was necessary to push the researchers' average magnitude above 10 in order to get the network to accept the superblock. We could just remove the 10-mag average requirement to begin with, but that would require a mandatory upgrade or the network would fork (those with the requirement would go one way and those without it would go another way).
A mandatory upgrade is like flipping the switch on a huge steampunk machine, think Howl's Moving Castle. Every exchange have to be taken into maintenance mode to avoid forking, hence the 2-week notice, so if a mandatory can be avoided it's a good thing. Since a mandatory upgrade was coming with the V8 stake engine anyway it was hard to justify another one that would just live for a week or two of exchange uptime. The magnitude boost removal is included in the 3.6.0.1 mandatory upgrade and is set to kick in at September 7 00:00 UTC.
Please let me know if I'm still not making any sense.
Quality post, thanks a lot for this!
Congratulations @ravonn! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOPCongratulations @ravonn! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOPAre you threatening me!? My bungole will not wait!