Utopian reputation and scoring systems
Utopian is constantly changing and sometimes it may be a bit difficult to keep track of everything that is going on. Utopian announces all changes so that everybody can learn which new features have been implemented, which features had to be edited and what has been fixed. To stay up-to-date one should follow @utopian-io, which is the main account of Utopian, and/or join our community on Discord.
A team of experienced people are there to answer your questions, as well as help you with your inquiries. Here you can meet many other people with the same interests as you. Meeting new people is very important in Utopian, which brings us to the main point of this post.
The recent changes in Utopian included a new questionnaire, scoring system and reputation system. Let's Score Contributions! and Welcome The New Voting Distribution for the Utopian Bot And Moderation Rewards! were published introducing these features. Unfortunately, many people were not able to understand them fully at first. The goal of this post is to explain the system in more details, or at least in a different way.
I'll start with explaining the reputation system and then the scoring system. They are tightly coupled together so it is important to understand them both properly.
Reputation system
Utopian used to use the same reputation as the one used on Steemit and other Steem-based apps. We have observed that their reputation system is not suited for our purposes because a user's reputation can easily be increased by a few valuable votes, even though they have been on Steem for a short time. I say this because the reputation is global across all Steem apps, which means that one can increase their reputation without Utopian. This brings some problems if we were to depend on such a system with further community impactful decisions.
Therefore, Utopian introduced its own reputation system. Everyone has their own reputation rank that gives them an influence within Utopian. This influence is important whilst scoring contributions. And the score of your approved contributions has an impact on your reputation growth. Can you now see how those two concepts are connected?
You need to provide quality contributions for your reputation to grow. In other words, you need to provide contributions that get approved by the Utopian moderation team and the higher the score those contributions get, the higher the impact they will have on your reputation. What about the contributions that existed before the scoring system? A default value is used for the calculation, so you can be sure your previous contributions are taken into account, but we will get to that later.
There is another way to increase your reputation/influence in Utopian and that is with Steem Power delegation. Delegations are not mandatory but very welcome as they help the community to grow, and it is the sponsorship of all involved people that allow Utopian to give such high rewards. The rewards have been very generous in my opinion.
| Rank | Level | Delegated SP | Influence |
|---|---|---|---|
| Newbie | 0 | - | 0 |
| Beginner | 1 | 100 | 5 |
| Advanced | 2 | 1,000 | 10 |
| Expert | 3 | 5,000 | 15 |
| Pro | 4 | 10,000 | 30 |
| Ninja | 5 | 15,000 | 45 |
| Guru | 6 | 20,000 | 60 |
| Hero | 7 | 25,000 | 75 |
| Legend | 8 | 50,000 | 90 |
| Elite | 9 | 100,000 | 100 |
Moderators have an influence of 60 (Guru) by default because of their position in Utopian. Community managers, Utopian staff and a few acknowledged people were given the Elite rank which means they are the most influential. Nevertheless, it does not mean that their decision is the only one and final. If the community sees that their decision is not the correct one or there are many other people involved in the scoring, the result may be overruled by the community.
How is the reputation calculated?
We introduced the ranks, but how can you know which rank you will be given? Let's forget the delegations for a while and focus on the first way of increasing your reputation.
The reputation system is dynamic. This means that if a contributor wants to increase his or her reputation, they must contribute regularly and make sure their contributions are of high quality. This is the reason why one should be aware of their current rank and what affects its current state.
First of all, the reputation score is computed. You can easily find the piece of code in the repository of Utopian's API. The algorithm takes all contributions submitted by a user and either adds or subtracts a portion of the score according to the number of approved and rejected contributions.
| Category | Divisor |
|---|---|
| Default | 3 |
| Development | 1 |
| Graphics | 2 |
| Analysis | 1 |
| Documentation | 1.5 |
| Translations | 2 |
| Tutorials | 2 |
| Video tutorials | 1.5 |
| Copywriting | 1.5 |
We also need to introduce a new category-wise constant, the divisor. This constant is called divider in the code. Many categories have the divisor explicitly defined. If that is not the case, we use the default divisor.
while ((post = await c.next()) !== null) {
const postScore = post.json_metadata.score;
const divider = customScoreDivider[post.json_metadata.type] ?
customScoreDivider[post.json_metadata.type].score_divider :
defaultScoreDivider;
if (postScore === undefined) { // for contributions before the scoring system
if (post.json_metadata.moderator.reviewed === true) {
score = (100 / divider) + score;
}
if (post.json_metadata.moderator.flagged === true) {
score = score - 100 / (divider);
}
}
if (postScore !== undefined) {
if (post.json_metadata.moderator.flagged === true) {
score = score - (100 / divider);
}
if (postScore >= 0 && post.json_metadata.moderator.reviewed === true) {
score = score + (postScore / divider);
}
}
}
We can see that people contributing to development and analysis categories can grow their reputation the fastest. On the other hand, if their contribution is rejected, their reputation score decreases the fastest as well.
The algorithm takes old contributions into account too. The default value is the maximum score of 100 that is divided by the category's divisor.
As you can see, the system has 10 levels. That's why the calculation of the reputation score is important; it has an impact on the contributors' ranking. Utopian currently takes the reputation score of the "best" contributor, which is essential in computing the ranks of all other contributors.
const scoreImpact = (score / topScore) * 100;
const levelImpact = Math.ceil((scoreImpact * (levels.length - 1)) / 100);
ScoreImpact is the scaled score reputation of each contributor, levelImpact is the level of a reputation rank, and topScore is the reputation score of the top contributor. So is it possible to know how much you need to level up? Not exactly, since the top contributor is not publicly known but you should be able to understand the principles of the algorithm.
Scoring system
We decided that we want to involve the community in the decision making. The scoring system is a good example of that as we expect experienced contributors in Utopian. The vote no longer depends solely on the decision of one moderator, but now everyone with a rank higher than Newbie can have an impact on the vote that could be given to the contributor.
Every user on Utopian can score a contribution. They can do this by answering the category specific questionnaire that determines the eventual score of the contribution. It is highly recommended that the user writes a comment and upvotes the contribution if they can see that the contribution is beneficial for the project.
We could say that the scoring system contains multiple polls — let's call it a "multipoll" system — where a person can give all his or her votes to a particular answer. The number of votes one person can give in this poll is predefined by the influence they have.
In the future, we might introduce a minimum score that you need to reach in order to receive a reward. The reason for this is that there currently aren't enough contributions, so Utopian does not spend all of its daily voting power as intended.
Demo
Let's assume that we want to score a bug hunting contribution. We will see the following questions.
Remember when I talked about the multipoll system? See, take each question as a separate poll where we are interested only in the answer with the most votes. Imagine that your influence is the number of votes you can put into the answer.
Let's also assume that there are 3 people that scored this contribution and their influences are 60, 60 and 10 respectively. This means that if all 3 people vote for the same answer, its maximum summed influence — the number of votes in our poll system — is 130. This would mean that at least 131 votes are needed to "overrule" their decision. However, there are also cases where 130 votes are enough to change the outcome. We look for the answer with the highest influence, or as we can say the highest number of votes. If there is more than one answer with the same summed influence, the first one of them is picked, which means the answer that gives the higher score.
We can also have a case where one of the scorers with the influence of 60 picked one answer and the other two, with a summed influence of 70 picked another one. The answer picked by those two scorers will become the most rated answer for the given question.
When will I be upvoted?
First of all, we mustn't forget that Utopian's votes are not promised and guaranteed. We had a long period when the votes were given to (almost) everyone whose contribution was approved and that has changed. We expect quality work that is not "trivial". This does not say much, right?
You see, the Utopian vote is valuable so we try to find contributions that deserve such a reward. Sometimes the contribution is just a small fix that saves the project a lot of time, but all the work is just changing one line of code, then it is not really enough to apply for the reward.
There are lots of similar cases when the contributor is of huge help to the project, but in that case, they should accumulate more of such small contributions and submit them at once if the specific category allows it, which is almost always possible.
The voting queue
Since the votes are not guaranteed and the scoring system exists, the voting queue has come into existence. The first and most basic requirement is that the contribution must be at least 2 days old to enter in the voting queue. Every contribution should be able to gather enough score during these 2 days. However, the contribution can be scored as long as it is not upvoted by Utopian or paid out. There are 2 basic scenarios on how to get into the queue.
Made by @andrejcibik
A moderator reviews the contribution
If the moderator checks a contribution and sees that everything is according to our rules and guidelines, the contribution holds value to the project and the open source movement, then the contribution is approved. The approval is not enough, though, as it is only one of the requirements.
Every contribution has to be scored by a person with some influence for the contribution to be eligible for the upvote. Currently, all approved contributions are also scored by the moderator that reviewed it.
A moderator has not reviewed the contribution yet
In the case that the moderator was not able to review the contribution, you can get into the queue with only the community scoring it and without the "permission" of the moderators. You will need a score of 80 or more and the total influence of all scorers must be at least 60.
Some uncertainty exists in this case as well. If it is found out that the contribution did not meet all expectations and requirements, it may still be rejected and unvoted after that.
What does it mean for you?
As a contributor, your main aim should be to deliver quality work that helps the project grow, promote it and teach other people what the best ways of utilizing it are.
The new system counts on you to spend time networking, meeting new people and scoring contributions of other Utopian users as well as upvoting quality contributions. The principles are not different from Steemit posts, just the tools that are used for achieving the best outcome are a little bit different.
If you are able to provide the needed work and publish a nice report within the Utopian post, you can be almost certain that you can reach the upvote. And that is what we want as well. We want to see a growing community where each member can help with different things.
Or if you just want to chat with Utopians, you can find us in Discord.
Very informative post... Thanks for let us know the true fact
will be great..
I may be one of those people who have been acknowledged. hope. :D
Hi podan, I meant the previous advisors in this case and I meant the Elite rank in particular. I personally appreciate all the work our previous members did for Utopian but I do not think that making "an exception" is the right way. You are one of few people who stayed in the Utopian community and made contributions for projects and I love your work, for example the logo for Amos.
We are working on ways how to incentivize the community more, which we believe will also support their progress of levelling up.
hehe .. yes, utopian has some many moderators before. and it's hard to make exceptions. but, Utopian has 4 previous supervisors. maybe 4 people get exceptions. :)
I've actually talked to Elear about this and the conversation got lost while we were busy with important issues. I'll bring it up again and see what he says.
I will wait. Hehe. Thanks brade.
You should get a dedicated Rank: elite designer ;-)
Hhh. I really hope. :D
How about Bug hunting?
If the divisor is not specified for the category, the default divisor is used.
Thanks espoem for all these clarifications. I'm so glad that you work so hard on quality contributions. I still strongly hope that there will come time for quality translation contributions as well.
What if a high quality contribution has zero scoring because no one scored it and has not been reviewed by mod which means It will not enter the voting cycle?
I think scoring system is missing too many things.
You see, the contributions are meant to be useful to someone. Some contributions are useful only to the project, some are useful also to the open source community in general, and some are useful to the community only. In every case, there are people who should be able to see the quality of the contribution.
Utopian has opened the doors to other people with deciding who can be voted and how high their reward can be. Since Utopian is built on top of Steem blockchain, the decision can be done only within several days.
I can see few cases when this can happen. There are many contributions so it may be overlooked by the readers, which means they will not score that contribution, or there is no person familiar with the project that could actually score the contribution.
Anyway, moderators currently score reviewed contributions, so this should not happen. A quality contribution can be discovered only when it is read. We are not able to predict which one is the good one and which one is worse just by reading the title.
As I mentioned it in the post, the networking and growing the Utopian fan base is important in this system. We are not looking only for the numbers, we are looking for meaningful comments and feedback too.
Which things do you mean are missing?
In my opinion current readers or even utopian users isn't enough or prepared for this system and any human can make mistakes, We can't achieve decentralization without justice.
Current scoring system can be abused in many ways by multiple accounts for low quality contributions or even plagiarism.
This means that system can be abused, Why you're waiting to be abused and waste power by unvote when you can use an algorithm in utopian through artificial intelligence or a even an advanced algorithm.
What if the platform users or daily topics going to be millions how mods can manage all that huge posts also humans can make mistakes but AI is learning and more accurate and faster and this is a perfect example of decentralization.
We have seen that translation category taken down because of abuse and the weakness of humans/mods with huge amounts of spam and I expect the same scenario with the current process without AI beside humans will take the hole system down.
I've also seen so many trivial suggestions/bugs posts has 60-50$ vote about 9 Lines post without any hardwork all that happened after scoring system.
Do you still think score system is fair ?
In my opinion current scoring system is useless and isn't professional and so easy to abuse by high score rank or newbie users It is irrelevant to a decentralized platform.
Utopian team needs to stop wasting time for system depends on humans and must be prepared for any abuse scenarios by using BOTS/AI.
Nice Information work continuously on steemit