Create a contest to encourage users to test new SOVRYN features on testnet

[light]

Questions

• Do users actually need this incentive? How has the volume and quality of user-generated bug reports been so far, without this incentive?

• Could the need for high quality bug reports be adequately fulfilled with an internal QA role that we do not have to micromanage as much as this contest program?

• If in any case we want to involve the community in the bug hunting process and reward their participation, would they be just as motivated with nonmonetary rewards e.g. unique NFTs, special Discord badges, etc?

Alternative workflow

I echo some of the other comments that suggest the workflow seems a bit overengineered for the task at hand. I don’t mean to denigrate the proposal, I think it’s well thought out it’s just that for such low amounts of value on the line I think there may be too much work involved for the effort to really be worth it.

Here’s an alternate workflow I came up with that might be “good enough” to at least test and validate the concept before putting further effort into refinement:

• Users are given a reporting period during which they can find and submit bug reports as new issues in the appropriate GitHub repo, with their RSK address included at the bottom of the report.

• A dev will apply the label “validated” if the bug can be reproduced.

• Bugs that cannot be reproduced will have their issues closed. Duplicate issues will also be closed, although if they include some helpful information that the first issue did not have then they may be considered for the prize pool as well.

• At the end of the reporting period, the RSK addresses included on all validated bug reports (and duplicates considered helpful) will go into a random drawing for 1st, 2nd, and 3rd place prizes to be sent to the winning RSK addresses.

[nbourbon]

Hi, here are my thoughts:

Based on the significant activity observed in the Discord server where users are currently reporting issues that are probably the results of their own tests directly in Discord without any special reward, we must think very carefully on what type of issues we would like to receive through this initiative.

This is very relevant because once you put SOVs as a reward, and if this is on the high side, you will have or may have a lot of submissions and that means, a lot of work for the jury to thoroughly review. What will be the reward to the Jury and why would the jury invest those hours there instead of contributing to the platform?.

If we think there is a special type of test that does not exactly fit the bug bounty requirement and does not exactly fit the type of issues reported through the discord channel, and we all believe are very relevant to the platform then I would agree with thinking of some sort of incentive that can be SOVs or NFts or something else

But at this point, I’m not sure what type of tests or issues we expect to find that is very different than the regular reports in #tech-support or #user-feedback channels for example?. If there are such then finding incentives is totally fine

[lactarius]

Hi, I have carefully read all the posts, very valuable. I am writing these lines to you:

I believe that the formality and efforts required to file a bug report are so demanding that anyone will not know how to do it and want to do it unless they are very committed. Indeed, I believe that few would do it if they were not incentivised to do so. Sovryn needs to incentivise users to find those bugs as soon as possible, in my view.

It’s one thing to report a bug on discord simply in a few words, but it’s quite another to explain it properly by following Julio’s proposed instructions.

I don’t think there will be many bug reports, because the developers are doing their work very well, and as these problems are discovered and fixed, the occurrence of bugs will decrease. However, if I am wrong and there are many bug reports, they are welcome, and that will justify the resources invested in discovering them, assessing them and solving them.

Regarding the quantity and form (SOV, NFT) or the frequency of the awards, I have full confidence in the judgement of the Sovryn team. I also trust the system they choose to value them. Actually for me it is more important to solve them. In my opinion, it would be good if the user could choose the incentive they want, whether monetary or not, if given the option.

On the other hand, the only thing that seems odd to me is to create periods with a start and an end for the reporting of a bug. From my point of view a bug that is discovered today should be reported today and fixed quickly. If I take a fortnight to write a brilliant report, for me it is not as important as revealing it as soon as possible, even if the way I make it public is not as spectacular.

On a day-to-day basis, perhaps a separate B-feedback channel could be opened in the discord, to differentiate normal queries and comments related to the use of the platform, from those caused by bugs in the platform. Perhaps some users will make simple alerts in that new B channel, which can be dealt with quickly before it leads to a reward report. Or if someone has already reported it before, the users of that channel themselves can help by explaining that the problem is known, has been reported, or is in the process of being reported and fixed. This reduces the possibility of ghost bugs, already reported bugs, or repeated bug reports, and avoids extra work for the jury by reducing background noise.

All in all, I consider this bug-finding programme to be very good, because it allows users to help Sovryn and be rewarded by Sovryn.

[nbourbon]

I think the key message some of us were trying to share is that, this effort would be worth if the type of bugs we are referring to are different than the ones included in the Bug Bounty that already exist and also different than the bugs that are being reported daily on Discord for example.
I’ve seen people doing the math on how much they were expecting to get and reporting that in Discord when numbers were not adding up etc.

So if, you foresee an area with possible issues not being covered between those two programs, then I agree there is an opportunity for another program but we may need to:

• Agree that there is an important type of issue not covered
• Define a template so the user that submits the report does a thorough job which at the same time filters out most of the spam
• I think that users can submit at anytime, but the SOV reward should be given with a certain cadence so there are enough reports to be compared and selected as winners
• Define who would be doing that job of reviewing and replicating those reports. Having the template will be critical because the reviewer should need to be able to replicate the error

But again, this is only valid if we think there are a number of issues not included in the bug bounty and as part of the error/support logs we get in Discord

[CEK]

I have been reading the proposal and each comment carefully and these are some thoughts.

-I totally agree with compensating early users who take their time to test the platform or who were victims of some bugs of it.
It is the early users who take risks and give value to the platform with their criticisms and discoveries.
It is vitally important to discover and correct all bugs or improve the platform in all aspects before being exposed to the public and running the risk of being exploited or criticized. Reputation is essential in the valuation of any platform.

-I disagree with some comments that suggest that users should not be rewarded financially for their discoveries or only be rewarded with NFT (speculative asset), because time invested is money. Time is the most valuable asset.
Maybe not many bugs/improvements are reported yet because not many people are actually using the platform.
I see a lot of people desperate for the SOV token but not many asking questions about the platform itself.
Myself, at the beginning of February, have had a 5X long with almost no loan, a short with an “opening price” lower than what is shown in the interface, and a few days ago a 5X short that should have more than 25% of gains that all the time it was negative it was negative because the “current price” (2,3k higher than market price) is not the one that really uses the platform (also higher) and other things that I was reporting.

-There is an aspect that I think it is the most important:
Users will only report bugs that harm them, they will NEVER report bugs that they can take advantage of, unless they have a greater incentive to report it (sufficient amount of SOVs). Game theory.

-Regarding the workload for the juries, an intermediate level could be included that is in charge of filtering the reports (duplicates, badly written, etc.) so that the jury receives fewer reports.
“Filtrator”'s tasks should include quick review of reports, discard the repeated ones, maintain a shared spreadsheet with incidents already reported, select those that qualify for the contest and forward them to the jury.

[yago]

Who would you suggest having as this intermediary level?

[CEK]

Someone who cares about the project and is properly SOV-monetarily incentivized…

[juliomoros]

Thank you very much for your comments; I really appreciate them very much!

It gives me to understand that all of you actually took time to understand it and who appreciate the importance of this proposal.

1.- Part of why it took me a while to answer is because I have carefully read all the objections, in detail. I took the time to analyze each response and each recommendation.

Unfortunately many of the suggestions do not solve the challenges that this proposal seeks to address.

2.- As a conclusion I have determined that the following worrisome issues do exist:

• What happens if too many reports are presented?
  It is an unknown parameter and the fear of receiving let’s say 3,000 reports for a contest is not an unfounded fear.

However: A “RANDOM SELECTION” OF REPORTS exposes us to discard high-quality ones, to simply reduce the number of reports to read.

THAT’S WHY: It makes sense to include a “Leaker” in the proposal with an incentive of 100 SOV per month.

If at the end of the Quarter no more than 30 reports per contest are received, this position will be eliminated.

• These contests are based on a continuous process.
  There is a more or less continuous rate of issue reports that is expected to be translated into a continuous rate of user reports.

THAT IS WHY A MONTHLY CONTEST will only make us read twice as many reports as in a bi-weekly contest; In addition, we expose ourselves to receiving reports far from the opportune moment. It is not desirable to receive a report of something that should have been tested 10 days ago. Sovryn is evolving at a very fast pace to expose us to that risk.

WHILE THERE ARE INCENTIVES to produce more reports by offering rewards for them; there is also the extra effort of the user to make a minimally acceptable report.

These are forces that tend to cancel each other out, producing as a result a number of reports not likely higher than the number of incidents that users report to us on a daily basis.

• A challenge posed by this proposal is to take advantage of user experience to anticipate the discovery of bugs in immunefi, which may results in significant savings for Sovryn.

THIS IS WHY GIT COIN does not provide added value:

a. Git Coin cannot offer rewards on SOV token, and they will charge their commission.
b. We already have a payment mechanism with the MultiSigWallet. We don’t need the one from Git Coin.
c. Git Coin requires an additional learning curve and admin functions (more seemingly unnecessary work, for Sovryn).
d. Git Coin is a tool, not an extra helper. (Git Coin doesn’t come with a funny dwarf in a box ready to do our work for us.)
e. Git Coin is ANOTHER platform with ITS own problems, with which we are going to bring our users face to face, instead of receiving them directly through our means; that complicates the picture more, especially for the user.

• It is not intended to create another one bug hunt, but a more complete user feedback; every day there are reports of issues brought by users, which will always be there because they are complaints about malfunctions of the platform, which is under evolving process.

These reports are free, yes, and they are so at the expense of having to chat with the user (which is very time consuming) to find out what is going on. The reports do not replace this, but they are expected to mitigate the situation. Reports can also enrich our Wiki with additional information.

A report saves Sovryn staff chat time. But, in any case, attending the user is a job that we cannot avoid.

3.- I’d love to have the time to respond to all the others concerns that were expressed, especially those of @light , @stefan and @exiledsurfer ; I am very grateful for your suggestions; unfortunately, for the reasons I have just mentioned and others, in my opinion those suggestions would not contribute to a real improvement towards the efficiency of the process already proposed.

4.-I am going to mention the latest modifications of SIP-0013.
(I know that due the number, no one will want to take me away the label … )

4.1. It is indicated (I did not do it originally) that the report must be made in English.

4.2. The report must include an Abstract, of no more than 300 words, with the headlines very summarized of what was covered in the report.

4.3. The report must include a conclusion, of no more than 400 words, with highly summarized headlines about what was discovered or recommended in the report.

4.4. A Leaker will be on the work team on a trial basis only for this quarter period. The aim of this “Leaker” is to save work to the jury.

4.5. The main responsibility of the leaker will be to maintain an excel sheet with the list of reported incidents

4.6. The Leaker will also have the responsibility to discard reports that do not meet the minimum required quality.

4.7. The Leaker will discard repeated reports, giving priority to the first to appear.

The Leaker can be any member of our community and will be selected from among those who show interest in carrying out the task, by the members of the jury.

Thanks to everyone for your interest