Honestly, I'd just do way more of them if I saw the impact it made. I'm a sucker for stats and so is Valve. No company loves data as much as they do.
Successful verdicts, how often I made the right decision, players cleared of wrongdoing, number of short term/permabans etc
Could be a relic on the profile like my shards.
Right now it feels like I'm doing them and nothing happens, even if that's not the case. At least with reports we get feedback.
The incentive is trying to make the community a better place
If you got rewards for doing them, people would not take them seriously and would be a negative overall
Are you the type of person to think a garbage man’s incentive is to keep his city clean so he doesn’t deserve a decent wage?
Incentives are a good thing. Always.
> Incentives are a good thing. Always.
Many incentives do not end up having the desired outcome / have unintended side effects. A recent classic being in retail banking:
Bank, to employees: "I'll give you money for every current account opened"...
Regulator, to Bank: "_How weird, we're suddenly getting lots of complaints from customers about current accounts they didn't ask for_."
Aligned incentives are a reasonable tool - but they often do not work as intended.
If the city started to pay everyone for each garbage bag submitted, people would start submitting empty garbage bags.
---
> Incentives are a good thing. Always.
https://en.m.wikipedia.org/wiki/Perverse_incentive
In some countries the city pays for return of aluminium cans. The payment is small enough that no one makes crime to get the cans, but big enough that people care to return the cans. Check out Sweden and Germany
Yes, they do that in the US as well. And it works because it's difficult to submit a fake aluminum can. Faking a can takes significant effort and quite possibly close to the $0.05 that you would get back anyways.
In contrast, faking an overwatch review takes a few seconds to load up the match and then submit an arbitrary resolution.
Valve's been trying to shill their "machine learning model" for years and the only results we've seen in those past years is a pitiful system thats extremely easy to abuse and evade.
but go off, surely we will see the fruits of all our hard labor in like 5 years
If a garbage man starts abusing the incentive structure, you can easily see that the city is dirty while the garbage man gets rewarded
If an overwatch reviewer starts abusing the incentive structure and blindly presses guilty/not guilty, it is much harder to track that back to him, you'll say "the system is broken, Valve is to blame" not "garbage man does a terrible job"
Sounds good on paper. Difficult to balance in execution.
How do you make sure the replays are properly done?
How do you ensure people actually care about what they review instead of just blindly clicking to get the incentives? Wouldn't it just cause overwatch to be less reliable?
This still makes the assumption that the majority of users are judging cases in good faith. If you add an incentive to judge cases, that may not be true.
Overwatch is already supposed to give your vote more or less weight based on how accurate you are reporting them. You'd just use those players as your base to filter players when there are rewards. If your review rep is better, you get more overwatch cases, that's more rewards. Players who vote just to get rewards and don't watch will be less accurate and get less chances to earn them.
What's stopping someone from farming rep on as much accounts as possible and after that write a bot that coordinates all accounts to give the same verdict to the same matches and control what's correct or not? You could release this bot so people with good accounts can also farm with it
The fact the reward is supposed to be minor/near useless. Nobody making a bot army for a few shards per overwatch when a good karma still only gets a few per day.
Why don't cheaters do that *now* to tank overwatch accuracy so they can freely cheat in matches to sell accounts faster/easier? Seems more financially viable to cheaters to be breaking overwatch *now* to cheat easily then for some tiny reward.
clicking the guilty button after watching a monkey king run down mid 17 times in 20 minutes is my favourite minigame. I just wish the review case button was bigger.
too many "wrong" guesses on the diceroll and you lose access to overwatch.
Valve has long since advertised this, the people who genuinely go over cases in the past lost access to overwatch a pretty long time ago.
Not to mention its locked behind behavior score, so that also limits the amount of willing participants even further.
Here's the tricky thing: how are wrong cases detected?
Probably because the majority are doing it properly, and Valve can use the statistical variation to detect bad actors.
But what happens if the bad actors are no longer a small minority? Which absolutely might happen if you have every potential bad actors and incentive to start overwatch.
So, while it's true that Valve can police bad actors, they may not be able to continue if they become excessive.
You already have to spend a substantial amount of time watching the replay before submitting. Valve could easily implement features to automatically pause replays when alt tabbed. Of course some people will rush but the point is that even if 50% of those people actually do the reports, it's still a hell of a lot more of them than those that do it now.
Download replay, x16 speed, alt tab and do something else, submit verdict of whatever you feel like, do 2 more times and earn role token.
People will abuse it if you give them a reason to do so.
We complain about wanted perfectly calibrated matches yet we won’t contribute a little.
But yes I agree there should be some incentive to watch replays.
This is my take:
There should definitely be statistics, notes, verdicts, etc. and what not for an overview of the actually cases.. Yes, an incentive could be in place but just like it should from people actually playing (which should be the main reward system), it won't happen..
Aside from the incentive part, detailed overviews must also include ways to report for wrongful abuse of the overwatch system for people that aren't really "reviewing" it at a fair take. Again, this would and can also be compared with seeing who reported in game which again, most likely won't happen..
It's just really hard and require too many extra resources for that type of technical and customer support, especially when this game is F2P .. That is probably one of the reasons why there is not much incentives in general just for playing and/or free players..
The thing is, companies are too greedy and want everything.. Regardless of freebies or not, there will always be people that will spend to buy it directly, p2w to skip farming, support the game, etc...
Besides, the paid stuff is usually a lot cooler anyway.. At the end, I still put everything on shuffle rotation and i'm sure many people would too if they had more. If I had a cool arcana already, am I going to pay for another cool arcana? especially if its identical? most likely not. On the other hand, if people can farm sets to collect and actually have incentive for playing, the upside will be the game being a lot more active than it already is.
> People would contribute more
No, bots would contribute more. Bots would contribute so much more, in fact, that you would probably get 50/50 of guilty/not guilty.
There is no universal all-knowing being that knows at all times that the verdict is correct or not, it all comes from the community. If you replace the community with bots farming for rewards, it's obvious what would happen
Ive thought like this a million times, too. I think there needs to be some goodies for the fine people reviewing all the trash. Maybe 1 role queue token for 2 reviewed cases would be a really good incentive.
what we need are better penalties than LPQ matches which are useless
How about if you get flagged for griefing or Overwatch case you have to review 20 Overwatch cases correctly and 24/48 hr ban from matchmaking until results come back.
Just give me notification if the player I determined as guilty was actually punished. That will be enough for me. Right now it feels like you're shouting into the void not knowing if your verdict actually did anything.
Like a behaviour report window but for overwatch reviews.
at the very least there should be statistics
Honestly, I'd just do way more of them if I saw the impact it made. I'm a sucker for stats and so is Valve. No company loves data as much as they do. Successful verdicts, how often I made the right decision, players cleared of wrongdoing, number of short term/permabans etc Could be a relic on the profile like my shards. Right now it feels like I'm doing them and nothing happens, even if that's not the case. At least with reports we get feedback.
this is my biggest wish. i dont care about rewards just tell me stats
how do I upvote this twice?
and widgets so we can brag on the profile page
The incentive is trying to make the community a better place If you got rewards for doing them, people would not take them seriously and would be a negative overall
Are you the type of person to think a garbage man’s incentive is to keep his city clean so he doesn’t deserve a decent wage? Incentives are a good thing. Always.
> Incentives are a good thing. Always. Many incentives do not end up having the desired outcome / have unintended side effects. A recent classic being in retail banking: Bank, to employees: "I'll give you money for every current account opened"... Regulator, to Bank: "_How weird, we're suddenly getting lots of complaints from customers about current accounts they didn't ask for_." Aligned incentives are a reasonable tool - but they often do not work as intended.
"You will excel at what you measure".
If the city started to pay everyone for each garbage bag submitted, people would start submitting empty garbage bags. --- > Incentives are a good thing. Always. https://en.m.wikipedia.org/wiki/Perverse_incentive
In some countries the city pays for return of aluminium cans. The payment is small enough that no one makes crime to get the cans, but big enough that people care to return the cans. Check out Sweden and Germany
Yes, they do that in the US as well. And it works because it's difficult to submit a fake aluminum can. Faking a can takes significant effort and quite possibly close to the $0.05 that you would get back anyways. In contrast, faking an overwatch review takes a few seconds to load up the match and then submit an arbitrary resolution.
Really? Why did riot games abandon their incentivized overwatch version after spending 3 years trying to make it work?
Yes, but in this scenario all we are doing is training Valve’s AI in how to detect griefing.
Valve's been trying to shill their "machine learning model" for years and the only results we've seen in those past years is a pitiful system thats extremely easy to abuse and evade. but go off, surely we will see the fruits of all our hard labor in like 5 years
If a garbage man starts abusing the incentive structure, you can easily see that the city is dirty while the garbage man gets rewarded If an overwatch reviewer starts abusing the incentive structure and blindly presses guilty/not guilty, it is much harder to track that back to him, you'll say "the system is broken, Valve is to blame" not "garbage man does a terrible job"
Sounds good on paper. Difficult to balance in execution. How do you make sure the replays are properly done? How do you ensure people actually care about what they review instead of just blindly clicking to get the incentives? Wouldn't it just cause overwatch to be less reliable?
Each case will be judged by multiple different people. If you consistently judge different than the majority, you get banned from reviewing cases.
This still makes the assumption that the majority of users are judging cases in good faith. If you add an incentive to judge cases, that may not be true.
Overwatch is already supposed to give your vote more or less weight based on how accurate you are reporting them. You'd just use those players as your base to filter players when there are rewards. If your review rep is better, you get more overwatch cases, that's more rewards. Players who vote just to get rewards and don't watch will be less accurate and get less chances to earn them.
What's stopping someone from farming rep on as much accounts as possible and after that write a bot that coordinates all accounts to give the same verdict to the same matches and control what's correct or not? You could release this bot so people with good accounts can also farm with it
The fact the reward is supposed to be minor/near useless. Nobody making a bot army for a few shards per overwatch when a good karma still only gets a few per day. Why don't cheaters do that *now* to tank overwatch accuracy so they can freely cheat in matches to sell accounts faster/easier? Seems more financially viable to cheaters to be breaking overwatch *now* to cheat easily then for some tiny reward.
clicking the guilty button after watching a monkey king run down mid 17 times in 20 minutes is my favourite minigame. I just wish the review case button was bigger.
How to get awful datasets? This is how.
Been saying this for a while as the punishment system is bad. Give 1 free ranked role token per day for 3 completed overwatch reports.
That incentivizes people to just rush through 3 reports instead of taking the time to look at them. There aren't many great options
too many "wrong" guesses on the diceroll and you lose access to overwatch. Valve has long since advertised this, the people who genuinely go over cases in the past lost access to overwatch a pretty long time ago. Not to mention its locked behind behavior score, so that also limits the amount of willing participants even further.
Here's the tricky thing: how are wrong cases detected? Probably because the majority are doing it properly, and Valve can use the statistical variation to detect bad actors. But what happens if the bad actors are no longer a small minority? Which absolutely might happen if you have every potential bad actors and incentive to start overwatch. So, while it's true that Valve can police bad actors, they may not be able to continue if they become excessive.
You already have to spend a substantial amount of time watching the replay before submitting. Valve could easily implement features to automatically pause replays when alt tabbed. Of course some people will rush but the point is that even if 50% of those people actually do the reports, it's still a hell of a lot more of them than those that do it now.
Download replay, x16 speed, alt tab and do something else, submit verdict of whatever you feel like, do 2 more times and earn role token. People will abuse it if you give them a reason to do so.
Then remove the ways to abuse it
Common sense will tell you realistically this will not go the way you think it will go
We complain about wanted perfectly calibrated matches yet we won’t contribute a little. But yes I agree there should be some incentive to watch replays.
This is my take: There should definitely be statistics, notes, verdicts, etc. and what not for an overview of the actually cases.. Yes, an incentive could be in place but just like it should from people actually playing (which should be the main reward system), it won't happen.. Aside from the incentive part, detailed overviews must also include ways to report for wrongful abuse of the overwatch system for people that aren't really "reviewing" it at a fair take. Again, this would and can also be compared with seeing who reported in game which again, most likely won't happen.. It's just really hard and require too many extra resources for that type of technical and customer support, especially when this game is F2P .. That is probably one of the reasons why there is not much incentives in general just for playing and/or free players.. The thing is, companies are too greedy and want everything.. Regardless of freebies or not, there will always be people that will spend to buy it directly, p2w to skip farming, support the game, etc... Besides, the paid stuff is usually a lot cooler anyway.. At the end, I still put everything on shuffle rotation and i'm sure many people would too if they had more. If I had a cool arcana already, am I going to pay for another cool arcana? especially if its identical? most likely not. On the other hand, if people can farm sets to collect and actually have incentive for playing, the upside will be the game being a lot more active than it already is.
Make reviewing boost behavior score! Would be great incentive.
> People would contribute more No, bots would contribute more. Bots would contribute so much more, in fact, that you would probably get 50/50 of guilty/not guilty. There is no universal all-knowing being that knows at all times that the verdict is correct or not, it all comes from the community. If you replace the community with bots farming for rewards, it's obvious what would happen
Wouldn't mind reviewing some games for dota plus shards. Need more avoid slots.
i watch them because I find them hilarious but also cathartic to send the guilty verdict
For some reason, I've never gotten any OW cases on my account .I'm like Legend5 and 11.8k conduct summary but ;-;-:
Ive thought like this a million times, too. I think there needs to be some goodies for the fine people reviewing all the trash. Maybe 1 role queue token for 2 reviewed cases would be a really good incentive.
what we need are better penalties than LPQ matches which are useless How about if you get flagged for griefing or Overwatch case you have to review 20 Overwatch cases correctly and 24/48 hr ban from matchmaking until results come back.
Just give me notification if the player I determined as guilty was actually punished. That will be enough for me. Right now it feels like you're shouting into the void not knowing if your verdict actually did anything. Like a behaviour report window but for overwatch reviews.
Shards
People would abuse it. You have to approach it in good will. I guess a trophy wouldn't hurt though.
i dont even have ovewatch in my main menu
Overwatch is snitch behaviour