• Valve justifies these changes to prevent tampering

  • In reality, this measure pays sinners justly.

  • Even if the key is legal, the analysis is no longer valid

For several months, Valve has been collecting feedback from Steam users to improve its rating system. The change they are announcing this week is very important, it means that the ratings of users who obtained the game through a key (or other means unrelated to Steam) will no longer be weighted.

Already in the month of May, the first major change occurs when Valve announces a new “Recent Reviews” section, which increases the visibility of the latest reviews (30 days), helping to reflect the development and current status of the product, which is particularly important is in the case of games released in Early Access format. A change of justice for the better, welcomed by both video game developers and consumers, but unrelated to what we are going to tell below.

THE BIG CHANGE IN THE RATING SYSTEM

This Monday, Valve announced that the ratings of users who received a product through a key will no longer be included in the game’s note. The main justification for this measure is to combat the use of fraudulent ratings and other strategies that artificially increase the rating of some games. In Valve’s words:

“An analysis of Steam games shows that at least 160 titles have a significantly higher percentage of positive reviews from users who activated the product with a key compared to users who purchased the game directly on Steam. Of course, these are legitimate reasons why this could be realistic for a game: some games have larger audiences, drawn heavily from Steam, and others have passionate early adopters or Kickstarter backers who are much more invested in the game.

But in many cases this abuse is clear and obvious, such as B. Duplicate and/or extensive verifications or verifications of developer accounts. In these cases, we have taken action by disapproving false reviews and will terminate any business relationship with developers who continue to violate our standards.”

CONSEQUENCES AT QUANTITATIVE AND QUALITATIVE LEVELS AND A PRACTICAL CASE WITH A WILDCAT

While I won’t be the one to question whether Valve’s intentions are noble when it comes to combating the use of fraudulent reviews on Steam with this measure, the most immediate consequences are what it entails on a quantitative level , for a developer the following:

  • They no longer weight game ratings from bundles like Humble, Indiegala, or Bundlestars.
  • Game ratings from crowdfunding projects such as Kickstarter or Verkami are no longer weighted.
  • Ratings from alternative shops that support Steam keys like Humble or Green Man Gaming are no longer weighted.
  • They no longer weight criticism of promotional keys provided by the developer or publisher to video game press professionals and/or influencers.
  • They no longer weight keys purchased directly from a developer or publisher in their own store.

Even assuming that the drop in ratings caused by this measure did not affect the percentage of positive reviews, it could be that a product loses its “very positive” or “extremely positive” rating, since Steam only uses these Ratings grants excellence from a certain number of critics, 50 get the first rating and 500 the second rating.

In this way, let’s hypothesize the previous assumption that the percentage of positive reviews does not change: Consider a game with 510 reviews, 95% of which are positive; this game would have an “extremely positive” rating, but if it lost more than 10 ratings (a number <500 total) after switching to Steam, its rating would automatically revert to "very positive", even if the new ratio of positive ratings was identical (95%) to the previous one.

What may seem like a problem of numbers is actually of paramount importance for the attractiveness or visibility (and consequently the sale) of a product on Steam, among other things because rising in the ratings helps to position products in the market offers in which Discovery lists and other algorithms that Valve has in place to create a virtuous cycle of sales.

spanish-games-humbly

At a qualitative level, the main problem with this measure is that Valve informs us about the elimination of fraudulent behavior, but in reality it achieves that it deprives users with a very high level of voice or the most qualified opinions of many indie games leaves the game aware that they have contributed not only financially but almost intellectually as well, leaving their “granite of feedback” from the pre-launch phases. I’m talking about projects that come from crowdfunding campaigns very diligent and/or that simply did not have sufficient access to funding to hire a high-level PR and that he was able to mobilize throughout the international press to be able to position himself in front empower its buyers and get you off to a good start.

Although Valve has its legitimate right, in the sense that it gives its keys whatever value it deems them, by taking this action you are telling the user that there is a market for first place finishers and one for runners up and you are betting people same, buy the keys in G2A , Humble or just win it by resorting to Steamgifts with Kickstarter backers and other crowdfunding programs.

Aside from debate and focusing on the impact this action has had on game ratings, the Steamspy Twitter account has just released a list of the 427 video games that suffered the biggest impact on their ratings with Valve’s action, with a minimal 6% change in its rating. Aside from checking for cheating and knowing which games have undergone a major quality change, it’s very interesting to see how more than 25% of the total (116 out of 427) have undergone a change… for the better!

The cases of games that have increased their ratings are probably due to the fact that a large part of the criticism they have lost has come from “bundle” format sales and/or outlets where the game’s target audience is very big is low qualified. In other words, at these outlets, people don’t buy so much because they’re interested in a game, but because it’s cheap or because it’s bundled with another that interests them, resulting in less affinity with the product and therefore therefore leads to lower grades in the assessment.

In our case, we noticed the following changes in the rating of Gato Salvaje Studio games:

  • AR-K: >20% critic loss and ratings increase from 60% (variable) to 62% (variable). AR-K is a product that was part of several bundles, hence the positive effect of not weighting its grades.
  • AR-K: THE GREAT ESCAPE: >20% loss of critics and ratings dropped from 98% (very positive) to 97% (positive) as the new total is under 50.
  • BULLSHOT: >25% loss of ratings and ratings drop from 80% (very positive) to 75% (mostly positive).

wild cat vapor

CONCLUSIONS

  • While we share the diagnosis, the recipe for eliminating “cheating” opinions has very important gaps, as it takes away the most qualified opinions from any Steam, Kickstarter project, and/or video game pro.
  • The change is good for Valve as it positions its store above Steam’s main distribution channels.
  • This is bad news for almost all indie developers, except for those who get a large percentage of criticism from package sales, in which case they tend to boost their products’ ratings.
  • Change is bad for some products even if they don’t lower their rating, as losing reviews due to missing metrics could lower their rating from “very positive” to “positive” or from “extremely positive” to “very positive” or 500 product points.
  • The measure confirms an open secret, the opinions from “bundle” environments are negatively skewed… and that leads us to a new idea, maybe fraudulent analytics should be regulated not only on the DEV side, but also on the user side with no relation to Product.
  • It would be good to explore other fairer alternatives such as B. the tightening of the requirements for submitting a review: either by playing time, eliminating part of the negative bias of users of “bundles”, or to have a minimum number of games in the library to avoid the phenomenon of multi-accounts.