Riot's surveys often have leading, misleading, biased, or incomplete response options.

LoL /r/leagueoflegends /u/akigo57 154 comments

Simple issues are relatively forgivable such as this question.

"When is the last time you purchased in-game content in League?

  1. Within the past month

  2. Within the past six months

  3. within the past year

  4. I've never purchased in-game content in league."

The missing option, "it's been greater than a year since I've purchased in-game content in league". It's omission is relatively innocuous but it is very telling. It reads like this survey wasn't made for people who avoid micro-transactions.

Let's look at the more leading sections.

Post image

By limiting options and responses Riot forces unfair conclusions.

The above survey options create inaccurate conclusions, and don't apply to every player.

I don't personally buy Hextech Chests so why would I ever select that option? The reason this is misleading is it creates false conclusions. Let's assume everyone who responded to this survey answered exactly as I did in the above example. Riot could then make the claim that the majority of players surveyed were satisfied with the content they get when buying Hextest chests which is probably an unfounded conclusion to reach.

Moving on.

Post image

Two options are essentially the same, practical responses aren't included, and Riot is insulting your intellect.

Hextech Crafting isn't confusing, it's just a dusty lootbox mechanic. Alternate currencies aren't difficult to understand, they're just yet another gimmick to make Orange essence and sometimes Blue essence irrelevant. Once again, by limiting options you're forced to essentially agree with two passively and indicate that you care slightly less about a third.

For a lot of us this whole page should probably look something like this:

Post image

Even after adding practical and real responses your voice is still limited.

Post image

Unrelated options.

Post image

Okay this one is actually important.

In the above example we see evidence of good survey practices. By repeating questions and options you can better ensure consistent responses to gain more accurate results. This combats people who just click through a survey without reading it. By randomizing responses you also get data on the choices that were not picked. The issue here is the number of instances you'd need for this to be accurate and precise is quite large. This survey doesn't really get close to that mark.


Post image

This is just confusing.

The wording on this section is terrible. It's functionally difficult to understand exactly what you're saying by any selection because of the response wording. Yes you could reason things through, but surveys in part need to be understandable at-a-glance or you risk inaccurate responses. If half of your respondents misunderstood a question then that question, and by extension your survey, is compromised.

TL:DR Riot's surveys tend to suck, are misleading, present only the illusion of choice, and at times confuse respondents.

1,001 Read the full article on Reddit


Be the first to comment.


This website uses cookies to ensure that you get the best experience Read more