Loading...

How to feed your trolls and influence people...

This post is an adapted version of the presentation I gave at the Montreal International Game Summit this year. So if you want to learn a little of what we talked about, grab a seat and let's get down to the subject of player feedback and how to intepret it.



Before we start it's important to not that this post is not exactly same as the session as some things just work differently when committing it to a page, as opposed to delivering it live. I am also one of those annoying people who insist on using animated presentations rather than simply being able to hand over the slides like you might with Powerpoint or Keynote, so this is an adaptation from the original presentation. It's a degree shorter when presented here, as with a forty minute presentation you end up with a few anecdotal asides that simply don't translate as well when it's written down. However it should still make for some interesting reading!


...so lets dive right in...
[Introduction]

Online communities present an interesting challenge for developers in this day and age. It is often an unpredictable and complex environment. However there is always one piece of advice I would give to absolutely anyone who intends to have a public role on the internet. It may sound simple, but it will be the only thing that allows you to retain your sanity...

Never take it personally...

...ever...

...people can be rude, mean and often downright abusive. Even if you could do everything perfectly, there will still be someone out there who gets a kick out of trolling you and generally questioning your genetic lineage. My forum handle has been 'Silirrion' for many years, and this is just an example of some of the fun  and cheerful feedback you will provoke at some stage....


 It's an unavoidable part of public engagement, and you simply have to let it slide off of you. If you aren;t able to disengage that part of your brain, you will eventually get driven down and worn out. It's a natural human weakness, and it takes some time and patience to  learn that you can't take it personally. 

So that established who are these trolls?

Any excuse to use Statler and Waldorf in a presentation slide is ok with me! These guys might have claim to being some of the original trolls in popular culture, but things have moved on since then!

There are now more channels for feedback than ever before. It should go without saying that if you are going to engage 

Ok, so once you are prepared and up to date on all your social media jargon and have established the lines of communication you will be using, what exactly is it that is so dangerous when it comes to player feedback? At a fundamental level there are three age-old traps that we see developers fall into time and time again when it comes to player feedback. They are inherent dangers that prey on some of the basics of human psychology. 

[Danger #1]

The first of these three dangers is, put quite simply, born out of arrogance, and that is the idea that the proverbial 'you' as a designer simply 'always know better' than your players. You can forgive people this one  on occasion, as sometimes the internet makes it all to easy to make you feel like you are a higher life form, and I'll never turn down an excuse to use an XKCD cartoon as an example... 


It is an easy trap to fall into. Feedback from players often tends to not be fully informed. After all, they don't have access to all of the information that you do as a designer. That often makes it easy to fall into the trap of ignoring their feedback on the grounds that it 'wasn't taking X into account' (even if X was something the players couldn't possibly know!). Now I'm not advocating you listen to every hair-brained idea that pops up on a forum thread or on a YouTube comment, but likewise you should never dismiss them out of hand, and use them as a way of testing your own convictions. Even asking yourself if the question is valid, is often worth the time in and of itself, because it gets you thinking about your own designs in critical terms. 

Someone's honest opinion is always worth considering...but we will get back to that point later!

So if that's the first danger, what's the second one?

[Danger #2]

Moving on you have a problem that is deeply rooted in a fundamental desire we all have - we all want to be liked! Alas, that can have a nasty side-effect...the Knee Jerk reaction...this is one of my own personal bugbears. 

While it is easy to understand why a developer might fall into the trap with this one, it can also indicate a fairly fundamental lack of understanding of what a designer is needed for in the process of making games. 

This angry mob of players wanted you to have more of the color red in your game. You have to keep them happy right? One of them wrote a lengthy, very articulate and impassioned plea on your forum for the inclusion of more red. Your CEO even read that one and want's to know what the harm is in adding more red...


...oh...wait...

..yeah...this isn't going to work...


...really? Crap...what the hell do we do now?


....sigh...

You can't win with that type of an approach!

Quite simply you can't please everyone all of the time. What's more, is that as a designer you shouldn't be trying to do so either! This is why this particular problem tends to irk me as a producer. When someone comes running to me with an issue simply because they read a forum thread about an issue, and wants to implement the players solution immediately it usually tells me two things

Firstly they didn't see and consider this issue themselves, and it took the players to point it out to them. Now not every issue is predictable, so this one can be forgiven on occasion. However then secondly you have the fact that they often haven't thought of the players feedback critically and objectively. They aren't actually approaching it as a design issue. They are reacting emotionally, and treating it more like it were a PR crisis.

They often want the players to like them, not be hating on them, and want to 'fix' that as quickly and as easily as possible, rather than actually considering the issue at hand, and all the implications of a solution.

Now saying this is a bad approach doesn't in the slightest mean it might not be an issue you have to take seriously. It often is something that has to be addressed. What is important here is that designers remember that they are the actual designers, and should be the ones that have the bigger picture, and be capable of assessing all the feedback, and coming up with an actual design solution. A solution which they have come to having carefully considered both the validity of the feedback and the impact of the possible solutions. 

That is why we have designers in the first place.

Having that instant emotional knee jerk reaction to someone not liking what you have done is rarely a good situation to place yourself in.

That leads us neatly onto the third danger...

[Danger #3]

In many ways this is the danger that will present you with the greatest risk to your project. It isn't as much of a flaw in the designer as the previous danger, but it is far more dangerous when it comes to how you spend your resources. In many ways it is an extension of the knee jerk reaction but takes on a much more cunningly dangerous form...and that is the attempt to actually plan to try and make everyone happy and end up with a weird design by committee approach. 

In most design projects focus is important. You need to have a clear design goal, a visions for your product. You should have a clear demographic in mind, a budget, some great unique selling points that you are going to anchor your design around. You should be aiming to take an idea, a specific something, and execute on that with a lazer focus.

The extension of the 'knee-jerk' reaction is actually planning to try and please all of that feedback, and to try and be more things to more people than your budget, resources or ambitions allow. This can lead to some very messy, unfocused development practices. Not every project needs to have every cool idea you can possibly imagine crammed into it.  

This is a trap that producers often find themselves slap bang in the middle of. Everyone has ideas...players, managers, marketing departments, CEOs, designers, artists, and coders alike. It is often the greatest challenge a producer faces. How to keep their team focused on the agreed goals, and not running off on tangents because a feedback channel suggested they might not like something.

Those of us of a certain age will remember what happened when someone let Homer Simpson load his 'perfect car' with every cool thing he could think of!


What's more is that this isn't even the start and end of a developers problems with community feedback!

If this all sounds very negative it isn't actually meant that way. A designers job is a difficult one because ideas are easy, freely available and, in effect, cheap. Designs on the other hand are complex beasts, that must pay homage to ideas, and often require the proverbial sacrifice in blood, sweat and tears. Feedback, and knowing how to deal with it, are essential to improving designs and refining them...but we aren't even finished with the complications just yet...there is one more very important one...

Users lie...

...unfortunately an element of collecting feedback is that not all of it is truthful....


On this subject a few pictures can spare a few hundred words at least - pictures from the good folk at Penny Arcade. Everyone knows that the internet has that fundamental weakness to it. The psychologists can have a field day theorizing as to reasons why we do it, but we can all agree that people often warp the truth into some evil unrecognizable form of the original information or opinion prior to it being posted.

So if we have established that we can't even trust the feedback to be honest and accurate how exactly can we view it?


Personally I always advise designers and those tasked with looking at feedback to treat them like some kind of smoke signals from your users. They are an indication of the areas that you need to investigate. There might be a raging fire beneath the smoke, or there might just be a small flame being manipulated by a man with a rug intent on sending a message that may or may not be true.

The true skill in using feedback effectively is trusting in your own ability to actually use feedback analytically...

...which leads us nicely into one of the more recent industry hot button topics


Metrics...some will tell you that they are the be all and end all of design analysis. Some will claim you can build entirely profitable enterprises purely from analysis of these metrics. Indeed some have already built very profitable empires out of a largely stats and metrics driven approach to design.

Personally I have always thought that metrics are a tool. A very important tool, but they are not the only tool available to a designer, and I would argue that a designer who only considers metrics is potentially just as short sighted as one who insists on only going with their 'gut feeling'

It is an interesting conflict that has been growing in the industry over the last few years. The creative instincts of the designers versus the cold hard metrics of user behavior. I love metrics, but I also find myself making decisions based on intuition, experience or belief in a design. The two do not have to be mutually exclusive of each other. 




Metrics serve a valuable purpose in allowing you to understand more about how people approach your designs, and how they interact with them. They often turn up things you wouldn't expect, and when approached as a tool to learning in that way, will usually mean that you are better informed and will make better designs in the future when armed with that information.

Likewise, this is a creative industry, and our natural instincts and creative sensibilities are important parts of the design process. If you distill your design down to a purely metrics driven formula then you may risk missing out on those all important creative inputs that can truly make the difference between a bad game and a good one, or a good game and a great one.

...and that leads on to the important part of assessing feedback, and that is how to read it. Before we get into this section I would invite you to watch this session from the TED lectures first.

This lecture is from a gentleman named Simon Sinek who wrote a very interesting book about the behavior of people and how business can use it that addressed a fundamental aspect of what motivates and inspires people.


...read on when you are finished...

Now while Sinek goes a lot further into what this kind of understanding can mean for a business. I was also struck by how fitting it is understanding players feedback. Just as in the examples he gives about why people are motivated and inspired by certain types of communication, you can benefit greatly from understanding why a user feels the way they do about a piece of feedback.



So in our content the key is in understanding why the user has felt compelled to give you a piece of feedback. You can take the 'Golden Circle' that Sinek talks about so eloquently and apply it to the feedback that you recieve from a design point of view. It will help you get to a deeper understanding of what is motivating the players feedback.


So taking the What, How and Why, I always like to challenge each piece of feedback with a question from each, and doing so soon reveals how much more important the WHY is, rather than the What and How. 

  • What is the problem the player has?
  • How has the players perceived that issue?
  • Why do they feel that way?
Sounds easy right? Maybe so, but you would be amazed how many times I have worked with, spoken to, or heard about designers who all too often fail to get to that all important third questions of WHY.

This also is a pretty robust approach for almost any piece of feedback. Sometimes it will be answered by the first question, or maybe the second, but going all the way to the third question will often enlighten you far more in terms of understanding how users understand and interact with your designs.

Importantly to often allows you to realize that the problem isn't actually what the user describes, but may lie elsewhere in your design.

Take an example. You have made a boss fight in your game. You are proud of the boss fight, feel it is pretty well balanced, and hope it is fun for the players. The players however find it way too hard and complain that several of the bosses attacks hit too hard and they die to much.

Now some designers see an easy answer and would say: "let's just reduce the damage on those attacks." 

However, maybe the issue isn't actually with the bosses attacks. In fact, you designed a special shield the players could activate to mitigate some of that damage, and upon closer inspection, the problem isn't the bosses attacks, but lies in the fact that your users aren't using their shield.

The question you have to ask is: Why did that happen?

Did you forget to introduce the mechanic?
Did you only make it useful so infrequently that it didn't occur to the player to use it?
Did you make the shield too difficult to execute? 
Did you fail to properly visually communicate when to use the shield?

These are all possible questions that you may have missed because you did not ask 'WHY?" and simply reacted to the feedback at the general level. It might be that actually solving one of those possible causes above would actually make that boss fight way more fun and a better experience when the players get to it....if you didn't ask yourself the why, then you might have been selling your encounter short...

So lets fit that into those three questions we asked at the start...

  • What is the problem the player has? 
    • The boss is killing them too many times and making it frustrating
  • How has the players perceived that issue?
    • They see those powerful attacks that kill them, and don't know how to deal with them.
  • Why do they feel that way?
    • Because you have failed as a designer to make the use of the shield intuitive enough (and that could be for any of the reasons above)
Many people might end up just asking the first two questions and conclude that reducing the damage inflicted by the bosses attacks is their best solution. It's quick, and it does indeed solve the problem that the player complained about...however in not asking the 'WHY?', you then miss the opportunity to make that boss fight much more fun and engaging for the user. 

Yes, that is a simplified example, but I think the point is clear enough. 




...is really the most important question to keep asking yourself when it comes to user feedback.

In Closing...

Before we part ways, and thanks for hanging in there, this has been a very long post, I wanted to close with an important point. This can all sound a little harsh on the users, and that isn't the case at all. Feedback is vital to a good design process, and while it is often imperfect, biased and not always constructive, it is something that a good designer wants to understand. You will benefit from learning to look at things objectively, impartially, and not taking anything personally. There is a wealth of good feedback out there, you just have to know how to harness it...

...because I want to circle back to the start and leave you with a final thought...in particular in relation to MMO communities...


We do all know these guys. Every game community has them. Many developers often hate having to interact with them...sometimes rightly so...however, it is also worth remembering that Statler and Waldorf were at every single one of the Muppet Shows. They showed up every night. They might not have shown it, or expressed it, but you don't show up every night like that unless you truly appreciate that entertainment. Your players do care, they do love your games, they just often don't agree with each choice you make. They might sometimes express that displeasure too strongly...but they do care...there is a reason they are still around that it isn't necessarily 'cool' to admit to...

Likewise Statler and Waldorf there have the best damn seats in the house. Balcony booth above the stage? Those guys aren't just some of your most dedicated fans, they are often your best customers as well...

..and that's why I firmly believe the best designers learn how to take feedback, look beyond the what and how of the complaints and truly understand the all important 'WHY?' and then use that knowledge to craft better games



MIGS 2011 3297852150422931631

Post a Comment

13 comments

Munir said...

Interesting read, thanks! Sounds like it would have been a good session to hear live as well. Half the fun is in those little annecdotes from the trenches!

Adamo said...

Hello,

I think things are not that easy and there are more points to consider:

- designer of course do NOT get flamed directly from com. Of course they have to deal with the situations and make best of it - for their circumstances. If a game has nothing else impemented than digging in an mmo..what shall they do if nothing is invested to revamp? If company avoids EFFECTIVE investments?

It is much more the overall mood, expecially at AGE OF CONAN this is the fact:

The designers, dev, gd, gm, etc are just the number 1 adress to express frustration. This is all.

The real problem is, that the COMPANY never does care about it. The Company never feels responsible at their decisions and do wonder that if they cut the dev team into half, why progress is too slow for comm-

We all are living in a "fast world" and of course pc games are cind of waste of time...waiting endless for improvements for things that waste time...of course is worse waste of time and THEN lots of ppl loose interests.

If the "heart" an "soul" of a computergame once gets lost, and devs are driven like assembly line workers and changed and replaced....what honestly, what do you expect for the product? And THIS is what a player FEELS.

Entertaining is much more than a business - and lots of people will learn this old lesson...

Its an easy calculation: An architect plans - the engineer realises it. So find out if you have less architects or probalbly lees engineers? Probably because most of them are working on TSW after 2 month after release?

Very often correct answers are not that far away...

AmandaP said...

Adamo I am pretty sure he isn't talking specifically about Age of Conan or any particular project. Businesses tend to care because they need to be successful to keep making a profit to keep a game around. I would say that if no-one at Funcom cared they would have shut AoC down about three years ago when it was in free fall. They didn't. They stuck at it, and tried to pick it back up and fix it.

I think designers who work for relatively low pay in gaming because they love it, or love their project probably care more than you think and I think that extends into management. Maybe just their motivations are different (money!!!)

Craig Morrison said...

@Munir aye, the stories from the front-lines can be amusing ;)

@Adamo I think you do underestimate how much passion and care goes into these games and generally that goes across the business as a whole. Maybe in our case a degree of that is due to the fact we are an independent studio and have a little more flexibility but there is never a lack of care there :)

...although not sure any of that is relevant to the subject :p

@AmandaP pretty much yes although as above I think most would be surprised at how much the senior management care as well...or maybe our company is just a little weird like that ;)

Anonymous said...

Great read, thank you for sharing. Very interesting to see things from a developers point of view. I guess you won;t answer this but what percentage of gamers do you think lie and give false feedback? We all know it happens but does it make you completely distrust gamer feedback?

Craig Morrison said...

@anonymous

I think its hard to say.

I don't think there are many that fall into the out and out troll category. Those who are simply misleading you for kicks. They make up a small portion of your overall population, but may make up a higher percentage of your vocal community, say on your forums for example, so it depends on how you look at it.

You could also include those users who give you false information unintentionally, maybe because they don't understand something (or were misinformed by either your design or a fellow player), so sometimes people might not even be aware they were giving you information that might be false.

So it isn't like you should immediately distrust XX% of feedback. You should just be aware that misinformation is relatively common to come across and, as a designer, it can be best practice to challenge the validity so that you understand it better, rather than always taking it at face value. It well be valid feedback, but you should test to see if it is.

Anonymous said...

Wow, this is one of the best Articles in your Blog Sil.

I am out of AoC but you dont pay a game years of sub if it has not a fewspecific strengths.
SWTOR is waiting at the door - a static standard old fashioned mmo with 2 outstanding strengths with the VO-Story and Companions.
Let the IP beside, some like it, some dont care.

Its you challenge to make TSW a success now, after the Hype about SWTOR calms down - you lessons will greatly help it.

Anonymous said...

PS: I also recommend you to take the chance and get yourself a complete edition of Ultima VII rereleased at GoG.com for only 6 Bucks.
Take a look at waht world design was possible in the past with only using up around 25 MB of ram.
Games today sorely miss this high art of designcraft.

Get inspired!

Anonymous said...

PPS: Have you gotten aware of Mechwarrior Online?
One guy made a survey there which seems pretty good:
http://mwomercs.com/forums/topic/1998-the-state-of-the-sphere-analysis/

At the end he concludes:
"And, as always, it turns out that a lot of the extremely vocal groups on these forums are the extreme minority when it comes to what people actually want. I'm not at all surprised."

Here is the link to your article - the noise poeple make is never an indication about their numbers - never fall victim to the noises.

;)

Adamo said...

Hello,

Mr. Morrison, Sir,

after the long aoc-"martyrdom"... I am pretty sure that FC with your person found the right man to bite into that unripe apple called aoc after Gaute left. If three years ago somebody would have told me what actually would have been done for aoc in the future, I would't have believe it, just to be fair - under those circumstances.

But one thing is for sure:

Your project, aoc when it was launched, had/has outraged basic features....and a unique combat system on the market(!).

To drive a product with such key-features against the wall is much more cind of arbitrariness - caused by the company:

Look at the TSW team, what they managed to create the last three years (in the meanwhile). Ok, its still not finished too...but imagine what it would have meant for aoc, if this team would have been FOCUSSED together on aoc for, lets say just for 1 another single full year....when the fire was "hot" yet. Not to mention the number of subscribers...

Missing content, new features, pvp maps, etc. (I really do not want to refresh all topics, I am happy that I could forget somehow (cause surely I never deal with it) that I paid for a dx10 product - without dx10 in it...once upon a time)


The question for the "MOTIVATION"....is the main key for any trolls and expressed frustrations of the playerbase.

So instead of trying to "deal" with trolls and developing strategies to "handle" them, I suggest to go deep inside (the company) and ask:
Is really everything ok with the product? Do we really have dx10 implemented at release? What are the reasons that can provocate anger in our customers, that actually had good intentions buying our product?

How can we "repair" this and give them the feeling back to offer good products?

You had to start from scratch&build with your team.
And even if I (as a player) do not 100% agree with the pve main-direction (which is cind of a "violation" to a pvp-layout mmo imho; but yes, you're doing good work overall) I want honor the courage of anybody that actually re-picked up aoc;

Keep in mind, we remember times when some young reporters of a pc magazine have been officially dissed by FC during an interview of a preview of the aoc-beta - just because they asked about some missing content...that partly is missing until that day...
(but yes, that was before you hired aoc)

To sum up:
Troll intentions are actually as different as what the word "content" means for different people with their "subjective point of view" (company, player, etc) to the same game:

Sometimes its "just" about whishes, but sometimes it can also be about "nogos". The better reasons are evaluated, the more clear will be the result and proper solutions can be offered - if actually wanted and supported enough by company of course...

Adamo said...

btw

Do you know what is worse than a troll in/for a game? It's not two trolls - it's no troll...

Craig Morrison said...

@anonymous Glad you enjoyed it :) ..and yes, I played all the Ultima games the first time around, and many others (Eye of the Beholder and Captive on the goold ol Amiga spring to mind)...definitely some great games there, and many formative inspirations :)

Richter said...

I have to agree with the guy above. This is one of the best posts on your blog so. Fascinating to see how a developer approaches listening to us as players.

I get the feeling though that far too many inside the industry fall foul of the first trap you mention above. Developers always seem to give off this 'we know better than you' feeling that is more arrogance than it is professional pride. I am glad you acknowledge that such behavior is a serious weakness.

Home item

Blog Archive

Flickr Photo

Labels

gaming (184) painting (153) Game Development (95) photography (86) MMO (69) 40k (66) TV (65) Games Industry (61) movies (59) Must Play (53) travel (49) games (48) writing (40) comics (34) featured (34) Indie Games (33) Age of Sigmar (27) Best of 2017 (25) Best of 2018 (25) Best of 2016 (24) Oslo (23) books (21) Game Design (20) community (20) art (19) Montreal (18) warmachine (16) hordes (15) short story (14) Music (13) blogging (13) random web stuff (13) tabletop (13) Infinity (9) kickstarter (9) storytelling (9) Norway (8) San Francisco (8) Anarchy Online (7) Blizzard (6) nanowrimo (6) Age of Conan (5) Learning (5) Los Angeles (5) zBrush (5) Audio Story (4) Blizzcon (4) California (4) GDC Europe 2012 (4) Guildball (4) MIGS 2011 (4) PAX East 2012 (4) PAX Prime 2012 (4) Teach Yourself (4) cosplay (4) inspiration (4) warhammer (4) zombies (4) Comic-con (3) Continue Magazine (3) Devcom (3) E3 2012 (3) GDC 2015 (3) GRAND (3) Gamasutra (3) Gamescom 2017 (3) World of Warcraft (3) steampunk (3) Board Games (2) Global Game Jam 2017 (2) Level Up 2018 (2) PVP (2) San Diego (2) extralife (2) lightning (2) nostalgia (2) ACAM (1) Armies on Parade 2017 (1) BIG Festival (1) Brutality (1) Comikaze 2014 (1) Conan (1) Deadzone (1) Detroit (1) Funcom (1) GDC 2012 (1) GDC 2013 (1) GDC 2016 (1) Gamescom 2012 (1) Global Game Jam 2015 (1) Grand Canyon (1) HATE (1) Halloween (1) Hawaii (1) Hero Forge (1) IGDA (1) Inktober (1) Isle of Skye (1) Korea (1) LA Comic Con 2016 (1) LVO 2018 (1) LVO 2019 (1) Las Vegas (1) Montreal Comiccon 2013 (1) New York (1) OCGRIP (1) Outside Lands (1) Podcasts (1) Quebec City (1) Rememberance (1) Rome (1) Scotland (1) Star Wars (1) Star Wars Celebration (1) Technology (1) Theatre Bizzare (1) Wild West Exodus (1) Yosemite (1) adventures (1) bloodbowl (1) creative (1) malifaux (1) nature (1) ottawa (1) personal (1) sleep no more (1) startmakinggames.com (1) storm (1) vancouver (1) wondercon 2014 (1) wondercon 2015 (1) wondercon 2016 (1) wondercon 2017 (1)

Follow by Email

Random Posts