July 18, 2008

Surveys, the Silent Truth

The fall-out from the Warhammer Online: Age of Reckoning class cuts continues, but Mythic is not sitting idly by. They are staunchly defending their decisions, unlike so many companies that make decisions and tell the community to shut up. The part that amazes me about all of this, is that so much of the change was directed by Mythic's stellar beta feedback systems.

One of the tools being used is surveys, which allow beta testers to provide feedback immediately from inside the beta test, right after an event occurs. A system I hope will make it into launch. I've argued in the past that the feedback mechanisms in beta tests are often lackluster and generic. Most games just put up a forum, e-mail address, and a standard feedback form. All of which are susceptible to the loud minority taking over.

Not so with surveys, and this quote from Mark Jacobs, WAR's top dog, really highlights the power of a good feedback:
I am really disappointed in losing the Choppa. Choppas are great. The Choppa rocks. However, in certain terms, the class wasn’t rocking. It’s so funny, because even on our Beta boards people are saying, "I cannot believe you’re taking it out. I know the feedback wasn’t great on it, I know that it wasn’t exciting, but why are you taking it out?" Well, that’s the whole point! We’re taking it out because we have gotten this kind of feedback, because we know that people aren’t playing them.

This happens all the time on boards. You get a small percentage of the population acting like they speak for the entire population. However, this time, we’ve got the data. We know just how many people were playing the Choppa or the Hammerer. We know just how long they took to level. We have the surveys. We have all this data going in. There was one post on the forums that said, "I know I have provided a lot of hard feedback on the class…" Well, that's the whole point!
Anyone arguing that Mythic is haphazardly, at the last minute, changing key game mechanics is wrong. Mythic is measured and consistent with their changes; with copious amounts of data to back up their decisions publicly. The only lacking aspect has been the rather slow release of information, which should pretty much be done now that Guild Beta is in full swing.

I truly love WAR that much more every time I read interviews from the development team. It is refreshing to see a development team that has been around a while, actually using the knowledge that has been gained through a dozen failed projects, along with the data gained from great in-house feedback mechanisms.

6 comments:

Notherenow1 said...

So, basically you are professing your love for something that you have not even had in your hands, played, touched...

That's kinda creepy...almost like stalking.
All I can hope for you is it does not let you down.

heartlessgamer said...

Why so serious?

Aaron Miller said...

I'm giving Mythic the benefit of the doubt, but recognizing a problem and finding a good solution are not the same thing.

It took a commitment to quality to acknowledge the problem. I respect that. But the fact that these huge changes were made so close to release has convinced me to cancel my preorder and postpone playing WAR for two or three months. They should be polishing right now... not making major changes to pivotal systems.

heartlessgamer said...

Sorry to hear that Aaron, but these changes were not made close to launch. They were only announced close to the features being discussed with the public.

I think there is enough evidence showing that this change in design is a long time coming, but went unannounced due to the community's ill attitude towards anything that Mythic announces. So, the announcements only came after Mythic had properly geared to respond to the unwarranted idiocy that did occur.

Anonymous said...

They changed classes alot in the early stages of DAoC and that caused me to cancel (my class getting nurfed). As a dedicated MMO player I'm sure I'll give WAR a try...but I don't have high hopes for a Mythic product right now. What I saw at Gen Con last year was pretty boring.

nerd.gond.bad

Anonymous said...

They changed classes alot in the early stages of DAoC and that caused me to cancel (my class getting nurfed). As a dedicated MMO player I'm sure I'll give WAR a try...but I don't have high hopes for a Mythic product right now. What I saw at Gen Con last year was pretty boring.

nerd.gond.bad