The most recent PTR release for Overwatch indicated a change of policy for Blizzard in their approach to updating the game. Previous PTRs had been more of a gentle introduction to the next update, a chance for players to see what life would be like on the other side and experiment with the small tweaks to heroes and gameplay. The latest PTR is different - it’s a bundle of larger tweaks, none of which are guaranteed or even designed to go into the live game unchanged. The PTR affects ultimate charge rate, the experience system, hero abilities, health pools, and more. It’s bringing up some interesting questions about how Blizzard should be gathering feedback on their ideas for updates and the best ways of testing these ideas.
This may seem like a minor step but it does have big implications for the game and its iterative update process. The previous policy of showcasing a future update in PTR was designed to allow for feedback, but in reality didn’t give time for anything useful. One of the only changes not to make it through PTR was a significant buff to Mercy’s damage boost, deemed too powerful to make it to the live game; practically every other change has been pushed into live no matter the feedback. On one hand it indicates that most of the changes have been good, but for the questionable ones, as in the case of Mercy’s buff, there wasn’t enough time or the right atmosphere to provide structured feedback and make iterative changes within the PTR.
New Policy
Blizzard admitted as much in their recent forum post on the matter, with Geoff Goodman saying, ”Previously we've been using the PTR, first and foremost, to make sure the game is solid and there aren't any heinous issues that would prevent us from releasing the patch. There was a nice secondary goal of gathering feedback on any changes we're planning and trying to iterate on them quickly, but we didn't always have time to try multiple different iterations of any particular change.” The lack of feedback and updates within PTR meant that it was purely an icebreaker, designed to give us some time to chat before the new guy was allowed into the party.
This had the secondary effect of making it hard work for Blizzard. If a change has to be game-ready before it even goes into PTR, it requires far more work on their end. Feedback in the PTR is a valuable asset to their development procedure, as they’re one of the most communicative developers in gaming.
The policy change this time was to make the PTR longer and change more before pushing the update live. Geoff posted in clear bold, right in the middle of his text, ”The PTR patch changes you guys see are unlikely to make it through PTR unchanged, unlike previous PTR cycles.” This allows them to put more drastic tweaks into the PTR and make further tweaks based on feedback from the players.
So what's the issue?
If Blizzard can properly pick out the relevant feedback and use it to make accurate specific changes, then this policy should work out far better. There are real downsides to it as well though, most to do with the structure of PTR.
The most obvious flaw with this policy is that PTR testers only have Quickplay. Without the ability to test these large changes in a hero limit 1 environment, a lot of the data will be totally useless for a big part of the community. For the first few days of any update, all that players want to do is use the newly changed heroes to get a feel for them. In this particular update it has resulted in roaming gangs of Soldier:76s, countered by equally stacked teams of D.Vas or Torbjorn turrets. Even if every player in the server has the same goal of seriously testing the new changes and providing feedback, together they create an environment that renders that testing useless.
This is still the case when the atmosphere of fevered excitement dwindles and players are back to stacking normal heroes, with the only fix being a mode that has hl1 as in ranked and competitive games. Jeff Kaplan acknowledged this problem a few hours after the current PTR was released, saying ” We'll see if we can get competitive up [in PTR]. If not, we might limit hero stacking on the PTR to get better testing results.”
That would be a great step to getting more accurate data about the changes. The other major problem though is intertwined with the concept of releasing many changes at once, an issue of confounding variables. The concern is that by making many changes at the same time, some may obscure others or interact with them in a way that disguises their unbalanced nature. When these changes, all interacting with each other, are separated for the push into the live game, there is a risk that individual tweaks may be unbalanced on their own. Will anyone be focused on the changes to Zarya’s tick rate in the midst of all the changes? These small pieces to the puzzle could easily be overlooked.
Another potential downside to this policy change is that players in the live release have no warning or soft introduction to the next update. With the PTR constantly shifting and changing and pieces being patched into the live version, there will be far less warning of what’s to come. So far there has been a very gentle and smooth transition from PTR to live, which alleviates the issues of having to play a tournament on multiple patches such as APAC was forced to experience. If a tournament is running and a patch hits which nobody fully tested in isolation, there is a far greater risk of disruption to the competition or the introduction of bugs into live play for thousands of dollars.
A potential answer
The solution to this is to slowly change the PTR so that it approaches the finished patch as time goes on. Each change, rather than trying something new, should be iteratively working its way towards a finished product, with the final PTR patch testing out the update that will go live.
Needless to say, this will take far more time. Letting the community have such a large role in testing out changes takes the workload away from Blizzard and could create amazing new ideas, but it certainly adds a lot of extra ladles into the broth. This policy change could be a great thing for the game, if Blizzard can pick out the most insightful feedback, make good changes, and don’t rush patches into live play.
But so far they’ve done a pretty good job on their own as the head cook.
For more competitive Overwatch news, follow us @GosuOverwatch.
