On being an effective beta tester

Public beta tests for video games have become, in the last five years, an overwhelmingly popular method of both advertising a game and gathering data about the game. Some beta tests, such as Red Alert 3 and Dawn of War II, have been primarily focused on the marketing aspect, and not as much for testing. However, tests such as Supreme Commander, Enemy Territory: Quake Wars, and WarCraft III have been focused on testing and feedback. Since StarCraft II is likely to be more focused on gathering feedback, this guide is aimed at maximizing the effectiveness of your participation in the StarCraft II beta.

A short introduction for myself. I have had a fair amount of experience doing beta tests for various games, including Supreme Commander, Enemy Territory: Quake Wars, and Universe at War: Earth Assault. Most of what I will write comes from my experiences and the experiences of other beta testers on those games.

This guide will be split into three parts:

1. Effective attitude
2. Effective testing methods
3. Effective communication methods


1. Effective attitude

This is the most important aspect of beta testing. In order to be effective as a beta tester, it is imperative that you abandon the idea that you are just doing this for fun. Beta testing is exactly that, testing. Not beta screwing around, not beta training, beta testing. While it is possible to enjoy beta tests, and even possible to get very good at a beta version of a game, these are not the main points of focus.

The main point of focus is to test everything about the game. You must be willing to try everything to bend, break, mess up, or otherwise use the game in unexpected ways. You must be willing to spend hours trying out strategies to test for balance. You must be willing to pay close attention to details of interactions. You must be willing to do this for free.

This is not an easy task, nor is it meant to be. Thankfully, you are not the only one, but your feedback is still important to make any game the best it can be.

2. Effective testing methods

If you have gone through the above section and are not scared, read through it again, carefully. If you agree with what I have written, and are willing to devote yourself to testing, then read on.

As mentioned above, testing involves a lot of time and a lot of different approaches. There is a lot to test in any game, so this section will cover it in parts.

Firstly, the easy stuff:
Technical testing

Technical testing is done by trying to break the game on your computer. Generally speaking, this is something that is usually done by pushing the limits of the game, and seeing if your machine crashes, glitches or slows down inordinately. Some methods for doing this include:
-Starting up the application(who knows, it might not even run).
-Setting all of the settings to highest and running an 8 player map with 7 AI players.
-Sending large groups of units around the map all at once, crossing paths.
-If a map editor is available, making a map with infinite loop object creation triggers.
-Alternately, creating a very large map and populating it with as many different triggers as possible. Generally creating an RPG style map should cover this pretty well.
-Switching graphics, sound, or input drivers and testing it out.
-Playing online from different network configurations.

The important thing about the above methods, and any other methods that may come to you, is that you keep pushing the limits with them until either the game blows up, or the game simply stops you. After all, there's only so far the settings go before they are maxed out.

Secondly, the hard stuff:
Gameplay testing

Gameplay testing is a lot more subjective, and a lot less clear cut than technical testing. With gameplay testing, it isn't enough for something to work or fail, it has to work well. Gameplay testing is primarily about balance testing, but finding bugs and glitches in unit behaviour or unit stats cannot be ignored.

Balance testing is generally the more important testing to be done, and should be occupying most of your time. Weeding out bugs and glitches is all well and good, but if the game is no fun, they make little difference. If they do make a difference, congratulations, you've found something to report.

Effective balance testing is accomplished by, at first, simply playing. I know I said the focus isn't to have fun or develop pro skills, but that doesn't mean they can't hurt. Developing a solid understanding of how to play the game can help immensely with this portion of the testing, though it is still not the primary goal. At any rate, when playing around, try to find units or unit combos that seem difficult to fight or easy to use to win. Try them against everyone you meet. Either someone will come up with a way to beat them, or it will become so popular that everyone will use it, prompting a patch.

Balance testing can be done with relative control as well. Simply find a few other players, and keep testing it, with each player alternating between trying the strategy and trying to counter the strategy. Eventually it will become clear how powerful the strategy is, and whether it warrants being nerfed.

As for bugs and glitches, the best way is to try different odd things. Try unburrowing a unit below another, or playing with worker mining collision, or killing a pylon during a warp gate warp-in. Attempt anything that seems like it could possibly cause the game to not know what to do. Thankfully, since we know what StarCraft's glitches are, we have a starting point from which we can test StarCraft II for glitches, but do not limit yourself to the known glitches. Blizzard has probably made an effort to remove as many of the old glitches as possible, except air unit stacking. Remember the goal is to test, and test exhaustively.

Also, for statistics bugs, pay attention to unit attacks and damages. This is a much harder thing to do while playing, so it needs to be done on its own. Basically, make sure that units are doing the amount of damage they should, with all the right bonuses. Make sure units are able to attack their correct targets. In short, test that all units are doing what they are meant to do. The easiest way to do this is to make a map in the editor with every unit and try attacking every other unit. Splitting this up among a group of friends will make the process a lot faster.

Third and lastly, the fluff stuff:
Graphics and Animation testing

Testing that graphics and animations are correct and glitch-free is important, but the least so. I mention it primarily to point out that while it is worth giving feedback, it is something that is generally finalized late in the beta testing anyway, so it is likely that you would be reporting problems with placeholders. Thus, there is little reason to worry about art problems until the last month or so of the test.


3. Effective communication methods

This is the most important part of being an effective beta tester. Without communicating your findings and problems, the company running the beta test has no idea what you have found. This is also one of the trickier aspects of beta testing. Generally speaking, most companies that run beta tests for feedback expect feedback. They often run official forums for this purpose, with official people reading them, officially. However, because these forums are for everyone in the beta test, they can get crowded very easily, which is the tricky part of communication. Many games also have feedback forms you can fill out to contact the company directly with a bug report or other issue.

Generally speaking, for balance and gameplay stuff, forums are the best bet. You can elaborate on your findings, possibly attach replays for evidence, and discuss the findings with others. Technical issues are better dealt with by direct bug report form, if available, but can be open for discussion depending on the issue. Keep in mind that making a bunch of bug report topics when a feedback form is available will make it harder for more discussion-oriented topics, like balance issues, to be noticed.

One channel that should not be overlooked is that of replays and unofficial forums. As mentioned before, if you find something you think is imbalanced, it can end up being used by everyone, necessitating a patch. Posting replays of the supposed imbalance online can shift the metagame overnight, and accomplish the task of correcting it with minimal individual effort. Even if the strategy gets a suitable counter, the metagame shift may still inspire another imbalance to be found, until finally the game achieves balance. Both unofficial and official forums can be useful for finding people with whom you can share different ideas for strategies, bugs, or technical tests.


To recap, in order to be an effective beta tester, you must be willing to test the game thoroughly, you must be testing the game both in terms of technical and game mechanics related issues, and you must be involved in the community and in communication with the company. It requires dedication and perseverence, but in the end, you gain a well-made, fun and deep game, along with the knowledge that you helped make it so. To whomever gets into the beta test for StarCraft II, I wish you well and encourage you to find and shoot as much trouble as you can.