[A] Benchmarker (BETA)

For first time players:

• Start by selecting a matchup to practice.
  
• Then select your supply benchmarks for 6:00, 8:00, and 10:00 minutes.
  
• Select the build completion objective (currently max supply or specified game time).
  
• Select the final build benchmark time or supply.
  
• *optional* Select whether to practice against the previous benchmark build order.
  
• Start your build.
  

Once you have completed a build you can elect to save it or start over. If you save your build it becomes available to practice against for that matchup next time you play that matchup.

Build Order steps are marked complete when the correct build-order-eligible unit or upgrade is made. They are marked failed when the incorrect build-order-eligible unit or upgrade is made.

This tool is designed around the idea of perfecting one standard “macro” build for each matchup. It is intended to help you with learning that build order, improving your macro mechanics, and refining your timings all as part of a single practice session. For most players below Master League this single build approach is commonly recommended as a great way to improve both on mechanics and understanding basic timings. So at least until multiple benchmark builds are added (and they almost certainly will be at some point), here are some of my recommended practice techniques:

Standard Macro Practice

• Pick a matchup.
  
• Think about what you consider your strongest “standard” macro build in that matchup.
  
• Play through that build a couple of times until you’re warmed up.
  
• Once you have a build run-through that you are happy with, save it as a benchmark build.
  
• For the next 10 - 30 minutes, attempt to refine that build by pushing yourself to exceed the supply benchmarks, and to shave off time from build step timings.
  
• Each time you execute the build more cleanly than the current benchmark save that new build as the benchmark to beat.
  
• After your chosen time is up or when you are satisfied with your current performance, move on to your next matchup and repeat steps 2 - 6 until done with each matchup.
  
• Go on the ladder and collect your wins.
  

Learning A New Build

• Pick a build.
  
• Select the appropriate matchup for the build.
  
• Uncheck practice against build order (if checked), or clear the existing build if desired.
  
• Play through the new build slowly at first. It is more important to get the build steps in the right order than to focus on execution or hitting timings the first time through.
  
• When the correct build order has been achieved (which you can check on the post-build report), save this build as the new benchmark.
  
• Practice against the newly saved benchmark build a couple of times, each time trying to shave a few seconds off the timings, squeezing a few more workers out, or use your resources more efficiently etc…
  
• Feel free to also use the [FORK] option to allow for diverging from the new build to add your own style and variations.
  
• Proceed to ladder.
  
• Slay nerds.
  

If you are a newer player, or you are learning a new build for the first time, you may choose to set your game speed to "Slower". Playing a perfect opening for 10 minutes can be quite hard on "Faster". After you have saved your build, set your game speed back to "Faster".

Also, whenever you are able to exceed one or more of the supply benchmarks during a build, don’t forget update the supply benchmarks to reflect the new record. From there you can tweak them by hand if desired.

More advanced players should ideally be comfortable cycling through different benchmark builds for a matchup since they are already familiar with them. If a large number of users start to ask for multiple different named benchmark builds it could be implemented before the 1.0 release.

One of the challenges in developing Benchmarker has been coming up with decent average defaults for supply benchmarks and alert thresholds. Any community feedback here is welcome. I’m talking about the default supply benchmarks for each matchup for an average (gold) level player. If each player could provide their league and recorded benchmarks for supply in each matchup, I can begin to build an average and create defaults by league per matchup.

I should also be able to do this with alerts if the community has opinions on what the right thresholds should be for various comfortable alert values by league. What APM and supply block thresholds seem challenging but fun to play with. With this option, newer players will not be practicing with the same intensity as the more hardcore players.

This is a beta release, so there are a couple of known issues that ideally should be addressed before Benchmarker is published across all of the current ladder maps. Some of these issues I know how to solve and just need more time to implement and test each fix/enhancement thoroughly, while others I would really appreciate any help or advice the community has.

Solved but Currently Unimplemented

• Not all Units/Upgrades/Abilities that perhaps should qualify as a build order step are being tracked. I expect more playtesting should fix this. Feedback from the community will expedite this getting resolved.
  
• Upgrade names are still being hammered out. If you see some weird looking ones, feel free to comment below so that I know about them.
  
• Playtesting has kind of shown that the alerts or hints can sometimes tend to fire at awkward times in the game where it doesn’t really make sense.
  
  • nexus/cc idle alert after probe/scv saturation on all bases
    
  • build hints still fire even if you’ve completed the step far before you did in the benchmark build
    
  • upgrade alert firing for upgrades you can't possibly research because you only have one upgrade facility
    
  
  
• Cancelling units from the build order can cause some odd behaviors when practicing against a benchmark build. Once implemented, cancelling a build order step should act effectively like an “UNDO” mechanism in the game to allow for correcting silly mistakes without restarting.
  
• There is no menu input validation to prevent crazy stuff from being entered except where not having it would throw exceptions. It can be added, but is somewhat tedious. So yes, at the moment if you try to break the game by entering text in a supply benchmark, it will break the game.
  
• The Benchmark Build Order evaluation logic is fairly complex and is still undergoing refinement as more and more edge cases are discovered where the existing logic doesn’t quite cover it. There will eventually be a page on the benchmarker wiki that covers this logic in more detail.
  
• You cannot correct a failed step if there are two of the same step back-to-back. There is a way to fix this, but it is a lot of work for very little gain at the moment.
  

Unsolved and Need Help

• I have implemented a Spending Quotient calculation that doesn’t appear to calculate correctly because Blizzard computes Income Rate and Average Unspent Resources in a non-intuitive way that I cannot replicate yet. If anyone has any information about this, it would be extremely helpful.
  
• Zerg creep doesn’t stick around after the build completes and all the units die, so if you wait long enough in the Post-Build Report screen it goes away on a new build and has to grow back and I’m not exactly sure how to “pre-creep” it yet. FIXED!
  
• Units which change icons upon upgrades (e.g. zergling -> speedling) are not reset back to the un-upgraded state. There may be a way to do this, but I am not familiar with how.[
  
• Some upgrades are not behaving as though reset correct despite the upgrade value being set to 0 on reset. Overlord speed is one known example of this.
  
• Fog of war is not reverted to its original coverage on reset. I’m not sure if there is actually any way to do this without exiting and reentering a game.FIXED!
  
• Build Hints and Performance Feedback sometimes compete for the Directive Message area of the screen. Not sure what the best solution to this is since only one message can be there at a time.
  

Won’t Fix because Reasons

• At the moment Benchmarker is only single player. As a trainer it doesn’t quite make as much sense as a multiplayer (since it’s ostensibly training your for the multiplayer version of the game). Never mind that the amount of effort to try to make it multiplayer would be ridiculous. But… There is a possibility of one day adding something like a “coach” player that could deploy and control harassment to assist in the primary player’s training if the community thinks that is something it might want.
  
• You cannot change race once you are in game. Most players only practice one race at a time, and the map editor isn’t really setup to support this feature very well.
  
• Benchmarker in its current form will not function as a Blizzard map mod. There are too many limitations on what you can do in a mod that prevent porting Benchmarker. There may be a trimmed down version based on the same fundamentals that could be used as a mod though. Contact me if interested in working on one.
  The current maximum build time is one hour of game time. There’s got to be a limit. I picked this one.
  
• Sometimes for events that occur near simultaneously, the trigger logic for build order evaluation seems to get confused. I assume this has to do with the concurrency of the game itself and the fact that many of the relevant variables used have global scope. It is unlikely there is anything that can be done through the map editor without incredibly tedious (and effectively untestable) dev work using locks etc... NEVERMIND - I stumbled across this thing called "Critical Section" and was able to isolate all build logic so that most all race conditions no should longer occur.
  
• When you attempt to correct a step such as 3x Barracks and you only complete 2 within 15 seconds, the third not only not correct the previous step, but will also fail the current step of the build if that is not also a Barracks. This is because of how the logic to increase the amount for a step works and there is really no way around this edge case that I have found yet.
  
• The in game timer cannot apparently be modified from within the game. This is the reason it keeps counting no matter what is going on. The build timer window was created to address this issue and it is located in the top right hand of the screen during a build.
  

• A Challenges section that gives you in-game mini-objectives to complete apart from the supply and build order objectives. These would be intended to challenge a player’s multi-tasking so that they still are alerted about macro and mechanics while taxing their APM and concentration. I openly acknowledge that this doesn’t simulate a realistic opponent because it is not intended to. It is intended to provide practice for the extra little things that tend to help you climb up the ladder. With multiple back-to-back challenges enabled in a single benchmarker training session, players will have a semi-realistic, but relatively stress-free practice trainer to really learn and test new ideas in. Challenge ideas include:
  
  • Worker stacking
    
  • Reach X minerals/income by Y time against an absolute scale of scores
    
  • Scouting for enemy tech at X, Y, Z time
    
  • Building a specific army composition by X time
    
  • Plus optional timing attack against fake selectable army with X army supply by Y time
    
  • Double/Triple pronged attacks by X time with Y units in each location
    
  • Minimap spotting of X proxy structures that each must be killed in Y seconds and spawn randomly on the map every Z seconds
    
  • Possibly can include an AI rush component or at least have AI units defending the structures with progressively more units over time to make killing in Y seconds harder
    
  • Defend X periodic AI drop harassment waves every Y seconds
    Simulated harassments of AI defended mineral lines to kill X workers while taking less than Y losses
    
  • Race-specific micro challenges to take place during the build at X time
    Possibly using logic donated by other micro training map creators
    
  
  
• An Alerts report tab on the Post-Build Report so that you can see how many alerts were triggered during the build and compare that against the benchmark build.
  
• Ability to save multiple benchmark builds per matchup
  pre-requisite for next feature idea...
  
• Importing/Exporting builds from the Benchmarker bank via a utility script. Similar to YABOT, this functionality would allow you to either share benchmark builds with friends for them to beat or to create finely tuned GSL based builds from studying VODs relentlessly
  
• A hardcore mode build option to fail benchmark build order objectives more aggressively for those players who want an extra challenge
  
• Configurable Supply Benchmark and Alert threshold defaults that can be selected based on the averaged feedback of what players in those leagues report (bronze, silver, gold, etc…)
  
• Map-specific builds
  may become requested once Benchmarker is released for multiple maps
  
• Enhanced GUI messages and layouts for things like Alerts and completing benchmarks
  
• A real-time Heads Up Display for information like APM and current worker count
  
• New Alerts such as:
  
  • Army Idle Too Long! for when you have a force out on the map and want to train yourself to keep it moving and jockeying for better position on the map
    
  • 2nd, 3rd, 4th… Expansion Delayed! to remind you to keep expanding on pace during macro practice
    
  
  
• A default “standard” build for each matchup in addition to benchmark builds based on the most recent GSL/WCS league games where players had the opportunity to play out a full standard build all the way to tier 3 without much enemy disruption of the build order
  
• A new tab on the Post-Build report for tracking how many alerts were triggered throughout the game
  

For a full list of proposed features, please checkout the issues page in the github project linked below.

These are questions that I never really was able to find a good answer for in my searching online. I’m hoping that there may be someone who reads this post who knows an answer or two. And if I get a couple of these answers, it will greatly help speed up some of the feature ideas above.

• I have struggled repeatedly to try and implement SQ somehow in the game, but I am absolutely stuck on one question: "How does Blizzard calculate their in-game averages for Resource Collection Rate and Average Unspent Minerals from the underlying game values?" Is there anyone who can explain this in enough mathematical detail to simulate a good income and resource collection rate without using blizzard’s values directly? (You can't use Blizzard's values because reset's don't affect them)
  
• Is there a good tutorial resource on, or anyone who can describe how to create a dialog that allows for selecting units similar to how a unit tester works without me having to go reverse engineer an old unit testing map? This would be used for creating builds from scratch as well as selecting units for challenges later on.
  
• Is there a particularly good resource on how to create/tune/control AI behaviors for SC2? This information would be very helpful in really making the challenges section more realistic.
  
• What are dialog panels useful for? I can’t find anything out there on dialog panels
  

One of the first goals of this project was to make sure that it could live on in the community even if I don’t have tons of time to devote to it myself. Therefore the entire project is available at:

https://github.com/benchmarker/benchmarker

I highly encourage anyone who is interested in helping to improve Benchmarker or who know how to implement any of the feature ideas above to get involved. I have gone to fairly extensive lengths to keep the map code clean and pleasant for others to work in. Development documentation and instructions for how to submit pull requests will be posted on the benchmarker github wiki shortly. I sincerely hope to grow this tool into something owned by the entire SC2 community and that can act as a launchpad to bring new players who don’t know where to start into the competitive 1v1 aspect of the game.

If while playing through the game you have any other ideas for how to fix or enhance Benchmarker, please feel free comment about them on this thread below. If you are interested in joining the Benchmarker project, please pm me on TL or contact via email at sc2benchmarker@gmail.com. Both experienced mapmakers and interested users wanting to try their hand will be welcomed enthusiastically.

As Benchmarker is still in beta there are likely to be a couple of bugs encountered from time to time. Hopefully none of them are game-breaking at this point and should at worst simply cause a build to need to be restarted most of the time. If you do come across one of these bugs and would like to report it to help me refine and polish the tool. Please follow these steps:

If it is an exception that pops up in red at the top of the screen, take a screenshot.
If instead it is a bug in gameplay that seems like it shouldn’t be happening, turn on one or more of the following debug flags and capture a screenshot with the debug messages that pertain to the buggy gameplay. You can enable debug flags by typing one of the following in the chat

• "-debug menus" - contains information for debugging menu behaviors.
  
• "-debug map" - helpful for debugging map mechanics like resetting the game.
  
• "-debug build" - displays a great deal (maybe too much) of information for how build order logic is being evaluated.
  
• "-debug alerts" - allows you to see what events and conditions are triggering alerts.
  
• "-debug bank" - can help to determine why values are not being saved or loaded correctly from the menus
  
• "-debug stats" - shows debug messaging for how stats are being computed throughout the game
  

Either post a comment in this thread or go to the issue submission page in benchmarker and open a new issue with the label of bug.
The issue description should ideally contain:

• A name that concisely describes the problem
  
• A screenshot of the affected behavior/exception message
  
• A short description of what you were doing when the bug happened
  
• A short description of what went wrong and how it affected the game.
  

A good example of a simple bug that takes approximately 15 seconds to create is at https://github.com/benchmarker/benchmarker/issues/190

Please search the existing issues of the project to make sure that you are opening has not already been submitted. Commenting on the same issue with more information is far more helpful in getting it solved.

The original inspiration to make Benchmarker came from a couple of places that definitely deserve a lot of the credit for the foundational ideas. Thanks to Day 9 and Lowko for their excellent videos on how to get newer players ramped up on competetive play It’s true that these videos (and perhaps even the map itself) will probably skew a bit Zerg, but the macro principles in these videos seem to me to be applicable across all of SC2 macro fairly evenly. If anyone has examples of similar tutorial videos that also go into the other race, comment below and they can be added.

- LowkoTV - How to Practise Zerg Macro
- Benchmarking in Zerg Versus Protoss
- Day[9] Daily #252 - Secrets of Hotkeys, APM and Mouse Movement

And a special thanks to Arondale for providing the splash screen and graphics!