|
In the past couple of months a lot of individuals in the Starcraft II community have started thinking in terms of the future of starcraft and number crunching becoming the future leading cause for victory. A recent relation was made between the 2002 Oakland A's (moneyball) and the game we've all come to know and love. Before i get the to the point i disagree with and will show you why i think that these types of relations cant be made im going to touch on a few things i find that are true about number crunching.
Number crunching for optimizing build orders, mineral/gas income, appropriate optimization for timing attacks, even what builds work best against the opposing build can all be consider legitimate number crunching. I believe to become a pro player a lot of this has to be taken into consideration. For example double mining on the nearest mineral patches in early game causes that tiny bit of an extra lead. This might not however cause you to win the game it was a result of number crunching that caused this to be discovered. Now with all that being said i want to elude the misconception that i think is growing in the community.
Number crunching i do not think can be used to figure out things like how to win tournaments out right like "moneyball" did with the playoffs in 2002. What the OA's did was that they figured out the minimum amount of runs needed to win each game based on the other teams averages. And to get those runs they took walks, balls, hits, and home runs into consideration. The reason this worked with baseballs is because of the fact that its a point based system. In starcraft you can be ahead the whole game and still lose the game even at a pro level. There is no points to judge how an individual is doing. For example lets say that player A is at 100 supply on 3 bases and player B is at 50 supply on 1 base. General knowledge would say that player A will win the game easily hands down no problem. Suddenly player B sneaks in 6 DT's into the 3 bases builds 10 canons on his front and takes out player A due to having no detection. This might seem like an easy case to solve and certainly any competent player could deal with this however we've seen time and time again this kill people.
A statistic was used in the article about creep spread if it was relevant to win rate. Even though it was an example and not a sound fact creep spread can be translated into so many different types of formats of data its not even usable. It could represent the players ability to multi-task, it could represent that the player prioritizes creep spread over everything else or it could even just straight mean that the player has been focusing on ONLY creep spread and is at 10 drones with 3 queens.
Now lets entertain the thought for a minute that the number crunching spoken of is something being used frequently right now. Lets say pro player A does all the numbers crunching and figures out the secret to winning the tournament. The vast amount of data that the brain will have to process and remember according to each and every situation that they come across would be enormous. Think about every build that the opposing player can do, then try and format a build that fits EVERY single one. Or lets say that you actually devised the perfect counter build for every build that the opposing player can use. Can any human being really remember that many builds??
To wrap up what im trying to say is that even though moneyballing starcraft sounds like a good idea, it does not seem realistic. Money ball works in situations where a point based system is used and not that many variables can be taken into account.
Reddit article for moneyballing: Here
|
Your post raises good points, but overlooks one or two big ones as well.
The guy involved in Moneyball was an economist who was trying to optimize baseball doing the steps you mentioned.
In starcraft 2, there are ways to optimize builds to be safe vs every opener at every time that it can hit with certain decision trees reacting off what you scout. Now, its possible, but HIGHLY UNLIKELY this will be achieved.
HOWEVER. U can see most ladder builds account for DT's at 7 minutes with turrets/spores at the front, or even a wall. almost all PvT builds came with a robo a dew months ago just in case banshees came out. In fact, most still do.
So ye, while optimizing builds will not work 100% in sc2, its coz the information in starcraft 2 is sooo far away from perfect information, too many assumptions need to be made to ensure that you CAN make a decision, but certain threats can still be accounted for.
A good example of a player who does not assume anything, afaik, is Nony. He plays safe, coz he doesnt assume the opponent will play a macro game. He doesnt assume the opponent wil cheese. He plays a game that can play against both of those, sacrificing some macro in early game for a shot at getting to the late game, and sacrificing a perfect defense to all cheeses for a shot at the macro game, and relying on HIS control and decision making to ensure he survives.
Even then, Nony is forgoing an important information source, but at the same time makes him a solid player.
(P.S A big problem I have with zerg in PvZ is that their tech tree and their ease of expanding allows them to safely outexpand protoss if they were to make 4 or 5 spines at every base, with a spore, a smallish standing army, a ling scout outside the P's base, control all 4 xelnagas, and expand conservatively but still quickly as creep reaches the base, and just make every P expansion painful to take
zerg doesnt do this, and instead drones like mad to get a bigger advantage, and then throws up defense last minute instead of building it up little by little, and then complain when they lose -_-)
|
I'm a Mathematics student at the University of Birmingham in England and while i do not necessarily agree with the entirety of your post, and excellent point is raised. The huge quantity of graphs and figures we are given relating to the growth of eSports, balance, player skill and pretty much anything and everything is wonderful, but there is a problem.
Just because something is shown in a mathematical/scientific way, does not mean it is accurate to the material it is trying to describe
Sometimes the method of analysis can be wrong, sometimes an inappropriate or wrongly sized/chosen dataset can be used, sometimes base assumptions are made that can skew the results. This phrase has been thrown around since the xkcd comic, but correlation DOES NOT prove causation. Companies who provide this data can skew it in a way that favours the point they are trying to make, and some can completely omit facts that go against the point they are trying to make in an attempt to win the argument. People, including professionals, get stats wrong all the time so always think about the info you are given, don't cling to figures blindly or you risk losing your track completely on the way to a deeper understanding of what you are trying to learn.
I would highly reccomend "Irrationality" by Stuart Sutherland for those interested in this field.
This aside, the topic title and content comes close to something I have been thinking about for a long time, and calling in to ItG/SotG to ask as well. Mathematical analysis has a myriad applications, but let's just look at one for the moment. Cybermetrics is a field of analysis used predominantly in baseball to evaluate players/teams in terms of skill and value. Since it's inception aspiring fantasy league winners have always been using their own statistical method try try to pick out the upcoming winners, and the winners of big fantasy leagues in america almost exclusively use these models.
In the present day, all major sporting teams have a score of mathematicians doing really complex maths (which is still well beyond my understanding unfortunately, but hopefully not for long!) to evaluate current players, opposing teams players for acquisition/to help beat them, and even for coaching purposes.
I wonder that in Starcraft, to what level has this taken effect? When EG acquired JYP, he is obviously an exceptionally talented player and already was excellent friends with Puma, but was their decision based on that, his percieved marketability, their current rosters' opinion and purely factors like this? Obviously this is the default way to decide these things, and i'm sure Scoots and all other managers and players have a better Idea than any of us on what you have to weigh in making these decisions, but do they have a statistician working in the background using standard methods of analysis to work with these factors and more as well?
Note here that i'm specifically not talking about build order optimisation, as while it is a numbers based problem, it (as far as I can tell) does not require particularly deep mathematical methodology. The decision-making in-game I'm sure would be benefited from having a mathematical background but frankly i'm so low level i don't even want to speculate. I'll leave it to the pros ^.^
Is there some "Starcraft Math", using a combination Starcraft specific factors such as mechanical skill, and standard models already researched in other sports and probably economics too which top Teams are using to make their decisions? If not, I'd probably choose this for my PhD thesis title and try to make a career from it as it combines my two favourite things in the world ^.^
But hey, I'll not get ahead of myself here, I've still got my masters degree to finish 
|
Thanks for the input. Well actually he didn't optimize the games he optimized the whole process of qualifying for the playoffs and making the bare minimum needed to win. Day9 has talked about this in optimizing units but that's different them what we are talking about. Think about all the random factors that need to be taken into account in a game that pits two fully functional beings with very low restrictions against each other. If you think in terms of what each side can do via baseball and the via starcraft you get that starcraft is a much more complex game even at its roots (sorry all baseball fans). Baseball starts off with a pitcher and a batter ultimately in the beginning. The batter can hit the ball, get a walk, or strike out. If the batter hits the ball it then branches into a slightly more complex game. From there it comes down to the person going to receive the ball he can drop or catch the ball then he can pass to one of the three bases that the runner is confined to. However in Starcraft even in the very beginning you have 3 options. Mine, build a building, or rush with all you're workers (lol naniwa). Then from there there and so many possibilities that haven't even been discovered yet. With every single unit you make you have countless other choices you could have made. I think that's why Zerg has trouble they think in rounds of larvae instead of each and every individual larvae.
|
Oh, one other point, while a mathematical approach may be useful, I really don't think that doing a whole bunch of theorising will be the number one way to win on the future. As far as i can tell, while it's obviously useful, you also need the pure mechanical skill, instinctual level knowledge, inspiration and mental and emotional stamina, not to mention other factors that only the pros would be able to eloquently describe from first-hand experience.
|
Number crunching is extremely useful for certain things. For example, last night on Day9, he and Sheth hashed out how much gas per minute you got from each extractor, therefore how much time it would take to get x infestors and y upgrades for a timing attack at z o'clock. Numbers matter. Its true that someone with amazing army control and tactics might still trounce you if your build is mathematically sound, but its not likely. Its one of the most overlooked things - korean builds with sick timings are generally so mathematically sound its incredible....
...... however you can just steal those builds yourself. Once the work is done, its done forever.
|
I don't really know anything about baseball so I'm probably missing something because I don't see how the moneyballing thing is related to what the reddit OP is talking about. What statistically advantageous system are you going to apply against an opponent with good creep spread based on the knowledge that players with above average creep spread have above average win rates for example?
|
valkyries man
User was warned for this post
|
There are a lot of variables involved in SC2, that's why it's not that simple to make a simple equation to win games.
However, if one can reduce them, then number-crunching and using equations and probabilities to win games long term is the way to go.
Fluid mechanics are used in swimming, probabilities in poker, solid mechanics in pool. Of it makes it less fun, but then instead of just players, each team could have engineers working to find best builds and the best way to respond to certain build and actions, while players just permorm what engineers tell them it's best. The good player is then that which has the best engineers and mechanics to execute and remember everything well.
Players can't be both, because most don't know mathematical analysis deep enough. They could if they studied, and then a player could be a SC2 engineer and player, knowing to build probability equations for each build and action and having their implications memorized for the games.
|
The author of that moneyballing article has good intentions, but seems to misunderstand why sabermetrics works. His suggestions is akin to study the win percentages correlating to "Knight to E5." Without any context, that information would be rather meaningless.
Personally, I'm curious over how some general TLPD statistics, like Elo, matchup Elo, and map biase percentage interact with each other to influence win percentages. Seems like an area ripe for research. I have found straight addition improve prediction accuracy (slightly but statistically significant), but a more meaningful algorithm seems possible. I just don't have the tools for it.
|
http://wellplayed.org/forum/articles/thread/fighting-conventional-wisdom-moneyballin-1
Here's the article in question. I agree it's not a perfect comparison (Baseball and SC2 are apples and oranges) but it's definitely a worthy topic to bring up, particularly the part about rejecting preconceived notions about a game and starting with a blank slate. If someone can get to that point, a lot of revelations about the metagame and the game itself could be made available, statistical or otherwise.
|
I am somewhat disturbed at the negativity the OP casts towards the use of quantitative analysis for improving processes and decision making.
Moneyball was one of my favourite books despite not being a baseball fan, and it's about much more than 1 guy turning around a team's performance in one year, as depicted in the movie. One part was the sabermetrics movement, where a bunch of geeks would analyse baseball statistics to derive superior quantitative measures of performance. One example is that of On-Base-Percentage, which is a better indicator of offensive reliability than conventional measures like runs, batting percentage or home runs.
The second part was the economics of things. Presumably, most players with high OBP have hefty contracts, but what Billy Beane did was get players that on one hand had good OBP, but also had deficiencies which reduced their value in the eyes of other organizations. For instance, he could get someone fat, or someone old and slow, then pay them much less than the usual going rate for a player with that OBP. As a result he gets a team with pretty good offensive capability but at a much reduced price - which was important because his team had a notoriously low payroll.
The OP is giving judging the use of analysis to aid decision making, citing excessive complexity in SC2. I think that's silly, how else can you confirm or deny your intuitions other than via rigorous critical thinking between trials? This is basically how science and applied works. Of course, some people arrive at faulty conclusions with their analysis, but that's the fault of the person doing the analysis and not of the methodology.
|
I'm not saying that math shouldn't be involved at all in Starcraft. I'm saying that with the mass amount of information and variables to take into consideration the amount of information that would be needed to be retained would be too immense for it to actually make the outcome 100% sure.
|
On February 18 2012 01:54 Doomville wrote:
I'm not saying that math shouldn't be involved at all in Starcraft. I'm saying that with the mass amount of information and variables to take into consideration the amount of information that would be needed to be retained would be too immense for it to actually make the outcome 100% sure.
When you want certainty, you stop looking at statistics.
|
@zyglog Sabermetrics is valuable for baseball, because a player's personal contribution to a team win is not trivial. SC2 is still a predominating 1v1 game, so the "moneyballing" article suggest going into the minute details of a player's play instead. While it could work, the examples that article raise was questionable. What can 15 or 25 creep tumor really tell you about a player or winrate, by itself? Seems like a poor line of initial research with obvious dead-ends, when there're plenty of lower hanging fruits.
StoicLoofah went a bit further on this issue:
StoicLoofah 9 points 19 hours ago The article poses a few close, but separate ideas. First, there exist better metrics in games to predict success (this is the baseball stuff). Second, unconventional strategies can be effective (Stephano stuff). Third, we can improve our play by looking more closely at data and coming up with interesting metrics (the speculative stuff). They roughly work together, but the analogy somewhat breaks down. Specifically, moneyball (the actual concepts behind it are called sabermetrics, but that's not important) is a strictly a measure for predicting a player's success. It isn't helpful in terms of making a player better. Knowing that you have a low OPS doesn't help you hit balls thrown on the outside corner. It may point out weaknesses in your game, but isn't directly a method for improvement. I think there are other analyses in baseball that are a more helpful analogy. For example, by looking at past data, we can determine whether the probability of winning is higher if you bunt or try for a hit in some circumstance. That doesn't quite fit into moneyball, but is an approach that is very slowly gaining traction in baseball. Unfortunately, most managers cling to their old truisms instead of looking at the data. TL;DR: make sure you distinguish between statistical analyses that predict success and those that impact how you play. http://www.reddit.com/r/starcraft/comments/psm2r/fighting_conventional_wisdom_moneyballing/
For example, how about we research into definitive cutoffs for early-, mid-, late- game (possibly per matchup); then we can make objective evaluations based on these cutoffs on when a player is most vulnerable?
What if we bin up players by their matchup strength, then see if we can find any discerting patterns on why some players are better in ZvT rather than ZvP? Their APM? Their openner? Timing of their Third?
Quantitative analysis is much needed in this sport. It's a fledging field that went unnotice for far too long. We all heard from the last lo3 how MLG missed their projections by 90%, maybe that'll be the warning shot that get this sport to pay attention to statistics.
|
The wellplayed.org writer makes a few very big mistakes because he doesn't truly understand the difference between Baseball and almost every other sport out there, including Starcraft. The interesting thing about Baseball is that it is a team sport and an individual sport at the same time. You play defense as a team and you try to knock in other teammates who are already on base or rely on teammates to hit you in once you're on, but the act of hitting and pitching takes place as an individual sport. What that means is that for hitting and pitching, you can very accurately assess a player's value statistically. However, since a player is a part of a team, a hit does not directly equal a win and a strikeout doesn't directly equal a loss. There needs to be some data interpretation to go from the individual's stats to a team's win%. Moneyball tries to take those statistics and translate them into win%.
Let's compare to other team sports and individual sports.
The author mentions basketball and I'll address it below in the spoiler.
+ Show Spoiler + It is a mistake to apply moneyball principles to a basketball team. Basketball is a pure team sport, unlike the hybrid that is Baseball. A basketball team acts as one unit. Every time a player takes a shot, he denies his teammates from taking that shot. So if you took something like shooting percentage (sh%) and tried to build a team of very high sh% guys, you'd have a problem. The problem is that many high sh% guys are inside players who pick up a ton of easy buckets based on the hard work of other players. Very few of those guys can effectively handle a ball or play outside defense.... both very important pieces towards winning. A basketball team needs to be built with roles in mind and if you don't fill those roles, you fail (there is some leeway on certain roles... like essentially playing with two centers instead of just one or going to a small lineup, but those have been attempted and very rarely work out better than the standard). If you broke players up by type and then applied sh% to try to find value, you still miss out on the idea that someone is a go-to guy. Those go-to guys often don't have the highest sh%, but they're the ones that are taking the bad shots as the clock runs down and lower their sh% in the process. On standard shots, they are often much better shooters. So it's really a mistake to over apply something like sh% even if broken down by role.
Now we get to the individual sport of Starcraft 2. The author suggest creep spread as an interesting metric to look at to see if he's a good player. Let's say that we found out that zergs who average placing 25+ creep tumors per game win an average of 55% of their tournament games and ones who placed less than 25 on average only won 45% of their tournament games. So, as a team manager, you'd go grab all the data for tournament players and try to find the one that places 25+ creep tumors per game and hire them, right? WRONG! You have a better statistic called win percentage right in front of you. A player's value as a winning player can be sumarized by win% because the only other factors that effect it are map and opponent. When it comes to creep spread, you still have map and opponent as a factor, but you also add a ton of other major factors and probably an infinite number of minor factors before you get to a win/loss.
Win% (accounting for opponent difficulty) is always going to be the best indicator of a winning player. This is true in every individual sport. You don't need to go through some statistical work to take an individual's stats and turn them into win%. You already have the final product readily available.
Now, that doesn't mean that you can't look at factors like creep spread, crunch some numbers, and then let your players know that they should work on spreading creep better. You absolutely can and should if you're a coach and do it for yourself as a player. I personally keep track of my own APM and something called SQ (spending quotient - look it up on TL) for every game I play to see how well I'm progressing on those fronts whereas my win% is heavily effected by the quality of my opponents. I also think that when optimizing a build order, you absolutely should look at the raw mathematical numbers to see how many units you could have out if your macro was perfect... it's much faster to flip some numbers around than to play through each build variant against a very easy computer to find out what works best.
Finally, player popularity is more important than player skill for a professional team. So raw win% still takes a backseat to the intangibles.
edit:
I want to make the point extra clear about the difference between Baseball, Basketball, and Starcraft 2. First, let's say you want SC2 to be more like Basketball as a true team sport. In that case, you'd play the game 4v4 and you're done. However, if you wanted SC2 to be a hybrid sport like Baseball, you would need one player to start the game, have a pause at preset intervals (5 min? 10?) and then shift in a new player for each team to play the next shift. If SC2 followed that model, then yes, you would need to start looking at various statistics other than win% to determine the effectiveness of each individual player. We have all sorts of team-league formats, but I just don't see that one happening.
|
On February 17 2012 23:48 Doomville wrote:
Now lets entertain the thought for a minute that the number crunching spoken of is something being used frequently right now. Lets say pro player A does all the numbers crunching and figures out the secret to winning the tournament. The vast amount of data that the brain will have to process and remember according to each and every situation that they come across would be enormous. Think about every build that the opposing player can do, then try and format a build that fits EVERY single one. Or lets say that you actually devised the perfect counter build for every build that the opposing player can use. Can any human being really remember that many builds??
Yes, they can remember every build. This varies by the person's intellect, but there's a saying..."In the beginner's mind there are many possibilities, in the expert's mind there are few." and it is absolutely true. The more experienced you are at Starcraft, the easier it becomes to guess what the opponent will do.
|
RenSC2, wouldn't the team leagues be sufficiently close to the hybrid sport you refer baseball as?
|
Number crunching can be very helpful for lower level/cheese play (e.g. computer optimized timing pushes) and to compute safe builds for the ladder. However at high level, safe play can be exploited as you spend money to be safe against things that never happen. An opponent reading you can take advantage of that.
At the very high level, players play much more reactive, so most of the time you can't pull off a predefined strat and/or get countered because opponent scouts what you are doing.
Doing some person-specific number crunching here (e.g. preferred openings, overlord routes, probability of cheese play) might give an edge then, but it won't be that game deciding imo.
Real top players are capable to adapt their build on the fly based on scouting and still are pretty near to an computer optimized build, they do the 'number crunching' ad hoc with their brain (frequently instinctive, without even rationalizing it).
|
I might be too old to understand, but is this "number crunching" what they used to call "data mining" in the olden days? You know, the diaper & beer story?
"number crunching" to me is a term so general I can't understand what you are talking about.
|
|
|
|
|
|
EPT
