EPT
Statistical Analysis of StarCraft 2 Balance - Page 4
| Forum Index > SC2 General |
|
Clearout
Norway1060 Posts
| ||
|
d_ijk_stra
United States36 Posts
On May 06 2011 05:20 hypnobean wrote: Anyone notice the paper identifies Jinro's race as Protoss? Oops!? I think I made it right in the introductory chapters...!? | ||
|
OrchidThief
Denmark2298 Posts
| ||
|
FrodaN
754 Posts
I'm not a betting man, but I wager that most people refuting this article didn't even read it thoroughly. | ||
|
d_ijk_stra
United States36 Posts
On May 06 2011 06:27 OrchidThief wrote: I know you've explained where beta is coming from in this thread, but it really wasn't apparent and seemed like it was just arbitrary numbers you'd fitted to players based on your assesment. I've taken several statistics courses but was still confused by it. Especially when it's such a significant part of your argument. Hmm... Actually I thought as soon as I declare the model as 'logistic regression', everybody will think it will estimates parameters (since this is Statistics ), but maybe it is not that natural for non-statistics people. But I'm glad that even non-statisticians are enjoying the article, and when preparing for the next version, I'll try to be more friendly to readers. | ||
|
WhiteDog
France8650 Posts
The analysis states that T > P, T > Z, and Z > P while the statistical significance is not very strong. Any feedback would be welcome. For me, as a zerg player, Z > T & P > Z... strange. Well I'm not a pro on this, but in sociology I think it was Bourdieu who explained that the few students coming from the lower class that actually goes through all the scholarship are statistically better than the student coming from the upper class. For him, this effect is caused because the few who succeed in school, despite coming from a social class that doesn't help them, develop special tactics or skills to overcome their handicap. What I'm trying to say is that the statistic, in this case, maybe reflect the exact opposite of the reality because, to overcome an imbalance (such as, it's just my point of view, but Z being a little bit over power against Terran) the players develop the skill they need to crush their opponent while their opponent don't because they are favored, so they obviously don't need to think to much to win. Twisting... my argument is pretty slant but I think it's true. | ||
|
AKspartan
United States126 Posts
| ||
|
Kazang
578 Posts
On May 06 2011 17:39 WhiteDog wrote: That's funny, each time I read that kind of analysis I think exactly the opposite For me, as a zerg player, Z > T & P > Z... strange. It just goes to show that neither your impression or the statistical analysis can conclude anything concrete as the variables of player skill outweigh any perceived or real "imbalance" between the races. | ||
|
DivineSC
United States128 Posts
| ||
|
Elean
689 Posts
Basicaly, it has exactly the same value as Idra saying "I'm the best player, I don't win, thus there is an imbalance". (actually, this model can converge to different solutions, the particular solution the author got was "protoss players are skilled and protoss are underpowered...", it could very well have converged to "protoss players have no skill and protoss are overpowered") All the people reading this should understand that this is not a scientific peer reviewed paper. There is no way, this would be accepted as it is now. If I were to review this paper I would ask for several modifications, and I would actually reject the paper unless the author answer this question: How can you tell there is no offset on the "skill parameter" of all the players of 1 race ? I would also ask a plot of the "skill pararemeter" distribution for each race. | ||
|
iNbluE
Switzerland674 Posts
On May 06 2011 21:57 DivineSC wrote: so you're saying that protoss is the worse race? LOL. Yeah that's a very strong point you're making... Seriously gtfo To the OP and everybody, the data was collected from GSL 1, several patches have been released since then. I think we just don't have enough datapoints from recent games to state anything yet. | ||
|
Smancer
United States379 Posts
How do you think this effects your model? Edit: I should be more careful when typing. | ||
|
nvrs
Greece481 Posts
Edit: If i was to start any sort of statistical analysis on the matter of balance, i would look at Blizzard first... | ||
|
maddogawl
United States63 Posts
, but to be honest taking the GSL from Oct 2010 to March 2011 causes some validity issues. There have been so many changes to the game in that time, remember the untargetable repairing scv's, the mass amount of bunker and pylon blocks, those are all things of the past. I feel anyone that takes this as the current balance of the game is sorely mistaken, the statistics stop in March 2011, since then we've had changes with the stim research time, bc speed, and infestor changes.My point is, the stats are nice to see the general trend of the GSL over that time period, but really nothing else. What we'd really need is a statistical analysis of each patch of the game, but even then talking about balance is something we should reserve to Blizzard. | ||
|
maddogawl
United States63 Posts
On May 06 2011 22:25 Elean wrote: It looks like this model assumes that protoss players are extremely skilled (6 Protoss in the top 10 skilled player), and get to the conclusion that Protoss is underpowered. Basicaly, it has exactly the same value as Idra saying "I'm the best player, I don't win, thus there is an imbalance". (actually, this model can converge to different solutions, the particular solution the author got was "protoss players are skilled and protoss are underpowered...", it could very well have converged to "protoss players have no skill and protoss are overpowered") All the people reading this should understand that this is not a scientific peer reviewed paper. There is no way, this would be accepted as it is now. If I were to review this paper I would ask for several modifications, and I would actually reject the paper unless the author answer this question: How can you tell there is no offset on the "skill parameter" of all the players of 1 race ? I would also ask a plot of the "skill pararemeter" distribution for each race. Precisely, there are some clear flaws with the statistical analysis, and theres no way to accurately indicate the skill of the player, in my opinion the statistics would be best served to say all GSL players are relatively equal in skill since they are in theory all the best in the world. | ||
|
Nakas
United States148 Posts
Secondly, the analysis of the adequacy of the model is inadequate. You show that your model is statistically better than the null model, but this is not enough. Much of the predictive power of the model is contained not just in BETA_race, but in BETA_player. Your analysis of the adequacy of the model needs to prove that your model can predict based on race, and not just players. | ||
|
Blacklizard
United States1194 Posts
On May 05 2011 15:05 d_ijk_stra wrote: I strongly agree with you and 'space-yes''s comment. At the time I was conducting the analysis, it was March and I didn't have good understandings on tournaments other than GSL. Moreover, gamers in GSL were isolated from others. But I didn't have enough GSL games per each patch, so I had to aggregate them all. I also feel very uncomfortable about this. Now the situation is a little different. There are many ongoing "global" leagues like NASL/TSL which I also enjoy to watch, thus I have more number of games worldwide and it might be enough to conduct a valid analysis. I hope I can do follow-up analysis anytime soon! NASL is fun to watch, but until global leagues have some way to fairly distribute lag... as in 5 matches, every other match is on another region's servers, you can't reliably believe the matches between North America and Korea (or something along those lines). | ||
|
KillerPenguin
United States516 Posts
The OP should say P>Z if the statement in your paper is correct. The paper only predicts the balance within the GSL. If 99% of players get into the GSL as Z and the 1% who make it in as P do well it would show P>Z balance. Luckily more T have made it into the GSL than any other race so your conclusion of T>Z and T>P are only further supported. The data comes from up to 7 months ago, since then the game has evolved and 2 patches have come out. The problem with only using new data is the sample size is significantly reduced, but honestly it was a completely different game pre 2011 most players would agree back then T was OP. PS: Stop complaining about sample size you guys don't know statistics. 800ish is fine if the balance is uneven enough, 10k is not necessary no one cares if the balance is off by that small of a margin. | ||
|
Krehlmar
Sweden1149 Posts
My point is; Yes, a test of 450 random games from random people would give some insight, but perhaps not even close to realistic, statistics on how the game balance is at the moment. | ||
|
Jayrod
1820 Posts
On May 05 2011 14:33 Zedders wrote: alright I've had enough of these graphs popping up everywhere and people statingthat the game is 'balanced now' because it's like 50/50/50.... It'd be interesting to see what the average game length is over time as well. Since cheeses have changed a lot since the game started (5rax reaper and whatnot), people have a) learned to deal with cheeses all-ins more adequately and b) developed more late game strategies, the games are probably as a result, longer. It isn't surprising to see that terran was so dominant at the beginning because of the number of people that started out playing terran. If i recall...the first GSL was vastly Terran populated. Not to mention vastly cheese populated too  Terran of course having the strongest tier one unit, the marine, had (has? i'm not sure anymore) the strongest early game. We all of course remember the BitByBit strategy (essentially all-inning...and if that all in doesnt work...all in again....and if that doesnt work...all in again...rinse and repeat). Since terran had the strongest early game...the game ended fast because cheeses were so powerful/prevalent. Therefore Terran won a lot. The games are getting longer now.... this of course results in more and more mistakes made by each player. Balance, in my opinion, should be weighted on how many mistakes the player can make in proportion to the other player's mistakes. What I mean by this is if one player makes less mistakes in his game decisions, he should ultimately win in a long game. Why you ask? Because Starcraft 2 is a game of decisions. And the longer the game goes on, the more decisions must be made. The more decisions that are made, the more mistakes there are, which should result in the degree of separation that makes one player better than the other. In context...let's say X race gets supply blocked 2 times (common macro mistake) but Y race never gets supply blocked. Y race then as a result has a larger army, larger economy etc. X race still wins simply because the units he made counter the units Y race made. Ok...this isn't imbalance...this is strategy right? Y makes a larger mistake by not scouting X and as a result his units crumble to X's. So we've established that theres different TYPEs of mistakes one can make. And some mistakes are weighed less than others. But at what point do these mistakes balance. What if X can get supply blocked twice, not scout opponent's army (+more mistakes) and still win. The severity of one race's total mistakes should not be much larger than another's. Ultimately I'd like to see X -not- win and I hope you agree with me, because X is clearly not the better player, his race is. --------------back to the graphs..... Ok so these graphs are representations of both races making an equal amount of mistakes since they are pros, and we are assuming that most pros compete at the same skill level regardless of race. So the degree of seperation of skill because of the mistakes that are made should be negligible. To sum up a little....... The average game length has increased (I'm pretty sure of this considering map size, cheese prevalance, spawn points). More game length means more potential for mistakes. Ultimately as e-sports fans, we want to see the better player win. This means the player that made the right call at the right time, with the right micro, while maintaining the right macro. Now it's super important to note...these graphs don't display anything about HOW the games were won. Looking at T v P... you might think "oh look it's balanced now because it's 50%/50% wins now" November2010 to jan 2011....Terran cheese prevails until protoss finally learns how to stop it (or they patched whatever). The game was balanced in january 2011 because Protoss learned how to stop strong terran all-ins? (the emergence of a 'safe build' to gain eco lead was developed) this isn't balance, this is metagame development, meaning half the people that are trying the old strategies that used to work 60% of the time, failed a lot. And the other half that realized this, tried new strategies (and not as developed and therefore not as good) won because it was something their opponent hadn't seen before. yay for meta game development! Despite almost your entire post not making any sense, I'll at least make the comment that you can't quantify the importance of various mistakes that are a made. A zerg player might play impeccably for 20 minutes, but then right click his army into his opponent and lose. That level of error has a different value from missing a larva injection or not having perfect creep spread. If you want to be able to have a real balance discussion someday its going to have to be something stupid and extreme like blizzard only allowing players to pick random for a year. Its not going to happen any time soon so we're just going to have to live with statistics like this mixed in with personal bias of the readers. Case and point, I bet most of the zerg players listening to idra vs. day9 on state of the game couldn't explain the most obvious counter-arguments against idras lack of scouting complaints. Day9 failed to make those points, but understand he doesn't think of the game in terms of balance while idra seems to constantly be coming up with ways to explain why the game is broken. | ||
| ||
