General Discussion - Page 542

On May 13 2012 16:53 Chiharu Harukaze wrote:
Ok, I'll bite. Just this once.


I'm sure my degree in statistics and analysis is useless then. Thanks for letting me know.


Interpretation is only as useful as the data you have, The data you present has far too many factors influencing it to be worth analysing. The matchmaking system is not perfect. This is a team game where every player's actions can have an effect on the game. The hero pool is still limited so even if DotA2 theoretically was a port of a perfectly balanced DotA, it would still be missing a lot of its internal components.

The best model you could make here on the influence of heros would still have far too many interaction components to run a significantly feasible generalised linear model. And then you also have the regular problems in non-parametric analysis.


Your own data. The values for games over 60 min or so is ridiculously small and the error bars start approaching +/- 50%. That's meaningless data.


I'm going to ignore the first fallacy that the error bars are a useful term since what is more useful in this case would be the distribution of the idiosyncratic risk (ε_i). Noise does not just mean variance here. Admittedly, I might be using the term loosely here. It's also about the explanatory variables you choose to develop the model, compensating for the covariance and interference between them, and dealing with the random terms because the effect of external factors could always potentially cause confounding errors, whether you recognise they exist or not. Specifically, the interaction effects are likely to be of the most concern.


The problem is that Starcraft2 is a solo game so the game can be balanced on a solo level between races. However, MOBA-style games are team games where the picks and bans can greatly influence the data. In public games, there is often a lack of strategy amongst the team in picking optimal selections and this can cause issues in determining the underlying causes. There's no use saying "Well, ZZZ is a useless hero" if further inspection of the data reveals that there is a extraneous variable that has not been properly accounted for in your analysis.

Also, again I bring up the point of it being a team game. If you can't match teams correctly in terms of skill level, you are going to face problems where one team is simply better than the other, which makes any resultant data from that game much less useful. And then even if the two teams are equal in skill, whether each player is plating a hero they are comfortable on, what positions they are playing, do they feel like try-harding or not, etc. matter.


If you define balance to be all about whether win-rates are at 50%, then yes you're completely correct. But that's an incredibly shallow interpretation of balance. Balance also involves the interaction of many other external variables such as team line ups and player skill and how much of the depth of knowledge players are aware of.

Let's theorise for example that hero ZZZ has a tendency to be picked when players of a higher skill level feel like they want to stomp people of inferior skill after tanking their Elo for fun, and not when they actually play serious games with people of their own skill level. People who are new to the game do not pick ZZZ because maybe there's a lot of depth of knowledge required to play ZZZ. You data will obvious be skewed because the real issue behind the >50% win rate will be the skill disparity between players and not the hero itself.

---

If you're interested in learning more about statistical analysis, you may find the following books useful:

• An introduction to generalized linear models ; Dobson, Annette J, Boca Raton : Chapman & Hall/CRC, 2002
  
• Robust diagnostic regression analysis; Atkinson, A. C, New York : Springer, 2000
  
• Time series : data analysis and theory; Brillinger, David R, Philadelphia : Society for Industrial and Applied Mathematics, 2001
  
• The analysis of time series : an introduction; Chatfield, Christopher, London : Chapman and Hall, 1989

On May 13 2012 17:20 paralleluniverse wrote:
No, you're degree in business statistics isn't impressive. Nor is your name dropping of completely unrelated and out of context terms like generalized linear models and idiosyncratic risk (this is related to uncertainty about a portfolio holding in finance, it isn't the error term in a regression model) make you sound learned. In fact, it just reinforces the fact that you have no idea what you're talking about, as you're not even using these words correctly.

On May 13 2012 17:20 paralleluniverse wrote:
Still, where are you getting the idea that the sample size is too small. Why don't you tell me what the sample size is, considering that you know it's small? And no, this isn't my data nor my analysis. I found it on the internet.

On May 13 2012 17:20 paralleluniverse wrote:
The only thing you learned in statistics is model fitting? And your main complaint is that I haven't fitted a good model to the data. What would be the point of model fitting here? This is sheer stupidity. Modelling has 2 basic purposes in statistics: firstly to determine what factors are significant, secondly to predict future outcomes. Neither is useful in determining whether or not a specific hero is balanced or not.

On May 13 2012 17:20 paralleluniverse wrote:
I can tell you what the most significant factors are without even needing to fit a model: kills and deaths.

On May 13 2012 17:20 paralleluniverse wrote:
What does this tell us about balance? Absolutely nothing. Suppose we can predict the outcome of a game based on a model (I have no doubt that is child's play given how lop-sided most matches are). The model I would use, and which will almost certainly be correct most of the time will only need to take into account kills and deaths by the 20 minute mark, again what does this tell us about balance? Again nothing. But it will tell us a lot about DotA, it's a horribly lopsided game.

On May 13 2012 17:20 paralleluniverse wrote:
My definition of balance, a close to 50% win rate, isn't arbitrary, it's the most useful indicator of balance and is mostly what Blizzard (correctly) used. This is because it says, given all the prior factors, i.e. before the start of the game: what is the win rate of a particular hero? If the hero wins much more than 50%, then it's overpowered, as a large determining factor of winning is what hero is picked before the game has even started, as such the game is not balanced.

On May 13 2012 17:20 paralleluniverse wrote:
Model fitting using spells and synergies as predictors will help shed light on what might cause imbalances, but that's not the point. The point is to determine whether or not there is imbalance with a hero, not what specifically about that hero makes it overpowered.

On May 13 2012 17:18 drew-chan wrote:


I don't even have a loss against ursa in my +-400 total games played > <

Not meaning to be cocky here, but I usually find just kiting ursa will be fine with forcestaffs and ghost scepters etc.

On May 13 2012 17:20 paralleluniverse wrote:
The only thing you learned in statistics is model fitting? And your main complaint is that I haven't fitted a good model to the data. What would be the point of model fitting here? This is sheer stupidity. Modelling has 2 basic purposes in statistics: firstly to determine what factors are significant, secondly to predict future outcomes. Neither is useful in determining whether or not a specific hero is balanced or not. I can tell you what the most significant factors are without even needing to fit a model: kills and deaths. What does this tell us about balance? Absolutely nothing. Suppose we can predict the outcome of a game based on a model (I have no doubt that this is child's play given how lop-sided most matches are). The model I would use, and which will almost certainly be correct most of the time will only need to take into account kills and deaths by the 20 minute mark, again what does this tell us about balance? Again nothing. But it will tell us a lot about DotA, it's a horribly lopsided game.

My definition of balance, a close to 50% skill-adjusted win rate, isn't arbitrary, it's the most useful indicator of balance and is mostly what Blizzard (correctly) uses. This is because it says, for 2 equally skilled players (or teams), given all the prior factors, i.e. before the start of the game: what is the win rate of a particular hero? If the hero wins much more than 50%, then it's overpowered since a large determining factor of winning is what hero is picked before the game has even started, as such the game is not balanced. Again, the unadjusted ratio with a matchmaking system is already awful, it will almost certainly be worse if it were skill-adjusted as in SC2.

On May 13 2012 17:36 Chiharu Harukaze wrote:
Ok, last post before I go back to lurking.


You're quite right here and I will concede the point. It really should be the error term, not idiosyncratic risk. I'm currently reading up on portfolios so I got them confused. My apologies.


No clue what the size is. I can't magic a number for you I'm afraid. But I do know that when Riot ran their analysis for LoL, they still found their millions of games insufficient for explaining many of the concerns they had for balance and matchmaking optimisation. I'm sure DotA has less games to analyse than LoL, especially over the 60 min mark as can be inferred from your graphs.


I agree we are not terribly interested in forecasting. However, you can't say "ZZZ is OP" without considering whether ZZZ is the driving explanatory variable behind wins. Is playing ZZZ more important, or is it secretly YYY who supports ZZZ? Is it the player skill of the person playing ZZZ? You can't ignore them and need to analyse that. Maybe WWW is the 100% counter to ZZZ but hasn't been ported in yet, or no one likes playing WWW so in lower level games ZZZ will stomp even though there is a correct counter.


That is most assuredly wrong. KDA is not the only thing that matters. It's important, sure. But that's not what drives the game.

Furthermore, there is the issue of unoptimised play. I'm sure many of the others here can tell you how to win games with a negative KDA so I won't do into that (relentless splitpush team, etc.) The question is how many players know what the correct thing to do is? Lower levelled players may not know what to do from a position where they are behind and splitpush for example and only have the mentality that they need to keep teamfighting. That already obscures a lot of the problem.

There are other issues too. Is it first blood being given up which demoralises the team which then leads to a snowball in KDA that actually matters and ZZZ just happens to be good at getting first-blood? Then the real issue is with teams not playing right after first-blood, not the hero for instance.


If that is how you wish to see it.


Again, you're missing out an important piece here. The reason why people pick heros also has an effect on the data. If ZZZ being picked won you the game is true, then yes we would see a skew in that direction. But we could also see skews in that direction for many other reasons including team composition, the support chosen (maybe certain support combinations work better than others?), maybe people playing into the strengths of ZZZ. For example, if everyone picks hard carries like Spectre on one team and the other team chooses all early-push, then that says nothing about the carry potential of a hero.


Correct. But you're not trying hard enough to falsify your own theory.

On May 13 2012 18:00 paralleluniverse wrote:

So from this we can conclude that:
1. You don't always use technical terms correctly.
2. You don't know what the sample size is, but somehow you do know that it's too small.
3. You don't understand the difference between determining whether a imbalance exist and determining what is the cause of the imbalance.
4. You don't understand the purpose and usage of models.

And most of your disagreement and your writing focuses on the last one.

Suppose that hero X has a 80% win rate. Is he overpowered? I would say yes. But what if it actually turned out that the reason why X has an 80% win rate is because he is usually paired with another hero Y which supports X very well. It was then found that if X is not paired with Y, then X has a 30% win rate. Now given that X has an 80% win rate overall, but a 30% win rate when not with Y, is X sill overpowered?

The answer is again, yes. Being paired with Y is how X is almost always played and this is overpowered. Given this fact, X is imbalanced as the game is mostly determined before it even starts. But finding the cause of the imbalance and determining how to nerf X if at all, we would need to see that the cause is actually Y, and nerfs should mostly focus on how well Y supports X, rather than a direct nerf to X.

This example illustrates the difference between showing that there is an imbalanced hero, and what the cause or recommended fix to an imbalance is. A distinction you do not understand.

As for 4, another thing you don't understand given your comments about modelling and winning with a kill deficit, is the following quote attributed to Box:
"All models are wrong, but some are useful."

On May 13 2012 17:45 Musou wrote:

Really, now. KD ratio is the most significant factor? Please. Dota is clearly not the game for you. KD is not nearly the most important factor in Dota. The single most important thing in the game is the ancient. If you can't throne the other team, you lose the game. Kills and deaths are only some of the means to that end. You state that Dota is a lopsided game, but what you fail to realize is that kills and deaths are not nearly as important as you seem to think. You clearly have not played or watched any high level matches (by which I mean clan wars or competitive level). Teams can be over 20,000 xp/gold behind and win games, because they make the right decisions. Win a single teamfight even if you're behind by 20-30 kills, force buyback or if the other team can't afford it or already used it, and push their base. Gank based lineups lose all the time because they spent all their time ganking the wrong heroes and the other players farmed enough to tip the teamfight into their favor. Heroes do not need to be balanced on a 1v1 level. The point is that it's a team game, and you may sacrifice certain things to invest in others.

Since you love to bring up SC2 so much, remember the holy trinity of Starcraft? You can choose economy, tech, or army, or any two of the three. However, you can never choose all three. This is your build order. Certain builds lose to other builds, hence push>carry>gank>push. Sometimes teams will sacrifice their supports so that their carries can farm. Other teams choose to go push and push the enemy base quickly to prevent carries from farming. This is all part of the drafting stage. SC2 is not balanced at the bronze level, nor should it be. No games are balanced around the low level, because players aren't skilled enough for it to matter. Using low level pub match data as an argument for balance is just completely ignorant.

Your definition of balance is completely arbitrary and unnecessary. You seem to think 1v1 balance is an important part of Dota, when in actuality, it's completely insignificant. It's all about the teamwork, and as stated before, drafting is part of the game. You don't just go into a game and pick whatever you want and expect your team to work. Drafting is your opening build order. As I am comparing hero drafting to a build order, it's quite obvious that it's not the equivalent of race balance in SC2. It's more like specific builds such as worker rushes, 1-1-1, 4 gate, roach-ling all-in. The individual heroes are simply the units/tech structures your team is choosing to build. In short, stop crying about the balance when it's perfectly fine as is, and go play something else if you don't like it. There are 20 million other people who clearly enjoy the game just fine the way it is.

On May 13 2012 18:00 paralleluniverse wrote:
Suppose that hero X has a 80% win rate. Is he overpowered? I would say yes. But what if it actually turned out that the reason why X has an 80% win rate is because he is usually paired with another hero Y which supports X very well. It was then found that if X is not paired with Y, then X has a 30% win rate. Now given that X has an 80% win rate overall, but a 30% win rate when not with Y, is X sill overpowered?

The answer is again, yes. Being paired with Y is how X is almost always played and this is overpowered. Given this fact, X is imbalanced as the game is mostly determined before it even starts. But finding the cause of the imbalance and determining how to nerf X if at all, we would need to see that the cause is actually Y, and nerfs should mostly focus on how well Y supports X, rather than a direct nerf to X.

This example illustrates the difference between showing that there is an imbalanced hero, and what the cause or recommended fix to an imbalance is. A distinction you do not understand.

On May 13 2012 18:10 paralleluniverse wrote:

You also don't understand the first thing about statistical modelling.

Modelling isn't about getting 100% correct predictions as that is not possible. If you really wanted I'm sure you can win a game with less kills than the loser. In fact, I've done it myself. But that doesn't change anything: nearly all games are won or loss based on kills and deaths, and this would be a highly accurate predictor.

If your model to predict the win and loss is dependent on whether or not the ancient is killed, as you suggest, then you're model is useless as you wouldn't know whether the ancient is killed, so that the model cannot be used for prediction until after the game is already over, i.e. that's not a prediction.

On May 13 2012 18:09 5-s wrote:

Suppose we found that in starcraft 2 across the spectrum, players making dark templars have a 65% winrate. We further investigate that players making dark templars have an 80% winrate in lower leagues, 60% in platinum, 50% in diamond, and 40% in masters / grandmasters. Would you consider this unit imbalanced? Because I believe most sensible people here, who are not looking for a technical (winrate alone) but a functional definition would say this unit in the above scenario is balanced.

On May 13 2012 18:20 paralleluniverse wrote:

Your post shows your understanding of balance and why it is important in game design is rather elementary. Your analogy doesn't apply to DotA.

To see this we need to ask, why does balance matter? In SC2, it is not desirable for a unit to be overpowered as nearly all strategies would use this unit at the exclusion of others. Balance within races only matters to this extend.

So DT's are overpowered in your example. But this doesn't matter as long as Protoss is overall not overpowered, and DTs are not used too often at the exclusion of other strategies. Basically balance within races doesn't matter as long as balance amongst races is OK, and the race doesn't only have a single dominant strategy.

But the balance between DotA heroes is more akin to balance between races (e.g. PvT win rates), rather than balance within races. The key is that picking a hero in DotA and picking a race in SC2 happens before the game starts, therefore if balance between races or balance between heroes isn't good, the game is mostly decided before it even starts.

On May 13 2012 18:18 Musou wrote:

I'm not talking about modeling. Dota2 is far too complex to ever be modeled. There are simply far too many variables, some of which include player skill, hero choice, item choice, skill order, teamwork, and decision making. Hero selection is only one of many factors. You state that "nearly all games" are won or lost based on kills and deaths. Please support that statement with proof. I disagree, and I believe most experience players would as well.

In addition, I'm not even talking about close games where you win with fewer kills. I'm talking about games where you are significantly behind, like down 14-1 in the first 10 minutes. These kinds of games typically occur because one side picks all early-mid game heroes that are good for ganking, while the other side has heroes that take some time to mature and become useful. Also, you may want to fix your usage of "you're" when you actually mean your. You keep insulting people about their lack of knowledge in a specific field, but you're using the wrong words.

On May 13 2012 18:09 Erasme wrote:
Ursa is the easiest hero to counter, he can't get a kill by himself. And ofc carries are better late game, that's why they carry the game.
Btw, saying that something is imbalanced will just stop you from having fun and getting better, because you'll blame the game instead of your own mistakes.
On topic

imbalanced ursa cant kill razor cuz he has no damage *evil laugh*
And it's fucking hard to lasthit as Razor lol, deceptive attack animation