|
Part 1: Introduction to Efficiency!
This blog is going to be the first in a series about my many musings and research on Dota stats. I’m introducing new metrics that tries to rethink the way people currently look at Dota stats in favor of a “means-to-objectives” based approach, and I'll expand it with later blogs. The idea is to explore how efficient Dota players can be with what they have in the game and build off of that. These metrics are called Efficiency Ratings, much like in basketball, but take the conclusions reached here with a grain of salt: what I am introducing here is a very naive and preliminary model of efficiency in Dota.
The Approach to a Naive Model
The starting point of this metric is answering the following question: What are the most easily measurable sources of heroes’ impact on the game that are currently recorded by replay parsers? There is KDA, Hero Damage, Building Damage, Damage Taken, Healing, and creep kill count (yes, creep kills). These are all useful stats, but some are less useful than others.
KDA is too concrete and creep kill count is too vague. Somebody with a low KDA could have a huge impact on the game and somebody with a high creep kill count may not have a great impact on the game. Looking at the damage stats they are far more useful than KDA and creep kill count alone because of the clearer granularity of the statistic, despite the fact that damage numbers can also be quite vague for a number of reasons. In spite of its weaknesses, it is the best of the simple statistics from game summaries and so we use damage as the naive measure of game impact.
Next is how we should think of efficiency. One of the first things that comes to my mind is that heroes can have different items and skills, and with different items and skills comes different potential of use. Heroes accumulate their potential by gaining more and more gold and levels during the game, so that they can have a greater impact. Therefore, we should define efficiency as some kind function of game impact over worth.
We have already chosen damage values to be our measure of game impact. But what is worth? We know that there are only two universal resources for heroes in Dota accessible in every game: net worth and experience. While experience should also be considered, again, to keep it simple for this model we simply choose gold worth only because it’s simple and very familiar with Dota viewers. I believe that more advanced models must a plethora of other factors.
Offense and Defense
Recall that we have chosen Hero Damage, Building Damage, Damage Taken, and Healing as our measures of game impact. Since it’s natural to use concepts of offense and defense in strategy, I am considering Hero Damage and Building Damage as offensive measures of impact and Damage Taken (Heroes and Buildings) and Healing as defensive measures of impact.
(Cost of Tango is just a fun scaling factor  )
Team Ratings:
![[image loading]](http://i.imgur.com/DSHGKTX.png)
![[image loading]](http://i.imgur.com/sjSWXdY.png)
![[image loading]](http://i.imgur.com/V9vzhv8.png)
![[image loading]](http://i.imgur.com/zPqHbXY.png)
Player Ratings:
![[image loading]](http://i.imgur.com/Tgm0pr2.png)
![[image loading]](http://i.imgur.com/eEkPOmL.png)
![[image loading]](http://i.imgur.com/IXNBSBb.png)
Before I continue I want to point out a few things:
The first is that I never defined a Building Defensive Rating for players. This is because you simply can’t measure how well a single player defends buildings; teams defend buildings, not players. Not only that, there's no way to tell if a player is creating space or just feeding.
The second is that the Player statistics are more than a little dubious, since again with current stats parsing of replays it’s very hard to trace many game mechanics to a player. For example, when a Dazzle casts Weave on a team we don’t know how much damage increase or decrease that Weave actually caused, and that any damage amplification or reduction is credited not to Dazzle but another hero. Likewise, we don't know how much damage Magnus actually contributes with his Empower, because the damage ends up getting credited to another hero. No doubt this would require a very detailed replay parser.
And one more thing: for Offensive Rating a higher value is stronger while with Defense Rating a lower value is stronger. Remember that we’re tallying damage done and damage taken.
Some Applications
Using our newly defined metrics we can then calculate it with some actual matches. I checked out DatDota and scraped all match data from Frankfurt Major to get the dataset, and then calculated average metrics for every team, every player, and every player-hero combo.
Team Ratings
Top 3 Average Team Offensive Hero Ratings
- Virtus.Pro
- Alliance
- CDEC
A quick look at the teams: Virtus.Pro: wildly aggressive team. CDEC: calmly aggressive team. Alliance: aggressive team? Interestingly enough, the rat lords Alliance adopted an aggressive teamfight mindset at Frankfurt that also funnily enough landed them with the lowest average Defensive Hero Rating of any team. Just watch the VoDs.
With that knowledge, then this list isn't really surprising.
Top 3 Average Team Offensive Building Ratings
- OG
- Secret
- Virtus.Pro
If you realize that that it’s the same exact order as the teams with the highest winrates at Frankfurt, and that teams that win usually kill much of the other team’s buildings to win, this order is really a no-brainer. Next.
Top 3 Average Team Defensive Hero Ratings
- Evil Geniuses
- CDEC
- OG
Arguably these were the three most disciplined and cleanest styled teams at Frankfurt. Ones that tried to do the most while taking as little losses as possible, so it is nice to see them show up all here.
Top 3 Average Team Defensive Building Ratings
- OG
- EHOME
- Secret
While OG being in the top spot isn’t any surprise since they won Major, the 4th place team EHOME snuck in ahead of 2nd place Secret. The likely explanations for this are that while Secret were not afraid of giving up buildings for more important objectives like Roshan, EHOME never let go of their buildings without any kind of resistance, and when EHOME's base was breached, the game typically ended very swiftly.
Player Ratings
Top 10 Picked-Hero-Player Average Ratings Rankings (filtered players with < 3 games):
I find that for players these sorts of rankings are pretty indicative of what kind of player they are, perhaps more than skill level. Sometimes you can see that a hero played in two different kinds of positions will excel at doing different things. (Personally I feel this part is more for fluff than meat atm, it is really too basic, but you can see some of the interesting tidbits)
![[image loading]](http://www.teamliquid.net/staff/rabidch/chart.png)
And before you guys ask, Julz played exactly 3 games of Dazzle, and all 3 games were wins but 2 were behind Kuku's Huskar.
I thought about including the top 10 rankings for the player statistics but the more I thought about it, I felt as if it was pretty meaningless information. Currently still working on something better.
Weaknesses and Things to Work On
I can think of a number of issues that the naive model proposed here doesn't bother with. Some include it not taking into account the importance of damage happening at the right time and place, effects of game lengths and certain periods of time on efficiency, how fast the teams accrue gold and levels, so on and soforth.
An example: an Antimage who doesn’t get to use his farm because his team lost before he could has terrible efficiency because they stuck all that worth on a hero that hasn’t come online yet. See OG versus Vega Game 2. Of course a better model of efficiency should take that into consideration or at least show that the team might not be effective at bringing Antimage online in time, making a gameplay or draft mistakes. Add on all of the things I mentioned in the earlier sections, there's a lot of situations the model can improve on.
Afterthoughts
We all know that Dota is a complex game. The amount of impact a player has on the game is more than just KDA and GPM and Net Worth and more. However, it’s likely impossible to encapsulate efficiency with just the current numbers we have, which is why I went with a very simple and naive approach. With some kind of start, hopefully better models down the road will be able to turn the numbers from the replay into numbers that mean at least a little bit something about performance and efficiency.
I think important it is to keep methodology open. It’s hard to verify and scrutinize a metric that is calculated in a black box and also it’s hard for other people to improve upon it. Nothing worse than going to a website looking at a stat and not being told how they obtain their rankings, because then its as good as their banter to you!
Part 2 is incoming where I look at the entire 6.85 patch and attempt to correlate efficiency and winrates with numbers and stuff more weighty than just words.
Tools used for analysis:
DatDota (which I may have to eventually stop using due to its limitations...), Python, and good ol' Excel.
***comic sans was intentional
|
|
|
Nice blog.
One thing I feel could be considered in stats is how meaningful your kills/deaths are. For example a simple starting point would be to assume that your goal as a team is to acquire more resources from the map than the opposing team, in the form of gold and experience. Each kill leads to certain events happening in the next minute or so. First there could be a kill where an important hero dies to a gank right in front of their tower that they were preparing to defend. Now the defending team is not farming but they can't defend either, resulting in free building kills and free farming time for the other team. Alternatively there could be a time where PLD is playing his BH and cuts off a smoke in a crucial location. This results in the BH dying but the rest of the team is free to farm, and the other team has to waste time running around the map to deal with the split push or finding their own farming locations. Essentially in that case the death wouldn't be considered bad at all because it can even result in a resource gain for your team in the next minute, while the death right in front of your tower is costly because it leads to a heavy resource loss. This way a simple KDA could be transformed to something that takes into account how good or bad your kills, deaths and assists are.
|
On January 18 2016 22:35 spudde123 wrote:
Nice blog.
One thing I feel could be considered in stats is how meaningful your kills/deaths are. For example a simple starting point would be to assume that your goal as a team is to acquire more resources from the map than the opposing team, in the form of gold and experience. Each kill leads to certain events happening in the next minute or so. First there could be a kill where an important hero dies to a gank right in front of their tower that they were preparing to defend. Now the defending team is not farming but they can't defend either, resulting in free building kills and free farming time for the other team. Alternatively there could be a time where PLD is playing his BH and cuts off a smoke in a crucial location. This results in the BH dying but the rest of the team is free to farm, and the other team has to waste time running around the map to deal with the split push or finding their own farming locations. Essentially in that case the death wouldn't be considered bad at all because it can even result in a resource gain for your team in the next minute, while the death right in front of your tower is costly because it leads to a heavy resource loss. This way a simple KDA could be transformed to something that takes into account how good or bad your kills, deaths and assists are.
It's very difficult to have an accurate mathematical model for it however. It's very difficult to decide what exchange rate you should have between gold and experience, and it always varies.
Dota 2 truly is such a complicated game in terms of math. When I was playing, I'd always do calculations for maximizing DPS to decide which items are ideal, but even then, there are so many variables to consider, saying whether Buterfly or Daedalus is preferential is extremely difficult because you have to take risk into consideration, and how you treat it (evasion or crit), the value of flutter, etc, etc.
edit: Really nice article btw, I'll finish reading it today.
|
On January 18 2016 22:18 dismiss wrote:
Are you Shikiyo?
his name is Shikyo
On January 18 2016 22:35 spudde123 wrote:
Nice blog.
One thing I feel could be considered in stats is how meaningful your kills/deaths are. For example a simple starting point would be to assume that your goal as a team is to acquire more resources from the map than the opposing team, in the form of gold and experience. Each kill leads to certain events happening in the next minute or so. First there could be a kill where an important hero dies to a gank right in front of their tower that they were preparing to defend. Now the defending team is not farming but they can't defend either, resulting in free building kills and free farming time for the other team. Alternatively there could be a time where PLD is playing his BH and cuts off a smoke in a crucial location. This results in the BH dying but the rest of the team is free to farm, and the other team has to waste time running around the map to deal with the split push or finding their own farming locations. Essentially in that case the death wouldn't be considered bad at all because it can even result in a resource gain for your team in the next minute, while the death right in front of your tower is costly because it leads to a heavy resource loss. This way a simple KDA could be transformed to something that takes into account how good or bad your kills, deaths and assists are.
i know where you're coming from, i've thought about this for a while really and really you'd have to code a rather advanced replay parser to take in the context of each event happening in game. very difficult and time consuming, since you need to determine positive and negative contexts, maybe if somebody hooked it up through yasp or something.
thats why stats people would rather work with the simple data of which they got a lot of and the opposite way of that which is work from the big picture to conjecturing things about small pictures
On January 19 2016 02:48 FiWiFaKi wrote:Show nested quote +On January 18 2016 22:35 spudde123 wrote:
Nice blog.
One thing I feel could be considered in stats is how meaningful your kills/deaths are. For example a simple starting point would be to assume that your goal as a team is to acquire more resources from the map than the opposing team, in the form of gold and experience. Each kill leads to certain events happening in the next minute or so. First there could be a kill where an important hero dies to a gank right in front of their tower that they were preparing to defend. Now the defending team is not farming but they can't defend either, resulting in free building kills and free farming time for the other team. Alternatively there could be a time where PLD is playing his BH and cuts off a smoke in a crucial location. This results in the BH dying but the rest of the team is free to farm, and the other team has to waste time running around the map to deal with the split push or finding their own farming locations. Essentially in that case the death wouldn't be considered bad at all because it can even result in a resource gain for your team in the next minute, while the death right in front of your tower is costly because it leads to a heavy resource loss. This way a simple KDA could be transformed to something that takes into account how good or bad your kills, deaths and assists are. It's very difficult to have an accurate mathematical model for it however. It's very difficult to decide what exchange rate you should have between gold and experience, and it always varies. Dota 2 truly is such a complicated game in terms of math. When I was playing, I'd always do calculations for maximizing DPS to decide which items are ideal, but even then, there are so many variables to consider, saying whether Buterfly or Daedalus is preferential is extremely difficult because you have to take risk into consideration, and how you treat it (evasion or crit), the value of flutter, etc, etc. edit: Really nice article btw, I'll finish reading it today.
i wish there was some sort of granularity i could get with my data so I could apply it to items but cant ask for everything i suppose
|
also secret is not too badly off compared to players on other positions. even pieliedie scores well by posting very good HR DR differences by having stupidly high HR to offset his high DR. the exception is misery who scores really poorly compared to other offlaners (except doom). some people already argue hes the weak link of secret.
|
Yea you can't get it straight away from datdota but the data for something like that is quite easy to get from replays using the open source parsers available. I've dabbled around with extending a replay parser and did some features that I wanted to try out for watching a replay out of client on a minimap view.
But of course depends on the feature how difficult it would be to extend. In case you are interested in studying something specific I might be able to help with parsing.
|
Why is a high hero defensive rating supposed to be good?
|
He says that with the defensive ratings a lower value is better.
Also am I getting confused or is there a difference between Hero Damage for team 1 and Damage Taken for team 2?
|
On January 19 2016 09:47 spudde123 wrote:
He says that with the defensive ratings a lower value is better.
Also am I getting confused or is there a difference between Hero Damage for team 1 and Damage Taken for team 2?
there is a difference
|
Which is what? I suppose damage taken can count damage from creeps, towers and whatever but I am not completely following how that relates to the defensive hero rating for teams.
|
On January 19 2016 10:10 spudde123 wrote:
Which is what? I suppose damage taken can count damage from creeps, towers and whatever but I am not completely following how that relates to the defensive hero rating for teams.
that is the difference.
i actually currently calculate both values since its easy (for the small dataset of frankfurt it didnt matter most of the time). of course i would want to use something more accurate but atm i am using damage taken because
it lines up with damage taken from players (which i also want to eventually move away from)
the damage instances taken from creeps and what not is probably important
|
Northern Ireland22208 Posts
where are the graphs rabidchchchch
|
On January 20 2016 01:15 ahswtini wrote:
where are the graphs rabidchchchch
part 2
|
|
|
An handy statistic would be #building_damage done during respawn_time+30 for each death, grouping data for networth_position, and trying to elaborate how it relates to winrate. Roshan should be counted as a full tower worth of damage.
|
On January 23 2016 00:11 Dekalinder wrote:
An handy statistic would be #building_damage done during respawn_time+30 for each death, grouping data for networth_position, and trying to elaborate how it relates to winrate. Roshan should be counted as a full tower worth of damage.
dont have the time to implement 1st point, but already working on the other 2 points. i hope to cover them better in the next few parts but its taking a bit of time
|
This seems nice. Do you know if any replay parser converts EHP changes? Raw HP changes do not fully reflect a hero's impact.
Dazzle, for example, doesn't just heal and damage, he changes EHP too. Healing someone with high damage mitigations has more value than healing the someone with low or even negative mitigations. He also boosts the damage of his allies. A well placed weave causes up to a (6%EHP per armour * 30 armour * 2 different directions)=360% Physical EHP difference between 2 opposing heroes affected by the ultimate (assuming the enemy had more than 30 armour as negative armour gives different EHP per value than positive armour).
Are there any plans to include things like "evaded" damage? A well placed Disruption from a Shadow Priest can prevent the damage of a Laguna Blade (it takes 0.25s for the blade to do damage). There is also things like Glimmer cape that disjoints projectiles and save allies due the invis.
Disables and Slows are also a form of damage prevented. Then, there are also things like force staves, Nether Swap and etc too which reduces potential damage thanks to kiting.
|
On January 23 2016 12:45 Clogon wrote:
This seems nice. Do you know if any replay parser converts EHP changes? Raw HP changes do not fully reflect a hero's impact.
Dazzle, for example, doesn't just heal and damage, he changes EHP too. Healing someone with high damage mitigations has more value than healing the someone with low or even negative mitigations. He also boosts the damage of his allies. A well placed weave causes up to a (6%EHP per armour * 30 armour * 2 different directions)=360% Physical EHP difference between 2 opposing heroes affected by the ultimate (assuming the enemy had more than 30 armour as negative armour gives different EHP per value than positive armour).
Are there any plans to include things like "evaded" damage? A well placed Disruption from a Shadow Priest can prevent the damage of a Laguna Blade (it takes 0.25s for the blade to do damage). There is also things like Glimmer cape that disjoints projectiles and save allies due the invis.
Disables and Slows are also a form of damage prevented. Then, there are also things like force staves, Nether Swap and etc too which reduces potential damage thanks to kiting.
atm i dont know of one that does and i dont plan to implement any better replay parsing method, because i simply dont have the time. as stated in other posts, i am currently working on correlating damage/gold with winrate with the entire 6.85 before i move onto more a more complicated model.
your second point is important and i already mentioned the effects of spells like dazzle's weave etc. as something that needs to be addressed. unfortunately yeah i cant just solve it on my own atm, which is really a killer for how seriously the player ratings can be taken.
for your third point, of course disables and slows are a form of damage prevented, but the simple and naive model actually takes care of it with a dumb and naive way: it simply just assumes that a better team would take less damage than a worse team in the same situation. of course not all teams do the same things, but maybe in the future items can be correlated with better defensive damage efficiency against certain heroes/items.
|
On January 23 2016 09:39 rabidch wrote:Show nested quote +On January 23 2016 00:11 Dekalinder wrote:
An handy statistic would be #building_damage done during respawn_time+30 for each death, grouping data for networth_position, and trying to elaborate how it relates to winrate. Roshan should be counted as a full tower worth of damage. dont have the time to implement 1st point, but already working on the other 2 points. i hope to cover them better in the next few parts but its taking a bit of time
I know man, if it was easy I would have done it myself.
Just kidding I'm to lazy, I would still have waited for someone else to do it ^_^
|
|
|
|
|
|
EPT
