EPTGeneral Discussion - Page 4468
| Forum Index > Dota 2 General |
IRC chatter should remain in IRC - http://webchat.quakenet.org/?channels=tl.dota2 Posts that relate to topics with their own thread, such as in News, Tournaments or Strategy should go in those threads. | ||
|
Kuroeeah
11696 Posts
| ||
|
BurningSera
Ireland19621 Posts
On December 05 2013 17:58 hariooo wrote: http://www.reddit.com/r/DotA2/comments/1s4ezx/mydotaskillcom_and_why_it_should_be_ignored/cdtubbc Hidden profiles shouldn't matter. Yes it does very much so matters, unless some people just wanted a number to say something meaningless about their skill/ranking then it shouldn't matter to them i guess (or anything matters to them in that case, hence the ridiculous ranking from the website). You can talk about fancy stats techniques etc but in the end of day I hope people know that the site is a joke lol (a bigger joke than DBR if you must, and i actually respect the stats guys behind dotabuff now because they actually have the self-respect and morale as some statisticians. say 1000 out of 100000 players are actual top class player, ill give you 800 of them have the profile opened for public because high level players surely care about their ranking (obviously i am being very generous here, go search dendi pub games you will almost always find 1-5 players that are anonymous), so in this case there are 200 top class players that will never be included in the stats results, ie the ranking of these 800 players will forever be not accurate because we know there are differences between top top top top class players and top class players. and then lets say 40000 out of the remaining 99000 players have made their profile public, that's a very generous number because that would mean almost 40% of the players are showing their profile. I think I'll stop here now because if you still think that DSR means anything then there is nothing more i say can persuade you to think otherwise lol. p/s: i am guessing that DSR is using somesort of 'curve fitting' model to 'predict' people rankings based on defining their games but in this case of dota2 player pool, it wont be a meaningful curve. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 18:14 Staboteur wrote: Wut? Correct you are, I have no background in statistics, but perhaps you could enlighten me? To those with a background in statistics (I assume!), 65% prediction accuracy over a thing with as many unpredictable variables as a dota 2 match is at the very least an indication that whatever changes they've been making to the formulas can be regarded as progress - The closer this number can get to the point where human error makes it impossible to predict further (A number I imagine to be higher than 65%?) the more accurate DSR can be considered to be. Looking at these numbers as a player, that level of accuracy should also be considered to be reflected in your rating; A system prone to being wrong 35% of the time might be "close" to giving you a rating of some value, but chances are if it's only shooting 15% above par, it's more of a ballpark than a pinpoint. It'd be interesting to hear from them about how accurate they are within specific margins... I.E. if a team with a DSR sum 500 less than their opponents results in a higher margin of correct predictions, or what the average DSR variances are in pub games / in their tests. Presumably the creator used DSR to predict the winner over a significant number of matches. He has commented that in essence, this 65% was calculated by looking at the two teams' combined DSR's and predicting that the higher DSR (by 1 or more) would win. To get a 65% prediction rate from that is highly significant. Over a large enough sample size of games, this becomes a statistical fact. Do not assume anything about statistics. It's not intuitive. It's a little like if I were to suggest to a doctor what I think the diagnosis should be based on no medical knowledge but just gut feeling. I'll give you a more relatable example. The House wins at Blackjack just 51% of the time. We know that, without counting cards, this edge is enough for the House to win consistently. It's certainly not a "ballpark" figure or anything like that. | ||
|
hadang
Germany941 Posts
| ||
|
Jetaap
France4814 Posts
As to why 65% is a very good rate... keep in mind that Valves matchmaking will try to make teams that are as even as possible so the odds of one team winning upon another should never be too far from 50%, therefore in order to have a 65% prediction rate DSR would have to be extremely highly correlated to the MMR or maybe be even better at find the favored team than ELO based MMR. Honestly, it's hard to believe that they could achieve such a high score on a big sample. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 18:45 BurningSera wrote: Yes it does very much so matters, unless some people just wanted a number to say something meaningless about their skill/ranking then it shouldn't matter to them i guess (or anything matters to them in that case, hence the ridiculous ranking from the website). You can talk about fancy stats techniques etc but in the end of day I hope people know that the site is a joke lol (a bigger joke than DBR if you must, and i actually respect the stats guys behind dotabuff now because they actually have the self-respect and morale as some statisticians. say 1000 out of 100000 players are actual top class player, ill give you 800 of them have the profile opened for public because high level players surely care about their ranking (obviously i am being very generous here, go search dendi pub games you will almost always find 1-5 players that are anonymous), so in this case there are 200 top class players that will never be included in the stats results, ie the ranking of these 800 players will forever be not accurate because we know there are differences between top top top top class players and top class players. and then lets say 40000 out of the remaining 99000 players have made their profile public, that's a very generous number because that would mean almost 40% of the players are showing their profile. I think I'll stop here now because if you still think that DSR means anything then there is nothing more i say can persuade you to think otherwise lol. p/s: i am guessing that DSR is using somesort of 'curve fitting' model to 'predict' people rankings but in this case of dota2 player pool, it wont be a meaningful curve. Holy fuck. You're trying to argue against something that already happens in real life. Let me see if I can break this down really simply for you with a braindead example. When your country (I'm assuming the UK does these) conducts a census of the population, they do not collect data from every single person living in the UK. They collect an INCOMPLETE data set. Yet they can still draw conclusions from that data set like who the expect certain demographics to vote for, if there is mass migration from rural to urban districts, etc. Here's another example because of how adamantly you want to argue on a topic with which you clearly have no academic or professional experience. When Nate Silver perfectly predicted the 9 swing states for the US 2012 election, I PROMISE YOU he did not collect data from every single voter in those states. Drawing accurate conclusions from incomplete data is done ubiquitously all over the world in many different fields. DOTA2 is just one of them. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 18:59 Jetaap wrote: Do you have the source for this 65% prediction rate? If that's true that sound quite good actually, even a little bit too good to be true to be honest. As to why 65% is a very good rate... keep in mind that Valves matchmaking will try to make teams that are as even as possible so the odds of one team winning upon another should never be too far from 50%, therefore in order to have a 65% prediction rate DSR would have to be extremely highly correlated to the MMR or maybe be even better at find the favored team than ELO based MMR. Honestly, it's hard to believe that they could achieve such a high score on a big sample. http://mydotaskill.com/faq.php | ||
|
Jetaap
France4814 Posts
Site is dead for me. Honestly I have trouble believing what this guy is saying, this whole thing needs much more transparency, i wish he would give more information on how the ratings are computed, how it's distributed etc.. . | ||
|
hariooo
Canada2830 Posts
On December 05 2013 19:04 Jetaap wrote: Site is dead for me. Honestly I have trouble believing what this guy is saying, this whole thing needs much more transparency, i wish he would give more information on how the ratings are computed, how it's distributed etc.. . http://webcache.googleusercontent.com/search?q=cache:http://mydotaskill.com/faq.php cached version People have trouble believing this guy because even the concept that the algorithm is based on requires an undergrad degree to understand. That doesn't mean he's right, but it also means people who don't know what the fuck is going on freak the fuck out. I'm sure most people don't even really understand how ELO works for 1v1 even though the formula is dead simple. They whine about that too in its various incarnations (in LoL, SC2, etc.) even though mathematically it's very rigorous. | ||
|
-Celestial-
United Kingdom3867 Posts
| ||
|
njt7
Sweden769 Posts
| ||
|
hfglgg
Germany5372 Posts
you can not compare the two ratings (DSR of both accounts) though. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 19:13 -Celestial- wrote: I think hariooo either has a really high DSR or is one of the guys behind DSR to be honest given the crusade to defend it here. To be honest if that's your takeaway when the half the thread who don't know what a confidence interval is decide to express what they think is statistically valid or not, that's pretty ignorant. Or maybe your DSR got dumpstered. | ||
|
Asha
United Kingdom38257 Posts
On December 05 2013 19:13 -Celestial- wrote: I think hariooo either has a really high DSR or is one of the guys behind DSR to be honest given the crusade to defend it here. Maybe he just likes that somebody has at least tried to make a rating system for the lots of people who want one? As long the guy is not doing it for personal profit I see no reason not to presume that he's genuinely trying to come up with a workable model for rating players. It's a bit sad some people slam him in completely un-constructive ways for putting himself out there and trying to offer a service nobody else is attempting. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 19:20 hfglgg wrote: even if they could predict the outcome of a match, it doesnt mean the individual scores are comparable. in an earlier version the two accounts of a friend of mine had a difference of about 500 points with the by valve lower ranked account having the higher number. its save to say that the account with the higher DSR rating, but lower MMR will win more often than the other account with lower DSR but higher MMR because it is underrated. you can not compare the two ratings (DSR of both accounts) though. There's really no point talking about earlier models. They were changed because they were bugged, according to the guy at least. And of course you can compare two accounts' DSR's? World of Tanks has a non-ELO rating system that their players use to compare skill level in a team game. DSR is not exactly a unique concept. The creator just has to adapt it to DOTA's unique performance metrics and determine what parameters are significant. | ||
|
Jetaap
France4814 Posts
On December 05 2013 19:11 hariooo wrote: http://webcache.googleusercontent.com/search?q=cache:http://mydotaskill.com/faq.php cached version People have trouble believing this guy because even the concept that the algorithm is based on requires an undergrad degree to understand. That doesn't mean he's right, but it also means people who don't know what the fuck is going on freak the fuck out. I'm sure most people don't even really understand how ELO works for 1v1 even though the formula is dead simple. They whine about that too in its various incarnations (in LoL, SC2, etc.) even though mathematically it's very rigorous. lol why are you so agressive and why do you automatically assume that nobody criticizing the DSR has no education whatsoever? And people have trouble believing the guy because all we know about his algorithm is that it uses a "few machine learning algorithm techniques". It can hardly be more vague than that. Apparently he is planning to open source it so i guess we'll see how it goes. For your information i do know what a confidence interval is, and i'm sure a lot of people on this forum do as a lot of them have a background in science/IT etc. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 19:29 Jetaap wrote: lol why are you so agressive and why do you automatically assume that nobody criticizing the DSR has no education whatsoever? And people have trouble believing the guy because all we know about his algorithm is that it uses a "few machine learning algorithm techniques". It can hardly be more vague than that. Apparently he is planning to open source it so i guess we'll see how it goes. Please point me to a single criticism from such an educated individual. I'm responding to people who think 20 is too small of a sample size based on instinct and people who think you need data from 100% of the population for analysis to be worthwhile. People are being aggressively ignorant and statistics is one of those topics where laymen aren't ashamed to "contribute" their own opinion for some reason.. | ||
|
hfglgg
Germany5372 Posts
On December 05 2013 19:27 hariooo wrote: There's really no point talking about earlier models. They were changed because they were bugged, according to the guy at least. And of course you can compare two accounts' DSR's? World of Tanks has a non-ELO rating system that their players use to compare skill level in a team game. DSR is not exactly a unique concept. The creator just has to adapt it to DOTA's unique performance metrics and determine what parameters are significant. my underlying point still stand that its not neccessarily possible to compare two ratings only because they can predict the outcome of a match. so the whole "we can predict xyz (even if true)" doesnt matter anything unless they provide full information about what DSR is. | ||
|
hariooo
Canada2830 Posts
On December 05 2013 19:34 hfglgg wrote: my underlying point still stand that its not neccessarily possible to compare two ratings only because they can predict the outcome of a match. so the whole "we can predict xyz (even if true)" doesnt matter anything unless they provide full information about what DSR is. What's the evidence for your statement? Can you give me a good example of when you can't compare 2 DSR's if they are valid parameters to predict wins? | ||
|
SnowfaLL
Canada730 Posts
Either way, I play only support so my KDA and GPM/XPM is much lower than others, therefor any metric calculating those in is false. The only thing that should matter is wins. | ||
| ||
