EPTWin
GSL Season 3 predictions using statistics - Page 2
| Forum Index > StarCraft 2 Tournaments |
|
MrPello
Sweden187 Posts
Win | ||
|
DarKFoRcE
Germany1215 Posts
Other flaws have already been pointed out by others: you forget that players got better over time, yet you raise their rating for past games using their future results... So all in all, it might be a good practise for your studies, and maybe you can summarize the past (although, when looking at the ranking your model produced, im not so sure about that  ). But please, dont use the model to try to make projections for the future, because i have a feeling that if i randomly assign winpercentages between 50/50 and 70/30 to the players, my "accuracy" would probably not that much lower than the one of your model. If the accuracy of a model is not much better than just randomly putting in numbers, maybe its not yet time to use it for projections. | ||
|
Myia
173 Posts
I actually like this analysis, and would be interested in more of the theory crafting behind this write up. Nice work  | ||
|
kazansky
Germany931 Posts
On December 08 2010 21:38 DarKFoRcE wrote: So all in all, it might be a good practise for your studies, and maybe you can summarize the past (although, when looking at the ranking your model produced, im not so sure about that ). But please, dont use the model to try to make projections for the future, because i have a feeling that if i randomly assign winpercentages between 50/50 and 70/30 to the players, my "accuracy" would probably not that much lower than the one of your model. If the accuracy of a model is not much better than just randomly putting in numbers, maybe its not yet time to use it for projections.The nice thing about mathematical models is that they do not claim to be truth, and interpretation oblies the reader (a fact that economist forget all the time). So when you don't think the model will be accurate enough (which I dont either), there is no need to trust it :-) After rethinking you're model, you should at least if you don't seperate matchups seperate the map score into 3 different sets. Because if you don't, if someone wins a tournament he gets 7 sets of 1.0 probability samples which hurts the reality by quite a measure. On advantage, weighting with the probabilities (ELO like) would be a possible improvement. | ||
|
aka_star
United Kingdom1546 Posts
| ||
|
kazansky
Germany931 Posts
On December 08 2010 22:20 aka_star wrote: I don't honestly know how you can model the probability of the players, it just blows my mind how complex putting a value on a player could be. It would says nothing about a winning strategy or the countless variables of real day events but seems to me that this system focuses more on averaging out past performance which following a market or a horse in its career is no guarantee. and even more sporadic the lesser the data. I suppose its a better guide than anything but I'm convinced this method would in itself require a probability of being right. You would be surprised. There are several professional booking companies in the UK that have specialized on betting on football matches. Their model does only incorporate past match data and does hit almost 90% for win tendencies, which is unbelievably high for football. The model is secret for obvious reasons but german journalist Christoph Biermann wrote a book about it. | ||
|
strongandbig
United States4858 Posts
| ||
|
Fa1nT
United States3423 Posts
I cracked a smile @ TSL_Rain havin a higher % than Jinro though,  | ||
|
traca
146 Posts
| ||
|
Heimatloser
Germany1494 Posts
i absolutely dont want to say that you did no nice work here, but i have to go with the common opionon in this thread and say that your rankings seem to be utter bullshit. PS: could you please add 12 more ppl to your rankings, i would love to see whom you would send to code S league! ( http://www.teamliquid.net/forum/viewmessage.php?topic_id=170463 ) | ||
|
Mip
United States63 Posts
To all wanting errors associated with predictions, I have them, the reason I didn't include them is purely because of not knowing how to make tables to post on these forums. I didn't want to dump too much unformatted data. If someone could give me a quick how-to on tables and images for these forums, I can make this more readable. I'm planning on doing another one of these for the finals, and I'd like to make it much more readable. The methodology is sound, and yeah, there should be a time effect in the data, but it's really just not feasible right now with the limited sample size. This makes predictions highly volitile The original post I really just threw together in the middle of the night while I was working through my output. I'll have some more time after this weekend to do a proper write-up in excruciatingly more detail. To Heimatloser, the predictions are based on draws from the posterior predictive distribution and significance doesn't mean very much in Bayesian statistics. Uncertainty about skill level is averaged out in the calculations. I will tell you that the variance associated with the predictions is large and I reported only the average. The results you see are highly sensitive to additional data, which as I've said repeatedly, is the biggest problem I have. To strongandbig, the problem with just adding a lot of non-korean tournaments is it adds a lot of more players where the biggest weakness of the model is not enough information on the current players. But I'm totally up for exploring more tournaments, and I want to go in that direction. If there's anyone out there that wants to help me find data, that would be awesome. Oh and thanks to whomever suggested I use my predictions on Liquidbets, that is actually an amazing data source because they have results + races + maps. If there's anyone out there that's really good at parsing html code or just the realizations on the webpages, I could really use help with that. Copy-pasting the brackets works pretty well, but it doesn't grab the race information, and the map information is not there at all. | ||
|
CrAzEdBaDgEr
Canada166 Posts
| ||
|
Centorian
United States95 Posts
Although to be realistic, historical data is only semi-valid in any sporting event (ie. upsets happen all the time). | ||
|
WeeKeong
United States282 Posts
| ||
|
greycubed
United States615 Posts
| ||
|
CrAzEdBaDgEr
Canada166 Posts
On December 09 2010 01:51 Centorian wrote: I think your model is sound enough. As stated some kind on time factor should be added... but you don't have enough data for that. In reality I think thats why there are such messed up stats. It's a matter of the amount of data that you have. I think a similar model (with some kind of time variable) could be very accurate once we have more games to base scores off of. Then when a lower ranked player take a series off of a highly ranked player it will make less of a difference because your lower score is based off of say 30-40 games instead of 5-10. Although to be realistic, historical data is only semi-valid in any sporting event (ie. upsets happen all the time). I would argue historical data is fully valid, and the very best measure to take when predicting future results. But no statistical model is perfect, and it will be wrong some of the time, especially when there's not a lot of data available. Just because there are upsets doesn't mean the favorite won't win most of the time, so if you had to make a prediction, unless there is some highly convincing information external to the model, it would make sense to bet on the favorite at 1:1 odds. | ||
|
odaxium
United States356 Posts
MLG would probably be a good, since it has double elimination and more tournaments to pool data from. | ||
|
Cel.erity
United States4890 Posts
On December 08 2010 22:24 kazansky wrote: You would be surprised. There are several professional booking companies in the UK that have specialized on betting on football matches. Their model does only incorporate past match data and does hit almost 90% for win tendencies, which is unbelievably high for football. The model is secret for obvious reasons but german journalist Christoph Biermann wrote a book about it. The difference between football and Starcraft is variance, especially in SC2. Football teams have a lot of players, so the impact of one players having a bad/good day is relatively low compared to a team of one. If the solo player has a bad/good day, it skews the results immensely. Also, football teams have faced each other many times in the professional arena, so there is a lot more data to draw upon. SC2 is also a new game with evolving strategies and nobody is at the top level yet, making the data even more inconsistent. Finally, I don't believe the formula accounts properly for player skill difference. In SC2, a player who is just slightly better than another will almost never lose on a favorable map, even though the data says it's 60/40. I think it's a good effort, but I don't believe there is any formula that can rate SC2 players right now with any degree of accuracy. This would be better applied to BW where the data, players, and maps are more consistent. | ||
|
SolonTLG
United States299 Posts
The ATP uses a 52-week cumulative, rolling point system that awards points by finishing place at each qualifying tournament. However, more important tournaments are awarded more points, and the weights are approximately determined by the financial rewards from the tournaments. That is, the potential financial gain determines the "level" of the tournament, but other feedback from the community could be used as well. I think a similar system would work well for SC2 because it and tennis share some (generally) common characteristics: 1) 1 vs. 1 tournaments with elimination brackets 2) Different surfaces (grass, clay, hard court) and conditions (day,night, hot, cold, windy) are approximate to different races and maps in SC2 3) Not all players play in all the tournaments In the same way that the "Upcoming Events" leverages the community's knowledge, one could develop database or results form where the community could help submit tournament results. Obviously, all ranking systems have their pros and cons, so only constructive criticism please! Finally, this ranking system (or an analogous one) can then inform the prediction models. | ||
|
SolonTLG
United States299 Posts
Obviously, since there are so many small SC2 tournaments, some subjective measures need to be used to build the pool of tournaments that would in an ATP-style ranking system. An initial list includes: GSL(s) MLG(s) IEM(s) Dreamhack Blizzcon I am sure there are others that would be good to also include, but I think a really good place to start would be the Blizzard-licensed events (all of the above are included I think). Does someone have a comprehensive list of the official Blizzard events? Other thoughts? | ||
| ||
