|
On August 02 2009 12:45 JWD wrote:Show nested quote +On August 02 2009 12:38 igotmyown wrote: There would be much less variance if you analyzed a lot more games, regardless of the population size. There's no way you can say TL is more accurate when changing one game could potentially flip the results by 2*(difference)*1/16 = .125*difference, assuming everyone votes. Of course it's blatantly obvious that I cannot conclude, based on a sample size of 24 games, that the .01% difference in accuracy between TL and YGosu voters is statistically significant. Are you suggesting that somewhere I said "TL is more accurate"?...I'm not sure what you're getting at.
I decided to pit TeamLiquid's Liquibetters against YGosu's Lottoers by comparing each group's collective predictive capability over the entire Bacchus OSL 2009 Ro16.
As the above graph shows, TL just barely edges out YGosu in predictive capability over the OSL Ro16 — but neither site's voting population demonstrated itself to be significantly more intelligent than a monkey (perfectly random voter). Uhhh...let's preserve our dignity by chalking that up to this Ro16's many upsets.
I dunno I kind of assumed predictive capability implied accuracy. That and all the WOOT we're slightly better! replies. Also if you look into the numbers, it's really hard to "bet" accurately your chance of winning is probably less than 60%. I'm sure there's some system that uses ranking odds.
I am saying that looking at a huge number of games (even if it meant looking at a smaller pool of voters) would been better.
|
United States12607 Posts
On August 02 2009 13:48 igotmyown wrote:Show nested quote +On August 02 2009 12:45 JWD wrote:On August 02 2009 12:38 igotmyown wrote: There would be much less variance if you analyzed a lot more games, regardless of the population size. There's no way you can say TL is more accurate when changing one game could potentially flip the results by 2*(difference)*1/16 = .125*difference, assuming everyone votes. Of course it's blatantly obvious that I cannot conclude, based on a sample size of 24 games, that the .01% difference in accuracy between TL and YGosu voters is statistically significant. Are you suggesting that somewhere I said "TL is more accurate"?...I'm not sure what you're getting at. Show nested quote +I decided to pit TeamLiquid's Liquibetters against YGosu's Lottoers by comparing each group's collective predictive capability over the entire Bacchus OSL 2009 Ro16. Show nested quote +As the above graph shows, TL just barely edges out YGosu in predictive capability over the OSL Ro16 — but neither site's voting population demonstrated itself to be significantly more intelligent than a monkey (perfectly random voter). Uhhh...let's preserve our dignity by chalking that up to this Ro16's many upsets. I dunno I kind of assumed predictive capability implied accuracy. That and all the WOOT we're slightly better! replies. Also if you look into the numbers, it's really hard to "bet" accurately your chance of winning is probably less than 60%. I'm sure there's some system that uses ranking odds. I am saying that looking at a huge number of games (even if it meant looking at a smaller pool of voters) would been better. Oh I agree more games would have been better...but also much more work Notice "over the entire Bacchus OSL 2009 Ro16" and "over the OSL Ro16"...I'm qualifying my statement that TL was more accurate by limiting its scope to just the OSL Ro16.
And I think you're misunderstanding something, because it is totally legitimate to say that TL was more accurate at predicting the OSL Ro16. That's a simple fact, as long as we define accuracy as % correct votes (...not that it could really mean anything else...). This isn't an issue of statistical significance because I am not extrapolating upon test results to make a statement about a larger population.
|
I think the korean votes were very weird... Are you sure they weren't getting odds to vote for the underdog?
|
i think it would be worthwhile to compare tl and ygosu over a longer period (imo atleast a year) to see if this .1% difference is a fluke or consistent.
|
Using TL's listed ELO ranks (the only reasonable way to statistically calculate odds) and a handy php script:
ELO predictions + Show Spoiler + Fantasy vs Typeb: 0.681321835166 Effort vs Yarnc: 0.56440049024 Fantasy vs Backho: 0.730245413297 Bisu vs Hwasin: 0.718756829899 Leta vs Hero: 0.579899760358 Leta vs Canata: 0.534483944727 Jaedong vs Effort: 0.58968350314 Zero vs Gogo: 0.582701884964 Stork vs Backho: 0.555890961117 Fantasy vs Stork: 0.683816314194 Jaedong vs Flash: 0.515824985265 Flash vs Effort: 0.574280203655 Canata vs Hero: 0.54592192278 Hero vs Kwanro: 0.502878199575 Canata vs Kwanro: 0.54877433586 Typeb vs Backho: 0.558731369325 Zero vs Hwasin: 0.531618297828 Gogo vs Hwasin: 0.448376452689 Kwanro vs Leta: 0.417298115036 Typeb vs Stork: 0.502878199575 Zero vs Bisu: 0.30753660742 Yarnc vs Flash: 0.363923100312 Gogo vs Bisu: 0.241305379853 Yarnc vs Jaedong: 0.349395372066
So the ELO approach predicted 18 out of 24. Averaging the odds gets 12.63 correct, or 0.526 with a SD of 0.1.
php code unintegrated with tlpd + Show Spoiler + <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> <title>ELO odds</title> </head>
<body> <form name="elo_odds" method="post" action="elo_odds.php"> Enter player 1 <select name = "player1"> <option value = "Jaedong">Jaedong</option> <option value = "Bisu">Bisu</option> <option value = "Flash">Flash</option> <option value = "Fantasy">Fantasy</option> <option value = "Effort">Effort</option> <option value = "Leta">Leta</option> <option value = "Yarnc">Yarnc</option> <option value = "Canata">Canata</option> <option value = "Zero" >Zero </option> <option value = "Kwanro" >Kwanro </option> <option value = "Gogo" >Gogo </option> <option value = "Backho" >Backho </option> <option value = "Typeb" >Typeb </option> <option value = "Hwasin" >Hwasin </option> <option value = "Hero" >Hero </option> <option value = "Stork" >Stork </option> </select> <!-- <input type = "text" name = "player1"> --> <br /> Enter player 2 <select name = "player2"> <option value = "Jaedong">Jaedong</option> <option value = "Bisu">Bisu</option> <option value = "Flash">Flash</option> <option value = "Fantasy">Fantasy</option> <option value = "Effort">Effort</option> <option value = "Leta">Leta</option> <option value = "Yarnc">Yarnc</option> <option value = "Canata">Canata</option> <option value = "Zero" >Zero </option> <option value = "Kwanro" >Kwanro </option> <option value = "Gogo" >Gogo </option> <option value = "Backho" >Backho </option> <option value = "Typeb" >Typeb </option> <option value = "Hwasin" >Hwasin </option> <option value = "Hero" >Hero </option> <option value = "Stork" >Stork </option> </select> <br /> <input type="submit" value="Process it!" /> <span style="color:#0000FF"><?php echo $alert ?></span> </form> <hr /> </body> </html>
<?php
if ( $_POST['player1'] && $_POST['player2'] ) { //echo $_POST['player1'] . $_POST['player2']; echo $_POST['player1'] . " vs ". $_POST['player2'] . ":\n" . odds( $_POST['player1'], $_POST['player2'] ); }
function odds( $player1, $player2 ) { $ELO[ Jaedong ]=2285;$ELO[Bisu]=2286;$ELO[Flash]=2274;$ELO[Fantasy]=2257; $ELO[ Effort ]=2222;$ELO[Leta]=2177;$ELO[Yarnc]=2177;$ELO[Canata]=2153; $ELO[Zero]=2145;$ELO[Kwanro]=2119;$ELO[Gogo]=2087;$ELO[Backho]=2084; $ELO[Typeb]=2125;$ELO[Hwasin]=2123;$ELO[Hero]=2121;$ELO[Stork]=2123; //print_r($ELO);
//echo $ELO[ $player1 ]; $Q1 = pow(10,($ELO[ $player1 ] / 400 )); $Q2 = pow(10,($ELO[ $player2 ] / 400 )); // echo $Q1 . $Q2; return ( $Q1/($Q1+$Q2) ); } /* $predictions = array(0.681321835166, 0.56440049024, 0.730245413297, 0.718756829899, 0.579899760358, 0.534483944727, 0.58968350314, 0.582701884964, 0.555890961117, 0.683816314194, 0.515824985265, 0.574280203655, 0.54592192278, 0.502878199575, 0.54877433586, 0.558731369325, 0.531618297828, 0.448376452689, 0.417298115036, 0.502878199575, 0.30753660742, 0.363923100312, 0.241305379853, 0.349395372066); $mean = 0; $variance = 0; foreach ( $predictions as $game ) { $mean += $game; $variance += $game * ( 1 - $game ); } echo "\n" . $mean . "\n" . sqrt( $variance ); */ ?>
|
On August 02 2009 16:50 igotmyown wrote:Using TL's listed ELO ranks (the only reasonable way to statistically calculate odds) and a handy php script: ELO predictions + Show Spoiler + Fantasy vs Typeb: 0.681321835166 Effort vs Yarnc: 0.56440049024 Fantasy vs Backho: 0.730245413297 Bisu vs Hwasin: 0.718756829899 Leta vs Hero: 0.579899760358 Leta vs Canata: 0.534483944727 Jaedong vs Effort: 0.58968350314 Zero vs Gogo: 0.582701884964 Stork vs Backho: 0.555890961117 Fantasy vs Stork: 0.683816314194 Jaedong vs Flash: 0.515824985265 Flash vs Effort: 0.574280203655 Canata vs Hero: 0.54592192278 Hero vs Kwanro: 0.502878199575 Canata vs Kwanro: 0.54877433586 Typeb vs Backho: 0.558731369325 Zero vs Hwasin: 0.531618297828 Gogo vs Hwasin: 0.448376452689 Kwanro vs Leta: 0.417298115036 Typeb vs Stork: 0.502878199575 Zero vs Bisu: 0.30753660742 Yarnc vs Flash: 0.363923100312 Gogo vs Bisu: 0.241305379853 Yarnc vs Jaedong: 0.349395372066
So the ELO approach predicted 18 out of 24. Averaging the odds gets 12.63 correct, or 0.526 with a SD of 0.1. php code unintegrated with tlpd + Show Spoiler + <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> <title>ELO odds</title> </head>
<body> <form name="elo_odds" method="post" action="elo_odds.php"> Enter player 1 <select name = "player1"> <option value = "Jaedong">Jaedong</option> <option value = "Bisu">Bisu</option> <option value = "Flash">Flash</option> <option value = "Fantasy">Fantasy</option> <option value = "Effort">Effort</option> <option value = "Leta">Leta</option> <option value = "Yarnc">Yarnc</option> <option value = "Canata">Canata</option> <option value = "Zero" >Zero </option> <option value = "Kwanro" >Kwanro </option> <option value = "Gogo" >Gogo </option> <option value = "Backho" >Backho </option> <option value = "Typeb" >Typeb </option> <option value = "Hwasin" >Hwasin </option> <option value = "Hero" >Hero </option> <option value = "Stork" >Stork </option> </select> <!-- <input type = "text" name = "player1"> --> <br /> Enter player 2 <select name = "player2"> <option value = "Jaedong">Jaedong</option> <option value = "Bisu">Bisu</option> <option value = "Flash">Flash</option> <option value = "Fantasy">Fantasy</option> <option value = "Effort">Effort</option> <option value = "Leta">Leta</option> <option value = "Yarnc">Yarnc</option> <option value = "Canata">Canata</option> <option value = "Zero" >Zero </option> <option value = "Kwanro" >Kwanro </option> <option value = "Gogo" >Gogo </option> <option value = "Backho" >Backho </option> <option value = "Typeb" >Typeb </option> <option value = "Hwasin" >Hwasin </option> <option value = "Hero" >Hero </option> <option value = "Stork" >Stork </option> </select> <br /> <input type="submit" value="Process it!" /> <span style="color:#0000FF"><?php echo $alert ?></span> </form> <hr /> </body> </html>
<?php
if ( $_POST['player1'] && $_POST['player2'] ) { //echo $_POST['player1'] . $_POST['player2']; echo $_POST['player1'] . " vs ". $_POST['player2'] . ":\n" . odds( $_POST['player1'], $_POST['player2'] ); }
function odds( $player1, $player2 ) { $ELO[ Jaedong ]=2285;$ELO[Bisu]=2286;$ELO[Flash]=2274;$ELO[Fantasy]=2257; $ELO[ Effort ]=2222;$ELO[Leta]=2177;$ELO[Yarnc]=2177;$ELO[Canata]=2153; $ELO[Zero]=2145;$ELO[Kwanro]=2119;$ELO[Gogo]=2087;$ELO[Backho]=2084; $ELO[Typeb]=2125;$ELO[Hwasin]=2123;$ELO[Hero]=2121;$ELO[Stork]=2123; //print_r($ELO);
//echo $ELO[ $player1 ]; $Q1 = pow(10,($ELO[ $player1 ] / 400 )); $Q2 = pow(10,($ELO[ $player2 ] / 400 )); // echo $Q1 . $Q2; return ( $Q1/($Q1+$Q2) ); } /* $predictions = array(0.681321835166, 0.56440049024, 0.730245413297, 0.718756829899, 0.579899760358, 0.534483944727, 0.58968350314, 0.582701884964, 0.555890961117, 0.683816314194, 0.515824985265, 0.574280203655, 0.54592192278, 0.502878199575, 0.54877433586, 0.558731369325, 0.531618297828, 0.448376452689, 0.417298115036, 0.502878199575, 0.30753660742, 0.363923100312, 0.241305379853, 0.349395372066); $mean = 0; $variance = 0; foreach ( $predictions as $game ) { $mean += $game; $variance += $game * ( 1 - $game ); } echo "\n" . $mean . "\n" . sqrt( $variance ); */ ?>
The ELOs have probably changed a bit since the matches were played.
|
You are an Oracle of sorts with that kind of foresight. Tried the commodities market?
|
Awesome write-up!
About Fantasy - Stork, I think koreans are more aware of the races and the slight favoring of different races over others. In this case meaning that Stork is protoss playing terran in this game afterall. Any terran can loose games to worse toss users on a given day, and we all know this. Even at pro level.
I also think TL idolizes players more and many times our judgement seems quite clouded. Some players are hated upon; gogo, BackHo, Hiya etc and this bias makes us bet against these players due to emotional reasons.
Also type-b vs stork is an example of knowing the particular matchup in depth. It's ZvP, and while Stork is awesome in long, out-drawn PvZ:s, zerg always has a slight upper hand in this matchup. Same thing as in PvT. Also I think TL bets on Stork here because he is "famous" and type-b isn't.
Let's not forget there are ALOT of noobs on TL too, like really new players. On yGosu maybe everyone voting isn't super gosu but many of the voters are likely to be pretty good players themselves and are probably very aware of how the different matchups work. Also they have starcraft on TV with in depth commentating.
Also I honestly think that alot of foreign people here who never have used the terran race, aren't aware of just how much that goes into playing a good TvP at pro level. Or a good ZvT. People take big names for granted, and while they suck themselves (obviously they are using protoss), they fail to see how strong protoss is in PvT for example. Same with ZvP newbies who suck and even think protoss is stronger PvZ, which is ridiculous. Anyways, these players will easily vote Terran over protoss, and protoss over zerg, based on their own understanding and not understanding that their bad skills is making it seem like the matchup is balanced against their favor, when in fact it's the other way around (PvT, ZvP, TvZ).
|
On August 02 2009 17:04 MannerMan wrote: The ELOs have probably changed a bit since the matches were played. They even changed while I was coding.
I just thought about it, and using probabilities to win only works if the system, aka liquibet, gives you odds. Like 1 out of 40 people are picking someone to win vs Jaedong, you should be earning 40x points. Statistically there's no point in picking underdogs if you don't get a bigger payoff, there's no line to bet off of like in sports, and using small correlation statistics ("he plays well on mondays") is pointless vs larger ones (ELO). I'm guessing people are far worse off than if they picked by Kespa rank or ELO, and success is statistical outliers.
Regardless, for anyone interested here's some php code so you can see the odds of a series. For example the OSL quarterfinals day 2 predictions (admittedly after it happened so the ELO's are probably affected)
Screenshot: + Show Spoiler + The code for elo_predict.php. You have to host it unfortunately. + Show Spoiler + <?php $url = "http://www.teamliquid.net/tlpd/players/detailed-elo"; $ch = curl_init(); curl_setopt( $ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt( $ch, CURLOPT_URL, $url); curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, TRUE); $tlpd = curl_exec( $ch ); curl_close( $ch ); //Captures table id="tblt_table, no subtables allowed preg_match( '/<table id="tblt_table"(.*?)<\/table>/si', $tlpd, $data); preg_match( '/<tr>\s+<td.*/si', $data[1], $table ); //Searches for individual players by "/tlpd/players...<" preg_match_all( '/\/tlpd\/players[^<]*/si', $table[0], $players );
//Creates array ELO indexed by the player names on the first page of TLPD's detailed_elo page. Entries are indexed ELO, vsT, vsZ, and vsP. //Uses html format to find data, so will not work if ELO is not " number ", and vsT, vsZ, and vsP are not ">number</span" foreach ($players[0] as $player) { $name = preg_replace( '/[^>]*?>/is', '', $player);
preg_match( '/>' . preg_quote( $name ) . '<.*?<\/tr>/is', $table[0], $details); preg_match( '/\s\s(\d+)\s\s/', $details[0], $temp ); $ELO[ $name ][ "ELO" ] = $temp[1]; unset( $temp ); preg_match_all( '/>(\d+)<\/span/', $details[0], $temp ); $ELO[ $name ][ "vsT"] = $temp[1][0]; $ELO[ $name ][ "vsZ"] = $temp[1][1]; $ELO[ $name ][ "vsP"] = $temp[1][2]; unset( $temp ); } unset( $player ); ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> <title>ELO_predict.php</title> </head>
<body> <form name="elo_predict.php" method="post" action="elo_predict.php"> Enter number of matches <input type= "int" name="matches" <? if ($_POST['matches']) { echo " value = " . $_POST['matches'] ; } echo '> <br/> <br/>'; for ( $i = 1; $i <= $_POST['matches']; $i++) { echo "Game $i"; echo '<br/> Enter player 1 <select name = "player1[]">';
foreach ( array_keys( $ELO ) as $name ) { echo '<option value = "' . $name . '" '; if ( $_POST['player1'][ $i-1 ] == $name ) { echo 'selected = "yes" '; } echo '>' . $name . '</option>\n'; } unset( $player); echo'</select> Enter player 2 <select name = "player2[]">'; foreach ( array_keys( $ELO ) as $name ) { echo '<option value = "' . $name . '" '; if ( $_POST['player2'][ $i-1 ] == $name ) { echo 'selected = "yes" '; } echo '>' . $name . '</option>\n';
} unset( $player);
echo '</select> <br />'; } ?>
<input type="submit" value="Process it!" /> <span style="color:#0000FF"><?php echo $alert ?></span> </form> <hr /> </body> </html>
<? //Print out odds if user has submitted player names if ( $_POST['matches'] && $_POST[ 'player1' ] && $_POST[ 'player2' ] ) { for ( $j = 0; $j < $_POST[ 'matches' ]; $j++ ) { echo $_POST[ 'player1'][ $j ] . " vs ". $_POST[ 'player2' ][ $j ] . ": " . odds( $_POST['player1'][ $j ], $_POST['player2'][ $j ] ); echo '<br />'; }
}
//Calculates odds of player 1 winning vs player 2 function odds( $player1, $player2 ) { global $ELO; $Q1 = pow(10,($ELO[ $player1 ][ "ELO" ] / 400 )); $Q2 = pow(10,($ELO[ $player2 ][ "ELO" ] / 400 )); return ( $Q1/($Q1+$Q2) ); }
?>
|
this is somewhat inaccurate because u didn't take into account the actual number of votes. Averaging the percent corrects doesn't give u the actual average. lets say 100 out of 100 people got one game correctly (100%) and 0 out of 1 person got another game correctly (0%). are you gonna average those two and say that they're 50%? no, of course not, it's 100 out of 101, which is 99% accuracy
|
|
|
|