EPTJust a possible avenue for exploration.
Match Making Rating Tool - Page 167
| Forum Index > SC2 General |
|
qdlbp
6 Posts
Just a possible avenue for exploration. | ||
|
korona
1098 Posts
On September 06 2013 02:35 Excalibur_Z wrote: Those are some interesting examples. How did you come by 42 days/6 weeks as the inactivity period, and are there similar results for 35 days, 28 days, 21 days, 14 days? I kind of have my doubts about them demoting players above X bonus pool forcibly, but they've done some dramatic changes with HotS so this may not be too far-fetched after all. Note I am going downwards with the data (but no time before next week to continue). Starting from accounts that had longest inactivity period. At the moment I chose all accounts that had more than 3 weeks of inactivity (probably will go even lower, but checking match lines manually for data errors takes time). There are some 400 to 500 such cases (inactivity longer than 3 weeks) that I have not yet checked other than graph inspection. There was at least 30 such cases in that data (for example I flagged lots with 25 days of inactivity), but last week when I went through it, I had no tools to speed up the verification (to immediately rule out e.g. cases that were actually missing matches and not inactive. Last week there was more than 900 accounts and almost half were ruled out with new tools), And no I don't think the bonus pool is the factor, but will record it just in case (it would be absurd, but the inactivity length on the other hand more likely. Then again having such intended feature is illogical (at least with so major drops) --> Would think it is more likely a bug than a feature). | ||
Excalibur_Z
United States12238 Posts
"The MMR system is designed to help keep you at a consistent level of competitive play, even if you step away from the game for a while, and this is likely what Korona is seeing. There are no plans to reveal activity metrics at this time." So... whatever that means =) Unfortunately it couldn't really be more vague than that. | ||
|
frankx999
United States5 Posts
I receive this message when attempting to add my profile to plugin MMR-Stats I copied it from the BNET page in the address bar | ||
|
Danglars
United States12133 Posts
On September 07 2013 09:24 Excalibur_Z wrote: This was the response I got: "The MMR system is designed to help keep you at a consistent level of competitive play, even if you step away from the game for a while, and this is likely what Korona is seeing. There are no plans to reveal activity metrics at this time." So... whatever that means =) Unfortunately it couldn't really be more vague than that. The line even if you step away from the game for a while suggests an intentional decay mechanism and not one resulting from bugs. Korona is seeing the intentional system in action. Whether it takes as arguments the length of absence (rather than just a start point) is still up in the air. | ||
|
korona
1098 Posts
On September 10 2013 07:39 frankx999 wrote: Error: Bnet Profile links not Valid I receive this message when attempting to add my profile to plugin MMR-Stats I copied it from the BNET page in the address bar Remember that the URL should point to your profile's main page & it should end with a slash ('/'). Also make sure there are no spaces in the beginning of the URL as whitespace characters seem not to be trimmed off automatically from the beginning of the address. The tool should be able to convert different language versions of the URL to the server's main language (for example Taiwanese profiles in Chinese are automatically converted to Korean profile addresses), but I have not checked that part of the code (potentially some language versions may not be supported if they have been added later). Thus I recommend to always use the server's main language when adding new URL:s. Here are examples for each server for imaginary account called 'ABC' whose bnet id is 123456 and 'sub-server code' is 1 (note that language code is between 'sc2' and 'profile'): EU, US, SEA: http://eu.battle.net/sc2/en/profile/123456/1/ABC/ http://us.battle.net/sc2/en/profile/123456/1/ABC/ http://sea.battle.net/sc2/en/profile/123456/1/ABC/ Korea & Taiwan: http://kr.battle.net/sc2/ko/profile/123456/1/ABC/ (Taiwanese version http://tw.battle.net/sc2/zh/profile/123456/1/ABC/ is automatically converted to the Korean version) China: http://www.battlenet.com.cn/sc2/zh/profile/123456/1/ABC/ | ||
|
Excalibur_Z
United States12238 Posts
On September 10 2013 09:58 Danglars wrote: The line even if you step away from the game for a while suggests an intentional decay mechanism and not one resulting from bugs. Korona is seeing the intentional system in action. Whether it takes as arguments the length of absence (rather than just a start point) is still up in the air. True, but it also says "keep you at a consistent level of competitive play" which suggests that you wouldn't decay. It's a rather confusing statement to say the least. | ||
|
frankx999
United States5 Posts
On September 10 2013 10:04 korona wrote: Remember that the URL should point to your profile's main page & it should end with a slash ('/'). Also make sure there are no spaces in the beginning of the URL as whitespace characters seem not to be trimmed off automatically from the beginning of the address. The tool should be able to convert different language versions of the URL to the server's main language (for example Taiwanese profiles in Chinese are automatically converted to Korean profile addresses), but I have not checked that part of the code (potentially some language versions may not be supported if they have been added later). Thus I recommend to always use the server's main language when adding new URL:s. Here are examples for each server for imaginary account called 'ABC' whose bnet id is 123456 and 'sub-server code' is 1 (note that language code is between 'sc2' and 'profile'): EU, US, SEA: http://eu.battle.net/sc2/en/profile/123456/1/ABC/ http://us.battle.net/sc2/en/profile/123456/1/ABC/ http://sea.battle.net/sc2/en/profile/123456/1/ABC/ Korea & Taiwan: http://kr.battle.net/sc2/ko/profile/123456/1/ABC/ (Taiwanese version http://tw.battle.net/sc2/zh/profile/123456/1/ABC/ is automatically converted to the Korean version) China: http://www.battlenet.com.cn/sc2/zh/profile/123456/1/ABC/ Thanks, I forgot the end slash | ||
|
vaderseven
United States2556 Posts
On September 07 2013 09:24 Excalibur_Z wrote: "There are no plans to reveal activity metrics at this time." Reveals there is an activity metric. You can not reveal what you do not have. The word is only used when it means to show something that is there and is hidden. On September 07 2013 09:24 Excalibur_Z wrote: ", and this is likely what Korona is seeing." This means that what precedes the , in this sentence is something that is being seen by korona. On September 07 2013 09:24 Excalibur_Z wrote: "The MMR system is designed to help keep you at a consistent level of competitive play, even if you step away from the game for a while This means that when you are inactive [as determined by Blizzard] the MMR system is designed to keep you at a certain level of competitive play. This means that they feel that your level of competitive play changes when you are inactive [as determined by Blizzard]. On September 07 2013 09:24 Excalibur_Z wrote: This was the response I got: "The MMR system is designed to help keep you at a consistent level of competitive play, even if you step away from the game for a while, and this is likely what Korona is seeing. There are no plans to reveal activity metrics at this time." So... whatever that means =) Unfortunately it couldn't really be more vague than that. Becomes: When a player has become inactive, as determined by a hidden metric, the MMR system is designed to keep you at a competitive win/lose ratio. What you have asked us concerning some data that Korona has seen is a result of this hidden metric. At this time, we do not want to reveal the math behind how this metric works. Blizzard wants 50% win rates and has probably noticed that overall (so from bronze to gm with weight given to player count) when players are inactive for a time they come back to the game at a lower MMR than they were at. This metric is just trying to make sure the great number of players that do go inactive and then come back can quickly be at the 50% win rate that the MMR system tries to achieve. On an unrelated note, is it possible to someone color code the graph so that I can see time instead of just # of games? I would love to be able to see the games from yesterday with a gray background and the game from today with a white background and the games from two days ago with a white background and the games from three days ago with a grey background etc etc. It would create a visual way to see how your MMR fared on different days. ![[image loading]](http://2.bp.blogspot.com/-r2mxRWXXPEc/UHwEjuNNwHI/AAAAAAAAS1Y/2KptVgeQ19U/s1600/RetailSept2012.jpg) Ignore the axis labels in that and just notice the white and blue backgrounds. I would love to be abel to shade different days with background effects similar to that. | ||
|
korona
1098 Posts
If somebody who played several games just after the season changed (Monday 26th of August) and had MMR tool open (when Blizzard's web profiles were not updated for new season but before latest MMR tool was released few days after) (+ has recorded data before and after too) & is Windows user would be willing to send me following 3 files in a zip package: 'games_i.data', 'games.arch', 'client.data'. They can be found from following folder: [PATH_TO]\Sc2gears\User content\Plugin file cache\mmr.plugin.MMRPlugin\ Please send me a link to the zip package as private message (PM) and do not add that link here. That data will be kept private (if the feature succeeds and your data has nice looking graphs I may use it for few example screenshots without account identifiers) When I have enough such data I will update this message. PS: Regarding analysis of the MMR decay feature / bug that Blizzard likely has implemented: I did not have time for analysis last week, but spent lots of time improving developer side analysis tools to make analysis more efficient & lessen possible analysis errors. And some of these tools have potential to be published in some form in future versions of MMR tool (no promises or timeframe)... | ||
|
Danglars
United States12133 Posts
I agree with vaderseven on the analysis that intially got Excalibur confused. The competitive play is now at a lower level, so the system keeps you at a level of competitive play by taking into consideration the absence. This is much to the chagrin of players that stop playing halfway through the season and resume play next season to find them in a league lower. Frequent cries of, "Blizzard dropped everybody a league this new season!!11!" (Combined with people whose MMR dipped and were not reclassified because it is impossible to be demoted and whose un-adjusted points stayed buoyed up ty blizzard) I only played 2 ladder games on August 26th, it was all just practice games!  | ||
|
Excalibur_Z
United States12238 Posts
On September 17 2013 17:07 Danglars wrote: Today I learned that my growth in MMR over the last couple of months looks a lot like US retail sales between 1992 and 2013! Maybe there's a correlation ... I agree with vaderseven on the analysis that intially got Excalibur confused. The competitive play is now at a lower level, so the system keeps you at a level of competitive play by taking into consideration the absence. This is much to the chagrin of players that stop playing halfway through the season and resume play next season to find them in a league lower. Frequent cries of, "Blizzard dropped everybody a league this new season!!11!" (Combined with people whose MMR dipped and were not reclassified because it is impossible to be demoted and whose un-adjusted points stayed buoyed up ty blizzard) I only played 2 ladder games on August 26th, it was all just practice games! My point was that it's a non-confirmation of MMR decay. Also, how far do they know to drop you? Are there tiered thresholds or is it more granular? Is this based off empirical evidence or are they just guessing? Won't this cause ripple effects if their guesses are wrong because based on activity projections this will lead to deflation? I have some theories but there are always holes. If you are inactive and come back to play, let's say your MMR drops by 300 as an inactivity penalty. That's fine, because your first few games will be you just getting back into the swing of things and warming up, so if you really do belong at your old MMR you will get back there. However, what happens when all your opponents are also people returning from inactivity? Relatively you're the same strength so you won't go up any. Instead you will earn rating from beating weaker players which will push those weaker players down further. When I was doing my activity metric studies, I found that about 28% of Master players, 32% of Diamond players, and 24% of Platinum players had less than a week's worth of bonus pool, but this number falls way off for other leagues: only 18% of Gold, 11% of Silver, and 9% of Bronze. If we were to expand this to less than 4 weeks' worth, the number differences become even more stark: 56% of Master, 58% of Diamond, 48% of Platinum, 37% of Gold, 22% of Silver, and only 13% of Bronze players had less than 4 weeks' worth of bonus pool by the end of last season. If we take the complement of these active percentages then we get the inactive percentages, but the active ones are useful in determining the odds of playing against an active player. Let's say all the inactive players get dropped exactly one league, to make things easy. We don't know what the activity threshold is exactly but let's just use 4 weeks to be generous. If you were in Plat and now you're in Gold, and there were only 37% of Gold players last season who met the activity criteria, then that means you're only playing against real Gold players about a third of the time. Over time you would get promoted back to Plat if you really belong there, but if you're playing against demoted Plats a majority of the time that could make it take a lot longer than expected. | ||
|
Pirfiktshon
United States1072 Posts
| ||
|
Nelvalhil
6 Posts
 Tha.link Theres the link of a screenshot of my MMR stats | ||
|
Cheren
United States2911 Posts
On August 29 2013 05:28 Val_ wrote: Korona: im top ml/low gml player and im playing like that - 2 months active, 2 months inactive (have such work) constantly, etc, for about 3 years and first 5-10 games after inactive there are always vs really weak players, (not top ml, at least) - even top dia (rarely) and low ml+ But only 5-10 easy games, half of a playday and im playing vs my level again (and games are not easy anymore) (and, for example, after some of my inactives i had my first ladder game vs my clan mate, i know exactly that he is lower) Sorry I dont have any data in your database because ive started to use MMR plugin a month ago, but keep investigating, something like that is present sry for my english :p From what I remember, ladder uses a Glicko system like aligulac where you have an Elo rating but you also have an uncertainty rating. High uncertainty = Elo goes up and down much faster. Sounds like when you don't play for a while in HotS your Elo goes down a little but your uncertainty goes up a lot. | ||
|
tenklavir
Slovakia116 Posts
| ||
|
Nelvalhil
6 Posts
On September 19 2013 05:04 tenklavir wrote: I was in Diamond, took 7 or 8 weeks off, came back for the new season, played my placement match, and ended up in Gold. That happens when you take some weeks off D: | ||
|
tenklavir
Slovakia116 Posts
On September 19 2013 06:02 Nelvalhil wrote: That happens when you take some weeks off D: Hehe I don't think I belong there though, that's the thing. I'm 22-1 since then and still not back in diamond, so maybe Blizz decay mechanism takes it a little too far. | ||
|
Mintograde
United States25 Posts
| ||
|
korona
1098 Posts
On September 19 2013 04:20 Cheren wrote: From what I remember, ladder uses a Glicko system like aligulac where you have an Elo rating but you also have an uncertainty rating. High uncertainty = Elo goes up and down much faster. Sounds like when you don't play for a while in HotS your Elo goes down a little but your uncertainty goes up a lot. No. Exact details of the MMR system that Blizzard uses internally are unknown. Some data regarding the system can be found e.g. from this lecture given by its designer Josh Menke (who has left Blizzard in March 2013 according to LinkedIn): + Show Spoiler + The main formula that MMR tool uses to solve the MMR is ELO based. It gives close approximations of the MMR, but there are special cases where it does not work (the capping mechanic). You can read more from not_that's original thread (since then there has been small changes to the capping mechanics and of course the league tiers have been removed): http://www.teamliquid.net/forum/viewmessage.php?topic_id=332391 And regarding uncertainty - It seems not to affect the results after some 20 to 50 games have been played after blank MMR start. Also regarding these MMR drop cases, the MMR seems to behave normally after the drop, just like it does before the drop. This suggests the uncertainty value does not change, but it would be direct decrease of the MMR value. It requires more research to understand how the likely MMR decay exactly works (of course also errors in cap detection are possible, but unlikely would cause so consistent effects. Also lots of accounts suggest there is decay - e.g. when account is placed in certain league you know it has required MMR for that league at that moment. Many such accounts have dropped league or leagues after inactivity followed by new placement match even if there seems not have been offset changes between S14 & S15). In many user graphs the drop has seemed quite similar in size (250 to 300 MMR points, which is typical size of each lower league). But logically such system does not make sense as It would erode the whole MMR system like Excalibur_Z said few posts earlier while speculating alternatives: On September 18 2013 03:22 Excalibur_Z wrote: I have some theories but there are always holes. If you are inactive and come back to play, let's say your MMR drops by 300 as an inactivity penalty. That's fine, because your first few games will be you just getting back into the swing of things and warming up, so if you really do belong at your old MMR you will get back there. However, what happens when all your opponents are also people returning from inactivity? Relatively you're the same strength so you won't go up any. Instead you will earn rating from beating weaker players which will push those weaker players down further. And many are observing such behavior. On September 19 2013 06:05 tenklavir wrote: Hehe I don't think I belong there though, that's the thing. I'm 22-1 since then and still not back in diamond, so maybe Blizz decay mechanism takes it a little too far. At least accounts that were inactive for 3 weeks or more seem to be in 'danger zone', but not all are affected. Yes it requires lots of matches. For example to cross whole platinum league it requires ~16 straight wins against similar MMR opponents. Here is example from one of my accounts from S12 & S13: http://imgur.com/WklUU5I I played that account to master (it had enough games so that MMR change rate had normalized). Just before season switch I started spending bonus pool and had a loss streak that dropped my MMR to mid-diam. Then had 29 days inactivity and my MMR had dropped to high gold. It took 61 games (41W-20L) to reach similar MMR than I had before the break (leagues were even smaller in S12 & S13 than now). One reason why I did not mention this observation few seasons ago was that there was long losing streak just before the inactivity (the MMR before the losing streak was known for certain) and was speculating if something was changed regarding such trends combined with season switch. But the later examples from last season suggest that it would also happen in middle of the season and can have wins on both sides of the inactivity. On September 19 2013 08:08 Mintograde wrote: I'm wondering what would happen if you had two consecutive inactivity periods separated only by a couple games. If there's some limit in place to how far your MMR can fall in a single inactivity period, it might still be possible to "chain" inactivity together, resulting in a much larger MMR drop. If that's the case, someone could conceivably go from Diamond to Gold, then Gold to Bronze. That would be frightening (and drops have usually been ~one league range in MMR), but more research is needed. I am not sure if I have time to continue analyzing last season's data this week, but will likely spent some time with the potential new MMR tool features on weekend. But for that I need test data from those who played several games just after this season started on August 26th (have got such data only from one user, but it did not contain the data I was hoping for, but it revealed another bug with my implementation). If somebody is willing to help please read my earlier post http://www.teamliquid.net/forum/viewmessage.php?topic_id=334561¤tpage=167#3330 Edit: Of course some MMR tool user might be willing to do a test regarding the inactivity. Play on a certain account that has plenty of games after blank MMR start (more than 50). Play enough to get several 'good games' (even better if there has been promotion few games earlier). Then go inactive for more than month with that account (not even logging in that server in between). After inactivity playing enough games to get several 'good games'. And then repeating the inactivity cycle. There should be no missing games (MMR tool should be always on when playing with that account) and memory reader should be on. Edit 2: Started reading Josh Menke's twitter (https://twitter.com/joshua_menke). There are interesting tweets such as: "I'm not at Blizzard anymore, but I think there may be some cases where that's possible" when somebody asked "For my own peace of mind, could you please confirm/refute that MMR in Starcraft 2 can change without playing?". Another: "SC2's skill detection system ran full speed, but it's player ranking system ran at Elo speed. Separated so we could pace how we wanted." (there were other related tweet to the last one), etc Edit 3: changed wording when mentioning just placed accounts dropping in next placements "... have dropped league after ..." --> "have dropped league or leagues after". When you have been just promoted you are just above the league threshold and MMR can drop to lower league range potentially with just one game. I am not referring such cases. | ||
| ||
