|
United States12249 Posts
It's long been established that the SC2 league populations center around the concept of "active players". If you're wondering why that target distribution of 2/18/20/32/20/8 is never what's reported on Nios or SC2Ranks, well, that's why. Those sites track all accounts and not solely active players. One of the main reasons for this is that we don't know what constitutes an "active player".
So, then, what is an "active player"?
Blizzard uses the bonus pool as their primary activity metric. The bonus pool accumulates at a fixed rate, so if you let it pile up, then you haven't been playing games and therefore aren't active. A couple of years ago I emailed the designer asking why our different forms of activity filters never matched up with Blizzard's distribution, and I received the response "if you were to look in terms of 'bonus weeks behind' you may find closer results." Now, I'm not sure about whether there is a secondary measurement being considered (perhaps games played), I'm just evaluating all possibilities.
+ Show Spoiler +
Side note: If Blizzard looks at league distributions on a continual basis--that is, in the middle of a season--then surely there is some additional filter in place like number of games played. It couldn't be one game or five games because just placing in a league shouldn't qualify you as an active player, even though early in a season your unspent bonus pool will be quite low.
Shadowed from SC2Ranks provided me with a data snapshot from this afternoon (Nios, your site is great but it doesn't have as many data points as SC2R, sorry!). Now that Shadowed has incorporated bonus pool tracking, I can finally look more closely at how Blizzard might be filtering for active users.
I created a spreadsheet based on that snapshot using the AM region, 1v1 bracket (150K 1v1 users, largest of all regions).
Link to Spreadsheet on Google Drive
Note: Even though I use column headers based on multiples of 180, the sub-Master leagues have already been adjusted for the 0.58 bonus pool accumulation rate. It was just easier to express "bonus weeks behind" in terms of the more round Master value (180 per week).
Each cell represents a league population percentage compared with the other leagues following the same criteria. That is, the B4 cell (Master, >=1 game, <=180 bonus pool) means that based on the snapshot from July 31, 2013, 6% of all users with >=1 game played and who have <=180 bonus pool are in Master league. This value is notably inflated over the target 2% value even though Nios and SC2Ranks find that 3% of the entire account population is in Master. You can see in the spreadsheet that a lot of the other leagues are closer to their targets, there are a lot of 18%s in the <=180 Diamond region, a lot of 20%s all over Platinum, a lot of 32%s all over Gold. What isn't apparent is one fixed coordinate that fits all the leagues. This leads me to believe that the league adjustments may in fact be seasonal.
Season Locks
The bonus pool stops accumulating during a season lock, which lasts for one week. Blizzard's stated reason for this is to allow players to spend their remaining bonus pool and establish final divisional rankings. However, the main reason for season locks may actually be far more practical. If Blizzard takes a snapshot of the remaining leagues at the time of the league lock, then that gives them a full week to analyze the current league populations. This would allow them plenty of time to run solves on whether they need to adjust the league thresholds for next season and what the new values should be.
Here's a look at a chart from the S4 Season Lock on August 20, 2013 which compares the Bonus Pool Population with the Actual Population and the Target Distribution:
![[image loading]](http://i.imgur.com/O12IoEi.png)
What you can see in this chart is that the Actual Population of all players is way off, but when you look in terms of remaining bonus pool, the numbers suddenly look a lot closer to the Target Distribution.
Here's how the data points break down:
1 bonus week: Master - 5.4%, Diamond - 15.27%, Platinum - 18.95%, Gold - 28.84%, Silver - 20.26%, Bronze - 11.26%
2 bonus weeks: Master - 5.56%, Diamond - 15.34%, Platinum - 19.37%, Gold - 29.58%, Silver - 20.07%, Bronze - 10.05%
3 bonus weeks: Master - 5.62%, Diamond - 15.06%, Platinum - 19.53%, Gold - 30.31%, Silver - 20.18%, Bronze - 9.26%
4 bonus weeks: Master - 5.64%, Diamond - 14.57%, Platinum - 19.46%, Gold - 31.27%, Silver - 20.46%, Bronze - 8.57%
5 bonus weeks: Master - 5.53%, Diamond - 13.87%, Platinum - 19.21%, Gold - 31.94%, Silver - 21.17%, Bronze - 8.25%
6 bonus weeks: Master - 5.09%, Diamond - 12.60%, Platinum - 18.35%, Gold - 31.96%, Silver - 22.93%, Bronze - 9.03%
What this tells us is that at the time of the S4 Season Lock, of the players who had less than 1080*0.58=626 bonus pool, 12.6% were in Diamond, far short of the 18% target. However, of the players who had less than 180*0.58=104 bonus pool, 15.27% were in Diamond. Master is always pretty steady at around 5%, above the 2% target. This means that out of all the active players, Master is inflated by the end of the season. This could mean that Blizzard might want to raise the requirements to get into Master league in order to make it more exclusive in the following season, which would also have an effect in making Diamond a little bigger.
What we don't know is which line reflects the actual activity metric, and that's why I have 6 different "bonus weeks behind" lines. In most of them, Diamond, Gold and Platinum are slightly underrepresented while Master, Silver and Bronze are slightly overrepresented.
|
Cool, thanks Excalibur_Z. Love these articles.
Blizzard behind the scene stuff is so awesome 
|
United States12249 Posts
Updated with a chart of the populations according to different bonus pool thresholds from the 2013 Season 4 league lock.
|
Wait what? Only 1 guy commented on this for an entire month? :o
|
On August 30 2013 04:32 Koshi wrote:
Wait what? Only 1 guy commented on this for an entire month? :o
:O
such an amazing research getting lost... 
|
On August 30 2013 04:32 Koshi wrote:
Wait what? Only 1 guy commented on this for an entire month? :o
Yeah, the bonus pool on this thread must be really high right now. Well perhaps since there is no useful information but a neat fact though.
|
On August 30 2013 04:32 Koshi wrote:
Wait what? Only 1 guy commented on this for an entire month? :o
Seriously, I missed this the first time around, but it is really interesting. Glad this got bumped!
Basically it appears that there are a lot of bronze/silver players who only play their placement games (plus maybe a few more) each season and then nothing else. It's amazing how all of 1 -> 6 bonus weeks are so similar to each other.
Thanks for the insight Excalibur_Z!
|
How did I miss this thread? Really interesting read
|
|
|
I ll give it 1 more bump. Cuz stats are for cool kids.
|
Very interesting, it's unfortunate that this thread got buried.
|
Missed the thread the two other times it got bumped, so thx Koshi.
What I read Blizzard sets the threshold manually? Can this also mean thats the reason why the WOL ladder is a bit f*ed up since a few seasons, since noone (except players like me) cares about WOL anymore? Maybe its also bc they removed GM-league without updating some of their other code.
[WOL league distribution NA: Masters 0.05% (5 Users) Diamond 1.20% (131 Users) Platinum 5.43% (592 Users) Gold 26.26% (2,864 Users) Silver 51.57% (5,625 Users) Bronze 15.49%(1,690 Users)]
|
On August 01 2013 09:37 Excalibur_Z wrote:
What this tells us is that at the time of the S4 Season Lock, of the players who had less than 1080*0.58=626 bonus pool, 12.6% were in Diamond, far short of the 18% target. However, of the players who had less than 180*0.58=104 bonus pool, 15.27% were in Diamond. Master is always pretty steady at around 5%, above the 2% target. This means that out of all the active players, Master is inflated by the end of the season. This could mean that Blizzard might want to raise the requirements to get into Master league in order to make it more exclusive in the following season, which would also have an effect in making Diamond a little bigger.
I have a feeling if you take a look right now, you'll see that the bolded part is true. I know of a number of people that were Master last season in the middle of their league that are now Diamond. After playing their placement match, it put them into Diamond instead of back into Masters, which it used to typically do.
It's interesting to think that Blizzard might actually be altering the criteria it takes to get into certain leagues based on the number of "active" players in each league from the previous season. This would of course make a lot of sense if you think about it. If they have set criteria to base whether or not a player should be in a certain league at the start of a season, it would make sense for the percentage of players in each league to be very close to what they are looking for. Then as the season goes on other players will meet that criteria and be promoted inflating the league once more and forcing Blizzard to set their criteria differently. Of course you should take into account players that do get demoted during the season, which might offset the inflation.
It would be interesting if Blizzard were to set an absolute set percentage for each league forcing people out of the league either by promotion of demotion.
That is of course, what I think is going on.
|
Personally I feel division should carry over to the start of next season, but have a simple promotion/demotion policy. Way too many people are being demoted 1-2 leagues at the start of every season after performing reasonably well in their former league, which sends a bad message to the players.
There is no reason demotions should be disable during the course of a season, particularly for the higher leagues. Win-rate, activity, and maybe even some 3rd criteria like a mixture of spending quotient/supply cap time/apm/etc could be used to assess whether a player should remain in their current league, receive a promotion, or a demotion.
Currently the ladder is a complete joke and does not in any way even resemble the term "ladder". It's simply Blizzard throwing you wherever they feel like at the start of the season with absolutely no explanation or clear reasoning. I can understand that the dynamics of the population shifts and league parameters need to be adjusted but that should be done during the course of a season based on player performance.
Is Diamond League too large? Fine. Demote the players occupying a seat in diamond that are either
a.) Not playing enough
b.) Not performing well
It is just such a terrible experience right now and impossible to maintain any sense of direction or to assess progression.
Edit: Or simply find a way to put this into the hands of the players or let us have a little control over our own fates.
Example - if a particular league has become too large and needs to be whittled down, maybe impose a requirement that all players outside of the Top 50 of their division will be demoted, or cut the players who simply aren't putting any time in, or maybe even have an informal tournament setup.
The way it is handled now just doesn't work and I can't think of anything that makes me less motivated to play than a random, undeserved demotion after a meaningless placement match. That system needs to be removed as well. I can understand the purpose of playing 5 placement matches, but that single match placement format proves absolutely nothing
|
On September 03 2013 07:23 NubainMuscle wrote:+ Show Spoiler +
Personally I feel division should carry over to the start of next season, but have a simple promotion/demotion policy. Way too many people are being demoted 1-2 leagues at the start of every season after performing reasonably well in their former league, which sends a bad message to the players.
There is no reason demotions should be disable during the course of a season, particularly for the higher leagues. Win-rate, activity, and maybe even some 3rd criteria like a mixture of spending quotient/supply cap time/apm/etc could be used to assess whether a player should remain in their current league, receive a promotion, or a demotion.
Currently the ladder is a complete joke and does not in any way even resemble the term "ladder". It's simply Blizzard throwing you wherever they feel like at the start of the season with absolutely no explanation or clear reasoning. I can understand that the dynamics of the population shifts and league parameters need to be adjusted but that should be done during the course of a season based on player performance.
Is Diamond League too large? Fine. Demote the players occupying a seat in diamond that are either a.) Not playing enough b.) Not performing well It is just such a terrible experience right now and impossible to maintain any sense of direction or to assess progression. + Show Spoiler +Edit: Or simply find a way to put this into the hands of the players or let us have a little control over our own fates.
Example - if a particular league has become too large and needs to be whittled down, maybe impose a requirement that all players outside of the Top 50 of their division will be demoted, or cut the players who simply aren't putting any time in, or maybe even have an informal tournament setup.
The way it is handled now just doesn't work and I can't think of anything that makes me less motivated to play than a random, undeserved demotion after a meaningless placement match. That system needs to be removed as well. I can understand the purpose of playing 5 placement matches, but that single match placement format proves absolutely nothing
This is what Blizzard does periodically. They do it at the end/start of ladder seasons. That's why you see the people being placed into different leagues instead of carrying over.
I think for a majority of players it's motivating to try to get back into *X* league each season (that's what we really want right? Encouraging people to play). If you were placed into *X* league but were constantly scared that you would be demoted all season you probably would not play until the season lock anyway (and then if your MMR dropped you'd still get placed into a lower league the following season).
I wouldn't mind if seasons were a bit shorter (we'll see how it goes with mirroring WCS seasons) or if there was a "mid season MMR/League/inactivity check" to promote/demote/kick players as needed.
|
Probably explains why I got bumped into plat after a crappy on and off season playing random :D
I'm guessing Diamond got too large because of how many people were demoted from masters the season before, and therefor people not playing well got kicked down.
Or am I misunderstanding?
|
Excalibur, I feasted upon your excellent OP.
What I would really love to see would be what the above graph looks like for WOL 1v1. I know much fewer people play WOL 1v1, but I feel quite confident that even the active player distribution for WOL would still look pretty screwed up (far off from blizzard's stated population targets). If the above graph IS from WOL then you will have successfully shocked me. I checked the distributions on SC2 ranks fairly regularly last season for WOL and HOTS, and they don't seem to have changed much, which strengthens my opinion that your graph is of HOTS 1v1 and that WOL 1v1 would still look pretty bad even after adjustment for active players.
I wonder if blizzard is adjusting MMR league boundaries JOINTLY for WOL and HOTS? It would make sense that fewer skilled players are left playing WOL... Still though, a grand total of 20 1v1 Masters in Americas region right now seems awfully low even if the boundaries vary jointly for WOL and HOTS. If blizzard IS doing this, then of course they would base the distribution off of HOTS and have that one look mostly correct (as it does in your graph) and leave WOL distribution naked to suffer the wrath of an eroded playerbase.
P.S. Bonus pool max at time of snapshot = 1337. That can't be a coincidence, right 
P.P.S. If you do make a WOL 1v1 graph, you might want to exclude China. It could be something else going on, but it looks like China is pretty different (unrepresentative) of the other regions so including them would skew the data massively.
|
On August 30 2013 04:52 Big J wrote:Show nested quote +On August 30 2013 04:32 Koshi wrote:
Wait what? Only 1 guy commented on this for an entire month? :o such an amazing research getting lost...
such amazing..
so research...
much statistic..
wow!
|
|
|
Why have seasons at all? Why have leagues? It's very silly. Just display an ELO rating.
|
|
|
|
|
|
EPT
