EPTPro Players Should Not Judge Maps
Maps generally get to the ladder and tournaments in the following way. Every so often a TLMC is announced, and mapmakers submit to it. A panel of judges consisting of pro players and community members narrow the submissions down to 16 finalists which play in a map contest tournament and are voted on publicly. Then Blizzard picks from among the maps from the contest to put them on ladder.
This process is terrible. Most mapmakers hate it. There are so many problems with the contest to the point where it's little more than a lottery that leans towards standard and pretty maps. There's no feedback from the judges, scores for maps aren't revealed to the mapmaker which keeps them guessing as to what they should submit next time. The entire process is non-transparent. Mapmakers have no idea if their maps even reached the judges or got cut earlier, no idea why a map succeeded or not. Judges give a single number from 1 to 5 per map, they average them and that's that. And because no one has any visibility, and judges are unwilling to bother with spending the time to give feedback into what's happening people don't have the info to complain.
There's endless problems to discuss with the TLMC which have mostly been hidden away from the community as a whole (TLMC 6, 8 and 11 in particular had some awful problems that are beyond the scope of this. TLMC 15 has had some scheduling and QA issues but nothing out of the ordinary awfulness). The one thing that I'll focus on here is the judging, particularly the judging from pro players. One of the judges, a professional Terran player, went over some of his judging on stream recently, and it was so bad that I decided to make this post. I'm not going to mention him by name (though I'm sure some of you know or can figure it out), because this post is not about him. The problems with TLMC judging have been going on for years, and this is just symptomatic of that. In fact I'm grateful to him for showing some transparency which is the number one thing that's entirely lacking in TLMCs.
The first reason why pro players judging maps is a problem is that:
They don't give a shit about judging maps.
Here's a direct quote from our judge:
I didn't take, like a, deep look into that though, always keep that in mind. It's not like I took like a deep look at the maps I just look at the picture and then I just looked at the base layout basically. So my points that I gave may not be that great.
Judging is a thankless task. It's an unrewarding chore to look at ~120 submissions and score them. However if you're going to half-ass things like that then you should just not judge things. No one's forcing you, and it would be better for everyone if you didn't judge.
In terms of effort put in, no one's expecting a judge to open up every map in the editor, or play on every single one of them. But if you're always only just glancing at overview pictures, you're going to get terrible results. In particular proportions and sizes are very hard to tell from just an overview pictures and that's one of the reasons why rush maps are almost always terrible (more on that later).
But this particular judge went beyond just not putting in a lot of effort. He also showed a complete disregard to the parameters of the judging. He likes standard maps (as most pros do), so he just scored the most standard maps in the freestyle category highly which defeats the purpose of the category (as seen by the fact that only one of the freestyle maps in TLMC 15 is in any way creative). He thinks 4p players map don't work, so he scored multi-spawn maps low (and given that he doesn't seem to know decimal points exist that means most 4p maps got scored basically the same). Now, I'm not the biggest fan of 4p maps personally, but regardless of the merits of multi-spawn maps, that's what ESL asked for, so as a judge he should have done his best to select from them the best of them. But no he didn't, so it's no surprise that we ended up with a bunch of not so great 4p rotational maps as finalists.
In general his reasoning was very arbitrary. He gave the following map (Misophist) a low score because "he likes maps that don't have all their bases on the edges", which is not a gameplay justification for anything.
![[image loading]](https://i.imgur.com/AVKf5Bd.jpeg)
Now what he may have been driving at is that bases on the edges result in this rush map being quite open which is bad for terran which leads us to the second reason why pros should not be judging maps.
They're biased and self-interested.
There's a very obvious conflict of interest in pros picking maps that affect their chances in future tournaments. And while I'm sure some of the judges attempt to be somewhat objective, many, including the one I'm using as an example aren't.
Giving credit for honesty, he wasn't hiding the fact that he was picking maps that are good for terran rather than maps that are good for the playing or viewing experience. He was just picking out features that are good from his point of view. This judge likes it when there's airspace behind the linear third so he scored those maps highly. There's nothing inherently wrong with airspace behind a linear third--sometimes a map needs that to help terran. However if a map is already good for terran, then a more objective judge wouldn't want that airspace. But that judge would score the map higher regardless. If a map that's imbalanced for terran makes ladder that's bad for the game, but it's fine for him.
Now you might say that having a bunch of self-interested pros pick maps that favor their race balances out in the end, but it doesn't always. Guess why a couple years ago the map pool was bad for terran, all the maps had overlord pillars and no reaper jump up spots? Yup, back then there were fewer terran judges so you got a zerg map pool.
But even if the number of judges of each race does balance out, you don't get the best maps making it to ladder. You get very standard maps that resemble those on ladder currently, or pretty maps, since that's the lowest common denominator among pros. Which leads me to the next point:
They (and the system) are awful at picking anything non-standard.
The format of the TLMC is set up to produce standard maps. Maps that deviate from the standard are usually cut even before they reach the judges (and no mapmakers aren't told if their map got cut) and this whole averaging of points from biased judges usually results in some safe boring maps that don't offend any one judge too much. And the TLMC is mostly successful at choosing not too bad standard maps (though admittedly the quality of standard maps is generally high (see https://imgur.com/a/HANtd46), so you can't go too wrong with your picks).
On the other hand the TLMC sucks at producing good non-standard maps. In particular the rush category (which isn't even that non-standard as it goes) has been an unmitigated disaster over the years.
The rush category has had maybe two successes (Dreamcatcher and Blackburn) and a long series of terrible maps. Submarine, Battle on the Boardwalk, Defender's Landing, Paladino Terminal... Kairos Junction was a standard map (since it had too many bases for rush), but was a better rush map than the lot of them.
To see how bad exactly these rush maps are we can get the winrates for all 90 LotV ladder maps from Liquipedia, calculate an imbalance score based on the square root of the sum of the squares for each match-up, and see how they stack up to maps from other categories. The higher the imbalance score, the worse the map. These are the results
(data here: https://docs.google.com/spreadsheets/d/1qQ4uyCJzUwsojDFLJc9jzEISkVIi3Ko3Emk17o-fHvc ):
![[image loading]](https://imgur.com/d5n8Rq2.jpg)
So on average a standard map is in the top third of LotV maps, while the average rush map is less balanced than 2/3rd of the maps in LotV. This is worse than the average map produced by Blizzard (most of which were made in early LotV where things were pretty crazy).
The reasons for why are rather complex (and this post is way too long already), and involves rush maps being harder to design for (though it's been done in the past and could be done again), and the TLMC categories being poorly defined, but mostly it's because the judges blindly cling to the current standards (no matter how bad they are because it's what they know). I'm not sure they even know there's a problem, and also many of the judges might like that status quo. Submarine and Oxide being broken maps for terran might be bad for the community and bad for the contest, but a terran judge might only see it as a good thing.
The rush map shown above? It could be broken. It could be a mistake. But at least it would be a new mistake that teaches us something. But terran judges are incentivized to have us keep making the same old mistakes with rush maps, so a rush map with an open middle like this is not on the menu, while Submarine 2.0 might be. For another example, a map like Promanus Grounds learns from successful maps like Dreamcatcher, but doesn't have too much of a chance in a TLMC where it's considered too rush for standard, too standard for rush and where judges cling to the status quo.
Likewise this results in freestyle/new/challenge maps rarely being good unless they're extremely standard (e.g. Deathaura was fine, but the speedzones barely mattered), and makes stuff like trying to get good multi-spawn maps out of a contest hopeless.
Note that the only reason we got Golden Wall as a finalist and thus on ladder was that the judging composition of TLMC13 was a bit different--there was a mapmaker in the judging panel, and he chose Golden Wall for his judge pick. We have him alone to thank for it. It wouldn't have made it based on the point average of the player judges.
But anyhow to end things here's one of the most egregious mistakes made in the TLMC 15 judging:
They don't give a shit about judging maps.
Here's one of the few map that got a 1 out of 5 from our judge:
![[image loading]](https://i.imgur.com/dV8O4wx.png)
It got this score because "the horizontal spawns are imbalanced". The problem? Horizontal spawns are obviously not enabled on the map. And the judge, if he had cared, would have known that. Apparently there was some sort of a mess-up on the admin side and not all the descriptions were given to the judges (this is another problem I haven't mentioned yet. Mapmakers are told to write long descriptions that rarely make it to the judges). But leaving the frequent administrative gaffes of the TLMC aside, the judge should have known the spawn restrictions even without that information if he had put in a modicum of effort. It's obvious from the layout. And even if it wasn't obvious to him, he could have easily checked. The map files are provided. But he didn't open that map or any other map. Because he didn't give a shit.
Conclusion/TL;DR
TL;DR The TLMC is how we get ladder maps, and it has a lot of problems with transparency and judging. One of the problems with judging is that a lot of pro judges half-ass their judging, and just pick maps that are good for their race. It's bad to the point where they miss-score maps because they can't be bothered to figure out the spawn restrictions of a map. This results in standard maps being boring, and non-standard maps being poorly balanced ultimately resulting in bad maps on ladder.
This type of awful judging from the pros, and all the other problems with the TLMC have been going on for years behind the scenes. Mapmakers have certainly been aware of how bad things are, and nothing that I've written should be new to them, but the community as a whole not so much. People don't truly understand (or particularly care about) how maps are sourced, so when maps are bad they blame various things, but not the source of problems which is how the TLMC is organized or judged. The TLMC doesn't serve the community or tournament organizers as well as it should, and I see no reason to expect it would change in the future. The system should be completely restructured and the judges changed, but it won't be. Mapmakers have been unhappy about the state of things and asking for changes for years, but nothing has changed. This post is just to get mapmaking off my chest and out of my life. And if you're one of the pro judges who did their due diligence, and truly put in the time and effort, I apologize. But there clearly haven't been enough of you, and given the complete lack of visibility into the TLMC I can't assume there are any of you at all.
