EPTRegarding my own experience judging hobbyist-level homebrewing competitions (beer and mead) and martial arts and kinetic art exhibitions (for lack of a better term) and tournaments, there are so many things TLMC could be doing differently.
For context: When judging beer or any beverage (fermented/alcoholic and otherwise), there are several critical elements (ingredients, appearance, smell, taste, feel) that define the style which are judged and scored objectively, along with flaws (added sugars, atypical yeast strains, off color or cloudiness, bad smells or tastes, particulates, etc.). However, some elements that are considered flaws by purists of one style of beer can be weighted lower among a panel of judges who see those "objective flaws" as subjective-but-enjoyable elements that make that beverage unique among other products in the same style, and these are the beers which, in my experience, often win local brewing competitions. (E.g., a beer or mead that accidentally got infected by another type of bacteria, or had a bad brew or fermentation that the brewer tried to save by adding more sugar or "adjuncts", but that got more fruity, floral, and tart smells and flavors, is objectively infected but is safe to drink and more interesting than if it wasn't "flawed".)
Regarding homebrew event judges, for friendly brewing events and competitions I have been a part of and observed from afar, judges are usually expected to be transparent about which styles they do not enjoy, and are expected to not judge those beers critically and to instead defer to the other judges at their table when scoring that beer as a group. Judges who are shown to be very picky are ignored (or shut down) during the table's "diplomatic" conversations during final scoring, or they get assigned by the organizers/admins to score only the final round of beer (the final round to decide which ones in a specific category get bronze/silver/gold or whatever).
Sometimes submissions are only rated across five to six elements within their category (look, smell, taste, feel, overall, and sometimes an "X factor") which is given back to the brewer (the scoring sheet being a literal piece of paper, a form-fillable PDF, or like an online survey). Other times the table that rated the beer comes to a decision on it and the head of the table (or whoever the table points to, or whoever volunteers) writes a brief description (one or two sentences, or a jumble of adjectives) on the table's score card. There is always information being shared about why it got the score it did, even if it's just a handful of adjectives like "too murky" or "grandpa's toes" or "grandma's kitchen during summer". And other times it's a conversation, and the judge who is also a brewer (or who knows one and immediately finds them) will tell you exactly what caused the off flavor that caused the low score and exactly how to avoid it the next time you brew, or they'll ask you how you brewed it or what ingredients you used if the org doesn't collect that info and they'll give you some opinions or take notes from you for themselves.
Regarding fun and audience participation: + Show Spoiler +
In contrast to that professionalism, sometimes completely random people from the crowd of strangers, that day's volunteers, and/or submitters who showed up are chosen to help "filter" through the bulk of submissions, or get asked to join a panel to try out and judge the most unique/weird beers and, at that table, are given just as much weight in their opinions as the rest of the invited judges, professional brewers, and retiring cicerones/sommeliers. These things don't happen all the time, and I assume they don't happen at all depending on where you go, especially the more prestigious and global/busy the competition gets, but they have happened and don't seem extraordinarily rare or strange to me. It's so fun to have seen. Which is important to me and is something that I feel could be applied to mapmaking contests.
Regarding problematic events I've been to that have led to conversations like this very TL thread: + Show Spoiler +
In the past several years, I have also seen a few competitions ran in a similar way as the TLMC, usually for very, very small, local events, which in my experience leads to many (pretty much all) of the same negative feelings and discussions of the crowd and participants. Tensions don't usually get very high because next to nothing is at stake at those events, or if they do heighten it prompts the organizers to take action and ban a participant or judge from the next event, or restructure the event to be more like other, longer lasting events that proove via their history, obvious excitement, and success stories (i.e. medalists going on to start or expand businesses or schools) that there are better ways to run these things. At the very lowest level, these kinds of competitions evolve to ensure that problems don't persist and people want to keep coming back, or the problem persists the next season/year and the event gets cancelled the year after that because nobody besides a few old purists want to bother submitting or judging anything to their own shindig.
Back to TLMC. If the pools of judges were bigger, the categories were better defined, the judging was divided among groups, and the job(s) of the judges were more specific, the entire contest process would be much better, and we would get much higher quality if not at least much more interesting maps during the public testing and voting stage.
On July 14 2021 08:09 Teoita wrote:
For what it's worth, my main mindset when looking at a map was to always first ask: "how would one defend a natural base against some all-in? how would you defend a third against a soultrain or roach max? Is it too easy to cut the map in half and turtle?".
For what it's worth, my main mindset when looking at a map was to always first ask: "how would one defend a natural base against some all-in? how would you defend a third against a soultrain or roach max? Is it too easy to cut the map in half and turtle?".
I strongly feel that this mindset should be encouraged through the rules of the judging process, and ideally the rules and goals for judges would change for each category. (All of my other paragraphs aside, I'd be interested to see this used alongside public voting as a first stage; i.e. make a survey that TL users or Bnet client users with X hours of SC2 played can open, and they are given a few questions for better information analysis before being given instructions about how to score each map they see which, UI/UX-wise, would be present throughout the survey as a sidebar or dropdown menu. It'd be incredible if this survey was connected to the Galaxy editor or SC2 client so any user could click "play map" and pop right into a bot game or invite a friend to play, but that's more than wishful thinking at this point.)
I do feel like tasking pro players with judging over a hundred maps is completely non-functional, on several levels, and it would be better if they were reserved for judging, say, a fourth or fifth of the total submissions and be assigned to pair down that remaining pool for the final public vote along with other judges and mapmakers (or just filter out/downvote maps based on explicit, specific criteria for each category). In other words, have a group of judges, those judges each score and filter through the chunk of submissions assigned to them, then another group of judges (maybe the same ones plus a few more, or an entirely different group that includes a few pro players and map makers) scores and filters it again and gives feedback and chooses a few favorite among their "table", then that is sorted and presented as the public vote.
Having more than a handful of judges would help a ton as well, and I frankly don't see how that can be an issue unless TLMC is ran by five people in the middle of nowhere with dialup internet and no phone service (because even that isn't an issue for a couple of homebrewing and martial arts events I've attended or observed in Nebraska and Kansas in towns with a couple thousand people max, if not a couple hundred...).
In my experience with homebrewing competitions: + Show Spoiler +
There are usually about ten to twelve people at a table tasting and judging one style of beer over one day, or a few cases or styles over a few days. Or there are several tables of four to six people tasting their own flights of beer (three to eight different things) throughout the day in addition to a few admins/moderators/volunteers who tally everything and tell which ones (out of the top ten) the three to five "veterans"/professionals will taste to determine the winners. Altogether, at least a dozen people, usually thirty or so, sometimes fifty to a hundred for the really big ones that incorporate online judging. Five people isn't okay unless you're running an event in which you're judging literally twelve to sixteen items in one evening; otherwise you physically cannot give any item the attention it deserves within the time available, and even ten people isn't very many for that much, which is why, at least for a time long before the pandemic, a lot of these events would use their audience to help with judging and use it as a bullet point when selling tickets (or the more premium tickets, depending on the event).
Any legal system incorporating a jury also helps prevent judge/jury burnout and reckless decision-making through their court case and jury selection systems (although the efficacy of such systems is outside the bounds of this rambling, and I'm not an expert by any stretch), right?
Any legal system incorporating a jury also helps prevent judge/jury burnout and reckless decision-making through their court case and jury selection systems (although the efficacy of such systems is outside the bounds of this rambling, and I'm not an expert by any stretch), right?
Investing into making a system that can collect and sort submissions into pools with A number of standard maps, B number of rush maps, C macro maps, D new maps (or whatever), then assign those pools to each "table" of judges (doesn't have to be people together at a physical or virtual table, just a group of judges that gets their own pool of maps to score), it could help so many problems. And again, I know tiny homebrewing events ran by elderly people who figured this out, or learned how to use Excel just for their competitions, or hired someone else to do it, so it seems proven to help with all kinds of contests like this, and I expect it to be comparatively easy for TL/ESL/whoever to figure out if it isn't already a tool in their box.
In other words: Have a sorting system that automatically assigns each submission to a bucket, with a little overlap between each bucket, and assign one bucket to each judge instead of assigning all submissions to each judge, and ensure that the sorting system either puts everything of one type into each bucket or puts an assortment of things into each bucket (for TLMC, for now, I would go with the latter, assorted bucket option).
Thinking again to homebrew judging and the number of people who are encouraged to try again and resubmit their brews: + Show Spoiler +
If maps were also not judged for ladder or tournament "effectiveness" based solely on whether or not they won the current TLMC and there was a more active or open discussion happening with organizers/admins (TLMC and tournaments) and judges, and some mapmakers were allowed to resubmit new versions of their maps (especially for non-standard categories), then it could allow for some awesome things to happen. But, the fairness or effectiveness of this or what-have-you also depends on the quality/structure of TLMC itself, and the quality/structure of its judging, so that should get fixed first.
+ Show Spoiler +
+ Show Spoiler +
Or... maybe we should just ask tournament organizers big and small to skip the TLMC entirely and directly hire mapmakers and game designers and their own casters as consultants, then commission map makers to work with their private panel to design maps specific to each tournament, allowing each tournament and its sponsors to find the right balance between standardization/stagnation and experimentation for their own brand, audience, formats, and players. This would also allow more than just TLMC to exist, as organizers could also run smaller-scale online cups for playtesting their own maps at diamond and lower level, and masters and higher level of play, though I acknowledge that I'm probably thinking too big at this point and only setting myself up to be disappointed when nobody with tons of money wants to do something really, really cool with it. In that case, someone should crowdfund this, and fail or succeed it would at least be something new to talk about.
Perhaps one good solution for the communication and quality and stagnation issues is to break up the TLMC into a few different sub-events, much like how the actual SC2 events are structured, and, again, having a little more automation of any kind would help a lot with organizing and running it. Or, and I know this was proposed a long time ago: there could be multiple TLMCs throughout the year, and each one would have a different focus (e.g. one new mechanic contest, one rush contest, one macro contest, one aesthetic design contest, one "Old Thing But Again/Different" or even T/P/Z/R and Multiplayer/Commander contests), and mapmakers and judge groups could devote themselves solely to the clear objectives of that specific contest without anyone stretching themselves too thin, and allowing creators to hone and show off their disciplines and skills and personal style.
Having not done things differently in the past makes a lot of these ideas difficult without someone swooping in and saying "We're funding a giant thing!" or "Look, this exists now, you're welcome!" But just restructuring the contest a little bit, or using the TL userbase to filter the submission pool, or getting just a few more people to judge maps would help; any one of those things would be better, all of them (or one of them and something else) would be so exciting. I mean, what if they just shot a survey up on TL and the Bnet client that let everybody vote on old goofy maps, then somebody adjusted it for current ladder considerations and it got popped back into the rotation for the current season (either without replacing any current maps, or replacing the most disliked/vetoed current map)? It's such a tiny thing that would be massively cool and fun.
Sad rant ending (ooh alternate blog ending, exciting!): + Show Spoiler +
I stopped trying to make maps for SC2 for the same reasons I did not stop homebrewing and submitting beers to homebrewing competitions, and did not stop participating in martial arts competitions and crosscountry and track: TLMC isn't fun enough to put up with the amount of time and energy it takes to participate, and the group of finalists often feels unfair (not even when you don't see your own map, but when you see no mention of some other map you liked or hoped for, or a dozen of them), and the lack of transparency is abysmal.
After I submitted the wrong map to a past TLMC, I realized my mistake way too late, but I kept paying attention to the contest because I was invested in the game as a player, viewer, and mapmaker, and fan of pro players and competing mapmakers. Afterwards, I had little motivation to continue participating besides my own drive and boredom. Whereas with homebrewing and crosscountry and all kinds of other things, everyone communicated, everyone helped me stay motivated (not only other struggling homebrewers or non-varsity kids, I mean everyone with rare exception), and everyone was always there to have fun and learn/teach/improve first and foremost. People gave a s***, from strangers who bumbled their way into the space to, in the case of homebrewing, regular bar patrons and faire attendees to every judge and organizer. And all of that showed—I felt all of that—because of how transparent and openly/actively engaging the process and organizers all were, compared to TLMC which just isn't. (And not just homebrewing either, again—art, music, dance, martial arts, LARP officiating, drum battles, skate battles, kids' and semi-pro ice skating exhibitions, even local gardening and baking and cake decorating workshops I attended as a child. Makes me kinda sad that my favorite game still hasn't figured it out, hence all the rambling.)
After I submitted the wrong map to a past TLMC, I realized my mistake way too late, but I kept paying attention to the contest because I was invested in the game as a player, viewer, and mapmaker, and fan of pro players and competing mapmakers. Afterwards, I had little motivation to continue participating besides my own drive and boredom. Whereas with homebrewing and crosscountry and all kinds of other things, everyone communicated, everyone helped me stay motivated (not only other struggling homebrewers or non-varsity kids, I mean everyone with rare exception), and everyone was always there to have fun and learn/teach/improve first and foremost. People gave a s***, from strangers who bumbled their way into the space to, in the case of homebrewing, regular bar patrons and faire attendees to every judge and organizer. And all of that showed—I felt all of that—because of how transparent and openly/actively engaging the process and organizers all were, compared to TLMC which just isn't. (And not just homebrewing either, again—art, music, dance, martial arts, LARP officiating, drum battles, skate battles, kids' and semi-pro ice skating exhibitions, even local gardening and baking and cake decorating workshops I attended as a child. Makes me kinda sad that my favorite game still hasn't figured it out, hence all the rambling.)
