• Log InLog In
  • Register
Liquid`
Team Liquid Liquipedia
EDT 09:41
CEST 15:41
KST 22:41
  • Home
  • Forum
  • Calendar
  • Streams
  • Liquipedia
  • Features
  • Store
  • EPT
  • TL+
  • StarCraft 2
  • Brood War
  • Smash
  • Heroes
  • Counter-Strike
  • Overwatch
  • Liquibet
  • Fantasy StarCraft
  • TLPD
  • StarCraft 2
  • Brood War
  • Blogs
Forum Sidebar
Events/Features
News
Featured News
[ASL19] Finals Recap: Standing Tall3HomeStory Cup 27 - Info & Preview18Classic wins Code S Season 2 (2025)16Code S RO4 & Finals Preview: herO, Rogue, Classic, GuMiho0TL Team Map Contest #5: Presented by Monster Energy6
Community News
Flash Announces Retirement From ASL19Weekly Cups (June 23-29): Reynor in world title form?12FEL Cracov 2025 (July 27) - $8000 live event16Esports World Cup 2025 - Final Player Roster14Weekly Cups (June 16-22): Clem strikes back1
StarCraft 2
General
The SCII GOAT: A statistical Evaluation Weekly Cups (June 23-29): Reynor in world title form? StarCraft Mass Recall: SC1 campaigns on SC2 thread How does the number of casters affect your enjoyment of esports? Esports World Cup 2025 - Final Player Roster
Tourneys
FEL Cracov 2025 (July 27) - $8000 live event HomeStory Cup 27 (June 27-29) WardiTV Mondays SOOPer7s Showmatches 2025 $200 Biweekly - StarCraft Evolution League #1
Strategy
How did i lose this ZvP, whats the proper response Simple Questions Simple Answers
Custom Maps
[UMS] Zillion Zerglings
External Content
Mutation # 480 Moths to the Flame Mutation # 479 Worn Out Welcome Mutation # 478 Instant Karma Mutation # 477 Slow and Steady
Brood War
General
Where did Hovz go? [ASL19] Finals Recap: Standing Tall Flash Announces Retirement From ASL ASL20 Preliminary Maps BW General Discussion
Tourneys
[Megathread] Daily Proleagues [BSL20] GosuLeague RO16 - Tue & Wed 20:00+CET The Casual Games of the Week Thread [BSL20] ProLeague LB Final - Saturday 20:00 CET
Strategy
Simple Questions, Simple Answers I am doing this better than progamers do.
Other Games
General Games
Stormgate/Frost Giant Megathread Nintendo Switch Thread Path of Exile What do you want from future RTS games? Beyond All Reason
Dota 2
Official 'what is Dota anymore' discussion
League of Legends
Heroes of the Storm
Simple Questions, Simple Answers Heroes of the Storm 2.0
Hearthstone
Heroes of StarCraft mini-set
TL Mafia
TL Mafia Community Thread Vanilla Mini Mafia
Community
General
US Politics Mega-thread Stop Killing Games - European Citizens Initiative Things Aren’t Peaceful in Palestine Trading/Investing Thread Russo-Ukrainian War Thread
Fan Clubs
SKT1 Classic Fan Club! Maru Fan Club
Media & Entertainment
Anime Discussion Thread [Manga] One Piece [\m/] Heavy Metal Thread Korean Music Discussion
Sports
2024 - 2025 Football Thread NBA General Discussion Formula 1 Discussion TeamLiquid Health and Fitness Initiative For 2023 NHL Playoffs 2024
World Cup 2022
Tech Support
Computer Build, Upgrade & Buying Resource Thread
TL Community
The Automated Ban List
Blogs
from making sc maps to makin…
Husyelt
Blog #2
tankgirl
Game Sound vs. Music: The Im…
TrAiDoS
StarCraft improvement
iopq
Heero Yuy & the Tax…
KrillinFromwales
Trip to the Zoo
micronesia
Customize Sidebar...

Website Feedback

Closed Threads



Active: 571 users

Re: Competition Judging

Blogs > blunderfulguy
Post a Reply
blunderfulguy
Profile Blog Joined April 2016
United States1415 Posts
Last Edited: 2021-07-14 06:04:47
July 14 2021 06:03 GMT
#1
This is a continuation of my post here in the "Pro Players Should Not Judge Maps" thread, digging into why I prefer my "pitch" in that comment, and some of my experience being a participant, audience member, and hobbyist- to enthusiast-level judge in various competitions, in this case mostly homebrewing events and beer judging. I also have some replies to comments in the SC2 General thread, which I felt was too long and rambly to put in my comment on that thread and fit better with everything else here.

Regarding my own experience judging hobbyist-level homebrewing competitions (beer and mead) and martial arts and kinetic art exhibitions (for lack of a better term) and tournaments, there are so many things TLMC could be doing differently.

For context: When judging beer or any beverage (fermented/alcoholic and otherwise), there are several critical elements (ingredients, appearance, smell, taste, feel) that define the style which are judged and scored objectively, along with flaws (added sugars, atypical yeast strains, off color or cloudiness, bad smells or tastes, particulates, etc.). However, some elements that are considered flaws by purists of one style of beer can be weighted lower among a panel of judges who see those "objective flaws" as subjective-but-enjoyable elements that make that beverage unique among other products in the same style, and these are the beers which, in my experience, often win local brewing competitions. (E.g., a beer or mead that accidentally got infected by another type of bacteria, or had a bad brew or fermentation that the brewer tried to save by adding more sugar or "adjuncts", but that got more fruity, floral, and tart smells and flavors, is objectively infected but is safe to drink and more interesting than if it wasn't "flawed".)

Regarding homebrew event judges, for friendly brewing events and competitions I have been a part of and observed from afar, judges are usually expected to be transparent about which styles they do not enjoy, and are expected to not judge those beers critically and to instead defer to the other judges at their table when scoring that beer as a group. Judges who are shown to be very picky are ignored (or shut down) during the table's "diplomatic" conversations during final scoring, or they get assigned by the organizers/admins to score only the final round of beer (the final round to decide which ones in a specific category get bronze/silver/gold or whatever).

Sometimes submissions are only rated across five to six elements within their category (look, smell, taste, feel, overall, and sometimes an "X factor") which is given back to the brewer (the scoring sheet being a literal piece of paper, a form-fillable PDF, or like an online survey). Other times the table that rated the beer comes to a decision on it and the head of the table (or whoever the table points to, or whoever volunteers) writes a brief description (one or two sentences, or a jumble of adjectives) on the table's score card. There is always information being shared about why it got the score it did, even if it's just a handful of adjectives like "too murky" or "grandpa's toes" or "grandma's kitchen during summer". And other times it's a conversation, and the judge who is also a brewer (or who knows one and immediately finds them) will tell you exactly what caused the off flavor that caused the low score and exactly how to avoid it the next time you brew, or they'll ask you how you brewed it or what ingredients you used if the org doesn't collect that info and they'll give you some opinions or take notes from you for themselves.

Regarding fun and audience participation: + Show Spoiler +
In contrast to that professionalism, sometimes completely random people from the crowd of strangers, that day's volunteers, and/or submitters who showed up are chosen to help "filter" through the bulk of submissions, or get asked to join a panel to try out and judge the most unique/weird beers and, at that table, are given just as much weight in their opinions as the rest of the invited judges, professional brewers, and retiring cicerones/sommeliers. These things don't happen all the time, and I assume they don't happen at all depending on where you go, especially the more prestigious and global/busy the competition gets, but they have happened and don't seem extraordinarily rare or strange to me. It's so fun to have seen. Which is important to me and is something that I feel could be applied to mapmaking contests.


Regarding problematic events I've been to that have led to conversations like this very TL thread: + Show Spoiler +
In the past several years, I have also seen a few competitions ran in a similar way as the TLMC, usually for very, very small, local events, which in my experience leads to many (pretty much all) of the same negative feelings and discussions of the crowd and participants. Tensions don't usually get very high because next to nothing is at stake at those events, or if they do heighten it prompts the organizers to take action and ban a participant or judge from the next event, or restructure the event to be more like other, longer lasting events that proove via their history, obvious excitement, and success stories (i.e. medalists going on to start or expand businesses or schools) that there are better ways to run these things. At the very lowest level, these kinds of competitions evolve to ensure that problems don't persist and people want to keep coming back, or the problem persists the next season/year and the event gets cancelled the year after that because nobody besides a few old purists want to bother submitting or judging anything to their own shindig.


Back to TLMC. If the pools of judges were bigger, the categories were better defined, the judging was divided among groups, and the job(s) of the judges were more specific, the entire contest process would be much better, and we would get much higher quality if not at least much more interesting maps during the public testing and voting stage.

On July 14 2021 08:09 Teoita wrote:
For what it's worth, my main mindset when looking at a map was to always first ask: "how would one defend a natural base against some all-in? how would you defend a third against a soultrain or roach max? Is it too easy to cut the map in half and turtle?".

I strongly feel that this mindset should be encouraged through the rules of the judging process, and ideally the rules and goals for judges would change for each category. (All of my other paragraphs aside, I'd be interested to see this used alongside public voting as a first stage; i.e. make a survey that TL users or Bnet client users with X hours of SC2 played can open, and they are given a few questions for better information analysis before being given instructions about how to score each map they see which, UI/UX-wise, would be present throughout the survey as a sidebar or dropdown menu. It'd be incredible if this survey was connected to the Galaxy editor or SC2 client so any user could click "play map" and pop right into a bot game or invite a friend to play, but that's more than wishful thinking at this point.)

I do feel like tasking pro players with judging over a hundred maps is completely non-functional, on several levels, and it would be better if they were reserved for judging, say, a fourth or fifth of the total submissions and be assigned to pair down that remaining pool for the final public vote along with other judges and mapmakers (or just filter out/downvote maps based on explicit, specific criteria for each category). In other words, have a group of judges, those judges each score and filter through the chunk of submissions assigned to them, then another group of judges (maybe the same ones plus a few more, or an entirely different group that includes a few pro players and map makers) scores and filters it again and gives feedback and chooses a few favorite among their "table", then that is sorted and presented as the public vote.

Having more than a handful of judges would help a ton as well, and I frankly don't see how that can be an issue unless TLMC is ran by five people in the middle of nowhere with dialup internet and no phone service (because even that isn't an issue for a couple of homebrewing and martial arts events I've attended or observed in Nebraska and Kansas in towns with a couple thousand people max, if not a couple hundred...).
In my experience with homebrewing competitions: + Show Spoiler +
There are usually about ten to twelve people at a table tasting and judging one style of beer over one day, or a few cases or styles over a few days. Or there are several tables of four to six people tasting their own flights of beer (three to eight different things) throughout the day in addition to a few admins/moderators/volunteers who tally everything and tell which ones (out of the top ten) the three to five "veterans"/professionals will taste to determine the winners. Altogether, at least a dozen people, usually thirty or so, sometimes fifty to a hundred for the really big ones that incorporate online judging. Five people isn't okay unless you're running an event in which you're judging literally twelve to sixteen items in one evening; otherwise you physically cannot give any item the attention it deserves within the time available, and even ten people isn't very many for that much, which is why, at least for a time long before the pandemic, a lot of these events would use their audience to help with judging and use it as a bullet point when selling tickets (or the more premium tickets, depending on the event).

Any legal system incorporating a jury also helps prevent judge/jury burnout and reckless decision-making through their court case and jury selection systems (although the efficacy of such systems is outside the bounds of this rambling, and I'm not an expert by any stretch), right?


Investing into making a system that can collect and sort submissions into pools with A number of standard maps, B number of rush maps, C macro maps, D new maps (or whatever), then assign those pools to each "table" of judges (doesn't have to be people together at a physical or virtual table, just a group of judges that gets their own pool of maps to score), it could help so many problems. And again, I know tiny homebrewing events ran by elderly people who figured this out, or learned how to use Excel just for their competitions, or hired someone else to do it, so it seems proven to help with all kinds of contests like this, and I expect it to be comparatively easy for TL/ESL/whoever to figure out if it isn't already a tool in their box.
In other words: Have a sorting system that automatically assigns each submission to a bucket, with a little overlap between each bucket, and assign one bucket to each judge instead of assigning all submissions to each judge, and ensure that the sorting system either puts everything of one type into each bucket or puts an assortment of things into each bucket (for TLMC, for now, I would go with the latter, assorted bucket option).

Thinking again to homebrew judging and the number of people who are encouraged to try again and resubmit their brews: + Show Spoiler +
If maps were also not judged for ladder or tournament "effectiveness" based solely on whether or not they won the current TLMC and there was a more active or open discussion happening with organizers/admins (TLMC and tournaments) and judges, and some mapmakers were allowed to resubmit new versions of their maps (especially for non-standard categories), then it could allow for some awesome things to happen. But, the fairness or effectiveness of this or what-have-you also depends on the quality/structure of TLMC itself, and the quality/structure of its judging, so that should get fixed first.

+ Show Spoiler +
Or... maybe we should just ask tournament organizers big and small to skip the TLMC entirely and directly hire mapmakers and game designers and their own casters as consultants, then commission map makers to work with their private panel to design maps specific to each tournament, allowing each tournament and its sponsors to find the right balance between standardization/stagnation and experimentation for their own brand, audience, formats, and players. This would also allow more than just TLMC to exist, as organizers could also run smaller-scale online cups for playtesting their own maps at diamond and lower level, and masters and higher level of play, though I acknowledge that I'm probably thinking too big at this point and only setting myself up to be disappointed when nobody with tons of money wants to do something really, really cool with it. In that case, someone should crowdfund this, and fail or succeed it would at least be something new to talk about.


Perhaps one good solution for the communication and quality and stagnation issues is to break up the TLMC into a few different sub-events, much like how the actual SC2 events are structured, and, again, having a little more automation of any kind would help a lot with organizing and running it. Or, and I know this was proposed a long time ago: there could be multiple TLMCs throughout the year, and each one would have a different focus (e.g. one new mechanic contest, one rush contest, one macro contest, one aesthetic design contest, one "Old Thing But Again/Different" or even T/P/Z/R and Multiplayer/Commander contests), and mapmakers and judge groups could devote themselves solely to the clear objectives of that specific contest without anyone stretching themselves too thin, and allowing creators to hone and show off their disciplines and skills and personal style.

Having not done things differently in the past makes a lot of these ideas difficult without someone swooping in and saying "We're funding a giant thing!" or "Look, this exists now, you're welcome!" But just restructuring the contest a little bit, or using the TL userbase to filter the submission pool, or getting just a few more people to judge maps would help; any one of those things would be better, all of them (or one of them and something else) would be so exciting. I mean, what if they just shot a survey up on TL and the Bnet client that let everybody vote on old goofy maps, then somebody adjusted it for current ladder considerations and it got popped back into the rotation for the current season (either without replacing any current maps, or replacing the most disliked/vetoed current map)? It's such a tiny thing that would be massively cool and fun.

Sad rant ending (ooh alternate blog ending, exciting!): + Show Spoiler +
I stopped trying to make maps for SC2 for the same reasons I did not stop homebrewing and submitting beers to homebrewing competitions, and did not stop participating in martial arts competitions and crosscountry and track: TLMC isn't fun enough to put up with the amount of time and energy it takes to participate, and the group of finalists often feels unfair (not even when you don't see your own map, but when you see no mention of some other map you liked or hoped for, or a dozen of them), and the lack of transparency is abysmal.

After I submitted the wrong map to a past TLMC, I realized my mistake way too late, but I kept paying attention to the contest because I was invested in the game as a player, viewer, and mapmaker, and fan of pro players and competing mapmakers. Afterwards, I had little motivation to continue participating besides my own drive and boredom. Whereas with homebrewing and crosscountry and all kinds of other things, everyone communicated, everyone helped me stay motivated (not only other struggling homebrewers or non-varsity kids, I mean everyone with rare exception), and everyone was always there to have fun and learn/teach/improve first and foremost. People gave a s***, from strangers who bumbled their way into the space to, in the case of homebrewing, regular bar patrons and faire attendees to every judge and organizer. And all of that showed—I felt all of that—because of how transparent and openly/actively engaging the process and organizers all were, compared to TLMC which just isn't. (And not just homebrewing either, again—art, music, dance, martial arts, LARP officiating, drum battles, skate battles, kids' and semi-pro ice skating exhibitions, even local gardening and baking and cake decorating workshops I attended as a child. Makes me kinda sad that my favorite game still hasn't figured it out, hence all the rambling.)


***
Blunder Man doing everything thing a blunder can.
Please log in or register to reply.
Live Events Refresh
Next event in 2h 19m
[ Submit Event ]
Live Streams
Refresh
StarCraft 2
Harstem 412
RotterdaM 325
StarCraft: Brood War
Rain 5140
EffOrt 1344
Larva 1018
Stork 425
Mini 386
Last 226
Zeus 192
BeSt 192
ZerO 159
hero 132
[ Show more ]
Light 131
Pusan 113
Snow 110
Mind 107
Hyun 106
Sharp 105
Rush 69
Aegong 48
JYJ32
Shinee 32
Movie 29
Barracks 26
NaDa 23
ajuk12(nOOB) 15
scan(afreeca) 11
HiyA 10
IntoTheRainbow 10
Noble 10
SilentControl 8
Icarus 7
Terrorterran 4
Hm[arnc] 2
Britney 0
Stormgate
NightEnD13
Dota 2
Gorgc7833
qojqva2360
XcaliburYe360
febbydoto1
League of Legends
singsing2616
Counter-Strike
markeloff115
edward44
Super Smash Bros
Mew2King147
Heroes of the Storm
Khaldor149
Other Games
hiko881
B2W.Neo655
DeMusliM422
crisheroes221
XaKoH 188
ArmadaUGS88
ZerO(Twitch)15
Organizations
StarCraft 2
Blizzard YouTube
StarCraft: Brood War
BSLTrovo
sctven
[ Show 11 non-featured ]
StarCraft 2
• AfreecaTV YouTube
• intothetv
• Kozan
• IndyKCrew
• LaughNgamezSOOP
• Migwel
• sooper7s
StarCraft: Brood War
• BSLYoutube
• STPLYoutube
• ZZZeroYoutube
Dota 2
• C_a_k_e 2451
Upcoming Events
uThermal 2v2 Circuit
2h 19m
OSC
5h 19m
SHIN vs Bunny
Cham vs MaNa
SKillous vs TBD
PAPI vs Jumy
Gerald vs Moja
ArT vs TBD
Replay Cast
10h 19m
The PondCast
20h 19m
RSL Revival
20h 19m
ByuN vs Classic
Clem vs Cham
WardiTV European League
1d 2h
Replay Cast
1d 10h
RSL Revival
1d 20h
herO vs SHIN
Reynor vs Cure
WardiTV European League
2 days
FEL
2 days
[ Show More ]
Korean StarCraft League
2 days
CranKy Ducklings
2 days
RSL Revival
2 days
FEL
3 days
Sparkling Tuna Cup
3 days
RSL Revival
3 days
FEL
3 days
BSL: ProLeague
4 days
Dewalt vs Bonyth
Replay Cast
5 days
Replay Cast
5 days
The PondCast
6 days
Liquipedia Results

Completed

Proleague 2025-06-28
HSC XXVII
Heroes 10 EU

Ongoing

JPL Season 2
BSL 2v2 Season 3
BSL Season 20
Acropolis #3
KCM Race Survival 2025 Season 2
CSL 17: 2025 SUMMER
Copa Latinoamericana 4
Championship of Russia 2025
RSL Revival: Season 1
Murky Cup #2
BLAST.tv Austin Major 2025
ESL Impact League Season 7
IEM Dallas 2025
PGL Astana 2025
Asian Champions League '25
BLAST Rivals Spring 2025
MESA Nomadic Masters
CCT Season 2 Global Finals
IEM Melbourne 2025
YaLLa Compass Qatar 2025

Upcoming

CSLPRO Last Chance 2025
CSLPRO Chat StarLAN 3
K-Championship
uThermal 2v2 Main Event
SEL Season 2 Championship
FEL Cracov 2025
Esports World Cup 2025
StarSeries Fall 2025
FISSURE Playground #2
BLAST Open Fall 2025
BLAST Open Fall Qual
Esports World Cup 2025
BLAST Bounty Fall 2025
BLAST Bounty Fall Qual
IEM Cologne 2025
FISSURE Playground #1
TLPD

1. ByuN
2. TY
3. Dark
4. Solar
5. Stats
6. Nerchio
7. sOs
8. soO
9. INnoVation
10. Elazer
1. Rain
2. Flash
3. EffOrt
4. Last
5. Bisu
6. Soulkey
7. Mini
8. Sharp
Sidebar Settings...

Advertising | Privacy Policy | Terms Of Use | Contact Us

Original banner artwork: Jim Warren
The contents of this webpage are copyright © 2025 TLnet. All Rights Reserved.