EPTPro Players Should Not Judge Maps - Page 9
| Forum Index > SC2 General |
|
NewSunshine
United States5938 Posts
| ||
Teoita
Italy12246 Posts
Pokebunny's suggestion is great. But in order to do that, we actually need DIFFERENT types of maps. With pros making the decisions, we aren't going to get different types of maps. All we are getting out of this contest is more of the same because the judges have a conflict of interest in this whole ordeal. Do you see how the way this contest is setup as a double edged sword so it that doesn't actually accomplish what it is set out to? I disagree. Pokebunny's idea works regardless of what's in the map pool - if the maps are standard, little is changed, if they are not, then we get to explore crazy ideas while mitigating side effects. It seems reasonable to me to start with what we can be confident is a good idea (changing map vetos), rather than with what can backfire horribly (radically different maps) BW korean pros were forced to play unorthodox maps all the time or what some pros would call ridiculous maps all the time. And you know what, those maps were also fun to play on as a player AND also provided exciting games for the viewers. The BW korean pros accepted it. True, but only to some extent. Some maps backfired spectacularly and ended up being mirror match fests, but indeed, it did work at times, provided the format was designed to account for these things. Which again is why I would emphasize re-looking at formats. Now that I think about it, imo the way forward is to try these new map formats in the TLMC Opens, and possibly see if we can get them in TSL or something (which would work well with its BW heritage anyway). And yes, Proleague was amazing. | ||
|
ReachTheSky
United States3294 Posts
On July 14 2021 09:29 Teoita wrote: I disagree. Pokebunny's idea works regardless of what's in the map pool - if the maps are standard, little is changed, if they are not, then we get to explore crazy ideas while mitigating side effects. It seems reasonable to me to start with what we can be confident is a good idea (changing map vetos), rather than with what can backfire horribly (radically different maps) True, but only to some extent. Some maps backfired spectacularly and ended up being mirror match fests, but indeed, it did work at times, provided the format was designed to account for these things. Which again is why I would emphasize re-looking at formats. Now that I think about it, imo the way forward is to try these new map formats in the TLMC Opens, and possibly see if we can get them in TSL or something (which would work well with its BW heritage anyway). And yes, Proleague was amazing. If you want radical change, you need to make what some might refer to as radical decisions such as putting radically different maps into the map pool. Your suggestion doesn't address what the majority of the player base wants, change on ladder. Instead it aims to compromise what the pros want and what the playerbase wants BUT only from a viewer perspective for tournaments, not as a non-pro who plays starcraft 2 on ladder. We want change where the majority of the playerbase plays, on ladder. If it's not gonna be typical balance changes via tweaking unit numbers/abilities/costs then the only alternative is to include multiple radically different or unorthodox maps on the ladder, if someone doesn't want to play them, they can just veto them. But in order to do this we actually need the different style maps put on the ladder. Other wise it's just more of the same. | ||
|
WombaT
Northern Ireland24512 Posts
Even helping run and casting local tournaments I’ve suggested for years bringing back some off-ladder maps (Golden Wall most recently) and got roundly voted down every time. Not because they hate these maps, but especially with us being increasingly busy these days it’s just easier to practice the ladder instead of finding practice partners (and we have a pretty incestuous scene so, you’d be practicing with your potential opponents). SC2 lacks a lot in making a wider map pool that’s widely played. Not many maps to start with, you can’t queue certain matchups, and there’s no kind of non-ladder matchmaking with custom parameters either. | ||
|
Semmo
Korea (South)627 Posts
On July 14 2021 09:22 NewSunshine wrote: I will also second my eternal love for what SC2 proleague tried to do. They were all in on trying out all kinds of crazy, experimental maps, and asking the world's best players to devise new strategies for their specific matchup and map. It was my favorite Starcraft 2 stream. They tried all kinds of shit, including some that made it to ladder like Planet S, King Sejong Station and Korhal Sky Island. They also ended up with maps that bombed in terms of balance, but they fucking tried it anyway. Naro Station, Arkanoid remake, Anaconda, Gwangalli Beach, Outboxer, Maze, and the list goes on. If we can come even kinda close to what they did with Proleague, I would be ecstatic. Man, Jacky’s proleague maps were the shit for sure. Would love to see similar balls to the walls maps. I think the proleague’s format definitely allowed for some more crazy maps though. | ||
|
blunderfulguy
United States1415 Posts
This is just a quick and dirty, late night example of how I would pitch a new mapmaking contest, or approach redesigning TLMC, thought up long before seeing this thread: Multiple rounds of judging. + Show Spoiler + Splitting the workload between groups of judges and separate phases allows each judge to spend more time and attention on each submission they review. Panels of contest/tournament administrators and moderators ensure the first pool of maps is free of issues, and that the final pool of maps retains a greater degree of creativity and uniqueness (gameplay-wise) than past competitions. ⦁Stage 1: Filtering. Maps are reorganized by type (standard and creative, standard and rush, and by number of players), and unplayable (not unfinished) maps are filtered out of the entire submission pool, then maps and judges are automatically distributed into groups. Maps without textures and doodads are not removed yet. + Show Spoiler + E.g., a panel of moderators/admins and/or public users looks at the entire submission pool, with each moderator or user being assigned an equal-sized pool/share of submissions to view; in this round, no scores are assigned, submisisons in incorrect or unclear categories are resorted into more accurate categories, and maps are only removed if they are unplayable (which could be automated using AI and bot matchmaking). Then submissions are split into pools (with a fair distribution of categories based on how many submissions are present in each category; i.e., the number of submissions of each category are proportionate within each pool) and those pools (with some overlap in submissions between pools) are assigned to each group/panel of judges. ⦁Stage 2: Judging Round 1. Each panel judges their pool (with each panelist judging and scoring independently from other judges in their panel). Low-scoring maps are then separated again and reviewed by the moderators for flaws or harsh criticism, then either removed from further judging or randomly assigned to a different panel to be scored again (then removed if they still score low). Panelists in this and each following stage are encouraged/instructed to give notes for at least half the submissions they judge. ⦁Stage 3: Judging Round 2. New panels and pools are generated and assigned to each other, then each panelist judges and scores each submission in their pool. Judges are instructed to tag their favorite maps and give comments on them. Submitters/mapmakers are notified when their submissions pass this stage. ⦁Stage 4: Judging Round 3. Each panel (as a group) reviews favorites from among all panels (including or excluding their own favorites), and scores submissions in their pool and the global favorites pool. Scoring in this stage is done categorically across multiple factors, and panels are encouraged to playtest a variety of submissions and favorites. Submitters/mapmakers are notified when their submissions do not pass this stage, and given comments on those submissions. ⦁Stage 5: Review. Scores are averaged and moderators review the surviving submissions and are tasked with breaking ties, forming the final pool and collecting notes from submitters/mapmakers and panelists/judges. The end of this round of judging also coincides with the deadline for aesthetic changes to submissions. + Show Spoiler + The panel of moderators during this stage can include mapmakers and pro players. ⦁Stage 6: Public Playtest. The final pool (and final aesthetics and creator descriptions for all submissions) is presented for public playtesting in a tournament with professional players. + Show Spoiler + Creators are encouraged to watch playtests and tournament matches, and are allowed to change their maps before the final round of public voting. However, they are also discouraged from making major changes that affect gameplay in any other way besides fixing significant flaws. Major changes to any maps during and after this stage should be reserved for after the maps have been played for weeks to months—if not a year—in order to make the map better for ladder and/or tournament play. ⦁Stage 7: Public Voting. The final round of judging is completed via public voting, determining the ranked winners of the mapmaking tournament. Take some workload off the judges by splitting the work into chunks, and getting more judges if possible. Use the audience to do some of the work wherever possible. Keep it interesting. Give feedback. Be transparent. On July 12 2021 04:02 hjpalpha wrote: if you would have seen all the maps they had to judge ...... well lets just say there are maps that are that bad As someone who has accidentally saved over the wrong file countless times, and who has submitted the entirely wrong file to the TLMC, I would say it's safe to say that a lot of submissions can be almost immediately handwaived away. However, this misses the point: many maps get handwaived away and thrown aside that are not completely unplayable, and deserve much more than a few seconds of attention and next to zero critical thought. I fumbled between bronze and gold league for years during Wings of Liberty and still understood the concept of vertical and horizontal spawn restrictions (and even understood that concept years and years before that from playing and watching Brood War, Warcraft 3, and even Halo and tabletop Dungeons & Dragons when I was ten...). Judges should be expected to understand these concepts, and be expected to respect submissions enough to not rate perfectly playable, average maps a 1/5. They should also be given enough time to do their job so they don't have to rush these decisions. On July 12 2021 07:15 Charoisaur wrote: Not only pros want to play on standard maps, I for my part don't want to play on a new Secret Spring ever again. Sure for spectactors and Gold league heroes it's "cool" and "interesting" but I don't think we should design the game for those people over the people who invest their life into the game. "Those people" do also invest their lives in the game, and comprise the majority of viewers, as far as I understand. If you ignore or actively disengage with them (in my experience), you'll turn the event into something only five old, lonely guys think they enjoy just because thinking about it doesn't infuriate them. It's supposed to involve some good, tested game design and structure and strategy, but not only that. It's also supposed to be fun, so let it be fun, and encourage the fun-ness when/where you can because otherwise nobody will care since there are so many other events that are better structured and more fun and engaging, and that produce excellent/interesting/engaging results more often than not. I have more thoughts in a blog post here, discussing my experience as a participant and judge in other competitions, along with much rambling, as is my style. This post might make more sense with the context of that blog post, but isn't required by any means, I don't think. I guess look at my blog if you're bored waiting for more replies in this thread? | ||
|
Callynn
Netherlands917 Posts
| ||
|
NeuralNet88
22 Posts
I've played on that map with friends and we all have enjoyed it quite a bit, so I don't really understand what makes that map a 1 in the judge's eyes exactly. There were others that I feel like he didn't give the correct score for, mainly I guess because he didn't really look at the maps with any kind of depth. I noticed that he didn't really have any notes for most of the maps as to why they received the score they did either. I guess if you're not really going to look at the maps in any kind of detail, there won't be any real detailed explanation as to why the map did or didn't do so well on the scoring system being used. I wonder if the other judges did this as well. Would explain why a lot of the more well designed maps didn't get a finalist spot. | ||
|
Kantuva
Uruguay206 Posts
On July 14 2021 08:51 ReachTheSky wrote: 5. We actually had an opportunity to switch things up, but we can't because a small group of less than 100 people want the game to revolve around their preferences, even though there are thousands and thousands of other players who play this game. I have been quite on the sidelines when it came to this discussion on TL, and far more active on the snuggly MapCave Discord Server. But I just need to note: It is not this. TLMC#15 was set up on the current way, with the current inertia of previous TLMC's in order to ensure map-pool stability from "Active Blizz DevTeam" era into "ESL taking some more of the reigns" era Scheduling a TLMC is quite a process, which needs to be telegraphed carefully to mapmakers with anticipation, the categories very much work on this favor, and that's what was done for the 4p map judge pickings, to telegraph to mapmakers that we would be doing things, so they, after many years of not doing so, would start producing +3p maps, the issue came from the natural clash of a system which was more or less meant for status quo into another to seek out outsiders, and this, per se "is kind of the point". TLMC15 was supposed from a macro perspective to not produce seismic changes because doing so would throw a wrench on balance and gameplay stability, but at the same time, it also, through the OP TLMC#15 itself set the strong expectations that it would bring about change and a breakage of inertia These from my perspective are the core of the conflict, but, I dont see people being dissapointed on how "non-nonstandard" the maps were as a bad thing, because again, there are bigger forces at stake here, DevTeam taking a step aside, and ESL with the community taking a step on carrying the weight, and that step forward needs to be stable. So I see the overall selection of more standard type maps as a net positive, where a net negative would have been increasing the risk of unbalance without an active devteam or a mature community-ESL-Blizz relationship to handle potential persistent map related problems. Now. Certainly, pro-players as influencers carry a huge amount of weight on both the collective mind of the public, and on the minds of tournament administrators, but it needs to be noted that TLMC is a process to get maps from the community into the ladder, and as Teoita mentioned, there have been different ways of setting up the TLMC and judging, and going forward we, on the mapmaking side of things, will be experimenting with a novel process to manage at least the early initial cut of +200 -> 40, 50 maps which can then be sampled by invited guests composed of casters, streamers, proplayers, tournament organizers, etc to create map pools out of. As an alternative to needing to parse ~180 maps at a time on a single week by a small amount of judges But to recap, this is a matter of the process, not of 100 or so "top people" holding the status quo. Now, I am certain that once the new process is in place, there will be riots from extremists whom dont want to play on anything other than a re-skinned version of Catalyst , but even they understand that the macro perspective and SC2 as E-Sports and Entertainment takes precedence, which is how the entire "thing" even began, as a Reality TV Show, as a spectacle | ||
Mizenhauer
United States1808 Posts
On July 19 2021 11:43 Kantuva wrote: I have been quite on the sidelines when it came to this discussion on TL, and far more active on the snuggly MapCave Discord Server. But I just need to note: It is not this. TLMC#15 was set up on the current way, with the current inertia of previous TLMC's in order to ensure map-pool stability from "Active Blizz DevTeam" era into "ESL taking some more of the reigns" era Scheduling a TLMC is quite a process, which needs to be telegraphed carefully to mapmakers with anticipation, the categories very much work on this favor, and that's what was done for the 4p map judge pickings, to telegraph to mapmakers that we would be doing things, so they, after many years of not doing so, would start producing +3p maps, the issue came from the natural clash of a system which was more or less meant for status quo into another to seek out outsiders, and this, per se "is kind of the point". TLMC15 was supposed from a macro perspective to not produce seismic changes because doing so would throw a wrench on balance and gameplay stability, but at the same time, it also, through the OP TLMC#15 itself set the strong expectations that it would bring about change and a breakage of inertia These from my perspective are the core of the conflict, but, I dont see people being dissapointed on how "non-nonstandard" the maps were as a bad thing, because again, there are bigger forces at stake here, DevTeam taking a step aside, and ESL with the community taking a step on carrying the weight, and that step forward needs to be stable. So I see the overall selection of more standard type maps as a net positive, where a net negative would have been increasing the risk of unbalance without an active devteam or a mature community-ESL-Blizz relationship to handle potential persistent map related problems. Now. Certainly, pro-players as influencers carry a huge amount of weight on both the collective mind of the public, and on the minds of tournament administrators, but it needs to be noted that TLMC is a process to get maps from the community into the ladder, and as Teoita mentioned, there have been different ways of setting up the TLMC and judging, and going forward we, on the mapmaking side of things, will be experimenting with a novel process to manage at least the early initial cut of +200 -> 40, 50 maps which can then be sampled by invited guests composed of casters, streamers, proplayers, tournament organizers, etc to create map pools out of. As an alternative to needing to parse ~180 maps at a time on a single week by a small amount of judges But to recap, this is a matter of the process, not of 100 or so "top people" holding the status quo. Now, I am certain that once the new process is in place, there will be riots from extremists whom dont want to play on anything other than a re-skinned version of Catalyst , but even they understand that the macro perspective and SC2 as E-Sports and Entertainment takes precedence, which is how the entire "thing" even began, as a Reality TV Show, as a spectacle I would like to double down on a lot of the difficulties facing map contest judges Kantu mentioned, as well as my own. I've never been a part of the 1v1 contest, but from experience I can tell you that every judge has pretty much an insurmountable task when it comes to pleasing every element of the player base. Just like you we want maps that are fun and interesting, but we also need to value stability above all (for 1v1 judges it's the fact that players livelihoods are determined by the maps that make it to ladder, for team judges it's that we need to weed out as much as the cheesiness as possible so people don't just get ling flood/phoenix or cannon rushed every game). But, as much as judging a team contest is difficult because of how cancerous the early game cheeses and late game comps are, the 1v1 judges are given a far more challenging task. They have to look at 100s of entries, most of which you simply have to dismiss at first site because you simply don't have the time to play games or do a really, really thorough examination on every map. People can whine about why the end products are often stale (I would prefer to say safe), but they aren't taking into account how difficult everything is, and how personal preference can alter judges scores significantly. I personally would never put the first three bases behind one choke, but maybe there's another judge that really values having one or two such maps in the pool. We don't have hundreds of judges to aggregate out a proper scoring, instead we're looking at a smile sample size where one person can completely skew what the end product is as far as rating. Then there's the fact that sometimes the judges are just wrong. Artic Dream and Efflorescence have no business being in the 2v2 pool, but given the limited time and resources available to judges the initial impression was that they would be great maps. Ultimately, it's sort of like this for pretty much every line of work. Judges cannot please everyone. Judges do not have enough time to give every map the review you think they deserve. Judges are not being compensated for their time. Judges cannot play test game and test game until they have hundreds of games worth of data and experience to go on. People especially need to give those involved in the organizational roles who have to communicate between Blizzard (or ESL) and judges, mapmakers and whoever else is involved in the process. The simple fact is with the resources available you're never going to get the perfect contest or the perfect judging. My final plea is this—please try to be understanding. Everyone involved in the contest wants to get the best possible maps on ladder, but it's simply an impossible task, especially when resources are less and less plentiful with each passing year. | ||
NonY
8748 Posts
for example, maybe a strategy is well suited for 5 or 6 of the current maps, but on 1 or 2 of the maps it doesn't work that well. for the next season, design the map pool to swap those numbers. i think the game balance right now is such that map pools can counter the most egregious common strats without matchup balance violating the "within 5% of 50/50" rule. if you target a strat that is often used by a race winning 53% right now and you nerf that strat via the map pool, that matchup is unlikely to dip below 45%, so it's fine. if you gather this data from pros before the mapmakers start making maps then you can give them guidelines on the features that the forthcoming map pool is designed to have. edit: i think having crazy maps for a proleague where a team works together to decide what player is going to get sent on that map (and therefore get to choose which race they're sending on the map), and then is informed ahead of time what player the opposing team is sending on that map, and as a group they brainstorm a strat, that's awesome. for a 1v1 tournament with bo5's and bo7's, especially if the map is being used for a tournament that happens all in one weekend, the zany maps are bad for the game. generally speaking the idea that "to break up stale tournament gameplay, we have to throw wildcards in there" is just low effort and dumb and ruins an otherwise good game. use targeted changes in the map pool as a whole to discourage players from playing the same strats on every map for season after season. if you dont know what changes those are, then ask a pro. | ||
|
Teoita
Italy12246 Posts
On July 20 2021 01:20 NonY wrote: edit: i think having crazy maps for a proleague where a team works together to decide what player is going to get sent on that map (and therefore get to choose which race they're sending on the map), and then is informed ahead of time what player the opposing team is sending on that map, and as a group they brainstorm a strat, that's awesome. for a 1v1 tournament with bo5's and bo7's, especially if the map is being used for a tournament that happens all in one weekend, the zany maps are bad for the game. Just wondering, what if we announced the map pool for each round of a tournament a few weeks in advance? E.g., in the ro8 the maps will be in order X, in the ro4 in order Y, and in the final in order Z? You could then throw in maybe one non-standard (whatever that means) map at the start of the series, but otherwise mitigate the effects of more experimental/potentially broken stuff. Even with weekend tournaments that may help players plan their way around weird/possibly annoying maps, and for e.g. GSL (which doesn't do this afaik?) it might definitely bring the format a bit closer to the hyper-preparation style of Proleague. | ||
|
Mizenhauer
United States1808 Posts
On July 20 2021 05:54 Teoita wrote: Yep I agree 100% with NonY Just wondering, what if we announced the map pool for each round of a tournament a few weeks in advance? E.g., in the ro8 the maps will be in order X, in the ro4 in order Y, and in the final in order Z? You could then throw in maybe one non-standard (whatever that means) map at the start of the series, but otherwise mitigate the effects of more experimental/potentially broken stuff. Even with weekend tournaments that may help players plan their way around weird/possibly annoying maps, and for e.g. GSL (which doesn't do this afaik?) it might definitely bring the format a bit closer to the hyper-preparation style of Proleague. In case you didn't know, players who participate in Code S finals pick the map order a day or two after the semifinals so they have about a week to prepare map specific builds. | ||
|
Teoita
Italy12246 Posts
| ||
|
NeuralNet88
22 Posts
On July 19 2021 22:36 Mizenhauer wrote: I would like to double down on a lot of the difficulties facing map contest judges Kantu mentioned, as well as my own. I've never been a part of the 1v1 contest, but from experience I can tell you that every judge has pretty much an insurmountable task when it comes to pleasing every element of the player base. Just like you we want maps that are fun and interesting, but we also need to value stability above all (for 1v1 judges it's the fact that players livelihoods are determined by the maps that make it to ladder, for team judges it's that we need to weed out as much as the cheesiness as possible so people don't just get ling flood/phoenix or cannon rushed every game). But, as much as judging a team contest is difficult because of how cancerous the early game cheeses and late game comps are, the 1v1 judges are given a far more challenging task. They have to look at 100s of entries, most of which you simply have to dismiss at first site because you simply don't have the time to play games or do a really, really thorough examination on every map. People can whine about why the end products are often stale (I would prefer to say safe), but they aren't taking into account how difficult everything is, and how personal preference can alter judges scores significantly. I personally would never put the first three bases behind one choke, but maybe there's another judge that really values having one or two such maps in the pool. We don't have hundreds of judges to aggregate out a proper scoring, instead we're looking at a smile sample size where one person can completely skew what the end product is as far as rating. Then there's the fact that sometimes the judges are just wrong. Artic Dream and Efflorescence have no business being in the 2v2 pool, but given the limited time and resources available to judges the initial impression was that they would be great maps. Ultimately, it's sort of like this for pretty much every line of work. Judges cannot please everyone. Judges do not have enough time to give every map the review you think they deserve. Judges are not being compensated for their time. Judges cannot play test game and test game until they have hundreds of games worth of data and experience to go on. People especially need to give those involved in the organizational roles who have to communicate between Blizzard (or ESL) and judges, mapmakers and whoever else is involved in the process. The simple fact is with the resources available you're never going to get the perfect contest or the perfect judging. My final plea is this—please try to be understanding. Everyone involved in the contest wants to get the best possible maps on ladder, but it's simply an impossible task, especially when resources are less and less plentiful with each passing year. Which is why there has been talk about improving the process to make sure the very best and most well designed maps make it to the ladder pool instead of the same boring "standard" stuff that, for the most part, we've been seeing for quite a while now. Instead of making excuses for the process, improve the process. Staying safe but having a stale product means less people stick around as opposed to when you take risks and try to make the product more interesting and engaging. | ||
|
NeuralNet88
22 Posts
I know Kantuva mentioned on the Pylon show how much he loved Agaton's maps for example. Yet his truly outstanding maps in Electron and Hardwire didn't make the cut over maps like Enchanted Isle and Sanguinite. Why is that? My guess is because the latter maps are much more "standard and safe" than Agaton's maps are. I think the game needs to start thinking about drifting away from what is considered "standard" and start thinking about maps that truly make the game more interesting to play. | ||
| ||
