| Forum Index > SC2 General |
|
Salteador Neo
Andorra5591 Posts
Air blocker in Ghettysburg means Terran can't float to the islands right? Makes it more fair and really awesome heh, I already liked it before knowing those were air blockers. Big fan of Frozen Zone and mad props to Avex for having 3 maps in the finalists. Basetradetv tourney hype! edit: The maps in the new map type are so bonkers, gotta love them too. | ||
monk
United States8477 Posts
On May 22 2016 04:42 Xenotolerance wrote: Not really, there's a salient difference between giving each map a 10 point rating then take the average from all judges, and having to decide on rankings 1-23 and filter the outcome, especially in how you can weigh the rankings. This is something we considered/are considering for future TLMCs, but it honestly makes very little difference. For this contest, it would have literally changed only one result. I'd want to see something like, Every judge's first choice is a finalist, every judge's second choice is a finalist unless two other judges veto, every judge's third choice is a finalist unless one other judge vetos. Fill in what's left by aggregate rankings. This way the judges can exercise their judgment more directly, and you'd get fewer Indifferent results. No, I'm confident this would give much worse results. | ||
|
Daswollvieh
5553 Posts
| ||
|
Xenotolerance
United States464 Posts
On May 22 2016 05:43 monk wrote: This is something we considered/are considering for future TLMCs, but it honestly makes very little difference. For this contest, it would have literally changed only one result. How can you know that without actually having those rankings? No, I'm confident this would give much worse results. Worse how? Please explain What TLMC has been doing so far has consistently produced underwhelming sets of finalists, with great but apparently unworthy maps totally overlooked in favor of mediocre maps that get ignored entirely after they place, so get a new system. Get rid of averaged point ratings, do anything else | ||
|
TelecoM
United States10834 Posts
| ||
|
NewSunshine
United States5938 Posts
If 1 judge had an abberant opinion, and really liked a map that nobody else liked, that system would allow what most people consider a bad map to instantly make finals. I don't need to go further, because that's already enough reason to abandon such a system. It would elect maps in a way that ignores discussion, where the current system more or less averages out how the judges feel. Just because the judges picked a few maps you didn't like so much doesn't invalidate their process. | ||
|
Xenotolerance
United States464 Posts
In the model I described, there would be five finalists insta-picked. They'd already have been vetted by three or more rounds of cuts and discussion, so it definitely doesn't ignore discussion. And in any event, the model can easily adapt, and move the vetoes up a step, such that the first choice can be stopped if two or three or w/e of the other judges say No. That way it can avoid the pitfall outlined in your first counterargument. You can also trust the judges to choose wisely, which they did for the most part here it seems, but when averaged, the judge's choices get muddled, and the cream fails to rise to the top. Plus, even if weak finalists don't invalidate the process, they definitely make obvious a need to reform it. Also... This whole discussion gets 100% better if the judges release their data. publish the list, the ratings, the whole nine yards please really though why haven't you already? | ||
|
ZigguratOfUr
Iraq16955 Posts
On May 22 2016 08:48 Xenotolerance wrote: Well I'm suggesting this process for the shortlist, so hopefully they've already filtered out the bad maps. In the model I described, there would be five finalists insta-picked. They'd already have been vetted by three or more rounds of cuts and discussion, so it definitely doesn't ignore discussion. And in any event, the model can easily adapt, and move the vetoes up a step, such that the first choice can be stopped if two or three or w/e of the other judges say No. That way it can avoid the pitfall outlined in your first counterargument. You can also trust the judges to choose wisely, which they did for the most part here it seems, but when averaged, the judge's choices get muddled, and the cream fails to rise to the top. Plus, even if weak finalists don't invalidate the process, they definitely make obvious a need to reform it. Also... This whole discussion gets 100% better if the judges release their data. publish the list, the ratings, the whole nine yards please really though why haven't you already? The only thing your system does is promote maps that one judge thinks is "excellent" and the other judges think is "fair", over maps that the judges overall think are "good". I'm not sure why you would want that. It adds even more variance to the final results than the current system. Most systems try to control for outlying judges, and your system empowers them. | ||
|
NewSunshine
United States5938 Posts
On May 22 2016 08:48 Xenotolerance wrote: You can also trust the judges to choose wisely, which they did for the most part here it seems, but when averaged, the judge's choices get muddled, and the cream fails to rise to the top. Ok, then what maps do you consider 'cream that failed to rise to the top'? Something to keep in mind is that categories were at play here, and while the Macro category was pretty stacked, the others were less so, Gold in particular was a weak one, that had not-so-great entries. On May 22 2016 08:48 Xenotolerance wrote: Plus, even if weak finalists don't invalidate the process, they definitely make obvious a need to reform it. Just because it's a map you don't like doesn't make it a weak finalist. I'm not in love with every map on the list, but I recognize that the judges only had so much to choose from, and they picked the best of what was in front of them. Besides, I thought they picked much better maps overall this time. I don't seem to be alone in this thinking either. What exactly is your problem? That your map's not on the list? On May 22 2016 08:48 Xenotolerance wrote: This whole discussion gets 100% better if the judges release their data. It really doesn't. There's not this magical wealth of data that will somehow break things wide-open and allow you to prove people wrong. In fact, there should never be anything in the background data that comes as a surprise, because the published results are merely a natural conclusion drawn from this data. Seeing it never helps as much as you want it to. | ||
|
Xenotolerance
United States464 Posts
the data will show whether my criticism is accurate or not. right now it's speculation, and numbers will confirm or deny to at least some concrete extent. I'm not interested in proving anybody wrong, I'm interested in getting TLMC right You're certainly not alone in thinking the pool is generally better, as I also think it is generally better. At the same time it still has its weaknesses, and I'm not alone in that idea either. good but needs improvement is an accurate way to put it imo anyway yeah I'm salty about my map not being a finalist, I made that clear in the other thread. it doesn't make me wrong On May 22 2016 09:00 ZigguratOfUr wrote: The only thing your system does is promote maps that one judge thinks is "excellent" and the other judges think is "fair", over maps that the judges overall think are "good". I'm not sure why you would want that. It adds even more variance to the final results than the current system. Most systems try to control for outlying judges, and your system empowers them. Right, that's the idea it should be less like committee democracy which tends toward compromise and more like curating an art gallery which tends toward bold statements Any map that a single judge thinks is the #1 best in the pool should be a finalist. This should be the overall tendency at the final stage compromise and averaging make for less-good maps passing in front of more-good maps, and I really can't believe any one of you out there actually prefers it that way. like, I get thinking my proposal wouldn't work, but thinking like 'ya it's worth the moderation if 5/15 finalists don't deserve to be there' I can't abide. I can't even | ||
|
ZigguratOfUr
Iraq16955 Posts
On May 22 2016 09:11 Xenotolerance wrote: Right, that's the idea it should be less like committee democracy which tends toward compromise and more like curating an art gallery which tends toward bold statements Any map that a single judge thinks is the #1 best in the pool should be a finalist. This should be the overall tendency at the final stage compromise and averaging make for less-good maps passing in front of more-good maps, and I really can't believe any one of you out there actually prefers it that way. like, I get thinking my proposal wouldn't work, but thinking like 'ya it's worth the moderation if 5/15 finalists don't deserve to be there' I can't abide. I can't even The likelihood of a single judge spotting a gem that the others didn't, is much less than that of any single judge mis-evaluating a map. Maybe moderation will result in 5/15 unworthy finalist, but your strategy of leaning towards the whims of any single judge will result in 10/15 unworthy ones. Consider the judging system of competitions. They usually have a panel of 5 judges, and they average the scores of the three with the middling scores. | ||
|
Xenotolerance
United States464 Posts
if you think the judges can't pick worthy maps, then you are outlining a different problem | ||
|
ZigguratOfUr
Iraq16955 Posts
On May 22 2016 09:27 Xenotolerance wrote: this ain't gymnastics if you think the judges can't pick worthy maps, then you are outlining a different problem Hardly. But there's some subjectivity in map evaluation and the current system controls for that better than yours. | ||
|
NewSunshine
United States5938 Posts
On May 22 2016 09:27 Xenotolerance wrote: this ain't gymnastics And what about judging SC2 maps is inherently different from judging in another context? And let me ask you again, in this contest, what do you think is an example of a mediocre map being chosen over a great one? This is more or less the entire basis of your... let's call it an argument. | ||
|
Xenotolerance
United States464 Posts
Eris got picked over Uvantak's three other maps Frozen Zone got picked over those same maps, plus say, Frostbite Gojira got picked for a category it wasn't submitted into (which, btw, makes me question WTH is up with that), over Dead Winter Reigns, and over any number of other maps in the other categories whose gold bases are used in a cool fashion Namaste got picked... at all. sorry avex I love you but it's not your best work New Gettysburg got picked in spite of glaring issues that I'm surprised Jacky put in the map. maybe the judges are banking on the iteration phase, but if that's the case, why wasn't a similar consideration made for other maps? so that's my personal take on it. what I want the data for is to parse whether these results are because of averaging or if the judges just picked them. it's the difference between just disagreeing with the finalists and seeing a reduction in quality due to process. one of them is worth fighting over, one of them isn't | ||
|
Xenotolerance
United States464 Posts
That's pretty low. don't go there. it's downright petty and mean, and we're above that, or we should be There's a pretty immense gulf between judging something like The Academy Awards, judging American Idol, judging Olympic trampoline, judging a supreme court case, and judging TLMC. You can't just slap a rating on something and call it a day if you're an art critic, and that's a core idea here. This isn't science, it's art*. It's not technical execution of specific concepts, it's creativity. It must be judged accordingly, and points will not satisfy but again to be clear, I'm thinking about reducing the shortlist. I do of course understand the technical elements and the importance thereof. I think that should be used for the first couple rounds, and deemphasized at the end Ziggurat - of course it's subjective. We shouldn't control for subjectivity, we should harness it. * I guess more accurately it's design, argument still holds | ||
|
Xenotolerance
United States464 Posts
| ||
|
NewSunshine
United States5938 Posts
On May 22 2016 09:41 Xenotolerance wrote: oh damn sunshine I didn't see your last comment before I wrote that. That's pretty low. don't go there. it's downright petty and mean, and we're above that, or we should be There's a pretty immense gulf between judging something like The Academy Awards, judging American Idol, judging Olympic trampoline, judging a supreme court case, and judging TLMC. You can't just slap a rating on something and call it a day if you're an art critic, and that's a core idea here. This isn't science, it's art*. It's not technical execution of specific concepts, it's creativity. It must be judged accordingly, and points will not satisfy but again to be clear, I'm thinking about reducing the shortlist. I do of course understand the technical elements and the importance thereof. I think that should be used for the first couple rounds, and deemphasized at the end Ziggurat - of course it's subjective. We shouldn't control for subjectivity, we should harness it. * I guess more accurately it's design, argument still holds Actually, the only thing that's any different would be a supreme court case, since proceedings are steeped in established laws and precedents, even then it's open to the judge's discretion where there is no precedent. There's a layer of subjectivity in all manner of judgement, that's why you have judges in the first place. | ||
|
ZigguratOfUr
Iraq16955 Posts
As for your concerns about the maps, that's a difference in opinion between you and the judges, not a concern with the system. | ||
|
Xenotolerance
United States464 Posts
I meant that for sunshine, but it applies to both of you | ||
| ||
GSL
2026 GSTL: Main Stage Day 2
[ Submit Event ] |
|
|
|
PiGosaur Cup
Afreeca Starleague
Mini vs Calm
Queen vs EffOrt
Replay Cast
The PondCast
Replay Cast
Escore
IntoTheTV X SOOP
Korean StarCraft League
GSL
Replay Cast
[ Show More ] |
|
|
EPT
