EPTOn November 09 2019 19:29 heqat wrote:
I think the problem is that DeepMind tends to overhype their results. I watched the Pylon show live event with TLO and it was all about how great is AlphaStar at SC2, etc. There would be less criticizing if they would spend some time talking about the limitiation of their SC2 agents which many people noticed (see the beastyqt video for instance). Of course, in the context of AI research it is a great achievement (which is what matters at the end), but in the SC2 context they should communicate more on the fact that SC2 is not mastered at a pro level (which was the case for Go and Chess).
I think the problem is that DeepMind tends to overhype their results. I watched the Pylon show live event with TLO and it was all about how great is AlphaStar at SC2, etc. There would be less criticizing if they would spend some time talking about the limitiation of their SC2 agents which many people noticed (see the beastyqt video for instance). Of course, in the context of AI research it is a great achievement (which is what matters at the end), but in the SC2 context they should communicate more on the fact that SC2 is not mastered at a pro level (which was the case for Go and Chess).
I think that DeepMind overhypes their results, because in the AI community they seem innovative and a step forward. The problem is, they don't know much about StarCraft 2. The role (mostly) TLO took so far, is so very important. Analysing the gameplay and "behaviour" of AlphaStar can teach us so much about how the AI works and what its limits are.
That's too much for one person to handle and that's why I am asking for a consistent overview of all released AlphaStar replays by the community.
Certainly most striking is that in this state AlphaStar knows no proper reaction to situations it has not "experienced" before.
It needs a more basic concept of individual units etc, that it can't learn by a dataset consisting only of regular high-mmr replays. Also since important, but only occasionally occuring subtleties don't make it into AlphaStars "mind".
I get the feeling it is tuned a bit too much to care about a high average winning rate across matches, instead of getting the best out of each individual game.
