EPTThe key here is that the AI trained with itself - AI vs AI in ~200 human years of training. That doesn't give it realistic perspective on what works best vs humans. If the AI was let to compete on ladder vs real humans, I bet it would focus exclusively on things like blink stalkers all the time - not just in some random instances - that give it inherent mechanical advantage no human can match.
StarCraft II: DeepMind Demonstration: Jan 24 - Page 29
| Forum Index > SC2 General |
|
figq
12519 Posts
The key here is that the AI trained with itself - AI vs AI in ~200 human years of training. That doesn't give it realistic perspective on what works best vs humans. If the AI was let to compete on ladder vs real humans, I bet it would focus exclusively on things like blink stalkers all the time - not just in some random instances - that give it inherent mechanical advantage no human can match. | ||
|
TaKeTV
Germany1199 Posts
On January 27 2019 21:25 figq wrote: To remind you that we've seen various different agents who trained together, not one entity. Some of them focused on blink stalkers and won by inhumanly good micro. Those seemed like easy victories. But really we've seen a great variety of approaches - aggressive, defensive, tactical and strategic. The key here is that the AI trained with itself - AI vs AI in ~200 human years of training. That doesn't give it realistic perspective on what works best vs humans. If the AI was let to compete on ladder vs real humans, I bet it would focus exclusively on things like blink stalkers all the time - not just in some random instances - that give it inherent mechanical advantage no human can match. On the one hand it obviously would be smart and the logical thing to do: exploit your own strength and abilities - aka unmatched mechanics. On the other hand that really wouldn't give a realistic approach of an Ai actually playing against a human. Humans need to physically move the mouse against friction, the distance etc. Also observation and reaction play a role as well. I find it interesting but what we saw is far away from actually an Ai playing / outplaying humans but rather a mechanically superior agent. And the biggest task will be learning to play the game with no knowledge of the map, actually having to fight for information and telling the difference between a fake, a human mistake etc. | ||
|
Polypoetes
20 Posts
On January 27 2019 21:25 figq wrote: If the AI was let to compete on ladder vs real humans, I bet it would focus exclusively on things like blink stalkers all the time - not just in some random instances - that give it inherent mechanical advantage no human can match. But it is not really possible to train the AI on the ladder. How do you get 200 years of game time worth of your bot playing vs humans on the ladder? Adjusting the weights of the NN based on how well it does vs humans is just really tricky because you will have a much smaller dataset. And the dataset could have all kinds of biases. And that even ignores the fact that people could be trolling. We have seen chat NNs say racist things because of people trolling. Secondly, even if you can train a NN to exploit mistakes humans make, maybe that is actually an inferior way to play. The NN will make deliberate mistakes 'knowing' that humans cannot exploit it. But another bot will. So you will just be muddying the way the NN plays with weakness. And third, things the NN does that seem like a mistake, once it gets so strong it beats humans, it is not clear we humans can recognize a mistake. Our criticism of what it does is only justified if we can show that we can exploit it as a weakness. Humans cut corners too. Maybe it knows it can cut corners. If the AI builds 7 observers vs Mana, maybe it does that because it is minimizing the risk to lose. And it judged the risk of losing by being outmacroed many magnitudes lower than being killed by DTs. So it just overproduced observers and then puts them all near it's biggest army. So while it seems like a mistake, and it probably is, humans stepping in and manually adjusting weights doesn't work because you have a very poor idea of what you are doing. | ||
|
Haukinger
Germany131 Posts
| ||
|
snakeeyez
United States1231 Posts
On January 27 2019 21:46 TaKeTV wrote: On the one hand it obviously would be smart and the logical thing to do: exploit your own strength and abilities - aka unmatched mechanics. On the other hand that really wouldn't give a realistic approach of an Ai actually playing against a human. Humans need to physically move the mouse against friction, the distance etc. Also observation and reaction play a role as well. I find it interesting but what we saw is far away from actually an Ai playing / outplaying humans but rather a mechanically superior agent. And the biggest task will be learning to play the game with no knowledge of the map, actually having to fight for information and telling the difference between a fake, a human mistake etc. I feel like it already showed some of this. It could see the entire map unlike a human, but it never had any different vision. It had to fight for and scout all of its vision just a person in all the games. It just didnt need the camera which is a big advantage. I think they still have a long ways to go though they need to play more matches with a limited camera, and there is a large variety of strats in this compared to chess. When it plays the best players its going to have a tough time if its limited in its micro to be more like a human. That is the point of this though overcoming these challenges. If it won with this demonstration easily then it was not much of a challenge to begin with. The fact it still has a ways to go is why they choose to try to win at starcraft 2. its probably one of the hardest games left to beat. The other was GO and they beat that which was amazing. | ||
|
Jasper_Ty
101 Posts
| ||
|
niteReloaded
Croatia5281 Posts
On January 27 2019 23:14 Haukinger wrote: It's funny how people deem a korean terran exploiting marines with superior mechanical skill the highest level of play but if an AI uses blink stalkers it's just stupid with high apm. It's not funny. Super, human micro requires very rare talent and huge amounts of practice. Super, AI micro requires coding it without APM limits. Strategy, metagame, balance are all built on the capacity and potential of individual units. Unit's utility is based on what a player can do with it. What a human player can do with a unit is vastly different from what a AI can do with it. Based on that, AI uses different tactics, different strategies, almost a different game. A human can't click 1500 in a minute, AlphaStar did. StarCraft is a realtime strategy game where speed is very important. They made a bot which has, at crucial times, played at least 2X faster than any human can. It's like building a sports playing bot which can run 2x faster. Suddenly, all tactics and strategies are affected and the game is almost not the same. --- DeepMind's only mistake is that they didn't put a proper peak-APM limit. Frankly, I would love to see games where AlphaStar is limited to 50 peak APM, then more games where the limit is 100 peak apm, then 150 etc. Would love to see the strategies and tactics used. | ||
|
Charoisaur
Germany15927 Posts
On January 27 2019 23:14 Haukinger wrote: It's funny how people deem a korean terran exploiting marines with superior mechanical skill the highest level of play but if an AI uses blink stalkers it's just stupid with high apm. it's not stupid, it's just worthless if we want to know how good the AI is strategically if it has unlimited apm because then it can win with anything. If the goal would be to just beat a human no matter how there would be no need for the Deepmind team to take this on, a regular micro bot would be enough. | ||
|
Xamo
Spain877 Posts
| ||
|
Polypoetes
20 Posts
I understand it is not interesting for you personally to improve your game. But we have had this discussion in the community ever since AlphaGo how long it would take for an AI to play properly and to beat top humans. I remember discussing with programmers who talked about how difficult of a problem it is to simulate combat before they make a move. Because that is what they tried to put into the BWAPI AIs. You see a game state, you see your units, you see the opponent units, you predict how the units will move and what they will attack, you calculate how much each side will lose, and then you know if this engagement is favorable. So when people say that they consider the AI microing like crazy 'trivial', I just have to laugh. Personally, I would like to see an unlimited APM AI. And I would love to see an AI play SC BW, because it seems to me that SC2 is way more straightforward and SC BW is way more on a knife edge and subtle. Maybe it is me being a former SC BW player, but I have this feeling that any play by an AI is way more exploitable in SC BW than in SC2. Honestly, this reminds me of the debate we had back when SC2 was announced and when we had this influx of AoE, Civ, and C&C players arguing that slowing down the game, or making the interface easier, would give the game a richer game of strategy. The game is not what you think it is. Imagine an AI figuring out how to play Civ3. It is just calculating which builds to build first under which circumstances. It is boring. There is one solution. You calculate what it is, and you just carry it out. Let's not forget that unlike Go or Chess, RTS games are convergent towards the end. | ||
|
Kafka777
361 Posts
| ||
|
fededevi
Italy45 Posts
A simple APM limit is not gonna do the trick, sometimes a player can have huge "useful" apm spikes so you will need a model that account for that and many other things. If you limit the AI too much, the AI will have to find workarounds to it's sub optimal micro by finding strategies that are sub-optimal for a player. If you do not limit the AI enough, the AI will come up with strategies that players will never be able to execute. Not to mention that each human player is different. Still it would be very interesting to see if the AI can come up with good/decent "very-low-micro" strategies. | ||
|
mierin
United States4943 Posts
| ||
|
fededevi
Italy45 Posts
But we do not know if that strategy is actually good for human players too because the AI is playing with different limitations. If over-probing was done by all agents on different APM limits then it would be an indication that it is a good idea to always build more probes. With "strategy" I mean the whole game strategy, you can't take out a single thing (like over-probing) and discard the rest, because everything is connected in a single game. For example the agent could be so good at winning with stalkers micro, that the only way to beat it is somehow to kill a lot of its probes. In this condition the agent would improve its chance of winning having a less than ideal number of probes. It would still have enough stalkers to win the engagements thanks to its god-like micro, but he would not lose to other kind of attacks to the mineral line. But if he was not so good at microing then it would have better chances by expanding its economy faster. This is just an example, I don't know if this is the case. | ||
|
Grumbels
Netherlands7031 Posts
This is a common “weakness” to AI’s: a complete inability to take the psychology of the opponent into consideration. Agents tend to converge to the same cautious, conservative style which shows maximum respect to the opponent. E.g. if you had to play a match versus an AI, it might be the case that if you just made blink stalkers it would vastly overestimate the strength of your army, because it doesn’t know you can’t actually micro blink stalkers at 1500 APM. And as a result it wouldn’t take any engagements. This is why chess engines have a “contempt” setting programmed into them, which forces them to make sub-optimal moves which nevertheless increase winning chances against weaker opponents. This is a must in any sort of tournament or league play in chess, since it avoids draws. I don’t know how you would program “contempt” into AlphaStar, other than training it versus agents with handicapped APM, such that it would develop more confidence in its micro, in order to better approach match conditions vs humans. But of course this doesn’t have any scientific value, only competitive utility. | ||
|
niteReloaded
Croatia5281 Posts
On January 29 2019 05:38 Grumbels wrote: An agent isn’t confident in its stalker micro, as it’s training vs bots, not humans. It won’t think it can win a four vs five stalker fight vs a human player because it is used to losing to other agents in such situations. This is a common “weakness” to AI’s: a complete inability to take the psychology of the opponent into consideration. Agents tend to converge to the same cautious, conservative style which shows maximum respect to the opponent. E.g. if you had to play a match versus an AI, it might be the case that if you just made blink stalkers it would vastly overestimate the strength of your army, because it doesn’t know you can’t actually micro blink stalkers at 1500 APM. And as a result it wouldn’t take any engagements. This is why chess engines have a “contempt” setting programmed into them, which forces them to make sub-optimal moves which nevertheless increase winning chances against weaker opponents. This is a must in any sort of tournament or league play in chess, since it avoids draws. I don’t know how you would program “contempt” into AlphaStar, other than training it versus agents with handicapped APM, such that it would develop more confidence in its micro, in order to better approach match conditions vs humans. But of course this doesn’t have any scientific value, only competitive utility. You release it on the ladder, with 1000s of instances playing vs humans 24/7. Machine learning and genetic algorithms (or whatnot) would cause bots to start taking some smaller sub optimal moves, just to gauge the reactions of the opponent. As the bots have played just other bots with same 'mechanical' capabilities, the ability to gauge opponent's mechanical skill wasn't necessary. | ||
|
Cyro
United Kingdom20285 Posts
You release it on the ladder, with 1000s of instances playing vs humans 24/7. There are not even that many humans on the ladder | ||
|
Deimos
Mexico134 Posts
| ||
|
Acrofales
Spain17977 Posts
On January 30 2019 09:11 Cyro wrote: There are not even that many humans on the ladder Especially not at GM level. Pretty sure it's already good enough that even knowing its weaknesses it'll still beat diamond players on mechanics alone. | ||
|
Plopus
Switzerland112 Posts
| ||
| ||
