I don't even think AI need high APM. Human have hundreds APM because they have many ineffective actions, selecting units multiple times, giving the same order multiple times.
I think a good AI probably already good enough with 60 - 120 APM. Because all of their action are effective actions (120 APM means 2 effective action per second), assuming the AI decide which action to take in order of their priority.
On December 13 2017 11:54 Kuraku wrote: I don't even think AI need high APM. Human have hundreds APM because they have many ineffective actions, selecting units multiple times, giving the same order multiple times.
I think a good AI probably already good enough with 60 - 120 APM. Because all of their action are effective actions (120 APM means 2 effective action per second), assuming the AI decide which action to take in order of their priority.
There's no misclicking either, and the APM isn't constrained by human limitations (such as doing consecutive actions on the same area of the map). Even with a small APM an AI would have some advantages of its own.
But yeah with no APM cap things are ridiculously easy for the AI.
On December 13 2017 11:54 Kuraku wrote: I don't even think AI need high APM. Human have hundreds APM because they have many ineffective actions, selecting units multiple times, giving the same order multiple times.
I think a good AI probably already good enough with 60 - 120 APM. Because all of their action are effective actions (120 APM means 2 effective action per second), assuming the AI decide which action to take in order of their priority.
There's no misclicking either, and the APM isn't constrained by human limitations (such as doing consecutive actions on the same area of the map). Even with a small APM an AI would have some advantages of its own.
But yeah with no APM cap things are ridiculously easy for the AI.
I agree that a good AI will have the highest effective APM, but the thing is, at 2 actions per second, you can't defend a double drop and manage a push at the front. Assuming that an action is changing the camera position, you would have used up your two moves/second by switching the camera and dealing with a single drop let alone dealing with two other threats.
Keep in mind that while the AI is doing this, they also have to macro. So while that double drop and push at the front is going on, they slip on their macro and forget to make units.
Or how does an ai stutter step or use blink micro with 120 apm while macroing? I'm not denying the effectiveness of a computer that know the optimal move to do at any given moment, but in times of stress, peoples' apm spikes up into the 400+ apm, and that includes effective actions like macro as well as sometimes less effective actions like in micro.
On December 13 2017 11:54 Kuraku wrote: I don't even think AI need high APM. Human have hundreds APM because they have many ineffective actions, selecting units multiple times, giving the same order multiple times.
I think a good AI probably already good enough with 60 - 120 APM. Because all of their action are effective actions (120 APM means 2 effective action per second), assuming the AI decide which action to take in order of their priority.
There's no misclicking either, and the APM isn't constrained by human limitations (such as doing consecutive actions on the same area of the map). Even with a small APM an AI would have some advantages of its own.
But yeah with no APM cap things are ridiculously easy for the AI.
I agree that a good AI will have the highest effective APM, but the thing is, at 2 actions per second, you can't defend a double drop and manage a push at the front. Assuming that an action is changing the camera position, you would have used up your two moves/second by switching the camera and dealing with a single drop let alone dealing with two other threats.
Keep in mind that while the AI is doing this, they also have to macro. So while that double drop and push at the front is going on, they slip on their macro and forget to make units.
Or how does an ai stutter step or use blink micro with 120 apm while macroing? I'm not denying the effectiveness of a computer that know the optimal move to do at any given moment, but in times of stress, peoples' apm spikes up into the 400+ apm, and that includes effective actions like macro as well as sometimes less effective actions like in micro.
Yeah, but conversely at 300 apm an AI can probably make widow mines completely useless. No matter what sufficiently large number you choose the AI will be able to do inhuman stuff, so limiting it to lower than a pro human's EAPM is probably fairer, though in truth there is no setup that will make everyone happy. Maybe 120 is too low though.
On December 13 2017 11:59 ZigguratOfUr wrote: There's no misclicking either, and the APM isn't constrained by human limitations (such as doing consecutive actions on the same area of the map). Even with a small APM an AI would have some advantages of its own.
That's probably why an apm cap is too simple. You need some kind of limiter for consecutive actions/mouse speed.
On December 13 2017 11:54 Kuraku wrote: I don't even think AI need high APM. Human have hundreds APM because they have many ineffective actions, selecting units multiple times, giving the same order multiple times.
I think a good AI probably already good enough with 60 - 120 APM. Because all of their action are effective actions (120 APM means 2 effective action per second), assuming the AI decide which action to take in order of their priority.
There's no misclicking either, and the APM isn't constrained by human limitations (such as doing consecutive actions on the same area of the map). Even with a small APM an AI would have some advantages of its own.
But yeah with no APM cap things are ridiculously easy for the AI.
I agree that a good AI will have the highest effective APM, but the thing is, at 2 actions per second, you can't defend a double drop and manage a push at the front. Assuming that an action is changing the camera position, you would have used up your two moves/second by switching the camera and dealing with a single drop let alone dealing with two other threats.
Keep in mind that while the AI is doing this, they also have to macro. So while that double drop and push at the front is going on, they slip on their macro and forget to make units.
Or how does an ai stutter step or use blink micro with 120 apm while macroing? I'm not denying the effectiveness of a computer that know the optimal move to do at any given moment, but in times of stress, peoples' apm spikes up into the 400+ apm, and that includes effective actions like macro as well as sometimes less effective actions like in micro.
Yeah, but conversely at 300 apm an AI can probably make widow mines completely useless. No matter what sufficiently large number you choose the AI will be able to do inhuman stuff, so limiting it to lower than a pro human's EAPM is probably fairer, though in truth there is no setup that will make everyone happy. Maybe 120 is too low though.
At 300 basically EPM, they can do a lot of things better than humans. I think 300 as an upper limit is probably too strong in that case. Having a limit at all, however, will make it so that in the most action-intensive moments, the AI will be at an intrinsic disadvantage because it cannot go higher, and even if a human has ineffective apm, I'm guessing that there are moments where their EPM has been higher than 300.
APM limits are kind of a trivial point right now since they haven't even gotten the AI to perform the proper actions. It could have infinite apm right now and it wouldn't make (much) of a difference.
120 is probably just the working limit they've set for now, I'm sure they can adjust it if necessary, after the AI is sufficiently trained so as to have an idea of what to actually do.
Perhaps a good idea would be to have APM/EAPM classes or ratings, like at 50, 60, 70, 80 and so on. The problem is matching the class to a player of the same ability or merely ramp up the AI class after every loss?.
Either way it's difficultbt come up with a fair playingfield for AI and player alike.
I think limiting the apm is overrated when even with unlimited apm the bots (for me they are bots until I see "smartness") are not able to beat even medium skilled humans. I played a little with the api and the bot had 148401 apm. And it is true, on a open field with lings and a few banelings not one baneling comes even close... but then there is a choke or ramp up etc and boom all marines are gone. And good human players are fast to adapt to something like this. Another example slow lings against a reaper. Just kiting backwards is the closest way into a wall/corner. And there even 150k apm don't help you (actually too fast move commands cancel the cliff jump, so they even harm you). Having situational awareness is whats needed for an AI. And this is not related to apm. So I would make two milestones out of it. But I know deepmind is limiting themselves to 300 apm(?).
Also keep in mind, the bots don't have the same universal access to the data as you have in the editor. For example there is no "projectile unit" where you simply blink backwards if it is near. You also don't have access to target unit or weapon cool down of enemy units. So many of the fancy micro bot videos are not easy transferable.
An AI just beat some of the world's best Dota players after only 2 weeks of training and you guys think it will take 5-10 years or more to develop one which can do the same in starcraft? Come on. Sure it's more strategically challenging, but it can also multitask perfectly. I was honestly surprised to find it hadn't already surpassed humans.
The C++ documentation looks wrong. It currently isn't being output for enemy units.
We've gotten feedback from a bunch of people that exposing this would be useful. We omitted it since it is in the grey area of what information a human player would be able to see. However you can roughly infer it from unit's animation.
If you think this is important, can you add it as an issue to the api GitHub page?
I don't know where it was said that Deepmind was limiting its APM. Does anyone have a link? Anyway, I had some new thoughts on this.
1. It might be hard to gauge the strength of a StarCraft AI. AlphaGo could test its strength by playing versus bots of reasonable strength, but these don't exist for StarCraft. The standard SC2 AI is just very easy to exploit and it can not learn from its mistakes or adapt when it sees that something is not working. That is very different from Go or Chess engines, which are vastly more sophisticated and are capable of reasonable decisions in any sort of situation. So given that it might be difficult to estimate its strength, having unlimited APM provides a safety feature in case they ever publicly challenge a human player
On the other hand, there is an acceptable way for an AI to test itself by playing on Ladder. AlphaGo, at one point, was allowed to play online for a week, where it defeated top pros by 60-0. If there was an opponent with odd decisions and inhuman levels of APM then a hypothetical AlphaSC could never enjoy anonymity, no matter the outcome of the games. If it had limited APM then it'd enjoy more security.
2. There is no real precedent for limiting APM. Existing AIs don't do this, and while chess engines might limit the hardware available, they do not limit themselves their calculation ability, which has historically been their main strength. If there is any question of whether machines can beat humans in a game of StarCraft, then neutering the engine so that their main advantage dissipates purely to prove a point about its superiority is at least dubious.
3. Deepmind has access to vastly powerful hardware, likely some of the best in the world. They need to reconfigure the learning process to efficiently use this hardware. So far they have tackled very slow-paced arcade games and board games with arbitrary time restrictions. StarCraft is a fast-paced real-time game, which requires a lot of computing power to simulate. Furthermore, it involves a lot of low level decisions which are fairly obvious and which consist of a sequence of steps to execute. If you try to learn chess it might make sense to very quickly play a lot of chess games, at a rate of, say, one per second. Your decisions might suffer, because you have only milliseconds per move to think, but that is okay because you compensate with volume of games. However, for SC2 it might be true that you can learn more quickly with less decisions and lower APM. But unlike for chess and go, the gameplay vastly differs depending on APM because there is a real-time component. A good move is still a good move in chess, even with different time controls. But if I only get to make 1 action per second, then I have to be very careful about my prioritizations in SC2.
4. Deepmind probably doesn't care as much if there are some whispers about how any showmatch is unfair. They had an unfair showmatch for chess, and people didn't care. If an AI can quickly crush any human player using some obscure marine rush, that will still provide headlines, even if from an AI perspective it is not as impressive as strategically outthinking humans. Deepmind probably keeps PR separate from its internal assessment of the quality of its AI. It's not like they will be done with SC2 the moment they beat a human player, because SC2 is so rich and complex.
On December 13 2017 17:32 Archiatrus wrote: Also keep in mind, the bots don't have the same universal access to the data as you have in the editor. For example there is no "projectile unit" where you simply blink backwards if it is near. You also don't have access to target unit or weapon cool down of enemy units. So many of the fancy micro bot videos are not easy transferable.
That is true, but they show what is possible in theory. You could do what the Zerg is doing here:+ Show Spoiler +
If you are just fast enough at reading which tank is shooting at which ling and then splitting all others if you have enough APM.
On December 13 2017 20:06 Excludos wrote: An AI just beat some of the world's best Dota players after only 2 weeks of training and you guys think it will take 5-10 years or more to develop one which can do the same in starcraft? Come on. Sure it's more strategically challenging, but it can also multitask perfectly. I was honestly surprised to find it hadn't already surpassed humans.
The DotA result did not seem that significant to me. The AI was eventually beaten by pro players, and as far as I know it was just 1v1 mid-only, which is not a serious category. Rote execution of last hit and deny mechanics with one single hero and a limited set of items is obviously something an AI would excel at, but this does not prove a serious ability to pick teams and evaluate item and strategy choices, nor does it prove that the AI(s) can coordinate effectively as a team, nor does it prove that the AI has some level of resilience vs exploitative and off-beat strategies designed to target its weaknesses.
Chess engines were unbeatable tactically long before they ever posed a serious threat to human players in a match.
Humans typically make between 30 and 300 actions per minute (APM), roughly increasing with player skill, with professional players often spiking above 500 APM. In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players.
Also about the chess showmatch, it was reputed that stockfish running for 8+ hours on some people's computers (laptops I suppose) was unable to find some of the moves alphazero played. I think I recall the presenter saying that once you showed stockfish the move however, then it liked it.