Humans typically make between 30 and 300 actions per minute (APM), roughly increasing with player skill, with professional players often spiking above 500 APM. In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players.
Also about the chess showmatch, it was reputed that stockfish running for 8+ hours on some people's computers (laptops I suppose) was unable to find some of the moves alphazero played. I think I recall the presenter saying that once you showed stockfish the move however, then it liked it.
This is all I know about it, I actually only heard about this a couple days ago and gleaned some information from that article. My assumption was that the Dota AI was 1v1 mid using a Shadowfiend and preset item choices. As far as I know, while the AI was self-learning, it was also given some specific subgoals such as that last hits and denies are good and so on. So it is not a completely general approach.
And yeah, I was just reading the paper as well looking for the APM limit. I guess I should read the information available before speculating. In any case, 180(E)APM seems reasonable because it is slightly above human capabilities while still keeping to some sort of limit so that learning can happen efficiently.
To be honest, at 180APM you can probably still easily devise unbeatable marine rushes if your execution is good enough.
Humans typically make between 30 and 300 actions per minute (APM), roughly increasing with player skill, with professional players often spiking above 500 APM. In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players.
Also about the chess showmatch, it was reputed that stockfish running for 8+ hours on some people's computers (laptops I suppose) was unable to find some of the moves alphazero played. I think I recall the presenter saying that once you showed stockfish the move however, then it liked it.
Whoa! Link to the dota AI being beaten ?
Yeah it didnt even consider some of the moves alphazero made. After A0 made the move Stockfish agreed that it was a good move
On December 13 2017 20:06 Excludos wrote: An AI just beat some of the world's best Dota players after only 2 weeks of training and you guys think it will take 5-10 years or more to develop one which can do the same in starcraft? Come on. Sure it's more strategically challenging, but it can also multitask perfectly. I was honestly surprised to find it hadn't already surpassed humans.
The DotA result did not seem that significant to me. The AI was eventually beaten by pro players, and as far as I know it was just 1v1 mid-only, which is not a serious category. Rote execution of last hit and deny mechanics with one single hero and a limited set of items is obviously something an AI would excel at, but this does not prove a serious ability to pick teams and evaluate item and strategy choices, nor does it prove that the AI(s) can coordinate effectively as a team, nor does it prove that the AI has some level of resilience vs exploitative and off-beat strategies designed to target its weaknesses.
Chess engines were unbeatable tactically long before they ever posed a serious threat to human players in a match.
Yes, it was beaten, and then it grew stronger. That's what training AI means.
But the significant part isn't that it just beat some players. The significant part is that it trained against itself and got good enough to beat top players within 2 weeks. During this time, all on its own, it learned to lasthit both enemy and friendly units, it learned what items to buy and when, it learned to make use of the donkey with both ferrying bottles and items, it learned to block friendly creeps, glyphs, dodging projectiles and a whole lot more.
A lot of these things are easily transferable to a game like starcraft. A good AI should easily be able to learn optimal build orders, multitasking armies with drops, perfect unit spread, map control, expansion, upgrades, etc. The one thing it might get trouble with is it's pitted against another race instead of a mirror. One of the sides might develop a strategy that the other simply isn't able to break out of, and the prior will simply stop developing because it keeps winning with this subpar strategy. But in a mirror, like I said, I'm surprised we haven't seen someone implement an AI already.
On December 13 2017 17:32 Archiatrus wrote: Also keep in mind, the bots don't have the same universal access to the data as you have in the editor. For example there is no "projectile unit" where you simply blink backwards if it is near. You also don't have access to target unit or weapon cool down of enemy units. So many of the fancy micro bot videos are not easy transferable.
That is true, but they show what is possible in theory. You could do what the Zerg is doing here:+ Show Spoiler +
If you are just fast enough at reading which tank is shooting at which ling and then splitting all others if you have enough APM.
I really hope they don't expose this in the api. Because that really feels like cheating. All the data you read is exact. As human you just assume what unit is targeted before the first shot. Building a heuristic to estimate the target is ok. But actually just knowing...
I know about the apm limit from the discord channel for the SC2 AI Ladder. There someone from Deepmind said that it is at the moment at 180. Constant number of actions per second.
On December 13 2017 17:32 Archiatrus wrote: Also keep in mind, the bots don't have the same universal access to the data as you have in the editor. For example there is no "projectile unit" where you simply blink backwards if it is near. You also don't have access to target unit or weapon cool down of enemy units. So many of the fancy micro bot videos are not easy transferable.
That is true, but they show what is possible in theory. You could do what the Zerg is doing here:+ Show Spoiler +
If you are just fast enough at reading which tank is shooting at which ling and then splitting all others if you have enough APM.
I really hope they don't expose this in the api. Because that really feels like cheating. All the data you read is exact. As human you just assume what unit is targeted before the first shot. Building a heuristic to estimate the target is ok. But actually just knowing...
I know about the apm limit from the discord channel for the SC2 AI Ladder. There someone from Deepmind said that it is at the moment at 180. Constant number of actions per second.
You have to remember that the API (that has been developped by Blizz' with the help of Deepmind), and Deepmind's development of an AI are two different things. Having this information exposed in the API will not mean AlphaSC is going to use it.
If I remember correctly, Deepmind always mentionned that the only information that their AI was going to use was the "raw input", which means the pixels from the screen, just like a human player would. That's how they worked on their "little" Atari project, and that's how they told people they were approaching SC2 as well.
Humans typically make between 30 and 300 actions per minute (APM), roughly increasing with player skill, with professional players often spiking above 500 APM. In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players.
Also about the chess showmatch, it was reputed that stockfish running for 8+ hours on some people's computers (laptops I suppose) was unable to find some of the moves alphazero played. I think I recall the presenter saying that once you showed stockfish the move however, then it liked it.
This is all I know about it, I actually only heard about this a couple days ago and gleaned some information from that article. My assumption was that the Dota AI was 1v1 mid using a Shadowfiend and preset item choices. As far as I know, while the AI was self-learning, it was also given some specific subgoals such as that last hits and denies are good and so on. So it is not a completely general approach.
And yeah, I was just reading the paper as well looking for the APM limit. I guess I should read the information available before speculating. In any case, 180(E)APM seems reasonable because it is slightly above human capabilities while still keeping to some sort of limit so that learning can happen efficiently.
To be honest, at 180APM you can probably still easily devise unbeatable marine rushes if your execution is good enough.
To give some frame of reference, most pro players in BW had ~180eapm, Flash and Jaedong had around 200-220 eapm at their peaks.
Humans typically make between 30 and 300 actions per minute (APM), roughly increasing with player skill, with professional players often spiking above 500 APM. In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players.
Also about the chess showmatch, it was reputed that stockfish running for 8+ hours on some people's computers (laptops I suppose) was unable to find some of the moves alphazero played. I think I recall the presenter saying that once you showed stockfish the move however, then it liked it.
This is all I know about it, I actually only heard about this a couple days ago and gleaned some information from that article. My assumption was that the Dota AI was 1v1 mid using a Shadowfiend and preset item choices. As far as I know, while the AI was self-learning, it was also given some specific subgoals such as that last hits and denies are good and so on. So it is not a completely general approach.
And yeah, I was just reading the paper as well looking for the APM limit. I guess I should read the information available before speculating. In any case, 180(E)APM seems reasonable because it is slightly above human capabilities while still keeping to some sort of limit so that learning can happen efficiently.
To be honest, at 180APM you can probably still easily devise unbeatable marine rushes if your execution is good enough.
To give some frame of reference, most pro players in BW had ~180eapm, Flash and Jaedong had around 200-220 eapm at their peaks.
Honestly, eapm as a metric does not really work for comparing an AI and a human. Because Flash is not carefully considering every action to find the correct one, so although a click by Flash might be distinguishable from pure spam, it is probably not as purely effective as an AI who can e.g. develop an internal sense of timing and only do some macro action the moment it is necessary. And I am sure that even with 60 apm you can do unnaturally good marine micro if you are precise.
Personally I would estimate about 100 apm for an AI is equivalent to 200 effective apm for a pro. If Deepmind uses 180apm, that puts it at around the maximum of potential human ability, or maybe a little beyond that.
Actually, I wonder if a selection counts as an action, I don’t think it does for Deepmind.
On December 13 2017 17:32 Archiatrus wrote: Also keep in mind, the bots don't have the same universal access to the data as you have in the editor. For example there is no "projectile unit" where you simply blink backwards if it is near. You also don't have access to target unit or weapon cool down of enemy units. So many of the fancy micro bot videos are not easy transferable.
That is true, but they show what is possible in theory. You could do what the Zerg is doing here:+ Show Spoiler +
If you are just fast enough at reading which tank is shooting at which ling and then splitting all others if you have enough APM.
I really hope they don't expose this in the api. Because that really feels like cheating. All the data you read is exact. As human you just assume what unit is targeted before the first shot. Building a heuristic to estimate the target is ok. But actually just knowing...
I know about the apm limit from the discord channel for the SC2 AI Ladder. There someone from Deepmind said that it is at the moment at 180. Constant number of actions per second.
You have to remember that the API (that has been developped by Blizz' with the help of Deepmind), and Deepmind's development of an AI are two different things. Having this information exposed in the API will not mean AlphaSC is going to use it.
If I remember correctly, Deepmind always mentionned that the only information that their AI was going to use was the "raw input", which means the pixels from the screen, just like a human player would. That's how they worked on their "little" Atari project, and that's how they told people they were approaching SC2 as well.
But deepmind saying they don't use it, does not mean others wont. I WILL use it in my bot once it is exposed. Does not mean I like it.
On December 14 2017 05:54 sc-darkness wrote: Can't AI win vs people just with high APM? Each unit could be micro managed individually just like the usual bots.
In the paper it says that Deepmind can execute one command per 8 frames, which translates to 180APM. As far as I understand, that is an arbitrary choice they made to improve the efficiency of the learning process, and is not inherent to the environment. That is to say, they could change the AI to have much higher APM, but they don't think it improves the learning process. And while they might limit themselves to 180APM even in real games, other AIs might not be so generous.
The built-in AI actually "cheats" by having access to the SC2 API, which allows for the superhuman actions that you can see in the automaton videos. Deepmind afaik seeks to somewhat mimic the human interface by having to interpret visual information and selecting units. However, the visual information is preprocessed to be more understandable.
StarCraft II also has a raw API, which is similar to the Broodwar API (BWAPI [1]). In this case, the observations are a list of all visible units on the map along with the properties (unit type, owner, coordinates, health, etc.), but without any visual component. Fog-of-war still exists, but there is no camera, so you can see all visible units simultaneously. This is a simpler and more precise representation, but it does not correspond to how humans perceive the game. For the purposes of comparing against humans this is considered “cheating” since it offers significant additional information. Using the raw API, actions control units or groups of units individually by a unit identifier. There is no need to select individuals or groups of units before issuing actions. This allows much more precise actions than the human interface allows, and thus yields the possibility of super-human behaviour via this API. Although we have not used the raw API for machine learning research, it is included as part of the release in order to support other use cases (e.g. scripted agents and visualisation) which the community may find useful.
This is what the AI sees, it has to select units and give them commands. I don't know if it has to use selection squares and such though.
Sorry for the multiple posts, but I found a recent article for Brood War AI research, which is also an active area of research. In fact, I wonder why Deepmind would even be interested in SC2 given that BW has a more active AI scene. For instance, Facebook targeted BW rather than SC2.
And on an apocalyptic note, in case you're wondering why google and facebook are interested in AI research, and why they don't actually care about the games themselves:
A Microsoft research paper on machine learning this year said that improving predictions of when a user will click on an ad by just 0.1 percent would yield hundreds of millions of dollars in new revenue.
. That's the end goal, enslaving humans to advertisements using machine learning.
These were the results for the recent BW AI competition. Note that the AI created by a hobbyist that just rushes you every game won the competition, and four out of the first six winners had really short average game times. Also, afaik, in 10 years of AI research, there hasn't been a single one that could compete on any level with a pro.
In the other article someone said they did not expect an AI to be able to beat a human player in the next five years. So honestly, if you're only casually interested in this I would go to sleep and wake up in two years before checking if there was any significant progress. SC2 is just vastly more complex than Chess or Go, and it's not even clear if a single AI based on a general learning algorithm is capable of learning it to the point of posing any sort of challenge to a pro player.
On December 14 2017 05:54 sc-darkness wrote: Can't AI win vs people just with high APM? Each unit could be micro managed individually just like the usual bots.
The goal isnt to win by straight up mechanics, but win by decision making. Hence why I know Deepmind, APM is throttled to near-human levels.
I think there are 3 issues for DeepMind in Starcraft as opposed to static games like chess/Go.
Too many degrees of freedom for movement. DeepMind learns by putting in random movements. Dumb moves in chess and Go are numerous, but dumb moves in SC2 somewhat dwarf that. The learning period will need to be more than 4 hours, more like thousands of hours.
Too many degrees of freedom with unit and resource interaction. Not just 2 bishops, but N bishops, and N pawns, and when and where should they be created? Will the computer be able to manipulate the resource balance like it can through position sense and material count for the board games?
Imperfect information and prediction. Predicting your opponent is somewhat easy in Go and Chess because you can force/expect near optimal responses for your opponent, SC2 has near optimal response too, theoretically, but it'll be less obvious due to positional and compositional/upgrade quirks and the sheer complexity due to the aforementioned points. DeepMind will need sophisticated, dynamic rule sets in real time.
I think it's possible, but I don't think the tech is there yet.
On December 14 2017 05:54 sc-darkness wrote: Can't AI win vs people just with high APM? Each unit could be micro managed individually just like the usual bots.
The goal isnt to win by straight up mechanics, but win by decision making. Hence why I know Deepmind, APM is throttled to near-human levels.
I would have perefered it to go under human levels, since it will be so probably perfectly efficient with it anyway.
On December 13 2017 20:06 Excludos wrote: An AI just beat some of the world's best Dota players after only 2 weeks of training and you guys think it will take 5-10 years or more to develop one which can do the same in starcraft? Come on. Sure it's more strategically challenging, but it can also multitask perfectly. I was honestly surprised to find it hadn't already surpassed humans.
The DotA result did not seem that significant to me. The AI was eventually beaten by pro players, and as far as I know it was just 1v1 mid-only, which is not a serious category. Rote execution of last hit and deny mechanics with one single hero and a limited set of items is obviously something an AI would excel at, but this does not prove a serious ability to pick teams and evaluate item and strategy choices, nor does it prove that the AI(s) can coordinate effectively as a team, nor does it prove that the AI has some level of resilience vs exploitative and off-beat strategies designed to target its weaknesses.
Chess engines were unbeatable tactically long before they ever posed a serious threat to human players in a match.
Yes, it was beaten, and then it grew stronger. That's what training AI means.
But the significant part isn't that it just beat some players. The significant part is that it trained against itself and got good enough to beat top players within 2 weeks. During this time, all on its own, it learned to lasthit both enemy and friendly units, it learned what items to buy and when, it learned to make use of the donkey with both ferrying bottles and items, it learned to block friendly creeps, glyphs, dodging projectiles and a whole lot more.
A lot of these things are easily transferable to a game like starcraft. A good AI should easily be able to learn optimal build orders, multitasking armies with drops, perfect unit spread, map control, expansion, upgrades, etc. The one thing it might get trouble with is it's pitted against another race instead of a mirror. One of the sides might develop a strategy that the other simply isn't able to break out of, and the prior will simply stop developing because it keeps winning with this subpar strategy. But in a mirror, like I said, I'm surprised we haven't seen someone implement an AI already.
I don't necessarily agree. Sometimes a learning process converges to a local maximum and can not move beyond that. The AI was only tested on the most strategically shallow version of the game, where execution is paramount. It was beaten both by outplaying it straight up and by exploiting its lack of understanding by using off-beat strategies. I'll believe that an AI can continue improving up to the point that you can no longer beat it straight up, even if you are a professional player. But it is not at all obvious that an AI can learn to defeat targeted anti-AI strategies.
So while it learned to hit glyphs and avoid projectiles, there is no direct reason to believe that it could understand about map control in a 5vs5 setting and that it could learn to understand what builds are good. Other than faith in the inevitability of AI progress, but that is so indiscriminate and only tells you that AI will be able to defeat humans "eventually". But companies can easily abandon AI research for DotA
These were the results for the recent BW AI competition. Note that the AI created by a hobbyist that just rushes you every game won the competition, and four out of the first six winners had really short average game times. Also, afaik, in 10 years of AI research, there hasn't been a single one that could compete on any level with a pro.
In the other article someone said they did not expect an AI to be able to beat a human player in the next five years. So honestly, if you're only casually interested in this I would go to sleep and wake up in two years before checking if there was any significant progress. SC2 is just vastly more complex than Chess or Go, and it's not even clear if a single AI based on a general learning algorithm is capable of learning it to the point of posing any sort of challenge to a pro player.
a few months before alphago was initially released there was an article in which members of the a.i.-go community stated they believe a go program that could beat professional players was at least a decade away. the reasoning sounded quite like what you are saying, basically that go was far more complex than chess and that was shown by the fact that the best go programs at that time were playing at a low intermediate level.
note that i'm not trying to say you're necessarily wrong, the games are very different, i just want to point out that i've read something similar before and therefore doubt we can make very reasonable guesses as outstanders not involved in the development.
These were the results for the recent BW AI competition. Note that the AI created by a hobbyist that just rushes you every game won the competition, and four out of the first six winners had really short average game times. Also, afaik, in 10 years of AI research, there hasn't been a single one that could compete on any level with a pro.
In the other article someone said they did not expect an AI to be able to beat a human player in the next five years. So honestly, if you're only casually interested in this I would go to sleep and wake up in two years before checking if there was any significant progress. SC2 is just vastly more complex than Chess or Go, and it's not even clear if a single AI based on a general learning algorithm is capable of learning it to the point of posing any sort of challenge to a pro player.
a few months before alphago was initially released there was an article in which members of the a.i.-go community stated they believe a go program that could beat professional players was at least a decade away. the reasoning sounded quite like what you are saying, basically that go was far more complex than chess and that was shown by the fact that the best go programs at that time were playing at a low intermediate level.
note that i'm not trying to say you're necessarily wrong, the games are very different, i just want to point out that i've read something similar before and therefore doubt we can make very reasonable guesses as outstanders not involved in the development.
You're not wrong, people underestimate things like this consistently. Again, have people already forgotten about the openAI beating top Dota players in 1v1 literally months ago? After only training for 2 weeks? Yes, you can argue that sc is more complex for sure, but it's not "decades away", or even "several years" away. AI research have absolutely skyrocketed these last few years. We are going to see an AI beat top sc players within 2018. If it's months or a year away I don't know, but it's right around the corner for sure.
These were the results for the recent BW AI competition. Note that the AI created by a hobbyist that just rushes you every game won the competition, and four out of the first six winners had really short average game times. Also, afaik, in 10 years of AI research, there hasn't been a single one that could compete on any level with a pro.
In the other article someone said they did not expect an AI to be able to beat a human player in the next five years. So honestly, if you're only casually interested in this I would go to sleep and wake up in two years before checking if there was any significant progress. SC2 is just vastly more complex than Chess or Go, and it's not even clear if a single AI based on a general learning algorithm is capable of learning it to the point of posing any sort of challenge to a pro player.
a few months before alphago was initially released there was an article in which members of the a.i.-go community stated they believe a go program that could beat professional players was at least a decade away. the reasoning sounded quite like what you are saying, basically that go was far more complex than chess and that was shown by the fact that the best go programs at that time were playing at a low intermediate level.
note that i'm not trying to say you're necessarily wrong, the games are very different, i just want to point out that i've read something similar before and therefore doubt we can make very reasonable guesses as outstanders not involved in the development.
You're not wrong, people underestimate things like this consistently. Again, have people already forgotten about the openAI beating top Dota players in 1v1 literally months ago? After only training for 2 weeks? Yes, you can argue that sc is more complex for sure, but it's not "decades away", or even "several years" away. AI research have absolutely skyrocketed these last few years. We are going to see an AI beat top sc players within 2018. If it's months or a year away I don't know, but it's right around the corner for sure.
It is true that the AI development will probably take place much faster than people anticipate, but the thing is, that Dota match is as representative of an actual match as coop is an accurate representation of what happens in Starcraft 2. There are things that transfer over like micro and in Starcraft, macro, to a certain extent, but they are wildly different things.
In that demonstration, it was 1 player versus 1 player, playing the same character, in one lane, with basically perfect map vision (at least in the sense that one player generally knows where the other player is and what they are doing) with the same creeps duking it out in a battle of sheer mechanics and a bit of tactics.
In an actual Dota game, there are 8 more players, playing several different characters that all have different spells and abilities with bosses and camps that they can defeat and different objectives not to mention managing their gold economy such that even if you became the best on the 1v1 simulation, that only means that in a very specific situation, you're going to really good.
And personally, I think SC2 is more complex than Dota, so while it may not take as long as some people are thinking, I think it will take longer than your estimation.
Maybe interesting add to the Micro tasks are simple for AIs and it is only the "strategy part": Table 1 of this paper. Of course the paper is now four month old. But I would have thought that for example CollectMineralShards should be easy for the Atari-net.
EPT
