Flash on DeepMind: "I think I can win"

On March 10 2016 23:36 Pandemona wrote:
Yea, i think AI would struggle in an RTS game. Yet i am still open to be surprised. Imagine God losing a bw series to an AI !!!

Aren't there already many bots for BW?

On March 10 2016 23:57 etofok wrote:


This is not just a bot mind you - it's a thinking AI that responds to situations accordingly to its accumulated experience. A bot is something that just has a decision making tree predetermined by its developer. This one develops its own decision making tree. On its own.

Watch 30 seconds of this guys.

On March 10 2016 23:57 WiSaGaN wrote:
I don't think this will become a strategy game.
It is more likely that the impeccable micro attack will just dominate the game.
How will Flash counter a perfect Drone hit-and-run?

On March 11 2016 00:00 Monochromatic wrote:
This is a pretty interesting proposition. I think it depends on how much google will let the ai go beyond human capabilities. There already exist stupid micro ai, and if google lets the ai micro flawlessly it shouldn't be a contest. Of course, if they limit it to human capabilities (Like a hard cap of 400 APM) , then I think a human will always have the ablity to win.

Of course, if they limit it to human capabilities (Like a hard cap of 400 APM)

On March 11 2016 00:11 Temporary Happiness wrote:
I think these 2 videos tell who's gonna win if this is done in Sc2:

https://www.youtube.com/watch?v=mrbYd4OFrWE

https://www.youtube.com/watch?v=IKVFZ28ybQs

When opponent microes like that there is no room for outplay him strategically i think..

On March 11 2016 00:42 BisuDagger wrote:
hero can operate at about 450-500 apm on a consistent basis. To make this fair, the actions per minute should be clamped to <=600

On March 11 2016 00:47 WiSaGaN wrote:

That looks like some interesting proposal.

On March 11 2016 01:13 Charoisaur wrote:

There are compositions where you can't micro that much for example roach vs roach zvz or roach ravager vs bio.
In those situations perfect micro doesnt give you that much of an advantage.

On March 11 2016 01:16 beg wrote:

Imagine 50 roaches, individually microed to create a perfect arc, pulling back before they die and burrowing, joining the battle again after regenerating.

Yea... no :D

On March 10 2016 23:36 Pandemona wrote:
Yea, i think AI would struggle in an RTS game. Yet i am still open to be surprised. Imagine God losing a bw series to an AI !!!

On March 11 2016 00:11 Temporary Happiness wrote:
I think these 2 videos tell who's gonna win if this is done in Sc2:

Video 1

Video 2

When opponent microes like that there is no room for outplay him strategically i think..

On March 11 2016 01:25 sCCrooked wrote:


Reasons like this are why they're considering Brood War and not SC2. BW is by far the only one that displays enough stability to produce a real result. Nobody with a strategically adept mind would pick SC2 for this.

Also, there should be absolutely no limits on the AI. Strategic depth should win out over pure mechanics/speed because in Brood War its much harder to completely counter someone's micro unlike its sequel where things are much more tic-tac-toe and then its over instead of constantly stacking areas you have to battle and defend.

On March 11 2016 00:42 BisuDagger wrote:
hero can operate at about 450-500 apm on a consistent basis. To make this fair, the actions per minute should be clamped to <=600

On March 11 2016 02:03 Goolpsy wrote:
Even with 400 apm limit, Neural network learning can easily teach it where to focus its micro most efficiently.

Game might seem like it, but Starcraft has far fewer strategies than chess.

On March 11 2016 02:11 eScaper-tsunami wrote:


The sad thing is it doesn't take 600 apm to have perfect micro. In the end it boils down to accuracy and efficiency.

On March 11 2016 03:44 The_Red_Viper wrote:
See the problem i have with this is that the Ai will have such a big advantage through 'mechanics' alone.
It's much more interesting in GO because there is no difference in execution, tactics and strategy is all that matters here.

Even though there are a lot more possible "board states" in sc2, i am not sure if that really matters in the end if you theoretically have a player with unlimited APM and attention.


But hey, i have obviously very little idea about it and when google says starcraft would be the next step, maybe it's harder than i think (or they really want to somewhat limit the AI in the mechanics department so it comes down to tactis/strategy, which would be weird though)

On March 11 2016 02:30 Cuce wrote:
Flash hit the key point I think.

Go and chess are games of complete information, where as starcraft is a game of incomplete information. furthermore, the more information you get usually means the mroe resouce you spent on getting information.

sure, a computer can calculate possible situations with given incomplete information, but to match what we call meta game and intuition of pro players, will take a really long time of self learning ana data collecting on computers part.

On March 11 2016 03:09 disciple wrote:
This match would have number of interesting implications chief among witch are BO decisions. If AI is strictly superior microing units theres no reason not to assume that it will try taking advantage of this and go for 1 base all-ins most of the time in order to force micro intensive early games. It would be cool if the AI has some doubt about his opponents skill and actually needs to confirm its superiority in micro in order to feel confident in winning and going for all ins. Humans already do that as we all know from Bisu being annoying as much as possible with his scouting probe. Now imagine AI controlling this, it will never die by mistake.

On March 11 2016 05:07 bITt.mAN wrote:
Lol.

1. They should do it with BWAPI because SC2 is lame like that (it doesn't have an API to interface code<->game).

2. There's been TONS of theorycrafting on RTS AI and their limitations. link Two big differences between turn-based games and RTS, are real-time computational optimizations (which figure far-less in turn-based AI), and, as Flash rightly states, finite information.

On March 11 2016 05:32 Grizvok wrote:


Their limitations NOW you mean. A sophisticated AI built to play SC2 (when it is ready) will destroy any player easily. Regardless you don't factor in the crazy levels of micro you can pull off with infinite APM. Dropping three areas at once while still macro'ing perfectly WHILE stutter step micro'ing each drop is something a human will never be able to do yet it is feasible that a computer could potentially do those things.

On March 11 2016 04:18 The_Red_Viper wrote:

Because mechanics are such a big part about starcraft. By far the biggest. So how do we really make sure that the Ai didn't win through mechanics? It's impossible (imo) to build it exactly at the sweet spot. Attention is probably even a bigger deal than apm itself.
The only real way to make sure "it is fair" is to make the AI use the same hardware, mouse, keyboard and monitor.
If you don't do that then the result is questionable at best as far as i can tell

edit: and even then you will get a device which is superior to human flesh, so i dunno..
AI vs AI would be interesting to watch though, i would imagine tactis and strategy would be a way bigger deal there because the mechanical part could be made exactly even

On March 11 2016 05:59 The_Red_Viper wrote:


I mean that the AI would have the same restrictions mechanically as the tpyical human. We only can interact with the game with the help of the hardware, mouse, keyboard and monitor.
The AI probably wouldn't do that, it could be everywhere at once (you as human cannot because the monitor simply doesn't make it possible, just as the mouse doen't make it possible to control different groups at once, etc)
If the human had another device (control the game directly with the brain or something similar) this maybe wouldn't be a limiting factor anymore.

But yeah if you can somehow make it so that the AI doesn't have better mechanics/multitasking/attention than the average pro player, then maybe this would be interesting (even though i am not so sure about that either, even though starcraft might have more possible "board states", i would imagine that most of them are completely irrelevant and that the actual depth of the game isn't anywhere near GO for example)
It being a game with limited information is the only interesting aspect about all of this i can see tbh

On March 11 2016 06:47 The_Red_Viper wrote:


Just to be clear, let's say you place building X at place Y or Z, that are two different "board states" right?
Even if it means that placing your first supply depot in the enemy base probably isn't all that smart?

I get that it isn't "intuitive" for the AI like for a human being, but there surely are tons and tons of these things in sc2.
Even something like: I move my army (or even single marine) a few tiles on the left, it probably won't be the biggest deal but it surely is considered a different "board state" ?
If we want to play 100% perfectly these things have to be considered, but overall it probably doesn't matter at all i would imagine.
I don't think the same is true for GO? (i have no idea about GO though)
My statement was probably just simply this: A high lvl GO players surely possesses more tactical/strategical understanding than a starcraft professional, you don't have to be highly intelligent to play starcraft at a high lvl, the same probably isn't true for GO/chess. i think? (i can see why this isn't all that relevant to the main topic though ^^)

On March 11 2016 07:11 disciple wrote:
Considering the careers savior and stork had, I think some APM between 80 and 120 will be sufficient

On March 11 2016 07:11 disciple wrote:
Considering the careers savior and stork had, I think some APM between 80 and 120 will be sufficient

On March 11 2016 08:37 Shinokuki wrote:
Why is this in sc2 section if alpha go is playing bw and flash mostly played bw and won lots of championships..

On March 11 2016 08:37 Shinokuki wrote:
Why is this in sc2 section if alpha go is playing bw and flash mostly played bw and won lots of championships..

On March 11 2016 09:25 ilikeredheads wrote:
Of course Flash can win, he's a terminator as well. Machine vs Machine

On March 11 2016 09:46 Shakattak wrote:

wouldnt that be so exciting to watch though?

On March 11 2016 02:36 Cuce wrote:



btw does broodwar has a hardlimit for command accepted per frame?
like 12 per frame or something.

I wouldnt be suprised if there was, at least a limit to buffering

On March 11 2016 10:54 vOdToasT wrote:
It's only a matter of time until artificial intelligence can defeat the greatest of us at all games.
After that, they will only lack the ability to artistically express and describe things that are valuable to humanity, because they don't know what it's like to be human.

Eventually, they may be able to understand that, and create art, too.

On March 11 2016 10:40 Superbanana wrote:
Imba Ai goes 3 rax reaper every game no matter what and wins every game

Don't say "solved". Chess is not solved, Go is not solved.

On March 11 2016 11:08 chipmonklord17 wrote:
Hey Google, instead of making an AI to beat a starcraft player, sponsor a starcraft team. It would cost less and probably be better received.

EDIT: Not saying this is poorly received, but imagine the hype if it was announced Google was getting into esports

On March 11 2016 10:35 rockslave wrote:
Everyone is missing the point (including Flash).

Go is already a game with an impossibly big search tree for brute force. Even chess is. The classical approach of heuristics coupled with brute force solved chess, but it was never even Platinum in Go.

The only reason for AIs starting to beat Go players is a somewhat recent innovation in AI: deep learning. From 10 years ago or so, there were several advancements to machine learning that made a gigantic leap in many fields for which computers always sucked. For instance: character recognition used to be a PitA, but nowadays you can write Python code that gets it right 99% of the time in a few minutes (the breakthrough was a particular optimization technique called backpropagation).

Even if you cap micro a lot, StarCraft isn't too much different from a combination of Go and a bunch of pattern recognition. That is precisely what machine learning solves. It's not easy though, there is a lot of clever training and parametrization to be done... But if they put it in their roadmap (with enough money), it will happen.

Oh, and imperfect information is not a problem at all. Even with a more standard (backtracking / brute force) approach, you only need to throw some probabilities around.

It's rather easy to write programs that play Poker well, for instance (discount the poker face though).

On March 11 2016 10:35 rockslave wrote:
Everyone is missing the point (including Flash).

Go is already a game with an impossibly big search tree for brute force. Even chess is. The classical approach of heuristics coupled with brute force solved chess, but it was never even Platinum in Go.

The only reason for AIs starting to beat Go players is a somewhat recent innovation in AI: deep learning. From 10 years ago or so, there were several advancements to machine learning that made a gigantic leap in many fields for which computers always sucked. For instance: character recognition used to be a PitA, but nowadays you can write Python code that gets it right 99% of the time in a few minutes (the breakthrough was a particular optimization technique called backpropagation).

Even if you cap micro a lot, StarCraft isn't too much different from a combination of Go and a bunch of pattern recognition. That is precisely what machine learning solves. It's not easy though, there is a lot of clever training and parametrization to be done... But if they put it in their roadmap (with enough money), it will happen.

Oh, and imperfect information is not a problem at all. Even with a more standard (backtracking / brute force) approach, you only need to throw some probabilities around.

It's rather easy to write programs that play Poker well, for instance (discount the poker face though).

On March 11 2016 14:47 beg wrote:
@ZAiNs: Aren't there many BW replays? Also, DeepMind is capable of learning from reading the graphics, so they could try using VoDs too.

On March 11 2016 15:09 ZAiNs wrote:

AlphaGo was fed 30 million moves and apparently the average number of moves per game is 200, meaning they gave it around 150,000 high-level games. Getting that number of BW games is impossible, and even if it were, I'm quite sure you'd need drastically more replays to get training results on par with AlphaGo's initial training set. I don't think VODs would even be useful because they show such little information about the game state at any point in time, I think a replay is needed so it can observe the entire game state at every point in time.

On March 11 2016 15:09 ZAiNs wrote:

AlphaGo was fed 30 million moves and apparently the average number of moves per game is 200, meaning they gave it around 150,000 high-level games. Getting that number of BW games is impossible, and even if it were, I'm quite sure you'd need drastically more replays to get training results on par with AlphaGo's initial training set. I don't think VODs would even be useful because they show such little information about the game state at any point in time, I think a replay is needed so it can observe the entire game state at every point in time.

On March 11 2016 13:01 ZAiNs wrote:

Deep learning needs a dataset for the AI to be trained though. For AlphaGo they trained two separate networks (one designed to predict the next move, and the other designed to predict the final winner) on 30 million discrete moves from games played by human experts. After that it trained itself by actually playing Go against itself a ridiculous number of times.

A Go game can be perfectly modelled by simple list of positions describing which square had a stone placed on it each turn, it's going to be very hard to get enough useful data (replays) to significantly help with the training. And without the initial training it's going to have to learn mostly by playing against itself which will be difficult because of the ridiculous number of game states. At least that's my understanding of things, I could be wrong, but it seems to be a lot harder than Go.

On March 11 2016 15:02 ETisME wrote:
Actually it makes me wonder what would two deepmind do if they were to play against each other.
We may even see a whole new meta developing

On March 11 2016 13:01 ZAiNs wrote:
Deep learning needs a dataset for the AI to be trained though. For AlphaGo they trained two separate networks (one designed to predict the next move, and the other designed to predict the final winner) on 30 million discrete moves from games played by human experts. After that it trained itself by actually playing Go against itself a ridiculous number of times.

A Go game can be perfectly modelled by simple list of positions describing which square had a stone placed on it each turn, it's going to be very hard to get enough useful data (replays) to significantly help with the training. And without the initial training it's going to have to learn mostly by playing against itself which will be difficult because of the ridiculous number of game states. At least that's my understanding of things, I could be wrong, but it seems to be a lot harder than Go.

On March 11 2016 22:47 MyLovelyLurker wrote:
I've been watching Broodwar for 15+ years, and programming reinforcement learning engines for a few. Here are a few thoughts on why the specific DeepMind approach is going to be very hard for SC, although it might well happen in around 10 years time :

1. We are assuming the AI teaches itself to play only from a realtime view of the pixels on the screen, and knows nothing about any score at all - *there is no score in Starcraft*, unlike in the arcade Atari 2600 games that have been mostly arcade games, with a clear numerical objective ( the score ) to be maximized by the playing agent. The act of playing thereby becomes a calculus problem ( to first order, set the gradients of the score function to zero ). Not impossible but harder in Starcraft.

2. Starcraft II is an imperfect information game, as opposed to chess or go where the board contains the whole information available to both players. Whilst it is possible to do reinforcement learning in that setting, it is a relatively new field and adds to the difficulty - articles are being published now on the subject.

3. The 60 or 120 APM barrier will not be broken easily. Right now in the Atari 2600 Deepmind simulations rely on one or two actions by frame, which imposes that your APM is limited by the FPS you render. Even with two policy networks - one for the keyboard and one for the mouse - you are headbutting against 120APM pretty much. It is not impossible to think about operating several policy networks in parallel in order to enable strong ( think Korean multiple drops ) multitasking, but it is a new area that needs to be explored - the connections between networks and their interaction would need to be thought through carefully. Some cutting-edge research with asynchronous networks goes in a similar direction.

4. Point-and-click games have not been tackled yet by RL ; they are joystick or keyboard-based, ergo with binary 'push or don't push' states, but no mouse game has been tackled by a policy network as far as I know. This brings its own sets of challenges ( the AI will have to figure out by itself, for instance, how to move the mouse in optimal ways, which includes making straight lines, position the cursor close to a Nexus or a pylon, etc ).

5. Starcraft is also 'multi-screen' - it requires frequently changing views with your F keys ( move to different bases and engagement battles ) in order to correctly represent the full state of the game. So far, Atari 2600 games have been mono-screen only. Again, it is not impossible to imagine this will be overcome in the future, it is just harder to do right now.

6. Combinatorial explosion in the number of unit compositions is also hard to tackle - every time you add a potential unit to the mix, the possibilities for army composition multiply, which is why in the campaign mode you learn to play from introduces units pretty much one at a time. It would objectively be much, much harder to start playing full games from laddering and without an instruction manual, which what the Deepmind approach is.

7. The meta in SC rotates on a regular basis - it is 'non stationary', which adds to the list of problems encountered by a machine that would learn by playing on ladder, as some of the strats and playstyles learned earlier could well be obsolete - and hard-countered - by the time they are assimilated. This happens with human players too ; they have to make a conscious effort to get out of a slump, learn more new information, and forget about the old. Some work on policy distillation or optimal brain damage in neural networks goes, very tentatively, in that direction. Again, this is hard.

For all those reasons, it would be an incredible achievement already to have a Starcraft deep reinforcement learning AI that can teach itself to play a very easy computer AI in a setting with only workers, and maybe a unit list restricted to just a couple, like zealots and dragoons.

If you look at the performance of reinforcement learning in 2d games such as Atari, 'mechanical' games like Pong or Breakout get to much higher skill levels than games with planning required such as Pacman. It is hence entirely possible that Starcraft Deepmind would play mechanically correctly, but overall pretty poorly, as one can only speculate. If you add up all the objection points above, you can get a feel for why there is quite a long way to go.

Happy to provide reference articles list if required.

[B]1. We are assuming the AI teaches itself to play only from a realtime view of the pixels on the screen, and knows nothing about any score at all - *there is no score in Starcraft*, unlike in the arcade Atari 2600 games that have been mostly arcade games, with a clear numerical objective ( the score ) to be maximized by the playing agent. The act of playing thereby becomes a calculus problem ( to first order, set the gradients of the score function to zero ). Not impossible but harder in Starcraft.

On March 12 2016 22:08 Superbanana wrote:
Hard? 10 years? Are you kidding?
+ Show Spoiler +

On March 10 2016 23:36 Pandemona wrote:
Yea, i think AI would struggle in an RTS game. Yet i am still open to be surprised. Imagine God losing a bw series to an AI !!!

On March 12 2016 09:06 BeStFAN wrote:


could anyone answer this?: what is the significance of AI's ability to master the game of Go in relation to what it means for it's ability to play BW at a high enough level?

in other words, before and after the developments required for the ability to beat sedol what tools has AI gained in relation to it's abilty to play SC?

On March 13 2016 00:58 75 wrote:
no way when there is no apm cap.

another question: can AIs beat top level poker players?

On March 12 2016 21:55 rockslave wrote:


I don't think your first hypothesis is true, the AI would be able to read the data in the replay files and judge plays accordingly (only in the training phase).

Also, there is a natural language to describe the moves: the one people use to describe AIs in BW (stuff like GTAI).

On March 12 2016 09:06 BeStFAN wrote:


could anyone answer this?: what is the significance of AI's ability to master the game of Go in relation to what it means for it's ability to play BW at a high enough level?

in other words, before and after the developments required for the ability to beat sedol what tools has AI gained in relation to it's abilty to play SC?

On March 13 2016 13:22 evilfatsh1t wrote:
is anyone really debating whether ai will be able to do something better than a human? i dont think anyone is naive enough to think humans will be able to defeat ai in something in the future. what flash, boxer are probably saying is if alphago could play starcraft NOW, the humans would win. of course if you gave google as much time as they wanted, the ai would win. its literally only a matter of time given the speed at which technology is advancing

On March 13 2016 18:56 WinterViewbot420 wrote:
I have never seen the official Fish bot say anything before, I didn't even know it could talk.

This is something about DeepMind, I don't know what some of these words mean!! HELP



edit: Clan members are telling me this is the first time the bot has talked in like ten years? Wtf lol

On March 14 2016 01:16 GGzerG wrote:
God speaks again.

Bots on fish can be made to speak by the Blizz / master of the channel, just sayin lol

On March 15 2016 09:43 Moose1 wrote:
but what race would the ai choose?

On March 15 2016 19:17 SlayerS_BunkiE wrote:

I think this is a very good topic to discuss. Which race would benefit most if you had "unlimited apm"?

On March 15 2016 19:17 SlayerS_BunkiE wrote:

I think this is a very good topic to discuss. Which race would benefit most if you had "unlimited apm"?

On March 17 2016 17:43 MrMischelito wrote:
I wouldn't be surprised if computers were better at video games. after all it's all just about numbers... 10110...
I'll continue following this discussion if/when computers start beating humans at basketball

On March 17 2016 20:51 Slayer91 wrote:


most ghetto post of this thread

On March 11 2016 04:06 Loccstana wrote:
This would be of interest to people interested in AI for Starcraft:

https://webdocs.cs.ualberta.ca/~cdavid/pdf/starcraft_survey.pdf

A conservative lower bound on the state space of brood war is 10^1685. This is many orders of magnitude above the state space of Go, which is 10^170. Whats more, the branching factor is 10^50 to 10^200, compared to <360 for Go.