It looks like we have confirmation that DeepMind will attempt to enter into the StarCraft arena next.
As of this moment, there are no details as to how long AlphaGo (AlphaStar?) will need before it's ready. There are also no representatives from the professional StarCraft II scene named as of yet.
Against Lee Sedol, Google had AlphaGo using 1920 CPUs and 280 GPUs. It remains to be seen how much more computing power Google will devote to this next challenge and the effect that large jump in resources has in AlphaGo's decision-making ability.
The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
The only way I see APM being limited would be if you did the following 1) Choose a designated 'esports keyboard' (some razer/steelseries/ect product) 2) Find the maximum amount of inputs the keyboard recognizes per minute 3) Make that the APM 'limit'
Each keyboard will stop responding after a certain number of inputs per second, and said input limit is probably much lower than the input limit that AlphaGo would have. At least I assume so.
Kinda disappointing they went for SCII instead of BW. BW is much more stable than the current version of SCII, especially that this is a new expansion that was out for only couple of months compared to 18 years of a stable version, heck, BW even have a stable map pool (or even a most single used map). It would have been more suitable for the AI to train on instead of a changing map pool and balance updates from time to time.
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
I don't think an apm cap would be completely arbitrary. Somewhere around the average apm of top pros or maybe a little higher I think is reasonable.
Choosing to cap APM based on top pros makes sense because it makes the measurement a lot more meaningful.
We already know that machines can have an unbelievably high APM. Clearly, machines can out micro humans. However, what we want to know is if machines can out-think humans. Capping APM based on human ability makes sense because it adds another layer of decision-making. What will the computer choose to do with that APM? Will some of that APM be wasted on micro with minimal gains? Or will the computer choose more efficient ways to use a limited resource? We could learn a lot if we limit APM that we would not learn if we let the machine use as many commands as possible.
APM is a limited resource in strategy games, just as turn based games have 1 move per turn. Having unlimited APM defeats the purpose of testing a computer's ability to learn, adjust, and out-think a human being.
Having the APM based on top pros would be pretty advantageous for DeepMind. Pros are trying to do things as fast as possible, not with the least APM possible, so there is obviously a large amount of "wasted" APM (though for a human it isn't wasted since it isn't like you could use it elsewhere). So a computer with a certain amount of APM can do a lot more than a human with the same APM.
A computer who is as "smart" as a human should be able to get by with less APM though exactly how much is hard to evaluate. Also computers can't misclick.
The more I think about it the more I'm surprised that Starcraft was chosen. Since there's so many mechanical advantages that a computer can leverage that are very hard to control for, any win by a computer will be beset by complaints and excuses, legitimate or not, that the computer isn't actually strategically or tactically superior.
without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
On March 28 2016 13:39 JimmyJRaynor wrote: without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
That was the prevailing stance before AlphaGo fought Lee Sedol... It didn't hold up to reality though.
APM is not the point. A large portion of human APM is meaningless and just for warming up purpose toward spikes in big fights.
SC is a totally different game to Go. People call it not "perfect information", meaning AI does not know exactly what its opponent doing in a SC game, not as completely as black and white stones on a Go board.
Suppose two medivacs are approaching the zerg AI's 2nd base, the AI does not know what's inside, marines, marines and mines, or empty? AI's need to predict the possible drop location, it is main base, or 2nd base, or one for each? All the possible scenarios requires defense strategies accordingly with consideration of limited resource and efficiency. All these scenarios are developing in real time and could shifting from each other within milliseconds as the medivacs make a boosted turn.
On March 28 2016 13:39 JimmyJRaynor wrote: without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
That was the prevailing stance before AlphaGo fought Lee Sedol... It didn't hold up to reality though.
who is the winner? the best Chess AI with heuristics or Alpha?
On March 28 2016 13:39 JimmyJRaynor wrote: without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
That was the prevailing stance before AlphaGo fought Lee Sedol... It didn't hold up to reality though.
who is the winner? the best Chess AI with heuristics or Alpha?
I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
BW AIs with no APM limits still can't beat decent humans, despite super muta micro and whatnot.
The Mouse-movement and Keyboard entry limitations would have to be imposed. Giving a computer full API access is to cheat, as the API could do things no mouse is actually capable of. We've seen some of the custom AIs over the years, but those functionally produce a new unit-movement control that goes around the game's actual functionality.
Thus, the first thing an "AlphaStar" would need to learn would be how to use the mouse. Haha.
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
BW AIs with no APM limits still can't beat decent humans, despite super muta micro and whatnot.
Well, Google also dumped somewhere in excess of several million dollars worth of equipment and probably more money in salaries at the issue. To the point that it wouldn't surprise me if the total "Man-Hours" applied to producing a Go-playing AI wasn't actually more total hours than it took for the Master to learn the game. (While it's impressive that it won, you can do a lot of impressive things with massive money expenditures.)
Though I did come up with a funny thought: what if someone only agreed to play the AI if it had to actually physically use a Keyboard & Mouse? Be sort of funny to see the complaints about insisting on that.
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
BW AIs with no APM limits still can't beat decent humans, despite super muta micro and whatnot.
The people programming the BW AIs don't have Google's resources.
Besides the goal really isn't to win at Starcraft, it's to develop AI research. They want to find new approaches and algorithms that would enable a computer to "out-think" humans at Starcraft. Figuring out ways to abuse the AIs mechanical advantages isn't all that meaningful.
Thus, the first thing an "AlphaStar" would need to learn would be how to use the mouse. Haha.
Yeah this is what I argued in the other thread. The AI would need the constraints of a mouse, keyboard and monitor or it is just cheating. The mouse, keyboard and monitor makes our play quite sloppy and limit our vision.
I still think the more interesting match is to let humans control the game with their mind and let the AI face off against that. Imagine perfect forcefields, and suddenly they appear. The only constraint would be the monitor (the AI wouldn't be able to view more than one area of the map at a time).
I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
This AI already beat Go world champions, that's a turn based game with no mechanics.
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
Extremely hyped for this, but honestly they need some SC pros working with them to set the AI, cause i highly doubt any computer can handle these billions of bazzillions possible scenarios in SC. In Go or Chess you can go the extensive way, just loading in all possible situations into the memory and the only thing AI should do in that case is to choose between a relatively small amout of choices. In SC2 though, the amount of these choices is uncomparably higher, i mean billion times higher. E.g. in chess there are only 64 squares, while in SC2 the amount of possible positions for units is beyond count, etc etc. If you add a mentioned "no info" factor eveything becomes even worse. There are a lot of situations in this game where you literally have to guess, and this is where common sense comes to help, which is obviously a problem for any AI. So i guess the only possible solution for this whould be working on some common "principles" or "basics" for AI to persue while playing. Yes that is a bit artificial as these "rules" would not be of AI origin, but this is how i see this.
On March 28 2016 13:39 JimmyJRaynor wrote: without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
That was the prevailing stance before AlphaGo fought Lee Sedol... It didn't hold up to reality though.
who is the winner? the best Chess AI with heuristics or Alpha?
That's a weird way to defend a point...
Are they playing Chess, Go or StarCraft?
here is why Alpha will have a tough time without heuristics in a game like Chess or Starcraft
On March 28 2016 15:39 insitelol wrote: In Go or Chess you can go the extensive way, just loading in all possible situations into the memory and the only thing AI should do in that case is to choose between a relatively small amout of choices.
You don't know what you are talking about. This doesn't work for Go, and this is why Alphago is a breakthrough.
[B]There are a lot of situations in this game where you literally have to guess, and this is where common sense comes to help, which is obviously a problem for any AI. So i guess the only possible solution for this whould be working on some common "principles" or "basics" for AI to persue while playing. Yes that is a bit artificial as these "rules" would not be of AI origin, but this is how i see this.
The problem is the same for Go, but they did NOT give Alphago a set of common principles. It has discovered these by itself.
On March 28 2016 15:39 insitelol wrote: In Go or Chess you can go the extensive way, just loading in all possible situations into the memory and the only thing AI should do in that case is to choose between a relatively small amout of choices.
You don't know what you are talking about. This doesn't work for Go, and this is why Alphago is a breakthrough.
[B]There are a lot of situations in this game where you literally have to guess, and this is where common sense comes to help, which is obviously a problem for any AI. So i guess the only possible solution for this whould be working on some common "principles" or "basics" for AI to persue while playing. Yes that is a bit artificial as these "rules" would not be of AI origin, but this is how i see this.
The problem is the same for Go, but they did NOT give Alphago a set of common principles. It has discovered these by itself.
Hm, ok, thanks. Didn't know that. It is hard to imagine though, i guess i should dig this a bit. edit: oh, its ANN right, should have guessed. So she whould accumulate these "principles" from enldess replay/vod base. It solves the "common sense" problem at least partially. Would be fun to watch.
Unless you limit AlphaGo it should be extremely easy to beat the best players in the world.
All you need is to make AlphaGo Terran, give it medivacs, stim, and the ability to micro each individual marine perfectly. Seriously it should be the easiest task ever.
Just make it like open reaper expand every game, then it only has to counter some cheeses. Go does not have mechanics, Starcraft 2 does, and there the AI would be infinitely superior.
[QUOTE]On March 28 2016 15:07 BronzeKnee wrote: [QUOTE]On March 28 2016 14:35 Taf the Ghost wrote:
I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful. [/QUOTE]
Isn't that the real issue here, though? Human brains are powerful BECAUSE they can think quickly, but the fact that we can think quickly means we're okay with being wrong if we come to a decision fast enough. The computer is on a totally different spectrum...it takes a long time to come to a conclusion, but in its mind it's always the move that has the highest probability of winning. The whole clash of fundamental ideals there is what makes watching AI vs. humans so interesting to me...really a 2nd race, if we could make the comparison.
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
China already has a supercomputer that has about the same computing power as the estimated computing power of the human brain. In about 10-20 years this computing power will cost like a couple thousand dollars (this is just based on a continuation of moore's law, which has held true for a very long time).
On March 28 2016 17:04 mierin wrote: Isn't that the real issue here, though? Human brains are powerful BECAUSE they can think quickly, but the fact that we can think quickly means we're okay with being wrong if we come to a decision fast enough. The computer is on a totally different spectrum...it takes a long time to come to a conclusion, but in its mind it's always the move that has the highest probability of winning. The whole clash of fundamental ideals there is what makes watching AI vs. humans so interesting to me...really a 2nd race, if we could make the comparison.
You can easily code an AI to take say 1 second to come up with a bunch of potential moves and then pick the one that looks the most promising so far. By no means does it have to evaluate all possiblities or any such nonsense. Obviously you can tinker with that decision making process any way you want; there could be thousands of decisions for different aspects of the game being evaluated in parallel.
Also humans don't think quickly when compared to computers. Brain cells operate at a much lower frequency as a modern cpu and signal transmission between cells is much slower than on a cpu.
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
China already has a supercomputer that has about the same computing power as the estimated computing power of the human brain. In about 10-20 years this computing power will cost like a couple thousand dollars (this is just based on a continuation of moore's law, which has held true for a very long time).
On March 28 2016 17:04 mierin wrote: Isn't that the real issue here, though? Human brains are powerful BECAUSE they can think quickly, but the fact that we can think quickly means we're okay with being wrong if we come to a decision fast enough. The computer is on a totally different spectrum...it takes a long time to come to a conclusion, but in its mind it's always the move that has the highest probability of winning. The whole clash of fundamental ideals there is what makes watching AI vs. humans so interesting to me...really a 2nd race, if we could make the comparison.
You can easily code an AI to take say 1 second to come up with a bunch of potential moves and then pick the one that looks the most promising so far. By no means does it have to evaluate all possiblities or any such nonsense. Obviously you can tinker with that decision making process any way you want; there could be thousands of decisions for different aspects of the game being evaluated in parallel.
Also humans don't think quickly when compared to computers. Brain cells operate at a much lower frequency as a modern cpu and signal transmission between cells is much slower than on a cpu.
I think we're kind of defining 'quickly' differently. Either way I think we agree with each other.
Deepmind bot vs archon mode maru/innovation is obviously why they're using sc2 instead of bw. I'm interested in how they are going to expose game data to the bot e.g. will they teach it to recognize images drawn on screen (or some way to capture image data being passed through directx render, not necessarily a camera pointed on the screen), or will they directly interface with sc2 api calls? It's in the same vein as questions about how is apm controlled in that a human might see a frame and miss some of the details going on.
I'd love to see what strategies the bot decides on with different contraints such as max apm, playing different races, playing different maps too!
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
if it doesn't have an APM limit then it is pointless since it'll have perfect macro. And it also needs to be affected by fog of war (no map hacks like insane AI has).
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
The only way I see APM being limited would be if you did the following 1) Choose a designated 'esports keyboard' (some razer/steelseries/ect product) 2) Find the maximum amount of inputs the keyboard recognizes per minute 3) Make that the APM 'limit'
Each keyboard will stop responding after a certain number of inputs per second, and said input limit is probably much lower than the input limit that AlphaGo would have. At least I assume so.
You can register 1300APM by holding down a rapid fire key!
On March 28 2016 17:49 DinosaurPoop wrote: Why do we need an apm limit you guys seriously overestimate the power of perfect micro.
I don't think anyone is even close to overestimating perfect micro. If anything people are not giving enough thought to the impact perfect micro(even with limited apm) will have on strategy.
On March 28 2016 16:57 TheoMikkelsen wrote: Unless you limit AlphaGo it should be extremely easy to beat the best players in the world.
All you need is to make AlphaGo Terran, give it medivacs, stim, and the ability to micro each individual marine perfectly. Seriously it should be the easiest task ever.
Just make it like open reaper expand every game, then it only has to counter some cheeses. Go does not have mechanics, Starcraft 2 does, and there the AI would be infinitely superior.
The point is precisely NOT THAT. If it's about implementing an AI that will go for a perfectly micro-ed cheese everytime, well, the technology is already here
The point of "AlphaStar" is that there is no "making" involved. The system must learn by itself the importance of scouting, the usefulness of cheese and what have you, just like a human beginner would, if he only knew the goal (make the opponent leave the game or destroy every building) and had a huge database of replays to watch.
The choice of interface that would be used is essential. A keyboard, mouse and screen set up (even virtual) means that the AI has to move the camera to issue commands, and is limited by the max mouse sensibility of the engine, which would effectively limit the APM count.
There is also the question of how APM will be perceived by the system through its learning phase. We think of automaton2000 and its godlike APM, but a "self-taught" AI may very well end up playing with a relatively low APM. Or spam keystrokes just as much as the worst of human APM spammers.
The questions that we (the starcraft community) ask about this AI are questions that DeepMind's people will have to think about, and make choices according to what they want their AI to do. Food for thoughts: given the "right" replays, it's entirely possible that "AlphaStar" might try to make the opponent leave by BMing in chat. The BM to end all BM, now wouldn't that be a funny sight?
On March 28 2016 17:49 DinosaurPoop wrote: Why do we need an apm limit you guys seriously overestimate the power of perfect micro.
I don't think anyone is even close to overestimating perfect micro. If anything people are not giving enough thought to the impact perfect micro(even with limited apm) will have on strategy.
Yeah, it would make any worker rush situation where the attacker has more workers for a brief moment a reliable way to win
On March 28 2016 18:42 Theosiel wrote: Food for thoughts: given the "right" replays, it's entirely possible that "AlphaStar" might try to make the opponent leave by BMing in chat. The BM to end all BM, now wouldn't that be a funny sight?
Indeed. Alphago played moves that no human would play, moves that shocked the Go community and made them question their knowledge of the game. What kind of BM would a super-AI come up with? Be afraid.
On March 28 2016 17:54 teamamerica wrote: I'm interested in how they are going to expose game data to the bot e.g. will they teach it to recognize images drawn on screen
That's pretty much what these Google AIs have been doing all this time.
The choice of interface that would be used is essential. A keyboard, mouse and screen set up (even virtual) means that the AI has to move the camera to issue commands, and is limited by the max mouse sensibility of the engine, which would effectively limit the APM count.
That doesn't actually limit nearly as much as you would think
If the cap the APM to that of a pro player I do not think DeepMind has a chance.
Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
On March 28 2016 18:42 Theosiel wrote: Food for thoughts: given the "right" replays, it's entirely possible that "AlphaStar" might try to make the opponent leave by BMing in chat. The BM to end all BM, now wouldn't that be a funny sight?
Indeed. Alphago played moves that no human would play, moves that shocked the Go community and made them question their knowledge of the game. What kind of BM would a super-AI come up with? Be afraid.
All this would lead to is a squad of KeSPA referees running in and disqualifying the AI.
On March 28 2016 18:42 Theosiel wrote: Food for thoughts: given the "right" replays, it's entirely possible that "AlphaStar" might try to make the opponent leave by BMing in chat. The BM to end all BM, now wouldn't that be a funny sight?
Indeed. Alphago played moves that no human would play, moves that shocked the Go community and made them question their knowledge of the game. What kind of BM would a super-AI come up with? Be afraid.
All this would lead to is a squad of KeSPA referees running in and disqualifying the AI.
if he actually watches too many replays and learns to type more than glhf and gg then he definitely will get DQ'd by kespa
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
On March 28 2016 18:42 Theosiel wrote: Food for thoughts: given the "right" replays, it's entirely possible that "AlphaStar" might try to make the opponent leave by BMing in chat. The BM to end all BM, now wouldn't that be a funny sight?
Indeed. Alphago played moves that no human would play, moves that shocked the Go community and made them question their knowledge of the game. What kind of BM would a super-AI come up with? Be afraid.
All this would lead to is a squad of KeSPA referees running in and disqualifying the AI.
if he actually watches too many replays and learns to type more than glhf and gg then he definitely will get DQ'd by kespa
On March 28 2016 18:42 Theosiel wrote: Food for thoughts: given the "right" replays, it's entirely possible that "AlphaStar" might try to make the opponent leave by BMing in chat. The BM to end all BM, now wouldn't that be a funny sight?
Indeed. Alphago played moves that no human would play, moves that shocked the Go community and made them question their knowledge of the game. What kind of BM would a super-AI come up with? Be afraid.
All this would lead to is a squad of KeSPA referees running in and disqualifying the AI.
It would foresee this move of course, so it would start by BM-ing the referees out of the building.
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
you should ignore people like them, they actually believe starcraft is the most complex game ever
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
But it is obvious that no human could beat an AI with even subpar decisionmaking and some logic to handle abusive corner-cases if it had perfect micro, macro and map awareness, microing every single marine at ten places at once. That's not interesting. It is about as boring as installing a wall behind a tennis net and let it compete against a human player. We already know who wins, and why.
The interesting part is the question whether humans can construct an AI that OUTPLAYS a human player due to better decisionmaking when it has roughly the same limitations as a human player (i.e. not being able to see every screen at once, apm cap, etc.).
It is not about being fair or unfair, it is about creating interesting scenarios that actually test the wit of the AI. Testing whether a computer can have inhuman micro and macro is pointless, we know the answer.
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
China already has a supercomputer that has about the same computing power as the estimated computing power of the human brain. In about 10-20 years this computing power will cost like a couple thousand dollars (this is just based on a continuation of moore's law, which has held true for a very long time).
On March 28 2016 17:04 mierin wrote: Isn't that the real issue here, though? Human brains are powerful BECAUSE they can think quickly, but the fact that we can think quickly means we're okay with being wrong if we come to a decision fast enough. The computer is on a totally different spectrum...it takes a long time to come to a conclusion, but in its mind it's always the move that has the highest probability of winning. The whole clash of fundamental ideals there is what makes watching AI vs. humans so interesting to me...really a 2nd race, if we could make the comparison.
You can easily code an AI to take say 1 second to come up with a bunch of potential moves and then pick the one that looks the most promising so far. By no means does it have to evaluate all possiblities or any such nonsense. Obviously you can tinker with that decision making process any way you want; there could be thousands of decisions for different aspects of the game being evaluated in parallel.
Also humans don't think quickly when compared to computers. Brain cells operate at a much lower frequency as a modern cpu and signal transmission between cells is much slower than on a cpu.
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
Well put! Wanted to respond myself, but you said it better than I could've
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
BW AIs with no APM limits still can't beat decent humans, despite super muta micro and whatnot.
That's the inbuilt BW AI, which uses a tiny fraction of the processing power AlphaGo/Star uses. You can't even remotely compare the two.
I wonder if Blizzard is going to allow other people to develop bots for StarCraft 2 as well. If I were to create one right now it would get banned from battle.net
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
China already has a supercomputer that has about the same computing power as the estimated computing power of the human brain. In about 10-20 years this computing power will cost like a couple thousand dollars (this is just based on a continuation of moore's law, which has held true for a very long time).
On March 28 2016 17:04 mierin wrote: Isn't that the real issue here, though? Human brains are powerful BECAUSE they can think quickly, but the fact that we can think quickly means we're okay with being wrong if we come to a decision fast enough. The computer is on a totally different spectrum...it takes a long time to come to a conclusion, but in its mind it's always the move that has the highest probability of winning. The whole clash of fundamental ideals there is what makes watching AI vs. humans so interesting to me...really a 2nd race, if we could make the comparison.
You can easily code an AI to take say 1 second to come up with a bunch of potential moves and then pick the one that looks the most promising so far. By no means does it have to evaluate all possiblities or any such nonsense. Obviously you can tinker with that decision making process any way you want; there could be thousands of decisions for different aspects of the game being evaluated in parallel.
Also humans don't think quickly when compared to computers. Brain cells operate at a much lower frequency as a modern cpu and signal transmission between cells is much slower than on a cpu.
Moore's law is not holding true anymore.
It's still close enough to true; computing power still grows exponentially. And in any case companies like Google will have easy access to human brain level computing power within 10-20 years.
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
BW AIs with no APM limits still can't beat decent humans, despite super muta micro and whatnot.
That's the inbuilt BW AI, which uses a tiny fraction of the processing power AlphaGo/Star uses. You can't even remotely compare the two.
He's talking about third party AIs. Those are much more elaborate than the simple built in AIs.
On March 28 2016 15:07 BronzeKnee wrote: I can't see the human ever losing in my lifetime with that constraint, the human mind would be too fast, too perfect, too powerful.
China already has a supercomputer that has about the same computing power as the estimated computing power of the human brain. In about 10-20 years this computing power will cost like a couple thousand dollars (this is just based on a continuation of moore's law, which has held true for a very long time).
On March 28 2016 17:04 mierin wrote: Isn't that the real issue here, though? Human brains are powerful BECAUSE they can think quickly, but the fact that we can think quickly means we're okay with being wrong if we come to a decision fast enough. The computer is on a totally different spectrum...it takes a long time to come to a conclusion, but in its mind it's always the move that has the highest probability of winning. The whole clash of fundamental ideals there is what makes watching AI vs. humans so interesting to me...really a 2nd race, if we could make the comparison.
You can easily code an AI to take say 1 second to come up with a bunch of potential moves and then pick the one that looks the most promising so far. By no means does it have to evaluate all possiblities or any such nonsense. Obviously you can tinker with that decision making process any way you want; there could be thousands of decisions for different aspects of the game being evaluated in parallel.
Also humans don't think quickly when compared to computers. Brain cells operate at a much lower frequency as a modern cpu and signal transmission between cells is much slower than on a cpu.
Moore's law is not holding true anymore.
It's still close enough to true; computing power still grows exponentially. And in any case companies like Google will have easy access to human brain level computing power within 10-20 years.
On March 28 2016 14:22 ThunderBum wrote: I would like to see the ai learn without any APM restraints to see what kind of micro it learns is the best. Will it learn when it's best to stand and fight for a trade and when it's best to run away?
I think limiting the APM puts a limitation on the ai that it shouldn't have. The question shouldn't be whether or not a human can beat a handicapped ai, it should be wether ai can leverage all of it's advantages to beat humans just as how humans will leverage their advantages to beat the ai.
Except with no APM limitations you don't really need any strategy or intelligence to beat humans. Just program it with a few rushes that will almost always succeed due to perfect micro.
BW AIs with no APM limits still can't beat decent humans, despite super muta micro and whatnot.
That's the inbuilt BW AI, which uses a tiny fraction of the processing power AlphaGo/Star uses. You can't even remotely compare the two.
He's talking about third party AIs. Those are much more elaborate than the simple built in AIs.
Yet they still don't run on20 CPUs like AlphaGo did. You really can't compare them.
I still don't see the point. SC2 is a game designed for human vs human battles where mechanics play the largest role since everyone does roughly the same shit, how can you decide if you limit the APM in a fair way, or the interface? (camera hotkey and stuff)
On March 28 2016 21:24 InfCereal wrote: Would the AI even have unbelievable APM?
Wasn't it taking alphago minutes to decide moves in go?
If Starcraft is even a fraction as complex as go, wouldn't its apm actually be quite slow because it's trying to figure out how to win every game?
I think the point is that there's not much decision making going on when executing close to perfect micro - think Automaton 2000 Marines vs Banelings or Tank pickup micro. This micro can be pretty much hardcoded and the AI would be instructed to execute it whenever possible if it had unlimited APM.
Perfect micro for any given situation due to making actual decisions would be computationally expensive, but inhuman, extremely good general micro algorithms would be good enough.
On March 28 2016 19:15 MockHamill wrote: If the cap the APM to that of a pro player I do not think DeepMind has a chance.
Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
What are you even saying here? How many permutations do you think there are in the first 3 minutes of StarCraft? Are you including the co-ordinates of each of your units in this count? Be realistic. Sort board states into discrete groups and do another count.
Do you have ANY idea how many possible board states there are in a single match of Go?
Possible board states in Chess: 10^120 Possible board states in Go: 10^761
If they do this they will probably limit the APM of the machine to something like what the top Korean pros can do (maybe use Innovation as a guide?).
When the machine played Jeopardy they made it think and gave it a human reaction time to hitting the buzzer, otherwise it would have just destroyed everyone by buzzing immediately on every question and then thinking of the answer in the next second or two.
On March 28 2016 13:39 JimmyJRaynor wrote: without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
That was the prevailing stance before AlphaGo fought Lee Sedol... It didn't hold up to reality though.
who is the winner? the best Chess AI with heuristics or Alpha?
That's a weird way to defend a point...
Are they playing Chess, Go or StarCraft?
here is why Alpha will have a tough time without heuristics in a game like Chess or Starcraft
years from now .. who knows.. maybe no heuristics are required. but for now.. Alpha will have a tough time.
Did you even read the paper? You can't just skim the abstract for keywords and post it if the article doesn't actually support your stance.
This paper outlines an alpha-beta pruning method which can be used in small group combats. The only relevant text in the entire article is:
These advantages come at a cost, namely increased runtime and/or memory requirements. Therefore, it can be challenging to adapt established search algorithms to large scale realtime decision problems, e.g. video games and robotics.
I'm not sure if I'm missing your real point or how familiar you are with the internal workings of AlphaGo. AlphaGo uses a policy network to decide which moves to consider. This plays the same role as heuristic-based pruning. The difference between AlphaGo's and previous Go playing AIs is that AlphaGo's policy network does not rely on domain knowledge (ie. special rules about Go) explicitly programmed in by the developers. AlphaGo learned how to play Go using an initial seed of sample human games and then it was set to play against itself.
We should really hype this as a community and help it become a mainstream Machines vs Humans event (for the sake of gaining a bigger fanbase for SCII at least).
Also, if people understand the depth and layers of a complex game such as SCII, people would be more inclined on giving it a try, you know, people like challenges.
On March 28 2016 23:44 alukarD wrote: We should really hype this as a community and help it become a mainstream Machines vs Humans event (for the sake of gaining a bigger fanbase for SCII at least).
Also, if people understand the depth and layers of a complex game such as SCII, people would be more inclined on giving it a try, you know, people like challenges.
This is years away from happening, if it does happen.
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
You miss the point.
Number of permutations is indeed important, in fact it's the most important factor when it comes to making a self learning AI. You still have to actually use the inputs of a game in order to train the gaming algorithm. The data set is way larger, and the task of purely training a neural network on this is highly intractable. The dimensionality of the space of all possible moves for starcraft is orders, and orders of magnitude larger than that of Go. For instance, in Go, at every turn you can move one unit. You have N units. Thus each turn, you have N possible choices of which units to move. In Starcraft, if you have N units, there are 2^N possible choices of which units to select. This includes buildings. In Go, the average number of turns is in the tens. In Starcraft there's 24 frames per second, and each frame has a possible set of actions. Go is a 19x19 board, Starcraft is a thousands by thousands board.
With Go, it is reasonable (though probably still extremely computationally expensive) to set your dimensionality of the data set as all possible moves, then take games as a data set, and just straightforwardly train a neural network to that. I'm sure the Alpha Go team was more sophisticated than that, and took a crazy amount of clever shortcuts. But most of the cleverness there came from finding ways to conserve computational resource, not from trying to think of the deep strategies. How deep the strategies are in Go had virtually nothing to do with the difficulty of their task. Starcraft will be insurmountably harder to train than Go, regardless of whether or not its strategies are deeper (and I agree with you, Go is much deeper in this regard). Like I said, with Go, even though the team was more clever, it is feasible to brute force train a NN on the exact moves. With Starcraft that isn't even remotely feasible. Not with the greatest super computers in the world. The number of possible moves scales exponentially with the number of units you have, and again exponentially with the number of frames the game takes. They'll have to take extremely different and far more clever strategies in training a SC2 AI, and it will be a hell of a lot more difficult.
I think a lot of people here are thinking too much about this as if it were like teaching a human. If that were so, then yes, Go would be a lot harder to teach the strategies to. But the problem of taking the best strategy when you have in mind the set of all strategies you can play is pretty much solved in machine learning already. Any maximum likelihood estimator can make a better decision than pretty much any human, as long as it has the right inputs. Giving it the set of strategies it can use is the real problem, and much, much harder to train for an AI than for a human. And ridiculously harder to train for Starcraft than for Go.
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
You miss the point.
Number of permutations is indeed important, in fact it's the most important factor when it comes to making a self learning AI. You still have to actually use the inputs of a game in order to train the gaming algorithm. The data set is way larger, and the task of purely training a neural network on this is highly intractable. The dimensionality of the space of all possible moves for starcraft is orders, and orders of magnitude larger than that of Go. For instance, in Go, at every turn you can move one unit. You have N units. Thus each turn, you have N possible choices of which units to move. In Starcraft, if you have N units, there are 2^N possible choices of which units to select. This includes buildings. In Go, the average number of turns is in the tens. In Starcraft there's 24 frames per second, and each frame has a possible set of actions. Go is a 19x19 board, Starcraft is a thousands by thousands board.
With Go, it is reasonable (though probably still extremely computationally expensive) to set your dimensionality of the data set as all possible moves, then take games as a data set, and just straightforwardly train a neural network to that. I'm sure the Alpha Go team was more sophisticated than that, and took a crazy amount of clever shortcuts. But most of the cleverness there came from finding ways to conserve computational resource, not from trying to think of the deep strategies. How deep the strategies are in Go had virtually nothing to do with the difficulty of their task. Starcraft will be insurmountably harder to train than Go, regardless of whether or not its strategies are deeper (and I agree with you, Go is much deeper in this regard). Like I said, with Go, even though the team was more clever, it is feasible to brute force train a NN on the exact moves. With Starcraft that isn't even remotely feasible. Not with the greatest super computers in the world. The number of possible moves scales exponentially with the number of units you have, and again exponentially with the number of frames the game takes. They'll have to take extremely different and far more clever strategies in training a SC2 AI, and it will be a hell of a lot more difficult.
Though I agree with your basic thoughts, I have to point out that it is NOT POSSIBLE to brute force solve the game of Go. There are 10^761 possible board states. That is a number beyond our capability to imagine. With current technologies it is impossible to store the entire game tree.
Starcraft 2 doesn't even deserve to have Starcraft in its name. I'd be surprised if Deepmind didn't get the better of the pros, and it will be a black eye for this game. This game used to be chess like and strategical. Now it's a game with 0 options that lives on the past of BW. Should have strategy stripped from the genre if SC 2 pros lose. Deepmind should challenge pros in Z vs P. I wonder how long it would take Deepmind to learn that every opponent will go phoenix into mass immortals.
A true test of AI here, ladies in gents. Is tac tac toe next? Look out world. My only question is will the AI bm the creators of this boring fraudulent strategy game. I'd like to see Deepmind BM someone for wasting its time.
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
You miss the point.
Number of permutations is indeed important, in fact it's the most important factor when it comes to making a self learning AI. You still have to actually use the inputs of a game in order to train the gaming algorithm. The data set is way larger, and the task of purely training a neural network on this is highly intractable. The dimensionality of the space of all possible moves for starcraft is orders, and orders of magnitude larger than that of Go. For instance, in Go, at every turn you can move one unit. You have N units. Thus each turn, you have N possible choices of which units to move. In Starcraft, if you have N units, there are 2^N possible choices of which units to select. This includes buildings. In Go, the average number of turns is in the tens. In Starcraft there's 24 frames per second, and each frame has a possible set of actions. Go is a 19x19 board, Starcraft is a thousands by thousands board.
With Go, it is reasonable (though probably still extremely computationally expensive) to set your dimensionality of the data set as all possible moves, then take games as a data set, and just straightforwardly train a neural network to that. I'm sure the Alpha Go team was more sophisticated than that, and took a crazy amount of clever shortcuts. But most of the cleverness there came from finding ways to conserve computational resource, not from trying to think of the deep strategies. How deep the strategies are in Go had virtually nothing to do with the difficulty of their task. Starcraft will be insurmountably harder to train than Go, regardless of whether or not its strategies are deeper (and I agree with you, Go is much deeper in this regard). Like I said, with Go, even though the team was more clever, it is feasible to brute force train a NN on the exact moves. With Starcraft that isn't even remotely feasible. Not with the greatest super computers in the world. The number of possible moves scales exponentially with the number of units you have, and again exponentially with the number of frames the game takes. They'll have to take extremely different and far more clever strategies in training a SC2 AI, and it will be a hell of a lot more difficult.
Though I agree with your basic thoughts, I have to point out that it is NOT POSSIBLE to brute force solve the game of Go. There are 10^761 possible board states. That is a number beyond our capability to imagine. With current technologies it is impossible to store the entire game tree.
The number of board states is irrelevant. It's the number of possible moves you can make, given a board state, that's relevant. That's all you really need to give for a training AI. My issue is the dimensionality of a training data set of moves. The number of dimensions of that set is the number of possible moves. For Go, with a given board state, it's actually not that much. (In the hundreds, I'd say.) For Starcraft, it's absurdly high. Think of the easiest machine learning problem (training a linear model in one dimension). The set of possible parameters is infinite, but there are only two parameters, and thus it's really quick and easy to train.
On March 29 2016 00:05 playa wrote: Starcraft 2 doesn't even deserve to have Starcraft in its name. I'd be surprised if Deepmind didn't get the better of the pros, and it will be a black eye for this game. This game used to be chess like and strategical. Now it's a game with 0 options that lives on the past of BW. Should have strategy stripped from the genre if SC 2 pros lose. Deepmind should challenge pros in Z vs P. I wonder how long it would take Deepmind to learn that every opponent will go phoenix into mass immortals.
A true test of AI here, ladies in gents. Is tac tac toe next? Look out world. My only question is will the AI bm the creators of this boring fraudulent strategy game. I'd like to see Deepmind BM someone for wasting its time.
I agree completely with you! The same happened with chess and now go, these games aren't worthy of the title "strategy game" when an AI can beat humans. I heard the first go players are already retiring and starting to play bw because they read the bw forums of tl, they came to the conclusion (based on the intelligent posts there) that bw is the epitome of strategic/tactical complexity. Preach it!
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
You miss the point.
Number of permutations is indeed important, in fact it's the most important factor when it comes to making a self learning AI. You still have to actually use the inputs of a game in order to train the gaming algorithm. The data set is way larger, and the task of purely training a neural network on this is highly intractable. The dimensionality of the space of all possible moves for starcraft is orders, and orders of magnitude larger than that of Go. For instance, in Go, at every turn you can move one unit. You have N units. Thus each turn, you have N possible choices of which units to move. In Starcraft, if you have N units, there are 2^N possible choices of which units to select. This includes buildings. In Go, the average number of turns is in the tens. In Starcraft there's 24 frames per second, and each frame has a possible set of actions. Go is a 19x19 board, Starcraft is a thousands by thousands board.
With Go, it is reasonable (though probably still extremely computationally expensive) to set your dimensionality of the data set as all possible moves, then take games as a data set, and just straightforwardly train a neural network to that. I'm sure the Alpha Go team was more sophisticated than that, and took a crazy amount of clever shortcuts. But most of the cleverness there came from finding ways to conserve computational resource, not from trying to think of the deep strategies. How deep the strategies are in Go had virtually nothing to do with the difficulty of their task. Starcraft will be insurmountably harder to train than Go, regardless of whether or not its strategies are deeper (and I agree with you, Go is much deeper in this regard). Like I said, with Go, even though the team was more clever, it is feasible to brute force train a NN on the exact moves. With Starcraft that isn't even remotely feasible. Not with the greatest super computers in the world. The number of possible moves scales exponentially with the number of units you have, and again exponentially with the number of frames the game takes. They'll have to take extremely different and far more clever strategies in training a SC2 AI, and it will be a hell of a lot more difficult.
Though I agree with your basic thoughts, I have to point out that it is NOT POSSIBLE to brute force solve the game of Go. There are 10^761 possible board states. That is a number beyond our capability to imagine. With current technologies it is impossible to store the entire game tree.
The number of board states is irrelevant. It's the number of possible moves you can make, given a board state, that's relevant. That's all you really need to give for a training AI. My issue is the dimensionality of a training data set of moves. The number of dimensions of that set is the number of possible moves. For Go, with a given board state, it's actually not that much. (In the hundreds, I'd say.) For Starcraft, it's absurdly high. Think of the easiest machine learning problem (training a linear model in one dimension). The set of possible parameters is infinite, but there are only two parameters, and thus it's really quick and easy to train.
Upon re-reading your previous post, I see that you were in fact discussing possible moves, not possible board states.
On March 28 2016 23:50 The Bottle wrote: Number of permutations is indeed important, in fact it's the most important factor when it comes to making a self learning AI. You still have to actually use the inputs of a game in order to train the gaming algorithm. The data set is way larger, and the task of purely training a neural network on this is highly intractable.
The problem is only more complex if you count every possible game state as a "possible move". That's a ridiculous approach to the problem, in the same way as it's ridiculous trying to do it in billiard, which you didn't comment. I have no notion that Starcraft is an easy problem for an AI, or even easier than Go.
This is a different problem, in the same way that Go is a different problem than chess, and that's no doubt why DeepMind think it's interesting. Starcraft is much more similar to real world problems than Go. The number of possible moves are for all practical purposes infinite, and there is no point in putting a number on it or saying that it has anything to do with complexity or how hard it is. It's a meaningless number. If we doubled the precision for all game state variables, the number of possible game states would increase dramatically. Does that mean that the problem becomes much harder? No. It's exactly the same problem. The infinite state space makes it belong to a different class of problems, an unsolved one. I'll not argue that.
On March 28 2016 23:50 The Bottle wrote: Number of permutations is indeed important, in fact it's the most important factor when it comes to making a self learning AI. You still have to actually use the inputs of a game in order to train the gaming algorithm. The data set is way larger, and the task of purely training a neural network on this is highly intractable.
The problem is only more complex if you count every possible game state as a "possible move". That's a ridiculous approach to the problem, in the same way as it's ridiculous trying to do it in billiard, which you didn't comment. I have no notion that Starcraft is an easy problem for an AI, or even easier than Go.
This is a different problem, in the same way that Go is a different problem than chess, and that's no doubt why DeepMind think it's interesting. Starcraft is much more similar to real world problems than Go. The number of possible moves are for all practical purposes infinite, and there is no point in putting a number on it or saying that it has anything to do with complexity or how hard it is. It's a meaningless number. If we doubled the precision for all game state variables, the number of possible game states would increase dramatically. Does that mean that the problem becomes much harder? No. It's exactly the same problem. The infinite state space makes it belong to a different class of problems, an unsolved one. I'l not argue that.
I'm not talking about the number of possible game states. That's not important for a machine learning algorithm. Number of possible moves you can make is important. That is, the way you encode a particular action. This is essential for making a training data set for your algorithm to learn.
Your billiard example doesn't work here, because there's no self-learning AI algorithm for billiard, at least not that I know of. (If there is, let me know.) The ones that exist simply do an on-the-spot calculation for each move. And while it is intractable to consider all possible moves, it takes a clever shortcut (one example is taking a large sample of N moves, properly separated in the angle-power space by use of k-means clustering, and picking the best one). It's scripting, similar to what Starcraft AI does now. The other problem with the billiard example is that each move can be completely encoded by three continuous variables (position along the rectangle, angle of shot, and power). Even though they're continuous, and have an infinite set of values, encoding this into a data set of moves for a given game is extremely easy. It's a Nx3 matrix of values, where N is the number of turns and 3 is the dimensions of moves. (Although given the nature of billiard, I'm not sure self learning would be any better than scripting; probably worse, in fact.) There's no analogous simple reduction like that in Starcraft.
The reason self learning is so much harder than scripting, is because you have to be so much more meticulous about giving your algorithm the necessary degrees of freedom in the former case. Because you can't feed it any particular strategy, it has to be able to infer those strategies from knowing the moves it can make, and using historic data to calculate which combination of moves maximizes the probability of success. They will have to find clever ways to transform the game input data in order to remove redundancies, and coarsen the scale of discrete moves. I'm sure they did something like that with Go already, but it will be substantially harder for SC2. You say it's just a different problem, sure. But a much, much harder problem, one I'm not quite sure they'll solve, even knowing their success with Go.
Why is everyone discussing APM, shouldn't EAPM be a more suitable metric. I doubt the AI will spam keys so even if you cap APM it will still outmatch a human easy. Because 500 APM from a human could be 200 EAPM while 500 APM from an AI could be actually be 500 EAPM and thus the AI are allowed double the actual actions than a human.
One aspect that people also misunderstand in comparing Go and SC2 is that of input/output.
For Go, it is relatively simple. The input is the game board at a given game state, which is all the current pieces in their positions. The output is a position on the board that corresponds to the move done once per turn. The cycle goes back and forth until the game is resolved. This means that the AI and the human have exactly the same input/output, and the game is 100% strategy.
For SC2, will that same stipulation hold? Will they require the AI to use the same input/output as a human being? I believe they should, otherwise it's too big an advantage for the AI and it will no longer be a test of strategy, but rather the efficiency of input/output that they are using.
First, the input. There's the screen, minimap, control groups, unit selection, command card, supplies. That's a lot of information that the AI must learn to interpret. They aren't going to program the AI what these things mean and how to use them, the AI has to learn that on its own. There's also the limitation that the AI can only look at one sector of the map at any given time. It has to learn to how to do that as well.
For output, it's just as difficult. How would they program the AI so that its outputs mimic that of a human being? I don't even know how they'd do that. Limiting APM is one way, but that's an artificial cap and I don't know if that's going to be effective. This is a problem they have to figure out and I doubt anyone here has an answer. The way they output has to be as similar as possible. Even then a machine will still have the advantage of 100% accuracy (perfect force fields, perfectly timed casting, perfect micro given APM restrictions).
The AI has to learn this input/output system on its own, from scratch. There is a lot more to consider compared to Go. This obstacle alone would be an amazing one to solve.
This is an important part of the discussion because the purpose of this test is to show advances in AI learning through strategy games. Unless the game is as close to 100% strategy as possible like Go, it's pointless to play.
On March 28 2016 19:15 MockHamill wrote: Starcraft is so much more complex than Go it is absurd. Basically the amount of permutations in the first 3 minutes of Starcraft is larger then any match of Go.
As someone who has played chess (no rank), Go (10 kyu) and Starcraft (platinum) at low amateur level I say you are wrong. At the strategic level Go is enormously more complex and deep than both chess and Starcraft. There is absolutely no comparison. Go completely dwarfs any game that I have come in contact with. Measuring complexity in "number of permutations" is not meaningful. How many permutations are there in a game of billiard? Do you think it would be hard for a computer?
Thank you for this post. It's so annoying to see so many people saying that Go is easy.
The reason why Go was such an interesting game for AI to tackle is precisely that it has so many possible moves that backtracking sucks too much.
On March 29 2016 02:08 lichter wrote: One aspect that people also misunderstand in comparing Go and SC2 is that of input/output.
For Go, it is relatively simple. The input is the game board at a given game state, which is all the current pieces in their positions. The output is a position on the board that corresponds to the move done once per turn. The cycle goes back and forth until the game is resolved. This means that the AI and the human have exactly the same input/output, and the game is 100% strategy.
For SC2, will that same stipulation hold? Will they require the AI to use the same input/output as a human being? I believe they should, otherwise it's too big an advantage for the AI and it will no longer be a test of strategy, but rather the efficiency of input/output that they are using.
First, the input. There's the screen, minimap, control groups, unit selection, command card, supplies. That's a lot of information that the AI must learn to interpret. They aren't going to program the AI what these things mean and how to use them, the AI has to learn that on its own. There's also the limitation that the AI can only look at one sector of the map at any given time. It has to learn to how to do that as well.
For output, it's just as difficult. How would they program the AI so that its outputs mimic that of a human being? I don't even know how they'd do that. Limiting APM is one way, but that's an artificial cap and I don't know if that's going to be effective. This is a problem they have to figure out and I doubt anyone here has an answer. The way they output has to be as similar as possible. Even then a machine will still have the advantage of 100% accuracy (perfect force fields, perfectly timed casting, perfect micro given APM restrictions).
The AI has to learn this input/output system on its own, from scratch. There is a lot more to consider compared to Go. This obstacle alone would be an amazing one to solve.
This is an important part of the discussion because the purpose of this test is to show advances in AI learning through strategy games. Unless the game is as close to 100% strategy as possible like Go, it's pointless to play.
I would bet they won't even try to solve that. It's besides the point. They will probably just use the API and have the AI access unit positioning directly.
On March 29 2016 02:08 lichter wrote: One aspect that people also misunderstand in comparing Go and SC2 is that of input/output.
For Go, it is relatively simple. The input is the game board at a given game state, which is all the current pieces in their positions. The output is a position on the board that corresponds to the move done once per turn. The cycle goes back and forth until the game is resolved. This means that the AI and the human have exactly the same input/output, and the game is 100% strategy.
For SC2, will that same stipulation hold? Will they require the AI to use the same input/output as a human being? I believe they should, otherwise it's too big an advantage for the AI and it will no longer be a test of strategy, but rather the efficiency of input/output that they are using.
First, the input. There's the screen, minimap, control groups, unit selection, command card, supplies. That's a lot of information that the AI must learn to interpret. They aren't going to program the AI what these things mean and how to use them, the AI has to learn that on its own. There's also the limitation that the AI can only look at one sector of the map at any given time. It has to learn to how to do that as well.
For output, it's just as difficult. How would they program the AI so that its outputs mimic that of a human being? I don't even know how they'd do that. Limiting APM is one way, but that's an artificial cap and I don't know if that's going to be effective. This is a problem they have to figure out and I doubt anyone here has an answer. The way they output has to be as similar as possible. Even then a machine will still have the advantage of 100% accuracy (perfect force fields, perfectly timed casting, perfect micro given APM restrictions).
The AI has to learn this input/output system on its own, from scratch. There is a lot more to consider compared to Go. This obstacle alone would be an amazing one to solve.
This is an important part of the discussion because the purpose of this test is to show advances in AI learning through strategy games. Unless the game is as close to 100% strategy as possible like Go, it's pointless to play.
I would bet they won't even try to solve that. It's besides the point. They will probably just use the API and have the AI access unit positioning directly.
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
On March 29 2016 01:33 The Bottle wrote: I'm not talking about the number of possible game states. That's not important for a machine learning algorithm. Number of possible moves you can make is important. That is, the way you encode a particular action. This is essential for making a training data set for your algorithm to learn.
Why do you assume that the best way to solve this problem is to throw every single game state variable or pixel at a neural net and then hope that it somehow works out?
Your billiard example doesn't work here, because there's no self-learning AI algorithm for billiard, at least not that I know of.
Then imagine one. One that learns for example by self play. Do you really think that it would have a hard time finding "the right moves" just because they are infinite in number? Edit: And the question is not if it can be better than a script, which can be near perfect. The question is if you think it's a hard problem.
They will have to find clever ways to transform the game input data in order to remove redundancies, and coarsen the scale of discrete moves. I'm sure they did something like that with Go already, but it will be substantially harder for SC2. You say it's just a different problem, sure. But a much, much harder problem, one I'm not quite sure they'll solve, even knowing their success with Go.
Yes, THIS is the problem, and once you have done this the "number of possible moves" in the original problem is irrelevant. That only tells you that you have THIS problem on your hands and that you can't solve it by ordinary search algorithms. You have to find a way to reduce the original problem by levels of abstraction. I don't know if there is a way for an AI find these abstractions by itself. Maybe that's how Alphago works, In any case, I'm not in the least convinced that it's as hard as you are trying to make it sound. In the simplest form you can have an ordinary scripted bot that asks an AI for advice. "Attack now", "build sentries", "expand there" etc. Or you could throw every pixel at it, like you want. I don't think that would work. Or something in between. How about that?
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
I don't think an apm cap would be completely arbitrary. Somewhere around the average apm of top pros or maybe a little higher I think is reasonable.
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
And if you don't allow the machine to have arbitrary APM, what APM will you allow? The choice is completely arbitrary.
I don't think an apm cap would be completely arbitrary. Somewhere around the average apm of top pros or maybe a little higher I think is reasonable.
EPM not APM.
Even EPM is "inflated" since many of the actions everyone does are tiny corrections on earlier clicks (due to misclicks and whatnot) which a computer wouldn't need.
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
Yeah, but SC2 requires a little more creativity than GO or Chess...I think eventually an AI could be better of course, but I don't think we're anywhere near that point yet. There aren't predetermined tiles or spots for pieces to move...SC2 units don't have simple rules, and there are almost infinite possibilities. I suppose you could teach an AI to macro like an animal and then A-move over someone, or maybe even attempt to setup concaves. But man, there are a lot more factors involved in winning an SC2 engagement than taking someone's rook in chess. And while I don't know the rules of Euro.Go, it looks like you have one type of unit and a grid to move on... No setting up army compositions, or gaining vision of your opponent, etc.
Maybe I'll be surprised as to how far AIs have come, but methinks they will be mechanically perfect but fairly retarded in many ways.
perfect macro and perfect micro will solve a lot of the 'strategical' issues by simply winning fights that, as humans, we believe you should lose. Easiest is TvZ, off creep perfect micro means you shouldn't lose any marines to banelings as long as you have stim.
On March 29 2016 02:58 Pseudorandom wrote: perfect macro and perfect micro will solve a lot of the 'strategical' issues by simply winning fights that, as humans, we believe you should lose. Easiest is TvZ, off creep perfect micro means you shouldn't lose any marines to banelings as long as you have stim.
Yeah I think it's possible that deepmind wins solely off of micro.
On March 29 2016 02:58 Pseudorandom wrote: perfect macro and perfect micro will solve a lot of the 'strategical' issues by simply winning fights that, as humans, we believe you should lose. Easiest is TvZ, off creep perfect micro means you shouldn't lose any marines to banelings as long as you have stim.
Yeah I think it's possible that deepmind wins solely off of micro.
Unless they implement an APM limiter or something
And all the zerg human hero needs to do is force a bunch of wasted stims and threaten to counter
On March 29 2016 02:58 Pseudorandom wrote: perfect macro and perfect micro will solve a lot of the 'strategical' issues by simply winning fights that, as humans, we believe you should lose. Easiest is TvZ, off creep perfect micro means you shouldn't lose any marines to banelings as long as you have stim.
Yeah I think it's possible that deepmind wins solely off of micro.
Unless they implement an APM limiter or something
And all the zerg human hero needs to do is force a bunch of wasted stims and threaten to counter
Deepmind will just send over his starting SCVs and win a 12 SCV vs 15 Drones fight.
For people wondering if blizzard will "tolerate" Google developing this since it is like a 3rd party program / mod, you don't realize that there isn't a single company that will spit on the amount of exposure a company like google can bring to you. A good AlphaSC2 would save Blizzard millions of $ in advertisement and will probably bring more players on the scene.
Oh and also, an AI cannot waste stims. You can program it so that it only stims when it knows it can reach yours units for sure even if they run away, and there is no turning back on individually controlled stimmed units lol.
The AI won't need a fancy late game strategy to win, simple rush builds with perfect micro should be enough to defeat humans. Doing the same in a long macro game would be a lot more difficult
On March 29 2016 01:33 The Bottle wrote: I'm not talking about the number of possible game states. That's not important for a machine learning algorithm. Number of possible moves you can make is important. That is, the way you encode a particular action. This is essential for making a training data set for your algorithm to learn.
Why do you assume that the best way to solve this problem is to throw every single game state variable or pixel at a neural net and then hope that it somehow works out?
Your billiard example doesn't work here, because there's no self-learning AI algorithm for billiard, at least not that I know of.
Then imagine one. One that learns for example by self play. Do you really think that it would have a hard time finding "the right moves" just because they are infinite in number? Edit: And the question is not if it can be better than a script, which can be near perfect. The question is if you think it's a hard problem.
They will have to find clever ways to transform the game input data in order to remove redundancies, and coarsen the scale of discrete moves. I'm sure they did something like that with Go already, but it will be substantially harder for SC2. You say it's just a different problem, sure. But a much, much harder problem, one I'm not quite sure they'll solve, even knowing their success with Go.
Yes, THIS is the problem, and once you have done this the "number of possible moves" in the original problem is irrelevant. That only tells you that you have THIS problem on your hands and that you can't solve it by ordinary search algorithms. You have to find a way to reduce the original problem by levels of abstraction. I don't know if there is a way for an AI find these abstractions by itself. Maybe that's how Alphago works, In any case, I'm not in the least convinced that it's as hard as you are trying to make it sound. In the simplest form you can have an ordinary scripted bot that asks an AI for advice. "Attack now", "build sentries", "expand there" etc. Or you could throw every pixel at it, like you want. I don't think that would work. Or something in between. How about that?
What you explained at the beginning is actually similar to how they train the initial state of the Go algorithm, before they get into the reinforced learning. It's not sufficient to create the algorithm as intelligent as it is, but that is what they do in their initial stages.
To clarify, they don't feed all possible permutations of the board, because that's obviously intractable. But they do feed it a large set of board positions from many games, with a target variable of which side won the game, and train a neural network on that. The input data for that is not hard to encode at all. It's simply a data set of 361 trinary points (black, white, or blank) and a binary target variable (which side won). Then in practice, given a board state, you calculate the probability of victory of all possible subsequent board states (all single turn moves you can make from the given state) using the NN trained by the above process. Such a method was used as the initial stage of the Go algorithm, as explained in this paper https://vk.com/doc-44016343_437229031?dl=56ce06e325d42fbc72 before they started the reinforced learning, but this would be impossible to do for Starcraft. (I mean this particular method, not any supervised learning method.)
But listen, because I think you're still misunderstanding me. I know that they're taking shortcuts to greatly reduce the search space of possible moves in Go, the paper states this pretty clearly. But the problem is still that, because of the sheer number of possible moves you can make, and the stark difference in outcome between those moves, it is incredibly difficult to reduce the space in an intelligent enough way to minimally reduce the information of best moves possible. The more complex the game is (and I mean this in terms of permutations of moves, not in terms of heuristic strategy), the harder this task is. Thus it will be monumentally difficult for Starcraft. Yes, they will have to take new shortcuts, of the sort that they didn't take in Go. But every time they do such a thing, they have to be incredibly careful not to remove certain specific crucial moves, or coarse grain it in with other moves that have drastically different results. (An example of this is in zvz ling bane wars, where a couple pixels different motion can be the difference between 2 dead lings and 20 dead lings, or aiming a disruptor shot, or things like that).
I should clarify, I don't think this task is impossible. For sure, in principle it's very feasible to imagine a self trained Starcraft algorithm that can beat any human. But I'm trying to explain why it's monumentally more difficult than training a Go algorithm. And why the actual difference in depth of strategy between the two games from a heuristic standpoint is not nearly as important as the complexity of move permutations. You say it's a different problem. Well it's a different problem in the same sense that doing long division and proving Fermat's Last Theorem are different problems.
As for the billiard example. I did explain how the training set of a deep learning NN algorithm of billiard can be encoded, and why it's incredibly easy to do this in comparison to the other problems. I can clarify, but from your response I feel like you didn't read that bit. In your defense, it was sneakily put in beside my other point, so maybe I'll let it sit a little longer.
On March 29 2016 03:49 The Bottle wrote: The more complex the game is (and I mean this in terms of permutations of moves, not in terms of heuristic strategy), the harder this task is. Thus it will be monumentally difficult for Starcraft.
And I'm saying that you're just making things up. An intractable large number of possible moves (or number of input variables) doesn't necessarily mean that the problem is hard (although it is a requirement) and reducing Starcraft to a problem on a higher level of abstraction than pixels or coordinates isn't necessarily very hard either. At least you haven't showed any arguments for it. Once you have moved to a high abstraction level Starcraft IS simple compared to Go, which cannot be reduced to builds or strategies in any similar way. This is the reason why I think it's at least possible that Starcraft is even easier to master than Go for an AI.
it is incredibly difficult to reduce the space in an intelligent enough way to minimally reduce the information of best moves possible
We are not trying to find the best moves possible. We are trying to beat the world champion, or someone similar. You are again making this harder than it is.
On March 28 2016 12:53 Circumstance wrote: The real-time aspect will be critical. Go, being turn-baseed, is purely a matchup of mind against "mind". If They allow the machine unlimited APM, then this won't be much of a match.
Also, you'd need representatives from all 3 races, wouldn't you?
Yeah, but SC2 requires a little more creativity than GO or Chess...I think eventually an AI could be better of course, but I don't think we're anywhere near that point yet. There aren't predetermined tiles or spots for pieces to move...SC2 units don't have simple rules, and there are almost infinite possibilities. I suppose you could teach an AI to macro like an animal and then A-move over someone, or maybe even attempt to setup concaves. But man, there are a lot more factors involved in winning an SC2 engagement than taking someone's rook in chess. And while I don't know the rules of Euro.Go, it looks like you have one type of unit and a grid to move on... No setting up army compositions, or gaining vision of your opponent, etc.
Maybe I'll be surprised as to how far AIs have come, but methinks they will be mechanically perfect but fairly retarded in many ways.
It seems like there is a huge divide in this thread between the people who know Go and those who don't...
On March 29 2016 03:49 The Bottle wrote: But listen, because I think you're still misunderstanding me. I know that they're taking shortcuts to greatly reduce the search space of possible moves in Go, the paper states this pretty clearly. But the problem is still that, because of the sheer number of possible moves you can make, and the stark difference in outcome between those moves, it is incredibly difficult to reduce the space in an intelligent enough way to minimally reduce the information of best moves possible. The more complex the game is (and I mean this in terms of permutations of moves, not in terms of heuristic strategy), the harder this task is. Thus it will be monumentally difficult for Starcraft. Yes, they will have to take new shortcuts, of the sort that they didn't take in Go. But every time they do such a thing, they have to be incredibly careful not to remove certain specific crucial moves, or coarse grain it in with other moves that have drastically different results. (An example of this is in zvz ling bane wars, where a couple pixels different motion can be the difference between 2 dead lings and 20 dead lings, or aiming a disruptor shot, or things like that).
Though I think you have an understanding of the workings of AlphaGo beyond the average layperson, I just want to point out that some of the language you use is incorrect. You seem to imply that DeepMind will influence AlphaGo's decisions which is incorrect.
Beyond a ladder (a large board-scale trap) calculator, the DeepMind team did not provide AlphaGo any hints, tips or tricks. Through many iterations, AlphaGo learned to place stronger weights on specific paths to search.
On March 29 2016 04:48 kingjames01 wrote: Beyond a ladder (a large board-scale trap) calculator, the DeepMind team did not provide AlphaGo any hints, tips or tricks. Through many iterations, AlphaGo learned to place stronger weights on specific paths to search.
I'm going off a slight tangent here, but someone on the DeepMind team spoke of a possible future development, and that would be doing Alphago again but without the first training step of the neural nets with lots of human games. While they didn't teach it any specific tricks, these games may have taught it bad habits. Go is actually a very little researched game, which Go Seigen proved in the middle of the last century by turning everything upside down. I would very VERY much want them to do this instead of trying Starcraft. It could have vast implications for our understanding of Go. Maybe the best starting move is right in the middle?
On March 29 2016 04:48 kingjames01 wrote: Beyond a ladder (a large board-scale trap) calculator, the DeepMind team did not provide AlphaGo any hints, tips or tricks. Through many iterations, AlphaGo learned to place stronger weights on specific paths to search.
I'm going off a slight tangent here, but someone on the DeepMind team spoke of a possible future development, and that would be doing Alphago again but without the first training step of the neural nets with lots of human games. While they didn't teach it any specific tricks, these games may have taught it bad habits. Go is actually a very little researched game, which Go Seigen proved in the middle of the last century by turning everything upside down. I would very VERY much want them to do this instead of trying Starcraft. It could have vast implications for our understanding of Go. Maybe the best starting move is right in the middle?
The issue with removing the first 'supervised' learning process for the policy network - namely, training AlphaGo without copying human moves at the very first - is that it might then take months and months before converging. Arguably policies learned them might be stronger, but it might be that networks take too long, or simply fail to converge. So, that approach is dependent on future progress in unsupervised machine learning methods.
On March 29 2016 03:49 The Bottle wrote: The more complex the game is (and I mean this in terms of permutations of moves, not in terms of heuristic strategy), the harder this task is. Thus it will be monumentally difficult for Starcraft.
And I'm saying that you're just making things up. An intractable large number of possible moves (or number of input variables) doesn't necessarily mean that the problem is hard (although it is a requirement) and reducing Starcraft to a problem on a higher level of abstraction than pixels or coordinates isn't necessarily very hard either. At least you haven't showed any arguments for it. Once you have moved to a high abstraction level Starcraft IS simple compared to Go, which cannot be reduced to builds or strategies in any similar way. This is the reason why I think it's at least possible that Starcraft is even easier to master than Go for an AI.
it is incredibly difficult to reduce the space in an intelligent enough way to minimally reduce the information of best moves possible
We are not trying to find the best moves possible. We are trying to beat the world champion, or someone similar. You are again making this harder than it is.
You said "reducing Starcraft to a problem on a higher level of abstraction than pixels or coordinates isn't necessarily very hard either". For scripting that's true. For supervised and reinforced learning... well... I really think you have to work a lot harder to justify this statement. Unless you think scripting is the way that they're going to beat a pro player, in which case... well, I agree with all of Boxer's objections to that.
I can try and explain why moving Starcraft to a high level of abstraction to make a training set for a learning algorithm is incredibly difficult, but it's hard to do so without explicitly explaining how a neural network or Monte Carlo search tree works. But without explaining it, I can say that with these algorithms, you need to associate some sort of metric with the set of legal moves for a given board state. For Alpha Go this is not hard to do, and in fact they explain exactly how they do it in the paper I linked in my last post. And if the space of actions is too large they can do sampling methods (hence the Monte Carlo tree search). However, the proportion of intelligent actions within the space of all actions cannot be so sparse that the vast majority of monte carlo iterations probably won't even find one in many samples. Not unless you significantly reduce that space, and coarse grain it.
The space reduction and coarse graining is extremely difficult, though. You can, for example, decide that sending a specific army composition to attack a general area is as fine as you want to make it. That would work fine for some army compositions (like a WoL-style protoss deathball) but it certainly wouldn't work for others (like ling-bane, tank-medivacs, or disruptors). You have to finely tune the degrees of freedom to quite an extent if you don't want an army to simply throw the game by not even using 10% of its utility. If you want to coarse grain the learning process to a very high level of macro strategy then it would be easy to train, but you would never get good micro (or any micro, really) without adding scripting to it. (Which may not be off the table.) However, if you think that with pure learning and no scripting, that if you coarse grain the training sets to very high level macro strategy and still train an algorithm that can beat pros, then we probably just completely disagree on the level of importance of small scale micro.
We are not trying to find the best moves possible. We are trying to beat the world champion, or someone similar. You are again making this harder than it is.
But you do that by approximating the best possible set of moves you can make. That's what a machine learning algorithm does. You start with an ideal (in this case, there exists a set of moves that are maximally good) and you try to approximate this ideal as well as you can.
On March 29 2016 04:15 WinterViewbot420 wrote: Terran drops could be very abusive by this robot if we assume it's not restricted.
Well, then the computer would be cheating. It needs to have the same restriction a human has, a mouse, a keyboard and monitor. Alternatively, I believe we'd be better off letting humans control the game with their mind. That would be a more fair match, with only the monitor being the limiting factor.
I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
On March 29 2016 05:24 The Bottle wrote: You said "reducing Starcraft to a problem on a higher level of abstraction than pixels or coordinates isn't necessarily very hard either". For scripting that's true. For supervised and reinforced learning... well... I really think you have to work a lot harder to justify this statement. Unless you think scripting is the way that they're going to beat a pro player, in which case... well, I agree with all of Boxer's objections to that.
No, I'm not talking about a pure scripted bot, nor a pure AI. I'm imagining a hybrid as I hinted about earlier. And yes, I have basic laymans knowledge about both neural nets and the Monte Carlo search method. I have followed the development of Go programs since the very beginning and have seen revolution of Monte Carlo search myself, but I'm not even sure Monte Carlo or any variant of current tree search methods are applicable for Starcraft. I can't see how. I know I'm not being very specific but todays Starcraft scripts can play a decent game without ANY AI techniques at all, just hard coded rules. I think that using just a policy network would help a lot. At what "granularity" you want to apply it depends on how much money you want to throw at the problem.
We are not trying to find the best moves possible. We are trying to beat the world champion, or someone similar. You are again making this harder than it is.
But you do that by approximating the best possible set of moves you can make. That's what a machine learning algorithm does. You start with an ideal (in this case, there exists a set of moves that are maximally good) and you try to approximate this ideal as well as you can.
With unlimited resources perhaps, but a compromise may be good enough to beat a world champion, which Lee Sedol discovered. The games used to train the policy network were games from public Go servers. They weren't even professional games. It was good enough.
Yes, input needs to be limited. Its similar to chess. If we let the machine check an infinite amount of moves, it will find the winning move by brute force. That is a weak form of intelligence and has more in common with a machine. The interesting question is if it is able to find solution in less than N moves and in case of sc2 in less than N inputs.
Second point is unit recognition. A human focuses generally on the center screen. The ai will focus on the whole screen and have the clear advantage.
Third point is adaptation on the fly in an incomplete knowledge problem. The advantage goes clear to human brain because it is literally made for that task. Imagine the micro you do, even if its muscle memory, you adapt on the fly, e.g. skill recognition: How good does he micro? Can i outmicro him? How can i minimize my costs?
The question, I am interested in, is if it finds a better strategy than what we call standard today.
It's like everyone is just willfully ignoring lichter and the fact that this AI will have to learn how to select its command center, how to create hotkeys, and how to navigate the interface based only on visual input. They will make it use at least an emulation of a keyboard and mouse. It will not have access to the game code. Any deviation from this would defeat the entire purpose of deepmind AI. If you watch the video where they show off the atari AI, the first two hundred games it played of Breakout, it could hardly hit the ball with the paddle.
If google can manage an AI that teaches itself how to become the automaton 2000 with scouting and decision making from scratch I will still be completely amazed.
Watch this video from the 9:14 mark, anyone who is having doubts.
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
as artosis put it on 2 articles on ESPN, not limiting apm would be like putting a world class runner in a race with a car. completely unfair. (good articles by the way)
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
as artosis put it on 2 articles on ESPN, not limiting apm would be like putting a world class runner in a race with a car. completely unfair. (good articles by the way)
Putting humans against computers in chess is also completely unfair. That doesn't make the AI problem illegitimate, and it's not like the technology is even there to make the right decision fast enough. If AI takes one second to decide an optimal move, how much APM does it really have?
On March 29 2016 05:24 The Bottle wrote: You said "reducing Starcraft to a problem on a higher level of abstraction than pixels or coordinates isn't necessarily very hard either". For scripting that's true. For supervised and reinforced learning... well... I really think you have to work a lot harder to justify this statement. Unless you think scripting is the way that they're going to beat a pro player, in which case... well, I agree with all of Boxer's objections to that.
No, I'm not talking about a pure scripted bot, nor a pure AI. I'm imagining a hybrid as I hinted about earlier. And yes, I have basic laymans knowledge about both neural nets and the Monte Carlo search method. I have followed the development of Go programs since the very beginning and have seen revolution of Monte Carlo search myself, but I'm not even sure Monte Carlo or any variant of current tree search methods are applicable for Starcraft. I can't see how. I know I'm not being very specific but todays Starcraft scripts can play a decent game without ANY AI techniques at all, just hard coded rules. I think that using just a policy network would help a lot. At what "granularity" you want to apply it depends on how much money you want to throw at the problem.
We are not trying to find the best moves possible. We are trying to beat the world champion, or someone similar. You are again making this harder than it is.
But you do that by approximating the best possible set of moves you can make. That's what a machine learning algorithm does. You start with an ideal (in this case, there exists a set of moves that are maximally good) and you try to approximate this ideal as well as you can.
With unlimited resources perhaps, but a compromise may be good enough to beat a world champion, which Lee Sedol discovered. The games used to train the policy network were games from public Go servers. They weren't even professional games. It was good enough.
A hybrid of scripting at the small scale and deep learning at the large scale may be the best way to go, but there's still a huge problem, one that doesn't exist in simply defined discrete board games like Go. As you can probably predict I'm going to say, that problem is the coarse graining of the data itself.I think you're really underestimating the scale of this problem because it is a huge problem.
The problem is divided into two parts. Actually defining the set of "rules" (i.e. possible actions) that the computer can take, and then getting an algorithm to recognize, from the metadata of historic games, when those actions have actually been taken (i.e. transforming the set of discrete actions in the game into the actions that you created in your state space of moves).
The first is problematic because it's difficult to make the choices. You have to strike a balance between making the set of actions robust enough that the algorithm can actually learn interesting strategies on its own, yet constrained enough that the computation is tractable. Any time you design a new "move" you have to ask yourself, "how badly did I just restrict the freedom of the algorithm? I just set move A and move B to be equivalent, but how many scenarios exist in which the outcome of said moves are drastically different?" With a game like Starcraft, that's extremely sensitive to small missteps, you're pretty much never going to do this the "right" way. But will the cumulative flaws of your coarse graining scheme be small enough not to completely botch the execution? That's hard to answer, and I honestly don't know. But this is a huge problem, and it doesn't even exist for Go.
The second part is actually recognizing when the set of "moves" that you defined are actually executed based on the metadata of a game. The only real objective way to store the data of a game is to store the exact actions, since machines don't recognize heuristics such as "he transitioned from ultras to mutas and then diverted his forces to his main to harass his fourth". So you have to be able to recognize what set of actions correspond to what "move" in your coarse grained space, and that's well out of the realm of the methods they used for the Go algorithm.
Now maybe the above that I described is doable. (Assuming they do a mix of deep learning for high level strategy and scripting for small scale actions). In fact I hope it is, and I hope they make an amazing algorithm that can beat anyone. But it's way harder. And, what I have been trying to argue here is that the reason that it's harder is because of the sheer complexity of the space of possible actions in Starcraft, compared to Go. The entire reason the problems I listed above exist is because of that complexity. And the more sensitive the outcome of the game is to small deviations in those discrete actions, the harder it is to coarse grain them. And this is all just to create a workable training set for the model, whereas doing the same in Go is pretty much trivial, and explained in a couple of sentences in the paper.
We see videos of micro where a moved banelings follow marines off creep and lose. Maybe a human player won't a move his banelings off creep? Or where zerglings avoid splash from tanks, can they also do that against marines? Or medivac+tank micro against projectile attacks, maybe the human will target the medivac or build air units?
The amazing micro tricks an ai can do are limited in application but we see the videos and focus so much on how amazing one side of the micro is that we neglect to consider that the other player wouldn't take that engagement.
On March 28 2016 13:46 a4bisu wrote: APM is not the point. A large portion of human APM is meaningless and just for warming up purpose toward spikes in big fights.
SC is a totally different game to Go. People call it not "perfect information", meaning AI does not know exactly what its opponent doing in a SC game, not as completely as black and white stones on a Go board.
Suppose two medivacs are approaching the zerg AI's 2nd base, the AI does not know what's inside, marines, marines and mines, or empty? AI's need to predict the possible drop location, it is main base, or 2nd base, or one for each? All the possible scenarios requires defense strategies accordingly with consideration of limited resource and efficiency. All these scenarios are developing in real time and could shifting from each other within milliseconds as the medivacs make a boosted turn.
People keep bringing this up but how do people do it? They use their experience to predict whats going to happen. Sometimes they are wrong, sometimes they are right. The AI will be the same, probably a bit better because recall of existing information in the game is perfect (unlike a person who could forget some small thing).
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
as artosis put it on 2 articles on ESPN, not limiting apm would be like putting a world class runner in a race with a car. completely unfair. (good articles by the way)
Putting humans against computers in chess is also completely unfair. That doesn't make the AI problem illegitimate, and it's not like the technology is even there to make the right decision fast enough. If AI takes one second to decide an optimal move, how much APM does it really have?
Invalid comparison, computers in chess are not AI. They're exactly what we're trying to avoid.
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
That kind of micro is so stupidly effective that it would hardly need to play intelligently to secure the win. Defeats the purpose of what we're trying to achieve here.
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
as artosis put it on 2 articles on ESPN, not limiting apm would be like putting a world class runner in a race with a car. completely unfair. (good articles by the way)
Putting humans against computers in chess is also completely unfair. That doesn't make the AI problem illegitimate, and it's not like the technology is even there to make the right decision fast enough. If AI takes one second to decide an optimal move, how much APM does it really have?
Invalid comparison, computers in chess are not AI. They're exactly what we're trying to avoid.
Please explain how computer chess is not AI if you want to invalidate the comparison.
On March 29 2016 05:43 chocorush wrote: I think it will be more interesting to see how the AI does first before we start worrying about handicapping for it to be "fair" (is the point of showing an AI that can play Starcraft supposed to be fair?).
Considering how many seconds even the simplest move took to compute in Go, I'm not convinced that we would need to worry about too high of an APM rate to worry about the engineering problem more than the computer science problem.
The problem is the following. If you allow unlimited apm, it will move all units away from splash and keep units at distance. Thats a few lines of code, they wont even need a real AI and the human will be helpless. But that has more in common with a machine than with an intelligence. Its not the question we seek. We want to know if it can solve the problem in less than N inputs. That requires strategy!
Is that really the end result of unlimited apm? I expect the computer to learn that focusing only on micro won't win games. And how will it attack if it's scripted to stay out of range? It will still need to learn army composition and learn how to engage strategically.
Just requiring it to follow the same rules as humans like only being able to micro units while its focus is on the screen will require it to have to learn to ration its focus as a resource.
I don't expect it to be able to micro while thinking efficiently, unless it learns to do a lot of meaningless spamming while it decides the next best optimal move, which is pretty humanlike already.
as artosis put it on 2 articles on ESPN, not limiting apm would be like putting a world class runner in a race with a car. completely unfair. (good articles by the way)
Putting humans against computers in chess is also completely unfair. That doesn't make the AI problem illegitimate, and it's not like the technology is even there to make the right decision fast enough. If AI takes one second to decide an optimal move, how much APM does it really have?
Invalid comparison, computers in chess are not AI. They're exactly what we're trying to avoid.
Please explain how computer chess is not AI if you want to invalidate the comparison.
Apologies for answering something not directed at me but one of the main goals of deepmind AI is to create "General AI" rather than "Focused AI" so while a chess bot is certainly 'AI' it's not the TYPE of AI that deepmind is looking to create. The video I linked earlier explains this more in depth before the atari part.
We're making a distinction between a computer explicitly programmed to replicate a set of moves in specific conditions and a computer that has learned through trial and error what a proper response might be.
Give the AI unlimited APM, it's fine imo. Would much rather see bw though, instead of sc2. On second, thought, I'd have to wait and see just HOW hard the AI could abuse this advantage, but to start with, I think the AI will have a hard enough time ever winning...
On March 29 2016 06:03 Fecalfeast wrote: It's like everyone is just willfully ignoring lichter
it's because i'm always right and no one wants to argue with me
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
The problem is that Starcraft 2, other than Go, is not purely a game of mind.
In fact, you can reach GM on mechanics alone. I do see the problem with "unlimited APM", so don't get me wrong, you got a point there: but to call something disingenuous to then go ahead and make another disingenuous statement is rather.. weird.
Some cornerpoints: no API. If the AI has all information all the time, it's dumb. I agree. However: apart from mouse/keyboard, how do you want to fiddle with the rest of the stuff that people consider "unfair"? Like "limitless awareness of minimap while macroing"?
It will never be a match on even terms. It's technically neither possible nor feasible. The reasoning, or rather, goal behind DeepMind playing Starcraft is that it's a game that, other than chess and Go, is based on "not having perfect information". The only thing they need to do really is limit "micro". That's it. Not even APM: just the ability (btw, no idea how) to micro the shit out of units to the point of abuse.
You're not playing another player. You're playing an AI. It will be mechanically stronger. It's pretty much literally a player who controls the game with his mind. I don't think that projecting human shortcomings on the AI somehow makes for an interesting match for anyone to watch. If the AI is limited to mimic(!) a human, might as well just train another human to play it.
So, all in all: limit not APM, but somehow the ability to micro. Nothing else needed. If the (non-cheating, like full vision) AI can beat a player without "infinite micro", it's simply stronger. It's nothing to frown upon, or to be concerned about. It will happen sooner or later anyway.
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
The problem is that Starcraft 2, other than Go, is not purely a game of mind.
In fact, you can reach GM on mechanics alone. I do see the problem with "unlimited APM", so don't get me wrong, you got a point there: but to call something disingenuous to then go ahead and make another disingenuous statement is rather.. weird.
The thing about a person's mechanical skill using a certain output is that it has to be spread around a thousand different decisions. Prioritizing what to do over what not to do is part of strategy. Attention and time are also limited resources that have to be accounted for. In Go, the computer and the human have the same amount of time to decide on a move and do the command. They can both do one move per turn. It's exactly the same. It's a resource, just like minerals, gas, supply, and it has to be dictated by strategy, not by output. A faster player is not necessarily better than a slower player if that faster player uses his APM incorrectly. You are right that mechanics play a huge part in StarCraft, and there's no easy to way to ensure the AI "plays the same game", but that doesn't change the fact that it should be one of the early goals of the project. I'm sure the DeepMind guys will surprise us with something ingenious and unexpected.
A hybrid of scripting at the small scale and deep learning at the large scale may be the best way to go, but there's still a huge problem, one that doesn't exist in simply defined discrete board games like Go. As you can probably predict I'm going to say, that problem is the coarse graining of the data itself.I think you're really underestimating the scale of this problem because it is a huge problem.
And I say you are overestimating it. Again, the bots that exist today are already doing this. They can scout, they can count and estimate enemy forces etc. Yes, it's on an coarse level, but I'm not buying that just because you are throwing away information it suddenly doesn't work. These bots are programmed by amateurs with no resources. This problem is only hard if you make it hard, and you can make it as hard as you want, but you are arguing that you can't do it just because you can make it hard, and that seems to have been your line all along. No, feeding an AI nothing but basic unit positions or pixel values probably won't work, but I didn't suggest that. No using the AlphaGo techniques unchanged with Monte Carlo search probably won't work, but I didn't suggest that either. Yes, the theoretical limits of an AI with simplified inuts is probably lower than an AI with access to raw data, but that doesn't necessarily make it bad.
An assumption I'm making is that this is done with direct access to an SC2 engine. Maybe you think it's cheating, but this (and the above) is why I think this isn't an interesting problem at all. You can make it easy or you can make it hard. All the argumentation is about where you draw the line. Limit the APM or not? There is no question in my mind that you can make an AI that can beat a world champion, if you choose to. That wasn't the case with Go. It was exceptionally hard. Monte Carlo wasn't new. Even Monte Carlo with neural nets wasn't new. Others had tried and failed. In this case people argue that a scripted bot with unlimited APM can do it. That makes it a non-problem to me.
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
The problem is that Starcraft 2, other than Go, is not purely a game of mind.
In fact, you can reach GM on mechanics alone.
no you can't. if you have no game knowledge you don't know any builds or what composition you should use, you don't know what builds your opponents race has and so you can't scout and prepare for them, hell you don't even know what are the most efficient ways to micro your units. many of those things are basic knowledge for us because we have spend a lot of time into this game but perfect mechanics with zero game knowledge doesn't take you far.
On March 29 2016 06:03 Fecalfeast wrote: It's like everyone is just willfully ignoring lichter
it's because i'm always right and no one wants to argue with me
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
The problem is that Starcraft 2, other than Go, is not purely a game of mind.
In fact, you can reach GM on mechanics alone. I do see the problem with "unlimited APM", so don't get me wrong, you got a point there: but to call something disingenuous to then go ahead and make another disingenuous statement is rather.. weird.
The thing about a person's mechanical skill using a certain output is that it has to be spread around a thousand different decisions. Prioritizing what to do over what not to do is part of strategy. Attention and time are also limited resources that have to be accounted for. In Go, the computer and the human have the same amount of time to decide on a move and do the command. They can both do one move per turn. It's exactly the same. It's a resource, just like minerals, gas, supply, and it has to be dictated by strategy, not by output. A faster player is not necessarily better than a slower player if that faster player uses his APM incorrectly. You are right that mechanics play a huge part in StarCraft, and there's no easy to way to ensure the AI "plays the same game", but that doesn't change the fact that it should be one of the early goals of the project. I'm sure the DeepMind guys will surprise us with something ingenious and unexpected.
The current Deepmind Atari games approach cannot have more than 2/3 actions per frame with a keyboard and mouse ( so max 120 APM / 180 APM at 60Hz rendering ), unless you deliberately do something silly to the final layer of your neural networks. As such most of the APM debate is moot.
On March 29 2016 06:03 Fecalfeast wrote: It's like everyone is just willfully ignoring lichter
it's because i'm always right and no one wants to argue with me
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
The problem is that Starcraft 2, other than Go, is not purely a game of mind.
In fact, you can reach GM on mechanics alone.
no you can't. if you have no game knowledge you don't know any builds or what composition you should use, you don't know what builds your opponents race has and so you can't scout and prepare for them, hell you don't even know what are the most efficient ways to micro your units. many of those things are basic knowledge for us because we have spend a lot of time into this game but perfect mechanics with zero game knowledge doesn't take you far.
Well, it depends on what you want to achieve in the game: Laddering up? I think this should be possible purely through perfect mechanics. There are quite a few examples of players who reached GM with just a couple of builds and an A.I. could do these builds better, most likely.
The problem arises if the A.I. is supposed to beat a progamer who's able to prepare for the vs. A.I. match. I don't think the A.I. is gonna win such a match in the near future, simply because if preparation is allowed, perfect mechanics aren't really the deciding factor anymore.
On March 29 2016 06:03 Fecalfeast wrote: It's like everyone is just willfully ignoring lichter
it's because i'm always right and no one wants to argue with me
I feel like that would be disingenuous to the purpose of this project. The amazing thing about Go is it's 100% a battle of the mind because input/output is the same regardless of computer or human. Unless they do that for SC2 as well, it won't be as meaningful as the breakthrough with Go.
The problem is that Starcraft 2, other than Go, is not purely a game of mind.
In fact, you can reach GM on mechanics alone.
no you can't. if you have no game knowledge you don't know any builds or what composition you should use, you don't know what builds your opponents race has and so you can't scout and prepare for them, hell you don't even know what are the most efficient ways to micro your units. many of those things are basic knowledge for us because we have spend a lot of time into this game but perfect mechanics with zero game knowledge doesn't take you far.
Well, it depends on what you want to achieve in the game: Laddering up? I think this should be possible purely through perfect mechanics. There are quite a few examples of players who reached GM with just a couple of builds and an A.I. could do these builds better, most likely.
The problem arises if the A.I. is supposed to beat a progamer who's able to prepare for the vs. A.I. match. I don't think the A.I. is gonna win such a match in the near future, simply because if preparation is allowed, perfect mechanics aren't really the deciding factor anymore.
even when executing a simple build there are many different ways you have to react to the way your opponent is playing. if you go for an immortal allin for example but your opponent turtles on 2 bases with spines while teching to mutas. what do you do then? there isn't a build that you can just blindly execute every game without reacting to your opponent.
On March 29 2016 18:58 Railgan wrote: Will Deepmind have to use mechanical hands to use a keyboard / mouse?
Unlikely, for the same reason they didn't use a camera to read the Go board in the game against LSD. It is just a technical complication that doesn't have much to do with the challenge they're trying to tackle.
On March 29 2016 18:58 Railgan wrote: Will Deepmind have to use mechanical hands to use a keyboard / mouse?
Unlikely, for the same reason they didn't use a camera to read the Go board in the game against LSD. It is just a technical complication that doesn't have much to do with the challenge they're trying to tackle.
Well a lot of people have already stated that the difficulty of selecting the units and ordering them around is like "half the challenge" of games like Broodwar or SC2.
If an AI doesn't need to "visually identify and select" his stuff, then progamers would need to be able to play with just their minds to make it a fair game.
I doubt there will go with image recognition software. That would even more complicate things. But it should be limited to only be aware of what is on screen and controlling units only on screen. Changing screen position and changing selecting counts as APM. Which should be limited to average progamer EPM. Hell they even can add reaction times and imperfect macro if they want to, it's very easy to do. The point is to make it win with strategy/tactics and not mechanics.
If anything, at least it will be much more entertaining then those GO matches were. Hoping they get some good casters for this. I wonder if they will have it play as random or a single race as well.
A hybrid of scripting at the small scale and deep learning at the large scale may be the best way to go, but there's still a huge problem, one that doesn't exist in simply defined discrete board games like Go. As you can probably predict I'm going to say, that problem is the coarse graining of the data itself.I think you're really underestimating the scale of this problem because it is a huge problem.
And I say you are overestimating it. Again, the bots that exist today are already doing this. They can scout, they can count and estimate enemy forces etc. Yes, it's on an coarse level, but I'm not buying that just because you are throwing away information it suddenly doesn't work. These bots are programmed by amateurs with no resources. This problem is only hard if you make it hard, and you can make it as hard as you want, but you are arguing that you can't do it just because you can make it hard, and that seems to have been your line all along. No, feeding an AI nothing but basic unit positions or pixel values probably won't work, but I didn't suggest that. No using the AlphaGo techniques unchanged with Monte Carlo search probably won't work, but I didn't suggest that either. Yes, the theoretical limits of an AI with simplified inuts is probably lower than an AI with access to raw data, but that doesn't necessarily make it bad.
An assumption I'm making is that this is done with direct access to an SC2 engine. Maybe you think it's cheating, but this (and the above) is why I think this isn't an interesting problem at all. You can make it easy or you can make it hard. All the argumentation is about where you draw the line. Limit the APM or not? There is no question in my mind that you can make an AI that can beat a world champion, if you choose to. That wasn't the case with Go. It was exceptionally hard. Monte Carlo wasn't new. Even Monte Carlo with neural nets wasn't new. Others had tried and failed. In this case people argue that a scripted bot with unlimited APM can do it. That makes it a non-problem to me.
No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro then, well that's another discussion entirely, but one that's completely useless in the context of Deepmind. If that's the method they end up using, then they will just abandon the project. Deepmind will only do this as long as it advances them in the problem of machine learning for a game without perfect information. Their end goal is not to beat a pro SC player, it's to advance the field of machine learning in an interesting way. They're not going to use the hiring time of their really highly paid research scientists to figure out how to script Starcraft AI well and give it good micro from the game strategy. This is why I have been assuming, the whole time, that some sort of learning is going to take place, and that that learning has to play an essential factor in the game being able to beat a pro player. And no, nothing like this has ever been done in any Starcraft game, or really any strategy video game (that I heard of). So you can't just dismiss the idea of a Starcraft AI learning a coarse grained strategy as "simple" because "it's been done". Nothing remotely like that has been done.
I never assumed it can't be done. Only that it's substantially more difficult than for Go, for all the reasons I explained. And I never assumed that they have to use the same methods as they did for Go. In fact I know that they can't. But as long as they program it to learn unscripted essential strategies for beating a pro (again, if that's not the case then this discussion is absolutely pointless, because it won't do anything for Deepmind so they just won't do it) then they have to use some sort of deep learning algorithm (be it a neural net, SVM, or some innovative combination of algorithms like they did for Go) then they need a data set that objectively defines the actions of the computer, and can be read from the metadata of all the games that can be played. In that case, all the problems I described exist, which are major problems that don't exist for Go.
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
No they're not, please give me an example of a single bot for a StarCraft game that uses input that's not given from an API to the game.
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
No they're not, please give me an example of a single bot for a StarCraft game that uses input that's not given from an API to the game.
Why do you think I mean "not given from an API to the game"?
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
I know that "some data exists". There is the entire build order of a player and the time each thing was built, which you see at the end game summary. If I were to guess, this is most likely the kind of data the AI uses, since it gets its data from an API. (But I don't claim to know how the current AI works, so if you have a better notion of what it's doing, tell me.) That kind of data is trivial to generate, and an AI that responds to this data from the API is easy to script. But it's useless for Deepmind's purpose. They want to construct a limited information AI algorithm. If, instead, we assumed that Deepmind decided to use an API and essentially map hack, then maybe they can train an algorithm on that "build order" data, and use scripting for the rest. But I think build order is just such a tiny and most uninteresting portion of the strategy involved in SC2. The resulting "learned strategies" we'd see from that are extremely uninteresting. Is the "existing data" that you're thinking of anything besides the build order data? If so, what?
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
I know that "some data exists". There is the entire build order of a player and the time each thing was built, which you see at the end game summary. If I were to guess, this is most likely the kind of data the AI uses, since it gets its data from an API. (But I don't claim to know how the current AI works, so if you have a better notion of what it's doing, tell me.) That kind of data is trivial to generate, and an AI that responds to this data from the API is easy to script. But it's useless for Deepmind's purpose. They want to construct a limited information AI algorithm. If, instead, we assumed that Deepmind decided to use an API and essentially map hack, then maybe they can train an algorithm on that "build order" data, and use scripting for the rest. But I think build order is just such a tiny and most uninteresting portion of the strategy involved in SC2 that the resulting "learned strategies" we'll see are extremely uninteresting. Is the "existing data" that you're thinking of anything besides the build order data? If so, what?
I'm not familiar with the script API either (or we should probably call it the map editor, because that's what I think it is), but I think it's pretty safe to assume that the API tells you if something is hidden behind the fog or war, so you don't have to map hack. The map editor seems to be very powerful and l think it's also safe to assume that there is a wealth of information available, but you don't have to use all of it. It's more than just build order anyway. The point is that the bot-scripts use this information today with pretty good results, and I don't think they cheat that much, unless you tell them to. Put this information into a DeepMind AI instead. Or why not have several. A self learning micro AI maybe, and a supervising macro AI. Maybe we should have one micro AI per unit. The possibilities are endless.
Considering that much of the average apm of pro's is produced via spamming keys as they warm up their fingers, even limiting an AI's APM to something like 400 (in SC2) would still be very overwhelming. If the computer can make 400 effective movements in a minute, I imagine it could do some crazy things. I'm not saying it will be able to beat the pro's, but making 6-7 actual, effective decisions per second is quite incredible.
On March 30 2016 02:53 ClanRH.TV wrote: Considering that much of the average apm of pro's is produced via spamming keys as they warm up their fingers, even limiting an AI's APM to something like 400 (in SC2) would still be very overwhelming. If the computer can make 400 effective movements in a minute, I imagine it could do some crazy things. I'm not saying it will be able to beat the pro's, but making 6-7 actual, effective decisions per second is quite incredible.
It most certainly would beat the pros. Remember that AI test of Zerglings showing if you perfectly micro Zerglings vs a siege tank line you can decimate it with fairly minimal losses because you can negate all splash? If weird part about Starcraft is that a majority of the skill comes from mechanics not really decision making. If the AI can execute perfectly microed blink-all ins I don't really see a human reliably stopping it. It isn't the AI is smarter than the human it is just it is mechanically better -- which makes sense its a computer.
On March 30 2016 02:53 ClanRH.TV wrote: Considering that much of the average apm of pro's is produced via spamming keys as they warm up their fingers, even limiting an AI's APM to something like 400 (in SC2) would still be very overwhelming. If the computer can make 400 effective movements in a minute, I imagine it could do some crazy things. I'm not saying it will be able to beat the pro's, but making 6-7 actual, effective decisions per second is quite incredible.
It most certainly would beat the pros. Remember that AI test of Zerglings showing if you perfectly micro Zerglings vs a siege tank line you can decimate it with fairly minimal losses because you can negate all splash? If weird part about Starcraft is that a majority of the skill comes from mechanics not really decision making. If the AI can execute perfectly microed blink-all ins I don't really see a human reliably stopping it. It isn't the AI is smarter than the human it is just it is mechanically better -- which makes sense its a computer.
This amongst the hundreds of other ridiculously stupid but possible once you have super computer doing the micro. For example, imagine if the only marines doing stutter step micro are the ones being targeted by the enemy units so that the other marines stand in place and eventually get a natural surround?
Imagine a medivac that cycles 4 marauders so that you never hit a single marauder. Mutalisks that will NEVER hit an AI controlled Phoenix. A bio ball where damaged units move to the back while the rest of the army keeps fighting to give maximum spread of damage before any single unit is killed? Perfect focus fire so that only the group within range of an enemy unit focuses their attack.
On March 28 2016 14:35 Taf the Ghost wrote: The Mouse-movement and Keyboard entry limitations would have to be imposed. Giving a computer full API access is to cheat, as the API could do things no mouse is actually capable of. We've seen some of the custom AIs over the years, but those functionally produce a new unit-movement control that goes around the game's actual functionality.
Thus, the first thing an "AlphaStar" would need to learn would be how to use the mouse. Haha.
I agree, I think the AI should be designed around the same limitations of a keyboard and mouse. Maybe programming the AI to operate a two hands that control the keyboard and mouse.
On March 30 2016 05:22 Xyik wrote: I don't know why everyone is still so fixated on micro, its been repeated by almost everyone in this thread, and isn't a very interesting discussion.
Because, in a way, its a revealing comment about the nature of RTS games. Most of the talent is because of people's ability to micro his units, workers, and buildings and not strategy. So when you can remove the limitation of control--then no RTS player can actually play well.
I think they key is that the computer should not outperform a human mechanically in any way, at least micro-wise. It should be coded in such a way that it approximates the mechanical skill of his opponent. Otherwise it would be just unfair. It's not interesting unless it is a battle of wits.
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
No they're not, please give me an example of a single bot for a StarCraft game that uses input that's not given from an API to the game.
Why do you think I mean "not given from an API to the game"?
You are suggesting that existing bots today use the same input data as deepmind would. Deepmind data is just the pixel output of the game. Not what the API provides.
On March 28 2016 14:35 Taf the Ghost wrote: The Mouse-movement and Keyboard entry limitations would have to be imposed. Giving a computer full API access is to cheat, as the API could do things no mouse is actually capable of. We've seen some of the custom AIs over the years, but those functionally produce a new unit-movement control that goes around the game's actual functionality.
Thus, the first thing an "AlphaStar" would need to learn would be how to use the mouse. Haha.
I agree, I think the AI should be designed around the same limitations of a keyboard and mouse. Maybe programming the AI to operate a two hands that control the keyboard and mouse.
Depends on if you constrain it to using a robotic arm to move the mouse, or just have something to move a mouse around. Stuff like laser cutters could be adapted to hold a mouse, with speed and accuracy far higher than that of a human. This one for example can do +-15um repeatedly.
Keyboard also, depending on how you set it up, it will hit keys so much faster than a human that it's not even funny.
It'd be easier to just set a keypress rate limit(10 keys/sec, with 10 seconds of burst up to 20kp/sec every minute), and an acceleration/speed limit on the mouse.
On March 30 2016 10:20 Iodem wrote: I thought they were supposed to have Flash play DeepMind. Is he gonna play SC2 again or what?
Then the matchup should be TvT and if Flash loses he will have to eat his golden mouse! If Flash wins however he gets to eat AlphaGo/AlphaStar!! You heard it here first people, it's a bet!
He won his golden mouse in BW. If anything it should be played there. I would love to see an A.I. abuse all the nifty tricks we came up with to break the game.
On March 30 2016 11:13 StarStruck wrote: He won his golden mouse in BW. If anything it should be played there. I would love to see an A.I. abuse all the nifty tricks we came up with to break the game.
If deepmind doesn't have to use input devices like a mouse and keyboard then it is at a tremendous advantage. Humans have to spend brainpower not only deciding what to do within the game, but also to move their own hands and fingers to make those things happen on screen. Just capping the APM of deepmind at that of a high level player will still still put deepmind at an advantage, because it will be able to use 100% of it's brainpower on the game itself. Humans have to spend a ton of their brainpower on manipulating input devices (I don't know what %, but would be interested to).
I think it will be difficult to create a true apples to apples comparison/competition here, short of building a robot that has a camera to watch the screen and an arm to move the mouse. AlphaGo had to have a human surrogate to place stones on the Go board.
I'll be very interested to see what terms Blizzard and Google decide on for the match.
Honestly I think the interesting part of the AI will be how it responds and gets stuff done depending on the input. AI's can already play engagements as perfectly as possible given APM constraints, the decision making that gets it there is much more interesting.
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
No they're not, please give me an example of a single bot for a StarCraft game that uses input that's not given from an API to the game.
Why do you think I mean "not given from an API to the game"?
You are suggesting that existing bots today use the same input data as deepmind would. Deepmind data is just the pixel output of the game. Not what the API provides.
There have been no details revealed about this so you are just guessing.
Would it be more interesting if we have DeepMind get setup, then have a pro and programmer decide what "strategy" deepmind should execute perfectly and have them face off another deepmind + pro combo?
Flash with DeepMind micro vs Jaedong with DeepMind micro?
On March 30 2016 00:58 The Bottle wrote: No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted. If you argue that a bot with unlimited APM and much more sophisticated scripting can beat an pro.
That is not what I meant at all. I said that todays bots are already using input from "coarse grained data", and there is no reason this data can't be used as input to a real self learning AI instead. Re-read what I said with that in mind. The data already exist. No-one have even tried to use it for self-learning.
I know that "some data exists". There is the entire build order of a player and the time each thing was built, which you see at the end game summary. If I were to guess, this is most likely the kind of data the AI uses, since it gets its data from an API. (But I don't claim to know how the current AI works, so if you have a better notion of what it's doing, tell me.) That kind of data is trivial to generate, and an AI that responds to this data from the API is easy to script. But it's useless for Deepmind's purpose. They want to construct a limited information AI algorithm. If, instead, we assumed that Deepmind decided to use an API and essentially map hack, then maybe they can train an algorithm on that "build order" data, and use scripting for the rest. But I think build order is just such a tiny and most uninteresting portion of the strategy involved in SC2 that the resulting "learned strategies" we'll see are extremely uninteresting. Is the "existing data" that you're thinking of anything besides the build order data? If so, what?
I'm not familiar with the script API either (or we should probably call it the map editor, because that's what I think it is), but I think it's pretty safe to assume that the API tells you if something is hidden behind the fog or war, so you don't have to map hack. The map editor seems to be very powerful and l think it's also safe to assume that there is a wealth of information available, but you don't have to use all of it. It's more than just build order anyway. The point is that the bot-scripts use this information today with pretty good results, and I don't think they cheat that much, unless you tell them to. Put this information into a DeepMind AI instead. Or why not have several. A self learning micro AI maybe, and a supervising macro AI. Maybe we should have one micro AI per unit. The possibilities are endless.
That's true, there is the information from a map editor (which unit/part of building occupies each grid coordinate, binary variable to indicate whether or not it's revealed by the fog of war for each player). I haven't thought of that. If they could think of a way to learn information by simply feeding a NN a large series of still frames from random games (like they do for the Alpha Go policy network stage), that would be feasible. Although I can't think off the bat what they could infer from that. In Go, they fed the network board position-action pairs, which you can't really do in this case, not with discrete actions. It still doesn't solve the two major problems I illustrated with coarse graining a set of actions. The first problem is solved if you just take any mapping and call it "good enough" (which might actually be ok, since the strategy in Starcraft isn't as deep as in Go) but I think you would have to work a lot harder than that if you want it to learn useful unscripted strategies that are essential for winning. And the second problem, well, I must say I have no idea how that would be solved.
Without getting too specific, I guess I can lay out the problem like this. If you asked me to write an algorithm for learning Go strategies given a large data set of played games, before even reading that paper by Deepmind, I could think of one really easily. Not a good one, mind you, not one that could come even close to performing as well as theirs. But the fact that the game is so simple (not the strategies, but the building blocks) with an extremely well defined discretized set of board states and actions, means I have to do almost no work in transforming the data in a way that can be read by a good supervised learning algorithm, or translating the problem to a metric for the algorithm. For Starcraft, I really don't know how I would go about it. I mean, I have ideas, but they seem extremely hard to implement, and it seems like I will constantly run into problems with the way I set things up, just to bring the data to the same level as Go already is right off the bat. You might say that this is for a lack of imagination on my part, and you might be right. But the scientists at Deepmind are not supermen, it's pretty easy to see that these issues will be a big hurdle for them.
I have an idea for a start at least. If its' feasible I don't know because I don't work with neural nets. If we start at the micro end I would want one type of neural net for each unit type. I would train them by simulating, or by using replays, local battles. A humongous amount of battles of course. Unlike Go the outcome of the battle is very easy to see. I wouldn't even need to play it to the end. Once one side start to win I would end it and record a win or loss. I would probably start with replays like the DeepMind team did, and then continue with self play. I would now expect (if I were good at this like the DeepMind guys) that the nets would learn micro, like the nets they used in the Atari games did. They would learn to cooperate with other. Medivacs heal marines and things like that. Now we have building blocks that a macro AI can use. This AI would not need to consider each unit, but could instead give orders, like "attack there".
It would probably be useful to organize the input to the nets in some more intelligent way than just coordinates, like "distance to closest enemy unit" and things like that. I think they did similar things for the Go problem.
I have no idea what I would do next. :-)
Edit: I want to remind you that the DeepMind team did NOT succeed at first. They used neural nets. They trained them. They used them in combination with Monte Carlo. It didn't work. Well, it worked of course, but other people had already created Go AIs that played at high amateur level. The breakthrough came first with the idea of a policy net and a value net. I may be that similar breakthroughs are needed for Starcraft. Just throwing nerual nets at a problem doesn't work.
My approach would probably to do something like the bag of visual words approach for vision to start building a feature set. Just take a huge DB of replays and let a pattern matching algorithm find commonalities. You might need multiple levels (such as first releasing it on the maps to analyse commonalities, and then use these commonalities in the higher level) such as "send 3 marines to ramp" instead of "send 3 marines to position (2012, 3437)". But you can easily test whether such patterns that it finds make sense by figuring out whether it finds common build orders, scouting patterns, etc. Obviously a gimongous number of those patterns are going to be complete nonsense to a human observer.
Once you have your features more or less working, you can take a similar approach to the way the Go system worked. You chop games into 5 second sequences (or half a second, or 2 second, or 10 second) and consider the "actions" the same as a move in Go (of course, it's real-time so you might need new methods to deal with the changing world, and unknowns in the game world, that occur while you consider a single player's actions), given only the additional label that the game was ultimately won or lost. That should at least give you a way to start understanding whether an approach like this is a viable way to "win" at SC2.
On March 31 2016 05:00 Mendelfist wrote: I have an idea for a start at least. If its' feasible I don't know because I don't work with neural nets. If we start at the micro end I would want one type of neural net for each unit type. I would train them by simulating, or by using replays, local battles. A humongous amount of battles of course. Unlike Go the outcome of the battle is very easy to see. I wouldn't even need to play it to the end. Once one side start to win I would end it and record a win or loss. I would probably start with replays like the DeepMind team did, and then continue with self play. I would now expect (if I were good at this like the DeepMind guys) that the nets would learn micro, like the nets they used in the Atari games did. They would learn to cooperate with other. Medivacs heal marines and things like that. Now we have building blocks that a macro AI can use. This AI would not need to consider each unit, but could instead give orders, like "attack there".
It would probably be useful to organize the input to the nets in some more intelligent way than just coordinates, like "distance to closest enemy unit" and things like that. I think they did similar things for the Go problem.
I have no idea what I would do next. :-)
Edit: I want to remind you that the DeepMind team did NOT succeed at first. They used neural nets. They trained them. They used them in combination with Monte Carlo. It didn't work. Well, it worked of course, but other people had already created Go AIs that played at high amateur level. The breakthrough came first with the idea of a policy net and a value net. I may be that similar breakthroughs are needed for Starcraft. Just throwing nerual nets at a problem doesn't work.
I am starting to see the disconnect between us. You are talking at a much higher level than I am. That is, you're thinking about it from a sort-of business sense, whereas I'm thinking about it as someone who has to actually write the algorithm. In your explanation, you didn't even attempt to explain what the form of an input vector or target variable would be, and what you're explicitly trying to predict. Is your input vector the board state in a local set of grid points around the unit? Is the target a sequence of actions taken by the unit itself? (That would be a very complicated target; you'd have to get into doing Markov Chain Monte Carlo simulations or something like that, and I don't know how well that would go in training an AI). Or is your input the local board state around a unit and the sequence of actions afterwards? You mentioned that you'd take, as an input, a small interval of time during a battle, for a unit, with your target being who won the battle. If you take that for the maximal possible information, this is already intractable for training a NN (for a typical NN with a single hidden layer and number of nodes equal to number of features, which should be your minimum, the complexity of its calculation goes as order(D^2) where D is your number of features). If you want to transform the data into one of less than maximal information for that battle, again this is an extremely difficult task.
In Go all my questions are extremely easy to answer. An input vector is a 361 vector of discrete variables having one of three values (0, 1, or 2 corresponding to black, white, or empty). The target variable is single categorical variable with 361 possible values (what action was taken). This would work well enough to train something at the very least, that's tractable and would give me an algorithm that can actually play Go. (In the case of training Alpha Go's policy network, their training set was more complicated; it's 361x11 as the size of the input vector, since they included variables for each square such as how many liberties there are around the set it's pointing to, how many moves were made in that square since the beginning, etc; still tractable and easy to code).
All you can do with your explanation is to find a machine learning expert, give them your basic outline, and tell them to figure out how to do this. However you can actually take my explanation of the Go learning process, find a database of Go games, and actually write the algorithm and apply it, because I gave all the relevant information needed. That's what I'm trying to figure out for Starcraft, and this is a really tough problem for all the reasons I explained.
Your example also requires somebody to go through every single training set and give it a label. You can't sample 20 million battles from the data base and expect there to be the data available of who won that battle... you can't even expect to sample 20 million random intervals and local grid spaces and expect them all to be a battle, so how would you find them automatically? You have to go through it personally, and then your training set would be on the order of hundreds (after you spend a really long time) rather than 20 million like Alpha Go's. And you have to do that for each unit type. In their case, you can easily take (and they did) 20 million randomly sampled board state-action pairs without needing anyone to figure out if it's what they want, or label the value of the target variable. (And before you tell me that "hundreds will do just fine", that's absolutely false. If you're training a NN with a large feature space of a unit's actions, your solution will be overfitted like crazy.)
On March 31 2016 07:28 The Bottle wrote: I am starting to see the disconnect between us. You are talking at a much higher level than I am. That is, you're thinking about it from a sort-of business sense, whereas I'm thinking about it as someone who has to actually write the algorithm. In your explanation, you didn't even attempt to explain what the form of an input vector or target variable would be, and what you're explicitly trying to predict.
Yes. You are correct. I don't see the problem. In the Atari example they used pixels as the input vector. It worked. How about this: The input vector could be distance and angle to each unit (within close range). You would also need health value, type of unit, and upgrade information. The target would just be the next action taken by the unit, not a sequence. I say the same as you, while I'm sure this isn't optimal why wouldn't it do *something*? They already have nets playing Atari games. I consider the basic problem already solved. I don't believe that SC2 units on a map would be more complicated than pixels in an Atari game.
However you can actually take my explanation of the Go learning process, find a database of Go games, and actually write the algorithm and apply it, because I gave all the relevant information needed. That's what I'm trying to figure out for Starcraft, and this is a really tough problem for all the reasons I explained.
Your idea for the Go problem would fail. It would play Go, but it would be no better than all the other attempts. It required an ingenious idea by DeepMind to make it actually work.
Your example also requires somebody to go through every single training set and give it a label. You can't sample 20 million battles from the data base and expect there to be the data available of who won that battle...
No. Seriously. You are trying very hard to make everything sound a lot harder than it is. This is NOT a hard problem. I could put a novice directly from the university at this for some time and he would come up with a decent idea for an automated process. I maybe even don't need human games. I could use the Expert AI and simulate a lot of battles. I only need this to get a starting point, because the rest would be done with self play.
On March 31 2016 07:28 The Bottle wrote: I am starting to see the disconnect between us. You are talking at a much higher level than I am. That is, you're thinking about it from a sort-of business sense, whereas I'm thinking about it as someone who has to actually write the algorithm. In your explanation, you didn't even attempt to explain what the form of an input vector or target variable would be, and what you're explicitly trying to predict.
Yes. You are correct. I don't see the problem. In the Atari example they used pixels as the input vector. It worked. How about this: The input vector could be distance and angle to each unit (within close range). You would also need health value, type of unit, and upgrade information. The target would just be the next action taken by the unit, not a sequence. I say the same as you, while I'm sure this isn't optimal why wouldn't it do *something*? They already have nets playing Atari games. I consider the basic problem already solved. I don't believe that SC2 units on a map would be more complicated than pixels in an Atari game.
My input vector suggestion was better (not the sequence of actions, but the local grid around the unit, of fixed size, with variables explaining the occupation of each grid element, with variables such as unit type, percentage of full health, etc). My way allows for a consistently structured vector whose dimensionality is the same for every input. You say the target is the unit's next action. What does that look like. Is it a softmax function whose levels consist of all possible actions? That's intractable. Is it the same, but with a large set of actions being restricted? Do you take the set of all possible actions and coarse grain it, then use the softmax of that? (Again a very difficult problem).
However you can actually take my explanation of the Go learning process, find a database of Go games, and actually write the algorithm and apply it, because I gave all the relevant information needed. That's what I'm trying to figure out for Starcraft, and this is a really tough problem for all the reasons I explained.
Your idea for the Go problem would fail. It would play Go, but it would be no better than all the other attempts. It required an ingenious idea by DeepMind to make it actually work.
No, you just misinterpreted the point I was making in that passage. Remember, I said,
This would work well enough to train something at the very least, that's tractable and would give me an algorithm that can actually play Go.
The process I described would get something that plays Go, and in a way that's better than random moves, maybe as good as a novice player. I never claimed it would do anywhere nearly as well as AlphaGo. My entire point in that section was just how the simplicity of Go makes for a structure that already gives you such readily available training data sets for training a tractable model. Something that Starcraft doesn't even come close to doing. This is the main reason why training a Starcraft AI is much, much harder than a Go AI.
Your example also requires somebody to go through every single training set and give it a label. You can't sample 20 million battles from the data base and expect there to be the data available of who won that battle...
No. Seriously. You are trying very hard to make everything sound a lot harder than it is. This is NOT a hard problem. I could put a novice directly from the university at this for some time and he would come up with a decent idea for an automated process. I maybe even don't need human games. I could use the Expert AI and simulate a lot of battles. I only need this to get a starting point, because the rest would be done with self play.
The reason I said that you would need to label the sets manually is because in your original quote on the idea,
If we start at the micro end I would want one type of neural net for each unit type. I would train them by simulating, or by using replays, local battles. A humongous amount of battles of course. Unlike Go the outcome of the battle is very easy to see. I wouldn't even need to play it to the end. Once one side start to win I would end it and record a win or loss.
you talked about using "local battles" and "win/loss" when referring to the outcome of the battle (rather than the game). These are both very subjective terms. There is no notion of a "battle" in the data of a Starcraft game, nor is there any such thing as a win when it comes to a battle. These are all subjective terms that human spectators give to what they're seeing, based on very ad hoc conditions. For example, if a probe picks at an SCV while it's building a ramp depot, nobody considers that a battle. But if there's 10 probes trying to pick at that SCV, and then 13 SCVs are taken off the line to fend off the probes, everyone would call that a battle. And what happens if 8 probes and 12 SCVs die? Well someone observing that battle in isolation might say the Protoss won, but in the bigger picture it could very well be the Terran, especially if this was an all-in by the Protoss and the Terran's intent wasn't so much to get a more efficient trade, but to hold off the probes as well as possible before a hellion is pumped out.
I will concede this though. You could automate the process as long as you come up with a very strict definition of what constitutes a battle, and what constitutes a win/loss of a battle. It would have to be something like "a unit takes damage from another faction within a square grid of size K - centred around the damaged unit - within which at least N supply of both factions are present" where K and N are adjustable parameters. And the victory condition could be "the player who lost less resources worth of units between the time of battle initiation (as defined above) and M seconds after no unit takes any damage." Of course there are major problems with this definition (for example an algorithm trained like this would never retreat its whole army), but it would give you something that works without manual interference. Or, instead of trying to define a "victory" for a battle, you could just use whoever won the actual game, which is what AlphaGo did. You don't need to play an entire replay for that, the metadata surely has some binary variable that you can access. In fact, I think that would actually be a lot better, because then you'd have the network trained to do things for bigger picture reasons (such as retreating an army) rather than very tunnel-vision reasons.
DeepMind has inspired me to learn both programming and setting up neural networks.
Honestly, I've never had that much fun for the last decades of my life. I'm loving this so much.
All of this is pure beauty.
I came to believe that neural networks (and recent techniques like deep learning) are among the most meaningful inventions we ever created. On the same level as steam engines, computers, the internet... :D
Sorry, fanboy talk. I just wanted to get this off my chest
No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted.
There are plenty of BW AI's which are using genetic algorithms to decide strategy and AFAIK there are ones which are also using neural nets to help shape some of it's behaviour.
I'm not familiar with the script API either (or we should probably call it the map editor, because that's what I think it is), but I think it's pretty safe to assume that the API tells you if something is hidden behind the fog or war, so you don't have to map hack.
All of the unit data in the game is available if you wanted to use it but for the sake of not cheating you can restrict yourself to only things which would be seen by a human player. For instance in my AI I have an array of known enemy units and an array of currently visible units. I have a function that runs every 0.x seconds (0.5 or some value like that) which calculates which units a human would be able to see. If it sees a unit for the first time it adds it to the known units array and takes a unit out when it dies. I deliberately only ever access data (such as position/health) about units which are currently visisble so as to restrict the AI in a way simillar to a human player. But I can still use the array of "known enemy units" to make strategic descissions such as what units to counter with. Even if Blizzard does not provide some kind of API which only gives the AI human knowledge it would be easy enough for Google to write their own API to interact with and release so others can verify that it isn't cheating.
I've also been able to get genetic algorithms to work for making macro descissions so that the macro is completely unscripted and decisions such as when to expand / when to drone up / when to rush / what to build / what tech paths to follow are decided on the fly. So it is not even true that SC2 bots are scripted.
This was a demo from a long time ago before I got genetic algorithms to work. (The AI in this vid is attempting to immitate builds it's copied from online replays).
Your AI sucks and the casting is unprofessional
yes I know my AI sucks and would be ranked in bronze league. it's only in the very first stages of development and this is one of the first games it had ever played. Pretty much as soon as I could get a field test to work I uploaded a replay to /r/castit for an aspiring caster to get some casting experience with.
Hopefully no one minds me pimping my own work. I just thought it was relevent to the discussion. I've had some limited experience playing with neural networks and always think they are cool.
As for what counts as cheating mechanically and what restrictions should be placed on deep mind that is going to be different for each person. But honestly I don't care. An APM cap of say 300 (very possible for a pro human) seems reasonable but otherwise what's the point of restricting it? If Google can build a bot which can showcase top tier strategic play that will be amazing! No matter what restrictions they impose upon themselves or none at all it will be cool enough for me and I garuntee I would watch that showmatch having nerdgasms through the whole event. Maybe they could even use Archon mode to let the AI vs two human pros simultaniously so it can out micro AND out strategize two humans at once :D
On April 01 2016 05:00 The Bottle wrote: My input vector suggestion was better
Fine. Lets go with you input vector. You apparently want me to go inte details so you can get something to shoot down. It's a pointless excercise. The point with my example and this: "Your idea for the Go problem would fail." Was that we can both come up with AIs that can play badly, and training a neural net to micro a single SC2 unit in a half-intelligent way can't be that much harder than training a net to play an Atari game. I will readily agree that there probably is a lot more details in making an SC2 AI than a Go AI, but that's beside the point.
The goal is however not to make an AI that plays badly. This has already been done, both for Go and SC2. I think a good indication for how hard a problem is is to see how far you can get with a moderate effort. How far would you get by trying to make a Go AI from scratch? You wouldn't get anywhere. Trust me. This was an unsolved problem for years. There was no solution in sight, until the Monte Carlo engines came. The AIs STILL played like amateurs. Another revolutionary idea was required, this time by the DeepMind team. No one else in the world has managed this. This is how hard Go is. In hindsight you can say that the solution isn't very complex, and therefore not very hard (compared to SC2) but that's dishonest.
How hard is it to make an SC2 AI then? With moderate effort you get pretty far. You need no revolutionary ideas. This tells me that SC2 is an easier problem. Yes, I'm extrapolating. Then we can argue to days end about how hard it is to select training data for nets. Fine. Don't use nets for micro then. Handcode it. I said from the beginning that the problem of making an SC2 AI is only as hard as you make it, and therefore not very interesting.
On March 31 2016 03:35 Naracs_Duc wrote: Would it be more interesting if we have DeepMind get setup, then have a pro and programmer decide what "strategy" deepmind should execute perfectly and have them face off another deepmind + pro combo?
Flash with DeepMind micro vs Jaedong with DeepMind micro?
No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted.
There are plenty of BW AI's which are using genetic algorithms to decide strategy and AFAIK there are ones which are also using neural nets to help shape some of it's behaviour.
[
Why are all these AI's in BW? Where are the SC2 AIs? The only one I know of is Green Tea AI...
Why are all these AI's in BW? Where are the SC2 AIs?
Part of it is history. There is a healthy and competitive scene for BW AI's which has been around long enough to foster a community.
Mostly it is because there is an interface available for 3rd party tools to interact with BW. Blizzard obviously does not want to cooperate with 3rd party tools which would make cheating and hacking easier on the ladder. So basically BW AI's exist because Blizzard allows them to.
It is possible to create an AI internally through their scripting tools but there are many restrictions. As powerful as the galaxy scripting language is it does not compare to any of the usual programming languages. A lot of AI teams are university teams which would have to worry about liscencing for SC2 as well.
No bot today in Starcraft is doing a strategy that it gained from supervised or reinforced learning. It's all scripted.
There are plenty of BW AI's which are using genetic algorithms to decide strategy and AFAIK there are ones which are also using neural nets to help shape some of it's behaviour.
[
Why are all these AI's in BW? Where are the SC2 AIs? The only one I know of is Green Tea AI...
Making custom StarCraft 2 AI violates the TOS, and the scripting language in SC2 doesn't have file I/O which some bots rely on to store/retrieve megabytes of data.
On November 05 2016 04:21 SCHWARZENEGGER wrote: what if AI will be too smart and refuse to play terran?
What if AI figures out the best way to win is to complain about balance on balance forums in order to manipulate blizzard into patching the game in a certain way.
Or what about developing memes such as Gom TvT in order to make sure blizzard doesn't buff terran/nerf zerg for the next 2 years no matter how needed it is.
It's not the "real time" aspect that's hard, real time for powerful computers is nothing more than a very fast series of turns (I can assure you that's literally how that's going to be implemented ;D). I think the "incomplete information" part of the game is what's going to be tough. But if they at least give the computer perfect mechanics, it should be feasible to win against most pro-players most of the time. How often will be the measure of this AI project's success.
On November 05 2016 05:57 ZenithM wrote: It's not the "real time" aspect that's hard, real time for powerful computers is nothing more than a very fast series of turns (I can assure you that's literally how that's going to be implemented ;D). I think the "incomplete information" part of the game is what's going to be tough. But if they at least give the computer perfect mechanics, it should be feasible to win against most pro-players most of the time. How often will be the measure of this AI project's success.
Learning algorithms thrive in situations with randomness, so it will probably thrive with unknown information. What makes it difficult is the massive state space that Sc2 has compared to board games.
On November 05 2016 04:21 SCHWARZENEGGER wrote: what if AI will be too smart and refuse to play terran?
Or what about developing memes such as Gom TvT in order to make sure blizzard doesn't buff terran/nerf zerg for the next 2 years no matter how needed it is.
On November 05 2016 05:57 ZenithM wrote: It's not the "real time" aspect that's hard, real time for powerful computers is nothing more than a very fast series of turns (I can assure you that's literally how that's going to be implemented ;D). I think the "incomplete information" part of the game is what's going to be tough. But if they at least give the computer perfect mechanics, it should be feasible to win against most pro-players most of the time. How often will be the measure of this AI project's success.
Learning algorithms thrive in situations with randomness, so it will probably thrive with unknown information. What makes it difficult is the massive state space that Sc2 has compared to board games.
I think they will rather easily find ways to smartly discretize the search space. What I meant by "incomplete information" is that it may happen that in a certain state of balance of SC2, Build A beats Build B no matter what you do. It would look silly if that AI got eliminated by a couple of cheeses that somehow hard counters its build 2 times in a row.
But I think SC2 might be stable enough that there aren't many of these scenarios, if at all.
Edit: And in my humble experience (I don't know what's your own experience with machine learning), I wouldn't say with a straight face that "learning algorithms thrive in randomness situations". The less random my test set is, the happier I am :D
On November 05 2016 05:57 ZenithM wrote: It's not the "real time" aspect that's hard, real time for powerful computers is nothing more than a very fast series of turns (I can assure you that's literally how that's going to be implemented ;D). I think the "incomplete information" part of the game is what's going to be tough. But if they at least give the computer perfect mechanics, it should be feasible to win against most pro-players most of the time. How often will be the measure of this AI project's success.
Learning algorithms thrive in situations with randomness, so it will probably thrive with unknown information. What makes it difficult is the massive state space that Sc2 has compared to board games.
Not really, the 'partially observable' part definitely makes reinforcement learning harder. Two bits strike me as really really hard. The infrequent rewards part ( if you define a reward function as per the number of adverse units killed minus yours, for instance ) makes it hard, as this function might not move for the first five minutes of the game, so you don't get that many training examples. The planning aspect as well, since the AI will have to infer series of increasingly complex concepts, from 'click the mouse' to 'fake proxy gate into expanding'.
Great that they're outsourcing this, excited to try some algorithms when it's out.
Edit: And in my humble experience (I don't know what's your own experience with machine learning), I wouldn't say with a straight face that "learning algorithms thrive in randomness situations". The less random my test set is, the happier I am :D
Agreed, this is why regularization was invented. He probably meant that gradient descent is robust to initialization & noise.
On November 05 2016 07:36 coverpunch wrote: I'm curious - is the native AI in the game limited in APM?
If Deepmind proceeds as they have in the past for Atari games AI, their SC2 AI would automatically be limited to 2 or 3 actions per frame, so at 60fps, that's max 180APM.
On November 05 2016 07:36 coverpunch wrote: I'm curious - is the native AI in the game limited in APM?
no but they use their apm differently, as they cant box units, and have to move them individually. So an insane AI, which is about plat/diamond level, is generally playing at more than 1000 apm average
On November 05 2016 07:36 coverpunch wrote: I'm curious - is the native AI in the game limited in APM?
If Deepmind proceeds as they have in the past for Atari games AI, their SC2 AI would automatically be limited to 2 or 3 actions per frame, so at 60fps, that's max 180APM.
wouldnt that be 180 actions per second? That would be cool to see.
On November 05 2016 07:36 coverpunch wrote: I'm curious - is the native AI in the game limited in APM?
If Deepmind proceeds as they have in the past for Atari games AI, their SC2 AI would automatically be limited to 2 or 3 actions per frame, so at 60fps, that's max 180APM.
wouldnt that be 180 actions per second? That would be cool to see.
They stacked frames in Atari to have the algorithm see animations, I'll assume they'll do the same here. 180 actions per second is 10,000 APM, that'd be too much.
On November 05 2016 07:36 coverpunch wrote: I'm curious - is the native AI in the game limited in APM?
If Deepmind proceeds as they have in the past for Atari games AI, their SC2 AI would automatically be limited to 2 or 3 actions per frame, so at 60fps, that's max 180APM.
wouldnt that be 180 actions per second? That would be cool to see.
Yes. But it shouldn't be hard to limit it to any reasonable number.
To the people who think SC2 is not solvable (or is more difficult than Go for that matter) because it has so many possible moves: Who said you'd have to evaluate every single possible move to become good at the game? It's obviously not what we humans do.
On November 05 2016 07:36 coverpunch wrote: I'm curious - is the native AI in the game limited in APM?
If Deepmind proceeds as they have in the past for Atari games AI, their SC2 AI would automatically be limited to 2 or 3 actions per frame, so at 60fps, that's max 180APM.
The sc2 simulation runs at ~22hz regardless of framerate so that may be a relevant number
If they need the API to program the interface for the AI, then the AI is cheating. Human brains have to work through visual clutter aswell. Will the Ai have to zoom in to snipe units under collosi? Will it have to rotate the camera to check for widowmines behind extractors?
With the Atari games I thought they just used keyboard input and pixelvalues of the screen, why not do the same for sc2?
I don't know, but do people that are familiar with the BW API know if opening up that SC2 API will make hacks more prevelant?
On November 06 2016 05:23 JWD[9] wrote: If they need the API to program the interface for the AI, then the AI is cheating. Human brains have to work through visual clutter aswell. Will the Ai have to zoom in to snipe units under collosi? Will it have to rotate the camera to check for widowmines behind extractors?
With the Atari games I thought they just used keyboard input and pixelvalues of the screen, why not do the same for sc2?
I don't know, but do people that are familiar with the BW API know if opening up that SC2 API will make hacks more prevelant?
Lol I've never heard of anyone zooming in to snipe units under a colossus.
On November 06 2016 05:23 JWD[9] wrote: If they need the API to program the interface for the AI, then the AI is cheating. Human brains have to work through visual clutter aswell. Will the Ai have to zoom in to snipe units under collosi? Will it have to rotate the camera to check for widowmines behind extractors?
With the Atari games I thought they just used keyboard input and pixelvalues of the screen, why not do the same for sc2?
I don't know, but do people that are familiar with the BW API know if opening up that SC2 API will make hacks more prevelant?
As mentioned in the talk at Blizzcon, there will be an API that uses visual information and an API that works like the BWAPI for Brood War
The SC2 API won't be capable to be used for ladder games.
SC2 can't be solved. I'd expect the AI to find some near unbeatable early game cheese, and ultimately it'll probably reduce to randomizing between executing a few strategies perfectly.
On November 06 2016 05:23 JWD[9] wrote: If they need the API to program the interface for the AI, then the AI is cheating. Human brains have to work through visual clutter aswell. Will the Ai have to zoom in to snipe units under collosi? Will it have to rotate the camera to check for widowmines behind extractors?
With the Atari games I thought they just used keyboard input and pixelvalues of the screen, why not do the same for sc2?
I don't know, but do people that are familiar with the BW API know if opening up that SC2 API will make hacks more prevelant?
As mentioned in the talk at Blizzcon, there will be an API that uses visual information and an API that works like the BWAPI for Brood War
The SC2 API won't be capable to be used for ladder games.
Ah, then I am on board, go ahead, ForScience (and more sc2 longlivity )
On November 06 2016 06:49 paralleluniverse wrote: SC2 can't be solved. I'd expect the AI to find some near unbeatable early game cheese, and ultimately it'll probably reduce to randomizing between executing a few strategies perfectly.
Games don't have to be solvable for an AI to play it better than any person. GO is an perfect example of that.
On March 28 2016 13:39 JimmyJRaynor wrote: without heuristic functions i think AlphaStar will have trouble "learning" the game. and with heuristic functions it'll play with the style of dictated by them.
The whole point of unsupervised learning is the AI picking up the right heuristics by itself. In the match against Lee Se Dol, AlphaGo made an attack that no pro could predict in a game that it won. It saw something in the board state that made it do the winning move, because some heuristic it developed "triggered" on said board state.
EDIT: I actually think that the best advantage of a potential AI is individual unit control and AoE-immune spreads. StarCraft 2 default AI (the one that players trigger by rightclicking) is VERY BAD in aoe-related pathfinding, so that alone might be enough to make all other advantages irrelevant.
On November 06 2016 08:03 BluzMan wrote: EDIT: I actually think that the best advantage of a potential AI is individual unit control and AoE-immune spreads. StarCraft 2 default AI (the one that players trigger by rightclicking) is VERY BAD in aoe-related pathfinding, so that alone might be enough to make all other advantages irrelevant.
They are not going to do that tho, they aren't going to spend a ton of money in a super AI only to make it be a automaton 1000 bot that only makes perfectly microed marines.
I hope they limit it to around 200 APM. 200 APM with 100% perfect mouse precision and never any errors in terms of building supplies is still very good and could battle most pro's in terms of mechanics.
This restriction will mean that they need to beat pro's through strategy.
I will be waiting eagerly how they will be designing this challenge to reduce the risk that the challenge will just be human versus god-tier micro-master robot.
What they did with go was amazing so its cool to see them try this. The only issue is how APM dependent this game is compared to say go. Its a huge huge advantage to have higher apm compared to say a person so they certainly need to account for that. I think a game like civilization 6 would be a better fit where its turn based and has good complexity but not dependant on apm. I guess there arent any replays or pro civilization 6 players though for them to use Also I think brood war would have been a better choice then starcraft 2 but I guess it doesnt really matter. Brood war uses a lot less resources and is more time tested with a longer history of competitive play
I don't get why so many people are focused on APM here. Much more interesting to me is the question how will the AI take its inputs. For GO it had an operator input the opponent's move in its interface, a no-go option for SC. For SC it may be image recognition based on video capture or framebuffer access. But I am very skeptical they can process that much raw data with a neural net in real time. If they manage to do it, that alone would be mighty impressive, regardless of if it beats the top human players or not. More likely to me seems that they would use some direct access to game state memory - probably with some intermediate layer that prevents it from getting data outside of the fog of war. It'll probably require Blizzard's help to develop and will always leave some doubt into the fairness of the accessible information.
On November 07 2016 05:28 georgir wrote: I don't get why so many people are focused on APM here. Much more interesting to me is the question how will the AI take its inputs. For GO it had an operator input the opponent's move in its interface, a no-go option for SC. For SC it may be image recognition based on video capture or framebuffer access. But I am very skeptical they can process that much raw data with a neural net in real time. If they manage to do it, that alone would be mighty impressive, regardless of if it beats the top human players or not. More likely to me seems that they would use some direct access to game state memory - probably with some intermediate layer that prevents it from getting data outside of the fog of war. It'll probably require Blizzard's help to develop and will always leave some doubt into the fairness of the accessible information.
I'm very curious about what they are planning...
Blizzard has stated that they have worked together with DeepMind to work on an API that they are going to release soon-ish. This API will give access to the game state as well as some different interfaces for the minimap and terrain ect.
On November 07 2016 04:38 snakeeyez wrote: What they did with go was amazing so its cool to see them try this. The only issue is how APM dependent this game is compared to say go. Its a huge huge advantage to have higher apm compared to say a person so they certainly need to account for that.
The AI will have an apm limit in some form.
On November 07 2016 04:38 snakeeyez wrote: I think a game like civilization 6 would be a better fit where its turn based and has good complexity but not dependant on apm. I guess there arent any replays or pro civilization 6 players though for them to use
GO proved that they can a turn based game that has a big complexity. Starcraft is way more interesting because it 3 asymmetric races and real time.
On November 07 2016 04:38 snakeeyez wrote: Also I think brood war would have been a better choice then starcraft 2 but I guess it doesnt really matter. Brood war uses a lot less resources and is more time tested with a longer history of competitive play
With SC2 they got the chance to work with blizzard and create an API conforming to their needs. Something that would have never happened with BW.
On November 07 2016 05:38 sabas123 wrote: GO proved that they can a turn based game that has a big complexity.
I don't quite agree. Go has a lot of complexity, but still input for it is very simple. The grid is tiny relative to a Civilization game. The scale of the neural net might very well be unfeasibly large if applied to Civilization. Or Starcraft, for that matter. I expect they will have to come up with an entirely new paradigm instead of the grid to represent game state for the AI.
I think civilization would be an interesting game for something like this although the problem is that its not really played competitively in the same sense that starcraft is so replays would be an issue. Its certainly a complicated game, and current AI struggles to do much of anything without large amounts of cheating. It would be even more impressive if they could take this system playing starcraft after winning, and feed it the civilization 6 rules for it to learn and win at that without any human changes. I dont know if its possible because then you have true AI which I think is still out of reach. Neural nets are tech from like 1960s its nothing new this deep learning stuff just a revisit to older stuff. Its interesting to see a team of real professional AI programmers with google resources try to tackle a game as complicated as starcraft. I think they will do it
On November 07 2016 04:38 snakeeyez wrote: What they did with go was amazing so its cool to see them try this. The only issue is how APM dependent this game is compared to say go. Its a huge huge advantage to have higher apm compared to say a person so they certainly need to account for that. I think a game like civilization 6 would be a better fit where its turn based and has good complexity but not dependant on apm. I guess there arent any replays or pro civilization 6 players though for them to use Also I think brood war would have been a better choice then starcraft 2 but I guess it doesnt really matter. Brood war uses a lot less resources and is more time tested with a longer history of competitive play
The thing about APM is that it's a challenge, but not a completely unsolved one. Watson, when it was doing Jeopardy, internally set up time limits for itself to just pick whatever "best" solutions in a given step it had after a given time so that it wouldn't take too long finding the best answer for a question. I imagine something similar could be implemented as a policy for a starcraft AI, where upon registering a new piece of information, internal deadlines are set up so that after a certain amount of time (probably variant) the best solution is picked and further computation is halted
Is such a bad idea test deepmind with SC2 instead of BW. At SC2 it can kill almost anything with 16 marines and 2 medivacs. Well, at least we will see how bad is it the balance with unlimited micro skill.
is the vod for the presentation available anywhere? I have a virtual ticket but I didn't watch the presentation live as I was expecting it to be put up in the video archive, but so far it hasn't. in fact, its the only streamed footage of the entire blizzcon that hasn't been put up in the archive. why?
On November 07 2016 05:38 sabas123 wrote: GO proved that they can a turn based game that has a big complexity.
I don't quite agree. Go has a lot of complexity, but still input for it is very simple. The grid is tiny relative to a Civilization game. The scale of the neural net might very well be unfeasibly large if applied to Civilization. Or Starcraft, for that matter. I expect they will have to come up with an entirely new paradigm instead of the grid to represent game state for the AI.
I did some rough calculations the other day:
The most important information you have to store in a game state is:
unit type unit position current unit health current command of unit weapon cooldown status, etc.
Considering that a command can have a target position, we'd have to store two positions. If we use tile resolution (e.g. 192x192 tiles for a map) and assuming 34 different unit types (includes buildings), max. 250 units per player, then we have to store roughly:
2x250 times: 2x192x192 (two positions) + 2x34 (unit type for two races) + 1500 (hit points) + current command / cooldown / etc.
+ some overhead for current upgrades / etc. which can be stored globally rather than per unit.
-> One unit state would probably fit into 48 bits, give or take a few. -> A game state would fit into 2x250 x 48 bits =< 24kbits give or take a few
A 256x256 pixel image with 10bit color depth requires 640kbits.
This is just to illustrate, without drawing any conclusions I could also have done some horrible mistakes, it's pretty late...
PS: note that this way the game state is not stored in a grid, but in an array of units
On November 07 2016 05:38 sabas123 wrote: GO proved that they can a turn based game that has a big complexity.
I don't quite agree. Go has a lot of complexity, but still input for it is very simple. The grid is tiny relative to a Civilization game. The scale of the neural net might very well be unfeasibly large if applied to Civilization. Or Starcraft, for that matter. I expect they will have to come up with an entirely new paradigm instead of the grid to represent game state for the AI.
I did some rough calculations the other day:
The most important information you have to store in a game state is:
unit type unit position current unit health current command of unit weapon cooldown status, etc.
Considering that a command can have a target position, we'd have to store two positions. If we use tile resolution (e.g. 192x192 tiles for a map) and assuming 34 different unit types (includes buildings), max. 250 units per player, then we have to store roughly:
2x250 times: 2x192x192 (two positions) + 2x34 (unit type for two races) + 1500 (hit points) + current command / cooldown / etc.
+ some overhead for current upgrades / etc. which can be stored globally rather than per unit.
-> One unit state would probably fit into 48 bits, give or take a few. -> A game state would fit into 2x250 x 48 bits =< 24kbits give or take a few
A 256x256 pixel image with 10bit color depth requires 640kbits.
This is just to illustrate, without drawing any conclusions I could also have done some horrible mistakes, it's pretty late...
PS: note that this way the game state is not stored in a grid, but in an array of units
You don't just account for the current game state, You account for all the possible branches to other states, and so on. Complexity increases exponentially.
I know this is about starcraft, but what about storing the game state for say civilization 6? Would civilization 6 be harder or easier then starcraft 2?
I echo the sentiment that they should have went with BW instead of SC2. One has a mature API that has gone through enough testing that it is pretty darn solid, with enough history that they have work to build on (most notably from the Berkeley research groups). With SC2 they are starting completely from scratch with a new API that is guaranteed to be glitchy and unreliable (the inevitable result of new software).
Frankly this choice strikes me as something that is probably made more from a PR perspective than from any intellectual merit of choosing SC2 as the platform for AI testing.
On November 07 2016 05:38 sabas123 wrote: GO proved that they can a turn based game that has a big complexity.
I don't quite agree. Go has a lot of complexity, but still input for it is very simple. The grid is tiny relative to a Civilization game. The scale of the neural net might very well be unfeasibly large if applied to Civilization. Or Starcraft, for that matter. I expect they will have to come up with an entirely new paradigm instead of the grid to represent game state for the AI.
I did some rough calculations the other day:
The most important information you have to store in a game state is:
unit type unit position current unit health current command of unit weapon cooldown status, etc.
Considering that a command can have a target position, we'd have to store two positions. If we use tile resolution (e.g. 192x192 tiles for a map) and assuming 34 different unit types (includes buildings), max. 250 units per player, then we have to store roughly:
2x250 times: 2x192x192 (two positions) + 2x34 (unit type for two races) + 1500 (hit points) + current command / cooldown / etc.
+ some overhead for current upgrades / etc. which can be stored globally rather than per unit.
-> One unit state would probably fit into 48 bits, give or take a few. -> A game state would fit into 2x250 x 48 bits =< 24kbits give or take a few
A 256x256 pixel image with 10bit color depth requires 640kbits.
This is just to illustrate, without drawing any conclusions I could also have done some horrible mistakes, it's pretty late...
PS: note that this way the game state is not stored in a grid, but in an array of units
You don't just account for the current game state, You account for all the possible branches to other states, and so on. Complexity increases exponentially.
My calculation was in response to game state size of go vs sc2. Game state and branching factor are two entirely different things. and no, you do not have to explore all possible branches, as I explained in a previous post.
On November 07 2016 13:55 LegalLord wrote: I echo the sentiment that they should have went with BW instead of SC2. One has a mature API that has gone through enough testing that it is pretty darn solid, with enough history that they have work to build on (most notably from the Berkeley research groups). With SC2 they are starting completely from scratch with a new API that is guaranteed to be glitchy and unreliable (the inevitable result of new software).
Frankly this choice strikes me as something that is probably made more from a PR perspective than from any intellectual merit of choosing SC2 as the platform for AI testing.
The bold part might be the reason they want to do SC2 instead of BW. They might not want to publish there work based on code or API done by other group of researchers so they went to SC2 instead. Plus, the work on BW would not be unique (which a lot of academics care about).
On November 07 2016 13:55 LegalLord wrote: I echo the sentiment that they should have went with BW instead of SC2. One has a mature API that has gone through enough testing that it is pretty darn solid, with enough history that they have work to build on (most notably from the Berkeley research groups). With SC2 they are starting completely from scratch with a new API that is guaranteed to be glitchy and unreliable (the inevitable result of new software).
Frankly this choice strikes me as something that is probably made more from a PR perspective than from any intellectual merit of choosing SC2 as the platform for AI testing.
The bold part might be the reason they want to do SC2 instead of BW. They might not want to publish there work based on code or API done by other group of researchers so they went to SC2 instead. Plus, the work on BW would not be unique (which a lot of academics care about).
To put that concern not-so-nicely, I would say that that sounds like they put their egos before the matter of doing what has the best chance of working. That is a distinct possibility but it's retreading old ground for really unproductive reasons.
It's been stated before, but I'm saying it again because so many people aren't reading about this and making conclusions. The AI deepmind wants to develop will be limited to play within the limits of human dexterity. This has been confirmed already.
The hardest part of writing this AI will be combining different types of information and how they are related to the probability of winning the game. AlphaGo was made out of three fairly basic pieces of software: A search tree to examine moves, a neural network that estimates how good a board position is, and another neural network that assigns probabilities to a certain players move based on that board position. Honestly AlphaGo is not even that impressive once you see what it really is, although the solution was rather ingenious.
Starcraft is different. Because the game has hidden information, knowing the full state of the game requires memory of past events. Also the quality of your information decays over time if you haven't scouted. There is definitely a whole meta-level aspect of the thought process that involves trying to figure out efficient ways to get more information about what your opponent is doing, because information itself is a resource. AlphaGo could be given an in-progress Go match and begin playing. Any Starcraft AI will depend more heavily on the internal state and to be put into the middle of a game with fog of war, and no prior knowledge, it will have to start from square one. Not that this would happen, however.
Starcraft also requires specific sub-goals that may contradict one-another. In Go you have a single task for every turn: place the piece that increases probability of winning the most. In Starcraft you have to do multiple things at the same time and decide which of them are the most important. Also these goals are abstract and require their own evaluation of success or failure. For example you may want to conduct a drop that kills workers. This action will have to be planned with the specific goal of killing workers and is basically totally separate from everything else you're doing, for the most part. However this still has to be baked into the overall evaluation of "what is my probability of winning." The thing about Starcraft is that there is basically infinite possibilities here. You could try hand-coding a bunch of different ways to look at the game, but that's not efficient nor does it really align with Deepmind's overall goals. Deepmind needs to figure out a way to make their AI have a sense about the game and create its own plan. Not an easy task.
I don't know why people are getting hung up on the issue of BW vs SC2. If you seriously think that Deepmind is going to fail because they chose SC2 then you're being really petty. For one thing they're working directly with the developers of SC2 to create this API so any concerns about how it isn't going to work are just ridiculous in my opinion. Do you really think that Blizzard devs don't know how to program or something?
I think the BW vs. SC2 is mainly because there are already AI contests for BW, and the structure for building AI is already in place. However, I don't see any problem working on SC2. The DeepMind people might even have gone to Blizzard and suggested BW due to the available computational infrastructure, but Blizzard wants to push the game they are still making money with, and suggested they can open up a similar API for SC2. With the support of DeepMind (Google) and Blizzard, I don't doubt that a lot of the current AI development for BW will port their solutions to SC2, and the AI community for SC2 will get a major boost, because lets face it: for AI development it makes very little difference whether it's SC2 or BW (the major problems that need to be addressed are the same for both), but the direct support of Blizzard (and Google) for one will be a major push for that game.
Or, just see it as dedgaem rancor flowing over into this thread
On November 07 2016 19:32 Acrofales wrote: I think the BW vs. SC2 is mainly because there are already AI contests for BW, and the structure for building AI is already in place. However, I don't see any problem working on SC2. The DeepMind people might even have gone to Blizzard and suggested BW due to the available computational infrastructure, but Blizzard wants to push the game they are still making money with, and suggested they can open up a similar API for SC2. With the support of DeepMind (Google) and Blizzard, I don't doubt that a lot of the current AI development for BW will port their solutions to SC2, and the AI community for SC2 will get a major boost, because lets face it: for AI development it makes very little difference whether it's SC2 or BW (the major problems that need to be addressed are the same for both), but the direct support of Blizzard (and Google) for one will be a major push for that game.
Or, just see it as dedgaem rancor flowing over into this thread
When I was watching it I thought that the game will have the possibility to regain some interest, not only in AI research scene with how ridiculously high levels of interest in that Go match against Lee Sedol were - especially in Korea.
On November 07 2016 19:32 Acrofales wrote: I think the BW vs. SC2 is mainly because there are already AI contests for BW, and the structure for building AI is already in place. However, I don't see any problem working on SC2. The DeepMind people might even have gone to Blizzard and suggested BW due to the available computational infrastructure, but Blizzard wants to push the game they are still making money with, and suggested they can open up a similar API for SC2. With the support of DeepMind (Google) and Blizzard, I don't doubt that a lot of the current AI development for BW will port their solutions to SC2, and the AI community for SC2 will get a major boost, because lets face it: for AI development it makes very little difference whether it's SC2 or BW (the major problems that need to be addressed are the same for both), but the direct support of Blizzard (and Google) for one will be a major push for that game.
Or, just see it as dedgaem rancor flowing over into this thread
Basically the argument here is that there is a PR advantage for Blizzard to push it from the SC2 side. That is also what I think it is, because for all intents and purposes the BW AI project is further along (though far, far from any feasible solution with current technology) and the technical argument would make BW be the better choice, easily.
This all makes me question the integrity of this as a project for AI from an academic perspective, and instead makes me feel like this is a "Google hotshots go after an even bigger leviathan than Go" PR stunt.
Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your question: yes. Blizzard is making an API for DeepMind to use here. Specifically for this purpose.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way. This is because by choosing SC2, they are throwing out a lot of highly valuable and relevant progress towards a well-developed and mature, if horribly incomplete, project. And Google isn't some genius that can simply toss aside that much progress, "pave their own path," and expect it will somehow work out for the best. It's not true.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
[...] they are throwing out a lot of highly valuable and relevant progress towards a well-developed and mature, if horribly incomplete, project.[...].
Yes it is incomplete. At the same time, after being involved in the community for (only) two months, I can pretty safely state the BW AI Bot community is currently at its peak, better than ever. And it will continue to go upwards. I will try hard myself to make my small contribution to make it more complete.
@Jett.Jack.Alvir: I did reply to your latest comment in the blog. Not sure if you are gonna be satisfied with the answer though
Well regardless whether they are or aren't pushing the boundaries of AI learning, I think doing SC2 will be more fun to watch.
Don't get me wrong, I love BW and watched it plenty of times, but I love SC2 more.
I see your point though, there is already so much progress with BW bots that choosing SC2 seems like an illogical choice.
edit: imp42, I read it and I am satisified. I really like your method of developing a bot. I feel like its capable of being ported into SC2 because of its simplistic approach. Keep up the awesome work!
but the focus on SC2 makes is questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way.
Why? What else do you think DeepMind is looking to do?
To be a PR stunt. To make the headlines look like: "Google is at it again! Geniuses who solved Go now move on to an even bigger leviathan: Starcraft!"
Google does some interesting research for sure, but ultimately it's a company that wants money. If Blizzard offered Google a sweetheart deal to push SC2 despite that being an academically foolhardy choice, then they will go with SC2. And that is what I think happened here.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that BW AI scene is now established? Can you provide solid reasons instead of what you believe is true?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe parts of the code alone.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
From an AI perspective, SC2 and BW are basically the same problem. If they "solve" one then it's a practical task of solving the other, not a research task. The problem of real-time decision making in an environment that many think approximates a general intelligence decision problem is far more daunting than training stuff on a new game.
What they throw out is substantial. The code, as you mentioned, is one of those things - and that is far from trivial (companies have been destroyed by the decision to rewrite their code from scratch while throwing out all their legacy code). The second is expertise - they lose the ability to benefit fully from hiring BW people who are BW experts who have worked on BW for years. And the third and related topic is community. They lose all the collaborators they have who have worked on BW for this long because while the approach is the same, they stunt their ability to collaborate when they aren't even playing the same game. And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
It just doesn't make sense. Not as an academic endeavor anyways. I think Blizzard just gave them a sweetheart deal to promote a newer game, and that's a losing strategy if they want real academic results.
Using DeepMind on SC2 over SC1 is super exciting. SC1 has had 2 decades of learning and discovery. It'd be great to see what a powerful AI can do with an RTS that is still in its infancy. So many new discoveries for pro players can come from this. The only reason for SC1 to have deep mind is so that the AI will have a true challenger. Like in Go, you would want the AI to play the best possible opponent and this case the AI could play versus Flash (I'd include Bisu, but clearly the AI would be too intimidated by his looks to concentrate)..
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
This is a missed argument since starter edition provides all the potential AI research needs. You just don't need to buy the full game and the API will be free and made - as they said - accessible to variety of people, not just academic researchers so the interest in potential research (or the game itself by the way) may rise among many people.
I completely disagree with your opinion that they made the deal to promote SC2 first.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that BW AI scene is now established? Can you provide solid reasons instead of what you believe is true?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I think I can give you plenty reasons:
- BW is much more stable. Who knows if Blizzard is going to release a new patch. Optimistically you could say that's the challenge that Deepmind wants. Realistically you have to admit it if they really want to deal with changing rules it would be much better for them to have those rules under their own control (by e.g. tweaking units in a map editor) without having any conflict of interest created by the fact Blizzard also has to support an active player base.
- Guess which program is going to be more light-weight to execute. A program developed in 1998, runnable on Windows98, or Sc2? There already exists a "headless" version of BW (no graphics) and you can easily create pretty much any API you want.
- compared to Sc2 BW is simpler in terms of possible moves, but arguably* deeper strategically. If you're really interested in "real" AI you would want to strip any unnecessary complexity and focus on the core issue.
- BW has a low resolution, making the jump from Atari games more reasonable if you want to go the pixel interpretation way.
- As has been said before, there is already research available on BW. The statement at the Blizzard panel that "all BW bots are scripted" is not a 100% true. Approaches using Neural Nets have been explored as well.
* arguably deeper: my personal opinion, no need to discuss it. If anybody thinks otherwise that's fine with me
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
This is a missed argument since starter edition provides all the potential AI research needs. You just don't need to buy the full game and the API will be free and made - as they said - accessible to variety of people, not just academic researchers so the interest in potential research (or the game itself by the way) may rise among many people.
I completely disagree with your opinion that they made the deal to promote SC2 first.
One may think that people doing research would want access to multiplayer since playing ladder games is a perfectly valid approach to training your AI.
It's not an insurmountable issue, but it's just not one that BW has. It's a game easily available for free.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that BW AI scene is now established? Can you provide solid reasons instead of what you believe is true?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I think I can give you plenty reasons:
- BW is much more stable. Who knows if Blizzard is going to release a new patch. Optimistically you could say that's the challenge that Deepmind wants. Realistically you have to admit it if they really want to deal with changing rules it would be much better for them to have those rules under their own control (by e.g. tweaking units in a map editor) without having any conflict of interest created by the fact Blizzard also has to support an active player base.
They said their goal with SC2 from now on is to not make any big changes and let the game grow on itself, with potential tweaks being just numbers, not big changes.
- Guess which program is going to be more light-weight to execute. A program developed in 1998, runnable on Windows98, or Sc2? There already exists a "headless" version of BW (no graphics) and you can easily create pretty much any API you want.
Valid reason behind BW, they still may do something about it for SC2 in the future though.
- compared to Sc2 BW is simpler in terms of possible moves, but arguably* deeper strategically. If you're really interested in "real" AI you would want to strip any unnecessary complexity and focus on the core issue.
I personally disagree since we do not know how the AI is going to evolve and how the game may be played differently when it may find more optimal way of playing than is considered a current highest level, but I didn't play BW much so I won't delve into discussion on comparing both games.
- BW has a low resolution, making the jump from Atari games more reasonable if you want to go the pixel interpretation way.
I personally think it may be a challenge for making the AI to more efficiently interpret visual cues and such on higher resolutions.
- As has been said before, there is already research available on BW. The statement at the Blizzard panel that "all BW bots are scripted" is not a 100% true. Approaches using Neural Nets have been explored as well.
* arguably deeper: my personal opinion, no need to discuss it. If anybody thinks otherwise that's fine with me
I won't argue on that since I've never been into AI research in BW more than an interesting thing to look at and think about.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your question: yes. Blizzard is making an API for DeepMind to use here. Specifically for this purpose.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way. This is because by choosing SC2, they are throwing out a lot of highly valuable and relevant progress towards a well-developed and mature, if horribly incomplete, project. And Google isn't some genius that can simply toss aside that much progress, "pave their own path," and expect it will somehow work out for the best. It's not true.
I kinda disagree that they are throwing it out. If it's interesting algorithms, they can pretty easily be assimilated. It shouldn't take much work to make learning algorithms designed for the BW API to work on the SC2 API (assuming Blizzard makes a decent SC2 API). If it's the work that has been done on perfect muta control bots, then I disagree with the premise, because that is not very interesting from a research point of view anyway (although it's pretty impressive from a mechanical perspective
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
This is a missed argument since starter edition provides all the potential AI research needs. You just don't need to buy the full game and the API will be free and made - as they said - accessible to variety of people, not just academic researchers so the interest in potential research (or the game itself by the way) may rise among many people.
I completely disagree with your opinion that they made the deal to promote SC2 first.
One may think that people doing research would want access to multiplayer since playing ladder games is a perfectly valid approach to training your AI.
It's not an insurmountable issue, but it's just not one that BW has. It's a game easily available for free.
They said they will never let the API to be used on ladder, but focus more on letting it interpret replays gathered from actual ladder games played by people and go from there.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your question: yes. Blizzard is making an API for DeepMind to use here. Specifically for this purpose.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way. This is because by choosing SC2, they are throwing out a lot of highly valuable and relevant progress towards a well-developed and mature, if horribly incomplete, project. And Google isn't some genius that can simply toss aside that much progress, "pave their own path," and expect it will somehow work out for the best. It's not true.
I kinda disagree that they are throwing it out. If it's interesting algorithms, they can pretty easily be assimilated. It shouldn't take much work to make learning algorithms designed for the BW API to work on the SC2 API (assuming Blizzard makes a decent SC2 API). If it's the work that has been done on perfect muta control bots, then I disagree with the premise, because that is not very interesting from a research point of view anyway (although it's pretty impressive from a mechanical perspective
Algorithms are primarily mathematical, so they would be able to use those with some primarily practical tweaking. The codebase already made, and the BW AI community to collaborate with, that is lost by going to SC2.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
This is a missed argument since starter edition provides all the potential AI research needs. You just don't need to buy the full game and the API will be free and made - as they said - accessible to variety of people, not just academic researchers so the interest in potential research (or the game itself by the way) may rise among many people.
I completely disagree with your opinion that they made the deal to promote SC2 first.
One may think that people doing research would want access to multiplayer since playing ladder games is a perfectly valid approach to training your AI.
It's not an insurmountable issue, but it's just not one that BW has. It's a game easily available for free.
They said they will never let the API to be used on ladder, but focus more on letting it interpret replays gathered from actual ladder games played by people and go from there.
Then that is a reduction of AI capabilities for stupid reasons. BW AI do sometimes train themselves in ladder games. At any rate they are capable of doing so.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
This is a missed argument since starter edition provides all the potential AI research needs. You just don't need to buy the full game and the API will be free and made - as they said - accessible to variety of people, not just academic researchers so the interest in potential research (or the game itself by the way) may rise among many people.
I completely disagree with your opinion that they made the deal to promote SC2 first.
One may think that people doing research would want access to multiplayer since playing ladder games is a perfectly valid approach to training your AI.
It's not an insurmountable issue, but it's just not one that BW has. It's a game easily available for free.
They said they will never let the API to be used on ladder, but focus more on letting it interpret replays gathered from actual ladder games played by people and go from there.
Then that is a reduction of AI capabilities for stupid reasons. BW AI do sometimes train themselves in ladder games. At any rate they are capable of doing so.
You don't know if there is not going to be a way of automating or simulating the matchmaking the ladder has in the package they are going to release. Why do you think letting bots into a game where people play is good?
On November 08 2016 02:27 LegalLord wrote: [quote] On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that the scene is now established?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I don't hate SC2 at all. I simply don't see their rationale, from an academic perspective, of actually doing SC2, precisely because they are throwing out years of progress by those who made AI before them. Among other things, there is a 24/7 AI test stream for BW and countless works by Berkeley and others, all done with BW.
DeepMind can make their own decisions, but I do not believe they are made from an academically wise perspective. They are basically retreading old ground for no academically justifiable reason.
I'd love to hear why is it wise to make the research behind each choice of games, and it's weird to me to think that the progress made in BW has to be largely scrapped and can't be moved over to a game of the same genre, but which has more complex and challenging choices on the example of macro mechanics which BW doesn't have, BW macro mechanic is ordering newly trained workers to mine which is not choice-based but mechanically demanding.
I can't understand how changing the game but staying within the same genre is "throwing out years of progress" in other thing than maybe code alone.
And I suppose fourth is availability - access to BW copies is much easier than access to SC2 copies, which makes it easier to collaborate with people who are conscious about spending money on lots of copies of SC2.
This is a missed argument since starter edition provides all the potential AI research needs. You just don't need to buy the full game and the API will be free and made - as they said - accessible to variety of people, not just academic researchers so the interest in potential research (or the game itself by the way) may rise among many people.
I completely disagree with your opinion that they made the deal to promote SC2 first.
One may think that people doing research would want access to multiplayer since playing ladder games is a perfectly valid approach to training your AI.
It's not an insurmountable issue, but it's just not one that BW has. It's a game easily available for free.
They said they will never let the API to be used on ladder, but focus more on letting it interpret replays gathered from actual ladder games played by people and go from there.
Then that is a reduction of AI capabilities for stupid reasons. BW AI do sometimes train themselves in ladder games. At any rate they are capable of doing so.
You don't know if there is not going to be a way of automating or simulating the matchmaking the ladder has in the package they are going to release. Why do you think letting bots into a game where people play is good?
Do you have a problem with playing with a robot that is about your skill level if you have no way of knowing whether or not it is actually a robot you're playing?
No one in BW ever has had a problem with that. It's so inconsequential relative to the size of the playerbase that it really just doesn't matter. But for training the AI it absolutely does matter.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way
On the other hand you may just hate Blizzard too much, why BW is better than SC2 for potential AI? Just because of the fact that BW AI scene is now established? Can you provide solid reasons instead of what you believe is true?
I think people at Deepmind are much more qualified to make such assumptions or comparisons, if they really thought BW would be better for it they would do it in BW. Or maybe evil Mike bought Google and their Deepmind team?
I think I can give you plenty reasons:
- BW is much more stable. Who knows if Blizzard is going to release a new patch. Optimistically you could say that's the challenge that Deepmind wants. Realistically you have to admit it if they really want to deal with changing rules it would be much better for them to have those rules under their own control (by e.g. tweaking units in a map editor) without having any conflict of interest created by the fact Blizzard also has to support an active player base.
- Guess which program is going to be more light-weight to execute. A program developed in 1998, runnable on Windows98, or Sc2? There already exists a "headless" version of BW (no graphics) and you can easily create pretty much any API you want.
- compared to Sc2 BW is simpler in terms of possible moves, but arguably* deeper strategically. If you're really interested in "real" AI you would want to strip any unnecessary complexity and focus on the core issue.
- BW has a low resolution, making the jump from Atari games more reasonable if you want to go the pixel interpretation way.
- As has been said before, there is already research available on BW. The statement at the Blizzard panel that "all BW bots are scripted" is not a 100% true. Approaches using Neural Nets have been explored as well.
* arguably deeper: my personal opinion, no need to discuss it. If anybody thinks otherwise that's fine with me
Stability is actually a point I agree with you on. But maybe Blizzard considers SC2 in its current version stable enough that big balance tweeks to multiplayer will no longer happen, just as they haven't happened to BW. Is SC2 perfectly balanced? No. But BW isn't either. Not sure what balance patches are planned, but major balance updates would mean AI needs retraining, which is a major pain in the ass.
SC2 is not heavy-weight enough to really worry about. Yes, SC2 will eat more CPU cycles. But when you're using a couple of orders of magnitude MORE cycles for your AI, the difference between SC2 and BW becomes a rounding error.
Strategic depth is a matter of taste. Lets not bring that discussion over here.
Resolution is not all that relevant for image recognition: it's more about how much stuff is on the screen, which would be about the same, given that the increased resolution is used by making the units look prettier, rather than fitting (significantly) more units on the screen. In any case, computer vision algorithms would have a bloody hard time on SC, and the initial phase will almost certainly be the API providing information about what is on the screen directly from the engine, rather than a CV module analyzing that. I do agree that the simplicity of being able to match a (very restricted) number of sprites makes the CV easier than the 3D model approach of SC2, but I am not enough of a CV expert to be able to say how much easier BW is in this department.
I already discussed the point about prior research above.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your question: yes. Blizzard is making an API for DeepMind to use here. Specifically for this purpose.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way. This is because by choosing SC2, they are throwing out a lot of highly valuable and relevant progress towards a well-developed and mature, if horribly incomplete, project. And Google isn't some genius that can simply toss aside that much progress, "pave their own path," and expect it will somehow work out for the best. It's not true.
I kinda disagree that they are throwing it out. If it's interesting algorithms, they can pretty easily be assimilated. It shouldn't take much work to make learning algorithms designed for the BW API to work on the SC2 API (assuming Blizzard makes a decent SC2 API). If it's the work that has been done on perfect muta control bots, then I disagree with the premise, because that is not very interesting from a research point of view anyway (although it's pretty impressive from a mechanical perspective
Algorithms are primarily mathematical, so they would be able to use those with some primarily practical tweaking. The codebase already made, and the BW AI community to collaborate with, that is lost by going to SC2.
Well, I presume that having both Blizzard and Google support a new SC2 AI community would cause some drive for the current BW AI community to switch. Note that most research groups working on BW AI are not particularly attached to BW. They just like the tools that the BW community has available. If Blizzard and Google make better tools available (and interesting tournaments), then research groups will switch faster than stalkers can blink.
On November 08 2016 03:59 Jett.Jack.Alvir wrote: that is only if they release the api to other institutions. it may be that google gets the api only, so no one else can switch from bw
That would be extremely dumb from a PR point of view. They also said the API would be publicly available.
BW is very close to being completely reverse engineered and open sourced. This is a screenshot the developer just released from his own BW engine: http://i.imgur.com/rdEcBYM.jpg
This is obviously big news and will deserve its own thread in due time. Take it as a teaser for now
I am not personally involved, just the messenger. The genius behind the project is an active BW Bot developer.
On November 08 2016 02:22 Jett.Jack.Alvir wrote: Well this would be a huge PR stunt for both Blizzard and Google. SC2 would get huge coverage, look at Go, and Google would push the boundaries of AI learning.
So Blizzard is allowing DeepMind to access SC2 api? I thought that DeepMind was looking at BW because there is no api restrictions.
On your question: yes. Blizzard is making an API for DeepMind to use here. Specifically for this purpose.
On your earlier statement: it's a PR stunt, yes, but the focus on SC2 makes it questionable whether or not they are really looking to "push the boundaries of AI learning" in any appreciable way. This is because by choosing SC2, they are throwing out a lot of highly valuable and relevant progress towards a well-developed and mature, if horribly incomplete, project. And Google isn't some genius that can simply toss aside that much progress, "pave their own path," and expect it will somehow work out for the best. It's not true.
I kinda disagree that they are throwing it out. If it's interesting algorithms, they can pretty easily be assimilated. It shouldn't take much work to make learning algorithms designed for the BW API to work on the SC2 API (assuming Blizzard makes a decent SC2 API). If it's the work that has been done on perfect muta control bots, then I disagree with the premise, because that is not very interesting from a research point of view anyway (although it's pretty impressive from a mechanical perspective
Algorithms are primarily mathematical, so they would be able to use those with some primarily practical tweaking. The codebase already made, and the BW AI community to collaborate with, that is lost by going to SC2.
Well, I presume that having both Blizzard and Google support a new SC2 AI community would cause some drive for the current BW AI community to switch. Note that most research groups working on BW AI are not particularly attached to BW. They just like the tools that the BW community has available. If Blizzard and Google make better tools available (and interesting tournaments), then research groups will switch faster than stalkers can blink.
And they might just draw more people in in general. If they do, I'm all for it, since another API won't hurt anyone. But Blizzard's track record of peaceful transitions is spotty, and there will still be a new community to rebuild, and a codebase too. It will take years before the community is as productive as what BW has now. So DeepMind takes a decidedly less productive approach here.
Though frankly, I think to a large extent Blizzard missed the boat by waiting this long to even announce that they are making the API. If this came out on SC2 release it would have caught on for sure. Now, it's a giant maybe. The community has waned since the release and that will not make a justification for the transition be as easy as it would be when the hype train was in full swing.
On November 06 2016 08:03 BluzMan wrote: EDIT: I actually think that the best advantage of a potential AI is individual unit control and AoE-immune spreads. StarCraft 2 default AI (the one that players trigger by rightclicking) is VERY BAD in aoe-related pathfinding, so that alone might be enough to make all other advantages irrelevant.
They are not going to do that tho, they aren't going to spend a ton of money in a super AI only to make it be a automaton 1000 bot that only makes perfectly microed marines.
Geez, you only read and quoted one half of the post. AlphaGo is based partly on unsupervised learning, basically, it invents algorithms by itself. If it ever finds a game mechanic loophole that gives it unholy winrates without really "understanding" the game, it stops learning, because it completely loses reference (you have no idea what to improve when you win all the time). The only thing it can do then is play with itself, but if it's basis isn't strong enough by that time, it will keep discovering "features" that are not relevant to real games. Just like those poor sods that discovered "tech to carriers and make 12 of them" to be a good strategy because of total lack of competition.
Allowing very high APM might be a concern because it might uncover something that completely breaks the game (I personally think it's AoE). But it's not known if such things exist, and an AI would need all advantages it has to hope to beat a human.
If AI get hax speed performance then humans get multicommand hacks and automated micro bots. If the computers only win because they are not input-constrained by having physical bodies that have to be moved to provide input into the game then the entire problem has been trivialized.
On November 08 2016 07:01 LegalLord wrote: If AI get hax speed performance then humans get multicommand hacks and automated micro bots. If the computers only win because they are not input-constrained by having physical bodies that have to be moved to provide input into the game then the entire problem has been trivialized.
I think everyone agrees that there needs to be a very low limit to APM. I'd even argue that extremely low APM (like 180) and a small "between actions" limit that is similar to the time it takes humans to touch two different keys in succession on a keyboard would be much more interesting because it would require the computer to "outthink" humans instead of "outdoing" us. Also, maybe having the program not "remember" how many resources it has unless it spends one of those actions to "look" at its money (and things like that) would be helpful too.
On November 08 2016 07:01 LegalLord wrote: If AI get hax speed performance then humans get multicommand hacks and automated micro bots. If the computers only win because they are not input-constrained by having physical bodies that have to be moved to provide input into the game then the entire problem has been trivialized.
I think everyone agrees that there needs to be a very low limit to APM. I'd even argue that extremely low APM (like 180) and a small "between actions" limit that is similar to the time it takes humans to touch two different keys in succession on a keyboard would be much more interesting because it would require the computer to "outthink" humans instead of "outdoing" us. Also, maybe having the program not "remember" how many resources it has unless it spends one of those actions to "look" at its money (and things like that) would be helpful too.
They mentioned a lot of things like that during the panel, actually about resources they said they may just pass it the numbers. It was also mentioned that the restrictions on the AI will also be entirely based on the game interface people use, so selecting things will involve selecting single units as well as boxing over them (with ctrl+click and control groups implied I think). I'm interested in how will they deal with reacting to things happening on the minimap. I bet APM constraints may change on how it all develops.
After all the project is about creating a self-learning AI that would simulate the process of how human thinks, not just creating a bot that plays the game as best as possible - although that is the goal, but it's a goal set for AI, not researchers.
I'm super excited thinking about how FPV of the AI will look like when significant progress is made. It has to be a regular stream on twitch at some point, haha.
You know what would be fucking cool? If Blizzard opened up a specific ladder for humans to play the AI. Of course the wait time would be incredibly long, unless Google invests an entire server farm to run the AI so it can play multiple instances.
On November 08 2016 07:01 LegalLord wrote: If AI get hax speed performance then humans get multicommand hacks and automated micro bots. If the computers only win because they are not input-constrained by having physical bodies that have to be moved to provide input into the game then the entire problem has been trivialized.
I think everyone agrees that there needs to be a very low limit to APM. I'd even argue that extremely low APM (like 180) and a small "between actions" limit that is similar to the time it takes humans to touch two different keys in succession on a keyboard would be much more interesting because it would require the computer to "outthink" humans instead of "outdoing" us. Also, maybe having the program not "remember" how many resources it has unless it spends one of those actions to "look" at its money (and things like that) would be helpful too.
They mentioned a lot of things like that during the panel, actually about resources they said they may just pass it the numbers. It was also mentioned that the restrictions on the AI will also be entirely based on the game interface people use, so selecting things will involve selecting single units as well as boxing over them (with ctrl+click and control groups implied I think). I'm interested in how will they deal with reacting to things happening on the minimap. I bet APM constraints may change on how it all develops.
After all the project is about creating a self-learning AI that would simulate the process of how human thinks, not just creating a bot that plays the game as best as possible - although that is the goal, but it's a goal set for AI, not researchers.
I'm super excited thinking about how FPV of the AI will look like when significant progress is made. It has to be a regular stream on twitch at some point, haha.
You can look at the stream I linked to see what an AI FPV looks like for BW. It would be like that, except SC2 instead.
It makes sense for Google to start with sc2, though whether they actually had a full understanding of the differences between games is highly uncertain. Blizzard's attitude here is of course obvious and has nothing to do with the spectator quality or strategic aspect. This is already a ridiculously difficult project, so why make it harder for themselves on taking on a game with much more strategic depth (and likely quality of opponents) first. The question is whether they will move onto BW afterwards for the next level of challenge.
When they entered Go it wasn't as if they challenged the best player in the world right away (for that matter they never did). They took it cautiously and challenged a second-tier player, and only challenged Lee Sedol after AlphaGo was proven and had plenty of time for feedback. If Lee Sedol got to play against the initial version of AlphaGo, he likely would have done very well. Then again, Go never had a dice roll aspect to it heh.
On November 08 2016 04:01 imp42 wrote: I will give you guys one more reason for BW:
BW is very close to being completely reverse engineered and open sourced. This is a screenshot the developer just released from his own BW engine: http://i.imgur.com/rdEcBYM.jpg
This is obviously big news and will deserve its own thread in due time. Take it as a teaser for now
I am not personally involved, just the messenger. The genius behind the project is an active BW Bot developer.
Not sure what the point of discussing why not BW is when it's clear that SC2 is the way they're going and that they've already put a lot of work into it both from the Blizzard side and Google side. One of the deepmind devs who spoke at the panel Oriol Vinyals was even one of the lead devs on the Berkeley Overmind bw bot so I'm sure he knows the pros and cons of BW vs SC2 and has a very good idea of what they want to achieve.
The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
Winning because you have 30000 APM is akin to winning because you have tanks against humans. Asymmetrical warfare is cheating.
Reasonable human-range AI limits on APM make perfect sense.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
They said they are going to give it the same game interface we have including simulating human-like unit selection with possibly only one exception of giving the AI resource numbers directly.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
They said they are going to give it the same game interface we have including simulating human-like unit selection with possibly only one exception of giving the AI resource numbers directly.
Yeah, I know, and that's quite fair. But Derpmallow is asking why it matters, and this is why.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
Winning because you have 30000 APM is akin to winning because you have tanks against humans. Asymmetrical warfare is cheating.
Reasonable human-range AI limits on APM make perfect sense.
I never said that APM limitation doesn't make sense for the showmatch, I just said that freaking out about all of that is silly. Deepmind's not going to make a program that is only impressive because it can micro perfectly, that doesn't give them any information to work off of for the stuff they actually care about. I am glad that they've thought about appropriate restrictions for the AI in order to make the showmatches fun to watch, but my core focus here is very similar to Deepmind themselves- seeing how complex they can make the cognition and decisionmaking for this AI.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
Winning because you have 30000 APM is akin to winning because you have tanks against humans. Asymmetrical warfare is cheating.
Reasonable human-range AI limits on APM make perfect sense.
I never said that APM limitation doesn't make sense for the showmatch, I just said that freaking out about all of that is silly. Deepmind's not going to make a program that is only impressive because it can micro perfectly, that doesn't give them any information to work off of for the stuff they actually care about. I am glad that they've thought about appropriate restrictions for the AI in order to make the showmatches fun to watch, but my core focus here is very similar to Deepmind themselves- seeing how complex they can make the cognition and decisionmaking for this AI.
Again, the entire point of all this is to demonstrate successful intelligent play by the AI that is comparable to that which humans have. To that end you have to have the AI succeed in a game against a human at a high level in a fair match. And in that sense the idea of a "fair match" is very important to rigorously define.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
Winning because you have 30000 APM is akin to winning because you have tanks against humans. Asymmetrical warfare is cheating.
Reasonable human-range AI limits on APM make perfect sense.
I never said that APM limitation doesn't make sense for the showmatch, I just said that freaking out about all of that is silly. Deepmind's not going to make a program that is only impressive because it can micro perfectly, that doesn't give them any information to work off of for the stuff they actually care about. I am glad that they've thought about appropriate restrictions for the AI in order to make the showmatches fun to watch, but my core focus here is very similar to Deepmind themselves- seeing how complex they can make the cognition and decisionmaking for this AI.
Again, the entire point of all this is to demonstrate successful intelligent play by the AI that is comparable to that which humans have. To that end you have to have the AI succeed in a game against a human at a high level in a fair match. And in that sense the idea of a "fair match" is very important to rigorously define.
I disagree with that being the core point of this project. This is a way for them to develop algorithms and techniques for using AI in a realtime environment with imperfect information. The showmatch is for publicity, but as far as the actual success of this project the AI never has to play against a single human being to be successful for Deepmind's goals. It's really cool that they're going to do so, but it is in no way necessary for the development of their AI projects. And thus, what game they're playing or how human-like they're playing it really doesn't matter, but I fully understand that for the sake of having the showmatch be engaging it's important for the AI to have extreme limitations and I'm sure they'll do a good job on that front.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
Winning because you have 30000 APM is akin to winning because you have tanks against humans. Asymmetrical warfare is cheating.
Reasonable human-range AI limits on APM make perfect sense.
I never said that APM limitation doesn't make sense for the showmatch, I just said that freaking out about all of that is silly. Deepmind's not going to make a program that is only impressive because it can micro perfectly, that doesn't give them any information to work off of for the stuff they actually care about. I am glad that they've thought about appropriate restrictions for the AI in order to make the showmatches fun to watch, but my core focus here is very similar to Deepmind themselves- seeing how complex they can make the cognition and decisionmaking for this AI.
Again, the entire point of all this is to demonstrate successful intelligent play by the AI that is comparable to that which humans have. To that end you have to have the AI succeed in a game against a human at a high level in a fair match. And in that sense the idea of a "fair match" is very important to rigorously define.
I disagree with that being the core point of this project. This is a way for them to develop algorithms and techniques for using AI in a realtime environment with imperfect information. The showmatch is for publicity, but as far as the actual success of this project the AI never has to play against a single human being to be successful for Deepmind's goals. It's really cool that they're going to do so, but it is in no way necessary for the development of their AI projects. And thus, what game they're playing or how human-like they're playing it really doesn't matter, but I fully understand that for the sake of having the showmatch be engaging it's important for the AI to have extreme limitations and I'm sure they'll do a good job on that front.
If they want to work on a simplified project they should. If the ultimate goal is to play humans it should have a similar physical constraint as a human.
Although, I will add one new perceived fault of SC2 over BW. I was never big into SC2 and I think my posting habits make it go without saying that I was mostly from BW. But as far as I can tell, custom maps never gained much traction in SC2, which means that a lot of custom scenarios (e.g. simplifications that you can test components of the AI on) will be less available. I know that in its brief foray into BW AI, the Facebook AI team used micro maps to train their system. Will an SC2 API have that possibility? Will it have it soon after release? I'm skeptical.
On November 09 2016 04:57 Derpmallow wrote: The amount of discussion here of APM and how to make the showmatch between man and machine fair and all of that is sort of baffling. This shouldn't need to be said, but I'll say it anyways: Deepmind's goal is not to make the perfect Starcraft bot. Their goal is to find new algorithms and techniques to apply to the quest for Artificial General Intelligence - that is, a robot that can fight you in Starcraft, do your taxes and discuss your marital issues without changing the underlying code itself.
This means that worries of it being too mechanically powerful for the showmatch are silly because, again, that's not their goal. They don't need to put millions of dollars and thousands of manhours into making an AI that can microbot a pro to death. Their task is to find a clever way of beating people since that helps their central goal. And, yes, this also means BW vs SC2 talk is silly because they aren't trying to prove that they can create the best Starcraft bot ever. The game is purely a testing ground to see how effective they can make the AI in thinking and planning and executing. There are a lot of cool AI projects for Brood War, yes, but they are not going to base their AI off those because, again, they're going for general. Neural Network AI for Brood War, while super cool, are very very narrow in what they can do. They play Starcraft. Deepmind wants to do a lot more than play Starcraft and other games.
The entire issue of speed is one of trivializing the problem. It's like building a football/soccer robot that is just a giant automated tank that will crush its human opposition. Yes it will win, but it didn't do so by virtue of its AI but rather by virtue of an interface that gives it an unfair advantage.
Yes, but the reality is that the showmatch is just that - a showmatch. I'm pretty sure they're aware that winning purely through mechanics is boring, but they're not going to spend hours and hours figuring out the exact amount of actions the average human is capable of in a period of time in order to make sure the games are perfectly fair. Odds are, they're going to do the same thing they did with Go and focus more on the AI playing against itself to see how effective the learning system is. Would it be nice if it became the most adept player of Starcraft 2, artificial or otherwise? Yeah, that'd be really cool. But that's not why they're doing this. This is about the journey, not the destination.
Winning because you have 30000 APM is akin to winning because you have tanks against humans. Asymmetrical warfare is cheating.
Reasonable human-range AI limits on APM make perfect sense.
I never said that APM limitation doesn't make sense for the showmatch, I just said that freaking out about all of that is silly. Deepmind's not going to make a program that is only impressive because it can micro perfectly, that doesn't give them any information to work off of for the stuff they actually care about. I am glad that they've thought about appropriate restrictions for the AI in order to make the showmatches fun to watch, but my core focus here is very similar to Deepmind themselves- seeing how complex they can make the cognition and decisionmaking for this AI.
Again, the entire point of all this is to demonstrate successful intelligent play by the AI that is comparable to that which humans have. To that end you have to have the AI succeed in a game against a human at a high level in a fair match. And in that sense the idea of a "fair match" is very important to rigorously define.
I disagree with that being the core point of this project. This is a way for them to develop algorithms and techniques for using AI in a realtime environment with imperfect information. The showmatch is for publicity, but as far as the actual success of this project the AI never has to play against a single human being to be successful for Deepmind's goals. It's really cool that they're going to do so, but it is in no way necessary for the development of their AI projects. And thus, what game they're playing or how human-like they're playing it really doesn't matter, but I fully understand that for the sake of having the showmatch be engaging it's important for the AI to have extreme limitations and I'm sure they'll do a good job on that front.
If they want to work on a simplified project they should. If the ultimate goal is to play humans it should have a similar physical constraint as a human.
Although, I will add one new perceived fault of SC2 over BW. I was never big into SC2 and I think my posting habits make it go without saying that I was mostly from BW. But as far as I can tell, custom maps never gained much traction in SC2, which means that a lot of custom scenarios (e.g. simplifications that you can test components of the AI on) will be less available. I know that in its brief foray into BW AI, the Facebook AI team used micro maps to train their system. Will an SC2 API have that possibility? Will it have it soon after release? I'm skeptical.
Like the person you quoted said, their ultimate goal is not to play good SC against humans, but to work towards general AI. I don't see how this is a "simplified project".
No clue why they're choosing SC2 over BW. I don't think this is PR for DeepMind/Google. I rather think this is PR for Blizzard. Dammit, DeepMind has the technology to cut global energy consumption by 30%. They really, really don't need any PR at this point.
And really, who cares? I'm excited to see what the geniuses behind DeepMind will deliver. We're alive and awake to witness one of the most important endeavours humankind has ever taken, the quest for general AI. And you're salty about them choosing SC2 over BW? Ah come on...
And yet, I'll agree with you. BW would have been a more reasonable choice, but not by a large margin. So I'm indifferent towards this. I'm looking forward to what DeepMind can do here.
Mendelfist, I like your abstract reasoning, but I think you just misunderstand what bottle has been trying to say. He is talking about chunking the gamestate and inputs into something reasonably learnable. You are talking about the structure of the AI once the chunking has been figured out. He is saying the problem is very hard because the task of chunking the game (which is essentially given for free by comparison in games like Chess, Go, and Billiards as he has explained) is very not straightforward, and probably needs a good deal of clever script/design and NN perception layer voodoo. You are saying that using NN (and/or other methods) would be straightforwardly applicable given proper chunking, which is mostly true -- I see there is still a large open question about what a policy even looks like for SC2 given dexterity-limited input.
As to the broader thought process about how hard SC2 is compared to Go, I think your "modest effort" argument is interesting but takes the wrong conclusion. Yes you can script great micro decent build order moderately okay lategame bots, but they would never beat a masters level player in a bo51 series. Probably even down to diamond or plat could handle them once they learn their setup and how to exploit it. The ultimate difficulty in SC2 AI is creating a decision maker than can adjust to enemy adaptation, not win by brute force with a killer strat and epic micro, in spite of how far those can get you. That readjusting decision making relies intimately on chunking the game state both for learning and for runtime operating; they influence each other deeply in a game like SC2, and the added wrinkle of limited inputs really makes a doozy of a problem.
On November 09 2016 06:28 EatThePath wrote: <3 @mendelfist and bottle
Mendelfist, I like your abstract reasoning, but I think you just misunderstand what bottle has been trying to say. He is talking about chunking the gamestate and inputs into something reasonably learnable. You are talking about the structure of the AI once the chunking has been figured out. He is saying the problem is very hard because the task of chunking the game (which is essentially given for free by comparison in games like Chess, Go, and Billiards as he has explained) is very not straightforward, and probably needs a good deal of clever script/design and NN perception layer voodoo. You are saying that using NN (and/or other methods) would be straightforwardly applicable given proper chunking, which is mostly true -- I see there is still a large open question about what a policy even looks like for SC2 given dexterity-limited input.
As to the broader thought process about how hard SC2 is compared to Go, I think your "modest effort" argument is interesting but takes the wrong conclusion. Yes you can script great micro decent build order moderately okay lategame bots, but they would never beat a masters level player in a bo51 series. Probably even down to diamond or plat could handle them once they learn their setup and how to exploit it. The ultimate difficulty in SC2 AI is creating a decision maker than can adjust to enemy adaptation, not win by brute force with a killer strat and epic micro, in spite of how far those can get you. That readjusting decision making relies intimately on chunking the game state both for learning and for runtime operating; they influence each other deeply in a game like SC2, and the added wrinkle of limited inputs really makes a doozy of a problem.
Neither is really true. Given the "voodoo magic" that deep learning can apply to chunking images, it's just a matter of scale. I don't think chunking SC2 is any harder than chunking images, and deep learning methods are better than anything else we have come up with. It just sucks that we have no clue what they are doing. So from an "explaining AI" point of view, deep learning is a disaster. But from getting good results, it's marvellous. Oh, you also need to not care about lower bounds, because absolutely nothing is provable (or at least insofar as we know right now) about NN as a function approximator (past the simplest of perceptron networks). It's quite possible that a very very low level detailed description of what is known about the gamestate every X milliseconds is a good input for a deep learning algorithm, just as the brightness level of every pixel is a good input for deep learning applied to CV.
On November 09 2016 06:28 EatThePath wrote: <3 @mendelfist and bottle
Mendelfist, I like your abstract reasoning, but I think you just misunderstand what bottle has been trying to say. He is talking about chunking the gamestate and inputs into something reasonably learnable. You are talking about the structure of the AI once the chunking has been figured out. He is saying the problem is very hard because the task of chunking the game (which is essentially given for free by comparison in games like Chess, Go, and Billiards as he has explained) is very not straightforward, and probably needs a good deal of clever script/design and NN perception layer voodoo. You are saying that using NN (and/or other methods) would be straightforwardly applicable given proper chunking, which is mostly true -- I see there is still a large open question about what a policy even looks like for SC2 given dexterity-limited input.
As to the broader thought process about how hard SC2 is compared to Go, I think your "modest effort" argument is interesting but takes the wrong conclusion. Yes you can script great micro decent build order moderately okay lategame bots, but they would never beat a masters level player in a bo51 series. Probably even down to diamond or plat could handle them once they learn their setup and how to exploit it. The ultimate difficulty in SC2 AI is creating a decision maker than can adjust to enemy adaptation, not win by brute force with a killer strat and epic micro, in spite of how far those can get you. That readjusting decision making relies intimately on chunking the game state both for learning and for runtime operating; they influence each other deeply in a game like SC2, and the added wrinkle of limited inputs really makes a doozy of a problem.
Neither is really true. Given the "voodoo magic" that deep learning can apply to chunking images, it's just a matter of scale. I don't think chunking SC2 is any harder than chunking images, and deep learning methods are better than anything else we have come up with. It just sucks that we have no clue what they are doing. So from an "explaining AI" point of view, deep learning is a disaster. But from getting good results, it's marvellous. Oh, you also need to not care about lower bounds, because absolutely nothing is provable (or at least insofar as we know right now) about NN as a function approximator (past the simplest of perceptron networks). It's quite possible that a very very low level detailed description of what is known about the gamestate every X milliseconds is a good input for a deep learning algorithm, just as the brightness level of every pixel is a good input for deep learning applied to CV.
It's quite possible but my point is that it's totally unknown right now and I assume figuring out what works well will be nontrivial. Image parsing was studied for decades before largescale parallel processing in the application of convolutional NN was thrown at it. And image perception (from what I understand) is based on fairly rudimentary low level layers that correspond quite naturally to geometric interpretation (edge detection/vertical/horizontal/lightVSdark etc). I think the inscrutable part of deep learning comes in the piles of abstraction and network scale. The ultimate outcome of an effective chunking perception scheme will appear not-that-hard, but I don't assume finding it will be straightforward.
imo the perception design is the first crux of the challenge, and the difficulty is you can't test that very well because it's such a convoluted road (no pun intended) to a playable AI. But who knows, maybe obvious chunking schemes will work well (even self learned maybe).
Btw by voodoo I meant the art/science of human designed CNN structure, but of course NN are pretty voodoo-seeming in general. XD
On November 09 2016 06:28 EatThePath wrote: <3 @mendelfist and bottle
Mendelfist, I like your abstract reasoning, but I think you just misunderstand what bottle has been trying to say. He is talking about chunking the gamestate and inputs into something reasonably learnable. You are talking about the structure of the AI once the chunking has been figured out. He is saying the problem is very hard because the task of chunking the game (which is essentially given for free by comparison in games like Chess, Go, and Billiards as he has explained) is very not straightforward, and probably needs a good deal of clever script/design and NN perception layer voodoo. You are saying that using NN (and/or other methods) would be straightforwardly applicable given proper chunking, which is mostly true -- I see there is still a large open question about what a policy even looks like for SC2 given dexterity-limited input.
As to the broader thought process about how hard SC2 is compared to Go, I think your "modest effort" argument is interesting but takes the wrong conclusion. Yes you can script great micro decent build order moderately okay lategame bots, but they would never beat a masters level player in a bo51 series. Probably even down to diamond or plat could handle them once they learn their setup and how to exploit it. The ultimate difficulty in SC2 AI is creating a decision maker than can adjust to enemy adaptation, not win by brute force with a killer strat and epic micro, in spite of how far those can get you. That readjusting decision making relies intimately on chunking the game state both for learning and for runtime operating; they influence each other deeply in a game like SC2, and the added wrinkle of limited inputs really makes a doozy of a problem.
Neither is really true. Given the "voodoo magic" that deep learning can apply to chunking images, it's just a matter of scale. I don't think chunking SC2 is any harder than chunking images, and deep learning methods are better than anything else we have come up with. It just sucks that we have no clue what they are doing. So from an "explaining AI" point of view, deep learning is a disaster. But from getting good results, it's marvellous. Oh, you also need to not care about lower bounds, because absolutely nothing is provable (or at least insofar as we know right now) about NN as a function approximator (past the simplest of perceptron networks). It's quite possible that a very very low level detailed description of what is known about the gamestate every X milliseconds is a good input for a deep learning algorithm, just as the brightness level of every pixel is a good input for deep learning applied to CV.
Whilst I agree with most of the spirit of what you're saying, it's slightly less true when reinforcement learning is concerned as various algorithms there will have various neuro-biological analogies ( experience replay and its prioritized variants, intrinsic motivation, etc).
Convolutional networks are indeed pretty black box at the moment but some bounds on extrema quality are beginning to appear from the connection with statistical physics. Warning : this is hardcore. arxiv.org
I read through about one quarter of the comments and am baffled at how many there are discussing APM restrictions and such. In the last pages I started seeing a lot more posts with understanding of what Deepmind's programme is doing. There is no pre-programmed micro, nor are there any rushes. The bot is supposed to learn by trial and error, as well as witnessing replays of games. The bot doesn't know what it should do with its APM. It might be just a lot of spamming for the first 6 months of training (the bot will be self-taught). The bot will know that the goal is to see the win screen, and not much more, when it starts.
This bot will teach itself how to play. No one will teach it strategies nor tactics. The bot will have access to replays and may play the game.
Deepmind want to see if their programme can learn how to play StarCraft. They hope for it to be able to beat some great player.
EPT
