On January 26 2019 08:43 TT1 wrote: I'd like to see how well AlphaStar does in BW, his main edge in the matches came from being aggressive and outmicroing his opponents w/ blink micro (obviously while having perfect macro/base management). BW has a much greater defenders advantage, it wouldn't be able to capitalize on the same aspects it did in SC2 (which to me was mainly exploiting micro and positioning).
BW games are slower paced and there's way less poking, AlphaStar wouldn't be able to exploit micro maneuvers which obviously benefits human players.
Edit: Well, AlphaStar could exploit microing multiple muta groups perfectly but that would only be viable in ZvT and ZvZ. Even then a progamer T could adapt and just open 1-1-1 into valks or fast vessel, AlphaStar would probably be godly in ZvZ tho.
AlphaStar in TvT could be extremely interesting imo.
BW TvT is very chess like tho, the longer the game goes the more it slows down (in terms of army movement). It comes down to positioning (tank lines) but there's a ton of decision making involved as well.
I'd like to see AlphaStar play other SC2 MU's first tho. The PvP wins felt like it was out-computing Mana/TLO more than anything . Blink micro is by far the most abusable micro aspect in SC2.
I'd personally love to watch AlphaStar 2h mass muta vs FlaSh. But again, even if it did win it would practically be like it's out-computing a human being by having no mechanical limits.. which is a given. Non ZvZ/ZvT MU's would be a different story tho.
Some observations of the game from me where that alpha-star is pretty good at figuring out army / build order stuff up until its maxed out, it's exceptionally good at the multi screen / angle micro management, but I think people are over estimating how good at is at actual intelligence related stuff.
There were many moments were alphastar was just running up choke points over and over and losing units even though it had just run into a strong army, it doesn't fell like it makes good combat decisions that aren't immediately micro related. At some point it also did nonsensical things like having six observers in the army or building out blink late in a predominantly stalker army.
I'm also not sure it really 'gets' economic management at a high level, it expands very conservatively and mostly to build more army. It would be interesting to see if it can make reasonable decisions in a past maxed out game, or if it can judge end games where resources are going to run out and adapt.
I would have really liked to see a ZvT to gauge this, because long term macro management and also sudden tech switches are definitely a much more strategic problem than the very micro focussed PvP games.
Honestly people aren't talking about his this AI was trained enough. This AI didn't create strategies on its own. The base agents were trained on replays, so effectively the first generation was just copying the replays exactly and so on. Those base agents would execute GM level builds with minimal training.
I'm really impressed by the optimizations it has found in the games. The disruptor game where it kills its own units seems to be an optimization where it realized it's probably better to sacrifice your own units as long as you kill the opponent's units. Sorta like banelings. But it does it very inefficiently. Some of the shots were plain mistakes, but we saw that even in it's build orders. In one game it build a fleet beacon to cancel it 1 second later.
As a proof of concept, the fact it can micromanage it's units according to the situation and context things are happening, the fact it can make decisions on it's own is proof of it's effectiveness. This is basically an alpha AI(no pun/context misinterpretation intended) that still has a lot of work to do.
A real test of it's effectiveness would be if it can play turn based SC2(not really turn based) where every .25 or .5 seconds the game is effectively paused and the player can decide what actions to take with every building and every unit. A single game would take a week of someone sitting there doing those actions. This is just to control for the perfect unit control the AI has, where clicking and focus firing units doesn't become a problem for the human player. There really isn't large strategic gains one could make even if they had 10x longer to think about the game since micromanaging the units is the most difficult part.
One optimization that it does and is really obvious is aggressive focus firing even in large battles. If you actually do the calculations, a perfectly efficient army vs the least efficient army(focused damaged on units, vs spreading the damage as much as possible) the AI can fight armies 40% bigger and win outright, trade evenly against armies around 60% larger. In "even" battles it will humiliate human players like a GM would humiliate a diamond level player.
Watching the replays now and am really impressed. Can't wait to see what this becomes into after a couple years.
Let's just make sure we don't piss HAL off, alright?
On January 26 2019 06:38 Eudorus wrote: For people who would doubt AI can beat strong players in AI, just read what people posted here a year ago, for example in the Boxer on AlphaGo thread. Secondly, so what kind of AI can beat a top players without an APM cap and also without a neural network?
So you say you think you need an APM cap to get an AI with 'smart decisions' rather than overwhelming the other player with superior mechanics? So what does that mean? I know what you are trying to say, but think about how you would define 'being smart' in the context of Starcraft. How is it 'smart' to play more human-like and have a finely tuned unit composition and play calm and macro carefully when the nature of the game is such that you should just mass stalkers, control them individually every time the game state updates, and simply just mass up, micro them around continuously, and go in for the kill when your opponent makes a mistake? The best way to define 'being smart' is that which leads to a higher winrate.
By the way, there are both simplifying aspects to having an APM cap, as well as real-life applications to not having one. It will be hard to train the same network to control all your 30 stalkers every game state while as well deciding upon when to switch tech or expand. So in that case you need a small fast micro network and a larger slower network. And there are real-world problems that require such NN's as well as real-world problems that require high speed. Furthermore, who knows what strategy the AI would come up with to properly micro infinite APM blink stalker vs blink stalker battles vs itself. That would need many more nodes than just moving the one that is hurt to the back while yourself attacking the lowest ph enemy stalker currently in range.
No, the APM cap was decided on to make the AI more 'human-like' and relatable. It is basically PR as well as seeing if you can train an AI to do tasks in human-like manner. There are many real-world problems that you can do effectively in eerie AI-like manners, which would be unacceptable for social reasons, or in a human-like manner. If you can mimic a human while outperforming them, than that is often better than doing the same thing slightly better, but being completely alien.
By "smart" I mean that AlphaStar should be able find appropriate solution to unexpected situations(like the Immortal drop one) and that it should be able to elaborate and recognize effective strategies; I'd say that the AI was more "efficient" than "smart".
As far as I know, AlphaZero was praised for some actually innovative and exceptionally good moves and AlphaStar should be able to do the same. What it has accomplished instead(and that's of course pretty impressive as well) is to win by mere mechanics; Its micro was not exactly perfect due to the cap but still inhuman at times(see blink Stalkers).
I am not a programmer but I guess you could create a super strong AI capable of beating humans with infinite apm even without the ability of learning by itself; by limitating AlphaStar you would ideally be able to make it win not eventually exploiting its mechanical advantages over humans but, possibly, outsmarting them. I don't think it's just a capriceous limitation imposed by silly humans who need to regard this AI as closer to them, it's that the very interesting part is for us to see if the AI can teach something to us, not just beat us.
If the objective here is just to develop an AI that can indipendently come up with ways to beat the top human players, simply you are right and I have nothing more to say.
On January 26 2019 08:43 TT1 wrote: I'd like to see how well AlphaStar does in BW, his main edge in the matches came from being aggressive and outmicroing his opponents w/ blink micro (obviously while having perfect macro/base management). BW has a much greater defenders advantage, it wouldn't be able to capitalize on the same aspects it did in SC2 (which to me was mainly exploiting micro and positioning).
BW games are slower paced and there's way less poking, AlphaStar wouldn't be able to exploit micro maneuvers which obviously benefits human players.
Watching all the replays I couldn't help but think the same thing. Obviously I get why they picked SC2, but building an agent for BW that could beat Flash in a Bo5 would be a real challenge.
On January 25 2019 07:02 Poopi wrote: But it got stuck really bad on the warp prism harass, and when MaNa started to attack near the 2nd base it was really indecisive.
Totally! Spotting with the observer played a big role, but this was the real critical point.
Well I guess it shouldn't be too surprising when a computer wins at a computer game.
If you consider that Alphastar needed "200 years of games" (for one race on one map), you can still appreciate that humans still learn way faster under real conditions and are able to adapt much more quickly and energy efficient.
I wonder if you can create AI decision making for applications outside of video games with the same approach. would it need to compute 200 years of trial-and-error in one second for every decision it makes in a new situation?
On January 26 2019 08:43 TT1 wrote: I'd like to see how well AlphaStar does in BW, his main edge in the matches came from being aggressive and outmicroing his opponents w/ blink micro (obviously while having perfect macro/base management). BW has a much greater defenders advantage, it wouldn't be able to capitalize on the same aspects it did in SC2 (which to me was mainly exploiting micro and positioning).
BW games are slower paced and there's way less poking, AlphaStar wouldn't be able to exploit micro maneuvers which obviously benefits human players.
Watching all the replays I couldn't help but think the same thing. Obviously I get why they picked SC2, but building an agent for BW that could beat Flash in a Bo5 would be a real challenge.
the popularity of the game in and outside of activision/blizzard like played a role. (promoting the game) I would imagine the deepmind guys would even prefer doing this with BW.
On January 26 2019 07:57 Grumbels wrote: Starcraft has this thing called “balance”, where unit strength is calibrated around human abilities to provide strategically rich gameplay. If you break the balance of the game then you can’t test the abilities of your agent, as it will just find out some silly unbeatable micro strat. APM caps and such is not just PR...
Exactly, it's playing another game, another of those games would have went to MaNa if the AI didnt do 1000+ constant APM from 3 different screens.
It's super cool and super impressive, but remember this is PvP on catalyst, I want to see non-mirror matchups specially with Zerg, ZvT/ZvP should be way more interesting, when does the AI build units?
About BW, it'd be cool, but BW APM also has to be capped, if the AI is getting ahead with mechanics on SC2, I can only imagine the monstermodemacro.
On January 25 2019 07:02 Poopi wrote: But it got stuck really bad on the warp prism harass, and when MaNa started to attack near the 2nd base it was really indecisive.
Totally! Spotting with the observer played a big role, but this was the real critical point.
Well I guess it shouldn't be too surprising when a computer wins at a computer game.
I think an important thing to take into account here is that the AI didn't spot the observer, and didn't even "think" to check for one. That allowed Mana full view of the ramp, and plenty of advance warning to pull his immortals back when a whole troop of stalkers came marching up the ramp, and also gave him knowledge that the whole stalker army was walking back out of the main. For some reason, the AI was completely unprepared for this harrassment and *never* left stalkers in that corner to defend for it. The first time, it could be in a state where it didn't know Mana had a warp prism, but the second time, it should have been in a different state where it did have that knowledge. And presumably in its 200 years of training, some games had been against an opposing bot that did some form of harrassment. And it would have learned better ways to deal with that. With the DeepMind guys saying they picked the agents that were "least exploitable", it's quite weird that it was so easily exploitable.
On January 26 2019 00:47 imp42 wrote: To sum up most of the debate so far:
- was the setup "fair" or not? - did the AI play well or not?
In the chess community we were blown away by the games AlphaZero played about a year ago. We had never seen anything like it. However, it was also very disappointing to realize Deepmind wasn't interested in chess at all. It was nothing but a playing field to demonstrate the capabilities of their neural net. As soon as the experiment concluded successfully, the Deepmind team moved on and left the chess world wondering what could have been, if they had access. Imagine somebody allowing you to peek into a treasure chest full of amazing content, but then closes it and stores it away, not to be opened again.
Realistically we are in the same situation with StarCraft of course. Once Deepmind "beats the game" they will move on without missing a beat.
Except they didn't. They actually continued the chess research, and released the new results in late 2018.
People forget that science (which is what they are doing) isn't a process where you - from the outside world - see results every day. When DeepMind first announced that AlphaZero had beaten Stockfish, it wasn't just the release of 10 games. There was a scientific paper included, which also had to be written and reviewed, and the researchers need to make sure that the conclusions they draw are correct.
Science isn't a field where the we, as the outside world, can expect to get daily or somewhat regular progress reports (although it can happen). This is, in part, to prevent people and the media from jumping to conclusions until the researchers feel certain they have accounted for whatever they feel they need to account for, and make sure the results aren't misinterpreted. And you are confusing this approach (which is a common and rather sound approach) with them having left "chess". They certainly haven't. And they certainly haven't left StarCraft II either. It just happens to be a research project, and they are doing it for the research - not for our entertainment. They are interested in StarCraft and Chess all right - just not at competitive sports, but as learning tools.
I, for one, are 100% certain that DeepMind isn't done with StarCraft. Since it's the first major attempt for an AI project (at least that i know of) that tries to solve a problem with limited information, i expect them to do put lot of work into this. For now, they have attempted training an AI that can win by executing powerful strategies. The next steps might be to attempt to make the AI more adaptable. This isn't just about training it more - it's about HOW to train it. That's actually the hard part of creating a good AI: you need to train it CORRECTLY and find the best approach. It's not just a bruce-force problem of letting it train for longer and longer.
To give an example of this, take this video by Codebullet. He is trying to train an AI to learn to play "Worlds Hardest Game" (that's what it's called). Initial attempts weren't getting him any results, so he had to change the way he was teaching it, and ended up solving the problem by incrementally expanding the amount of moves the AI was allowed to make before it would die. In that game, it was an easy solution. But for a game like StarCraft, this is actually a really complex problem.
On January 25 2019 19:30 Jockmcplop wrote: I don't think there's much you can do with a preexisting game like Starcraft to get a 'fair' game between ai and humans. Maybe intense tweaking of balance and ai ability but even then I don't think it would seem like a real opponent. You would have to design a game from the ground up with ai in mind I think.
I'm very impressed with the decision making of this ai. Its streets ahead of anything else I've seen.
I think its just super micro potential units that are broken for AI. I promise with zerg AI would not look nearly that impressive. THey would probably have a hard time droning and making units at the right times. Then you dont have things like warp prism micro or blink micro that can scale like crazy. What are they gonna do, dance their zerglings? They cant jump accross a wall. What are they gonna do, shoot an oracle or void ray or banshee with some roaches?
The AI would probably be forced in a ling bane hydra game, and then it would come down to how good they can micro hydras against AOE.
Zerg is the true race where human intelligence shines. Its about being one step ahead, predicting what the opponent will do, where he will send his warp prism,etc. You have to know when to drone and make units sometimes based just on instincts alone or knowing your opponent or current meta trends.
Protoss is you pick a build, execute it perfectly, and if you do execute it perfectly or very close, you probably win,unless your build order was too coin flippy.
In fact, PVP is by far the easiest matchup for an AI like deepmind to win consistently against humans. edit: Actually, ZvZ might be even easier for A.I now that i think about it... Most definitely maybe?
FUN FACT: I wonder if deepmind AI could play billions of matches of all matchups to determine which is the potential best race. If someone is unbiased and would know , its an AI. I bet they already know lol.
This kind if AI playing billions of games against same skill opponent could probably easily conclude if certain units are overpowered in certain matchups. Very interesting stuff when you think about it....
There's some truth to what you're saying here, I'm especially interested in how an AI Zerg approaches drone production.
Overall though, I think a Zerg AI would have a flock of mutas constantly poking away at their opponent in ways that no human can.
I also think that splitting up their zerglings would increase their effectiveness.
EPT
. Blink micro is by far the most abusable micro aspect in SC2.
