On January 27 2019 15:23 Dumbledore wrote: The alphastar agents that went 10-0 had ability to see the map zoomed out all the time. Not fair at all. The camera based agent got slam dunked.
Humans evolve too, and Mana found a way to abuse the AI. That is expected to happen. He came into this match prepared.
Fact is that according to DeepMind, the camera-based agent is actually only slightly weaker than the non-camera based agent. You can't judge something from one game. If they had gone to a 10-game match, i would still expect the AI to win 6-7 out of 10 games.
Here's the strength-graphs of the camera-agent versus the camera-less agent.
I think we need to see a lot more games. I would like to see mana get some rematches against the same agents. Every match really needs to be limited to camera to make it legit. having the entire map is a pretty large cheat code.
Is Alphastar using physical keyboard/Mouse? If not then there is no point to do this since fatigue /coordination error is a big part of starcraft games.
Do this when AI uses real time camera, physical keyboard/mouse.
One thing that I thought that was pretty interesting was, how does the AI know that you're not going DT?
In the one game Mana did go DT it seemed like he had a pretty decent edge with map control and unit attacks, but eventually didn't maximize that lead.
I read that the AI can see the blur on the map, and since it has full map vision it can essentially see any cloaked unit, even if it's undetected.
Would removing this attribute, or at least making the camera focus on the blur as a requirement vs cloaked units make them much stronger vs the AI, and furthermore, alter the builds it does substantially?
Like, if for example, Mana had feigned an expo and just went straight to DTs, other than the agent that went mass disruptor, I don't see how this wouldn't just be a serious flaw in the builds it does.
Making mass stalkers/blink and microing them is insane, but it doesn't beat a variety of other builds that were unable to be tested.
Thusly, it would be great to, as others have said, see the pros be able to play against the same agents multiple times to see if their MMR is 7000 in a BO1, or if it's actually 7000 across X number of games vs pros.
On January 27 2019 07:19 Xitah wrote: Not impressed. Why?
Let's compare to chess: when alphazero beat stockfish, a lot of chess analysts thought alphazero played beautifully and very different from the typical brute force AI. Youtube is full of game analysis of those games because GMs truly find those games beautiful and interesting.
Here it looks like the AI can get away with suboptimal decisions, bad decisions just because it can click simultaneously on 100+ units at a time anywhere on the map.
This feels like giving an average Joe an aimbot in a FPS game.
That's a superficial and wrong analysis. You're missing many things the AI did right, and that isn't related to micro.
For example, the AI had an absolutely perfect understanding or ability to predict the outcome of a battle based on army sizes and unit composition. This shouldn't really come as a surprise, since it's something you would expect an AI to do really well after 200 years of accumulated playing-experience, but it's really an important point. Even if you reduce the micro-abilities of the AI even further, humans can't hope to ever be as good at judging the outcome of an engagement as an AI. You often see this reflected in commentators in professional matches, when a medium-sized to large battle ensues, and they can't tell who is gonna win it until the end.
And you also saw this in these games: there were several situations where TLO/Mana misjudged the winning chances of an engagement, and got crushed, both on defense and offense. The AI understood the army compositions better, and knew when to move in or when it had enough defense, where's the human player made a sub-optimal judgement.
Trying to reduce the outcome of this match to simply being about superior micro is plain ignorant.
I don't know if you can call the AI better at judging an engagement. If you download the replays and take control of the fights vs an equally skilled opponent the human player is usually the one who has the better army at the end. The AI doesn't evaluate the fight as winnable because it has better army, it knows it can win because it can outmicro the opponent with its composition.
Having 3 stalkers, 1 zealot and 4 phoenixes vs 8 stalkers, 1 adept and 2 phoenixes.
Not impressed. Situation repeats with Kasparov vs Deep blue. Human side were in the dark. Human side played like it plays vs human. And human side won and showed how "stupid" a. I. Is in last game after having enough experience.
On January 28 2019 01:34 scwish wrote: Just my thoughts from the point of view of a casual sc:bw/sc2 player. I take it that AS learned "how to play" by getting an initial seed of pro replays, thus probably picking up the initial concept of "mining is good" and "kill enemy stuff to win", and then evolved by playing countless games versus instances of itself, right? If so, I think that this is a major flaw in the eventual evolution of its decisionmaking ability.
No. This is mathematically required. The phase space game moves is way too big for it to randomly get near the phase space of playing properly. There is no point in spending tens of thousands or more hours of game play of agents just randomly clicking like monkeys, where no iteration of the NN is any better than the other.
It´s more or less the same if I was watching some replays and played a tutorial with my friend, and then never took a glance outside the box but only played against him, over and over again, for, well... 200 years.
You can't be serious.
The result is a tree of winning techniques (not necessarily strategies or refined builds) that proved to be good within the scope of the learning environment,
That is all that training a neural network can mathematically ever do. But I thought your point would be that it only copies replays and never does anything new or creative.
... but it doesn´t have too much of a similarity to the evolution of the PvP meta over the years, or even within a patch. It´s not trying to be a better gamer (c), but trying to reach higher win-% with what it´s doing. This if of course backed by an access to mechanics way beond human capabilites.
Why would you expect the learning of a neural network to mirror the human meta? I think you should understand that it never should. A meta is something completely random anyway. That's why there can be different meta's if populations of gamers are cut off from each other. And why do we humans have a meta anyway? Because we copy each others strategies. In training a NN, you never ever copy the weights and biases of the agent that beat you. So why even bring it up?
So it should not happen. And you certainly don't want it to happen either.
Well, so far so good. It´s still nice to see that AS figured out things like flanking, proxies, blink micro etc. more or less by itself. What it failed to learn about is any consistent form of reactionary play depending on opponents' actions, see manas prism harass.
It did respond to Mana's warp prism and it did so perfectly consistently, which was exactly the problem. Do you even know the meaning of the words you are using? This was a clear example of a human recognizing a tendency of the AI and exploiting it and the AI because of it's nature unable to adapt to this.
But why should it, as long as pushing hard with near-perfect macro at home while aiming for an economy lead still works? It´s been practicing against other versions of itself that (mostly) didn´t execute any major tech switches after all.
Good insight. The AI has shown us that the correct way to play SC2 is not to try to out-think the opponent. Instead, you play a game where it becomes irrelevant what your opponent is thinking.
You say you know that it didn't know to adapt because it didn't face agents beating it because they did learn to do tech switches. But you have no reason to believe this. And neither TLO or Mana won by doing a tech switch. So I don't get this.
Also, I see its heavy emphasis on blink stalkers in one form or another as a result of AS "understanding" that perfect micro with them is freaking awesome, or rather: cost/return wise they are oftentimes more effective than other protoss compositions, given the fact that fueling hundreds of apm into microing them makes them a dozen times better over a-moving them, or just focus firing.
This sentence is an absolute mess, but I think I get the jest of it. Yes, AS has a deeper understanding of Starcraft than all people here claiming it should do techswitches and diverse unit compositions.
I assume that in the history of ASvAS games, any AI instance that tried to play a conservative composition against 80-100% blink stalkers eventuelly got wrecked, so this tree of decisions eventually died, or rather fell behind too much in priority to be considered.
Which means those agents were playing the game wrong and the ones that won and were selected had a deeper understanding.
Basically, with unlimited eapm, SC2 would need an independent set of balance patches around the mechanical capabilities of AS, or at least AS-PvP would sooner or later evolve to a point where all it needs are nexi, gates, core, forge, probes and stalkers. kappa.
Why? How many more balance patches has SC2 had compared to SC BW. And it still isn't balanced. Or is the argument here: "AS shows that SC2 sucks."
But: what would have happened if AS didn´t get to learn in an environment where it can spike eapm to 1500 and beyond, but only to a human level? Or even as low as maybe 50ish eapm? Obviously letting it learn at high eapm and then hardcapping it way below that for an actual match will distort the result big time. We will probably never know, but I guess that with different cost/return weights on units, we would have seen totally different preferred unit compositions. Maybe 11/11 soul train games.
The AS would likely have to overcome some technical hurdle but then the outcome would be AS winning. And you would come up with something new to whine about.
When I for myself try to improve, I play people that are seeded somewhere between "a bit better" and "quite a bit better", because otherwise, I am unlikely to get a glimpse of my own flaws, and all I get out of a game is a tiny bit of mechanical routine.
That's just your own rationalization about how you learn. People on TL have made much stronger cases about the benefits of playing against people weaker than you.
Obviously, it´s quite hard to find a human opponent that´s stronger than AS and able to play hundreds of thousands of games per hour, simultaneously.
After this statement, I am not quite sure if you mean what 'quite hard' means. But say this was somehow possible, would we then even need to develop deeplearning AI's? We just get a bunch of people better than a deeplearning algorithm to do the same thing for hundreds of thousands of hours.
So what do you think would happen if AS wouldn´t play-to-learn against itself, but rather against conventional bots with a gazillion of apm, perfect micro and strats that are designed to crush it with random multi-front pushes, while itself being constrained to screen vision and, say, 200-250 eapm?
Do such bots exist? It would simply overfit the NN to beat these very weak bots.
And once it starts to consistently deal with it, improve the sparring bot?
How would it improve the non-deeplearning sparring bot? They are hard-coded line by line.
Would AS learn to be more of an e-serral than a blink-god? Or would AS just evolve into developing a pushing technique that is mostly unstoppable by this kind of play, thus basically ending up where it is right now?
It would waste a whole bunch of Google's invested dollars.
On January 28 2019 15:33 cpower wrote: Is Alphastar using physical keyboard/Mouse? If not then there is no point to do this since fatigue /coordination error is a big part of starcraft games.
But an AI doesn't get fatigued. Why would you hard-code in artificial fatigue so that the NN develops to avoid the effect of fatigue that it doesn't suffer from in the first place? Also, I don't think even for a human playing a Bo5, fatigue plays a big role. Unless you are jet-lagged or something. I assume you mean mental fatigue, which is hard to notice yourself. From my experience, humans have no obvious problems concentrating for 5x30 minutes.
I don't understand why you say that an AI is not useful unless it has all the flaws humans have.
On January 28 2019 16:06 -Kyo- wrote: I read that the AI can see the blur on the map, and since it has full map vision it can essentially see any cloaked unit, even if it's undetected.
So can humans. They just removed an interface limitation from the AI so that it would learn to actually play the game first. It is silly to simultaneously have the AI learn to play the game and 'fight the interface'. So they gave all the info theoretically available to the player and fed it straight into an input matrix. If you put in these limitations then the AI will just learn to be really really good at camera control while right now it can focus straight on strategizing.
Would removing this attribute, or at least making the camera focus on the blur as a requirement vs cloaked units make them much stronger vs the AI, and furthermore, alter the builds it does substantially?
What?
Thusly, it would be great to, as others have said, see the pros be able to play against the same agents multiple times to see if their MMR is 7000 in a BO1, or if it's actually 7000 across X number of games vs pros.
But this is also tricky because a human GM playing a BO1 vs a random top agent will result in a completely different MMR for AS than a human playing over and over vs the same AI until it knows exactly how to exploit it. An AI can be no.1 on the ladder while at the same time any human can be shown how to beat it with a specific strategy if humans are allowed to play over and over vs the exact same AI until they find how to exploit it.
Same sensationalised story when AlphaZero "beat" stockfish. Basically playing against a shelf bought stockfish with no tablebases. Yeh you can make your AI seem impressive when you give it massively favourable conditions.
AI is a powerful tool but I stopped taking this at all seriously after the list of conditions and the fact it sees the whole map. Pretty much a marketing gimmick, cool but not that impressive.
On January 28 2019 21:51 KelsierSC wrote: Same sensationalised story when AlphaZero "beat" stockfish. Basically playing against a shelf bought stockfish with no tablebases. Yeh you can make your AI seem impressive when you give it massively favourable conditions.
But the result was 27-0, 60-something draws... Do you think something could help Stockfish take a game from AlphaZero?
On January 28 2019 21:51 KelsierSC wrote: Same sensationalised story when AlphaZero "beat" stockfish. Basically playing against a shelf bought stockfish with no tablebases. Yeh you can make your AI seem impressive when you give it massively favourable conditions.
But the result was 27-0, 60-something draws... Do you think something could help Stockfish take a game from AlphaZero?
When Alphazero played 1000 games against stockfish and Stockfish had access to its opening books SF was able to consistently win games. Alphazero won the majority of the games but the margin was significantly smaller.
afaik there have been no fundamental advances in physics since the 60’s, yet look at today’s technology versus 50 years ago. There’s a lot of cool things to be done using current AI tech.
On January 26 2019 08:00 shabby wrote: I wonder if AlphaStar would play differently if it didn't learn anything from replays, but had to figure everything out by itself. Would be more interesting strategywise to see what it does when it doesnt "mimic humans".
Considering the move space, initially a neural network with random weights will have purely random actions. It will be as if your cat is walking across your keyboard, or as if a monkey is clicking your mouse. At that point you depend on the AI accidentally building a pylon, building a gateway, and having a zealot move towards the enemy base. So you would have hundreds of thousands of games lasting for a very long time with literally nothing happening. In other words, the phase space of tremendously huge and only a very tiny segment of that phase space has engines that actually attempt to play the game. And if you initiate a random neural net, it will be out there in a flat desert of completely random clicking. If you are there and move in any direction of the phase space, you aren't suddenly winning more games. All your agents would just randomly click while the clock counts down to a draw. So all your bots draw vs all of them because the phase space is so huge, no neural network gets initialized with the proper weights. That's why you first imitate human play. We already know how a game of Starcraft should look like. So there is no sense in exploring the vast flat desert of useless neural nets. Maybe you are copying things from humans that are bad and you don't unlearn them. Hard to know. But it makes no sense to try to train neural nets when only 0.0000001% of them actually send their probes to mine minerals.
Well, yeah, heh. Seems your point is that it would take a lot of processing power/time, my point was what would happen if you did it. If it floats your boat, you could clone the machine a million times, lower the timelimit for a draw in games where nothing happens, and change the game speed drastically for the initial learning or something. Idk, you could probably even "weight" (if thats the right word) decisions in the beginning to favor mining/building to speed up the first million games of random clicks. The point is that after it figures out that mining minerals and building buildings and an army to attack with is good, it would "evolve" without any interference from how humans have decided SC should look. Saying that we already know how a game of starcraft should look like is arrogant imo, and kinda makes the point of an AI moot, besides the technical research, which most of us arent interested in. People thought they knew how chess and DoTA should be played too, turns out it should be played a lot more aggressively, with sacrifices being made along the way for the greater good.
Yeah, the correct way to play SC2 is to rely on micro unattainable by human players. The same way flying at high speed is the correct way to play American football... Great insight.
You would think it would be obvious, right? Well, apparently not to some.
Maybe Blizzard can use this NN in the future to make sure the game doesn't require too much micro.
As for AlphaZero in chess, yeah Deepmind ran a weak version of Stockfish against AlphaZero. Just like they picked TLO out of all people to play vs AlphaStar. And they blamed Blizzard for that, lol. Also, why would you even ask Blizzard? Isn't TLO a Supreme Commander player that retired years ago? And if you go through the Stockfish games, many games have moved that no Stockfish engine would ever make. It is as if they manually entered a bad movie. Or the way they ran Stockfish resulted in it blundering. And if you look on Youtube, there's even people that claim that for some games they released, they switched around the names and the game won by AlphaZero was actually won by Stockfish.
On January 28 2019 21:51 KelsierSC wrote: Same sensationalised story when AlphaZero "beat" stockfish. Basically playing against a shelf bought stockfish with no tablebases. Yeh you can make your AI seem impressive when you give it massively favourable conditions.
But the result was 27-0, 60-something draws... Do you think something could help Stockfish take a game from AlphaZero?
When Alphazero played 1000 games against stockfish and Stockfish had access to its opening books SF was able to consistently win games. Alphazero won the majority of the games but the margin was significantly smaller.
Alright, I pointed this out before, but... They played 1200 games in that series. The margin was "significantly smaller" for two reasons: 1. Any non-zero number would make for a "significant" difference 2. They didn't just "have access to" opening books. The opening positions were determined beforehand. So AZ and SF might have actually started playing after 7 moves, or however many moves took them to the pre-determined starting position. Some of those positions would be intrinsically disadvantaged. If you watch some of the games that AZ lost, you can see that this actually did make a difference, and as far as I can tell, those may have been the only games AZ lost.
Despite number 2 above, the score was 290-24 in that match. That's the significantly smaller margin you're pointing to. When you combine this with the increasingly competitive Leela Chess Zero in TCEC, it's hard to argue that DeepMind failed to come up with an algorithm that yields a superior agent in chess. I get that you were responding to a post talking about their earlier complete shutout series, but if we go to the root comment, the fundamental idea that these are "sensationalised" results is just... not tenable.
Getting back to StarCraft and the root comment, the A* results are pretty impressive, any way you cut it. I think where people are getting confused is that they want to see something even more impressive. Well, give them time and let's see. In the meanwhile, they've accomplished something real here, even if it's not exactly what some people want them to have accomplished. There now exists a bot that can beat SCII pros. Am I wrong in thinking that prior to this, nothing else had even come close, even when taking full advantage of bot abilities compared to humans? But I guess that even though they tried to make its abilities somewhat more human-like, it's not an accomplishment because, you know, they didn't succeed fully, so they haven't proven that a bot can totally destroy all humans every game under perfectly human-like conditions. Until they do that, they are just a bunch of over-hyped sensationalists, right?
On January 29 2019 08:48 Polypoetes wrote: You would think it would be obvious, right? Well, apparently not to some.
Maybe Blizzard can use this NN in the future to make sure the game doesn't require too much micro.
As for AlphaZero in chess, yeah Deepmind ran a weak version of Stockfish against AlphaZero. Just like they picked TLO out of all people to play vs AlphaStar. And they blamed Blizzard for that, lol. Also, why would you even ask Blizzard? Isn't TLO a Supreme Commander player that retired years ago? And if you go through the Stockfish games, many games have moved that no Stockfish engine would ever make. It is as if they manually entered a bad movie. Or the way they ran Stockfish resulted in it blundering. And if you look on Youtube, there's even people that claim that for some games they released, they switched around the names and the game won by AlphaZero was actually won by Stockfish.
They picked TLO because he is great for PR. Also they picked him because he actually is a legit Pro and plays well enough to test the waters. They then went on to Protoss Pro Mana to see how good AlphaStar actually can be. So what's the problem?
TLO played SupCom ages ago and deliberately switched to Broodwar in order to prepare for SC2 which he knew would be a real esport opportunity.
On January 27 2019 07:19 Xitah wrote: Not impressed. Why?
Let's compare to chess: when alphazero beat stockfish, a lot of chess analysts thought alphazero played beautifully and very different from the typical brute force AI. Youtube is full of game analysis of those games because GMs truly find those games beautiful and interesting.
Here it looks like the AI can get away with suboptimal decisions, bad decisions just because it can click simultaneously on 100+ units at a time anywhere on the map.
This feels like giving an average Joe an aimbot in a FPS game.
That's a superficial and wrong analysis. You're missing many things the AI did right, and that isn't related to micro.
For example, the AI had an absolutely perfect understanding or ability to predict the outcome of a battle based on army sizes and unit composition. This shouldn't really come as a surprise, since it's something you would expect an AI to do really well after 200 years of accumulated playing-experience, but it's really an important point. Even if you reduce the micro-abilities of the AI even further, humans can't hope to ever be as good at judging the outcome of an engagement as an AI. You often see this reflected in commentators in professional matches, when a medium-sized to large battle ensues, and they can't tell who is gonna win it until the end.
And you also saw this in these games: there were several situations where TLO/Mana misjudged the winning chances of an engagement, and got crushed, both on defense and offense. The AI understood the army compositions better, and knew when to move in or when it had enough defense, where's the human player made a sub-optimal judgement.
Trying to reduce the outcome of this match to simply being about superior micro is plain ignorant.
Good point. Just to add to this a bit: while it is certainly true that medium- and large-scale battles are visually hard to follow and difficult to "score" as the fast action unfurls for casters, another major reason they often won't know who's going to win a fight is because the unit control from each player is also highly variable. Some players have different unit targeting priority tendencies and that can influence outcomes, some are better at positioning, some can more effectively exploit their favorite units, some click faster, some click more accurately, some react faster, and so on.
AlphaStar chooses to take an engagement against an enemy force knowing that it will have near-100% unit efficiency and assuming that the enemy will have the same (since it has prior experience playing against itself). Human play can only introduce mistakes, resulting in relative inefficiency. There was only one moment that I saw where AlphaStar made a critical blunder in a skirmish, and that was in the Blink Stalker game (Game 4 vs MaNa) where it blinked forward and lost like 7 Stalkers in exchange for 1 Immortal. By the odds, if AlphaStar pushes forward against you, it's doing so with extremely high confidence backed up by centuries of experience.
EPT![[image loading]](https://storage.googleapis.com/deepmind-live-cms/images/SCII-BlogPost-Fig10_02.width-1500.png)
