For those who missed, in 2017 Facebook announced their AI project to compete with Google Deep Mind right after Google success to defeat the GO champion Lee Sedol.
The latest update that I managed to find is about FB participating in another Canadian tourney one year ago managing to take the second place. The first place was taken by a Samsung bot and the third place by a Chinese researchers team.
I'm quite passionate about Neural Network AI since the AlphaGo vs Lee Sedol stream so If anyone have any updates on the state of FB AI project I would love to know more on the topic
As someone kind of deep in the BW AI scene, I feel I have to make some corrections here.
First of all, it's Neural, not Neuronal network
Second, this is not really fair to both of those companies. "defeated by a team of amateurs" - It's a clickbait headline, and does not tell the whole story. The Student Starcraft AI Tournament is a long-running competition, and it has a running ladder. You can watch the games 0/24 here: (I cast those regularly on the official SSCAIT channel) https://sscaitournament.com/
As soon as you upload your bot, you have to face that the other bots will begin learning, and adapt their strategies to yours. For this reason, most authors withhold their entries until the time of the actual competition. You don't know what to expect, and can't possibly adapt to every situation, the problem space is just too large. And machine learning is not a silver bullet, it is not general learning, more like fine-tuning some values. There are a lot of tactics that bots can use, and they can't just improvise like a human can.
The Facebook and Samsung teams are performed very well, given their time of entry (The FB team especially had very little time to prepare). SAIDA (The Samsung bot) actually won. Then a lot of hobbyist developers spent a lot of time figuring out how to beat it, and eventually, the meta just evolved. (I think it was discovered early that it is vulnerable to bunker rushes).
And I feel that these companies just can't get a win here. If they win the tournament "omg it's easy because they have money", if they don't then "omg they lost to amateurs". And they didn't lose, they got in the top 3.
I interviewed Dave Churchill about this as well (The organizer of these tournaments) in my podcast about Brood War AI.
If you really want updates about the BW AI scene, you can join the SSCAIT Discord, I think that's the best place, and most bot authors frequent it. As for the Facebook team's progress, there aren't many updates - I would imagine exactly because the reactions. I feel they have to present a big win, or else they take a PR hit.
Also, there is Jay Scott's blog with detailed analysis of bots and results: http://satirist.org/ai/starcraft/blog/ And lastly, there is mine, where I write articles about bot development, and host the aforementioned podcast - it's called The Undermind. www.makingcomputerdothings.com
As someone kind of deep in the BW AI scene, I feel I have to make some corrections here.
First of all, it's Neural, not Neuronal network
Second, this is not really fair to both of those companies. "defeated by a team of amateurs" - It's a clickbait headline, and does not tell the whole story. The Student Starcraft AI Tournament is a long-running competition, and it has a running ladder. You can watch the games 0/24 here: (I cast those regularly on the official SSCAIT channel) https://sscaitournament.com/
As soon as you upload your bot, you have to face that the other bots will begin learning, and adapt their strategies to yours. For this reason, most authors withhold their entries until the time of the actual competition. You don't know what to expect, and can't possibly adapt to every situation, the problem space is just too large. And machine learning is not a silver bullet, it is not general learning, more like fine-tuning some values. There are a lot of tactics that bots can use, and they can't just improvise like a human can.
The Facebook and Samsung teams are performed very well, given their time of entry (The FB team especially had very little time to prepare). SAIDA (The Samsung bot) actually won. Then a lot of hobbyist developers spent a lot of time figuring out how to beat it, and eventually, the meta just evolved. (I think it was discovered early that it is vulnerable to bunker rushes).
And I feel that these companies just can't get a win here. If they win the tournament "omg it's easy because they have money", if they don't then "omg they lost to amateurs". And they didn't lose, they got in the top 3.
I interviewed Dave Churchill about this as well (The organizer of these tournaments) in my podcast about Brood War AI.
If you really want updates about the BW AI scene, you can join the SSCAIT Discord, I think that's the best place, and most bot authors frequent it. As for the Facebook team's progress, there aren't many updates - I would imagine exactly because the reactions. I feel they have to present a big win, or else they take a PR hit.
Also, there is Jay Scott's blog with detailed analysis of bots and results: http://satirist.org/ai/starcraft/blog/ And lastly, there is mine, where I write articles about bot development, and host the aforementioned podcast - it's called The Undermind. www.makingcomputerdothings.com
Thanks for the interesting input I'm curious to see how long it will take until an AI beats Flash (under certain fair APM/reaction restrictions that we can agree on)
APM restriction is not really a good metric. In SC2 it's different - the game handles a lot of things well and smoothly. In BW, you kinda need a lot of APM as a bot to get basic things done. And also, the APM does not represent the actual processing behind decision making.
In the SSCAIT replays, APM goes up to tens of thousands routinely. Still, the best of these bots are at about B- level in human terms.
If you think that's not fair, then I encourage you to write a bot yourself, and try to beat Flash. I can pretty much guarantee that you won't be able to do that.
I'm willing to make a bet that the first bot that reliably beats human pros won't be using any tactic developed by humans, but rather, something that's only feasible to execute for bots.
And before the booing begins, look at the date. This was before the existenc of AlphaStar was public.
Are people pairing human and AI yet in BW? So you have the AI running constantly making SCVs and microing perfectly, but the human player has total override so he can intervene to control what's happening strategically? And facing them off to create HIGH LEVEL games?
Are people pairing human and AI yet in BW? So you have the AI running constantly making SCVs and microing perfectly, but the human player has total override so he can intervene to control what's happening strategically? And facing them off to create HIGH LEVEL games?
It is possible, but I think it's impractical, and would not be that effective. The problem i see is communicating with the AI, and allocating resources. You can enable user input when developing a bot, but that's mostly used for debugging. I would be curious what an "assisted AI" could do against a human player, but I do believe it's harder than you imagine.
Wow thank you so much for the updates @Sonko!! A great read for a sunday afternoon
On the "Neuronal" mistake, unfortunately I can't edit the thread title... but you know, my profile quote says it all xD
You're very welcome Everyone makes mistakes, and especially if you're not familiar with these terms.
Are people pairing human and AI yet in BW? So you have the AI running constantly making SCVs and microing perfectly, but the human player has total override so he can intervene to control what's happening strategically? And facing them off to create HIGH LEVEL games?
It is possible, but I think it's impractical, and would not be that effective. The problem i see is communicating with the AI, and allocating resources. You can enable user input when developing a bot, but that's mostly used for debugging. I would be curious what an "assisted AI" could do against a human player, but I do believe it's harder than you imagine.
This is already done with success in SC2 in fact, by illegal hack(+assist) software. Mostly very easy things to automatise for a bot (perfect injection, marine splits, creeps, minimap incursion highlight, ... ), with probably a setting to enable/disable it in game. On can argue even things like auto-pathfinding (or auto targeting higher priori target, or smart casting) is already a human+bot/ai fusion. I know nearly nothings about BW but in SC2 all theses automations are real and mostly works fine. (well, pathfinding is sometimes a bit strange tbh ). So yep if one choose well the tasks to automatise, it's doable and produce a stronger "player" than an human in a bare-bones BW-esque situation.
On November 11 2019 00:09 Sonko wrote: Still, the best of these bots are at about B- level in human terms.
I've been wondering for a while, how fast is the level of the best bots increasing? I know the best bot are getting updated relatively regularly so it should be actually quite fast
FYI, after the AIIDE 2018 competition, Facebook published a couple of papers in Nov 2018: "High-Level Strategy Selection under Partial Observability in StarCraft: Brood War" at https://arxiv.org/abs/1811.08568 "Forward Modeling for Partial Observation Strategy Games - A StarCraft Defogger" at https://arxiv.org/abs/1812.00054
I am very grateful to Facebook for releasing the installers for and and fully open-sourcing their Brood War AI architecture (TorchCraft) and their bot (CherryPi), for the time they spent to document them, and all their papers. I wish DeepMind had taken more of that approach with AlphaStar for Starcraft 2. The paper, pseudocode and detailed technical architecture info etc for AlphaStar are great, but if they had released the installer and full source code, that would have been fantastic.
On November 11 2019 00:09 Sonko wrote: APM restriction is not really a good metric. In SC2 it's different - the game handles a lot of things well and smoothly. In BW, you kinda need a lot of APM as a bot to get basic things done. And also, the APM does not represent the actual processing behind decision making.
In the SSCAIT replays, APM goes up to tens of thousands routinely. Still, the best of these bots are at about B- level in human terms.
If you think that's not fair, then I encourage you to write a bot yourself, and try to beat Flash. I can pretty much guarantee that you won't be able to do that.
I'm willing to make a bet that the first bot that reliably beats human pros won't be using any tactic developed by humans, but rather, something that's only feasible to execute for bots.
And before the booing begins, look at the date. This was before the existenc of AlphaStar was public.
I think you've misunderstood my position on the matter. There shouldn't be any booing. When I wrote "under certain fair APM/reaction restrictions that we can agree on" I did not mean that it's not an achievement to win the best player without exploiting advantages an AI would have. I've done work in machine learning too (data classification and prediction) so I know how hard these things are, and I have respect for everyone involved.
Your conjecture is easily relatable imo, as that is an advantage the AI can exploit. I'd still be impressed if the AI can defeat Flash (or any other top tier progamer) by exploiting mechanics (eg with crazy dropship projectile evasion tactics), but the ultimate goal, I think we can agree on, is to be able to win without having to rely on exploiting superhuman mechanics. That was what I was trying to say.
The day that it happens, new efficiencies of BW might be revealed to us similar to what Open AI has done for DotA recently.
I'm waiting for the day where I can see Larva bash his keyboard after losing against some superhuman AI micro haha
only thing that would pique my interest (and the one im sure that would be considered the development in the field of AI) is when AI able to beat a pro human using tactics/decisionmaking/mindgames rather than banking on inhuman mechanics.
see that game soulkey vs sharp in ksl not too long ago where he trick sharp into thinking hes going for a lurker bust? thats the sort of tactics i want to see rather than inhuman micro that some rudimentary bot like Automaton 2000 can also do.
On November 11 2019 17:10 ggsimida wrote: only thing that would pique my interest (and the one im sure that would be considered the development in the field of AI) is when AI able to beat a pro human using tactics/decisionmaking/mindgames rather than banking on inhuman mechanics.
see that game soulkey vs sharp in ksl not too long ago where he trick sharp into thinking hes going for a lurker bust? thats the sort of tactics i want to see rather than inhuman micro that some rudimentary bot like Automaton 2000 can also do.
I think you've misunderstood my position on the matter. There shouldn't be any booing. When I wrote "under certain fair APM/reaction restrictions that we can agree on" I did not mean that it's not an achievement to win the best player without exploiting advantages an AI would have. I've done work in machine learning too (data classification and prediction) so I know how hard these things are, and I have respect for everyone involved.
Fair point, and I wasn't reacting to your statement, but rather, trying to prevent the predictable reaction. So mea culpa. (Or I should insert a "nothing personnel, kid" meme here )
I've been wondering for a while, how fast is the level of the best bots increasing? I know the best bot are getting updated relatively regularly so it should be actually quite fast
It's hard to tell, because most of the bot improvements are done for bot vs. bot matches, and evolving that meta - not necessarily getting better vs humans.
is there any way we can play the SSCAIT AI with the current bw remaster?
On November 11 2019 17:10 ggsimida wrote: only thing that would pique my interest (and the one im sure that would be considered the development in the field of AI) is when AI able to beat a pro human using tactics/decisionmaking/mindgames rather than banking on inhuman mechanics.
see that game soulkey vs sharp in ksl not too long ago where he trick sharp into thinking hes going for a lurker bust? thats the sort of tactics i want to see rather than inhuman micro that some rudimentary bot like Automaton 2000 can also do.
On November 11 2019 17:10 ggsimida wrote: only thing that would pique my interest (and the one im sure that would be considered the development in the field of AI) is when AI able to beat a pro human using tactics/decisionmaking/mindgames rather than banking on inhuman mechanics.
see that game soulkey vs sharp in ksl not too long ago where he trick sharp into thinking hes going for a lurker bust? thats the sort of tactics i want to see rather than inhuman micro that some rudimentary bot like Automaton 2000 can also do.
That's right, those games vs Mana on SC2 were so disappointing, this AI was doing nothing more than lifting more weight than a human could ever do. it is really misguiding to talk about artificial intelligence, really there is nothing intelligent about an AI, it is dumb as brick. The day it can quantify a mind game into binary code we could talk about intelligence, for now it is artificial whatever. Good luck to facebook dealing with hold lurker.
does anybody here know how to let two bots play each other with me just ob'sing? I tried 2v2 (only option since i gotta create the game) and ally with one bot as terran (lift cc) and at times i try use map settings creating on an Obs map but the screen gets stuck and the "start game" that needs to be clicked to commence is all greyed out. it's really frustrating because I want to watch different bots duke it out and play 2v2 if possible.
Note: I can play remastered with real people blah blah but botting is interesting to me so
On November 11 2019 17:10 ggsimida wrote: only thing that would pique my interest (and the one im sure that would be considered the development in the field of AI) is when AI able to beat a pro human using tactics/decisionmaking/mindgames rather than banking on inhuman mechanics.
see that game soulkey vs sharp in ksl not too long ago where he trick sharp into thinking hes going for a lurker bust? thats the sort of tactics i want to see rather than inhuman micro that some rudimentary bot like Automaton 2000 can also do.
That's right, those games vs Mana on SC2 were so disappointing, this AI was doing nothing more than lifting more weight than a human could ever do. it is really misguiding to talk about artificial intelligence, really there is nothing intelligent about an AI, it is dumb as brick. The day it can quantify a mind game into binary code we could talk about intelligence, for now it is artificial whatever. Good luck to facebook dealing with hold lurker.
Winning by "cheating with mechanics" is still a scientific achievement imo. I believe that we have to start with winning by "cheating with mechanics" and slowly progress to being able to win without exploiting mechanics.
EPT
