Hi Guys, I'm one of the organizers of the upcoming Starcraft AI Competition. This year's tournament is being run at the University of Alberta and will again use Broodwar.
This is the 2nd annual running of this competition, and will have a maximum of 30 teams. It was announced around 6 months ago, and submissions have finally started rolling in. I will be maintaining this thread for those who are interested in seeing what the state of the art in computer Starcraft AI is doing.
Some quick notes about the tournament for those who don't visit the website:
- Maximum of 30 teams - Round robin tournament - Each game has 1 hour time limit, with Starcraft score used at that point to determine the winner (to avoid situations where bots break and games don't end) - Games will be run on 40 machines simultaneously to get as many relevant results as possible
Here is a video of our current tournament set-up, using automatic game scheduling software to play the maximum number of games:
We will be keeping a page online showing the progress of the tournament as games are played, however specific results of the games will be kept secret until the tournament is over.
Any questions or comments about the tournament are welcome!
Would be cool if you could post the replays and have someone cast them (maybe you can ask Sayle if he is willing and able), or send us some of the games casted. These are always fun to watch, as each year the bots get a little trickier.
Looks pretty cool - It would be nice to pick a couple of good games - if there's any - and get them casted for people to see just how advanced (or not) bots are right now.
Nice stuff. Btw - why are you not releasing the results until after the tournament completes? You fear sabotage or hacking or...?
On August 08 2011 10:54 Essbee wrote: Wow this is really amazing. I showed this to a friend, he found it really great too O_O
Edit: Just a quick question: In which language these bots are made? C++?
The bots can be written in any language, but the main API is in C++. The two main ways of creating a bot are:
1) Write a native bot completely in C++ which is then injected into StarCraft using Chaoslauncher 2) Write a 'proxy bot' in a language which talks to StarCraft via memory sharing or sockets
I think you need to do 2 things. One is make it easy for players to play against these bots on 1 computer. Two would be some articles and hype like videos casts and things like that. having the bots all open source means someone can pick up with a real bot instead of having to start all over.
Is there any way to download some bots (like Berkley's overmind for example)? I'd really like to play against them and compare with the BWAPI ones. I've watched some videos on ucscbwebber's youtube channel and it appeared to me that a bot written "from scratch" performed only comparable (if not worse) to the unidimensional build-order-script oriented non-cheating BWAPI bots (raynor, panda, etc).
So i get it this tournament is to attract people who wanted to give them selves a shot in AI programming is this for those who are experience only or for the beginners too ?
Well if it helps I'll volunteer to cast some shit^^
AH I always love these bot competitions, mainly because they have the potential to show us the best MnM micro that has ever been seen. Y U NO MAKE Marines micro? screw mutas, screw dragoons, move dem marines back 'n forth individually and you win EZ!
On August 10 2011 19:34 Holy Check wrote: Is there any way to download some bots (like Berkley's overmind for example)? I'd really like to play against them and compare with the BWAPI ones. I've watched some videos on ucscbwebber's youtube channel and it appeared to me that a bot written "from scratch" performed only comparable (if not worse) to the unidimensional build-order-script oriented non-cheating BWAPI bots (raynor, panda, etc).
I think they perform better than the non-cheating BWAPI bots based on what I see so far on videos of these stuff.
On August 10 2011 21:44 bITt.mAN wrote: Well if it helps I'll volunteer to cast some shit^^
AH I always love these bot competitions, mainly because they have the potential to show us the best MnM micro that has ever been seen. Y U NO MAKE Marines micro? screw mutas, screw dragoons, move dem marines back 'n forth individually and you win EZ!
I believe MUTAR is the way to go when talking 1000 - 2000 APMs. If you allow a bot to get a decent amount of mutas he will rape you sideways. I believe this was the principle behind the Overmind bot.
I think that given enough time and effort, a BW bot will beat top progamers. If for example the BW departments of a few universities work together, I'm sure we'll see a B level bot pretty soon raping shit on ICCUP.
And you are seriously underestimating programmers. See what I did there?
Just by harassing multiple places at once, a computer can demolish a human player. Not to mention imbalanced muta micro, 10000 consume-plague-swarm combos at the same time, perfect macro, etc.
Sure, a human has the edge in the early game, a better grasp of strategy and better all-round decision making. And a computer can't match that. But that's thinking through the perspective of a human's style of play, since a computer-oriented style would definitely focus on mechanics and use extra-safe builds.
Plus the whole BW programming scene has been around for just a couple of years. Unlike chess for example, it hasn't benefited from the expertise of big companies, experienced programmers (no offense to university students) and expert gamers. But despite all this, Berkley's Overmind can beat a C- player.
If a handful of students from a handful of universities managed to make an AI that is much better than the Blizzard one in just a couple of years, WHILE having to struggle with the internals (the API), I don't see why we won't have a capable BOT in another couple of years.
I'm sure people had this exact same discussion back in '95 when talking about chess. Sure, chess and Starcraft are quite different, but at that time a supercomputer was as powerful as a Pentium 4 and it was facing a world champion.
On August 12 2011 01:12 Holy Check wrote: If a handful of students from a handful of universities managed to make an AI that is much better than the Blizzard one in just a couple of years, WHILE having to struggle with the internals (the API), I don't see why we won't have a capable BOT in another couple of years.
I'm sure people had this exact same discussion back in '95 when talking about chess. Sure, chess and Starcraft are quite different, but at that time a supercomputer was as powerful as a Pentium 4 and it was facing a world champion.
My credentials in the area are as follows:
- I am doing my phd in computer science in RTS ai - I submitted a bot for last year's tournament, have followed the scene since it started - Am organizing / running this year's tournament, and have seen all the entries in testing - Follow the pro BW / SC2 scene fairly closely - Am a diamond SC2 player (lol)
The things that bots are good at so far are:
- Micromanagement - Keeping track of information - High APM - Build order planning (this is my current research)
The things bots are REALLY bad at are:
- Learning / Adapting to opponent's playstyle - Opponent modeling (is he aggressive, etc) - Building placements is still poor - Multi unit-type pathfinding and micromanagement
The only way bots beat humans right now is by surprise. Our bot has beaten humans in matches, but it loses every subsequent match because it is still very difficult to incorporate true AI solutions into bots, so most things like micro are all scripted (to a point). This makes them very easy to exploit once you figure them out. Things like detecting that you are being kited are very difficult to do algorithmically.
Humans are very good at looking at a situation and reacting to it, bots are very very very bad at this right now, and until some major work is done to true ai solutions such as adversarial search for high level RTS planning (which is something I plan on looking at next year) then humans will always be able to 'figure out' bots and beat them trivially. (which is the current state of the art). Even berkley's bot from last year (which won) had very simple rules to attempt to react to opponent's actions which basically amounted to a large finite state machines of gameplay rules.
tl; dr: Bots can beat humans by surprising them, but humans eventually figure out their weakness and exploit them, something bots are not good at. This will be the case for years to come.
Edit: Chess is a different problem than StarCraft, because StarCraft's state space and action space are so much larger than chess. Chess in '95 was already in a state where all they needed to beat humans was a computer that was 20x as powerful. As soon as that hardware came out they beat the best humans in the world. StarCraft would need hardware that could do more calculations per second than there are atoms in the universe to play StarCraft as well as chess programs play chess
I have seen that more and more people choose to further their studies (master's degree, doctoral degree) in RTS and Starcraft in particular. That is excellent news, since a dedicated team working a few years continually towards a goal like this is bound to have results.
You say bots are bad at adapting at their opponent's playstyle. But what if the approach is the other way around? What if bots are built in with a few personalities (pretty much like chess) which are randomly fielded? Something like: turtle zerg, aggressive zerg, all-in zerg, etc. That would pretty much by itself mix things up sufficiently in a BO3 or BO5 series.
To further the comparison with chess, I'd say that AI research itself isn't in its infancy stages, so it's not like Starcraft AI programming starts from scratch. Also, SC has a huge advantage over chess: it can be compartimentalized. You can have independent teams working on micro, macro, build orders, scouting + reacting, spell casting, harassing, etc.
So maybe a higher-stakes more international competition would encourage people to work together more and even attract a few high class players to help as consultants.
I just hope you stick to SCBW and not switch over to SC2.
On August 12 2011 17:37 Holy Check wrote: I have seen that more and more people choose to further their studies (master's degree, doctoral degree) in RTS and Starcraft in particular. That is excellent news, since a dedicated team working a few years continually towards a goal like this is bound to have results.
You say bots are bad at adapting at their opponent's playstyle. But what if the approach is the other way around? What if bots are built in with a few personalities (pretty much like chess) which are randomly fielded? Something like: turtle zerg, aggressive zerg, all-in zerg, etc. That would pretty much by itself mix things up sufficiently in a BO3 or BO5 series.
To further the comparison with chess, I'd say that AI research itself isn't in its infancy stages, so it's not like Starcraft AI programming starts from scratch. Also, SC has a huge advantage over chess: it can be compartimentalized. You can have independent teams working on micro, macro, build orders, scouting + reacting, spell casting, harassing, etc.
So maybe a higher-stakes more international competition would encourage people to work together more and even attract a few high class players to help as consultants.
I just hope you stick to SCBW and not switch over to SC2.
The reason we think it can be broken down into smaller areas is because this is the way humans visualize the game. If you look at chess AI, computers play chess far differently (and now far better) than humans do. Humans are very good at pruning the game tree based on learned heuristics, while computers are very bad at this in comparison, so they do massive searches for the best response to opponent strategies using some sort of minimax algorithm. For the time being we are sticking to breaking down RTS into its human recognizable components because this is a good heuristic, but it may not be the best way to play starcarft 20 years from now (from an AI point of view).
And don't worry, we'll be sticking to BW for a long time, Activision's Blizzard doesn't let us touch SC2
On August 12 2011 19:04 djbhINDI wrote: This is such a cool idea. Wonder how much more is built into SC2 AI by default.
The SC2 AI is not much better by comparison. It has a few opening builds it randomly chooses from and then has set attack timings. Also, the insane AI knows where you are and gets additional resources. You can test this by flying your command center at the start of the game and watching the insane AI go right for it without scouting it.
Everybody goes to starcraft 2 but really brood war is much better balanced with tons more information on it. Starcraft 2 being online only, and the system requirements are way to high to devote the kind of cpu you need to run real AI algorithms. People must not understand the amount of CPU you need to do AI you can never have enough. Even using trees and the fastest data structures in existence it just takes tons of CPU to get AI to work at difficult complex problems such as ones in starcraft.
I didn't realize that the lack of computing power is the thing keeping BW AI back. Always thought it was more of an algorithm problem. Perhaps after this competition we will see a paralel BW AI (that would be pretty baller). Placing the different modules of the AI on different computers should be able to make a big difference if the algorithms are good.
On August 13 2011 16:37 Holy Check wrote: I didn't realize that the lack of computing power is the thing keeping BW AI back. Always thought it was more of an algorithm problem. Perhaps after this competition we will see a paralel BW AI (that would be pretty baller). Placing the different modules of the AI on different computers should be able to make a big difference if the algorithms are good.
At this point it's still an algorithms issue. We have nothing that can cope with the scale of RTS search space. It only becomes about "CPU Power" when you get the algorithms working to within a few orders of magnitude of running in real-time.
The bottom line is, the API needs to have a CULA or some other gpu-accelerated port so the real muscle of modern computing could flex its muscles. Remember, starcraft analysis is largely a parallel-able process due to multidimensional nature of map.
40 seems like really really low number for testing AI and running them concurrently means any adaptive AI that can use previous knowledge to improve will be pretty much worthless. Of course I suspect most if not all the AI are just rush build and micro the units to victory types so it wouldn't matter, still very cool.
I was just wondering if it was possible for a programmer to make a launcher that let's human players play starcraft at 2x fastest speed. It would make practicing more interesting.
On August 14 2011 06:38 littlefighter wrote: I was just wondering if it was possible for a programmer to make a launcher that let's human players play starcraft at 2x fastest speed. It would make practicing more interesting.
Yes, just download the BWAPI example module, set speed to whatever you want and enable user input. The bot does nothing while you control it at NX speed
On August 14 2011 06:38 littlefighter wrote: I was just wondering if it was possible for a programmer to make a launcher that let's human players play starcraft at 2x fastest speed. It would make practicing more interesting.
Yes, just download the BWAPI example module, set speed to whatever you want and enable user input. The bot does nothing while you control it at NX speed
Ah thank you for replying, I need a bit of help with it.
I downloaded BWAPI 3.6.1, I found ExampleAIModule.cpp in BWAPI_3.6.1\ExampleAIModule\Source.
And the line Broodwar->enableFlag(Flag::UserInput); is in it, but ctrl-f speed doesn't find anything. So I'm completely lost o___o I don't program.
On August 12 2011 14:14 Serendib wrote: As soon as that hardware came out they beat the best humans in the world. StarCraft would need hardware that could do more calculations per second than there are atoms in the universe to play StarCraft as well as chess programs play chess
Sounds like a challenge to Intel and AMD!
Haha I'm looking forward to this as always, usually some pretty cool games, hope you release the finals or whatever reps/vods of some of the coolest stuff.
It might take a while, but every bit of understanding garnered will be a step forward to understanding the complexity of calculating BW.
On August 14 2011 06:38 littlefighter wrote: I was just wondering if it was possible for a programmer to make a launcher that let's human players play starcraft at 2x fastest speed. It would make practicing more interesting.
Yes, just download the BWAPI example module, set speed to whatever you want and enable user input. The bot does nothing while you control it at NX speed
Ah thank you for replying, I need a bit of help with it.
I downloaded BWAPI 3.6.1, I found ExampleAIModule.cpp in BWAPI_3.6.1\ExampleAIModule\Source.
And the line Broodwar->enableFlag(Flag::UserInput); is in it, but ctrl-f speed doesn't find anything. So I'm completely lost o___o I don't program.
I downloaded skynet that was in this competition and its pretty good. It beat 2 built in AI computers easily, but it loses every time to a quick gas steal. A planner like overmind had seems almost mandatory to deal with things like say blocked expansions, pylons in mineral line, or gas steals. I also think you should cast or give some articles on the best bots to get people to support it.
Any update? The competition is supposed to end tomorrow.
Also, I am quite the confused camper... From what I've seen on the website only 13 out of the 30 bots are working? What happened? Also what happened to the Berkeley Overmind. The only entry I see for Berkeley is the Undermind, and it's a Terran, not a Zerg bot.
I played some games against skynet and while it is a good bot it has problems that overmind solved last year. If you play terran the bot goes dragoons, but if you steal its gas it stops building units and loses. The overmind dealt with gas steal so it seems bots need to learn from others so each bot does not have the same problems as other bots.
I think that in order to compete fairly with humans, the bots must have some APM limit at around 350 (but keep selecting several units as one action, not equal to the number of selected units like it is now in BWAPI if I'm not mistaken). It will be much harder task for bots, because they will need to reduce the amount of actions spent on micro. In this way they will play more like a human, decide how to spend the actions efficiently. A more better way to create bots will be not writing exact algorithms for reacting to the input, but make bots to behave randomly in order to get experience - bots remember random actions, analyze the results and produce algorithms based on it for future use. Of course analyzing is a difficult word, but on the first steps the programmer can participate in some analysis by implementing some basic concepts.
I really want these bots to improve just so I can play brood war again without any lag. Playing people brings in lag, and its hard to find games against people your skill. A good bot could solve that along with advancing AI.
On October 17 2011 07:09 snakeeyez wrote: I really want these bots to improve just so I can play brood war again without any lag. Playing people brings in lag, and its hard to find games against people your skill. A good bot could solve that along with advancing AI.
We would love our bots to be played more against humans. Have you tried it yet?
When I try to play the bots using the bwapi and the multiple instances I pick the local pc option. Then I play one of the instances, but after about 15 minutes we both get dropped out of the game so I can never finish any games against the bots. I played against skynet and after stealing his gas he stopped making units. I think if it was a little easier to setup and play against the bots more people would do it.
EPT
