The AIIDE 2010 StarCraft Competition has come to a close. The challenge given to competitors was to build the best performing bot for an immensely popular, commercial game. The competition consisted of four tournaments of varying complexity. This was the first year the competition was held and it turned out to be a success. Even though no prizes were offered, over twenty-eight teams participated in the competition.
The showcase game of the competition was a bot versus human match. In the exhibition match, =DoGo=, a World Cyber Games 2001 competitor played against the top ranking bot of the competition. The result was an exciting man versus machine match highlighting the state-of-the-art in RTS AI.
While the expert player was capable of defeating the top performing bots in the competition, the results are quite encouraging.
Tournament 1: Micromanagement The first tournament evaluated bots in unit micromanagement scenarios.The winner of this tournament was FreSCBot, which uses multi-agent finite-state machines. The runner-up was Sherbrooke, which also uses state machines. Complete tournament details are available here and the full results are posted here. The video below shows the final match between FreSCBot (red) and Sherbrooke (blue).
Tournament 2: Small-Scale Combat The second tournament evaluated bots in small-scale combat scenarios. The results of this competition mirrored the results of the first tournament: FreSCBot won and Sherbrooke was runner-up. Complete tournament details are available here and the full results are posted here. The video below shows the final match between FreSCBot (purple) and Sherbrooke (orange).
Tournament 3: Tech-Limited Game The third tournament tested bots in a tech-limited StarCraft environment which requires reasoning at strategic and tactical levels, but omits much of the complexity of the full version of StarCraft. The winner of this tournament was Mimic Bot, which attempted to mirror the opponent's strategy while also performing a gas steal and applying fallback strategies. The runner up was Botnik, which executes a Zealot rush strategy. Complete tournament details are available here and the full results are posted here. The video below shows the final match between MimicBot (red) andBotnik (teal).
Tournament 4: Full Gameplay The final tournament was a best-of-five, double-elimination tournament simulating a professional gaming competition. The winner of this tournament was Overmind, a Zerg bot that effectively scouted its opponents, interrupted their economy, and performed Mutalisk harassment until victory was ensured. The runner-up was Krasi0, a Terran bot with excellent defense and pushing capabilities. Complete tournament details are available here and the full results are posted here. The video below shows an exert from one of the games between Overmind (yellow) and Krasi0 (brown).
Highlight #1: Krasi0 versus Skynet During this minute of action between bots Krasi0 and Skynet, it was difficult to distinguish whether the game was being played by skilled human players or the StarCraft equivalent of Deep Blue.
Highlight #2: Berkeley's Mutalisk Evolution Contemporary StarCraft wisdom tells us that the best way to use mutalisks is to clump them. In human versus human battles, this makes it difficult to single out the weaker mutalisks, because the units are stacked on top of each other. However, UC Berkeley's team identified a flaw in this tactic; it reduces the damage output of each individual mutalisk, because not all mutalisks will fire when using this tactic. Instead, they employed a model in which mutalisk are always moving, maximizing damage output while simultaneously maximizing movement.
Conclusion The inaugural StarCraft AI Competition was a huge success! While no prizes were planned originally, Blizzard sponsored the competition and the winner of each tournament received a collectors edition of StarCraft 2 signed by the dev team. I would like to thank Blizzard, BWAPI, AAAI, TL, and all of the participants in making this event a reality. The next competition will be hosted by the University of Alberta at AIIDE 2011.
Update: VODs from the Competition I've recorded a few of the more interesting games from the StarCraft AI Competition. All replays from the competition are available and I encourage people to post additional videos and commentaries: http://eis.ucsc.edu/StarCraftAICompetition#Results
Here's one of the matches from the finals between Krasi0 and Overmind (Replay):
This video shows the initial match between =DoGo= and Krasi0, with DoGo playing as Protoss (Replay):
One of the highlights of the competition, Skynet versus Krasi0. This video shows the match-up on Heartbreak Ridge (Replay):
The semi-finals between Chronos and Overmind (Replay):
Edit: Post in this thread if you upload any videos!
WOW!!! FreSCBot in the Micro Tournament totally dominated the competition. 6 zealots left! It looked like it didn't even lose a mutalisk either but I couldn't really tell. I wish they had the units selected so we could see the health bars.
EDIT: LOL I just watched the small-scale tournament. Same bot(FreSCBot) lost a SINGLE marine out of 20. Embarrassing really.
What the teams managed to accomplish in terms of micro control is fairly impressive, but from what I've seen of the actual game replays they seem to have trouble with keeping decent macro.
This is a bit surprising considering that macro is often though of as what an AI would do best. This makes me wonder what it is that caused a lot of the AIs to run at 1k+ minerals for 5+ minutes.
It's rather funny how there actually are AI scripts (using Blizzards script language) that macro better but would probably loose to one of these bots because the AI script micro is horrendous.
Maybe if I have some time and can figure out how to get Overmind zerg bot setup I'll test how it fairs against one of my Terran anti-zerg scripts.
On October 14 2010 14:56 Equalizer wrote: What the teams managed to accomplish in terms of micro control is fairly impressive, but from what I've seen of the actual game replays they seem to have trouble with keeping decent macro.
This is a bit surprising considering that macro is often though of as what an AI would do best. This makes me wonder what it is that caused a lot of the AIs to run at 1k+ minerals for 5+ minutes.
It's rather funny how there actually are AI scripts (using Blizzards script language) that macro better but would probably loose to one of these bots because the AI script micro is horrendous.
Maybe if I have some time and can figure out how to get Overmind zerg bot setup I'll test how it fairs against one of my Terran anti-zerg scripts.
Overmind does have good macro. It's just we have to account for cheese that may or maynot show up, so we always make some super early sunken colonies that leads to starvation. Also, bots are bad at determine if an attack is life threatening or not, so a lot of times the defenses are being put up prematurely. That combined with our 2 hatch muta opening does not lend to a smooth early game macro. However, if you try to play a long game with Overmind it will actually take over the map while saturating each expansion with drones and, if it is smart enough not to suicide mutalisks, it can max 200/200 in reasonable time.
PvP matchups were unique, and fun. My bot, ZotBot, used one of two openings, each had a 100% win rate vs Protoss. I believe my ZotBot had 9 PvP matches.
It just occured to me that in a way, computers cheat when they micro, in that they take advantage of their more direct interface with the game in a way humans can't. Humans either have to locate a unit by sight and then select it by hand, or keep it hotkeyed to one of just 10 groups, while i bet the computer can just magically call up a unit anywhere on the map. I think for a computer to truly legitimately beat a human player, it'd have to get information from the game only by means of a camera facing the moniter, and operate the mouse with a robotic arm.
On October 15 2010 09:25 zobz wrote: It just occured to me that in a way, computers cheat when they micro, in that they take advantage of their more direct interface with the game in a way humans can't. Humans either have to locate a unit by sight and then select it by hand, or keep it hotkeyed to one of just 10 groups, while i bet the computer can just magically call up a unit anywhere on the map. I think for a computer to truly legitimately beat a human player, it'd have to get information from the game only by means of a camera facing the moniter, and operate the mouse with a robotic arm.
that's ridiculous. the point is whether it's an "AI" that can think well enough to play. which is 99% of the challenge and a ridiculously difficult problem.
you could take your argumentt a level further and say that the robotic arm has an advantage because it never gets tired, or is too precise, or whatever, so to be "fair" you have to emulate a human arm cell by cell.
On October 15 2010 09:25 zobz wrote: it'd have to get information from the game only by means of a camera facing the moniter, and operate the mouse with a robotic arm.
As far as video processing, there's already a pacman competition that requires this process.
On October 15 2010 09:25 zobz wrote: It just occured to me that in a way, computers cheat when they micro, in that they take advantage of their more direct interface with the game in a way humans can't. Humans either have to locate a unit by sight and then select it by hand, or keep it hotkeyed to one of just 10 groups, while i bet the computer can just magically call up a unit anywhere on the map. I think for a computer to truly legitimately beat a human player, it'd have to get information from the game only by means of a camera facing the moniter, and operate the mouse with a robotic arm.
that's ridiculous. the point is whether it's an "AI" that can think well enough to play. which is 99% of the challenge and a ridiculously difficult problem.
you could take your argumentt a level further and say that the robotic arm has an advantage because it never gets tired, or is too precise, or whatever, so to be "fair" you have to emulate a human arm cell by cell.
Of course rescripting to make a computer understand the flow of the game is a momentus challenge in itself ffs. But i'm sure you wouldn't claim that the physical aspect of starcraft is negligable. What i'm asking is, what do you think two robotic arms operating a mouse and keyboard at 20 000 apm would look like? Unimpressive? The implication is both that a robot would have a much harder time beating a human if it had to physically perform its apm, and that it would probably take a substantial amount of reprogramming to get it to use the minimap and hotkeys effectively, so that it didn't have to be impossibly fast. The thing is that an sc robot shouldn't exclusively be compared to a chess robot, but also to a soccer robot because it is a physical sport.
On October 15 2010 09:25 zobz wrote: It just occured to me that in a way, computers cheat when they micro, in that they take advantage of their more direct interface with the game in a way humans can't. Humans either have to locate a unit by sight and then select it by hand, or keep it hotkeyed to one of just 10 groups, while i bet the computer can just magically call up a unit anywhere on the map. I think for a computer to truly legitimately beat a human player, it'd have to get information from the game only by means of a camera facing the moniter, and operate the mouse with a robotic arm.
that's ridiculous. the point is whether it's an "AI" that can think well enough to play. which is 99% of the challenge and a ridiculously difficult problem.
you could take your argumentt a level further and say that the robotic arm has an advantage because it never gets tired, or is too precise, or whatever, so to be "fair" you have to emulate a human arm cell by cell.
Of course rescripting to make a computer understand the flow of the game is a momentus challenge in itself ffs. But i'm sure you wouldn't claim that the physical aspect of starcraft is negligable. What i'm asking is, what do you think two robotic arms operating a mouse and keyboard at 20 000 apm would look like? Unimpressive? The implication is both that a robot would have a much harder time beating a human if it had to physically perform its apm, and that it would probably take a substantial amount of reprogramming to get it to use the minimap and hotkeys effectively, so that it didn't have to be impossibly fast. The thing is that an sc robot shouldn't exclusively be compared to a chess robot, but also to a soccer robot because it is a physical sport.
again, you are missing the point. This is an AI competition. If you want a sensor and actuator competition (that's what you are describing) there are such that exist. But that's not the point of this competition. Your point is valid, as in, they are right, the computer do have advantages that it can query units across the map, but your point is irrelevant to the competition itself.
On October 15 2010 09:25 zobz wrote: It just occured to me that in a way, computers cheat when they micro, in that they take advantage of their more direct interface with the game in a way humans can't. Humans either have to locate a unit by sight and then select it by hand, or keep it hotkeyed to one of just 10 groups, while i bet the computer can just magically call up a unit anywhere on the map. I think for a computer to truly legitimately beat a human player, it'd have to get information from the game only by means of a camera facing the moniter, and operate the mouse with a robotic arm.
that's ridiculous. the point is whether it's an "AI" that can think well enough to play. which is 99% of the challenge and a ridiculously difficult problem.
you could take your argumentt a level further and say that the robotic arm has an advantage because it never gets tired, or is too precise, or whatever, so to be "fair" you have to emulate a human arm cell by cell.
Of course rescripting to make a computer understand the flow of the game is a momentus challenge in itself ffs. But i'm sure you wouldn't claim that the physical aspect of starcraft is negligable. What i'm asking is, what do you think two robotic arms operating a mouse and keyboard at 20 000 apm would look like? Unimpressive? The implication is both that a robot would have a much harder time beating a human if it had to physically perform its apm, and that it would probably take a substantial amount of reprogramming to get it to use the minimap and hotkeys effectively, so that it didn't have to be impossibly fast. The thing is that an sc robot shouldn't exclusively be compared to a chess robot, but also to a soccer robot because it is a physical sport.
again, you are missing the point. This is an AI competition. If you want a sensor and actuator competition (that's what you are describing) there are such that exist. But that's not the point of this competition. Your point is valid, as in, they are right, the computer do have advantages that it can query units across the map, but your point is irrelevant to the competition itself.
I didn't miss anything said by the person to whom i was responding.
It would be great to get some feedback on how the bots performed.
I've seen the reps posted in this thread. What can i say the AIs are cute furry little noobs: - they seems to scout but that is just for the show eg: after scouting the Overmind AI attacked the wrong base with some lings lolz -the terran ai knows to pull scvs to defend, it also made a cute wall with 3 scvs and marines on heartbrake ridge -they will not suicide units in defenses eg: terran pulled back after losing 2 marines to sunkens -mutalisk micro will give you a ":D" , basically every mutalisk dances like crazy however its still bad eg: 30 mutalisks will refuse to kill a lonely goliath because they are busy microing to kill a command center lolz -terran can lay mines here and there but the push is horrible, it needs more units, also it sends vultures at expos for harass but they die to cannons -terran is very efficient when it comes to repairing units
This is what got my attention during thereplays, i would like to see the showmatch. I wonder how bad it got raped or maybe the human went easy on him. xD hehehe
You can see how the AI makes bad decisions especially in the last video. Mutas focused fired down a bunker over everything else and what the terran AI decided to build
Arstechnica had a really nice write up about the development of the winner of the Full Gameplay competition. Was an interesting read. Thought others might enjoy as well.
Yeah, while there were a few mistakes (namely some statements that implied that mutalisk clumping was bad) it was really cool. Also, that things muta micro vs archons was sick.
Did AI's consider worker micro for harvesting purposes? With constant micro AI could achieve maximum harvesting with 2 workers per mineral patch, hence faster macro mode.
EPT
He's scary enough with his mutas already! D:
.
