|
On December 08 2010 09:17 jacobman wrote:Show nested quote +On December 08 2010 02:08 Skrag wrote:On December 07 2010 20:14 jacobman wrote:On December 07 2010 04:56 Skrag wrote:
Good news! After doing some research on sc2mapster, I found the set of commands that will let me set up the AI to execute perfectly at every point, including exact worker gather micro and transfer logic. It'll be a little bit of work to get things set up to execute various build orders, but now that I know where to look, it's pretty easy to get the AI to follow exact directions. Do you know how to set up a drone transfer and a rally point change in the galaxy scripts? I'm trying to set the computer up to run the builds in a game, but currently I don't know how to do those two parts. Rally points can be changed by issuing an order to the unit/building you want to change the rally for. Transfering could be achieved by giving the drones you want to transfer a harvest order targeting the mineral patches at the expansion. (That was the key order I couldn't figure out how to do before btw, sending a specific drone to a specific mineral patch) Do you know what the syntax is for that? It's really hard to find the names for abilities since there isn't complete documentation yet. Here are my guesses so far. Mineral Patch: either C_MineralField or C_CU_MineralField Gather Minerals: C_AB_DroneHarvest Move: C_AB_Move Can you confirm any of those so I might know which terms aren't causing syntax errors?
I can't answer that question, because I'm using triggers exclusively, not writing scripts. I'm *very* close to having a fully working 11 overpool though. (so close that it might be done, I just have to test out the final results) Once that's done, adding other build orders should be relatively easy, since the overpool does some fairly complex things that won't be needed for other builds.
|
My AI-driven 11overpool is FINALLY working. I ran into a problem at the very end where the AI would have the queen cast inject as long as the queen was at the main hatchery, but wouldn't do it automatically at the expansion hatch, so I had to manually watch for energy and inject.
The 6 minute results are better than my previous "best" replay by 65 minerals, a completed drone (bumping the total up to 45 drones completed), 5/17 better on in-progress drone completion, and 9/40 better on injects.
After that huge headache, and figuring out how to do everything I needed to be able to do, it should be fairly easy to add other build orders, as well as a frequent state output to a file which should smooth out a lot of the problems that can be caused by your choice of when to actually measure.
|
On December 08 2010 13:35 Skrag wrote:
My AI-driven 11overpool is FINALLY working. I ran into a problem at the very end where the AI would have the queen cast inject as long as the queen was at the main hatchery, but wouldn't do it automatically at the expansion hatch, so I had to manually watch for energy and inject.
The 6 minute results are better than my previous "best" replay by 65 minerals, a completed drone (bumping the total up to 45 drones completed), 5/17 better on in-progress drone completion, and 9/40 better on injects.
After that huge headache, and figuring out how to do everything I needed to be able to do, it should be fairly easy to add other build orders, as well as a frequent state output to a file which should smooth out a lot of the problems that can be caused by your choice of when to actually measure.
I haven't tried to optimize miner micro past the first 9 workers, though (9 total, so the 6 you start with plus the next 3). It's pretty difficult to figure out which patch to send them to, and involves a lot of restarting. I may do that at some point, but if not, the micro for the first 9 will work just the same for other builds, and still give a reasonable comparison.
|
Ok, so here's the bad news. Even having the AI do the same things every time, and even using a fixed random seed when testing from the map editor, I'm seeing small differences between individual runs.
On the 11 pool, there's been as much as a 55 mineral and a full drone difference by the 6 minute mark, and I have no idea why. 
|
Hm, that is really odd. I would have expected that sc2 plus AI are a purely deterministic system and given the same starting parameters the output is identical every time. Is the AI perhaps running in different threads, so that it is not guaranteed that the exact order of execution of AI and sc2 commands might vary slightly? But good to know that there is this uncertainty.
However, on a more practical note, when it is impossible do obtain deterministic results, one either accepts an error of say 50 minerals and a drone, or one has to do some statistical averaging, giving say the mean and min/max over 10 trials. That is a lot of work of course.
|
Skrag, that is some very impressive testing. Having the AI do the builds is really the only way to comparable results.
I'm guessing the small differences are down to larva positions and drone movement AI. You should still get very good results for comparing builds if you do multiple runs and taking the averages.
I would love to see what you results you get for a 14 hatch 15 pool. Again, great job!
|
On December 08 2010 18:44 Ctuchik wrote:
Skrag, that is some very impressive testing. Having the AI do the builds is really the only way to comparable results.
I'm guessing the small differences are down to larva positions and drone movement AI. You should still get very good results for comparing builds if you do multiple runs and taking the averages.
I would love to see what you results you get for a 14 hatch 15 pool. Again, great job!
Yeah, it certainly makes it easier to get multiple test points. Doing runs on super slow over and over again is annoying. I'm really intrigued that there is still variations in the end results. I had figured the computer would do things exactly the same every time.... so weird.
I'm basically at the same point as you right now. I'm pretty much done with the maynard function, and when that is done I'll be able to get the computer to run any builds too. I'm curious to test out this variation in results you mentioned. I wonder where the differences are.
PS. I figured out the script question I had earlier. SC2mapster is awesome. Thanks for mentioning it earlier.
|
On December 08 2010 19:07 jacobman wrote:Show nested quote +On December 08 2010 18:44 Ctuchik wrote:
Skrag, that is some very impressive testing. Having the AI do the builds is really the only way to comparable results.
I'm guessing the small differences are down to larva positions and drone movement AI. You should still get very good results for comparing builds if you do multiple runs and taking the averages.
I would love to see what you results you get for a 14 hatch 15 pool. Again, great job! Yeah, it certainly makes it easier to get multiple test points. Doing runs on super slow over and over again is annoying. I'm really intrigued that there is still variations in the end results. I had figured the computer would do things exactly the same every time.... so weird. I'm basically at the same point as you right now. I'm pretty much done with the maynard function, and when that is done I'll be able to get the computer to run any builds too. I'm curious to test out this variation in results you mentioned. I wonder where the differences are. PS. I figured out the script question I had earlier. SC2mapster is awesome. Thanks for mentioning it earlier.
Okay so I've finished doing a few preliminary tests with the AI running a build order. I chose the 13 Pool 15 Hatch build since I have the most experience with the results of that build.
Just like with you, the AI improved on my best attempt (which I had started to think the max) by 70 minerals. The good news is that I tried a few attempts and each time it ended with the same minerals mined at the 6:00 mark. I think you should double check your replays. You might not be taking the data at the exact beginning of the 6:00 mark. Even small fractions of a second can lead to some variation, like you were experiencing.
I'm going to start applying the AI to all the build orders I have. I'll update some time tomorrow.
|
From what Idra and ret said, you can't hold off terran 2 rax unless you go hatch first, not for the econ, but for the larva,
they tested it too, but didn't post results though.
|
On December 09 2010 00:50 johanngrunt wrote:
From what Idra and ret said, you can't hold off terran 2 rax unless you go hatch first, not for the econ, but for the larva,
they tested it too, but didn't post results though.
Interesting that getting the hatch up is so important.
If you have a specific build order, with overlord timings to 52 supply, let me know. I'm going to be doing build order runs later with the AI.
|
Shouldn't you add
15 hatch and 14 pool. ? it's the most common thing psy does(the dude from utube)
|
On December 09 2010 01:02 nTwLegy wrote:
Shouldn't you add
15 hatch and 14 pool. ? it's the most common thing psy does(the dude from utube)
Sure thing. Do you have overlord timings I could use, or do you mind getting some good overlord timings to test?
If not I may not have the results up for a couple days. I'm a little bit busy currently, but I will check out that build order too. It would be helpful if you knew the overlord timings though.
|
On December 08 2010 17:44 Skrag wrote:Ok, so here's the bad news. Even having the AI do the same things every time, and even using a fixed random seed when testing from the map editor, I'm seeing small differences between individual runs. On the 11 pool, there's been as much as a 55 mineral and a full drone difference by the 6 minute mark, and I have no idea why.
So it turns out that I just got lucky on the first two tests I did. There is some variance between tests. I thought about it a little bit, and I realized the difference is due to the random walk of the larva. That causes drones to spawn at different locations, which leads to slightly different mining behavior each time. I'm going to do like ten trials for each BO eventually to make up for this a bit. For now though, I'm just doing one test for each one to get started. Here are my results so far at the 6:20 mark.
14Hatch/15Pool
Minerals: 5305
Larva: 55.95
Pool Finish: 3:39
Hatch Finish: 3:40
13Pool/15Hatch
Minerals: 5187.5
Larva: 57.49
Pool Finish: 2:51
Hatch Finish: 4:13
|
On December 09 2010 00:35 jacobman wrote:
Okay so I've finished doing a few preliminary tests with the AI running a build order. I chose the 13 Pool 15 Hatch build since I have the most experience with the results of that build.
Just like with you, the AI improved on my best attempt (which I had started to think the max) by 70 minerals. The good news is that I tried a few attempts and each time it ended with the same minerals mined at the 6:00 mark. I think you should double check your replays. You might not be taking the data at the exact beginning of the 6:00 mark. Even small fractions of a second can lead to some variation, like you were experiencing.
I'm going to start applying the AI to all the build orders I have. I'll update some time tomorrow.
I saw you came to the same conclusion I did, which is that the random larvae walk is causing the variance between runs. It should be exactly the same with a fixed random seed, but for some reason it's not, and I've definitely been seeing drones pop up at different spots. It's also possible that the AI itself has a small amount of inconsistency, like maybe the AI doesn't run at the same rate as the simulation and there can be differences in exactly what frames it runs on.
I'm currently using SetStock in some places, and my own custom build triggers in others (because SetStock builds things in the order that you specify most of the time, but changes the order sometimes occasionally, and queens seem to be delayed by a second or two when using SetStock for them). I might try to figure out a cleaner way to make sure that everything gets built the instant it's possible to build it, because it seems like there might be a slight delay on building things that are waiting on larvae.
How are you doing things in script? I don't actually know how to create a custom AI script. The only instructions I managed to find online involve modifying the MPQ files, and I really don't want to do that if I can avoid it.
As far as measuring inconsistencies go, I'm 100% positive I'm measuring at exactly the same time every run, because I'm using a timer and printing the stats on-screen (and to a debug file, which will make it really easy to make graphs based on data spit out every 2-3 seconds) rather than getting data from the replay. There's a player property for resources collected, which shows the total amount of resources that have actually been taken from the patches.
One problem I'm running into is that 6 minutes is too long for a build that creates only drones on XelNaga caverns. Many maps only have 2 close spots, giving a maximum number of drones at the base of 22, but XelNaga has 4 close spots, so the maximum is 20, and there can be *very* long delays between the time the 19th and 20th drones are added and the time the workers all settle into a routine. In the tests I've been running, I've been cutting off the main base at 19 workers, but in almost every test, the drones still haven't settled into a routine at the 6 minute mark. Since you can only have 19-20 drones max at the main, this is oversaturating the expansion well before the 6 minute mark.
|
Want to exchange maps? Sounds like we've both taken very different approaches, and it might be interesting to peek over the fence. 
|
I used to go 17 or 18 hatch all the time, but at diamond level, this ain't happening! The best build ever is 13 hatch, 12 pool. Best of both worlds. you are safe, and have a good economy. Pool first, sucks, unless you have to defend against a rush.
|
To mitigate the oversaturation problems, and to be able to do some real comparisons of builds in action rather than just drone races, I'm thinking about adding things like gas, a pair of zerglings, and a spine crawler or two at the appropriate points in the various builds, while trying to do all the "extra" things as close as possible to the same time for each build.
|
Also, I decided to go ahead and try to optimize the drone micro on mineral patches for all of the drones going to minerals at the main. This definitely made a much larger difference than you were thinking it would. The difference between minimal micro (the first 9 workers) and full optimal micro (or as close as possible, it's *really* hard to get drones 17+ to go to a spot without causing any interruption in mining) was almost 100 minerals at the 6 minute mark in both 11p18h and 13p15h. And when microing optimally, the 11pool went down a second sooner, and the hatchery went down a full 5 seconds sooner.
It's actually kind of discouraging that these very small things can make such large differences, because once again, it means that the small nitty-gritty details can have a larger impact than what build you choose to begin with.
|
Some results, after optimizing drone micro as much as possible for all the drones that end up at the main hatchery. (expansion drones were all just rallied to a center patch)
The numbers in these reports are minerals collected, vespene collected, supply used, supply available, completed drone counts, drone count including partial drones in progress, larvae count, and inject progress seconds (so 79 would mean two injects are almost done, 40 could mean one just started and one almost done, or two midway).
11 OverPool
+ Show Spoiler +
0:00 report: 0 0 6 10 6 6.00 0 0
0:10 report: 30 0 7 10 6 6.58 2 0
0:20 report: 70 0 8 10 7 7.38 2 0
0:30 report: 120 0 9 10 7 8.19 2 0
0:40 report: 180 0 10 10 8 9.17 1 0
0:50 report: 230 0 11 10 9 10.06 1 0
1:00 report: 300 0 11 10 10 10.70 2 0
1:10 report: 375 0 11 10 11 11.00 1 0
1:20 report: 450 0 11 10 11 11.00 2 0
1:30 report: 530 0 10 18 10 10.00 3 0
1:40 report: 605 0 11 18 10 10.33 2 0
1:50 report: 670 0 13 18 10 11.60 1 0
2:00 report: 755 0 14 18 12 13.08 0 0
2:10 report: 835 0 15 18 13 14.26 0 0
2:20 report: 925 0 16 18 14 14.92 0 0
2:30 report: 1035 0 16 18 15 15.60 0 0
2:40 report: 1145 0 18 18 16 16.00 1 0
2:50 report: 1245 0 18 18 16 16.00 2 0
3:00 report: 1345 0 17 18 15 15.00 2 0
3:10 report: 1455 0 18 18 15 15.21 1 0
3:20 report: 1555 0 18 18 15 15.80 2 0
3:30 report: 1665 0 20 26 16 16.47 0 6.625
3:40 report: 1770 0 22 26 16 17.65 0 16.625
3:50 report: 1885 0 23 26 18 18.04 0 26.625
4:00 report: 2000 0 23 26 18 18.63 0 36.625
4:10 report: 2125 0 27 34 19 19.98 0 2.625
4:20 report: 2240 0 27 34 19 22.33 0 12.625
4:30 report: 2370 0 28 34 23 23.55 0 22.625
4:40 report: 2490 0 30 36 24 24.69 0 32.625
4:50 report: 2650 0 32 36 25 25.89 2 2.1875
5:00 report: 2790 0 35 36 26 28.78 0 20.375
5:10 report: 2975 0 36 36 30 31.43 0 40.375
5:20 report: 3135 0 37 44 32 32.17 0 60.375
5:30 report: 3325 0 39 44 32 33.18 3 39.375
5:40 report: 3545 0 48 52 34 38.20 0 14.9375
5:50 report: 3740 0 49 52 39 43.47 0 34.9375
6:00 report: 3975 0 50 52 44 44.99 0 54.9375
Started Spawning Pool at 87.6875
Finished Spawning Pool at 152.625
Started Hatchery at 172.063
Finished Hatchery at 272.063
13P15H
+ Show Spoiler +
0:00 report: 0 0 6 10 6 6.00 0 0
0:10 report: 30 0 7 10 6 6.58 2 0
0:20 report: 70 0 8 10 7 7.38 2 0
0:30 report: 120 0 9 10 7 8.19 2 0
0:40 report: 180 0 9 10 8 8.81 2 0
0:50 report: 230 0 9 10 9 9.00 2 0
1:00 report: 300 0 10 10 9 9.45 2 0
1:10 report: 365 0 12 18 10 10.05 0 0
1:20 report: 430 0 13 18 10 11.49 0 0
1:30 report: 505 0 13 18 12 12.85 1 0
1:40 report: 590 0 13 18 13 13.00 1 0
1:50 report: 675 0 13 18 12 12.25 1 0
2:00 report: 765 0 14 18 12 13.32 1 0
2:10 report: 850 0 15 18 14 14.58 0 0
2:20 report: 950 0 15 18 15 15.00 1 0
2:30 report: 1050 0 15 18 15 15.00 2 0
2:40 report: 1145 0 14 18 14 14.00 2 0
2:50 report: 1250 0 17 18 14 14.04 2 0
3:00 report: 1355 0 17 18 14 14.63 1 0
3:10 report: 1450 0 18 18 15 15.35 1 0
3:20 report: 1565 0 18 18 15 15.93 2 0
3:30 report: 1670 0 20 26 16 16.60 0 0
3:40 report: 1785 0 23 26 16 18.10 0 3.5625
3:50 report: 1900 0 23 26 18 18.91 0 13.5625
4:00 report: 2020 0 23 26 19 19.00 0 23.5625
4:10 report: 2135 0 24 26 19 19.32 0 33.5625
4:20 report: 2255 0 29 36 19 20.73 0 0
4:30 report: 2375 0 31 36 21 24.42 0 12.375
4:40 report: 2515 0 31 36 26 26.84 0 32.375
4:50 report: 2645 0 32 36 27 27.54 0 52.375
5:00 report: 2815 0 34 36 28 28.78 0 72.375
5:10 report: 2995 0 42 44 29 31.81 0 4.625
5:20 report: 3180 0 43 44 32 37.14 0 22.9375
5:30 report: 3385 0 44 44 39 39.41 1 42.9375
5:40 report: 3620 0 46 52 39 40.47 0 62.9375
5:50 report: 3850 0 51 52 40 42.66 0 38.5
6:00 report: 4095 0 52 52 43 46.29 5 14.5
Started Spawning Pool at 101.25
Started Hatchery at 150.625
Finished Spawning Pool at 166.188
Finished Hatchery at 250.625
14H15P
+ Show Spoiler +
0:00 report: 0 0 6 10 6 6.00 0 0
0:10 report: 30 0 7 10 6 6.58 2 0
0:20 report: 70 0 8 10 7 7.38 2 0
0:30 report: 120 0 9 10 7 8.19 2 0
0:40 report: 180 0 9 10 8 8.81 2 0
0:50 report: 230 0 9 10 9 9.00 2 0
1:00 report: 300 0 10 10 9 9.45 2 0
1:10 report: 365 0 12 18 10 10.05 0 0
1:20 report: 430 0 13 18 10 11.49 0 0
1:30 report: 505 0 13 18 12 12.85 1 0
1:40 report: 590 0 14 18 13 13.54 0 0
1:50 report: 680 0 14 18 14 14.00 1 0
2:00 report: 775 0 13 18 13 13.00 2 0
2:10 report: 865 0 15 18 13 13.60 0 0
2:20 report: 960 0 15 18 14 14.72 1 0
2:30 report: 1055 0 14 18 14 14.00 2 0
2:40 report: 1155 0 16 18 14 14.47 0 0
2:50 report: 1260 0 17 18 14 15.94 0 0
3:00 report: 1370 0 17 18 16 16.88 1 0
3:10 report: 1475 0 17 18 17 17.00 0 0
3:20 report: 1600 0 18 18 17 17.25 0 0
3:30 report: 1720 0 18 26 17 17.83 1 0
3:40 report: 1830 0 24 28 18 18.64 0 0
3:50 report: 1950 0 25 28 19 19.92 0 0
4:00 report: 2080 0 26 28 20 21.26 1 0
4:10 report: 2215 0 27 28 22 22.12 0 0
4:20 report: 2355 0 28 28 22 23.00 0 0
4:30 report: 2505 0 29 36 23 24.13 1 5.4375
4:40 report: 2670 0 30 36 24 25.36 0 25.4375
4:50 report: 2825 0 30 36 26 26.00 0 45.4375
5:00 report: 2995 0 31 36 26 26.40 1 65.4375
5:10 report: 3170 0 37 44 26 28.14 4 0
5:20 report: 3365 0 42 52 28 33.74 0 16
5:30 report: 3525 0 43 52 37 38.08 0 36
5:40 report: 3740 0 44 52 38 38.93 0 56
5:50 report: 3955 0 45 52 39 39.80 0 76
6:00 report: 4200 0 54 60 40 43.68 0 7.5625
Started Hatchery at 117.813
Started Spawning Pool at 149.313
Finished Spawning Pool at 214.313
Finished Hatchery at 217.75
14H14P
+ Show Spoiler +
0:00 report: 0 0 6 10 6 6.00 0 0
0:10 report: 30 0 7 10 6 6.58 2 0
0:20 report: 70 0 8 10 7 7.38 2 0
0:30 report: 120 0 9 10 7 8.19 2 0
0:40 report: 180 0 9 10 8 8.81 2 0
0:50 report: 230 0 9 10 9 9.00 2 0
1:00 report: 300 0 10 10 9 9.45 2 0
1:10 report: 365 0 12 18 10 10.05 0 0
1:20 report: 430 0 13 18 10 11.49 0 0
1:30 report: 505 0 13 18 12 12.85 1 0
1:40 report: 590 0 14 18 13 13.54 0 0
1:50 report: 680 0 14 18 14 14.00 1 0
2:00 report: 775 0 13 18 13 13.00 2 0
2:10 report: 865 0 14 18 13 13.47 1 0
2:20 report: 960 0 14 18 14 14.00 2 0
2:30 report: 1050 0 14 18 13 13.04 2 0
2:40 report: 1145 0 15 18 13 13.98 1 0
2:50 report: 1240 0 17 18 14 15.75 0 0
3:00 report: 1350 0 17 18 16 16.83 1 0
3:10 report: 1460 0 17 18 17 17.00 0 0
3:20 report: 1575 0 18 18 17 17.24 0 0
3:30 report: 1700 0 20 26 17 17.83 1 0
3:40 report: 1810 0 24 28 18 18.62 0 0
3:50 report: 1930 0 25 28 19 19.95 0 0
4:00 report: 2055 0 26 28 20 21.27 1 0
4:10 report: 2205 0 27 28 21 22.07 0 0
4:20 report: 2330 0 28 28 22 22.94 0 1
4:30 report: 2490 0 29 36 23 24.12 1 12
4:40 report: 2650 0 30 36 24 25.42 0 32
4:50 report: 2805 0 30 36 26 26.00 0 52
5:00 report: 2975 0 31 36 26 26.43 3 31
5:10 report: 3150 0 36 44 27 29.11 3 6.5625
5:20 report: 3340 0 41 52 31 33.91 0 22.5625
5:30 report: 3510 0 42 52 36 37.07 0 42.5625
5:40 report: 3730 0 43 52 37 38.35 1 62.5625
5:50 report: 3955 0 48 52 39 40.13 0 38.125
6:00 report: 4175 0 52 52 39 44.00 1 14.125
Started Hatchery at 117.813
Started Spawning Pool at 143.938
Finished Spawning Pool at 208.875
Finished Hatchery at 217.75
Conclusions:
11P vs 13P: It looks to me like 13pool is slightly better off at pretty much every point in both resources and partial drone count, as well as being ahead on coompleted drones a good percentage of the time.
14H15P vs 14H14P: I've had a suspicion that you could pool on 14 after a 14 hatch, only sacrifice a small amount of economy, and gain a larva advantage. The numbers appear to bear that idea out, but the larva advantage of 14pool seems really small, and doesn't seem to kick in until 5 minutes or so.
Adding the drone micro seems to have made the results *much* more consistent, by the way.
Next stop will be adding some more typical choices, like 14p16h, which seems a lot more common than 13p15h.
|
|
|
|
|
|
|
|
EPT
