|
![[image loading]](http://skillcraft.ca/wp-content/uploads/2011/08/skillcraft-ad-600x200.png)
The data and scientific paper for the SkillCraft project are available online. In case you don’t know, the SkillCraft project is the largest scientific study of expertise ever conducted using StarCraft 2 players. Over three thousand players from bronze leaguers to professionals contributed replays to the project.
The published scientific paper is available to download here. One of the reasons we chose the scientific journal PLoS ONE is that they are open access, and thus make the articles available without a paywall.
The dataset we used to create the paper is available here. It's also available from the UCI machine learning repository here.
The dataset would be great to use in a project for a machine learning course, or for just poking around. The dataset consists of twenty variables for each of the 3395 games. The data are anonymized to protect participants’ privacy. There is documentation with the dataset at those links, and more details in the paper itself. If you have questions about either the paper, or the data, please post comments in this thread, and we’ll try to answer them quickly.
If you use the data for a project, or find anything interesting in there, we’d love to hear about it here, (or email us at cognitive-science-lab*AT*sfu.ca)
Our next project involves looking at how individual players develop over time. If you have more than 300 replays, please go to skillcraft.ca and contribute them to the study. Also, we now have ethics approval for participants younger than 16 to contribute. If you know anyone younger (perhaps your children, even) it would be awesome to get those replays, even if there are fewer than 300. There are interesting affects of age on performance and those data would be invaluable to understanding how age influences cognitive motor abilities. We have a paper under review now on some aspects of this. We'll post it when it's reviewed.
Thanks to all those players who have contributed replay files to the project, we couldn’t have done it without you!
The SkillCraft team (skillcraft.ca, Facebook page, Twitter)
|
|
|
Always very exciting 
|
This is really phenomenal!! I am actually very surprised at how important APM is. I thought workers produced would be more important, too...
|
Im honestly really confused at what this shows. All I really understood was that people at higher ranks tend to play with higher APM which I assumed was generally true. Someone want to explain to me what all this means in laymans terms?
|
Northern Ireland26773 Posts
Sick, going to have a proper comb over this
|
Dang, I should spend some time reading this!
|
I just read the article in most of its length. I think it's quite academic but I suggest that to use primarily figure one to figure out what areas you should focus on as a player. For example, if you are a player in Platium, you should read the Plat-master column and then base your time management on the priorities explained.
|
On December 07 2013 13:38 lailaiwd wrote:
I just read the article in most of its length. I think it's quite academic but I suggest that to use primarily figure one to figure out what areas you should focus on as a player. For example, if you are a player in Platium, you should read the Plat-master column and then base your time management on the priorities explained.
I think this is a very sensible approach, because it emphasizes the "normal" way skills in SC2 seem to develop. This is not to say that you couldn't emphasize other things earlier and have good results. We'd really need to do some experimental work to see how different training methods (emphasizing hotkeys earlier, for instance) might impact the development of the player as a whole.
Anyone out there got a good training map and what to see how it affects peoples game?
|
I wasn't able to find it: what is the average and std dev of people's apm per league? does it vary considerably based on race?
edit: eps file viewer. Very nasty to find a free program. Why not just bmp files that everyone can easily open? It's also hard to see actual numbers to calculate a mean and std dev.
|
On January 18 2014 17:08 CutTheEnemy wrote:
I wasn't able to find it: what is the average and std dev of people's apm per league? does it vary considerably based on race?
edit: eps file viewer. Very nasty to find a free program. Why not just bmp files that everyone can easily open? It's also hard to see actual numbers to calculate a mean and std dev.
The authors of the study have uploaded the complete dataset, that is amazing (thanks thanks thanks!) and something you barely see in any publications. I personally think you should not blame them because you don't know how to read the data. I suggest using R, if you are interested I can provide you with the code to get started using their dataset.
I guess here is what you are looking for:
![[image loading]](http://i.imgur.com/LqWKFXf.png)
I love boxplots that's why I used them (the fat line is the median, the box includes 50% of people, from 25% quantile to 75% quantile but I also included mean and sd at the bottom).
Race was not include in the dataset, sorry.
|
Perhaps with this data we come up with an absolute metric of skill that does not rely on MMR.
|
On January 19 2014 07:15 Loccstana wrote:
Perhaps with this data we come up with an absolute metric of skill that does not rely on MMR.
You could come up with a metric of MECHANICAL skill.
If you combine APM, Action Latency and Assign to Hotkeys you can predict 54% of the variance of what league someone is in (including ANY other measure in the study [including age] you never exceed ~58%...).
I think that's fucking good - especially considering none of them has anything to do with strategy.
But - assuming two players of similar mechanical skill - there will still be variation in strategy, game knowledge/understanding and adequate (or even creative) decision making - and none of the measures used in the study can really capture those... so... yes, based on data like this you can do a damn good job of predicting what leagues someone is in but you won't be able to replace MMR.
|
I think the following graphs will help interested readers in understanding the content of the study on another level (assuming you have read it and have some kind of grasp on what the measures mean)... a level that is not dependent on a thorough understanding of the statistical methods used in the study but still allows to understand the basis for the conclusions drawn... and possibly even a quarter-decent informed opinion concerning some of the conclusions...
As seen in my previous post, there is an interesting "APM" spike for Pros.
![[image loading]](http://i.imgur.com/Wi3Rwlp.png)
This spike in APM for pros seems to be based a lot on "Select by Hotkeys" (the correlation between "Select by Hotkeys" and "APM" increases from 0.56 in Bronze to 0.88 in Pros).
![[image loading]](http://i.imgur.com/HcAwj42.png)
The pattern for Action Latency is impressively linear. Even though APM increases prediction accuracy at the highest level, Action Latency never loses its value.
![[image loading]](http://i.imgur.com/II2gnr4.png)
Worker Production kind of sucks, knowing when to stop building workers could be a relevant skill as well 
|
thanks for the diagrams
|
|
|
|
|
|
EPT
