Scientific American article about SC2+Science

In a paper published this year, cognitive scientist Joshua Lewis and colleagues at the University of California – San Diego analyzed what actions players took in 2000 games to see if certain capabilities stood out as hallmarks of success. Unlike previous studies, which tested participants before and after they played games to see if their behaviors changed, the approach taken by Lewis and colleagues allowed them to look for specific differences in what players are doing and perceiving.

They tracked several measures, including how many actions players took per minute and the distances between the locations where actions occurred across the map. Not surprisingly, they found that players who made the most moves tended to win. Of more interest was the second calculation. Distributing actions more widely across a map, which the authors argue reflects a player’s ability to distribute attention, also correlated highly with winning.

Now the question is whether people can learn to divide their attention more effectively. Professional Starcraft players belong to teams, with coaches and practice schedules, and they devote the majority of their time to developing their abilities. “If there is some methodology for building up multitasking skills, we might be able to figure out a way to train people to better distribute their attention,” Lewis says. “Maybe these teams have learned that implicitly.”

Both Blair and Lewis see parallels between the game and emergency management systems. In a high-stress crisis situation, the people in charge of coordinating a response may find themselves facing competing demands. Alarms might be alerting them to a fire burning in one part of town, a riot breaking out a few streets over, and the contamination of drinking water elsewhere. The mental task of keeping cool and distributing attention among equally urgent activities might closely resemble the core challenge of Starcraft 2.

On December 02 2011 10:01 FallDownMarigold wrote:
Very interesting read, thank you for the heads up about the article. The fact that you are a researcher who likes SC2 tempts me to ask you something, so I'd love to see you thoughts

I'm wondering, what do you think about the idea of removing the motor requirement for SC2? In other words, what do you think about direct control over SC2 units via neuromotor prostheses (NMP)? The reason I'm asking is because one of the important things the article highlights is the motor aspect of SC2's difficulty. It's not like chess, where motor skills aren't really that relevant. Sure, you need basic motor connectivity to play chess, but I would argue that you don't need the refined, pruned, and much more specified corticospinal motor neuronal plasticity that you do need at high level SC2.

So, on to my rationale: It has been demonstrated in mouse, macaque, and human that neuronal ensembles (activation patterns of many neurons firing in concert to some end) unique to specific, intentional motor commands exist at the primary cortex (M1) even when corresponding limbs are no longer present or innervated. Recently, researchers have developed novel strategies combining brain-computer interface technology with algorithms to recognize and record unique, intentional neuronal ensembles at M1 corresponding to unique, intentional movements. Test patients have had electrodes implanted at M1 capable of recording and analyzing cortical motor activity. Using advances in brain-machine interface, NMPs have been developed whereby patients fitted with M1 electrodes can transfer intentional thoughts directly to a machine in order to achieve movement. For example, a patient can move a cursor around a screen based on neuronal firing patterns at the patient's M1 cortical region. First researchers determine which patterns correspond to which commands. Researchers then input these instructions, and then the NMP can match neuronal patterns to "learned" commands, thereby enabling direct neuronal control over machine-assisted movement. Therefore, it would be reasonable to apply this concept of NMP-assisted control of movement to SC2. Specifically, it would be reasonable to develop an NMP that can record neuronal patterns unique to commands in SC2, such as "stutter step marines", or "move units X distance". In theory, any number of complex commands could be matched by statistical analysis to a unique neuronal firing pattern. These patterns could be read by the NMP directly off the player's brain, enabling the player to issue commands to SC2 unfettered by motor-requirements.

So do you think it would be interesting to study SC2 if APM was no issue? It might be a novel way to examine players executing strategy at the highest possible level where literally, there are no mechanical barriers. So long as the player knows what must be done, the player can think, and the task will be executed (provided the NMP is made very well)

From the perspective of the cognitive motor system, StarCraft is the most interesting thing you could do online

On December 02 2011 10:08 talismania wrote:


What makes you think APM is no longer an issue? BCIs are usually placed on the motor cortex, and are thought to primarily decode the urge to execute a command. Therefore a command still has to be executed.

APM is not dependent on finger speed - everyone can move their fingers fast enough. APM depends on the brain telling the fingers when to move, and adding a BCI does not change that.

Furthermore I've yet to see a BCI that has good enough spatial acuity for resolving individual finger movements at the speeds required. Maybe in the future.

Each player begins with a small base of one of three species—terran (humans), zerg (insectoid creatures), or protoss (photosynthetic aliens).

On December 02 2011 10:55 EnderCraft wrote:

Anyone who has been watching Day9 for a while will remember his very newb friendly "Mental Checklist" day9 daily. I truly believe he illustrates what it means to effectively use APM and play the game in that very daily. You're constantly checking for idle production facilities, checking supply, watching the mini map, assessing scouting information, choosing when to tech/upgrade, and the list goes on. What separates the pros from your average player is their ability to effectively utilize that mental checklist. Moreover, a consequent result of utilizing an effective mental checklist is higher APM than your average player.

On December 02 2011 10:01 FallDownMarigold wrote:
...

So do you think it would be interesting to study SC2 if APM was no issue? It might be a novel way to examine players executing strategy at the highest possible level where literally, there are no mechanical barriers. So long as the player knows what must be done, the player can think, and the task will be executed (provided the NMP is made very well)

On December 02 2011 09:44 W2 wrote:
The way he explained sc2 ...

On December 02 2011 10:42 BroboCop wrote:
on a side note: didn't really know anything about skillcraft... now i do. to skillcraft, if you need replays from a complexity academy member (i play at grandmaster/high master level) send me a msg and i'll send them over.

On December 02 2011 10:08 talismania wrote:


What makes you think APM is no longer an issue? BCIs are usually placed on the motor cortex, and are thought to primarily decode the urge to execute a command. Therefore a command still has to be executed.

APM is not dependent on finger speed - everyone can move their fingers fast enough. APM depends on the brain telling the fingers when to move, and adding a BCI does not change that.

Furthermore I've yet to see a BCI that has good enough spatial acuity for resolving individual finger movements at the speeds required. Maybe in the future.

On December 02 2011 13:19 Insomni7 wrote:
Is the link broken for anyone else? Or is it the site?

On December 02 2011 13:18 FallDownMarigold wrote:

Oh but an NMP does change that! Re: BrainGate

Consider the simple fact that signals conduct much slower down cortical projection neurons than they do via electrical/machine circuitry. APM would be near instantaneous. It would coincide with thought, thus the physical limit imposed by mashing keys would be gone.

Obviously there are no current algorithms designed to recognize and categorize inputs to SC2 games -- yes, it's complicated work. It's not impossible at all though, and to me it's so exciting.

On December 02 2011 13:28 juicyjames wrote:

+ Show Spoiler [Image of Article] +
When the article is back up I do recommend visiting the actual site and maybe even commenting to show Scientific American that we want more articles like this on their site.

On December 02 2011 13:31 fush wrote:


it'd be cool, but i can't see the logic behind your suggestion. yes - it removes the latency between the motor signal and the actual button press by the fingers, but this is the pathway that everyone has to go through and isn't the issue that you're trying to get at (which, if i'm understanding correctly, is the cognitive aspect of the game).

what talismania is saying is that there simply no mechanical limitation to finger speed in SC2, which i think is true. anyone can focus solely on tapping their fingers and it's likely far faster than they can play. i think it's more likely that the roadblock is higher up, somewhere between visual processing and executive control. removing the latency between the final decision to move the finger and the finger press itself will likely not provide a different result.

On December 02 2011 13:54 Stropheum wrote:
Link is broken for me, so that's pretty much all I can say about this thread.

On December 02 2011 13:18 FallDownMarigold wrote:

Oh but an NMP does change that! Re: BrainGate

Consider the simple fact that signals conduct much slower down cortical projection neurons than they do via electrical/machine circuitry. APM would be near instantaneous. It would coincide with thought, thus the physical limit imposed by mashing keys would be gone. As thought commands -- represented by unique neuronal ensembles -- are recognized by the machine, their signal is synchronously transmitted into the game as a command. There is no holdup imposed by the fact that chemical/electrical signaling down a pyramidal neuron is MUCH slower!

Obviously there are no current algorithms designed to recognize and categorize inputs to SC2 games -- yes, it's complicated work. It's not impossible at all though, and to me it's so exciting.

On December 02 2011 14:03 FallDownMarigold wrote:


I think a significant reduction in latency could be achieved via NMP based on ensemble studies in humans. I think it would affect SC2 playing ability. In other words, I think a player with a perfected NMP could achieve better results due to the fact that he would issue more commands per minute.

If you retrain the way in which you issue commands, you achieve a faster rate of issuing commands based on overcoming the need to input signals to the periphery. Rather, you input directly to a computer interface, thus there is a significant reduction in latency. It might not seem like this would be significant, but in fact it could allow a player to totally re-learn how to play SC2, faster. What if a concerted thought, along with its recognized, temporally-dynamic ensemble, achieves several actions normally requiring several motor actions? You can't argue that several motor functions vs. no motor functions is not significant. Depending on how advanced the software or analysis techniques become, you could have a lot of different things being performed simultaneously. Microing separate groups of units, near-synchronously? The fact is that you could achieve this type of near synchronous control of two groups on a screen much more perfectly with no motor requirement. You are right that you can tap your fingers furiously faster than you can issue commands or play... But issuing SC2 commands is not as simple as tapping fingers together as fast as possible. Complex, coordinated movements are required between eye, fingers, and mouse-arm. It would be significant to eliminate all those movements.

On December 02 2011 11:26 TastyMuffins wrote:


lol protoss are photosynthetic aliens

On December 02 2011 16:04 ELA wrote:


They are.. They dont have mouths dude, they dont eat, they absorb light and feed that way, aka photosynthetic

On December 02 2011 16:07 Snuggles wrote:
The link isnt working for me =(

On December 02 2011 16:04 Gamegene wrote:


don't they absorb water from their skin?

On December 02 2011 15:10 fush wrote:


my point originally was not NMP-trained vs. normal SC2 playing. rather, assuming there are significant differences between two groups of players with differing skills, the difference between these two groups under the normal conditions won't differ significantly from the difference between these two groups after they've been NMP trained. basically, i'm having difficulty seeing how removing a single component that has little to do with the actual information processing can affect the outcome when the argument is that the roadblock to perfect play is a mental rather than mechanical one.

there's absolutely no doubt that should the technology be perfected and optimized for sc2, that performance will improve as you said. near-simultaneous actions are possible whereas they aren't when you have to queue commands normally. my point is that the point of difference probably lies upstream of when the command to move is given, and if you take out this component entirely, barring any motor deficiencies one might have, the respective changes to performance in each skill group may not be that much different from other groups pre- and post- NMP training.

On December 02 2011 12:08 CrushDog5 wrote:
There is also a cognitive processing limit on incoming information. Experts are accessing different areas of the map more often than novices (based on preliminary analyses) so the trend for faster is better holds for screen movements as well. Nevertheless, you have to look and decide what to do (how many zealots to warp in to fend off a drop, for example), and that takes cognitive time. My instinct is that pros are probably already nearing the cognitive limits, at least in the most intense parts of the game. You'd still have something like APM, it would just be a bit faster.

With so many moving parts, even a top-level player can succumb to paralysis.

On December 03 2011 00:19 bLah. wrote:
By Sandra Upson
-this part blew my mind

On December 02 2011 22:22 FallDownMarigold wrote:



I see what you are saying. I also appreciate you being clear on what I am unclear about: Here's a better way to present basic idea without letting the details confuse things:

Although the motor component of SC2 -- the requirement of inputting commands into the game -- does not constitute all of the difficulty involved in playing SC2, there is no doubt that it does play an important role in performance given the fact that there is a relatively huge time delay between instantaneously issuing multiple commands vs. quickly issuing single commands.

The article reinforces this concept and makes a point of highlighting the fact that there is a fine motor requirement in SC2. Someone with awful motor capacity will never compete with someone able to finely control and issue hundreds of finger commands based on suspected, highly plastic corticospinal neuronal plasticity. I agree with you and Talismania that obviously there is higher-order, pre-frontal processing that occurs, and this processing must happen very quickly and efficiently in order to be "the best" SC2 player. I agree that in order to REALLY be the absolute best at the game, one would have to perfect the processing that occurs on the higher-order level of thought. However, that is only relevant to the "strategy" component of RTS. The issue I'm addressing is relevant to the "real-time" component of SC2. In other words, NMP technology will not improve a chess player, given that chess has no "real-time" component like SC2 -- there is no "motor requirement" in chess. In SC2, however, the "real-time" component is tackled by endowing players with the ability to immediately enact commands at much higher rates than normal rate. Consequently, their play is only fettered by the higher order cognition to which you refer.

So in a nutshell: If this tech is pursued the way I see it, you'd have a "real-time" strategy game game whereby the "real-time" component is actually closer to real-time in that every command you desire is realized immediately, rather than lagging by the process of you issuing single, rapid commands towards achieving a larger action or immediate goal. With regard to your second emboldened point, I argue that due to this lightening of burden, there is an increase in focus that can be given to the higher-order, strategical planning center. As a result, players can execute better, more fluid play. I agree that these changes would be consistent with each group, but my goal is not to pit NMP-assisted players vs. normal players. The goal is to analyze NMP-assisted players in comparison to themselves. One could analyze 1,000 games from Mvp using an NMP and see if his overall gameplay is significantly effected. I would be extremely surprised if it wasn't altered vastly due to all the reasons above, thus I'm really interested in this idea.


Thanks for the reply.

Cortical plasticity has been demonstrated in humans and other animals in response to altered behavior. If you had a drop off in one requirement of the game -- the motor requirement -- do you agree that it would be possible for one to become better at the other aspects of the game (intake and processing ability)? The cortical topography corresponding to higher-order processing could be expanded in response to a loss of the specificity required to implement such motor functions playing SC2. Therefore I think an NMP's effect on playing SC2 transcends mere mechanical advantage. Instead, I think the mechanical advantage conferred by an NMP will then further enable a player to drastically improve on the "cognitive", abstract processing involved in make choices in the game, given that the player is free to devote all playing capacity to that one area. I do believe this area could be pushed to a greater limit with less neuronal circuitry devoted to the motor component of the game.

On December 02 2011 18:07 Derity wrote:
The article doesn't tell anything new. Brain activity should look like the one of a musician. Pilots should have a pretty good 3d imagination as well and also have a good crisis management.
I don't expect anything new from these studies about multitasking, only that games can be useful to train multitasking at a specific environment. So playing starcraft doesn't give you any better multitasking as team sports or sth else.

On December 02 2011 09:44 Zzoram wrote:
Basically they found APM and multitasking were the most strongly associated with winning. This suggests that mechanics are still the most important thing for a player in SC2.

On December 09 2011 03:59 Snuggles wrote:
Jenny Bates comments on that article are such BS =_=. At the highest levels of play there are very very legitimate examples of multitasking. Not mindless spamming, god we finished that argument a while ago.