WCSDemo of build 0.8.0
Latest episode ( Mission 10 on Zerus - Hard difficulty )
Youtube playlist of all the episodes
The challenge I set out for myself is finishing the Heart of the Swarm campaign with increasing difficulty settings. Further limitations are: I cannot use the mouse or keyboard during a mission, I must rely on the eyetracker and the audio recognition that I have built to control the inputs.
I also cannot develop the software to gain advantages that other players do not have ( build hotkey macros, do inputs faster than the 30hz of the keyboard, automated mouse movement, read out game information ). That means it should abide by the Blizzard End User License Agreement. I want to develop accessibility features so that I can play SC2, I have no desire whatsoever to create software that can be construed as cheating. Relevant area of the agreement:
+ Show Spoiler +
License Limitations. Blizzard may suspend or revoke your license to use the Platform, or parts, components and/or single features thereof, if you violate, or assist others in violating, the license limitations set forth below. You agree that you will not, in whole or in part or under any circumstances, do the following:
- Cheating: Create, use, offer, promote, advertise, make available and/or distribute the following or assist therein:
- cheats; i.e. methods not expressly authorized by Blizzard, influencing and/or facilitating the gameplay, including exploits of any in-game bugs, and thereby granting you and/or any other user an advantage over other players not using such methods;
- bots; i.e. any code and/or software, not expressly authorized by Blizzard, that allows the automated control of a Game, or any other feature of the Platform, e.g. the automated control of a character in a Game;
- hacks; i.e. accessing or modifying the software of the Platform in any manner not expressly authorized by Blizzard; and/or
- any code and/or software, not expressly authorized by Blizzard, that can be used in connection with the Platform and/or any component or feature thereof which changes and/or facilitates the gameplay or other functionality;
Brief history
About two years ago I started getting problems with my arms and hands. During that time I started looking for ways to mitigate the aches by reaching out for software to help me out. Obviously playing SC2 was out of the question considering the amount of keystrokes and mouse movements involved.
I started fooling around with machine learning at the same time, doing simple recognition tasks. One day I had an idea to make browsing more easy on the hands by doing audio recognition, and then binding mouse clicks and scroll actions to them to let my hands rest after a days work.
After a while, I figured I could try the same with games requiring few inputs to play, like Heroes of the Storm. I had moderate success with it and eventually managed to beat the Elite AI. I thought about using it to go on the ladder, but my interest in Heroes waned after the collapse of the pro scene.
That's when my interest in SC2 reignited, and I had the idea to see if it was even possible to make an APM heavy RTS game playable. So I set a goal for myself and figured I would document my way through the development by making youtube videos. That's where we are now.
Current program specifications ( v0.8.0 )
- Use eyetracker to move the mouse around
- Recognize and perform an action in roughly 60 miliseconds
Sound/keybindings
- Hissing to left-mouse drag ( selecting units )
- Tongue clicking to right-mouse click ( and build units / buildings )
- Whistling to attack-move ( patrol, hold position and stop bound to areas of the screen when whistling )
- Oh sound to hold the control key ( for unit selection )
- Sh sound to hold the shift key ( for unit deselection and adding to control groups )
- ü + looking at top left to Select Kerrigan
- ü + looking at the center area to Jump between bases
- ü + looking at the command card to Follow the selection
- ü + looking at the bottom left to Jump to Alert
- iy + looking somewhere in a 3 x 3 grid on the screen to press a number from 1 till 9 for control groups
- ah + looking somewhere in a 4 x 3 grid on the screen to press a letter from q till v for unit creation
- Oah sound to press Q ( first ability + creep spread )
- Ae sound to press W ( second ability )
- F sound to Burrow
- Finger snap to activate speech recognition ( for GL HF and GGs )
