I'm kinda reluctant to dive into this at all - the hash retrieval, not the s2gs reversing, that is.
I've checked through the traffic preceeding the s2gs GET and it appears to be typical battle.net 2.0 protocol chatter, most of which has been reversed elsewhere. It's probably possible to get into this somehow, of course, but Blizzard isn't friendly to people attempting to reverse their protocols or implementing anything that makes use of it, as we should all know.
And since every other file in the cache, including the index, look clean it's pretty safe to assume that it's somewhere in the payload after the auth - and again, I'd rather not dive into that, especially in public.
I've also taken a look through all the web frontend stuff on battle.net, since they're at least displaying the match history there, but to no avail.
imho, the best bet would be to grab Morhaime on an upcoming Blizzard event and shake him until his team prioritizes a SC2 API. They just recently confirmed a D3 API (http://us.battle.net/wow/en/forum/topic/4217171883#10) and shortly before that confessed that while they'd like to do a SC2 API, it's not happening currently.
They seem to be slacking on these fronts ..I remember when I was racing Thott, reading from WoW memory after the savedvariables got cut in one of the beta pushes. Took them more than 5+ years, scraping terrabytes off the armory, until they had an API going. And all that while they have the massively popular WoW UI community going, necessarily realizing that third party stuff is 100% win-win. Gah. The lengths through which we have to go to obtain these bits of data is excruciating sometimes.
On April 23 2012 23:01 SirPsychoMantis wrote: Can anyone upload some of the .s2gs files with the corrisponding basic info (like workers built and average unspent resources)? I'm at work with no access to sc2.
EDIT: Or just the file name / info, since it looks like we can request it from b.net servers
I did. Look at the page 2.
Not sure how I completely missed that, thanks. I'll do some poking.
Edit : I put values of 1: and 2: in Hexadecimal, its easier to get values like this
I think 0x84 is for drones, 0x86 for overlords the 0: is the supply (7,8,9,10) the 1: not sure... the ending A might be the total supply (10) the 2: I think is the time where 0x1C = 00:02 0x32 = 00:03 0x49 = 00:05 0x11C = 00:18 0x12F = 00:19 0x167 = 00:22 0x1C4 = 00:28 0x1DE = 00:30 0x276 = 00:39 0x2C4 = 00:44 0x316 = 00:49 0x378 = 00:56 0x38C = 00:57
anyone to get the general formula ?
Edit : Found ! i's in 16th of seconds ! 0x1DE = 478 and 478/16 = 29.875 so 00:30
How's it going? I've been trying to parse it but it seems to be incomplete.. Some units are missing.
Also, any idea what the 2nd structure contains? Lots of entries like this:
After carefully recording the times of various actions I took in the SC2 client, and comparing to the timestamps of files in my Cache, I don't think any of the files in the Cache contain a usable reference to an s2gs file.
Try it yourself! If any of you are trying this on OS X I'd be happy to share the command I used for retrieving the cache files in timestamp order.
In particular what you should try is pulling up a friend's profile, then their Match History, then hover your mouse over the magnifying glass, and then click on the match. What I found, and let me know if you find differently, if you do it carefully and make a note of the times of your actions, is that the s2gs is retrieved when you hover your mouse over the magnifying glass. Prior to that, nothing else you've done with the friend causes any files to be retrieved into the cache.
To me, this strongly implies that the s2gs filename must be being sent over the wire, but not stored in any cache file.
On April 24 2012 11:18 dsjoerg wrote: To me, this strongly implies that the s2gs filename must be being sent over the wire, but not stored in any cache file.
Not only that, but the HTTP requests to *.depot.battle.net are also part of the general Battle.net 2.0 protocol. Authentication and Warden utilize that CDN in all current Blizzard games. The only thing obviously referenced are the map information files that are in the index. As I said above, dangerous territory to get into. Let's find a blizzard rep, a blunt object and get an official API from him
I just tought about this : imagine if you could at the start of a ladder game get all your opponent previous build orders of his match history against your race with s2gs files, you could metagame your opponent lol. (oh your last 5 games against my race you did fast expand, then i will fast expand too).
I made a hacky prototype demonstrating how to get the hash files and automating a little of the SC2 client. It's a windows C#.NET app that takes a character name and code as input, and outputs the .s2gs file locations for their last 50 games.
Obviously I don't expect people to actually use the tool by itself, but you can take the code from it and perhaps expose a web service interface to it or integrate it with other tools etc.
The client automation is a very brittle proof of concept - it requires that SC2 be run in windowed mode and positioned in the top left of your screen. If the client lags more than it expects, it'll get confused since it doesn't bother trying to figure out what state the client is in before moving to the next step. Adding that functionality wouldn't be hard, you could just take samples of critical regions of the screen and compare average color values, for example.
On April 27 2012 19:46 Prillan wrote: Nice. But how does it trigger the download? By just looking at the match history?
Yes. Just to clarify though, the download in this case is a ~5 kilobyte payload that just has references to the .s2gs files on their CDN. To download the actual s2gs files, you'd still have to make the http requests (which is fast and trivial so I didn't include it).
I read all of this and still dont understand what is so great about it. What could you possibly get from this that you caouldnt get from a replay? You can get you spending from a replay.
On April 27 2012 19:46 Prillan wrote: Nice. But how does it trigger the download? By just looking at the match history?
Yes. Just to clarify though, the download in this case is a ~5 kilobyte payload that just has references to the .s2gs files on their CDN. To download the actual s2gs files, you'd still have to make the http requests (which is fast and trivial so I didn't include it).
Ok, looked at the source. How did you know where in the memory to look? ^^
On April 27 2012 20:20 treekiller wrote: I read all of this and still dont understand what is so great about it. What could you possibly get from this that you caouldnt get from a replay? You can get you spending from a replay.
No, actually you can't. Stats not possible to get from replays (might be a incomplete list): * Resource Collection Rate * Average Unspent Resources * Workers Created (you can get pretty close with the replay) * Army Value over time (graph) * Income over time (graph) * Units Killed * Structures Razed
Ok, looked at the source. How did you know where in the memory to look? ^^
Well I had a basic understanding of how the client caches the interface data in memory from when I was pulling ladder data from it for starcraftrankings.com - the rest was just taking hex dumps/diffing them/guessing and checking a little. This data was particularly easy since they always have an 8 byte prefix and all you have to do is take the 32 bytes immediately following it.
On April 27 2012 20:20 treekiller wrote: I read all of this and still dont understand what is so great about it. What could you possibly get from this that you caouldnt get from a replay? You can get you spending from a replay.
Nope. A replay file contains only the match information (players, maps, etc.) and user input actions. It does not contain any processed information about the game itself.
Spending, resource collection rate and the other info in score screen all require SC2 game engine to be calculated accurately. The replay data contains a lot of noise and simulating even the simplest things, such as build order from replay is pretty much impossible to do without running SC2.
Ok, looked at the source. How did you know where in the memory to look? ^^
Well I had a basic understanding of how the client caches the interface data in memory from when I was pulling ladder data from it for starcraftrankings.com - the rest was just taking hex dumps/diffing them/guessing and checking a little. This data was particularly easy since they always have an 8 byte prefix and all you have to do is take the 32 bytes immediately following it.
I made some additions to your code ^^. I think you should have a option of skipping the macro process to allow the user to navigate by himself to the match history screen and also automate the downloading process as well. (I could do it but Avengers will take up my time today ^^)
On April 27 2012 22:22 Prillan wrote: I made some additions to your code ^^. I think you should have a option of skipping the macro process to allow the user to navigate by himself to the match history screen and also automate the downloading process as well. (I could do it but Avengers will take up my time today ^^)
Thanks for the pull request, merged! Meh, the code is already separate enough to just do the memory reading if you want (just comment out the call to SC2Macros.openMatchHistory()). You should add the support for sending the http requests if you want it
On April 27 2012 22:25 dsjoerg wrote: Gibybo, that prototype is red-hot. Nice work!
On April 27 2012 22:22 Prillan wrote: I made some additions to your code ^^. I think you should have a option of skipping the macro process to allow the user to navigate by himself to the match history screen and also automate the downloading process as well. (I could do it but Avengers will take up my time today ^^)
Thanks for the pull request, merged! Meh, the code is already separate enough to just do the memory reading if you want (just comment out the call to SC2Macros.openMatchHistory()). You should add the support for sending the http requests if you want it
EPT
