|
On April 22 2012 23:39 Aelfric wrote:
I think it can be implemented in sc2gears or some program to create that file for every auto saved replay after the update.
This file is downloaded from the bnet depots. Someone could packet sniff their computer and get the request URL. From there it will probably be possible to automatically download the S2GS files at will based on information from the replay.
I am fairly confident saying this because that is how map downloading currently works. You can grab the exact map file game was played on from the replay file via bnet.
|
On April 22 2012 23:57 ShadesofGraylin wrote:Show nested quote +On April 22 2012 23:39 Aelfric wrote:
I think it can be implemented in sc2gears or some program to create that file for every auto saved replay after the update. This file is downloaded from the bnet depots. Someone could packet sniff their computer and get the request URL. From there it will probably be possible to automatically download the S2GS files at will based on information from the replay. I am fairly confident saying this because that is how map downloading currently works. You can grab the exact map file game was played on from the replay file via bnet.
I'm on it!
EDIT:
Found it.
GET /{hash}.s2gs HTTP/1.1
User-Agent: Battle.net Web Client
Host: EU.depot.battle.net:1119
Accept: */*
Now to figuring out the hash. Maybe it's contained in the replay somewhere?
|
*looks down on nerd army*
Yes yes, continue working.
*ascends back into the internet*
|
Nerds unite! This is just simply amazing. Not only the work itself but the collaboration! Yay Go Team Nerd!
|
On April 23 2012 00:04 Prillan wrote:Found it.
GET /{hash}.s2gs HTTP/1.1
User-Agent: Battle.net Web Client
Host: EU.depot.battle.net:1119
Accept: */*
Now to figuring out the hash. Maybe it's contained in the replay somewhere?
Awesome. Can you supply a set of replay files and their corresponding .s2gs hashes?
It'd help to have a common set of data for everyone to be working with, possibly with screenshots of the score screens as well. Ideally a mix of 1v1,2v2,4v4, FFA games to see how/if the player count affects the structure.
I haven't booted SC in weeks so this would be particuarly helpful to me...
|
On April 23 2012 00:30 ShadesofGraylin wrote:Show nested quote +On April 23 2012 00:04 Prillan wrote:Found it.
GET /{hash}.s2gs HTTP/1.1
User-Agent: Battle.net Web Client
Host: EU.depot.battle.net:1119
Accept: */*
Now to figuring out the hash. Maybe it's contained in the replay somewhere? Awesome. Can you supply a set of replay files and their corresponding .s2gs hashes? It'd help to have a common set of data for everyone to be working with, possibly with screenshots of the score screens as well. Ideally a mix of 1v1,2v2,4v4, FFA games to see how/if the player count affects the structure. I haven't booted SC in weeks so this would be particuarly helpful to me...
Sure, I need to structure my work a bit first.. It's a mess ^^
Do anyone know how the map hash is calculated?
|
On April 23 2012 00:38 Prillan wrote:
Do anyone know how the map hash is calculated?
The map hash is stored in the replay.initData file in the .SC2Replay MPQ archive. We don't calculate it, we just use it. The initData file actually stores a bunch of resource identifiers but none of them (that I know of) are this .s2gs file though so it is probably stored elsewhere.
|
On April 23 2012 00:46 ShadesofGraylin wrote:Show nested quote +On April 23 2012 00:38 Prillan wrote:
Do anyone know how the map hash is calculated? The map hash is stored in the replay.initData file in the .SC2Replay MPQ archive. We don't calculate it, we just use it. The initData file actually stores a bunch of resource identifiers but none of them (that I know of) are this .s2gs file though so it is probably stored elsewhere.
Sad news: The hash is actually the SHA-256 hash of the file itself.
I'm trying to find some kind of index file to grab from the same source.
|
The s2gs file is named after a hexdigest of it's contents. E.g.
+ Show Spoiler +>>> import hashlib
>>> s2gs = open('775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46.s2gs','rb')
>>> print hashlib.sha256(s2gs.read()).hexdigest()
'775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46'
I would guess that map files are named the same way. This means that if we can't find an explicit reference from the replay file we might be out of luck with the automatic downloading.
|
On April 23 2012 01:09 ShadesofGraylin wrote:The s2gs file is named after a hexdigest of it's contents. E.g. + Show Spoiler +>>> import hashlib
>>> s2gs = open('775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46.s2gs','rb')
>>> print hashlib.sha256(s2gs.read()).hexdigest()
'775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46'
I would guess that map files are named the same way. This means that if we can't find an explicit reference from the replay file we might be out of luck with the automatic downloading.
There is no date or timestamp that could be compared/checked? I'm novice programmer, but I figure there has to be something between the replay file and the s2gs file that would match and that would be unique that could be cross referenced. Date/Time seems like a good place to start..but again i'm novice, and as my sig says "I'm Nub."
|
This is really great work guys!
|
On April 23 2012 01:13 thurst0n wrote:Show nested quote +On April 23 2012 01:09 ShadesofGraylin wrote:The s2gs file is named after a hexdigest of it's contents. E.g. + Show Spoiler +>>> import hashlib
>>> s2gs = open('775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46.s2gs','rb')
>>> print hashlib.sha256(s2gs.read()).hexdigest()
'775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46'
I would guess that map files are named the same way. This means that if we can't find an explicit reference from the replay file we might be out of luck with the automatic downloading. There is no date or timestamp that could be compared/checked? I'm novice programmer, but I figure there has to be something between the replay file and the s2gs file that would match and that would be unique that could be cross referenced. Date/Time seems like a good place to start..but again i'm novice, and as my sig says "I'm Nub."
Yeah, that's not the problem ^^
We wanted to find a way to calculate the hash so we could download the file given a replay. Now we can only use new .s2gs files and those already in the Cache folder. The other ones are lost forever 
|
On April 23 2012 01:13 thurst0n wrote:Show nested quote +On April 23 2012 01:09 ShadesofGraylin wrote:The s2gs file is named after a hexdigest of it's contents. E.g. + Show Spoiler +>>> import hashlib
>>> s2gs = open('775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46.s2gs','rb')
>>> print hashlib.sha256(s2gs.read()).hexdigest()
'775e7f7858721decd2c20c479691fba3e00d23cc5165aea8538b5b5e1dc5ba46'
I would guess that map files are named the same way. This means that if we can't find an explicit reference from the replay file we might be out of luck with the automatic downloading. There is no date or timestamp that could be compared/checked? I'm novice programmer, but I figure there has to be something between the replay file and the s2gs file that would match and that would be unique that could be cross referenced. Date/Time seems like a good place to start..but again i'm novice, and as my sig says "I'm Nub."
The issue of correlating two files is different from the issue of using one file to get the other.
The two files might both have identical player/time/initdata information that allows us to definitively say that the two files we have are for the same replay. That information is certainly useful but requires that we have physically obtained a copy of each file.
It is a different issue to find one file given that we have the other. Since we know how to grab replay files easily, the issue becomes grabbing the s2gs file from the replay file. If files are named according to a hash of their contents (not reproducable without the file), the only way we could find one from the other is if there were embedded links. Map files for instance, have their id embedded in the replay file for retrieval at a later point.
|
Great work. I don't know enough to meaningfully contribute to the work but I can say that you are doing a great job.
Implementing the findings into programs like SC2Gears would be amazing for even more replay and multiple replay analysis.
Thanks for the good work and I hope you can figure it all out.
|
I don't think the s2gs hash is in the replay, or computable from the replay. Think about how the SC2 client works. There must be some kind of "player profile" data that the SC2 client retrieves, which must contain a list-like data structure of the Match History. Inside the Match History there must be information for the games that are shown on the Match History. Inside the data for a match must be the s2gs filename.
I expect that only wiresharking will tell us for sure.
|
On April 23 2012 02:41 dsjoerg wrote:
I don't think the s2gs hash is in the replay, or computable from the replay. Think about how the SC2 client works. There must be some kind of "player profile" data that the SC2 client retrieves, which must contain a list-like data structure of the Match History. Inside the Match History there must be information for the games that are shown on the Match History. Inside the data for a match must be the s2gs filename.
I expect that only wiresharking will tell us for sure.
I've done that and didn't find anything interesting. I haven't found any other interesting file in the Cache folder which contains anything close to a list of s2gs files
|
Searching through hexdumps of the replay datafiles confirms that there isn't an explicit reference in the replay file.
Also, I agree with your analysis @dsjoerg. Wireshark can provide us with the URLs we need to simulate moving through the client application. I'd suspect that most of the files will be in the same serialized data format. Not sure how much Blizzard would appreciate our pinging of their data servers but I suppose we can find out the hard way.
@Prillan want to track down some of the other data feeds triggered by the profile GUI?
|
Prillan, although the "player profile" may not be in the Cache file, it must be transmitted over the wire to the SC2 client... otherwise it wouldn't know s2gs filename to request.
|
I can see this ending up in the formations of Liquid's Eleven. A grand heist to get Blizzards data.
|
|
|
|
|
|
|
|
EPT
