s2gs file -- a fun little mystery for us - Page 4

Prillan

Sweden350 Posts

April 22 2012 18:39 GMT

#61

On April 23 2012 03:02 dsjoerg wrote:
Prillan, although the "player profile" may not be in the Cache file, it must be transmitted over the wire to the SC2 client... otherwise it wouldn't know s2gs filename to request.

I know. The problem is that almost all communication is encrypted. Only file downloads are not and from that I can't find anything useful, unfortunately.

As for GUI -> File download triggers. Nothing interesting.
+ Show Spoiler +


.s2ml - Map localization
.s2mi - Map info
.s2mv - Map image
.s2mh - Map ???
.unlk - JSON - ???
.s2ma - Map file
.ldpl - JSON - ???
.pfty - A lot of utf-8 encoded swear words (language filter?)
.s2ql - XML - Some kind of localization file
.s2qh - JSON - ???
.wafl - (waffles??) - Image file
.ldpt - JSON - Something about textures
.s2gs - Score screen stats
.auth - ???
.achv - JSON - Achivements
.xml - Depends on the file 
.txt - [Breaking News/PTR Available]
.dll - Scan.dll (No idea what it does)

and with that I mean "No interesting file downloads when I look around"

I found one interesting .xml file: Division Names

Mixopteryx

Norway4 Posts

April 22 2012 18:55 GMT

#62

When I open a players match history a bunch of files are created in my cache folder. Most of them have the extension "s2ml", and there is also some with extensions "s2mh" and "sc2mi". This type of files are also created when I click "show more entries" at the bottom of the Match history. They could maybe contain information about which s2gs files that are needed.

I do not understand the methods for decoding the files, so I collected the files in the linked zip-file and hope it can be useful to someone else.

The files created by clicking at "show more entries" and some related s2gs files(created when I clicked on matches that appeared after showing more entries):
http://folk.ntnu.no/toringb/moreentries.zip

EDIT: Great work guys! Im excited to see if this will be figured out!

ShadesofGraylin

United States32 Posts

April 22 2012 18:56 GMT

#63

What specifically is transmitted when you click "Match History"?

EDIT: you are awesome @Mixopteryx.

Prillan

Sweden350 Posts

April 22 2012 19:16 GMT

#64

On April 23 2012 03:56 ShadesofGraylin wrote:
What specifically is transmitted when you click "Match History"?

EDIT: you are awesome @Mixopteryx.

C Array dump and packet summary from wireshark: data.c, packets.txt

peer_0_X = To Battle.Net #X
peer_1_Y = From Battle.Net #Y

Mixopteryx

Norway4 Posts

April 22 2012 19:29 GMT

#65

I think I have something better! When I sort the match history by type(1v1 etc) only one file changes! It has the file extension "s2mi".

I have gathered the files changed in all sorting versions and some related s2gs files. The files in the folder 1v1 is collected when sorting 1v1.
http://folk.ntnu.no/toringb/Sorting.zip

My method is this:
Go to profile.
Wait a minute so the clock changes.
Click match history
Search /Users/Shared/Blizzard for files with . in the filename and are changed within the last minute. (Playing on mac)

Prillan

Sweden350 Posts

April 22 2012 19:42 GMT

#66

On April 23 2012 04:29 Mixopteryx wrote:
I think I have something better! When I sort the match history by type(1v1 etc) only one file changes! It has the file extension "s2mi".

I have gathered the files changed in all sorting versions and some related s2gs files. The files in the folder 1v1 is collected when sorting 1v1.
http://folk.ntnu.no/toringb/Sorting.zip

My method is this:
Go to profile.
Wait a minute so the clock changes.
Click match history
Search /Users/Shared/Blizzard for files with . in the filename and are changed within the last minute. (Playing on mac)

I still believe that .s2mi is StarCraft 2 Map Information.
This is what I could extract from your

1v1\cee7efb71099820c79571bdef6261f69d982630e6f92e79844e5fa58733f67b3.s2mi

{ 0: { 0: { 0: 66893, 1: 65538},
       1: 's2mh\x00\x00EU\xa0\xbd\x1dT\x10\\\xca\nL\xb8\x91\xc8U\x06h\xc6\xeaL\xa3E_\x82\xed}\xc7\x9e\x04Ji\x1a\xbe\xeb',
       2: 1316575949L,
       3: 0,
       4: 0,
       5: 0,
       6: 2724134,
       7: "Tal'darim Altar LE",
       8: { 0: 3405691582L, 1: 15189410102982475776L},
       9: 0,
       10: 0,
       11: { 0: 2, 1: '\x00\x00S2', 2: 1, 3: 'blizzmaps#1'},
       12: 0,
       13: 'enUS',
       14: { 0: 2, 1: '\x00\x00S2', 2: 1, 3: 165251},
       15: 0,
       16: 0,
       17: ['BLIZ']},
  1: 0}

ShadesofGraylin

United States32 Posts

April 22 2012 20:14 GMT

#67

Just as an update. The s2mi (s2 match information) files point to a s2mh (s2 match history) file with the struct[1] value.

The s2mh is a rather large file you can request with the normal http://<gateway>.depot.battle.net:1119/<hash>.s2mh.

I can't currently parse this file, there some subtle bug with the parser I'm having trouble with.

dsjoerg

United States384 Posts

April 22 2012 20:23 GMT

#68

I'm buying an extra box of tissues just to be ready.

ShadesofGraylin

United States32 Posts

April 22 2012 20:28 GMT

#69

On April 23 2012 05:23 dsjoerg wrote:
I'm buying an extra box of tissues just to be ready.

What are we getting ready for exactly David?

netherh

United Kingdom333 Posts

April 22 2012 20:28 GMT

#70

Here's what an s2mh file looks like for me:

+ Show Spoiler +


[{0: {0: {0: 22686, 1: 65537},
      1: 'Random_Unit_Wars_1-3.SC2Map',
      2: 's2ma\x00\x00EU\xbbk\x89n7L.I\xc1 (\xa4\xa2\xdfsV\xbd\x1cic<0\x1cw\xbdmC
\xae\x8e\xd1\x1d\xf6',
      3: 34875,
      4: {0: {0: 0, 1: 0, 2: 1},
          1: {0: 0, 1: 0, 2: 2},
          2: 0,
          3: {0: 0, 1: 0, 2: 0, 3: 512, 4: 468},
          4: 5,
          5: 22,
          6: [{0: {0: 999, 1: 500},
               1: [{0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0}]},
              {0: {0: 999, 1: 3007},
               1: [{0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0},
                   {0: 1, 1: 0}]},
              {0: {0: 999, 1: 3001},
               1: [{0: 1, 1: 1},
                   {0: 1, 1: 1},
                   {0: 1, 1: 1},
                   {0: 1, 1: 1},
                   {0: 1, 1: 1},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0}]},
              {0: {0: 999, 1: 3002},
               1: [{0: 0, 1: 0},
                   {0: 1, 1: 0},
                   {0: 2, 1: 0},
                   {0: 3, 1: 0},
                   {0: 4, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0},
                   {0: 0, 1: 0}]}],
          7: ['s2mv\x00\x00EU\xd8\x165\x82\x98N\xd0\xd48\x84Y\xfcv\xde\xad\xe6W\x
cb\xe9\xe6\x05\xfe`a\x15\x7fS\xcd457\x96'],
          8: [{0: 'enGB',
               1: ['s2ml\x00\x00EUd\xd1#<\xff\xd9\x02\x0c\xef\xda\x833\xc8\x89\x9
eFq|\x9a\x96HA\xd1\xc6\xd7\xee2\xf8MQ\xb8N']}]},
      5: [],
      6: [{0: 7, 1: 0}, {0: 21, 1: 0}, {0: 19, 1: 0}],
      7: [],
      8: [{0: 'enGB',
           1: ['s2ml\x00\x00EUd\xd1#<\xff\xd9\x02\x0c\xef\xda\x833\xc8\x89\x9eFq|
\x9a\x96HA\xd1\xc6\xd7\xee2\xf8MQ\xb8N']}],
      9: {0: 153, 1: 168},
      10: {0: 0, 1: 0, 2: 3},
      11: 0,
      12: 0,
      13: [{0: {0: 1, 1: 1},
            1: {0: 0, 1: 0, 2: 6},
            2: {0: 0, 1: 0, 2: 8},
            3: {0: 0, 1: 0, 2: 7},
            4: {0: 0, 1: 0, 2: 9},
            5: {0: {0: 0, 1: 0, 2: 4}, 1: {0: 0, 1: 0, 2: 5}, 2: 0},
            6: [{0: {0: 999, 1: 3010}, 1: {0: 1, 1: 0}},
                {0: {0: 999, 1: 3000}, 1: {0: 4, 1: 0}},
                {0: {0: 999, 1: 1001}, 1: {0: 0, 1: 0}},
                {0: {0: 999, 1: 2000}, 1: {0: 4, 1: 0}},
                {0: {0: 999, 1: 3007},
                 1: [{0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0}]},
                {0: {0: 999, 1: 2018},
                 1: [{0: 0, 1: 0},
                     {0: 5, 1: 0},
                     {0: 10, 1: 0},
                     {0: 1, 1: 0},
                     {0: 6, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0}]},
                {0: {0: 999, 1: 500},
                 1: [{0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 1, 1: 0},
                     {0: 0, 1: 0},
                     {0: 0, 1: 0}]}],
            7: [{0: {0: 999, 1: 1001}, 1: ' \x00\x00'},
                {0: {0: 999, 1: 2000}, 1: ' \x00\x00'}],
            8: 3}]},
  1: {0: 0}}]

I'm sure it's not 100% correct though (and some bits may be missing). I just added this in utils.py to get the output (not sure it's even reading the right amount of chars or doing anything like the correct thing).

elif datatype == 0x01:
#0x01 is an unknown data type...
return self.read_chars(3)

Still. You can see that it points to the s2ml, s2mv and s2ma files.

netherh

United Kingdom333 Posts

April 22 2012 22:08 GMT

#71

Here's a list of all the files created for various actions. I'm not sure it really tells us anything we don't already know.


Click on profile:

\bd\36\bd36dbb24cd0b3bb889865d62a53176a678f54a7acfb766d71052db383f94a6a.s2mi
\6e\e1\6ee188f2fd7cc5d30ffdd1c554c7a6b3ac4790462aaa0306e1c6a3f0a2132c7c.s2mi
\ab\9a\ab9afe2eee122b5ef7dfa730a55620fc7a69829854b92013b2a1a86abc55e901.s2mi
\a1\d4\a1d497696b8fca14134b6e84b14c42416d0dd2418ddfc9bb9e4c7e204f328991.s2mi
\b1\e9\b1e940a6841e8e780a0c081638175e08a1407f5201e1df4272b285dea7c773be.s2mi
\00\75\00750f9cda5ed2966ad2aaf927bc57845833ad53b36a64c125b8ae09ae6bcdbf.s2mi
\15\02\1502d6d0f7f49856a8e165e6d81a8e4111e7d960507e6c9cd49b4f78eacd9ee8.s2mi
\2e\47\2e4713d3e512d4ea059c8b871b8dfe6f91a3faedef4f52acd465b66a95dff4af.s2mi
\c0\22\c022fd2c2174a31f153132366d540cf0e81c715f048992bf3b93fa2e925930ce.s2mi

\1f\96\1f96e46830b69f767787b5201332f57a3fa4f6c52c3168d8f2added232c5501d.s2qh
\12\ec\12ecf9bd03a897bac2dd4bd7eb41ee243c17ce67b32e4611781b60ce37557481.s2qh

\b2\c9\b2c9caa9c3eeefa3c0f203cf10ce2e1fa8e32bea682bfb59ad1e7dbd252548d2.s2ql
\ee\44\ee44ec744ebb90c638494b78b331c233eaea5d2d133d3c9672e2e05a81cecec3.s2ql
\df\08\df081da0b52f88f92ac10cecb48c74b274f830e51bcb6f04cedb374060cf35b2.s2ql
\8a\bc\8abc2724c7e245031b83f3615ae1bdf9f9796c56bda49b2a327c1fe420826b75.s2ql
\3c\b6\3cb6ead29e58b7905fb0ad34a4a2cbe7ed357d8b08d20e9809c7517d4b16aee3.s2ql
\76\b5\76b5865163516632d2f7d9a7729265a1023fbdef3afb031e8957b4c84183da4d.s2ql


Click on match history:

\1f\cc\1fcc6dd0f9d05eec6b184e73bcd647075f6dfe2cf6a1b07fc3ded23f927f55d2.s2mh
\7f\8b\7f8b94b1910dd7024d7c8024dbe2aef523edb6d613846480f770eef3f473b2b3.s2ml


Mouse over a magnifying glass, or click on a match:

\56\87\5687ebfa6845a9fffa3aba01843e55210ad6ce3725859d3356404c888cc63ca2.s2gs


Corresponding s2gs request:

GET /5687ebfa6845a9fffa3aba01843e55210ad6ce3725859d3356404c888cc63ca2.s2gs HTTP/1.1
User-Agent: Battle.net Web Client
Host: EU.depot.battle.net:1119
Accept: */*

Used this code to monitor the Cache folder (adapted from here: http://msdn.microsoft.com/en-us/library/System.IO.FileSystemWatcher.aspx ). You also need to set the working directory to the Cache folder in the project properties.

+ Show Spoiler +


using System;
using System.IO;
using System.Security.Permissions;

namespace watcher
{
	public class Watcher
	{

		public static void Main()
		{
			Run();
		}

		[PermissionSet(SecurityAction.Demand, Name = "FullTrust")]
		public static void Run()
		{
			// Create a new FileSystemWatcher and set its properties.
			FileSystemWatcher watcher = new FileSystemWatcher
			{
				Path = ".",
				NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.FileName | NotifyFilters.DirectoryName,
				Filter = "*.*",
				IncludeSubdirectories = true,
			};

			watcher.Changed += new FileSystemEventHandler(OnChanged);
			watcher.Created += new FileSystemEventHandler(OnChanged);
			watcher.Deleted += new FileSystemEventHandler(OnChanged);
			watcher.Renamed += new RenamedEventHandler(OnRenamed);

			watcher.EnableRaisingEvents = true;

			Console.WriteLine("Press \'q\' to quit the sample.");
			while (Console.Read() != 'q') ;
		}

		private static void OnChanged(object source, FileSystemEventArgs e)
		{
			// Specify what is done when a file is changed, created, or deleted.
			Console.WriteLine("File: " + e.ChangeType + " " + e.FullPath);
		}

		private static void OnRenamed(object source, RenamedEventArgs e)
		{
			// Specify what is done when a file is renamed.
			Console.WriteLine("File: {0} renamed to {1}", e.OldFullPath, e.FullPath);
		}
	}
}

Prillan

Sweden350 Posts

April 22 2012 22:17 GMT

#72

Yeah, that's what I did.

I guess we should work on parsing the .s2gs file completely now ^^

NagAfightinG

United Kingdom270 Posts

April 22 2012 22:44 GMT

#73

I have no idea what you guys are talking about, but you should keep doing it :D

Linwelin

Ireland7554 Posts

April 22 2012 22:51 GMT

#74

I really don't know what you're doing there, but it's awesome

DeCoup

Australia1933 Posts

April 23 2012 01:29 GMT

#75

You guys are amazing. I am only a laymen but im having a great time tracking your progress here. I have a good feeling that what you achieve here will become very useful on future data mining too.

windsupernova

Mexico5280 Posts

April 23 2012 02:41 GMT

#76

As long as you don't use it to cheat or pirate the game Blizzard doesn't care too much about this kind of stuff.

dsjoerg

United States384 Posts

April 23 2012 10:32 GMT

#77

Prillan mentioned that "The problem is that almost all communication is encrypted." One thing that is probably not encrypted are the communications between the SC2 client and its Scaleform/Flash GFX UI. If anyone knows how to get in there, that seems like natural point from which to get a commanding view of the terrain.

I've seen the flash code inside one of the MPQs. gfx files IIRC.

Prillan

Sweden350 Posts

April 23 2012 11:07 GMT

#78

On April 23 2012 19:32 dsjoerg wrote:
Prillan mentioned that "The problem is that almost all communication is encrypted." One thing that is probably not encrypted are the communications between the SC2 client and its Scaleform/Flash GFX UI. If anyone knows how to get in there, that seems like natural point from which to get a commanding view of the terrain.

I've seen the flash code inside one of the MPQs. gfx files IIRC.

I looked into it. The .gfx files is only templates that the client fills. We probably need the client code :/

SirPsychoMantis

United States180 Posts

April 23 2012 14:01 GMT

#79

Can anyone upload some of the .s2gs files with the corrisponding basic info (like workers built and average unspent resources)? I'm at work with no access to sc2.

EDIT: Or just the file name / info, since it looks like we can request it from b.net servers

Prillan

Sweden350 Posts

April 23 2012 14:31 GMT

#80

On April 23 2012 23:01 SirPsychoMantis wrote:
Can anyone upload some of the .s2gs files with the corrisponding basic info (like workers built and average unspent resources)? I'm at work with no access to sc2.

EDIT: Or just the file name / info, since it looks like we can request it from b.net servers

I did. Look at the page 2.

Prev 1 2 3 4 5 6 7 Next All

Please or register to reply.

s2gs file -- a fun little mystery for us - Page 4

Completed

Ongoing

Upcoming