|
Hey folks,
I think it is time to announce my new project, WARP. I have been working on it for some time now. It aims to be a replay manager (and parser) for RTS games, but it currently "only" supports SC2Replay files.
Features: - Displays map preview of the played map - Displays players, including their color and race - Displays observers - Displays chat log - Displays game duration - Displays replay version - Rename functionality for replay files - Filtering replays by player name, race or file name - Sorting replays for quick access - Hiding chat messages of certain players
Although these are pretty basic functions, WARP is still in development and will provide an overview of the players' actions soon, like in BWChart.
Additionally, WARP is cross-platform, as it is entirely written in java.
Get it at http://trac.erichseifert.de/warp. The current version (0.4) was tested with just under 500 replays and yielded good results.
Feel free to register and create tickets, if you would like to file a bug report or suggest changes/improvements.
Greetings, Hailstorm
|
Except that confusing open file/folder dialog, works nicely. GJ
|
holy crap, sweet!
thanks dude, I was hoping for something like this to come up :D
|
I click open and go to the directory containing my SC2 replays, but it appears empty and there is nothing to open. I have the folder open and there most certainly is replays in it.
|
On March 26 2010 01:32 Grobyc wrote: I click open and go to the directory containing my SC2 replays, but it appears empty and there is nothing to open. I have the folder open and there most certainly is replays in it. The dialog where you choose your files only accepts directories. Just select your replay directory and click "open". This should also work, if you have already jumped into the folder.
Now that's a usability problem I have got to get rid of ^^
Edit: WARP will dig into the selected folder and include all replays, even if they are contained in subfolders.
|
Can't get it working... I assume because this is fairly recent install of Windows (2 weeks) and I probably don't have java running yet.
(Bump... interested in seeing what others think of this.)
|
awesome program keep working on it man need a new bwchart :D
does anyone know how many replays blizzards Recent folder supports before it starts erasing? maybe someone could do a program that copies all Recent games into the Autoreplay folder so we never have to worry about our replays getting removed?
|
This looks pretty awesome!
|
United States4126 Posts
I wish you luck in trying to parse players' actions. It seems like a difficult task
I have one question. Since replay files contain people's first person views, would it be possible to make a number for the total distance someone's screen traveled? I think that would be a pretty sweet statistic to measure.
|
|
On March 26 2010 03:27 MorroW wrote: awesome program keep working on it man need a new bwchart :D
does anyone know how many replays blizzards Recent folder supports before it starts erasing? maybe someone could do a program that copies all Recent games into the Autoreplay folder so we never have to worry about our replays getting removed?
wait what? They start erasing? =.=
|
On March 26 2010 04:59 Antimage wrote:Show nested quote +On March 26 2010 03:27 MorroW wrote: awesome program keep working on it man need a new bwchart :D
does anyone know how many replays blizzards Recent folder supports before it starts erasing? maybe someone could do a program that copies all Recent games into the Autoreplay folder so we never have to worry about our replays getting removed? wait what? They start erasing? =.=
Doubt that, ve 937 games played (recent replays) and it just gets stored there till u delete it yourself.
|
sounds bad ass dude but you forgot to hide replay progress bar
|
United States4126 Posts
Why would this hide the replay progress bar? It has nothing to do with watching replays within SC2.
|
o i see...should read more carefully next time T_T
|
so how do i rename files ? do i have to do it manually or there is some way to do it automatically (with some set manner ie. player 1-player 2-map or something like this) ?
btw. once u select file and try to change its name it seems like u cant accept it. enter or trying to pick other file dont work u have to resize window ; /
|
Is there any way to be able to tell who won? Is there like a leave action like there was in bw?
|
Sweet tool, looking forward to seeing this grow. I hope it will expand to take into account a lot the stuff that BWchart and Wc3chart did. I'm also really interested in APM (don't hate) to see my own progression over time.
|
I have one question. Since replay files contain people's first person views, would it be possible to make a number for the total distance someone's screen traveled? I think that would be a pretty sweet statistic to measure. You are right, that would be possible. The replay contains a file named something like "smartcam.events". I assume that camera movements are stored there. Although it is an interesting request, I am likely to put emphasis on some other features, though
o how do i rename files ? do i have to do it manually or there is some way to do it automatically (with some set manner ie. player 1-player 2-map or something like this) ? I am afraid you have to do it manually for now, but renaming a bunch of replays by a kind of template would be really nice, indeed.
btw. once u select file and try to change its name it seems like u cant accept it. enter or trying to pick other file dont work u have to resize window ; / Thanks, I will have a look at this issue.
By the way: If there are any problems concerning WARP, feel free to sign up at the project page and file a bug report.
Is there any way to be able to tell who won? Is there like a leave action like there was in bw? I guess there is some kind of leave action. I did not look much into the actions, though, but parsing the actions will be one of the next steps. Additionally, it is rather difficult for me to get hands on proper test replays, as I do not have a beta-key :/ plus the guy, who provided some test replays does not respond anymore...
|
What would be great is the ability to have a full-fledged database of replays. And you could be able to tag files in the database with your own personal information. Things like, "This is a good example of my macro," or "Lost due to personal stupidity." and so forth. It would also allow you to do searches based on this information, so that you can categorize replays as you see fit.
You should look into having this program set up an SQLite database for doing this kind of thing.
|
Now we just need you to make it to actually watch the replays without bnet 2.0 and everyone will love you :D
|
Are you planning to open source any of it? Even just your parsing logic?
Nevermind I'm retarded.
|
Running warp.bat on windows 7 64-bit (latest java version) has the following error
'javaw' is not recognized as an internal or external command, operable program or batch file. However running WARP-0.4.jar directly works. Edit: And btw exellent work. I can't wait to see this project evolve.
|
Nice work
I guess you're using libmpq to parse the file first ?
I would like just for fun to make a library for php but i think i need to start with libmpq ..
|
I'm gonna port this to .NET for my new team website I'm working on. Thank you so much for publishing this.
|
|
I guess you're using libmpq to parse the file first ? No, I am using my own java based parser, MPQParser, for the extraction of the files. Although the class files are included in the WARP jar, the mpq parser is not really ready for release yet, as there is only very limited support for all the unsigned data types, which do not exist in java.
I would like just for fun to make a library for php but i think i need to start with libmpq .. There is already someone, who is working on a parser for PHP. Have a look at http://phpmpq.codeplex.com/.
|
This program ran fine the first time. But now when i try to run it nothing happens at all. No extra process and no application.
|
Would be complete enough for me if you could get the map name (but I've looked at the file and I understand why this is difficult). Nibbits seem to be able to do it fine.
|
On March 26 2010 14:48 DeCoup wrote: This program ran fine the first time. But now when i try to run it nothing happens at all. No extra process and no application. Could you please start the console and start WARP with the command:
java -jar path/to/WARP.jar and submit the output? Do this either by filing a ticket at the project page or by submitting the console output via PM.
On March 26 2010 15:17 prOxi.swAMi wrote: Would be complete enough for me if you could get the map name (but I've looked at the file and I understand why this is difficult). Nibbits seem to be able to do it fine. Daniel Cramer from Nibbits also states that my code is completely wrong. He uses bit parsing instead of byte parsing. I don't see the point, why Blizzard should create bit based formats... Both of our attempts seem to work though.
|
Can you give me some hints to learn how to parse replay files (for any game, any langage, just .. the way you do it) ? thx !
|
Nice to see this is already going on!
Great work! Looking forward to hearing updates =)
|
Shell script doesn't load the terminal on OS X (tries to launch XCode, prolly just a file/app association issue), but loading the jar on it's own works just fine anyway.
|
Can you hide the replay bar? I am so sad Blizzard hasn't done anything about that yet. =/
|
Bad news, community!
You probably already know it: As of patch #9 (April 26th 2010), which updated starcraft 2 to version 0.11, the replay format seems to have changed. I do not know how far-reaching these changes are, because I have not had a closer look at them, yet.
Currently, I am relying on another project which aims at parsing SC2Replay files in C++. They work in a team, not as a single person like I do, and achieved several successes. For now, the release of the much anticipated version 0.5 of WARP, which should support graphs of the players' APM will most likely be delayed for some time...
I will keep you up to date, though.
Sorry! Hailstorm
|
Are you dead yet? No, WARP is alive and the development is flourishing!
I am proud to announce WARP v0.6.
Features: - Support of replays post patch 9 - APM Chart for each player - Revamped user interface - Database for storing parsed replays, which also loads on startup. - Status reports of WARP
How to improve WARP? I once more appeal to you: Do not hesitate to file tickets on http://trac.erichseifert.de/warp. It doesn't matter, whether you find a bug, or you just want to propose an enhancement or a new feature. You are always welcome.
Contribution Some of the people I asked said that they are too unexperienced with Java and do not want to contribute to WARP. Don't underestimate your skills! Any code is good enough. I am still looking for helping hands. Just let me know, if you are interested in contribution.
|
Well, i can't run this, i downloaded the required Java application and when i run the .bat i get my cmd pop up for a split second and then nothing. The .jar will open up a blank Java program. No console options or anything.
|
On June 15 2010 20:34 Mecha_cl wrote: Well, i can't run this, i downloaded the required Java application and when i run the .bat i get my cmd pop up for a split second and then nothing. The .jar will open up a blank Java program. No console options or anything.
- Try opening warp.bat with a text editor. - Replace "javaw" with "java"
3 minutes ago, I had some other guy having the problem that the javaw command is not known to windows. This will possibliy solve the issue.
I do find it strange, though, that you receive an empty window when starting the .jar file directly. Please excuse the question, but do you have the lastest java verison installed (also available from the project page)? I know this sounds like a typical response from any tech support, but sometimes it is the easy solutions that solve problems best :D
|
On June 15 2010 20:50 Hailst0rm wrote:Show nested quote +On June 15 2010 20:34 Mecha_cl wrote: Well, i can't run this, i downloaded the required Java application and when i run the .bat i get my cmd pop up for a split second and then nothing. The .jar will open up a blank Java program. No console options or anything.
- Try opening warp.bat with a text editor. - Replace "javaw" with "java" 3 minutes ago, I had some other guy having the problem that the javaw command is not known to windows. This will possibliy solve the issue. I do find it strange, though, that you receive an empty window when starting the .jar file directly. Please excuse the question, but do you have the lastest java verison installed (also available from the project page)? I know this sounds like a typical response from any tech support, but sometimes it is the easy solutions that solve problems best :D For some reason, around 50 replays work that were taken BEFORE the 6/01/2010... so its starcraft replays being awful!
|
On June 15 2010 22:31 Mecha_cl wrote: For some reason, around 50 replays work that were taken BEFORE the 6/01/2010... so its starcraft replays being awful!
I have spotted the mistake: The replay parser contains a list of replay versions it is able to parse. As the version of the newest patch was not included, the parser skipped these replays.
Replays up to the third of june (excluding) should work, though. Do they?
|
Hi community,
once more, I would like to announce a version bump to WARP v0.7.
Check this feature list for information about WARP's capabilities. Unfortunately, new replays do not work, yet, but are planned for the next release...
As some Windows users experienced problems when trying to start WARP, I included an executable file, which does work now.
By the way, I am still looking for people that submit bug reports Contributors are also welcome of course!
Greetings, Hailstorm
|
So I run WARP, and open my replay folder. It parses however many replays are in there, and then I see this:
What's going on?
|
On August 05 2010 09:49 Biochemist wrote: What's going on?
Cannot say that on first sight... Which version are the replays you are trying to parse? Unfortunately, replays since the second beta phase up to now are not supported, yet. Sorry!
|
Ah, that would explain it. They're all retail.
|
I'd post this as new thread, but apparently someone had a great idea of not allowing users to make threads. So here goes:
After tinkering for a bit with replays, I found a proper way to parse replay.details file inside replay MPQ. I'm posting it here because there is no information about this part of replays on the internet (apart from obviously incomplete info like http://code.google.com/p/starcraft2replay/wiki/ReplayDetails ): replay sites that are working are hoarding what they know to themselves (to kill off competition?). This file has data in it, that is stored in a data serialization format very similar to JSON, apart from the fact that JSON is text format while replay.details is binary. User data of MPQ archive (the MPQ\x1b part) is also using this format, while in beta it was just plain binary. Basically, whole file can be translated into array of values that are either arrays or scalars (strings or numbers). Any modern language can work natively with these values so working on replays becomes a simple and enjoyable task.
Before I start explaining format, I'll first describe how integers are stored in replays. Blizzard chose a variable-length format (different from the one used in in other parts of replays by the way, why?) that is in a way similar to utf8: a number consists of a number of bytes; a byte that has 0 in most significant bit is the last one. So, to read a number, you read bytes until you encounter the one less than 0x80, and put their 7 least significant bytes into your variable, first byte at offset 0, second - at offset 7, third - at 14, etc. Least significant bit of result is sign (two's complements were a bit too difficult for talented sc2 team?). Shift result right by one byte and multiply by -1 if least significant bit was set. This is the number stored in replay. I am very sure it is the correct way to handle it because I got numbers like 144000000000 (decimal) from some replays, as well as 1337 (each player had this number attached to him in beta). I'll refer to this format as VLF (variable length format) later on. Here are few examples:
bytes values (dec) ===== ============ 00 => 0 01 => 0 02 => 1 03 => -1 04 => 2 7e => 63 7f => -63 80 => INVALID (last byte can't have most significant bit set) 80 00 => 0 (but you'll never find it in replays since 0 can be represented by just 00) 80 01 => 64 81 01 => -64 82 01 => 65 80 02 => 128 81 02 => -128 80 80 80 10 => 16777216
Now, replay.details format itself is pretty simple. It consists of one data type marker followed by data itself. Data type marker is a single byte and can have following values:
02: Binary data. 02 is followed by size of binary data in VLF after that comes data itself; in most cases it's text in utf8. 02 12 41 55 54 4F 4D 41 54 49 43 is a string AUTOMATIC.
04: An array. 04 is alway followed by 01 00, after that there is number of elements in array in VLF, and elements follow. Each element is a data type marker followed by data, all in same format.
05: An array. Followed by number in VLF which indicates how many pairs will follow -- a pair consists of array element index in VLF and element itself (data type marker followed by data).
06: A number. Its value follows in one byte.
07: A number. Its value follows in four bytes.
09: A number. Its value follows in VLF.
Here are examples:
binary data (right align) equivalent in JSON ========================= ================== 02 04 68 69 => "hi" 05 02 00 02 04 68 69 => ["hi"] 05 04 00 02 04 68 69 02 02 04 68 69 => ["hi","hi"] 05 06 00 09 02 02 09 04 08 09 06 => [1,2,null,3] 05 04 00 05 02 09 02 02 05 02 09 04 => [[1],[2]] 05 04 00 06 01 02 07 02 00 00 00 => [1,2]
Here is an example of random parsed replay.details I downloaded:
[[["IdrA",[1,844300288,1,null,693604],"Zerg",[255,22,128,0],2,1,100,0,2],["Silver",[1,844300288,1,null,445448],"Terran",[255,0,66,255],0,0,0,0,1]],"Lost Temple","",["Minimap.tga"],1,129251921670000000,-144000000000,"","","",[<unrealted unreadable binary data>],0,4,2]
It's very easy to work with and part of program that works with binary data not only becomes small but it almost doesn't change when blizz releases new version of replays. You can see that [255,22,128,0] is color, "Silver" is player name, "Lost Temple" is map name, and after studying some other replays it's easy to notice that last digit in player array is team number and 129251921670000000 is date.
|
Dakota_Fanning
Hungary2325 Posts
On August 06 2010 19:48 noko1111 wrote:+ Show Spoiler +I'd post this as new thread, but apparently someone had a great idea of not allowing users to make threads. So here goes: After tinkering for a bit with replays, I found a proper way to parse replay.details file inside replay MPQ. I'm posting it here because there is no information about this part of replays on the internet (apart from obviously incomplete info like http://code.google.com/p/starcraft2replay/wiki/ReplayDetails ): replay sites that are working are hoarding what they know to themselves (to kill off competition?). This file has data in it, that is stored in a data serialization format very similar to JSON, apart from the fact that JSON is text format while replay.details is binary. User data of MPQ archive (the MPQ\x1b part) is also using this format, while in beta it was just plain binary. Basically, whole file can be translated into array of values that are either arrays or scalars (strings or numbers). Any modern language can work natively with these values so working on replays becomes a simple and enjoyable task. Before I start explaining format, I'll first describe how integers are stored in replays. Blizzard chose a variable-length format (different from the one used in in other parts of replays by the way, why?) that is in a way similar to utf8: a number consists of a number of bytes; a byte that has 0 in most significant bit is the last one. So, to read a number, you read bytes until you encounter the one less than 0x80, and put their 7 least significant bytes into your variable, first byte at offset 0, second - at offset 7, third - at 14, etc. Least significant bit of result is sign (two's complements were a bit too difficult for talented sc2 team?). Shift result right by one byte and multiply by -1 if least significant bit was set. This is the number stored in replay. I am very sure it is the correct way to handle it because I got numbers like 144000000000 (decimal) from some replays, as well as 1337 (each player had this number attached to him in beta). I'll refer to this format as VLF (variable length format) later on. Here are few examples: bytes values (dec) ===== ============ 00 => 0 01 => 0 02 => 1 03 => -1 04 => 2 7e => 63 7f => -63 80 => INVALID (last byte must have most significant bit set) 80 00 => 0 (but you'll never find it in replays since 0 can be represented by just 00) 80 01 => 64 81 01 => -64 82 01 => 65 80 02 => 128 81 02 => -128 80 80 80 10 => 16777216 Now, replay.details format itself is pretty simple. It consists of one data type marker followed by data itself. Data type marker is a single byte and can have following values: 02: Binary data. 02 is followed by size of binary data in VLF after that comes data itself; in most cases it's text in utf8. 02 12 41 55 54 4F 4D 41 54 49 43 is a string AUTOMATIC. 04: An array. 04 is alway followed by 01 00, after that there is number of elements in array in VLF, and elements follow. Each element is a data type marker followed by data, all in same format. 05: An array. Followed by number in VLF which indicates how many pairs will follow -- a pair consists of array element index in VLF and element itself (data type marker followed by data). 06: A number. Its value follows in one byte. 07: A number. Its value follows in four bytes. 09: A number. Its value follows in VLF. Here are examples: binary data (right align) equivalent in JSON ========================= ================== 02 04 68 69 => "hi" 05 02 00 02 04 68 69 => ["hi"] 05 04 00 02 04 68 69 02 02 04 68 69 => ["hi","hi"] 05 06 00 09 02 02 09 04 08 09 06 => [1,2,null,3] 05 04 00 05 02 09 02 02 05 02 09 04 => [[1],[2]] 05 04 00 06 01 02 07 02 00 00 00 => [1,2] Here is an example of random parsed replay.details I downloaded: [[["IdrA",[1,844300288,1,null,693604],"Zerg",[255,22,128,0],2,1,100,0,2],["Silver",[1,844300288,1,null,445448],"Terran",[255,0,66,255],0,0,0,0,1]],"Lost Temple","",["Minimap.tga"],1,129251921670000000,-144000000000,"","","",[<unrealted unreadable binary data>],0,4,2] It's very easy to work with and part of program that works with binary data not only becomes small but it almost doesn't change when blizz releases new version of replays. You can see that [255,22,128,0] is color, "Silver" is player name, "Lost Temple" is map name, and after studying some other replays it's easy to notice that last digit in player array is team number and 129251921670000000 is date. That is some valuable info, I wasn't able to find the replay save time. Thanks.
Edit: I put the quote in spoiler.
|
This indeed is very valuable.
Am I allowed to reuse your information? I would like to put this down in my wiki for public access and work it into WARP. Is this ok for you? I already started a documentation of the sc2replay format, but due to the format changes, among other factors, I put more emphasis on the development of WARP.
Maybe some of the other developers might join in and help improving the format's documentation
|
Hi guys,
finally, WARP v0.8 is out.
Version 0.8 supports all replays from the start of the beta, i.e. patches 0.2.0 to 1.0.2! More changes: - Fixed a bug that caused replays to be displayed a second time, if the user resumes parsing on startup - Users are notified which replays could not have been parsed - Number of parsed replays is shown correctly - A player's tooltip shows his realID, if available - Numerous updates to architecture and source code, allowing better maintenance and enhanced development work flow
Again, it is available on http://trac.erichseifert.de/warp.
Greetings, Hailstorm
|
Due to the new patch that was released today, a small version bump to WARP v0.8.1 just occurred. This version also supports replays of 1.0.3
|
Many thanks noko1111, that <unrealted unreadable binary data> is actually a hash for the map. It's to distinguish between what map was used. It's consistent based on the map it was played on.
|
Hi everyone!
Thanks a lot for that posting, Noko1111. I'm currently working on some kind of tool where i want to work with those Starcraft 2 replay files.
I tried to understand the logic behind all that stuff how integers are stored in replay files, but i'm kinda stuck here and don't seem to quiet get it yet.
I think i know how to convert the read data from the replay file back to a "normal" number (so far only to a length of 2 bytes - haven't tried the bigger ones yet).
My problem is, that i have absolutely no idea how to encode a normal number into the replay format number. I tried lots of things sitting here with pen and paper shifting those bits around but i couldn't find it out yet.
Can someone please try to explain that hole thing again maybe with some examples how to calculate it in the two directions (replay format => normal number and backwards)? Maybe there is some kind of mathematical formular to use?
Thanks and greetings Skubidus
|
Nevermind figured it out finally.
|
hello, it stucks at 67% when it tries to parse replays, what's going on?
|
So if this is entirely made in java, could there be a port to android. I recommend going to XDA developers if you need help with it.
|
On October 02 2010 11:37 qnoob wrote: hello, it stucks at 67% when it tries to parse replays, what's going on? Which version of WARP are you using? This bug should have been fixed with version 0.8.1. It occurs due to some error when parsing the replays. Most likely it is the new patch that causes trouble here. The replay format changed once again and broke all the parsers.
If you are using the WARP 0.8.1 and the error still occurs, please follow the following steps (I assume you are running WARP on a Windows machine): - Start the command line (Start menu -> run... -> type cmd) - Step into the folder where you installed WARP (cd folder\to\WARP) - Run WARP with the command java -cp "bin\*" de.erichseifert.warp.gui.swing.WARP - Parse the replay folder again until it gets stuck - Copy the output in the command line and send it to me (preferrably via PM)
On October 02 2010 12:15 L1F3 wrote: So if this is entirely made in java, could there be a port to android. I recommend going to XDA developers if you need help with it. I always thought that Java applications run without problems on Android... I am not familiar with this, but according to Wikipedia, there is a cross-assembler for Java that allows applications to run on Android. I will look into this with the next release. Thanks for the hint!
Greetings, Hailstorm
|
Hey there - any updates on WARP? It's been a few patches that it can't parse replays Any plans to update it? Keep up the good work
|
Yes there are. Last sunday, the guy from phpsc2replay figured outthe new format. I must admit that I didn't have a look at it this time... I will try to start implementing the changes this weekend, but I am currently short on time.
|
Good stuff, I'll have to check this out
|
On October 14 2010 03:25 Hailst0rm wrote:Yes there are. Last sunday, the guy from phpsc2replay figured outthe new format. I must admit that I didn't have a look at it this time... I will try to start implementing the changes this weekend, but I am currently short on time.
No worries - keep up the good work
|
can this be implemented into a website ? purpose of cup website - autowin loss and info about replay online
|
Hey,
i can't load any of my replays. Does the parser not work with the new patch yet?
|
|
|
|