• Log InLog In
  • Register
Liquid`
Team Liquid Liquipedia
EDT 19:56
CEST 01:56
KST 08:56
  • Home
  • Forum
  • Calendar
  • Streams
  • Liquipedia
  • Features
  • Store
  • EPT
  • TL+
  • StarCraft 2
  • Brood War
  • Smash
  • Heroes
  • Counter-Strike
  • Overwatch
  • Liquibet
  • Fantasy StarCraft
  • TLPD
  • StarCraft 2
  • Brood War
  • Blogs
Forum Sidebar
Events/Features
News
Featured News
Code S RO4 & Finals Preview: herO, Rogue, Classic, GuMiho0TL Team Map Contest #5: Presented by Monster Energy4Code S RO8 Preview: herO, Zoun, Bunny, Classic7Code S RO8 Preview: Rogue, GuMiho, Solar, Maru3BGE Stara Zagora 2025: Info & Preview27
Community News
Classic & herO RO8 Interviews: "I think it’s time to teach [Rogue] a lesson."2Rogue & GuMiho RO8 interviews: "Lifting that trophy would be a testament to all I’ve had to overcome over the years and how far I’ve come on this journey.4Code S RO8 Results + RO4 Bracket (2025 Season 2)12BGE Stara Zagora 2025 - Replay Pack2Weekly Cups (June 2-8): herO doubles down1
StarCraft 2
General
Rogue & GuMiho RO8 interviews: "Lifting that trophy would be a testament to all I’ve had to overcome over the years and how far I’ve come on this journey. Classic & herO RO8 Interviews: "I think it’s time to teach [Rogue] a lesson." Code S RO8 Results + RO4 Bracket (2025 Season 2) Code S RO4 & Finals Preview: herO, Rogue, Classic, GuMiho I have an extra ticket to the GSL Ro4/finals
Tourneys
[GSL 2025] Code S: Season 2 - Semi Finals & Finals $3,500 WardiTV European League 2025 Sea Duckling Open (Global, Bronze-Diamond) SOOPer7s Showmatches 2025 RSL: Revival, a new crowdfunded tournament series
Strategy
[G] Darkgrid Layout Simple Questions Simple Answers [G] PvT Cheese: 13 Gate Proxy Robo
Custom Maps
[UMS] Zillion Zerglings
External Content
Mutation # 477 Slow and Steady Mutation # 476 Charnel House Mutation # 475 Hard Target Mutation # 474 Futile Resistance
Brood War
General
TRUSTWORTHY BITCOIN RECOVERY EXPERT HIRE SLAYER CO ASL20 Preliminary Maps BGH Auto Balance -> http://bghmmr.eu/ Recent recommended BW games BW General Discussion
Tourneys
[Megathread] Daily Proleagues [BSL 2v2] ProLeague Season 3 - Friday 21:00 CET Small VOD Thread 2.0 [BSL20] ProLeague Bracket Stage - Day 4
Strategy
I am doing this better than progamers do. [G] How to get started on ladder as a new Z player
Other Games
General Games
Path of Exile Nintendo Switch Thread Stormgate/Frost Giant Megathread Beyond All Reason What do you want from future RTS games?
Dota 2
Official 'what is Dota anymore' discussion
League of Legends
Heroes of the Storm
Simple Questions, Simple Answers Heroes of the Storm 2.0
Hearthstone
Heroes of StarCraft mini-set
TL Mafia
Vanilla Mini Mafia TL Mafia Community Thread
Community
General
US Politics Mega-thread Things Aren’t Peaceful in Palestine UK Politics Mega-thread Russo-Ukrainian War Thread Vape Nation Thread
Fan Clubs
Maru Fan Club Serral Fan Club
Media & Entertainment
Korean Music Discussion [Manga] One Piece
Sports
Formula 1 Discussion 2024 - 2025 Football Thread NHL Playoffs 2024 TeamLiquid Health and Fitness Initiative For 2023
World Cup 2022
Tech Support
Computer Build, Upgrade & Buying Resource Thread
TL Community
The Automated Ban List
Blogs
A Better Routine For Progame…
TrAiDoS
StarCraft improvement
iopq
Heero Yuy & the Tax…
KrillinFromwales
I was completely wrong ab…
jameswatts
Need Your Help/Advice
Glider
Trip to the Zoo
micronesia
Customize Sidebar...

Website Feedback

Closed Threads



Active: 32310 users

Looking for a replay parser

Forum Index > SC2 General
Post a Reply
Deleted User 3420
Profile Blog Joined May 2003
24492 Posts
Last Edited: 2019-04-12 18:35:27
April 12 2019 13:29 GMT
#1
I am building a deep learning agent for sc2, and part of the agent involves outcome prediction (just like alphastar).

The way Deepmind did it is they took a huge dataset of replays, and then actually ran the replays, and recorded observations frame by frame. They then used those observations as their dataset to train with.

The problem is, for people like me with normal PCs, this isn't really feasible. Deepmind did it with many cores, each one running a replay at the same time. So they were able to record observations much faster than I will be able to with my home PC.

I am wondering if there are any advanced replay parsers at the moment (particularly ones that can work with various patches).


What I want is a bit demanding, though. I don't just want to parse actions, I also need, at minimum:

my units positions
my unit deaths
enemy units killed
gas,minerals,time
available and current supply


even better would be "seen enemy units" or "current seen enemy positions"




Does anyone know about this? Thank you.
alpenrahm
Profile Blog Joined December 2010
Germany628 Posts
April 12 2019 14:48 GMT
#2
is there even an official way to run replays without the video interface?

I think these guys (http://ggtracker.com/landing_tour) might be able to help you out?
in anycase it´s a pretty cool project. keep us updated on your progress!



loginn
Profile Blog Joined January 2011
France815 Posts
Last Edited: 2019-04-12 16:25:55
April 12 2019 16:18 GMT
#3
github.com is the official client to parse replays.

I think has the kind of information you're looking for though not at every tick but rather every 70 ticks I think.

Breath20 from sc2replaystats has a lot of experience, try his discord maybe ?
Stephano, Taking skill to the bank since IPL3. Also Lucifron and FBH
Senkii
Profile Joined December 2012
Hungary37 Posts
April 12 2019 16:32 GMT
#4
sc2reader is a bit outdated but has most of this. Replays contain some tracker events, like checking supply, mineral, gas etc every 10 sec as well as units created/killed with their positions. If you want to run the reps you can only use the blizz interface which can only run reps at 16x speed one by one as far as i know (never tried)
Cyro
Profile Blog Joined June 2011
United Kingdom20284 Posts
Last Edited: 2019-04-12 17:47:28
April 12 2019 17:41 GMT
#5
What I want is a bit demanding, though. I don't just want to parse actions, I also need, at minimum:

my units positions
my unit deaths


I don't think that this can be done the way that you want it to. The replay file (and internet traffic) contains surprisingly little information. It records a bare minimum of data and then the game simulates the game state deterministically from there.

If you create a unit at a certain position and tell it to move across the map then the information recorded is the unit creation time/position/etc and the move command, not any of its locations along the way. Pathfinding, collisions with other units and so on - even fights and unit deaths IIRC - are handled in the simulation.

Since it's deterministic the end result will always be the same given the same input but you do have to do the simulation on each PC if you're playing the game or want to usefully see much of the information that the replay contains.
"oh my god my overclock... I got a single WHEA error on the 23rd hour, 9 minutes" -Belial88
Deleted User 3420
Profile Blog Joined May 2003
24492 Posts
April 12 2019 17:49 GMT
#6
Cyro - I know this but my hope was that someone had created a tool that would process some of that information from the replay file itself. As in, "play the game out" to the extent of getting various game state information - without actually playing it with the SC2 client.

How that could be done, I don't really know.

That said I am playing around with SC2reader now. I had played with s2protocol and was having some issues, but I'll try it again too if sc2reader doesn't have enough to work for me.

Thank you for the suggestions everyone.
Cyro
Profile Blog Joined June 2011
United Kingdom20284 Posts
April 12 2019 17:59 GMT
#7
It would have to be a perfectly accurate simulation, every time there is even a tiny tweak we see blizzard make replays unplayable in the new version of the game because they no longer play out as they should
"oh my god my overclock... I got a single WHEA error on the 23rd hour, 9 minutes" -Belial88
AKnopf
Profile Blog Joined March 2011
Germany259 Posts
April 12 2019 18:09 GMT
#8
During the alphastar presentation they said, that Blizzard supplied them with a special build that was created just for that use case. It runs replays (and apparently even AI games) at high speed without the GUI.

My guess is, they paid Blizzard good money to create that build so the chances of them sharing it with the public is pretty slim. Unfortunately, that also means that there is no such tool available anywhere, or else Blizzard would not have created it custom for deep mind.

Good luck with your project, though!
The world - its a funny place
loginn
Profile Blog Joined January 2011
France815 Posts
April 12 2019 20:00 GMT
#9
On April 13 2019 03:09 AKnopf wrote:
During the alphastar presentation they said, that Blizzard supplied them with a special build that was created just for that use case. It runs replays (and apparently even AI games) at high speed without the GUI.

My guess is, they paid Blizzard good money to create that build so the chances of them sharing it with the public is pretty slim. Unfortunately, that also means that there is no such tool available anywhere, or else Blizzard would not have created it custom for deep mind.

Good luck with your project, though!


I believe this is open source
Stephano, Taking skill to the bank since IPL3. Also Lucifron and FBH
Deleted User 3420
Profile Blog Joined May 2003
24492 Posts
April 12 2019 22:29 GMT
#10
Well in the end I have decided that a replay parser probably just won't work, the quality of the data I need is just too important to not utilize everything that can be observed in a given moment of the game. plus my predictor needs to reflect all the data a player would have available at that given moment.


the parsers are still useful to delete the thousands of replays where the main player is not zerg or where a player leaves right at the beginning, though
Please log in or register to reply.
Live Events Refresh
Cheesadelphia
15:00
Cheeseadelphia 2025
Gerald vs EpicLIVE!
TBD vs FuturE
CranKy Ducklings444
davetesta101
LiquipediaDiscussion
[ Submit Event ]
Live Streams
Refresh
StarCraft 2
RuFF_SC2 91
StarCraft: Brood War
Britney 16202
NaDa 31
Dota 2
LuMiX1
League of Legends
Grubby2867
Dendi1495
Counter-Strike
Fnx 1329
Stewie2K900
Foxcn609
Super Smash Bros
Mew2King39
Other Games
tarik_tv13357
summit1g9975
C9.Mang0782
ViBE160
Trikslyr79
Organizations
Other Games
gamesdonequick1128
StarCraft 2
Blizzard YouTube
StarCraft: Brood War
BSLTrovo
sctven
[ Show 20 non-featured ]
StarCraft 2
• Hupsaiya 84
• RyuSc2 75
• musti20045 30
• HeavenSC 24
• tFFMrPink 11
• IndyKCrew
• AfreecaTV YouTube
• intothetv
• Kozan
• sooper7s
• LaughNgamezSOOP
• Migwel
StarCraft: Brood War
• RayReign 15
• STPLYoutube
• ZZZeroYoutube
• BSLYoutube
Dota 2
• Ler91
League of Legends
• Doublelift5491
Other Games
• imaqtpie1843
• Shiphtur233
Upcoming Events
GSL Code S
8h 5m
Rogue vs herO
Classic vs GuMiho
Sparkling Tuna Cup
10h 5m
WardiTV Qualifier
16h 5m
BSL: ProLeague
18h 5m
Bonyth vs Dewalt
Cross vs Doodle
MadiNho vs Dragon
Replay Cast
1d
Wardi Open
1d 11h
Replay Cast
2 days
Replay Cast
2 days
RSL Revival
2 days
Cure vs Percival
ByuN vs Spirit
RSL Revival
3 days
herO vs sOs
Zoun vs Clem
[ Show More ]
Replay Cast
4 days
The PondCast
4 days
RSL Revival
4 days
Serral vs SHIN
Solar vs Cham
Replay Cast
5 days
RSL Revival
5 days
Reynor vs Scarlett
ShoWTimE vs Classic
uThermal 2v2 Circuit
5 days
SC Evo League
6 days
Circuito Brasileiro de…
6 days
Liquipedia Results

Completed

Proleague 2025-06-11
BGE Stara Zagora 2025
Heroes 10 EU

Ongoing

JPL Season 2
BSL 2v2 Season 3
BSL Season 20
KCM Race Survival 2025 Season 2
NPSL S3
Rose Open S1
CSL 17: 2025 SUMMER
2025 GSL S2
Murky Cup #2
BLAST.tv Austin Major 2025
ESL Impact League Season 7
IEM Dallas 2025
PGL Astana 2025
Asian Champions League '25
BLAST Rivals Spring 2025
MESA Nomadic Masters
CCT Season 2 Global Finals
IEM Melbourne 2025
YaLLa Compass Qatar 2025
PGL Bucharest 2025

Upcoming

Copa Latinoamericana 4
CSLPRO Last Chance 2025
CSLPRO Chat StarLAN 3
K-Championship
SEL Season 2 Championship
Esports World Cup 2025
HSC XXVII
Championship of Russia 2025
BLAST Open Fall 2025
Esports World Cup 2025
BLAST Bounty Fall 2025
BLAST Bounty Fall Qual
IEM Cologne 2025
FISSURE Playground #1
TLPD

1. ByuN
2. TY
3. Dark
4. Solar
5. Stats
6. Nerchio
7. sOs
8. soO
9. INnoVation
10. Elazer
1. Rain
2. Flash
3. EffOrt
4. Last
5. Bisu
6. Soulkey
7. Mini
8. Sharp
Sidebar Settings...

Advertising | Privacy Policy | Terms Of Use | Contact Us

Original banner artwork: Jim Warren
The contents of this webpage are copyright © 2025 TLnet. All Rights Reserved.