ESEA Invite League

On October 08 2014 07:44 R1CH wrote:
There's a lot of issues affecting tournament games recently in Dota 2, and a lot of people are quick to blame it on DDoS. However there's also some misunderstanding of what a DDoS is and what it can affect, so I thought I'd write a post to clear some of it up.

What's a DDoS?
As some background, I used to run game servers back in 2001 for several years and dealt with DDoS attacks against the game servers, associated websites and my ISP. Obviously these days the attacks are a lot larger, but the principle of a DDoS attack is still the same: overwhelm a network connection with data to the point where it becomes unusable.

DDoS stands for Distributed Denial of Service. Distributed means the attack comes from many IPs distributed around the world, in the form of compromised home users, websites, dedicated servers, routers and more. A non-distributed DoS attack is quite trivial to block once you isolate the offending IP, however a DDoS is much harder to block since the IPs are mixed in with legitimate IPs.

How It Works
For a DDoS attack to work, the crucial piece of information an attacker needs is an IP address. Without an IP, there's nowhere to send the attack traffic. Thus it's very important to make sure IP addresses remain private to be safe from an attack.

Once an attacker has an IP, they command their network of compromised PCs to flood it with garbage data. Most attackers don't actually own such a network and instead pay to rent out a botnet to launch attacks, or use a pay-to-DDoS service.

How Can I Tell If I'm Being DDoSed?
Generally since your internet connection will be completely flooded, a DDoS attack will manifest as your game having unplayable latency and packet loss, disconnections from voip software, websites failing to load and inability to really do much of anything with your internet connection. Eventually the game may time out once no legitimate data is able to get through.

If you lose the connection to the game server but can still browse websites and chat on teamspeak etc, it's not a DDoS attack but likely a routing issue to the game server - unfortunately there isn't anything you can do about this but wait for it to subside. While it could also be the game server crashed or an attack on the game server, you'd have to confirm with other players to know for sure.

How To Mess Up A Dota 2 Game
There's several ways someone can interfere with a Dota 2 game.

1) Attack the players
By finding the IP of a player, you can trivially knock them offline with a minimal amount of bandwidth. Since most players are playing from residential connections with limited download speed, a DDoS attack can easily saturate a typical 100mbps connection.

2) Attack the Dota 2 network
If the attacker has enough bandwidth, they can try to destablize the Dota 2 servers. Since the IP of the servers are public, it's just a case of throwing enough bandwidth at them and hoping one of the upstream links gets saturated, causing lag and disconnections.

3) Attack the Dota 2 game server
If the IP address of the game server is able to be discovered, the attacker can directly hit that specific game server. Much like a home user, each individual server has less available bandwidth than the entire ISP, likely limited by a 1gbps ethernet connection. If you're able to figure out the server IP, it's quite easy to cause lag and disconnections by saturating the uplink to the game server.

Update: A recent Dota 2 update fixed a few issues related to how the game server IP could be discovered, but there may be other methods that aren't as well known.

4) Attack random servers
Similar to the above attack, multiple servers share a physical switch, which itself has limited upstream bandwidth. Even if you don't know the game server IP, you can hope to get lucky and find another server on the same switch, and overload the bandwidth of the switch. Since each game server IP hosts multiple Dota 2 games, one switch could represent several hundred active games.

As far as I can tell, most attacks against tournament games fall into categories 1) and 3).

Examples
It's not always possible to determine if something is the result of a DDoS or other network problem. For example, someone constantly disconnecting from a game could just mean their internet is being flaky rather than an attack.

Example: All the players in a gaming house drop at the same time
Cannot be determined. Either their internet connection is having trouble or it's an attack. The only way to know is for the players to monitor incoming bandwidth at their router.

Example: Several unrelated people drop from a game at the same time
Most likely an internet routing issue. There are many different routes to the servers, and if a route experiences a problem, anyone using that route would be affected. There's a small possibility it could be an attack that is only able to saturate that specific route.

Example: Everyone on the server lags or drops at the same time
This is probably a DDoS attack on the server, or an internal network issue inside the Dota 2 network.

Keep in mind that online gaming is much more demanding of your internet connection than web browsing or downloading files. Routes can change all the time and you'd normally never even notice, but during a game even a few hundred milliseconds of dropped packets during a re-routing event or other network issue can be very noticeable.

Protection
The key way to stay protected is to hide your IP - without an IP, there's nothing to attack. Players should never have to reveal their IP, and should stay away from IRC and P2P networks that might expose it. Skype is a big problem as it can reveal your IP very easily, but there are ways around that.

Anyone on your Steam friends list or in the same steam group as you can also find your IP using the voice call feature. The Steam Beta has an update to protect against this, if you don't use the Beta then it's best to remove anyone you don't trust from your friends list and leave any public groups.

For Valve and their game servers, this becomes a bit harder. The IP of the servers is public, since people have to know the game server IPs to be able to connect and play games on them. To avoid attack scenario 3) above, it's important that players and spectators in tournament games don't reveal the IP of the game server they're playing on. Simply opening the console will show this information, so avoid the temptation to check pings via the console or similar while streaming.

There are hardware mitigation devices that can attempt to identify and filter out DDoS traffic, but these are expensive and scale poorly - you need enough bandwidth plus filter capacity to handle the biggest DDoS attack against you.

Other service providers offer scrubbing services, where the real IP of the game server is essentially protected through a high bandwidth proxy and all traffic is examined and filtered before being routed to the real server. Unfortunately this doesn't work well for game servers, since the scrubbing and re-routing can add significant latency.

Another option is simply to throw more bandwidth at the problem. If you have more bandwidth than the attackers then you won't suffer a complete network outage, but individual servers can still be targeted if the IP is found.

As you can see, unfortunately there aren't really many good solutions for protecting game servers. Large DDoS attacks do get investigated and it's possible the bigger attack networks will get shut down, but tracing an attack is often very hard if not impossible without detailed forensic investigation of the compromised systems that are part of the attack.

It's also difficult to know if something is the result of a DDoS attack or legitimate network issues. Players would need to be able to check incoming bandwidth on their router to confirm an attack, which is not something many routers offer. Game servers lagging could also be due to poor routing or internal Dota 2 network issues.