SYN Flood Attack: How Unfinished TCP Handshakes Work

A SYN flood attack exploits the TCP three-way handshake by sending a flood of SYN packets but never completing the handshake, forcing the target server to hold half-open connections open until its resources are exhausted. The attacker sends many connection requests with spoofed or unreachable source IP addresses, and the server reserves memory and state for each one while waiting for an ACK that never arrives. This article explains the exact mechanism, the role of unfinished handshakes, and how defenses like SYN cookies mitigate the attack.

The TCP Three-Way Handshake

Before any TCP connection carries data, the client and server must agree on initial sequence numbers and parameters through a three-step process:

1. SYN: The client sends a packet with the SYN flag set, announcing an initial sequence number (ISN) and requesting a connection.
2. SYN-ACK: The server responds with a packet that sets both SYN and ACK flags, acknowledging the client’s ISN and offering its own ISN.
3. ACK: The client sends a packet with the ACK flag set, acknowledging the server’s ISN. At this point, the connection is established and data can flow.

During step 2, the server allocates a Transmission Control Block (TCB), which holds the connection’s state, sequence numbers, buffer sizes, and timers. The TCB is stored in a kernel table called the connection table or SYN queue. The server keeps this entry for a period called the SYN timeout (typically between 30 and 120 seconds, depending on the OS) waiting for the final ACK.

How a SYN Flood Exploits the Handshake

In a SYN flood, the attacker sends a continuous stream of SYN packets, each with a forged source IP address that either does not exist or is not reachable. The server responds to each with a SYN-ACK and then waits for the ACK that will never come. Because the source IP is fake, the server’s SYN-ACK goes into the void, and the TCB remains in the SYN-RECEIVED state until the timeout expires.

The attacker does not need to send many packets per second to be effective. Each half-open connection consumes:

• Memory for the TCB (typically a few hundred bytes to a few kilobytes).
• A slot in the SYN queue, which is usually limited to a few hundred or thousand entries.
• CPU time to process the incoming SYN and generate the SYN-ACK.

Once the queue is full, the server drops new incoming SYN packets, including those from legitimate users. The result is a denial of service: the server cannot accept any new connections, even though it may still be able to process existing ones.

Why the Handshake Is Never Completed

The attacker deliberately never sends the final ACK. In a classic direct SYN flood, the attacker uses raw sockets to craft packets with random source IPs. In a distributed reflection attack, the attacker sends SYN packets with the victim’s IP as the source to a large number of innocent servers, which then all send SYN-ACKs to the victim, overwhelming it with unsolicited traffic.

Some variants use the attacker’s real IP but simply do not respond to the SYN-ACK. This is less common because the server can eventually identify and block that IP, and the attacker’s own resources are consumed by the incoming SYN-ACKs. Spoofing the source IP is the core trick: it makes the server’s responses go nowhere and makes it impossible for the server to distinguish a malicious SYN from a legitimate one that was lost in the network.

Common Misconception: The Server Crashes

Many people assume a SYN flood causes a kernel panic or a total system crash. In practice, the OS remains running, but the network stack becomes unresponsive to new connections. The machine may still respond to pings and even to established connections, but it cannot open new ones. The system’s memory may be heavily consumed by TCBs, but a modern OS has limits on the SYN queue, so the flood typically fills that queue rather than exhausting all RAM. The effect is a denial of service for new users, not a system-wide crash.

Defenses and Mitigations

Several techniques have been developed to counter SYN floods:

• SYN cookies: Instead of storing a TCB for each incoming SYN, the server encodes the connection state into the sequence number of the SYN-ACK. When the final ACK arrives, the server reconstructs the state from the sequence number, so no memory is allocated until the handshake completes. This is the most common defense on Linux and many firewalls.
• Shortening the SYN timeout: Reducing the time the server waits for the final ACK shrinks the window in which a half-open connection occupies resources. This helps but does not prevent the flood.
• Rate limiting: Firewalls can limit the number of SYN packets per second per source IP, dropping excess packets. This is effective against single-source floods but less so against distributed attacks.
• Filtering spoofed IPs: Network ingress filtering (RFC 2827) blocks packets that claim to come from addresses that are not routable from the source network. This prevents many spoofed SYN floods at the network edge.
• Specialized DDoS protection: Cloud-based scrubbing services and dedicated DDoS appliances can detect and drop malicious SYN floods before they reach the server.

Real-World Impact and Scale

SYN floods are one of the oldest and most common DDoS attack types, first documented in the late 1990s. They remain relevant because TCP is everywhere and the attack is cheap to launch: a single computer can generate thousands of SYN packets per second using simple tools. Distributed attacks using botnets can generate millions of packets per second, overwhelming even large servers.

The impact depends on the target’s capacity and the attack volume. A small web server might be taken down by a few thousand half-open connections, while a large cloud provider with SYN cookies enabled might shrug off a multi-gigabit flood. Modern operating systems have SYN cookies enabled by default (Linux) or offer them as a configurable option (Windows, macOS), which greatly reduces the effectiveness of simple floods.

Frequently Asked Questions

What is a half-open connection?

A half-open connection is a TCP connection in which the server has sent a SYN-ACK but has not yet received the final ACK. The server has allocated resources for the connection, but it is not fully established. In a SYN flood, the attacker creates many half-open connections and never completes them.

How does a SYN cookie work?

A SYN cookie is a cryptographic hash of the client’s IP, port, and a secret value, placed in the sequence number of the SYN-ACK. The server does not store any state for the connection. When the client sends the final ACK, the server verifies the sequence number and, if valid, creates the TCB at that moment. This eliminates the need to hold resources for unverified connections.

Can a SYN flood be launched from a single computer?

Yes. A single computer with a modest internet connection can generate thousands of SYN packets per second, which is enough to fill the SYN queue of a typical server. However, modern defenses like SYN cookies make it much harder for a single source to cause significant damage, so attackers often use botnets for distributed floods.

What is the difference between a SYN flood and a DDoS attack?

A SYN flood is a specific type of denial-of-service (DoS) attack that targets the TCP handshake. A distributed denial-of-service (DDoS) attack is any DoS attack launched from multiple sources. A SYN flood can be a DDoS if it is launched from a botnet, but it can also be a simple DoS from one machine.

How can I tell if my server is under a SYN flood?

Common signs include a large number of connections in the SYN-RECEIVED state (visible with netstat or ss), a full SYN queue, and an inability to accept new connections while existing ones work. Network traffic analysis shows a high rate of incoming SYN packets with no corresponding ACKs.

Is a SYN flood illegal?

In most jurisdictions, launching a SYN flood against a system you do not own is illegal under computer fraud and abuse laws, as it is a form of unauthorized denial of service. It is also a violation of most ISPs’ terms of service. This article is for educational purposes only.

Related

• Forge Captain Rowan Quest Guide
• SIM Sent a Text Message? Meaning & Fixes
• Check Your PC’s Full? Disk Space Guide