AoE: What Hosts and Storage Targets Exchange

ATA over Ethernet (AoE) is a network protocol that lets a host computer access a block storage device (a target) directly over an Ethernet network, as if it were a locally attached hard drive. In the simplest terms, the host sends ATA commands wrapped in Ethernet frames to the target, and the target replies with the requested data or status information. This exchange is purely about reading and writing data blocks — there is no TCP/IP, no routing, and no filesystem logic in the protocol itself.

Formula

The calculator above estimates the effective throughput of an AoE transfer. The formula is:

Effective throughput (GB/s) = (Data size in GB × 8) / Time in seconds

Where:

• Data size (GB) – the amount of data you move, in gigabytes (1 GB = 10⁹ bytes).
• Time (s) – the duration of the transfer, in seconds.
• Network bandwidth (Gb/s) – the nominal link speed, in gigabits per second (used to compute the percentage of bandwidth utilized).

The second output, Bandwidth used (%), is (effective throughput × 8) / bandwidth × 100. It shows how much of the link’s nominal capacity you actually achieve, accounting for overhead.

Worked Example

Using the defaults: transfer 100 GB in 60 seconds over a 1 Gb/s link.

1. Convert GB to bits: 100 GB × 8 = 800 Gb.
2. Divide by time: 800 Gb / 60 s = 13.333 Gb/s.
3. Convert to GB/s: 13.333 / 8 = 1.667 GB/s.
4. Bandwidth used: (1.667 × 8) / 1 × 100 = 1333% — this exceeds 100%, indicating the transfer is faster than the link’s nominal speed, which is impossible in practice. In reality, the link would cap the throughput at ~1 Gb/s (0.125 GB/s), so the time input would be larger.

The calculator gives 1.667 GB/s and 1333% for the default values, which is a clear sign that the input time is too short for the data size on a 1 Gb/s link. Adjust the time or bandwidth to realistic values.

Background: Why AoE Exists

ATA over Ethernet was developed in the early 2000s by Brantley Coile of Coraid, Inc. as a way to build storage area networks (SANs) without the overhead of TCP/IP. Unlike iSCSI, which encapsulates SCSI commands in TCP packets, AoE sends raw ATA commands directly over Ethernet frames (EtherType 0x88A2). This eliminates TCP/IP processing, reducing CPU load on the host and lowering latency. AoE is designed for local area networks (LANs) only — it cannot be routed across subnets because it has no IP layer.

The protocol is simple: the host (initiator) sends a command frame to a target’s MAC address, and the target responds with a data frame or a status frame. There is no authentication or encryption in the base protocol, so it is typically used in trusted, isolated storage networks.

How the Exchange Works

The exchange between a host and a target follows a request-response pattern. Here is the sequence for a typical read operation:

1. The host’s AoE driver constructs an AoE header with the target’s major and minor address, a command code (e.g., ATA_CMD_READ), and a logical block address (LBA).
2. The frame is sent to the target’s MAC address, with the EtherType set to 0x88A2.
3. The target receives the frame, performs the ATA command on its attached disk, and sends back a response frame containing the requested data (for reads) or a status code (for writes).
4. The host’s driver delivers the data to the operating system, which sees the target as a local block device (e.g., /dev/sdb).

For writes, the host sends the data in the same frame as the command, and the target replies with a status frame indicating success or failure. The protocol also supports tag numbers to match responses to requests, and a “query� command to discover targets on the network.

What Hosts and Targets Actually Exchange

At the wire level, the exchange consists of Ethernet frames with an AoE header. The AoE header contains:

• Version (4 bits) – always 1 for AoE.
• Flags (4 bits) – indicate response or command, and error conditions.
• Major and minor target addresses (16 bits each) – identify the specific target and its LUN (logical unit).
• Command (8 bits) – the ATA command (read, write, identify, etc.).
• Tag (16 bits) – a unique identifier for the request, used to match responses.
• Sector count (16 bits) – number of 512-byte sectors to transfer.
• LBA (48 bits) – starting logical block address.

The payload for a write command contains the data to be written (up to 16 sectors per frame in the original spec, but larger frames are possible with jumbo frames). For reads, the response frame carries the data. The target also sends an “identify� response with device parameters (capacity, model, etc.) when the host issues an ATA IDENTIFY command.

How to Choose Between AoE and iSCSI

Choosing AoE vs. iSCSI depends on your network and requirements:

• Use AoE if you need the lowest possible latency and CPU overhead, and your storage network is a single LAN segment with no routing. AoE is simpler and often faster for pure block storage.
• Use iSCSI if you need to route traffic across subnets, use authentication (CHAP), or want to leverage existing TCP/IP management tools. iSCSI is more feature-rich and widely supported.
• Consider hardware – AoE is natively supported in Linux and some BSDs, but Windows requires third-party initiators. iSCSI has built-in support in Windows, VMware, and most OSes.

For most modern deployments, iSCSI is more common because of its flexibility and security features. AoE remains a niche choice for dedicated high-performance storage networks.

Common Problems and Misconceptions

One common misconception is that AoE is “like iSCSI but without TCP.� While true, that also means AoE lacks congestion control, retransmission, and error recovery. If a frame is lost, the host must time out and retry, which can cause performance drops on unreliable links. Another issue is that AoE frames cannot cross routers, so you cannot extend an AoE SAN across subnets without layer-2 tunneling.

Another problem is security: AoE has no authentication, so any host on the network can read or write to a target. Always place AoE on a separate VLAN or physically isolated network. Also, some switches may block EtherType 0x88A2 or handle jumbo frames inconsistently, so verify your switch supports the frame sizes you use.

Costs and Alternatives

AoE is free and open-source (the spec is public, and Linux has a built-in driver). The main cost is hardware: you need a dedicated Ethernet network (preferably 10GbE) and storage targets (which can be software-defined using Linux + AoE target software). Alternatives include:

• iSCSI – more features, but higher CPU overhead.
• Fibre Channel – dedicated hardware, high cost, highest performance.
• NFS/SMB – file-level access, not block-level, but easier to manage.
• NVMe-oF – modern protocol for NVMe over fabrics, lower latency than AoE.

For most users, iSCSI is the practical choice. AoE is best for specialized low-overhead SANs in a controlled environment.

Safety and Best Practices

While AoE itself is not dangerous, misconfiguration can cause data loss. Always ensure that only authorized hosts can access targets. Use VLANs or dedicated switches to isolate AoE traffic. When testing, use non-production data. Also, remember that AoE has no built-in error correction – rely on the underlying disk’s integrity and back up critical data.

Frequently Asked Questions

Can AoE work over Wi-Fi? Technically yes, but it is not recommended. Wi-Fi adds latency and packet loss, and AoE has no retransmission, so performance would be poor and errors likely.

Does AoE support multiple hosts accessing the same target? Yes, but you must use a cluster-aware filesystem (like GFS2 or OCFS2) if multiple hosts write simultaneously. Otherwise, use it as a single-host device.

What is the maximum size of an AoE frame? The standard Ethernet MTU is 1500 bytes, but jumbo frames up to 9000 bytes are common. The AoE spec allows up to 16 sectors (8 KB) per frame, but with jumbo frames you can transfer more per frame.

Is AoE still used today? Yes, but it is niche. It is used in some high-performance computing and embedded systems where simplicity and low latency are critical.

How do I set up AoE on Linux? Load the aoe kernel module, then use aoe-discover to find targets. The devices appear as /dev/etherd/eX.Y. For targets, use the vblade software or similar.

Related

• Keep Orca Agents Running Remotely: 6 VPS Options
• Satellite Messaging vs Cellular in Emergencies
• Tab Scheduler: When to Suggest Popular Tags

in t | Time to transfer (s) | s | 60 | 0
in d | Data size | GB | 100 | 0 | 100000
in b | Network bandwidth | Gb/s | 1 | 0.1 | 100
out eff | Effective throughput | GB/s | (d * 8) / t | 3
out pct | Bandwidth used | % | ((d * 8) / (b * t)) * 100 | 1