Asymmetric Routing

A standard architecture for remote sites

When a remote site (and often a critical one) with many users is “connected” to the Data Center, there are very often 2 links that provide a 24/7 service guarantee to access the servers.

Depending on the company’s choice, the 2 links can be used in:

• Active/Active. Both links are used;

• One of the links is Active and the other is Backup.

In the first case, there are various rules for making the best use of the 2 links: IP parity, number of active sessions, link usage rate, choice of protocols, … In short, there’s an embarrassment of choice!!

In the second solution (Active/Backup), there is only one possible path, but the 2 links are not used optimally.

Let’s study the case of 2 linksActive/Active. Users complain of slowness when using applications on the DC or simply when sharing files under Windows.

Let’s start from the principle that the first analyses showed that the servers were not saturated, that the application metrics showed no alerts and that the user workstations are recent… So it must be the network!! The metrics provided by the ISP do not show any link saturation.

So, where’s the problem?

What is really going on with the TCP/IP exchanges between the user and a server? And what is the impact on response times for the latter?

To see and above all understand the data exchanges between the users (nearly 1000 in our case) and the DC, we installed a network probe on the remote site that allows us to see these exchanges.

First observation: over 1 day of analysis, the packet loss rate in IN and OUT is particularly high. Observation confirmed over several weeks of recording.

As a reminder: a loss rate of more than 3% is considered critical for the network. Between 1.5% and 3%, it already seems abnormal even on a WAN. We also note an important point: loss is no greater in one direction (OUT) than in the other (IN).

There are two possibilities:

• There is a single path between a user and the server:

The user will always take the same path for the duration of the exchanges with a server. The TCP/IP packets will take a single path and some don’t make it through in one direction.

• There are two possible paths for the same conversation:

First case, the data may take one path for a certain time for the same client TCP port, until the arrival of areset(RST). Then take another one for a new TCP port.

Second case, the FORWARD data take one path (red in the diagram) and the RETURN data (acknowledgment packet, for example ACK) take another path (purple in the diagram).

An analysisWiresharkwill allow us to dig into this point.

In this file .pcap, there is a change of equipment during a conversation between a client IP and a server.

First, the communication TCP/55173 to TCP/8080 takes place between:

Client ßà Server: HP (–:–:–:–:1c:b7) and the CISCO (–:–:–:–:d0:82)

That is, a single path.

Then we notice that for a similar flow TCP/55179 to TCP/8080:

Client to Server: HP (–:–:–:–:1c:b7) and the CISCO (–:–:–:–:d0:82)

Server to Client: CISCO (–:–:–:–:df:02) and the HP (–:–:–:–:81:21)

That is, 2 different paths: one for the outbound and the other for the return.

The fact that on the TCP/55179 connection there are two paths should not, in theory, cause any disruption during data exchanges. TCP is robust enough to compensate for network problems. But it turns out that we observed that for each asymmetric communication, whatever the conversations, the packets going through one of the two paths were largely retransmitted.

The reasons for this phenomenon can be many:

• path B is longer than path A: the ACKs arrive too late and the data is already retransmitted;

• path B is shared by the ISP with other clients, it is saturated;

• path B has a technical failure,…

But the consequences are very real.