Previous

Content

 
5.0.- One TCP flow and One UDP flow  

Google
Warning: this page has a bundle of png and gif pictures; it will last a little if you have an slow Internet connection.
Continuing with our study we will see now how one TCP flow and one UDP flow compete for network resources. The topology will be the same we used before but having two sources; one for TCP and another for UDP.

 
In both hosts we have TCP and UDP. Host A has a FTP source that always has a packet to send and a CBR source (Constant Bandwidth Rate) that sends packets at a fixed rate of 0.95 Mbps. . The packets are 1000-byte size. Host B is in charge to acknowledging TCP packets and destroying UDP packets it receives. Ack packets are 40-byte size. The link is a 2 Mbps link having a propagation delay of 10 milliseconds. The buffer size is 20 packets, then our queue can have a maximum of 20 packets. The queue is DropTail (just a FIFO queue). Most of the actual routers and hosts in the Internet use this kind of queue. The TCP implementation is Reno.
 
The tcl code to feed the ns-2 simulator is in tcp+udp.tcl.
 
At time 0.1 seconds the FTP and CBR sources in A start to send packets, and at time 10.0 seconds both stop to send packets.
 
To get information from the simulation we use, besides the ns-2 simulator, the awk program to parse the ns-2 output, and the gnuplot utility to plot the results.
 
For each test we are going to get the following information:
 
  1. Instant throughput in kbps
  2. System throughput in kbps
  3. Latency in ms
  4. Jitter in ms
  5. Bytes trasferred
  6. Losses
To filter noises we use EWMA (Exponential Weighted Moving Average) to smooth the outputs using α = 0.9, before ploting.
 
The first graph obtained from the test is instant throughput in kbps:
 
   

 

It's a good idea to have a look to this scenario when one TCP flow or one UDP flow is being transmitted. TCP and UDP interaction generates oscillation in both flows when they are competing for network resources. As you see the share is almost fair with each flow trying to get 50% of the available bandwidth. What really occurs is that CBR sources was set at 0.95Mbps and it tries to keep its constant rate through UDP, and TCP, on the contrary, tries to use for itself the rest of the available bandwidth, just 1.05Mbps. Then shares are very close to 50-50%.
But competition is unavoidable. When TCP backs off, UDP takes advantage and tries to control the available bandwidth; a peak results then. But inmediately TCP tries to test network resources increasing its bandwidth consumption and UDP cedes and enters in a valley. This cycle repeats again and again.
Let's see now the system throughput. Exactly as before, the instant throughput is calculated dividing the bytes of each packet (1000 bytes) by the time required to send the packet. The system throughput is calculated dividing the total bytes sent up to a given time by the total time transcurred in seconds. The graph is:

Here we can see better the loss episode in the TCP life. But finally both flows accomodate well and each one takes about 50% of the total available bandwidth. Despite some oscillation performance is very acceptable.
Let's see how latency goes for this scenario; here we have the graph:

Well, this variable suffers a lot. For one TCP flow the latency was close to 73ms and for one UDP flow it was 14ms. Now it is like we got before when two TCP flows were competing. UDP was harmed because even with two UDP flows its latency never was higher than 18ms. If you compare TCP behavior with two competing flows with this scenario, we have to come to a conclusion that UDP carries the worst thing in its interaction with TCP. It lost more.
Let's check jitter; the graph is this:

We have a large dispersion this time. Even worst than when two TCP flows are competing.
Now is time to see what happen with total bytes transferred; for TCP we have:

KB sent: 1150.742188
And for UDP:

KB sent: 1148.437500
Total bytes transferred are then:

KB sent: 2299.180288
For UDP observe that total bytes transferred is exactly the same than before when two UDP flows were tested. We don't have in this case de-rating for UDP.  TCP is almost the same too. When two TCP flows were competing in this scenario partial bytes transferred were 1109KB and 1069KB. Now we have 1150KB, even better. Great loser here is then UDP because its mean latency and jitter were seriously damaged.
Finally let's check losses; for TCP we have:

   Packets sent:  1158
Packets dropped:    24
   Total losses:     2.072539%
Very bad; this result is even worst than we got before when testing two TCP flows.
For UDP we have:

   Packets sent:  1176
Packets dropped:     0
   Total losses:     0.000000%
Eureka!! Exactly the same response than before with two UDP flows competing. In this new competition TCP lost more packets and UDP lost its very low latency and jittering. But I think UDP lost more. Because UDP is designed to be a very lightweight protocol for real-time traffic that is very sensible to high latencies and jitters.  Having higher values of these variables spoils UDP reason to be.
Okay, now we are going to make new tests selecting small packets of 100-byte size instead of packets of 1000-byte size. The instant throughput graph is:

New smaller packet size generates a denser oscillation. However, observe that again the bandwidth distribution is almost fair. Each protocol takes about 50% of the available bandwidth. Let's verify the system throughput:

Bandwidth distribution is very fair. UDP takes its predetermined fixed rate of 0.95Mbps and TCP takes all left available bandwidth, i.e., 1.05Mbps. Observe also that heavy TCP loss episode we saw before with 1000-byte size packets is not present this time. Smaller packets avoid serious conflicts and perhaps, we will check this later looking at the losses result, TCP saves itself now from having some dropped packets.
It's time to see latency:

Well, a great improve using these small packets. 11.5ms is good enough even for real-time traffic. Also latency is very stable then we can expect a very, very low jitter. Let's check:

Really very good. Packet size reduction helps lowering significantly delay response. Let's check now the number of bytes transferred. For TCP flow we have:

KB sent: 1259.902344
And for UDP:

KB sent: 1148.144531
Total bytes transferred are then:

KB sent: 2408.046875
Okay. UDP bytes transferred are almost the same but we've got a better TCP response, just 9.48% more bytes transferred. Moving smaller size packets helps TCP to avoid a loss episode that affect its performance.
Let's finish these tests checking losses. Here we have them for TCP flow:

   Packets sent:  9216
Packets dropped:     0
   Total losses:     0.000000%
Observe that packet size reduction dropped to zero TCP losses. Let's see if this improved behavior affected UDP negatively; losses in this case are:

   Packets sent: 11757
Packets dropped:     0
   Total losses:     0.000000%
Well, no. UDP maintains its good performance with no losses at all.
Okay, fellows. We have seen how reducing the packet size has been beneficial for TCP response having no effects on UDP behavior. We have to suppose that increasing the packet size will be terrible, at least, for TCP. Let's check now this scenario. Next graph is instant throughput for 4000-byte size packets:

It seems that both flows are having now a bad time. The graph shows even a worst scenario when it is compared with two TCP flows competing for network resources. Let's see system throughput:

At a glance it seems that both flows are having losses now. We will check this below having a look to the loss results. Latency should be terrible in this scenario; the graph is this:

It's true. It climbed up almost to 350ms. And now we have jittering. The graph for jitter is:

We have a band reaching up to 30-50ms. It's proved again than having bigger packets more latency and jittering is induced.
Let's see now the number of bytes transferred; for TCP we have

KB sent: 1286.210938
More bytes transferred but probably we have losses in this scenario; for UDP we have:

KB sent: 1101.562500
UDP suffered more than TCP. Number of bytes transferred are now 4.09% less. Total bytes transferred are then:

KB sent: 2387.773438
Number of bytes transferred is a little bit more than when using 1000-byte size packets. We have an increase of 3.85%. Despite of the observable loss episodes, using larger size packets we improve our system throughput.
Finally, let's check losses in each flow; for TCP we have:

   Packets sent:   334
Packets dropped:     7
   Total losses:     2.095808%
and for UDP:

   Packets sent:   294
Packets dropped:    12
   Total losses:     4.081633%
As you see UDP took the worst piece of the pie in this sharing.
Next step in our investigation will be to see what happen when the link bandwidth is reduced having the two flows competing for resources. How TCP will react? UDP will be receiving packets almost to the link bandwidth capacity practically leaving no availability for TCP. Will TCP support this scenario?  We will adjust the link bandwidth to 1 Mbps instead of the original value of 2 Mbps. The propagation delay will be kept in 10ms. Here we have the instant throughput for this case:

TCP can't support this scenario; its starvation is clearly observable. It tries to recover but falling is unavoidable. Let's see system throughput:

In this graph TCP starvation is better observed. Final throughput is in the 100kbps range. It can't fight against UDP to preserve some significant share of the total available bandwidth.
Let's see how latency behaves for this environment:

Both seriously spoiled. UDP is in the range of 200-240ms, some really unacceptable for real-time traffic over UDP. TCP is even worst. It has some episodes with packets delayed up to 4 seconds and even more.
Jittering should reflect this situation; let's have a look to it:

Some week dispersion but having deviations up to +/- 80ms.
Next we will check the total bytes transferred for each flow; for TCP we have:

KB sent: 92.460938
Really terrible for TCP. UDP starved it to having less than 10% of the total available bandwidth.
For UDP we have:

KB sent: 1133.789062
UDP almost maintained its original throughput (having a 2Mbps link) of 1148 KB sent, a reduction of only 1.31%.
Let's see what about with losses; for TCP we have:

   Packets sent:   120
Packets dropped:    28
   Total losses:    23.333333%
Packets sent suffered a reduction of 89.64% from 1158 packets to just 120 packets but indeed number of dropped packets climbs to an unacceptable value of 23.33%.
For UDP we have:

  Packets sent:  1176
Packets dropped:    15
   Total losses:     1.275510%
In this case number of packets sent are the same but we have small losses of 1.28%.
These tests show us something really very important. When we have TCP and UDP flows competing for network resources and UDP covers or almost covers the available bandwidth, then TCP flows are seriously starved not finding enough space to control some reasonable share of the resources. Again, we have to be sure when dealing with UDP transporting constant bit rate flows, that available bandwidth should be good enough to support the UDP flows plus good enough again to allow TCP flows to find some place in the party.
 
Okay, dear lectors, following with our study let's check now what happen if we increase the buffer size in the link. Our buffer is actually 20 packets and we will test with a new scenario of 50 packets?  The instant throughput is:

 
   

 
Well, again some additional buffering serves to relieve pressure against traffic. Now curves are soften and less oscillation is present. Share is fair having UDP its 0.95Mbps and TCP the available bandwidth left.
Let's check the system throughput. The graph is:

TCP has better performance in this scenario. The heavy loss episode when flow was starting using 20-packets buffering has disappeared. Let's check if latency was hardly affected; the graph is:

Well, latency is a little higher now but very stable. Let's check queues for both scenarios; for 20-packets buffering is:

It seems that sometimes total number of packets are very close to the limit. Let's check the total queue (both flows):

Yes, very close. Buffer is almost full most of the time. Let's check now queues for 50-packets buffereing:

Now. again, response is soften. But total queue reaches almost 30-packets. This means that original 20-packets buffering was not good enough for our scenario. Let's check total queue (both flows):

Very nice. Buffer is filled most of the time with 32-33 packets. Minimum buffering should be at least 35 packets for this scenario (two flows, TCP + UDP).
Of course, having a longer queue implies a higher latency. But in this case is better because we have less pressure against the flows, perhaps no losses (we will check this below) and an improved response at all.
Jitter should be almost zero because latency is very stable; the graph is:

Small peaks and valleys in the latency graph above are responsible for this jittering that oscillate between +/- 4ms. Nevertheless, this behavior is a little bit better than it was obtained using a 20-packets buffer. Extra-buffering helps to have an smoothed response.
Okay, it's time to check the number of bytes transferred; for TCP we have:

KB sent: 1298.007812
Very well!! An improve of almost 13% when comparing with a 20-packets buffer test. Just again having the right buffer helps to have even an improved throughput.
For UDP we have:

KB sent: 1148.437500
Exactly the same response we got before using an smaller buffer. We can conclude that mainly for TCP a right buffering selection is very important to have a better tunned system.
Finally let's check losses for this environment; for TCP we have:

   Packets sent:  1279
Packets dropped:     0
   Total losses:     0.000000%
Definitively TCP took advantage from the higher buffering. Now it sent more packets (10.45% more) and it didn't have losses. For UDP we have:

   Packets sent:  1176
Packets dropped:     0
   Total losses:     0.000000%
Same number of packets sent with no losses again.
Okay, fellows; it has been proved that selecting a right buffering is very important to get as much as is possible from your system; then, it is self-evident that having the opposite, i.e., not enough buffering should be hardly bad for the system; let's check this. Using an under-buffering scenario of 5-packets we have the following instant throughput graph:

First, we have less output. Using 20-packets buffer peaks reach up to 1.8Mbps. Now they have a top of about 1.25Mbps. However share "seems" as being fair as before. Let's see the system throughput:

Well, share really is not fair. Soften with EWMA tends to generate some distorsion of the reality when reading instant throughput, but system throughput gets back us to the real scenario. TCP has been starved again, now not as severely as was before when having a 1 Mbps link but phenomenon is clearly observable. Final TCP bandwidth oscillates about 250Kbps, i.e., 25% of what it should be. On the contrary UDP kepts its share. Again as before, reducing the queue's buffers we have a less fair environment, this time affecting more severely to TCP.
Let's have a look to latency for this scenario:

Latency is lower than it was when using a 20 packets buffer. It is also more stable. Very similar to what we got before with two TCP flows competing. Packets are now having to make less queue in buffers. Let's check jitter this time:

Some dispersion but centered on zero axis. We will check below how having this little improve in latency and jittering will affect bytes transferred and losses. But let's check previously our queues. Do you remember the 20-packets buffer queue? It was:

Both flows oscillates about 6 packets. Now that we have only 5 packets as buffering how will this graph look; let's see:

Queues are now very constrained and each flow seems to have no more than one packet in queue. Let's check how all this affect number of bytes transferred. For TCP we have:

KB sent: 282.382812
Terrible!! This value is 75.48% less than we got using a 20-packets buffer. TCP was hardly starved in this scenario. For UDP we have:

KB sent: 1148.437500
On the contrary UDP have maintained exactly the same number of bytes transferred. Reducing the buffer only affected TCP behavior. Observe that total bytes transferred are now:

KB sent: 1430.8203
And we have now our link bandwidth under utilized.
Finally let's check losses; for TCP we have:

   Packets sent:   304
Packets dropped:    25
   Total losses:     8.223684%
We have not only a lot of less packets sent (1158 packets before vs 304 packets now) but losses increase from 2% to 8%. A very bad response for TCP. For UDP we have:

   Packets sent:  1176
Packets dropped:     0
   Total losses:     0.000000%
No de-rating for UDP. Same number of packets with no losses.
TCP starving is the main decision here. Buffer reduction was not convenient because marginal latency and jittering reduction is unjustifiable. Then, as long as we are talking about TCP it is better to have enough buffering to avoid starving. Issue is then trying to find the right choice to have enough buffering without increasing too much latency and jittering response.
Our last test will be to see what happen when we select a propagation delay of 100ms instead of 10ms, almost a ten times longer link. The instant throughput is:

We have an scenary very similar to what we got before using a 50-packets buffer. Oscillations are soften now. Something we can expect because having a longer link is like having more buffering available for packet allocation. But beware, let's check system throughput to be sure EMWA is not making some tricks; here we have:

Okay, but just again, TCP is carrying with the worst behavior. We saw before that TCP is more sensible to longer links than UDP showing for these scenarios more oscillations and a poorer response. On the contrary UDP seems to be as being in home.
Let's see latency:

Well, at least 100ms of this is due to the link propagation delay. There is more dispersion now. We can suppose as before than the queue was translated from the buffer to the link. Let's have a look to the queue graph:

Confirmed!! Now the queue is very low. Packets are now traveling very happy throughout the link. Let's have a look to the jitter to see how this dispersion influence its behavior:
 
Well, the response is supportable. +/-5ms but most of packets are very close to zero.
 
Let's check now the total bytes transferred; for TCP we have:
 

KB sent: 904.960938
 
A reduction of about 21.4% comparing with what we got before using a 10ms link. For UDP we have:
 

KB sent: 1148.437500
 
Well, UDP is not well aware of what is happening. It maintains its same response. Having a shorter or a longer link is not a matter for this protocol. TCP, on the contrary, has to be treated with higher care because its response is very influenced by the link length.
 
   

 
Okay, it's time to check losses. For TCP we have:

Packets sent:   892
Packets dropped:     0
   Total losses:     0.000000%
Okay, no losses because longer link implies a higher buffer to receive packets and soften oscillations but paying with a reduction of about 22.4% in the number of packets sent with TCP. For UDP we have:

   Packets sent:  1176
Packets dropped:     0
   Total losses:     0.000000%
UDP is very fresh drinking a glass of delicious juice because its response is exactly the same. Same number of packets with no losses.
It's time to repeat again what we said before and before. Conclusions are the same, i.e.:
Comparing both protocols, UDP surpassed in most of the tests to TCP.  TCP is deeply affected by many factors like available bandwidth, network congestion, buffering availability, link propagation delay,  packet size, interaction with other flows and protocols, bad luck at startup,  etc., etc.  UDP is stronger because it is simple and of constant rate by nature.  Problem to select UDP instead of TCP as a viable Internet transport protocol is how to control losses when the network get congested.
If we could have a controlled network where Call Admission Control (CAC) could be implemented, and if we could shape UDP traffic to be sure it respects the initial solicited bandwidth, then UDP would be a very nice solution for traffic transport. But, because this ideal conditions can't be implemented yet in Internet, and being the main goal to protect the health of the network against congestion disease, the only solution is to run it with the safer TCP.
Okay, that's all folks. Hoping this document can be of some usefulness for you in your daily work.
 

 
Previous

Content