[Planetlab-devel] Network virtualization

[Daniel Lezcano]

Hello Sapan,

thank you very much for your answer. It is a very interesting analysis.
A have a few questions :)

What is the link speed between the two machines ?
What were the benchmarking tools ?
What is the meaning of the Native Linux - 19, NetNS - 1, OpenVz - 7 ?
Do you have some graphics showing the throughput/cpu/jitter vs nb 
containers ?

I am very surprised to read the throughput is not impacted by the 
containers number while the CPU utilization is much more important. Do 
you have any explication on that ?

Thank you.
Regards.

   -- Daniel

Sapan Bhatia wrote:
> Hello Daniel,
>
> Here's a summary of the results. Please feel free to contact me for any
> questions or clarifications.
> The experiments were run between two 3GHz Xeons with 4 megabytes of 
> cache.
>
> Sapan
>
> ----
>
> The evaluation strategy used is to compare the effective performance
> of containers running with their own copy of the protocol stack with
> that of processes on native Linux. Any significant deviation should
> imply a major overhead in the implementation. Specifically, we run
> the following steps:
>
> (i) We start with 1 container pushing out or routing data at full
> throttle and progressively increase the number of containers,
> measuring the overall throughput, CPU utilization and jitter. This is
> not a good microbenchmark because the length of a time slice allows
> containers to send out a few dozen packets in every burst - meaning
> that packet sends are not interleaved, and there is no added
> contention in the stack. But it's a good macrobenchmark, because this
> is what we'd like to do in the context of PlanetLab (run many slices
> at once). The only additional thing it brings into play is the
> context-switch overhead.
>
> Results:
> - There is no degradation in throughput compared to native Linux even
> when we drop the packet size from 1024->256->64b. This is surprising
> - so I double checked that the sessions were not bottlenecked on the
> receiver (which uses a kernel-mode packet accounter or because of
> Ethernet flow control, which was disabled).
>
> - In general, jitter degrades drastically beyond about 20 concurrent
> containers or processes (i.e., this happens on native Linux as well).
> The degradation happens sooner with containers and is slightly higher
> (about 10-15%). At first, it seemed that this might be the 'visible'
> performance impact of using containers, but it turned out to be an
> effect of (poor) scheduling.
>
> Jitter degrades with concurrency because of the difference in packet
> delays across and between context switches. As long as processes are
> scheduled more-or-less in a round-robin order, it is proportional to
> the concurrency. However, with heavy loads, some processes start with
> a disadvantage because they get preempted frequently at launch time,
> and this effect ripples through till things stablize. It takes
> between 30 seconds and 7 minutes to stablize. This time is
> independent of the implementation and is a function of the
> concurrency (and HZ).
>
> Jitter is slightly worse for containers because the context-switch
> overhead is higher - which is because the cache footprint is higher.
>
> This (unstable scheduling) effect should hopefully go away, as Marc
> suggested, with a better scheduler (eg. the one used for PlanetLab).
>
> - CPU utilization varies considerably. We measure the concurrency at
> which the CPU utilization hits the ceiling.
>
> Results: For native Linux - 19
> . NetNS - 1 (surprising, I mailed Eric
> about this and after exchanging a few messages, he was satisfied but
> said that things would improve. Also note that this is his first release,
> and to the best of my knowledge the first time somebody other than
> him has used his code). OpenVZ - 7.
>
> The increase in CPU utilization from 18-19 and 6-7 happens in a giant
> leap (from 45% or so to 100%). It also corresponds to a jump in the
> no. of L2 cache misses (which shoot up at this point) suggesting that
> the cache footprint of all the processes now challenge the L2 cache.
>
> =>This number is a measure of the cache footprint of the
> implementation rather than a measure of the actual overhead.
>
> This is a preliminary set of experiments which I'll enhance and rerun
> continually with
> more complex setups.
>
>
>
>
>
> On 2/20/07, Daniel Lezcano <dlezcano at fr.ibm.com> wrote:
>>
>> Hi all,
>>
>> I heard some of you did some benchmarking on the network virtualization
>> provided by OpenVZ and by the Eric's Biederman patchset. Is it possible
>> to have more informations on the results ?
>>
>> Thanks in advance
>>
>>         -- Daniel
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at lists.planet-lab.org
>> https://lists.planet-lab.org/mailman/listinfo/devel
>>
>