[Planetlab-users] Re: [Planetlab-devel] Network virtualization

[Sapan Bhatia]

Hello Daniel,

Here's a summary of the results. Please feel free to contact me for any
questions or clarifications.
The experiments were run between two 3GHz Xeons with 4 megabytes of cache.

Sapan

----

The evaluation strategy used is to compare the effective performance
of containers running with their own copy of the protocol stack with
that of processes on native Linux. Any significant deviation should
imply a major overhead in the implementation. Specifically, we run
the following steps:

(i) We start with 1 container pushing out or routing data at full
throttle and progressively increase the number of containers,
measuring the overall throughput, CPU utilization and jitter. This is
not a good microbenchmark because the length of a time slice allows
containers to send out a few dozen packets in every burst - meaning
that packet sends are not interleaved, and there is no added
contention in the stack. But it's a good macrobenchmark, because this
is what we'd like to do in the context of PlanetLab (run many slices
at once). The only additional thing it brings into play is the
context-switch overhead.

Results:
- There is no degradation in throughput compared to native Linux even
when we drop the packet size from 1024->256->64b. This is surprising
- so I double checked that the sessions were not bottlenecked on the
receiver (which uses a kernel-mode packet accounter or because of
Ethernet flow control, which was disabled).

- In general, jitter degrades drastically beyond about 20 concurrent
containers or processes (i.e., this happens on native Linux as well).
The degradation happens sooner with containers and is slightly higher
(about 10-15%). At first, it seemed that this might be the 'visible'
performance impact of using containers, but it turned out to be an
effect of (poor) scheduling.

Jitter degrades with concurrency because of the difference in packet
delays across and between context switches. As long as processes are
scheduled more-or-less in a round-robin order, it is proportional to
the concurrency. However, with heavy loads, some processes start with
a disadvantage because they get preempted frequently at launch time,
and this effect ripples through till things stablize. It takes
between 30 seconds and 7 minutes to stablize. This time is
independent of the implementation and is a function of the
concurrency (and HZ).

Jitter is slightly worse for containers because the context-switch
overhead is higher - which is because the cache footprint is higher.

This (unstable scheduling) effect should hopefully go away, as Marc
suggested, with a better scheduler (eg. the one used for PlanetLab).

- CPU utilization varies considerably. We measure the concurrency at
which the CPU utilization hits the ceiling.

Results: For native Linux - 19
. NetNS - 1 (surprising, I mailed Eric
about this and after exchanging a few messages, he was satisfied but
said that things would improve. Also note that this is his first release,
and to the best of my knowledge the first time somebody other than
him has used his code). OpenVZ - 7.

The increase in CPU utilization from 18-19 and 6-7 happens in a giant
leap (from 45% or so to 100%). It also corresponds to a jump in the
no. of L2 cache misses (which shoot up at this point) suggesting that
the cache footprint of all the processes now challenge the L2 cache.

=>This number is a measure of the cache footprint of the
implementation rather than a measure of the actual overhead.

This is a preliminary set of experiments which I'll enhance and rerun
continually with
more complex setups.





On 2/20/07, Daniel Lezcano <dlezcano at fr.ibm.com> wrote:
>
> Hi all,
>
> I heard some of you did some benchmarking on the network virtualization
> provided by OpenVZ and by the Eric's Biederman patchset. Is it possible
> to have more informations on the results ?
>
> Thanks in advance
>
>         -- Daniel
>
> _______________________________________________
> Devel mailing list
> Devel at lists.planet-lab.org
> https://lists.planet-lab.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.planet-lab.org/pipermail/users/attachments/20070220/ab568703/attachment.html