[Planetlab-devel] advice on adding in-node emulation support ?

[Luigi Rizzo]

On Mon, Jun 01, 2009 at 10:03:39AM -0400, Marc Fiuczynski wrote:
> Hi Luigi and Martha,
> 
> I remember making a mental note about your really cool addition, but  
> didn't respond to your inquiry because I thought others would pipe  
> up.  Seems to me that it would be best to capture your work into at  
> least two SVN modules that result in at least two rpms.  One rpm for  
> root context related code and another to make it easy for folks to  
> install it in their slice.  We could package your new tools into the  
> vserver-reference image, but that would only be visible to new slices  
> -- old slice need to explicitly install those separately. Finally,  
> I'd like to see these rpms built and made available via a MyPLC rpm  
> repository
> 
> I guess before going further, can you update us on where you are now  
> with this work?
> 
> Marc

Hi all,
first of all, sorry for not answering earlier but we spent quite
some time tracking a nasty bug (kernel stack overflow, now fixed)
in our dummynet module for PlanetLab/OneLab nodes.

To summarise, and trying to capture your feedback so far we have
the following pieces (the code is all ready and mostly tested):

------- RPM #1 (root context) -----------------------------------------
  This RPM contains all the parts that should be visible
  to the root context. You can build them as described at the end
  of this email.  It contains:

    /lib/modules/2.6.22/ipfw_mod.ko
        the kernel module implementing ipfw and the firewall

    /sbin/ipfw
        the control program (binary), talking to the kernel
        via a setsockopt. Needs to be run by the root context only.

    /etc/cron.d/ipfw.cron
        crond configuration to call the cleanup script below and
        remove expired entries from the emulator's configuration.

    /usr/bin/ipfw-cleanup
        This is a script that should be run either periodically
        (say every 1..5 minutes) or on demand, to remove expired
        entries from the emulator configuration.
        It should be run in the root context only.
-----------------------------------------------------------------------

---- VSYS STUFF -------------------------------------------------------
  These two pieces are part of the vsys backend used to configure
  the emulation:

    /vsys/ipfw-be       http://XXX
        The script implementing the backend for the (vsys-based)
        configuration of the emulation. Run in the root context only.

    /vsys/ipfw-be.acl (PROBABLY UNNECESSARY, SEE BELOW)
        the acl file for the ipfw-be vsys component.
For the ACL, we found that there is a NodeManager plugin (vsys.py)
that can be used to periodically update the vsys.conf on the nodes,
reflecting the slices that are instantiated on the node.

To this purpose, at least so it seems, a slices XXX who wants to use
the service YYY add (or are created with) a tag 'vsys=YYY'
and the plugin will automatically add slice XXX to /vsys/YYY.acl
at the next round.
So for newly created slices we can force this tag, and for
existing users we can run a pass on the database to add the tag.
-----------------------------------------------------------------------

---- RPM #2 (for users to install) ------------------------------------
    /usr/sbin/netconfig
        The frontend for the (vsys-based) program for the configuration
        of the emulation. This is a shell script that
        should be available by default to all slices (and normally
        run in the slice context).

If there are things that are installed on a node and made available
to users (I guess there must be a minimal set of things), then we
could add this RPM to the set of components that are installed
by default. Is this the "vserver reference" that Mark was mentioning ?
-----------------------------------------------------------------------

-----------------------------------------------------------------------
INSTRUCTIONS TO BUILD THE RPM #1:

1. prepare a build system, as described in the wiki
   https://svn.planet-lab.org/wiki/VserverCentos

2. edit onelab.mk and add the following lines:

#
# ipfw
#
ipfw-MODULES := ipfw
ipfw-SPEC := ipfw.spec
ipfw-DEPEND-DEVEL-RPMS := kernel-devel
ipfw-SPECVARS = kernel_version=$(kernel.rpm-version) \
        kernel_release=$(kernel.rpm-release) \
        kernel_arch=$(kernel.rpm-arch)
IN_BOOTSTRAPFS += ipfw
ALL += ipfw


3. edit onelab-tags.mk and add the following line

ipfw-SVNPATH            := http://onelab1.iet.unipi.it/svn/ipfw-stable/
-----------------------------------------------------------------------

Makes sense ?

	cheers
	luigi and marta