Fwd: [Planetlab-users] Networking-related issues

Sun Aug 3 19:40:53 EDT 2008

Hi Jawwad,

This was my response to Cristian and Randolph, who apparently had a similar
problem to what I think you're facing. They made this change, and it fixed
their problem

---------- Forwarded message ----------
From: Sapan Bhatia <sapanb at cs.princeton.edu>
Date: Thu, Jul 31, 2008 at 4:36 PM
Subject: Re: [Planetlab-users] Networking-related issues
To: Cristian Lumezanu <lume at cs.umd.edu>
Cc: Randolph Baden <randofu at cs.umd.edu>

Hi Cristian,

Basically, packets in the Linux kernel are processed by a series of 'packet
handlers' that implement various packet-processing engines, e.g. TCP/IP, ARP
etc. One of these engines is the 'packet socket a.k.a PF_PACKET' interface.

Every incoming packet goes through the PF_PACKET interface. If it's an IP
packet, then it goes through the TCP/IP stack next.

For outgoing packets, there are two possibilites:
- If you use the PF_PACKET interface, then a packet is treated _only_ by the
PF_PACKET layer. This layer does not know anything about IP, it just writes
the packet to the buffer of the device driver.
- If you use the PF_INET family, then your packets are treated by the TCP/IP
stack and subsequently send to PF_PACKET.

Our tracking/multiplexing code -- the functionality that detects that a
returning packet corresponds to packet that was previously transmitted by a
slice -- is implemented in the TCP/IP stack. So in the first case for
outgoing packets, our system is blind to the fact that the returning packet
should be handed over to your slice. On the other hand, if you were to use
the PF_INET interface, then our kernel would create a connection record for
the echo request/response and the returning packet would get classified as
part of that connection.

Version 4.1 of PlanetLab implemented a lightweight TCP/IP stack within
packet sockets. Unfortunately, it did not support SMP and was hard to keep
up with the mainline kernel, so we got rid of it in 4.2.

The workaround for you in such a case is to replace the socket family in the
call to socket() from PF_PACKET to PF_INET. There will be some other
changes, but I expect them to be restricted to a few lines of code.

You can even use library forwarding to trick your application into using the
other interface. That way you won't have to modify pyxida.

Sapan

On Thu, Jul 31, 2008 at 4:20 PM, Cristian Lumezanu <lume at cs.umd.edu> wrote:

> Sorry, I am confused. We are sending raw packets and we are using the pcap
> library to capture the responses. We are not doing any connection tracking.
>
> Did you see any packets in the logs. Our tcpdump on princeton-7 shows many
> outgoing icmp requests to 8.4.80.20
>
> Cristian
>
>
> Sapan Bhatia wrote:
>
>> Ah, I see.
>>
>> Since you're using packet sockets to send data, the packets are not being
>> tracked. In order for this to work, you'll need to use the PF_INET
>> interfance for sending packets. I don't think this is going to be too
>> difficult. You can continue using packet sockets with recv().
>>
>> Unfortunately, I don't foresee 4.2 supporting connection tracking for this
>> interface.
>>
>> Sapan
>>
>> On Thu, Jul 31, 2008 at 3:46 PM, Sapan Bhatia <sapanb at cs.princeton.edu
>> >wrote:
>>
>>
>>
>>> Hi,
>>>
>>> The logging is still running, so you can generate the traffic now.
>>>
>>> Sapan
>>>
>>> On Thu, Jul 31, 2008 at 3:43 PM, Cristian Lumezanu <lume at cs.umd.edu
>>> >wrote:
>>>
>>>
>>>
>>>> Pyxida is sending pings at random intervals and it may be that it just
>>>> did
>>>> not send any at that point. We know for sure that it sends a lot of
>>>> pings
>>>> when it starts (we can also see them in tcpdump) so please set up the
>>>> log
>>>> again and then tell us when you have done so. We will start Pyxida then
>>>> and
>>>> the pings should show. Thanks.
>>>>
>>>> Cristian
>>>>
>>>> Sapan Bhatia wrote:
>>>>
>>>>
>>>>
>>>>> Hi,
>>>>>
>>>>> I don't see any return packets from these IPs.
>>>>>
>>>>> [root at planetlab-7 ~]# iptables -A INPUT -s 8.14.56.43 -j LOG
>>>>> [root at planetlab-7 ~]# dmesg
>>>>> [root at planetlab-7 ~]# dmesg
>>>>> [root at planetlab-7 ~]# iptables -A INPUT -s 8.4.80.20 -j LOG
>>>>>
>>>>> Similarly, I don't see any outgoing packets headed to these
>>>>> destinations.
>>>>>
>>>>> Sapan
>>>>>
>>>>>
>>>>> On Thu, Jul 31, 2008 at 2:37 PM, Randolph Baden <randofu at cs.umd.edu>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Pyxida is currently running on planetlab-7.  Among the IPs that it is
>>>>>> trying to ping are the following two:
>>>>>>
>>>>>> 8.14.56.43, and 8.4.80.20
>>>>>>
>>>>>> There are others, but hopefully that's enough to figure something out.
>>>>>>
>>>>>> On Thu, Jul 31, 2008 at 2:24 PM, Sapan Bhatia <
>>>>>> sapanb at cs.princeton.edu>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Actually, I think a better setup for debugging would be if I logged
>>>>>>> all
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> the
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> echo packets you're sending and see if they're getting tagged
>>>>>>> properly.
>>>>>>>
>>>>>>> If you can give me the address of the destination host, I can set up
>>>>>>> a
>>>>>>> logging rule on PLANETLAB-7. Then we can re-run your experiment to
>>>>>>> see
>>>>>>> what's happening.
>>>>>>>
>>>>>>> Sapan
>>>>>>>
>>>>>>> On Thu, Jul 31, 2008 at 2:01 PM, Randolph Baden <randofu at cs.umd.edu>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> This is just the strace output from running it for a few tens of
>>>>>>>
>>>>>>>
>>>>>>>> seconds, but it's pretty large.  I did this on
>>>>>>>> salt.planetlab.cs.umd.edu.
>>>>>>>>
>>>>>>>> I would try on planetlab-7.cs.princeton.edu, but strace is not
>>>>>>>> installed there.  This is the message yum spits out when I try to
>>>>>>>> install it:
>>>>>>>>
>>>>>>>> http://boot.planet-lab.org/install-rpms/3rdparty/repodata/repomd.xml
>>>>>>>> :
>>>>>>>> [Errno 14] HTTP Error 404: Not Found
>>>>>>>> Trying other mirror.
>>>>>>>> Error: Cannot retrieve repository metadata (repomd.xml) for
>>>>>>>> repository: ThirdParty. Please verify its path and try again
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Sapan Bhatia
>>>>>>> www.cs.princeton.edu/~sapanb <http://www.cs.princeton.edu/%7Esapanb><
>>>>>>> http://www.cs.princeton.edu/%7Esapanb>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>> --
>>> Sapan Bhatia
>>> www.cs.princeton.edu/~sapanb <http://www.cs.princeton.edu/%7Esapanb>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>

-- 
Sapan Bhatia
www.cs.princeton.edu/~sapanb <http://www.cs.princeton.edu/%7Esapanb>

-- 
Sapan Bhatia
www.cs.princeton.edu/~sapanb <http://www.cs.princeton.edu/%7Esapanb>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.planet-lab.org/pipermail/users/attachments/20080803/2630a8e6/attachment.html