[OLSR-users] Problems when losing one link in a dual link setup (long mail)

[Andreas Tønnesen]

Roar,

One thing to try would be to change the lines of code found in
src/process_routes.c line 222 from:
   olsr_add_routes_in_kernel(add_kernel_list);
   olsr_delete_routes_from_kernel(delete_kernel_list);
to:
   olsr_delete_routes_from_kernel(delete_kernel_list);
   olsr_add_routes_in_kernel(add_kernel_list);

Adding new routes prior to deleting the old ones was added as a 
optimization some time ago - it turned out it only worked in Linux, but 
I suspect it might cause trouble there as well... this is probably why
you see stuff like:
 > (ioctl) Adding route: 10.10.10.3 gw 10.10.10.3 (hopc 1)
 > (ioctl) Adding route: 192.168.100.3 gw 10.10.10.3 (hopc 1)
 > (ioctl) Adding route: 10.10.10.4 gw 10.10.10.3 (hopc 2)
 > (ioctl) Deleting route: 10.10.10.4 (hopc 3)
 > (ioctl) Deleting route: 10.10.10.3 (hopc 2)
 > (ioctl) Deleting route: 192.168.100.3 (hopc 2)

If that does not work it would be great if you could provide us with 
debuglevel 9 output from the same parts of logging that you sent in your 
previous mail.

- Andreas

Roar Bjørgum Rotvik wrote:
> Hello,
> 
> I have some problems with a multi link setup that I hope someone may 
> answer :)
> 
> Setup:
> I'm testing a setup with 4 nodes A-B-C-D, where all nodes has WLAN
> (eth1). In addition B and C has an Ethernet wire between them (eth0).
> 
> Addresses:
> A: eth1: 10.10.10.1
> B: eth0: 192.168.100.2, eth1: 10.10.10.2
> C: eth0: 192.168.100.3, eth1: 10.10.10.3
> D: eth1: 10.10.10.4
> 
> Both nodes have default olsrd.conf, with the following interface config:
> Interface "eth1"
> {
> }
> Interface "eth0"
> {
> }
> 
> I.e. no special configuration, just defining the interfaces. Olsrd is
> started with no options and "Debug 1" in the config file.
> 
> During startup on B and C I see the following:
> B: Using interface eth1 10.10.10.2 as main interface
> C: Using interface eth1 10.10.10.3 as main interface
> 
> Question:
> I also see the following after startup:
> B: (after discovering C on both interfaces):
> "Adding MID alias main 10.10.10.3 -> 192.168.100.3 based on HELLO"
> "Inserting alias 192.168.100.3 for 10.10.10.3"
> C: (after discovering B on both interfaces):
> "Adding MID alias main 10.10.10.2 -> 192.168.100.2 based on HELLO"
> "Inserting alias 192.168.100.2 for 10.10.10.2"
> 
> I guess that this means that B discovers that both 10.10.10.3 and
> 192.168.100.3 belongs with node C and therefore uses 192.168.100.3 as an 
> alias for 10.10.10.3.
> 
> But then I see some problems after letting the system settle (after 
> starting all the nodes):
> Routing table on B:
> Destination        GW        Iface
> 10.10.10.1        0.0.0.0        eth1
> 192.168.100.3        0.0.0.0        eth0
> 10.10.10.4        192.168.100.3    eth0
> 
> Routing table on C:
> Destination        GW        Iface
> 10.10.10.1        192.168.100.2    eth1
> 192.168.100.2        0.0.0.0        eth0
> 10.10.10.4        0.0.0.0        eth1
> 
> The first problem I see is there is no route over WLAN interfaces 
> between B and C (B has no route to 10.10.10.3 and C has no route to 
> 10.10.10.2).
> I guess the reason the LAN-link is chosen between B and C is that 
> 192.168.100.x is an alias for 10.10.10.x on both nodes as shown earlier?
> 
> But still there should have been a route entry for the other interfaces, 
> preferably with another priority or weight, since we prefer LAN over 
> WLAN if available.
> But there may exist applications that need to address C on both 
> IP-addresses (WLAN/LAN) from B and the route should be present.
> The solution could be as easy (?) as to add this route on B (and similar 
> on C):
> 10.10.10.3    192.168.100.3        eth0
> 
> Then we force access to WLAN addresses between B and C to use the 
> LAN-link if available. At least a direct routing should be present.
> 
> Second problem is what happens when I pull out the LAN cable.
> The routing tables changes like this (waiting a while before printing 
> routing table):
> Routing table on B:
> Destination        GW        Iface
> 10.10.10.3        0.0.0.0        eth0
>                     ^^^^
> 10.10.10.1        0.0.0.0        eth1
> 192.168.100.3        0.0.0.0        eth0
> 10.10.10.4        10.10.10.3    eth0
>                     ^^^^
> Routing table on C:
> Destination        GW        Iface
> 10.10.10.2        0.0.0.0        eth0
>                     ^^^^
> 10.10.10.1        10.10.10.2    eth0
>                     ^^^^
> 192.168.100.2        0.0.0.0        eth0
> 10.10.10.4        0.0.0.0        eth1
> 
> Problems here:
> a) Between B and C new routes for the WLAN IP addresses is now added 
> (those I missed in the first place), but check the interface used!
> It is defined to use eth0 (LAN) interface (that is not working now) 
> instead of the eth1 interface for the current IP address.
> The interface for the 2-hop neighbors (10.10.10.4 on B and 10.10.10.1 on 
> C) is therefore also using the wrong interface.
> 
> b) The LAN routes are still present..
> The route 192.168.100.3 on B (and 192.168.100.2 on C) is no longer valid 
> and should have been removed.
> 
>  From the debug-console (debug level 1) on B we can see this when the 
> cable is pulled out (handwritten, so it may not be verbatim correct):
> [192.168.100.3] HELLO timeout
> [192.168.100.3] HELLO timeout
> [192.168.100.3] HELLO timeout
> Setting 10.10.10.3 as MPR
> (ioctl) Adding route: 10.10.10.3 gw 10.10.10.3 (hopc 1)
> (ioctl) Adding route: 192.168.100.3 gw 10.10.10.3 (hopc 1)
> (ioctl) Adding route: 10.10.10.4 gw 10.10.10.3 (hopc 2)
> <so far so good, this is what we want, except for 10.10.10.3 using eth0 
> as shown earlier, then this happen:>
> (ioctl) Deleting route: 10.10.10.4 (hopc 3)
> (ioctl) Deleting route: 10.10.10.3 (hopc 2)
> (ioctl) Deleting route: 192.168.100.3 (hopc 2)
> 
> Why does olsrd first install correct links and then remove the links 
> after that? I also see that the hopc in the deletion seems wrong (one 
> more than real hop count...).
> But as the routing table after pulling the cable shows this "Deleting 
> route" didn't seem to do anything wrong..
> 
> Then I insert the cable again and wait some more to let the links 
> stabilize.
> Routing table on B:
> Destination        GW        Iface
> 10.10.10.3        0.0.0.0        eth0
> 10.10.10.1        0.0.0.0        eth1
> 192.168.100.3        0.0.0.0        eth0
> 10.10.10.4        192.168.100.3    eth0
> 
> Routing table on C:
> Destination        GW        Iface
> 10.10.10.2        0.0.0.0        eth0
> 10.10.10.1        10.10.10.2    eth0
> 192.168.100.2        0.0.0.0        eth0
> 10.10.10.4        0.0.0.0        eth1
> 
> This is not the routing table as when olsrd started with the LAN cable 
> plugged in. Now eth1 from B and C is in the routing table, but still 
> using the LAN-interface (eth0) to communicate.
> Another strange thing is that B changed the routing for 10.10.10.4 to 
> use 192.168.100.3 as GW, while C continues to use 10.10.10.2 as GW to 
> 10.10.10.1 as when the cable was unplugged.
> 
> On B this is also shown in the debug trace (C didn't have anything extra 
> than detecting link changes):
> (ioctl) Adding route: 10.10.10.3 gw 192.168.100.3 (hopc 1)
> (ioctl) Adding route: 192.168.100.3 gw 192.168.100.3 (hopc 1)
>     File exists
> (ioctl) Adding route: 192.168.100.3 gw 192.168.100.3 (hopc 1)
>     File exists
> (ioctl) Adding route: 10.10.10.4 gw 192.168.100.3 (hopc 2)
> (ioctl) Deleting route: 10.10.10.4 (hopc 3)
> (ioctl) Deleting route: 10.10.10.3 (hopc 2)
> (ioctl) Deleting route: 192.168.100.3 (hopc 2)
> Delete route: No such process
> (ioctl) Deleting route: 192.168.100.3 (hopc 2)
> Delete route: No such process
> 
> I can give more info if needed (more debug info, change parameters and 
> so on).
> 

-- 
Andreas Tønnesen
http://www.olsr.org