[OLSR-users] Problems when losing one link in a dual link setup (long mail)

[Roar Bjørgum Rotvik]

Hello,

I have some problems with a multi link setup that I hope someone may 
answer :)

Setup:
I'm testing a setup with 4 nodes A-B-C-D, where all nodes has WLAN
(eth1). In addition B and C has an Ethernet wire between them (eth0).

Addresses:
A: eth1: 10.10.10.1
B: eth0: 192.168.100.2, eth1: 10.10.10.2
C: eth0: 192.168.100.3, eth1: 10.10.10.3
D: eth1: 10.10.10.4

Both nodes have default olsrd.conf, with the following interface config:
Interface "eth1"
{
}
Interface "eth0"
{
}

I.e. no special configuration, just defining the interfaces. Olsrd is
started with no options and "Debug 1" in the config file.

During startup on B and C I see the following:
B: Using interface eth1 10.10.10.2 as main interface
C: Using interface eth1 10.10.10.3 as main interface

Question:
I also see the following after startup:
B: (after discovering C on both interfaces):
"Adding MID alias main 10.10.10.3 -> 192.168.100.3 based on HELLO"
"Inserting alias 192.168.100.3 for 10.10.10.3"
C: (after discovering B on both interfaces):
"Adding MID alias main 10.10.10.2 -> 192.168.100.2 based on HELLO"
"Inserting alias 192.168.100.2 for 10.10.10.2"

I guess that this means that B discovers that both 10.10.10.3 and
192.168.100.3 belongs with node C and therefore uses 192.168.100.3 as an 
alias for 10.10.10.3.

But then I see some problems after letting the system settle (after 
starting all the nodes):
Routing table on B:
Destination		GW		Iface
10.10.10.1		0.0.0.0		eth1
192.168.100.3		0.0.0.0		eth0
10.10.10.4		192.168.100.3	eth0

Routing table on C:
Destination		GW		Iface
10.10.10.1		192.168.100.2	eth1
192.168.100.2		0.0.0.0		eth0
10.10.10.4		0.0.0.0		eth1

The first problem I see is there is no route over WLAN interfaces 
between B and C (B has no route to 10.10.10.3 and C has no route to 
10.10.10.2).
I guess the reason the LAN-link is chosen between B and C is that 
192.168.100.x is an alias for 10.10.10.x on both nodes as shown earlier?

But still there should have been a route entry for the other interfaces, 
preferably with another priority or weight, since we prefer LAN over 
WLAN if available.
But there may exist applications that need to address C on both 
IP-addresses (WLAN/LAN) from B and the route should be present.
The solution could be as easy (?) as to add this route on B (and similar 
on C):
10.10.10.3	192.168.100.3		eth0

Then we force access to WLAN addresses between B and C to use the 
LAN-link if available. At least a direct routing should be present.

Second problem is what happens when I pull out the LAN cable.
The routing tables changes like this (waiting a while before printing 
routing table):
Routing table on B:
Destination		GW		Iface
10.10.10.3		0.0.0.0		eth0
					^^^^
10.10.10.1		0.0.0.0		eth1
192.168.100.3		0.0.0.0		eth0
10.10.10.4		10.10.10.3	eth0
					^^^^
Routing table on C:
Destination		GW		Iface
10.10.10.2		0.0.0.0		eth0
					^^^^
10.10.10.1		10.10.10.2	eth0
					^^^^
192.168.100.2		0.0.0.0		eth0
10.10.10.4		0.0.0.0		eth1

Problems here:
a) Between B and C new routes for the WLAN IP addresses is now added 
(those I missed in the first place), but check the interface used!
It is defined to use eth0 (LAN) interface (that is not working now) 
instead of the eth1 interface for the current IP address.
The interface for the 2-hop neighbors (10.10.10.4 on B and 10.10.10.1 on 
C) is therefore also using the wrong interface.

b) The LAN routes are still present..
The route 192.168.100.3 on B (and 192.168.100.2 on C) is no longer valid 
and should have been removed.

 From the debug-console (debug level 1) on B we can see this when the 
cable is pulled out (handwritten, so it may not be verbatim correct):
[192.168.100.3] HELLO timeout
[192.168.100.3] HELLO timeout
[192.168.100.3] HELLO timeout
Setting 10.10.10.3 as MPR
(ioctl) Adding route: 10.10.10.3 gw 10.10.10.3 (hopc 1)
(ioctl) Adding route: 192.168.100.3 gw 10.10.10.3 (hopc 1)
(ioctl) Adding route: 10.10.10.4 gw 10.10.10.3 (hopc 2)
<so far so good, this is what we want, except for 10.10.10.3 using eth0 
as shown earlier, then this happen:>
(ioctl) Deleting route: 10.10.10.4 (hopc 3)
(ioctl) Deleting route: 10.10.10.3 (hopc 2)
(ioctl) Deleting route: 192.168.100.3 (hopc 2)

Why does olsrd first install correct links and then remove the links 
after that? I also see that the hopc in the deletion seems wrong (one 
more than real hop count...).
But as the routing table after pulling the cable shows this "Deleting 
route" didn't seem to do anything wrong..

Then I insert the cable again and wait some more to let the links stabilize.
Routing table on B:
Destination		GW		Iface
10.10.10.3		0.0.0.0		eth0
10.10.10.1		0.0.0.0		eth1
192.168.100.3		0.0.0.0		eth0
10.10.10.4		192.168.100.3	eth0

Routing table on C:
Destination		GW		Iface
10.10.10.2		0.0.0.0		eth0
10.10.10.1		10.10.10.2	eth0
192.168.100.2		0.0.0.0		eth0
10.10.10.4		0.0.0.0		eth1

This is not the routing table as when olsrd started with the LAN cable 
plugged in. Now eth1 from B and C is in the routing table, but still 
using the LAN-interface (eth0) to communicate.
Another strange thing is that B changed the routing for 10.10.10.4 to 
use 192.168.100.3 as GW, while C continues to use 10.10.10.2 as GW to 
10.10.10.1 as when the cable was unplugged.

On B this is also shown in the debug trace (C didn't have anything extra 
than detecting link changes):
(ioctl) Adding route: 10.10.10.3 gw 192.168.100.3 (hopc 1)
(ioctl) Adding route: 192.168.100.3 gw 192.168.100.3 (hopc 1)
	File exists
(ioctl) Adding route: 192.168.100.3 gw 192.168.100.3 (hopc 1)
	File exists
(ioctl) Adding route: 10.10.10.4 gw 192.168.100.3 (hopc 2)
(ioctl) Deleting route: 10.10.10.4 (hopc 3)
(ioctl) Deleting route: 10.10.10.3 (hopc 2)
(ioctl) Deleting route: 192.168.100.3 (hopc 2)
Delete route: No such process
(ioctl) Deleting route: 192.168.100.3 (hopc 2)
Delete route: No such process

I can give more info if needed (more debug info, change parameters and 
so on).

-- 
Roar B. Rotvik