[Olsr-dev] Fwd: Smarter gateway
Teco Boot
(spam-protected)
Fri Jan 13 17:19:45 CET 2012
Back on discussion on improved SmartGateway
Op 14 dec. 2011, om 23:26 heeft Markus Kittenberger het volgende geschreven:
>
>> e.g. to seperate between stable gateways for long lasting connections that should no change gateway, and other traffic classes where it does not hurt much to change gateways, and additinoal maybe even use no tunnel (and always the nearest gateway) for some traffic for example dns or ntp requests
>>
>> (and this needs no conntrack support to realize, just some simple traffic classification, and anhancements to olsrd to maintain/configure multiple SGW tunnels)
> So new long lifetime connections take 2nd best path, or worse?
> yes, but at least they can live long,.. (-;
>
> I didn't try conntrack for this, but what is wrong with it? (other than make it happen...)
> olsrd currently does not need a patched kernel, for any of its features (which is great!)
>
> but having conntrack and SGW (and routing) work seemless together would!
> (or at least i have no idea to make it happen without kernel modifications)
Finally found some time to play with this.
I've a working prototype, with two gateways and one client.
No kernel modification required. Just iptables & conntrack.
On each router, I can have 0, 1 or 2 border interfaces: ppp0 and eth0. Both with
masquerade. I need connection pinning for these interfaces also. If eth0 comes
up, why kill connections via ppp0? OK, there are reasons, see below.
I support up to 8 SGW tunnels. I used names tnl_101 to tnl_108. Number of tunnels
would be config parameter.
For this test, SmartGateway on the client is disabled. I create tunnels and
rules by hand. A new SGW plugin shall automate it.
1: Set up permanent iptables & ip rules.
========================================
I use markings, rules and tables 91 - 92 for border interfaces and
101 - 108 for tunnels. This fixed config is part of my fw setup. I flush
all rules before, for fw restart.
# Indicate which egress interface was selected on connection setup
# This egress interface will be used for future packets of this connection
iptables -t mangle -A POSTROUTING -m conntrack --ctstate NEW -o ppp0 -j CONNMARK --set-mark 91
iptables -t mangle -A POSTROUTING -m conntrack --ctstate NEW -o eth1 -j CONNMARK --set-mark 92
for i in `seq 101 108`; do
iptables -t mangle -A POSTROUTING -m conntrack --ctstate NEW -o tnl_$i -j CONNMARK --set-mark $i
done
# Also for incoming connections on border interface
iptables -t mangle -A INPUT -m conntrack --ctstate NEW -i ppp0 -j CONNMARK --set-mark 91
iptables -t mangle -A INPUT -m conntrack --ctstate NEW -i eth1 -j CONNMARK --set-mark 92
#
# Enforce egress interface selection based on previous set mark
# Earlier saved ctmark for this connection is copied to this packet fwmark
iptables -t mangle -A PREROUTING -i br0 -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -i tunl0 -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
#
# Set up ip rules
# One time only action...
if [ `ip rule show | grep "lookup 91" | wc -l` == 0 ] ; then
# Link fwmark to routing tables
for i in 91 92 `seq 101 108`; do
ip rule add fwmark $i table $i pref $i
done
fi
2: Maintain tunnels when GW detected & set up routing
==================================================
For this experiment, I do this manual. I know which gw is present.
It is not shown with gateway, because smartgateway is disabled.
It is also not shown with HNA, because I use default routes in main table.
# Add tnl_101 interface
ip tunnel add tnl_101 mode ipip remote 172.31.175.173
ip address add 172.31.175.166/32 dev tnl_101
ip link set tnl_101 up
ip route add default dev tnl_101 table 101
# Add tnl_102 interface
ip tunnel add tnl_102 mode ipip remote 172.31.175.179
ip address add 172.31.175.166/32 dev tnl_102
ip link set tnl_102 up
ip route add default dev tnl_102 table 102
Now activate one of the tunnels as default.
And update the default route every now and then :-)
(spam-protected):~# while true ; do
> ip route replace default dev tnl_101
> echo default gateway is now tnl_101
> sleep 10
> ip route replace default dev tnl_102
> echo default gateway is now tnl_102
> sleep 10
> done
default gateway is now tnl_101
default gateway is now tnl_102
default gateway is now tnl_101
default gateway is now tnl_102
default gateway is now tnl_101
default gateway is now tnl_102
default gateway is now tnl_101
default gateway is now tnl_102
...
Set up some test connections, e.g. ping 8.8.8.8
Tested with SSH, OpenVPN en surfing.
No problems seen :-))
(spam-protected):~# tcpdump -ni tnl_101 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tnl_101, link-type RAW (Raw IP), capture size 65535 bytes
14:22:25.010922 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 0, length 64
14:22:26.011600 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 1, length 64
14:22:27.011714 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 2, length 64
14:22:28.012823 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 3, length 64
14:22:29.012857 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 4, length 64
14:22:30.013023 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 5, length 64
14:22:31.013399 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 6, length 64
14:22:32.014351 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 7, length 64
14:22:33.014502 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 8, length 64
14:22:34.015078 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 9, length 64
14:22:35.015921 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 10, length 64
14:22:36.015966 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19124, seq 11, length 64
...
(spam-protected):~# tcpdump -ni tnl_102 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tnl_102, link-type RAW (Raw IP), capture size 65535 bytes
14:22:31.066048 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19380, seq 0, length 64
14:22:32.066917 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19380, seq 1, length 64
14:22:33.067682 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19380, seq 2, length 64
14:22:34.068468 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19380, seq 3, length 64
14:22:35.069620 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19380, seq 4, length 64
14:22:36.069815 IP 10.175.166.38 > 8.8.8.8: ICMP echo request, id 19380, seq 5, length 64
...
(spam-protected):~# conntrack -L -m 101
tcp 6 431864 ESTABLISHED src=10.175.166.38 dst=x.x.x.x sport=60046 dport=22 src=x.x.x.x dst=10.175.166.38 sport=22 dport=60046 [ASSURED] mark=101 use=1
udp 17 172 src=10.175.166.38 dst=x.x.x.x sport=65054 dport=1194 src=x.x.x.x dst=10.175.166.38 sport=1194 dport=65054 [ASSURED] mark=101 use=1
tcp 6 431662 ESTABLISHED src=10.175.166.38 dst=x.x.x.x sport=60037 dport=443 src=x.x.x.x dst=10.175.166.38 sport=443 dport=60037 [ASSURED] mark=101 use=1
icmp 1 29 src=10.175.166.38 dst=8.8.8.8 type=8 code=0 id=19124 src=8.8.8.8 dst=10.175.166.38 type=0 code=0 id=19124 mark=101 use=2
conntrack v0.9.14 (conntrack-tools): 4 flow entries have been shown.
(spam-protected):~# conntrack -L -m 102
tcp 6 431685 ESTABLISHED src=10.175.166.38 dst=x.x.x.x sport=60032 dport=443 src=x.x.x.x dst=10.175.166.38 sport=443 dport=60032 [ASSURED] mark=102 use=1
tcp 6 430851 ESTABLISHED src=10.175.166.38 dst=x.x.x.x sport=59997 dport=5223 src=x.x.x.x dst=10.175.166.38 sport=5223 dport=59997 [ASSURED] mark=102 use=1
udp 17 177 src=10.175.166.38 dst=x.x.x.x sport=65054 dport=1194 src=x.x.x.x dst=10.175.166.38 sport=1194 dport=65054 [ASSURED] mark=102 use=1
icmp 1 29 src=10.175.166.38 dst=8.8.8.8 type=8 code=0 id=19380 src=8.8.8.8 dst=10.175.166.38 type=0 code=0 id=19380 mark=102 use=1
conntrack v0.9.14 (conntrack-tools): 4 flow entries have been shown.
(spam-protected):~# iptables -vL -t mangle | grep -v "for me only"
Chain PREROUTING (policy ACCEPT 18972 packets, 2360K bytes)
pkts bytes target prot opt in out source destination
3737 678K CONNMARK all -- br0 any anywhere anywhere CONNMARK restore
0 0 CONNMARK all -- tunl0 any anywhere anywhere CONNMARK restore
Chain INPUT (policy ACCEPT 17305 packets, 1795K bytes)
pkts bytes target prot opt in out source destination
0 0 CONNMARK all -- ppp0 any anywhere anywhere ctstate NEW CONNMARK set 0x5b
0 0 CONNMARK all -- eth1 any anywhere anywhere ctstate NEW CONNMARK set 0x5c
Chain FORWARD (policy ACCEPT 5343 packets, 1238K bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 18980 packets, 2205K bytes)
pkts bytes target prot opt in out source destination
19000 2206K CONNMARK all -- any any anywhere anywhere CONNMARK restore
Chain POSTROUTING (policy ACCEPT 22807 packets, 3643K bytes)
pkts bytes target prot opt in out source destination
0 0 CONNMARK all -- any ppp0 anywhere anywhere ctstate NEW CONNMARK set 0x5b
0 0 CONNMARK all -- any eth1 anywhere anywhere ctstate NEW CONNMARK set 0x5c
184 14105 CONNMARK all -- any tnl_101 anywhere anywhere ctstate NEW CONNMARK set 0x65
73 5311 CONNMARK all -- any tnl_102 anywhere anywhere ctstate NEW CONNMARK set 0x66
0 0 CONNMARK all -- any tnl_103 anywhere anywhere ctstate NEW CONNMARK set 0x67
0 0 CONNMARK all -- any tnl_104 anywhere anywhere ctstate NEW CONNMARK set 0x68
0 0 CONNMARK all -- any tnl_105 anywhere anywhere ctstate NEW CONNMARK set 0x69
0 0 CONNMARK all -- any tnl_106 anywhere anywhere ctstate NEW CONNMARK set 0x6a
0 0 CONNMARK all -- any tnl_107 anywhere anywhere ctstate NEW CONNMARK set 0x6b
0 0 CONNMARK all -- any tnl_108 anywhere anywhere ctstate NEW CONNMARK set 0x6c
0 0 DROP all -- any ppp0 anywhere 10.0.0.0/8
0 0 DROP all -- any ppp0 anywhere 172.16.0.0/12
0 0 DROP all -- any ppp0 anywhere 192.168.0.0/16
39 2496 TCPMSS tcp -- any tnl_+ anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS set 1440
0 0 TCPMSS tcp -- any ppp0 anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS set 1440
0 0 TCPMSS tcp -- any eth1 anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS set 1440
Other maintenance procedures:
=============================
I stored the default route in the main table. This needs more thoughts,
e.g. not let OLSR advertise an HNA and how to utilize another node's fast
exit link in favor of own turtle|$$$ links (wired broadband prio over UMTS,
UMTS prio over GRPS/satcom).
The route maintenance could be very basic (just add better GWs, limit to
8 tunnels, throw away the lousy ones).
The more advanced method would smoothly kill connections via turtle|$$$ links,
if something better is in place. This could be done by removing conntrack state,
e.g. for applications that recover easily, such as ping & OpenVPN. DNS & NTP
are short-lived, so they take best path automatically. SSH (and unknown?)
doesn't survive, so let them go.
(spam-protected):~# conntrack -D -m 102
tcp 6 431620 ESTABLISHED src=10.175.166.38 dst=x.x.x.x sport=60032 dport=443 src=x.x.x.x dst=10.175.166.38 sport=443 dport=60032 [ASSURED] mark=102 use=1
tcp 6 430786 ESTABLISHED src=10.175.166.38 dst=x.x.x.x sport=59997 dport=5223 src=x.x.x.x dst=10.175.166.38 sport=5223 dport=59997 [ASSURED] mark=102 use=1
udp 17 179 src=10.175.166.38 dst=x.x.x.x sport=65054 dport=1194 src=x.x.x.x dst=10.175.166.38 sport=1194 dport=65054 [ASSURED] mark=102 use=1
icmp 1 29 src=10.175.166.38 dst=8.8.8.8 type=8 code=0 id=19380 src=8.8.8.8 dst=10.175.166.38 type=0 code=0 id=19380 mark=102 use=1
conntrack v0.9.14 (conntrack-tools): 4 flow entries have been deleted.
Cleanup conntrack shall be performed also when a tunnel is deleted.
Opinions?
Teco
More information about the Olsr-dev
mailing list