Project

General

Profile

Issue #3058

Strongswan in a dual WAN scenario

Added by Ramon PH 10 days ago. Updated 4 days ago.

Status:
Closed
Priority:
Normal
Category:
configuration
Affected version:
5.3.5
Resolution:
No change required

Description

Good morning to everyone.

I am using Strongswan to try to set up dual WAN connectivity between two OpenFlow switches (Ubuntu 16.04 servers with OVS 2.9.0 installed, forwarding enabled and managed by an SDN controller) by using GRE over IPsec tunnels (encapsulate GRE traffic with IPsec). Specifically, the connection between these devices is intended to follow a point-to-multipoint topology, where one of the devices only uses one WAN port and the other one uses two, and there would be two IPsec tunnels, each of them connecting the WAN port of the first device with one of the WAN ports of the second device. All the traffic present in the scenario is encapsulated with GRE. The scenario including its IP addresses would be the following:

*******************                                    192.168.150.3/24 ***********************
*                 *                                  ********************br0*                 *
*                 *****  192.168.150.4/24            *                  *****                 *
*    device1      *br0*************************(L2 switch)                  *     device2     *
*                 *****                              * 192.168.150.1/24 *****                 *
*                 *                                  ********************br1*                 *
*******************                                                     ***********************

Without using IPsec (only GRE encapsulation), the traffic is forwarded correctly, because device2 has some iptables rules in the mangle table in order to mark packets depending on the "key" field of the GRE header, whose value is fixed by configuring the OVS. In our case, the traffic between 192.168.150.3 and 192.168.150.4 will use the key=1, and the traffic between 192.168.150.1 and 192.168.150.4 will use the key=3. After marking the packets, device2 has two ip rules installed in order to forward traffic depending on the key value (use br0 for key=1 and br1 for key=3).

The configuration aforementioned in device2 is the following:

ip rule
0:    from all lookup local 
218:    from all fwmark 0x3 lookup tunnel3 
219:    from all fwmark 0x1 lookup tunnel1 
32766:    from all lookup main 
32767:    from all lookup default 

sudo ip r show table tunnel1
192.168.150.0/24 dev br0  scope link

sudo ip r show table tunnel3
192.168.150.0/24 dev br1  scope link 

sudo iptables -L -v -t mangle
Chain PREROUTING (policy ACCEPT 343 packets, 28499 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x1" MARK set 0x1
    1   112 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x3" MARK set 0x3

Chain INPUT (policy ACCEPT 343 packets, 28499 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 336 packets, 56519 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x1" MARK set 0x1
    1   112 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x3" MARK set 0x3

Chain POSTROUTING (policy ACCEPT 336 packets, 56519 bytes)
 pkts bytes target     prot opt in     out     source               destination      

However, after activating IPsec, the traffic is not being forwarded in the same way. First of all, I am aware of the table 220 managed by IPsec, which controls the IPsec forwarding. I could check that table 220 has only one routing entry in device2, which is the following:

sudo ip r show table 220
192.168.150.4 via 192.168.150.4 dev br0  proto static  src 192.168.150.3 

In the case of device1, it has two routing entries, which correspond to the two IPsec tunnels established:

sudo ip r show table 220
192.168.150.1 via 192.168.150.1 dev br0  proto static  src 192.168.150.4 
192.168.150.3 via 192.168.150.3 dev br0  proto static  src 192.168.150.4

With these rules, the traffic from device1 to device2 is forwarded correctly, but the traffic from device2 to device1 always uses the br0 interface, even forcing device2 to use the device1 interface (as the ESP packet use the br0 MAC address, the traffic is sent from br0 interface in device2, and not from br1).

After some thoughts, I tried to use custom forwarding rules instead of using the table 220. I set charon.install_routes = no in /etc/strongswan.conf, verifying that table 220 was empty after deploying the IPsec tunnels. I installed some iptables rules in the mangle table, matching packets by the protocol (ESP) and the source and destination IP addresses, and setting a mark in a similar way to GRE packets, so that the ESP packets should use the tunnel1 or tunnel3 routing table depending on the IPsec tunnel to be used.

Despite that, the traffic has not been forwarded correctly yet. The most surprising thing is that the iptables rules had matches on the ESP rules, as you can see here:

sudo iptables -L -v -t mangle
Chain PREROUTING (policy ACCEPT 343 packets, 28499 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x1" MARK set 0x1
    1   112 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x3" MARK set 0x3
    0     0 MARK       esp  --  any    any     192.168.150.4        192.168.150.3        MARK set 0x1
    1   188 MARK       esp  --  any    any     192.168.150.4        192.168.150.1        MARK set 0x3

Chain INPUT (policy ACCEPT 343 packets, 28499 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 336 packets, 56519 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x1" MARK set 0x1
    1   112 MARK       gre  --  any    any     anywhere             anywhere             u32 "0x14&0x20000000=0x20000000&&0x18&0xffffffff=0x3" MARK set 0x3
    1   188 MARK       esp  --  any    any     192.168.150.1        192.168.150.4        MARK set 0x3
    0     0 MARK       esp  --  any    any     192.168.150.3        192.168.150.4        MARK set 0x1

Chain POSTROUTING (policy ACCEPT 336 packets, 56519 bytes)
 pkts bytes target     prot opt in     out     source               destination       

Right now, I am completely stuck in this issue, and I do not know how to continue in order to solve this problem. I think it should be due to the management of the routing module in the Linux kernel, but I am not sure about that. Could you give me some ideas in order to try to fix this problem? Thank you very much in advance for your help and support.

The IPsec configuration is the following one (version 5.3.5, installed with apt-get and with the default configuration in other files):

device1

cat /etc/ipsec.conf 
config setup
        charondebug=all
        uniqueids=yes
        strictcrlpolicy=no
conn %default
          ike=aes256-sha2_256-modp1024!
            esp=aes256-sha2_256!
            keyingtries=0
            ikelifetime=1h
            lifetime=8h
            dpddelay=30
            dpdtimeout=120
            dpdaction=restart
            authby=secret
            auto=start
            keyexchange=ikev2
            type=tunnel

conn A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona
    left=192.168.150.4
    right=192.168.150.3
    leftsubnet=%dynamic[gre]
    rightsubnet=%dynamic[gre]

conn A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two
    left=192.168.150.4
    right=192.168.150.1
    leftsubnet=%dynamic[gre]
    rightsubnet=%dynamic[gre]

sudo ipsec status
Security Associations (2 up, 0 connecting):
A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two[2]: ESTABLISHED 14 minutes ago, 192.168.150.4[192.168.150.4]...192.168.150.1[192.168.150.1]
A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two{2}:  INSTALLED, TUNNEL, reqid 2, ESP SPIs: cf568860_i c15f35b7_o
A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two{2}:   192.168.150.4/32[gre] === 192.168.150.1/32[gre]
A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona[1]: ESTABLISHED 14 minutes ago, 192.168.150.4[192.168.150.4]...192.168.150.3[192.168.150.3]
A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona{1}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: c4fd4788_i cc009cc2_o
A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona{1}:   192.168.150.4/32[gre] === 192.168.150.3/32[gre]

device2

cat /etc/ipsec.conf
config setup
        charondebug=all
        uniqueids=yes
        strictcrlpolicy=no
conn %default
          ike=aes256-sha2_256-modp1024!
            esp=aes256-sha2_256!
            keyingtries=0
            ikelifetime=1h
            lifetime=8h
            dpddelay=30
            dpdtimeout=120
            dpdaction=restart
            authby=secret
            auto=start
            keyexchange=ikev2
            type=tunnel

conn A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona
    left=192.168.150.3
    right=192.168.150.4
    leftsubnet=%dynamic[gre]
    rightsubnet=%dynamic[gre]

conn A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two
    left=192.168.150.1
    right=192.168.150.4
    leftsubnet=%dynamic[gre]
    rightsubnet=%dynamic[gre]

sudo ipsec status
Security Associations (2 up, 0 connecting):
A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two[4]: ESTABLISHED 14 minutes ago, 192.168.150.1[192.168.150.1]...192.168.150.4[192.168.150.4]
A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two{4}:  INSTALLED, TUNNEL, reqid 4, ESP SPIs: c15f35b7_i cf568860_o
A.spain.Madrid.gre.spain.Barcelona.two-B.spain.Madrid.gre.spain.Barcelona.two{4}:   192.168.150.1/32[gre] === 192.168.150.4/32[gre]
A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona[3]: ESTABLISHED 14 minutes ago, 192.168.150.3[192.168.150.3]...192.168.150.4[192.168.150.4]
A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona{3}:  INSTALLED, TUNNEL, reqid 3, ESP SPIs: cc009cc2_i c4fd4788_o
A.spain.Madrid.gre.spain.Barcelona-B.spain.Madrid.gre.spain.Barcelona{3}:   192.168.150.3/32[gre] === 192.168.150.4/32[gre]

Best regards,
Ramón.

History

#1 Updated by Tobias Brunner 9 days ago

  • Status changed from New to Feedback
  • Priority changed from High to Normal

Without using IPsec (only GRE encapsulation), the traffic is forwarded correctly

Is your goal to use GRE or to use IPsec? Because you can do the latter without the former if you already mark traffic. When using marks (with GRE interfaces they might not really be necessary) make sure the SAs and policies have matching marks.

First of all, I am aware of the table 220 managed by IPsec, which controls the IPsec forwarding.

It doesn't really. That routing table and IPsec handling are not directly related (the latter is controlled by IPsec policies). Routes are mainly needed if there are no matching routes yet, or if a specific source address should be used.

#2 Updated by Ramon PH 4 days ago

Tobias Brunner wrote:

Without using IPsec (only GRE encapsulation), the traffic is forwarded correctly

Is your goal to use GRE or to use IPsec? Because you can do the latter without the former if you already mark traffic. When using marks (with GRE interfaces they might not really be necessary) make sure the SAs and policies have matching marks.

First of all, I am aware of the table 220 managed by IPsec, which controls the IPsec forwarding.

It doesn't really. That routing table and IPsec handling are not directly related (the latter is controlled by IPsec policies). Routes are mainly needed if there are no matching routes yet, or if a specific source address should be used.

Dear Tobias,

Thank you for your support. Regarding your first comment, my goal is to use GRE over IPsec, in this specific case. However, after reading your comment, I could check that, removing the iptables rules for GRE traffic and only using the rules for IPsec traffic, the scenario works completely and the traffic is forwarded correctly, so I do not have more doubts for the moment. This issue can be closed.

Thank you again for your help.
Best regards,
Ramón.

#3 Updated by Tobias Brunner 4 days ago

  • Status changed from Feedback to Closed
  • Assignee set to Tobias Brunner
  • Resolution set to No change required

Also available in: Atom PDF