strongSwan

Thanks Tobias I think you're right to suggest that using multiple passthrough policies is the way to go here.

There are a couple of hundred separate subnets for both Netflix and AWS that I'll have to exclude from the tunnel so I think that will mean the same number of separate passthrough policies. Even so if this is the case, I think this is going to be more readable than a big list of narrowing subnets.

As long as there is no performance penalty for configuring this many passthrough policies I think it will be perfectly fine.

Taking 1 given subnet as an example: 108.175.32.0/20, I assume something like this should work:

conn passthrough_base
    left=127.0.0.1
    right=127.0.0.1    
    type=passthrough
    auto=route

conn passthrough_1
    also=passthrough_base
    leftsubnet=108.175.32.0/20
    rightsubnet=108.175.32.0/20

There are a couple of hundred separate subnets for both Netflix and AWS that I'll have to exclude from the tunnel so I think that will mean the same number of separate passthrough policies. Even so if this is the case, I think this is going to be more readable than a big list of narrowing subnets.

You can just set a list of the excluded IPs/subnets in rightsubnet (comma-separated).

As long as there is no performance penalty for configuring this many passthrough policies I think it will be perfectly fine.

There is (you have to try how noticeable it really is), as policies are stored in a list (sorted by priority) in the kernel, unless, you configure charon.plugins.kernel-netlink.spdh_thresh in strongswan.conf, which enables hashing parts of the addresses/selectors to lookup policies (only works on newer kernels and with newer strongSwan releases).

Taking 1 given subnet as an example: 108.175.32.0/20, I assume something like this should work:

Set leftsubnet to your actual local subnet (or just 0.0.0.0/0), and as mentioned above you can set rightsubnet to a list of subnets.

I think there's something not quite working as expected.

As a test I added the IP addresses for api.ipify.org to a test passthrough connection to test that curl api.ipify.org would return my local LAN public IP address and not the public IP of my remote VPN server.

Unfortunately it didn't seem to work because I see the IP of my remote VPN server returned from curl api.ipify.org which would seem to indicate the request to api.ipify.org is still going down the tunnel regardless of my configured passthrough connection.

Here's my ipsec.conf and ipsec statusall output:

# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
    # strictcrlpolicy=yes
    # uniqueids=never

conn base
    fragmentation=yes

    closeaction=restart
    dpdaction=restart
    # dpddelay=35s
    keyingtries=%forever

    keyexchange=ikev2    
    ike=aes128gcm16-prfsha512-ecp256,aes128-sha2_512-prfsha512-ecp256,aes128-sha2_384-prfsha384-ecp256!
    esp=aes128gcm16-ecp256,aes128-sha2_512-prfsha512-ecp256!

    auto=start

conn vpn
    also=base

    right=46.101.30.31
    rightid=46.101.30.31
    rightsubnet=0.0.0.0/0
    rightauth=pubkey    

    leftsourceip=%config     
    leftauth=pubkey
    leftcert=joe.crt    
    left=%defaultroute
    leftsubnet=%dynamic,192.168.0.1/24
    leftupdown=/usr/lib/ipsec/strongswan-updown

conn lan_base
    also=base

    left=46.101.30.31
    leftid=46.101.30.31
    leftsubnet=0.0.0.0/0
    leftauth=pubkey

    rightauth=pubkey
    rightcert=joe.crt
    rightsubnet=%dynamic,192.168.0.1/24

conn my_laptop
    also=lan_base
    rightsourceip=192.168.0.11
    rightid=192.168.0.11

conn my_phone
    also=lan_base
    rightsourceip=192.168.0.12
    rightid=192.168.0.12

conn sony_bravia_tv
    also=lan_base
    rightsourceip=192.168.0.15
    rightid=192.168.0.15

# VPN passthrough / tunnel bypass rules
conn passthrough_base
    left=127.0.0.1
    right=127.0.0.1
    leftsubnet=192.168.0.1/24
    authby=never
    type=passthrough
    auto=route

# exclude all LAN IPs from the tunnel
conn lan_traffic_passthrough
    also=passthrough_base    
    rightsubnet=192.168.0.1/24

# exclude ipify subnets from the tunnel
conn ipify_passthrough
    also=passthrough_base
    rightsubnet=174.129.241.106/32,23.23.170.235/32,184.73.220.206/32

root@nanopineo2:~/strongswan-5.6.1# ipsec statusall
Status of IKE charon daemon (strongSwan 5.6.1, Linux 4.11.1-sun50iw2, aarch64):
  uptime: 10 minutes, since Nov 28 23:58:50 2017
  malloc: sbrk 2555904, mmap 0, used 463696, free 2092208
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 4
  loaded plugins: charon aes des rc2 sha2 sha1 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf curve25519 agent xcbc cmac hmac attr kernel-netlink resolve socket-default stroke vici updown eap-identity eap-md5 eap-gtc eap-mschapv2 xauth-generic counters
Virtual IP pools (size/online/offline):
  192.168.0.11: 1/0/0
  192.168.0.12: 1/0/0
  192.168.0.15: 1/0/0
Listening IP addresses:
  192.168.0.10
Connections:
        base:  %any...%any  IKEv2, dpddelay=30s
        base:   local:  uses public key authentication
        base:   remote: uses public key authentication
        base:   child:  dynamic === dynamic TUNNEL, dpdaction=restart
         vpn:  %any...46.101.30.31  IKEv2, dpddelay=30s
         vpn:   local:  [CN=joe] uses public key authentication
         vpn:    cert:  "CN=joe" 
         vpn:   remote: [46.101.30.31] uses public key authentication
         vpn:   child:  dynamic 192.168.0.0/24 === 0.0.0.0/0 TUNNEL, dpdaction=restart
    lan_base:  46.101.30.31...%any  IKEv2, dpddelay=30s
    lan_base:   local:  [46.101.30.31] uses public key authentication
    lan_base:   remote: [CN=joe] uses public key authentication
    lan_base:    cert:  "CN=joe" 
    lan_base:   child:  0.0.0.0/0 === dynamic 192.168.0.0/24 TUNNEL, dpdaction=restart
   my_laptop:  46.101.30.31...%any  IKEv2, dpddelay=30s
   my_laptop:   local:  [46.101.30.31] uses public key authentication
   my_laptop:   remote: [CN=joe] uses public key authentication
   my_laptop:    cert:  "CN=joe" 
   my_laptop:   child:  0.0.0.0/0 === dynamic 192.168.0.0/24 TUNNEL, dpdaction=restart
    my_phone:  46.101.30.31...%any  IKEv2, dpddelay=30s
    my_phone:   local:  [46.101.30.31] uses public key authentication
    my_phone:   remote: [CN=joe] uses public key authentication
    my_phone:    cert:  "CN=joe" 
    my_phone:   child:  0.0.0.0/0 === dynamic 192.168.0.0/24 TUNNEL, dpdaction=restart
sony_bravia_tv:  46.101.30.31...%any  IKEv2, dpddelay=30s
sony_bravia_tv:   local:  [46.101.30.31] uses public key authentication
sony_bravia_tv:   remote: [CN=joe] uses public key authentication
sony_bravia_tv:    cert:  "CN=joe" 
sony_bravia_tv:   child:  0.0.0.0/0 === dynamic 192.168.0.0/24 TUNNEL, dpdaction=restart
passthrough_base:  127.0.0.1...127.0.0.1  IKEv1/2
passthrough_base:   local:  [127.0.0.1] uses public key authentication
passthrough_base:   remote: [127.0.0.1] uses public key authentication
passthrough_base:   child:  dynamic === 192.168.0.0/24 PASS
lan_traffic_passthrough:   child:  192.168.0.0/24 === 192.168.0.0/24 PASS
ipify_passthrough:   child:  174.129.241.106/32 23.23.170.235/32 184.73.220.206/32 === 192.168.0.0/24 PASS
Shunted Connections:
passthrough_base:  dynamic === 192.168.0.0/24 PASS
lan_traffic_passthrough:  192.168.0.0/24 === 192.168.0.0/24 PASS
ipify_passthrough:  174.129.241.106/32 23.23.170.235/32 184.73.220.206/32 === 192.168.0.0/24 PASS
Security Associations (1 up, 0 connecting):
         vpn[2]: ESTABLISHED 10 minutes ago, 192.168.0.10[CN=joe]...46.101.30.31[46.101.30.31]
         vpn[2]: IKEv2 SPIs: 9dc40f7c15b43f2e_i* ef44c3bfe2308aab_r, public key reauthentication in 2 hours
         vpn[2]: IKE proposal: AES_GCM_16_128/PRF_HMAC_SHA2_512/ECP_256
         vpn{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: cfcc58cf_i c78c7398_o
         vpn{1}:  AES_GCM_16_128, 7431042 bytes_i (8183 pkts, 2s ago), 560903 bytes_o (5604 pkts, 2s ago), rekeying in 31 minutes
         vpn{1}:   10.19.48.2/32 === 0.0.0.0/0

These passthrough policies are reversed (there is a log message on level 2 in cfg that shows when/why, basically the reason is right=127.0.0.1, which is a local address). You might want to disable charon.plugins.stroke.allow_swap or switch to swanctl.conf.

Thanks Tobias, clearly I wasn't 100% on the "l"eft ("l"ocal) / "r"ight("r"emote) convention in strongswan. I have rearranged my configuration so it's more clear that left is local and right is remote now.

Although I have for the moment decided to leave charon.plugins.stroke.allow_swap in it's default state (yes, allow_swap), my log appears to confirm that no left/right swap is occurring because there's no mention of either "left is other host, swapping ends" or "left nor right host is our side, assuming left=local" in my log which is running at level 2.

So it would appear that my test ipify passthrough just isn't working here in 5.6.1. Either that or my configuration has confused strongswan so it doesn't work.

I'd like to try and stay with using ipsec.conf if at all possible. It seems like what I'm trying to do should be possible: configure a passthrough for LAN traffic and also configure a passthrough for public subnets.

So it would appear that my test ipify passthrough just isn't working here in 5.6.1. Either that or my configuration has confused strongswan so it doesn't work.

Just look at the output of ipsec statusall or ip xfrm policy. If it still looks like the output you posted above the local and remote traffic selectors are clearly reversed (those on the left of === are local, those on the right remote). If you don't see a log message your logging configuration might be incorrect. Instead of disabling allow_swap (why are you reluctant to change it?) you can also remove right=127.0.0.1 (so it defaults to 0.0.0.0), left=127.0.0.1 should probably be enough to avoid that the connection is used for incoming packets.

I believe I'm getting closer to understanding what's going on now (even though it's still not working :))

I can see that strongswan is doing it's best to enable the passthrough to work by adding the required routes to table 220:

root@nanopineo2:~# ip route show table 220
default via 192.168.0.1 dev eth0  proto static  src 192.168.0.10 
23.23.170.235 via 192.168.0.1 dev eth0  proto static  src 192.168.0.10 
174.129.241.106 via 192.168.0.1 dev eth0  proto static  src 192.168.0.10 
184.73.220.206 via 192.168.0.1 dev eth0  proto static  src 192.168.0.10 
192.168.0.0/24 dev eth0  proto static  src 192.168.0.10

..however I believe it's my custom updown script up-client section which is 'overriding' these routes, resulting in traffic being pushed down the tunnel regardless due to my SNAT rule:

up-client:)
    # custom iptables nat rule insert for LAN clients    
    iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -m policy --dir out --pol ipsec -j ACCEPT
    iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j SNAT --to-source $PLUTO_MY_SOURCEIP   

    # custom iptables MSS adjust rule insert for LAN clients
    iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1280

    if [ $VPN_LOGGING ]
    then
      echo "iptables up-client rules done" 

      logger -t $TAG -p $FAC_PRIO \
      "+ $PLUTO_PEER_ID $PLUTO_PEER -- $PLUTO_ME" + "iptables up-client rules done" 

    fi

    ;;

Putting the problem with the passthrough to one side for the moment, under normal circumstances my SNAT rule here works because I generally do want all traffic pushed down the tunnel. However I'm now trying to implement an exemption for some subnets.

Perhaps what I need to do here is take control of the passthrough subnets within my updown script and somehow add a PREROUTING exit rule to prevent the POSTROUTING SNAT rule for the target passthrough subnets.

Clearly I'm no expert with this stuff and any help / advice is very much appreciated. I can't help but think this is much harder than it should be and perhaps I'm doing everything completely wrong.

It appears that I'm on the right lines because introducing these new -j RETURN exit rules into the POSTROUTING chain is making the passthrough work as it should:

up-client:)
# custom iptables nat rule insert for LAN clients    
  iptables -t nat -A POSTROUTING -d 174.129.241.106/32 -j RETURN
  iptables -t nat -A POSTROUTING -d 23.23.170.235/32 -j RETURN
  iptables -t nat -A POSTROUTING -d 184.73.220.206/32 -j RETURN

  iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -m policy --dir out --pol ipsec -j ACCEPT        
  iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j SNAT --to-source $PLUTO_MY_SOURCEIP   

  # custom iptables MSS adjust rule insert for LAN clients
  iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1280

  if [ $VPN_LOGGING ]
  then
    echo "iptables up-client rules done" 

    logger -t $TAG -p $FAC_PRIO \
    "+ $PLUTO_PEER_ID $PLUTO_PEER -- $PLUTO_ME" + "iptables up-client rules done" 
  fi

  ;;

If this is the solution I probably just need a better way of managing the list of exemption subnets now.

If this is the solution I probably just need a better way of managing the list of exemption subnets now.

You might only need these rules to avoid that packets to certain locations are tunneled (the outbound IPsec policy will only match if the source address matches the virtual IP, which will not be the case if the NAT rule is not applied). So passthrough policies might not be necessary (for local traffic generated on the server itself that could be different if the route installed in table 220 forces the virtual IP as source address, in the output above there is no such route, though). You can also combine multiple subnets in a single rule (just comma-separate them: -d 174.129.241.106/32,23.23.170.235/32,...).

Thanks Tobias, I have success.

You were right there's no need for a passthrough policy here, with my setup at least, and a POSTROUTING exit rule alone is enough to prevent traffic being routed down the tunnel. Thank you for all your help.

For the record I boiled it down to look like the following using ipset (http://ipset.netfilter.org/ipset.man.html) where my subnet exemption rows are contained within the strongswan_updown_exemption_subnets.txt file:

up-client:)
  # custom iptables nat rule insert for LAN clients

  ipset -L tunnel_exemptions >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    ipset create tunnel_exemptions hash:ip
    for subnet in $(cat /usr/lib/ipsec/strongswan_updown_exemption_subnets.txt); do 
      ipset add tunnel_exemptions $subnet;
    done
  fi

  iptables -t nat -A POSTROUTING -m set --match-set tunnel_exemptions dst -j RETURN

  iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -m policy --dir out --pol ipsec -j ACCEPT        
  iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j SNAT --to-source $PLUTO_MY_SOURCEIP   

  # custom iptables MSS adjust rule insert for LAN clients
  iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1280

  if [ $VPN_LOGGING ]
  then
    echo "iptables up-client rules done" 

    logger -t $TAG -p $FAC_PRIO \
    "+ $PLUTO_PEER_ID $PLUTO_PEER -- $PLUTO_ME" + "iptables up-client rules done" 
  fi

  ;;