strongSwan

I have a roadwarrior connection using "auto=route" on the initiator. After restarting Strongswan, the routing table 220 contains this:

$ ip -4 route show table 220
10.153.139.0/24 via 172.24.27.1 dev wlp2s0  proto static  src 172.24.27.100

After "up'ing" the conn or generating traffic, no route is added/changed so the source IP is wrong and doesn't match the assigned virtual IP. This means I cannot reach the remote network 10.153.139.0/24.

Obfuscated configuration on the responder (Strongswan 5.3.5):

conn base
  left=192.2.0.1
  leftsubnet=10.153.139.0/24
  leftid="C=CA, O=example.net, CN=vpn.example.net" 
  leftcert=vpn.example.net.crt
  leftsendcert=ifasked
  leftauth=pubkey
  rightdns=10.153.139.1,8.8.8.8
  right=%any
  rightsendcert=never
  rightauth=eap-mschapv2
  eap_identity=%identity
  fragmentation=yes
  keyexchange=ikev2
  ike=aes128gcm128-prfsha512-modp4096,aes256-sha1-modp1024,3des-sha1-modp1024!
  esp=aes128gcm128-modp4096,aes128-sha1,aes256-sha1,3des-sha1!
  dpddelay=60
  dpdtimeout=300
  dpdaction=clear

# XXX: user per-user entries to have static IP assigned (is there a better way?)
conn sdeziel
  eap_identity=sdeziel
  rightsourceip=192.168.139.100/32
  also=base
  auto=add

conn jdoe
  eap_identity=jdoe
  rightsourceip=192.168.139.101/32
  also=base
  auto=add

and on the initiator (Strongswan 5.3.5):

conn sdeziel
  right=192.2.0.1
  rightsubnet=10.153.139.0/24
  rightid="C=CA, O=example.net, CN=vpn.example.net" 
  rightauth=pubkey
  left=%any
  leftsendcert=never
  leftauth=eap-mschapv2
  keyexchange=ikev2
  ike=aes128gcm128-prfsha512-modp4096!
  esp=aes128gcm128-modp4096!
  dpddelay=60
  dpdtimeout=300
  dpdaction=restart
  leftsourceip=%config4
  leftid=sdeziel
  auto=route

I also tried assigning a virtual IPv4 + IPv6 and the problem is the same with the difference that the IP version I use to establish the tunnel defines which virtual IP version will be broken. Establishing over IPv6 means the virutal IPv6 won't work but the IPv4 will and vice-versa.