Project

General

Profile

Issue #3571

ipv6 over ipv4 ESP packet not be decrypted correctly

Added by Sean Yuan about 1 month ago. Updated about 1 month ago.

Status:
Closed
Priority:
Normal
Category:
kernel-interface
Affected version:
5.6.0
Resolution:
No change required

Description

Hi,

I'm trying to setup ipv6 over ipv4 ipsec tunnel using strong-swan.
Setup is similar to :
https://www.strongswan.org/testing/testresults/ipv6-stroke/rw-ip6-in-ip4-ikev2/

So my setup is : alice <---> SeGW <----> carol

alice: fec1::10
SeGW : eth0 is 192.168.2.157, eth1 is fec1::1
carol: eth0 is 192.168.2.158 and fec3::8

After Ike SA and child sa are established i try to ping from carol(strongswan machine)
to alice with ping6 fec1::10.But the ping6 hungup.

I see on moon that esp packet is received (echo request) and sent(echo reply).
Also I dump packets on carol eth0 interface with tcpdump save as attached file ping.pcap.

I see on carol that esp packet is sent and received.After decryped the esp packets with wiresharks,
it is the correct ICMPv6 request and reply.

So why the esp packets cannot be decrypted on carol and ping failed.
Any ideas on how to solve this problem ?

carol ipsec.conf

config setup
    strictcrlpolicy=no

conn %default
    keyingtries=0
    dpdaction=clear
    dpddelay=120s
    auto=start
    keyexchange=ikev2
    reauth=no
    mobike=no
    rekeymargin=1m
    ikelifetime=10m
    keylife=60m
    leftupdown=/root/scripts/updown.tenpin
    replay_window=0

conn conn-1
    keyexchange=ikev2
    ike=aes128-sha1-modp1024
    esp=aes128-sha1,aes256-sha1
    right=192.168.2.157
    rightid=%any 
    leftcert=carol.crt.pem
    leftsendcert=ifasked
    authby=pubkey
    rightsubnet=fec1::/16
    left=%defaultroute
    leftsourceip=%config
    leftdns=%config
    fragmentation=no
    leftikeport=500
    rightikeport=500
    type=tunnel

carol: ipsec statusall

Status of IKE charon daemon (strongSwan 5.6.0, Linux 3.10.84-perf, armv7l):
  uptime: 82 minutes, since Sep 18 09:09:50 2020
  malloc: sbrk 1216512, mmap 0, used 232240, free 984272
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 10
  loaded plugins: charon aes des rc2 sha2 sha1 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp curve25519 xcbc cmac hmac attr kernel-pfkey kernel-netlink resolve socket-default stroke vici updown eap-identity eap-sim eap-aka xauth-generic
Listening IP addresses:
  192.168.0.100
  192.168.2.158
Connections:
      conn-1:  %any...192.168.2.157  IKEv2, dpddelay=120s
      conn-1:   local:  [C=com, O=myvpn, CN=VPN Client] uses public key authentication
      conn-1:    cert:  "C=com, O=myvpn, CN=VPN Client" 
      conn-1:   remote: uses public key authentication
      conn-1:   child:  dynamic === fec1::/16 TUNNEL, dpdaction=clear
Security Associations (1 up, 0 connecting):
      conn-1[11]: ESTABLISHED 2 minutes ago, 192.168.2.158[C=com, O=myvpn, CN=VPN Client]...192.168.2.157[C=com, O=myvpn, CN=211.75.141.110]
      conn-1[11]: IKEv2 SPIs: a821a2f1fc4dc3aa_i* 113153c91ba68e39_r, rekeying in 5 minutes, public key reauthentication in 25 minutes
      conn-1[11]: IKE proposal: AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
      conn-1{6}:  INSTALLED, TUNNEL, reqid 2, ESP SPIs: c10f426c_i c11b745a_o
      conn-1{6}:  AES_CBC_128/HMAC_SHA1_96, 0 bytes_i, 0 bytes_o, rekeying in 45 minutes
      conn-1{6}:   fec3::8/128 === fec1::/16

iptables-save

# Generated by iptables-save v1.6.0 on Fri Sep 18 10:32:24 2020
*mangle
:PREROUTING ACCEPT [11829:1944818]
:INPUT ACCEPT [11829:1944818]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [9101:1610313]
:POSTROUTING ACCEPT [9101:1610313]
:nglog_chain - [0:0]
:tcpmss_chain - [0:0]
-A PREROUTING -i eth0 -p tcp -m tcp --tcp-flags SYN SYN -g tcpmss_chain
-A PREROUTING -i eth0 -p udp -m udp --sport 2152 -m u32 --u32 "0x1a&0xff=0xff&&0x21&0xff=0x45&&0x27&0x1f=0x0&&0x28&0xff=0x0&&0x2a&0xff=0x6&&0x42&0x3=0x1:0x3" -j nglog_chain
-A PREROUTING -i eth0 -p udp -m udp --sport 2152 -m u32 --u32 "0x1a&0xff=0xff&&0x21&0xff=0x45&&0x27&0x1f=0x0&&0x28&0xff=0x0&&0x2a&0xff=0x1" -j nglog_chain
-A PREROUTING -i eth0 -p udp -m udp --sport 2152 -m u32 --u32 "0x1a&0xff=0x1:0xfe" -j nglog_chain
-A PREROUTING -i eth0 -p sctp -m u32 ! --u32 "0x0&0xff=0x30" -m u32 ! --u32 "0x2e&0xff=0xa" -j nglog_chain
-A OUTPUT -o lo -p udp -m udp --dport 17054 -j nglog_chain
-A OUTPUT -o lo -p udp -m udp --dport 9999 -m u32 ! --u32 "0x19&0xff=0x5" -j nglog_chain
-A POSTROUTING -o eth0 -p tcp -m tcp --tcp-flags SYN SYN -g tcpmss_chain
-A POSTROUTING -o eth0 -p udp -m udp --dport 2152 -m u32 --u32 "0x1a&0xff=0xff&&0x21&0xff=0x45&&0x27&0x1f=0x0&&0x28&0xff=0x0&&0x2a&0xff=0x6&&0x42&0x3=0x1:0x3" -j nglog_chain
-A POSTROUTING -o eth0 -p udp -m udp --dport 2152 -m u32 --u32 "0x1a&0xff=0xff&&0x21&0xff=0x45&&0x27&0x1f=0x0&&0x28&0xff=0x0&&0x2a&0xff=0x1" -j nglog_chain
-A POSTROUTING -o eth0 -p udp -m udp --dport 2152 -m u32 --u32 "0x1a&0xff=0x1:0xfe" -j nglog_chain
-A POSTROUTING -o eth0 -p sctp -m u32 ! --u32 "0x0&0xff=0x30" -m u32 ! --u32 "0x2e&0xff=0xa" -j nglog_chain
-A nglog_chain -j NFLOG --nflog-group 5
-A tcpmss_chain -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1300
COMMIT
# Completed on Fri Sep 18 10:32:24 2020

ip6tables-save

# Generated by ip6tables-save v1.6.0 on Fri Sep 18 10:32:24 2020
*filter
:INPUT ACCEPT [5036:518224]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [5039:519656]
-A INPUT -d fec3::7/128 -j ACCEPT
-A OUTPUT -s fec3::7/128 -j ACCEPT
COMMIT
# Completed on Fri Sep 18 10:32:24 2020
# Generated by ip6tables-save v1.6.0 on Fri Sep 18 10:32:24 2020
*mangle
:PREROUTING ACCEPT [5118:526672]
:INPUT ACCEPT [5118:526672]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [5122:528216]
:POSTROUTING ACCEPT [5122:528216]
:nglog_chain - [0:0]
:tcpmss_chain - [0:0]
-A PREROUTING -i eth0 -p tcp -m tcp --tcp-flags SYN SYN -g tcpmss_chain
-A PREROUTING -i eth0 -p udp -m udp --sport 2152 -m u32 --u32 "0x2e&0xff=0xff&&0x35&0xff=0x45&&0x3b&0x1f=0x0&&0x3c&0xff=0x0&&0x3e&0xff=0x6&&0x56&0x3=0x1:0x3" -j nglog_chain
-A PREROUTING -i eth0 -p udp -m udp --sport 2152 -m u32 --u32 "0x2e&0xff=0xff&&0x35&0xff=0x45&&0x3b&0x1f=0x0&&0x3c&0xff=0x0&&0x3e&0xff=0x1" -j nglog_chain
-A PREROUTING -i eth0 -p udp -m udp --sport 2152 -m u32 --u32 "0x2e&0xff=0x1:0xfe" -j nglog_chain
-A PREROUTING -i eth0 -p sctp -m u32 ! --u32 "0x0&0xff=0x44" -m u32 ! --u32 "0x42&0xff=0xa" -j nglog_chain
-A POSTROUTING -o eth0 -p tcp -m tcp --tcp-flags SYN SYN -g tcpmss_chain
-A POSTROUTING -o eth0 -p udp -m udp --dport 2152 -m u32 --u32 "0x2e&0xff=0xff&&0x35&0xff=0x45&&0x3b&0x1f=0x0&&0x3c&0xff=0x0&&0x3e&0xff=0x6&&0x56&0x3=0x1:0x3" -j nglog_chain
-A POSTROUTING -o eth0 -p udp -m udp --dport 2152 -m u32 --u32 "0x2e&0xff=0xff&&0x35&0xff=0x45&&0x3b&0x1f=0x0&&0x3c&0xff=0x0&&0x3e&0xff=0x1" -j nglog_chain
-A POSTROUTING -o eth0 -p udp -m udp --dport 2152 -m u32 --u32 "0x2e&0xff=0x1:0xfe" -j nglog_chain
-A POSTROUTING -o eth0 -p sctp -m u32 ! --u32 "0x0&0xff=0x44" -m u32 ! --u32 "0x42&0xff=0xa" -j nglog_chain
-A nglog_chain -j NFLOG --nflog-group 5
-A tcpmss_chain -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1300
COMMIT
# Completed on Fri Sep 18 10:32:24 2020

ip route show table all

broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.0.0 dev eth1 table local proto kernel scope link src 192.168.0.100 
local 192.168.0.100 dev eth1 table local proto kernel scope host src 192.168.0.100 
broadcast 192.168.0.255 dev eth1 table local proto kernel scope link src 192.168.0.100 
broadcast 192.168.2.0 dev eth0 table local proto kernel scope link src 192.168.2.158 
local 192.168.2.158 dev eth0 table local proto kernel scope host src 192.168.2.158 
broadcast 192.168.2.255 dev eth0 table local proto kernel scope link src 192.168.2.158 
default via 192.168.2.1 dev eth0 
127.0.0.0/8 dev lo scope link 
192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.100 
192.168.2.0/24 dev eth0 proto kernel scope link src 192.168.2.158 
local ::1 dev lo proto none metric 0 
local fe80::7a29:edff:feb9:f4d7 dev lo proto none metric 0 
local fe80::7a29:edff:feb9:f4d7 dev lo proto none metric 0 
local fe80::7a29:edff:feb9:f4d7 dev lo proto none metric 0 
fe80::/64 dev eth3 proto kernel metric 256 
fe80::/64 dev eth1 proto kernel metric 256 
fe80::/64 dev eth0 proto kernel metric 256 
fec1::/16 dev eth0 proto static src fec3::8 metric 1024 
local fec3::8 dev lo proto none metric 0 
fec3::8 dev eth0 proto kernel metric 256 
ff00::/8 dev eth3 metric 256 
ff00::/8 dev eth1 metric 256 
ff00::/8 dev eth0 metric 256 
unreachable default dev lo proto kernel metric 4294967295  error -101

ip xfrm state

src 192.168.2.158 dst 192.168.2.157
    proto esp spi 0xc073bdcc reqid 3 mode tunnel
    replay-window 0 
    auth-trunc hmac(sha1) 0xa0c9794a2459e8d32408b1f170a731b24979be21 96
    enc cbc(aes) 0xe0ac7c0f10cc3c62f95f071b6960e8aa
    sel src 0.0.0.0/0 dst 0.0.0.0/0 
src 192.168.2.157 dst 192.168.2.158
    proto esp spi 0xcc653cdb reqid 3 mode tunnel
    replay-window 0 
    auth-trunc hmac(sha1) 0x545fdf3d2e265c1367c79333d5f9367fb34464ff 96
    enc cbc(aes) 0xe5b2c3a2b1fbb5609c1a3cb948a0af41
    sel src 0.0.0.0/0 dst 0.0.0.0/0

ping.pcap (2.44 KB) ping.pcap Sean Yuan, 18.09.2020 05:49

History

#1 Updated by Tobias Brunner about 1 month ago

  • Status changed from New to Feedback

carol: ipsec statusall

Status of IKE charon daemon (strongSwan 5.6.0, Linux 3.10.84-perf, armv7l):

These are pretty old strongSwan and Linux versions, so might be due to that.

ip xfrm state

...
    sel src 0.0.0.0/0 dst 0.0.0.0/0 
...

This doesn't look right. Tunnel mode SAs should not have a selector (and the family would be wrong). I don't think we ever installed them for tunnel mode SAs, so this might be an issue with the kernel or iproute2 is just reporting this incorrectly.

Check for errors in /proc/net/xfrm_stat (if available), or in ip -s xfrm state.

#2 Updated by Sean Yuan about 1 month ago

Tobias Brunner wrote:

carol: ipsec statusall
[...]

These are pretty old strongSwan and Linux versions, so might be due to that.

ip xfrm state
[...]

This doesn't look right. Tunnel mode SAs should not have a selector (and the family would be wrong). I don't think we ever installed them for tunnel mode SAs, so this might be an issue with the kernel or iproute2 is just reporting this incorrectly.

Check for errors in /proc/net/xfrm_stat (if available), or in ip -s xfrm state.

/proc/net/xfrm_stat is not enabled on carol, here is the output of ip -s xfrm state.It seems no error counted.

# ip -s xfrm state
src 192.168.2.158 dst 192.168.2.157
    proto esp spi 0xc0d759b4(3235338676) reqid 9(0x00000009) mode tunnel
    replay-window 0 seq 0x00001015 flag  (0x00000000)
    auth-trunc hmac(sha1) 0xfd751287bab12efd954026265560d154cc23e606 (160 bits) 96
    enc cbc(aes) 0x8929946b0bb191f715f1d7cac96e57eb (128 bits)
    sel src 0.0.0.0/0 dst 0.0.0.0/0 uid 0
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 3483(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      34112(bytes), 328(packets)
      add 2020-09-21 08:05:17 use 2020-09-21 08:06:25
    stats:
      replay-window 0 replay 0 failed 0
src 192.168.2.157 dst 192.168.2.158
    proto esp spi 0xcf4263ce(3477234638) reqid 9(0x00000009) mode tunnel
    replay-window 0 seq 0x00001014 flag  (0x00000000)
    auth-trunc hmac(sha1) 0x3f80c3c67a3d01cd9a85ff6cb43c212b0c70d31b (160 bits) 96
    enc cbc(aes) 0xe0c7cb179120324cf9c6d59fef930e5b (128 bits)
    sel src 0.0.0.0/0 dst 0.0.0.0/0 uid 0
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 3537(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      34112(bytes), 328(packets)
      add 2020-09-21 08:05:17 use 2020-09-21 08:06:25
    stats:
      replay-window 0 replay 0 failed 0

#3 Updated by Tobias Brunner about 1 month ago

It seems no error counted.

Yes, and even the packet counters increase (wasn't the case in your previous output of ipsec statusall), so this looks OK.

#4 Updated by Sean Yuan about 1 month ago

Tobias Brunner wrote:

It seems no error counted.

Yes, and even the packet counters increase (wasn't the case in your previous output of ipsec statusall), so this looks OK.

Sorry, I had make a mistake that I did not ping alice in the previous ipsec statusall log. Here is the correct state.
Both ipsec statusall and ip -s xfrm state have input packets counted.

# ip -s xfrm state
src 192.168.2.158 dst 192.168.2.157
    proto esp spi 0xc9d090db(3385888987) reqid 1(0x00000001) mode tunnel
    replay-window 0 seq 0x00000006 flag  (0x00000000)
    auth-trunc hmac(sha1) 0x55f6064ffac0b5e8665842078e742795f026ae81 (160 bits) 96
    enc cbc(aes) 0x1dae38c8f7ecd30239215872c5a07f38 (128 bits)
    sel src 0.0.0.0/0 dst 0.0.0.0/0 uid 0
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 3481(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      25688(bytes), 247(packets)
      add 2020-09-21 15:11:02 use 2020-09-21 15:12:01
    stats:
      replay-window 0 replay 0 failed 0
src 192.168.2.157 dst 192.168.2.158
    proto esp spi 0xc02e49ef(3224259055) reqid 1(0x00000001) mode tunnel
    replay-window 0 seq 0x00000005 flag  (0x00000000)
    auth-trunc hmac(sha1) 0xcd2131e32ab1c63ba2085531a3521e475714695c (160 bits) 96
    enc cbc(aes) 0x73be83c42aa91c6cea38e0f9ab642ce0 (128 bits)
    sel src 0.0.0.0/0 dst 0.0.0.0/0 uid 0
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 3534(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      25688(bytes), 247(packets)
      add 2020-09-21 15:11:02 use 2020-09-21 15:12:01
    stats:
      replay-window 0 replay 0 failed 0
# ipsec statusall
Status of IKE charon daemon (strongSwan 5.6.0, Linux 3.10.84-perf, armv7l):
  uptime: 6 minutes, since Sep 21 15:10:14 2020
  malloc: sbrk 1216512, mmap 0, used 227400, free 989112
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 4
  loaded plugins: charon aes des rc2 sha2 sha1 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp curve25519 xcbc cmac hmac attr kernel-pfkey kernel-netlink resolve socket-default stroke vici updown eap-identity eap-sim eap-aka xauth-generic
Listening IP addresses:
  192.168.0.100
  192.168.2.158
Connections:
      conn-1:  %any...192.168.2.157  IKEv2, dpddelay=120s
      conn-1:   local:  [C=com, O=myvpn, CN=VPN Client] uses public key authentication
      conn-1:    cert:  "C=com, O=myvpn, CN=VPN Client" 
      conn-1:   remote: uses public key authentication
      conn-1:   child:  dynamic === fec1::/16 TUNNEL, dpdaction=clear
Security Associations (1 up, 0 connecting):
      conn-1[1]: ESTABLISHED 5 minutes ago, 192.168.2.158[C=com, O=myvpn, CN=VPN Client]...192.168.2.157[C=com, O=myvpn, CN=211.75.141.110]
      conn-1[1]: IKEv2 SPIs: 15c546b36b5ccbea_i* d5ccd4f4256d4c21_r, rekeying in 2 minutes, public key reauthentication in 50 minutes
      conn-1[1]: IKE proposal: AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
      conn-1{1}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: c02e49ef_i c9d090db_o
      conn-1{1}:  AES_CBC_128/HMAC_SHA1_96, 27248 bytes_i, 27248 bytes_o (262 pkts, 0s ago), rekeying in 52 minutes
      conn-1{1}:   fec3::1/128 === fec1::/16

#5 Updated by Tobias Brunner about 1 month ago

Both ipsec statusall and ip -s xfrm state have input packets counted.

Actually, the ipsec statusall output does not show an inbound packet count, only outbound. However, it does show transmitted bytes for both directions. This indicates that for some reason the inbound policy was not used (the packet count and time since the last packet is only shown if the last use time from the policy is known). This could be due to the selector on the SA, i.e. the packet is dropped because it doesn't match it right after decryption before it even hits the inbound policy.

As mentioned before, the selector is not set for tunnel mode SAs, we also set XFRM_STATE_AF_UNSPEC on such SAs so the kernel should not assign a default address family to the selector (that flag was added with 2.6.26, so it should be supported by your kernel, and is set by strongSwan since 4.2.5). And since iproute2 only lists a selector if it is non-zero and all listed data is apparently 0, it presumably has the family set to AF_INET and not AF_UNSPEC. What the output of ip xfrm state also shows is that the XFRM_STATE_AF_UNSPEC flag is not actually set on the SAs (flag (0x00000000)), which means the kernel copied the address family of the SA to the selector when the SA was created.

The reason for this this can be seen in the output of ipsec statusall. You have the kernel-pfkey plugin loaded, which you should not on Linux as it has several limitations. This flag is apparently one of them as it simply can't be set via PF_KEYv2. So make sure you only load the kernel-netlink plugin.

#6 Updated by Sean Yuan about 1 month ago

Tobias Brunner wrote:

Both ipsec statusall and ip -s xfrm state have input packets counted.

Actually, the ipsec statusall output does not show an inbound packet count, only outbound. However, it does show transmitted bytes for both directions. This indicates that for some reason the inbound policy was not used (the packet count and time since the last packet is only shown if the last use time from the policy is known). This could be due to the selector on the SA, i.e. the packet is dropped because it doesn't match it right after decryption before it even hits the inbound policy.

As mentioned before, the selector is not set for tunnel mode SAs, we also set XFRM_STATE_AF_UNSPEC on such SAs so the kernel should not assign a default address family to the selector (that flag was added with 2.6.26, so it should be supported by your kernel, and is set by strongSwan since 4.2.5). And since iproute2 only lists a selector if it is non-zero and all listed data is apparently 0, it presumably has the family set to AF_INET and not AF_UNSPEC. What the output of ip xfrm state also shows is that the XFRM_STATE_AF_UNSPEC flag is not actually set on the SAs (flag (0x00000000)), which means the kernel copied the address family of the SA to the selector when the SA was created.

The reason for this this can be seen in the output of ipsec statusall. You have the kernel-pfkey plugin loaded, which you should not on Linux as it has several limitations. This flag is apparently one of them as it simply can't be set via PF_KEYv2. So make sure you only load the kernel-netlink plugin.

With the kernel-pfkey plugin not loaded, it works now.
Thanks for all your help.

#7 Updated by Tobias Brunner about 1 month ago

  • Category set to kernel-interface
  • Status changed from Feedback to Closed
  • Assignee set to Tobias Brunner
  • Resolution set to No change required

Also available in: Atom PDF