Project

General

Profile

Issue #2913

Need help VPN VTI connection strongswan

Added by shunsuke takahashi 3 months ago. Updated 2 months ago.

Status:
Closed
Priority:
Normal
Category:
configuration
Affected version:
5.7.2
Resolution:
No change required

Description

We want strong swan to cisco(ASA5525X) VPN VTI mode connection
ipsec statu fine, but can not access target ip

■required
- loacal ip: 52.192.162.65(AWS strong swan)
- local tunnel interface ip: 169.254.8.146
- remote ip: 202.242.25.167(cisco)
- remote tunnnel interface ip: 169.254.8.145

we wrong setting???

create vit device

$ ip tunnel add vti0 mode vti local 52.192.162.65 remote 202.242.25.167 okey 32 ikey 32
$ ip link set vti0 up
$ ip addr add 169.254.8.146/30 remote 169.254.8.145/30 dev vti0
$ echo "0" > /proc/sys/net/ipv4/conf/vti0/rp_filter
$ ip route del table 220 default
RTNETLINK answers: No such process
$ ip -s tunnel show vti0
vti0: ip/ip  remote 202.242.25.167  local 52.192.162.65  ttl inherit  key 32

check vti tunnel

$ ip -s tunnel show
ip_vti0: ip/ip  remote any  local any  ttl inherit  nopmtudisc key 0
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0
vti0: ip/ip  remote 202.242.25.167  local 52.192.162.65  ttl inherit  key 32
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0

check ipcsec status

$ sudo /usr/sbin/ipsec status
Security Associations (1 up, 0 connecting):
   ikev2-vpn[1]: ESTABLISHED 64 seconds ago, 172.30.83.192[52.192.162.65]...202.242.25.167[202.242.25.167]
   ikev2-vpn{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: c351f3c4_i 12874dec_o
   ikev2-vpn{1}:   0.0.0.0/0 === 0.0.0.0/0

strongswan config

conn ikev2-vpn
    type=tunnel
    ikelifetime=24h
    authby=secret
    auto=add
    ike=aes256-sha256-modp1024
    esp=aes256-sha256
    keyexchange=ikev2
    leftid=52.192.162.65
    leftsubnet=0.0.0.0/0
    rightid=202.242.25.167
    rightsubnet=0.0.0.0/0
    mark=32

History

#1 Updated by Tobias Brunner 2 months ago

  • Description updated (diff)
  • Category set to configuration
  • Status changed from New to Feedback
  • Assignee deleted (shunsuke takahashi)
  • Priority changed from High to Normal

The local IP on the VTI interface is definitely not correct. As can be seen in the output of ipsec status the local IP is apparently 172.30.83.192.

#2 Updated by shunsuke takahashi 2 months ago

Can we know which not correct setting, [create vit device] , [strongswan config] or something else?
If you need other info , I add it.

more info
- local server(AWS strong swan)
public ip: 52.192.162.65
private ip: 172.30.83.192

strongswan.conf config


# strongswan.conf - strongSwan configuration file
#
# Refer to the strongswan.conf(5) manpage for details
#
# Configuration changes should be made in the included files

charon {
    load_modular = yes
    install_routes=0
    plugins {
        include strongswan.d/charon/*.conf
    }
}

include strongswan.d/*.conf

ip xfrm state


$ sudo ip xfrm state
src 172.30.83.192 dst 202.242.25.167
    proto esp spi 0x3fd65112 reqid 4 mode tunnel

    replay-window 32 flag af-unspec
    mark 0x20/0xffffffff  # <<<<---- 
    auth-trunc hmac(sha256) 0x6d5e534d20d1d979887eecfe5a07bd2c80ee5417367f1fd8be52d403b1d5a1bc 128
    enc cbc(aes) 0x5a8b2b2a10d920a1bea7243add90a602b5a735285477754e7361b04d0ab52999
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 202.242.25.167 dst 172.30.83.192
    proto esp spi 0xc1c81e4b reqid 4 mode tunnel
    replay-window 32 flag af-unspec
    mark 0x20/0xffffffff # <<<<---- 
    auth-trunc hmac(sha256) 0xee90ed6be79f6a15e898ee3e7d57b0d1d2c8ffc79db2ee832bfe12da3453eee3 128
    enc cbc(aes) 0x5cd0e577ee8884bbc2dde7bf85730594042aa03470cedef6bf1fa437a5d237c7
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000

ipsec statusall


$ sudo ipsec statusall
Status of IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-1074-aws, x86_64):
  uptime: 2 hours, since Feb 06 05:30:30 2019
  malloc: sbrk 1486848, mmap 0, used 346160, free 1140688
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 9
  loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark stroke updown
Listening IP addresses:
  172.30.83.192
  169.254.8.146
Connections:
   ikev2-vpn:  %any...%any  IKEv2
   ikev2-vpn:   local:  [52.192.162.65] uses pre-shared key authentication
   ikev2-vpn:   remote: [202.242.25.167] uses pre-shared key authentication
   ikev2-vpn:   child:  0.0.0.0/0 === 0.0.0.0/0 TUNNEL
Security Associations (1 up, 0 connecting):
   ikev2-vpn[4]: ESTABLISHED 21 minutes ago, 172.30.83.192[52.192.162.65]...202.242.25.167[202.242.25.167]
   ikev2-vpn[4]: IKEv2 SPIs: 28bcfcf7c3f457b5_i 8200264846676f8f_r*, pre-shared key reauthentication in 23 hours
   ikev2-vpn[4]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
   ikev2-vpn{19}:  INSTALLED, TUNNEL, reqid 4, ESP in UDP SPIs: c1c81e4b_i 3fd65112_o
   ikev2-vpn{19}:  AES_CBC_256/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 22 minutes
   ikev2-vpn{19}:   0.0.0.0/0 === 0.0.0.0/0

#3 Updated by Tobias Brunner 2 months ago

Can we know which not correct setting, [create vit device] , [strongswan config] or something else?

What's unclear about my response? You configured a local IP address on the VTI interface (local 52.192.162.65) that does not match the local IP of the negotiated SAs (172.30.83.192). So change the VTI's local address to the one that's actually used.

#4 Updated by shunsuke takahashi 2 months ago

thank you for replay.
We recreated vti device, bad not access target ip.
other incorrect settings or not enough settings?

recreate vti device

$ ip tunnel del vti0
$ ip tunnel add vti0 mode vti local 172.30.83.192 remote 202.242.25.167 okey 32 ikey 32
$ ip link set vti0 up
$ ip addr add 169.254.8.146/30 remote 169.254.8.145/30 dev vti0
$ echo "0" > /proc/sys/net/ipv4/conf/vti0/rp_filter
$ ip route del table 220 default
$ ip -s tunnel show vti0
vti0: ip/ip  remote 202.242.25.167  local 172.30.83.192  ttl inherit  key 32
$ ip -s tunnel show
ip_vti0: ip/ip  remote any  local any  ttl inherit  nopmtudisc key 0
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0
vti0: ip/ip  remote 202.242.25.167  local 172.30.83.192  ttl inherit  key 32
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0
ipsec.conf
```
conn ikev2-vpn
    type=tunnel
    ikelifetime=24h
    authby=secret
    auto=add
    ike=aes256-sha256-modp1024
    esp=aes256-sha256
    keyexchange=ikev2
    leftid=172.30.83.192
    leftsubnet=0.0.0.0/0
    rightid=202.242.25.167
    rightsubnet=0.0.0.0/0
    mark=32

restart strong swan

$ sudo systemctl stop strongswan
sudo: unable to resolve host vpn-phase1
$ sudo systemctl start strongswan
sudo: unable to resolve host vpn-phase1
$ /usr/sbin/ipsec status
Security Associations (1 up, 0 connecting):
   ikev2-vpn[1]: ESTABLISHED 0 seconds ago, 172.30.83.192[172.30.83.192]...202.242.25.167[202.242.25.167]
   ikev2-vpn{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: cab49836_i 80c98195_o
   ikev2-vpn{1}:   0.0.0.0/0 === 0.0.0.0/0

testing

$ ping 172.30.3.193
PING 172.30.3.193 (172.30.3.193) 56(84) bytes of data.
^C
--- 172.30.3.193 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1008ms

$ ping 169.254.8.145
PING 169.254.8.145 (169.254.8.145) 56(84) bytes of data.
^C
--- 169.254.8.145 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1007ms

$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 172.30.3.193 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6048ms

$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 169.254.8.145 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5038ms

$ ip -s tunnel show
ip_vti0: ip/ip  remote any  local any  ttl inherit  nopmtudisc key 0
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0
vti0: ip/ip  remote 202.242.25.167  local 172.30.83.192  ttl inherit  key 32
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    13         1092         0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    13         1092         0      0        0        0
$ ip xfrm state
src 172.30.83.192 dst 202.242.25.167
    proto esp spi 0x80c98195 reqid 1 mode tunnel
    replay-window 32 flag af-unspec
    mark 0x20/0xffffffff
    auth-trunc hmac(sha256) 0xb8e92570a53da5b6067f62aa03ac274b77bf3ff7f1ea6415260bd90e856b3490 128
    enc cbc(aes) 0x26c14556703d378bb879ced2bc24f465cb58914f844320aa28176a566f3fe9e4
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0xd, bitmap 0x00000000
src 202.242.25.167 dst 172.30.83.192
    proto esp spi 0xcab49836 reqid 1 mode tunnel
    replay-window 32 flag af-unspec
    mark 0x20/0xffffffff
    auth-trunc hmac(sha256) 0x287d5cf84ed5a4f04a38899568ab27ff3bc27a274fc6f83ffd8e4406527c95c3 128
    enc cbc(aes) 0xaa14c96bbdaca2c996fe31c34034823efd18a9e986315831b1a69d2cb7bb1e29
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0xd, oseq 0x0, bitmap 0x00001fff
$ ip xfrm policy
src 0.0.0.0/0 dst 0.0.0.0/0
    dir fwd priority 3075
    mark 0x20/0xffffffff
    tmpl src 202.242.25.167 dst 172.30.83.192
        proto esp reqid 1 mode tunnel
src 0.0.0.0/0 dst 0.0.0.0/0
    dir in priority 3075
    mark 0x20/0xffffffff
    tmpl src 202.242.25.167 dst 172.30.83.192
        proto esp reqid 1 mode tunnel
src 0.0.0.0/0 dst 0.0.0.0/0
    dir out priority 3075
    mark 0x20/0xffffffff
    tmpl src 172.30.83.192 dst 202.242.25.167
        proto esp reqid 1 mode tunnel
src 0.0.0.0/0 dst 0.0.0.0/0
    socket in priority 0
src 0.0.0.0/0 dst 0.0.0.0/0
    socket out priority 0
src 0.0.0.0/0 dst 0.0.0.0/0
    socket in priority 0
src 0.0.0.0/0 dst 0.0.0.0/0
    socket out priority 0
src ::/0 dst ::/0
    socket in priority 0
src ::/0 dst ::/0
    socket out priority 0
src ::/0 dst ::/0
    socket in priority 0
src ::/0 dst ::/0
    socket out priority 0

#6 Updated by Tobias Brunner 2 months ago

Did you already read RouteBasedVPN? Anyway, according to the interface stats some packets are exchanged (check with ipsec statusall or ip -s xfrm state whether they were tunneled). Maybe there is a firewall rule that drops them. Checking for errors might also be a good idea (ip -s xfrm state and cat /proc/net/xfrm_stat). Other than that you need to debug this further (check where packets get dropped, maybe the problem is on the other end etc.).

#7 Updated by Tobias Brunner 2 months ago

Did you already read RouteBasedVPN?

yes, alredy read.

Then why, for instance, this command:

$ ip route del table 220 default

Other than that you need to debug this further (check where packets get dropped, maybe the problem is on the other end etc.).

How to debug?

Counters, iptables debugging, tcpdump/wireshark, whatever is available on the other end etc.

check result ↓

Did you run these commands after the different ping commands you tried before? What about the counters of the VTI interface?

#8 Updated by shunsuke takahashi 2 months ago

Then why, for instance, this command:

sorry other site see this command.

we change VTI device command.
Are there match configration , we uploaded attachment?

$ ip tunnel del vti0
$ ip tunnel add vti0 mode vti local 172.30.83.192 remote 202.242.25.167 key 32
$ sysctl -w net.ipv4.conf.vti0.disable_policy=1
$ ip link set vti0 up
$ ip addr add 169.254.8.146/30 remote 169.254.8.145/30 dev vti0

Did you run these commands after the different ping commands you tried before?

yes there results after ping commands.

What about the counters of the VTI interface?

you say mean the counters of the VTI interface is this?

$ ip -s tunnel show
ip_vti0: ip/ip  remote any  local any  ttl inherit  nopmtudisc key 0
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0
vti0: ip/ip  remote 202.242.25.167  local 172.30.83.192  ttl inherit  key 32
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    3          252          0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    3          252          0      0        0        0

send ping

$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 169.254.8.145 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6047ms

$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 172.30.3.193 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 2999ms

reply?!? bad ping command failed.
why???

$ tcpdump -i vti0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti0, link-type RAW (Raw IP), capture size 262144 bytes
10:20:00.093801 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 2649, seq 1, length 64
10:20:00.103587 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 2649, seq 1, length 64
10:20:01.101358 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 2649, seq 2, length 64
10:20:01.111084 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 2649, seq 2, length 64
10:20:02.109344 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 2649, seq 3, length 64
10:20:02.118935 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 2649, seq 3, length 64
10:20:03.117316 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 2649, seq 4, length 64
10:20:03.126596 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 2649, seq 4, length 64
10:20:04.125336 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 2649, seq 5, length 64

10:26:19.335955 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 2660, seq 1, length 64
10:26:19.346138 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 2660, seq 1, length 64
10:26:20.335182 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 2660, seq 2, length 64
10:26:20.345390 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 2660, seq 2, length 64
10:26:21.335201 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 2660, seq 3, length 64
10:26:21.352308 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 2660, seq 3, length 64
10:26:22.335183 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 2660, seq 4, length 64
10:26:22.344750 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 2660, seq 4, length 64

#9 Updated by Tobias Brunner 2 months ago

Did you run these commands after the different ping commands you tried before?

yes there results after ping commands.

That means no IPsec packets were sent or received, even though...

What about the counters of the VTI interface?

you say mean the counters of the VTI interface is this?
...

vti0: ip/ip  remote 202.242.25.167  local 172.30.83.192  ttl inherit  key 32
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    3          252          0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    3          252          0      0        0        0

...the VTI interface processed three packets (in both directions) and no errors are counted. That seems strange.

$ tcpdump -i vti0

This shows that ICMP packets are exchanged over the interface. What do you see on the physical interface? Any ESP packets? If so, the counters on the SA should increase too (which your previous commands didn't show). And it's strange that ping doesn't recognize the response (maybe it is dropped by the firewall, or some other mechanism in the kernel, e.g. routing).

#10 Updated by shunsuke takahashi 2 months ago

■we want to see there.
1. created enabled VTI device `169.254.8.146 ~ 169.254.8.145`
2. success communication test `169.254.8.146 to 172.30.3.193` and `169.254.8.146` to `169.254.8.145`

sorry I not speak English,
I am very grateful of your cooperation.

#11 Updated by shunsuke takahashi 2 months ago

sorry I misstake delete command comment.

re write↓


$ ipsec statusall
Status of IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-1074-aws, x86_64):
  uptime: 2 hours, since Feb 08 10:01:38 2019
  malloc: sbrk 1486848, mmap 0, used 346304, free 1140544
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 9
  loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark stroke updown
Listening IP addresses:
  172.30.83.192
  169.254.8.146
Connections:
   ikev2-vpn:  %any...%any  IKEv2
   ikev2-vpn:   local:  [172.30.83.192] uses pre-shared key authentication
   ikev2-vpn:   remote: [202.242.25.167] uses pre-shared key authentication
   ikev2-vpn:   child:  0.0.0.0/0 === 0.0.0.0/0 TUNNEL
Security Associations (1 up, 0 connecting):
   ikev2-vpn[4]: ESTABLISHED 9 minutes ago, 172.30.83.192[172.30.83.192]...202.242.25.167[202.242.25.167]
   ikev2-vpn[4]: IKEv2 SPIs: b7c1c97f47b17736_i 3dfc2857bde80253_r*, pre-shared key reauthentication in 23 hours
   ikev2-vpn[4]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
   ikev2-vpn{19}:  INSTALLED, TUNNEL, reqid 4, ESP in UDP SPIs: c92a576b_i 313476e4_o
   ikev2-vpn{19}:  AES_CBC_256/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 38 minutes
   ikev2-vpn{19}:   0.0.0.0/0 === 0.0.0.0/0
root@vpn-phase1:/etc#
root@vpn-phase1:/etc#
$ ip -s xfrm state
src 172.30.83.192 dst 202.242.25.167
    proto esp spi 0x313476e4(825521892) reqid 4(0x00000004) mode tunnel
    replay-window 32 seq 0x00000000 flag af-unspec (0x00100000)
    mark 0x20/0xffffffff
    auth-trunc hmac(sha256) 0xf0def01705dc1d800bbe8686ac0c5dd012d70888a644b36d925e49516f404cdc (256 bits) 128
    enc cbc(aes) 0x0c094ec4e928cddcb174509f0365a60479fef5eed1b82eca2b205f838dfd8c90 (256 bits)
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 2851(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      0(bytes), 0(packets)
      add 2019-02-08 12:18:58 use -
    stats:
      replay-window 0 replay 0 failed 0
src 202.242.25.167 dst 172.30.83.192
    proto esp spi 0xc92a576b(3374995307) reqid 4(0x00000004) mode tunnel
    replay-window 32 seq 0x00000000 flag af-unspec (0x00100000)
    mark 0x20/0xffffffff
    auth-trunc hmac(sha256) 0xcc8735dcd4b34ed3d0d33b843ecd48a8402904e717c2fda28fdbf0e54479616a (256 bits) 128
    enc cbc(aes) 0xb7b4f4a61ef57a3700b30eb368ec54e4421fe3e6618e76eec140c6f6bbfd7c84 (256 bits)
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 3030(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      0(bytes), 0(packets)
      add 2019-02-08 12:18:58 use -
    stats:
      replay-window 0 replay 0 failed 0
root@vpn-phase1:/etc#
root@vpn-phase1:/etc#
$ cat /proc/net/xfrm_stat
XfrmInError                 0
XfrmInBufferError           0
XfrmInHdrError              0
XfrmInNoStates              0
XfrmInStateProtoError       0
XfrmInStateModeError        0
XfrmInStateSeqError         0
XfrmInStateExpired          0
XfrmInStateMismatch         0
XfrmInStateInvalid          0
XfrmInTmplMismatch          0
XfrmInNoPols                0
XfrmInPolBlock              0
XfrmInPolError              0
XfrmOutError                0
XfrmOutBundleGenError       0
XfrmOutBundleCheckError     0
XfrmOutNoStates             0
XfrmOutStateProtoError      0
XfrmOutStateModeError       0
XfrmOutStateSeqError        0
XfrmOutStateExpired         0
XfrmOutPolBlock             0
XfrmOutPolDead              0
XfrmOutPolError             0
XfrmFwdHdrError             0
XfrmOutStateInvalid         0
XfrmAcquireError            0

Is this route correct??

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.31.80.1     0.0.0.0         UG    0      0        0 eth0
169.254.8.144   0.0.0.0         255.255.255.252 U     0      0        0 vti0
172.31.80.0     0.0.0.0         255.255.240.0   U     0      0        0 eth0

why this route?

169.254.8.144   0.0.0.0         255.255.255.252 U     0      0        0 vti0

Im setting this command

ip addr add 169.254.8.146/30 remote 169.254.8.145/30 dev vti0

#12 Updated by Tobias Brunner 2 months ago

Again, this is very strange. While the interface shows packets getting processed, the output of these commands does not show any processed packets:

...
   ikev2-vpn{19}:  AES_CBC_256/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 38 minutes
...
    lifetime current:
      0(bytes), 0(packets)
      add 2019-02-08 12:18:58 use -
...
    lifetime current:
      0(bytes), 0(packets)
      add 2019-02-08 12:18:58 use -

So what happens to the packets sent to the VTI if they are not tunneled, and where does the response come from you see in tcpdump?

Is this route correct??

Please don't use the route command, use ip route instead.

why this route?
[...]

It's probably added by the command you quoted here:

Im setting this command

ip addr add 169.254.8.146/30 remote 169.254.8.145/30 dev vti0

Why do you set remote 169.254.8.145/30? (Not sure if that's even valid, peer is usually used for point-to-point interfaces, which VTIs are not really.) You can also just use ip addr add 169.254.8.146/32 dev vti0 and add a route to the other IP with ip route add 169.254.8.145/32 dev vti0 src 169.254.8.146/32 (and maybe ip route add 172.30.3.193/32 dev vti0 src 169.254.8.146/32 if you want to reach that IP too).

#13 Updated by shunsuke takahashi 2 months ago

Why do you set remote 169.254.8.145/30? (Not sure if that's even valid, peer is usually used for point-to-point interfaces, which VTIs are not really.) You can also just use ip addr add 169.254.8.146/32 dev vti0 and add a route to the other IP with ip route add 169.254.8.145/32 dev vti0 src 169.254.8.146/32 (and maybe ip route add 172.30.3.193/32 dev vti0 src 169.254.8.146/32 if you want to reach that IP too).

we recreate vti device.
but failed...

$ ip tunnel del vti0
$ ip tunnel add vti0 mode vti local 172.30.83.192 remote 202.242.25.167 key 32
$ ip link set vti0 up
$ ip addr add 169.254.8.146 dev vti0
$ ip route add 169.254.8.145 dev vti0 src 169.254.8.146
$ ip route add 172.30.3.193 dev vti0 src 169.254.8.146
$ sysctl -w net.ipv4.conf.vti0.disable_policy=1
$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 172.30.3.193 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms

$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 169.254.8.145 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms

$ ip -s tunnel show
ip_vti0: ip/ip  remote any  local any  ttl inherit  nopmtudisc key 0
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0
vti0: ip/ip  remote 202.242.25.167  local 172.30.83.192  ttl inherit  key 32
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    11         924          0      0        0        0
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    11         924          0      0        0        0
$ ip -s xfrm state
src 172.30.83.192 dst 202.242.25.167
    proto esp spi 0x25b87c84(632847492) reqid 1(0x00000001) mode tunnel
    replay-window 32 seq 0x00000000 flag af-unspec (0x00100000)
    mark 0x20/0xffffffff
    auth-trunc hmac(sha256) 0x08b0bbd75f7f73c47f80acc60515b4d85acb4d7b0b3a2b86c8ad52f9142a7e30 (256 bits) 128
    enc cbc(aes) 0x55a16dbdaa0c528cf2f72bfab13ca0e476e7d522a0bd5546a67d20b6a97d2eb7 (256 bits)
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x1a, bitmap 0x00000000
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 2737(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      2184(bytes), 26(packets)
      add 2019-02-08 14:30:55 use 2019-02-08 14:46:35
    stats:
      replay-window 0 replay 0 failed 0
src 202.242.25.167 dst 172.30.83.192
    proto esp spi 0xc91151ec(3373355500) reqid 1(0x00000001) mode tunnel
    replay-window 32 seq 0x00000000 flag af-unspec (0x00100000)
    mark 0x20/0xffffffff
    auth-trunc hmac(sha256) 0xb3e340f0fa84ba703295ce0a2d78eb799cde6d0063bec05125d581d62e205158 (256 bits) 128
    enc cbc(aes) 0x0009417de03595cb64e8e5715d34cdee22c3164ca0cf78c5cd7502ae24f2e65c (256 bits)
    encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x1a, oseq 0x0, bitmap 0x03ffffff
    lifetime config:
      limit: soft (INF)(bytes), hard (INF)(bytes)
      limit: soft (INF)(packets), hard (INF)(packets)
      expire add: soft 2970(sec), hard 3600(sec)
      expire use: soft 0(sec), hard 0(sec)
    lifetime current:
      2184(bytes), 26(packets)
      add 2019-02-08 14:30:55 use 2019-02-08 14:46:35
    stats:
      replay-window 0 replay 0 failed 0
$ cat /proc/net/xfrm_stat
XfrmInError                 0
XfrmInBufferError           0
XfrmInHdrError              0
XfrmInNoStates              0
XfrmInStateProtoError       0
XfrmInStateModeError        0
XfrmInStateSeqError         0
XfrmInStateExpired          0
XfrmInStateMismatch         0
XfrmInStateInvalid          0
XfrmInTmplMismatch          0
XfrmInNoPols                0
XfrmInPolBlock              0
XfrmInPolError              0
XfrmOutError                0
XfrmOutBundleGenError       0
XfrmOutBundleCheckError     0
XfrmOutNoStates             0
XfrmOutStateProtoError      0
XfrmOutStateModeError       0
XfrmOutStateSeqError        0
XfrmOutStateExpired         0
XfrmOutPolBlock             0
XfrmOutPolDead              0
XfrmOutPolError             0
XfrmFwdHdrError             0
XfrmOutStateInvalid         0
XfrmAcquireError            0
$ ipsec statusall
Status of IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-1074-aws, x86_64):
  uptime: 24 minutes, since Feb 08 14:30:52 2019
  malloc: sbrk 1486848, mmap 0, used 344864, free 1141984
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 3
  loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark stroke updown
Listening IP addresses:
  172.30.83.192
  169.254.8.146
Connections:
   ikev2-vpn:  %any...%any  IKEv2
   ikev2-vpn:   local:  [172.30.83.192] uses pre-shared key authentication
   ikev2-vpn:   remote: [202.242.25.167] uses pre-shared key authentication
   ikev2-vpn:   child:  0.0.0.0/0 === 0.0.0.0/0 TUNNEL
Security Associations (1 up, 0 connecting):
   ikev2-vpn[1]: ESTABLISHED 24 minutes ago, 172.30.83.192[172.30.83.192]...202.242.25.167[202.242.25.167]
   ikev2-vpn[1]: IKEv2 SPIs: 2ec75cacf4190f61_i 488b558b171490c3_r*, pre-shared key reauthentication in 23 hours
   ikev2-vpn[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
   ikev2-vpn{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: c91151ec_i 25b87c84_o
   ikev2-vpn{1}:  AES_CBC_256/HMAC_SHA2_256_128, 2184 bytes_i (26 pkts, 47s ago), 2184 bytes_o (26 pkts, 47s ago), rekeying in 20 minutes
   ikev2-vpn{1}:   0.0.0.0/0 === 0.0.0.0/0
$ ip route
default via 172.31.80.1 dev eth0
169.254.8.145 dev vti0  scope link  src 169.254.8.146
172.30.3.193 dev vti0  scope link  src 169.254.8.146
172.31.80.0/20 dev eth0  proto kernel  scope link  src 172.30.83.192

tunneled tcpdump

$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 172.30.3.193 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms

$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 169.254.8.145 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms

$ tcpdump -i vti0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti0, link-type RAW (Raw IP), capture size 262144 bytes
14:54:43.920520 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5165, seq 1, length 64
14:54:43.932033 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 5165, seq 1, length 64
14:54:44.929337 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5165, seq 2, length 64
14:54:44.952911 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 5165, seq 2, length 64
14:54:45.937336 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5165, seq 3, length 64
14:54:45.946980 IP ip-172-30-3-193.ap-northeast-1.compute.internal > 169.254.8.146: ICMP echo reply, id 5165, seq 3, length 64
14:54:50.660648 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5166, seq 1, length 64
14:54:50.669750 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 5166, seq 1, length 64
14:54:51.669298 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5166, seq 2, length 64
14:54:51.678468 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 5166, seq 2, length 64
14:54:52.677351 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5166, seq 3, length 64
14:54:52.686532 IP 169.254.8.145 > 169.254.8.146: ICMP echo reply, id 5166, seq 3, length 64

not tunneled tcpdump

systemctl stop strongswan
$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
From 169.254.8.146 icmp_seq=1 Destination Host Unreachable
From 169.254.8.146 icmp_seq=2 Destination Host Unreachable
From 169.254.8.146 icmp_seq=3 Destination Host Unreachable
From 169.254.8.146 icmp_seq=4 Destination Host Unreachable
^C
--- 172.30.3.193 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 2999ms

$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
From 169.254.8.146 icmp_seq=1 Destination Host Unreachable
From 169.254.8.146 icmp_seq=2 Destination Host Unreachable
From 169.254.8.146 icmp_seq=3 Destination Host Unreachable
From 169.254.8.146 icmp_seq=4 Destination Host Unreachable
^C
--- 169.254.8.145 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 2997ms

15:05:52.156097 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5253, seq 1, length 64
15:05:53.155197 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5253, seq 2, length 64
15:05:54.155193 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5253, seq 3, length 64
15:05:55.155185 IP 169.254.8.146 > ip-172-30-3-193.ap-northeast-1.compute.internal: ICMP echo request, id 5253, seq 4, length 64
15:06:01.186322 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5254, seq 1, length 64
15:06:02.185335 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5254, seq 2, length 64
15:06:03.184337 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5254, seq 3, length 64
15:06:04.183339 IP 169.254.8.146 > 169.254.8.145: ICMP echo request, id 5254, seq 4, length 64

#14 Updated by Tobias Brunner 2 months ago

Now everything looks good (in regards to the counters, they don't all match up, but at least all of them increase). Only your ping command still thinks there has not been any response, even though tcpdump clearly shows one (as do the counters that increase in both directions). Did you check your firewall rules (please post iptables-save)?

#15 Updated by shunsuke takahashi 2 months ago

Now everything looks good

so good!

iptables-save response

$ iptables-save

root@vpn-phase1:/home/ubuntu# iptables-save
# Generated by iptables-save v1.6.0 on Fri Feb  8 17:10:32 2019
*filter
:INPUT ACCEPT [255569:36216784]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [627954:250490889]
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 2 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 2 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 1 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 1 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 56 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 56 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 55 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 55 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 54 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 54 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 53 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 53 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 52 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 52 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 51 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 51 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 50 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 50 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 49 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 49 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 48 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 48 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 47 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 47 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 46 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 46 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 45 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 45 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 44 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 44 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 43 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 43 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 42 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 42 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 41 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 41 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 40 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 40 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 39 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 39 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 38 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 38 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 37 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 37 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 36 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 36 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 35 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 35 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 34 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 34 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 33 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 33 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 32 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 32 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 31 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 31 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 30 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 30 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 29 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 29 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 28 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 28 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 27 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 27 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 26 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 26 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 25 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 25 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 24 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 24 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 23 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 23 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 22 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 22 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 21 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 21 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 20 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 20 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 19 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 19 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 18 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 18 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 17 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 17 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 16 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 16 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 15 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 15 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 14 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 14 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 13 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 13 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 12 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 12 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 11 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 11 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 10 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 10 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 9 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 9 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 8 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 8 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 7 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 7 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 6 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 6 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 5 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 5 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 4 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 4 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 3 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 3 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 2 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 2 --proto esp -j ACCEPT
-A FORWARD -i eth0 -m policy --dir in --pol ipsec --reqid 1 --proto esp -j ACCEPT
-A FORWARD -o eth0 -m policy --dir out --pol ipsec --reqid 1 --proto esp -j ACCEPT
COMMIT
# Completed on Fri Feb  8 17:10:32 2019

#17 Updated by Tobias Brunner 2 months ago

iptables rules look OK (you don't filter anything at all, so these FORWARD rules are not really necessary and all inbound packets should be accepted).

AWS route table setting

AWS Security group setting

Not sure if these have any effect on this (but I don't know AWS at all). I'd guess these settings are more for traffic outside the actual Linux VM (and tcpdump shows that ICMPs are sent/received locally).

Anyway, this does not really explain why the ping command thinks there has not been any response. Is rp_filter still disabled (e.g. sysctl -w net.ipv4.conf.all.rp_filter=0 or try setting it to 2 for loose mode)? Could also be a kernel issue (e.g. with marks, VTIs, or specific to ICMP). Did you try traffic other than ICMP (UDP/TCP, e.g. SSH or something)? Or a different/newer kernel?

#18 Updated by shunsuke takahashi 2 months ago

..Fail
$ sysctl -w net.ipv4.conf.all.rp_filter=0

$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 172.30.3.193 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
^C
--- 169.254.8.145 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1009ms

.. Oh success!! What does this mean?
Is it dangerous to be in this setting?
$ sysctl -w net.ipv4.conf.all.rp_filter=2

$ ping 172.30.3.193 -I vti0
PING 172.30.3.193 (172.30.3.193) from 169.254.8.146 vti0: 56(84) bytes of data.
64 bytes from 172.30.3.193: icmp_seq=1 ttl=255 time=13.1 ms
64 bytes from 172.30.3.193: icmp_seq=2 ttl=255 time=11.2 ms
^C
--- 172.30.3.193 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 11.233/12.181/13.130/0.954 ms
$ ping 169.254.8.145 -I vti0
PING 169.254.8.145 (169.254.8.145) from 169.254.8.146 vti0: 56(84) bytes of data.
64 bytes from 169.254.8.145: icmp_seq=1 ttl=255 time=9.85 ms
64 bytes from 169.254.8.145: icmp_seq=2 ttl=255 time=10.1 ms
^C
--- 169.254.8.145 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 9.858/10.028/10.199/0.197 ms

#19 Updated by Tobias Brunner 2 months ago

.. Oh success!! What does this mean?

Not sure exactly, because it seems strange to me that disabling the filter completely (0) doesn't work, while using loose mode (2) does (it's why I suggested trying both). It's possible that some interface specific setting overrides this to strict mode (1) as the kernel apparently uses the maximum number that's configured.

Is it dangerous to be in this setting?
$ sysctl -w net.ipv4.conf.all.rp_filter=2
[...]

You can read more about this filtering in RFC 3704. The loose mode is described in section 2.4, some notes are in section 4.1 and the security considerations in section 5. It might be possible to set this only for the VTI with sysctl -w net.ipv4.conf.vti0.rp_filter=2, so it wouldn't affect the inbound packets on the physical interface.

#20 Updated by shunsuke takahashi 2 months ago

Hi, I found rp_filter setting (0) doesn't work reason.
rp_filter default value is (1),
both rp_filter set (0) when succeed.

sysctl -w net.ipv4.conf.all.rp_filter=0
sysctl -w net.ipv4.conf.vti0.rp_filter=0

but , we will setting is this.

sysctl -w net.ipv4.conf.all.rp_filter=1
sysctl -w net.ipv4.conf.vti0.rp_filter=2

by the way, today tried both connection test.

169.254.8.146(aws VTI) to 169.254.8.145(cisco VTI) -> ok
169.254.8.146(aws VTI) to 172.30.3.193(cisco local server) -> ok
169.254.8.145(cisco VTI) to 169.254.8.146(aws VTI) -> ng
169.254.8.145(cisco VTI) to 172.30.66.3(aws local server) -> ng

tcpdump -i vti0 result.
aws strong swan server not reply.
which not enough setting???

07:18:55.213210 IP 169.254.8.145 > 169.254.8.146: ICMP echo request, id 14086, seq 37136, length 80
07:18:57.213364 IP 169.254.8.145 > 169.254.8.146: ICMP echo request, id 14087, seq 37136, length 80
07:18:59.212774 IP 169.254.8.145 > 169.254.8.146: ICMP echo request, id 14088, seq 37136, length 80

#21 Updated by Tobias Brunner 2 months ago

No idea. Seems strange that it would work in one direction but not the other.

#22 Updated by shunsuke takahashi 2 months ago

just resolved issue.

after try this command, succeed ping.
ip route del table 220 default
reference by this page
https://lists.strongswan.org/pipermail/users/2014-December/007156.html

#23 Updated by Tobias Brunner 2 months ago

  • Status changed from Feedback to Closed
  • Assignee set to Tobias Brunner
  • Resolution set to No change required

after try this command, succeed ping.
ip route del table 220 default

Once again this shows that you haven't fully read RouteBasedVPN. Because there it clearly states that route installation by strongSwan must be disabled via charon.install_routes.

Anyway, I'm glad you got it working.

Also available in: Atom PDF