Project

General

Profile

Issue #3151

Forecast stops forwarding multicast

Added by Steve Wright over 1 year ago. Updated over 1 year ago.

Status:
Feedback
Priority:
Normal
Assignee:
-
Category:
-
Affected version:
5.8.0
Resolution:

Description

Have a pretty basic setup, end user connects to strongswan 5.8, windows installs subnet route into VPN and is able to access the secure network. Unicast routing works without any interruptions. When we test multicast routing using iperf, the client is able to connect and receive the multicast traffic, initially. After roughly 4 minutes and 20 seconds, the client no longer receives the multicast despite still being sent from the host within the secure network. Looking at the logs and packet captures, there is no message that stands out, like an IGMP timeout or PIM timeout (from client or server). After that timeout, I am no longer able to receive any multicast, even after restarting iperf. If I restart the VPN client connection, I am able to receive multicast once again, but only for the first 4 minutes. I have tried changing the DPD timers and actions to leave the VPN connection up with no change to the issue. The VPN connection itself seems stable, and unicast continues to function once multicast stops. The sender is 10.60.4.2 (Secure Network) to group 239.60.1.1, being received by 192.168.100.1(VPN Client). Am I missing a setting or plugin for this scenario? Files attached.

iptables-save.txt (746 Bytes) iptables-save.txt Steve Wright, 16.08.2019 21:39
ipsec.statusall.txt (1.68 KB) ipsec.statusall.txt Steve Wright, 16.08.2019 21:39
ipsec.conf.txt (777 Bytes) ipsec.conf.txt Steve Wright, 16.08.2019 21:39
ip route table all.txt (2 KB) ip route table all.txt Steve Wright, 16.08.2019 21:40
ip address.txt (1.03 KB) ip address.txt Steve Wright, 16.08.2019 21:40
log.txt (5 MB) log.txt Steve Wright, 16.08.2019 22:08
charon_debug1.txt (3.17 MB) charon_debug1.txt Steve Wright, 19.08.2019 16:05
charon_debug2.txt.txt (3.03 MB) charon_debug2.txt.txt Steve Wright, 19.08.2019 16:05

History

#1 Updated by Tobias Brunner over 1 year ago

  • Status changed from New to Feedback

Have a pretty basic setup

I'd say, every setup that involves stuff like the experimental forecast plugin is not "pretty basic".

When we test multicast routing using iperf, the client is able to connect and receive the multicast traffic, initially. After roughly 4 minutes and 20 seconds, the client no longer receives the multicast despite still being sent from the host within the secure network. Looking at the logs and packet captures, there is no message that stands out, like an IGMP timeout or PIM timeout (from client or server). After that timeout, I am no longer able to receive any multicast, even after restarting iperf. If I restart the VPN client connection, I am able to receive multicast once again, but only for the first 4 minutes. I have tried changing the DPD timers and actions to leave the VPN connection up with no change to the issue.

If the log is any indication, the server itself or the plugin's AF_PACKET socket doesn't receive any multicasts from 10.60.4.2 to 239.60.1.1 anymore after a while. That a reconnect fixes it seems strange, because that really doesn't change anything in regards to the socket or the group registration (only firewall rules and internal filter lists are updated, which shouldn't have an effect on how multicasts are handled).

#2 Updated by Steve Wright over 1 year ago

Sorry, simply a figure of speech. You are correct, anything with the forecast plugin or the likes is not a pretty basic setup. Strange in deed since the host is still sending the traffic and unicast is unaffected. But this is why I opened an issue, due to the fact that everything works great initially, then stops unexpectedly. Based on your initial assessment, does it appear that all the correct plugins are loaded and the ipsec.conf file should provide the functionality desired? Are there any additional logs that could assist in troubleshooting?

#3 Updated by Tobias Brunner over 1 year ago

does it appear that all the correct plugins are loaded and the ipsec.conf file should provide the functionality desired?

I don't see anything that pops out.

Are there any additional logs that could assist in troubleshooting?

As I said, if you see inbound multicast packets on the server (e.g. in tcpdump/wireshark), but the daemon stops logging that it intercepts them that's really weird because it would either mean the kernel doesn't deliver them to the packet socket, or that the daemon stopped processing them after it read them. In either case it's even weirder that a reconnect should fix it (as I said, that doesn't affect the socket, kernel, or group registration - it should also not affect any middleboxes). Just to make sure, you don't restart the daemon in between, right? Do you have a log that shows it stop forwarding after about 4 minutes, the reconnect and then the newly working intercepts?

#4 Updated by Steve Wright over 1 year ago

Correct, I do not restart the daemon. Attached is the log showing the daemon starting and the client connecting. From the log you can see that the strongswan continues to receive the multicast from the source (10.60.4.2), but stops forwarding to the client (192.168.100.1) at roughly 4 minutes and some seconds into his connection. Once I restart the VPN on the client, and restart the multicast tool, I begin to receive the multicast traffic once again, without restarting the daemon.

#5 Updated by Tobias Brunner over 1 year ago

From the log you can see that the strongswan continues to receive the multicast from the source (10.60.4.2), but stops forwarding to the client (192.168.100.1) at roughly 4 minutes and some seconds into his connection.

Where do you see that? According to the log, packets are forwarded until the connection is terminated (at least according to the timestamps in the log). The connection is established at 09:48 and ends at 09:53 (second file) and until then packets are forwarded (or is there some delay at the end, within the same minute, where nothing happens?).

Once I restart the VPN on the client, and restart the multicast tool, I begin to receive the multicast traffic once again, without restarting the daemon.

What multicast tool? On the client? Restarting that alone doesn't change anything?

#6 Updated by Steve Wright over 1 year ago

Where do you see that? According to the log, packets are forwarded until the connection is terminated (at least according to the timestamps in the log). The connection is established at 09:48 and ends at 09:53 (second file) and until then packets are forwarded (or is there some delay at the end, within the same minute, where nothing happens?).
I guess I am misreading the log. But I did notice this log statement "Mon, 2019-08-19 09:49 09[NET] forwarding a 239.255.255.250 multicast from peer 192.168.100.1 to internal network" and my question is what is the internal network? Is that to the strongswan server? What I do not see on my secure network is the IGMP request from the client, only from the strongswan server.

What multicast tool? On the client? Restarting that alone doesn't change anything?

We are using iperf to generate multicast traffic, but I have also used another tool that has worked for us. Doesnt have a name, just "Multicast Test Tool". But, no, restarting that alone on the client does not change anything. It has to be restarted once the VPN connection is restarted.

#7 Updated by Steve Wright over 1 year ago

Is it possible that the forecast plugin is not forwarding IGMP requests from the client through the server to the secure network? Notice, I am not running any NAT (masquerade) on the server, so that I can track users within the secure network, but only see the IGMP request from the strongswan server. Just a thought on this issue.

#8 Updated by Tobias Brunner over 1 year ago

my question is what is the internal network? Is that to the strongswan server? What I do not see on my secure network is the IGMP request from the client, only from the strongswan server.

Yes, the "internal network" is basically what's connected to the configured internal interface. When a broad-/multicast message is received that is sent by a client, the daemon replicates the message on that interface via packet socket. I guess IGMPs from the client are technically not really necessary if the daemon already forwards multicast directed at it to all clients that include a matching address in their traffic selector.

Is it possible that the forecast plugin is not forwarding IGMP requests from the client through the server to the secure network?

At least the log says it does (those messages addressed to 224.0.0.22 seem to be IGMPs).

Notice, I am not running any NAT (masquerade) on the server, so that I can track users within the secure network, but only see the IGMP request from the strongswan server.

If you assign virtual IPs from the server's LAN, the farp plugin (which you have loaded) should make sure packets are sent via gateway (see ForwardingAndSplitTunneling).

#9 Updated by Steve Wright over 1 year ago

Tobias,

I was able to get the multicast to stay up by installing smcroute and statically joining the multicast group on the interface connected to the secure network. I also tried just adding a static multicast route for the 224.0.0.22 address to the same interface and receive any random multicast group I generated traffic on. So it seems there is a routing issue that is not sticking out between the daemon and kernel. Thanks for your input

#10 Updated by Tobias Brunner over 1 year ago

I was able to get the multicast to stay up by installing smcroute and statically joining the multicast group on the interface connected to the secure network. I also tried just adding a static multicast route for the 224.0.0.22 address to the same interface and receive any random multicast group I generated traffic on. So it seems there is a routing issue that is not sticking out between the daemon and kernel.

Interesting, so the kernel somehow lost the group registration? Or was there perhaps a conflict with the client's own group registration?

#11 Updated by Steve Wright over 1 year ago

That is what I am trying to figure out. While 224.0.0.22 does allow me to dynamically join any group without restarting anything, I still lose the traffic after 260 seconds. The kernel never even registers the group when I look at the mroutes. If I statically join the multicast group though, I can receive it for as long as I am sending (Was able to receive traffic over a 12 hour period with no dropped packets) and the kernel registers the group accordingly. I would say that there are really two problems with that information, the hosts IGMP request isnt getting forwarded for whatever reason and the IGMP querier/report message isnt either. Given the large amount of groups I would be dealing with in production, having this work dynamically would be the ideal solution.

Also available in: Atom PDF