Bug 218412

Summary: Low throughput in WireGuard VPN
Product: Networking Reporter: Aleksei Zuev (wads31566)
Component: OtherAssignee: Stephen Hemminger (stephen)
Status: NEW ---    
Severity: normal    
Priority: P3    
Hardware: Intel   
OS: Linux   
Kernel Version: 6.8.0-1rc Subsystem:
Regression: No Bisected commit-id:
Attachments: kernel config
lshw on server with updated kernel 6.8.0-1rc
lspci on server
wireguard config server
wireguard client 1
wireguard client 2
wireguard client 3
wireguard client 4
wireguard client 5
network graph

Description Aleksei Zuev 2024-01-23 15:07:14 UTC
Created attachment 305758 [details]
kernel config

Issue: iperf3 shows low throughput using WireGuard VPN
Host hardware: Intel(R) Xeon(R) Gold 6226R (64 cores), 700GB RAM.
VM configurations: OpenWRT latest stable release, 2 cores from Intel(R) Xeon(R) Gold 6226R, 128MB RAM.
Network config: Two physical host (same hardware on both, configs in attachment) connected via Ethernet switch. On host A (IP ends with 33.101)I run few VMs via libvirt, every VM is endpoint of one of VPN tunnels. On host B (IP ends with 33.102) I run VPN server, 1 interface with many VPN endpoint peers.
Throughput between A & B is 10Gb/s, the same between VMs and host B (non-VPN connection).
Detail description: I run different test configurations, for better understanding I attached all configs and simple  network graph. 
Test 1: I run "iperf3 -s" on host B and run one iperf3 client on VM using as address destination VPN server IP address "iperf3 -c 10.10.10.56 -b 0 -t 0". Result: about 3Gb/s.
Test 2: I run two iperf3 servers with different ports "iperf3 -s -p *port number*" on host B and two iperf3 clients on VM using as address destination VPN server IP address, but for every client is different port "iperf3 -c 10.10.10.56 -b 0 -t 0 -p *port number*". Result: summary throughput about 3Gb/s and about 1.5Gb/s
Other tests: Same as Test 2, but with 3,4 and 5 clients. Result: summary throughput about 3Gb/s and throughput for every client is evenly dividing by clients' amount.
On B host I use kernel version 6.8.0-1rc compiled from scratch by myself (kernel config is also attached), on host A is usual Debian 12. Before updating kernel on host B there also was Debian 12 and here's Debian bug report link for the same config: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1060912
Comment 1 Aleksei Zuev 2024-01-23 15:08:56 UTC
Created attachment 305759 [details]
lshw on server with updated kernel 6.8.0-1rc
Comment 2 Aleksei Zuev 2024-01-23 15:09:32 UTC
Created attachment 305760 [details]
lspci on server
Comment 3 Aleksei Zuev 2024-01-23 15:09:59 UTC
Created attachment 305761 [details]
wireguard config server
Comment 4 Aleksei Zuev 2024-01-23 15:10:36 UTC
Created attachment 305762 [details]
wireguard client 1
Comment 5 Aleksei Zuev 2024-01-23 15:10:58 UTC
Created attachment 305763 [details]
wireguard client 2
Comment 6 Aleksei Zuev 2024-01-23 15:12:08 UTC
Created attachment 305764 [details]
wireguard client 3
Comment 7 Aleksei Zuev 2024-01-23 15:12:30 UTC
Created attachment 305765 [details]
wireguard client 4
Comment 8 Aleksei Zuev 2024-01-23 15:12:54 UTC
Created attachment 305766 [details]
wireguard client 5
Comment 9 Aleksei Zuev 2024-01-23 15:13:57 UTC
Created attachment 305767 [details]
network graph
Comment 10 Aleksei Zuev 2024-01-23 19:51:36 UTC
Throughput for 64 cores is very low