Hello,
can someone help me with routing between interfaces?
I need ping between servers without nat.
I have 1 server directly connected to GW (10.0.3.9/24) and 1 server via wireguard VPN (10.1.0.3/24).
I can ping GW from both servers (and ping servers from gw) but I can’t ping between servers (any firewall rules applied, wireguard have allowed IP range).
Hello,
If you use nat, exclude the 10.1.0.0/24 network.
Example:
set nat source rule 100 destination address !10.1.0.0/24
Then all traffic will pass to nat except the network 10.1.0.0/24.
Best regards, Volodymyr Rybak
Excluding network not helped (I also tried disable nat rule)
GW: sudo tcpdump -i any icmp -vv
Ping from server2
10:46:15.601853 IP (tos 0x0, ttl 64, id 753, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 90, seq 1, length 64
10:46:15.601947 IP (tos 0x0, ttl 63, id 753, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 90, seq 1, length 64
10:46:16.617569 IP (tos 0x0, ttl 64, id 945, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 90, seq 2, length 64
Ping from server1
10:46:26.943880 IP (tos 0x0, ttl 64, id 43061, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 30, seq 2, length 64
10:46:27.967776 IP (tos 0x0, ttl 64, id 43274, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 30, seq 3, length 64
10:46:28.991727 IP (tos 0x0, ttl 64, id 43338, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 30, seq 4, length 64Does server2 know about 10.0.3.0/24 network?
From which interface that dump? Is it wiregurd interface?
Share your configuration.
Dump is from GW, any interface, only ICMP. Server2 know about network
Routes from s2:
10.1.0.0/24 dev wg1 proto kernel scope link src 10.1.0.3
10.0.3.0/24 dev wg1 scope link
Routes from GW:
vyos@gw# ip route list
default nhid 32 via xxxxxxx dev eth0 proto static metric 20
xxxxxx dev eth0 proto kernel scope link src xxxxx
10.0.3.0/24 dev eth2 proto kernel scope link src 10.0.3.1
10.1.0.0/24 dev wg02 proto kernel scope link src 10.1.0.2
From both servers ping arrive to GW
WG config on s2:
[Interface]
Address = 10.1.0.3/24
ListenPort = 51821
PrivateKey = xxxxx
[Peer]
Endpoint = xxxxxx:51821
PublicKey = xxxx
AllowedIPs = 10.1.0.2/32, 10.0.3.0/24
PersistentKeepalive = 5
WG config on GW:
set interfaces wireguard wg02 address '10.1.0.2/24'
set interfaces wireguard wg02 description 'VPN wg02'
set interfaces wireguard wg02 peer s2 address 'xxxxxx'
set interfaces wireguard wg02 peer s2 allowed-ips '10.1.0.3/32'
set interfaces wireguard wg02 peer s2 port '51821'
set interfaces wireguard wg02 peer s2 pubkey 'xxxxx'
set interfaces wireguard wg02 port '51821'
set interfaces wireguard wg02 private-key 'wg02'
Ping from s1 (on eth2):
vyos@gw# sudo tcpdump -i eth2 icmp -vv
tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes
13:23:38.923351 IP (tos 0x0, ttl 64, id 47410, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 31, seq 1, length 64
13:23:39.928243 IP (tos 0x0, ttl 64, id 47426, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 31, seq 2, length 64
Ping from s2 (internet on eth0, wg02):
vyos@gw# sudo tcpdump -i eth0 icmp -vv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:23:49.477406 IP (tos 0x0, ttl 63, id 7786, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 94, seq 1, length 64
13:23:50.505948 IP (tos 0x0, ttl 63, id 7797, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 94, seq 2, length 64
vyos@gw# sudo tcpdump -i wg02 icmp -vv
tcpdump: listening on wg02, link-type RAW (Raw IP), capture size 262144 bytes
13:24:03.788424 IP (tos 0x0, ttl 64, id 9916, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 95, seq 1, length 64
13:24:04.809904 IP (tos 0x0, ttl 64, id 9973, offset 0, flags [DF], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo request, id 95, seq 2, length 64GW config:
set firewall all-ping 'enable'
set firewall broadcast-ping 'disable'
set firewall config-trap 'disable'
set firewall ip-src-route 'disable'
set firewall log-martians 'enable'
set firewall name WAN-GW default-action 'reject'
set firewall name WAN-GW description 'Communication from WAN to GW'
set firewall name WAN-GW enable-default-log
set firewall name WAN-GW rule 4 action 'accept'
set firewall name WAN-GW rule 4 description 'Accept infra wireguard connection'
set firewall name WAN-GW rule 4 destination port '51821'
set firewall name WAN-GW rule 4 log 'enable'
set firewall name WAN-GW rule 4 protocol 'udp'
set firewall name WAN-GW rule 10 action 'accept'
set firewall name WAN-GW rule 10 description 'Accept already established connections'
set firewall name WAN-GW rule 10 log 'enable'
set firewall name WAN-GW rule 10 state established 'enable'
set firewall name WAN-GW rule 10 state related 'enable'
set firewall name WAN-IN default-action 'reject'
set firewall name WAN-IN description 'Communication from WAN through GW'
set firewall name WAN-IN enable-default-log
set firewall name WAN-IN rule 10 action 'accept'
set firewall name WAN-IN rule 10 description 'Accept already established connections'
set firewall name WAN-IN rule 10 log 'enable'
set firewall name WAN-IN rule 10 state established 'enable'
set firewall name WAN-IN rule 10 state related 'enable'
set firewall receive-redirects 'disable'
set firewall send-redirects 'enable'
set firewall source-validation 'disable'
set firewall state-policy established action 'accept'
set firewall state-policy established log enable
set firewall state-policy invalid action 'reject'
set firewall state-policy invalid log enable
set firewall state-policy related action 'accept'
set firewall state-policy related log enable
set firewall syn-cookies 'enable'
set firewall twa-hazards-protection 'disable'
set interfaces ethernet eth0 address 'xxxxxxxxx'
set interfaces ethernet eth0 description 'WAN (eth0)'
set interfaces ethernet eth0 firewall in name 'WAN-IN'
set interfaces ethernet eth0 firewall local name 'WAN-GW'
set interfaces ethernet eth0 mtu '1500'
set interfaces ethernet eth2 address '10.0.3.1/24'
set interfaces ethernet eth2 description 'LAN (eth2)'
set interfaces ethernet eth2 mtu '9000'
set interfaces loopback lo
set interfaces wireguard wg02 address '10.1.0.2/24'
set interfaces wireguard wg02 description 'VPN wg02'
set interfaces wireguard wg02 peer s2 address 'xxxxxxxx'
set interfaces wireguard wg02 peer s2 allowed-ips '10.1.0.3/32'
set interfaces wireguard wg02 peer s2 port '51821'
set interfaces wireguard wg02 peer s2 pubkey 'xxxxxxxxxxxxxxxxxxxxxxxxxx'
set interfaces wireguard wg02 port '51821'
set interfaces wireguard wg02 private-key 'wg02'
set nat source rule 100 destination address '!10.1.0.0/24'
set nat source rule 100 outbound-interface 'eth0'
set nat source rule 100 translation address 'xxxxxxxxxxxx'
set protocols static route 0.0.0.0/0 next-hop xxxxxxxxxxxx
set service snmp location 'Prague'
set service ssh disable-host-validation
set service ssh port '22'
set system config-management commit-revisions '100'
set system console device ttyS0 speed '115200'
set system host-name 'gw'Hello,
Please configure nat:
set nat source rule 100 destination address ‘!10.1.0.0/24’
set nat source rule 100 source address 10.0.3.0/24
set nat source rule 100 outbound-interface ‘eth0’
set nat source rule 100 translation address ‘xxxxxxxxxxxx’
Try it.
Best regards, Volodymyr Rybak
I don’t actually need to set up NAT (and this configuration didn’t change anything anyway). NAT to the internet works fine. (To understand: xxxxxxxxxxxx in NAT is a public IP).
And the communication between servers cannot be NATed
Try dump traffic from server1 or server2
Your configuration in the test lab works fine
Pings from server1
vyos@serv1:~$ ping 10.1.0.3
PING 10.1.0.3 (10.1.0.3) 56(84) bytes of data.
64 bytes from 10.1.0.3: icmp_seq=1 ttl=63 time=8.48 ms
64 bytes from 10.1.0.3: icmp_seq=2 ttl=63 time=3.17 ms
64 bytes from 10.1.0.3: icmp_seq=3 ttl=63 time=0.453 ms
Dump from server2
vyos@serv2:~$ sudo tcpdump -ntvi wg01
tcpdump: listening on wg01, link-type RAW (Raw IP), capture size 262144 bytes
IP (tos 0x0, ttl 63, id 24181, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 2259, seq 1, length 64
IP (tos 0x0, ttl 64, id 61964, offset 0, flags [none], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo reply, id 2259, seq 1, length 64
IP (tos 0x0, ttl 63, id 24352, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 2259, seq 2, length 64
IP (tos 0x0, ttl 64, id 62039, offset 0, flags [none], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo reply, id 2259, seq 2, length 64
IP (tos 0x0, ttl 63, id 24593, offset 0, flags [DF], proto ICMP (1), length 84)
10.0.3.9 > 10.1.0.3: ICMP echo request, id 2259, seq 3, length 64
IP (tos 0x0, ttl 64, id 62144, offset 0, flags [none], proto ICMP (1), length 84)
10.1.0.3 > 10.0.3.9: ICMP echo reply, id 2259, seq 3, length 64
One thing, you shouldn’t see such packets on eth0
10.1.0.3 > 10.0.3.9
You should see external ip addresses and port 51821
No packets came to the servers (in both directions s1->s2, s2->s1).
It wrong. The packets should be going via wgX interface.
On eth0 interface you should see only external addresses + port 51821