Bad UDP Checksum on VyOS replies (guest VM)

General questions
1

Hello,
I’m running VyOS 1.5 rolling as edge router on a Proxmox host.
Checking the traffic with tcpdump, I see that every UDP reply from VyOS to any host is reporting [bad udp cksum 0x83d6 -> 0xc6f3!].
For reference:

10:14:12.908225 IP (tos 0x0, ttl 64, id 54170, offset 0, flags [DF], proto UDP (17), length 56)
    192.168.1.50.46062 > 192.168.1.1.53: [udp sum ok] 53802+ A? google.com. (28)
10:14:12.908519 IP (tos 0x0, ttl 64, id 17691, offset 0, flags [DF], proto UDP (17), length 95)
    192.168.1.1.53 > 192.168.1.50.46062: [bad udp cksum 0x83e0 -> 0xeaa0!] 53802 q: A? google.com. 1/0/1 google.com. A 142.250.185.206 ar: . OPT UDPsize=512 [ECS 185.40.234.254/32/0] (67)
10:14:12.929054 IP (tos 0x0, ttl 63, id 60393, offset 0, flags [DF], proto UDP (17), length 62)
    192.168.1.31.45992 > 192.168.1.1.53: [udp sum ok] 14768+ A? adsbexchange.com. (34)
10:14:12.929076 IP (tos 0x0, ttl 63, id 26935, offset 0, flags [DF], proto UDP (17), length 62)
    192.168.1.31.44355 > 192.168.1.1.53: [udp sum ok] 46002+ AAAA? adsbexchange.com. (34)
10:14:12.929285 IP (tos 0x0, ttl 64, id 17622, offset 0, flags [DF], proto UDP (17), length 155)
    192.168.1.1.53 > 192.168.1.31.44355: [bad udp cksum 0x8409 -> 0x67b4!] 46002 q: AAAA? adsbexchange.com. 0/1/1 ns: adsbexchange.com. SOA kim.ns.cloudflare.com. dns.cloudflare.com. 2334219126 10000 2400 604800 1800 ar: . OPT UDPsize=512 [ECS 2a00:dd80:20::6d5/128/0] (127)
10:14:12.929308 IP (tos 0x0, ttl 64, id 17623, offset 0, flags [DF], proto UDP (17), length 133)
    192.168.1.1.53 > 192.168.1.31.45992: [bad udp cksum 0x83f3 -> 0x39a6!] 14768 q: A? adsbexchange.com. 3/0/1 adsbexchange.com. A 104.26.4.191, adsbexchange.com. A 104.26.5.191, adsbexchange.com. A 172.67.69.138 ar: . OPT UDPsize=512 [ECS 185.40.234.254/32/0] (105)
10:14:14.642274 IP (tos 0x0, ttl 64, id 63982, offset 0, flags [DF], proto UDP (17), length 68)
    192.168.1.39.53944 > 192.168.1.1.53: [udp sum ok] 7744+ [1au] AAAA? grafana.com. ar: . OPT UDPsize=1232 (40)
10:14:14.642467 IP (tos 0x0, ttl 64, id 39278, offset 0, flags [DF], proto UDP (17), length 68)
    192.168.1.39.33993 > 192.168.1.1.53: [udp sum ok] 62049+ [1au] A? grafana.com. ar: . OPT UDPsize=1232 (40)
10:14:14.664473 IP (tos 0x0, ttl 64, id 55097, offset 0, flags [DF], proto UDP (17), length 84)
    192.168.1.1.53 > 192.168.1.39.33993: [bad udp cksum 0x83ca -> 0x34f8!] 62049 q: A? grafana.com. 1/0/1 grafana.com. A 34.120.177.193 ar: . OPT UDPsize=512 (56)
10:14:14.664487 IP (tos 0x0, ttl 64, id 55098, offset 0, flags [DF], proto UDP (17), length 96)
    192.168.1.1.53 > 192.168.1.39.53944: [bad udp cksum 0x83d6 -> 0xc6f3!] 7744 q: AAAA? grafana.com. 1/0/1 grafana.com. AAAA 2600:1901:0:b3ea:: ar: . OPT UDPsize=512 (68)

The DNS is currently provided by a ControlD container in VyOS but I see this behaviour also with different containers. I saw the same also with DHCP packets (server in container).
The container is configured with cap-add net-raw and net-bind-services (listening on port 53).

VyOS VM is using 2 virtio bridges, LAN and WAN (PPPoE over VLAN).

Is this behaviour expected? Is the checksum being re-calculated for every packet?
Thank you!

2

It usually fixes with

sudo ethtool -K ethX tx-checksumming off
sudo ethtool -K ethX rx off tx off

disable hardware offload or ignore it.

1 Like
3

Thanks for your prompt reply.
It seems indeed to fix the error according to tcpdump output.
Is this change permanent?

4

Personally I would recommend to have this offloading enabled.

The drawback is since the checksum is applied upon transmit by the NIC itself (hence offloading as in the system CPU doesnt have to calculate it) the checksum is incorrect according to Wireshark/Tshark/Tcpdump since that is looking at the packet before its sent to the NIC.

There is a setting in Wireshark where you can ignore the checksum.

I would only disable this offloading during a troubleshoot where you are suspecting a bad link (and then reenable this offloading once the troubleshooting is complete).