Incorrect flow-accounting on subinterfaces

I have problem with flow-accounting data on subinterface: all information except of destination address is wrong.

$ sh flow-accounting interface eth1.334
flow-accounting for [eth1.334]
Src Addr        Dst Addr        Sport Dport Proto    Packets      Bytes   Flows
64.6.70.15      172.16.0.100    0     0       194          1         63       1
64.6.70.21      172.16.0.100    0     0       188          1         63       1
64.6.70.13      172.16.0.100    0     0       196          1         63       1
64.6.70.9       172.16.0.100    0     0       200          1         63       1
64.6.70.19      172.16.0.100    0     0       190          1         63       1
64.6.70.11      172.16.0.100    0     0       198          1         63       1
64.6.70.17      172.16.0.100    0     0       192          1         63       1
64.6.70.23      172.16.0.100    0     0       186          1         63       1
64.6.70.9       172.16.0.100    0     0       202          1         61       1
64.6.70.37      172.16.0.100    0     0       195          1         40       1
64.6.70.43      172.16.0.100    0     0       189          1         40       1
64.6.70.29      172.16.0.100    0     0       203          1         40       1
64.6.70.35      172.16.0.100    0     0       197          1         40       1
64.6.70.41      172.16.0.100    0     0       191          1         40       1
64.6.70.33      172.16.0.100    0     0       199          1         40       1
64.6.70.39      172.16.0.100    0     0       193          1         40       1
64.6.70.31      172.16.0.100    0     0       201          1         40       1
64.6.70.45      172.16.0.100    0     0       187          1         40       1

$ sh nat source translations address 172.16.0.100
Pre-NAT src          Pre-NAT dst        Post-NAT src         Post-NAT dst
172.16.0.100:5896    192.70.196.45:2061 xxx.yyy.223.194:5896 192.70.196.45:2061
  tcp: 172.16.0.100 ==> xxx.yyy.223.194  timeout: 299 use: 1

$ monitor traffic interface eth1.334 filter "host 172.16.0.100"
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1.334, link-type EN10MB (Ethernet), capture size 262144 bytes
09:31:11.749977 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [P.], seq 157720286:157720313, ack 2053498746, win 32120, length 27
09:31:11.753586 IP 192.70.196.45.2061 > 172.16.0.100.5896: Flags [P.], seq 1:23, ack 27, win 64737, length 22
09:31:12.420381 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [.], ack 23, win 32120, length 0
09:31:13.019616 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [P.], seq 27:54, ack 23, win 32120, length 27
09:31:13.061320 IP 192.70.196.45.2061 > 172.16.0.100.5896: Flags [.], ack 54, win 64710, length 0
09:31:13.068682 IP 192.70.196.45.2061 > 172.16.0.100.5896: Flags [P.], seq 23:45, ack 54, win 64710, length 22
09:31:13.724740 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [.], ack 45, win 32120, length 0
09:31:14.339969 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [P.], seq 54:82, ack 45, win 32120, length 28
09:31:14.344559 IP 192.70.196.45.2061 > 172.16.0.100.5896: Flags [P.], seq 45:67, ack 82, win 64682, length 22
09:31:14.959218 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [.], ack 67, win 32120, length 0
09:31:15.566137 IP 172.16.0.100.5896 > 192.70.196.45.2061: Flags [P.], seq 82:109, ack 67, win 32120, length 27
09:31:15.569549 IP 192.70.196.45.2061 > 172.16.0.100.5896: Flags [P.], seq 67:89, ack 109, win 64655, length 22

Version: VyOS 1.2.0-rolling+201904190439
Hardware model: KVM

there is one open task in phabricator:
https://phabricator.vyos.net/T446

sadly the forum link in the task don’t work anymore. Maybe this is the problem.
Or does it work in other releases?

EDIT:

i see the src address is wrong so the task in phabricator is maybe not your issue… mh
can you provide your flow-accounting configuration?

system {
<skipped...>
    flow-accounting {
        interface eth2
        interface eth1.334
        netflow {
            engine-id 70
            sampling-rate 1
            server 10.99.0.123 {
                port 2055
            }
            source-ip 10.99.0.70
            version 9
        }
    }
<skipped...>
}

or “command style”:

set system flow-accounting interface 'eth2'
set system flow-accounting interface 'eth1.334'
set system flow-accounting netflow engine-id '70'
set system flow-accounting netflow sampling-rate '1'
set system flow-accounting netflow server 10.99.0.123 port '2055'
set system flow-accounting netflow source-ip '10.99.0.70'
set system flow-accounting netflow version '9'

Hey,

i tryed it with the official 1.2.1 release and the latest rolling vyos-1.2.0-rolling+201905280337-amd64.iso

both worked for me, in this setup:

ISP  <--- 192.168.122.0/24 eth0 vyos eth1 10.1.1.0/24 ---> client

set nat source rule 1 outbound-interface 'eth0'
set nat source rule 1 translation address 'masquerade'


vyos@vyos:~$ show flow-accounting
flow-accounting for [eth0]
Src Addr        Dst Addr        Sport Dport Proto    Packets      Bytes   Flows
185.144.208.249 192.168.122.182 443   39768   tcp     155628  436672883       1
1.1.1.1         192.168.122.182 0     0      icmp         11        924       1
216.58.207.67   192.168.122.182 80    57442   tcp          4        742       1
145.239.0.197   192.168.122.182 123   123     udp          6        456       6
193.141.27.6    192.168.122.182 123   123     udp          6        456       6
192.168.122.1   192.168.122.182 53    55669   udp          2        202       1
1.1.1.1         192.168.122.182 53    47405   udp          2        154       1
192.168.122.1   192.168.122.182 53    46800   udp          2        142       1
78.46.53.2      192.168.122.182 123   123     udp          1         76       1
185.144.208.249 192.168.122.182 443   39770   tcp          1         60       1

Total entries: 10
Total flows  : 20
Total pkts   : 155,663
Total bytes  : 436,676,095

flow-accounting for [eth1]
Src Addr        Dst Addr        Sport Dport Proto    Packets      Bytes   Flows
10.1.1.10       1.1.1.1         0     0      icmp         11        924       1
10.1.1.10       216.58.207.67   57442 80      tcp          6        393       1
10.1.1.10       1.1.1.1         47405 53      udp          2        110       1

i download the rolling release on the vyos machine and do icmp and a “curl google.com

is this setup like yours?

please, can you test the latest rolling?

Hi, Rob.

If I get flow-accounting from whole ethernet interface there is no problems.
But I need flow-accounting from subinterfaces (vif) as I wrote in first message.

oh i see sorry, will test it later again with vif

Hi @sergy.silk,

i tested it again with a vif interface and it worked

vyos@vyos# run show flow-accounting interface eth1.222
flow-accounting for [eth1.222]
Src Addr        Dst Addr        Sport Dport Proto    Packets      Bytes   Flows
10.1.1.13       1.1.1.1         0     0      icmp         92       7728       1
10.1.1.14       172.217.18.163  35018 80      tcp          6        393       1
10.1.1.14       10.1.1.1        0     0      icmp          2        168       1
10.1.1.14       193.99.144.80   47718 80      tcp          2        112       1
10.1.1.14       1.1.1.1         36845 53      udp          2        110       1
10.1.1.14       1.1.1.1         33733 53      udp          2        108       1 

can you test the latest rolling release?

I tested it on latest rolling release:

$ sh ver
Version:          VyOS 1.2-rolling-201910150117
Built by:         autobuild@vyos.net
Built on:         Tue 15 Oct 2019 01:17 UTC
Build UUID:       ece9a491-935e-4e1a-9be3-4a9e2cfe45e1
Build Commit ID:  25bb74bc51f7ee

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  Red Hat
Hardware model:   KVM
Hardware S/N:
Hardware UUID:    981110f7-8aaf-4d3c-8f3b-2e9c54121e62

Copyright:        VyOS maintainers and contributors

The result was incorrect:

$ sh flow-accounting interface eth1.17
flow-accounting for [eth1.17]
Src Addr        Dst Addr        Sport Dport Proto    Packets      Bytes   Flows
128.6.126.87    172.16.1.17     0     0       106          1       1400       1
128.6.228.16    172.16.1.17     0     0       239          1       1248       1
128.6.52.195    172.16.1.17     0     0     ipx-in-ip          1        617       1
128.6.52.105    172.16.1.17     0     0       201          1        617       1
128.6.50.52     172.16.1.17     0     0       254          1        617       1
128.6.51.6      172.16.1.17     0     0     ipv6-frag          1        617       1
128.6.52.35     172.16.1.17     0     0        15          1        617       1
128.6.51.110    172.16.1.17     0     0       196          1        617       1
128.6.51.86     172.16.1.17     0     0       220          1        617       1
128.6.53.17     172.16.1.17     0     0        33          1        617       1
128.6.51.77     172.16.1.17     0     0       229          1        617       1
128.6.52.54     172.16.1.17     0     0       252          1        617       1
128.6.52.125    172.16.1.17     0     0       181          1        617       1
128.6.51.45     172.16.1.17     0     0         5          1        617       1

Also I checked flow-accounting on VyOS VM deployed on VMware Hypervisor.
I got correct result:

$ sh flow-accounting interface eth1.800
flow-accounting for [eth1.800]
Src Addr        Dst Addr        Sport Dport Proto    Packets      Bytes   Flows
10.99.0.123     10.99.252.20    0     0      icmp         41       3444       1
10.99.252.17    10.99.252.20    0     0      icmp          3        252       1

I conclude that the problem is observed on virtual machines deployed on KVM hypervisors.

Problem is fixed by changing network interface driver from virtio to rtl8139 or e1000.

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.