Hi, I have a problem with BGP eating up RAM on my 2 BGP routers, so i used docker and build 1.2.4 iso image and installed it on both routers.
One of them if fine, but second will crash tomorrow because memory ( configuration is almost the same )
BGP process take 22GB of RAM, please advice
$ show version
Version: VyOS 1.2.4
Built by: your@email.eu
Built on: Mon 16 Mar 2020 22:11 UTC
Build UUID: 144f761a-1278-4f74-b085-119b1807a44a
Build Commit ID: ad20b8d243b654
Architecture: x86_64
Boot via: installed image
System type: bare metal
Hardware vendor: HP
Hardware model: ProLiant DL360e Gen8
Hardware S/N: CZ3350W920
Hardware UUID: 38395242-4131-5a43-3333-353057393230
Copyright: VyOS maintainers and contributors
$ sudo vtysh -c “show memory”
Memory statistics for zebra:
System allocator statistics:
Total heap allocated: 354 MiB
Holding block headers: 16 MiB
Used small blocks: 0 bytes
Used ordinary blocks: 346 MiB
Free small blocks: 2886 KiB
Free ordinary blocks: 8307 KiB
Ordinary blocks: 14021
Small blocks: 30570
Holding blocks: 1
(see system documentation for ‘mallinfo’ for meaning)
— qmem libfrr —
Type : Current# Size Total Max# MaxBytes
Buffer : 12 24 288 12 288
Buffer data : 1 4120 4120 8 32960
Host config : 3 variable 152 3 152
Command Tokens : 3899 72 280856 3907 281432
Command Token Text : 2845 variable 94024 2852 94256
Command Token Help : 2845 variable 68328 2852 68496
Command Argument : 2 variable 48 10 240
Command Argument Name : 894 variable 21616 899 21752
RCU thread : 9 128 1224 9 1224
FRR POSIX Thread : 18 variable 1440 18 1440
POSIX sync primitives : 18 variable 864 18 864
Graph : 26 8 624 27 648
Graph Node : 4565 32 183000 4567 183080
Hash : 1618 variable 77840 1618 77840
Hash Bucket : 1944 32 87904 1952 93216
Hash Index : 809 variable 240712 810 244816
Hook entry : 18 48 1024 18 1024
Interface : 11 264 2904 11 2904
Connected : 11 48 616 11 616
Link List : 1120 40 48880 1125 49128
Link Node : 66426 24 1594944 88875 2133752
Logging : 1 72 72 1 72
Temporary memory : 16 variable 26752 26 27168
Module loading name : 1 5 24 1 24
Nexthop : 906 120 109152 3253 390712
NetNS Context : 2 variable 128 2 128
NetNS Name : 1 18 24 1 24
Nexthop Group : 539 8 12936 1050 25200
Northbound Node : 5 400 2056 5 2056
Northbound Configuration : 2 16 64 2 64
Prefix List : 6 80 528 6 528
Prefix List Str : 6 variable 144 6 144
Prefix List Entry : 25 112 3272 25 3272
Prefix List Trie Table : 15 4096 61560 15 61560
Prefix : 11 48 648 11 648
Privilege information : 4 variable 160 4 160
Route map : 13 104 1352 13 1352
Route map name : 13 variable 312 14 336
Route map index : 15 136 2040 15 2040
Stream : 18422 variable 2080928 74891 10702336
Stream FIFO : 17 64 1224 25 1800
Route table : 22 56 1232 22 1232
Route node : 1489631 variable 178756056 1490441 178853256
Thread : 42 168 7056 42 7056
Thread master : 39 variable 167336 39 167336
Thread Poll Info : 20 8192 164000 20 164000
Thread stats : 41 72 2952 41 2952
Typed-hash bucket : 11 variable 16798720 11 16798720
Typed-heap array : 1 576 584 1 584
Vector : 9189 16 220936 9192 221008
Vector index : 9189 variable 300936 9192 301160
VRF : 1 200 200 1 200
VRF bit-map : 6 8 144 6 144
VTY : 4 variable 19744 4 19776
Work queue : 2 variable 208 2 208
Work queue item : 1 24 24 1 24
Work queue name string : 1 22 24 1 24
YANG module : 1 48 56 1 56
— qmem Label Manager —
Type : Current# Size Total Max# MaxBytes
— qmem zebra —
Type : Current# Size Total Max# MaxBytes
Zebra Interface Information : 11 360 3960 11 3960
Route Entry : 882937 80 77701016 905318 79671376
RIB destination : 817886 88 71974464 818325 72013096
Zebra DPlane Provider : 1 232 232 1 232
Nexthop Group Entry : 539 80 47480 1049 92368
Nexthop Group Connected : 738 40 34384 776 39248
Zebra Name Space : 5 variable 552 5 552
PTM BFD process registration table.: 3 32 120 3 120
RIB table info : 4 16 96 4 96
Nexthop tracking object : 518 232 121392 519 121624
ZEBRA VRF : 1 4744 4744 1 4744
— qmem Table Manager —
Type : Current# Size Total Max# MaxBytes
Memory statistics for ripngd:
System allocator statistics:
Total heap allocated: 1320 KiB
Holding block headers: 0 bytes
Used small blocks: 0 bytes
Used ordinary blocks: 1229 KiB
Free small blocks: 1616 bytes
Free ordinary blocks: 91 KiB
Ordinary blocks: 2
Small blocks: 43
Holding blocks: 0
(see system documentation for ‘mallinfo’ for meaning)
— qmem libfrr —
Type : Current# Size Total Max# MaxBytes
Buffer : 5 24 136 5 152
Buffer data : 1 4120 4120 1 4120
Host config : 3 variable 152 3 152
Command Tokens : 1829 72 131720 1832 131952
Command Token Text : 1387 variable 43672 1389 43800
Command Token Help : 1387 variable 33304 1389 33352
Command Argument : 2 variable 48 5 120
Command Argument Name : 316 variable 7600 317 7624
Graph : 18 8 432 19 472
Graph Node : 2186 32 87648 2189 87768
Hash : 244 variable 11744 244 11744
Hash Bucket : 436 32 17440 436 17440
Hash Index : 122 variable 35664 123 35736
Hook entry : 4 48 224 4 224
Interface : 11 264 2904 11 2904
Connected : 11 48 616 11 616
Link List : 32 40 1296 37 1528
Link Node : 152 24 3696 194 4848
Logging : 1 72 72 1 72
Temporary memory : 4 variable 96 6 208
Northbound Node : 48 400 19584 48 19584
Northbound Configuration : 2 16 48 2 48
Prefix : 11 48 616 11 616
Privilege information : 2 variable 80 2 80
Route map : 13 104 1352 13 1352
Route map name : 13 variable 312 14 336
Route map index : 15 136 2040 15 2040
Stream : 2 28824 57648 2 57648
Thread : 5 168 856 5 856
Thread master : 3 variable 16712 3 16712
Thread Poll Info : 2 8192 16400 2 16400
Thread stats : 6 72 432 6 432
Typed-hash bucket : 1 variable 520 1 520
Vector : 4415 16 106232 4420 106352
Vector index : 4415 variable 140424 4420 140544
VRF : 1 200 200 1 200
VRF bit-map : 3 8 72 3 72
VTY : 4 variable 19744 4 19744
YANG module : 2 48 128 2 128
Zclient : 1 3312 3320 1 3320
Redistribution instance IDs : 3 2 72 3 72
— qmem ripngd —
Type : Current# Size Total Max# MaxBytes
ripng interface : 11 96 1176 11 1176
Memory statistics for ospfd:
System allocator statistics:
Total heap allocated: 3088 KiB
Holding block headers: 0 bytes
Used small blocks: 0 bytes
Used ordinary blocks: 2892 KiB
Free small blocks: 1968 bytes
Free ordinary blocks: 196 KiB
Ordinary blocks: 32
Small blocks: 44
Holding blocks: 0
(see system documentation for ‘mallinfo’ for meaning)
— qmem libfrr —
Type : Current# Size Total Max# MaxBytes
Buffer : 5 24 120 5 136
Buffer data : 1 4120 4120 2 8240
Host config : 3 variable 152 3 152
Command Tokens : 3600 72 259360 3608 259936
Command Token Text : 2559 variable 81432 2566 81664
Command Token Help : 2559 variable 61432 2566 61600
Command Argument : 2 variable 48 10 256
Command Argument Name : 510 variable 12272 515 12392
Graph : 19 8 456 20 480
Graph Node : 4212 32 168976 4214 169056
Hash : 248 variable 11920 248 11920
Hash Bucket : 717 32 28712 717 28712
Hash Index : 124 variable 40032 125 40864
Hook entry : 11 48 632 11 632
Interface : 11 264 2904 11 2904
Connected : 11 48 616 11 616
Link List : 54 40 2176 63 2520
Link Node : 182 24 4400 233 5856
Logging : 1 72 72 1 72
Temporary memory : 4 variable 96 14 544
Module loading name : 1 5 24 1 24
Northbound Node : 5 400 2056 5 2056
Northbound Configuration : 2 16 64 2 64
Prefix List : 6 80 528 6 528
Prefix List Str : 6 variable 144 6 144
Prefix List Entry : 25 112 3288 25 3288
Prefix List Trie Table : 15 4096 61560 15 61560
Prefix : 11 48 632 11 632
Privilege information : 2 variable 96 2 96
Route map : 13 104 1352 13 1352
Route map name : 13 variable 312 14 336
Route map index : 15 136 2040 15 2040
Stream : 2 28824 57648 2 57648
Route table : 24 56 1344 24 1344
Thread : 9 168 1512 9 1512
Thread master : 3 variable 16712 3 16712
Thread Poll Info : 2 8192 16400 2 16400
Thread stats : 8 72 576 8 576
Typed-hash bucket : 1 variable 520 1 520
Typed-heap array : 1 576 584 1 584
Vector : 8469 16 203640 8472 203712
Vector index : 8469 variable 269880 8472 270104
VRF : 1 200 200 1 200
VRF bit-map : 3 8 72 3 72
VTY : 4 variable 19744 4 19744
YANG module : 1 48 72 1 72
Zclient : 1 3312 3320 1 3320
Redistribution instance IDs : 3 2 72 3 72
— qmem ospfd —
Type : Current# Size Total Max# MaxBytes
OSPF if info : 11 40 456 11 456
OSPF if params : 11 104 1144 11 1144
OSPF MPLS parameters : 11 248 2728 11 2728
OSPF Extended parameters : 11 136 1496 11 1496
OSPF opaque function table : 3 112 360 3 360
Memory statistics for ospf6d:
System allocator statistics:
Total heap allocated: 2780 KiB
Holding block headers: 0 bytes
Used small blocks: 0 bytes
Used ordinary blocks: 2520 KiB
Free small blocks: 1680 bytes
Free ordinary blocks: 260 KiB
Ordinary blocks: 12
Small blocks: 43
Holding blocks: 0
(see system documentation for ‘mallinfo’ for meaning)
— qmem libfrr —
Type : Current# Size Total Max# MaxBytes
Buffer : 5 24 120 5 120
Buffer data : 1 4120 4120 1 4120
Host config : 3 variable 152 3 152
Command Tokens : 2862 72 206128 2865 206360
Command Token Text : 2111 variable 68328 2113 68456
Command Token Help : 2111 variable 50680 2113 50728
Command Argument : 2 variable 48 5 120
Command Argument Name : 390 variable 9376 391 9400
Graph : 19 8 456 20 480
Graph Node : 3379 32 135672 3381 135752
Hash : 246 variable 11904 246 11904
Hash Bucket : 599 32 24024 599 24024
Hash Index : 123 variable 38056 124 38296
Hook entry : 5 48 280 5 280
Interface : 11 264 2904 11 2904
Connected : 11 48 632 11 632
Link List : 32 40 1296 36 1488
Link Node : 156 24 3824 198 5008
Logging : 1 72 72 1 72
Temporary memory : 4 variable 96 6 224
Module loading name : 1 5 24 1 24
Northbound Node : 5 400 2056 5 2056
Northbound Configuration : 2 16 48 2 48
Prefix : 11 48 616 11 616
Privilege information : 3 variable 104 3 104
Route map : 13 104 1352 13 1352
Route map name : 13 variable 312 14 336
Route map index : 15 136 2040 15 2040
Stream : 2 28824 57648 2 57648
Thread : 10 168 1680 10 1680
Thread master : 3 variable 16712 3 16712
Thread Poll Info : 2 8192 16400 2 16400
Thread stats : 9 72 648 9 648
Typed-hash bucket : 1 variable 520 1 520
Typed-heap array : 1 576 584 1 584
Vector : 6804 16 163632 6807 163704
Vector index : 6804 variable 222128 6807 222168
VRF : 1 200 200 1 200
VRF bit-map : 3 8 72 3 72
VTY : 4 variable 19744 4 19744
YANG module : 1 48 72 1 72
Zclient : 1 3312 3320 1 3320
Redistribution instance IDs : 3 2 72 3 72
— qmem ospf6d —
Type : Current# Size Total Max# MaxBytes
Memory statistics for bgpd:
System allocator statistics:
Total heap allocated: > 2GB
Holding block headers: 32 MiB
Used small blocks: 0 bytes
Used ordinary blocks: > 2GB
Free small blocks: 15 KiB
Free ordinary blocks: 264 MiB
Ordinary blocks: 261386
Small blocks: 176
Holding blocks: 3
(see system documentation for ‘mallinfo’ for meaning)
— qmem libfrr —
Type : Current# Size Total Max# MaxBytes
Buffer : 6 24 144 8 192
Buffer data : 1 4120 4120 21441 88336920
Host config : 3 variable 152 3 152
Command Tokens : 10526 72 758416 10532 758848
Command Token Text : 7743 variable 265896 7749 266120
Command Token Help : 7743 variable 186488 7749 186632
Command Argument : 2 variable 48 10 240
Command Argument Name : 1669 variable 40216 1673 40312
RCU thread : 2 128 272 2 272
FRR POSIX Thread : 4 variable 320 4 320
POSIX sync primitives : 4 variable 192 4 192
Graph : 39 8 952 40 976
Graph Node : 12527 32 502632 12529 502712
Hash : 559 variable 26952 559 26952
Hash Bucket : 1399770 32 56033888 1680398 67309312
Hash Index : 280 variable 19367488 281 23565872
Hook entry : 9 48 504 9 504
Interface : 11 264 2904 11 2904
Connected : 11 48 632 11 632
Link List : 66 40 2672 74 2992
Link Node : 356 24 8592 478 11520
Logging : 1 72 72 1 72
Temporary memory : 24 variable 640 28 816
Module loading name : 2 5 48 2 48
Nexthop : 8 120 976 519 63608
Northbound Configuration : 2 16 48 2 48
Prefix List : 6 80 528 6 528
Prefix List Str : 6 variable 144 6 144
Prefix List Entry : 25 112 3240 25 3240
Prefix List Trie Table : 15 4096 61560 15 61560
Prefix : 11 48 616 11 616
Privilege information : 3 variable 136 3 136
Ring buffer : 16 variable 328064 18 369072
Route map : 13 104 1352 13 1352
Route map name : 16 variable 384 17 408
Route map index : 15 136 2040 15 2040
Route map rule : 16 40 640 16 640
Route map rule str : 16 variable 384 16 384
Route map compiled : 11 variable 264 11 264
Route map dependency : 1 24 24 1 24
Route map dependency data : 2 16 48 2 48
Skip List : 2 56 128 2 128
Skip Node : 4 160 672 4 672
Socket union : 12 28 480 14 560
Stream : 12997 variable 3891704 2014033 264683528
Stream FIFO : 16 64 1184 18 1328
Route table : 106 56 5984 106 5984
Thread : 37 168 6216 43 7224
Thread master : 11 variable 50184 11 50184
Thread Poll Info : 6 8192 49200 6 49200
Thread stats : 30 72 2192 30 2192
Typed-hash bucket : 4 variable 16798280 4 16798280
Typed-heap array : 1 576 584 1 584
Vector : 25139 16 604376 25143 604472
Vector index : 25139 variable 814568 25143 814792
VRF : 1 200 200 1 200
VRF bit-map : 3 8 72 3 72
VTY : 4 variable 19744 6 29632
Work queue : 8 152 1232 9 1384
Work queue name string : 8 variable 192 9 216
Zclient : 2 3312 6640 2 6640
Redistribution instance IDs : 6 2 144 6 144
— qmem bgpd —
Type : Current# Size Total Max# MaxBytes
Mac Hash Entry : 7 16 168 7 168
Mac Hash Entry Interface String: 11 variable 264 11 264
BGP instance : 2 variable 5888 2 5888
BGP listen socket details : 2 48 112 2 112
BGP peer : 12 20928 251232 13 272168
BGP peer hostname : 14 variable 336 16 384
BGP peer af : 7 80 616 8 704
BGP update group : 4 104 416 5 520
BGP update subgroup : 4 240 1008 4 1008
BGP packet : 4 56 224 45 3096
BGP attribute : 854307 240 212024680 883372 219234240
BGP aspath : 103211223 40 4136982456 103222250 4137429408
BGP aspath seg : 103246192 24 2612694400 103257238 2612982544
BGP aspath segment data : 103246192 variable 3413357216 103257238 3413689520
BGP aspath str : 103211224 variable 9128411472 103222251 9129198616
BGP table : 93 40 3720 93 3720
BGP node : 1489824 184 274539888 1490632 274687344
BGP route : 2406475 112 290649896 2407944 290834832
BGP connected : 4 4 112 4 112
BGP synchronise : 172 72 12496 193 14024
BGP adj in : 2406908 48 155152816 2408387 155382832
BGP adj out : 1635760 88 144011888 1636632 144089776
BGP AS list : 1 48 56 1 56
BGP AS filter : 1 40 40 1 40
BGP AS filter str : 1 7 24 1 24
community : 4501 40 180232 4591 183832
community val : 4501 variable 180296 4591 184912
community str : 4501 variable 491848 4590 504856
extcommunity : 402 32 16112 417 16712
extcommunity val : 402 variable 11920 417 12512
extcommunity str : 402 variable 60112 416 64488
community-list handler : 1 120 136 1 136
Cluster list : 2 24 48 2 48
Cluster list val : 2 variable 48 2 48
BGP transit attr : 8 24 192 9 216
BGP transit val : 8 variable 192 9 216
BGP nexthop : 518 72 37328 519 37400
BGP peer update interface : 1 9 24 1 24
BGP regexp : 1 64 72 1 72
BGP own address : 4 56 224 4 224
BGP Filter Information : 28 variable 672 30 720
Large Community : 239 40 9576 250 10016
Large Community display string: 239 variable 22152 249 24056
Large Community value : 239 variable 9400 250 10384
BGP Martian Address Intf String: 4 variable 96 4 96
BGP PBR Context : 1 16 24 1 24
BGP EVPN instance information : 1 56 56 1 56
— qmem rfapi —
Type : Current# Size Total Max# MaxBytes
NVE Configuration : 1 2896 2904 1 2904
RFAPI Generic : 1 296 296 1 296
RFAPI Import Table : 1 208 216 1 216