kernel panic with pf_ring 8.8.0

[wz200210]

Hi

i am using pf_ring 8.8.0 with suricata

Linux version 3.10.0-1160.62.1.el7.x86_64

got kernel panic randomly as below:

[2685596.677234] [PF_RING] failure creating lo in /proc [net=0000000000000000]
[2685596.677240] ------------[ cut here ]------------
[2685596.677242] WARNING: CPU: 7 PID: 13189 at fs/proc/generic.c:399 proc_register+0x8e/0xb0
[2685596.677243] proc_dir_entry '/proc/eth0' already registered
[2685596.677244] Modules linked in: pf_ring(OE+) xt_REDIRECT nf_nat_redirect xt_owner veth vxlan ip6_udp_tunnel udp_tunnel xt_recent xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs xt_comment xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c br_netfilter bridge stp llc overlay(T) iptable_mangle AliSecGuard(OE) udp_diag nfsv3 nfs_acl nfs lockd grace fscache tcp_diag inet_diag sunrpc nfit libnvdimm intel_powerclamp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel parport_pc parport lrw virtio_balloon gf128mul glue_helper i2c_piix4 ablk_helper cryptd pcspkr ip_tables ext4 mbcache jbd2 virtio_net net_failover virtio_console
[2685596.677269] virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy [last unloaded: pf_ring]
[2685596.677278] CPU: 7 PID: 13189 Comm: insmod Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
[2685596.677279] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 2221b89 04/01/2014
[2685596.677280] Call Trace:
[2685596.677282] [] dump_stack+0x19/0x1b
[2685596.677285] [] __warn+0xd8/0x100
[2685596.677287] [] warn_slowpath_fmt+0x5f/0x80
[2685596.677290] [] ? pde_subdir_insert+0x58/0xb0
[2685596.677292] [] proc_register+0x8e/0xb0
[2685596.677294] [] proc_mkdir_data+0x66/0xa0
[2685596.677296] [] proc_mkdir+0x15/0x20
[2685596.677299] [] add_device_to_proc+0x28/0x110 [pf_ring]
[2685596.677302] [] add_device_to_ring_list+0xfa/0x2e0 [pf_ring]
[2685596.677305] [] ? schedule+0x29/0x70
[2685596.677307] [] ? schedule_timeout+0x221/0x2d0
[2685596.677309] [] ? ida_get_new_above+0x230/0x2a0
[2685596.677312] [] ring_notifier+0x298/0x71a [pf_ring]
[2685596.677314] [] ? wait_for_completion+0x11d/0x140
[2685596.677317] [] ? _rcu_barrier+0x259/0x370
[2685596.677319] [] ? kfree+0x127/0x140
[2685596.677322] [] ? add_device_to_ring_list+0x2e0/0x2e0 [pf_ring]
[2685596.677323] [] __register_netdevice_notifier+0xa8/0x200
[2685596.677326] [] ? 0xffffffffc0771fff
[2685596.677328] [] register_netdevice_notifier+0x13/0x20
[2685596.677331] [] ring_init+0x287/0x1000 [pf_ring]
[2685596.677334] [] ? 0xffffffffc0771fff
[2685596.677336] [] ? 0xffffffffc0771fff
[2685596.677338] [] do_one_initcall+0xba/0x240
[2685596.677340] [] load_module+0x271a/0x2bb0
[2685596.677342] [] ? ddebug_proc_write+0x100/0x100
[2685596.677344] [] ? fput+0x13/0x20
[2685596.677346] [] ? copy_module_from_fd.isra.44+0x53/0x150
[2685596.677349] [] SyS_finit_module+0xa6/0xd0
[2685596.677351] [] system_call_fastpath+0x25/0x2a
[2685596.677353] ---[ end trace 23bb2a1e5d21259d ]---
[2685596.677354] [PF_RING] failure creating eth0 in /proc [net=0000000000000000]
[2685596.677365] general protection fault: 0000 [#1] SMP
[2685596.677502] Modules linked in: pf_ring(OE+) xt_REDIRECT nf_nat_redirect xt_owner veth vxlan ip6_udp_tunnel udp_tunnel xt_recent xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs xt_comment xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c br_netfilter bridge stp llc overlay(T) iptable_mangle AliSecGuard(OE) udp_diag nfsv3 nfs_acl nfs lockd grace fscache tcp_diag inet_diag sunrpc nfit libnvdimm intel_powerclamp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel parport_pc parport lrw virtio_balloon gf128mul glue_helper i2c_piix4 ablk_helper cryptd pcspkr ip_tables ext4 mbcache jbd2 virtio_net net_failover virtio_console
[2685596.679393] virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy [last unloaded: pf_ring]
[2685596.679943] CPU: 7 PID: 13189 Comm: insmod Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
[2685596.680187] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 2221b89 04/01/2014
[2685596.680362] task: ffff9aa94df8d280 ti: ffff9a9eeca10000 task.ti: ffff9a9eeca10000
[2685596.680516] RIP: 0010:[] [] pde_subdir_insert+0x3f/0xb0
[2685596.680678] RSP: 0018:ffff9a9eeca13a10 EFLAGS: 00010206
[2685596.680788] RAX: ffff9aabd0300048 RBX: 61613937ffffffff RCX: 0000000000000000
[2685596.680918] RDX: 0000000000000002 RSI: ffff9ab3b64c7840 RDI: ffff9aabd0300010
[2685596.681049] RBP: ffff9a9eeca13a30 R08: 000000000001f380 R09: ffffffff9a5888e0
[2685596.681186] R10: ffff9ab5c0bdf380 R11: fffff3735806da00 R12: ffff9ab3b64c78c5
[2685596.681316] R13: ffff9aabd0300048 R14: ffff9ab3b64c7840 R15: ffff9aabd0300000
[2685596.681448] FS: 00007f3954850740(0000) GS:ffff9ab5c0bc0000(0000) knlGS:0000000000000000
[2685596.681595] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[2685596.681704] CR2: 00007fba9c34fd70 CR3: 00000000757a4000 CR4: 00000000007606e0
[2685596.681838] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[2685596.681973] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[2685596.682104] PKRU: 55555554
[2685596.682158] Call Trace:
[2685596.682209] [] proc_register+0x4c/0xb0
[2685596.682311] [] proc_mkdir_data+0x66/0xa0
[2685596.682414] [] proc_mkdir+0x15/0x20
[2685596.682514] [] add_device_to_proc+0x28/0x110 [pf_ring]
[2685596.682643] [] add_device_to_ring_list+0xfa/0x2e0 [pf_ring]
[2685596.682790] [] ? schedule+0x29/0x70
[2685596.682887] [] ? schedule_timeout+0x221/0x2d0
[2685596.682987] [] ? ida_get_new_above+0x230/0x2a0
[2685596.683082] [] ring_notifier+0x298/0x71a [pf_ring]
[2685596.683182] [] ? wait_for_completion+0x11d/0x140
[2685596.683281] [] ? _rcu_barrier+0x259/0x370
[2685596.683370] [] ? kfree+0x127/0x140
[2685596.683926] [] ? add_device_to_ring_list+0x2e0/0x2e0 [pf_ring]
[2685596.684502] [] __register_netdevice_notifier+0xa8/0x200
[2685596.684984] [] ? 0xffffffffc0771fff
[2685596.685414] [] register_netdevice_notifier+0x13/0x20
[2685596.685862] [] ring_init+0x287/0x1000 [pf_ring]
[2685596.686320] [] ? 0xffffffffc0771fff
[2685596.686861] [] ? 0xffffffffc0771fff
[2685596.687324] [] do_one_initcall+0xba/0x240
[2685596.687745] [] load_module+0x271a/0x2bb0
[2685596.688159] [] ? ddebug_proc_write+0x100/0x100
[2685596.688594] [] ? fput+0x13/0x20
[2685596.688963] [] ? copy_module_from_fd.isra.44+0x53/0x150
[2685596.689358] [] SyS_finit_module+0xa6/0xd0
[2685596.689731] [] system_call_fastpath+0x25/0x2a
[2685596.690096] Code: 6f 38 41 54 4c 89 e8 4c 8d a6 85 00 00 00 53 eb 0a 0f 1f 00 48 8d 43 10 48 89 da 48 8b 18 48 85 db 74 31 41 0f b6 96 84 00 00 00 <0f> b6 43 44 0f b6 ca 39 c1 72 de 77 12 48 8d 73 45 4c 89 e7 e8
[2685596.691111] RIP [] pde_subdir_insert+0x3f/0xb0
[2685596.691472] RSP

crash> bt
PID: 13189 TASK: ffff9aa94df8d280 CPU: 7 COMMAND: "insmod"
#0 [ffff9a9eeca13788] machine_kexec at ffffffff9a2662f4
#1 [ffff9a9eeca137e8] __crash_kexec at ffffffff9a322b82
#2 [ffff9a9eeca138b8] crash_kexec at ffffffff9a322c70
#3 [ffff9a9eeca138d0] oops_end at ffffffff9a991798
#4 [ffff9a9eeca138f8] die at ffffffff9a230a7b
#5 [ffff9a9eeca13928] do_general_protection at ffffffff9a991092
#6 [ffff9a9eeca13960] general_protection at ffffffff9a990718
[exception RIP: pde_subdir_insert+63]
RIP: ffffffff9a4cd79f RSP: ffff9a9eeca13a10 RFLAGS: 00010206
RAX: ffff9aabd0300048 RBX: 61613937ffffffff RCX: 0000000000000000
RDX: 0000000000000002 RSI: ffff9ab3b64c7840 RDI: ffff9aabd0300010
RBP: ffff9a9eeca13a30 R8: 000000000001f380 R9: ffffffff9a5888e0
R10: ffff9ab5c0bdf380 R11: fffff3735806da00 R12: ffff9ab3b64c78c5
R13: ffff9aabd0300048 R14: ffff9ab3b64c7840 R15: ffff9aabd0300000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffff9a9eeca13a38] proc_register at ffffffff9a4cd99c
#8 [ffff9a9eeca13a60] proc_mkdir_data at ffffffff9a4cdb26
#9 [ffff9a9eeca13a88] proc_mkdir at ffffffff9a4cdb95
#10 [ffff9a9eeca13a98] add_device_to_proc at ffffffffc0874b18 [pf_ring]
#11 [ffff9a9eeca13ac8] add_device_to_ring_list at ffffffffc0874cfa [pf_ring]
#12 [ffff9a9eeca13bc8] ring_notifier at ffffffffc0875178 [pf_ring]
#13 [ffff9a9eeca13c90] __register_netdevice_notifier at ffffffff9a8533c8
#14 [ffff9a9eeca13ce0] register_netdevice_notifier at ffffffff9a853533
#15 [ffff9a9eeca13cf0] init_module at ffffffffc0772287 [pf_ring]
#16 [ffff9a9eeca13d38] do_one_initcall at ffffffff9a20210a
#17 [ffff9a9eeca13d68] load_module at ffffffff9a31ea7a
#18 [ffff9a9eeca13eb8] sys_finit_module at ffffffff9a31f0f6
#19 [ffff9a9eeca13f50] system_call_fastpath at ffffffff9a999f92
RIP: 00007f3953d12e29 RSP: 00007ffc2e831e18 RFLAGS: 00010207
RAX: 0000000000000139 RBX: 0000000001bee2c0 RCX: 00007f3953d84f90
RDX: 0000000000000000 RSI: 000000000041a96e RDI: 0000000000000005
RBP: 000000000041a96e R8: 0000000000000000 R9: 00007ffc2e832028
R10: 0000000000000005 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000001bee280 R14: 0000000000000000 R15: 0000000000000000
ORIG_RAX: 0000000000000139 CS: 0033 SS: 002b
crash>

[wz200210]

attach bt -f

crash> bt -f
PID: 13189 TASK: ffff9aa94df8d280 CPU: 7 COMMAND: "insmod"
#0 [ffff9a9eeca13788] machine_kexec at ffffffff9a2662f4
ffff9a9eeca13790: 00009a9eeca137f0 ffff9a9e40000000
ffff9a9eeca137a0: 000000002b001000 ffff9a9e6b001000
ffff9a9eeca137b0: 000000002b000000 ffff9a9eeca137d0
ffff9a9eeca137c0: 000000006caea0ca ffff9a9eeca13968
ffff9a9eeca137d0: ffff9a9eeca137f0 ffff9a9eeca13968
ffff9a9eeca137e0: ffff9a9eeca138b0 ffffffff9a322b82
#1 [ffff9a9eeca137e8] __crash_kexec at ffffffff9a322b82
ffff9a9eeca137f0: ffff9aabd0300000 ffff9ab3b64c7840
ffff9a9eeca13800: ffff9aabd0300048 ffff9ab3b64c78c5
ffff9a9eeca13810: ffff9a9eeca13a30 61613937ffffffff
ffff9a9eeca13820: fffff3735806da00 ffff9ab5c0bdf380
ffff9a9eeca13830: ffffffff9a5888e0 000000000001f380
ffff9a9eeca13840: ffff9aabd0300048 0000000000000000
ffff9a9eeca13850: 0000000000000002 ffff9ab3b64c7840
ffff9a9eeca13860: ffff9aabd0300010 ffffffffffffffff
ffff9a9eeca13870: ffffffff9a4cd79f 0000000000000010
ffff9a9eeca13880: 0000000000010206 ffff9a9eeca13a10
ffff9a9eeca13890: 0000000000000018 000000006caea0ca
ffff9a9eeca138a0: 0000000000000246 000000000000000b
ffff9a9eeca138b0: ffff9a9eeca138c8 ffffffff9a322c70
#2 [ffff9a9eeca138b8] crash_kexec at ffffffff9a322c70
ffff9a9eeca138c0: ffffffffb8296f00 ffff9a9eeca138f0
ffff9a9eeca138d0: ffffffff9a991798
#3 [ffff9a9eeca138d0] oops_end at ffffffff9a991798
ffff9a9eeca138d8: ffff9a9eeca13968 0000000000000246
ffff9a9eeca138e8: ffffffff9ac5f10c ffff9a9eeca13920
ffff9a9eeca138f8: ffffffff9a230a7b
#4 [ffff9a9eeca138f8] die at ffffffff9a230a7b
ffff9a9eeca13900: ffff9a9eeca13968 ffff9aa94df8d280
ffff9a9eeca13910: 0000000000000000 0000000000000000
ffff9a9eeca13920: ffff9a9eeca13958 ffffffff9a991092
#5 [ffff9a9eeca13928] do_general_protection at ffffffff9a991092
ffff9a9eeca13930: 0000000000000001 ffff9ab3b64c78c5
ffff9a9eeca13940: ffff9aabd0300048 ffff9ab3b64c7840
ffff9a9eeca13950: ffff9aabd0300000 ffff9a9eeca13a30
ffff9a9eeca13960: ffffffff9a990718
#6 [ffff9a9eeca13960] general_protection at ffffffff9a990718
[exception RIP: pde_subdir_insert+63]
RIP: ffffffff9a4cd79f RSP: ffff9a9eeca13a10 RFLAGS: 00010206
RAX: ffff9aabd0300048 RBX: 61613937ffffffff RCX: 0000000000000000
RDX: 0000000000000002 RSI: ffff9ab3b64c7840 RDI: ffff9aabd0300010
RBP: ffff9a9eeca13a30 R8: 000000000001f380 R9: ffffffff9a5888e0
R10: ffff9ab5c0bdf380 R11: fffff3735806da00 R12: ffff9ab3b64c78c5
R13: ffff9aabd0300048 R14: ffff9ab3b64c7840 R15: ffff9aabd0300000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
ffff9a9eeca13968: ffff9aabd0300000 ffff9ab3b64c7840
ffff9a9eeca13978: ffff9aabd0300048 ffff9ab3b64c78c5
ffff9a9eeca13988: ffff9a9eeca13a30 61613937ffffffff
ffff9a9eeca13998: fffff3735806da00 ffff9ab5c0bdf380
ffff9a9eeca139a8: ffffffff9a5888e0 000000000001f380
ffff9a9eeca139b8: ffff9aabd0300048 0000000000000000
ffff9a9eeca139c8: 0000000000000002 ffff9ab3b64c7840
ffff9a9eeca139d8: ffff9aabd0300010 ffffffffffffffff
ffff9a9eeca139e8: ffffffff9a4cd79f 0000000000000010
ffff9a9eeca139f8: 0000000000010206 ffff9a9eeca13a10
ffff9a9eeca13a08: 0000000000000018 0000000000000000
ffff9a9eeca13a18: ffff9ab3b64c7840 ffff9aabd0300010
ffff9a9eeca13a28: ffff9aa94df92d60 ffff9a9eeca13a58
ffff9a9eeca13a38: ffffffff9a4cd99c
#7 [ffff9a9eeca13a38] proc_register at ffffffff9a4cd99c
ffff9a9eeca13a40: ffff9ab3b64c7840 0000000000000000
ffff9a9eeca13a50: ffff9aabd0300000 ffff9a9eeca13a80
ffff9a9eeca13a60: ffffffff9a4cdb26
#8 [ffff9a9eeca13a60] proc_mkdir_data at ffffffff9a4cdb26
ffff9a9eeca13a68: ffff9aabd0300010 ffff9aa94df92d00
ffff9a9eeca13a78: ffff9aa94df92d08 ffff9a9eeca13a90
ffff9a9eeca13a88: ffffffff9a4cdb95
#9 [ffff9a9eeca13a88] proc_mkdir at ffffffff9a4cdb95
ffff9a9eeca13a90: ffff9a9eeca13ac0 ffffffffc0874b18
#10 [ffff9a9eeca13a98] add_device_to_proc at ffffffffc0874b18 [pf_ring]
ffff9a9eeca13aa0: ffff9ab297e58000 ffff9aa94df92d00
ffff9a9eeca13ab0: ffff9ab297e58000 0000000000000001
ffff9a9eeca13ac0: ffff9a9eeca13bc0 ffffffffc0874cfa
#11 [ffff9a9eeca13ac8] add_device_to_ring_list at ffffffffc0874cfa [pf_ring]
ffff9a9eeca13ad0: ffff9a9eeca13fd8 000000000001acc0
ffff9a9eeca13ae0: ffffffff9ae18480 ffffffff9ae8ea98
ffff9a9eeca13af0: 7fffffffffffffff ffffffff9ae8ea90
ffff9a9eeca13b00: ffff9aa94df8d280 000000000000000c
ffff9a9eeca13b10: ffff9a9eeca13b20 ffffffff9a98c3d9
ffff9a9eeca13b20: ffff9a9eeca13bc8 ffffffff9a98a0a1
ffff9a9eeca13b30: 7fffffffffffffff ffff9a9eeca13c88
ffff9a9eeca13b40: 0000000000000282 fffff3735806da00
ffff9a9eeca13b50: ffff9ab5c0bdf380 ffffffff9a5888e0
ffff9a9eeca13b60: ffff9aa94df92de0 000000000000013b
ffff9a9eeca13b70: ffff9ab4395a3d80 ffff9ab4433f1c00
ffff9a9eeca13b80: 0000000000000001 ffff9aabcff00028
ffff9a9eeca13b90: 000000006caea0ca ffff9ab297e58000
ffff9a9eeca13ba0: ffffffffc088e830 ffffffffc087df00
ffff9a9eeca13bb0: 0000000000000000 ffffffffc08919e0
ffff9a9eeca13bc0: ffff9a9eeca13c88 ffffffffc0875178
#12 [ffff9a9eeca13bc8] ring_notifier at ffffffffc0875178 [pf_ring]
ffff9a9eeca13bd0: ffffffff9a98c7ad 0000000000000001
ffff9a9eeca13be0: ffff9aa94df8d280 ffffffff00000001
ffff9a9eeca13bf0: dead000000000100 dead000000000200
ffff9a9eeca13c00: 000000006caea0ca ffffffff9ae7a380
ffff9a9eeca13c10: ffffffff9ae8ea90 ffffffff9af5e300
ffff9a9eeca13c20: ffffffff9ae8ea60 ffff9a9eeca13c78
ffff9a9eeca13c30: ffffffff9a359e99 ffff9a9eeca13c60
ffff9a9eeca13c40: ffffffff9a426af7 ffffffff9ae8ea88
ffff9a9eeca13c50: ffff9ab466150000 000000006caea0ca
ffff9a9eeca13c60: ffff9ab297e58000 ffffffffc087d040
ffff9a9eeca13c70: ffffffffc0874ee0 0000000000000000
ffff9a9eeca13c80: ffff9ab450919550 ffff9a9eeca13cd8
ffff9a9eeca13c90: ffffffff9a8533c8
#13 [ffff9a9eeca13c90] __register_netdevice_notifier at ffffffff9a8533c8
ffff9a9eeca13c98: ffffffff9b464b00 ffff9ab450919480
ffff9a9eeca13ca8: ffff9ab500000000 ffffffff9ae18020
ffff9a9eeca13cb8: ffff9ab16c4627a0 ffffffffc0772000
ffff9a9eeca13cc8: 0000000000000000 ffffffffc087d3c0
ffff9a9eeca13cd8: ffff9a9eeca13ce8 ffffffff9a853533
#14 [ffff9a9eeca13ce0] register_netdevice_notifier at ffffffff9a853533
ffff9a9eeca13ce8: ffff9a9eeca13d30 ffffffffc0772287
#15 [ffff9a9eeca13cf0] init_module at ffffffffc0772287 [pf_ring]
ffff9a9eeca13cf8: 000000006c462c01 0000000000000001
ffff9a9eeca13d08: 0000000000000032 ffffffffc0772000
ffff9a9eeca13d18: ffffffff9ae18020 ffff9ab16c4627a0
ffff9a9eeca13d28: ffffffffc0772000 ffff9a9eeca13d60
ffff9a9eeca13d38: ffffffff9a20210a
#16 [ffff9a9eeca13d38] do_one_initcall at ffffffff9a20210a
ffff9a9eeca13d40: ffffffffc087d3d8 ffff9a9eeca13ec0
ffff9a9eeca13d50: 0000000000000000 0000000000000001
ffff9a9eeca13d60: ffff9a9eeca13eb0 ffffffff9a31ea7a
#17 [ffff9a9eeca13d68] load_module at ffffffff9a31ea7a
ffff9a9eeca13d70: ffffffff9a5b4cd0 ffff9aabbaeaca10
ffff9a9eeca13d80: ffffbdab4d0cff10 ffff9aa94df8d280
ffff9a9eeca13d90: ffff9aa94df8d280 ffffffffc0773000
ffff9a9eeca13da0: 0000000000002718 ffffbdab4d0cf610
ffff9a9eeca13db0: ffffffffc087d3d8 ffffffffc087d5d0
ffff9a9eeca13dc0: ffff9aac1d983df0 ffff9aac1d983de0
ffff9a9eeca13dd0: ffff9aab00000001 ffff9aaf7d4ec300
ffff9a9eeca13de0: 00000000baeaca00 ffffffff9ad46d08
ffff9a9eeca13df0: ffff9a9eeca13e00 ffffffff9a450b73
ffff9a9eeca13e00: ffff9a9eeca13eb0 ffffffff9a31a603
ffff9a9eeca13e10: ffffffffc0876030 0000000000000002
ffff9a9eeca13e20: 0000000000000000 0000000000000000
ffff9a9eeca13e30: 0000000000000000 0000000000000000
ffff9a9eeca13e40: 00006c656e72656b 0000000000000000
ffff9a9eeca13e50: 0000000000000000 0000000000000000
ffff9a9eeca13e60: 0000000000000000 0000000000000000
ffff9a9eeca13e70: 0000000000000000 000000006caea0ca
ffff9a9eeca13e80: 000000006caea0ca ffff9a9eeca13ec0
ffff9a9eeca13e90: 0000000000000000 0000000000000005
ffff9a9eeca13ea0: 000000000041a96e 0000000000000000
ffff9a9eeca13eb0: ffff9a9eeca13f48 ffffffff9a31f0f6
#18 [ffff9a9eeca13eb8] sys_finit_module at ffffffff9a31f0f6
ffff9a9eeca13ec0: ffffbdab4d09f000 0000000000030fd0
ffff9a9eeca13ed0: ffffbdab4d0cf610 ffffbdab4d0cf4a8
ffff9a9eeca13ee0: ffffbdab4d0bbcf8 000000000002d000
ffff9a9eeca13ef0: 000000000002e650 0000000000000000
ffff9a9eeca13f00: 0000000000000000 0000002500000024
ffff9a9eeca13f10: 0000001c0000001f 0000000000000016
ffff9a9eeca13f20: 000000006caea0ca 0000000000000000
ffff9a9eeca13f30: 0000000000000000 0000000000000000
ffff9a9eeca13f40: 0000000000000000 0000000000000000
ffff9a9eeca13f50: ffffffff9a999f92
#19 [ffff9a9eeca13f50] system_call_fastpath at ffffffff9a999f92
RIP: 00007f3953d12e29 RSP: 00007ffc2e831e18 RFLAGS: 00010207
RAX: 0000000000000139 RBX: 0000000001bee2c0 RCX: 00007f3953d84f90
RDX: 0000000000000000 RSI: 000000000041a96e RDI: 0000000000000005
RBP: 000000000041a96e R8: 0000000000000000 R9: 00007ffc2e832028
R10: 0000000000000005 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000001bee280 R14: 0000000000000000 R15: 0000000000000000
ORIG_RAX: 0000000000000139 CS: 0033 SS: 002b
crash>

[wz200210]

Is this issue related to containers? If I don't care about the network interfaces of the container, is there any configuration to workaround？

[cardigliano]

@wz200210 please provide more details about the setup. e.g. are you running in a container? what capture interfaces are you using? are you running any zc driver? model? Thank you.

[wz200210]

suricata_conf.txt

Here is the suricata config

I run suricata in a virtualmachine , some other containers also run on this virtualmachine

PFring capture eth0 only. default mode.

[wz200210]

seconds before the crash something like page alloc failed accur like this

[2685596.666280] insmod: page allocation failure: order:8, mode:0xc0d0

596.567874] [PF_RING] Welcome to PF_RING 8.8.0 ($Revision: unknown$)
(C) 2004-24 ntop
[2685596.567876] LINUX_VERSION_CODE 00030A00
[2685596.567878] [PF_RING] Min # ring slots 4096
[2685596.567879] [PF_RING] Slot version 21
[2685596.567880] [PF_RING] Capture TX Yes [RX+TX]
[2685596.567881] [PF_RING] IP Defragment No
[2685596.567891] NET: Registered protocol family 27
[2685596.666280] insmod: page allocation failure: order:8, mode:0xc0d0
[2685596.666284] CPU: 0 PID: 13189 Comm: insmod Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
[2685596.666285] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 2221b89 04/01/2014
[2685596.666286] Call Trace:
[2685596.666292] [] dump_stack+0x19/0x1b
[2685596.666295] [] warn_alloc_failed+0x110/0x180
[2685596.666297] [] __alloc_pages_nodemask+0x9df/0xbe0
[2685596.666300] [] alloc_pages_current+0x98/0x110
[2685596.666302] [] kmalloc_order+0x18/0x40
[2685596.666305] [] kmalloc_order_trace+0x26/0xa0
[2685596.666306] [] __kmalloc+0x211/0x230
[2685596.666310] [] ops_init+0x8a/0x150
[2685596.666312] [] register_pernet_operations+0x10b/0x1b0
[2685596.666325] [] ? 0xffffffffc0771fff
[2685596.666326] [] register_pernet_subsys+0x28/0x40
[2685596.666331] [] ring_init+0x27b/0x1000 [pf_ring]
[2685596.666402] [] ? 0xffffffffc0771fff
[2685596.666405] [] ? 0xffffffffc0771fff
[2685596.666408] [] do_one_initcall+0xba/0x240
[2685596.666411] [] load_module+0x271a/0x2bb0
[2685596.666415] [] ? ddebug_proc_write+0x100/0x100
[2685596.666419] [] ? fput+0x13/0x20
[2685596.666422] [] ? copy_module_from_fd.isra.44+0x53/0x150
[2685596.666424] [] SyS_finit_module+0xa6/0xd0
[2685596.666427] [] system_call_fastpath+0x25/0x2a

[cardigliano]

OS version?

[wz200210]

OS:
CentOS 7.9
Linux version 3.10.0-1160.62.1.el7.x86_64

NET info while crash:

crash> net
NET_DEVICE NAME IP ADDRESS(ES)
ffff9ab5c0db4000 lo 127.0.0.1
ffff9ab5bd2f5000 eth0 10.180.0.64
ffff9ab5b6338000 docker0 172.17.0.1
ffff9ab4660c0000 flannel.1 10.248.5.0
ffff9ab439710000 cni0 10.248.5.1
ffff9ab432907000 veth220e9b20
ffff9ab4476da000 veth77830082
ffff9ab46650c000 vethf7fe77d7
ffff9ab40e2f9000 vethca39530f
ffff9ab428f07000 veth6afab18b
ffff9ab2af94f000 vethdc962263
ffff9ab420417000 veth1f65ed3c
ffff9ab3a9f44000 vethe5d9d559
ffff9a9efb9ba000 veth12beb84a
ffff9ab0990fc000 vethe464b47d
ffff9ab27d182000 veth833cbda2
ffff9aaf1f75d000 veth80a0e945
ffff9aaf8923c000 vethd7d083c6
ffff9aabd3c86000 veth5cb23c62
ffff9aabdb068000 veth343137be
ffff9ab04756d000 vethb0bfe5da
ffff9aa0b1941000 vethb7c87aaf
ffff9a9ec0ba0000 veth8e82cf6e
ffff9a9f4ff23000 veth71e3a523
ffff9a9f53a7f000 veth9aeeac1d
ffff9a9e9abc3000 veth1499a6e5
ffff9aa454f8c000 veth7f56184a
ffff9a9f5c426000 veth73601f19
ffff9aa9986f6000 vetha1fa65c0
ffff9a9e67ef3000 vethd827fa9e
ffff9ab59c714000 veth3d620855
ffff9a9eb33de000 vethc53cba3c
ffff9ab4661fb000 vetha5381e0b

[wz200210]

I've tried to insmod/rmmod pfring, turn on/off eth0 promisc, up and down other containers repeatablly continuously at the same time. But it's hard to reproduce the issue

[wz200210]

I Got these information when I analysis the vmcore：

seams like the parent dir of /proc/net/pf_ring/dev/lo 's proc_dir_entry is broken

Crash near pde_subdir_insert

/usr/src/debug/kernel-3.10.0-1160.62.1.el7/linux-3.10.0-1160.62.1.el7.x86_64/fs/proc/generic.c: 397
0xffffffff9a4cd991 <proc_register+65>: mov %r12,%rsi
0xffffffff9a4cd994 <proc_register+68>: mov %r13,%rdi
0xffffffff9a4cd997 <proc_register+71>: callq 0xffffffff9a4cd760 <pde_subdir_insert>

RSI: ffff9ab3b64c7840 RDI: ffff9aabd0300010

Register RSI: ffff9ab3b64c7840
/proc/net/pf_ring/dev/lo： looks fine，and it's parent point to 0xffff9aabd0300010 same as register RDI

crash> struct proc_dir_entry ffff9ab3b64c7840
struct proc_dir_entry {
low_ino = 4026534068,
mode = 16749,
nlink = 2,
uid = {
val = 0
},
gid = {
val = 0
},
size = 0,
proc_iops = 0xffffffff9aa35ec0,
proc_fops = 0xffffffff9aa35f80,
parent = 0xffff9aabd0300010,
subdir = {
rb_node = 0x0
},
subdir_node = {
__rb_parent_color = 0,
rb_right = 0x0,
rb_left = 0x0
},
data = 0x0,
count = {
counter = 1
},
in_use = {
counter = 0
},
pde_unload_completion = 0x0,
pde_openers = {
next = 0xffff9ab3b64c78b0,
prev = 0xffff9ab3b64c78b0
},
pde_unload_lock = {
{
rlock = {
raw_lock = {
val = {
counter = 0
}
}
}
}
},
namelen = 2 '\002',
name = 0xffff9ab3b64c78c5 "lo"
}

Register RDI: ffff9aabd0300010
/proc/net/pf_ring/dev: seams the parent node of lo is broken

crash> struct proc_dir_entry ffff9aabd0300010
struct proc_dir_entry {
low_ino = 3492806672,
mode = 39595,
nlink = 3492806673,
uid = {
val = 4294941355
},
gid = {
val = 0
},
size = 0,
proc_iops = 0x100000101,
proc_fops = 0x0,
parent = 0x0,
subdir = {
rb_node = 0x61613937ffffffff
},
subdir_node = {
__rb_parent_color = 8724194240,
rb_right = 0xffff9ab5b25fa200,
rb_left = 0xffff9ab3de52bce0
},
data = 0xffff9aabd03004c0,
count = {
counter = 0
},
in_use = {
counter = 0
},
pde_unload_completion = 0x0,
pde_openers = {
next = 0xffff9ab5b76732f8,
prev = 0xffff9ab5b76732e8
},
pde_unload_lock = {
{
rlock = {
raw_lock = {
val = {
counter = 0
}
}
}
}
},
namelen = 6 '\006',
name = 0xffff9aabd0300095 ""
}

[cardigliano]

Thank you for the trace, I am taking a look at the code to see if there is any obvious bug,
however please note Centos 7 is no longer supported (we still have the repo online, but no updates are released and we are unable to support it). Thus I suggest to move to a more recent version and install latest pf_ring stable (9.0) or dev (9.1) to be able to get updates and fixes.

[cardigliano]

It would be great if you have the chance to clone the dev branch from github and test it. In the meantime I also added additional safety checks on that branch to figure out what's going on.

[wz200210]

thanks for your great support!
this is my fork https://github.com/wz200210/PF_RING, and i have noticed your safety check commit

The test system is provided by cloud vendor so i am not able to update the system to a newer version. it will still be Centos 7
It will take some time for me to reproduce the issue

[wz200210]

hello cardigliano

I try to reproduce this issue with your safety check commit with debug mode
I run 3 scripts at the same time：
script A：insmod/rmmod pfring, repeatablly continuously
script B：up and down common containers repeatablly continuously
script C：up and down host-net containers repeatablly continuously

add_device_to_ring_list crash is not reproduced
but i got another remove_device_from_proc crash on two VM：

[314159.467897] [PF_RING] Welcome to PF_RING 8.8.0 ($Revision: unknown$)
(C) 2004-24 ntop
[314159.467910] LINUX_VERSION_CODE 00030A00
[314159.467917] [PF_RING] Min # ring slots 4096
[314159.467923] [PF_RING] Slot version 21
[314159.467929] [PF_RING] Capture TX Yes [RX+TX]
[314159.467936] [PF_RING] IP Defragment No
[314159.468004] NET: Registered protocol family 27
[314159.616287] [PF_RING][DEBUG] ring_net_init:9084 init network namespace [net=ffffffff88316180]
[314159.616398] [PF_RING][DEBUG] ring_proc_init:1893 registered /proc/net/pf_ring [net=ffffffff88316180]
[314159.616885] [PF_RING][DEBUG] ring_net_init:9084 init network namespace [net=ffff9793b76e3d80]
[314159.616991] [PF_RING][DEBUG] ring_proc_init:1893 registered /proc/net/pf_ring [net=ffff9793b76e3d80]
[314159.617333] [PF_RING][DEBUG] ring_net_init:9084 init network namespace [net=ffff979706f45200]
[314159.617438] [PF_RING][DEBUG] ring_proc_init:1893 registered /proc/net/pf_ring [net=ffff979706f45200]
[314159.617478] [PF_RING] lo: NETDEV_REGISTER Type=772 IfIndex=1 Ptr=ffff979475d18000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617495] [PF_RING][DEBUG] ring_notifier:8966 lo: [REGISTER][ifindex: 1]
[314159.617518] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=lo ifindex=1 (1)
[314159.617530] [PF_RING][DEBUG] add_device_to_proc:8761 created lo in /proc [net=ffffffff88316180] [entry=ffff97968e68fc00]
[314159.617542] [PF_RING][DEBUG] add_device_to_proc:8775 created lo/info in /proc [net=ffffffff88316180] [entry=ffff97968e68ff00]
[314159.617551] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device lo does NOT support hardware packet filtering (2)
[314159.617564] [PF_RING] lo: NETDEV_UP Type=772 IfIndex=1 Ptr=ffff979475d18000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617578] [PF_RING] ens192: NETDEV_REGISTER Type=1 IfIndex=2 Ptr=ffff97972bb44000 Namespace=ffffffff88316180 Addr=0050569E2007
[314159.617586] [PF_RING][DEBUG] ring_notifier:8966 ens192: [REGISTER][ifindex: 2]
[314159.617604] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=ens192 ifindex=2 (1)
[314159.617616] [PF_RING][DEBUG] add_device_to_proc:8761 created ens192 in /proc [net=ffffffff88316180] [entry=ffff97968e68f240]
[314159.617632] [PF_RING][DEBUG] add_device_to_proc:8775 created ens192/info in /proc [net=ffffffff88316180] [entry=ffff97968e68f900]
[314159.617640] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device ens192 does NOT support hardware packet filtering (2)
[314159.617653] [PF_RING] ens192: NETDEV_UP Type=1 IfIndex=2 Ptr=ffff97972bb44000 Namespace=ffffffff88316180 Addr=0050569E2007
[314159.617667] [PF_RING] docker0: NETDEV_REGISTER Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617675] [PF_RING][DEBUG] ring_notifier:8966 docker0: [REGISTER][ifindex: 3]
[314159.617694] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=docker0 ifindex=3 (1)
[314159.617705] [PF_RING][DEBUG] add_device_to_proc:8761 created docker0 in /proc [net=ffffffff88316180] [entry=ffff97968e68f480]
[314159.617718] [PF_RING][DEBUG] add_device_to_proc:8775 created docker0/info in /proc [net=ffffffff88316180] [entry=ffff97940ddc0240]
[314159.617727] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device docker0 does NOT support hardware packet filtering (2)
[314159.617740] [PF_RING] docker0: NETDEV_UP Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617753] [PF_RING] flannel.1: NETDEV_REGISTER Type=1 IfIndex=4 Ptr=ffff97972d344000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617766] [PF_RING][DEBUG] ring_notifier:8966 flannel.1: [REGISTER][ifindex: 4]
[314159.617786] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=flannel.1 ifindex=4 (1)
[314159.617798] [PF_RING][DEBUG] add_device_to_proc:8761 created flannel.1 in /proc [net=ffffffff88316180] [entry=ffff97940ddc0b40]
[314159.617810] [PF_RING][DEBUG] add_device_to_proc:8775 created flannel.1/info in /proc [net=ffffffff88316180] [entry=ffff97940ddc03c0]
[314159.617819] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device flannel.1 does NOT support hardware packet filtering (2)
[314159.617832] [PF_RING] flannel.1: NETDEV_UP Type=1 IfIndex=4 Ptr=ffff97972d344000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617845] [PF_RING] cni0: NETDEV_REGISTER Type=1 IfIndex=5 Ptr=ffff97972d0b2000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617853] [PF_RING][DEBUG] ring_notifier:8966 cni0: [REGISTER][ifindex: 5]
[314159.617872] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=cni0 ifindex=5 (1)
[314159.617883] [PF_RING][DEBUG] add_device_to_proc:8761 created cni0 in /proc [net=ffffffff88316180] [entry=ffff97940ddc0e40]
[314159.617899] [PF_RING][DEBUG] add_device_to_proc:8775 created cni0/info in /proc [net=ffffffff88316180] [entry=ffff97940ddc0180]
[314159.617906] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device cni0 does NOT support hardware packet filtering (2)
[314159.617919] [PF_RING] cni0: NETDEV_UP Type=1 IfIndex=5 Ptr=ffff97972d0b2000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617933] [PF_RING] vethcd103a93: NETDEV_REGISTER Type=1 IfIndex=6 Ptr=ffff97972d234000 Namespace=ffffffff88316180 Addr=000000000000
[314159.617941] [PF_RING][DEBUG] ring_notifier:8966 vethcd103a93: [REGISTER][ifindex: 6]
[314159.617960] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=vethcd103a93 ifindex=6 (1)
[314159.617972] [PF_RING][DEBUG] add_device_to_proc:8761 created vethcd103a93 in /proc [net=ffffffff88316180] [entry=ffff97940ddc0540]
[314159.617984] [PF_RING][DEBUG] add_device_to_proc:8775 created vethcd103a93/info in /proc [net=ffffffff88316180] [entry=ffff97940ddc0300]
[314159.617993] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device vethcd103a93 does NOT support hardware packet filtering (2)
[314159.618006] [PF_RING] vethcd103a93: NETDEV_UP Type=1 IfIndex=6 Ptr=ffff97972d234000 Namespace=ffffffff88316180 Addr=000000000000
[314159.618024] [PF_RING] veth930dc200: NETDEV_REGISTER Type=1 IfIndex=7 Ptr=ffff9796ab392000 Namespace=ffffffff88316180 Addr=000000000000
[314159.618032] [PF_RING][DEBUG] ring_notifier:8966 veth930dc200: [REGISTER][ifindex: 7]
[314159.618055] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=veth930dc200 ifindex=7 (1)
[314159.618069] [PF_RING][DEBUG] add_device_to_proc:8761 created veth930dc200 in /proc [net=ffffffff88316180] [entry=ffff97940ddc0c00]
[314159.618085] [PF_RING][DEBUG] add_device_to_proc:8775 created veth930dc200/info in /proc [net=ffffffff88316180] [entry=ffff97940ddc0900]
[314159.618093] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device veth930dc200 does NOT support hardware packet filtering (2)
[314159.618106] [PF_RING] veth930dc200: NETDEV_UP Type=1 IfIndex=7 Ptr=ffff9796ab392000 Namespace=ffffffff88316180 Addr=000000000000
[314159.618120] [PF_RING] lo: NETDEV_REGISTER Type=772 IfIndex=1 Ptr=ffff979688879000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314159.618131] [PF_RING][DEBUG] ring_notifier:8966 lo: [REGISTER][ifindex: 1]
[314159.618260] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=lo ifindex=1 (1)
[314159.618286] [PF_RING][DEBUG] add_device_to_proc:8761 created lo in /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc06c0]
[314159.618302] [PF_RING][DEBUG] add_device_to_proc:8775 created lo/info in /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc00c0]
[314159.618314] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device lo does NOT support hardware packet filtering (2)
[314159.618328] [PF_RING] lo: NETDEV_UP Type=772 IfIndex=1 Ptr=ffff979688879000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314159.618345] [PF_RING] eth0: NETDEV_REGISTER Type=1 IfIndex=3 Ptr=ffff97972d236000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314159.618357] [PF_RING][DEBUG] ring_notifier:8966 eth0: [REGISTER][ifindex: 3]
[314159.618376] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=eth0 ifindex=3 (1)
[314159.618395] [PF_RING][DEBUG] add_device_to_proc:8761 created eth0 in /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc0000]
[314159.618411] [PF_RING][DEBUG] add_device_to_proc:8775 created eth0/info in /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc0d80]
[314159.618423] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device eth0 does NOT support hardware packet filtering (2)
[314159.618438] [PF_RING] eth0: NETDEV_UP Type=1 IfIndex=3 Ptr=ffff97972d236000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314159.618453] [PF_RING] lo: NETDEV_REGISTER Type=772 IfIndex=1 Ptr=ffff979688b96000 Namespace=ffff979706f45200 Addr=000000000000
[314159.618461] [PF_RING][DEBUG] ring_notifier:8966 lo: [REGISTER][ifindex: 1]
[314159.618484] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=lo ifindex=1 (1)
[314159.618500] [PF_RING][DEBUG] add_device_to_proc:8761 created lo in /proc [net=ffff979706f45200] [entry=ffff97940ddc0480]
[314159.618514] [PF_RING][DEBUG] add_device_to_proc:8775 created lo/info in /proc [net=ffff979706f45200] [entry=ffff979424b1f600]
[314159.618531] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device lo does NOT support hardware packet filtering (2)
[314159.618545] [PF_RING] lo: NETDEV_UP Type=772 IfIndex=1 Ptr=ffff979688b96000 Namespace=ffff979706f45200 Addr=000000000000
[314159.618561] [PF_RING] eth0: NETDEV_REGISTER Type=1 IfIndex=3 Ptr=ffff9796ab393000 Namespace=ffff979706f45200 Addr=000000000000
[314159.618573] [PF_RING][DEBUG] ring_notifier:8966 eth0: [REGISTER][ifindex: 3]
[314159.618592] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=eth0 ifindex=3 (1)
[314159.618606] [PF_RING][DEBUG] add_device_to_proc:8761 created eth0 in /proc [net=ffff979706f45200] [entry=ffff979424b1f540]
[314159.618618] [PF_RING][DEBUG] add_device_to_proc:8775 created eth0/info in /proc [net=ffff979706f45200] [entry=ffff979424b1fa80]
[314159.618630] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device eth0 does NOT support hardware packet filtering (2)
[314159.618644] [PF_RING] eth0: NETDEV_UP Type=1 IfIndex=3 Ptr=ffff9796ab393000 Namespace=ffff979706f45200 Addr=000000000000
[314159.618652] [PF_RING] pf_ring initialized correctly
[314160.254231] [PF_RING] docker0: NETDEV_CHANGE Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.178177] [PF_RING] veth361ac4f: NETDEV_POST_INIT Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffffffff88316180 Addr=000000000000
[314162.178580] [PF_RING] veth361ac4f: NETDEV_REGISTER Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffffffff88316180 Addr=000000000000
[314162.178589] [PF_RING][DEBUG] ring_notifier:8966 veth361ac4f: [REGISTER][ifindex: 7744]
[314162.178612] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=veth361ac4f ifindex=7744 (1)
[314162.178627] [PF_RING][DEBUG] add_device_to_proc:8761 created veth361ac4f in /proc [net=ffffffff88316180] [entry=ffff97972a9eeb40]
[314162.178638] [PF_RING][DEBUG] add_device_to_proc:8775 created veth361ac4f/info in /proc [net=ffffffff88316180] [entry=ffff97972a9eec00]
[314162.178646] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device veth361ac4f does NOT support hardware packet filtering (2)
[314162.178961] [PF_RING] vethed0ba5d: NETDEV_POST_INIT Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.179366] [PF_RING] vethed0ba5d: NETDEV_REGISTER Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.179374] [PF_RING][DEBUG] ring_notifier:8966 vethed0ba5d: [REGISTER][ifindex: 7745]
[314162.179393] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=vethed0ba5d ifindex=7745 (1)
[314162.179406] [PF_RING][DEBUG] add_device_to_proc:8761 created vethed0ba5d in /proc [net=ffffffff88316180] [entry=ffff979406cda0c0]
[314162.179417] [PF_RING][DEBUG] add_device_to_proc:8775 created vethed0ba5d/info in /proc [net=ffffffff88316180] [entry=ffff979406cdab40]
[314162.179425] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device vethed0ba5d does NOT support hardware packet filtering (2)
[314162.180333] docker0: port 1(vethed0ba5d) entered blocking state
[314162.180343] docker0: port 1(vethed0ba5d) entered disabled state
[314162.180364] [PF_RING] vethed0ba5d: NETDEV_JOIN Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.180468] [PF_RING] vethed0ba5d: Unknown msg 26 Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.180499] [PF_RING] vethed0ba5d: Unknown msg 21 Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.180600] device vethed0ba5d entered promiscuous mode
[314162.180706] [PF_RING] docker0: NETDEV_FEAT_CHANGE Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.180938] [PF_RING] vethed0ba5d: NETDEV_PRE_UP Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.181275] [PF_RING] vethed0ba5d: NETDEV_UP Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.181289] IPv6: ADDRCONF(NETDEV_UP): vethed0ba5d: link is not ready
[314162.181298] docker0: port 1(vethed0ba5d) entered blocking state
[314162.181307] docker0: port 1(vethed0ba5d) entered forwarding state
[314162.182931] [PF_RING] vethed0ba5d: NETDEV_CHANGE Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.182945] docker0: port 1(vethed0ba5d) entered disabled state
[314162.183287] [PF_RING] docker0: NETDEV_CHANGE Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.353274] IPVS: Creating netns size=2048 id=3940
[314162.353526] [PF_RING][DEBUG] ring_net_init:9084 init network namespace [net=ffff97972d0a9480]
[314162.353622] [PF_RING][DEBUG] ring_proc_init:1893 registered /proc/net/pf_ring [net=ffff97972d0a9480]
[314162.353673] [PF_RING] lo: NETDEV_POST_INIT Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.354023] [PF_RING] lo: NETDEV_REGISTER Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.354031] [PF_RING][DEBUG] ring_notifier:8966 lo: [REGISTER][ifindex: 1]
[314162.354052] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=lo ifindex=1 (1)
[314162.354069] [PF_RING][DEBUG] add_device_to_proc:8761 created lo in /proc [net=ffff97972d0a9480] [entry=ffff979335b65c00]
[314162.354108] [PF_RING][DEBUG] add_device_to_proc:8775 created lo/info in /proc [net=ffff97972d0a9480] [entry=ffff979335b65000]
[314162.354115] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device lo does NOT support hardware packet filtering (2)
[314162.372145] [PF_RING] lo: NETDEV_PRE_UP Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.372249] [PF_RING] lo: NETDEV_UP Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.745746] [PF_RING][DEBUG] remove_device_from_proc:8664 removing lo/info from /proc [net=ffff97972d0a9480] [entry=ffff979335b65000]
[314162.745764] [PF_RING][DEBUG] remove_device_from_proc:8673 removing lo from /proc [net=ffff97972d0a9480] [entry=ffff979335b65c00]
[314162.745779] [PF_RING] veth361ac4f: NETDEV_UNREGISTER Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffffffff88316180 Addr=000000000000
[314162.745787] [PF_RING][DEBUG] remove_device_from_proc:8664 removing vethed0ba5d/info from /proc [net=ffffffff88316180] [entry=ffff979406cdab40]
[314162.745803] [PF_RING][DEBUG] remove_device_from_proc:8673 removing vethed0ba5d from /proc [net=ffffffff88316180] [entry=ffff979406cda0c0]
[314162.745807] [PF_RING][DEBUG] ring_notifier:8994 veth361ac4f: [UNREGISTER][ifindex: 7744]
[314162.745820] [PF_RING][DEBUG] remove_device_from_ring_list:8698 removing dev=veth361ac4f ifindex=7744 (1)
[314162.745832] [PF_RING][DEBUG] remove_device_from_proc:8664 removing veth361ac4f/info from /proc [net=ffffffff88316180] [entry=ffff97972a9eec00]
[314162.745838] [PF_RING][DEBUG] remove_device_from_proc:8664 removing veth361ac4f/info from /proc [net=ffffffff88316180] [entry=ffff97972a9eec00]
[314162.745853] [PF_RING][DEBUG] remove_device_from_proc:8673 removing veth361ac4f from /proc [net=ffffffff88316180] [entry=ffff97972a9eeb40]
[314162.745855] ------------[ cut here ]------------
[314162.745944] WARNING: CPU: 3 PID: 11200 at fs/proc/generic.c:620 remove_proc_entry+0x18d/0x1a0
[314162.745950] name 'info'
[314162.745954] Modules linked in: pf_ring(OE-) veth vxlan ip6_udp_tunnel udp_tunnel ipt_REJECT nf_reject_ipv4 xt_statistic xt_nat nf_tables ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nfnetlink_queue xt_NFQUEUE xt_comment xt_mark iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac vfat fat iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev vmw_balloon ablk_helper joydev cryptd pcspkr sg nfit libnvdimm parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect
[314162.746150] sysimgblt fb_sys_fops ttm sd_mod ahci crc_t10dif crct10dif_generic libahci drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: pf_ring]
[314162.746201] CPU: 3 PID: 11200 Comm: rmmod Kdump: loaded Tainted: G OE ------------ T 3.10.0-1160.88.1.el7.x86_64 #1
[314162.746209] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/09/2019
[314162.746216] Call Trace:
[314162.746232] [] dump_stack+0x19/0x1f
[314162.746247] [] __warn+0xd8/0x100
[314162.746260] [] warn_slowpath_fmt+0x5f/0x80
[314162.746273] [] remove_proc_entry+0x18d/0x1a0
[314162.746294] [] remove_device_from_proc+0x40/0xf0 [pf_ring]
[314162.746318] [] ring_exit+0x5a/0x1e1 [pf_ring]
[314162.746333] [] SyS_delete_module+0x19e/0x320
[314162.746347] [] system_call_fastpath+0x25/0x2a
[314162.746357] ---[ end trace 86aa192f276d2831 ]---
[314162.746367] [PF_RING][DEBUG] remove_device_from_proc:8673 removing veth361ac4f from /proc [net=ffffffff88316180] [entry=0000000000000000]
[314162.746374] ------------[ cut here ]------------
[314162.746384] WARNING: CPU: 3 PID: 11200 at fs/proc/generic.c:620 remove_proc_entry+0x18d/0x1a0
[314162.746394] name 'veth361ac4f'
[314162.746398] Modules linked in: pf_ring(OE-) veth vxlan ip6_udp_tunnel udp_tunnel ipt_REJECT nf_reject_ipv4 xt_statistic xt_nat nf_tables ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nfnetlink_queue xt_NFQUEUE xt_comment xt_mark iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac vfat fat iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev vmw_balloon ablk_helper joydev cryptd pcspkr sg nfit libnvdimm parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect
[314162.746534] sysimgblt fb_sys_fops ttm sd_mod ahci crc_t10dif crct10dif_generic libahci drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: pf_ring]
[314162.746579] CPU: 3 PID: 11200 Comm: rmmod Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.88.1.el7.x86_64 #1
[314162.746587] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/09/2019
[314162.746593] Call Trace:
[314162.746606] [] dump_stack+0x19/0x1f
[314162.746617] [] __warn+0xd8/0x100
[314162.746630] [] warn_slowpath_fmt+0x5f/0x80
[314162.746643] [] remove_proc_entry+0x18d/0x1a0
[314162.746662] [] remove_device_from_proc+0x68/0xf0 [pf_ring]
[314162.746681] [] ring_exit+0x5a/0x1e1 [pf_ring]
[314162.746695] [] SyS_delete_module+0x19e/0x320
[314162.746708] [] system_call_fastpath+0x25/0x2a
[314162.746718] ---[ end trace 86aa192f276d2832 ]---
[314162.746723] ------------[ cut here ]------------
[314162.746737] WARNING: CPU: 3 PID: 11200 at lib/list_debug.c:53 __list_del_entry+0x6b/0xe0
[314162.746744] list_del corruption, ffff979412ed3760->next is LIST_POISON1 (dead000000000100)
[314162.746748] Modules linked in: pf_ring(OE-) veth vxlan ip6_udp_tunnel udp_tunnel ipt_REJECT nf_reject_ipv4 xt_statistic xt_nat nf_tables ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nfnetlink_queue xt_NFQUEUE xt_comment xt_mark iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac vfat fat iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev vmw_balloon ablk_helper joydev cryptd pcspkr sg nfit libnvdimm parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect
[314162.746885] sysimgblt fb_sys_fops ttm sd_mod ahci crc_t10dif crct10dif_generic libahci drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: pf_ring]
[314162.746935] CPU: 3 PID: 11200 Comm: rmmod Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.88.1.el7.x86_64 #1
[314162.746942] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/09/2019
[314162.746948] Call Trace:
[314162.746958] [] dump_stack+0x19/0x1f
[314162.746970] [] __warn+0xd8/0x100
[314162.746983] [] warn_slowpath_fmt+0x5f/0x80
[314162.746998] [] __list_del_entry+0x6b/0xe0
[314162.747009] [] list_del+0xd/0x30
[314162.747026] [] ring_exit+0x65/0x1e1 [pf_ring]
[314162.747039] [] SyS_delete_module+0x19e/0x320
[314162.747055] [] system_call_fastpath+0x25/0x2a
[314162.747079] ---[ end trace 86aa192f276d2833 ]---
[314162.747090] [PF_RING][DEBUG] remove_device_from_proc:8664 removing eth0/info from /proc [net=ffff979706f45200] [entry=ffff979424b1fa80]
[314162.747101] [PF_RING][DEBUG] remove_device_from_proc:8673 removing eth0 from /proc [net=ffff979706f45200] [entry=ffff979424b1f540]
[314162.747113] [PF_RING][DEBUG] remove_device_from_proc:8664 removing lo/info from /proc [net=ffff979706f45200] [entry=ffff979424b1f600]
[314162.747123] [PF_RING][DEBUG] remove_device_from_proc:8673 removing lo from /proc [net=ffff979706f45200] [entry=ffff97940ddc0480]
[314162.747137] [PF_RING][DEBUG] remove_device_from_proc:8664 removing eth0/info from /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc0d80]
[314162.747147] [PF_RING][DEBUG] remove_device_from_proc:8673 removing eth0 from /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc0000]
[314162.747158] [PF_RING][DEBUG] remove_device_from_proc:8664 removing lo/info from /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc00c0]
[314162.747168] [PF_RING][DEBUG] remove_device_from_proc:8673 removing lo from /proc [net=ffff9793b76e3d80] [entry=ffff97940ddc06c0]
[314162.747179] [PF_RING][DEBUG] remove_device_from_proc:8664 removing veth930dc200/info from /proc [net=ffffffff88316180] [entry=ffff97940ddc0900]
[314162.747189] [PF_RING][DEBUG] remove_device_from_proc:8673 removing veth930dc200 from /proc [net=ffffffff88316180] [entry=ffff97940ddc0c00]
[314162.747201] [PF_RING][DEBUG] remove_device_from_proc:8664 removing vethcd103a93/info from /proc [net=ffffffff88316180] [entry=ffff97940ddc0300]
[314162.747211] [PF_RING][DEBUG] remove_device_from_proc:8673 removing vethcd103a93 from /proc [net=ffffffff88316180] [entry=ffff97940ddc0540]
[314162.747224] [PF_RING][DEBUG] remove_device_from_proc:8664 removing cni0/info from /proc [net=ffffffff88316180] [entry=ffff97940ddc0180]
[314162.747233] [PF_RING][DEBUG] remove_device_from_proc:8673 removing cni0 from /proc [net=ffffffff88316180] [entry=ffff97940ddc0e40]
[314162.747245] [PF_RING][DEBUG] remove_device_from_proc:8664 removing flannel.1/info from /proc [net=ffffffff88316180] [entry=ffff97940ddc03c0]
[314162.747255] [PF_RING][DEBUG] remove_device_from_proc:8673 removing flannel.1 from /proc [net=ffffffff88316180] [entry=ffff97940ddc0b40]
[314162.747266] [PF_RING][DEBUG] remove_device_from_proc:8664 removing docker0/info from /proc [net=ffffffff88316180] [entry=ffff97940ddc0240]
[314162.747276] [PF_RING][DEBUG] remove_device_from_proc:8673 removing docker0 from /proc [net=ffffffff88316180] [entry=ffff97968e68f480]
[314162.747288] [PF_RING][DEBUG] remove_device_from_proc:8664 removing ens192/info from /proc [net=ffffffff88316180] [entry=ffff97968e68f900]
[314162.747309] [PF_RING][DEBUG] remove_device_from_proc:8673 removing ens192 from /proc [net=ffffffff88316180] [entry=ffff97968e68f240]
[314162.747320] [PF_RING][DEBUG] remove_device_from_proc:8664 removing lo/info from /proc [net=ffffffff88316180] [entry=ffff97968e68ff00]
[314162.747330] [PF_RING][DEBUG] remove_device_from_proc:8673 removing lo from /proc [net=ffffffff88316180] [entry=ffff97968e68fc00]
[314162.756059] [PF_RING] veth361ac4f: Unknown msg 17 Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffffffff88316180 Addr=000000000000
[314162.756347] [PF_RING] veth361ac4f: NETDEV_REGISTER Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.756356] [PF_RING][DEBUG] ring_notifier:8966 veth361ac4f: [REGISTER][ifindex: 7744]
[314162.756375] [PF_RING][DEBUG] add_device_to_ring_list:8799 adding dev=veth361ac4f ifindex=7744 (1)
[314162.756389] [PF_RING][DEBUG] add_device_to_proc:8761 created veth361ac4f in /proc [net=ffff97972d0a9480] [entry=ffff97972d9863c0]
[314162.756402] [PF_RING][DEBUG] add_device_to_proc:8775 created veth361ac4f/info in /proc [net=ffff97972d0a9480] [entry=ffff97972d986cc0]
[314162.756410] [PF_RING][DEBUG] add_device_to_ring_list:8835 Device veth361ac4f does NOT support hardware packet filtering (2)
[314162.765170] [PF_RING] eth0: NETDEV_CHANGENAME Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.765180] [PF_RING][DEBUG] ring_notifier:9012 Device changed name to eth0 [ifindex: 7744]
[314162.765189] [PF_RING][DEBUG] ring_notifier:9030 Updating device name veth361ac4f to eth0
[314162.765200] [PF_RING] removing dev=veth361ac4f ifindex=7744 (it changed name to eth0)
[314162.765223] [PF_RING][DEBUG] remove_device_from_proc:8664 removing veth361ac4f/info from /proc [net=ffff97972d0a9480] [entry=ffff97972d986cc0]
[314162.765235] [PF_RING][DEBUG] remove_device_from_proc:8673 removing veth361ac4f from /proc [net=ffff97972d0a9480] [entry=ffff97972d9863c0]
[314162.765244] [PF_RING][DEBUG] ring_notifier:9046 adding dev=eth0 ifindex=7744 (2)
[314162.765257] [PF_RING][DEBUG] add_device_to_proc:8761 created eth0 in /proc [net=ffff97972d0a9480] [entry=ffff9793b75b2b40]
[314162.765268] [PF_RING][DEBUG] add_device_to_proc:8775 created eth0/info in /proc [net=ffff97972d0a9480] [entry=ffff9793b75b26c0]
[314162.765575] [PF_RING] eth0: NETDEV_CHANGEADDR Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.765821] [PF_RING] eth0: NETDEV_PRE_UP Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.765869] [PF_RING] eth0: NETDEV_UP Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.765931] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[314162.769641] [PF_RING] eth0: NETDEV_CHANGE Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.769651] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[314162.769712] [PF_RING] vethed0ba5d: NETDEV_CHANGE Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.769720] IPv6: ADDRCONF(NETDEV_CHANGE): vethed0ba5d: link becomes ready
[314162.769817] docker0: port 1(vethed0ba5d) entered blocking state
[314162.769826] docker0: port 1(vethed0ba5d) entered forwarding state
[314162.770126] [PF_RING] docker0: NETDEV_CHANGE Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.770850] [PF_RING] lo: NETDEV_GOING_DOWN Type=772 IfIndex=1 Ptr=ffff979475d18000 Namespace=ffffffff88316180 Addr=000000000000
[314162.770863] [PF_RING] lo: NETDEV_DOWN Type=772 IfIndex=1 Ptr=ffff979475d18000 Namespace=ffffffff88316180 Addr=000000000000
[314162.770880] [PF_RING] lo: NETDEV_UNREGISTER Type=772 IfIndex=1 Ptr=ffff979475d18000 Namespace=ffffffff88316180 Addr=000000000000
[314162.770899] [PF_RING][DEBUG] ring_notifier:8994 lo: [UNREGISTER][ifindex: 1]
[314162.770912] [PF_RING] ens192: NETDEV_GOING_DOWN Type=1 IfIndex=2 Ptr=ffff97972bb44000 Namespace=ffffffff88316180 Addr=0050569E2007
[314162.770925] [PF_RING] ens192: NETDEV_DOWN Type=1 IfIndex=2 Ptr=ffff97972bb44000 Namespace=ffffffff88316180 Addr=0050569E2007
[314162.770937] [PF_RING] ens192: NETDEV_UNREGISTER Type=1 IfIndex=2 Ptr=ffff97972bb44000 Namespace=ffffffff88316180 Addr=0050569E2007
[314162.770945] [PF_RING][DEBUG] ring_notifier:8994 ens192: [UNREGISTER][ifindex: 2]
[314162.770957] [PF_RING] docker0: NETDEV_GOING_DOWN Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.770969] [PF_RING] docker0: NETDEV_DOWN Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.770993] [PF_RING] docker0: NETDEV_UNREGISTER Type=1 IfIndex=3 Ptr=ffff97972d0b6000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771002] [PF_RING][DEBUG] ring_notifier:8994 docker0: [UNREGISTER][ifindex: 3]
[314162.771016] [PF_RING] flannel.1: NETDEV_GOING_DOWN Type=1 IfIndex=4 Ptr=ffff97972d344000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771028] [PF_RING] flannel.1: NETDEV_DOWN Type=1 IfIndex=4 Ptr=ffff97972d344000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771041] [PF_RING] flannel.1: NETDEV_UNREGISTER Type=1 IfIndex=4 Ptr=ffff97972d344000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771048] [PF_RING][DEBUG] ring_notifier:8994 flannel.1: [UNREGISTER][ifindex: 4]
[314162.771091] [PF_RING] cni0: NETDEV_GOING_DOWN Type=1 IfIndex=5 Ptr=ffff97972d0b2000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771104] [PF_RING] cni0: NETDEV_DOWN Type=1 IfIndex=5 Ptr=ffff97972d0b2000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771116] [PF_RING] cni0: NETDEV_UNREGISTER Type=1 IfIndex=5 Ptr=ffff97972d0b2000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771124] [PF_RING][DEBUG] ring_notifier:8994 cni0: [UNREGISTER][ifindex: 5]
[314162.771136] [PF_RING] vethcd103a93: NETDEV_GOING_DOWN Type=1 IfIndex=6 Ptr=ffff97972d234000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771154] [PF_RING] vethcd103a93: NETDEV_DOWN Type=1 IfIndex=6 Ptr=ffff97972d234000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771167] [PF_RING] vethcd103a93: NETDEV_UNREGISTER Type=1 IfIndex=6 Ptr=ffff97972d234000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771174] [PF_RING][DEBUG] ring_notifier:8994 vethcd103a93: [UNREGISTER][ifindex: 6]
[314162.771187] [PF_RING] veth930dc200: NETDEV_GOING_DOWN Type=1 IfIndex=7 Ptr=ffff9796ab392000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771203] [PF_RING] veth930dc200: NETDEV_DOWN Type=1 IfIndex=7 Ptr=ffff9796ab392000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771219] [PF_RING] veth930dc200: NETDEV_UNREGISTER Type=1 IfIndex=7 Ptr=ffff9796ab392000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771231] [PF_RING][DEBUG] ring_notifier:8994 veth930dc200: [UNREGISTER][ifindex: 7]
[314162.771246] [PF_RING] vethed0ba5d: NETDEV_GOING_DOWN Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771260] [PF_RING] vethed0ba5d: NETDEV_DOWN Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771275] [PF_RING] vethed0ba5d: NETDEV_UNREGISTER Type=1 IfIndex=7745 Ptr=ffff97973f721000 Namespace=ffffffff88316180 Addr=000000000000
[314162.771286] [PF_RING][DEBUG] ring_notifier:8994 vethed0ba5d: [UNREGISTER][ifindex: 7745]
[314162.771301] [PF_RING] lo: NETDEV_GOING_DOWN Type=772 IfIndex=1 Ptr=ffff979688879000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314162.771316] [PF_RING] lo: NETDEV_DOWN Type=772 IfIndex=1 Ptr=ffff979688879000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314162.771332] [PF_RING] lo: NETDEV_UNREGISTER Type=772 IfIndex=1 Ptr=ffff979688879000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314162.771343] [PF_RING][DEBUG] ring_notifier:8994 lo: [UNREGISTER][ifindex: 1]
[314162.771356] [PF_RING] eth0: NETDEV_GOING_DOWN Type=1 IfIndex=3 Ptr=ffff97972d236000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314162.771370] [PF_RING] eth0: NETDEV_DOWN Type=1 IfIndex=3 Ptr=ffff97972d236000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314162.771385] [PF_RING] eth0: NETDEV_UNREGISTER Type=1 IfIndex=3 Ptr=ffff97972d236000 Namespace=ffff9793b76e3d80 Addr=000000000000
[314162.771397] [PF_RING][DEBUG] ring_notifier:8994 eth0: [UNREGISTER][ifindex: 3]
[314162.771409] [PF_RING] lo: NETDEV_GOING_DOWN Type=772 IfIndex=1 Ptr=ffff979688b96000 Namespace=ffff979706f45200 Addr=000000000000
[314162.771423] [PF_RING] lo: NETDEV_DOWN Type=772 IfIndex=1 Ptr=ffff979688b96000 Namespace=ffff979706f45200 Addr=000000000000
[314162.771437] [PF_RING] lo: NETDEV_UNREGISTER Type=772 IfIndex=1 Ptr=ffff979688b96000 Namespace=ffff979706f45200 Addr=000000000000
[314162.771453] [PF_RING][DEBUG] ring_notifier:8994 lo: [UNREGISTER][ifindex: 1]
[314162.771465] [PF_RING] eth0: NETDEV_GOING_DOWN Type=1 IfIndex=3 Ptr=ffff9796ab393000 Namespace=ffff979706f45200 Addr=000000000000
[314162.771481] [PF_RING] eth0: NETDEV_DOWN Type=1 IfIndex=3 Ptr=ffff9796ab393000 Namespace=ffff979706f45200 Addr=000000000000
[314162.771494] [PF_RING] eth0: NETDEV_UNREGISTER Type=1 IfIndex=3 Ptr=ffff9796ab393000 Namespace=ffff979706f45200 Addr=000000000000
[314162.771506] [PF_RING][DEBUG] ring_notifier:8994 eth0: [UNREGISTER][ifindex: 3]
[314162.771518] [PF_RING] lo: NETDEV_GOING_DOWN Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.771534] [PF_RING] lo: NETDEV_DOWN Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.771555] [PF_RING] lo: NETDEV_UNREGISTER Type=772 IfIndex=1 Ptr=ffff97973fd2a000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.771563] [PF_RING][DEBUG] ring_notifier:8994 lo: [UNREGISTER][ifindex: 1]
[314162.771579] [PF_RING] eth0: NETDEV_GOING_DOWN Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.771595] [PF_RING] eth0: NETDEV_DOWN Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.771612] [PF_RING] eth0: NETDEV_UNREGISTER Type=1 IfIndex=7744 Ptr=ffff97973f724000 Namespace=ffff97972d0a9480 Addr=000000000000
[314162.771620] [PF_RING][DEBUG] ring_notifier:8994 eth0: [UNREGISTER][ifindex: 7744]
[314162.771635] [PF_RING][DEBUG] remove_device_from_ring_list:8698 removing dev=eth0 ifindex=7744 (1)
[314162.771645] [PF_RING][DEBUG] remove_device_from_proc:8664 removing eth0/info from /proc [net=ffff97972d0a9480] [entry=ffff9793b75b26c0]
[314162.771661] [PF_RING][DEBUG] remove_device_from_proc:8673 removing eth0 from /proc [net=ffff97972d0a9480] [entry=ffff9793b75b2b40]
[314162.771683] [PF_RING][DEBUG] ring_net_exit:9093 exit network namespace [net=ffffffff88316180]
[314162.771691] [PF_RING][DEBUG] ring_proc_term:1903 Removing /proc/net/pf_ring [net=ffffffff88316180]
[314162.771706] [PF_RING][DEBUG] ring_proc_term:1911 deregistered /proc/net/pf_ring [net=ffffffff88316180]
[314162.771713] [PF_RING][DEBUG] ring_net_exit:9093 exit network namespace [net=ffff9793b76e3d80]
[314162.771720] [PF_RING][DEBUG] ring_proc_term:1903 Removing /proc/net/pf_ring [net=ffff9793b76e3d80]
[314162.771733] [PF_RING][DEBUG] ring_proc_term:1911 deregistered /proc/net/pf_ring [net=ffff9793b76e3d80]
[314162.771740] [PF_RING][DEBUG] ring_net_exit:9093 exit network namespace [net=ffff979706f45200]
[314162.771751] [PF_RING][DEBUG] ring_proc_term:1903 Removing /proc/net/pf_ring [net=ffff979706f45200]
[314162.771765] [PF_RING][DEBUG] ring_proc_term:1911 deregistered /proc/net/pf_ring [net=ffff979706f45200]
[314162.771772] [PF_RING][DEBUG] ring_net_exit:9093 exit network namespace [net=ffff97972d0a9480]
[314162.771787] [PF_RING][DEBUG] ring_proc_term:1903 Removing /proc/net/pf_ring [net=ffff97972d0a9480]
[314162.771802] [PF_RING][DEBUG] ring_proc_term:1911 deregistered /proc/net/pf_ring [net=ffff97972d0a9480]
[314162.797111] NET: Unregistered protocol family 27
[314162.797121] [PF_RING] Module unloaded
[314162.860388] BUG: unable to handle kernel NULL pointer dereference at (null)
[314162.860733] IP: [] skb_queue_tail+0x33/0x50
[314162.860932] PGD 800000040f641067 PUD 40f705067 PMD 0
[314162.861100] Oops: 0002 [#1] SMP
[314162.861232] Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel ipt_REJECT nf_reject_ipv4 xt_statistic xt_nat nf_tables ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nfnetlink_queue xt_NFQUEUE xt_comment xt_mark iptable_mangle xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype iptable_filter br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac vfat fat iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev vmw_balloon ablk_helper joydev cryptd pcspkr sg nfit libnvdimm parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect sysimgblt
[314162.864039] fb_sys_fops ttm sd_mod ahci crc_t10dif crct10dif_generic libahci drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel serio_raw libata vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: pf_ring]
[314162.865001] CPU: 1 PID: 114 Comm: kauditd Kdump: loaded Tainted: G W OE ------------ T 3.10.0-1160.88.1.el7.x86_64 #1
[314162.865411] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/09/2019
[314162.865796] task: ffff9796acf78000 ti: ffff9796acfec000 task.ti: ffff9796acfec000
[314162.866013] RIP: 0010:[] [] skb_queue_tail+0x33/0x50
[314162.866262] RSP: 0018:ffff9796acfefd40 EFLAGS: 00010046
[314162.866420] RAX: 0000000000000246 RBX: ffff979335fe2890 RCX: ffff9796ad4e0800
[314162.866630] RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff979335fe28a4
[314162.866839] RBP: ffff9796acfefd58 R08: 0000000000000764 R09: 0000000000000000
[314162.867049] R10: 0000000200000000 R11: 0000000000000000 R12: ffff979335fe2890
[314162.867550] R13: ffff979335fe28a4 R14: 0000000000000764 R15: 0000000000000000
[314162.867761] FS: 0000000000000000(0000) GS:ffff97973fc80000(0000) knlGS:0000000000000000
[314162.868030] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[314162.868200] CR2: 0000000000000000 CR3: 000000040f7d0000 CR4: 00000000001607e0
[314162.875653] Call Trace:
[314162.882792] [] __netlink_sendskb+0x4d/0x180
[314162.890009] [] ? security_sock_rcv_skb+0x1c/0x30
[314162.897514] [] netlink_unicast+0x1eb/0x220
[314162.904912] [] kauditd_send_skb+0x57/0x140
[314162.912222] [] ? task_rq_unlock+0x20/0x20
[314162.919591] [] kauditd_thread+0xbf/0x1e0
[314162.926757] [] ? wake_up_state+0x20/0x20
[314162.933972] [] ? audit_printk_skb+0x70/0x70
[314162.940913] [] kthread+0xd1/0xe0
[314162.947694] [] ? insert_kthread_work+0x40/0x40
[314162.954300] [] ret_from_fork_nospec_begin+0x7/0x21
[314162.960847] [] ? insert_kthread_work+0x40/0x40
[314162.967126] Code: e5 41 55 4c 8d 6f 14 41 54 49 89 f4 53 48 89 fb 4c 89 ef e8 50 33 15 00 48 8b 53 08 49 89 1c 24 4c 89 ef 48 89 c6 49 89 54 24 08 <4c> 89 22 83 43 10 01 4c 89 63 08 e8 1d 2f 15 00 5b 41 5c 41 5d
[314162.986296] RIP [] skb_queue_tail+0x33/0x50
[314162.992622] RSP
[314162.998738] CR2: 0000000000000000