KFUNC_PROBE: invalid bpf_context access

[t-coron]

Hi, I'm writing simple tracing codes with kfunc.

When I attach kfunc probe to __netif_receive_skb (as shown in A), it works fine.
However, if I attach it to __netif_receive_skb_core (as shown in B), I get an error message shown in C.

A:

b = BPF(text="""
#include <linux/tcp.h>
#include <linux/ip.h>

KFUNC_PROBE(__netif_receive_skb, struct sk_buff *skb){
    u8 proto;
    struct iphdr *iph;
    bpf_probe_read_kernel(&iph, sizeof(iph), &skb->data);
    bpf_probe_read_kernel(&proto, sizeof(proto), &iph->protocol);
    if (proto == IPPROTO_TCP) {
        bpf_trace_printk("hit!");
    }
    return 0;
}
""")

B:

KFUNC_PROBE(__netif_receive_skb_core, struct sk_buff **pskb){
    struct sk_buff *skb;
    u8 proto;
    struct iphdr *iph;
    bpf_probe_read_kernel(&skb, sizeof(skb), pskb);
    bpf_probe_read_kernel(&iph, sizeof(iph), &skb->data);
    bpf_probe_read_kernel(&proto, sizeof(proto), &iph->protocol);
    if (proto == IPPROTO_TCP) {
        bpf_trace_printk("hit!");
    }
    return 0;
}

C:

0: (79) r3 = *(u64 *)(r1 +0)
func '__netif_receive_skb_core' arg0 type PTR is not a struct
invalid bpf_context access off=0 size=8
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

I think __netif_receive_skb and __netif_receive_skb_core support kfunc.
But, I wonder in case B, eBPF verifier might be rejecting the code.

I suspect it's a BTF issue with struct sk_buff **pskb. When I use kprobe instead, case B works fine.

Does anyone have idea about this ?

[chenhengqi]

Please post the full verifier logs here.

[t-coron]

(C) is the only verifier log I can see.

With debug = 0x8 | 0x10, I can get the following log

Disassembly of section .bpf.fn.kfunc____netif_receive_skb_core:
kfunc____netif_receive_skb_core:
; KFUNC_PROBE(__netif_receive_skb_core, struct sk_buff **pskb){ // Line  19
   0:   79 13 00 00 00 00 00 00         r3 = *(u64 *)(r1 + 0)
   1:   bf a1 00 00 00 00 00 00         r1 = r10
   2:   07 01 00 00 f8 ff ff ff         r1 += -8
; bpf_probe_read_kernel(&skb, sizeof(skb), pskb); // Line  23
   3:   b7 02 00 00 08 00 00 00         r2 = 8
   4:   85 00 00 00 71 00 00 00         call 113
; bpf_probe_read_kernel(&iph, sizeof(iph), &skb->data); // Line  24
   5:   79 a3 f8 ff 00 00 00 00         r3 = *(u64 *)(r10 - 8)
   6:   07 03 00 00 c8 00 00 00         r3 += 200
   7:   bf a1 00 00 00 00 00 00         r1 = r10
; KFUNC_PROBE(__netif_receive_skb_core, struct sk_buff **pskb){ // Line  19
   8:   07 01 00 00 e8 ff ff ff         r1 += -24
; bpf_probe_read_kernel(&iph, sizeof(iph), &skb->data); // Line  24
   9:   b7 02 00 00 08 00 00 00         r2 = 8
  10:   85 00 00 00 71 00 00 00         call 113
; bpf_probe_read_kernel(&proto, sizeof(proto), &iph->protocol); // Line  25
  11:   79 a3 e8 ff 00 00 00 00         r3 = *(u64 *)(r10 - 24)
  12:   07 03 00 00 09 00 00 00         r3 += 9
  13:   bf a1 00 00 00 00 00 00         r1 = r10
; KFUNC_PROBE(__netif_receive_skb_core, struct sk_buff **pskb){ // Line  19
  14:   07 01 00 00 f7 ff ff ff         r1 += -9
; bpf_probe_read_kernel(&proto, sizeof(proto), &iph->protocol); // Line  25
  15:   b7 02 00 00 01 00 00 00         r2 = 1
  16:   85 00 00 00 71 00 00 00         call 113
; if (proto == IPPROTO_TCP) { // Line  26
  17:   71 a1 f7 ff 00 00 00 00         r1 = *(u8 *)(r10 - 9)
  18:   55 01 08 00 06 00 00 00         if r1 != 6 goto +8
  19:   b7 01 00 00 21 00 00 00         r1 = 33
; ({ char _fmt[] = "hit!!"; bpf_trace_printk_(_fmt, sizeof(_fmt)); }); // Line  27
  20:   6b 1a e4 ff 00 00 00 00         *(u16 *)(r10 - 28) = r1
  21:   b7 01 00 00 68 69 74 21         r1 = 561277288
  22:   63 1a e0 ff 00 00 00 00         *(u32 *)(r10 - 32) = r1
  23:   bf a1 00 00 00 00 00 00         r1 = r10
  24:   07 01 00 00 e0 ff ff ff         r1 += -32
  25:   b7 02 00 00 06 00 00 00         r2 = 6
  26:   85 00 00 00 06 00 00 00         call 6
; KFUNC_PROBE(__netif_receive_skb_core, struct sk_buff **pskb){ // Line  19
  27:   b7 00 00 00 00 00 00 00         r0 = 0
  28:   95 00 00 00 00 00 00 00         exit

bpf: Failed to load program: Permission denied
0: (79) r3 = *(u64 *)(r1 +0)
func '__netif_receive_skb_core' arg0 type PTR is not a struct
invalid bpf_context access off=0 size=8
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

[yonghong-song]

This is a limitation in verifier, it only supports ptr to void/int/struct for ptr function arguments.

[t-coron]

Thanks for explaining.
where Is it documented? ( Is there an obvious reason for this ?)