linux/kernel/bpf
Alexei Starovoitov 6fcd486b3a bpf: Refactor RCU enforcement in the verifier.
bpf_rcu_read_lock/unlock() are only available in clang compiled kernels. Lack
of such key mechanism makes it impossible for sleepable bpf programs to use RCU
pointers.

Allow bpf_rcu_read_lock/unlock() in GCC compiled kernels (though GCC doesn't
support btf_type_tag yet) and allowlist certain field dereferences in important
data structures like tast_struct, cgroup, socket that are used by sleepable
programs either as RCU pointer or full trusted pointer (which is valid outside
of RCU CS). Use BTF_TYPE_SAFE_RCU and BTF_TYPE_SAFE_TRUSTED macros for such
tagging. They will be removed once GCC supports btf_type_tag.

With that refactor check_ptr_to_btf_access(). Make it strict in enforcing
PTR_TRUSTED and PTR_UNTRUSTED while deprecating old PTR_TO_BTF_ID without
modifier flags. There is a chance that this strict enforcement might break
existing programs (especially on GCC compiled kernels), but this cleanup has to
start sooner than later. Note PTR_TO_CTX access still yields old deprecated
PTR_TO_BTF_ID. Once it's converted to strict PTR_TRUSTED or PTR_UNTRUSTED the
kfuncs and helpers will be able to default to KF_TRUSTED_ARGS. KF_RCU will
remain as a weaker version of KF_TRUSTED_ARGS where obj refcnt could be 0.

Adjust rcu_read_lock selftest to run on gcc and clang compiled kernels.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: David Vernet <void@manifault.com>
Link: https://lore.kernel.org/bpf/20230303041446.3630-7-alexei.starovoitov@gmail.com
2023-03-03 17:42:20 +01:00
..
preload bpf: iterators: Split iterators.lskel.h into little- and big- endian versions 2023-01-28 12:45:15 -08:00
arraymap.c bpf: Do btf_record_free outside map_free callback 2022-11-17 19:11:31 -08:00
bloom_filter.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
bpf_cgrp_storage.c bpf: Fix a compilation failure with clang lto build 2022-11-30 17:13:25 -08:00
bpf_inode_storage.c bpf: Fix a compilation failure with clang lto build 2022-11-30 17:13:25 -08:00
bpf_iter.c bpf: Initialize the bpf_run_ctx in bpf_iter_run_prog() 2022-08-18 17:06:13 -07:00
bpf_local_storage.c bpf: Support kptrs in local storage maps 2023-03-01 10:24:33 -08:00
bpf_lru_list.c
bpf_lru_list.h
bpf_lsm.c bpf: Fix the kernel crash caused by bpf_setsockopt(). 2023-01-26 23:26:40 -08:00
bpf_struct_ops.c mm: Introduce set_memory_rox() 2022-12-15 10:37:26 -08:00
bpf_struct_ops_types.h
bpf_task_storage.c bpf: Fix a compilation failure with clang lto build 2022-11-30 17:13:25 -08:00
btf.c bpf: Refactor RCU enforcement in the verifier. 2023-03-03 17:42:20 +01:00
cgroup.c bpf: Make bpf_get_current_[ancestor_]cgroup_id() available for all program types 2023-03-02 22:37:07 -08:00
cgroup_iter.c bpf: Pin the start cgroup in cgroup_iter_seq_init() 2022-11-21 17:40:42 +01:00
core.c Networking changes for 6.3. 2023-02-21 18:24:12 -08:00
cpumap.c net: skbuff: drop the word head from skb cache 2023-02-10 09:10:28 +00:00
cpumask.c bpf: Refactor RCU enforcement in the verifier. 2023-03-03 17:42:20 +01:00
devmap.c bpf: devmap: check XDP features in __xdp_enqueue routine 2023-02-02 20:48:24 -08:00
disasm.c
disasm.h
dispatcher.c bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func 2022-12-14 12:02:14 -08:00
hashtab.c bpf: Support kptrs in percpu hashmap and percpu LRU hashmap 2023-03-01 10:24:33 -08:00
helpers.c bpf: Introduce kptr_rcu. 2023-03-03 17:42:20 +01:00
inode.c fs: port inode_init_owner() to mnt_idmap 2023-01-19 09:24:28 +01:00
Kconfig
link_iter.c
local_storage.c bpf: Consolidate spin_lock, timer management into btf_record 2022-11-03 22:19:40 -07:00
lpm_trie.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
Makefile bpf: Enable cpumasks to be queried and used as kptrs 2023-01-25 07:57:49 -08:00
map_in_map.c bpf: Add comments for map BTF matching requirement for bpf_list_head 2022-11-17 19:22:14 -08:00
map_in_map.h
map_iter.c
memalloc.c bpf: Zeroing allocated object from slab in bpf memory allocator 2023-02-15 15:40:06 -08:00
mmap_unlock_work.h
net_namespace.c
offload.c bpf: Drop always true do_idr_lock parameter to bpf_map_free_id 2023-02-02 20:26:12 -08:00
percpu_freelist.c bpf: Initialize same number of free nodes for each pcpu_freelist 2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c bpf: Remove unneeded memset in queue_stack_map creation 2022-08-10 11:48:22 -07:00
reuseport_array.c net: Fix suspicious RCU usage in bpf_sk_reuseport_detach() 2022-08-17 16:42:59 -07:00
ringbuf.c bpf: Rename MEM_ALLOC to MEM_RINGBUF 2022-11-14 21:52:45 -08:00
stackmap.c perf/bpf: Always use perf callchains if exist 2022-09-13 15:03:22 +02:00
syscall.c bpf: Support kptrs in local storage maps 2023-03-01 10:24:33 -08:00
sysfs_btf.c
task_iter.c bpf: keep a reference to the mm, in case the task is dead. 2022-12-28 14:11:48 -08:00
tnum.c
trampoline.c bpf: Fix panic due to wrong pageattr of im->image 2022-12-28 13:46:28 -08:00
verifier.c bpf: Refactor RCU enforcement in the verifier. 2023-03-03 17:42:20 +01:00