linux/io_uring
Jens Axboe fa3595523d io_uring: get rid of alloc cache init_once handling
init_once is called when an object doesn't come from the cache, and
hence needs initial clearing of certain members. While the whole
struct could get cleared by memset() in that case, a few of the cache
members are large enough that this may cause unnecessary overhead if
the caches used aren't large enough to satisfy the workload. For those
cases, some churn of kmalloc+kfree is to be expected.

Ensure that the 3 users that need clearing put the members they need
cleared at the start of the struct, and wrap the rest of the struct in
a struct group so the offset is known.

While at it, improve the interaction with KASAN such that when/if
KASAN writes to members inside the struct that should be retained over
caching, it won't trip over itself. For rw and net, the retaining of
the iovec over caching is disabled if KASAN is enabled. A helper will
free and clear those members in that case.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-01-23 11:32:28 -07:00
..
advise.c io_uring/advise: support 64-bit lengths 2024-06-16 14:54:55 -06:00
advise.h
alloc_cache.h io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
cancel.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
cancel.h io_uring/cancel: get rid of init_hash_table() helper 2024-10-29 13:43:27 -06:00
epoll.c io_uring: undeprecate epoll_ctl support 2023-05-26 20:22:41 -06:00
epoll.h
eventfd.c io_uring/eventfd: ensure io_eventfd_signal() defers another RCU period 2025-01-09 07:16:45 -07:00
eventfd.h io_uring/eventfd: move eventfd handling to separate file 2024-06-16 14:54:55 -06:00
fdinfo.c io_uring/fdinfo: fix io_uring_show_fdinfo() misuse of ->d_iname 2025-01-19 07:28:37 -07:00
fdinfo.h
filetable.c io_uring/rsrc: remove unused parameter ctx for io_rsrc_node_alloc() 2025-01-21 07:07:21 -07:00
filetable.h io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers 2024-11-07 15:24:33 -07:00
fs.c io_uring/fs: consider link->flags when getting path for LINKAT 2023-11-20 09:01:42 -07:00
fs.h
futex.c io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
futex.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
io-wq.c exec: Make sure task->comm is always NUL-terminated 2024-12-16 16:53:00 -08:00
io-wq.h io_uring/io-wq: make io_wq_work flags atomic 2024-06-16 14:54:55 -06:00
io_uring.c io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
io_uring.h io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
kbuf.c for-6.14/io_uring-20250119 2025-01-20 20:27:33 -08:00
kbuf.h io_uring/kbuf: use region api for pbuf rings 2024-12-23 08:17:16 -07:00
Makefile io_uring: add GCOV_PROFILE_URING Kconfig option 2024-08-30 10:52:02 -06:00
memmap.c io_uring/memmap: unify io_uring mmap'ing code 2024-12-23 08:17:16 -07:00
memmap.h io_uring/kbuf: use region api for pbuf rings 2024-12-23 08:17:16 -07:00
msg_ring.c io_uring/msg_ring: don't leave potentially dangling ->tctx pointer 2025-01-22 17:10:45 -07:00
msg_ring.h io_uring/msg_ring: Drop custom destructor 2024-12-27 10:08:21 -07:00
napi.c io_uring/napi: add static napi tracking strategy 2024-11-06 13:55:38 -07:00
napi.h io_uring/napi: add static napi tracking strategy 2024-11-06 13:55:38 -07:00
net.c io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
net.h io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
nop.c io_uring/nop: ensure nop->fd is always initialized 2024-11-21 07:15:30 -07:00
nop.h
notif.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
notif.h io_uring/notif: implement notification stacking 2024-04-22 19:31:18 -06:00
opdef.c for-6.13-rc6-tag 2025-01-09 10:16:45 -08:00
opdef.h io_uring: Fix probe of disabled operations 2024-06-19 08:58:00 -06:00
openclose.c io_uring: enable audit and restrict cred override for IORING_OP_FIXED_FD_INSTALL 2024-01-23 15:25:14 -07:00
openclose.h io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL 2023-12-12 07:42:57 -07:00
poll.c io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
poll.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
refs.h io_uring: kill dead code in io_req_complete_post 2024-04-15 08:10:26 -06:00
register.c io_uring: clean up io_uring_register_get_file() 2025-01-21 07:07:17 -07:00
register.h io_uring: temporarily disable registered waits 2024-11-15 09:58:34 -07:00
rsrc.c io_uring/rsrc: Move lockdep assert from io_free_rsrc_node() to caller 2025-01-21 07:07:26 -07:00
rsrc.h io_uring/rsrc: Move lockdep assert from io_free_rsrc_node() to caller 2025-01-21 07:07:26 -07:00
rw.c io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
rw.h io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
slist.h io_uring: silence variable ‘prev’ set but not used warning 2023-03-09 10:10:58 -07:00
splice.c io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers 2024-11-07 15:24:33 -07:00
splice.h io_uring/splice: open code 2nd direct file assignment 2024-10-29 13:43:28 -06:00
sqpoll.c execve updates for v6.14-rc1 2025-01-20 13:27:58 -08:00
sqpoll.h io_uring/sqpoll: statistics of the true utilization of sq threads 2024-03-01 06:28:19 -07:00
statx.c io_statx_prep(): use getname_uflags() 2024-11-13 11:44:30 -05:00
statx.h
sync.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
sync.h
tctx.c io_uring/tctx: work around xa_store() allocation error issue 2024-11-29 07:20:28 -07:00
tctx.h
timeout.c for-6.14/io_uring-20250119 2025-01-20 20:27:33 -08:00
timeout.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
truncate.c io_uring: add support for ftruncate 2024-02-09 09:04:39 -07:00
truncate.h io_uring: add support for ftruncate 2024-02-09 09:04:39 -07:00
uring_cmd.c io_uring: get rid of alloc cache init_once handling 2025-01-23 11:32:28 -07:00
uring_cmd.h for-6.13-rc6-tag 2025-01-09 10:16:45 -08:00
waitid.c io_uring: Move old async data allocation helper to header 2024-12-27 10:08:11 -07:00
waitid.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
xattr.c replace do_getxattr() with saner helpers. 2024-11-06 12:59:39 -05:00
xattr.h