linux/kernel
Dr. Thomas Orgis 0e0af57e0e taskstats: version 12 with thread group and exe info
The task exit struct needs some crucial information to be able to provide
an enhanced version of process and thread accounting.  This change
provides:

1. ac_tgid in additon to ac_pid
2. thread group execution walltime in ac_tgetime
3. flag AGROUP in ac_flag to indicate the last task
   in a thread group / process
4. device ID and inode of task's /proc/self/exe in
   ac_exe_dev and ac_exe_inode
5. tools/accounting/procacct as demonstrator

When a task exits, taskstats are reported to userspace including the
task's pid and ppid, but without the id of the thread group this task is
part of.  Without the tgid, the stats of single tasks cannot be correlated
to each other as a thread group (process).

The taskstats documentation suggests that on process exit a data set
consisting of accumulated stats for the whole group is produced.  But such
an additional set of stats is only produced for actually multithreaded
processes, not groups that had only one thread, and also those stats only
contain data about delay accounting and not the more basic information
about CPU and memory resource usage.  Adding the AGROUP flag to be set
when the last task of a group exited enables determination of process end
also for single-threaded processes.

My applicaton basically does enhanced process accounting with summed
cputime, biggest maxrss, tasks per process.  The data is not available
with the traditional BSD process accounting (which is not designed to be
extensible) and the taskstats interface allows more efficient on-the-fly
grouping and summing of the stats, anyway, without intermediate disk
writes.

Furthermore, I do carry statistics on which exact program binary is used
how often with associated resources, getting a picture on how important
which parts of a collection of installed scientific software in different
versions are, and how well they put load on the machine.  This is enabled
by providing information on /proc/self/exe for each task.  I assume the
two 64-bit fields for device ID and inode are more appropriate than the
possibly large resolved path to keep the data volume down.

Add the tgid to the stats to complete task identification, the flag AGROUP
to mark the last task of a group, the group wallclock time, and
inode-based identification of the associated executable file.

Add tools/accounting/procacct.c as a simplified fork of getdelays.c to
demonstrate process and thread accounting.

[thomas.orgis@uni-hamburg.de: fix version number in comment]
  Link: https://lkml.kernel.org/r/20220405003601.7a5f6008@plasteblaster
Link: https://lkml.kernel.org/r/20220331004106.64e5616b@plasteblaster
Signed-off-by: Dr. Thomas Orgis <thomas.orgis@uni-hamburg.de>
Reviewed-by: Ismael Luceno <ismael@iodev.co.uk>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: xu xin <xu.xin16@zte.com.cn>
Cc: Yang Yang <yang.yang29@zte.com.cn>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-04-29 14:38:03 -07:00
..
bpf bpf: Fix maximum permitted number of arguments check 2022-03-28 19:08:17 -07:00
cgroup Driver core changes for 5.18-rc1 2022-03-28 12:41:28 -07:00
configs Char/Misc and other driver updates for 5.18-rc1 2022-03-28 12:27:35 -07:00
debug kdb: Fix the putarea helper function 2022-03-24 16:39:47 +00:00
dma dma-direct: avoid redundant memory sync for swiotlb 2022-04-14 06:30:39 +02:00
entry entry: Fix compile error in dynamic_irqentry_exit_cond_resched() 2022-04-05 09:59:36 +02:00
events perf/core: Fix perf_mmap fail when CONFIG_PERF_USE_VMALLOC enabled 2022-04-19 21:15:42 +02:00
futex mm/truncate: Inline invalidate_complete_page() into its one caller 2022-03-21 12:59:01 -04:00
gcov gcov: Remove compiler version check 2021-12-02 17:25:21 +09:00
irq genirq/affinity: Consider that CPUs on nodes can be unbalanced 2022-04-11 09:58:03 +02:00
kcsan KCSAN updates for v5.17 2022-01-11 09:51:26 -08:00
livepatch ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
locking Changes in this cycle were: 2022-03-22 13:44:21 -07:00
power for-5.18/block-2022-03-18 2022-03-21 16:48:55 -07:00
printk printk changes for 5.18 2022-03-23 10:54:27 -07:00
rcu Changes in this cycle were: 2022-03-22 14:39:12 -07:00
sched sched/pelt: Fix attach_entity_load_avg() corner case 2022-04-19 21:15:41 +02:00
time tick/sched: Fix non-kernel-doc comment 2022-04-10 12:23:34 +02:00
trace Networking fixes for 5.18-rc2, including fixes from bpf and netfilter 2022-04-07 19:01:47 -10:00
.gitignore
acct.c kernel: remove spurious blkdev.h includes 2021-10-18 06:17:01 -06:00
async.c Revert "module, async: async_synchronize_full() on module init iff async is used" 2022-02-03 11:20:34 -08:00
audit.c audit: improve audit queue handling when "audit=1" on cmdline 2022-01-25 13:22:51 -05:00
audit.h audit: log AUDIT_TIME_* records only from rules 2022-02-22 13:51:40 -05:00
audit_fsnotify.c fsnotify: clarify contract for create event hooks 2021-10-27 12:32:34 +02:00
audit_tree.c audit: use struct_size() helper in kmalloc() 2021-12-14 17:39:42 -05:00
audit_watch.c \n 2021-11-06 16:43:20 -07:00
auditfilter.c audit/stable-5.17 PR 20220110 2022-01-11 13:08:21 -08:00
auditsc.c audit/stable-5.18 PR 20220321 2022-03-21 20:53:11 -07:00
backtracetest.c
bounds.c
capability.c xfs: don't generate selinux audit messages for capability testing 2022-03-09 10:32:06 -08:00
cfi.c
compat.c arch: remove compat_alloc_user_space 2021-09-08 15:32:35 -07:00
configs.c
context_tracking.c
cpu.c cpu/hotplug: Remove the 'cpu' member of cpuhp_cpu_state 2022-04-13 21:25:40 +02:00
cpu_pm.c PM: cpu: Make notifier chain use a raw_spinlock_t 2021-08-16 18:55:32 +02:00
crash_core.c kernel/crash_core: suppress unknown crashkernel parameter warning 2021-12-25 12:20:55 -08:00
crash_dump.c
cred.c x86: Mark __invalid_creds() __noreturn 2022-03-15 10:32:44 +01:00
delayacct.c delayacct: track delays from memory compact 2022-01-20 08:52:55 +02:00
dma.c
exec_domain.c
exit.c ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
extable.c lkdtm: Really write into kernel text in WRITE_KERN 2022-02-16 23:25:12 +11:00
fail_function.c
fork.c kasan, arm64: reset pointer tags of vmapped stacks 2022-03-24 19:06:47 -07:00
freezer.c
gen_kheaders.sh
groups.c
hung_task.c lib/Kconfig.debug: remove more CONFIG_..._VALUE indirections 2022-04-29 14:38:00 -07:00
iomem.c
irq_work.c irq_work: use kasan_record_aux_stack_noalloc() record callstack 2022-04-15 14:49:55 -07:00
jump_label.c
kallsyms.c kallsyms: Skip the name search for empty string 2022-03-17 20:17:18 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks locking/rwlock: Provide RT variant 2021-08-17 17:50:51 +02:00
Kconfig.preempt Revert "signal, x86: Delay calling signals in atomic on RT enabled kernels" 2022-03-31 10:36:55 +02:00
kcov.c kcov: don't generate a warning on vm_insert_page()'s failure 2022-04-21 20:01:10 -07:00
kexec.c kexec: avoid compat_alloc_user_space 2021-09-08 15:32:34 -07:00
kexec_core.c kexec: remove redundant assignments 2022-04-29 14:38:03 -07:00
kexec_elf.c
kexec_file.c memblock: add MEMBLOCK_DRIVER_MANAGED to mimic IORESOURCE_SYSRAM_DRIVER_MANAGED 2021-11-06 13:30:42 -07:00
kexec_internal.h
kheaders.c
kmod.c
kprobes.c kprobes: Fix KRETPROBES when CONFIG_KRETPROBE_ON_RETHOOK is set 2022-04-26 16:09:36 +02:00
ksysfs.c kernel/ksysfs.c: use helper macro __ATTR_RW 2022-03-23 19:00:33 -07:00
kthread.c asm-generic updates for 5.18 2022-03-23 18:03:08 -07:00
latencytop.c
Makefile static_call: Don't make __static_call_return0 static 2022-04-05 09:59:38 +02:00
module-internal.h module: add in-kernel support for decompressing 2022-01-11 18:45:02 -08:00
module.c NFSD: Remove svc_serv_ops::svo_module 2022-02-28 10:26:40 -05:00
module_decompress.c module: fix building with sysfs disabled 2022-02-16 12:51:32 -08:00
module_signature.c
module_signing.c
notifier.c notifier: Return an error when a callback has already been registered 2021-12-29 10:37:33 +01:00
nsproxy.c memcg: enable accounting for new namesapces and struct nsproxy 2021-09-03 09:58:12 -07:00
padata.c padata: replace cpumask_weight with cpumask_empty in padata.c 2022-01-31 11:21:46 +11:00
panic.c panic: move panic_print before kmsg dumpers 2022-03-23 19:00:35 -07:00
params.c kobject: remove kset from struct kset_uevent_ops callbacks 2021-12-28 11:26:18 +01:00
pid.c pid: add pidfd_get_task() helper 2021-10-14 13:29:18 +02:00
pid_namespace.c kernel: pid_namespace: use NULL instead of using plain integer as pointer 2022-04-29 14:38:00 -07:00
profile.c exit: Remove profile_handoff_task 2022-01-08 12:43:57 -06:00
ptrace.c ptrace: remove redudant check of #ifdef PTRACE_SINGLESTEP 2022-04-29 14:38:02 -07:00
range.c
reboot.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input 2021-11-12 11:53:16 -08:00
regset.c
relay.c
resource.c kernel/resource: fix kfree() of bootmem memory again 2022-03-23 19:00:35 -07:00
resource_kunit.c
rseq.c rseq: Remove broken uapi field layout on 32-bit little endian 2022-02-02 13:11:34 +01:00
scftorture.c scftorture: Always log error message 2021-12-07 16:36:17 -08:00
scs.c kasan, vmalloc: only tag normal vmalloc allocations 2022-03-24 19:06:48 -07:00
seccomp.c ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
signal.c Revert "signal, x86: Delay calling signals in atomic on RT enabled kernels" 2022-03-31 10:36:55 +02:00
smp.c smp: Fix offline cpu check in flush_smp_call_function_queue() 2022-04-13 18:44:35 +02:00
smpboot.c
smpboot.h
softirq.c genirq, softirq: Use in_hardirq() instead of in_irq() 2022-02-02 21:34:19 +01:00
stackleak.c gcc-plugins/stackleak: Use noinstr in favor of notrace 2022-02-03 17:02:21 -08:00
stacktrace.c uaccess: remove CONFIG_SET_FS 2022-02-25 09:36:06 +01:00
static_call.c static_call: Don't make __static_call_return0 static 2022-04-05 09:59:38 +02:00
static_call_inline.c static_call: Don't make __static_call_return0 static 2022-04-05 09:59:38 +02:00
stop_machine.c
sys.c prlimit: do not grab the tasklist_lock 2022-03-08 14:33:36 -06:00
sys_ni.c mm/mempolicy: wire up syscall set_mempolicy_home_node 2022-01-15 16:30:30 +02:00
sysctl-test.c
sysctl.c Merge branch 'akpm' (patches from Andrew) 2022-03-22 16:11:53 -07:00
task_work.c resume_user_mode: Move to resume_user_mode.h 2022-03-10 16:51:50 -06:00
taskstats.c taskstats: version 12 with thread group and exe info 2022-04-29 14:38:03 -07:00
torture.c torture: Wake up kthreads after storing task_struct pointer 2022-02-01 17:24:39 -08:00
tracepoint.c tracepoint: Fix kerneldoc comments 2021-08-16 11:39:51 -04:00
tsacct.c taskstats: version 12 with thread group and exe info 2022-04-29 14:38:03 -07:00
ucount.c ucounts: Handle wrapping in is_ucounts_overlimit 2022-02-17 09:11:57 -06:00
uid16.c
uid16.h
umh.c
up.c
user-return-notifier.c
user.c fs/epoll: use a per-cpu counter for user's watches count 2021-09-08 11:50:27 -07:00
user_namespace.c ucounts: Fix systemd LimitNPROC with private users regression 2022-02-25 10:40:14 -06:00
usermode_driver.c
utsname.c
utsname_sysctl.c
watch_queue.c watch_queue: Free the page array when watch_queue is dismantled 2022-04-02 10:37:39 -07:00
watchdog.c lib/Kconfig.debug: remove more CONFIG_..._VALUE indirections 2022-04-29 14:38:00 -07:00
watchdog_hld.c
workqueue.c Merge branch 'for-5.18' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq 2022-03-23 12:40:51 -07:00
workqueue_internal.h workqueue: Assign a color to barrier work items 2021-08-17 07:49:10 -10:00