public-mirrors/linux
1
0
Fork 0
mirror of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2025-08-05 16:54:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
linux/arch/riscv/include/asm/vdso/processor.h

29 lines
578 B
C
Raw Permalink Normal View History

riscv: use vDSO common flow to reduce the latency of the time-related functions Even if RISC-V has supported the vDSO feature, the latency of the functions for obtaining the system time is still expensive. It is because these functions still trigger a corresponding system call in the process, which slows down the response time. If we want to remove the system call to reduce the latency, the kernel should have the ability to output the system clock information to userspace. This patch introduces the vDSO common flow to enable the kernel to achieve the above feature and uses "rdtime" instruction to obtain the current time in the user space. Under this condition, the latency cost by the ecall from U-mode to S-mode can be eliminated. After applying this patch, the latency of gettimeofday() measured on the HiFive unleashed board can be reduced by %61. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-09 22:14:48 +08:00
/* SPDX-License-Identifier: GPL-2.0-only */
#ifndef __ASM_VDSO_PROCESSOR_H
#define __ASM_VDSO_PROCESSOR_H
#ifndef __ASSEMBLY__
RISC-V: fix barrier() use in <vdso/processor.h> riscv's <vdso/processor.h> uses barrier() so it should include <asm/barrier.h> Fixes this build error: CC [M] drivers/net/ethernet/emulex/benet/be_main.o In file included from ./include/vdso/processor.h:10, from ./arch/riscv/include/asm/processor.h:11, from ./include/linux/prefetch.h:15, from drivers/net/ethernet/emulex/benet/be_main.c:14: ./arch/riscv/include/asm/vdso/processor.h: In function 'cpu_relax': ./arch/riscv/include/asm/vdso/processor.h:14:2: error: implicit declaration of function 'barrier' [-Werror=implicit-function-declaration] 14 | barrier(); This happens with a total of 5 networking drivers -- they all use <linux/prefetch.h>. rv64 allmodconfig now builds cleanly after this patch. Fixes fallout from: 815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive") Fixes: ad5d1122b82f ("riscv: use vDSO common flow to reduce the latency of the time-related functions") Reported-by: Andreas Schwab <schwab@linux-m68k.org> Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-11-16 17:39:51 -08:00
#include <asm/barrier.h>
riscv: Provide a definition for 'pause' If we're going to provide the encoding for 'pause' in cpu_relax() anyway, then we can drop the toolchain checks and just always use it. The advantage of doing this is that other code that need pause don't need to also define it (yes, another use is coming). Add the definition to insn-def.h since it's an instruction definition and also because insn-def.h doesn't include much, so it's safe to include from asm/vdso/processor.h without concern for circular dependencies. Signed-off-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20240426100820.14762-9-ajones@ventanamicro.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-04-26 12:08:21 +02:00
#include <asm/insn-def.h>
RISC-V: fix barrier() use in <vdso/processor.h> riscv's <vdso/processor.h> uses barrier() so it should include <asm/barrier.h> Fixes this build error: CC [M] drivers/net/ethernet/emulex/benet/be_main.o In file included from ./include/vdso/processor.h:10, from ./arch/riscv/include/asm/processor.h:11, from ./include/linux/prefetch.h:15, from drivers/net/ethernet/emulex/benet/be_main.c:14: ./arch/riscv/include/asm/vdso/processor.h: In function 'cpu_relax': ./arch/riscv/include/asm/vdso/processor.h:14:2: error: implicit declaration of function 'barrier' [-Werror=implicit-function-declaration] 14 | barrier(); This happens with a total of 5 networking drivers -- they all use <linux/prefetch.h>. rv64 allmodconfig now builds cleanly after this patch. Fixes fallout from: 815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive") Fixes: ad5d1122b82f ("riscv: use vDSO common flow to reduce the latency of the time-related functions") Reported-by: Andreas Schwab <schwab@linux-m68k.org> Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Arvind Sankar <nivedita@alum.mit.edu> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-11-16 17:39:51 -08:00
riscv: use vDSO common flow to reduce the latency of the time-related functions Even if RISC-V has supported the vDSO feature, the latency of the functions for obtaining the system time is still expensive. It is because these functions still trigger a corresponding system call in the process, which slows down the response time. If we want to remove the system call to reduce the latency, the kernel should have the ability to output the system clock information to userspace. This patch introduces the vDSO common flow to enable the kernel to achieve the above feature and uses "rdtime" instruction to obtain the current time in the user space. Under this condition, the latency cost by the ecall from U-mode to S-mode can be eliminated. After applying this patch, the latency of gettimeofday() measured on the HiFive unleashed board can be reduced by %61. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-09 22:14:48 +08:00
static inline void cpu_relax(void)
{
#ifdef __riscv_muldiv
riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y commit 8eb060e10185 ("arch/riscv: add Zihintpause support") broke building with CONFIG_CC_OPTIMIZE_FOR_SIZE enabled (gcc 11.1.0): CC arch/riscv/kernel/vdso/vgettimeofday.o In file included from <command-line>: ./arch/riscv/include/asm/jump_label.h: In function 'cpu_relax': ././include/linux/compiler_types.h:285:33: warning: 'asm' operand 0 probably does not match constraints 285 | #define asm_volatile_goto(x...) asm goto(x) | ^~~ ./arch/riscv/include/asm/jump_label.h:41:9: note: in expansion of macro 'asm_volatile_goto' 41 | asm_volatile_goto( | ^~~~~~~~~~~~~~~~~ ././include/linux/compiler_types.h:285:33: error: impossible constraint in 'asm' 285 | #define asm_volatile_goto(x...) asm goto(x) | ^~~ ./arch/riscv/include/asm/jump_label.h:41:9: note: in expansion of macro 'asm_volatile_goto' 41 | asm_volatile_goto( | ^~~~~~~~~~~~~~~~~ make[1]: *** [scripts/Makefile.build:249: arch/riscv/kernel/vdso/vgettimeofday.o] Error 1 make: *** [arch/riscv/Makefile:128: vdso_prepare] Error 2 Having a static branch in cpu_relax() is problematic because that function is widely inlined, including in some quite complex functions like in the VDSO. A quick measurement shows this static branch is responsible by itself for around 40% of the jump table. Drop the static branch, which ends up being the same number of instructions anyway. If Zihintpause is supported, we trade the nop from the static branch for a div. If Zihintpause is unsupported, we trade the jump from the static branch for (what gets interpreted as) a nop. Fixes: 8eb060e10185 ("arch/riscv: add Zihintpause support") Signed-off-by: Samuel Holland <samuel@sholland.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-22 01:09:58 -05:00
int dummy;
/* In lieu of a halt instruction, induce a long-latency stall. */
__asm__ __volatile__ ("div %0, %0, zero" : "=r" (dummy));
riscv: use vDSO common flow to reduce the latency of the time-related functions Even if RISC-V has supported the vDSO feature, the latency of the functions for obtaining the system time is still expensive. It is because these functions still trigger a corresponding system call in the process, which slows down the response time. If we want to remove the system call to reduce the latency, the kernel should have the ability to output the system clock information to userspace. This patch introduces the vDSO common flow to enable the kernel to achieve the above feature and uses "rdtime" instruction to obtain the current time in the user space. Under this condition, the latency cost by the ecall from U-mode to S-mode can be eliminated. After applying this patch, the latency of gettimeofday() measured on the HiFive unleashed board can be reduced by %61. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-09 22:14:48 +08:00
#endif
Merge patch "riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y" This is a single fix, but it conflicts with some recent features. I'm merging it on top of the commit it fixes to ease backporting. * b4-shazam-merge: riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y Link: https://lore.kernel.org/r/20220922060958.44203-1-samuel@sholland.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-31 21:55:43 -08:00
riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y commit 8eb060e10185 ("arch/riscv: add Zihintpause support") broke building with CONFIG_CC_OPTIMIZE_FOR_SIZE enabled (gcc 11.1.0): CC arch/riscv/kernel/vdso/vgettimeofday.o In file included from <command-line>: ./arch/riscv/include/asm/jump_label.h: In function 'cpu_relax': ././include/linux/compiler_types.h:285:33: warning: 'asm' operand 0 probably does not match constraints 285 | #define asm_volatile_goto(x...) asm goto(x) | ^~~ ./arch/riscv/include/asm/jump_label.h:41:9: note: in expansion of macro 'asm_volatile_goto' 41 | asm_volatile_goto( | ^~~~~~~~~~~~~~~~~ ././include/linux/compiler_types.h:285:33: error: impossible constraint in 'asm' 285 | #define asm_volatile_goto(x...) asm goto(x) | ^~~ ./arch/riscv/include/asm/jump_label.h:41:9: note: in expansion of macro 'asm_volatile_goto' 41 | asm_volatile_goto( | ^~~~~~~~~~~~~~~~~ make[1]: *** [scripts/Makefile.build:249: arch/riscv/kernel/vdso/vgettimeofday.o] Error 1 make: *** [arch/riscv/Makefile:128: vdso_prepare] Error 2 Having a static branch in cpu_relax() is problematic because that function is widely inlined, including in some quite complex functions like in the VDSO. A quick measurement shows this static branch is responsible by itself for around 40% of the jump table. Drop the static branch, which ends up being the same number of instructions anyway. If Zihintpause is supported, we trade the nop from the static branch for a div. If Zihintpause is unsupported, we trade the jump from the static branch for (what gets interpreted as) a nop. Fixes: 8eb060e10185 ("arch/riscv: add Zihintpause support") Signed-off-by: Samuel Holland <samuel@sholland.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-22 01:09:58 -05:00
/*
* Reduce instruction retirement.
* This assumes the PC changes.
*/
riscv: Provide a definition for 'pause' If we're going to provide the encoding for 'pause' in cpu_relax() anyway, then we can drop the toolchain checks and just always use it. The advantage of doing this is that other code that need pause don't need to also define it (yes, another use is coming). Add the definition to insn-def.h since it's an instruction definition and also because insn-def.h doesn't include much, so it's safe to include from asm/vdso/processor.h without concern for circular dependencies. Signed-off-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20240426100820.14762-9-ajones@ventanamicro.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-04-26 12:08:21 +02:00
__asm__ __volatile__ (RISCV_PAUSE);
riscv: use vDSO common flow to reduce the latency of the time-related functions Even if RISC-V has supported the vDSO feature, the latency of the functions for obtaining the system time is still expensive. It is because these functions still trigger a corresponding system call in the process, which slows down the response time. If we want to remove the system call to reduce the latency, the kernel should have the ability to output the system clock information to userspace. This patch introduces the vDSO common flow to enable the kernel to achieve the above feature and uses "rdtime" instruction to obtain the current time in the user space. Under this condition, the latency cost by the ecall from U-mode to S-mode can be eliminated. After applying this patch, the latency of gettimeofday() measured on the HiFive unleashed board can be reduced by %61. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-09 22:14:48 +08:00
barrier();
}
#endif /* __ASSEMBLY__ */
#endif /* __ASM_VDSO_PROCESSOR_H */
Reference in a new issue Copy permalink