public-mirrors/linux
1
0
Fork 0
mirror of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2025-08-21 06:50:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
linux/arch/s390/include/asm/asm-extable.h

88 lines
3.3 KiB
C
Raw Permalink Normal View History

s390/extable: move EX_TABLE define to asm-extable.h Follow arm64 and riscv and move the EX_TABLE define to asm-extable.h which is a lot less generic than the current linkage.h. Also make sure that all files which contain EX_TABLE usages actually include the new header file. This should make sure that the files always compile and there won't be any random compile breakage due to other header file dependencies. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 11:22:12 +01:00
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_EXTABLE_H
#define __ASM_EXTABLE_H
#include <linux/stringify.h>
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
#include <linux/bits.h>
s390/extable: move EX_TABLE define to asm-extable.h Follow arm64 and riscv and move the EX_TABLE define to asm-extable.h which is a lot less generic than the current linkage.h. Also make sure that all files which contain EX_TABLE usages actually include the new header file. This should make sure that the files always compile and there won't be any random compile breakage due to other header file dependencies. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 11:22:12 +01:00
#include <asm/asm-const.h>
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
#define EX_TYPE_NONE 0
#define EX_TYPE_FIXUP 1
#define EX_TYPE_BPF 2
#define EX_TYPE_UA_STORE 3
#define EX_TYPE_UA_LOAD_MEM 4
#define EX_TYPE_UA_LOAD_REG 5
s390/extable: add EX_TABLE_UA_LOAD_REGPAIR() macro Add new exception table type which is able to handle register pairs. If an exception is recognized on such an instruction the specified register pair will be zeroed, and the specified error register will be modified so it contains -EFAULT, similar to the existing EX_TABLE_UA_LOAD_REG() macro. Link: https://lore.kernel.org/r/Y2J8RSW2khWLgpPo@osiris Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-11-02 15:18:45 +01:00
#define EX_TYPE_UA_LOAD_REGPAIR 6
s390: add support for DCACHE_WORD_ACCESS Implement load_unaligned_zeropad() and enable DCACHE_WORD_ACCESS to speed up string operations in fs/dcache.c and fs/namei.c. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:42 +02:00
#define EX_TYPE_ZEROPAD 7
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
#define EX_DATA_REG_ERR_SHIFT 0
#define EX_DATA_REG_ERR GENMASK(3, 0)
#define EX_DATA_REG_ADDR_SHIFT 4
#define EX_DATA_REG_ADDR GENMASK(7, 4)
#define EX_DATA_LEN_SHIFT 8
#define EX_DATA_LEN GENMASK(11, 8)
s390/extable: convert to relative table with data Follow arm64, riscv, and x86 and change extable layout to common "relative table with data". This allows to get rid of s390 specific code in sorttable.c. The main difference to before is that extable entries do not contain a relative function pointer anymore. Instead data and type fields are added. The type field is used to indicate which exception handler needs to be called, while the data field is currently unused. Acked-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 14:52:42 +01:00
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
#define __EX_TABLE(_section, _fault, _target, _type, _regerr, _regaddr, _len) \
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
stringify_in_c(.section _section,"a";) \
s390: consistently use .balign instead of .align The .align directive has inconsistent behavior across architectures. Use .balign instead everywhere. This is a no-op for s390, but with this there is no mix in using .align and .balign anymore. Future code is supposed to use only .balign. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
2023-06-22 13:24:40 +02:00
stringify_in_c(.balign 4;) \
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
stringify_in_c(.long (_fault) - .;) \
stringify_in_c(.long (_target) - .;) \
stringify_in_c(.short (_type);) \
stringify_in_c(.macro extable_reg regerr, regaddr;) \
stringify_in_c(.set .Lfound, 0;) \
stringify_in_c(.set .Lcurr, 0;) \
stringify_in_c(.irp rs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15;) \
stringify_in_c( .ifc "\regerr", "%%r\rs";) \
stringify_in_c( .set .Lfound, 1;) \
stringify_in_c( .set .Lregerr, .Lcurr;) \
stringify_in_c( .endif;) \
stringify_in_c( .set .Lcurr, .Lcurr+1;) \
stringify_in_c(.endr;) \
stringify_in_c(.ifne (.Lfound != 1);) \
stringify_in_c( .error "extable_reg: bad register argument1";) \
stringify_in_c(.endif;) \
stringify_in_c(.set .Lfound, 0;) \
stringify_in_c(.set .Lcurr, 0;) \
stringify_in_c(.irp rs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15;) \
stringify_in_c( .ifc "\regaddr", "%%r\rs";) \
stringify_in_c( .set .Lfound, 1;) \
stringify_in_c( .set .Lregaddr, .Lcurr;) \
stringify_in_c( .endif;) \
stringify_in_c( .set .Lcurr, .Lcurr+1;) \
stringify_in_c(.endr;) \
stringify_in_c(.ifne (.Lfound != 1);) \
stringify_in_c( .error "extable_reg: bad register argument2";) \
stringify_in_c(.endif;) \
stringify_in_c(.short .Lregerr << EX_DATA_REG_ERR_SHIFT | \
.Lregaddr << EX_DATA_REG_ADDR_SHIFT | \
_len << EX_DATA_LEN_SHIFT;) \
stringify_in_c(.endm;) \
stringify_in_c(extable_reg _regerr,_regaddr;) \
stringify_in_c(.purgem extable_reg;) \
s390/extable: add dedicated uaccess handler This is more or less a combination of commit 2e77a62cb3a6 ("arm64: extable: add a dedicated uaccess handler") and commit 4b5305decc84 ("x86/extable: Extend extable functionality"). To describe the problem that needs to solved let's cite the full arm64 commit message: ------ For inline assembly, we place exception fixups out-of-line in the `.fixup` section such that these are out of the way of the fast path. This has a few drawbacks: * Since the fixup code is anonymous, backtraces will symbolize fixups as offsets from the nearest prior symbol, currently `__entry_tramp_text_end`. This is confusing, and painful to debug without access to the relevant vmlinux. * Since the exception handler adjusts the PC to execute the fixup, and the fixup uses a direct branch back into the function it fixes, backtraces of fixups miss the original function. This is confusing, and violates requirements for RELIABLE_STACKTRACE (and therefore LIVEPATCH). * Inline assembly and associated fixups are generated from templates, and we have many copies of logically identical fixups which only differ in which specific registers are written to and which address is branched to at the end of the fixup. This is potentially wasteful of I-cache resources, and makes it hard to add additional logic to fixups without significant bloat. This patch address all three concerns for inline uaccess fixups by adding a dedicated exception handler which updates registers in exception context and subsequent returns back into the function which faulted, removing the need for fixups specialized to each faulting instruction. Other than backtracing, there should be no functional change as a result of this patch. ------ Acked-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 15:02:46 +01:00
stringify_in_c(.previous)
s390/extable: move EX_TABLE define to asm-extable.h Follow arm64 and riscv and move the EX_TABLE define to asm-extable.h which is a lot less generic than the current linkage.h. Also make sure that all files which contain EX_TABLE usages actually include the new header file. This should make sure that the files always compile and there won't be any random compile breakage due to other header file dependencies. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 11:22:12 +01:00
#define EX_TABLE(_fault, _target) \
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_FIXUP, __stringify(%%r0), __stringify(%%r0), 0)
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
s390/extable: move EX_TABLE define to asm-extable.h Follow arm64 and riscv and move the EX_TABLE define to asm-extable.h which is a lot less generic than the current linkage.h. Also make sure that all files which contain EX_TABLE usages actually include the new header file. This should make sure that the files always compile and there won't be any random compile breakage due to other header file dependencies. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 11:22:12 +01:00
#define EX_TABLE_AMODE31(_fault, _target) \
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
__EX_TABLE(.amode31.ex_table, _fault, _target, EX_TYPE_FIXUP, __stringify(%%r0), __stringify(%%r0), 0)
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
#define EX_TABLE_UA_STORE(_fault, _target, _regerr) \
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_UA_STORE, _regerr, _regerr, 0)
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
#define EX_TABLE_UA_LOAD_MEM(_fault, _target, _regerr, _regmem, _len) \
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_UA_LOAD_MEM, _regerr, _regmem, _len)
s390/uaccess: use exception handler to zero result on get_user() failure Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-01 21:26:06 +02:00
#define EX_TABLE_UA_LOAD_REG(_fault, _target, _regerr, _regzero) \
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_UA_LOAD_REG, _regerr, _regzero, 0)
s390/extable: move EX_TABLE define to asm-extable.h Follow arm64 and riscv and move the EX_TABLE define to asm-extable.h which is a lot less generic than the current linkage.h. Also make sure that all files which contain EX_TABLE usages actually include the new header file. This should make sure that the files always compile and there won't be any random compile breakage due to other header file dependencies. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 11:22:12 +01:00
s390/extable: add EX_TABLE_UA_LOAD_REGPAIR() macro Add new exception table type which is able to handle register pairs. If an exception is recognized on such an instruction the specified register pair will be zeroed, and the specified error register will be modified so it contains -EFAULT, similar to the existing EX_TABLE_UA_LOAD_REG() macro. Link: https://lore.kernel.org/r/Y2J8RSW2khWLgpPo@osiris Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-11-02 15:18:45 +01:00
#define EX_TABLE_UA_LOAD_REGPAIR(_fault, _target, _regerr, _regzero) \
s390/extable: reduce number of extable macros Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:40 +02:00
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_UA_LOAD_REGPAIR, _regerr, _regzero, 0)
s390/extable: add EX_TABLE_UA_LOAD_REGPAIR() macro Add new exception table type which is able to handle register pairs. If an exception is recognized on such an instruction the specified register pair will be zeroed, and the specified error register will be modified so it contains -EFAULT, similar to the existing EX_TABLE_UA_LOAD_REG() macro. Link: https://lore.kernel.org/r/Y2J8RSW2khWLgpPo@osiris Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-11-02 15:18:45 +01:00
s390: add support for DCACHE_WORD_ACCESS Implement load_unaligned_zeropad() and enable DCACHE_WORD_ACCESS to speed up string operations in fs/dcache.c and fs/namei.c. Reviewed-by: Sven Schnelle <svens@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2023-10-06 15:42:42 +02:00
#define EX_TABLE_ZEROPAD(_fault, _target, _regdata, _regaddr) \
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_ZEROPAD, _regdata, _regaddr, 0)
s390/extable: move EX_TABLE define to asm-extable.h Follow arm64 and riscv and move the EX_TABLE define to asm-extable.h which is a lot less generic than the current linkage.h. Also make sure that all files which contain EX_TABLE usages actually include the new header file. This should make sure that the files always compile and there won't be any random compile breakage due to other header file dependencies. Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2022-02-28 11:22:12 +01:00
#endif /* __ASM_EXTABLE_H */
Reference in a new issue Copy permalink