public-mirrors/linux
1
0
Fork 0
mirror of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2025-08-05 16:54:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
linux/arch/loongarch/include/asm/inst.h

788 lines
20 KiB
C
Raw Permalink Normal View History

LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2020-2022 Loongson Technology Corporation Limited
*/
#ifndef _ASM_INST_H
#define _ASM_INST_H
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#include <linux/bitops.h>
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
#include <linux/types.h>
#include <asm/asm.h>
LoongArch: Simulate branch and PC* instructions According to LoongArch Reference Manual, simulate branch and PC* instructions, this is preparation for later patch. Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#branch-instructions Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#_pcaddi_pcaddu121_pcaddu18l_pcalau12i Tested-by: Jeff Xie <xiehuan09@gmail.com> Co-developed-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
#include <asm/ptrace.h>
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
LoongArch: Add alternative runtime patching mechanism Introduce the "alternative" mechanism from ARM64 and x86 for LoongArch to apply runtime patching. The main purpose of this patch is to provide a framework. In future we can use this mechanism (i.e., the ALTERNATIVE and ALTERNATIVE_2 macros) to optimize hotspot functions according to cpu features. Signed-off-by: Jun Yi <yijun@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
#define INSN_NOP 0x03400000
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
#define INSN_BREAK 0x002a0000
LoongArch: KVM: Add software breakpoint support When VM runs in kvm mode, system will not exit to host mode when executing a general software breakpoint instruction such as INSN_BREAK, trap exception happens in guest mode rather than host mode. In order to debug guest kernel on host side, one mechanism should be used to let VM exit to host mode. Here a hypercall instruction with a special code is used for software breakpoint usage. VM exits to host mode and kvm hypervisor identifies the special hypercall code and sets exit_reason with KVM_EXIT_DEBUG. And then let qemu handle it. Idea comes from ppc kvm, one api KVM_REG_LOONGARCH_DEBUG_INST is added to get the hypercall code. VMM needs get sw breakpoint instruction with this api and set the corresponding sw break point for guest kernel. Signed-off-by: Bibo Mao <maobibo@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2024-05-06 22:00:47 +08:00
#define INSN_HVCL 0x002b8000
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
#define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
#define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
LoongArch: modules/ftrace: Initialize PLT at load time This patch implements ftrace trampolines through plt entry. Tested by forcing ftrace_make_call() to use the module PLT, and then loading up a module after setting up ftrace with: | echo ":mod:<module-name>" > set_ftrace_filter; | echo function > current_tracer; | modprobe <module-name> Since FTRACE_ADDR/FTRACE_REGS_ADDR is only defined when CONFIG_DYNAMIC_ FTRACE is selected, we wrap their usage in module_init_ftrace_plt() with ifdeffery rather than using IS_ENABLED(). Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:40:21 +08:00
#define ADDR_IMMMASK_LU12IW 0x00000000FFFFF000
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#define ADDR_IMMMASK_ORI 0x0000000000000FFF
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
#define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
#define ADDR_IMMSHIFT_LU52ID 52
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#define ADDR_IMMSBIDX_LU52ID 11
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
#define ADDR_IMMSHIFT_LU32ID 32
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#define ADDR_IMMSBIDX_LU32ID 19
LoongArch: modules/ftrace: Initialize PLT at load time This patch implements ftrace trampolines through plt entry. Tested by forcing ftrace_make_call() to use the module PLT, and then loading up a module after setting up ftrace with: | echo ":mod:<module-name>" > set_ftrace_filter; | echo function > current_tracer; | modprobe <module-name> Since FTRACE_ADDR/FTRACE_REGS_ADDR is only defined when CONFIG_DYNAMIC_ FTRACE is selected, we wrap their usage in module_init_ftrace_plt() with ifdeffery rather than using IS_ENABLED(). Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:40:21 +08:00
#define ADDR_IMMSHIFT_LU12IW 12
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#define ADDR_IMMSBIDX_LU12IW 19
#define ADDR_IMMSHIFT_ORI 0
#define ADDR_IMMSBIDX_ORI 63
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
#define ADDR_IMMSHIFT_ADDU16ID 16
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#define ADDR_IMMSBIDX_ADDU16ID 15
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
#define ADDR_IMM(addr, INSN) \
(sign_extend64(((addr & ADDR_IMMMASK_##INSN) >> ADDR_IMMSHIFT_##INSN), ADDR_IMMSBIDX_##INSN))
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
LoongArch: Add kprobes support Kprobes allows you to trap at almost any kernel address and execute a callback function, this commit adds kprobes support for LoongArch. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
enum reg0i15_op {
break_op = 0x54,
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
enum reg0i26_op {
b_op = 0x14,
bl_op = 0x15,
};
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
enum reg1i20_op {
lu12iw_op = 0x0a,
lu32id_op = 0x0b,
LoongArch: Add alternative runtime patching mechanism Introduce the "alternative" mechanism from ARM64 and x86 for LoongArch to apply runtime patching. The main purpose of this patch is to provide a framework. In future we can use this mechanism (i.e., the ALTERNATIVE and ALTERNATIVE_2 macros) to optimize hotspot functions according to cpu features. Signed-off-by: Jun Yi <yijun@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
pcaddi_op = 0x0c,
LoongArch: Simulate branch and PC* instructions According to LoongArch Reference Manual, simulate branch and PC* instructions, this is preparation for later patch. Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#branch-instructions Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#_pcaddi_pcaddu121_pcaddu18l_pcalau12i Tested-by: Jeff Xie <xiehuan09@gmail.com> Co-developed-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
pcalau12i_op = 0x0d,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
pcaddu12i_op = 0x0e,
pcaddu18i_op = 0x0f,
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
};
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
enum reg1i21_op {
beqz_op = 0x10,
bnez_op = 0x11,
LoongArch: Add alternative runtime patching mechanism Introduce the "alternative" mechanism from ARM64 and x86 for LoongArch to apply runtime patching. The main purpose of this patch is to provide a framework. In future we can use this mechanism (i.e., the ALTERNATIVE and ALTERNATIVE_2 macros) to optimize hotspot functions according to cpu features. Signed-off-by: Jun Yi <yijun@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
bceqz_op = 0x12, /* bits[9:8] = 0x00 */
bcnez_op = 0x12, /* bits[9:8] = 0x01 */
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
enum reg2_op {
revb2h_op = 0x0c,
revb4h_op = 0x0d,
revb2w_op = 0x0e,
revbd_op = 0x0f,
revh2w_op = 0x10,
revhd_op = 0x11,
LoongArch: Add more instruction opcodes and emit_* helpers This patch adds more instruction opcodes and their corresponding emit_* helpers which will be used in later patches. Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-11-08 14:12:15 +08:00
extwh_op = 0x16,
extwb_op = 0x17,
LoongArch: KVM: Add cpucfg area for kvm hypervisor Instruction cpucfg can be used to get processor features. And there is a trap exception when it is executed in VM mode, and also it can be used to provide cpu features to VM. On real hardware cpucfg area 0 - 20 is used by now. Here one specified area 0x40000000 -- 0x400000ff is used for KVM hypervisor to provide PV features, and the area can be extended for other hypervisors in future. This area will never be used for real HW, it is only used by software. Signed-off-by: Bibo Mao <maobibo@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2024-05-06 22:00:47 +08:00
cpucfg_op = 0x1b,
LoongArch: KVM: Implement handle iocsr exception Implement kvm handle vcpu iocsr exception, setting the iocsr info into vcpu_run and return to user space to handle it. Reviewed-by: Bibo Mao <maobibo@loongson.cn> Tested-by: Huacai Chen <chenhuacai@loongson.cn> Signed-off-by: Tianrui Zhao <zhaotianrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-10-02 10:01:28 +08:00
iocsrrdb_op = 0x19200,
iocsrrdh_op = 0x19201,
iocsrrdw_op = 0x19202,
iocsrrdd_op = 0x19203,
iocsrwrb_op = 0x19204,
iocsrwrh_op = 0x19205,
iocsrwrw_op = 0x19206,
iocsrwrd_op = 0x19207,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
};
enum reg2i5_op {
slliw_op = 0x81,
srliw_op = 0x89,
sraiw_op = 0x91,
};
enum reg2i6_op {
sllid_op = 0x41,
srlid_op = 0x45,
sraid_op = 0x49,
};
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
enum reg2i12_op {
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
addiw_op = 0x0a,
addid_op = 0x0b,
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
lu52id_op = 0x0c,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
andi_op = 0x0d,
ori_op = 0x0e,
xori_op = 0x0f,
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
ldb_op = 0xa0,
ldh_op = 0xa1,
ldw_op = 0xa2,
ldd_op = 0xa3,
stb_op = 0xa4,
sth_op = 0xa5,
stw_op = 0xa6,
std_op = 0xa7,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
ldbu_op = 0xa8,
ldhu_op = 0xa9,
ldwu_op = 0xaa,
LoongArch: Add unaligned access support Loongson-2 series (Loongson-2K500, Loongson-2K1000) don't support unaligned access in hardware, while Loongson-3 series (Loongson-3A5000, Loongson-3C5000) are configurable whether support unaligned access in hardware. This patch add unaligned access emulation for those LoongArch processors without hardware support. Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
flds_op = 0xac,
fsts_op = 0xad,
fldd_op = 0xae,
fstd_op = 0xaf,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
};
enum reg2i14_op {
llw_op = 0x20,
scw_op = 0x21,
lld_op = 0x22,
scd_op = 0x23,
ldptrw_op = 0x24,
stptrw_op = 0x25,
ldptrd_op = 0x26,
stptrd_op = 0x27,
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
};
enum reg2i16_op {
jirl_op = 0x13,
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
beq_op = 0x16,
bne_op = 0x17,
blt_op = 0x18,
bge_op = 0x19,
bltu_op = 0x1a,
bgeu_op = 0x1b,
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
enum reg2bstrd_op {
bstrinsd_op = 0x2,
bstrpickd_op = 0x3,
};
enum reg3_op {
LoongArch: Relay BCE exceptions to userland as SIGSEGV with si_code=SEGV_BNDERR SEGV_BNDERR was introduced initially for supporting the Intel MPX, but fell into disuse after the MPX support was removed. The LoongArch bounds-checking instructions behave very differently than MPX, but overall the interface is still kind of suitable for conveying the information to userland when bounds-checking assertions trigger, so we wouldn't have to invent more UAPI. Specifically, when the BCE triggers, a SEGV_BNDERR is sent to userland, with si_addr set to the out-of-bounds address or value (in asrt{gt,le}'s case), and one of si_lower or si_upper set to the configured bound depending on the faulting instruction. The other bound is set to either 0 or ULONG_MAX to resemble a range with both lower and upper bounds. Note that it is possible to have si_addr == si_lower in case of a failing asrtgt or {ld,st}gt, because those instructions test for strict greater-than relationship. This should not pose a problem for userland, though, because the faulting PC is available for the application to associate back to the exact instruction for figuring out the expectation. Example exception context generated by a faulting `asrtgt.d t0, t1` (assert t0 > t1 or BCE) with t0=100 and t1=200: > pc 00005555558206a4 ra 00007ffff2d854fc tp 00007ffff2f2f180 sp 00007ffffbf9fb80 > a0 0000000000000002 a1 00007ffffbf9fce8 a2 00007ffffbf9fd00 a3 00007ffff2ed4558 > a4 0000000000000000 a5 00007ffff2f044c8 a6 00007ffffbf9fce0 a7 fffffffffffff000 > t0 0000000000000064 t1 00000000000000c8 t2 00007ffffbfa2d5e t3 00007ffff2f12aa0 > t4 00007ffff2ed6158 t5 00007ffff2ed6158 t6 000000000000002e t7 0000000003d8f538 > t8 0000000000000005 u0 0000000000000000 s9 0000000000000000 s0 00007ffffbf9fce8 > s1 0000000000000002 s2 0000000000000000 s3 00007ffff2f2c038 s4 0000555555820610 > s5 00007ffff2ed5000 s6 0000555555827e38 s7 00007ffffbf9fd00 s8 0000555555827e38 > ra: 00007ffff2d854fc > ERA: 00005555558206a4 > CRMD: 000000b0 (PLV0 -IE -DA +PG DACF=CC DACM=CC -WE) > PRMD: 00000007 (PPLV3 +PIE -PWE) > EUEN: 00000000 (-FPE -SXE -ASXE -BTE) > ECFG: 0007181c (LIE=2-4,11-12 VS=7) > ESTAT: 000a0000 [BCE] (IS= ECode=10 EsubCode=0) > PRID: 0014c010 (Loongson-64bit, Loongson-3A5000) Signed-off-by: WANG Xuerui <git@xen0n.name> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-05-01 17:19:27 +08:00
asrtle_op = 0x02,
asrtgt_op = 0x03,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
addw_op = 0x20,
addd_op = 0x21,
subw_op = 0x22,
subd_op = 0x23,
nor_op = 0x28,
and_op = 0x29,
or_op = 0x2a,
xor_op = 0x2b,
orn_op = 0x2c,
andn_op = 0x2d,
sllw_op = 0x2e,
srlw_op = 0x2f,
sraw_op = 0x30,
slld_op = 0x31,
srld_op = 0x32,
srad_op = 0x33,
mulw_op = 0x38,
mulhw_op = 0x39,
mulhwu_op = 0x3a,
muld_op = 0x3b,
mulhd_op = 0x3c,
mulhdu_op = 0x3d,
divw_op = 0x40,
modw_op = 0x41,
divwu_op = 0x42,
modwu_op = 0x43,
divd_op = 0x44,
modd_op = 0x45,
divdu_op = 0x46,
moddu_op = 0x47,
ldxb_op = 0x7000,
ldxh_op = 0x7008,
ldxw_op = 0x7010,
ldxd_op = 0x7018,
stxb_op = 0x7020,
stxh_op = 0x7028,
stxw_op = 0x7030,
stxd_op = 0x7038,
ldxbu_op = 0x7040,
ldxhu_op = 0x7048,
ldxwu_op = 0x7050,
LoongArch: Add unaligned access support Loongson-2 series (Loongson-2K500, Loongson-2K1000) don't support unaligned access in hardware, while Loongson-3 series (Loongson-3A5000, Loongson-3C5000) are configurable whether support unaligned access in hardware. This patch add unaligned access emulation for those LoongArch processors without hardware support. Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
fldxs_op = 0x7060,
fldxd_op = 0x7068,
fstxs_op = 0x7070,
fstxd_op = 0x7078,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
amswapw_op = 0x70c0,
amswapd_op = 0x70c1,
amaddw_op = 0x70c2,
amaddd_op = 0x70c3,
amandw_op = 0x70c4,
amandd_op = 0x70c5,
amorw_op = 0x70c6,
amord_op = 0x70c7,
amxorw_op = 0x70c8,
amxord_op = 0x70c9,
LoongArch: Check for AMO instructions in insns_not_supported() Like llsc instructions, the atomic memory access instructions shouldn't be supported for probing, so check for them in insns_not_supported(). Closes: https://lore.kernel.org/all/SY4P282MB351877A70A0333C790FE85A5C09C9@SY4P282MB3518.AUSP282.PROD.OUTLOOK.COM/ Tested-by: Jeff Xie <xiehuan09@gmail.com> Reported-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:44 +08:00
ammaxw_op = 0x70ca,
ammaxd_op = 0x70cb,
amminw_op = 0x70cc,
ammind_op = 0x70cd,
ammaxwu_op = 0x70ce,
ammaxdu_op = 0x70cf,
amminwu_op = 0x70d0,
ammindu_op = 0x70d1,
amswapdbw_op = 0x70d2,
amswapdbd_op = 0x70d3,
amadddbw_op = 0x70d4,
amadddbd_op = 0x70d5,
amanddbw_op = 0x70d6,
amanddbd_op = 0x70d7,
amordbw_op = 0x70d8,
amordbd_op = 0x70d9,
amxordbw_op = 0x70da,
amxordbd_op = 0x70db,
ammaxdbw_op = 0x70dc,
ammaxdbd_op = 0x70dd,
ammindbw_op = 0x70de,
ammindbd_op = 0x70df,
ammaxdbwu_op = 0x70e0,
ammaxdbdu_op = 0x70e1,
ammindbwu_op = 0x70e2,
ammindbdu_op = 0x70e3,
LoongArch: Relay BCE exceptions to userland as SIGSEGV with si_code=SEGV_BNDERR SEGV_BNDERR was introduced initially for supporting the Intel MPX, but fell into disuse after the MPX support was removed. The LoongArch bounds-checking instructions behave very differently than MPX, but overall the interface is still kind of suitable for conveying the information to userland when bounds-checking assertions trigger, so we wouldn't have to invent more UAPI. Specifically, when the BCE triggers, a SEGV_BNDERR is sent to userland, with si_addr set to the out-of-bounds address or value (in asrt{gt,le}'s case), and one of si_lower or si_upper set to the configured bound depending on the faulting instruction. The other bound is set to either 0 or ULONG_MAX to resemble a range with both lower and upper bounds. Note that it is possible to have si_addr == si_lower in case of a failing asrtgt or {ld,st}gt, because those instructions test for strict greater-than relationship. This should not pose a problem for userland, though, because the faulting PC is available for the application to associate back to the exact instruction for figuring out the expectation. Example exception context generated by a faulting `asrtgt.d t0, t1` (assert t0 > t1 or BCE) with t0=100 and t1=200: > pc 00005555558206a4 ra 00007ffff2d854fc tp 00007ffff2f2f180 sp 00007ffffbf9fb80 > a0 0000000000000002 a1 00007ffffbf9fce8 a2 00007ffffbf9fd00 a3 00007ffff2ed4558 > a4 0000000000000000 a5 00007ffff2f044c8 a6 00007ffffbf9fce0 a7 fffffffffffff000 > t0 0000000000000064 t1 00000000000000c8 t2 00007ffffbfa2d5e t3 00007ffff2f12aa0 > t4 00007ffff2ed6158 t5 00007ffff2ed6158 t6 000000000000002e t7 0000000003d8f538 > t8 0000000000000005 u0 0000000000000000 s9 0000000000000000 s0 00007ffffbf9fce8 > s1 0000000000000002 s2 0000000000000000 s3 00007ffff2f2c038 s4 0000555555820610 > s5 00007ffff2ed5000 s6 0000555555827e38 s7 00007ffffbf9fd00 s8 0000555555827e38 > ra: 00007ffff2d854fc > ERA: 00005555558206a4 > CRMD: 000000b0 (PLV0 -IE -DA +PG DACF=CC DACM=CC -WE) > PRMD: 00000007 (PPLV3 +PIE -PWE) > EUEN: 00000000 (-FPE -SXE -ASXE -BTE) > ECFG: 0007181c (LIE=2-4,11-12 VS=7) > ESTAT: 000a0000 [BCE] (IS= ECode=10 EsubCode=0) > PRID: 0014c010 (Loongson-64bit, Loongson-3A5000) Signed-off-by: WANG Xuerui <git@xen0n.name> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-05-01 17:19:27 +08:00
fldgts_op = 0x70e8,
fldgtd_op = 0x70e9,
fldles_op = 0x70ea,
fldled_op = 0x70eb,
fstgts_op = 0x70ec,
fstgtd_op = 0x70ed,
fstles_op = 0x70ee,
fstled_op = 0x70ef,
ldgtb_op = 0x70f0,
ldgth_op = 0x70f1,
ldgtw_op = 0x70f2,
ldgtd_op = 0x70f3,
ldleb_op = 0x70f4,
ldleh_op = 0x70f5,
ldlew_op = 0x70f6,
ldled_op = 0x70f7,
stgtb_op = 0x70f8,
stgth_op = 0x70f9,
stgtw_op = 0x70fa,
stgtd_op = 0x70fb,
stleb_op = 0x70fc,
stleh_op = 0x70fd,
stlew_op = 0x70fe,
stled_op = 0x70ff,
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
};
enum reg3sa2_op {
alslw_op = 0x02,
alslwu_op = 0x03,
alsld_op = 0x16,
};
LoongArch: Add kprobes support Kprobes allows you to trap at almost any kernel address and execute a callback function, this commit adds kprobes support for LoongArch. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
struct reg0i15_format {
unsigned int immediate : 15;
unsigned int opcode : 17;
};
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
struct reg0i26_format {
unsigned int immediate_h : 10;
unsigned int immediate_l : 16;
unsigned int opcode : 6;
};
struct reg1i20_format {
unsigned int rd : 5;
unsigned int immediate : 20;
unsigned int opcode : 7;
};
struct reg1i21_format {
unsigned int immediate_h : 5;
unsigned int rj : 5;
unsigned int immediate_l : 16;
unsigned int opcode : 6;
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
struct reg2_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int opcode : 22;
};
struct reg2i5_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int immediate : 5;
unsigned int opcode : 17;
};
struct reg2i6_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int immediate : 6;
unsigned int opcode : 16;
};
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
struct reg2i12_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int immediate : 12;
unsigned int opcode : 10;
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
struct reg2i14_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int immediate : 14;
unsigned int opcode : 8;
};
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
struct reg2i16_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int immediate : 16;
unsigned int opcode : 6;
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
struct reg2bstrd_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int lsbd : 6;
unsigned int msbd : 6;
unsigned int opcode : 10;
};
LoongArch: KVM: Implement handle iocsr exception Implement kvm handle vcpu iocsr exception, setting the iocsr info into vcpu_run and return to user space to handle it. Reviewed-by: Bibo Mao <maobibo@loongson.cn> Tested-by: Huacai Chen <chenhuacai@loongson.cn> Signed-off-by: Tianrui Zhao <zhaotianrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-10-02 10:01:28 +08:00
struct reg2csr_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int csr : 14;
unsigned int opcode : 8;
};
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
struct reg3_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int rk : 5;
unsigned int opcode : 17;
};
struct reg3sa2_format {
unsigned int rd : 5;
unsigned int rj : 5;
unsigned int rk : 5;
unsigned int immediate : 2;
unsigned int opcode : 15;
};
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
union loongarch_instruction {
unsigned int word;
LoongArch: Add kprobes support Kprobes allows you to trap at almost any kernel address and execute a callback function, this commit adds kprobes support for LoongArch. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
struct reg0i15_format reg0i15_format;
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
struct reg0i26_format reg0i26_format;
struct reg1i20_format reg1i20_format;
struct reg1i21_format reg1i21_format;
struct reg2_format reg2_format;
struct reg2i5_format reg2i5_format;
struct reg2i6_format reg2i6_format;
struct reg2i12_format reg2i12_format;
struct reg2i14_format reg2i14_format;
struct reg2i16_format reg2i16_format;
struct reg2bstrd_format reg2bstrd_format;
LoongArch: KVM: Implement handle iocsr exception Implement kvm handle vcpu iocsr exception, setting the iocsr info into vcpu_run and return to user space to handle it. Reviewed-by: Bibo Mao <maobibo@loongson.cn> Tested-by: Huacai Chen <chenhuacai@loongson.cn> Signed-off-by: Tianrui Zhao <zhaotianrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-10-02 10:01:28 +08:00
struct reg2csr_format reg2csr_format;
LoongArch: Add some instruction opcodes and formats According to the "Table of Instruction Encoding" in LoongArch Reference Manual [1], add some instruction opcodes and formats which are used in the BPF JIT for LoongArch. [1] https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#table-of-instruction-encoding Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
struct reg3_format reg3_format;
struct reg3sa2_format reg3sa2_format;
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
};
#define LOONGARCH_INSN_SIZE sizeof(union loongarch_instruction)
enum loongarch_gpr {
LOONGARCH_GPR_ZERO = 0,
LOONGARCH_GPR_RA = 1,
LOONGARCH_GPR_TP = 2,
LOONGARCH_GPR_SP = 3,
LOONGARCH_GPR_A0 = 4, /* Reused as V0 for return value */
LOONGARCH_GPR_A1, /* Reused as V1 for return value */
LOONGARCH_GPR_A2,
LOONGARCH_GPR_A3,
LOONGARCH_GPR_A4,
LOONGARCH_GPR_A5,
LOONGARCH_GPR_A6,
LOONGARCH_GPR_A7,
LOONGARCH_GPR_T0 = 12,
LOONGARCH_GPR_T1,
LOONGARCH_GPR_T2,
LOONGARCH_GPR_T3,
LOONGARCH_GPR_T4,
LOONGARCH_GPR_T5,
LOONGARCH_GPR_T6,
LOONGARCH_GPR_T7,
LOONGARCH_GPR_T8,
LOONGARCH_GPR_FP = 22,
LOONGARCH_GPR_S0 = 23,
LOONGARCH_GPR_S1,
LOONGARCH_GPR_S2,
LOONGARCH_GPR_S3,
LOONGARCH_GPR_S4,
LOONGARCH_GPR_S5,
LOONGARCH_GPR_S6,
LOONGARCH_GPR_S7,
LOONGARCH_GPR_S8,
LOONGARCH_GPR_MAX
};
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
#define is_imm12_negative(val) is_imm_negative(val, 12)
static inline bool is_imm_negative(unsigned long val, unsigned int bit)
{
return val & (1UL << (bit - 1));
}
LoongArch: Add kprobes support Kprobes allows you to trap at almost any kernel address and execute a callback function, this commit adds kprobes support for LoongArch. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
static inline bool is_break_ins(union loongarch_instruction *ip)
{
return ip->reg0i15_format.opcode == break_op;
}
LoongArch: Add alternative runtime patching mechanism Introduce the "alternative" mechanism from ARM64 and x86 for LoongArch to apply runtime patching. The main purpose of this patch is to provide a framework. In future we can use this mechanism (i.e., the ALTERNATIVE and ALTERNATIVE_2 macros) to optimize hotspot functions according to cpu features. Signed-off-by: Jun Yi <yijun@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
static inline bool is_pc_ins(union loongarch_instruction *ip)
{
return ip->reg1i20_format.opcode >= pcaddi_op &&
ip->reg1i20_format.opcode <= pcaddu18i_op;
}
LoongArch: Add prologue unwinder support It unwind the stack frame based on prologue code analyze. CONFIG_KALLSYMS is needed, at least the address and length of each function. Three stages when we do unwind, 1) unwind_start(), the prapare of unwinding, fill unwind_state. 2) unwind_done(), judge whether the unwind process is finished or not. 3) unwind_next_frame(), unwind the next frame. Dividing unwinder helps to add new unwinders in the future, e.g.: unwinder_frame, unwinder_orc, .etc. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-08-06 16:10:03 +08:00
static inline bool is_branch_ins(union loongarch_instruction *ip)
{
return ip->reg1i21_format.opcode >= beqz_op &&
ip->reg1i21_format.opcode <= bgeu_op;
}
static inline bool is_ra_save_ins(union loongarch_instruction *ip)
{
/* st.d $ra, $sp, offset */
return ip->reg2i12_format.opcode == std_op &&
ip->reg2i12_format.rj == LOONGARCH_GPR_SP &&
ip->reg2i12_format.rd == LOONGARCH_GPR_RA &&
!is_imm12_negative(ip->reg2i12_format.immediate);
}
static inline bool is_stack_alloc_ins(union loongarch_instruction *ip)
{
/* addi.d $sp, $sp, -imm */
return ip->reg2i12_format.opcode == addid_op &&
ip->reg2i12_format.rj == LOONGARCH_GPR_SP &&
ip->reg2i12_format.rd == LOONGARCH_GPR_SP &&
is_imm12_negative(ip->reg2i12_format.immediate);
}
LoongArch: ptrace: Add hardware single step support Use the generic ptrace_resume code for PTRACE_SYSCALL, PTRACE_CONT, PTRACE_KILL and PTRACE_SINGLESTEP handling. This implies defining arch_has_single_step() and implementing the user_enable_single_step() and user_disable_single_step() functions. LoongArch cannot do hardware single-stepping per se, the hardware single-stepping it is achieved by configuring the instruction fetch watchpoints (FWPS) and specifies that the next instruction must trigger the watch exception by setting the mask bit. In some scenarios CSR.FWPS.Skip is used to ignore the next hit result, avoid endless repeated triggering of the same watchpoint without canceling it. Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
static inline bool is_self_loop_ins(union loongarch_instruction *ip, struct pt_regs *regs)
{
switch (ip->reg0i26_format.opcode) {
case b_op:
case bl_op:
if (ip->reg0i26_format.immediate_l == 0
&& ip->reg0i26_format.immediate_h == 0)
return true;
}
switch (ip->reg1i21_format.opcode) {
case beqz_op:
case bnez_op:
case bceqz_op:
if (ip->reg1i21_format.immediate_l == 0
&& ip->reg1i21_format.immediate_h == 0)
return true;
}
switch (ip->reg2i16_format.opcode) {
case beq_op:
case bne_op:
case blt_op:
case bge_op:
case bltu_op:
case bgeu_op:
if (ip->reg2i16_format.immediate == 0)
return true;
break;
case jirl_op:
if (regs->regs[ip->reg2i16_format.rj] +
((unsigned long)ip->reg2i16_format.immediate << 2) == (unsigned long)ip)
return true;
}
return false;
}
LoongArch: Simulate branch and PC* instructions According to LoongArch Reference Manual, simulate branch and PC* instructions, this is preparation for later patch. Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#branch-instructions Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#_pcaddi_pcaddu121_pcaddu18l_pcalau12i Tested-by: Jeff Xie <xiehuan09@gmail.com> Co-developed-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-02-25 15:52:57 +08:00
void simu_pc(struct pt_regs *regs, union loongarch_instruction insn);
void simu_branch(struct pt_regs *regs, union loongarch_instruction insn);
LoongArch: Move three functions from kprobes.c to inst.c The three functions insns_not_supported(), insns_need_simulation() and arch_simulate_insn() will be used for uprobes, move them from kprobes.c to inst.c, this is preparation for later patch, no functionality change. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:44 +08:00
bool insns_not_supported(union loongarch_instruction insn);
bool insns_need_simulation(union loongarch_instruction insn);
void arch_simulate_insn(union loongarch_instruction insn, struct pt_regs *regs);
LoongArch/ftrace: Add dynamic function tracer support The compiler has inserted 2 NOPs before the regular function prologue. T series registers are available and safe because of LoongArch's psABI. At runtime, we can replace nop with bl to enable ftrace call and replace bl with nop to disable ftrace call. The bl instruction requires us to save the original RA value, so it saves RA at t0 here. Details are: | Compiled | Disabled | Enabled | +------------+------------------------+------------------------+ | nop | move t0, ra | move t0, ra | | nop | nop | bl ftrace_caller | | func_body | func_body | func_body | The RA value will be recovered by ftrace_regs_entry, and restored into RA before returning to the regular function prologue. When a function is not being traced, the "move t0, ra" is not harmful. 1) ftrace_make_call, ftrace_make_nop (in kernel/ftrace.c) The two functions turn each recorded call site of filtered functions into a call to ftrace_caller or nops. 2) ftracce_update_ftrace_func (in kernel/ftrace.c) turns the nops at ftrace_call into a call to a generic entry for function tracers. 3) ftrace_caller (in kernel/mcount_dyn.S) The entry where each _mcount call sites calls to once they are filtered to be traced. Co-developed-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:40:15 +08:00
int larch_insn_read(void *addr, u32 *insnp);
int larch_insn_write(void *addr, u32 insn);
int larch_insn_patch_text(void *addr, u32 insn);
u32 larch_insn_gen_nop(void);
u32 larch_insn_gen_b(unsigned long pc, unsigned long dest);
u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest);
LoongArch: Add larch_insn_gen_break() to generate break insns There exist various break insns such as BRK_KPROBE_BP, BRK_KPROBE_SSTEPBP, BRK_UPROBE_BP and BRK_UPROBE_XOLBP, add larch_insn_gen_break() to generate break insns simpler, this is preparation for later patch. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:44 +08:00
u32 larch_insn_gen_break(int imm);
LoongArch/ftrace: Add dynamic function tracer support The compiler has inserted 2 NOPs before the regular function prologue. T series registers are available and safe because of LoongArch's psABI. At runtime, we can replace nop with bl to enable ftrace call and replace bl with nop to disable ftrace call. The bl instruction requires us to save the original RA value, so it saves RA at t0 here. Details are: | Compiled | Disabled | Enabled | +------------+------------------------+------------------------+ | nop | move t0, ra | move t0, ra | | nop | nop | bl ftrace_caller | | func_body | func_body | func_body | The RA value will be recovered by ftrace_regs_entry, and restored into RA before returning to the regular function prologue. When a function is not being traced, the "move t0, ra" is not harmful. 1) ftrace_make_call, ftrace_make_nop (in kernel/ftrace.c) The two functions turn each recorded call site of filtered functions into a call to ftrace_caller or nops. 2) ftracce_update_ftrace_func (in kernel/ftrace.c) turns the nops at ftrace_call into a call to a generic entry for function tracers. 3) ftrace_caller (in kernel/mcount_dyn.S) The entry where each _mcount call sites calls to once they are filtered to be traced. Co-developed-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Jinyang He <hejinyang@loongson.cn> Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:40:15 +08:00
u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongarch_gpr rk);
u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj);
LoongArch: modules/ftrace: Initialize PLT at load time This patch implements ftrace trampolines through plt entry. Tested by forcing ftrace_make_call() to use the module PLT, and then loading up a module after setting up ftrace with: | echo ":mod:<module-name>" > set_ftrace_filter; | echo function > current_tracer; | modprobe <module-name> Since FTRACE_ADDR/FTRACE_REGS_ADDR is only defined when CONFIG_DYNAMIC_ FTRACE is selected, we wrap their usage in module_init_ftrace_plt() with ifdeffery rather than using IS_ENABLED(). Signed-off-by: Qing Zhang <zhangqing@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:40:21 +08:00
u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int imm);
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
LoongArch: Add guard for the larch_insn_gen_xxx functions Add guard for the larch_insn_gen_xxx functions to verify whether the immediate operand is within the acceptable range. Signed-off-by: WANG Rui <wangrui@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:42 +08:00
u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
LoongArch: Move {signed,unsigned}_imm_check() to inst.h {signed,unsigned}_imm_check() will also be used in the bpf jit, so move them from module.c to inst.h, this is preparation for later patches. Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:19 +08:00
static inline bool signed_imm_check(long val, unsigned int bit)
{
return -(1L << (bit - 1)) <= val && val < (1L << (bit - 1));
}
static inline bool unsigned_imm_check(unsigned long val, unsigned int bit)
{
return val < (1UL << bit);
}
LoongArch: Add larch_insn_gen_break() to generate break insns There exist various break insns such as BRK_KPROBE_BP, BRK_KPROBE_SSTEPBP, BRK_UPROBE_BP and BRK_UPROBE_XOLBP, add larch_insn_gen_break() to generate break insns simpler, this is preparation for later patch. Tested-by: Jeff Xie <xiehuan09@gmail.com> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-06-29 20:58:44 +08:00
#define DEF_EMIT_REG0I15_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
int imm) \
{ \
insn->reg0i15_format.opcode = OP; \
insn->reg0i15_format.immediate = imm; \
}
DEF_EMIT_REG0I15_FORMAT(break, break_op)
LoongArch: Make the users of larch_insn_gen_break() constant LoongArch defines UPROBE_SWBP_INSN as a function call and this breaks arch_uprobe_trampoline() which uses it to initialize a static variable. Add the new "__builtin_constant_p" helper, __emit_break(), and redefine the current users of larch_insn_gen_break() to use it. Fixes: ff474a78cef5 ("uprobe: Add uretprobe syscall to speed up return probe") Reported-by: Nathan Chancellor <nathan@kernel.org> Closes: https://lore.kernel.org/all/20240614174822.GA1185149@thelio-3990X/ Suggested-by: Andrii Nakryiko <andrii@kernel.org> Tested-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Oleg Nesterov <oleg@redhat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2024-07-20 22:41:07 +08:00
/* like emit_break(imm) but returns a constant expression */
#define __emit_break(imm) ((u32)((imm) | (break_op << 15)))
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
#define DEF_EMIT_REG0I26_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
int offset) \
{ \
unsigned int immediate_l, immediate_h; \
\
immediate_l = offset & 0xffff; \
offset >>= 16; \
immediate_h = offset & 0x3ff; \
\
insn->reg0i26_format.opcode = OP; \
insn->reg0i26_format.immediate_l = immediate_l; \
insn->reg0i26_format.immediate_h = immediate_h; \
}
DEF_EMIT_REG0I26_FORMAT(b, b_op)
LoongArch: Simplify larch_insn_gen_xxx implementation Simplify larch_insn_gen_xxx implementation by reusing emit_xxx. Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-01-17 11:42:16 +08:00
DEF_EMIT_REG0I26_FORMAT(bl, bl_op)
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
#define DEF_EMIT_REG1I20_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, int imm) \
{ \
insn->reg1i20_format.opcode = OP; \
insn->reg1i20_format.immediate = imm; \
insn->reg1i20_format.rd = rd; \
}
DEF_EMIT_REG1I20_FORMAT(lu12iw, lu12iw_op)
DEF_EMIT_REG1I20_FORMAT(lu32id, lu32id_op)
DEF_EMIT_REG1I20_FORMAT(pcaddu18i, pcaddu18i_op)
#define DEF_EMIT_REG2_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj) \
{ \
insn->reg2_format.opcode = OP; \
insn->reg2_format.rd = rd; \
insn->reg2_format.rj = rj; \
}
DEF_EMIT_REG2_FORMAT(revb2h, revb2h_op)
DEF_EMIT_REG2_FORMAT(revb2w, revb2w_op)
DEF_EMIT_REG2_FORMAT(revbd, revbd_op)
LoongArch: Add more instruction opcodes and emit_* helpers This patch adds more instruction opcodes and their corresponding emit_* helpers which will be used in later patches. Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-11-08 14:12:15 +08:00
DEF_EMIT_REG2_FORMAT(extwh, extwh_op)
DEF_EMIT_REG2_FORMAT(extwb, extwb_op)
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
#define DEF_EMIT_REG2I5_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
int imm) \
{ \
insn->reg2i5_format.opcode = OP; \
insn->reg2i5_format.immediate = imm; \
insn->reg2i5_format.rd = rd; \
insn->reg2i5_format.rj = rj; \
}
DEF_EMIT_REG2I5_FORMAT(slliw, slliw_op)
DEF_EMIT_REG2I5_FORMAT(srliw, srliw_op)
DEF_EMIT_REG2I5_FORMAT(sraiw, sraiw_op)
#define DEF_EMIT_REG2I6_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
int imm) \
{ \
insn->reg2i6_format.opcode = OP; \
insn->reg2i6_format.immediate = imm; \
insn->reg2i6_format.rd = rd; \
insn->reg2i6_format.rj = rj; \
}
DEF_EMIT_REG2I6_FORMAT(sllid, sllid_op)
DEF_EMIT_REG2I6_FORMAT(srlid, srlid_op)
DEF_EMIT_REG2I6_FORMAT(sraid, sraid_op)
#define DEF_EMIT_REG2I12_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
int imm) \
{ \
insn->reg2i12_format.opcode = OP; \
insn->reg2i12_format.immediate = imm; \
insn->reg2i12_format.rd = rd; \
insn->reg2i12_format.rj = rj; \
}
DEF_EMIT_REG2I12_FORMAT(addiw, addiw_op)
DEF_EMIT_REG2I12_FORMAT(addid, addid_op)
DEF_EMIT_REG2I12_FORMAT(lu52id, lu52id_op)
DEF_EMIT_REG2I12_FORMAT(andi, andi_op)
DEF_EMIT_REG2I12_FORMAT(ori, ori_op)
DEF_EMIT_REG2I12_FORMAT(xori, xori_op)
LoongArch: Add more instruction opcodes and emit_* helpers This patch adds more instruction opcodes and their corresponding emit_* helpers which will be used in later patches. Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-11-08 14:12:15 +08:00
DEF_EMIT_REG2I12_FORMAT(ldb, ldb_op)
DEF_EMIT_REG2I12_FORMAT(ldh, ldh_op)
DEF_EMIT_REG2I12_FORMAT(ldw, ldw_op)
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
DEF_EMIT_REG2I12_FORMAT(ldbu, ldbu_op)
DEF_EMIT_REG2I12_FORMAT(ldhu, ldhu_op)
DEF_EMIT_REG2I12_FORMAT(ldwu, ldwu_op)
DEF_EMIT_REG2I12_FORMAT(ldd, ldd_op)
DEF_EMIT_REG2I12_FORMAT(stb, stb_op)
DEF_EMIT_REG2I12_FORMAT(sth, sth_op)
DEF_EMIT_REG2I12_FORMAT(stw, stw_op)
DEF_EMIT_REG2I12_FORMAT(std, std_op)
#define DEF_EMIT_REG2I14_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
int imm) \
{ \
insn->reg2i14_format.opcode = OP; \
insn->reg2i14_format.immediate = imm; \
insn->reg2i14_format.rd = rd; \
insn->reg2i14_format.rj = rj; \
}
DEF_EMIT_REG2I14_FORMAT(llw, llw_op)
DEF_EMIT_REG2I14_FORMAT(scw, scw_op)
DEF_EMIT_REG2I14_FORMAT(lld, lld_op)
DEF_EMIT_REG2I14_FORMAT(scd, scd_op)
DEF_EMIT_REG2I14_FORMAT(ldptrw, ldptrw_op)
DEF_EMIT_REG2I14_FORMAT(stptrw, stptrw_op)
DEF_EMIT_REG2I14_FORMAT(ldptrd, ldptrd_op)
DEF_EMIT_REG2I14_FORMAT(stptrd, stptrd_op)
#define DEF_EMIT_REG2I16_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rj, \
enum loongarch_gpr rd, \
int offset) \
{ \
insn->reg2i16_format.opcode = OP; \
insn->reg2i16_format.immediate = offset; \
insn->reg2i16_format.rj = rj; \
insn->reg2i16_format.rd = rd; \
}
DEF_EMIT_REG2I16_FORMAT(beq, beq_op)
DEF_EMIT_REG2I16_FORMAT(bne, bne_op)
DEF_EMIT_REG2I16_FORMAT(blt, blt_op)
DEF_EMIT_REG2I16_FORMAT(bge, bge_op)
DEF_EMIT_REG2I16_FORMAT(bltu, bltu_op)
DEF_EMIT_REG2I16_FORMAT(bgeu, bgeu_op)
LoongArch: BPF: Adjust the parameter of emit_jirl() The branch instructions beq, bne, blt, bge, bltu, bgeu and jirl belong to the format reg2i16, but the sequence of oprand is different for the instruction jirl. So adjust the parameter order of emit_jirl() to make it more readable correspond with the Instruction Set Architecture manual. Here are the instruction formats: beq rj, rd, offs16 bne rj, rd, offs16 blt rj, rd, offs16 bge rj, rd, offs16 bltu rj, rd, offs16 bgeu rj, rd, offs16 jirl rd, rj, offs16 Link: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#branch-instructions Suggested-by: Huacai Chen <chenhuacai@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2024-12-02 16:42:08 +08:00
static inline void emit_jirl(union loongarch_instruction *insn,
enum loongarch_gpr rd,
enum loongarch_gpr rj,
int offset)
{
insn->reg2i16_format.opcode = jirl_op;
insn->reg2i16_format.immediate = offset;
insn->reg2i16_format.rd = rd;
insn->reg2i16_format.rj = rj;
}
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
#define DEF_EMIT_REG2BSTRD_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
int msbd, \
int lsbd) \
{ \
insn->reg2bstrd_format.opcode = OP; \
insn->reg2bstrd_format.msbd = msbd; \
insn->reg2bstrd_format.lsbd = lsbd; \
insn->reg2bstrd_format.rj = rj; \
insn->reg2bstrd_format.rd = rd; \
}
DEF_EMIT_REG2BSTRD_FORMAT(bstrpickd, bstrpickd_op)
#define DEF_EMIT_REG3_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
enum loongarch_gpr rk) \
{ \
insn->reg3_format.opcode = OP; \
insn->reg3_format.rd = rd; \
insn->reg3_format.rj = rj; \
insn->reg3_format.rk = rk; \
}
LoongArch: Add more instruction opcodes and emit_* helpers This patch adds more instruction opcodes and their corresponding emit_* helpers which will be used in later patches. Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-11-08 14:12:15 +08:00
DEF_EMIT_REG3_FORMAT(addw, addw_op)
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
DEF_EMIT_REG3_FORMAT(addd, addd_op)
DEF_EMIT_REG3_FORMAT(subd, subd_op)
DEF_EMIT_REG3_FORMAT(muld, muld_op)
LoongArch: Add more instruction opcodes and emit_* helpers This patch adds more instruction opcodes and their corresponding emit_* helpers which will be used in later patches. Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-11-08 14:12:15 +08:00
DEF_EMIT_REG3_FORMAT(divd, divd_op)
DEF_EMIT_REG3_FORMAT(modd, modd_op)
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
DEF_EMIT_REG3_FORMAT(divdu, divdu_op)
DEF_EMIT_REG3_FORMAT(moddu, moddu_op)
DEF_EMIT_REG3_FORMAT(and, and_op)
DEF_EMIT_REG3_FORMAT(or, or_op)
DEF_EMIT_REG3_FORMAT(xor, xor_op)
DEF_EMIT_REG3_FORMAT(sllw, sllw_op)
DEF_EMIT_REG3_FORMAT(slld, slld_op)
DEF_EMIT_REG3_FORMAT(srlw, srlw_op)
DEF_EMIT_REG3_FORMAT(srld, srld_op)
DEF_EMIT_REG3_FORMAT(sraw, sraw_op)
DEF_EMIT_REG3_FORMAT(srad, srad_op)
LoongArch: Add more instruction opcodes and emit_* helpers This patch adds more instruction opcodes and their corresponding emit_* helpers which will be used in later patches. Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2023-11-08 14:12:15 +08:00
DEF_EMIT_REG3_FORMAT(ldxb, ldxb_op)
DEF_EMIT_REG3_FORMAT(ldxh, ldxh_op)
DEF_EMIT_REG3_FORMAT(ldxw, ldxw_op)
LoongArch: Add BPF JIT support BPF programs are normally handled by a BPF interpreter, add BPF JIT support for LoongArch to allow the kernel to generate native code when a program is loaded into the kernel. This will significantly speed-up processing of BPF programs. Co-developed-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Youling Tang <tangyouling@loongson.cn> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-10-12 16:36:20 +08:00
DEF_EMIT_REG3_FORMAT(ldxbu, ldxbu_op)
DEF_EMIT_REG3_FORMAT(ldxhu, ldxhu_op)
DEF_EMIT_REG3_FORMAT(ldxwu, ldxwu_op)
DEF_EMIT_REG3_FORMAT(ldxd, ldxd_op)
DEF_EMIT_REG3_FORMAT(stxb, stxb_op)
DEF_EMIT_REG3_FORMAT(stxh, stxh_op)
DEF_EMIT_REG3_FORMAT(stxw, stxw_op)
DEF_EMIT_REG3_FORMAT(stxd, stxd_op)
DEF_EMIT_REG3_FORMAT(amaddw, amaddw_op)
DEF_EMIT_REG3_FORMAT(amaddd, amaddd_op)
DEF_EMIT_REG3_FORMAT(amandw, amandw_op)
DEF_EMIT_REG3_FORMAT(amandd, amandd_op)
DEF_EMIT_REG3_FORMAT(amorw, amorw_op)
DEF_EMIT_REG3_FORMAT(amord, amord_op)
DEF_EMIT_REG3_FORMAT(amxorw, amxorw_op)
DEF_EMIT_REG3_FORMAT(amxord, amxord_op)
DEF_EMIT_REG3_FORMAT(amswapw, amswapw_op)
DEF_EMIT_REG3_FORMAT(amswapd, amswapd_op)
#define DEF_EMIT_REG3SA2_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
enum loongarch_gpr rd, \
enum loongarch_gpr rj, \
enum loongarch_gpr rk, \
int imm) \
{ \
insn->reg3sa2_format.opcode = OP; \
insn->reg3sa2_format.immediate = imm; \
insn->reg3sa2_format.rd = rd; \
insn->reg3sa2_format.rj = rj; \
insn->reg3sa2_format.rk = rk; \
}
DEF_EMIT_REG3SA2_FORMAT(alsld, alsld_op)
LoongArch: Add unaligned access support Loongson-2 series (Loongson-2K500, Loongson-2K1000) don't support unaligned access in hardware, while Loongson-3 series (Loongson-3A5000, Loongson-3C5000) are configurable whether support unaligned access in hardware. This patch add unaligned access emulation for those LoongArch processors without hardware support. Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-12-10 22:39:59 +08:00
struct pt_regs;
void emulate_load_store_insn(struct pt_regs *regs, void __user *addr, unsigned int *pc);
unsigned long unaligned_read(void __user *addr, void *value, unsigned long n, bool sign);
unsigned long unaligned_write(void __user *addr, unsigned long value, unsigned long n);
LoongArch: Add other common headers Add some other common headers for basic LoongArch support. Reviewed-by: WANG Xuerui <git@xen0n.name> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
2022-05-31 18:04:11 +08:00
#endif /* _ASM_INST_H */
Reference in a new issue Copy permalink