public-mirrors/linux
1
0
Fork 0
mirror of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2025-08-20 06:20:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
linux/arch/powerpc/platforms/pseries/hvCall.S

371 lines
6.8 KiB
ArmAsm
Raw Permalink Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation either version 2 of the license or at your option any later version extracted by the scancode license scanner the SPDX license identifier GPL-2.0-or-later has been chosen to replace the boilerplate/reference in 3029 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-27 08:55:01 +02:00
/* SPDX-License-Identifier: GPL-2.0-or-later */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
/*
* This file contains the generic code to perform a call to the
* pSeries LPAR hypervisor.
*/
powerpc, jump_label: Include linux/jump_label.h to get HAVE_JUMP_LABEL define Commit 1bc9e47aa8e4 ("powerpc/jump_label: Use HAVE_JUMP_LABEL") converted uses of CONFIG_JUMP_LABEL to HAVE_JUMP_LABEL in some assembly files. HAVE_JUMP_LABEL is defined in linux/jump_label.h, so we need to include this or we always get the non jump label fallback code. Signed-off-by: Anton Blanchard <anton@samba.org> Acked-by: Michael Ellerman <mpe@ellerman.id.au> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: benh@kernel.crashing.org Cc: catalin.marinas@arm.com Cc: davem@davemloft.net Cc: heiko.carstens@de.ibm.com Cc: jbaron@akamai.com Cc: linux@arm.linux.org.uk Cc: linuxppc-dev@lists.ozlabs.org Cc: liuj97@gmail.com Cc: mgorman@suse.de Cc: mmarek@suse.cz Cc: paulus@samba.org Cc: ralf@linux-mips.org Cc: rostedt@goodmis.org Cc: schwidefsky@de.ibm.com Cc: will.deacon@arm.com Fixes: 1bc9e47aa8e4 ("powerpc/jump_label: Use HAVE_JUMP_LABEL") Link: http://lkml.kernel.org/r/1428551492-21977-3-git-send-email-anton@samba.org Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-04-09 13:51:32 +10:00
#include <linux/jump_label.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
#include <asm/hvcall.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
#include <asm/asm-offsets.h>
powerpc: Remove second definition of STACK_FRAME_OVERHEAD Since STACK_FRAME_OVERHEAD is defined in asm/ptrace.h and that is ASSEMBER safe, we can just include that instead of going via asm-offsets.h. Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2010-11-18 15:06:17 +00:00
#include <asm/ptrace.h>
powerpc: clean inclusions of asm/feature-fixups.h files not using feature fixup don't need asm/feature-fixups.h files using feature fixup need asm/feature-fixups.h Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-07-05 16:25:01 +00:00
#include <asm/feature-fixups.h>
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
.section ".text"
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
#ifdef CONFIG_TRACEPOINTS
jump_label: move 'asm goto' support test to Kconfig Currently, CONFIG_JUMP_LABEL just means "I _want_ to use jump label". The jump label is controlled by HAVE_JUMP_LABEL, which is defined like this: #if defined(CC_HAVE_ASM_GOTO) && defined(CONFIG_JUMP_LABEL) # define HAVE_JUMP_LABEL #endif We can improve this by testing 'asm goto' support in Kconfig, then make JUMP_LABEL depend on CC_HAS_ASM_GOTO. Ugly #ifdef HAVE_JUMP_LABEL will go away, and CONFIG_JUMP_LABEL will match to the real kernel capability. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
2018-12-31 00:14:15 +09:00
#ifndef CONFIG_JUMP_LABEL
powerpc/pseries: move hcall_tracepoint_refcount out of .toc The .toc section is not really intended for arbitrary data. Writable data in particular prevents making the TOC read-only after relocation. Move hcall_tracepoint_refcount into the .data section. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20220926053823.2668799-1-npiggin@gmail.com
2022-09-26 15:38:23 +10:00
.data
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
.globl hcall_tracepoint_refcount
hcall_tracepoint_refcount:
powerpc/asm: Convert .llong directives to .8byte .llong is an undocumented PPC specific directive. The generic equivalent is .quad, but even better (because it's self describing) is .8byte. Convert all .llong directives to .8byte. Signed-off-by: Tobin C. Harding <me@tobin.cc> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-03-09 16:42:12 +11:00
.8byte 0
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
.section ".text"
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
#endif
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
/*
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
* precall must preserve all registers. use unused STK_PARAM()
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
* areas to save snapshots and opcode. STK_PARAM() in the caller's
* frame will be available even on ELFv2 because these are all
* variadic functions.
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
*/
powerpc: tracing: Give hypervisor call tracepoints access to arguments While most users of the hcall tracepoints will only want the opcode and return code, some will want all the arguments. To avoid the complexity of using varargs we pass a pointer to the register save area, which contains all the arguments. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:51:09 +00:00
#define HCALL_INST_PRECALL(FIRST_REG) \
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
mflr r0; \
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
std r3,STK_PARAM(R3)(r1); \
std r4,STK_PARAM(R4)(r1); \
std r5,STK_PARAM(R5)(r1); \
std r6,STK_PARAM(R6)(r1); \
std r7,STK_PARAM(R7)(r1); \
std r8,STK_PARAM(R8)(r1); \
std r9,STK_PARAM(R9)(r1); \
std r10,STK_PARAM(R10)(r1); \
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
std r0,16(r1); \
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
addi r4,r1,STK_PARAM(FIRST_REG); \
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
stdu r1,-STACK_FRAME_MIN_SIZE(r1); \
powerpc: add CFUNC assembly label annotation This macro is to be used in assembly where C functions are called. pcrel addressing mode requires branches to functions with a localentry value of 1 to have either a trailing nop or @notoc. This macro permits the latter without changing callers. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Add dummy definitions to fix selftests build] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://msgid.link/20230408021752.862660-5-npiggin@gmail.com
2023-04-08 12:17:50 +10:00
bl CFUNC(__trace_hcall_entry); \
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
ld r3,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \
ld r4,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1); \
ld r5,STACK_FRAME_MIN_SIZE+STK_PARAM(R5)(r1); \
ld r6,STACK_FRAME_MIN_SIZE+STK_PARAM(R6)(r1); \
ld r7,STACK_FRAME_MIN_SIZE+STK_PARAM(R7)(r1); \
ld r8,STACK_FRAME_MIN_SIZE+STK_PARAM(R8)(r1); \
ld r9,STACK_FRAME_MIN_SIZE+STK_PARAM(R9)(r1); \
ld r10,STACK_FRAME_MIN_SIZE+STK_PARAM(R10)(r1)
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
/*
* postcall is performed immediately before function return which
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
* allows liberal use of volatile registers.
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
*/
powerpc: tracing: Give hypervisor call tracepoints access to arguments While most users of the hcall tracepoints will only want the opcode and return code, some will want all the arguments. To avoid the complexity of using varargs we pass a pointer to the register save area, which contains all the arguments. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:51:09 +00:00
#define __HCALL_INST_POSTCALL \
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
ld r0,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \
std r3,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \
powerpc: tracing: Add hypervisor call tracepoints Add hcall_entry and hcall_exit tracepoints. This replaces the inline assembly HCALL_STATS code and converts it to use the new tracepoints. To keep the disabled case as quick as possible, we embed a status word in the TOC so we can get at it with a single load. By doing so we keep the overhead at a minimum. Time taken for a null hcall: No tracepoint code: 135.79 cycles Disabled tracepoints: 137.95 cycles For reference, before this patch enabling HCALL_STATS resulted in a null hcall of 201.44 cycles! Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:50:29 +00:00
mr r4,r3; \
powerpc/pseries: Optimise hcall tracepoints Now that we execute the hcall tracepoint entry and exit code out of line, we can use the same stack across both functions. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:56 +10:00
mr r3,r0; \
powerpc: add CFUNC assembly label annotation This macro is to be used in assembly where C functions are called. pcrel addressing mode requires branches to functions with a localentry value of 1 to have either a trailing nop or @notoc. This macro permits the latter without changing callers. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Add dummy definitions to fix selftests build] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://msgid.link/20230408021752.862660-5-npiggin@gmail.com
2023-04-08 12:17:50 +10:00
bl CFUNC(__trace_hcall_exit); \
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
ld r0,STACK_FRAME_MIN_SIZE+16(r1); \
addi r1,r1,STACK_FRAME_MIN_SIZE; \
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r3,STK_PARAM(R3)(r1); \
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
mtlr r0
powerpc: tracing: Give hypervisor call tracepoints access to arguments While most users of the hcall tracepoints will only want the opcode and return code, some will want all the arguments. To avoid the complexity of using varargs we pass a pointer to the register save area, which contains all the arguments. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:51:09 +00:00
#define HCALL_INST_POSTCALL_NORETS \
li r5,0; \
__HCALL_INST_POSTCALL
#define HCALL_INST_POSTCALL(BUFREG) \
mr r5,BUFREG; \
__HCALL_INST_POSTCALL
jump_label: move 'asm goto' support test to Kconfig Currently, CONFIG_JUMP_LABEL just means "I _want_ to use jump label". The jump label is controlled by HAVE_JUMP_LABEL, which is defined like this: #if defined(CC_HAVE_ASM_GOTO) && defined(CONFIG_JUMP_LABEL) # define HAVE_JUMP_LABEL #endif We can improve this by testing 'asm goto' support in Kconfig, then make JUMP_LABEL depend on CC_HAS_ASM_GOTO. Ugly #ifdef HAVE_JUMP_LABEL will go away, and CONFIG_JUMP_LABEL will match to the real kernel capability. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
2018-12-31 00:14:15 +09:00
#ifdef CONFIG_JUMP_LABEL
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
#define HCALL_BRANCH(LABEL) \
ARCH_STATIC_BRANCH(LABEL, hcall_tracepoint_key)
#else
/*
* We branch around this in early init (eg when populating the MMU
* hashtable) by using an unconditional cpu feature.
*/
#define HCALL_BRANCH(LABEL) \
BEGIN_FTR_SECTION; \
b 1f; \
END_FTR_SECTION(0, 1); \
powerpc/pseries: move hcall_tracepoint_refcount out of .toc The .toc section is not really intended for arbitrary data. Writable data in particular prevents making the TOC read-only after relocation. Move hcall_tracepoint_refcount into the .data section. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20220926053823.2668799-1-npiggin@gmail.com
2022-09-26 15:38:23 +10:00
LOAD_REG_ADDR(r12, hcall_tracepoint_refcount) ; \
powerpc/pseries: Fix hcall tracepoints with JUMP_LABEL=n With JUMP_LABEL=n, hcall_tracepoint_refcount's address is being tested instead of its value. This results in the tracing slowpath always being taken unnecessarily. Fixes: 9a10ccb29c0a2 ("powerpc/pseries: move hcall_tracepoint_refcount out of .toc") Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://msgid.link/20230509091600.70994-1-npiggin@gmail.com
2023-05-09 19:15:59 +10:00
ld r12,0(r12); \
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
cmpdi r12,0; \
bne- LABEL; \
1:
#endif
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
#else
powerpc: tracing: Give hypervisor call tracepoints access to arguments While most users of the hcall tracepoints will only want the opcode and return code, some will want all the arguments. To avoid the complexity of using varargs we pass a pointer to the register save area, which contains all the arguments. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:51:09 +00:00
#define HCALL_INST_PRECALL(FIRST_ARG)
#define HCALL_INST_POSTCALL_NORETS
#define HCALL_INST_POSTCALL(BUFREG)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
#define HCALL_BRANCH(LABEL)
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
#endif
powerpc/pseries: Fix hcall tracing recursion in pv queued spinlocks The paravit queued spinlock slow path adds itself to the queue then calls pv_wait to wait for the lock to become free. This is implemented by calling H_CONFER to donate cycles. When hcall tracing is enabled, this H_CONFER call can lead to a spin lock being taken in the tracing code, which will result in the lock to be taken again, which will also go to the slow path because it queues behind itself and so won't ever make progress. An example trace of a deadlock: __pv_queued_spin_lock_slowpath trace_clock_global ring_buffer_lock_reserve trace_event_buffer_lock_reserve trace_event_buffer_reserve trace_event_raw_event_hcall_exit __trace_hcall_exit plpar_hcall_norets_trace __pv_queued_spin_lock_slowpath trace_clock_global ring_buffer_lock_reserve trace_event_buffer_lock_reserve trace_event_buffer_reserve trace_event_raw_event_rcu_dyntick rcu_irq_exit irq_exit __do_irq call_do_irq do_IRQ hardware_interrupt_common_virt Fix this by introducing plpar_hcall_norets_notrace(), and using that to make SPLPAR virtual processor dispatching hcalls by the paravirt spinlock code. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Reviewed-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210508101455.1578318-2-npiggin@gmail.com
2021-05-08 20:14:52 +10:00
_GLOBAL_TOC(plpar_hcall_norets_notrace)
HMT_MEDIUM
mfcr r0
stw r0,8(r1)
HVSC /* invoke the hypervisor */
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
powerpc/pseries: Fix hcall tracing recursion in pv queued spinlocks The paravit queued spinlock slow path adds itself to the queue then calls pv_wait to wait for the lock to become free. This is implemented by calling H_CONFER to donate cycles. When hcall tracing is enabled, this H_CONFER call can lead to a spin lock being taken in the tracing code, which will result in the lock to be taken again, which will also go to the slow path because it queues behind itself and so won't ever make progress. An example trace of a deadlock: __pv_queued_spin_lock_slowpath trace_clock_global ring_buffer_lock_reserve trace_event_buffer_lock_reserve trace_event_buffer_reserve trace_event_raw_event_hcall_exit __trace_hcall_exit plpar_hcall_norets_trace __pv_queued_spin_lock_slowpath trace_clock_global ring_buffer_lock_reserve trace_event_buffer_lock_reserve trace_event_buffer_reserve trace_event_raw_event_rcu_dyntick rcu_irq_exit irq_exit __do_irq call_do_irq do_IRQ hardware_interrupt_common_virt Fix this by introducing plpar_hcall_norets_notrace(), and using that to make SPLPAR virtual processor dispatching hcalls by the paravirt spinlock code. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Reviewed-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210508101455.1578318-2-npiggin@gmail.com
2021-05-08 20:14:52 +10:00
lwz r0,8(r1)
mtcrf 0xff,r0
blr /* return r3 = status */
powerpc/pseries: hcall functions are exported to modules, need _GLOBAL_TOC() The hcall macros may call out to c code for tracing, so we need to set up a valid r2. This fixes an oops found when testing ibmvscsi as a module. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-05-13 20:48:57 +10:00
_GLOBAL_TOC(plpar_hcall_norets)
[PATCH] ppc64: enforce medium thread priority in hypervisor calls Calls into the hypervisor do not raise the thread priority. Ensure we are running at medium priority upon entry to the hypervisor. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-01 08:58:46 -07:00
HMT_MEDIUM
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
mfcr r0
stw r0,8(r1)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
HCALL_BRANCH(plpar_hcall_norets_trace)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
HVSC /* invoke the hypervisor */
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
lwz r0,8(r1)
mtcrf 0xff,r0
blr /* return r3 = status */
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
#ifdef CONFIG_TRACEPOINTS
plpar_hcall_norets_trace:
HCALL_INST_PRECALL(R4)
HVSC
HCALL_INST_POSTCALL_NORETS
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
lwz r0,8(r1)
mtcrf 0xff,r0
blr
#endif
powerpc/pseries: hcall functions are exported to modules, need _GLOBAL_TOC() The hcall macros may call out to c code for tracing, so we need to set up a valid r2. This fixes an oops found when testing ibmvscsi as a module. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-05-13 20:48:57 +10:00
_GLOBAL_TOC(plpar_hcall)
[PATCH] ppc64: enforce medium thread priority in hypervisor calls Calls into the hypervisor do not raise the thread priority. Ensure we are running at medium priority upon entry to the hypervisor. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-01 08:58:46 -07:00
HMT_MEDIUM
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
mfcr r0
stw r0,8(r1)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
HCALL_BRANCH(plpar_hcall_trace)
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
std r4,STK_PARAM(R4)(r1) /* Save ret buffer */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
[POWERPC] clean up pseries hcall interfaces Our pseries hcall interfaces are out of control: plpar_hcall_norets plpar_hcall plpar_hcall_8arg_2ret plpar_hcall_4out plpar_hcall_7arg_7ret plpar_hcall_9arg_9ret Create 3 interfaces to cover all cases: plpar_hcall_norets: 7 arguments no returns plpar_hcall: 6 arguments 4 returns plpar_hcall9: 9 arguments 9 returns There are only 2 cases in the kernel that need plpar_hcall9, hopefully we can keep it that way. Pass in a buffer to stash return parameters so we avoid the &dummy1, &dummy2 madness. Signed-off-by: Anton Blanchard <anton@samba.org> -- Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-07-19 08:01:28 +10:00
mr r4,r5
mr r5,r6
mr r6,r7
mr r7,r8
mr r8,r9
mr r9,r10
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-16 15:20:36 -07:00
HVSC /* invoke the hypervisor */
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r12,STK_PARAM(R4)(r1)
[POWERPC] clean up pseries hcall interfaces Our pseries hcall interfaces are out of control: plpar_hcall_norets plpar_hcall plpar_hcall_8arg_2ret plpar_hcall_4out plpar_hcall_7arg_7ret plpar_hcall_9arg_9ret Create 3 interfaces to cover all cases: plpar_hcall_norets: 7 arguments no returns plpar_hcall: 6 arguments 4 returns plpar_hcall9: 9 arguments 9 returns There are only 2 cases in the kernel that need plpar_hcall9, hopefully we can keep it that way. Pass in a buffer to stash return parameters so we avoid the &dummy1, &dummy2 madness. Signed-off-by: Anton Blanchard <anton@samba.org> -- Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-07-19 08:01:28 +10:00
std r4, 0(r12)
std r5, 8(r12)
std r6, 16(r12)
std r7, 24(r12)
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
lwz r0,8(r1)
mtcrf 0xff,r0
blr /* return r3 = status */
#ifdef CONFIG_TRACEPOINTS
plpar_hcall_trace:
HCALL_INST_PRECALL(R5)
mr r4,r5
mr r5,r6
mr r6,r7
mr r7,r8
mr r8,r9
mr r9,r10
HVSC
powerpc/pseries: Fix STK_PARAM access in the hcall tracing code In powerpc pseries system, below behaviour is observed while enabling tracing on hcall: # cd /sys/kernel/debug/tracing/ # cat events/powerpc/hcall_exit/enable 0 # echo 1 > events/powerpc/hcall_exit/enable # ls -bash: fork: Bad address Above is from power9 lpar with latest kernel. Past this, softlockup is observed. Initially while attempting via perf_event_open to use "PERF_TYPE_TRACEPOINT", kernel panic was observed. perf config used: ================ memset(&pe[1],0,sizeof(struct perf_event_attr)); pe[1].type=PERF_TYPE_TRACEPOINT; pe[1].size=96; pe[1].config=0x26ULL; /* 38 raw_syscalls/sys_exit */ pe[1].sample_type=0; /* 0 */ pe[1].read_format=PERF_FORMAT_TOTAL_TIME_ENABLED|PERF_FORMAT_TOTAL_TIME_RUNNING|PERF_FORMAT_ID|PERF_FORMAT_GROUP|0x10ULL; /* 1f */ pe[1].inherit=1; pe[1].precise_ip=0; /* arbitrary skid */ pe[1].wakeup_events=0; pe[1].bp_type=HW_BREAKPOINT_EMPTY; pe[1].config1=0x1ULL; Kernel panic logs: ================== Kernel attempted to read user page (8) - exploit attempt? (uid: 0) BUG: Kernel NULL pointer dereference on read at 0x00000008 Faulting instruction address: 0xc0000000004c2814 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries Modules linked in: nfnetlink bonding tls rfkill sunrpc dm_service_time dm_multipath pseries_rng xts vmx_crypto xfs libcrc32c sd_mod t10_pi crc64_rocksoft crc64 sg ibmvfc scsi_transport_fc ibmveth dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 1431 Comm: login Not tainted 6.4.0+ #1 Hardware name: IBM,8375-42A POWER9 (raw) 0x4e0202 0xf000005 of:IBM,FW950.30 (VL950_892) hv:phyp pSeries NIP page_remove_rmap+0x44/0x320 LR wp_page_copy+0x384/0xec0 Call Trace: 0xc00000001416e400 (unreliable) wp_page_copy+0x384/0xec0 __handle_mm_fault+0x9d4/0xfb0 handle_mm_fault+0xf0/0x350 ___do_page_fault+0x48c/0xc90 hash__do_page_fault+0x30/0x70 do_hash_fault+0x1a4/0x330 data_access_common_virt+0x198/0x1f0 --- interrupt: 300 at 0x7fffae971abc git bisect tracked this down to below commit: 'commit baa49d81a94b ("powerpc/pseries: hvcall stack frame overhead")' This commit changed STACK_FRAME_OVERHEAD (112 ) to STACK_FRAME_MIN_SIZE (32 ) since 32 bytes is the minimum size for ELFv2 stack. With the latest kernel, when running on ELFv2, STACK_FRAME_MIN_SIZE is used to allocate stack size. During plpar_hcall_trace, first call is made to HCALL_INST_PRECALL which saves the registers and allocates new stack frame. In the plpar_hcall_trace code, STK_PARAM is accessed at two places. 1. To save r4: std r4,STK_PARAM(R4)(r1) 2. To access r4 back: ld r12,STK_PARAM(R4)(r1) HCALL_INST_PRECALL precall allocates a new stack frame. So all the stack parameter access after the precall, needs to be accessed with +STACK_FRAME_MIN_SIZE. So the store instruction should be: std r4,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1) If the "std" is not updated with STACK_FRAME_MIN_SIZE, we will end up with overwriting stack contents and cause corruption. But instead of updating 'std', we can instead remove it since HCALL_INST_PRECALL already saves it to the correct location. similarly load instruction should be: ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1) Fix the load instruction to correctly access the stack parameter with +STACK_FRAME_MIN_SIZE and remove the store of r4 since the precall saves it correctly. Cc: stable@vger.kernel.org # v6.2+ Fixes: baa49d81a94b ("powerpc/pseries: hvcall stack frame overhead") Co-developed-by: Naveen N Rao <naveen@kernel.org> Signed-off-by: Naveen N Rao <naveen@kernel.org> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://msgid.link/20230929172337.7906-1-atrajeev@linux.vnet.ibm.com
2023-09-29 22:53:36 +05:30
ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
std r4,0(r12)
std r5,8(r12)
std r6,16(r12)
std r7,24(r12)
powerpc: tracing: Give hypervisor call tracepoints access to arguments While most users of the hcall tracepoints will only want the opcode and return code, some will want all the arguments. To avoid the complexity of using varargs we pass a pointer to the register save area, which contains all the arguments. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:51:09 +00:00
HCALL_INST_POSTCALL(r12)
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
lwz r0,8(r1)
mtcrf 0xff,r0
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
blr
#endif
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
[POWERPC] Avoid hypervisor statistics calculation in real mode kexec invokes plpar_hcall hypervisor call in real mode. plpar_hcall refers to per cpu variables for accounting hypervisor statistics. These variables may not be in the RMO region, so accesses to them in real mode may result in a data storage exception. This fixes this problem by using a new plpar_hcall_raw function which does not update the hypervisor call statistics. Thanks to Anton for suggesting this idea. Signed-off-by: Mohan Kumar M <mohan@in.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-03-21 11:21:32 +05:30
/*
* plpar_hcall_raw can be called in real mode. kexec/kdump need some
* hypervisor calls to be executed in real mode. So plpar_hcall_raw
* does not access the per cpu hypervisor call statistics variables,
* since these variables may not be present in the RMO region.
*/
_GLOBAL(plpar_hcall_raw)
HMT_MEDIUM
mfcr r0
stw r0,8(r1)
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
std r4,STK_PARAM(R4)(r1) /* Save ret buffer */
[POWERPC] Avoid hypervisor statistics calculation in real mode kexec invokes plpar_hcall hypervisor call in real mode. plpar_hcall refers to per cpu variables for accounting hypervisor statistics. These variables may not be in the RMO region, so accesses to them in real mode may result in a data storage exception. This fixes this problem by using a new plpar_hcall_raw function which does not update the hypervisor call statistics. Thanks to Anton for suggesting this idea. Signed-off-by: Mohan Kumar M <mohan@in.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-03-21 11:21:32 +05:30
mr r4,r5
mr r5,r6
mr r6,r7
mr r7,r8
mr r8,r9
mr r9,r10
HVSC /* invoke the hypervisor */
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r12,STK_PARAM(R4)(r1)
[POWERPC] Avoid hypervisor statistics calculation in real mode kexec invokes plpar_hcall hypervisor call in real mode. plpar_hcall refers to per cpu variables for accounting hypervisor statistics. These variables may not be in the RMO region, so accesses to them in real mode may result in a data storage exception. This fixes this problem by using a new plpar_hcall_raw function which does not update the hypervisor call statistics. Thanks to Anton for suggesting this idea. Signed-off-by: Mohan Kumar M <mohan@in.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-03-21 11:21:32 +05:30
std r4, 0(r12)
std r5, 8(r12)
std r6, 16(r12)
std r7, 24(r12)
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
[POWERPC] Avoid hypervisor statistics calculation in real mode kexec invokes plpar_hcall hypervisor call in real mode. plpar_hcall refers to per cpu variables for accounting hypervisor statistics. These variables may not be in the RMO region, so accesses to them in real mode may result in a data storage exception. This fixes this problem by using a new plpar_hcall_raw function which does not update the hypervisor call statistics. Thanks to Anton for suggesting this idea. Signed-off-by: Mohan Kumar M <mohan@in.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-03-21 11:21:32 +05:30
lwz r0,8(r1)
mtcrf 0xff,r0
blr /* return r3 = status */
powerpc/pseries: hcall functions are exported to modules, need _GLOBAL_TOC() The hcall macros may call out to c code for tracing, so we need to set up a valid r2. This fixes an oops found when testing ibmvscsi as a module. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-05-13 20:48:57 +10:00
_GLOBAL_TOC(plpar_hcall9)
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
HMT_MEDIUM
mfcr r0
stw r0,8(r1)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
HCALL_BRANCH(plpar_hcall9_trace)
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
std r4,STK_PARAM(R4)(r1) /* Save ret buffer */
[POWERPC] clean up pseries hcall interfaces Our pseries hcall interfaces are out of control: plpar_hcall_norets plpar_hcall plpar_hcall_8arg_2ret plpar_hcall_4out plpar_hcall_7arg_7ret plpar_hcall_9arg_9ret Create 3 interfaces to cover all cases: plpar_hcall_norets: 7 arguments no returns plpar_hcall: 6 arguments 4 returns plpar_hcall9: 9 arguments 9 returns There are only 2 cases in the kernel that need plpar_hcall9, hopefully we can keep it that way. Pass in a buffer to stash return parameters so we avoid the &dummy1, &dummy2 madness. Signed-off-by: Anton Blanchard <anton@samba.org> -- Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-07-19 08:01:28 +10:00
mr r4,r5
mr r5,r6
mr r6,r7
mr r7,r8
mr r8,r9
mr r9,r10
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r10,STK_PARAM(R11)(r1) /* put arg7 in R10 */
ld r11,STK_PARAM(R12)(r1) /* put arg8 in R11 */
ld r12,STK_PARAM(R13)(r1) /* put arg9 in R12 */
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
HVSC /* invoke the hypervisor */
[POWERPC] Fix corruption in hcall9 It looks to me like we are corrupting r12 in the hcall9 function. Although we have r0 free we cant use offsets against it, so save away r12 in there instead. r12 holds the ninth return value from the hypervisor call, so without this fix, the caller will see the wrong value for the ninth element in the array that gets the return values. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-01-09 02:37:16 +11:00
mr r0,r12
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r12,STK_PARAM(R4)(r1)
[POWERPC] clean up pseries hcall interfaces Our pseries hcall interfaces are out of control: plpar_hcall_norets plpar_hcall plpar_hcall_8arg_2ret plpar_hcall_4out plpar_hcall_7arg_7ret plpar_hcall_9arg_9ret Create 3 interfaces to cover all cases: plpar_hcall_norets: 7 arguments no returns plpar_hcall: 6 arguments 4 returns plpar_hcall9: 9 arguments 9 returns There are only 2 cases in the kernel that need plpar_hcall9, hopefully we can keep it that way. Pass in a buffer to stash return parameters so we avoid the &dummy1, &dummy2 madness. Signed-off-by: Anton Blanchard <anton@samba.org> -- Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-07-19 08:01:28 +10:00
std r4, 0(r12)
std r5, 8(r12)
std r6, 16(r12)
std r7, 24(r12)
std r8, 32(r12)
std r9, 40(r12)
std r10,48(r12)
std r11,56(r12)
[POWERPC] Fix corruption in hcall9 It looks to me like we are corrupting r12 in the hcall9 function. Although we have r0 free we cant use offsets against it, so save away r12 in there instead. r12 holds the ninth return value from the hypervisor call, so without this fix, the caller will see the wrong value for the ninth element in the array that gets the return values. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2007-01-09 02:37:16 +11:00
std r0, 64(r12)
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
lwz r0,8(r1)
mtcrf 0xff,r0
blr /* return r3 = status */
#ifdef CONFIG_TRACEPOINTS
plpar_hcall9_trace:
HCALL_INST_PRECALL(R5)
mr r4,r5
mr r5,r6
mr r6,r7
mr r7,r8
mr r8,r9
mr r9,r10
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
ld r10,STACK_FRAME_MIN_SIZE+STK_PARAM(R11)(r1)
ld r11,STACK_FRAME_MIN_SIZE+STK_PARAM(R12)(r1)
ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R13)(r1)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
HVSC
mr r0,r12
powerpc/pseries: hvcall stack frame overhead This call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20221127124942.1665522-6-npiggin@gmail.com
2022-11-27 22:49:30 +10:00
ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1)
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
std r4,0(r12)
std r5,8(r12)
std r6,16(r12)
std r7,24(r12)
std r8,32(r12)
std r9,40(r12)
std r10,48(r12)
std r11,56(r12)
std r0,64(r12)
powerpc: tracing: Give hypervisor call tracepoints access to arguments While most users of the hcall tracepoints will only want the opcode and return code, some will want all the arguments. To avoid the complexity of using varargs we pass a pointer to the register save area, which contains all the arguments. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Paul Mackerras <paulus@samba.org>
2009-10-26 18:51:09 +00:00
HCALL_INST_POSTCALL(r12)
[POWERPC] powerpc: Instrument Hypervisor Calls Add instrumentation for hypervisor calls on pseries. Call statistics include number of calls, wall time and cpu cycles (if available) and are made available via debugfs. Instrumentation code is behind the HCALL_STATS config option and has no impact if not enabled. Signed-off-by: Mike Kravetz <kravetz@us.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-09-06 16:23:12 -07:00
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
[PATCH] powerpc: Extends HCALL interface for InfiniBand usage This extends the HCALL interface for InfiniBand usage. I've made the patch against the linux-2.6 git tree and Segher's patch: [PATCH] Change H_StudlyCaps to H_SHOUTING_CAPS We moved this into the common powerpc code based on comments we got after posting the first eHCA InfiniBand device driver patch. Signed-off-by: Heiko j Schick <schickhj@de.ibm.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-03-30 22:47:14 +02:00
lwz r0,8(r1)
mtcrf 0xff,r0
powerpc/pseries: Use jump labels for hcall tracepoints hcall tracepoints add quite a few instructions to our hcall path: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) b 164 <---- start ld r12,0(r2) std r12,32(r1) cmpdi r12,0 beq 164 <---- end ... We have an unconditional branch that gets noped out during boot and a load/compare/branch. We also store the tracepoint value to the stack for the hcall_exit path to use. By using jump labels we can simplify this to just a single nop that gets replaced with a branch when the tracepoint is enabled: plpar_hcall: mr r2,r2 mfcr r0 stw r0,8(r1) nop <---- ... If jump labels are not enabled, we fall back to the old method. Signed-off-by: Anton Blanchard <anton@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2014-07-03 15:52:03 +10:00
blr
#endif
powerpc/pseries: Add hcall to read 4 ptes at a time in real mode This adds plpar_pte_read_4_raw() which can be used read 4 PTEs from PHYP at a time, while in real mode. It also creates a new hcall9 which can be used in real mode. It's the same as plpar_hcall9 but minus the tracing hcall statistics which may require variables outside the RMO. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2010-05-10 20:28:26 +00:00
/* See plpar_hcall_raw to see why this is needed */
_GLOBAL(plpar_hcall9_raw)
HMT_MEDIUM
mfcr r0
stw r0,8(r1)
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
std r4,STK_PARAM(R4)(r1) /* Save ret buffer */
powerpc/pseries: Add hcall to read 4 ptes at a time in real mode This adds plpar_pte_read_4_raw() which can be used read 4 PTEs from PHYP at a time, while in real mode. It also creates a new hcall9 which can be used in real mode. It's the same as plpar_hcall9 but minus the tracing hcall statistics which may require variables outside the RMO. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2010-05-10 20:28:26 +00:00
mr r4,r5
mr r5,r6
mr r6,r7
mr r7,r8
mr r8,r9
mr r9,r10
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r10,STK_PARAM(R11)(r1) /* put arg7 in R10 */
ld r11,STK_PARAM(R12)(r1) /* put arg8 in R11 */
ld r12,STK_PARAM(R13)(r1) /* put arg9 in R12 */
powerpc/pseries: Add hcall to read 4 ptes at a time in real mode This adds plpar_pte_read_4_raw() which can be used read 4 PTEs from PHYP at a time, while in real mode. It also creates a new hcall9 which can be used in real mode. It's the same as plpar_hcall9 but minus the tracing hcall statistics which may require variables outside the RMO. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2010-05-10 20:28:26 +00:00
HVSC /* invoke the hypervisor */
mr r0,r12
powerpc: Merge STK_REG/PARAM/FRAMESIZE Merge the defines of STACKFRAMESIZE, STK_REG, STK_PARAM from different places. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 13:33:14 +00:00
ld r12,STK_PARAM(R4)(r1)
powerpc/pseries: Add hcall to read 4 ptes at a time in real mode This adds plpar_pte_read_4_raw() which can be used read 4 PTEs from PHYP at a time, while in real mode. It also creates a new hcall9 which can be used in real mode. It's the same as plpar_hcall9 but minus the tracing hcall statistics which may require variables outside the RMO. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2010-05-10 20:28:26 +00:00
std r4, 0(r12)
std r5, 8(r12)
std r6, 16(r12)
std r7, 24(r12)
std r8, 32(r12)
std r9, 40(r12)
std r10,48(r12)
std r11,56(r12)
std r0, 64(r12)
powerpc/64s: avoid reloading (H)SRR registers if they are still valid When an interrupt is taken, the SRR registers are set to return to where it left off. Unless they are modified in the meantime, or the return address or MSR are modified, there is no need to reload these registers when returning from interrupt. Introduce per-CPU flags that track the validity of SRR and HSRR registers. These are cleared when returning from interrupt, when using the registers for something else (e.g., OPAL calls), when adjusting the return address or MSR of a context, and when context switching (which changes the return address and MSR). This improves the performance of interrupt returns. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> [mpe: Fold in fixup patch from Nick] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20210617155116.2167984-5-npiggin@gmail.com
2021-06-18 01:51:03 +10:00
li r4,0
stb r4,PACASRR_VALID(r13)
powerpc/pseries: Add hcall to read 4 ptes at a time in real mode This adds plpar_pte_read_4_raw() which can be used read 4 PTEs from PHYP at a time, while in real mode. It also creates a new hcall9 which can be used in real mode. It's the same as plpar_hcall9 but minus the tracing hcall statistics which may require variables outside the RMO. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2010-05-10 20:28:26 +00:00
lwz r0,8(r1)
mtcrf 0xff,r0
blr /* return r3 = status */
Reference in a new issue Copy permalink