2017-12-04 10:39:38 +01:00
|
|
|
// SPDX-License-Identifier: GPL-2.0
|
2024-07-16 13:50:53 +02:00
|
|
|
|
2025-02-07 15:49:11 +01:00
|
|
|
#ifndef pr_fmt
|
|
|
|
#define pr_fmt(fmt) "alt: " fmt
|
|
|
|
#endif
|
|
|
|
|
2024-07-16 13:50:53 +02:00
|
|
|
#include <linux/uaccess.h>
|
2025-02-07 15:49:11 +01:00
|
|
|
#include <linux/printk.h>
|
2024-07-16 13:50:55 +02:00
|
|
|
#include <asm/nospec-branch.h>
|
s390: Add infrastructure to patch lowcore accesses
The s390 architecture defines two special per-CPU data pages
called the "prefix area". In s390-linux terminology this is usually
called "lowcore". This memory area contains system configuration
data like old/new PSW's for system call/interrupt/machine check
handlers and lots of other data. It is normally mapped to logical
address 0. This area can only be accessed when in supervisor mode.
This means that kernel code can dereference NULL pointers, because
accesses to address 0 are allowed. Parts of lowcore can be write
protected, but read accesses and write accesses outside of the write
protected areas are not caught.
To remove this limitation for debugging and testing, remap lowcore to
another address and define a function get_lowcore() which simply
returns the address where lowcore is mapped at. This would normally
introduce a pointer dereference (=memory read). As lowcore is used
for several very often used variables, add code to patch this function
during runtime, so we avoid the memory reads.
For C code get_lowcore() has to be used, for assembly code it is
the GET_LC macro. When using this macro/function a reference is added
to alternative patching. All these locations will be patched to the
actual lowcore location when the kernel is booted or a module is loaded.
To make debugging/bisecting problems easier, this patch adds all the
infrastructure but the lowcore address is still hardwired to 0. This
way the code can be converted on a per function basis, and the
functionality is enabled in a patch after all the functions have
been converted.
Note that this requires at least z16 because the old lpsw instruction
only allowed a 12 bit displacement. z16 introduced lpswey which allows
20 bits (signed), so the lowcore can effectively be mapped from
address 0 - 0x7e000. To use 0x7e000 as address, a 6 byte lgfi
instruction would have to be used in the alternative. To save two
bytes, llilh can be used, but this only allows to set bits 16-31 of
the address. In order to use the llilh instruction, use 0x70000 as
alternative lowcore address. This is still large enough to catch
NULL pointer dereferences into large arrays.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2024-07-22 15:41:14 +02:00
|
|
|
#include <asm/abs_lowcore.h>
|
s390: introduce CPU alternatives
Implement CPU alternatives, which allows to optionally patch newer
instructions at runtime, based on CPU facilities availability.
A new kernel boot parameter "noaltinstr" disables patching.
Current implementation is derived from x86 alternatives. Although
ideal instructions padding (when altinstr is longer then oldinstr)
is added at compile time, and no oldinstr nops optimization has to be
done at runtime. Also couple of compile time sanity checks are done:
1. oldinstr and altinstr must be <= 254 bytes long,
2. oldinstr and altinstr must not have an odd length.
alternative(oldinstr, altinstr, facility);
alternative_2(oldinstr, altinstr1, facility1, altinstr2, facility2);
Both compile time and runtime padding consists of either 6/4/2 bytes nop
or a jump (brcl) + 2 bytes nop filler if padding is longer then 6 bytes.
.altinstructions and .altinstr_replacement sections are part of
__init_begin : __init_end region and are freed after initialization.
Signed-off-by: Vasily Gorbik <gor@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2017-10-12 13:01:47 +02:00
|
|
|
#include <asm/alternative.h>
|
|
|
|
#include <asm/facility.h>
|
2025-02-07 15:48:56 +01:00
|
|
|
#include <asm/sections.h>
|
|
|
|
#include <asm/machine.h>
|
|
|
|
|
2025-02-07 15:49:11 +01:00
|
|
|
#ifndef a_debug
|
|
|
|
#define a_debug pr_debug
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#ifndef __kernel_va
|
|
|
|
#define __kernel_va(x) (void *)(x)
|
|
|
|
#endif
|
|
|
|
|
2025-02-07 15:48:56 +01:00
|
|
|
unsigned long __bootdata_preserved(machine_features[1]);
|
s390: introduce CPU alternatives
Implement CPU alternatives, which allows to optionally patch newer
instructions at runtime, based on CPU facilities availability.
A new kernel boot parameter "noaltinstr" disables patching.
Current implementation is derived from x86 alternatives. Although
ideal instructions padding (when altinstr is longer then oldinstr)
is added at compile time, and no oldinstr nops optimization has to be
done at runtime. Also couple of compile time sanity checks are done:
1. oldinstr and altinstr must be <= 254 bytes long,
2. oldinstr and altinstr must not have an odd length.
alternative(oldinstr, altinstr, facility);
alternative_2(oldinstr, altinstr1, facility1, altinstr2, facility2);
Both compile time and runtime padding consists of either 6/4/2 bytes nop
or a jump (brcl) + 2 bytes nop filler if padding is longer then 6 bytes.
.altinstructions and .altinstr_replacement sections are part of
__init_begin : __init_end region and are freed after initialization.
Signed-off-by: Vasily Gorbik <gor@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2017-10-12 13:01:47 +02:00
|
|
|
|
2025-02-07 15:49:11 +01:00
|
|
|
struct alt_debug {
|
|
|
|
unsigned long facilities[MAX_FACILITY_BIT / BITS_PER_LONG];
|
|
|
|
unsigned long mfeatures[MAX_MFEATURE_BIT / BITS_PER_LONG];
|
|
|
|
int spec;
|
|
|
|
};
|
|
|
|
|
|
|
|
static struct alt_debug __bootdata_preserved(alt_debug);
|
|
|
|
|
|
|
|
static void alternative_dump(u8 *old, u8 *new, unsigned int len, unsigned int type, unsigned int data)
|
|
|
|
{
|
|
|
|
char oinsn[33], ninsn[33];
|
|
|
|
unsigned long kptr;
|
|
|
|
unsigned int pos;
|
|
|
|
|
|
|
|
for (pos = 0; pos < len && 2 * pos < sizeof(oinsn) - 3; pos++)
|
|
|
|
hex_byte_pack(&oinsn[2 * pos], old[pos]);
|
|
|
|
oinsn[2 * pos] = 0;
|
|
|
|
for (pos = 0; pos < len && 2 * pos < sizeof(ninsn) - 3; pos++)
|
|
|
|
hex_byte_pack(&ninsn[2 * pos], new[pos]);
|
|
|
|
ninsn[2 * pos] = 0;
|
|
|
|
kptr = (unsigned long)__kernel_va(old);
|
|
|
|
a_debug("[%d/%3d] %016lx: %s -> %s\n", type, data, kptr, oinsn, ninsn);
|
|
|
|
}
|
|
|
|
|
2024-07-16 13:50:53 +02:00
|
|
|
void __apply_alternatives(struct alt_instr *start, struct alt_instr *end, unsigned int ctx)
|
s390: introduce CPU alternatives
Implement CPU alternatives, which allows to optionally patch newer
instructions at runtime, based on CPU facilities availability.
A new kernel boot parameter "noaltinstr" disables patching.
Current implementation is derived from x86 alternatives. Although
ideal instructions padding (when altinstr is longer then oldinstr)
is added at compile time, and no oldinstr nops optimization has to be
done at runtime. Also couple of compile time sanity checks are done:
1. oldinstr and altinstr must be <= 254 bytes long,
2. oldinstr and altinstr must not have an odd length.
alternative(oldinstr, altinstr, facility);
alternative_2(oldinstr, altinstr1, facility1, altinstr2, facility2);
Both compile time and runtime padding consists of either 6/4/2 bytes nop
or a jump (brcl) + 2 bytes nop filler if padding is longer then 6 bytes.
.altinstructions and .altinstr_replacement sections are part of
__init_begin : __init_end region and are freed after initialization.
Signed-off-by: Vasily Gorbik <gor@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2017-10-12 13:01:47 +02:00
|
|
|
{
|
2025-02-07 15:49:11 +01:00
|
|
|
struct alt_debug *d;
|
2024-07-16 13:50:53 +02:00
|
|
|
struct alt_instr *a;
|
2025-02-07 15:49:11 +01:00
|
|
|
bool debug, replace;
|
|
|
|
u8 *old, *new;
|
s390: introduce CPU alternatives
Implement CPU alternatives, which allows to optionally patch newer
instructions at runtime, based on CPU facilities availability.
A new kernel boot parameter "noaltinstr" disables patching.
Current implementation is derived from x86 alternatives. Although
ideal instructions padding (when altinstr is longer then oldinstr)
is added at compile time, and no oldinstr nops optimization has to be
done at runtime. Also couple of compile time sanity checks are done:
1. oldinstr and altinstr must be <= 254 bytes long,
2. oldinstr and altinstr must not have an odd length.
alternative(oldinstr, altinstr, facility);
alternative_2(oldinstr, altinstr1, facility1, altinstr2, facility2);
Both compile time and runtime padding consists of either 6/4/2 bytes nop
or a jump (brcl) + 2 bytes nop filler if padding is longer then 6 bytes.
.altinstructions and .altinstr_replacement sections are part of
__init_begin : __init_end region and are freed after initialization.
Signed-off-by: Vasily Gorbik <gor@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2017-10-12 13:01:47 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* The scan order should be from start to end. A later scanned
|
|
|
|
* alternative code can overwrite previously scanned alternative code.
|
|
|
|
*/
|
2025-02-07 15:49:11 +01:00
|
|
|
d = &alt_debug;
|
s390: introduce CPU alternatives
Implement CPU alternatives, which allows to optionally patch newer
instructions at runtime, based on CPU facilities availability.
A new kernel boot parameter "noaltinstr" disables patching.
Current implementation is derived from x86 alternatives. Although
ideal instructions padding (when altinstr is longer then oldinstr)
is added at compile time, and no oldinstr nops optimization has to be
done at runtime. Also couple of compile time sanity checks are done:
1. oldinstr and altinstr must be <= 254 bytes long,
2. oldinstr and altinstr must not have an odd length.
alternative(oldinstr, altinstr, facility);
alternative_2(oldinstr, altinstr1, facility1, altinstr2, facility2);
Both compile time and runtime padding consists of either 6/4/2 bytes nop
or a jump (brcl) + 2 bytes nop filler if padding is longer then 6 bytes.
.altinstructions and .altinstr_replacement sections are part of
__init_begin : __init_end region and are freed after initialization.
Signed-off-by: Vasily Gorbik <gor@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2017-10-12 13:01:47 +02:00
|
|
|
for (a = start; a < end; a++) {
|
2024-07-16 13:50:53 +02:00
|
|
|
if (!(a->ctx & ctx))
|
|
|
|
continue;
|
|
|
|
switch (a->type) {
|
|
|
|
case ALT_TYPE_FACILITY:
|
2024-07-16 13:50:56 +02:00
|
|
|
replace = test_facility(a->data);
|
2025-02-07 15:49:11 +01:00
|
|
|
debug = __test_facility(a->data, d->facilities);
|
2024-07-16 13:50:53 +02:00
|
|
|
break;
|
2025-02-07 15:48:56 +01:00
|
|
|
case ALT_TYPE_FEATURE:
|
|
|
|
replace = test_machine_feature(a->data);
|
2025-02-07 15:49:11 +01:00
|
|
|
debug = __test_machine_feature(a->data, d->mfeatures);
|
2025-02-07 15:48:56 +01:00
|
|
|
break;
|
2024-07-16 13:50:55 +02:00
|
|
|
case ALT_TYPE_SPEC:
|
|
|
|
replace = nobp_enabled();
|
2025-02-07 15:49:11 +01:00
|
|
|
debug = d->spec;
|
2024-07-16 13:50:55 +02:00
|
|
|
break;
|
2024-07-16 13:50:53 +02:00
|
|
|
default:
|
|
|
|
replace = false;
|
2025-02-07 15:49:11 +01:00
|
|
|
debug = false;
|
2024-07-16 13:50:53 +02:00
|
|
|
}
|
|
|
|
if (!replace)
|
|
|
|
continue;
|
2025-02-07 15:49:11 +01:00
|
|
|
old = (u8 *)&a->instr_offset + a->instr_offset;
|
|
|
|
new = (u8 *)&a->repl_offset + a->repl_offset;
|
|
|
|
if (debug)
|
|
|
|
alternative_dump(old, new, a->instrlen, a->type, a->data);
|
|
|
|
s390_kernel_write(old, new, a->instrlen);
|
s390: introduce CPU alternatives
Implement CPU alternatives, which allows to optionally patch newer
instructions at runtime, based on CPU facilities availability.
A new kernel boot parameter "noaltinstr" disables patching.
Current implementation is derived from x86 alternatives. Although
ideal instructions padding (when altinstr is longer then oldinstr)
is added at compile time, and no oldinstr nops optimization has to be
done at runtime. Also couple of compile time sanity checks are done:
1. oldinstr and altinstr must be <= 254 bytes long,
2. oldinstr and altinstr must not have an odd length.
alternative(oldinstr, altinstr, facility);
alternative_2(oldinstr, altinstr1, facility1, altinstr2, facility2);
Both compile time and runtime padding consists of either 6/4/2 bytes nop
or a jump (brcl) + 2 bytes nop filler if padding is longer then 6 bytes.
.altinstructions and .altinstr_replacement sections are part of
__init_begin : __init_end region and are freed after initialization.
Signed-off-by: Vasily Gorbik <gor@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2017-10-12 13:01:47 +02:00
|
|
|
}
|
|
|
|
}
|