License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 15:07:57 +01:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0 */
|
2016-04-29 23:25:41 +10:00
|
|
|
#ifndef _ASM_POWERPC_BOOK3S_64_MMU_H_
|
|
|
|
|
#define _ASM_POWERPC_BOOK3S_64_MMU_H_
|
|
|
|
|
|
2018-11-29 14:06:57 +00:00
|
|
|
#include <asm/page.h>
|
|
|
|
|
|
2016-04-29 23:25:41 +10:00
|
|
|
#ifndef __ASSEMBLY__
|
|
|
|
|
/*
|
|
|
|
|
* Page size definition
|
|
|
|
|
*
|
|
|
|
|
* shift : is the "PAGE_SHIFT" value for that page size
|
|
|
|
|
* sllp : is a bit mask with the value of SLB L || LP to be or'ed
|
|
|
|
|
* directly to a slbmte "vsid" value
|
|
|
|
|
* penc : is the HPTE encoding mask for the "LP" field:
|
|
|
|
|
*
|
|
|
|
|
*/
|
|
|
|
|
struct mmu_psize_def {
|
|
|
|
|
unsigned int shift; /* number of bits */
|
|
|
|
|
int penc[MMU_PAGE_COUNT]; /* HPTE encoding */
|
|
|
|
|
unsigned int tlbiel; /* tlbiel supported for that page size */
|
|
|
|
|
unsigned long avpnm; /* bits to mask out in AVPN in the HPTE */
|
2021-06-21 14:19:59 +05:30
|
|
|
unsigned long h_rpt_pgsize; /* H_RPT_INVALIDATE page size encoding */
|
2016-04-29 23:25:58 +10:00
|
|
|
union {
|
|
|
|
|
unsigned long sllp; /* SLB L||LP (exact mask to use in slbmte) */
|
|
|
|
|
unsigned long ap; /* Ap encoding used by PowerISA 3.0 */
|
|
|
|
|
};
|
2016-04-29 23:25:41 +10:00
|
|
|
};
|
|
|
|
|
extern struct mmu_psize_def mmu_psize_defs[MMU_PAGE_COUNT];
|
|
|
|
|
#endif /* __ASSEMBLY__ */
|
|
|
|
|
|
|
|
|
|
/* 64-bit classic hash table MMU */
|
|
|
|
|
#include <asm/book3s/64/mmu-hash.h>
|
|
|
|
|
|
|
|
|
|
#ifndef __ASSEMBLY__
|
2016-04-29 23:25:42 +10:00
|
|
|
/*
|
2017-02-27 14:28:55 -08:00
|
|
|
* ISA 3.0 partition and process table entry format
|
2016-04-29 23:25:42 +10:00
|
|
|
*/
|
|
|
|
|
struct prtb_entry {
|
|
|
|
|
__be64 prtb0;
|
|
|
|
|
__be64 prtb1;
|
|
|
|
|
};
|
|
|
|
|
extern struct prtb_entry *process_tb;
|
|
|
|
|
|
|
|
|
|
struct patb_entry {
|
|
|
|
|
__be64 patb0;
|
|
|
|
|
__be64 patb1;
|
|
|
|
|
};
|
|
|
|
|
extern struct patb_entry *partition_tb;
|
|
|
|
|
|
2017-01-30 21:21:37 +11:00
|
|
|
/* Bits in patb0 field */
|
2016-04-29 23:25:42 +10:00
|
|
|
#define PATB_HR (1UL << 63)
|
2017-02-27 11:51:37 +11:00
|
|
|
#define RPDB_MASK 0x0fffffffffffff00UL
|
2016-04-29 23:25:42 +10:00
|
|
|
#define RPDB_SHIFT (1UL << 8)
|
2017-01-30 21:21:37 +11:00
|
|
|
#define RTS1_SHIFT 61 /* top 2 bits of radix tree size */
|
|
|
|
|
#define RTS1_MASK (3UL << RTS1_SHIFT)
|
|
|
|
|
#define RTS2_SHIFT 5 /* bottom 3 bits of radix tree size */
|
|
|
|
|
#define RTS2_MASK (7UL << RTS2_SHIFT)
|
|
|
|
|
#define RPDS_MASK 0x1f /* root page dir. size field */
|
|
|
|
|
|
|
|
|
|
/* Bits in patb1 field */
|
|
|
|
|
#define PATB_GR (1UL << 63) /* guest uses radix; must match HR */
|
|
|
|
|
#define PRTS_MASK 0x1f /* process table size field */
|
2017-02-27 11:51:37 +11:00
|
|
|
#define PRTB_MASK 0x0ffffffffffff000UL
|
2017-01-30 21:21:37 +11:00
|
|
|
|
2021-11-29 13:09:15 +10:00
|
|
|
/* Number of supported LPID bits */
|
|
|
|
|
extern unsigned int mmu_lpid_bits;
|
|
|
|
|
|
powerpc/mm/radix: Workaround prefetch issue with KVM
There's a somewhat architectural issue with Radix MMU and KVM.
When coming out of a guest with AIL (Alternate Interrupt Location, ie,
MMU enabled), we start executing hypervisor code with the PID register
still containing whatever the guest has been using.
The problem is that the CPU can (and will) then start prefetching or
speculatively load from whatever host context has that same PID (if
any), thus bringing translations for that context into the TLB, which
Linux doesn't know about.
This can cause stale translations and subsequent crashes.
Fixing this in a way that is neither racy nor a huge performance
impact is difficult. We could just make the host invalidations always
use broadcast forms but that would hurt single threaded programs for
example.
We chose to fix it instead by partitioning the PID space between guest
and host. This is possible because today Linux only use 19 out of the
20 bits of PID space, so existing guests will work if we make the host
use the top half of the 20 bits space.
We additionally add support for a property to indicate to Linux the
size of the PID register which will be useful if we eventually have
processors with a larger PID space available.
There is still an issue with malicious guests purposefully setting the
PID register to a value in the hosts PID range. Hopefully future HW
can prevent that, but in the meantime, we handle it with a pair of
kludges:
- On the way out of a guest, before we clear the current VCPU in the
PACA, we check the PID and if it's outside of the permitted range
we flush the TLB for that PID.
- When context switching, if the mm is "new" on that CPU (the
corresponding bit was set for the first time in the mm cpumask), we
check if any sibling thread is in KVM (has a non-NULL VCPU pointer
in the PACA). If that is the case, we also flush the PID for that
CPU (core).
This second part is needed to handle the case where a process is
migrated (or starts a new pthread) on a sibling thread of the CPU
coming out of KVM, as there's a window where stale translations can
exist before we detect it and flush them out.
A future optimization could be added by keeping track of whether the
PID has ever been used and avoid doing that for completely fresh PIDs.
We could similarily mark PIDs that have been the subject of a global
invalidation as "fresh". But for now this will do.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
[mpe: Rework the asm to build with CONFIG_PPC_RADIX_MMU=n, drop
unneeded include of kvm_book3s_asm.h]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-07-24 14:26:06 +10:00
|
|
|
/* Number of supported PID bits */
|
|
|
|
|
extern unsigned int mmu_pid_bits;
|
|
|
|
|
|
|
|
|
|
/* Base PID to allocate from */
|
|
|
|
|
extern unsigned int mmu_base_pid;
|
|
|
|
|
|
2023-08-01 10:14:46 +05:30
|
|
|
extern unsigned long __ro_after_init memory_block_size;
|
2020-07-09 18:49:25 +05:30
|
|
|
|
powerpc/mm/radix: Workaround prefetch issue with KVM
There's a somewhat architectural issue with Radix MMU and KVM.
When coming out of a guest with AIL (Alternate Interrupt Location, ie,
MMU enabled), we start executing hypervisor code with the PID register
still containing whatever the guest has been using.
The problem is that the CPU can (and will) then start prefetching or
speculatively load from whatever host context has that same PID (if
any), thus bringing translations for that context into the TLB, which
Linux doesn't know about.
This can cause stale translations and subsequent crashes.
Fixing this in a way that is neither racy nor a huge performance
impact is difficult. We could just make the host invalidations always
use broadcast forms but that would hurt single threaded programs for
example.
We chose to fix it instead by partitioning the PID space between guest
and host. This is possible because today Linux only use 19 out of the
20 bits of PID space, so existing guests will work if we make the host
use the top half of the 20 bits space.
We additionally add support for a property to indicate to Linux the
size of the PID register which will be useful if we eventually have
processors with a larger PID space available.
There is still an issue with malicious guests purposefully setting the
PID register to a value in the hosts PID range. Hopefully future HW
can prevent that, but in the meantime, we handle it with a pair of
kludges:
- On the way out of a guest, before we clear the current VCPU in the
PACA, we check the PID and if it's outside of the permitted range
we flush the TLB for that PID.
- When context switching, if the mm is "new" on that CPU (the
corresponding bit was set for the first time in the mm cpumask), we
check if any sibling thread is in KVM (has a non-NULL VCPU pointer
in the PACA). If that is the case, we also flush the PID for that
CPU (core).
This second part is needed to handle the case where a process is
migrated (or starts a new pthread) on a sibling thread of the CPU
coming out of KVM, as there's a window where stale translations can
exist before we detect it and flush them out.
A future optimization could be added by keeping track of whether the
PID has ever been used and avoid doing that for completely fresh PIDs.
We could similarily mark PIDs that have been the subject of a global
invalidation as "fresh". But for now this will do.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
[mpe: Rework the asm to build with CONFIG_PPC_RADIX_MMU=n, drop
unneeded include of kvm_book3s_asm.h]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-07-24 14:26:06 +10:00
|
|
|
#define PRTB_SIZE_SHIFT (mmu_pid_bits + 4)
|
|
|
|
|
#define PRTB_ENTRIES (1ul << mmu_pid_bits)
|
2017-03-29 22:36:56 +11:00
|
|
|
|
2021-11-29 13:09:15 +10:00
|
|
|
#define PATB_SIZE_SHIFT (mmu_lpid_bits + 4)
|
|
|
|
|
#define PATB_ENTRIES (1ul << mmu_lpid_bits)
|
2016-04-29 23:25:41 +10:00
|
|
|
|
|
|
|
|
typedef unsigned long mm_context_id_t;
|
|
|
|
|
struct spinlock;
|
|
|
|
|
|
2017-04-03 19:51:44 +10:00
|
|
|
/* Maximum possible number of NPUs in a system. */
|
|
|
|
|
#define NV_MAX_NPUS 8
|
|
|
|
|
|
2016-04-29 23:25:41 +10:00
|
|
|
typedef struct {
|
2018-03-26 15:34:48 +05:30
|
|
|
union {
|
|
|
|
|
/*
|
|
|
|
|
* We use id as the PIDR content for radix. On hash we can use
|
|
|
|
|
* more than one id. The extended ids are used when we start
|
|
|
|
|
* having address above 512TB. We allocate one extended id
|
|
|
|
|
* for each 512TB. The new id is then used with the 49 bit
|
|
|
|
|
* EA to build a new VA. We always use ESID_BITS_1T_MASK bits
|
|
|
|
|
* from EA and new context ids to build the new VAs.
|
|
|
|
|
*/
|
|
|
|
|
mm_context_id_t id;
|
2021-12-02 00:41:52 +10:00
|
|
|
#ifdef CONFIG_PPC_64S_HASH_MMU
|
2018-03-26 15:34:48 +05:30
|
|
|
mm_context_id_t extended_id[TASK_SIZE_USER64/TASK_CONTEXT_SIZE];
|
2021-12-02 00:41:52 +10:00
|
|
|
#endif
|
2018-03-26 15:34:48 +05:30
|
|
|
};
|
2016-04-29 23:25:41 +10:00
|
|
|
|
2017-07-24 14:28:02 +10:00
|
|
|
/* Number of bits in the mm_cpumask */
|
|
|
|
|
atomic_t active_cpus;
|
|
|
|
|
|
2018-03-23 09:29:05 +11:00
|
|
|
/* Number of users of the external (Nest) MMU */
|
|
|
|
|
atomic_t copros;
|
|
|
|
|
|
2020-04-15 23:08:11 -07:00
|
|
|
/* Number of user space windows opened in process mm_context */
|
|
|
|
|
atomic_t vas_windows;
|
|
|
|
|
|
2021-12-02 00:41:52 +10:00
|
|
|
#ifdef CONFIG_PPC_64S_HASH_MMU
|
2019-04-17 18:33:50 +05:30
|
|
|
struct hash_mm_context *hash_context;
|
2021-12-02 00:41:52 +10:00
|
|
|
#endif
|
2017-04-03 19:51:44 +10:00
|
|
|
|
2020-09-27 09:16:29 +00:00
|
|
|
void __user *vdso;
|
2018-04-16 16:57:20 +05:30
|
|
|
/*
|
|
|
|
|
* pagetable fragment support
|
|
|
|
|
*/
|
2016-04-29 23:25:41 +10:00
|
|
|
void *pte_frag;
|
2018-04-16 16:57:22 +05:30
|
|
|
void *pmd_frag;
|
2016-04-29 23:25:41 +10:00
|
|
|
#ifdef CONFIG_SPAPR_TCE_IOMMU
|
|
|
|
|
struct list_head iommu_group_mem_list;
|
|
|
|
|
#endif
|
2018-01-18 17:50:25 -08:00
|
|
|
|
|
|
|
|
#ifdef CONFIG_PPC_MEM_KEYS
|
|
|
|
|
/*
|
|
|
|
|
* Each bit represents one protection key.
|
|
|
|
|
* bit set -> key allocated
|
|
|
|
|
* bit unset -> key available for allocation
|
|
|
|
|
*/
|
|
|
|
|
u32 pkey_allocation_map;
|
2018-01-18 17:50:32 -08:00
|
|
|
s16 execute_only_pkey; /* key holding execute-only protection */
|
2018-01-18 17:50:25 -08:00
|
|
|
#endif
|
2016-04-29 23:25:41 +10:00
|
|
|
} mm_context_t;
|
|
|
|
|
|
2021-12-02 00:41:52 +10:00
|
|
|
#ifdef CONFIG_PPC_64S_HASH_MMU
|
2019-04-17 18:33:48 +05:30
|
|
|
static inline u16 mm_ctx_user_psize(mm_context_t *ctx)
|
|
|
|
|
{
|
2019-04-17 18:33:50 +05:30
|
|
|
return ctx->hash_context->user_psize;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void mm_ctx_set_user_psize(mm_context_t *ctx, u16 user_psize)
|
|
|
|
|
{
|
2019-04-17 18:33:50 +05:30
|
|
|
ctx->hash_context->user_psize = user_psize;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline unsigned char *mm_ctx_low_slices(mm_context_t *ctx)
|
|
|
|
|
{
|
2019-04-17 18:33:50 +05:30
|
|
|
return ctx->hash_context->low_slices_psize;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline unsigned char *mm_ctx_high_slices(mm_context_t *ctx)
|
|
|
|
|
{
|
2019-04-17 18:33:50 +05:30
|
|
|
return ctx->hash_context->high_slices_psize;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline unsigned long mm_ctx_slb_addr_limit(mm_context_t *ctx)
|
|
|
|
|
{
|
2019-04-17 18:33:50 +05:30
|
|
|
return ctx->hash_context->slb_addr_limit;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void mm_ctx_set_slb_addr_limit(mm_context_t *ctx, unsigned long limit)
|
|
|
|
|
{
|
2019-04-17 18:33:50 +05:30
|
|
|
ctx->hash_context->slb_addr_limit = limit;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
|
2019-04-25 14:29:30 +00:00
|
|
|
static inline struct slice_mask *slice_mask_for_size(mm_context_t *ctx, int psize)
|
|
|
|
|
{
|
|
|
|
|
#ifdef CONFIG_PPC_64K_PAGES
|
|
|
|
|
if (psize == MMU_PAGE_64K)
|
2019-04-25 14:29:31 +00:00
|
|
|
return &ctx->hash_context->mask_64k;
|
2019-04-25 14:29:30 +00:00
|
|
|
#endif
|
|
|
|
|
#ifdef CONFIG_HUGETLB_PAGE
|
|
|
|
|
if (psize == MMU_PAGE_16M)
|
2019-04-25 14:29:31 +00:00
|
|
|
return &ctx->hash_context->mask_16m;
|
2019-04-25 14:29:30 +00:00
|
|
|
if (psize == MMU_PAGE_16G)
|
2019-04-25 14:29:31 +00:00
|
|
|
return &ctx->hash_context->mask_16g;
|
2019-04-25 14:29:30 +00:00
|
|
|
#endif
|
|
|
|
|
BUG_ON(psize != MMU_PAGE_4K);
|
|
|
|
|
|
2019-04-25 14:29:31 +00:00
|
|
|
return &ctx->hash_context->mask_4k;
|
2019-04-25 14:29:30 +00:00
|
|
|
}
|
|
|
|
|
|
2019-04-17 18:33:48 +05:30
|
|
|
#ifdef CONFIG_PPC_SUBPAGE_PROT
|
|
|
|
|
static inline struct subpage_prot_table *mm_ctx_subpage_prot(mm_context_t *ctx)
|
|
|
|
|
{
|
2019-04-17 18:33:51 +05:30
|
|
|
return ctx->hash_context->spt;
|
2019-04-17 18:33:48 +05:30
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
2016-04-29 23:25:41 +10:00
|
|
|
/*
|
|
|
|
|
* The current system page and segment sizes
|
|
|
|
|
*/
|
|
|
|
|
extern int mmu_virtual_psize;
|
|
|
|
|
extern int mmu_vmalloc_psize;
|
|
|
|
|
extern int mmu_io_psize;
|
2021-12-02 00:41:52 +10:00
|
|
|
#else /* CONFIG_PPC_64S_HASH_MMU */
|
|
|
|
|
#ifdef CONFIG_PPC_64K_PAGES
|
|
|
|
|
#define mmu_virtual_psize MMU_PAGE_64K
|
|
|
|
|
#else
|
|
|
|
|
#define mmu_virtual_psize MMU_PAGE_4K
|
|
|
|
|
#endif
|
|
|
|
|
#endif
|
2022-03-01 17:47:43 -03:00
|
|
|
extern int mmu_linear_psize;
|
2021-12-02 00:41:52 +10:00
|
|
|
extern int mmu_vmemmap_psize;
|
2016-04-29 23:25:41 +10:00
|
|
|
|
2016-04-29 23:25:57 +10:00
|
|
|
/* MMU initialization */
|
2016-07-26 20:09:30 +10:00
|
|
|
void mmu_early_init_devtree(void);
|
2016-07-26 21:31:59 +10:00
|
|
|
void hash__early_init_devtree(void);
|
2016-07-26 21:55:27 +10:00
|
|
|
void radix__early_init_devtree(void);
|
2020-11-27 10:14:05 +05:30
|
|
|
#ifdef CONFIG_PPC_PKEY
|
2020-07-09 08:59:36 +05:30
|
|
|
void pkey_early_init_devtree(void);
|
|
|
|
|
#else
|
|
|
|
|
static inline void pkey_early_init_devtree(void) {}
|
|
|
|
|
#endif
|
|
|
|
|
|
2016-04-29 23:25:57 +10:00
|
|
|
extern void hash__early_init_mmu(void);
|
2016-04-29 23:25:58 +10:00
|
|
|
extern void radix__early_init_mmu(void);
|
2020-04-29 17:02:47 +10:00
|
|
|
static inline void __init early_init_mmu(void)
|
2016-04-29 23:25:57 +10:00
|
|
|
{
|
2016-04-29 23:25:58 +10:00
|
|
|
if (radix_enabled())
|
|
|
|
|
return radix__early_init_mmu();
|
2016-04-29 23:25:57 +10:00
|
|
|
return hash__early_init_mmu();
|
|
|
|
|
}
|
|
|
|
|
extern void hash__early_init_mmu_secondary(void);
|
2016-04-29 23:25:58 +10:00
|
|
|
extern void radix__early_init_mmu_secondary(void);
|
2016-04-29 23:25:57 +10:00
|
|
|
static inline void early_init_mmu_secondary(void)
|
|
|
|
|
{
|
2016-04-29 23:25:58 +10:00
|
|
|
if (radix_enabled())
|
|
|
|
|
return radix__early_init_mmu_secondary();
|
2016-04-29 23:25:57 +10:00
|
|
|
return hash__early_init_mmu_secondary();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
extern void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base,
|
|
|
|
|
phys_addr_t first_memblock_size);
|
|
|
|
|
static inline void setup_initial_memory_limit(phys_addr_t first_memblock_base,
|
|
|
|
|
phys_addr_t first_memblock_size)
|
|
|
|
|
{
|
2020-08-28 15:38:52 +05:30
|
|
|
/*
|
|
|
|
|
* Hash has more strict restrictions. At this point we don't
|
|
|
|
|
* know which translations we will pick. Hence go with hash
|
|
|
|
|
* restrictions.
|
|
|
|
|
*/
|
2021-12-02 00:41:52 +10:00
|
|
|
if (!early_radix_enabled())
|
|
|
|
|
hash__setup_initial_memory_limit(first_memblock_base,
|
|
|
|
|
first_memblock_size);
|
2016-04-29 23:25:57 +10:00
|
|
|
}
|
2016-08-04 15:32:06 +10:00
|
|
|
|
2017-01-30 21:21:36 +11:00
|
|
|
#ifdef CONFIG_PPC_PSERIES
|
2021-12-16 17:00:27 -05:00
|
|
|
void __init radix_init_pseries(void);
|
2017-01-30 21:21:36 +11:00
|
|
|
#else
|
2021-01-25 17:53:38 +08:00
|
|
|
static inline void radix_init_pseries(void) { }
|
2017-01-30 21:21:36 +11:00
|
|
|
#endif
|
|
|
|
|
|
powerpc/64s: Trim offlined CPUs from mm_cpumasks
When offlining a CPU, powerpc/64s does not flush TLBs, rather it just
leaves the CPU set in mm_cpumasks, so it continues to receive TLBIEs
to manage its TLBs.
However the exit_flush_lazy_tlbs() function expects that after
returning, all CPUs (except self) have flushed TLBs for that mm, in
which case TLBIEL can be used for this flush. This breaks for offline
CPUs because they don't get the IPI to flush their TLB. This can lead
to stale translations.
Fix this by clearing the CPU from mm_cpumasks, then flushing all TLBs
before going offline.
These offlined CPU bits stuck in the cpumask also prevents the cpumask
from being trimmed back to local mode, which means continual broadcast
IPIs or TLBIEs are needed for TLB flushing. This patch prevents that
situation too.
A cast of many were involved in working this out, but in particular
Milton, Aneesh, Paul made key discoveries.
Fixes: 0cef77c7798a7 ("powerpc/64s/radix: flush remote CPUs out of single-threaded mm_cpumask")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Debugged-by: Milton Miller <miltonm@us.ibm.com>
Debugged-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Debugged-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20201126102530.691335-5-npiggin@gmail.com
2020-11-26 20:25:30 +10:00
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
|
#define arch_clear_mm_cpumask_cpu(cpu, mm) \
|
|
|
|
|
do { \
|
|
|
|
|
if (cpumask_test_cpu(cpu, mm_cpumask(mm))) { \
|
2023-05-24 16:08:19 +10:00
|
|
|
dec_mm_active_cpus(mm); \
|
powerpc/64s: Trim offlined CPUs from mm_cpumasks
When offlining a CPU, powerpc/64s does not flush TLBs, rather it just
leaves the CPU set in mm_cpumasks, so it continues to receive TLBIEs
to manage its TLBs.
However the exit_flush_lazy_tlbs() function expects that after
returning, all CPUs (except self) have flushed TLBs for that mm, in
which case TLBIEL can be used for this flush. This breaks for offline
CPUs because they don't get the IPI to flush their TLB. This can lead
to stale translations.
Fix this by clearing the CPU from mm_cpumasks, then flushing all TLBs
before going offline.
These offlined CPU bits stuck in the cpumask also prevents the cpumask
from being trimmed back to local mode, which means continual broadcast
IPIs or TLBIEs are needed for TLB flushing. This patch prevents that
situation too.
A cast of many were involved in working this out, but in particular
Milton, Aneesh, Paul made key discoveries.
Fixes: 0cef77c7798a7 ("powerpc/64s/radix: flush remote CPUs out of single-threaded mm_cpumask")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Debugged-by: Milton Miller <miltonm@us.ibm.com>
Debugged-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Debugged-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20201126102530.691335-5-npiggin@gmail.com
2020-11-26 20:25:30 +10:00
|
|
|
cpumask_clear_cpu(cpu, mm_cpumask(mm)); \
|
|
|
|
|
} \
|
|
|
|
|
} while (0)
|
|
|
|
|
|
|
|
|
|
void cleanup_cpu_mmu_context(void);
|
|
|
|
|
#endif
|
|
|
|
|
|
2021-12-02 00:41:52 +10:00
|
|
|
#ifdef CONFIG_PPC_64S_HASH_MMU
|
2018-09-20 14:03:57 +05:30
|
|
|
static inline int get_user_context(mm_context_t *ctx, unsigned long ea)
|
2018-03-26 15:34:48 +05:30
|
|
|
{
|
|
|
|
|
int index = ea >> MAX_EA_BITS_PER_CONTEXT;
|
|
|
|
|
|
|
|
|
|
if (likely(index < ARRAY_SIZE(ctx->extended_id)))
|
|
|
|
|
return ctx->extended_id[index];
|
|
|
|
|
|
|
|
|
|
/* should never happen */
|
|
|
|
|
WARN_ON(1);
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline unsigned long get_user_vsid(mm_context_t *ctx,
|
|
|
|
|
unsigned long ea, int ssize)
|
|
|
|
|
{
|
2018-09-20 14:03:57 +05:30
|
|
|
unsigned long context = get_user_context(ctx, ea);
|
2018-03-26 15:34:48 +05:30
|
|
|
|
|
|
|
|
return get_vsid(context, ea, ssize);
|
|
|
|
|
}
|
2021-12-02 00:41:52 +10:00
|
|
|
#endif
|
2018-03-26 15:34:48 +05:30
|
|
|
|
2016-04-29 23:25:41 +10:00
|
|
|
#endif /* __ASSEMBLY__ */
|
|
|
|
|
#endif /* _ASM_POWERPC_BOOK3S_64_MMU_H_ */
|