License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 15:07:57 +01:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0 */
|
2005-04-16 15:20:36 -07:00
|
|
|
#ifndef _PARISC_PGTABLE_H
|
|
|
|
|
#define _PARISC_PGTABLE_H
|
|
|
|
|
|
2019-08-24 17:54:43 -07:00
|
|
|
#include <asm/page.h>
|
2019-12-04 16:54:12 -08:00
|
|
|
|
|
|
|
|
#if CONFIG_PGTABLE_LEVELS == 3
|
|
|
|
|
#include <asm-generic/pgtable-nopud.h>
|
|
|
|
|
#elif CONFIG_PGTABLE_LEVELS == 2
|
|
|
|
|
#include <asm-generic/pgtable-nopmd.h>
|
|
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
#include <asm/fixmap.h>
|
|
|
|
|
|
2025-03-14 08:09:56 +01:00
|
|
|
#ifndef __ASSEMBLER__
|
2005-04-16 15:20:36 -07:00
|
|
|
/*
|
|
|
|
|
* we simulate an x86-style page table for the linux mm code
|
|
|
|
|
*/
|
|
|
|
|
|
2007-10-18 23:40:25 -07:00
|
|
|
#include <linux/bitops.h>
|
2011-01-16 23:06:14 +01:00
|
|
|
#include <linux/spinlock.h>
|
2013-01-14 19:45:00 -05:00
|
|
|
#include <linux/mm_types.h>
|
2005-04-16 15:20:36 -07:00
|
|
|
#include <asm/processor.h>
|
|
|
|
|
#include <asm/cache.h>
|
|
|
|
|
|
2019-04-28 00:09:53 +02:00
|
|
|
/* This is for the serialization of PxTLB broadcasts. At least on the N class
|
|
|
|
|
* systems, only one PxTLB inter processor broadcast can be active at any one
|
2021-02-12 16:38:52 +01:00
|
|
|
* time on the Merced bus. */
|
2019-04-28 00:09:53 +02:00
|
|
|
extern spinlock_t pa_tlb_flush_lock;
|
|
|
|
|
#if defined(CONFIG_64BIT) && defined(CONFIG_SMP)
|
|
|
|
|
extern int pa_serialize_tlb_flushes;
|
|
|
|
|
#else
|
|
|
|
|
#define pa_serialize_tlb_flushes (0)
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#define purge_tlb_start(flags) do { \
|
|
|
|
|
if (pa_serialize_tlb_flushes) \
|
|
|
|
|
spin_lock_irqsave(&pa_tlb_flush_lock, flags); \
|
|
|
|
|
else \
|
|
|
|
|
local_irq_save(flags); \
|
|
|
|
|
} while (0)
|
|
|
|
|
#define purge_tlb_end(flags) do { \
|
|
|
|
|
if (pa_serialize_tlb_flushes) \
|
|
|
|
|
spin_unlock_irqrestore(&pa_tlb_flush_lock, flags); \
|
|
|
|
|
else \
|
|
|
|
|
local_irq_restore(flags); \
|
|
|
|
|
} while (0)
|
|
|
|
|
|
|
|
|
|
/* Purge data and instruction TLB entries. The TLB purge instructions
|
|
|
|
|
* are slow on SMP machines since the purge must be broadcast to all CPUs.
|
2015-07-01 17:18:37 -04:00
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
static inline void purge_tlb_entries(struct mm_struct *mm, unsigned long addr)
|
|
|
|
|
{
|
2019-04-28 00:09:53 +02:00
|
|
|
unsigned long flags;
|
|
|
|
|
|
|
|
|
|
purge_tlb_start(flags);
|
2022-02-17 15:26:27 +01:00
|
|
|
mtsp(mm->context.space_id, SR_TEMP1);
|
|
|
|
|
pdtlb(SR_TEMP1, addr);
|
|
|
|
|
pitlb(SR_TEMP1, addr);
|
2019-04-28 00:09:53 +02:00
|
|
|
purge_tlb_end(flags);
|
2015-07-01 17:18:37 -04:00
|
|
|
}
|
|
|
|
|
|
2021-11-08 16:48:16 -05:00
|
|
|
extern void __update_cache(pte_t pte);
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
/* Certain architectures need to do special things when PTEs
|
|
|
|
|
* within a page table are directly modified. Thus, the following
|
|
|
|
|
* hook is made available.
|
|
|
|
|
*/
|
2021-02-12 16:38:52 +01:00
|
|
|
#define set_pte(pteptr, pteval) \
|
|
|
|
|
do { \
|
|
|
|
|
*(pteptr) = (pteval); \
|
2021-11-08 16:48:16 -05:00
|
|
|
mb(); \
|
2021-02-12 16:38:52 +01:00
|
|
|
} while(0)
|
|
|
|
|
|
2025-03-14 08:09:56 +01:00
|
|
|
#endif /* !__ASSEMBLER__ */
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
#define pte_ERROR(e) \
|
|
|
|
|
printk("%s:%d: bad pte %08lx.\n", __FILE__, __LINE__, pte_val(e))
|
2019-12-04 16:54:12 -08:00
|
|
|
#if CONFIG_PGTABLE_LEVELS == 3
|
2005-04-16 15:20:36 -07:00
|
|
|
#define pmd_ERROR(e) \
|
|
|
|
|
printk("%s:%d: bad pmd %08lx.\n", __FILE__, __LINE__, (unsigned long)pmd_val(e))
|
2019-12-04 16:54:12 -08:00
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
#define pgd_ERROR(e) \
|
|
|
|
|
printk("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, (unsigned long)pgd_val(e))
|
|
|
|
|
|
2006-04-20 20:40:23 +00:00
|
|
|
/* This is the size of the initially mapped kernel memory */
|
2016-10-09 09:57:54 +02:00
|
|
|
#if defined(CONFIG_64BIT)
|
|
|
|
|
#define KERNEL_INITIAL_ORDER 26 /* 1<<26 = 64MB */
|
2015-11-20 11:17:27 +01:00
|
|
|
#else
|
2016-10-09 09:57:54 +02:00
|
|
|
#define KERNEL_INITIAL_ORDER 25 /* 1<<25 = 32MB */
|
2015-11-20 11:17:27 +01:00
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
#define KERNEL_INITIAL_SIZE (1 << KERNEL_INITIAL_ORDER)
|
|
|
|
|
|
2015-04-14 15:45:54 -07:00
|
|
|
#if CONFIG_PGTABLE_LEVELS == 3
|
2021-07-15 14:46:12 +01:00
|
|
|
#define PMD_TABLE_ORDER 1
|
2022-07-03 17:12:02 +03:00
|
|
|
#define PGD_TABLE_ORDER 0
|
2005-04-16 15:20:36 -07:00
|
|
|
#else
|
2022-07-03 17:12:02 +03:00
|
|
|
#define PGD_TABLE_ORDER 1
|
2005-04-16 15:20:36 -07:00
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
/* Definitions for 3rd level (we use PLD here for Page Lower directory
|
|
|
|
|
* because PTE_SHIFT is used lower down to mean shift that has to be
|
|
|
|
|
* done to get usable bits out of the PTE) */
|
|
|
|
|
#define PLD_SHIFT PAGE_SHIFT
|
|
|
|
|
#define PLD_SIZE PAGE_SIZE
|
|
|
|
|
#define BITS_PER_PTE (PAGE_SHIFT - BITS_PER_PTE_ENTRY)
|
|
|
|
|
#define PTRS_PER_PTE (1UL << BITS_PER_PTE)
|
|
|
|
|
|
|
|
|
|
/* Definitions for 2nd level */
|
2019-12-04 16:54:12 -08:00
|
|
|
#if CONFIG_PGTABLE_LEVELS == 3
|
2005-04-16 15:20:36 -07:00
|
|
|
#define PMD_SHIFT (PLD_SHIFT + BITS_PER_PTE)
|
|
|
|
|
#define PMD_SIZE (1UL << PMD_SHIFT)
|
|
|
|
|
#define PMD_MASK (~(PMD_SIZE-1))
|
2021-07-15 14:46:12 +01:00
|
|
|
#define BITS_PER_PMD (PAGE_SHIFT + PMD_TABLE_ORDER - BITS_PER_PMD_ENTRY)
|
2019-12-04 16:54:12 -08:00
|
|
|
#define PTRS_PER_PMD (1UL << BITS_PER_PMD)
|
2005-04-16 15:20:36 -07:00
|
|
|
#else
|
|
|
|
|
#define BITS_PER_PMD 0
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
/* Definitions for 1st level */
|
2019-12-04 16:54:12 -08:00
|
|
|
#define PGDIR_SHIFT (PLD_SHIFT + BITS_PER_PTE + BITS_PER_PMD)
|
2022-07-03 17:12:02 +03:00
|
|
|
#if (PGDIR_SHIFT + PAGE_SHIFT + PGD_TABLE_ORDER - BITS_PER_PGD_ENTRY) > BITS_PER_LONG
|
parisc: fix usage of 32bit PTE page table entries on 32bit kernels
This patch fixes a long outstanding bug on 32bit parisc linux kernels
which prevented us from using 32bit PTE table entries (instead of 64bit
entries of which 32bit were unused).
The problem was caused by this assembler statement in the L2_ptep
macro in arch/parisc/kernel/entry.S:447:
EXTR \va,31-ASM_PGDIR_SHIFT,ASM_BITS_PER_PGD,\index
which expanded to
extrw,u r8,9,11,r1
and which has undefined behavior since the length value (11) extends
beyond the leftmost bit (11-1 > 9).
Interestingly PA2.0 processors seem to don't care and just zero-extend
the value, while PA1.1 processors don't.
Fix this problem by detecting an address space overflow with ASM_BITS_PER_PGD
and adjusting it accordingly. To prevent such problems in the future,
some compile time sanity checks in arch/parisc/mm/init.c were added.
Since the page table now only consumes half of it's old size, we can
use the freed memory to harmonize 32- and 64bit kernels and let both
map 16MB for the initial page table.
Signed-off-by: Helge Deller <deller@gmx.de>
Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
2009-01-18 19:16:16 +01:00
|
|
|
#define BITS_PER_PGD (BITS_PER_LONG - PGDIR_SHIFT)
|
|
|
|
|
#else
|
2022-07-03 17:12:02 +03:00
|
|
|
#define BITS_PER_PGD (PAGE_SHIFT + PGD_TABLE_ORDER - BITS_PER_PGD_ENTRY)
|
parisc: fix usage of 32bit PTE page table entries on 32bit kernels
This patch fixes a long outstanding bug on 32bit parisc linux kernels
which prevented us from using 32bit PTE table entries (instead of 64bit
entries of which 32bit were unused).
The problem was caused by this assembler statement in the L2_ptep
macro in arch/parisc/kernel/entry.S:447:
EXTR \va,31-ASM_PGDIR_SHIFT,ASM_BITS_PER_PGD,\index
which expanded to
extrw,u r8,9,11,r1
and which has undefined behavior since the length value (11) extends
beyond the leftmost bit (11-1 > 9).
Interestingly PA2.0 processors seem to don't care and just zero-extend
the value, while PA1.1 processors don't.
Fix this problem by detecting an address space overflow with ASM_BITS_PER_PGD
and adjusting it accordingly. To prevent such problems in the future,
some compile time sanity checks in arch/parisc/mm/init.c were added.
Since the page table now only consumes half of it's old size, we can
use the freed memory to harmonize 32- and 64bit kernels and let both
map 16MB for the initial page table.
Signed-off-by: Helge Deller <deller@gmx.de>
Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
2009-01-18 19:16:16 +01:00
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
#define PGDIR_SIZE (1UL << PGDIR_SHIFT)
|
|
|
|
|
#define PGDIR_MASK (~(PGDIR_SIZE-1))
|
|
|
|
|
#define PTRS_PER_PGD (1UL << BITS_PER_PGD)
|
|
|
|
|
#define USER_PTRS_PER_PGD PTRS_PER_PGD
|
|
|
|
|
|
parisc: fix usage of 32bit PTE page table entries on 32bit kernels
This patch fixes a long outstanding bug on 32bit parisc linux kernels
which prevented us from using 32bit PTE table entries (instead of 64bit
entries of which 32bit were unused).
The problem was caused by this assembler statement in the L2_ptep
macro in arch/parisc/kernel/entry.S:447:
EXTR \va,31-ASM_PGDIR_SHIFT,ASM_BITS_PER_PGD,\index
which expanded to
extrw,u r8,9,11,r1
and which has undefined behavior since the length value (11) extends
beyond the leftmost bit (11-1 > 9).
Interestingly PA2.0 processors seem to don't care and just zero-extend
the value, while PA1.1 processors don't.
Fix this problem by detecting an address space overflow with ASM_BITS_PER_PGD
and adjusting it accordingly. To prevent such problems in the future,
some compile time sanity checks in arch/parisc/mm/init.c were added.
Since the page table now only consumes half of it's old size, we can
use the freed memory to harmonize 32- and 64bit kernels and let both
map 16MB for the initial page table.
Signed-off-by: Helge Deller <deller@gmx.de>
Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
2009-01-18 19:16:16 +01:00
|
|
|
#ifdef CONFIG_64BIT
|
2005-04-16 15:20:36 -07:00
|
|
|
#define MAX_ADDRBITS (PGDIR_SHIFT + BITS_PER_PGD)
|
|
|
|
|
#define MAX_ADDRESS (1UL << MAX_ADDRBITS)
|
2006-04-20 20:40:23 +00:00
|
|
|
#define SPACEID_SHIFT (MAX_ADDRBITS - 32)
|
parisc: fix usage of 32bit PTE page table entries on 32bit kernels
This patch fixes a long outstanding bug on 32bit parisc linux kernels
which prevented us from using 32bit PTE table entries (instead of 64bit
entries of which 32bit were unused).
The problem was caused by this assembler statement in the L2_ptep
macro in arch/parisc/kernel/entry.S:447:
EXTR \va,31-ASM_PGDIR_SHIFT,ASM_BITS_PER_PGD,\index
which expanded to
extrw,u r8,9,11,r1
and which has undefined behavior since the length value (11) extends
beyond the leftmost bit (11-1 > 9).
Interestingly PA2.0 processors seem to don't care and just zero-extend
the value, while PA1.1 processors don't.
Fix this problem by detecting an address space overflow with ASM_BITS_PER_PGD
and adjusting it accordingly. To prevent such problems in the future,
some compile time sanity checks in arch/parisc/mm/init.c were added.
Since the page table now only consumes half of it's old size, we can
use the freed memory to harmonize 32- and 64bit kernels and let both
map 16MB for the initial page table.
Signed-off-by: Helge Deller <deller@gmx.de>
Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
2009-01-18 19:16:16 +01:00
|
|
|
#else
|
|
|
|
|
#define MAX_ADDRBITS (BITS_PER_LONG)
|
2022-04-05 21:28:37 +02:00
|
|
|
#define MAX_ADDRESS (1ULL << MAX_ADDRBITS)
|
parisc: fix usage of 32bit PTE page table entries on 32bit kernels
This patch fixes a long outstanding bug on 32bit parisc linux kernels
which prevented us from using 32bit PTE table entries (instead of 64bit
entries of which 32bit were unused).
The problem was caused by this assembler statement in the L2_ptep
macro in arch/parisc/kernel/entry.S:447:
EXTR \va,31-ASM_PGDIR_SHIFT,ASM_BITS_PER_PGD,\index
which expanded to
extrw,u r8,9,11,r1
and which has undefined behavior since the length value (11) extends
beyond the leftmost bit (11-1 > 9).
Interestingly PA2.0 processors seem to don't care and just zero-extend
the value, while PA1.1 processors don't.
Fix this problem by detecting an address space overflow with ASM_BITS_PER_PGD
and adjusting it accordingly. To prevent such problems in the future,
some compile time sanity checks in arch/parisc/mm/init.c were added.
Since the page table now only consumes half of it's old size, we can
use the freed memory to harmonize 32- and 64bit kernels and let both
map 16MB for the initial page table.
Signed-off-by: Helge Deller <deller@gmx.de>
Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
2009-01-18 19:16:16 +01:00
|
|
|
#define SPACEID_SHIFT 0
|
|
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
/* This calculates the number of initial pages we need for the initial
|
|
|
|
|
* page tables */
|
2022-12-14 22:17:57 +01:00
|
|
|
#if (KERNEL_INITIAL_ORDER) >= (PLD_SHIFT + BITS_PER_PTE)
|
|
|
|
|
# define PT_INITIAL (1 << (KERNEL_INITIAL_ORDER - PLD_SHIFT - BITS_PER_PTE))
|
2006-04-20 20:40:23 +00:00
|
|
|
#else
|
|
|
|
|
# define PT_INITIAL (1) /* all initial PTEs fit into one page */
|
|
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* pgd entries used up by user/kernel:
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/* NB: The tlb miss handlers make certain assumptions about the order */
|
|
|
|
|
/* of the following bits, so be careful (One example, bits 25-31 */
|
|
|
|
|
/* are moved together in one instruction). */
|
|
|
|
|
|
|
|
|
|
#define _PAGE_READ_BIT 31 /* (0x001) read access allowed */
|
|
|
|
|
#define _PAGE_WRITE_BIT 30 /* (0x002) write access allowed */
|
|
|
|
|
#define _PAGE_EXEC_BIT 29 /* (0x004) execute access allowed */
|
|
|
|
|
#define _PAGE_GATEWAY_BIT 28 /* (0x008) privilege promotion allowed */
|
|
|
|
|
#define _PAGE_DMB_BIT 27 /* (0x010) Data Memory Break enable (B bit) */
|
|
|
|
|
#define _PAGE_DIRTY_BIT 26 /* (0x020) Page Dirty (D bit) */
|
|
|
|
|
#define _PAGE_REFTRAP_BIT 25 /* (0x040) Page Ref. Trap enable (T bit) */
|
|
|
|
|
#define _PAGE_NO_CACHE_BIT 24 /* (0x080) Uncached Page (U bit) */
|
|
|
|
|
#define _PAGE_ACCESSED_BIT 23 /* (0x100) Software: Page Accessed */
|
|
|
|
|
#define _PAGE_PRESENT_BIT 22 /* (0x200) Software: translation valid */
|
2015-11-20 15:46:52 +01:00
|
|
|
#define _PAGE_HPAGE_BIT 21 /* (0x400) Software: Huge Page */
|
2005-04-16 15:20:36 -07:00
|
|
|
#define _PAGE_USER_BIT 20 /* (0x800) Software: User accessible page */
|
2022-10-14 10:18:53 +02:00
|
|
|
#ifdef CONFIG_HUGETLB_PAGE
|
|
|
|
|
#define _PAGE_SPECIAL_BIT _PAGE_DMB_BIT /* DMB feature is currently unused */
|
|
|
|
|
#else
|
|
|
|
|
#define _PAGE_SPECIAL_BIT _PAGE_HPAGE_BIT /* use unused HUGE PAGE bit */
|
|
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
/* N.B. The bits are defined in terms of a 32 bit word above, so the */
|
|
|
|
|
/* following macro is ok for both 32 and 64 bit. */
|
|
|
|
|
|
|
|
|
|
#define xlate_pabit(x) (31 - x)
|
|
|
|
|
|
|
|
|
|
/* this defines the shift to the usable bits in the PTE it is set so
|
|
|
|
|
* that the valid bits _PAGE_PRESENT_BIT and _PAGE_USER_BIT are set
|
|
|
|
|
* to zero */
|
|
|
|
|
#define PTE_SHIFT xlate_pabit(_PAGE_USER_BIT)
|
|
|
|
|
|
2006-04-20 20:40:23 +00:00
|
|
|
/* PFN_PTE_SHIFT defines the shift of a PTE value to access the PFN field */
|
|
|
|
|
#define PFN_PTE_SHIFT 12
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
#define _PAGE_READ (1 << xlate_pabit(_PAGE_READ_BIT))
|
|
|
|
|
#define _PAGE_WRITE (1 << xlate_pabit(_PAGE_WRITE_BIT))
|
|
|
|
|
#define _PAGE_RW (_PAGE_READ | _PAGE_WRITE)
|
|
|
|
|
#define _PAGE_EXEC (1 << xlate_pabit(_PAGE_EXEC_BIT))
|
|
|
|
|
#define _PAGE_GATEWAY (1 << xlate_pabit(_PAGE_GATEWAY_BIT))
|
|
|
|
|
#define _PAGE_DMB (1 << xlate_pabit(_PAGE_DMB_BIT))
|
|
|
|
|
#define _PAGE_DIRTY (1 << xlate_pabit(_PAGE_DIRTY_BIT))
|
|
|
|
|
#define _PAGE_REFTRAP (1 << xlate_pabit(_PAGE_REFTRAP_BIT))
|
|
|
|
|
#define _PAGE_NO_CACHE (1 << xlate_pabit(_PAGE_NO_CACHE_BIT))
|
|
|
|
|
#define _PAGE_ACCESSED (1 << xlate_pabit(_PAGE_ACCESSED_BIT))
|
|
|
|
|
#define _PAGE_PRESENT (1 << xlate_pabit(_PAGE_PRESENT_BIT))
|
2015-11-20 15:46:52 +01:00
|
|
|
#define _PAGE_HUGE (1 << xlate_pabit(_PAGE_HPAGE_BIT))
|
2005-04-16 15:20:36 -07:00
|
|
|
#define _PAGE_USER (1 << xlate_pabit(_PAGE_USER_BIT))
|
2022-10-14 10:18:53 +02:00
|
|
|
#define _PAGE_SPECIAL (1 << xlate_pabit(_PAGE_SPECIAL_BIT))
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2018-09-20 22:31:48 -04:00
|
|
|
#define _PAGE_TABLE (_PAGE_PRESENT | _PAGE_READ | _PAGE_WRITE | _PAGE_DIRTY | _PAGE_ACCESSED)
|
2021-12-08 11:06:52 +01:00
|
|
|
#define _PAGE_CHG_MASK (PAGE_MASK | _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_SPECIAL)
|
2011-04-14 18:25:21 -05:00
|
|
|
#define _PAGE_KERNEL_RO (_PAGE_PRESENT | _PAGE_READ | _PAGE_DIRTY | _PAGE_ACCESSED)
|
|
|
|
|
#define _PAGE_KERNEL_EXEC (_PAGE_KERNEL_RO | _PAGE_EXEC)
|
|
|
|
|
#define _PAGE_KERNEL_RWX (_PAGE_KERNEL_EXEC | _PAGE_WRITE)
|
|
|
|
|
#define _PAGE_KERNEL (_PAGE_KERNEL_RO | _PAGE_WRITE)
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2023-01-13 18:10:16 +01:00
|
|
|
/* We borrow bit 23 to store the exclusive marker in swap PTEs. */
|
|
|
|
|
#define _PAGE_SWP_EXCLUSIVE _PAGE_ACCESSED
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
/* The pgd/pmd contains a ptr (in phys addr space); since all pgds/pmds
|
|
|
|
|
* are page-aligned, we don't care about the PAGE_OFFSET bits, except
|
|
|
|
|
* for a few meta-information bits, so we shift the address to be
|
2006-04-20 20:40:23 +00:00
|
|
|
* able to effectively address 40/42/44-bits of physical address space
|
|
|
|
|
* depending on 4k/16k/64k PAGE_SIZE */
|
2005-04-16 15:20:36 -07:00
|
|
|
#define _PxD_PRESENT_BIT 31
|
2021-02-12 16:38:52 +01:00
|
|
|
#define _PxD_VALID_BIT 30
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
#define PxD_FLAG_PRESENT (1 << xlate_pabit(_PxD_PRESENT_BIT))
|
|
|
|
|
#define PxD_FLAG_VALID (1 << xlate_pabit(_PxD_VALID_BIT))
|
|
|
|
|
#define PxD_FLAG_MASK (0xf)
|
|
|
|
|
#define PxD_FLAG_SHIFT (4)
|
2015-11-20 15:46:52 +01:00
|
|
|
#define PxD_VALUE_SHIFT (PFN_PTE_SHIFT-PxD_FLAG_SHIFT)
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2025-03-14 08:09:56 +01:00
|
|
|
#ifndef __ASSEMBLER__
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2018-09-20 22:31:48 -04:00
|
|
|
#define PAGE_NONE __pgprot(_PAGE_PRESENT | _PAGE_USER)
|
|
|
|
|
#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_READ | _PAGE_WRITE)
|
2005-04-16 15:20:36 -07:00
|
|
|
/* Others seem to make this executable, I don't know if that's correct
|
|
|
|
|
or not. The stack is mapped this way though so this is necessary
|
|
|
|
|
in the short term - dhd@linuxcare.com, 2000-08-08 */
|
2018-09-20 22:31:48 -04:00
|
|
|
#define PAGE_READONLY __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_READ)
|
|
|
|
|
#define PAGE_WRITEONLY __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_WRITE)
|
|
|
|
|
#define PAGE_EXECREAD __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_READ | _PAGE_EXEC)
|
2005-04-16 15:20:36 -07:00
|
|
|
#define PAGE_COPY PAGE_EXECREAD
|
2018-09-20 22:31:48 -04:00
|
|
|
#define PAGE_RWX __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)
|
2005-04-16 15:20:36 -07:00
|
|
|
#define PAGE_KERNEL __pgprot(_PAGE_KERNEL)
|
2011-04-14 18:25:21 -05:00
|
|
|
#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL_EXEC)
|
|
|
|
|
#define PAGE_KERNEL_RWX __pgprot(_PAGE_KERNEL_RWX)
|
|
|
|
|
#define PAGE_KERNEL_RO __pgprot(_PAGE_KERNEL_RO)
|
2005-04-16 15:20:36 -07:00
|
|
|
#define PAGE_KERNEL_UNC __pgprot(_PAGE_KERNEL | _PAGE_NO_CACHE)
|
2018-09-20 22:31:48 -04:00
|
|
|
#define PAGE_GATEWAY __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_GATEWAY| _PAGE_READ)
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We could have an execute only page using "gateway - promote to priv
|
|
|
|
|
* level 3", but that is kind of silly. So, the way things are defined
|
|
|
|
|
* now, we must always have read permission for pages with execute
|
|
|
|
|
* permission. For the fun of it we'll go ahead and support write only
|
|
|
|
|
* pages.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*xwr*/
|
2006-04-20 20:40:23 +00:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
extern pgd_t swapper_pg_dir[]; /* declared in init_task.c */
|
|
|
|
|
|
|
|
|
|
/* initial page tables for 0-8MB for kernel */
|
|
|
|
|
|
|
|
|
|
extern pte_t pg0[];
|
|
|
|
|
|
|
|
|
|
/* zero page used for uninitialized stuff */
|
|
|
|
|
|
|
|
|
|
extern unsigned long *empty_zero_page;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* ZERO_PAGE is a global shared page that is always zero: used
|
|
|
|
|
* for zero-mapped memory areas etc..
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#define ZERO_PAGE(vaddr) (virt_to_page(empty_zero_page))
|
|
|
|
|
|
2010-12-22 10:24:36 -06:00
|
|
|
#define pte_none(x) (pte_val(x) == 0)
|
2005-04-16 15:20:36 -07:00
|
|
|
#define pte_present(x) (pte_val(x) & _PAGE_PRESENT)
|
2021-11-08 16:48:16 -05:00
|
|
|
#define pte_user(x) (pte_val(x) & _PAGE_USER)
|
2025-07-21 16:06:21 -04:00
|
|
|
#define pte_clear(mm, addr, xp) set_pte_at((mm), (addr), (xp), __pte(0))
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
#define pmd_flag(x) (pmd_val(x) & PxD_FLAG_MASK)
|
|
|
|
|
#define pmd_address(x) ((unsigned long)(pmd_val(x) &~ PxD_FLAG_MASK) << PxD_VALUE_SHIFT)
|
2019-12-04 16:54:12 -08:00
|
|
|
#define pud_flag(x) (pud_val(x) & PxD_FLAG_MASK)
|
|
|
|
|
#define pud_address(x) ((unsigned long)(pud_val(x) &~ PxD_FLAG_MASK) << PxD_VALUE_SHIFT)
|
2005-04-16 15:20:36 -07:00
|
|
|
#define pgd_flag(x) (pgd_val(x) & PxD_FLAG_MASK)
|
|
|
|
|
#define pgd_address(x) ((unsigned long)(pgd_val(x) &~ PxD_FLAG_MASK) << PxD_VALUE_SHIFT)
|
|
|
|
|
|
|
|
|
|
#define pmd_none(x) (!pmd_val(x))
|
|
|
|
|
#define pmd_bad(x) (!(pmd_flag(x) & PxD_FLAG_VALID))
|
|
|
|
|
#define pmd_present(x) (pmd_flag(x) & PxD_FLAG_PRESENT)
|
|
|
|
|
static inline void pmd_clear(pmd_t *pmd) {
|
2019-12-04 16:54:12 -08:00
|
|
|
set_pmd(pmd, __pmd(0));
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-14 15:45:54 -07:00
|
|
|
#if CONFIG_PGTABLE_LEVELS == 3
|
2021-07-07 18:09:53 -07:00
|
|
|
#define pud_pgtable(pud) ((pmd_t *) __va(pud_address(pud)))
|
|
|
|
|
#define pud_page(pud) virt_to_page((void *)pud_pgtable(pud))
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
/* For 64 bit we have three level tables */
|
|
|
|
|
|
2019-12-04 16:54:12 -08:00
|
|
|
#define pud_none(x) (!pud_val(x))
|
|
|
|
|
#define pud_bad(x) (!(pud_flag(x) & PxD_FLAG_VALID))
|
|
|
|
|
#define pud_present(x) (pud_flag(x) & PxD_FLAG_PRESENT)
|
|
|
|
|
static inline void pud_clear(pud_t *pud) {
|
|
|
|
|
set_pud(pud, __pud(0));
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The following only work if pte_present() is true.
|
|
|
|
|
* Undefined behaviour if not..
|
|
|
|
|
*/
|
2007-10-16 14:24:59 -07:00
|
|
|
static inline int pte_dirty(pte_t pte) { return pte_val(pte) & _PAGE_DIRTY; }
|
|
|
|
|
static inline int pte_young(pte_t pte) { return pte_val(pte) & _PAGE_ACCESSED; }
|
|
|
|
|
static inline int pte_write(pte_t pte) { return pte_val(pte) & _PAGE_WRITE; }
|
2021-12-08 11:06:52 +01:00
|
|
|
static inline int pte_special(pte_t pte) { return pte_val(pte) & _PAGE_SPECIAL; }
|
2007-10-16 14:24:59 -07:00
|
|
|
|
|
|
|
|
static inline pte_t pte_mkclean(pte_t pte) { pte_val(pte) &= ~_PAGE_DIRTY; return pte; }
|
|
|
|
|
static inline pte_t pte_mkold(pte_t pte) { pte_val(pte) &= ~_PAGE_ACCESSED; return pte; }
|
|
|
|
|
static inline pte_t pte_wrprotect(pte_t pte) { pte_val(pte) &= ~_PAGE_WRITE; return pte; }
|
|
|
|
|
static inline pte_t pte_mkdirty(pte_t pte) { pte_val(pte) |= _PAGE_DIRTY; return pte; }
|
|
|
|
|
static inline pte_t pte_mkyoung(pte_t pte) { pte_val(pte) |= _PAGE_ACCESSED; return pte; }
|
2023-06-12 17:10:27 -07:00
|
|
|
static inline pte_t pte_mkwrite_novma(pte_t pte) { pte_val(pte) |= _PAGE_WRITE; return pte; }
|
2021-12-08 11:06:52 +01:00
|
|
|
static inline pte_t pte_mkspecial(pte_t pte) { pte_val(pte) |= _PAGE_SPECIAL; return pte; }
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2015-11-20 15:46:52 +01:00
|
|
|
/*
|
|
|
|
|
* Huge pte definitions.
|
|
|
|
|
*/
|
|
|
|
|
#ifdef CONFIG_HUGETLB_PAGE
|
|
|
|
|
#define pte_huge(pte) (pte_val(pte) & _PAGE_HUGE)
|
2015-12-06 21:25:20 +01:00
|
|
|
#define pte_mkhuge(pte) (__pte(pte_val(pte) | \
|
|
|
|
|
(parisc_requires_coherency() ? 0 : _PAGE_HUGE)))
|
2015-11-20 15:46:52 +01:00
|
|
|
#else
|
|
|
|
|
#define pte_huge(pte) (0)
|
|
|
|
|
#define pte_mkhuge(pte) (pte)
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
#define __mk_pte(addr,pgprot) \
|
|
|
|
|
({ \
|
|
|
|
|
pte_t __pte; \
|
|
|
|
|
\
|
2006-04-20 20:40:23 +00:00
|
|
|
pte_val(__pte) = ((((addr)>>PAGE_SHIFT)<<PFN_PTE_SHIFT) + pgprot_val(pgprot)); \
|
2005-04-16 15:20:36 -07:00
|
|
|
\
|
|
|
|
|
__pte; \
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
static inline pte_t pfn_pte(unsigned long pfn, pgprot_t pgprot)
|
|
|
|
|
{
|
|
|
|
|
pte_t pte;
|
2006-04-20 20:40:23 +00:00
|
|
|
pte_val(pte) = (pfn << PFN_PTE_SHIFT) | pgprot_val(pgprot);
|
2005-04-16 15:20:36 -07:00
|
|
|
return pte;
|
|
|
|
|
}
|
|
|
|
|
|
2007-10-16 14:24:59 -07:00
|
|
|
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
|
2005-04-16 15:20:36 -07:00
|
|
|
{ pte_val(pte) = (pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot); return pte; }
|
|
|
|
|
|
|
|
|
|
/* Permanent address of a page. On parisc we don't have highmem. */
|
|
|
|
|
|
2006-04-20 20:40:23 +00:00
|
|
|
#define pte_pfn(x) (pte_val(x) >> PFN_PTE_SHIFT)
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
#define pte_page(pte) (pfn_to_page(pte_pfn(pte)))
|
|
|
|
|
|
2020-06-08 21:33:10 -07:00
|
|
|
static inline unsigned long pmd_page_vaddr(pmd_t pmd)
|
|
|
|
|
{
|
|
|
|
|
return ((unsigned long) __va(pmd_address(pmd)));
|
|
|
|
|
}
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2022-02-04 13:49:20 -05:00
|
|
|
#define pmd_pfn(pmd) (pmd_address(pmd) >> PAGE_SHIFT)
|
2005-04-16 15:20:36 -07:00
|
|
|
#define __pmd_page(pmd) ((unsigned long) __va(pmd_address(pmd)))
|
|
|
|
|
#define pmd_page(pmd) virt_to_page((void *)__pmd_page(pmd))
|
|
|
|
|
|
|
|
|
|
/* Find an entry in the second-level page table.. */
|
|
|
|
|
|
|
|
|
|
extern void paging_init (void);
|
|
|
|
|
|
2023-08-02 16:13:48 +01:00
|
|
|
static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
|
|
|
|
|
pte_t *ptep, pte_t pte, unsigned int nr)
|
|
|
|
|
{
|
|
|
|
|
if (pte_present(pte) && pte_user(pte))
|
|
|
|
|
__update_cache(pte);
|
|
|
|
|
for (;;) {
|
|
|
|
|
*ptep = pte;
|
|
|
|
|
purge_tlb_entries(mm, addr);
|
|
|
|
|
if (--nr == 0)
|
|
|
|
|
break;
|
|
|
|
|
ptep++;
|
|
|
|
|
pte_val(pte) += 1 << PFN_PTE_SHIFT;
|
|
|
|
|
addr += PAGE_SIZE;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
#define set_ptes set_ptes
|
2025-07-21 16:06:21 -04:00
|
|
|
#define set_pte_at(mm, addr, ptep, pte) set_ptes(mm, addr, ptep, pte, 1)
|
2023-08-02 16:13:48 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
/* Used for deferring calls to flush_dcache_page() */
|
|
|
|
|
|
|
|
|
|
#define PG_dcache_dirty PG_arch_1
|
|
|
|
|
|
2023-08-02 16:13:48 +01:00
|
|
|
#define update_mmu_cache_range(vmf, vma, addr, ptep, nr) __update_cache(*ptep)
|
|
|
|
|
#define update_mmu_cache(vma, addr, ptep) __update_cache(*ptep)
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2023-01-13 18:10:16 +01:00
|
|
|
/*
|
|
|
|
|
* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that
|
|
|
|
|
* are !pte_none() && !pte_present().
|
|
|
|
|
*
|
|
|
|
|
* Format of swap PTEs (32bit):
|
|
|
|
|
*
|
|
|
|
|
* 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
|
|
|
|
|
* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
|
|
|
|
* <---------------- offset -----------------> P E <ofs> < type ->
|
|
|
|
|
*
|
|
|
|
|
* E is the exclusive marker that is not stored in swap entries.
|
|
|
|
|
* _PAGE_PRESENT (P) must be 0.
|
|
|
|
|
*
|
|
|
|
|
* For the 64bit version, the offset is extended by 32bit.
|
|
|
|
|
*/
|
2005-04-16 15:20:36 -07:00
|
|
|
#define __swp_type(x) ((x).val & 0x1f)
|
2023-05-13 22:30:06 +02:00
|
|
|
#define __swp_offset(x) ( (((x).val >> 5) & 0x7) | \
|
|
|
|
|
(((x).val >> 10) << 3) )
|
2023-01-13 18:10:16 +01:00
|
|
|
#define __swp_entry(type, offset) ((swp_entry_t) { \
|
|
|
|
|
((type) & 0x1f) | \
|
2023-05-13 22:30:06 +02:00
|
|
|
((offset & 0x7) << 5) | \
|
|
|
|
|
((offset >> 3) << 10) })
|
2005-04-16 15:20:36 -07:00
|
|
|
#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
|
|
|
|
|
#define __swp_entry_to_pte(x) ((pte_t) { (x).val })
|
|
|
|
|
|
2025-02-18 18:55:14 +01:00
|
|
|
static inline bool pte_swp_exclusive(pte_t pte)
|
2023-01-13 18:10:16 +01:00
|
|
|
{
|
|
|
|
|
return pte_val(pte) & _PAGE_SWP_EXCLUSIVE;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline pte_t pte_swp_mkexclusive(pte_t pte)
|
|
|
|
|
{
|
|
|
|
|
pte_val(pte) |= _PAGE_SWP_EXCLUSIVE;
|
|
|
|
|
return pte;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline pte_t pte_swp_clear_exclusive(pte_t pte)
|
|
|
|
|
{
|
|
|
|
|
pte_val(pte) &= ~_PAGE_SWP_EXCLUSIVE;
|
|
|
|
|
return pte;
|
|
|
|
|
}
|
|
|
|
|
|
parisc: Try to fix random segmentation faults in package builds
PA-RISC systems with PA8800 and PA8900 processors have had problems
with random segmentation faults for many years. Systems with earlier
processors are much more stable.
Systems with PA8800 and PA8900 processors have a large L2 cache which
needs per page flushing for decent performance when a large range is
flushed. The combined cache in these systems is also more sensitive to
non-equivalent aliases than the caches in earlier systems.
The majority of random segmentation faults that I have looked at
appear to be memory corruption in memory allocated using mmap and
malloc.
My first attempt at fixing the random faults didn't work. On
reviewing the cache code, I realized that there were two issues
which the existing code didn't handle correctly. Both relate
to cache move-in. Another issue is that the present bit in PTEs
is racy.
1) PA-RISC caches have a mind of their own and they can speculatively
load data and instructions for a page as long as there is a entry in
the TLB for the page which allows move-in. TLBs are local to each
CPU. Thus, the TLB entry for a page must be purged before flushing
the page. This is particularly important on SMP systems.
In some of the flush routines, the flush routine would be called
and then the TLB entry would be purged. This was because the flush
routine needed the TLB entry to do the flush.
2) My initial approach to trying the fix the random faults was to
try and use flush_cache_page_if_present for all flush operations.
This actually made things worse and led to a couple of hardware
lockups. It finally dawned on me that some lines weren't being
flushed because the pte check code was racy. This resulted in
random inequivalent mappings to physical pages.
The __flush_cache_page tmpalias flush sets up its own TLB entry
and it doesn't need the existing TLB entry. As long as we can find
the pte pointer for the vm page, we can get the pfn and physical
address of the page. We can also purge the TLB entry for the page
before doing the flush. Further, __flush_cache_page uses a special
TLB entry that inhibits cache move-in.
When switching page mappings, we need to ensure that lines are
removed from the cache. It is not sufficient to just flush the
lines to memory as they may come back.
This made it clear that we needed to implement all the required
flush operations using tmpalias routines. This includes flushes
for user and kernel pages.
After modifying the code to use tmpalias flushes, it became clear
that the random segmentation faults were not fully resolved. The
frequency of faults was worse on systems with a 64 MB L2 (PA8900)
and systems with more CPUs (rp4440).
The warning that I added to flush_cache_page_if_present to detect
pages that couldn't be flushed triggered frequently on some systems.
Helge and I looked at the pages that couldn't be flushed and found
that the PTE was either cleared or for a swap page. Ignoring pages
that were swapped out seemed okay but pages with cleared PTEs seemed
problematic.
I looked at routines related to pte_clear and noticed ptep_clear_flush.
The default implementation just flushes the TLB entry. However, it was
obvious that on parisc we need to flush the cache page as well. If
we don't flush the cache page, stale lines will be left in the cache
and cause random corruption. Once a PTE is cleared, there is no way
to find the physical address associated with the PTE and flush the
associated page at a later time.
I implemented an updated change with a parisc specific version of
ptep_clear_flush. It fixed the random data corruption on Helge's rp4440
and rp3440, as well as on my c8000.
At this point, I realized that I could restore the code where we only
flush in flush_cache_page_if_present if the page has been accessed.
However, for this, we also need to flush the cache when the accessed
bit is cleared in ptep_clear_flush_young to keep things synchronized.
The default implementation only flushes the TLB entry.
Other changes in this version are:
1) Implement parisc specific version of ptep_get. It's identical to
default but needed in arch/parisc/include/asm/pgtable.h.
2) Revise parisc implementation of ptep_test_and_clear_young to use
ptep_get (READ_ONCE).
3) Drop parisc implementation of ptep_get_and_clear. We can use default.
4) Revise flush_kernel_vmap_range and invalidate_kernel_vmap_range to
use full data cache flush.
5) Move flush_cache_vmap and flush_cache_vunmap to cache.c. Handle
VM_IOREMAP case in flush_cache_vmap.
At this time, I don't know whether it is better to always flush when
the PTE present bit is set or when both the accessed and present bits
are set. The later saves flushing pages that haven't been accessed,
but we need to flush in ptep_clear_flush_young. It also needs a page
table lookup to find the PTE pointer. The lpa instruction only needs
a page table lookup when the PTE entry isn't in the TLB.
We don't atomically handle setting and clearing the _PAGE_ACCESSED bit.
If we miss an update, we may miss a flush and the cache may get corrupted.
Whether the current code is effectively atomic depends on process control.
When CONFIG_FLUSH_PAGE_ACCESSED is set to zero, the page will eventually
be flushed when the PTE is cleared or in flush_cache_page_if_present. The
_PAGE_ACCESSED bit is not used, so the problem is avoided.
The flush method can be selected using the CONFIG_FLUSH_PAGE_ACCESSED
define in cache.c. The default is 0. I didn't see a large difference
in performance.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Cc: <stable@vger.kernel.org> # v6.6+
Signed-off-by: Helge Deller <deller@gmx.de>
2024-06-10 18:47:07 +00:00
|
|
|
static inline pte_t ptep_get(pte_t *ptep)
|
|
|
|
|
{
|
|
|
|
|
return READ_ONCE(*ptep);
|
|
|
|
|
}
|
|
|
|
|
#define ptep_get ptep_get
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
static inline int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
|
|
|
|
|
{
|
2013-04-23 22:42:07 +02:00
|
|
|
pte_t pte;
|
|
|
|
|
|
parisc: Try to fix random segmentation faults in package builds
PA-RISC systems with PA8800 and PA8900 processors have had problems
with random segmentation faults for many years. Systems with earlier
processors are much more stable.
Systems with PA8800 and PA8900 processors have a large L2 cache which
needs per page flushing for decent performance when a large range is
flushed. The combined cache in these systems is also more sensitive to
non-equivalent aliases than the caches in earlier systems.
The majority of random segmentation faults that I have looked at
appear to be memory corruption in memory allocated using mmap and
malloc.
My first attempt at fixing the random faults didn't work. On
reviewing the cache code, I realized that there were two issues
which the existing code didn't handle correctly. Both relate
to cache move-in. Another issue is that the present bit in PTEs
is racy.
1) PA-RISC caches have a mind of their own and they can speculatively
load data and instructions for a page as long as there is a entry in
the TLB for the page which allows move-in. TLBs are local to each
CPU. Thus, the TLB entry for a page must be purged before flushing
the page. This is particularly important on SMP systems.
In some of the flush routines, the flush routine would be called
and then the TLB entry would be purged. This was because the flush
routine needed the TLB entry to do the flush.
2) My initial approach to trying the fix the random faults was to
try and use flush_cache_page_if_present for all flush operations.
This actually made things worse and led to a couple of hardware
lockups. It finally dawned on me that some lines weren't being
flushed because the pte check code was racy. This resulted in
random inequivalent mappings to physical pages.
The __flush_cache_page tmpalias flush sets up its own TLB entry
and it doesn't need the existing TLB entry. As long as we can find
the pte pointer for the vm page, we can get the pfn and physical
address of the page. We can also purge the TLB entry for the page
before doing the flush. Further, __flush_cache_page uses a special
TLB entry that inhibits cache move-in.
When switching page mappings, we need to ensure that lines are
removed from the cache. It is not sufficient to just flush the
lines to memory as they may come back.
This made it clear that we needed to implement all the required
flush operations using tmpalias routines. This includes flushes
for user and kernel pages.
After modifying the code to use tmpalias flushes, it became clear
that the random segmentation faults were not fully resolved. The
frequency of faults was worse on systems with a 64 MB L2 (PA8900)
and systems with more CPUs (rp4440).
The warning that I added to flush_cache_page_if_present to detect
pages that couldn't be flushed triggered frequently on some systems.
Helge and I looked at the pages that couldn't be flushed and found
that the PTE was either cleared or for a swap page. Ignoring pages
that were swapped out seemed okay but pages with cleared PTEs seemed
problematic.
I looked at routines related to pte_clear and noticed ptep_clear_flush.
The default implementation just flushes the TLB entry. However, it was
obvious that on parisc we need to flush the cache page as well. If
we don't flush the cache page, stale lines will be left in the cache
and cause random corruption. Once a PTE is cleared, there is no way
to find the physical address associated with the PTE and flush the
associated page at a later time.
I implemented an updated change with a parisc specific version of
ptep_clear_flush. It fixed the random data corruption on Helge's rp4440
and rp3440, as well as on my c8000.
At this point, I realized that I could restore the code where we only
flush in flush_cache_page_if_present if the page has been accessed.
However, for this, we also need to flush the cache when the accessed
bit is cleared in ptep_clear_flush_young to keep things synchronized.
The default implementation only flushes the TLB entry.
Other changes in this version are:
1) Implement parisc specific version of ptep_get. It's identical to
default but needed in arch/parisc/include/asm/pgtable.h.
2) Revise parisc implementation of ptep_test_and_clear_young to use
ptep_get (READ_ONCE).
3) Drop parisc implementation of ptep_get_and_clear. We can use default.
4) Revise flush_kernel_vmap_range and invalidate_kernel_vmap_range to
use full data cache flush.
5) Move flush_cache_vmap and flush_cache_vunmap to cache.c. Handle
VM_IOREMAP case in flush_cache_vmap.
At this time, I don't know whether it is better to always flush when
the PTE present bit is set or when both the accessed and present bits
are set. The later saves flushing pages that haven't been accessed,
but we need to flush in ptep_clear_flush_young. It also needs a page
table lookup to find the PTE pointer. The lpa instruction only needs
a page table lookup when the PTE entry isn't in the TLB.
We don't atomically handle setting and clearing the _PAGE_ACCESSED bit.
If we miss an update, we may miss a flush and the cache may get corrupted.
Whether the current code is effectively atomic depends on process control.
When CONFIG_FLUSH_PAGE_ACCESSED is set to zero, the page will eventually
be flushed when the PTE is cleared or in flush_cache_page_if_present. The
_PAGE_ACCESSED bit is not used, so the problem is avoided.
The flush method can be selected using the CONFIG_FLUSH_PAGE_ACCESSED
define in cache.c. The default is 0. I didn't see a large difference
in performance.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Cc: <stable@vger.kernel.org> # v6.6+
Signed-off-by: Helge Deller <deller@gmx.de>
2024-06-10 18:47:07 +00:00
|
|
|
pte = ptep_get(ptep);
|
2013-04-23 22:42:07 +02:00
|
|
|
if (!pte_young(pte)) {
|
2005-04-16 15:20:36 -07:00
|
|
|
return 0;
|
2013-04-23 22:42:07 +02:00
|
|
|
}
|
2025-07-21 16:06:21 -04:00
|
|
|
set_pte_at(vma->vm_mm, addr, ptep, pte_mkold(pte));
|
2005-04-16 15:20:36 -07:00
|
|
|
return 1;
|
|
|
|
|
}
|
|
|
|
|
|
parisc: Try to fix random segmentation faults in package builds
PA-RISC systems with PA8800 and PA8900 processors have had problems
with random segmentation faults for many years. Systems with earlier
processors are much more stable.
Systems with PA8800 and PA8900 processors have a large L2 cache which
needs per page flushing for decent performance when a large range is
flushed. The combined cache in these systems is also more sensitive to
non-equivalent aliases than the caches in earlier systems.
The majority of random segmentation faults that I have looked at
appear to be memory corruption in memory allocated using mmap and
malloc.
My first attempt at fixing the random faults didn't work. On
reviewing the cache code, I realized that there were two issues
which the existing code didn't handle correctly. Both relate
to cache move-in. Another issue is that the present bit in PTEs
is racy.
1) PA-RISC caches have a mind of their own and they can speculatively
load data and instructions for a page as long as there is a entry in
the TLB for the page which allows move-in. TLBs are local to each
CPU. Thus, the TLB entry for a page must be purged before flushing
the page. This is particularly important on SMP systems.
In some of the flush routines, the flush routine would be called
and then the TLB entry would be purged. This was because the flush
routine needed the TLB entry to do the flush.
2) My initial approach to trying the fix the random faults was to
try and use flush_cache_page_if_present for all flush operations.
This actually made things worse and led to a couple of hardware
lockups. It finally dawned on me that some lines weren't being
flushed because the pte check code was racy. This resulted in
random inequivalent mappings to physical pages.
The __flush_cache_page tmpalias flush sets up its own TLB entry
and it doesn't need the existing TLB entry. As long as we can find
the pte pointer for the vm page, we can get the pfn and physical
address of the page. We can also purge the TLB entry for the page
before doing the flush. Further, __flush_cache_page uses a special
TLB entry that inhibits cache move-in.
When switching page mappings, we need to ensure that lines are
removed from the cache. It is not sufficient to just flush the
lines to memory as they may come back.
This made it clear that we needed to implement all the required
flush operations using tmpalias routines. This includes flushes
for user and kernel pages.
After modifying the code to use tmpalias flushes, it became clear
that the random segmentation faults were not fully resolved. The
frequency of faults was worse on systems with a 64 MB L2 (PA8900)
and systems with more CPUs (rp4440).
The warning that I added to flush_cache_page_if_present to detect
pages that couldn't be flushed triggered frequently on some systems.
Helge and I looked at the pages that couldn't be flushed and found
that the PTE was either cleared or for a swap page. Ignoring pages
that were swapped out seemed okay but pages with cleared PTEs seemed
problematic.
I looked at routines related to pte_clear and noticed ptep_clear_flush.
The default implementation just flushes the TLB entry. However, it was
obvious that on parisc we need to flush the cache page as well. If
we don't flush the cache page, stale lines will be left in the cache
and cause random corruption. Once a PTE is cleared, there is no way
to find the physical address associated with the PTE and flush the
associated page at a later time.
I implemented an updated change with a parisc specific version of
ptep_clear_flush. It fixed the random data corruption on Helge's rp4440
and rp3440, as well as on my c8000.
At this point, I realized that I could restore the code where we only
flush in flush_cache_page_if_present if the page has been accessed.
However, for this, we also need to flush the cache when the accessed
bit is cleared in ptep_clear_flush_young to keep things synchronized.
The default implementation only flushes the TLB entry.
Other changes in this version are:
1) Implement parisc specific version of ptep_get. It's identical to
default but needed in arch/parisc/include/asm/pgtable.h.
2) Revise parisc implementation of ptep_test_and_clear_young to use
ptep_get (READ_ONCE).
3) Drop parisc implementation of ptep_get_and_clear. We can use default.
4) Revise flush_kernel_vmap_range and invalidate_kernel_vmap_range to
use full data cache flush.
5) Move flush_cache_vmap and flush_cache_vunmap to cache.c. Handle
VM_IOREMAP case in flush_cache_vmap.
At this time, I don't know whether it is better to always flush when
the PTE present bit is set or when both the accessed and present bits
are set. The later saves flushing pages that haven't been accessed,
but we need to flush in ptep_clear_flush_young. It also needs a page
table lookup to find the PTE pointer. The lpa instruction only needs
a page table lookup when the PTE entry isn't in the TLB.
We don't atomically handle setting and clearing the _PAGE_ACCESSED bit.
If we miss an update, we may miss a flush and the cache may get corrupted.
Whether the current code is effectively atomic depends on process control.
When CONFIG_FLUSH_PAGE_ACCESSED is set to zero, the page will eventually
be flushed when the PTE is cleared or in flush_cache_page_if_present. The
_PAGE_ACCESSED bit is not used, so the problem is avoided.
The flush method can be selected using the CONFIG_FLUSH_PAGE_ACCESSED
define in cache.c. The default is 0. I didn't see a large difference
in performance.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Cc: <stable@vger.kernel.org> # v6.6+
Signed-off-by: Helge Deller <deller@gmx.de>
2024-06-10 18:47:07 +00:00
|
|
|
int ptep_clear_flush_young(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep);
|
|
|
|
|
pte_t ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep);
|
2005-04-16 15:20:36 -07:00
|
|
|
|
parisc: Try to fix random segmentation faults in package builds
PA-RISC systems with PA8800 and PA8900 processors have had problems
with random segmentation faults for many years. Systems with earlier
processors are much more stable.
Systems with PA8800 and PA8900 processors have a large L2 cache which
needs per page flushing for decent performance when a large range is
flushed. The combined cache in these systems is also more sensitive to
non-equivalent aliases than the caches in earlier systems.
The majority of random segmentation faults that I have looked at
appear to be memory corruption in memory allocated using mmap and
malloc.
My first attempt at fixing the random faults didn't work. On
reviewing the cache code, I realized that there were two issues
which the existing code didn't handle correctly. Both relate
to cache move-in. Another issue is that the present bit in PTEs
is racy.
1) PA-RISC caches have a mind of their own and they can speculatively
load data and instructions for a page as long as there is a entry in
the TLB for the page which allows move-in. TLBs are local to each
CPU. Thus, the TLB entry for a page must be purged before flushing
the page. This is particularly important on SMP systems.
In some of the flush routines, the flush routine would be called
and then the TLB entry would be purged. This was because the flush
routine needed the TLB entry to do the flush.
2) My initial approach to trying the fix the random faults was to
try and use flush_cache_page_if_present for all flush operations.
This actually made things worse and led to a couple of hardware
lockups. It finally dawned on me that some lines weren't being
flushed because the pte check code was racy. This resulted in
random inequivalent mappings to physical pages.
The __flush_cache_page tmpalias flush sets up its own TLB entry
and it doesn't need the existing TLB entry. As long as we can find
the pte pointer for the vm page, we can get the pfn and physical
address of the page. We can also purge the TLB entry for the page
before doing the flush. Further, __flush_cache_page uses a special
TLB entry that inhibits cache move-in.
When switching page mappings, we need to ensure that lines are
removed from the cache. It is not sufficient to just flush the
lines to memory as they may come back.
This made it clear that we needed to implement all the required
flush operations using tmpalias routines. This includes flushes
for user and kernel pages.
After modifying the code to use tmpalias flushes, it became clear
that the random segmentation faults were not fully resolved. The
frequency of faults was worse on systems with a 64 MB L2 (PA8900)
and systems with more CPUs (rp4440).
The warning that I added to flush_cache_page_if_present to detect
pages that couldn't be flushed triggered frequently on some systems.
Helge and I looked at the pages that couldn't be flushed and found
that the PTE was either cleared or for a swap page. Ignoring pages
that were swapped out seemed okay but pages with cleared PTEs seemed
problematic.
I looked at routines related to pte_clear and noticed ptep_clear_flush.
The default implementation just flushes the TLB entry. However, it was
obvious that on parisc we need to flush the cache page as well. If
we don't flush the cache page, stale lines will be left in the cache
and cause random corruption. Once a PTE is cleared, there is no way
to find the physical address associated with the PTE and flush the
associated page at a later time.
I implemented an updated change with a parisc specific version of
ptep_clear_flush. It fixed the random data corruption on Helge's rp4440
and rp3440, as well as on my c8000.
At this point, I realized that I could restore the code where we only
flush in flush_cache_page_if_present if the page has been accessed.
However, for this, we also need to flush the cache when the accessed
bit is cleared in ptep_clear_flush_young to keep things synchronized.
The default implementation only flushes the TLB entry.
Other changes in this version are:
1) Implement parisc specific version of ptep_get. It's identical to
default but needed in arch/parisc/include/asm/pgtable.h.
2) Revise parisc implementation of ptep_test_and_clear_young to use
ptep_get (READ_ONCE).
3) Drop parisc implementation of ptep_get_and_clear. We can use default.
4) Revise flush_kernel_vmap_range and invalidate_kernel_vmap_range to
use full data cache flush.
5) Move flush_cache_vmap and flush_cache_vunmap to cache.c. Handle
VM_IOREMAP case in flush_cache_vmap.
At this time, I don't know whether it is better to always flush when
the PTE present bit is set or when both the accessed and present bits
are set. The later saves flushing pages that haven't been accessed,
but we need to flush in ptep_clear_flush_young. It also needs a page
table lookup to find the PTE pointer. The lpa instruction only needs
a page table lookup when the PTE entry isn't in the TLB.
We don't atomically handle setting and clearing the _PAGE_ACCESSED bit.
If we miss an update, we may miss a flush and the cache may get corrupted.
Whether the current code is effectively atomic depends on process control.
When CONFIG_FLUSH_PAGE_ACCESSED is set to zero, the page will eventually
be flushed when the PTE is cleared or in flush_cache_page_if_present. The
_PAGE_ACCESSED bit is not used, so the problem is avoided.
The flush method can be selected using the CONFIG_FLUSH_PAGE_ACCESSED
define in cache.c. The default is 0. I didn't see a large difference
in performance.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Cc: <stable@vger.kernel.org> # v6.6+
Signed-off-by: Helge Deller <deller@gmx.de>
2024-06-10 18:47:07 +00:00
|
|
|
struct mm_struct;
|
2005-04-16 15:20:36 -07:00
|
|
|
static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
|
|
|
|
|
{
|
2025-07-21 16:06:21 -04:00
|
|
|
set_pte_at(mm, addr, ptep, pte_wrprotect(*ptep));
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#define pte_same(A,B) (pte_val(A) == pte_val(B))
|
|
|
|
|
|
2025-03-14 08:09:56 +01:00
|
|
|
#endif /* !__ASSEMBLER__ */
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2006-04-20 20:40:23 +00:00
|
|
|
|
|
|
|
|
/* TLB page size encoding - see table 3-1 in parisc20.pdf */
|
|
|
|
|
#define _PAGE_SIZE_ENCODING_4K 0
|
2006-04-21 02:20:37 +00:00
|
|
|
#define _PAGE_SIZE_ENCODING_16K 1
|
|
|
|
|
#define _PAGE_SIZE_ENCODING_64K 2
|
2006-04-20 20:40:23 +00:00
|
|
|
#define _PAGE_SIZE_ENCODING_256K 3
|
|
|
|
|
#define _PAGE_SIZE_ENCODING_1M 4
|
|
|
|
|
#define _PAGE_SIZE_ENCODING_4M 5
|
2006-04-21 02:20:37 +00:00
|
|
|
#define _PAGE_SIZE_ENCODING_16M 6
|
|
|
|
|
#define _PAGE_SIZE_ENCODING_64M 7
|
2006-04-20 20:40:23 +00:00
|
|
|
|
|
|
|
|
#if defined(CONFIG_PARISC_PAGE_SIZE_4KB)
|
|
|
|
|
# define _PAGE_SIZE_ENCODING_DEFAULT _PAGE_SIZE_ENCODING_4K
|
|
|
|
|
#elif defined(CONFIG_PARISC_PAGE_SIZE_16KB)
|
|
|
|
|
# define _PAGE_SIZE_ENCODING_DEFAULT _PAGE_SIZE_ENCODING_16K
|
|
|
|
|
#elif defined(CONFIG_PARISC_PAGE_SIZE_64KB)
|
|
|
|
|
# define _PAGE_SIZE_ENCODING_DEFAULT _PAGE_SIZE_ENCODING_64K
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
|
2005-10-21 22:54:20 -04:00
|
|
|
#define pgprot_noncached(prot) __pgprot(pgprot_val(prot) | _PAGE_NO_CACHE)
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
/* We provide our own get_unmapped_area to provide cache coherency */
|
|
|
|
|
|
|
|
|
|
#define HAVE_ARCH_UNMAPPED_AREA
|
2014-01-31 22:19:52 +01:00
|
|
|
#define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
|
parisc: Try to fix random segmentation faults in package builds
PA-RISC systems with PA8800 and PA8900 processors have had problems
with random segmentation faults for many years. Systems with earlier
processors are much more stable.
Systems with PA8800 and PA8900 processors have a large L2 cache which
needs per page flushing for decent performance when a large range is
flushed. The combined cache in these systems is also more sensitive to
non-equivalent aliases than the caches in earlier systems.
The majority of random segmentation faults that I have looked at
appear to be memory corruption in memory allocated using mmap and
malloc.
My first attempt at fixing the random faults didn't work. On
reviewing the cache code, I realized that there were two issues
which the existing code didn't handle correctly. Both relate
to cache move-in. Another issue is that the present bit in PTEs
is racy.
1) PA-RISC caches have a mind of their own and they can speculatively
load data and instructions for a page as long as there is a entry in
the TLB for the page which allows move-in. TLBs are local to each
CPU. Thus, the TLB entry for a page must be purged before flushing
the page. This is particularly important on SMP systems.
In some of the flush routines, the flush routine would be called
and then the TLB entry would be purged. This was because the flush
routine needed the TLB entry to do the flush.
2) My initial approach to trying the fix the random faults was to
try and use flush_cache_page_if_present for all flush operations.
This actually made things worse and led to a couple of hardware
lockups. It finally dawned on me that some lines weren't being
flushed because the pte check code was racy. This resulted in
random inequivalent mappings to physical pages.
The __flush_cache_page tmpalias flush sets up its own TLB entry
and it doesn't need the existing TLB entry. As long as we can find
the pte pointer for the vm page, we can get the pfn and physical
address of the page. We can also purge the TLB entry for the page
before doing the flush. Further, __flush_cache_page uses a special
TLB entry that inhibits cache move-in.
When switching page mappings, we need to ensure that lines are
removed from the cache. It is not sufficient to just flush the
lines to memory as they may come back.
This made it clear that we needed to implement all the required
flush operations using tmpalias routines. This includes flushes
for user and kernel pages.
After modifying the code to use tmpalias flushes, it became clear
that the random segmentation faults were not fully resolved. The
frequency of faults was worse on systems with a 64 MB L2 (PA8900)
and systems with more CPUs (rp4440).
The warning that I added to flush_cache_page_if_present to detect
pages that couldn't be flushed triggered frequently on some systems.
Helge and I looked at the pages that couldn't be flushed and found
that the PTE was either cleared or for a swap page. Ignoring pages
that were swapped out seemed okay but pages with cleared PTEs seemed
problematic.
I looked at routines related to pte_clear and noticed ptep_clear_flush.
The default implementation just flushes the TLB entry. However, it was
obvious that on parisc we need to flush the cache page as well. If
we don't flush the cache page, stale lines will be left in the cache
and cause random corruption. Once a PTE is cleared, there is no way
to find the physical address associated with the PTE and flush the
associated page at a later time.
I implemented an updated change with a parisc specific version of
ptep_clear_flush. It fixed the random data corruption on Helge's rp4440
and rp3440, as well as on my c8000.
At this point, I realized that I could restore the code where we only
flush in flush_cache_page_if_present if the page has been accessed.
However, for this, we also need to flush the cache when the accessed
bit is cleared in ptep_clear_flush_young to keep things synchronized.
The default implementation only flushes the TLB entry.
Other changes in this version are:
1) Implement parisc specific version of ptep_get. It's identical to
default but needed in arch/parisc/include/asm/pgtable.h.
2) Revise parisc implementation of ptep_test_and_clear_young to use
ptep_get (READ_ONCE).
3) Drop parisc implementation of ptep_get_and_clear. We can use default.
4) Revise flush_kernel_vmap_range and invalidate_kernel_vmap_range to
use full data cache flush.
5) Move flush_cache_vmap and flush_cache_vunmap to cache.c. Handle
VM_IOREMAP case in flush_cache_vmap.
At this time, I don't know whether it is better to always flush when
the PTE present bit is set or when both the accessed and present bits
are set. The later saves flushing pages that haven't been accessed,
but we need to flush in ptep_clear_flush_young. It also needs a page
table lookup to find the PTE pointer. The lpa instruction only needs
a page table lookup when the PTE entry isn't in the TLB.
We don't atomically handle setting and clearing the _PAGE_ACCESSED bit.
If we miss an update, we may miss a flush and the cache may get corrupted.
Whether the current code is effectively atomic depends on process control.
When CONFIG_FLUSH_PAGE_ACCESSED is set to zero, the page will eventually
be flushed when the PTE is cleared or in flush_cache_page_if_present. The
_PAGE_ACCESSED bit is not used, so the problem is avoided.
The flush method can be selected using the CONFIG_FLUSH_PAGE_ACCESSED
define in cache.c. The default is 0. I didn't see a large difference
in performance.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Cc: <stable@vger.kernel.org> # v6.6+
Signed-off-by: Helge Deller <deller@gmx.de>
2024-06-10 18:47:07 +00:00
|
|
|
#define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
|
|
|
|
|
#define __HAVE_ARCH_PTEP_CLEAR_FLUSH
|
2005-04-16 15:20:36 -07:00
|
|
|
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
|
|
|
|
|
#define __HAVE_ARCH_PTE_SAME
|
|
|
|
|
|
|
|
|
|
#endif /* _PARISC_PGTABLE_H */
|