linux/arch/arm/mm
Ard Biesheuvel 50596b7559 ARM: smp: Store current pointer in TPIDRURO register if available
Now that the user space TLS register is assigned on every return to user
space, we can use it to keep the 'current' pointer while running in the
kernel. This removes the need to access it via thread_info, which is
located at the base of the stack, but will be moved out of there in a
subsequent patch.

Use the __builtin_thread_pointer() helper when available - this will
help GCC understand that reloading the value within the same function is
not necessary, even when using the per-task stack protector (which also
generates accesses via the TLS register). For example, the generated
code below loads TPIDRURO only once, and uses it to access both the
stack canary and the preempt_count fields.

<do_one_initcall>:
       e92d 41f0       stmdb   sp!, {r4, r5, r6, r7, r8, lr}
       ee1d 4f70       mrc     15, 0, r4, cr13, cr0, {3}
       4606            mov     r6, r0
       b094            sub     sp, #80 ; 0x50
       f8d4 34e8       ldr.w   r3, [r4, #1256] ; 0x4e8  <- stack canary
       9313            str     r3, [sp, #76]   ; 0x4c
       f8d4 8004       ldr.w   r8, [r4, #4]             <- preempt count

Co-developed-by: Keith Packard <keithpac@amazon.com>
Signed-off-by: Keith Packard <keithpac@amazon.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
2021-09-27 16:54:02 +02:00
..
abort-ev4.S
abort-ev4t.S
abort-ev5t.S
abort-ev5tj.S
abort-ev6.S
abort-ev7.S ARM: 9091/1: Revert "mm: qsd8x50: Fix incorrect permission faults" 2021-06-13 18:16:43 +01:00
abort-lv4t.S
abort-macro.S
abort-nommu.S
alignment.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
cache-b15-rac.c ARM: mm: Remove virtual address print from B15 RAC driver 2020-05-06 11:12:12 -07:00
cache-fa.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cache-feroceon-l2.c
cache-l2x0-pmu.c
cache-l2x0.c ARM: 9007/1: l2c: fix prefetch bits init in L2X0_AUX_CTRL using DT values 2020-09-15 14:35:53 +01:00
cache-nop.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cache-tauros2.c
cache-tauros3.h
cache-uniphier.c
cache-v4.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cache-v4wb.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cache-v4wt.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cache-v6.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cache-v7.S ARM: 9059/1: cache-v7: get rid of mini-stack 2021-03-09 10:25:18 +00:00
cache-v7m.S sched/rt, ARM: Use CONFIG_PREEMPTION 2019-12-08 14:37:32 +01:00
cache-xsc3l2.c
context.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
copypage-fa.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
copypage-feroceon.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
copypage-v4mc.c mm: move page_mapping_file to pagemap.h 2021-04-30 11:20:37 -07:00
copypage-v4wb.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
copypage-v4wt.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
copypage-v6.c mm: move page_mapping_file to pagemap.h 2021-04-30 11:20:37 -07:00
copypage-xsc3.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
copypage-xscale.c mm: move page_mapping_file to pagemap.h 2021-04-30 11:20:37 -07:00
dma-mapping-nommu.c ARM/nommu: use the generic dma-direct code for non-coherent devices 2021-08-18 16:24:10 +02:00
dma-mapping.c ARM/dma-mapping: don't set failed sg dma_address to DMA_MAPPING_ERROR 2021-08-09 17:13:05 +02:00
dma.h
dump.c ARM: 9073/1: ptdump: add __init section marker to three functions 2021-04-18 19:15:12 +01:00
extable.c
fault-armv.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
fault.c mm/arm: use general page fault accounting 2020-08-12 10:58:03 -07:00
fault.h ARM: 8898/1: mm: Don't treat faults reported from cache maintenance as writes 2019-08-23 11:39:34 +01:00
flush.c mm: remove flush_kernel_dcache_page 2021-09-03 09:58:13 -07:00
fsr-2level.c
fsr-3level.c
hugetlbpage.c mm: remove unneeded includes of <asm/pgalloc.h> 2020-08-07 11:33:26 -07:00
idmap.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
init.c arm: extend pfn_valid to take into account freed memory map alignment 2021-06-30 11:39:00 +03:00
iomap.c ARM: 8923/1: mm: include <asm/vga.h> for vga_base 2019-10-31 16:58:54 +00:00
ioremap.c arm: ioremap: don't abuse pfn_valid() to check if pfn is in RAM 2021-07-06 09:01:47 +03:00
kasan_init.c ARM: 9016/2: Initialize the mapping of KASan shadow memory 2020-10-27 12:11:10 +00:00
Kconfig ARM: 9091/1: Revert "mm: qsd8x50: Fix incorrect permission faults" 2021-06-13 18:16:43 +01:00
l2c-common.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
l2c-l2x0-resume.S
Makefile ARM updates for 5.11: 2020-12-22 13:34:27 -08:00
mm.h mm: pgtable: add shortcuts for accessing kernel PMD and PTE 2020-06-09 09:39:13 -07:00
mmap.c arm: Use the generic devmem_is_allowed() 2020-12-11 12:28:24 -08:00
mmu.c ARM: 9104/2: Fix Keystone 2 kernel mapping regression 2021-08-10 12:17:25 +01:00
nommu.c mm: remove flush_kernel_dcache_page 2021-09-03 09:58:13 -07:00
pabort-legacy.S
pabort-v6.S
pabort-v7.S
pageattr.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
pgd.c ARM: 9016/2: Initialize the mapping of KASan shadow memory 2020-10-27 12:11:10 +00:00
physaddr.c
pmsa-v7.c ARM: 9069/1: NOMMU: Fix conversion for_each_membock() to for_each_mem_range() 2021-03-25 10:25:20 +00:00
pmsa-v8.c ARM: 9069/1: NOMMU: Fix conversion for_each_membock() to for_each_mem_range() 2021-03-25 10:25:20 +00:00
proc-arm7tdmi.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm9tdmi.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm720.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm740.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm920.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm922.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm925.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm926.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm940.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm946.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm1020.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm1020e.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm1022.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-arm1026.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-fa526.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-feroceon.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-macros.S ARM: smp: Store current pointer in TPIDRURO register if available 2021-09-27 16:54:02 +02:00
proc-mohawk.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-sa110.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-sa1100.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-syms.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
proc-v6.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-v7-2level.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
proc-v7-3level.S
proc-v7-bugs.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
proc-v7.S ARM: 9059/1: cache-v7: get rid of mini-stack 2021-03-09 10:25:18 +00:00
proc-v7m.S ARM: 8933/1: replace Sun/Solaris style flag on section directive 2019-11-15 22:21:19 +00:00
proc-xsc3.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
proc-xscale.S mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
ptdump_debugfs.c ARM: 9074/1: ptdump: convert to DEFINE_SHOW_ATTRIBUTE 2021-04-18 19:15:13 +01:00
pv-fixup-asm.S ARM: 9104/2: Fix Keystone 2 kernel mapping regression 2021-08-10 12:17:25 +01:00
tcm.h
tlb-fa.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
tlb-v4.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
tlb-v4wb.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
tlb-v4wbi.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
tlb-v6.S mm: update legacy flush_tlb_* to use vma 2021-06-29 10:53:52 -07:00
tlb-v7.S mm: update legacy flush_tlb_* to use vma 2021-06-29 10:53:52 -07:00