linux/scripts/package/install-extmod-build

87 lines
2.6 KiB
Text
Raw Normal View History

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-only
set -eu
destdir=${1}
is_enabled() {
grep -q "^$1=y" include/config/auto.conf
}
find_in_scripts() {
find scripts \
\( -name atomic -o -name dtc -o -name kconfig -o -name package \) -prune -o \
! -name unifdef -a ! -name mk_elfconfig -a \( -type f -o -type l \) -print
}
mkdir -p "${destdir}"
(
cd "${srctree}"
echo Makefile
find "arch/${SRCARCH}" -maxdepth 1 -name 'Makefile*'
find "arch/${SRCARCH}" -name generated -prune -o -name include -type d -print
find "arch/${SRCARCH}" -name Kbuild.platforms -o -name Platform
find include \( -name config -o -name generated \) -prune -o \( -type f -o -type l \) -print
find_in_scripts
) | tar -c -f - -C "${srctree}" -T - | tar -xf - -C "${destdir}"
{
if is_enabled CONFIG_OBJTOOL; then
echo tools/objtool/objtool
fi
echo Module.symvers
echo "arch/${SRCARCH}/include/generated"
echo include/config/auto.conf
echo include/config/kernel.release
echo include/generated
find_in_scripts
if is_enabled CONFIG_GCC_PLUGINS; then
find scripts/gcc-plugins -name '*.so'
fi
} | tar -c -f - -T - | tar -xf - -C "${destdir}"
kbuild: deb-pkg: add pkg.linux-upstream.nokernelheaders build profile Since commit f1d87664b82a ("kbuild: cross-compile linux-headers package when possible"), 'make bindeb-pkg' may attempt to cross-compile the linux-headers package, but it fails under certain circumstances. For example, when CONFIG_MODULE_SIG_FORMAT is enabled on Debian, the following command fails: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg [ snip ] Rebuilding host programs with aarch64-linux-gnu-gcc... HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/kallsyms HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/sorttable HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/asn1_compiler HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/sign-file In file included from /usr/include/openssl/opensslv.h:109, from debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/sign-file.c:25: /usr/include/openssl/macros.h:14:10: fatal error: openssl/opensslconf.h: No such file or directory 14 | #include <openssl/opensslconf.h> | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. This commit adds a new profile, pkg.linux-upstream.nokernelheaders, to guard the linux-headers package. There are two options to fix the above issue. Option 1: Set the pkg.linux-upstream.nokernelheaders build profile $ DEB_BUILD_PROFILES=pkg.linux-upstream.nokernelheaders \ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg This skips the building of the linux-headers package. Option 2: Install the necessary build dependencies If you want to cross-compile the linux-headers package, you need to install additional packages. For example, on Debian, the packages necessary for cross-compiling it to arm64 can be installed with the following commands: # dpkg --add-architecture arm64 # apt update # apt install gcc-aarch64-linux-gnu libssl-dev:arm64 Fixes: f1d87664b82a ("kbuild: cross-compile linux-headers package when possible") Reported-by: Ron Economos <re@w6rz.net> Closes: https://lore.kernel.org/all/b3d4f49e-7ddb-29ba-0967-689232329b53@w6rz.net/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Ron Economos <re@w6rz.net> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2024-10-23 03:16:58 +09:00
# When ${CC} and ${HOSTCC} differ, rebuild host programs using ${CC}.
kbuild: cross-compile linux-headers package when possible A long standing issue in the upstream kernel packaging is that the linux-headers package is not cross-compiled. For example, you can cross-build Debian packages for arm64 by running the following command: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg However, the generated linux-headers-*_arm64.deb is useless because the host programs in it were built for your build machine architecture (likely x86), not arm64. The Debian kernel maintains its own Makefiles to cross-compile host tools without relying on Kbuild. [1] Instead of adding such full custom Makefiles, this commit adds a small piece of code to cross-compile host programs located under the scripts/ directory. A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it would also cross-compile scripts/basic/fixdep, which needs to be native to process the if_changed_dep macro. (This approach may work under some circumstances; you can execute foreign architecture programs with the help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, but it would require installing QEMU and libc for that architecture.) A trick is to use the external module build (KBUILD_EXTMOD=), which does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link userspace programs (CONFIG_CC_CAN_LINK=y). There are known limitations: - GCC plugins It would possible to rebuild GCC plugins for the target architecture by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages installed, but gcc on the installed system emits "cc1: error: incompatible gcc/plugin versions". - objtool and resolve_btfids These are built by the tools build system. They are not covered by the current solution. The resulting linux-headers package is broken if CONFIG_OBJTOOL or CONFIG_DEBUG_INFO_BTF is enabled. I only tested this with Debian, but it should work for other package systems as well. [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2024-07-27 16:42:04 +09:00
#
# This caters to host programs that participate in Kbuild. objtool and
# resolve_btfids are out of scope.
kbuild: deb-pkg: add pkg.linux-upstream.nokernelheaders build profile Since commit f1d87664b82a ("kbuild: cross-compile linux-headers package when possible"), 'make bindeb-pkg' may attempt to cross-compile the linux-headers package, but it fails under certain circumstances. For example, when CONFIG_MODULE_SIG_FORMAT is enabled on Debian, the following command fails: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg [ snip ] Rebuilding host programs with aarch64-linux-gnu-gcc... HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/kallsyms HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/sorttable HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/asn1_compiler HOSTCC debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/sign-file In file included from /usr/include/openssl/opensslv.h:109, from debian/linux-headers-6.12.0-rc4/usr/src/linux-headers-6.12.0-rc4/scripts/sign-file.c:25: /usr/include/openssl/macros.h:14:10: fatal error: openssl/opensslconf.h: No such file or directory 14 | #include <openssl/opensslconf.h> | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. This commit adds a new profile, pkg.linux-upstream.nokernelheaders, to guard the linux-headers package. There are two options to fix the above issue. Option 1: Set the pkg.linux-upstream.nokernelheaders build profile $ DEB_BUILD_PROFILES=pkg.linux-upstream.nokernelheaders \ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg This skips the building of the linux-headers package. Option 2: Install the necessary build dependencies If you want to cross-compile the linux-headers package, you need to install additional packages. For example, on Debian, the packages necessary for cross-compiling it to arm64 can be installed with the following commands: # dpkg --add-architecture arm64 # apt update # apt install gcc-aarch64-linux-gnu libssl-dev:arm64 Fixes: f1d87664b82a ("kbuild: cross-compile linux-headers package when possible") Reported-by: Ron Economos <re@w6rz.net> Closes: https://lore.kernel.org/all/b3d4f49e-7ddb-29ba-0967-689232329b53@w6rz.net/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Ron Economos <re@w6rz.net> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2024-10-23 03:16:58 +09:00
if [ "${CC}" != "${HOSTCC}" ]; then
kbuild: cross-compile linux-headers package when possible A long standing issue in the upstream kernel packaging is that the linux-headers package is not cross-compiled. For example, you can cross-build Debian packages for arm64 by running the following command: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg However, the generated linux-headers-*_arm64.deb is useless because the host programs in it were built for your build machine architecture (likely x86), not arm64. The Debian kernel maintains its own Makefiles to cross-compile host tools without relying on Kbuild. [1] Instead of adding such full custom Makefiles, this commit adds a small piece of code to cross-compile host programs located under the scripts/ directory. A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it would also cross-compile scripts/basic/fixdep, which needs to be native to process the if_changed_dep macro. (This approach may work under some circumstances; you can execute foreign architecture programs with the help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, but it would require installing QEMU and libc for that architecture.) A trick is to use the external module build (KBUILD_EXTMOD=), which does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link userspace programs (CONFIG_CC_CAN_LINK=y). There are known limitations: - GCC plugins It would possible to rebuild GCC plugins for the target architecture by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages installed, but gcc on the installed system emits "cc1: error: incompatible gcc/plugin versions". - objtool and resolve_btfids These are built by the tools build system. They are not covered by the current solution. The resulting linux-headers package is broken if CONFIG_OBJTOOL or CONFIG_DEBUG_INFO_BTF is enabled. I only tested this with Debian, but it should work for other package systems as well. [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2024-07-27 16:42:04 +09:00
echo "Rebuilding host programs with ${CC}..."
kbuild: change working directory to external module directory with M= Currently, Kbuild always operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory. This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts. The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel. The appearance of the build log will change as follows: [Before] $ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux' [After] $ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux' Printing "Entering directory" twice is cumbersome. This will be addressed later. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de>
2024-11-10 10:34:33 +09:00
# This leverages external module building.
# - Clear sub_make_done to allow the top-level Makefile to redo sub-make.
# - Filter out --no-print-directory to print "Entering directory" logs
# when Make changes the working directory.
unset sub_make_done
MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//)
kbuild: cross-compile linux-headers package when possible A long standing issue in the upstream kernel packaging is that the linux-headers package is not cross-compiled. For example, you can cross-build Debian packages for arm64 by running the following command: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg However, the generated linux-headers-*_arm64.deb is useless because the host programs in it were built for your build machine architecture (likely x86), not arm64. The Debian kernel maintains its own Makefiles to cross-compile host tools without relying on Kbuild. [1] Instead of adding such full custom Makefiles, this commit adds a small piece of code to cross-compile host programs located under the scripts/ directory. A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it would also cross-compile scripts/basic/fixdep, which needs to be native to process the if_changed_dep macro. (This approach may work under some circumstances; you can execute foreign architecture programs with the help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, but it would require installing QEMU and libc for that architecture.) A trick is to use the external module build (KBUILD_EXTMOD=), which does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link userspace programs (CONFIG_CC_CAN_LINK=y). There are known limitations: - GCC plugins It would possible to rebuild GCC plugins for the target architecture by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages installed, but gcc on the installed system emits "cc1: error: incompatible gcc/plugin versions". - objtool and resolve_btfids These are built by the tools build system. They are not covered by the current solution. The resulting linux-headers package is broken if CONFIG_OBJTOOL or CONFIG_DEBUG_INFO_BTF is enabled. I only tested this with Debian, but it should work for other package systems as well. [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2024-07-27 16:42:04 +09:00
cat <<-'EOF' > "${destdir}/Kbuild"
subdir-y := scripts
EOF
# HOSTCXX is not overridden. The C++ compiler is used to build:
# - scripts/kconfig/qconf, which is unneeded for external module builds
# - GCC plugins, which will not work on the installed system even after
# being rebuilt.
#
# Use the single-target build to avoid the modpost invocation, which
# would overwrite Module.symvers.
"${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
cat <<-'EOF' > "${destdir}/scripts/Kbuild"
subdir-y := basic
hostprogs-always-y := mod/modpost
mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o)
EOF
# Run once again to rebuild scripts/basic/ and scripts/mod/modpost.
"${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/
rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild"
fi
find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete