2019-05-30 15:03:44 +03:00
# SPDX-License-Identifier: GPL-2.0
2005-07-26 00:10:36 +04:00
####
# kbuild: Generic definitions
2007-02-09 00:48:51 +03:00
# Convenient variables
2005-07-26 00:10:36 +04:00
comma := ,
2014-03-20 06:08:20 +04:00
quote := "
2006-01-07 00:35:59 +03:00
squote := '
2005-07-26 00:10:36 +04:00
empty :=
space := $(empty) $(empty)
kbuild: fix if_change and friends to consider argument order
Currently, arg-check is implemented as follows:
arg-check = $(strip $(filter-out $(cmd_$(1)), $(cmd_$@)) \
$(filter-out $(cmd_$@), $(cmd_$(1))) )
This does not care about the order of arguments that appear in
$(cmd_$(1)) and $(cmd_$@). So, if_changed and friends never rebuild
the target if only the argument order is changed. This is a problem
when the link order is changed.
Apparently,
obj-y += foo.o
obj-y += bar.o
and
obj-y += bar.o
obj-y += foo.o
should be distinguished because the link order determines the probe
order of drivers. So, built-in.o should be rebuilt when the order
of objects is changed.
This commit fixes arg-check to compare the old/current commands
including the argument order.
Of course, this change has a side effect; Kbuild will react to the
change of compile option order. For example, "-DFOO -DBAR" and
"-DBAR -DFOO" should give no difference to the build result, but
false positive should be better than false negative.
I am moving space_escape to the top of Kbuild.include just for a
matter of preference. In practical terms, space_escape can be
defined after arg-check because arg-check uses "=" flavor, not ":=".
Having said that, collecting convenient variables in one place makes
sense from the point of readability.
Chaining "%%%SPACE%%%" to "_-_SPACE_-_" is also a matter of taste
at this point. Actually, it can be arbitrary as long as it is an
unlikely used string. The only problem I see in "%%%SPACE%%%" is
that "%" is a special character in "$(patsubst ...)" context. This
commit just uses "$(subst ...)" for arg-check, but I am fixing it now
in case we might want to use it in $(patsubst ...) context in the
future.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-07 09:48:26 +03:00
space_escape := _-_SPACE_-_
Kbuild: fix # escaping in .cmd files for future Make
I tried building using a freshly built Make (4.2.1-69-g8a731d1), but
already the objtool build broke with
orc_dump.c: In function ‘orc_dump’:
orc_dump.c:106:2: error: ‘elf_getshnum’ is deprecated [-Werror=deprecated-declarations]
if (elf_getshdrnum(elf, &nr_sections)) {
Turns out that with that new Make, the backslash was not removed, so cpp
didn't see a #include directive, grep found nothing, and
-DLIBELF_USE_DEPRECATED was wrongly put in CFLAGS.
Now, that new Make behaviour is documented in their NEWS file:
* WARNING: Backward-incompatibility!
Number signs (#) appearing inside a macro reference or function invocation
no longer introduce comments and should not be escaped with backslashes:
thus a call such as:
foo := $(shell echo '#')
is legal. Previously the number sign needed to be escaped, for example:
foo := $(shell echo '\#')
Now this latter will resolve to "\#". If you want to write makefiles
portable to both versions, assign the number sign to a variable:
C := \#
foo := $(shell echo '$C')
This was claimed to be fixed in 3.81, but wasn't, for some reason.
To detect this change search for 'nocomment' in the .FEATURES variable.
This also fixes up the two make-cmd instances to replace # with $(pound)
rather than with \#. There might very well be other places that need
similar fixup in preparation for whatever future Make release contains
the above change, but at least this builds an x86_64 defconfig with the
new make.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=197847
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-09 00:35:28 +03:00
pound := \#
2022-12-11 05:54:47 +03:00
define newline
endef
2005-07-26 00:10:36 +04:00
kbuild: add test-{ge,gt,le,lt} macros
GNU Make 4.4 introduced $(intcmp ...), which is useful to compare two
integers without forking a new process.
Add test-{ge,gt,le,lt} macros, which work more efficiently with GNU
Make >= 4.4. For older Make versions, they fall back to the 'test'
shell command.
The first two parameters to $(intcmp ...) must not be empty. To avoid
the syntax error, I appended '0' to them. Fortunately, '00' is treated
as '0'. This is needed because CONFIG options may expand to an empty
string when the kernel configuration is not included.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com> # RISC-V
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2022-12-11 05:46:47 +03:00
###
# Comparison macros.
# Usage: $(call test-lt, $(CONFIG_LLD_VERSION), 150000)
#
# Use $(intcmp ...) if supported. (Make >= 4.4)
# Otherwise, fall back to the 'test' shell command.
ifeq ($(intcmp 1,0,,,y),y)
test-ge = $(intcmp $(strip $1)0, $(strip $2)0,,y,y)
test-gt = $(intcmp $(strip $1)0, $(strip $2)0,,,y)
else
test-ge = $(shell test $(strip $1)0 -ge $(strip $2)0 && echo y)
test-gt = $(shell test $(strip $1)0 -gt $(strip $2)0 && echo y)
endif
test-le = $(call test-ge, $2, $1)
test-lt = $(call test-gt, $2, $1)
2006-07-23 21:37:44 +04:00
###
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
dot-target = $(dir $@).$(notdir $@)
2022-05-27 13:01:49 +03:00
###
# Name of target with a '.tmp_' as filename prefix. foo/bar.o => foo/.tmp_bar.o
tmp-target = $(dir $@).tmp_$(notdir $@)
2005-07-26 00:10:36 +04:00
###
2020-04-23 17:23:53 +03:00
# The temporary file to save gcc -MMD generated dependencies must not
2005-07-26 00:10:36 +04:00
# contain a comma
2006-07-23 21:37:44 +04:00
depfile = $(subst $(comma),_,$(dot-target).d)
2005-07-26 00:10:36 +04:00
2006-07-01 11:58:02 +04:00
###
# filename of target with directory and extension stripped
basetarget = $(basename $(notdir $@))
2019-01-17 13:02:43 +03:00
###
# real prerequisites without phony targets
real-prereqs = $(filter-out $(PHONY), $^)
2006-01-07 00:35:59 +03:00
###
# Escape single quote for use in echo statements
escsq = $(subst $(squote),'\$(squote)',$1)
2019-12-19 11:33:27 +03:00
###
# Quote a string to pass it to C files. foo => '"foo"'
stringify = $(squote)$(quote)$1$(quote)$(squote)
2022-11-18 22:15:50 +03:00
###
# The path to Kbuild or Makefile. Kbuild has precedence over Makefile.
kbuild-dir = $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
kbuild-file = $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
2022-12-11 05:54:47 +03:00
###
# Read a file, replacing newlines with spaces
#
# Make 4.2 or later can read a file by using its builtin function.
ifneq ($(filter-out 3.% 4.0 4.1, $(MAKE_VERSION)),)
read-file = $(subst $(newline),$(space),$(file < $1))
else
read-file = $(shell cat $1 2>/dev/null)
endif
2008-11-06 11:31:34 +03:00
###
# Easy method for doing a status message
kecho := :
quiet_kecho := echo
silent_kecho := :
kecho := $($(quiet)kecho)
2005-07-26 00:10:36 +04:00
###
# filechk is used to check if the content of a generated file is updated.
# Sample usage:
2019-01-03 04:16:54 +03:00
#
# filechk_sample = echo $(KERNELRELEASE)
# version.h: FORCE
2005-07-26 00:10:36 +04:00
# $(call filechk,sample)
2019-01-03 04:16:54 +03:00
#
2005-07-26 00:10:36 +04:00
# The rule defined shall write to stdout the content of the new file.
# The existing file will be compared with the new one.
# - If no file exist it is created
# - If the content differ the new file is used
# - If they are equal no change, and no timestamp update
define filechk
2021-08-13 09:30:05 +03:00
$(check-FORCE)
2020-01-10 08:02:24 +03:00
$(Q)set -e; \
mkdir -p $(dir $@); \
2023-01-09 16:40:40 +03:00
trap "rm -f $(tmp-target)" EXIT; \
{ $(filechk_$(1)); } > $(tmp-target); \
if [ ! -r $@ ] || ! cmp -s $@ $(tmp-target); then \
2020-01-10 08:02:24 +03:00
$(kecho) ' UPD $@'; \
2023-01-09 16:40:40 +03:00
mv -f $(tmp-target) $@; \
2005-07-26 00:10:36 +04:00
fi
endef
2007-02-09 00:48:51 +03:00
###
2005-07-26 00:10:36 +04:00
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.build obj=
# Usage:
# $(Q)$(MAKE) $(build)=dir
2014-09-09 15:03:58 +04:00
build := -f $(srctree)/scripts/Makefile.build obj
2005-07-26 00:10:36 +04:00
2014-09-03 17:29:24 +04:00
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.dtbinst obj=
# Usage:
# $(Q)$(MAKE) $(dtbinst)=dir
2018-09-12 07:52:54 +03:00
dtbinst := -f $(srctree)/scripts/Makefile.dtbinst obj
2014-09-03 17:29:24 +04:00
2014-11-26 13:31:13 +03:00
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.clean obj=
# Usage:
# $(Q)$(MAKE) $(clean)=dir
clean := -f $(srctree)/scripts/Makefile.clean obj
2022-12-22 19:25:32 +03:00
# pring log
#
# If quiet is "silent_", print nothing and sink stdout
# If quiet is "quiet_", print short log
# If quiet is empty, print short log and whole command
silent_log_print = exec >/dev/null;
quiet_log_print = $(if $(quiet_cmd_$1), echo ' $(call escsq,$(quiet_cmd_$1)$(why))';)
2022-12-22 19:25:34 +03:00
log_print = echo '$(pound) $(call escsq,$(or $(quiet_cmd_$1),cmd_$1 $@)$(why))'; \
2022-12-22 19:25:32 +03:00
echo ' $(call escsq,$(cmd_$1))';
kbuild: sink stdout from cmd for silent build
When building with 'make -s', no output to stdout should be printed.
As Arnd Bergmann reported [1], mkimage shows the detailed information
of the generated images.
I think this should be suppressed by the 'cmd' macro instead of by
individual scripts.
Insert 'exec >/dev/null;' in order to redirect stdout to /dev/null for
silent builds.
[Note about this implementation]
'exec >/dev/null;' may look somewhat tricky, but this has a reason.
Appending '>/dev/null' at the end of command line is a common way for
redirection, so I first tried this:
cmd = @set -e; $(echo-cmd) $(cmd_$(1)) >/dev/null
... but it would not work if $(cmd_$(1)) itself contains a redirection.
For example, cmd_wrap in scripts/Makefile.asm-generic redirects the
output from the 'echo' command into the target file.
It would be expanded into:
echo "#include <asm-generic/$*.h>" > $@ >/dev/null
Then, the target file gets empty because the string will go to /dev/null
instead of $@.
Next, I tried this:
cmd = @set -e; $(echo-cmd) { $(cmd_$(1)); } >/dev/null
The form above would be expanded into:
{ echo "#include <asm-generic/$*.h>" > $@; } >/dev/null
This works as expected. However, it would be a syntax error if
$(cmd_$(1)) is empty.
When CONFIG_TRIM_UNUSED_KSYMS is disabled, $(call cmd,gen_ksymdeps) in
scripts/Makefile.build would be expanded into:
set -e; { ; } >/dev/null
..., which causes an syntax error.
I also tried this:
cmd = @set -e; $(echo-cmd) ( $(cmd_$(1)) ) >/dev/null
... but this causes a syntax error for the same reason.
So, finally I adopted:
cmd = @set -e; $(echo-cmd) exec >/dev/null; $(cmd_$(1))
[1]: https://lore.kernel.org/lkml/20210514135752.2910387-1-arnd@kernel.org/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-05-17 10:03:13 +03:00
kbuild: remove the target in signal traps when interrupted
When receiving some signal, GNU Make automatically deletes the target if
it has already been changed by the interrupted recipe.
If the target is possibly incomplete due to interruption, it must be
deleted so that it will be remade from scratch on the next run of make.
Otherwise, the target would remain corrupted permanently because its
timestamp had already been updated.
Thanks to this behavior of Make, you can stop the build any time by
pressing Ctrl-C, and just run 'make' to resume it.
Kbuild also relies on this feature, but it is equivalently important
for any build systems that make decisions based on timestamps (if you
want to support Ctrl-C reliably).
However, this does not always work as claimed; Make immediately dies
with Ctrl-C if its stderr goes into a pipe.
[Test Makefile]
foo:
echo hello > $@
sleep 3
echo world >> $@
[Test Result]
$ make # hit Ctrl-C
echo hello > foo
sleep 3
^Cmake: *** Deleting file 'foo'
make: *** [Makefile:3: foo] Interrupt
$ make 2>&1 | cat # hit Ctrl-C
echo hello > foo
sleep 3
^C$ # 'foo' is often left-over
The reason is because SIGINT is sent to the entire process group.
In this example, SIGINT kills 'cat', and 'make' writes the message to
the closed pipe, then dies with SIGPIPE before cleaning the target.
A typical bad scenario (as reported by [1], [2]) is to save build log
by using the 'tee' command:
$ make 2>&1 | tee log
This can be problematic for any build systems based on Make, so I hope
it will be fixed in GNU Make. The maintainer of GNU Make stated this is
a long-standing issue and difficult to fix [3]. It has not been fixed
yet as of writing.
So, we cannot rely on Make cleaning the target. We can do it by
ourselves, in signal traps.
As far as I understand, Make takes care of SIGHUP, SIGINT, SIGQUIT, and
SITERM for the target removal. I added the traps for them, and also for
SIGPIPE just in case cmd_* rule prints something to stdout or stderr
(but I did not observe an actual case where SIGPIPE was triggered).
[Note 1]
The trap handler might be worth explaining.
rm -f $@; trap - $(sig); kill -s $(sig) $$
This lets the shell kill itself by the signal it caught, so the parent
process can tell the child has exited on the signal. Generally, this is
a proper manner for handling signals, in case the calling program (like
Bash) may monitor WIFSIGNALED() and WTERMSIG() for WCE although this may
not be a big deal here because GNU Make handles SIGHUP, SIGINT, SIGQUIT
in WUE and SIGTERM in IUE.
IUE - Immediate Unconditional Exit
WUE - Wait and Unconditional Exit
WCE - Wait and Cooperative Exit
For details, see "Proper handling of SIGINT/SIGQUIT" [4].
[Note 2]
Reverting 392885ee82d3 ("kbuild: let fixdep directly write to .*.cmd
files") would directly address [1], but it only saves if_changed_dep.
As reported in [2], all commands that use redirection can potentially
leave an empty (i.e. broken) target.
[Note 3]
Another (even safer) approach might be to always write to a temporary
file, and rename it to $@ at the end of the recipe.
<command> > $(tmp-target)
mv $(tmp-target) $@
It would require a lot of Makefile changes, and result in ugly code,
so I did not take it.
[Note 4]
A little more thoughts about a pattern rule with multiple targets (or
a grouped target).
%.x %.y: %.z
<recipe>
When interrupted, GNU Make deletes both %.x and %.y, while this solution
only deletes $@. Probably, this is not a big deal. The next run of make
will execute the rule again to create $@ along with the other files.
[1]: https://lore.kernel.org/all/YLeot94yAaM4xbMY@gmail.com/
[2]: https://lore.kernel.org/all/20220510221333.2770571-1-robh@kernel.org/
[3]: https://lists.gnu.org/archive/html/help-make/2021-06/msg00001.html
[4]: https://www.cons.org/cracauer/sigint.html
Fixes: 392885ee82d3 ("kbuild: let fixdep directly write to .*.cmd files")
Reported-by: Ingo Molnar <mingo@kernel.org>
Reported-by: Rob Herring <robh@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2022-08-07 03:48:09 +03:00
# Delete the target on interruption
#
# GNU Make automatically deletes the target if it has already been changed by
# the interrupted recipe. So, you can safely stop the build by Ctrl-C (Make
# will delete incomplete targets), and resume it later.
#
# However, this does not work when the stderr is piped to another program, like
# $ make >&2 | tee log
# Make dies with SIGPIPE before cleaning the targets.
#
# To address it, we clean the target in signal traps.
#
# Make deletes the target when it catches SIGHUP, SIGINT, SIGQUIT, SIGTERM.
# So, we cover them, and also SIGPIPE just in case.
#
# Of course, this is unneeded for phony targets.
delete-on-interrupt = \
$(if $(filter-out $(PHONY), $@), \
$(foreach sig, HUP INT QUIT TERM PIPE, \
trap 'rm -f $@; trap - $(sig); kill -s $(sig) $$$$' $(sig);))
2022-12-22 19:25:32 +03:00
# print and execute commands
cmd = @$(if $(cmd_$(1)),set -e; $($(quiet)log_print) $(delete-on-interrupt) $(cmd_$(1)),:)
2005-07-26 00:10:36 +04:00
###
2007-02-06 04:18:21 +03:00
# if_changed - execute command if any prerequisite is newer than
2005-07-26 00:10:36 +04:00
# target, or command line has changed
# if_changed_dep - as if_changed, but uses fixdep to reveal dependencies
# including used config symbols
# if_changed_rule - as if_changed but execute rule instead
2019-06-12 20:52:48 +03:00
# See Documentation/kbuild/makefiles.rst for more info
2005-07-26 00:10:36 +04:00
ifneq ($(KBUILD_NOCMDDEP),1)
2019-06-22 19:07:03 +03:00
# Check if both commands are the same including their order. Result is empty
kbuild: fix if_change and friends to consider argument order
Currently, arg-check is implemented as follows:
arg-check = $(strip $(filter-out $(cmd_$(1)), $(cmd_$@)) \
$(filter-out $(cmd_$@), $(cmd_$(1))) )
This does not care about the order of arguments that appear in
$(cmd_$(1)) and $(cmd_$@). So, if_changed and friends never rebuild
the target if only the argument order is changed. This is a problem
when the link order is changed.
Apparently,
obj-y += foo.o
obj-y += bar.o
and
obj-y += bar.o
obj-y += foo.o
should be distinguished because the link order determines the probe
order of drivers. So, built-in.o should be rebuilt when the order
of objects is changed.
This commit fixes arg-check to compare the old/current commands
including the argument order.
Of course, this change has a side effect; Kbuild will react to the
change of compile option order. For example, "-DFOO -DBAR" and
"-DBAR -DFOO" should give no difference to the build result, but
false positive should be better than false negative.
I am moving space_escape to the top of Kbuild.include just for a
matter of preference. In practical terms, space_escape can be
defined after arg-check because arg-check uses "=" flavor, not ":=".
Having said that, collecting convenient variables in one place makes
sense from the point of readability.
Chaining "%%%SPACE%%%" to "_-_SPACE_-_" is also a matter of taste
at this point. Actually, it can be arbitrary as long as it is an
unlikely used string. The only problem I see in "%%%SPACE%%%" is
that "%" is a special character in "$(patsubst ...)" context. This
commit just uses "$(subst ...)" for arg-check, but I am fixing it now
in case we might want to use it in $(patsubst ...) context in the
future.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-07 09:48:26 +03:00
# string if equal. User may override this check using make KBUILD_NOCMDDEP=1
2022-12-29 12:15:01 +03:00
# If the target does not exist, the *.cmd file should not be included so
# $(savedcmd_$@) gets empty. Then, target will be built even if $(newer-prereqs)
# happens to become empty.
2022-12-29 12:15:00 +03:00
cmd-check = $(filter-out $(subst $(space),$(space_escape),$(strip $(savedcmd_$@))), \
kbuild: fix if_change and friends to consider argument order
Currently, arg-check is implemented as follows:
arg-check = $(strip $(filter-out $(cmd_$(1)), $(cmd_$@)) \
$(filter-out $(cmd_$@), $(cmd_$(1))) )
This does not care about the order of arguments that appear in
$(cmd_$(1)) and $(cmd_$@). So, if_changed and friends never rebuild
the target if only the argument order is changed. This is a problem
when the link order is changed.
Apparently,
obj-y += foo.o
obj-y += bar.o
and
obj-y += bar.o
obj-y += foo.o
should be distinguished because the link order determines the probe
order of drivers. So, built-in.o should be rebuilt when the order
of objects is changed.
This commit fixes arg-check to compare the old/current commands
including the argument order.
Of course, this change has a side effect; Kbuild will react to the
change of compile option order. For example, "-DFOO -DBAR" and
"-DBAR -DFOO" should give no difference to the build result, but
false positive should be better than false negative.
I am moving space_escape to the top of Kbuild.include just for a
matter of preference. In practical terms, space_escape can be
defined after arg-check because arg-check uses "=" flavor, not ":=".
Having said that, collecting convenient variables in one place makes
sense from the point of readability.
Chaining "%%%SPACE%%%" to "_-_SPACE_-_" is also a matter of taste
at this point. Actually, it can be arbitrary as long as it is an
unlikely used string. The only problem I see in "%%%SPACE%%%" is
that "%" is a special character in "$(patsubst ...)" context. This
commit just uses "$(subst ...)" for arg-check, but I am fixing it now
in case we might want to use it in $(patsubst ...) context in the
future.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-07 09:48:26 +03:00
$(subst $(space),$(space_escape),$(strip $(cmd_$1))))
2011-05-16 18:37:34 +04:00
else
2022-12-29 12:15:01 +03:00
# We still need to detect missing targets.
2022-12-29 12:15:00 +03:00
cmd-check = $(if $(strip $(savedcmd_$@)),,1)
2005-07-26 00:10:36 +04:00
endif
2014-08-07 23:39:57 +04:00
# Replace >$< with >$$< to preserve $ when reloading the .cmd file
# (needed for make)
Kbuild: fix # escaping in .cmd files for future Make
I tried building using a freshly built Make (4.2.1-69-g8a731d1), but
already the objtool build broke with
orc_dump.c: In function ‘orc_dump’:
orc_dump.c:106:2: error: ‘elf_getshnum’ is deprecated [-Werror=deprecated-declarations]
if (elf_getshdrnum(elf, &nr_sections)) {
Turns out that with that new Make, the backslash was not removed, so cpp
didn't see a #include directive, grep found nothing, and
-DLIBELF_USE_DEPRECATED was wrongly put in CFLAGS.
Now, that new Make behaviour is documented in their NEWS file:
* WARNING: Backward-incompatibility!
Number signs (#) appearing inside a macro reference or function invocation
no longer introduce comments and should not be escaped with backslashes:
thus a call such as:
foo := $(shell echo '#')
is legal. Previously the number sign needed to be escaped, for example:
foo := $(shell echo '\#')
Now this latter will resolve to "\#". If you want to write makefiles
portable to both versions, assign the number sign to a variable:
C := \#
foo := $(shell echo '$C')
This was claimed to be fixed in 3.81, but wasn't, for some reason.
To detect this change search for 'nocomment' in the .FEATURES variable.
This also fixes up the two make-cmd instances to replace # with $(pound)
rather than with \#. There might very well be other places that need
similar fixup in preparation for whatever future Make release contains
the above change, but at least this builds an x86_64 defconfig with the
new make.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=197847
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-09 00:35:28 +03:00
# Replace >#< with >$(pound)< to avoid starting a comment in the .cmd file
2014-08-07 23:39:57 +04:00
# (needed for make)
# Replace >'< with >'\''< to be able to enclose the whole string in '...'
# (needed for the shell)
Kbuild: fix # escaping in .cmd files for future Make
I tried building using a freshly built Make (4.2.1-69-g8a731d1), but
already the objtool build broke with
orc_dump.c: In function ‘orc_dump’:
orc_dump.c:106:2: error: ‘elf_getshnum’ is deprecated [-Werror=deprecated-declarations]
if (elf_getshdrnum(elf, &nr_sections)) {
Turns out that with that new Make, the backslash was not removed, so cpp
didn't see a #include directive, grep found nothing, and
-DLIBELF_USE_DEPRECATED was wrongly put in CFLAGS.
Now, that new Make behaviour is documented in their NEWS file:
* WARNING: Backward-incompatibility!
Number signs (#) appearing inside a macro reference or function invocation
no longer introduce comments and should not be escaped with backslashes:
thus a call such as:
foo := $(shell echo '#')
is legal. Previously the number sign needed to be escaped, for example:
foo := $(shell echo '\#')
Now this latter will resolve to "\#". If you want to write makefiles
portable to both versions, assign the number sign to a variable:
C := \#
foo := $(shell echo '$C')
This was claimed to be fixed in 3.81, but wasn't, for some reason.
To detect this change search for 'nocomment' in the .FEATURES variable.
This also fixes up the two make-cmd instances to replace # with $(pound)
rather than with \#. There might very well be other places that need
similar fixup in preparation for whatever future Make release contains
the above change, but at least this builds an x86_64 defconfig with the
new make.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=197847
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-09 00:35:28 +03:00
make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
2006-01-30 12:04:27 +03:00
kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild
The incremental build of Linux kernel is pretty slow when lots of
objects are compiled. The rebuild of allmodconfig may take a few
minutes even when none of the objects needs to be rebuilt.
The time-consuming part in the incremental build is the evaluation of
if_changed* macros since they are used in the recipes to compile C and
assembly source files into objects.
I notice the following code in if_changed* is expensive:
$(filter-out $(PHONY) $(wildcard $^),$^)
In the incremental build, every object has its .*.cmd file, which
contains the auto-generated list of included headers. So, $^ are
expanded into the long list of the source file + included headers,
and $(wildcard $^) checks whether they exist.
It may not be clear why this check exists there.
Here is the record of my research.
[1] The first code addition into Kbuild
This code dates back to 2002. It is the pre-git era. So, I copy-pasted
it from the historical git tree.
| commit 4a6db0791528c220655b063cf13fefc8470dbfee (HEAD)
| Author: Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
| Date: Mon Jun 17 00:22:37 2002 -0500
|
| kbuild: Handle removed headers
|
| New and old way to handle dependencies would choke when a file
| #include'd by other files was removed, since the dependency on it was
| still recorded, but since it was gone, make has no idea what to do about
| it (and would complain with "No rule to make <file> ...")
|
| We now add targets for all the previously included files, so make will
| just ignore them if they disappear.
|
| diff --git a/Rules.make b/Rules.make
| index 6ef827d3df39..7db5301ea7db 100644
| --- a/Rules.make
| +++ b/Rules.make
| @@ -446,7 +446,7 @@ if_changed = $(if $(strip $? \
| # execute the command and also postprocess generated .d dependencies
| # file
|
| -if_changed_dep = $(if $(strip $? \
| +if_changed_dep = $(if $(strip $? $(filter-out FORCE $(wildcard $^),$^)\
| $(filter-out $(cmd_$(1)),$(cmd_$@))\
| $(filter-out $(cmd_$@),$(cmd_$(1)))),\
| @set -e; \
| diff --git a/scripts/fixdep.c b/scripts/fixdep.c
| index b5d7bee8efc7..db45bd1888c0 100644
| --- a/scripts/fixdep.c
| +++ b/scripts/fixdep.c
| @@ -292,7 +292,7 @@ void parse_dep_file(void *map, size_t len)
| exit(1);
| }
| memcpy(s, m, p-m); s[p-m] = 0;
| - printf("%s: \\\n", target);
| + printf("deps_%s := \\\n", target);
| m = p+1;
|
| clear_config();
| @@ -314,7 +314,8 @@ void parse_dep_file(void *map, size_t len)
| }
| m = p + 1;
| }
| - printf("\n");
| + printf("\n%s: $(deps_%s)\n\n", target, target);
| + printf("$(deps_%s):\n", target);
| }
|
| void print_deps(void)
The "No rule to make <file> ..." error can be solved by passing -MP to
the compiler, but I think the detection of header removal is a good
feature. When a header is removed, all source files that previously
included it should be re-compiled. This makes sure we has correctly
got rid of #include directives of it.
This is also related with the behavior of $?. The GNU Make manual says:
$?
The names of all the prerequisites that are newer than the target,
with spaces between them.
This does not explain whether a non-existent prerequisite is considered
to be newer than the target.
At this point of time, GNU Make 3.7x was used, where the $? did not
include non-existent prerequisites. Therefore,
$(filter-out FORCE $(wildcard $^),$^)
was useful to detect the header removal, and to rebuild the related
objects if it is the case.
[2] Change of $? behavior
Later, the behavior of $? was changed (fixed) to include prerequisites
that did not exist.
First, GNU Make commit 64e16d6c00a5 ("Various changes getting ready for
the release of 3.81.") changed it, but in the release test of 3.81, it
turned out to break the kernel build.
See these:
- http://lists.gnu.org/archive/html/bug-make/2006-03/msg00003.html
- https://savannah.gnu.org/bugs/?16002
- https://savannah.gnu.org/bugs/?16051
Then, GNU Make commit 6d8d9b74d9c5 ("Numerous updates to tests for
issues found on Cygwin and Windows.") reverted it for the 3.81 release
to give Linux kernel time to adjust to the new behavior.
After the 3.81 release, GNU Make commit 7595f38f62af ("Fixed a number
of documentation bugs, plus some build/install issues:") re-added it.
[3] Adjustment to the new $? behavior on Kbuild side
Meanwhile, the kernel build was changed by commit 4f1933620f57 ("kbuild:
change kbuild to not rely on incorrect GNU make behavior") to adjust to
the new $? behavior.
[4] GNU Make 3.82 released in 2010
GNU Make 3.82 was the first release that integrated the correct $?
behavior. At this point, Kbuild dealt with GNU Make versions with
different $? behaviors.
3.81 or older:
$? does not contain any non-existent prerequisite.
$(filter-out $(PHONY) $(wildcard $^),$^) was useful to detect
removed include headers.
3.82 or newer:
$? contains non-existent prerequisites. When a header is removed,
it appears in $?. $(filter-out $(PHONY) $(wildcard $^),$^) became
a redundant check.
With the correct $? behavior, we could have dropped the expensive
check for 3.82 or later, but we did not. (Maybe nobody noticed this
optimization.)
[5] The .SECONDARY special target trips up $?
Some time later, I noticed $? did not work as expected under some
circumstances. As above, $? should contain non-existent prerequisites,
but the ones specified as SECONDARY do not appear in $?.
I asked this in GNU Make ML, and it seems a bug:
https://lists.gnu.org/archive/html/bug-make/2019-01/msg00001.html
Since commit 8e9b61b293d9 ("kbuild: move .SECONDARY special target to
Kbuild.include"), all files, including headers listed in .*.cmd files,
are treated as secondary.
So, we are back into the incorrect $? behavior.
If we Kbuild want to react to the header removal, we need to keep
$(filter-out $(PHONY) $(wildcard $^),$^) but this makes the rebuild
so slow.
[Summary]
- I believe noticing the header removal and recompiling related objects
is a nice feature for the build system.
- If $? worked correctly, $(filter-out $(PHONY),$?) would be enough
to detect the header removal.
- Currently, $? does not work correctly when used with .SECONDARY,
and Kbuild is hit by this bug.
- I filed a bug report for this, but not fixed yet as of writing.
- Currently, the header removal is detected by the following expensive
code:
$(filter-out $(PHONY) $(wildcard $^),$^)
- I do not want to revert commit 8e9b61b293d9 ("kbuild: move
.SECONDARY special target to Kbuild.include"). Specifying
.SECONDARY globally is clean, and it matches to the Kbuild policy.
This commit proactively removes the expensive check since it makes the
incremental build faster. A downside is Kbuild will no longer be able
to notice the header removal.
You can confirm it by the full-build followed by a header removal, and
then re-build.
$ make defconfig all
[ full build ]
$ rm include/linux/device.h
$ make
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
Kernel: arch/x86/boot/bzImage is ready (#11)
Building modules, stage 2.
MODPOST 12 modules
Previously, Kbuild noticed a missing header and emits a build error.
Now, Kbuild is fine with it. This is an unusual corner-case, not a big
deal. Once the $? bug is fixed in GNU Make, everything will work fine.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-07 18:09:44 +03:00
# Find any prerequisites that are newer than target or that do not exist.
2006-07-23 21:37:44 +04:00
# PHONY targets skipped in both cases.
2022-12-29 12:15:01 +03:00
# If there is no prerequisite other than phony targets, $(newer-prereqs) becomes
# empty even if the target does not exist. cmd-check saves this corner case.
2019-11-07 18:09:45 +03:00
newer-prereqs = $(filter-out $(PHONY),$?)
2006-07-23 21:37:44 +04:00
2021-08-13 09:30:05 +03:00
# It is a typical mistake to forget the FORCE prerequisite. Check it here so
# no more breakage will slip in.
check-FORCE = $(if $(filter FORCE, $^),,$(warning FORCE prerequisite is missing))
if-changed-cond = $(newer-prereqs)$(cmd-check)$(check-FORCE)
2021-08-13 09:30:04 +03:00
2007-02-06 04:18:21 +03:00
# Execute command if command has changed or prerequisite(s) are updated.
2022-05-27 13:01:54 +03:00
if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:)
cmd_and_savecmd = \
2018-11-30 04:05:29 +03:00
$(cmd); \
2022-12-29 12:15:00 +03:00
printf '%s\n' 'savedcmd_$@ := $(make-cmd)' > $(dot-target).cmd
2005-07-26 00:10:36 +04:00
2007-02-06 04:18:21 +03:00
# Execute the command and also postprocess generated .d dependencies file.
2021-08-13 09:30:04 +03:00
if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:)
kbuild: add fine grained build dependencies for exported symbols
Like with kconfig options, we now have the ability to compile in and
out individual EXPORT_SYMBOL() declarations based on the content of
include/generated/autoksyms.h. However we don't want the entire
world to be rebuilt whenever that file is touched.
Let's apply the same build dependency trick used for CONFIG_* symbols
where the time stamp of empty files whose paths matching those symbols
is used to trigger fine grained rebuilds. In our case the key is the
symbol name passed to EXPORT_SYMBOL().
However, unlike config options, we cannot just use fixdep to parse
the source code for EXPORT_SYMBOL(ksym) because several variants exist
and parsing them all in a separate tool, and keeping it in synch, is
not trivially maintainable. Furthermore, there are variants such as
EXPORT_SYMBOL_GPL(pci_user_read_config_##size);
that are instanciated via a macro for which we can't easily determine
the actual exported symbol name(s) short of actually running the
preprocessor on them.
Storing the symbol name string in a special ELF section doesn't work
for targets that output assembly or preprocessed source.
So the best way is really to leverage the preprocessor by having it
output actual symbol names anchored by a special sequence that can be
easily filtered out. Then the list of symbols is simply fed to fixdep
to be merged with the other dependencies.
That implies the preprocessor is executed twice for each source file.
A previous attempt relied on a warning pragma for each EXPORT_SYMBOL()
instance that was filtered apart from stderr by the build system with
a sed script during the actual compilation pass. Unfortunately the
preprocessor/compiler diagnostic output isn't stable between versions
and this solution, although more efficient, was deemed too fragile.
Because of the lowercasing performed by fixdep, there might be name
collisions triggering spurious rebuilds for similar symbols. But this
shouldn't be a big issue in practice. (This is the case for CONFIG_*
symbols and I didn't want to be different here, whatever the original
reason for doing so.)
To avoid needless build overhead, the exported symbol name gathering is
performed only when CONFIG_TRIM_UNUSED_KSYMS is selected.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
2016-01-22 21:41:57 +03:00
2016-02-17 23:50:06 +03:00
cmd_and_fixdep = \
2018-11-30 04:05:27 +03:00
$(cmd); \
kbuild: let fixdep directly write to .*.cmd files
Currently, fixdep writes dependencies to .*.tmp, which is renamed to
.*.cmd after everything succeeds. This is a very safe way to avoid
corrupted .*.cmd files. The if_changed_dep has carried this safety
mechanism since it was added in 2002.
If fixdep fails for some reasons or a user terminates the build while
fixdep is running, the incomplete output from the fixdep could be
troublesome.
This is my insight about some bad scenarios:
[1] If the compiler succeeds to generate *.o file, but fixdep fails
to write necessary dependencies to .*.cmd file, Make will miss
to rebuild the object when headers or CONFIG options are changed.
In this case, fixdep should not generate .*.cmd file at all so
that 'arg-check' will surely trigger the rebuild of the object.
[2] A partially constructed .*.cmd file may not be a syntactically
correct makefile. The next time Make runs, it would include it,
then fail to parse it. Once this happens, 'make clean' is be the
only way to fix it.
In fact, [1] is no longer a problem since commit 9c2af1c7377a ("kbuild:
add .DELETE_ON_ERROR special target"). Make deletes a target file on
any failure in its recipe. Because fixdep is a part of the recipe of
*.o target, if it fails, the *.o is deleted anyway. However, I am a
bit worried about the slight possibility of [2].
So, here is a solution. Let fixdep directly write to a .*.cmd file,
but allow makefiles to include it only when its corresponding target
exists.
This effectively reverts commit 2982c953570b ("kbuild: remove redundant
$(wildcard ...) for cmd_files calculation"), and commit 00d78ab2ba75
("kbuild: remove dead code in cmd_files calculation in top Makefile")
because now we must check the presence of targets.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-11-30 04:05:22 +03:00
scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;\
2018-11-30 04:05:28 +03:00
rm -f $(depfile)
kbuild: add fine grained build dependencies for exported symbols
Like with kconfig options, we now have the ability to compile in and
out individual EXPORT_SYMBOL() declarations based on the content of
include/generated/autoksyms.h. However we don't want the entire
world to be rebuilt whenever that file is touched.
Let's apply the same build dependency trick used for CONFIG_* symbols
where the time stamp of empty files whose paths matching those symbols
is used to trigger fine grained rebuilds. In our case the key is the
symbol name passed to EXPORT_SYMBOL().
However, unlike config options, we cannot just use fixdep to parse
the source code for EXPORT_SYMBOL(ksym) because several variants exist
and parsing them all in a separate tool, and keeping it in synch, is
not trivially maintainable. Furthermore, there are variants such as
EXPORT_SYMBOL_GPL(pci_user_read_config_##size);
that are instanciated via a macro for which we can't easily determine
the actual exported symbol name(s) short of actually running the
preprocessor on them.
Storing the symbol name string in a special ELF section doesn't work
for targets that output assembly or preprocessed source.
So the best way is really to leverage the preprocessor by having it
output actual symbol names anchored by a special sequence that can be
easily filtered out. Then the list of symbols is simply fed to fixdep
to be merged with the other dependencies.
That implies the preprocessor is executed twice for each source file.
A previous attempt relied on a warning pragma for each EXPORT_SYMBOL()
instance that was filtered apart from stderr by the build system with
a sed script during the actual compilation pass. Unfortunately the
preprocessor/compiler diagnostic output isn't stable between versions
and this solution, although more efficient, was deemed too fragile.
Because of the lowercasing performed by fixdep, there might be name
collisions triggering spurious rebuilds for similar symbols. But this
shouldn't be a big issue in practice. (This is the case for CONFIG_*
symbols and I didn't want to be different here, whatever the original
reason for doing so.)
To avoid needless build overhead, the exported symbol name gathering is
performed only when CONFIG_TRIM_UNUSED_KSYMS is selected.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
2016-01-22 21:41:57 +03:00
2005-07-26 00:10:36 +04:00
# Usage: $(call if_changed_rule,foo)
2007-02-09 00:48:51 +03:00
# Will check if $(cmd_foo) or any of the prerequisites changed,
# and if so will execute $(rule_foo).
2021-08-13 09:30:04 +03:00
if_changed_rule = $(if $(if-changed-cond),$(rule_$(1)),@:)
2006-07-23 21:37:44 +04:00
2006-08-08 23:35:14 +04:00
###
2017-08-02 05:31:06 +03:00
# why - tell why a target got built
2006-08-08 23:35:14 +04:00
# enabled by make V=2
# Output (listed in the order they are checked):
# (1) - due to target is PHONY
# (2) - due to target missing
# (3) - due to: file1.h file2.h
# (4) - due to command line change
# (5) - due to missing .cmd file
# (6) - due to target not in $(targets)
# (1) PHONY targets are always build
# (2) No target, so we better build it
# (3) Prerequisite is newer than target
# (4) The command line stored in the file named dir/.target.cmd
# differed from actual command line. This happens when compiler
# options changes
# (5) No dir/.target.cmd file (used to store command line)
# (6) No dir/.target.cmd file and target not listed in $(targets)
# This is a good hint that there is a bug in the kbuild file
2022-12-22 19:25:34 +03:00
ifneq ($(findstring 2, $(KBUILD_VERBOSE)),)
2022-12-22 19:25:32 +03:00
_why = \
2006-08-08 23:35:14 +04:00
$(if $(filter $@, $(PHONY)),- due to target is PHONY, \
$(if $(wildcard $@), \
2019-11-07 18:09:45 +03:00
$(if $(newer-prereqs),- due to: $(newer-prereqs), \
2019-06-22 19:07:03 +03:00
$(if $(cmd-check), \
2022-12-29 12:15:00 +03:00
$(if $(savedcmd_$@),- due to command line change, \
2006-08-08 23:35:14 +04:00
$(if $(filter $@, $(targets)), \
- due to missing .cmd file, \
- due to $(notdir $@) not in $$(targets) \
) \
) \
) \
), \
- due to target missing \
) \
)
2022-12-22 19:25:32 +03:00
why = $(space)$(strip $(_why))
2006-08-08 23:35:14 +04:00
endif
2015-08-14 18:17:16 +03:00
###############################################################################
2018-07-20 10:46:33 +03:00
# delete partially updated (i.e. corrupted) files on error
.DELETE_ON_ERROR:
2018-12-01 03:27:15 +03:00
# do not delete intermediate files automatically
kbuild: use .NOTINTERMEDIATE for future GNU Make versions
In Kbuild, some files are generated by chains of pattern/implicit rules.
For example, *.dtb.o files in drivers/of/unittest-data/Makefile are
generated by the chain of 3 pattern rules, like this:
%.dts -> %.dtb -> %.dtb.S -> %.dtb.o
Here, %.dts is the real source, %.dtb.o is the final target.
%.dtb and %.dtb.S are called "intermediate files".
As GNU Make manual [1] says, intermediate files are treated differently
in two ways:
(a) The first difference is what happens if the intermediate file does
not exist. If an ordinary file 'b' does not exist, and make considers
a target that depends on 'b', it invariably creates 'b' and then
updates the target from 'b'. But if 'b' is an intermediate file, then
make can leave well enough alone: it won't create 'b' unless one of
its prerequisites is out of date. This means the target depending
on 'b' won't be rebuilt either, unless there is some other reason
to update that target: for example the target doesn't exist or a
different prerequisite is newer than the target.
(b) The second difference is that if make does create 'b' in order to
update something else, it deletes 'b' later on after it is no longer
needed. Therefore, an intermediate file which did not exist before
make also does not exist after make. make reports the deletion to
you by printing a 'rm' command showing which file it is deleting.
The combination of these is problematic for Kbuild because most of the
build rules depend on FORCE and the if_changed* macros really determine
if the target should be updated. So, all missing files, whether they
are intermediate or not, are always rebuilt.
To see the problem, delete ".SECONDARY:" from scripts/Kbuild.include,
and repeat this command:
$ make allmodconfig drivers/of/unittest-data/
The intermediate files will be deleted, which results in rebuilding
intermediate and final objects in the next run of make.
In the old days, people suppressed (b) in inconsistent ways.
As commit 54a702f70589 ("kbuild: mark $(targets) as .SECONDARY and
remove .PRECIOUS markers") noted, you should not use .PRECIOUS because
.PRECIOUS has the following behavior (c), which is not likely what you
want.
(c) If make is killed or interrupted during the execution of their
recipes, the target is not deleted. Also, the target is not deleted
on error even if .DELETE_ON_ERROR is specified.
.SECONDARY is a much better way to disable (b), but a small problem
is that .SECONDARY enables (a), which gives a side-effect to $?;
prerequisites marked as .SECONDARY do not appear in $?. This is a
drawback for Kbuild.
I thought it was a bug and opened a bug report. As Paul, the GNU Make
maintainer, concluded in [2], this is not a bug.
A good news is that, GNU Make 4.4 added the perfect solution,
.NOTINTERMEDIATE, which cancels both (a) and (b).
For clarificaton, my understanding of .INTERMEDIATE, .SECONDARY,
.PRECIOUS and .NOTINTERMEDIATE are as follows:
(a) (b) (c)
.INTERMEDIATE enable enable disable
.SECONDARY enable disable disable
.PRECIOUS disable disable enable
.NOTINTERMEDIATE disable disable disable
However, GNU Make 4.4 has a bug for the global .NOTINTERMEDIATE. [3]
It was fixed by commit 6164608900ad ("[SV 63417] Ensure global
.NOTINTERMEDIATE disables all intermediates"), and will be available
in the next release of GNU Make.
The following is the gain for .NOTINTERMEDIATE:
[Current Make]
$ make allnoconfig vmlinux
[ full build ]
$ rm include/linux/device.h
$ make vmlinux
CALL scripts/checksyscalls.sh
Make does not notice the removal of <linux/device.h>.
[Future Make]
$ make-latest allnoconfig vmlinux
[ full build ]
$ rm include/linux/device.h
$ make-latest vmlinux
CC arch/x86/kernel/asm-offsets.s
In file included from ./include/linux/writeback.h:13,
from ./include/linux/memcontrol.h:22,
from ./include/linux/swap.h:9,
from ./include/linux/suspend.h:5,
from arch/x86/kernel/asm-offsets.c:13:
./include/linux/blk_types.h:11:10: fatal error: linux/device.h: No such file or directory
11 | #include <linux/device.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make-latest[1]: *** [scripts/Makefile.build:114: arch/x86/kernel/asm-offsets.s] Error 1
make-latest: *** [Makefile:1282: prepare0] Error 2
Make notices the removal of <linux/device.h>, and rebuilds objects
that depended on <linux/device.h>. There exists a source file that
includes <linux/device.h>, and it raises an error.
To see detailed background information, refer to commit 2d3b1b8f0da7
("kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild").
[1]: https://www.gnu.org/software/make/manual/make.html#Chained-Rules
[2]: https://savannah.gnu.org/bugs/?55532
[3]: https://savannah.gnu.org/bugs/?63417
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-12-11 06:10:59 +03:00
#
# .NOTINTERMEDIATE is more correct, but only available on newer Make versions.
# Make 4.4 introduced .NOTINTERMEDIATE, and it appears in .FEATURES, but the
# global .NOTINTERMEDIATE does not work. We can use it on Make > 4.4.
# Use .SECONDARY for older Make versions, but "newer-prereq" cannot detect
# deleted files.
ifneq ($(and $(filter notintermediate, $(.FEATURES)),$(filter-out 4.4,$(MAKE_VERSION))),)
.NOTINTERMEDIATE:
else
2018-12-01 03:27:15 +03:00
.SECONDARY:
kbuild: use .NOTINTERMEDIATE for future GNU Make versions
In Kbuild, some files are generated by chains of pattern/implicit rules.
For example, *.dtb.o files in drivers/of/unittest-data/Makefile are
generated by the chain of 3 pattern rules, like this:
%.dts -> %.dtb -> %.dtb.S -> %.dtb.o
Here, %.dts is the real source, %.dtb.o is the final target.
%.dtb and %.dtb.S are called "intermediate files".
As GNU Make manual [1] says, intermediate files are treated differently
in two ways:
(a) The first difference is what happens if the intermediate file does
not exist. If an ordinary file 'b' does not exist, and make considers
a target that depends on 'b', it invariably creates 'b' and then
updates the target from 'b'. But if 'b' is an intermediate file, then
make can leave well enough alone: it won't create 'b' unless one of
its prerequisites is out of date. This means the target depending
on 'b' won't be rebuilt either, unless there is some other reason
to update that target: for example the target doesn't exist or a
different prerequisite is newer than the target.
(b) The second difference is that if make does create 'b' in order to
update something else, it deletes 'b' later on after it is no longer
needed. Therefore, an intermediate file which did not exist before
make also does not exist after make. make reports the deletion to
you by printing a 'rm' command showing which file it is deleting.
The combination of these is problematic for Kbuild because most of the
build rules depend on FORCE and the if_changed* macros really determine
if the target should be updated. So, all missing files, whether they
are intermediate or not, are always rebuilt.
To see the problem, delete ".SECONDARY:" from scripts/Kbuild.include,
and repeat this command:
$ make allmodconfig drivers/of/unittest-data/
The intermediate files will be deleted, which results in rebuilding
intermediate and final objects in the next run of make.
In the old days, people suppressed (b) in inconsistent ways.
As commit 54a702f70589 ("kbuild: mark $(targets) as .SECONDARY and
remove .PRECIOUS markers") noted, you should not use .PRECIOUS because
.PRECIOUS has the following behavior (c), which is not likely what you
want.
(c) If make is killed or interrupted during the execution of their
recipes, the target is not deleted. Also, the target is not deleted
on error even if .DELETE_ON_ERROR is specified.
.SECONDARY is a much better way to disable (b), but a small problem
is that .SECONDARY enables (a), which gives a side-effect to $?;
prerequisites marked as .SECONDARY do not appear in $?. This is a
drawback for Kbuild.
I thought it was a bug and opened a bug report. As Paul, the GNU Make
maintainer, concluded in [2], this is not a bug.
A good news is that, GNU Make 4.4 added the perfect solution,
.NOTINTERMEDIATE, which cancels both (a) and (b).
For clarificaton, my understanding of .INTERMEDIATE, .SECONDARY,
.PRECIOUS and .NOTINTERMEDIATE are as follows:
(a) (b) (c)
.INTERMEDIATE enable enable disable
.SECONDARY enable disable disable
.PRECIOUS disable disable enable
.NOTINTERMEDIATE disable disable disable
However, GNU Make 4.4 has a bug for the global .NOTINTERMEDIATE. [3]
It was fixed by commit 6164608900ad ("[SV 63417] Ensure global
.NOTINTERMEDIATE disables all intermediates"), and will be available
in the next release of GNU Make.
The following is the gain for .NOTINTERMEDIATE:
[Current Make]
$ make allnoconfig vmlinux
[ full build ]
$ rm include/linux/device.h
$ make vmlinux
CALL scripts/checksyscalls.sh
Make does not notice the removal of <linux/device.h>.
[Future Make]
$ make-latest allnoconfig vmlinux
[ full build ]
$ rm include/linux/device.h
$ make-latest vmlinux
CC arch/x86/kernel/asm-offsets.s
In file included from ./include/linux/writeback.h:13,
from ./include/linux/memcontrol.h:22,
from ./include/linux/swap.h:9,
from ./include/linux/suspend.h:5,
from arch/x86/kernel/asm-offsets.c:13:
./include/linux/blk_types.h:11:10: fatal error: linux/device.h: No such file or directory
11 | #include <linux/device.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make-latest[1]: *** [scripts/Makefile.build:114: arch/x86/kernel/asm-offsets.s] Error 1
make-latest: *** [Makefile:1282: prepare0] Error 2
Make notices the removal of <linux/device.h>, and rebuilds objects
that depended on <linux/device.h>. There exists a source file that
includes <linux/device.h>, and it raises an error.
To see detailed background information, refer to commit 2d3b1b8f0da7
("kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild").
[1]: https://www.gnu.org/software/make/manual/make.html#Chained-Rules
[2]: https://savannah.gnu.org/bugs/?55532
[3]: https://savannah.gnu.org/bugs/?63417
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-12-11 06:10:59 +03:00
endif
