2021-07-03 17:42:57 +03:00
# SPDX-License-Identifier: GPL-2.0
2023-07-18 18:15:33 +03:00
# Where to place rustdoc generated documentation
rustdoc_output := $( objtree) /Documentation/output/rust/rustdoc
2021-07-03 17:42:57 +03:00
obj-$(CONFIG_RUST) += core.o compiler_builtins.o
always-$(CONFIG_RUST) += exports_core_generated.h
# Missing prototypes are expected in the helpers since these are exported
# for Rust only, thus there is no header nor prototypes.
obj-$(CONFIG_RUST) += helpers.o
CFLAGS_REMOVE_helpers.o = -Wmissing-prototypes -Wmissing-declarations
always-$(CONFIG_RUST) += libmacros.so
no-clean-files += libmacros.so
always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
obj-$(CONFIG_RUST) += alloc.o bindings.o kernel.o
always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
exports_kernel_generated.h
2023-04-03 12:33:52 +03:00
always-$(CONFIG_RUST) += uapi/uapi_generated.rs
obj-$(CONFIG_RUST) += uapi.o
2022-11-10 19:41:37 +03:00
i f d e f C O N F I G _ R U S T _ B U I L D _ A S S E R T _ A L L O W
obj-$(CONFIG_RUST) += build_error.o
e l s e
always-$(CONFIG_RUST) += build_error.o
e n d i f
2021-07-03 17:42:57 +03:00
obj-$(CONFIG_RUST) += exports.o
rust: support running Rust documentation tests as KUnit ones
Rust has documentation tests: these are typically examples of
usage of any item (e.g. function, struct, module...).
They are very convenient because they are just written
alongside the documentation. For instance:
/// Sums two numbers.
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
In userspace, the tests are collected and run via `rustdoc`.
Using the tool as-is would be useful already, since it allows
to compile-test most tests (thus enforcing they are kept
in sync with the code they document) and run those that do not
depend on in-kernel APIs.
However, by transforming the tests into a KUnit test suite,
they can also be run inside the kernel. Moreover, the tests
get to be compiled as other Rust kernel objects instead of
targeting userspace.
On top of that, the integration with KUnit means the Rust
support gets to reuse the existing testing facilities. For
instance, the kernel log would look like:
KTAP version 1
1..1
KTAP version 1
# Subtest: rust_doctests_kernel
1..59
# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
ok 1 rust_doctest_kernel_build_assert_rs_0
# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
ok 2 rust_doctest_kernel_build_assert_rs_1
# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
ok 3 rust_doctest_kernel_init_rs_0
...
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
ok 59 rust_doctest_kernel_types_rs_2
# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
# Totals: pass:59 fail:0 skip:0 total:59
ok 1 rust_doctests_kernel
Therefore, add support for running Rust documentation tests
in KUnit. Some other notes about the current implementation
and support follow.
The transformation is performed by a couple scripts written
as Rust hostprogs.
Tests using the `?` operator are also supported as usual, e.g.:
/// ```
/// # use kernel::{spawn_work_item, workqueue};
/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
/// # Ok::<(), Error>(())
/// ```
The tests are also compiled with Clippy under `CLIPPY=1`, just
like normal code, thus also benefitting from extra linting.
The names of the tests are currently automatically generated.
This allows to reduce the burden for documentation writers,
while keeping them fairly stable for bisection. This is an
improvement over the `rustdoc`-generated names, which include
the line number; but ideally we would like to get `rustdoc` to
provide the Rust item path and a number (for multiple examples
in a single documented Rust item).
In order for developers to easily see from which original line
a failed doctests came from, a KTAP diagnostic line is printed
to the log, containing the location (file and line) of the
original test (i.e. instead of the location in the generated
Rust file):
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
This line follows the syntax for declaring test metadata in the
proposed KTAP v2 spec [1], which may be used for the proposed
KUnit test attributes API [2]. Thus hopefully this will make
migration easier later on (suggested by David [3]).
The original line in that test attribute is figured out by
providing an anchor (suggested by Boqun [4]). The original file
is found by walking the filesystem, checking directory prefixes
to reduce the amount of combinations to check, and it is only
done once per file. Ambiguities are detected and reported.
A notable difference from KUnit C tests is that the Rust tests
appear to assert using the usual `assert!` and `assert_eq!`
macros from the Rust standard library (`core`). We provide
a custom version that forwards the call to KUnit instead.
Importantly, these macros do not require passing context,
unlike the KUnit C ones (i.e. `struct kunit *`). This makes
them easier to use, and readers of the documentation do not need
to care about which testing framework is used. In addition, it
may allow us to test third-party code more easily in the future.
However, a current limitation is that KUnit does not support
assertions in other tasks. Thus we presently simply print an
error to the kernel log if an assertion actually failed. This
should be revisited to properly fail the test, perhaps saving
the context somewhere else, or letting KUnit handle it.
Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1]
Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2]
Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3]
Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-18 08:27:51 +03:00
always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.rs
always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.c
obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.o
obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o
2021-07-03 17:42:57 +03:00
# Avoids running `$(RUSTC)` for the sysroot when it may not be available.
i f d e f C O N F I G _ R U S T
# `$(rust_flags)` is passed in case the user added `--sysroot`.
rustc_sysroot := $( shell $( RUSTC) $( rust_flags) --print sysroot)
rustc_host_target := $( shell $( RUSTC) --version --verbose | grep -F 'host: ' | cut -d' ' -f2)
RUST_LIB_SRC ?= $( rustc_sysroot) /lib/rustlib/src/rust/library
i f e q ( $( quiet ) , s i l e n t _ )
cargo_quiet = -q
rust_test_quiet = -q
rustdoc_test_quiet = --test-args -q
rust: support running Rust documentation tests as KUnit ones
Rust has documentation tests: these are typically examples of
usage of any item (e.g. function, struct, module...).
They are very convenient because they are just written
alongside the documentation. For instance:
/// Sums two numbers.
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
In userspace, the tests are collected and run via `rustdoc`.
Using the tool as-is would be useful already, since it allows
to compile-test most tests (thus enforcing they are kept
in sync with the code they document) and run those that do not
depend on in-kernel APIs.
However, by transforming the tests into a KUnit test suite,
they can also be run inside the kernel. Moreover, the tests
get to be compiled as other Rust kernel objects instead of
targeting userspace.
On top of that, the integration with KUnit means the Rust
support gets to reuse the existing testing facilities. For
instance, the kernel log would look like:
KTAP version 1
1..1
KTAP version 1
# Subtest: rust_doctests_kernel
1..59
# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
ok 1 rust_doctest_kernel_build_assert_rs_0
# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
ok 2 rust_doctest_kernel_build_assert_rs_1
# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
ok 3 rust_doctest_kernel_init_rs_0
...
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
ok 59 rust_doctest_kernel_types_rs_2
# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
# Totals: pass:59 fail:0 skip:0 total:59
ok 1 rust_doctests_kernel
Therefore, add support for running Rust documentation tests
in KUnit. Some other notes about the current implementation
and support follow.
The transformation is performed by a couple scripts written
as Rust hostprogs.
Tests using the `?` operator are also supported as usual, e.g.:
/// ```
/// # use kernel::{spawn_work_item, workqueue};
/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
/// # Ok::<(), Error>(())
/// ```
The tests are also compiled with Clippy under `CLIPPY=1`, just
like normal code, thus also benefitting from extra linting.
The names of the tests are currently automatically generated.
This allows to reduce the burden for documentation writers,
while keeping them fairly stable for bisection. This is an
improvement over the `rustdoc`-generated names, which include
the line number; but ideally we would like to get `rustdoc` to
provide the Rust item path and a number (for multiple examples
in a single documented Rust item).
In order for developers to easily see from which original line
a failed doctests came from, a KTAP diagnostic line is printed
to the log, containing the location (file and line) of the
original test (i.e. instead of the location in the generated
Rust file):
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
This line follows the syntax for declaring test metadata in the
proposed KTAP v2 spec [1], which may be used for the proposed
KUnit test attributes API [2]. Thus hopefully this will make
migration easier later on (suggested by David [3]).
The original line in that test attribute is figured out by
providing an anchor (suggested by Boqun [4]). The original file
is found by walking the filesystem, checking directory prefixes
to reduce the amount of combinations to check, and it is only
done once per file. Ambiguities are detected and reported.
A notable difference from KUnit C tests is that the Rust tests
appear to assert using the usual `assert!` and `assert_eq!`
macros from the Rust standard library (`core`). We provide
a custom version that forwards the call to KUnit instead.
Importantly, these macros do not require passing context,
unlike the KUnit C ones (i.e. `struct kunit *`). This makes
them easier to use, and readers of the documentation do not need
to care about which testing framework is used. In addition, it
may allow us to test third-party code more easily in the future.
However, a current limitation is that KUnit does not support
assertions in other tasks. Thus we presently simply print an
error to the kernel log if an assertion actually failed. This
should be revisited to properly fail the test, perhaps saving
the context somewhere else, or letting KUnit handle it.
Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1]
Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2]
Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3]
Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-18 08:27:51 +03:00
rustdoc_test_kernel_quiet = >/dev/null
2021-07-03 17:42:57 +03:00
e l s e i f e q ( $( quiet ) , q u i e t _ )
rust_test_quiet = -q
rustdoc_test_quiet = --test-args -q
rust: support running Rust documentation tests as KUnit ones
Rust has documentation tests: these are typically examples of
usage of any item (e.g. function, struct, module...).
They are very convenient because they are just written
alongside the documentation. For instance:
/// Sums two numbers.
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
In userspace, the tests are collected and run via `rustdoc`.
Using the tool as-is would be useful already, since it allows
to compile-test most tests (thus enforcing they are kept
in sync with the code they document) and run those that do not
depend on in-kernel APIs.
However, by transforming the tests into a KUnit test suite,
they can also be run inside the kernel. Moreover, the tests
get to be compiled as other Rust kernel objects instead of
targeting userspace.
On top of that, the integration with KUnit means the Rust
support gets to reuse the existing testing facilities. For
instance, the kernel log would look like:
KTAP version 1
1..1
KTAP version 1
# Subtest: rust_doctests_kernel
1..59
# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
ok 1 rust_doctest_kernel_build_assert_rs_0
# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
ok 2 rust_doctest_kernel_build_assert_rs_1
# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
ok 3 rust_doctest_kernel_init_rs_0
...
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
ok 59 rust_doctest_kernel_types_rs_2
# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
# Totals: pass:59 fail:0 skip:0 total:59
ok 1 rust_doctests_kernel
Therefore, add support for running Rust documentation tests
in KUnit. Some other notes about the current implementation
and support follow.
The transformation is performed by a couple scripts written
as Rust hostprogs.
Tests using the `?` operator are also supported as usual, e.g.:
/// ```
/// # use kernel::{spawn_work_item, workqueue};
/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
/// # Ok::<(), Error>(())
/// ```
The tests are also compiled with Clippy under `CLIPPY=1`, just
like normal code, thus also benefitting from extra linting.
The names of the tests are currently automatically generated.
This allows to reduce the burden for documentation writers,
while keeping them fairly stable for bisection. This is an
improvement over the `rustdoc`-generated names, which include
the line number; but ideally we would like to get `rustdoc` to
provide the Rust item path and a number (for multiple examples
in a single documented Rust item).
In order for developers to easily see from which original line
a failed doctests came from, a KTAP diagnostic line is printed
to the log, containing the location (file and line) of the
original test (i.e. instead of the location in the generated
Rust file):
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
This line follows the syntax for declaring test metadata in the
proposed KTAP v2 spec [1], which may be used for the proposed
KUnit test attributes API [2]. Thus hopefully this will make
migration easier later on (suggested by David [3]).
The original line in that test attribute is figured out by
providing an anchor (suggested by Boqun [4]). The original file
is found by walking the filesystem, checking directory prefixes
to reduce the amount of combinations to check, and it is only
done once per file. Ambiguities are detected and reported.
A notable difference from KUnit C tests is that the Rust tests
appear to assert using the usual `assert!` and `assert_eq!`
macros from the Rust standard library (`core`). We provide
a custom version that forwards the call to KUnit instead.
Importantly, these macros do not require passing context,
unlike the KUnit C ones (i.e. `struct kunit *`). This makes
them easier to use, and readers of the documentation do not need
to care about which testing framework is used. In addition, it
may allow us to test third-party code more easily in the future.
However, a current limitation is that KUnit does not support
assertions in other tasks. Thus we presently simply print an
error to the kernel log if an assertion actually failed. This
should be revisited to properly fail the test, perhaps saving
the context somewhere else, or letting KUnit handle it.
Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1]
Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2]
Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3]
Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-18 08:27:51 +03:00
rustdoc_test_kernel_quiet = >/dev/null
2021-07-03 17:42:57 +03:00
e l s e
cargo_quiet = --verbose
e n d i f
core-cfgs = \
--cfg no_fp_fmt_parse
alloc-cfgs = \
2022-12-06 04:05:19 +03:00
--cfg no_borrow \
2021-07-03 17:42:57 +03:00
--cfg no_fmt \
--cfg no_global_oom_handling \
--cfg no_macros \
--cfg no_rc \
--cfg no_str \
--cfg no_string \
--cfg no_sync \
--cfg no_thin
quiet_cmd_rustdoc = RUSTDOC $( if $( rustdoc_host) ,H, ) $<
cmd_rustdoc = \
OBJTREE = $( abspath $( objtree) ) \
$( RUSTDOC) $( if $( rustdoc_host) ,$( rust_common_flags) ,$( rust_flags) ) \
$( rustc_target_flags) -L$( objtree) /$( obj) \
2023-07-18 18:15:33 +03:00
--output $( rustdoc_output) \
2021-07-03 17:42:57 +03:00
--crate-name $( subst rustdoc-,,$@ ) \
2023-10-31 23:10:14 +03:00
$( if $( rustdoc_host) ,,--sysroot= /dev/null) \
2021-07-03 17:42:57 +03:00
@$( objtree) /include/generated/rustc_cfg $<
# The `html_logo_url` and `html_favicon_url` forms of the `doc` attribute
# can be used to specify a custom logo. However:
# - The given value is used as-is, thus it cannot be relative or a local file
# (unlike the non-custom case) since the generated docs have subfolders.
# - It requires adding it to every crate.
# - It requires changing `core` which comes from the sysroot.
#
# Using `-Zcrate-attr` would solve the last two points, but not the first.
# The https://github.com/rust-lang/rfcs/pull/3226 RFC suggests two new
# command-like flags to solve the issue. Meanwhile, we use the non-custom case
# and then retouch the generated files.
rustdoc : rustdoc -core rustdoc -macros rustdoc -compiler_builtins \
rustdoc-alloc rustdoc-kernel
2023-10-18 18:55:27 +03:00
$( Q) cp $( srctree) /Documentation/images/logo.svg $( rustdoc_output) /static.files/
$( Q) cp $( srctree) /Documentation/images/COPYING-logo $( rustdoc_output) /static.files/
2023-07-18 18:15:33 +03:00
$( Q) find $( rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \
2023-10-18 18:55:27 +03:00
-e 's:rust-logo-[0-9a-f]+\.svg:logo.svg:g' \
-e 's:favicon-[0-9a-f]+\.svg:logo.svg:g' \
2023-12-16 02:54:28 +03:00
-e 's:<link rel="alternate icon" type="image/png" href="[/.]+/static\.files/favicon-(16x16|32x32)-[0-9a-f]+\.png">::g' \
-e 's:<a href="srctree/([^"]+)">:<a href="$(abs_srctree)/\1">:g'
2023-10-18 18:55:27 +03:00
$( Q) for f in $( rustdoc_output) /static.files/rustdoc-*.css; do \
echo ".logo-container > img { object-fit: contain; }" >> $$ f; done
2021-07-03 17:42:57 +03:00
rustdoc-macros : private rustdoc_host = yes
rustdoc-macros : private rustc_target_flags = --crate -type proc -macro \
--extern proc_macro
rustdoc-macros : $( src ) /macros /lib .rs FORCE
$( call if_changed,rustdoc)
rustdoc-core : private rustc_target_flags = $( core -cfgs )
rustdoc-core : $( RUST_LIB_SRC ) /core /src /lib .rs FORCE
$( call if_changed,rustdoc)
rustdoc-compiler_builtins : $( src ) /compiler_builtins .rs rustdoc -core FORCE
$( call if_changed,rustdoc)
# We need to allow `rustdoc::broken_intra_doc_links` because some
# `no_global_oom_handling` functions refer to non-`no_global_oom_handling`
# functions. Ideally `rustdoc` would have a way to distinguish broken links
# due to things that are "configured out" vs. entirely non-existing ones.
rustdoc-alloc : private rustc_target_flags = $( alloc -cfgs ) \
-Arustdoc::broken_intra_doc_links
rustdoc-alloc : $( src ) /alloc /lib .rs rustdoc -core rustdoc -compiler_builtins FORCE
$( call if_changed,rustdoc)
rustdoc-kernel : private rustc_target_flags = --extern alloc \
2022-11-10 19:41:37 +03:00
--extern build_error --extern macros = $( objtree) /$( obj) /libmacros.so \
2023-04-03 12:33:52 +03:00
--extern bindings --extern uapi
2021-07-03 17:42:57 +03:00
rustdoc-kernel : $( src ) /kernel /lib .rs rustdoc -core rustdoc -macros \
rustdoc-compiler_builtins rustdoc-alloc $( obj) /libmacros.so \
$( obj) /bindings.o FORCE
$( call if_changed,rustdoc)
quiet_cmd_rustc_test_library = RUSTC TL $<
cmd_rustc_test_library = \
OBJTREE = $( abspath $( objtree) ) \
$( RUSTC) $( rust_common_flags) \
@$( objtree) /include/generated/rustc_cfg $( rustc_target_flags) \
--crate-type $( if $( rustc_test_library_proc) ,proc-macro,rlib) \
--out-dir $( objtree) /$( obj) /test --cfg testlib \
--sysroot $( objtree) /$( obj) /test/sysroot \
-L$( objtree) /$( obj) /test \
--crate-name $( subst rusttest-,,$( subst rusttestlib-,,$@ ) ) $<
2022-11-10 19:41:37 +03:00
rusttestlib-build_error : $( src ) /build_error .rs rusttest -prepare FORCE
$( call if_changed,rustc_test_library)
2021-07-03 17:42:57 +03:00
rusttestlib-macros : private rustc_target_flags = --extern proc_macro
rusttestlib-macros : private rustc_test_library_proc = yes
rusttestlib-macros : $( src ) /macros /lib .rs rusttest -prepare FORCE
$( call if_changed,rustc_test_library)
rusttestlib-bindings : $( src ) /bindings /lib .rs rusttest -prepare FORCE
$( call if_changed,rustc_test_library)
2023-04-03 12:33:52 +03:00
rusttestlib-uapi : $( src ) /uapi /lib .rs rusttest -prepare FORCE
$( call if_changed,rustc_test_library)
2021-07-03 17:42:57 +03:00
quiet_cmd_rustdoc_test = RUSTDOC T $<
cmd_rustdoc_test = \
OBJTREE = $( abspath $( objtree) ) \
$( RUSTDOC) --test $( rust_common_flags) \
@$( objtree) /include/generated/rustc_cfg \
$( rustc_target_flags) $( rustdoc_test_target_flags) \
--sysroot $( objtree) /$( obj) /test/sysroot $( rustdoc_test_quiet) \
2023-07-18 18:15:33 +03:00
-L$( objtree) /$( obj) /test --output $( rustdoc_output) \
2021-07-03 17:42:57 +03:00
--crate-name $( subst rusttest-,,$@ ) $<
rust: support running Rust documentation tests as KUnit ones
Rust has documentation tests: these are typically examples of
usage of any item (e.g. function, struct, module...).
They are very convenient because they are just written
alongside the documentation. For instance:
/// Sums two numbers.
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
In userspace, the tests are collected and run via `rustdoc`.
Using the tool as-is would be useful already, since it allows
to compile-test most tests (thus enforcing they are kept
in sync with the code they document) and run those that do not
depend on in-kernel APIs.
However, by transforming the tests into a KUnit test suite,
they can also be run inside the kernel. Moreover, the tests
get to be compiled as other Rust kernel objects instead of
targeting userspace.
On top of that, the integration with KUnit means the Rust
support gets to reuse the existing testing facilities. For
instance, the kernel log would look like:
KTAP version 1
1..1
KTAP version 1
# Subtest: rust_doctests_kernel
1..59
# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
ok 1 rust_doctest_kernel_build_assert_rs_0
# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
ok 2 rust_doctest_kernel_build_assert_rs_1
# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
ok 3 rust_doctest_kernel_init_rs_0
...
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
ok 59 rust_doctest_kernel_types_rs_2
# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
# Totals: pass:59 fail:0 skip:0 total:59
ok 1 rust_doctests_kernel
Therefore, add support for running Rust documentation tests
in KUnit. Some other notes about the current implementation
and support follow.
The transformation is performed by a couple scripts written
as Rust hostprogs.
Tests using the `?` operator are also supported as usual, e.g.:
/// ```
/// # use kernel::{spawn_work_item, workqueue};
/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
/// # Ok::<(), Error>(())
/// ```
The tests are also compiled with Clippy under `CLIPPY=1`, just
like normal code, thus also benefitting from extra linting.
The names of the tests are currently automatically generated.
This allows to reduce the burden for documentation writers,
while keeping them fairly stable for bisection. This is an
improvement over the `rustdoc`-generated names, which include
the line number; but ideally we would like to get `rustdoc` to
provide the Rust item path and a number (for multiple examples
in a single documented Rust item).
In order for developers to easily see from which original line
a failed doctests came from, a KTAP diagnostic line is printed
to the log, containing the location (file and line) of the
original test (i.e. instead of the location in the generated
Rust file):
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
This line follows the syntax for declaring test metadata in the
proposed KTAP v2 spec [1], which may be used for the proposed
KUnit test attributes API [2]. Thus hopefully this will make
migration easier later on (suggested by David [3]).
The original line in that test attribute is figured out by
providing an anchor (suggested by Boqun [4]). The original file
is found by walking the filesystem, checking directory prefixes
to reduce the amount of combinations to check, and it is only
done once per file. Ambiguities are detected and reported.
A notable difference from KUnit C tests is that the Rust tests
appear to assert using the usual `assert!` and `assert_eq!`
macros from the Rust standard library (`core`). We provide
a custom version that forwards the call to KUnit instead.
Importantly, these macros do not require passing context,
unlike the KUnit C ones (i.e. `struct kunit *`). This makes
them easier to use, and readers of the documentation do not need
to care about which testing framework is used. In addition, it
may allow us to test third-party code more easily in the future.
However, a current limitation is that KUnit does not support
assertions in other tasks. Thus we presently simply print an
error to the kernel log if an assertion actually failed. This
should be revisited to properly fail the test, perhaps saving
the context somewhere else, or letting KUnit handle it.
Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1]
Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2]
Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3]
Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-18 08:27:51 +03:00
quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $<
cmd_rustdoc_test_kernel = \
rm -rf $( objtree) /$( obj) /test/doctests/kernel; \
mkdir -p $( objtree) /$( obj) /test/doctests/kernel; \
OBJTREE = $( abspath $( objtree) ) \
$( RUSTDOC) --test $( rust_flags) \
@$( objtree) /include/generated/rustc_cfg \
-L$( objtree) /$( obj) --extern alloc --extern kernel \
--extern build_error --extern macros \
--extern bindings --extern uapi \
--no-run --crate-name kernel -Zunstable-options \
2023-10-31 23:10:14 +03:00
--sysroot= /dev/null \
rust: support running Rust documentation tests as KUnit ones
Rust has documentation tests: these are typically examples of
usage of any item (e.g. function, struct, module...).
They are very convenient because they are just written
alongside the documentation. For instance:
/// Sums two numbers.
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
In userspace, the tests are collected and run via `rustdoc`.
Using the tool as-is would be useful already, since it allows
to compile-test most tests (thus enforcing they are kept
in sync with the code they document) and run those that do not
depend on in-kernel APIs.
However, by transforming the tests into a KUnit test suite,
they can also be run inside the kernel. Moreover, the tests
get to be compiled as other Rust kernel objects instead of
targeting userspace.
On top of that, the integration with KUnit means the Rust
support gets to reuse the existing testing facilities. For
instance, the kernel log would look like:
KTAP version 1
1..1
KTAP version 1
# Subtest: rust_doctests_kernel
1..59
# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
ok 1 rust_doctest_kernel_build_assert_rs_0
# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
ok 2 rust_doctest_kernel_build_assert_rs_1
# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
ok 3 rust_doctest_kernel_init_rs_0
...
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
ok 59 rust_doctest_kernel_types_rs_2
# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
# Totals: pass:59 fail:0 skip:0 total:59
ok 1 rust_doctests_kernel
Therefore, add support for running Rust documentation tests
in KUnit. Some other notes about the current implementation
and support follow.
The transformation is performed by a couple scripts written
as Rust hostprogs.
Tests using the `?` operator are also supported as usual, e.g.:
/// ```
/// # use kernel::{spawn_work_item, workqueue};
/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
/// # Ok::<(), Error>(())
/// ```
The tests are also compiled with Clippy under `CLIPPY=1`, just
like normal code, thus also benefitting from extra linting.
The names of the tests are currently automatically generated.
This allows to reduce the burden for documentation writers,
while keeping them fairly stable for bisection. This is an
improvement over the `rustdoc`-generated names, which include
the line number; but ideally we would like to get `rustdoc` to
provide the Rust item path and a number (for multiple examples
in a single documented Rust item).
In order for developers to easily see from which original line
a failed doctests came from, a KTAP diagnostic line is printed
to the log, containing the location (file and line) of the
original test (i.e. instead of the location in the generated
Rust file):
# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
This line follows the syntax for declaring test metadata in the
proposed KTAP v2 spec [1], which may be used for the proposed
KUnit test attributes API [2]. Thus hopefully this will make
migration easier later on (suggested by David [3]).
The original line in that test attribute is figured out by
providing an anchor (suggested by Boqun [4]). The original file
is found by walking the filesystem, checking directory prefixes
to reduce the amount of combinations to check, and it is only
done once per file. Ambiguities are detected and reported.
A notable difference from KUnit C tests is that the Rust tests
appear to assert using the usual `assert!` and `assert_eq!`
macros from the Rust standard library (`core`). We provide
a custom version that forwards the call to KUnit instead.
Importantly, these macros do not require passing context,
unlike the KUnit C ones (i.e. `struct kunit *`). This makes
them easier to use, and readers of the documentation do not need
to care about which testing framework is used. In addition, it
may allow us to test third-party code more easily in the future.
However, a current limitation is that KUnit does not support
assertions in other tasks. Thus we presently simply print an
error to the kernel log if an assertion actually failed. This
should be revisited to properly fail the test, perhaps saving
the context somewhere else, or letting KUnit handle it.
Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1]
Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2]
Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3]
Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-18 08:27:51 +03:00
--test-builder $( objtree) /scripts/rustdoc_test_builder \
$< $( rustdoc_test_kernel_quiet) ; \
$( objtree) /scripts/rustdoc_test_gen
%/doctests_kernel_generated.rs %/doctests_kernel_generated_kunit.c : \
$( src) /kernel/lib.rs $( obj) /kernel.o \
$( objtree) /scripts/rustdoc_test_builder \
$( objtree) /scripts/rustdoc_test_gen FORCE
$( call if_changed,rustdoc_test_kernel)
2021-07-03 17:42:57 +03:00
# We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
# so for the moment we skip `-Cpanic=abort`.
quiet_cmd_rustc_test = RUSTC T $<
cmd_rustc_test = \
OBJTREE = $( abspath $( objtree) ) \
$( RUSTC) --test $( rust_common_flags) \
@$( objtree) /include/generated/rustc_cfg \
$( rustc_target_flags) --out-dir $( objtree) /$( obj) /test \
--sysroot $( objtree) /$( obj) /test/sysroot \
-L$( objtree) /$( obj) /test \
--crate-name $( subst rusttest-,,$@ ) $<; \
$( objtree) /$( obj) /test/$( subst rusttest-,,$@ ) $( rust_test_quiet) \
$( rustc_test_run_flags)
rusttest : rusttest -macros rusttest -kernel
# This prepares a custom sysroot with our custom `alloc` instead of
# the standard one.
#
# This requires several hacks:
# - Unlike `core` and `alloc`, `std` depends on more than a dozen crates,
# including third-party crates that need to be downloaded, plus custom
# `build.rs` steps. Thus hardcoding things here is not maintainable.
# - `cargo` knows how to build the standard library, but it is an unstable
# feature so far (`-Zbuild-std`).
# - `cargo` only considers the use case of building the standard library
# to use it in a given package. Thus we need to create a dummy package
# and pick the generated libraries from there.
# - Since we only keep a subset of upstream `alloc` in-tree, we need
# to recreate it on the fly by putting our sources on top.
# - The usual ways of modifying the dependency graph in `cargo` do not seem
# to apply for the `-Zbuild-std` steps, thus we have to mislead it
# by modifying the sources in the sysroot.
# - To avoid messing with the user's Rust installation, we create a clone
# of the sysroot. However, `cargo` ignores `RUSTFLAGS` in the `-Zbuild-std`
# steps, thus we use a wrapper binary passed via `RUSTC` to pass the flag.
#
# In the future, we hope to avoid the whole ordeal by either:
# - Making the `test` crate not depend on `std` (either improving upstream
# or having our own custom crate).
# - Making the tests run in kernel space (requires the previous point).
# - Making `std` and friends be more like a "normal" crate, so that
# `-Zbuild-std` and related hacks are not needed.
quiet_cmd_rustsysroot = RUSTSYSROOT
cmd_rustsysroot = \
rm -rf $( objtree) /$( obj) /test; \
mkdir -p $( objtree) /$( obj) /test; \
cp -a $( rustc_sysroot) $( objtree) /$( obj) /test/sysroot; \
cp -r $( srctree) /$( src) /alloc/* \
$( objtree) /$( obj) /test/sysroot/lib/rustlib/src/rust/library/alloc/src; \
echo '\#!/bin/sh' > $( objtree) /$( obj) /test/rustc_sysroot; \
echo " $( RUSTC) --sysroot= $( abspath $( objtree) /$( obj) /test/sysroot) \"\$ $@ \" " \
>> $( objtree) /$( obj) /test/rustc_sysroot; \
chmod u+x $( objtree) /$( obj) /test/rustc_sysroot; \
$( CARGO) -q new $( objtree) /$( obj) /test/dummy; \
RUSTC = $( objtree) /$( obj) /test/rustc_sysroot $( CARGO) $( cargo_quiet) \
test -Zbuild-std --target $( rustc_host_target) \
--manifest-path $( objtree) /$( obj) /test/dummy/Cargo.toml; \
rm $( objtree) /$( obj) /test/sysroot/lib/rustlib/$( rustc_host_target) /lib/*; \
cp $( objtree) /$( obj) /test/dummy/target/$( rustc_host_target) /debug/deps/* \
$( objtree) /$( obj) /test/sysroot/lib/rustlib/$( rustc_host_target) /lib
rusttest-prepare : FORCE
$( call if_changed,rustsysroot)
rusttest-macros : private rustc_target_flags = --extern proc_macro
rusttest-macros : private rustdoc_test_target_flags = --crate -type proc -macro
rusttest-macros : $( src ) /macros /lib .rs rusttest -prepare FORCE
$( call if_changed,rustc_test)
$( call if_changed,rustdoc_test)
rusttest-kernel : private rustc_target_flags = --extern alloc \
2023-04-03 12:33:52 +03:00
--extern build_error --extern macros --extern bindings --extern uapi
2021-07-03 17:42:57 +03:00
rusttest-kernel : $( src ) /kernel /lib .rs rusttest -prepare \
2023-04-03 12:33:52 +03:00
rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
rusttestlib-uapi FORCE
2021-07-03 17:42:57 +03:00
$( call if_changed,rustc_test)
$( call if_changed,rustc_test_library)
i f d e f C O N F I G _ C C _ I S _ C L A N G
bindgen_c_flags = $( c_flags)
e l s e
# bindgen relies on libclang to parse C. Ideally, bindgen would support a GCC
# plugin backend and/or the Clang driver would be perfectly compatible with GCC.
#
# For the moment, here we are tweaking the flags on the fly. This is a hack,
# and some kernel configurations may not work (e.g. `GCC_PLUGIN_RANDSTRUCT`
# if we end up using one of those structs).
bindgen_skip_c_flags := -mno-fp-ret-in-387 -mpreferred-stack-boundary= % \
-mskip-rax-setup -mgeneral-regs-only -msign-return-address= % \
-mindirect-branch= thunk-extern -mindirect-branch-register \
-mfunction-return= thunk-extern -mrecord-mcount -mabi= lp64 \
-mindirect-branch-cs-prefix -mstack-protector-guard% -mtraceback= no \
-mno-pointers-to-nested-functions -mno-string \
-mno-strict-align -mstrict-align \
-fconserve-stack -falign-jumps= % -falign-loops= % \
-femit-struct-debug-baseonly -fno-ipa-cp-clone -fno-ipa-sra \
-fno-partial-inlining -fplugin-arg-arm_ssp_per_task_plugin-% \
-fno-reorder-blocks -fno-allow-store-data-races -fasan-shadow-offset= % \
-fzero-call-used-regs= % -fno-stack-clash-protection \
2023-07-11 10:19:14 +03:00
-fno-inline-functions-called-once -fsanitize= bounds-strict \
2023-08-15 09:53:46 +03:00
-fstrict-flex-arrays= % \
2021-07-03 17:42:57 +03:00
--param= % --param asan-%
# Derived from `scripts/Makefile.clang`.
BINDGEN_TARGET_x86 := x86_64-linux-gnu
BINDGEN_TARGET := $( BINDGEN_TARGET_$( SRCARCH) )
# All warnings are inhibited since GCC builds are very experimental,
# many GCC warnings are not supported by Clang, they may only appear in
# some configurations, with new GCC versions, etc.
bindgen_extra_c_flags = -w --target= $( BINDGEN_TARGET)
2023-02-11 00:51:41 +03:00
# Auto variable zero-initialization requires an additional special option with
# clang that is going to be removed sometime in the future (likely in
# clang-18), so make sure to pass this option only if clang supports it
# (libclang major version < 16).
#
# https://github.com/llvm/llvm-project/issues/44842
# https://github.com/llvm/llvm-project/blob/llvmorg-16.0.0-rc2/clang/docs/ReleaseNotes.rst#deprecated-compiler-flags
i f d e f C O N F I G _ I N I T _ S T A C K _ A L L _ Z E R O
libclang_maj_ver = $( shell $( BINDGEN) $( srctree) /scripts/rust_is_available_bindgen_libclang.h 2>& 1 | sed -ne 's/.*clang version \([0-9]*\).*/\1/p' )
i f e q ( $( shell expr $ ( libclang_maj_ver ) \< 16) , 1 )
bindgen_extra_c_flags += -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
e n d i f
e n d i f
2021-07-03 17:42:57 +03:00
bindgen_c_flags = $( filter-out $( bindgen_skip_c_flags) , $( c_flags) ) \
$( bindgen_extra_c_flags)
e n d i f
i f d e f C O N F I G _ L T O
bindgen_c_flags_lto = $( filter-out $( CC_FLAGS_LTO) , $( bindgen_c_flags) )
e l s e
bindgen_c_flags_lto = $( bindgen_c_flags)
e n d i f
bindgen_c_flags_final = $( bindgen_c_flags_lto) -D__BINDGEN__
quiet_cmd_bindgen = BINDGEN $@
cmd_bindgen = \
$( BINDGEN) $< $( bindgen_target_flags) \
--use-core --with-derive-default --ctypes-prefix core::ffi --no-layout-tests \
--no-debug '.*' \
rust: bindgen: upgrade to 0.65.1
In LLVM 16, anonymous items may return names like `(unnamed union at ..)`
rather than empty names [1], which breaks Rust-enabled builds because
bindgen assumed an empty name instead of detecting them via
`clang_Cursor_isAnonymous` [2]:
$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
RUSTC L rust/core.o
BINDGEN rust/bindings/bindings_generated.rs
BINDGEN rust/bindings/bindings_helpers_generated.rs
BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to
a more recent version, 0.65.1, to support LLVM 16.
Since bindgen 0.58.0 changed the `--{white,black}list-*` flags to
`--{allow,block}list-*` [3], update them on our side too.
In addition, bindgen 0.61.0 moved its CLI utility into a binary crate
called `bindgen-cli` [4]. Thus update the installation command in the
Quick Start guide.
Moreover, bindgen 0.61.0 changed the default functionality to bind
`size_t` to `usize` [5] and added the `--no-size_t-is-usize` flag
to not bind `size_t` as `usize`. Then bindgen 0.65.0 removed
the `--size_t-is-usize` flag [6]. Thus stop passing the flag to bindgen.
Finally, bindgen 0.61.0 added support for the `noreturn` attribute (in
its different forms) [7]. Thus remove the infinite loop in our Rust
panic handler after calling `BUG()`, since bindgen now correctly
generates a `BUG()` binding that returns `!` instead of `()`.
Link: https://github.com/llvm/llvm-project/commit/19e984ef8f49bc3ccced15621989fa9703b2cd5b [1]
Link: https://github.com/rust-lang/rust-bindgen/pull/2319 [2]
Link: https://github.com/rust-lang/rust-bindgen/pull/1990 [3]
Link: https://github.com/rust-lang/rust-bindgen/pull/2284 [4]
Link: https://github.com/rust-lang/rust-bindgen/commit/cc78b6fdb6e829e5fb8fa1639f2182cb49333569 [5]
Link: https://github.com/rust-lang/rust-bindgen/pull/2408 [6]
Link: https://github.com/rust-lang/rust-bindgen/issues/2094 [7]
Signed-off-by: Aakash Sen Sharma <aakashsensharma@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/issues/1013
Tested-by: Ariel Miculas <amiculas@cisco.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com
[ Reworded commit message. Mentioned the `bindgen-cli` binary crate
change, linked to it and updated the Quick Start guide. Re-added a
deleted "as" word in a code comment and reflowed comment to respect
the maximum length. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
2023-06-12 22:43:11 +03:00
-o $@ -- $( bindgen_c_flags_final) -DMODULE \
2021-07-03 17:42:57 +03:00
$( bindgen_target_cflags) $( bindgen_target_extra)
$(obj)/bindings/bindings_generated.rs : private bindgen_target_flags = \
2023-09-28 23:14:20 +03:00
$( shell grep -Ev '^#|^$$' $( srctree) /$( src) /bindgen_parameters)
2023-11-04 17:56:56 +03:00
$(obj)/bindings/bindings_generated.rs : private bindgen_target_extra = ; \
sed -Ei 's/pub const RUST_CONST_HELPER_([a-zA-Z0-9_]*)/pub const \1/g' $@
2021-07-03 17:42:57 +03:00
$(obj)/bindings/bindings_generated.rs : $( src ) /bindings /bindings_helper .h \
$( src) /bindgen_parameters FORCE
$( call if_changed_dep,bindgen)
2023-04-03 12:33:52 +03:00
$(obj)/uapi/uapi_generated.rs : private bindgen_target_flags = \
2023-09-28 23:14:20 +03:00
$( shell grep -Ev '^#|^$$' $( srctree) /$( src) /bindgen_parameters)
2023-04-03 12:33:52 +03:00
$(obj)/uapi/uapi_generated.rs : $( src ) /uapi /uapi_helper .h \
$( src) /bindgen_parameters FORCE
$( call if_changed_dep,bindgen)
2021-07-03 17:42:57 +03:00
# See `CFLAGS_REMOVE_helpers.o` above. In addition, Clang on C does not warn
# with `-Wmissing-declarations` (unlike GCC), so it is not strictly needed here
# given it is `libclang`; but for consistency, future Clang changes and/or
# a potential future GCC backend for `bindgen`, we disable it too.
$(obj)/bindings/bindings_helpers_generated.rs : private bindgen_target_flags = \
rust: bindgen: upgrade to 0.65.1
In LLVM 16, anonymous items may return names like `(unnamed union at ..)`
rather than empty names [1], which breaks Rust-enabled builds because
bindgen assumed an empty name instead of detecting them via
`clang_Cursor_isAnonymous` [2]:
$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
RUSTC L rust/core.o
BINDGEN rust/bindings/bindings_generated.rs
BINDGEN rust/bindings/bindings_helpers_generated.rs
BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to
a more recent version, 0.65.1, to support LLVM 16.
Since bindgen 0.58.0 changed the `--{white,black}list-*` flags to
`--{allow,block}list-*` [3], update them on our side too.
In addition, bindgen 0.61.0 moved its CLI utility into a binary crate
called `bindgen-cli` [4]. Thus update the installation command in the
Quick Start guide.
Moreover, bindgen 0.61.0 changed the default functionality to bind
`size_t` to `usize` [5] and added the `--no-size_t-is-usize` flag
to not bind `size_t` as `usize`. Then bindgen 0.65.0 removed
the `--size_t-is-usize` flag [6]. Thus stop passing the flag to bindgen.
Finally, bindgen 0.61.0 added support for the `noreturn` attribute (in
its different forms) [7]. Thus remove the infinite loop in our Rust
panic handler after calling `BUG()`, since bindgen now correctly
generates a `BUG()` binding that returns `!` instead of `()`.
Link: https://github.com/llvm/llvm-project/commit/19e984ef8f49bc3ccced15621989fa9703b2cd5b [1]
Link: https://github.com/rust-lang/rust-bindgen/pull/2319 [2]
Link: https://github.com/rust-lang/rust-bindgen/pull/1990 [3]
Link: https://github.com/rust-lang/rust-bindgen/pull/2284 [4]
Link: https://github.com/rust-lang/rust-bindgen/commit/cc78b6fdb6e829e5fb8fa1639f2182cb49333569 [5]
Link: https://github.com/rust-lang/rust-bindgen/pull/2408 [6]
Link: https://github.com/rust-lang/rust-bindgen/issues/2094 [7]
Signed-off-by: Aakash Sen Sharma <aakashsensharma@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/issues/1013
Tested-by: Ariel Miculas <amiculas@cisco.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com
[ Reworded commit message. Mentioned the `bindgen-cli` binary crate
change, linked to it and updated the Quick Start guide. Re-added a
deleted "as" word in a code comment and reflowed comment to respect
the maximum length. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
2023-06-12 22:43:11 +03:00
--blocklist-type '.*' --allowlist-var '' \
--allowlist-function 'rust_helper_.*'
2021-07-03 17:42:57 +03:00
$(obj)/bindings/bindings_helpers_generated.rs : private bindgen_target_cflags = \
-I$( objtree) /$( obj) -Wno-missing-prototypes -Wno-missing-declarations
$(obj)/bindings/bindings_helpers_generated.rs : private bindgen_target_extra = ; \
sed -Ei 's/pub fn rust_helper_([a-zA-Z0-9_]*)/#[link_name="rust_helper_\1"]\n pub fn \1/g' $@
$(obj)/bindings/bindings_helpers_generated.rs : $( src ) /helpers .c FORCE
$( call if_changed_dep,bindgen)
quiet_cmd_exports = EXPORTS $@
cmd_exports = \
$( NM) -p --defined-only $< \
2023-09-28 23:49:25 +03:00
| awk '/ (T|R|D) / {printf "EXPORT_SYMBOL_RUST_GPL(%s);\n",$$3}' > $@
2021-07-03 17:42:57 +03:00
$(obj)/exports_core_generated.h : $( obj ) /core .o FORCE
$( call if_changed,exports)
$(obj)/exports_alloc_generated.h : $( obj ) /alloc .o FORCE
$( call if_changed,exports)
$(obj)/exports_bindings_generated.h : $( obj ) /bindings .o FORCE
$( call if_changed,exports)
$(obj)/exports_kernel_generated.h : $( obj ) /kernel .o FORCE
$( call if_changed,exports)
quiet_cmd_rustc_procmacro = $( RUSTC_OR_CLIPPY_QUIET) P $@
cmd_rustc_procmacro = \
$( RUSTC_OR_CLIPPY) $( rust_common_flags) \
2023-10-06 00:39:58 +03:00
-Clinker-flavor= gcc -Clinker= $( HOSTCC) \
-Clink-args= '$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
kbuild: specify output names separately for each emission type from rustc
In Kbuild, two different rules must not write to the same file, but
it happens when compiling rust source files.
For example, set CONFIG_SAMPLE_RUST_MINIMAL=m and run the following:
$ make -j$(nproc) samples/rust/rust_minimal.o samples/rust/rust_minimal.rsi \
samples/rust/rust_minimal.s samples/rust/rust_minimal.ll
[snip]
RUSTC [M] samples/rust/rust_minimal.o
RUSTC [M] samples/rust/rust_minimal.rsi
RUSTC [M] samples/rust/rust_minimal.s
RUSTC [M] samples/rust/rust_minimal.ll
mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
make[3]: *** [scripts/Makefile.build:334: samples/rust/rust_minimal.ll] Error 1
make[3]: *** Waiting for unfinished jobs....
mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
make[3]: *** [scripts/Makefile.build:309: samples/rust/rust_minimal.o] Error 1
mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
make[3]: *** [scripts/Makefile.build:326: samples/rust/rust_minimal.s] Error 1
make[2]: *** [scripts/Makefile.build:504: samples/rust] Error 2
make[1]: *** [scripts/Makefile.build:504: samples] Error 2
make: *** [Makefile:2008: .] Error 2
The reason for the error is that 4 threads running in parallel renames
the same file, samples/rust/rust_minimal.d.
This does not happen when compiling C or assembly files because
-Wp,-MMD,$(depfile) explicitly specifies the dependency filepath.
$(depfile) is a unique path for each target.
Currently, rustc is only given --out-dir and --emit=<list-of-types>
So, all the rust build rules output the dep-info into the default
<CRATE_NAME>.d, which causes the path conflict.
Fortunately, the --emit option is able to specify the output path
individually, with the form --emit=<type>=<path>.
Add --emit=dep-info=$(depfile) to the common part. Also, remove the
redundant --out-dir because the output path is specified for each type.
The code gets much cleaner because we do not need to rename *.d files.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-07 12:18:15 +03:00
--emit= dep-info= $( depfile) --emit= link = $@ --extern proc_macro \
--crate-type proc-macro \
2023-01-07 12:18:17 +03:00
--crate-name $( patsubst lib%.so,%,$( notdir $@ ) ) $<
2021-07-03 17:42:57 +03:00
# Procedural macros can only be used with the `rustc` that compiled it.
# Therefore, to get `libmacros.so` automatically recompiled when the compiler
# version changes, we add `core.o` as a dependency (even if it is not needed).
$(obj)/libmacros.so : $( src ) /macros /lib .rs $( obj ) /core .o FORCE
$( call if_changed_dep,rustc_procmacro)
quiet_cmd_rustc_library = $( if $( skip_clippy) ,RUSTC,$( RUSTC_OR_CLIPPY_QUIET) ) L $@
cmd_rustc_library = \
OBJTREE = $( abspath $( objtree) ) \
$( if $( skip_clippy) ,$( RUSTC) ,$( RUSTC_OR_CLIPPY) ) \
$( filter-out $( skip_flags) ,$( rust_flags) $( rustc_target_flags) ) \
kbuild: specify output names separately for each emission type from rustc
In Kbuild, two different rules must not write to the same file, but
it happens when compiling rust source files.
For example, set CONFIG_SAMPLE_RUST_MINIMAL=m and run the following:
$ make -j$(nproc) samples/rust/rust_minimal.o samples/rust/rust_minimal.rsi \
samples/rust/rust_minimal.s samples/rust/rust_minimal.ll
[snip]
RUSTC [M] samples/rust/rust_minimal.o
RUSTC [M] samples/rust/rust_minimal.rsi
RUSTC [M] samples/rust/rust_minimal.s
RUSTC [M] samples/rust/rust_minimal.ll
mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
make[3]: *** [scripts/Makefile.build:334: samples/rust/rust_minimal.ll] Error 1
make[3]: *** Waiting for unfinished jobs....
mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
make[3]: *** [scripts/Makefile.build:309: samples/rust/rust_minimal.o] Error 1
mv: cannot stat 'samples/rust/rust_minimal.d': No such file or directory
make[3]: *** [scripts/Makefile.build:326: samples/rust/rust_minimal.s] Error 1
make[2]: *** [scripts/Makefile.build:504: samples/rust] Error 2
make[1]: *** [scripts/Makefile.build:504: samples] Error 2
make: *** [Makefile:2008: .] Error 2
The reason for the error is that 4 threads running in parallel renames
the same file, samples/rust/rust_minimal.d.
This does not happen when compiling C or assembly files because
-Wp,-MMD,$(depfile) explicitly specifies the dependency filepath.
$(depfile) is a unique path for each target.
Currently, rustc is only given --out-dir and --emit=<list-of-types>
So, all the rust build rules output the dep-info into the default
<CRATE_NAME>.d, which causes the path conflict.
Fortunately, the --emit option is able to specify the output path
individually, with the form --emit=<type>=<path>.
Add --emit=dep-info=$(depfile) to the common part. Also, remove the
redundant --out-dir because the output path is specified for each type.
The code gets much cleaner because we do not need to rename *.d files.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-07 12:18:15 +03:00
--emit= dep-info= $( depfile) --emit= obj = $@ \
--emit= metadata = $( dir $@ ) $( patsubst %.o,lib%.rmeta,$( notdir $@ ) ) \
--crate-type rlib -L$( objtree) /$( obj) \
2023-01-07 12:18:17 +03:00
--crate-name $( patsubst %.o,%,$( notdir $@ ) ) $< \
2023-10-31 23:10:14 +03:00
--sysroot= /dev/null \
2021-07-03 17:42:57 +03:00
$( if $( rustc_objcopy) ,; $( OBJCOPY) $( rustc_objcopy) $@ )
rust-analyzer :
2023-04-11 12:17:15 +03:00
$( Q) $( srctree) /scripts/generate_rust_analyzer.py \
2023-08-04 20:14:39 +03:00
--cfgs= 'core=$(core-cfgs)' --cfgs= 'alloc=$(alloc-cfgs)' \
2023-04-11 12:17:15 +03:00
$( abs_srctree) $( abs_objtree) \
$( RUST_LIB_SRC) $( KBUILD_EXTMOD) > \
$( if $( KBUILD_EXTMOD) ,$( extmod_prefix) ,$( objtree) ) /rust-project.json
2021-07-03 17:42:57 +03:00
2022-12-06 00:50:00 +03:00
redirect-intrinsics = \
rust: upgrade to Rust 1.71.1
This is the second upgrade to the Rust toolchain, from 1.68.2 to 1.71.1
(i.e. the latest).
See the upgrade policy [1] and the comments on the first upgrade in
commit 3ed03f4da06e ("rust: upgrade to Rust 1.68.2").
# Unstable features
No unstable features (that we use) were stabilized.
Therefore, the only unstable feature allowed to be used outside
the `kernel` crate is still `new_uninit`, though other code to be
upstreamed may increase the list.
Please see [2] for details.
# Required changes
For the upgrade, this patch requires the following changes:
- Removal of the `__rust_*` allocator functions, together with
the addition of the `__rust_no_alloc_shim_is_unstable` static.
See [3] for details.
- Some more compiler builtins added due to `<f{32,64}>::midpoint()`
that got added in Rust 1.71 [4].
# `alloc` upgrade and reviewing
The vast majority of changes are due to our `alloc` fork being upgraded
at once.
There are two kinds of changes to be aware of: the ones coming from
upstream, which we should follow as closely as possible, and the updates
needed in our added fallible APIs to keep them matching the newer
infallible APIs coming from upstream.
Instead of taking a look at the diff of this patch, an alternative
approach is reviewing a diff of the changes between upstream `alloc` and
the kernel's. This allows to easily inspect the kernel additions only,
especially to check if the fallible methods we already have still match
the infallible ones in the new version coming from upstream.
Another approach is reviewing the changes introduced in the additions in
the kernel fork between the two versions. This is useful to spot
potentially unintended changes to our additions.
To apply these approaches, one may follow steps similar to the following
to generate a pair of patches that show the differences between upstream
Rust and the kernel (for the subset of `alloc` we use) before and after
applying this patch:
# Get the difference with respect to the old version.
git -C rust checkout $(linux/scripts/min-tool-version.sh rustc)
git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
cut -d/ -f3- |
grep -Fv README.md |
xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
git -C linux diff --patch-with-stat --summary -R > old.patch
git -C linux restore rust/alloc
# Apply this patch.
git -C linux am rust-upgrade.patch
# Get the difference with respect to the new version.
git -C rust checkout $(linux/scripts/min-tool-version.sh rustc)
git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
cut -d/ -f3- |
grep -Fv README.md |
xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
git -C linux diff --patch-with-stat --summary -R > new.patch
git -C linux restore rust/alloc
Now one may check the `new.patch` to take a look at the additions (first
approach) or at the difference between those two patches (second
approach). For the latter, a side-by-side tool is recommended.
Link: https://rust-for-linux.com/rust-version-policy [1]
Link: https://github.com/Rust-for-Linux/linux/issues/2 [2]
Link: https://github.com/rust-lang/rust/pull/86844 [3]
Link: https://github.com/rust-lang/rust/pull/92048 [4]
Closes: https://github.com/Rust-for-Linux/linux/issues/68
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Reviewed-by: Trevor Gross <tmgross@umich.edu>
Link: https://lore.kernel.org/r/20230729220317.416771-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
2023-07-30 01:03:16 +03:00
__addsf3 __eqsf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __unordsf2 \
__adddf3 __ledf2 __ltdf2 __muldf3 __unorddf2 \
2022-12-06 00:50:00 +03:00
__muloti4 __multi3 \
__udivmodti4 __udivti3 __umodti3
i f n e q ( $( or $ ( CONFIG_ARM 64) ,$ ( and $ ( CONFIG_RISCV ) ,$ ( CONFIG_ 64BIT ) ) ) , )
# These intrinsics are defined for ARM64 and RISCV64
redirect-intrinsics += \
__ashrti3 \
__ashlti3 __lshrti3
e n d i f
2021-07-03 17:42:57 +03:00
$(obj)/core.o : private skip_clippy = 1
$(obj)/core.o : private skip_flags = -Dunreachable_pub
2022-12-06 00:50:00 +03:00
$(obj)/core.o : private rustc_objcopy = $( foreach sym ,$ ( redirect -intrinsics ) ,--redefine -sym $ ( sym ) =__rust $ ( sym ) )
2021-07-03 17:42:57 +03:00
$(obj)/core.o : private rustc_target_flags = $( core -cfgs )
2023-01-07 12:45:45 +03:00
$(obj)/core.o : $( RUST_LIB_SRC ) /core /src /lib .rs scripts /target .json FORCE
2021-07-03 17:42:57 +03:00
$( call if_changed_dep,rustc_library)
$(obj)/compiler_builtins.o : private rustc_objcopy = -w -W '__ *'
$(obj)/compiler_builtins.o : $( src ) /compiler_builtins .rs $( obj ) /core .o FORCE
$( call if_changed_dep,rustc_library)
$(obj)/alloc.o : private skip_clippy = 1
$(obj)/alloc.o : private skip_flags = -Dunreachable_pub
$(obj)/alloc.o : private rustc_target_flags = $( alloc -cfgs )
$(obj)/alloc.o : $( src ) /alloc /lib .rs $( obj ) /compiler_builtins .o FORCE
$( call if_changed_dep,rustc_library)
2022-11-10 19:41:37 +03:00
$(obj)/build_error.o : $( src ) /build_error .rs $( obj ) /compiler_builtins .o FORCE
$( call if_changed_dep,rustc_library)
2021-07-03 17:42:57 +03:00
$(obj)/bindings.o : $( src ) /bindings /lib .rs \
$( obj) /compiler_builtins.o \
$( obj) /bindings/bindings_generated.rs \
$( obj) /bindings/bindings_helpers_generated.rs FORCE
$( call if_changed_dep,rustc_library)
2023-04-03 12:33:52 +03:00
$(obj)/uapi.o : $( src ) /uapi /lib .rs \
$( obj) /compiler_builtins.o \
$( obj) /uapi/uapi_generated.rs FORCE
$( call if_changed_dep,rustc_library)
2021-07-03 17:42:57 +03:00
$(obj)/kernel.o : private rustc_target_flags = --extern alloc \
2023-04-03 12:33:52 +03:00
--extern build_error --extern macros --extern bindings --extern uapi
2022-11-10 19:41:37 +03:00
$(obj)/kernel.o : $( src ) /kernel /lib .rs $( obj ) /alloc .o $( obj ) /build_error .o \
2023-04-03 12:33:52 +03:00
$( obj) /libmacros.so $( obj) /bindings.o $( obj) /uapi.o FORCE
2021-07-03 17:42:57 +03:00
$( call if_changed_dep,rustc_library)
e n d i f # CONFIG_RUST
