5
0
mirror of git://git.proxmox.com/git/proxmox-backup-restore-image.git synced 2025-01-19 18:03:35 +03:00

43 Commits

Author SHA1 Message Date
Fabian Grünbichler
53dbb29d6b kernel: switch to ntfs3 driver
it's the more modern version, the legacy 'ntfs' one is already removed upstream
nowadays.. initial feedback on a test build was positive throughout - while it
still doesn't support all features of the Windows version, it does handle a lot
of backups better than the previous one..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-08 17:40:03 +02:00
Stoiko Ivanov
95e95f36a5 update kernel to 6.5.11 and ZFS to 2.2.1, refresh patches
* for the kernel-patch this includes a rename from 0003+0004 to
  0001+0002
* for ZFS there was a change in upstream's autotools-setup - I
  referenced the commit in the actual patch-file

minimally tested with a VM with a zfs-pool and an ext4 disk
* restore of a directory on ext4 containing 160MB of debian packages
  as tar.zstd
* restore of a small folder (/root in a debian container) on zfs
both worked

restoring files from a Windows guest - worked, however there is an
independent issue with tpmstate not being found:
`given image 'drive-tpmstate0-backup.img.fidx' not found (400)`

directories with 10 million files also still cause the restore-shim to
run into OOM (but this is independent of the restore-image)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2023-11-28 16:31:32 +01:00
Thomas Lamprecht
d9d80a1f50 copy over base package state for debug one
Fixes a small regression in 0cec05b ("build initramfs: download
packages into per-flavor dirs"), which while taking a better
approach in general, missed that the debug package list is selected
with the assumption that it builds on top of the base list, but with
the split into two separate directories (to fix pre-caching), this
was lost.

While the packages selected pulled in a surprising amount of reverse
dependencies, it was still missing some like lvm2 tools (dmeventd and
co) and most notably busybox - which is really nice to have in the
debug image.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Thomas Lamprecht
6dbf509bb3 fix clean build again, use cache if actually there
whit dropping this in 0cec05b ("build initramfs: download packages
into per-flavor dirs"), the pre-caching was made effectively useless,
and building without internet during the build-step failed.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Stoiko Ivanov
925fec6e32 build initramfs: strip (zfs) binaries and libs after installing
this reduces the needed memory quite significantly (most extreme
example is libzpool from 17M to 4.3M, and init itself 15M->475K).

With this change the regular image is able to boot with
proxmox-file-restore and 192M.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Stoiko Ivanov
40187818f4 build initramfs: add busybox to base image as well
commit 83c46462ac6a99ce386088587c57e8a5131c36ca requires busybox to be
installed, which was only present in the debug image.

switching to busybox from busybox-static reduces the memory-footprint
quite a bit (699k vs. 1.9M) - and busybox only depends on libc6)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Stoiko Ivanov
322f01f6cb buildsys: disable zfs features during configure
this should save a small bit of compiletime and should reduce the
space needed for the initramfs image

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Stoiko Ivanov
fed9401c33 build initramfs: add libtirpc3 to basic packages
since libc6 in bookworm does not provide use '--enable-obsolete-rpc'
anymore and zfs needs it (building '--without-libtirpc' fails) - we
add it to both images.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Stoiko Ivanov
fb04ffde9a build initramfs: download packages into per-flavor dirs
commit f773602bd35c84c292649fb1c6e215050f25c21e introduced pre-caching
packages in the source-package, but resulted in both initrds getting
all packages installed:
* DOWNLOAD_ONLY=1 .../build_intiramfs.sh puts all packages in to pkgs
* installation when running w/o DOWNLOAD_ONLY installs all files
  matching '*.deb' - which results in all packages being installed

Additionally the TO_DOWNLOAD variable contained all packages anyways
(apt-rdepends yields packagenames - the file-existance check matches
on deb-filenames (containing version+arch) - so this was stripped out

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-01 17:41:47 +02:00
Thomas Lamprecht
96aa35260a buildsys: clean-up cached packages in top-level clean target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 10:09:21 +02:00
Thomas Lamprecht
09e46f60f1 update kernel to 6.2.16 & ZFS to 2.1.11, drop outdated patches
switch to our new unified ubuntu-kernels git origin, allows for
simpler update (no buildsys change required anymore)

The two patches dropped are:

- removal of xr-usb-uart, which was patched in by ubuntu (thus no
  KConfig switch, and got now removed, see:
  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1945938
- a fix for a syntax error in Ubuntu patches, which is fixed in newer
  repo already

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 10:09:05 +02:00
Thomas Lamprecht
52e7aecda3 buildsys: add kernel-img convenience target and complete prerequisites
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 10:09:05 +02:00
Thomas Lamprecht
be0a14256e buildsys: split out applying ZFS patches, allows build continuation
as trying to applying them twice won't work, so rather move that out
to its own target, that won't be repeated if new enough.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 10:09:05 +02:00
Thomas Lamprecht
c0a631babd buildsys: output the patch being applied
as otherwise it's unnecessarily harder to map errors from `patch` to
the respective patch file.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 10:09:05 +02:00
Thomas Lamprecht
f773602bd3 build initramfs: add options for pre-caching dependencies and allow using them
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-27 17:57:48 +02:00
Thomas Lamprecht
2695c25369 makefile: convert to use simple parenthesis
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-27 13:16:01 +02:00
Max Carrara
83c46462ac fix #4477: init: symlink busybox binaries on init
This change makes Busybox symbolically link its inbuilt utils to their
respective locations, allowing them to be invoked through their
absolute path. This means that Busybox inbuilts like e.g. `mount`
and `env` become available at `/bin/mount` and `/usr/bin/env` etc.

In particular, when entering any of the snapshot directories in
`.zfs/snapshot` (e.g. via `cd` or `ls`), ZFS normally mounts the
respective snapshot(s) on demand, using `/usr/bin/env mount [...]`
internally. Before this change, this would fail, rendering ZFS
snapshots unavailable when restoring individual files.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-05-17 11:32:46 +02:00
Thomas Lamprecht
64f3a452f0 init shim: print booted Linux kernel version
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-12 12:30:49 +02:00
Thomas Lamprecht
175f8d975b init shim: print package version
use the one from the debian package, as that's what we actually track
but fall back to the cargo one from the init shim (which we normally
don't bump) for developer convenience (env! fails the build if the
variable isn't set).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-12 12:29:39 +02:00
Thomas Lamprecht
ea328f216c init shim: code style touch up
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-12 12:28:39 +02:00
Fabian Grünbichler
dd8070a7f8 build: use system toolchain and rust dependencies
instead of using crates.io - the build still requires network access and
depends on configured APT sources in the build environment, so there's room for
further improvement.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-30 11:13:02 +01:00
Fabian Grünbichler
0bc9e13908 init: update nix to 0.26
and stop tracking Cargo.lock file

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-30 11:13:02 +01:00
Thomas Lamprecht
addaaa2608 init shim: drop ttyS1 driver check
the path changed to not include serial8250 directly anymore in jammy
kernel and we just don't really care, not much winning by dying here
if the base ttyS1 dev exists..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 17:15:02 +01:00
Thomas Lamprecht
0187961a74 init shim: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 17:15:02 +01:00
Thomas Lamprecht
5e968dfd03 update ZFS to 2.1.6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 15:14:45 +01:00
Thomas Lamprecht
4269959011 update kernel to Ubuntu-5.15.0-54.60
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-15 15:14:45 +01:00
Thomas Lamprecht
00749b1253 update kernel to Ubuntu Jammy based on 5.15.12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-08 09:50:54 +01:00
Thomas Lamprecht
0d4dc051f2 update ZFS to 2.1.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-08 09:50:54 +01:00
Dominik Csapak
a47c5cede1 add kernel config options for memory hotplug
so that we can hotplug memory in the guest

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-07 13:36:13 +01:00
Stefan Reiter
0faebc35eb add LVM (thin) tooling
FILE_LOCKING is required for LVM, the 4 lib* dependencies are not
automatically resolved for some reason, but are required.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-03 01:55:21 +02:00
Stefan Reiter
03894a933a build custom ZFS tools without udev requirement
We already include the required sources with the zfsonlinux submodule,
so apply a patch to disable linking against libudev (as I couldn't find
a working configure flag for it?) and build the user space part as well.

Includes dependencies as well as 'strace' for the debug initramfs, which
proved quite useful for debugging.

The init-shim automatically creates the necessary /dev/zfs device node,
and additionally /dev/null to make rust's std::process::Command happy.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-28 12:04:00 +02:00
Dominik Csapak
ac66a42c1a kconfig: enable V4 xfs support for kernel in restore image
since kernel 5.10, xfs v4 format is deprecated, reenable it for our
restore image kernel

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-10 13:09:37 +02:00
Stefan Reiter
2a1ef81fca add debug initramfs as seperate package
"proxmox-backup-restore-image-debug", containing only the debug
initramfs, so depends on the base "proxmox-backup-restore-image" for the
kernel.

Adapt the init-shim to start an agetty on ttyS1, which the host
can use to connect to a root shell for debugging, and use
create_dir_all, since some debug packages seem to create /sys and /proc
as empty dirs already.

The build_initramfs.sh script is modified to include dependency
resolution via apt-rdepends, so debug packages like agetty (util-linux),
busybox and gdb can easily be added. This now builds both the regular
and the debug binary at once, to avoid downloading shared packages
twice.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 13:25:02 +02:00
Stefan Reiter
37456b06e8 kernel: power off on panic
Instead of just rebooting, which may lead to an infinite loop, try to
resolve the situation by just powering off the VM - it can be restarted
any time anyway.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 13:25:02 +02:00
Thomas Lamprecht
653dfd0f6e update kernel to Ubuntu-5.11.0-18.19 / v5.11.17
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-25 13:24:53 +02:00
Stefan Reiter
37ee813020 add workaround kernel patch for vsock panics
Allocation failures for vsock packet buffers occur routinely when
downloading more than one stream at the same time, with less then 512
MiB of RAM it sometimes even occurs for single downloads.

This appears to fix it in all of my reproducer scenarios, tested with up
to 6 downloads at once in a 128 MiB RAM machine.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-26 18:47:01 +02:00
Thomas Lamprecht
e6f991e994 build initramfs: do not hardcode package versions
While it can be good to have those tracked, handling that should be
less friction. (e.g., simply output them a file tracked in git)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 14:58:17 +02:00
Thomas Lamprecht
469393baea update ZFS to 2.0.4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 12:07:25 +02:00
Thomas Lamprecht
02a834beeb update submodule to Ubuntu-5.11.0-14.15
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 11:51:23 +02:00
Thomas Lamprecht
a93dbade55 update to 5.11 based kernel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 16:48:28 +01:00
Stefan Reiter
69c58cb92f fixup "test-run" target
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 16:47:54 +01:00
Thomas Lamprecht
03dbac97da actually commit the new split-out make file
... -.-

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 15:23:27 +01:00
Thomas Lamprecht
275f3083a8 update/rework packaging and buildsys
try to split packaging from build and lots of such stuff

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-15 19:24:27 +01:00