12125 Commits

Author SHA1 Message Date
Richard W.M. Jones
413f5ccabf Version 2.5.9. v2.5.9 2024-08-29 15:29:06 +01:00
Richard W.M. Jones
950e82dfb7 convert: windows: Ignore sriov drivers on virtio-win disk
Update common submodule to get:

  Richard W.M. Jones (1):
      mlcustomize: Ignore sriov drivers on virtio-win disk
2024-08-29 15:09:41 +01:00
Richard W.M. Jones
67dd3de654 test-data/phony-guests: "Update" phony Windows to 2022 Server
For the phony Windows guest we used the registry information from a
32 bit Windows 7 instance.  This version of Windows is EOL.  Newer
Windows changed the versioning scheme so we now use the build ID to
distinguish between versions, but we have not tested this in virt-v2v
upstream before.

I grabbed HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion
from a real 64 bit Windows 2022 Server instance, minus identifying strings.

Various tests had to be updated because they contained references to
Windows 7, or because virtio-win contains more drivers for Windows
2022.
2024-08-29 15:09:10 +01:00
Richard W.M. Jones
2e78fb4d10 output/create_libvirt_xml.ml: Hard-code libosinfo URL for W2K22
Also add a comment that we ought to get this from the libosinfo
information provided by libguestfs.  I'm not sure why we aren't doing
that already.
2024-08-29 15:09:10 +01:00
Richard W.M. Jones
9b69cde051 po-docs: Update podfiles
Adds the two v2v-customize-*.pod snippets.

Also update the common submodule to get:

  Richard W.M. Jones (1):
      mlcustomize: Include v2v-customize-*.pod in EXTRA_DIST

Updates: commit f720aff7d690ead787b2ec9d4dabdff2b2923efd
2024-08-29 15:08:42 +01:00
Richard W.M. Jones
cb56f6f94d convert: windows: Online all virtio disks at first boot
Windows 2022 (and possibly earlier versions back to around 2019) will
force offline any non-boot disks which change bus, apparently as a
security mitigation.  The effect of this is that although the system
drive (C:) is present after conversion, other drives may seem to
disappear.

Running a Powershell script to bring all disks online seems risky.
The compromise is to bring online only virtio disks at first boot.

To further reduce risk, we only do this if there are non-system disks
(ie. > 1 disks in total), and only if we installed virtio drivers.

Fixes: https://issues.redhat.com/browse/RHEL-55763
Fixes: https://issues.redhat.com/browse/RHEL-55837
Related: https://issues.redhat.com/browse/MTV-1299
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1662286
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Thanks: Martin Necas
Acked-by: Martin Necas
2024-08-27 13:19:36 +01:00
Richard W.M. Jones
aa91135688 tests: Add a test of customizing a VM during conversion
Updates: commit ade62ebdb50e32a05c336ffcf4c08c6996a3af36
2024-08-20 21:32:54 +01:00
Richard W.M. Jones
f720aff7d6 v2v: Don't include virt-customize --inject* options
Commit ade62ebdb5 ("v2v: Add support for virt-customize options")
included --inject* options that covered facilities for injecting QEMU
guest agent etc which are already part of virt-v2v.  We shouldn't add
the options for that.

The generator in libguestfs was changed so we can now exclude those
options from virt-v2v.

Update common submodule to include:

Richard W.M. Jones (1):
      mlcustomize: Update generated options for virt-v2v

Fixes: commit ade62ebdb50e32a05c336ffcf4c08c6996a3af36
2024-08-20 21:24:00 +01:00
Richard W.M. Jones
170d6ba42e Version 2.5.8. v2.5.8 2024-08-20 12:07:40 +01:00
Richard W.M. Jones
ade62ebdb5 v2v: Add support for virt-customize options
For higher level tools making further arbitrary changes to the guest
post-conversion, especially adding more firstboot scripts, it's useful
if we have various features from virt-customize.  Particularly the
ability to upload and run scripts.  Since we have the virt-customize
code in the common/ submodule, the easiest thing to do is to add the
virt-customize options, the same way it is done in virt-builder and
virt-sysprep.

Also updates the common submodule to get:

Richard W.M. Jones (1):
      mlcustomize: Move virt-customize modules to mlcustomize/
2024-08-20 11:46:31 +01:00
Richard W.M. Jones
373ee61ab5 convert: Move SELinux relabelling step to common conversion code
Simple code motion.  SELinux relabelling is a no-op for
non-Linux/non-SELinux guests.
2024-08-20 10:14:06 +01:00
Richard W.M. Jones
0e3fb1ba37 convert: windows: Allow increasing verboseness of driver installation
To try to chase an issue with installing drivers on 32 bit Windows 10,
let's increase the LogLevel to maximum to get maximum logging.  As I
understand it we shouldn't leave this on all the time, so it is
commented out by default.

This setting is explained further here:
https://learn.microsoft.com/en-us/windows-hardware/drivers/install/setting-setupapi-logging-levels

Thanks: Vadim Rozenfeld
Related: https://issues.redhat.com/browse/RHEL-53990
2024-08-14 16:30:25 +01:00
Richard W.M. Jones
2471e29626 Version 2.5.7. v2.5.7 2024-08-13 11:57:12 +01:00
Richard W.M. Jones
bc845c124c convert: windows: Don't wait indefinitely for netkvm.sys
In the network configuration script, don't wait indefinitely for the
netkvm.sys (virtio-net) adapter to become available.  It might not
have been installed correctly or there could be some other reason why
it doesn't work.  This of course indicates a bug in the guest, but
it's still better than looping forever.

Reported-by: Ming Xie
Related: https://stackoverflow.com/a/40841942
Related: https://issues.redhat.com/browse/RHEL-50731
2024-08-13 10:47:51 +01:00
Richard W.M. Jones
e9c0d63b22 docs: Restate position on removal of VMware Tools
With the "reboot after every step" change made in the previous commit,
it seems removal of VMware Tools is now more reliable.  Update the
notes about this.

Updates: commit c952f310c902e438a8b0b5240a4b486b698bede8
2024-08-08 10:56:16 +01:00
Richard W.M. Jones
50f005f8b7 Pull in a fix to make Windows firstboot more reliable.
Update the common submodule to get:

Richard W.M. Jones (2):
      mlcustomize: Add some comments to firstboot batch file
      mlcustomize: Reboot Windows between each firstboot script

Document that Windows may now reboot several times after conversion.
2024-08-08 10:49:36 +01:00
Richard W.M. Jones
7bc9b115ba Update common submodule
Richard W.M. Jones (1):
      mlcustomize: Write qemu-ga log file name to log.txt
2024-08-05 10:18:31 +01:00
Richard W.M. Jones
c952f310c9 docs: Add a note about removal of VMware Tools on Windows
We use VMware's recommended method for removal, but we know from
experience that this rarely works.  The alternate method being
proposed involves making very invasive changes to the Registry and
filesystem, which are inappropriate for virt-v2v to do and highly
risky.  Therefore simply document this, with notes for the virt-v2v
user if they want to try the risky method.

Fixes: https://issues.redhat.com/browse/RHEL-51169
Thanks: Yan Vugenfirer
Reported-by: Ming Xie
2024-08-05 10:08:34 +01:00
Richard W.M. Jones
168eacf977 common: mlcustomize: Inject qemu-ga & blnsvr into <firstboot_dir>/Temp
Update common submodule to pick up this further change which should
mean that for all Windows conversions, everything is confined to
C:\Program Files\Guestfs\Firstboot (except the virtio drivers
themselves).

Richard W.M. Jones (1):
      mlcustomize: Inject qemu-ga & blnsvr into <firstboot_dir>/Temp
2024-08-01 11:03:42 +01:00
Richard W.M. Jones
c57ec4fd5d convert: More robust qemu-ga installation, change paths
Add a commit from the common submodule to attempt to make qemu-ga
installation more robust on Windows.

Rename network configuration Powershell script from "v2vnetcf" to
"network-configuration".  I also dropped the ".ps1" extension as the
modified Firstboot.add_firstboot_powershell function now adds this.

Update the common submodule to get these changes:

Richard W.M. Jones (3):
      mlcustomize: Use Start-Process -Wait to run qemu-ga installer
      mlcustomize: Add Firstboot.firstboot_dir function
      mlcustomize: Place powershell scripts into <firstboot_dir>\Temp
2024-08-01 10:12:28 +01:00
Richard W.M. Jones
7800049c56 docs: Note that --mac len field is now optional
Reported-by: Ming Xie
Updates: commit 159fda411d2f75b087106e7293d273ae142c9fbe
2024-07-31 10:09:50 +01:00
Richard W.M. Jones
74dda022ee Version 2.5.6. v2.5.6 2024-07-30 17:00:18 +01:00
Richard W.M. Jones
d33fa8822b convert: Collect drive mappings in 'inspect' object
Currently we only print it in debugging output, but we might use it in
future.  It costs nothing to get it as it is already collected for
Windows guests (and is empty for other guests).
2024-07-30 16:33:20 +01:00
Richard W.M. Jones
652766b412 convert: Dump more information about the guest during conversion
As with the previous commit, this will match searches for ^info:
2024-07-30 16:07:34 +01:00
Richard W.M. Jones
0950ae3d75 convert: Tag some important information in debug output with ^info:
When we get large virt-v2v logs back from customers it is often
difficult to find relevant information about the guest.  Try to fix
this by tagging some important information so that it will match this
regular expression in searches:

  ^info:

Note I'm not including information about input and output drivers
here.  If we suspect a problem there, then more detailed review of the
log is always needed, plus that information is usually much easier to
find (eg. finding the kubevirt yaml or libvirt XML is obvious even in
large log files).

The main complexity here was to ensure that all of the nbdinfo
information appears together.  We need to run the command and collect
the output, otherwise it gets mixed in with debugging from nbdkit
which resulted from running nbdinfo.

Also update the common submodule to include:

  Richard W.M. Jones (1):
      mldrivers/linux_kernels.ml: Prefix general information with ^info:
2024-07-30 16:07:34 +01:00
Richard W.M. Jones
7a82e130c2 lib: Only get nbdkit config and version once
The configuration (--dump-config) and version of nbdkit won't change
while we are running, so we only need to get it once.  We also don't
need the config parameter to Nbdkit.version so drop that.
2024-07-30 15:57:57 +01:00
Richard W.M. Jones
159fda411d v2v: --mac: Allow gw and len fields to be empty
Even if these appear in the middle of the list, allow them to be
empty.  For example this is now permitted whereas previously it was an
error:

  virt-v2v --mac <MAC>:ip:<ADDR>,,,nameserver

Reported-by: Arik Hadas
2024-07-28 14:49:42 +01:00
Richard W.M. Jones
4ba18d2d6d Update common submodule
This pulls in the commits below which simplify the installation of
Qemu Guest Agent on Windows.

Richard W.M. Jones (4):
      mlcustomize: firstboot: Use Linux path for Powershell script path
      mlcustomize: firstboot: Use powershell.exe instead of path
      mlcustomize: firstboot: Use Powershell -NoProfile flag
      mlcustomize: Revert delay installation of qemu-ga MSI

Fixes: https://issues.redhat.com/browse/RHEL-49761
2024-07-25 09:08:41 +01:00
Richard W.M. Jones
3d26e48af3 -i ova: Ignore dot-underscore-files in OVA files
I received an OVA created by a mac which contained various files
prefixed by "._" that contain some sort of extra information.  Ignore
those files when decoding OVAs:

$ tar tvf win22ktest.tar 2>/dev/null
drwxr-xr-x markd/staff       0 2024-07-23 18:23 win22test/
-rw-r--r-- markd/staff     619 2024-07-23 15:16 win22test/._win22test.mf
-rw-r--r-- markd/staff     271 2024-07-23 15:16 win22test/win22test.mf
-rw-r--r-- markd/staff     623 2024-07-23 15:16 win22test/._win22test-1.vmdk
-rw-r--r-- markd/staff 8348649984 2024-07-23 15:16 win22test/win22test-1.vmdk
-rw-r--r-- markd/staff        624 2024-07-23 15:00 win22test/._win22test-2.nvram
-rw-r--r-- markd/staff     270840 2024-07-23 15:00 win22test/win22test-2.nvram
-rw-r--r-- markd/staff        620 2024-07-23 15:00 win22test/._win22test.ovf
-rw-r--r-- markd/staff      12052 2024-07-23 15:00 win22test/win22test.ovf
2024-07-25 08:55:59 +01:00
Richard W.M. Jones
0119315c8d Version 2.5.5. v2.5.5 2024-07-11 12:22:47 +01:00
Richard W.M. Jones
2b00025238 -o kubevirt: Create a VirtualMachine instead of a VirtualMachineInstance
VMs converted by virt-v2v are pets not cattle so we want to create
permanent non-running machines.  (They may later or even soon
afterwards be started up by some other tool, at which point they also
become Instances.)  Therefore use a different kind and slightly
restructure the yaml around this.

Thanks: Lee Yarwood
2024-07-11 12:14:21 +01:00
Richard W.M. Jones
e0d79e9a2e -o kubevirt: Add metadata labels
Add various information which we have but cannot add to the Kubevirt
yaml in any other place.

Example output:

...
metadata:
  name: fedora-39
  labels:
    libguestfs.org/virt-v2v-version: "2.5.4"
    libguestfs.org/osinfo: fedora39
...

Thanks: Bella Khizgiyaev, Liran Rotenberg
Fixes: https://issues.redhat.com/browse/RHEL-45992
2024-07-11 12:14:21 +01:00
Richard W.M. Jones
9cedbf00c8 docs: Update -oo "?" example to use libvirt instead of RHV mode 2024-07-08 10:04:46 +01:00
Richard W.M. Jones
2a6c242273 Revert "docs: Remove paragraph about -ip passwords and ssh/scp"
Previously we removed this paragraph, believing that the -ip option
now copes with all cases.  However this still isn't true because
libvirt runs this ssh command:

  ssh -l root -T -e none -- [XEN-HOST] sh -c 'which virt-ssh-helper 1>/dev/null 2>&1; if test $? = 0; then     virt-ssh-helper 'xen://'; else    if 'nc' -q 2>&1 | grep "requires an argument" >/dev/null 2>&1; then ARG=-q0;else ARG=;fi;'nc' $ARG -U /var/run/libvirt/libvirt-sock; fi'

I checked with Dan and there is no way to suppress this or pass in a
password, so we still need ssh-agent even with -ip.  Note this applies
to any libvirt ssh connection, thus to Xen or VMware over SSH.

Reported-by: Ming Xie
Fixes: https://issues.redhat.com/browse/RHEL-45527
Thanks: Daniel Berrange

This reverts commit 67fcf66904c7f1f6da858eba35e95dad670427c0.
2024-07-04 11:01:24 +01:00
Richard W.M. Jones
4df3dcf64d inspector: Add more fields to the output
Add more inspection fields in the <operatingsystem> element of
virt-v2v-inspector output.  I've tried to keep these field names
consistent with virt-inspector (if there is a difference then it's a
bug in this tool).

See: https://issues.redhat.com/browse/MTV-1079
2024-07-03 11:39:52 +01:00
Richard W.M. Jones
9f7bb5fc33 docs: Add VDDK prereq that server must not be in maintenance mode
Fixes: https://issues.redhat.com/browse/RHEL-33699
2024-05-16 13:43:29 +01:00
Richard W.M. Jones
70eec57765 convert: windows: Install blnsvr from virtio-win
Also update the common module to get these two commits for the
implementation:

Richard W.M. Jones (2):
      mlcustomize: Add virt-customize --inject-blnsvr generated files
      mlcustomize: Add Inject_virtio_win.inject_blnsvr implementation

Fixes: https://issues.redhat.com/browse/RHEL-36591
2024-05-16 13:29:40 +01:00
Richard W.M. Jones
6513fcbec9 Update common submodule
Pulls in these commits, but they are not thought to be relevant to
virt-v2v.

  Ben Brown (1):
      Initialise bar->fp as NULL

  Richard W.M. Jones (2):
      mlcustomize: Update virt-customize generated files
      options: Allow nbd+unix:// URIs
2024-05-16 12:32:00 +01:00
Richard W.M. Jones
eb3ace8ff7 Version 2.5.4. v2.5.4 2024-04-25 15:04:36 +01:00
Richard W.M. Jones
cc0e640fa0 test-data/binaries: Remove +x attribute
These binaries are not meant to be run, they are purely data files
used for testing.  Remove the +x attribute to prevent accidentally
running them.

However to avoid breaking the phony guests, we need to chmod +x the
files when we upload them into those guests.

(cherry picked from libguestfs commit 574a87f889c7b48b231fcabf0d45b88bd48b53f8)
2024-04-25 13:30:02 +01:00
Richard W.M. Jones
5eb5edf9b5 test-data/binaries: Remove unused binary blobs
These blobs were used by the libguestfs tests, but not used in
guestfs-tools or virt-v2v.  Only two of them are required in
guestfs-tools and virt-v2v, and those are used for building phony
guests.  Remove the other, unused ones.

(cherry picked from guestfs-tools commit de9ab80c1af9fc10dca8bad3dc6201c56545f232)
2024-04-25 11:03:31 +01:00
Richard W.M. Jones
1bffa15a8a test-data: Don't generate test.iso
It is only used by libguestfs tests.

(cherry picked from guestfs-tools commit 202a6ee8f3f95e7c12c799180231332fb7077c9b)
2024-04-25 10:26:20 +01:00
Richard W.M. Jones
bfa517cb6e test-data/files: Remove this directory
It was only used by libguestfs tests and is not relevant in
guestfs-tools or virt-v2v.

(cherry picked from guestfs-tools commit 51673f9d7ef53860397788a7530c00c1ea8979db)
2024-04-25 10:24:47 +01:00
Jerry James
5eeae696b2 Fix bytecode compilation to output whole exe instead of using -custom
These binaries are more resistant to being stripped.

Reported-by: Ilya Leoshkevich
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2274708
(cherry picked from guestfs-tools commit c249fd5caca8f1eed439f83e2bb1af46c8a28a27)
2024-04-12 20:14:34 +01:00
Richard W.M. Jones
58f1639d6c Version 2.5.3. v2.5.3 2024-04-11 17:48:04 +01:00
Bella Khizgiyaev
bf33133c05
-i ova: Use the detected firmware type when absent rather than default (#53)
Signed-off-by: Bella Khizgiyaev <bkhizgiy@redhat.com>
2024-04-11 14:25:35 +01:00
Thorsten Bolten
089d9a235e -o rhv-upload: Add O_TRUNC to truncate JSON file if it exists
Fixes: https://github.com/libguestfs/virt-v2v/issues/46
Thanks: Nir Soffer
2024-04-06 17:57:42 +01:00
Nir Soffer
5672d89531
-o rhv-upload: Don't share transfer.json (#49)
Use unique v2vtransferN.json for every transfer, like like
out.paramsN.json. If we keep the temporary files after the import, the
content of the file can help debugging.

Related to #46
2024-04-06 15:15:45 +01:00
Nir Soffer
ff341481eb -o rhv-upload: Improve host id logging
This log[1] is proved confusing for developers:

    cannot read /etc/vdsm/vdsm.id, using any host: [Errno 2] No such file or directory: '/etc/vdsm/vdsm.id'

This is actually a debug message, that can help RHV developers to
understand where virt-v2v was running. To make this more clear, in the
expected case when running on non-oVirt host, we log now:

    not an oVirt host, using non-oVirt host

For any other error, we can degrade to remote transfer, but we want to
warn the user about this, so we log:

    warning: cannot read host id, using non-oVirt host: [Errno 2] No such file or directory: '/etc/vdsm/vdsm.id'

[1] https://github.com/libguestfs/virt-v2v/issues/46#issuecomment-2039604719

RWMJ: Changed "any host" to "non-oVirt host" to make it clearer.
2024-04-06 07:38:25 +01:00
Nir Soffer
0ea92cc029 -o rhv-upload: Output outside of the with block
We write the json inside a with block, so what actually happen is:

1. Computing the values
2. Dumping the json to stdout
3. Closing the connection

If some code in sdk.Connection.close() write something to stdout, it
will corrupt our json output. Avoid this by moving the output out of the
with block.

This will not prevent bad code to write something before the json. We
need to log bad json in the Ocaml code to reveal such issue.
2024-04-06 07:34:31 +01:00