12113 Commits

Author SHA1 Message Date
Richard W.M. Jones
2471e29626 Version 2.5.7. v2.5.7 2024-08-13 11:57:12 +01:00
Richard W.M. Jones
bc845c124c convert: windows: Don't wait indefinitely for netkvm.sys
In the network configuration script, don't wait indefinitely for the
netkvm.sys (virtio-net) adapter to become available.  It might not
have been installed correctly or there could be some other reason why
it doesn't work.  This of course indicates a bug in the guest, but
it's still better than looping forever.

Reported-by: Ming Xie
Related: https://stackoverflow.com/a/40841942
Related: https://issues.redhat.com/browse/RHEL-50731
2024-08-13 10:47:51 +01:00
Richard W.M. Jones
e9c0d63b22 docs: Restate position on removal of VMware Tools
With the "reboot after every step" change made in the previous commit,
it seems removal of VMware Tools is now more reliable.  Update the
notes about this.

Updates: commit c952f310c902e438a8b0b5240a4b486b698bede8
2024-08-08 10:56:16 +01:00
Richard W.M. Jones
50f005f8b7 Pull in a fix to make Windows firstboot more reliable.
Update the common submodule to get:

Richard W.M. Jones (2):
      mlcustomize: Add some comments to firstboot batch file
      mlcustomize: Reboot Windows between each firstboot script

Document that Windows may now reboot several times after conversion.
2024-08-08 10:49:36 +01:00
Richard W.M. Jones
7bc9b115ba Update common submodule
Richard W.M. Jones (1):
      mlcustomize: Write qemu-ga log file name to log.txt
2024-08-05 10:18:31 +01:00
Richard W.M. Jones
c952f310c9 docs: Add a note about removal of VMware Tools on Windows
We use VMware's recommended method for removal, but we know from
experience that this rarely works.  The alternate method being
proposed involves making very invasive changes to the Registry and
filesystem, which are inappropriate for virt-v2v to do and highly
risky.  Therefore simply document this, with notes for the virt-v2v
user if they want to try the risky method.

Fixes: https://issues.redhat.com/browse/RHEL-51169
Thanks: Yan Vugenfirer
Reported-by: Ming Xie
2024-08-05 10:08:34 +01:00
Richard W.M. Jones
168eacf977 common: mlcustomize: Inject qemu-ga & blnsvr into <firstboot_dir>/Temp
Update common submodule to pick up this further change which should
mean that for all Windows conversions, everything is confined to
C:\Program Files\Guestfs\Firstboot (except the virtio drivers
themselves).

Richard W.M. Jones (1):
      mlcustomize: Inject qemu-ga & blnsvr into <firstboot_dir>/Temp
2024-08-01 11:03:42 +01:00
Richard W.M. Jones
c57ec4fd5d convert: More robust qemu-ga installation, change paths
Add a commit from the common submodule to attempt to make qemu-ga
installation more robust on Windows.

Rename network configuration Powershell script from "v2vnetcf" to
"network-configuration".  I also dropped the ".ps1" extension as the
modified Firstboot.add_firstboot_powershell function now adds this.

Update the common submodule to get these changes:

Richard W.M. Jones (3):
      mlcustomize: Use Start-Process -Wait to run qemu-ga installer
      mlcustomize: Add Firstboot.firstboot_dir function
      mlcustomize: Place powershell scripts into <firstboot_dir>\Temp
2024-08-01 10:12:28 +01:00
Richard W.M. Jones
7800049c56 docs: Note that --mac len field is now optional
Reported-by: Ming Xie
Updates: commit 159fda411d2f75b087106e7293d273ae142c9fbe
2024-07-31 10:09:50 +01:00
Richard W.M. Jones
74dda022ee Version 2.5.6. v2.5.6 2024-07-30 17:00:18 +01:00
Richard W.M. Jones
d33fa8822b convert: Collect drive mappings in 'inspect' object
Currently we only print it in debugging output, but we might use it in
future.  It costs nothing to get it as it is already collected for
Windows guests (and is empty for other guests).
2024-07-30 16:33:20 +01:00
Richard W.M. Jones
652766b412 convert: Dump more information about the guest during conversion
As with the previous commit, this will match searches for ^info:
2024-07-30 16:07:34 +01:00
Richard W.M. Jones
0950ae3d75 convert: Tag some important information in debug output with ^info:
When we get large virt-v2v logs back from customers it is often
difficult to find relevant information about the guest.  Try to fix
this by tagging some important information so that it will match this
regular expression in searches:

  ^info:

Note I'm not including information about input and output drivers
here.  If we suspect a problem there, then more detailed review of the
log is always needed, plus that information is usually much easier to
find (eg. finding the kubevirt yaml or libvirt XML is obvious even in
large log files).

The main complexity here was to ensure that all of the nbdinfo
information appears together.  We need to run the command and collect
the output, otherwise it gets mixed in with debugging from nbdkit
which resulted from running nbdinfo.

Also update the common submodule to include:

  Richard W.M. Jones (1):
      mldrivers/linux_kernels.ml: Prefix general information with ^info:
2024-07-30 16:07:34 +01:00
Richard W.M. Jones
7a82e130c2 lib: Only get nbdkit config and version once
The configuration (--dump-config) and version of nbdkit won't change
while we are running, so we only need to get it once.  We also don't
need the config parameter to Nbdkit.version so drop that.
2024-07-30 15:57:57 +01:00
Richard W.M. Jones
159fda411d v2v: --mac: Allow gw and len fields to be empty
Even if these appear in the middle of the list, allow them to be
empty.  For example this is now permitted whereas previously it was an
error:

  virt-v2v --mac <MAC>:ip:<ADDR>,,,nameserver

Reported-by: Arik Hadas
2024-07-28 14:49:42 +01:00
Richard W.M. Jones
4ba18d2d6d Update common submodule
This pulls in the commits below which simplify the installation of
Qemu Guest Agent on Windows.

Richard W.M. Jones (4):
      mlcustomize: firstboot: Use Linux path for Powershell script path
      mlcustomize: firstboot: Use powershell.exe instead of path
      mlcustomize: firstboot: Use Powershell -NoProfile flag
      mlcustomize: Revert delay installation of qemu-ga MSI

Fixes: https://issues.redhat.com/browse/RHEL-49761
2024-07-25 09:08:41 +01:00
Richard W.M. Jones
3d26e48af3 -i ova: Ignore dot-underscore-files in OVA files
I received an OVA created by a mac which contained various files
prefixed by "._" that contain some sort of extra information.  Ignore
those files when decoding OVAs:

$ tar tvf win22ktest.tar 2>/dev/null
drwxr-xr-x markd/staff       0 2024-07-23 18:23 win22test/
-rw-r--r-- markd/staff     619 2024-07-23 15:16 win22test/._win22test.mf
-rw-r--r-- markd/staff     271 2024-07-23 15:16 win22test/win22test.mf
-rw-r--r-- markd/staff     623 2024-07-23 15:16 win22test/._win22test-1.vmdk
-rw-r--r-- markd/staff 8348649984 2024-07-23 15:16 win22test/win22test-1.vmdk
-rw-r--r-- markd/staff        624 2024-07-23 15:00 win22test/._win22test-2.nvram
-rw-r--r-- markd/staff     270840 2024-07-23 15:00 win22test/win22test-2.nvram
-rw-r--r-- markd/staff        620 2024-07-23 15:00 win22test/._win22test.ovf
-rw-r--r-- markd/staff      12052 2024-07-23 15:00 win22test/win22test.ovf
2024-07-25 08:55:59 +01:00
Richard W.M. Jones
0119315c8d Version 2.5.5. v2.5.5 2024-07-11 12:22:47 +01:00
Richard W.M. Jones
2b00025238 -o kubevirt: Create a VirtualMachine instead of a VirtualMachineInstance
VMs converted by virt-v2v are pets not cattle so we want to create
permanent non-running machines.  (They may later or even soon
afterwards be started up by some other tool, at which point they also
become Instances.)  Therefore use a different kind and slightly
restructure the yaml around this.

Thanks: Lee Yarwood
2024-07-11 12:14:21 +01:00
Richard W.M. Jones
e0d79e9a2e -o kubevirt: Add metadata labels
Add various information which we have but cannot add to the Kubevirt
yaml in any other place.

Example output:

...
metadata:
  name: fedora-39
  labels:
    libguestfs.org/virt-v2v-version: "2.5.4"
    libguestfs.org/osinfo: fedora39
...

Thanks: Bella Khizgiyaev, Liran Rotenberg
Fixes: https://issues.redhat.com/browse/RHEL-45992
2024-07-11 12:14:21 +01:00
Richard W.M. Jones
9cedbf00c8 docs: Update -oo "?" example to use libvirt instead of RHV mode 2024-07-08 10:04:46 +01:00
Richard W.M. Jones
2a6c242273 Revert "docs: Remove paragraph about -ip passwords and ssh/scp"
Previously we removed this paragraph, believing that the -ip option
now copes with all cases.  However this still isn't true because
libvirt runs this ssh command:

  ssh -l root -T -e none -- [XEN-HOST] sh -c 'which virt-ssh-helper 1>/dev/null 2>&1; if test $? = 0; then     virt-ssh-helper 'xen://'; else    if 'nc' -q 2>&1 | grep "requires an argument" >/dev/null 2>&1; then ARG=-q0;else ARG=;fi;'nc' $ARG -U /var/run/libvirt/libvirt-sock; fi'

I checked with Dan and there is no way to suppress this or pass in a
password, so we still need ssh-agent even with -ip.  Note this applies
to any libvirt ssh connection, thus to Xen or VMware over SSH.

Reported-by: Ming Xie
Fixes: https://issues.redhat.com/browse/RHEL-45527
Thanks: Daniel Berrange

This reverts commit 67fcf66904c7f1f6da858eba35e95dad670427c0.
2024-07-04 11:01:24 +01:00
Richard W.M. Jones
4df3dcf64d inspector: Add more fields to the output
Add more inspection fields in the <operatingsystem> element of
virt-v2v-inspector output.  I've tried to keep these field names
consistent with virt-inspector (if there is a difference then it's a
bug in this tool).

See: https://issues.redhat.com/browse/MTV-1079
2024-07-03 11:39:52 +01:00
Richard W.M. Jones
9f7bb5fc33 docs: Add VDDK prereq that server must not be in maintenance mode
Fixes: https://issues.redhat.com/browse/RHEL-33699
2024-05-16 13:43:29 +01:00
Richard W.M. Jones
70eec57765 convert: windows: Install blnsvr from virtio-win
Also update the common module to get these two commits for the
implementation:

Richard W.M. Jones (2):
      mlcustomize: Add virt-customize --inject-blnsvr generated files
      mlcustomize: Add Inject_virtio_win.inject_blnsvr implementation

Fixes: https://issues.redhat.com/browse/RHEL-36591
2024-05-16 13:29:40 +01:00
Richard W.M. Jones
6513fcbec9 Update common submodule
Pulls in these commits, but they are not thought to be relevant to
virt-v2v.

  Ben Brown (1):
      Initialise bar->fp as NULL

  Richard W.M. Jones (2):
      mlcustomize: Update virt-customize generated files
      options: Allow nbd+unix:// URIs
2024-05-16 12:32:00 +01:00
Richard W.M. Jones
eb3ace8ff7 Version 2.5.4. v2.5.4 2024-04-25 15:04:36 +01:00
Richard W.M. Jones
cc0e640fa0 test-data/binaries: Remove +x attribute
These binaries are not meant to be run, they are purely data files
used for testing.  Remove the +x attribute to prevent accidentally
running them.

However to avoid breaking the phony guests, we need to chmod +x the
files when we upload them into those guests.

(cherry picked from libguestfs commit 574a87f889c7b48b231fcabf0d45b88bd48b53f8)
2024-04-25 13:30:02 +01:00
Richard W.M. Jones
5eb5edf9b5 test-data/binaries: Remove unused binary blobs
These blobs were used by the libguestfs tests, but not used in
guestfs-tools or virt-v2v.  Only two of them are required in
guestfs-tools and virt-v2v, and those are used for building phony
guests.  Remove the other, unused ones.

(cherry picked from guestfs-tools commit de9ab80c1af9fc10dca8bad3dc6201c56545f232)
2024-04-25 11:03:31 +01:00
Richard W.M. Jones
1bffa15a8a test-data: Don't generate test.iso
It is only used by libguestfs tests.

(cherry picked from guestfs-tools commit 202a6ee8f3f95e7c12c799180231332fb7077c9b)
2024-04-25 10:26:20 +01:00
Richard W.M. Jones
bfa517cb6e test-data/files: Remove this directory
It was only used by libguestfs tests and is not relevant in
guestfs-tools or virt-v2v.

(cherry picked from guestfs-tools commit 51673f9d7ef53860397788a7530c00c1ea8979db)
2024-04-25 10:24:47 +01:00
Jerry James
5eeae696b2 Fix bytecode compilation to output whole exe instead of using -custom
These binaries are more resistant to being stripped.

Reported-by: Ilya Leoshkevich
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2274708
(cherry picked from guestfs-tools commit c249fd5caca8f1eed439f83e2bb1af46c8a28a27)
2024-04-12 20:14:34 +01:00
Richard W.M. Jones
58f1639d6c Version 2.5.3. v2.5.3 2024-04-11 17:48:04 +01:00
Bella Khizgiyaev
bf33133c05
-i ova: Use the detected firmware type when absent rather than default (#53)
Signed-off-by: Bella Khizgiyaev <bkhizgiy@redhat.com>
2024-04-11 14:25:35 +01:00
Thorsten Bolten
089d9a235e -o rhv-upload: Add O_TRUNC to truncate JSON file if it exists
Fixes: https://github.com/libguestfs/virt-v2v/issues/46
Thanks: Nir Soffer
2024-04-06 17:57:42 +01:00
Nir Soffer
5672d89531
-o rhv-upload: Don't share transfer.json (#49)
Use unique v2vtransferN.json for every transfer, like like
out.paramsN.json. If we keep the temporary files after the import, the
content of the file can help debugging.

Related to #46
2024-04-06 15:15:45 +01:00
Nir Soffer
ff341481eb -o rhv-upload: Improve host id logging
This log[1] is proved confusing for developers:

    cannot read /etc/vdsm/vdsm.id, using any host: [Errno 2] No such file or directory: '/etc/vdsm/vdsm.id'

This is actually a debug message, that can help RHV developers to
understand where virt-v2v was running. To make this more clear, in the
expected case when running on non-oVirt host, we log now:

    not an oVirt host, using non-oVirt host

For any other error, we can degrade to remote transfer, but we want to
warn the user about this, so we log:

    warning: cannot read host id, using non-oVirt host: [Errno 2] No such file or directory: '/etc/vdsm/vdsm.id'

[1] https://github.com/libguestfs/virt-v2v/issues/46#issuecomment-2039604719

RWMJ: Changed "any host" to "non-oVirt host" to make it clearer.
2024-04-06 07:38:25 +01:00
Nir Soffer
0ea92cc029 -o rhv-upload: Output outside of the with block
We write the json inside a with block, so what actually happen is:

1. Computing the values
2. Dumping the json to stdout
3. Closing the connection

If some code in sdk.Connection.close() write something to stdout, it
will corrupt our json output. Avoid this by moving the output out of the
with block.

This will not prevent bad code to write something before the json. We
need to log bad json in the Ocaml code to reveal such issue.
2024-04-06 07:34:31 +01:00
Richard W.M. Jones
19ac262ceb -o rhv-upload: Dump the JSON before parsing
Related: https://github.com/libguestfs/virt-v2v/issues/46
2024-04-05 22:13:40 +01:00
Richard W.M. Jones
d8f09872a6 -o rhv-upload: clarify debug message
Thanks: Nir Soffer
Related: https://github.com/libguestfs/virt-v2v/issues/46
2024-04-05 21:53:24 +01:00
Nir Soffer
cfeebf0f12
-o rhv-upload: Add context if parsing params fails (#47)
If parsing params fail we get unhelpful error:

    virt-v2v: error: internal error: invalid argument:
    /tmp/v2v.CtVpBL/v2vtransfer.json: JSON parse error: end of file expected
    near 'e'

Change the parsing code to read the entire json text and include it in
the error message. This will make it clear why the json could not be
parsed, and help to find the root cause for having invalid json.

Example error:

    $ cat out.params0.json
    This is not a json document

    $ python output/rhv-upload-transfer.py out.params0.json
    Traceback (most recent call last):
      File "/home/nsoffer/src/virt-v2v/output/rhv-upload-transfer.py", line 261, in <module>
        raise RuntimeError(f"Cannot parse params {data!r}: {e}").with_traceback(
      File "/home/nsoffer/src/virt-v2v/output/rhv-upload-transfer.py", line 259, in <module>
        params = json.loads(data)
                 ^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.12/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.12/json/decoder.py", line 337, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.12/json/decoder.py", line 355, in raw_decode
        raise JSONDecodeError("Expecting value", s, err.value) from None
    RuntimeError: Cannot parse params 'This is not a json document\n': Expecting value: line 1 column 1 (char 0)

Related to #46
2024-04-05 20:56:40 +01:00
Richard W.M. Jones
a659b334f5 tests/test-v2v-o-kubevirt-windows.sh: Set fake virtio-win
When converting this phony Windows guest we didn't set the VIRTIO_WIN
environment variable.  As a result the test output could change (and
break) depending on if the virtio-win package was installed or not.

Point VIRTIO_WIN to the fake virtio-win directory (as we do with other
Windows tests).  We also need to modify the test output since the fake
virtio-win does contain enough for virt-v2v to think that virtio
driver installation was successful.

Fixes: commit af1f7b571fb21fdbfa2974ee54a22b4fdcbc886a
2024-03-15 11:38:39 +00:00
Richard W.M. Jones
ce67e6e66f tests: Don't rebuild real_guests_scripts after clean
Move this rule to 'make distclean' so we don't rebuild these scripts
all the time.  It's not necessary as they hardly ever change.
2024-03-12 09:56:55 +00:00
Richard W.M. Jones
a8ece4e72d Version 2.5.2. v2.5.2 2024-03-12 09:51:59 +00:00
Richard W.M. Jones
7845d21c65 tests: Fix EXTRA_DIST
Fixes: commit af1f7b571fb21fdbfa2974ee54a22b4fdcbc886a
2024-03-12 09:51:33 +00:00
Richard W.M. Jones
86c43f70ac -o kubevirt: Update comment about displays
I believe we don't need to explicitly enable a display, as Kubevirt
will enable one by default.  Add these findings as a comment.

Thanks: Arik Hadas
2024-03-11 16:40:57 +00:00
Richard W.M. Jones
54ca2da251 -o kubevirt: Add support for sound device
Thanks: Arik Hadas
2024-03-11 16:37:08 +00:00
Richard W.M. Jones
53c9177a78 -o kubevirt: Use virtio-transitional for ancient guests 2024-03-11 16:04:29 +00:00
Richard W.M. Jones
353a9348ea -o kubevirt: Add an RNG device if virtio-rng driver in the guest 2024-03-11 15:49:56 +00:00
Richard W.M. Jones
eee069f8da -o kubevirt: Add network interfaces to output 2024-03-11 15:49:12 +00:00