5600 Commits

Author SHA1 Message Date
Cole Robinson
4205272384 tests: Add explicit test_dist target
Things like po validation don't need to be done on every test run
2018-02-20 12:33:57 -05:00
Cole Robinson
98d880a179 tests: Differentiate conn cache based on actual open URI
Since what is actually passed to libvirt here is what actually matters,
not all our fakeuri trickery
2018-02-20 12:21:45 -05:00
Cole Robinson
059fb7d0ba xmlbuilder: Order child props before serializing
We were implicitly depending on dict hash ordering here, which
was causing some different XML output in centos CI
2018-02-20 11:41:04 -05:00
Cole Robinson
a8fb49dbe0 xmlapi: Fix some pylint 2018-02-20 11:27:55 -05:00
Cole Robinson
dfa0fa34b4 tests: xmlparse: Add a roundtrip metadata test
To ensure the xml engine doesn't mangle unknown XML
2018-02-20 11:23:37 -05:00
Cole Robinson
bc94666778 tests: xmlparse: Add qemu xmlns tests
Since reproducing via manual clitest is hard since it requires
XML on stdin
2018-02-20 11:23:37 -05:00
Cole Robinson
2bea4ecbf6 xmlapi: Some libxml2 simplifications 2018-02-20 11:23:37 -05:00
Cole Robinson
f04b284f21 xmlbuilder: Abstract libxml2 API and cleanup
Moves the libxml2 bits to a separate xmlapi file and class, a bunch
of cleanups to xmlbuilder internals dealing with XML stuff.

The main point is to experiment with different XML library impls,
since libxml2 is unfun to deal with and we are having python3
issues like

https://bugzilla.gnome.org/show_bug.cgi?id=776815
2018-02-20 11:23:37 -05:00
Cole Robinson
1302156265 setup: Add explicit error if running with python2 2018-02-16 12:37:54 -05:00
Cole Robinson
a20230091b xmlbuilder: Move xpath string ops to their own class
This streamlines the actual libxml2 interaction quite a bit
2018-02-14 14:49:35 -05:00
Cole Robinson
dce3484e95 xmlbuilder: Move all libxml2 interaction to a single class
Makes the boundaries clearer
2018-02-14 11:29:12 -05:00
Cole Robinson
a853387866 xmlbuilder: Remove more redundant xml state
We don't need to carry around a separate xml_node. Dropping it simplifies
a few things
2018-02-14 11:08:09 -05:00
Cole Robinson
d108bbe143 Convert docstrings to standard reStructuredText param format 2018-02-14 11:08:09 -05:00
Cole Robinson
7cf0b310cc capabilities: Move machine canonicalizing to capsinfo
Rather than overright XML content on every parse
2018-02-14 11:08:09 -05:00
Cole Robinson
c4fdd1d56a xmlbuilder: Remove unused code 2018-02-14 11:08:09 -05:00
Cole Robinson
8917305a6e Don't use count() for substring checking
Use the idiomatic 'X in Y'
2018-02-14 11:08:09 -05:00
Cole Robinson
ab884b620f virtconv: ovf: convert to ElementTree 2018-02-14 09:57:39 -05:00
Cole Robinson
fe728f899d tests: virtconvtest: Accumulate more errors before exiting 2018-02-14 09:57:39 -05:00
Chen Hanxiao
a9596c7492 pylint: fix logging-too-few-args issue
From: Chen Hanxiao <chenhanxiao@gmail.com>

pylint complain:
  Not enough arguments for logging format string (logging-too-few-args)

Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
2018-02-13 16:42:29 +08:00
Cole Robinson
4c7c45908f manager: Fix mem, disk, net stats graphs (bz 1543896)
python2/3 division compat messed this up

https://bugzilla.redhat.com/show_bug.cgi?id=1543896
2018-02-09 14:02:04 -05:00
Cole Robinson
1550e70a4e uitests: Silence noisy harmless warnings
Could mean a code error but more likely is just slightly racy
UI bits disappearing that we weren't looking for anyways, so hide
it behind a debug statement
2018-02-08 14:23:11 -05:00
Cole Robinson
8e7b5ed2d8 Drop __future__ imports, no longer needed 2018-02-08 14:08:23 -05:00
Cole Robinson
dae8642bb1 xmlbuilder: centralize adding child new child prop instances
Currently the domain CPU class has a child property like:

  siblings = XMLChildProperty(_CPUCellSibling)

If a user wants to add a new sibling, we add a convenience function:

    def add_sibling(self):
        obj = _CPUCellSibling(self.conn)
        self.add_child(obj)
        return obj

Rather than require every child property to define a similar matching
helper function, this adds infrastructure in xmlbuilder to do this
generically for every child property. Now callers can do

    obj = guest.cpu.siblings.add_new()
2018-02-08 14:03:47 -05:00
Cole Robinson
8819176ad4 xmlbuilder: Drop unused node_top_xpath 2018-02-07 15:45:38 -05:00
Cole Robinson
598be36862 xmlbuilder: drop is_build monkey patching
It's not needed
2018-02-07 15:38:30 -05:00
Cole Robinson
48037fcf6f spec: Remove obsolete icon/desktop cache commands
Handled in Fedora for a while now
2018-02-06 19:02:53 -05:00
Cole Robinson
3cb33d381c Update docs to use new setup.py invocation 2018-02-06 19:02:53 -05:00
Cole Robinson
b42e37be1f Fix last bits of python3 pylint 2018-02-06 19:02:53 -05:00
Cole Robinson
de791136c1 setup: Use pylint-3 2018-02-06 19:02:53 -05:00
Cole Robinson
03971cef11 spec: Update for python3
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2018-02-06 19:02:53 -05:00
Cole Robinson
ddba9c1702 Switch to python3 in script shebang
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2018-02-06 18:56:15 -05:00
Cole Robinson
e47b34c05a Fix initrdinject and urltests with py3
A few random issues scattered about
2018-02-06 18:56:15 -05:00
Cole Robinson
b8e2952a9c connect: Fix urllib usage on py3 2018-02-06 18:56:15 -05:00
Cole Robinson
3086c7fda9 Drop python3 compat imports
We are going completely python3
2018-02-06 18:56:15 -05:00
Cole Robinson
e90d728380 urlfetcher: Open local files as binary
Needed for proper functioning on python3
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
15bd408570 Replace raw_input() with input()
raw_input() was renamed to input() in Python 3

See https://www.python.org/dev/peps/pep-3111/
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
e3d35b2fb9 tests: cli: Handle UnicodeError when logging 2018-02-06 18:49:17 -05:00
Radostin Stoyanov
81d68e6100 Use StringIO instad of BytesIO in Py 3
The print function in Python will convert printed arguments to text
strings [1] and thus print() cannot be used with binary mode file
objects.

In Python 2 the write() method of BytesIO() takes as input *bytes* object
which refers to *str* in Python 3. [2] The write() method of StringIO()
takes *unicode* object. [3] Therefore, StringIO() object cannot be used with
the print() function.

In Python 3 the write() method of BytesIO() takes a *bytes* object (not
*str*). [4] Therefore, the BytesIO() obj cannot be used with print().
However, the write() method of StringIO() in Python 3 takes a *str* as
input.[5]

Example of the issue:

    from __future__ import print_function
    import io

    a = io.BytesIO()
    b = io.StringIO()

    print("test", file=a)  <- Fails for Python 3
    print("test", file=b)  <- Fails for Python 2

    a.write('%s\n' % "test")  <- Fails for Python 3
    b.write('%s\n' % "test")  <- Fails for Python 2

[1] https://docs.python.org/3.5/library/functions.html#print
[2] https://docs.python.org/2/library/io.html#io.BufferedWriter.write
[3] https://docs.python.org/2/library/io.html#io.TextIOBase.write
[4] https://docs.python.org/3/library/io.html#io.BufferedWriter.write
[5] https://docs.python.org/3/library/io.html#io.TextIOBase.write
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
5146d66126 python3 compat: python3 strings have no decode() 2018-02-06 18:49:17 -05:00
Radostin Stoyanov
4d9c6141dd Convert the output of range() to list
In Python 2 the range() function returns a list [1].
In Python 3 the range function returns a range type [2] which
represents an immutable sequence of numbers [3].

[1] https://docs.python.org/2.7/library/functions.html#range
[2] https://docs.python.org/3/library/functions.html#func-range
[3] https://docs.python.org/3/library/stdtypes.html#typesseq-range
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
5854b7bb17 Replace "Queue" with "queue"
The Queue module has been renamed to queue in Python 3. [1]

[1] https://docs.python.org/2/library/queue.html
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
08c23477fd Fix re-raising exception
In Python 2 the raise command can take 3 objects as arguments which
are used to specify the exception to be raise as well as detailed
information and traceback [1].

In Python 3 the sintax is changed and now the keyword "from" and
"with_traceback" keyworkds are used to specify this information. [2]

Example which has the same behaviour for both Python 2 and 3:

    try:
        a = 1/0
    except Exception:
        import sys
        err = sys.exc_info()

        try:
                raise Exception("Test")
        except:
                pass

        if sys.version_info[0] == 2:
            exec("raise err[0], err[1], err[2]")
        else:
            raise
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
95c695d774 Convert long to int
- Python 2 only
k = 9223372036854775808L

- Python 2 and 3:
k = 9223372036854775808

See http://python-future.org/compatible_idioms.html#long-integers
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
6de65ab328 urlfetcher: Python 2/3 compatability for 'urllib'
The `urllib2` module has been split across several modules in Python 3
named `urllib.request` and `urllib.error`. [1]

[1] https://docs.python.org/2.7/library/urllib2.html
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
5553cbeb38 Replace ipaddr module with ipaddress
The `ipaddress` is available in Python 3.3+ [1] and backport for
Python 2 is available on PyPI [2].

The main differences between ipaddr and ipaddress are:

- ipaddress *Network classes are equivalent to the ipaddr *Network
  class counterparts with the strict flag set to True.
- ipaddress *Interface classes are equivalent to the ipaddr *Network
  class counterparts with the strict flag set to False.
- The factory functions in ipaddress were renamed to disambiguate them
  from classes.
- A few attributes were renamed to disambiguate their purpose as well.
  (eg. network -> network_address, numhosts -> num_addresses)
- A number of methods and functions which returned containers in ipaddr
  now return iterators. This includes subnets, address_exclude,
  summarize_address_range and collapse_address_list.

Another major difference is that in Python 2 the `ipaddress` module
must use unicode. [3]

[1] https://www.python.org/dev/peps/pep-3144/
[2] https://pypi.python.org/pypi/ipaddress
[3] https://github.com/phihag/ipaddress
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
978fb25ac7 Wrap keys(), values() in a list
In Python 3 dict.values() [1] , dict.keys() [2] and dict.items() [3]
return a view [4] of the dictionary’s values, keys and items.

In Python 2 these functions return a list. [5] [6] [7]

To resolve this we can convert the result of these function to a list.

[1] https://docs.python.org/3/library/stdtypes.html#dict.values
[2] https://docs.python.org/3/library/stdtypes.html#dict.keys
[3] https://docs.python.org/3/library/stdtypes.html#dict.items
[4] https://docs.python.org/3/library/stdtypes.html#dict-views
[5] https://docs.python.org/2/library/stdtypes.html#dict.items
[6] https://docs.python.org/2/library/stdtypes.html#dict.keys
[7] https://docs.python.org/2/library/stdtypes.html#dict.values
2018-02-06 18:49:17 -05:00
Radostin Stoyanov
810ee09292 Replace ConfigParser with configparser
The ConfigParser module has been renamed to configparser in
Python 3. [1] Backport of this changes is also available for
Python 2. [2]

[1] https://docs.python.org/2/library/configparser.html
[2] https://pypi.python.org/pypi/configparser/3.2.0r3
2018-02-06 18:49:17 -05:00
Cole Robinson
40c678783e Prep for release 1.5.0 v1.5.0 2018-02-06 18:03:49 -05:00
Cole Robinson
5141e73416 man: Fix libvirt doc reference for startup_policy
Reported-by: Digimer <lists@alteeve.ca>
2018-02-06 10:24:32 -05:00
Wim ten Have
708516c39e virtinst: add vcpupin support
Add vcpupin support to virt-install so that it can create guest
domains with statically allocated vcpu pinning towards a given cpuset.

Syntax: to pin vcpu=0 to cpuset="1,3" and vcpu=1 to cpuset=2

  --cputune vcpupin0.vcpu=0,vcpupin0.cpuset=1,3,vcpupin1.vcpu=1,vcpupin1.cpuset=2

generates below XML description for the guest domain.

  <cputune>
    <vcpupin vcpu="0" cpuset="1,3"/>
    <vcpupin vcpu="1" cpuset="2"/>
  </cputune>

Signed-off-by: Wim ten Have <wim.ten.have@oracle.com>
Signed-off-by: Menno Lageman <menno.lageman@oracle.com>
2018-02-06 10:21:56 -05:00