Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python patches

A few fixes to the Python CI tests, a few fixes to the (async) QMP
library, and a set of pat

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python patches

A few fixes to the Python CI tests, a few fixes to the (async) QMP
library, and a set of patches that begin to shift us towards using the
new qmp lib.

# gpg: Signature made Sat 22 Jan 2022 00:07:58 GMT
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request:
scripts/render-block-graph: switch to AQMP
scripts/cpu-x86-uarch-abi: switch to AQMP
scripts/cpu-x86-uarch-abi: fix CLI parsing
python: move qmp-shell under the AQMP package
python: move qmp utilities to python/qemu/utils
python/qmp: switch qmp-shell to AQMP
python/qmp: switch qom tools to AQMP
python/qmp: switch qemu-ga-client to AQMP
python/qemu-ga-client: don't use deprecated CLI syntax in usage comment
python/aqmp: rename AQMPError to QMPError
python/aqmp: add SocketAddrT to package root
python/aqmp: copy type definitions from qmp
python/aqmp: handle asyncio.TimeoutError on execute()
python/aqmp: add __del__ method to legacy interface
python/aqmp: fix docstring typo
python: use avocado's "new" runner
python: pin setuptools below v60.0.0

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

python: move qmp utilities to python/qemu/utils

In order to upload a QMP package to PyPI, I want to remove any scripts
that I am not 100% confident I want to support upstream, beyond our
castle wall

python: move qmp utilities to python/qemu/utils

In order to upload a QMP package to PyPI, I want to remove any scripts
that I am not 100% confident I want to support upstream, beyond our
castle walls.

Move most of our QMP utilities into the utils package so we can split
them out from the PyPI upload.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python Pull request

Moves QMP-related tools not used for build or automatic testing from
scripts/ to python

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python Pull request

Moves QMP-related tools not used for build or automatic testing from
scripts/ to python/qemu/qmp/ where they will be protected from bitrot by
the check-python-* CI jobs.

stub forwarders are left in the old locations for now.

# gpg: Signature made Sat 19 Jun 2021 00:02:40 BST
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request: (72 commits)
scripts/qmp-shell: add redirection shim
python: add qmp-shell entry point
scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py
scripts/qmp-shell: add docstrings
scripts/qmp-shell: make QMPShellError inherit QMPError
scripts/qmp-shell: remove double-underscores
scripts/qmp-shell: convert usage comment to docstring
scripts/qmp-shell: Remove too-broad-exception
scripts/qmp-shell: Fix empty-transaction invocation
scripts/qmp-shell: remove TODO
scripts/qmp-shell: use logging to show warnings
scripts/qmp-shell: Use context manager instead of atexit
python/qmp: return generic type from context manager
scripts/qmp-shell: unprivatize 'pretty' property
scripts/qmp-shell: Accept SocketAddrT instead of string
scripts/qmp-shell: add mypy types
python/qmp: add QMPObject type alias
scripts/qmp-shell: initialize completer early
scripts/qmp-shell: refactor QMPCompleter
scripts/qmp-shell: Fix "FuzzyJSON" parser
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/

Redirect to the new qom scripts. These forwarders can be deleted
eventually when there has been more time for the dust on the Python
packagi

scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/

Redirect to the new qom scripts. These forwarders can be deleted
eventually when there has been more time for the dust on the Python
packaging to settle and people understand how to find these commands.

Note: You can run these by setting $PYTHONPATH in your shell and then
running "python3 -m qemu.qmp.qom", or you can install the qemu namespace
package and use the "qom" or "qom-set" scripts.

I've written how to install the package elsewhere, but for the sake of
git-blame, cd to ./python, and then do:

- pip3 install [--user] [-e] .

--user will install to your local user install (will not work inside of
a venv), omitting this flag installs to your system-wide packages
(outside of a venv) or to your current virtual environment (inside the
venv).

When installing to a venv or to your system-wide packages, "qom"
should be in your $PATH already. If you do a user install, you may
need to add ~/.local/bin to your $PATH if you haven't already.

-e installs in editable mode: the installed package is effectively just
a symlink to this folder; so changes to your git working tree are
reflected in the installed package.

Note: installing these packages to an environment outside a venv can be
dangerous: Many QEMU scripts will begin to prefer the installed version
instead of the version directly in the tree. Use with caution. editable
mode is recommended when working outside of a venv.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200531' into staging

Python queue:

* migration acceptance test fix
* introduce pylintrc & flake8 config
* various cleanups (Py

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200531' into staging

Python queue:

* migration acceptance test fix
* introduce pylintrc & flake8 config
* various cleanups (Python3, style)
* vm-test can set QEMU_LOCAL=1 to use locally built binaries
* refactored BootLinuxBase & LinuxKernelTest acceptance classes

https://gitlab.com/philmd/qemu/pipelines/151323210
https://travis-ci.org/github/philmd/qemu/builds/693157969

# gpg: Signature made Sun 31 May 2020 17:37:35 BST
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200531: (25 commits)
tests/acceptance: refactor boot_linux to allow code reuse
tests/acceptance: refactor boot_linux_console test to allow code reuse
tests/acceptance: allow console interaction with specific VMs
tests/acceptance/migration.py: Wait for both sides
tests/migration/guestperf: Use Python 3 interpreter
tests/vm: allow wait_ssh() to specify command
tests/vm: Add ability to select QEMU from current build
tests/vm: Pass --debug through for vm-boot-ssh
python/qemu/qtest: Check before accessing _qtest
python/qemu/qmp: assert sockfile is not None
python/qemu/qmp: use True/False for non/blocking modes
python/qemu: Adjust traceback typing
python/qemu: fix socket.makefile() typing
python/qemu: remove Python2 style super() calls
python/qemu: delint; add flake8 config
python/qemu: delint and add pylintrc
python/qemu/machine: remove logging configuration
python/qemu/machine: add kill() method
python: remove more instances of sys.version_info
scripts/qmp: Fix shebang and imports
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

scripts/qmp: Fix shebang and imports

There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.

This just mechanically correct

scripts/qmp: Fix shebang and imports

There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.

This just mechanically corrects the imports and the shebangs, as part of
ensuring that the python/qemu/lib refactoring didn't break anything
needlessly.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20200528222129.23826-2-jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

show more ...

scripts/qmp: Use Python 3 interpreter

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <202005

scripts/qmp: Use Python 3 interpreter

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200207' into staging

- Python 3 cleanups:
. Remove text about Python 2 in qemu-deprecated (Thomas)
. Remove shebang header

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200207' into staging

- Python 3 cleanups:
. Remove text about Python 2 in qemu-deprecated (Thomas)
. Remove shebang header (Paolo, Philippe)
. scripts/checkpatch.pl now allows Python 3 interpreter (Philippe)
. Explicit usage of Python 3 interpreter in scripts (Philippe)
. Fix Python scripts permissions (Paolo, Philippe)
. Drop 'from __future__ import print_function' (Paolo)
. Specify minimum python requirements in ReadTheDocs configuration (Alex)
- Test UNIX/EXEC transports with migration (Oksana)
- Added extract_from_rpm helper, improved extract_from_deb (Liam)
- Allow to use other serial consoles than default one (Philippe)
- Various improvements in QEMUMonitorProtocol (Wainer)
- Fix kvm_available() on ppc64le (Wainer)

# gpg: Signature made Fri 07 Feb 2020 15:01:56 GMT
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200207: (46 commits)
.readthedocs.yml: specify some minimum python requirements
drop "from __future__ import print_function"
make all Python scripts executable
scripts/signrom: remove Python 2 support, add shebang
tests/qemu-iotests/check: Only check for Python 3 interpreter
scripts: Explicit usage of Python 3 (scripts without __main__)
tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__)
tests/vm: Remove shebang header
tests/acceptance: Remove shebang header
scripts/tracetool: Remove shebang header
scripts/minikconf: Explicit usage of Python 3
scripts: Explicit usage of Python 3 (scripts with __main__)
tests: Explicit usage of Python 3
tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)
tests/qemu-iotests/check: Allow use of python3 interpreter
scripts/checkpatch.pl: Only allow Python 3 interpreter
tests/acceptance/migration: Default to -nodefaults
tests/acceptance/migration: Add the 'migration' tag
tests/acceptance/migration: Test EXEC transport when migrating
tests/acceptance/migration: Test UNIX transport when migrating
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

drop "from __future__ import print_function"

This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini

drop "from __future__ import print_function"

This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

show more ...

Merge branch 'master' of git://git.qemu.org/qemu into x86-next

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-06-22

* Partially revert commit that broke qmp-shell and other scripts
inside ./scrip

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-06-22

* Partially revert commit that broke qmp-shell and other scripts
inside ./scripts/qmp

# gpg: Signature made Fri 22 Jun 2018 19:47:23 BST
# gpg: using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
Partially revert "python: futurize -f libfuturize.fixes.fix_absolute_import"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Partially revert "python: futurize -f libfuturize.fixes.fix_absolute_import"

Since commit 068cf7a44cd4d65c05aa877dbebced295be5ce44, qmp-shell
is broken:

$ ./scripts/qmp/qmp-shell
Traceback (mos

Partially revert "python: futurize -f libfuturize.fixes.fix_absolute_import"

Since commit 068cf7a44cd4d65c05aa877dbebced295be5ce44, qmp-shell
is broken:

$ ./scripts/qmp/qmp-shell
Traceback (most recent call last):
File "./scripts/qmp/qmp-shell", line 70, in <module>
from . import qmp
ValueError: Attempted relative import in non-package

Relative imports don't work on scripts that are executed
directly, so revert the change on the scripts inside scripts/qmp.

Fixes: 068cf7a44cd4d65c05aa877dbebced295be5ce44
Reported-by: John Snow <jsnow@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180621175451.7948-1-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-06-11

* Make code compatible with Python 3 using 'futurize --stage1'
* Require Python >

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-06-11

* Make code compatible with Python 3 using 'futurize --stage1'
* Require Python >= 2.7 and remove Python 2.6 compatibility
modules

# gpg: Signature made Mon 11 Jun 2018 18:41:26 BST
# gpg: using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
python: Remove scripts/ordereddict.py
python: Remove scripts/argparse.py
configure: Require Python 2.7 or newer
python: futurize -f lib2to3.fixes.fix_numliterals
python: futurize -f lib2to3.fixes.fix_except
python: futurize -f lib2to3.fixes.fix_renames
python: futurize -f lib2to3.fixes.fix_tuple_params
python: futurize -f lib2to3.fixes.fix_reduce
python: futurize -f lib2to3.fixes.fix_standarderror
python: futurize -f lib2to3.fixes.fix_has_key
python: futurize -f libfuturize.fixes.fix_next_call
python: futurize -f libfuturize.fixes.fix_absolute_import
python: futurize -f libfuturize.fixes.fix_print_with_import

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

python: futurize -f lib2to3.fixes.fix_has_key

Change "dict.has_key(key)" to "key in dict"

This is necessary for Python 3 compatibility.

Done using:

$ py=$( (g grep -l -E '^#!.*python';find -nam

python: futurize -f lib2to3.fixes.fix_has_key

Change "dict.has_key(key)" to "key in dict"

This is necessary for Python 3 compatibility.

Done using:

$ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
sort -u | grep -v README.sh4)
$ futurize -w -f lib2to3.fixes.fix_has_key $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-5-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

python: futurize -f libfuturize.fixes.fix_absolute_import

Make implicit relative imports explicit and add "from __future__ import
absolute_import" at the top of each relevant module.

This is necess

python: futurize -f libfuturize.fixes.fix_absolute_import

Make implicit relative imports explicit and add "from __future__ import
absolute_import" at the top of each relevant module.

This is necessary for Python 3 compatibility.

Done using:

$ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
sort -u | grep -v README.sh4)
$ futurize -w -f libfuturize.fixes.fix_absolute_import $py

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-3-ehabkost@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

python: futurize -f libfuturize.fixes.fix_print_with_import

Change all Python code to use print as a function.

This is necessary for Python 3 compatibility.

Done using:

$ py=$( (g grep -l -E '^

python: futurize -f libfuturize.fixes.fix_print_with_import

Change all Python code to use print as a function.

This is necessary for Python 3 compatibility.

Done using:

$ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
sort -u | grep -v README.sh4)
$ futurize -w -f libfuturize.fixes.fix_print_with_import $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-2-ehabkost@redhat.com>
[ehabkost: fixup tests/docker/docker.py]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

Merge remote-tracking branch 'armbru/tags/pull-qapi-2017-05-23' into staging

QAPI patches for 2017-05-23

# gpg: Signature made Tue 23 May 2017 12:33:32 PM BST
# gpg: using RSA key 0x

Merge remote-tracking branch 'armbru/tags/pull-qapi-2017-05-23' into staging

QAPI patches for 2017-05-23

# gpg: Signature made Tue 23 May 2017 12:33:32 PM BST
# gpg: using RSA key 0x3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* armbru/tags/pull-qapi-2017-05-23:
qapi-schema: Remove obsolete note from ObjectTypeInfo
block: Use QDict helpers for --force-share
shutdown: Expose bool cause in SHUTDOWN and RESET events
shutdown: Add source information to SHUTDOWN and RESET
shutdown: Preserve shutdown cause through replay
shutdown: Prepare for use of an enum in reset/shutdown_request
shutdown: Simplify shutdown_signal
sockets: Plug memory leak in socket_address_flatten()
scripts/qmp/qom-set: fix the value argument passed to srv.command()

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

scripts/qmp/qom-set: fix the value argument passed to srv.command()

When invoking the script with -s, we end up passing a bogus value
to QEMU:

$ ./scripts/qmp/qom-set -s /var/tmp/qmp-sock-exp /mach

scripts/qmp/qom-set: fix the value argument passed to srv.command()

When invoking the script with -s, we end up passing a bogus value
to QEMU:

$ ./scripts/qmp/qom-set -s /var/tmp/qmp-sock-exp /machine.accel kvm
{}
$ ./scripts/qmp/qom-get -s /var/tmp/qmp-sock-exp /machine.accel
/var/tmp/qmp-sock-exp

This happens because sys.argv[2] isn't necessarily the command line
argument that holds the value. It is sys.argv[4] when -s was also
passed.

Actually, the code already has a variable to handle that. This patch
simply uses it.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <149373610338.5144.9635049015143453288.stgit@bahia.lan>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

Merge remote-tracking branch 'luiz/queue/qmp' into staging

# By Luiz Capitulino
# Via Luiz Capitulino
* luiz/queue/qmp:
QMP: qmp-events.txt: alphabetical order fix and other minor changes
QMP: U

Merge remote-tracking branch 'luiz/queue/qmp' into staging

# By Luiz Capitulino
# Via Luiz Capitulino
* luiz/queue/qmp:
QMP: qmp-events.txt: alphabetical order fix and other minor changes
QMP: Update qmp-spec.txt
QMP: Update README file
QMP: QMP/ -> docs/qmp/
QMP: fix qmp-commands.txt generation path
QMP: add scripts/qmp

Message-id: 1379509422-29115-1-git-send-email-lcapitulino@redhat.com

show more ...

QMP: add scripts/qmp

Populate it with all scripts stored in QMP/. Also fixes trailing
whitespaces in qmp-shell and qmp.py.

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: Eric

QMP: add scripts/qmp

Populate it with all scripts stored in QMP/. Also fixes trailing
whitespaces in qmp-shell and qmp.py.

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...