6a59e3c5 | 15-Jul-2025 |
Daniel P. Berrangé <berrange@redhat.com> |
python: ensure QEMUQtestProtocol closes its socket
While QEMUQtestMachine closes the socket that was passed to QEMUQtestProtocol, the python resource leak manager still believes that the copy QEMUQt
python: ensure QEMUQtestProtocol closes its socket
While QEMUQtestMachine closes the socket that was passed to QEMUQtestProtocol, the python resource leak manager still believes that the copy QEMUQtestProtocol holds is open. We must explicitly call close to avoid this leak warnnig.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit 6ccb48ffc19fe25511313a246d4a8bad51114ea9) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
7c9d65f9 | 03-Sep-2025 |
John Snow <jsnow@redhat.com> |
python: backport 'avoid creating additional event loops per thread'
This commit is two backports squashed into one to avoid regressions.
python: *really* remove get_event_loop
A prior commit, aa1f
python: backport 'avoid creating additional event loops per thread'
This commit is two backports squashed into one to avoid regressions.
python: *really* remove get_event_loop
A prior commit, aa1ff990, switched away from using get_event_loop *by default*, but this is not good enough to avoid deprecation warnings as `asyncio.get_event_loop_policy().get_event_loop()` is *also* deprecated. Replace this mechanism with explicit calls to asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to match.
python: avoid creating additional event loops per thread
"Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated behavior altogether by calling new_event_loop() directly if there was no loop currently running, but this has the unfortunate side effect of potentially creating multiple event loops per thread if tests instantiate multiple QMP connections in a single thread. This behavior is apparently not well-defined and causes problems in some, but not all, combinations of Python interpreter version and platform environment.
Partially revert to Daniel Berrange's original patch, which calls get_event_loop and simply suppresses the deprecation warning in Python<=3.13. This time, however, additionally register new loops created with new_event_loop() so that future calls to get_event_loop() will return the loop already created.
Reported-by: Richard W.M. Jones <rjones@redhat.com> Reported-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@21ce2ee4f2df87efe84a27b9c5112487f4670622 cherry picked from commit python-qemu-qmp@c08fb82b38212956ccffc03fc6d015c3979f42fe Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit 85f223e5b031eb8ab63fbca314a4fb296a3a2632) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
1034cd16 | 13-Aug-2024 |
John Snow <jsnow@redhat.com> |
python: backport 'Remove deprecated get_event_loop calls'
This method was deprecated in 3.12 because it ordinarily should not be used from coroutines; if there is not a currently running event loop,
python: backport 'Remove deprecated get_event_loop calls'
This method was deprecated in 3.12 because it ordinarily should not be used from coroutines; if there is not a currently running event loop, this automatically creates a new event loop - which is usually not what you want from code that would ever run in the bottom half.
In our case, we do want this behavior in two places:
(1) The synchronous shim, for convenience: this allows fully sync programs to use QEMUMonitorProtocol() without needing to set up an event loop beforehand. This is intentional to fully box in the async complexities into the legacy sync shim.
(2) The qmp_tui shell; instead of relying on asyncio.run to create and run an asyncio program, we need to be able to pass the current asyncio loop to urwid setup functions. For convenience, again, we create one if one is not present to simplify the creation of the TUI appliance.
The remaining user of get_event_loop() was in fact one of the erroneous users that should not have been using this function: if there's no running event loop inside of a coroutine, you're in big trouble :)
Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@aa1ff9907603a3033296027e1bd021133df86ef1 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit 5d99044d09db0fa8c2b3294e301927118f9effc9) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
67d93471 | 22-Jul-2022 |
John Snow <jsnow@redhat.com> |
python: backport 'qmp-tui: Do not crash if optional dependencies are not met'
Based on the discussion at https://github.com/pypa/pip/issues/9726 - even though the setuptools documentation implies th
python: backport 'qmp-tui: Do not crash if optional dependencies are not met'
Based on the discussion at https://github.com/pypa/pip/issues/9726 - even though the setuptools documentation implies that it is possible to guard script execution with optional dependency groups, this is not true in practice with the scripts generated by pip.
Just do the simple thing and guard the import statements.
Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@df520dcacf9a75dd4c82ab1129768de4128b554c Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit fd0ed46d4effbf2700804657bad9c6db086527c4) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
677a4e9d | 22-Jul-2022 |
John Snow <jsnow@redhat.com> |
python: backport 'qmp-shell-wrap: handle missing binary gracefully'
Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@9c889dcbd58817b0c917a9d2dd16161f48ac8203 Sig
python: backport 'qmp-shell-wrap: handle missing binary gracefully'
Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@9c889dcbd58817b0c917a9d2dd16161f48ac8203 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit fcaeeb7653d2c6f38183170e1cae5729adb7875c) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
5f395651 | 06-Jun-2023 |
John Snow <jsnow@redhat.com> |
python: backport 'Use @asynciocontextmanager'
This removes a non-idiomatic use of a "coroutine callback" in favor of something a bit more standardized.
Signed-off-by: John Snow <jsnow@redhat.com> c
python: backport 'Use @asynciocontextmanager'
This removes a non-idiomatic use of a "coroutine callback" in favor of something a bit more standardized.
Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@commit 97f7ffa3be17a50544b52767d14b6fd478c07b9e Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit 0408b8d7a086486f5c1887798be744b2d73bcda9) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
f246e6ef | 06-Jun-2023 |
John Snow <jsnow@redhat.com> |
python: backport 'drop Python3.6 workarounds'
Now that the minimum version is 3.7, drop some of the 3.6-specific hacks we've been carrying. A single remaining compatibility hack concerning 3.6's lac
python: backport 'drop Python3.6 workarounds'
Now that the minimum version is 3.7, drop some of the 3.6-specific hacks we've been carrying. A single remaining compatibility hack concerning 3.6's lack of @asynccontextmanager is addressed in the following commit.
Signed-off-by: John Snow <jsnow@redhat.com> cherry picked from commit python-qemu-qmp@3e8e34e594cfc6b707e6f67959166acde4b421b8 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (cherry picked from commit f9d2e0a3bd7ba2a693a892881f91cf53fa90cc71) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
show more ...
|
64e4375b | 15-Jul-2025 |
John Snow <jsnow@redhat.com> |
python: fix editable installs for modern pip/setuptools
The way editable installs work has changed at some point since Fedora 40 was released. Generally, we should be opting to use pyproject.toml in
python: fix editable installs for modern pip/setuptools
The way editable installs work has changed at some point since Fedora 40 was released. Generally, we should be opting to use pyproject.toml installs (PEP517/518) - but those are not fully supported until v61 of setuptools, and CentOS Stream 9 ships v53.
Until that time, we can make use of a transitional feature in pip/setuptools to use "legacy" editable installs, which is enough to fix "make check-dev" on modern local workstations for now.
By using the environment variable approach to configure pip, we avoid any problems for older versions of pip that don't recognize this option, so it's harmless. The config-settings option first appeared in v23 of pip. editable_mode was first supported by setuptools in v64.
(I'm not currently precisely aware of when the default behavior of '-e' switched away from 'compat', but it appears to be a joint effect between setuptools and pip versions.)
Version information for supported build platforms:
distro python3 pip setuptools sphinx -------------------------------------------------------- centos_stream_9 3.9.23 21.3.1 53.0.0 3.4.3 ubuntu_22_04 3.10.12 22.0.2 59.6.0 4.3.2
** pyproject.toml installs supported as of here **
freebsd 3.11.13 23.3.2 63.1.0 5.3.0 debian_12 3.11.2 23.0.1 66.1.1 5.3.0 ubuntu_24_04 3.12.3 24.0 68.1.2 7.2.6 centos_stream_10 3.12.11 23.3.2 69.0.3 7.2.6 fedora_41 3.13.5 24.2 69.2.0 7.3.7 alpine_3_19 3.11.13 23.3.1 70.3.0 6.2.1 alpine_3_20 3.12.11 24.0 70.3.0 7.2.6 alpine_3_21 3.12.11 24.3.1 70.3.0 8.1.3 ubuntu_24_10 3.12.7 24.2 74.1.2 7.4.7 fedora_42 3.13.5 24.3.1 74.1.3 8.1.3 ubuntu_25_04 3.13.3 25.0 75.8.0 8.1.3 macports 3.13.5 25.1.1 78.1.1 8.2.3 openbsd 3.12.11 25.1.1 79.0.1 8.2.3 alpine_3_22 3.12.11 25.1.1 80.9.0 8.2.3 homebrew 3.13.5 --- 80.9.0 8.2.3 pkgsrc_current 3.12.11 25.1.1 80.9.0 8.2.3
Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250715222548.198888-1-jsnow@redhat.com> Tested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
show more ...
|
781e7305 | 04-Jun-2025 |
Markus Armbruster <armbru@redhat.com> |
python: Drop redundant warn_unused_configs = True
strict = True implies warn_unused_configs = True.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
python: Drop redundant warn_unused_configs = True
strict = True implies warn_unused_configs = True.
Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20250604200354.459501-6-jsnow@redhat.com
show more ...
|
65aa0a17 | 04-Jun-2025 |
John Snow <jsnow@redhat.com> |
python: add qapi static analysis tests
Update the python tests to also check QAPI and the QAPI Sphinx extensions. The docs/sphinx/qapidoc_legacy.py file is not included in these checks, as it is des
python: add qapi static analysis tests
Update the python tests to also check QAPI and the QAPI Sphinx extensions. The docs/sphinx/qapidoc_legacy.py file is not included in these checks, as it is destined for removal soon. mypy is also not called on the QAPI Sphinx extensions, owing to difficulties supporting Sphinx 3.x - 8.x while maintaining static type checking support. mypy *is* called on all of the QAPI tools themselves, though.
flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in scripts/qapi/ - that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date.
The list of pylint ignores is also updated, owing again to the wide window of pylint version support: newer versions require pragmas to occasionally silence the "too many positional arguments" warning, but older versions do not have such a warning category and will instead yelp about an unrecognized option. Silence that warning, too.
As a result of this patch, one would be able to run any of the following tests locally from the qemu.git/python directory and have it cover the QAPI tooling as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters.
- "make check-minreqs" Run tests specifically under our oldest supported Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do not regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test helps catch bleeding-edge problems before they become problems for developer workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions.
- "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to have installed. This is a quick convenience check that does not map to any particular GitLab test.
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt)
Finally, finally, finally: this means that QAPI tooling will be linted and type-checked from the GitLab pipelines.
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20250604200354.459501-5-jsnow@redhat.com [Edited license choice per review --js] Signed-off-by: John Snow <jsnow@redhat.com>
show more ...
|
05fd7214 | 01-Nov-2024 |
John Snow <jsnow@redhat.com> |
python: silence pylint raising-non-exception error
As of (at least) pylint 3.3.1, this code trips pylint up into believing we are raising something other than an Exception. We are not: the first two
python: silence pylint raising-non-exception error
As of (at least) pylint 3.3.1, this code trips pylint up into believing we are raising something other than an Exception. We are not: the first two values may indeed be "None", but the last and final value must by definition be a SystemExit exception.
Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20241101173700.965776-5-jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
show more ...
|
45b14be9 | 26-Jun-2024 |
John Snow <jsnow@redhat.com> |
python: enable testing for 3.13
Python 3.13 is in beta and Fedora 41 is preparing to make it the default system interpreter; enable testing for it.
(In the event problems develop prior to release,
python: enable testing for 3.13
Python 3.13 is in beta and Fedora 41 is preparing to make it the default system interpreter; enable testing for it.
(In the event problems develop prior to release, it should only impact the check-python-tox job, which is not run by default and is allowed to fail.)
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 20240626232230.408004-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
show more ...
|
c5be2445 | 26-Jun-2024 |
John Snow <jsnow@redhat.com> |
python: Do not use pylint 3.2.4 with python 3.8
There is a bug in this version, see: https://github.com/pylint-dev/pylint/issues/9751
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Alex B
python: Do not use pylint 3.2.4 with python 3.8
There is a bug in this version, see: https://github.com/pylint-dev/pylint/issues/9751
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 20240626232230.408004-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
show more ...
|