fc00123f | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
python: mkvenv: remove ensure command
This was used to bootstrap the venv with a TOML parser, after which ensuregroup is used. Now that we expect it to be present as a system package (either tomli
python: mkvenv: remove ensure command
This was used to bootstrap the venv with a TOML parser, after which ensuregroup is used. Now that we expect it to be present as a system package (either tomli or, for Python 3.11, tomllib), it is not needed anymore.
Note that this means that, when implemented, the hypothetical "isolated" mode that does not use any system packages will only work with Python 3.11+.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
0a88ac96 | 16-May-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
Revert "mkvenv: work around broken pip installations on Debian 10"
Debian 10 has Python 3.7, so it is not possible to use it anymore now that Python 3.8 is required.
Signed-off-by: Paolo Bonzini <p
Revert "mkvenv: work around broken pip installations on Debian 10"
Debian 10 has Python 3.7, so it is not possible to use it anymore now that Python 3.8 is required.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
3e4b6b0a | 06-Jun-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
mkvenv: assume presence of importlib.metadata
importlib.metadata is included in Python 3.8, so there is no need to fallback to either importlib-metadata or pkgresources when generating console scrip
mkvenv: assume presence of importlib.metadata
importlib.metadata is included in Python 3.8, so there is no need to fallback to either importlib-metadata or pkgresources when generating console script shims.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
c03f57fd | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.
ensuregroup allows to specify both the acceptable versions of avocado, and
Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.
ensuregroup allows to specify both the acceptable versions of avocado, and a locked version to be used when avocado is not installed as a system pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and reuse the distro package on Fedora and CentOS Stream (the only distros where it's available).
ensuregroup's usage of "(>=..., <=...)" constraints when evaluating the distro package, and "==" constraints when installing it from PyPI, makes it possible to avoid conflicts between the known-good version and a package plugins included in the distro.
This is because package plugins have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in an error:
avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
But at the same time, if the venv does not include a system distribution of avocado then we can install a known-good version and stick to LTS releases.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
71ed611c | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
python: mkvenv: add ensuregroup command
Introduce a new subcommand that retrieves the packages to be installed from a TOML file. This allows being more flexible in using the system version of a pack
python: mkvenv: add ensuregroup command
Introduce a new subcommand that retrieves the packages to be installed from a TOML file. This allows being more flexible in using the system version of a package, while at the same time using a known-good version when installing the package. This is important for packages that sometimes have backwards-incompatible changes or that depend on specific versions of their dependencies.
Compared to JSON, TOML is more human readable and easier to edit. A parser is available in 3.11 but also available as a small (12k) package for older versions, tomli. While tomli is bundled with pip, this is only true of recent versions of pip. Of all the supported OSes pretty much only FreeBSD has a recent enough version of pip while staying on Python <3.11. So we cannot use the same trick that is in place for distlib.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
0f1ec070 | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
python: mkvenv: introduce TOML-like representation of dependencies
We would like to place all Python dependencies in the same file, so that we can add more information without having long and comple
python: mkvenv: introduce TOML-like representation of dependencies
We would like to place all Python dependencies in the same file, so that we can add more information without having long and complex command lines. The plan is to have a TOML file with one entry per package, for example
[avocado] avocado-framework = { accepted = "(>=88.1, <93.0)", installed = "88.1", canary = "avocado" }
Each TOML section will thus be a dictionary of dictionaries. Modify mkvenv.py's workhorse function, _do_ensure, to already operate on such a data structure. The "ensure" subcommand is modified to separate the depspec into a name and a version part, and use the result (plus the --diagnose argument) to build a dictionary for each command line argument.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
d37c21b5 | 19-May-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
mkvenv: pass first missing package to diagnose()
If sphinx is present but the theme is not, mkvenv will print an inaccurate diagnostic:
ERROR: Could not find a version that satisfies the requiremen
mkvenv: pass first missing package to diagnose()
If sphinx is present but the theme is not, mkvenv will print an inaccurate diagnostic:
ERROR: Could not find a version that satisfies the requirement sphinx-rtd-theme>=0.5.0 (from versions: none) ERROR: No matching distribution found for sphinx-rtd-theme>=0.5.0
'sphinx>=1.6.0' not found: • Python package 'sphinx' version '5.3.0' was found, but isn't suitable. • mkvenv was configured to operate offline and did not check PyPI.
Instead, ignore the packages that were found to be present, and report an error based on the first absent package.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
3b087f79 | 23-Dec-2021 |
Paolo Bonzini <pbonzini@redhat.com> |
meson: require 0.63.0
This version allows cleanups in modinfo collection, but they only work with Ninja 1.9.x and 1.8.x is still supported. It also supports the equivalent of QEMU's --static option
meson: require 0.63.0
This version allows cleanups in modinfo collection, but they only work with Ninja 1.9.x and 1.8.x is still supported. It also supports the equivalent of QEMU's --static option to configure.
The wheel file is bumped to 0.63.3, the last release in the 0.63 branch.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
2274817f | 10-May-2023 |
John Snow <jsnow@redhat.com> |
python: add vendor.py utility
This is a teeny-tiny script that just downloads any packages we want to vendor from PyPI and stores them in qemu.git/python/wheels/. If I'm hit by a meteor, it'll be ea
python: add vendor.py utility
This is a teeny-tiny script that just downloads any packages we want to vendor from PyPI and stores them in qemu.git/python/wheels/. If I'm hit by a meteor, it'll be easy to replicate what I have done in order to udpate the vendored source.
We don't really care which python runs it; it exists as a meta-utility with no external dependencies and we won't package or install it. It will be monitored by the linters/type checkers, though; so it's guaranteed safe on python 3.6+.
Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-15-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
c8049626 | 16-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: work around broken pip installations on Debian 10
This is a workaround intended for Debian 10, where the debian-patched pip does not function correctly if accessed from within a virtual envi
mkvenv: work around broken pip installations on Debian 10
This is a workaround intended for Debian 10, where the debian-patched pip does not function correctly if accessed from within a virtual environment.
We don't support Debian 10 as a build platform any longer, though we do still utilize it for our build-tricore-softmmu CI test. It's also possible that this bug might appear on other derivative platforms and this workaround may prove useful.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-11-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
68ea6d17 | 16-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: use pip's vendored distlib as a fallback
distlib is usually not installed on Linux distribution, but it is vendored into pip. Because the virtual environment has pip via ensurepip, we can p
mkvenv: use pip's vendored distlib as a fallback
distlib is usually not installed on Linux distribution, but it is vendored into pip. Because the virtual environment has pip via ensurepip, we can piggy-back on pip's vendored version. This could break if they move our cheese in the future, but the fix would be simply to require distlib.
If it is debundled, as it is on msys, it is simply available directly.
Signed-off-by: John Snow <jsnow@redhat.com> [Move to toplevel. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
92834894 | 16-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: add console script entry point generation
When creating a virtual environment that inherits system packages, script entry points (like "meson", "sphinx-build", etc) are not re-generated with
mkvenv: add console script entry point generation
When creating a virtual environment that inherits system packages, script entry points (like "meson", "sphinx-build", etc) are not re-generated with the correct shebang. When you are *inside* of the venv, this is not a problem, but if you are *outside* of it, you will not have a script that engages the virtual environment appropriately.
Add a mechanism that generates new entry points for pre-existing packages so that we can use these scripts to run "meson", "sphinx-build", "pip", unambiguously inside the venv.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-9-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
4695a22e | 10-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: add --diagnose option to explain "ensure" failures
This is a routine that is designed to print some usable info for human beings back out to the terminal if/when "mkvenv ensure" fails to loc
mkvenv: add --diagnose option to explain "ensure" failures
This is a routine that is designed to print some usable info for human beings back out to the terminal if/when "mkvenv ensure" fails to locate or install a package during configure time, such as meson or sphinx.
Since we are requiring that "meson" and "sphinx" are installed to the same Python environment as QEMU is configured to build with, this can produce some surprising failures when things are mismatched. This method is here to try and ease that sting by offering some actionable diagnosis.
Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-8-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
c5538eed | 10-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: add ensure subcommand
This command is to be used to add various packages (or ensure they're already present) into the configure-provided venv in a modular fashion.
Examples:
mkvenv ensure
mkvenv: add ensure subcommand
This command is to be used to add various packages (or ensure they're already present) into the configure-provided venv in a modular fashion.
Examples:
mkvenv ensure --online --dir "${source_dir}/python/wheels/" "meson>=0.61.5" mkvenv ensure --online "sphinx>=1.6.0" mkvenv ensure "qemu.qmp==0.0.2"
It's designed to look for packages in three places, in order:
(1) In system packages, if the version installed is already good enough. This way your distribution-provided meson, sphinx, etc are always used as first preference.
(2) In a vendored packages directory. Here I am suggesting qemu.git/python/wheels/ as that directory. This is intended to serve as a replacement for vendoring the meson source for QEMU tarballs. It is also highly likely to be extremely useful for packaging the "qemu.qmp" package in source distributions for platforms that do not yet package qemu.qmp separately.
(3) Online, via PyPI, ***only when "--online" is passed***. This is only ever used as a fallback if the first two sources do not have an appropriate package that meets the requirement. The ability to build QEMU and run tests *completely offline* is not impinged.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-7-jsnow@redhat.com> [Use distlib to lookup distributions. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|