| fc00123f | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
python: mkvenv: remove ensure command
This was used to bootstrap the venv with a TOML parser, after which
ensuregroup is used. Now that we expect it to be present as a system
package (either tomli
python: mkvenv: remove ensure command
This was used to bootstrap the venv with a TOML parser, after which
ensuregroup is used. Now that we expect it to be present as a system
package (either tomli or, for Python 3.11, tomllib), it is not needed
anymore.
Note that this means that, when implemented, the hypothetical "isolated"
mode that does not use any system packages will only work with Python
3.11+.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 0a88ac96 | 16-May-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
Revert "mkvenv: work around broken pip installations on Debian 10"
Debian 10 has Python 3.7, so it is not possible to use it anymore
now that Python 3.8 is required.
Signed-off-by: Paolo Bonzini <p
Revert "mkvenv: work around broken pip installations on Debian 10"
Debian 10 has Python 3.7, so it is not possible to use it anymore
now that Python 3.8 is required.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 3e4b6b0a | 06-Jun-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
mkvenv: assume presence of importlib.metadata
importlib.metadata is included in Python 3.8, so there is no
need to fallback to either importlib-metadata or pkgresources
when generating console scrip
mkvenv: assume presence of importlib.metadata
importlib.metadata is included in Python 3.8, so there is no
need to fallback to either importlib-metadata or pkgresources
when generating console script shims.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| c03f57fd | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.
ensuregroup allows to specify both the acceptable versions of avocado,
and
Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.
ensuregroup allows to specify both the acceptable versions of avocado,
and a locked version to be used when avocado is not installed as a system
pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and
reuse the distro package on Fedora and CentOS Stream (the only distros
where it's available).
ensuregroup's usage of "(>=..., <=...)" constraints when evaluating
the distro package, and "==" constraints when installing it from PyPI,
makes it possible to avoid conflicts between the known-good version and
a package plugins included in the distro.
This is because package plugins have "==" constraints on the version
that is included in the distro, and, using "pip install avocado==88.1"
on a venv that includes system packages will result in an error:
avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
But at the same time, if the venv does not include a system distribution
of avocado then we can install a known-good version and stick to LTS
releases.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 71ed611c | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
python: mkvenv: add ensuregroup command
Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a pack
python: mkvenv: add ensuregroup command
Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a package, while at the same time using a known-good version
when installing the package. This is important for packages that
sometimes have backwards-incompatible changes or that depend on
specific versions of their dependencies.
Compared to JSON, TOML is more human readable and easier to edit. A
parser is available in 3.11 but also available as a small (12k) package
for older versions, tomli. While tomli is bundled with pip, this is only
true of recent versions of pip. Of all the supported OSes pretty much
only FreeBSD has a recent enough version of pip while staying on Python
<3.11. So we cannot use the same trick that is in place for distlib.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 0f1ec070 | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
python: mkvenv: introduce TOML-like representation of dependencies
We would like to place all Python dependencies in the same file, so that
we can add more information without having long and comple
python: mkvenv: introduce TOML-like representation of dependencies
We would like to place all Python dependencies in the same file, so that
we can add more information without having long and complex command lines.
The plan is to have a TOML file with one entry per package, for example
[avocado]
avocado-framework = {
accepted = "(>=88.1, <93.0)",
installed = "88.1",
canary = "avocado"
}
Each TOML section will thus be a dictionary of dictionaries. Modify
mkvenv.py's workhorse function, _do_ensure, to already operate on such
a data structure. The "ensure" subcommand is modified to separate the
depspec into a name and a version part, and use the result (plus the
--diagnose argument) to build a dictionary for each command line argument.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| d37c21b5 | 19-May-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
mkvenv: pass first missing package to diagnose()
If sphinx is present but the theme is not, mkvenv will print an
inaccurate diagnostic:
ERROR: Could not find a version that satisfies the requiremen
mkvenv: pass first missing package to diagnose()
If sphinx is present but the theme is not, mkvenv will print an
inaccurate diagnostic:
ERROR: Could not find a version that satisfies the requirement sphinx-rtd-theme>=0.5.0 (from versions: none)
ERROR: No matching distribution found for sphinx-rtd-theme>=0.5.0
'sphinx>=1.6.0' not found:
• Python package 'sphinx' version '5.3.0' was found, but isn't suitable.
• mkvenv was configured to operate offline and did not check PyPI.
Instead, ignore the packages that were found to be present, and report
an error based on the first absent package.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 3b087f79 | 23-Dec-2021 |
Paolo Bonzini <pbonzini@redhat.com> |
meson: require 0.63.0
This version allows cleanups in modinfo collection, but they only
work with Ninja 1.9.x and 1.8.x is still supported. It also supports the
equivalent of QEMU's --static option
meson: require 0.63.0
This version allows cleanups in modinfo collection, but they only
work with Ninja 1.9.x and 1.8.x is still supported. It also supports the
equivalent of QEMU's --static option to configure.
The wheel file is bumped to 0.63.3, the last release in the 0.63 branch.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 2274817f | 10-May-2023 |
John Snow <jsnow@redhat.com> |
python: add vendor.py utility
This is a teeny-tiny script that just downloads any packages we want to
vendor from PyPI and stores them in qemu.git/python/wheels/. If I'm hit
by a meteor, it'll be ea
python: add vendor.py utility
This is a teeny-tiny script that just downloads any packages we want to
vendor from PyPI and stores them in qemu.git/python/wheels/. If I'm hit
by a meteor, it'll be easy to replicate what I have done in order to
udpate the vendored source.
We don't really care which python runs it; it exists as a meta-utility
with no external dependencies and we won't package or install it. It
will be monitored by the linters/type checkers, though; so it's
guaranteed safe on python 3.6+.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-15-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| c8049626 | 16-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: work around broken pip installations on Debian 10
This is a workaround intended for Debian 10, where the debian-patched
pip does not function correctly if accessed from within a virtual
envi
mkvenv: work around broken pip installations on Debian 10
This is a workaround intended for Debian 10, where the debian-patched
pip does not function correctly if accessed from within a virtual
environment.
We don't support Debian 10 as a build platform any longer, though we do
still utilize it for our build-tricore-softmmu CI test. It's also
possible that this bug might appear on other derivative platforms and
this workaround may prove useful.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-11-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 68ea6d17 | 16-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: use pip's vendored distlib as a fallback
distlib is usually not installed on Linux distribution, but it is vendored
into pip. Because the virtual environment has pip via ensurepip, we
can p
mkvenv: use pip's vendored distlib as a fallback
distlib is usually not installed on Linux distribution, but it is vendored
into pip. Because the virtual environment has pip via ensurepip, we
can piggy-back on pip's vendored version. This could break if they move
our cheese in the future, but the fix would be simply to require distlib.
If it is debundled, as it is on msys, it is simply available directly.
Signed-off-by: John Snow <jsnow@redhat.com>
[Move to toplevel. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 92834894 | 16-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: add console script entry point generation
When creating a virtual environment that inherits system packages,
script entry points (like "meson", "sphinx-build", etc) are not
re-generated with
mkvenv: add console script entry point generation
When creating a virtual environment that inherits system packages,
script entry points (like "meson", "sphinx-build", etc) are not
re-generated with the correct shebang. When you are *inside* of the
venv, this is not a problem, but if you are *outside* of it, you will
not have a script that engages the virtual environment appropriately.
Add a mechanism that generates new entry points for pre-existing
packages so that we can use these scripts to run "meson",
"sphinx-build", "pip", unambiguously inside the venv.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-9-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| 4695a22e | 10-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: add --diagnose option to explain "ensure" failures
This is a routine that is designed to print some usable info for human
beings back out to the terminal if/when "mkvenv ensure" fails to loc
mkvenv: add --diagnose option to explain "ensure" failures
This is a routine that is designed to print some usable info for human
beings back out to the terminal if/when "mkvenv ensure" fails to locate
or install a package during configure time, such as meson or sphinx.
Since we are requiring that "meson" and "sphinx" are installed to the
same Python environment as QEMU is configured to build with, this can
produce some surprising failures when things are mismatched. This method
is here to try and ease that sting by offering some actionable
diagnosis.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-8-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
| c5538eed | 10-May-2023 |
John Snow <jsnow@redhat.com> |
mkvenv: add ensure subcommand
This command is to be used to add various packages (or ensure they're
already present) into the configure-provided venv in a modular fashion.
Examples:
mkvenv ensure
mkvenv: add ensure subcommand
This command is to be used to add various packages (or ensure they're
already present) into the configure-provided venv in a modular fashion.
Examples:
mkvenv ensure --online --dir "${source_dir}/python/wheels/" "meson>=0.61.5"
mkvenv ensure --online "sphinx>=1.6.0"
mkvenv ensure "qemu.qmp==0.0.2"
It's designed to look for packages in three places, in order:
(1) In system packages, if the version installed is already good
enough. This way your distribution-provided meson, sphinx, etc are
always used as first preference.
(2) In a vendored packages directory. Here I am suggesting
qemu.git/python/wheels/ as that directory. This is intended to serve as
a replacement for vendoring the meson source for QEMU tarballs. It is
also highly likely to be extremely useful for packaging the "qemu.qmp"
package in source distributions for platforms that do not yet package
qemu.qmp separately.
(3) Online, via PyPI, ***only when "--online" is passed***. This is only
ever used as a fallback if the first two sources do not have an
appropriate package that meets the requirement. The ability to build
QEMU and run tests *completely offline* is not impinged.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-7-jsnow@redhat.com>
[Use distlib to lookup distributions. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|