qapi: Fix undocumented return values by generating something

Generated command documentation lacks information on return value in
several cases, e.g. query-tpm.

The obvious fix would be to require

qapi: Fix undocumented return values by generating something

Generated command documentation lacks information on return value in
several cases, e.g. query-tpm.

The obvious fix would be to require a Returns: section when a command
returns something.

However, note that many existing Returns: sections are pretty useless:
the description is basically the return type, which then gets rendered
like "Return: <Type> – <basically the return type>". This suggests
that a description is often not really necessary, and requiring one
isn't useful.

Instead, generate the obvious minimal thing when Returns: is absent:
"Return: <Type>".

This auto-generated Return documentation is placed is as follows:

1. If we have arguments, return goes right after them.
2. Else if we have errors, return goes right before them.
3. Else if we have features, return goes right before them.
4. Else return goes right after the intro

To facilitate this algorithm, a "TODO:" hack line is used to separate
the intro from the remainder of the documentation block in cases where
there are no other sections to separate the intro from e.g. examples and
additional detail meant to appear below the key sections of interest.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250711051045.51110-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[_insert_near_kind() code replaced by something simpler, commit
message amended to explain why we're doing this]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: lift restriction on using '=' in doc blocks

We reject lines starting with '=' in definition documentation. This
made sense when such lines were headings in free-form documentation,
but not in

qapi: lift restriction on using '=' in doc blocks

We reject lines starting with '=' in definition documentation. This
made sense when such lines were headings in free-form documentation,
but not in definition documentation.

Before the previous commit, lines starting with '=' were headings in
free-form documentation, and rejected in definition documentation,
where such headings could not work.

The previous commit dropped the headings feature from free-form
documentation, because we can simply use plain rST headings.
Rejecting them in definition documentation no longer makes sense, so
drop that, too.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Amend commit message to explain why]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/sphinx: remove special parsing for freeform sections

Remove the QAPI doc section heading syntax, use plain rST section
headings instead.

Tests and documentation are updated to match.

Interest

docs/sphinx: remove special parsing for freeform sections

Remove the QAPI doc section heading syntax, use plain rST section
headings instead.

Tests and documentation are updated to match.

Interestingly, Plain rST headings work fine before this patch, except
for over- and underlining with '=', which the doc parser rejected as
invalid QAPI doc section heading in free-form comments.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Add more detail to commit message]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: delete un-needed python static analysis configs

Since the previous commit, python/setup.cfg applies to scripts/qapi/ as
well. Configuration files in scripts/qapi/ override python/setup.cfg.

qapi: delete un-needed python static analysis configs

Since the previous commit, python/setup.cfg applies to scripts/qapi/ as
well. Configuration files in scripts/qapi/ override python/setup.cfg.

scripts/qapi/.flake8 and scripts/qapi/.isort.cfg actually match
python/setup.cfg exactly, and can go.

The differences between scripts/qapi/mypy.ini and python/setup.cfg are
harmless: namespace_packages being set to True is a requirement for the
PEP420 nested package structure of QEMU but not for scripts/qapi, but
has no effect on type checking the QAPI code. warn_unused_ignores is
used in python/ to be able to target a wide variety of mypy versions;
some of which that have added new ignore categories that are not present
in older versions.

Ultimately, scripts/qapi/mypy.ini can be removed without any real change
in behavior to how mypy enforces type safety there.

The pylint config is being left in place because the settings differ
enough from the python/ directory settings that we need a chit-chat on
how to merge them O:-)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.459501-7-jsnow@redhat.com

show more ...

python: add qapi static analysis tests

Update the python tests to also check QAPI and the QAPI Sphinx
extensions. The docs/sphinx/qapidoc_legacy.py file is not included in
these checks, as it is des

python: add qapi static analysis tests

Update the python tests to also check QAPI and the QAPI Sphinx
extensions. The docs/sphinx/qapidoc_legacy.py file is not included in
these checks, as it is destined for removal soon. mypy is also not
called on the QAPI Sphinx extensions, owing to difficulties supporting
Sphinx 3.x - 8.x while maintaining static type checking support. mypy
*is* called on all of the QAPI tools themselves, though.

flake8, isort and mypy use the tool configuration from the existing
python directory (in setup.cfg). pylint continues to use the special
configuration located in scripts/qapi/ - that configuration is more
permissive. If we wish to unify the two configurations, that's a
separate series and a discussion for a later date.

The list of pylint ignores is also updated, owing again to the wide
window of pylint version support: newer versions require pragmas to
occasionally silence the "too many positional arguments" warning, but
older versions do not have such a warning category and will instead yelp
about an unrecognized option. Silence that warning, too.

As a result of this patch, one would be able to run any of the following
tests locally from the qemu.git/python directory and have it cover the
QAPI tooling as well. All of the following options run the python tests,
static analysis tests, and linter checks; but with different
combinations of dependencies and interpreters.

- "make check-minreqs" Run tests specifically under our oldest supported
Python and our oldest supported dependencies. This is the test that
runs on GitLab as "check-python-minreqs". This helps ensure we do not
regress support on older platforms accidentally.

- "make check-tox" Runs the tests under the newest supported
dependencies, but under each supported version of Python in turn. At
time of writing, this is Python 3.8 to 3.13 inclusive. This test helps
catch bleeding-edge problems before they become problems for developer
workstations. This is the GitLab test "check-python-tox" and is an
optionally run, may-fail test due to the unpredictable nature of new
dependencies being released into the ecosystem that may cause
regressions.

- "make check-dev" Runs the tests under the newest supported
dependencies using whatever version of Python the user happens to have
installed. This is a quick convenience check that does not map to any
particular GitLab test.

(Note! check-dev may be busted on Fedora 41 and bleeding edge versions
of setuptools. That's unrelated to this patch and I'll address it
separately and soon. Thank you for your patience, --mgmt)

Finally, finally, finally: this means that QAPI tooling will be linted
and type-checked from the GitLab pipelines.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.459501-5-jsnow@redhat.com
[Edited license choice per review --js]
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

qapi: Add some pylint ignores

This restores the linting baseline in QAPI.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.4

qapi: Add some pylint ignores

This restores the linting baseline in QAPI.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.459501-2-jsnow@redhat.com

show more ...

Drop support for Python 3.8

Python 3.8 went "end of life" in October 2024 and Fedora 42 dropped
this version already, so the "python" CI job is currently failing.
Thus it's time to drop support for

Drop support for Python 3.8

Python 3.8 went "end of life" in October 2024 and Fedora 42 dropped
this version already, so the "python" CI job is currently failing.
Thus it's time to drop support for this Python version in QEMU, too.

While we're at it, also look for "python3.13" in the configure script.

Message-ID: <20250425120710.879518-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

scripts/qapi/backend: Clean up create_backend()'s failure mode

create_backend()'s caller catches QAPIError, and returns non-zero exit
code on catch. The caller's caller passes the exit code to
sys.

scripts/qapi/backend: Clean up create_backend()'s failure mode

create_backend()'s caller catches QAPIError, and returns non-zero exit
code on catch. The caller's caller passes the exit code to
sys.exit().

create_backend() doesn't care: it reports errors to stderr and
sys.exit()s.

Change it to raise QAPIError instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250311065352.992307-1-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

show more ...

qapi/parser: add undocumented stub members to all_sections

Parser and doc generator cooperate on generating stub documentation for
undocumented members. The parser makes up an ArgSection with an em

qapi/parser: add undocumented stub members to all_sections

Parser and doc generator cooperate on generating stub documentation for
undocumented members. The parser makes up an ArgSection with an empty
description, and the doc generator makes up a description.

Right now, the made-up ArgSections go into doc.args. However, the new
doc generator uses .all_sections, not .args. So put them into
.all_sections, too.

Insert them right after existing 'member' sections. If there are none,
insert directly after the leading section.

Doesn't affect the old generator, because that one doesn't use
.all_sections.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-60-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message rewritten]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/source: allow multi-line QAPISourceInfo advancing

This is for the sake of the new rST generator (the "transmogrifier") so
we can advance multiple lines on occasion while keeping the
generated<-

qapi/source: allow multi-line QAPISourceInfo advancing

This is for the sake of the new rST generator (the "transmogrifier") so
we can advance multiple lines on occasion while keeping the
generated<-->source mappings accurate.

next_line now simply takes an optional n parameter which chooses the
number of lines to advance.

The next patch will use this when converting section syntax in free-form
documentation to more traditional rST section header syntax, which does
not always line up 1:1 for line counts.

For example:

```
##
# = Section <-- Info is pointing here, "L1"
#
# Lorem Ipsum
##
```

would be transformed to rST as:

```
======= <-- L1
Section <-- L1
======= <-- L1
<-- L2
Lorem Ipsum <-- L3
```

After consuming the single "Section" line from the source, we want to
advance the source pointer to the next non-empty line which requires
jumping by more than one line.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250311034303.75779-42-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/schema: add __repr__ to QAPIDoc.Section

Makes debugging far more pleasant when you can just print(section) and
get something reasonable to display.

Signed-off-by: John Snow <jsnow@redhat.com>

qapi/schema: add __repr__ to QAPIDoc.Section

Makes debugging far more pleasant when you can just print(section) and
get something reasonable to display.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-35-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: clean up encoding of section kinds

We have several kinds of sections, and to tell them apart, we use
Section attribute @tag and also the section object's Python type:

type

qapi: clean up encoding of section kinds

We have several kinds of sections, and to tell them apart, we use
Section attribute @tag and also the section object's Python type:

type @tag
untagged Section None
@foo: ArgSection 'foo'
Returns: Section 'Returns'
Errors: Section 'Errors'
Since: Section 'Since'
TODO: Section 'TODO'

Note:

* @foo can be a member or a feature description, depending on context.

* tag == 'Since' can be a Since: section or a member or feature
description. If it's a Section, it's the former, and if it's an
ArgSection, it's the latter.

Clean this up as follows. Move the member or feature name to new
ArgSection attribute @name, and replace @tag by enum @kind like this:

type kind name
untagged Section PLAIN
@foo: ArgSection MEMBER 'foo' if member or argument
ArgSection FEATURE 'foo' if feature
Returns: Section RETURNS
Errors: Section ERRORS
Since: Section SINCE
TODO: Section TODO

The qapi-schema tests are updated to account for the new section names;
"TODO" becomes "Todo" and `None` becomes "Plain" there.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-34-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: adjust info location for doc body section

Instead of using the info object for the doc block as a whole (which
always points to the very first line of the block), update the info
pointe

qapi/parser: adjust info location for doc body section

Instead of using the info object for the doc block as a whole (which
always points to the very first line of the block), update the info
pointer for each call to ensure_untagged_section when the existing
section is otherwise empty. This way, Sphinx error information will
match precisely to where the text actually starts.

For example, this patch will move the info pointer for the "Hello!"
untagged section ...

> ## <-- from here ...
> # Hello! <-- ... to here.
> ##

This doesn't seem to improve error reporting now. It will with the
forthcoming QAPI doc transmogrifier.

If I stick bad rST into qapi/block-core.json like this:

> ##
> # @SnapshotInfo:
> #
> +# rST syntax error: *ahh!
> +#
> # @id: unique shapshot id
> #
> # @name: user chosen name

The existing code's error message will point to the beginning of the doc
comment, which is less than helpful. The transmogrifier's message will
point to the erroneous line, but to accomplish this, it needs this
patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-33-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>