qapi: Fix undocumented return values by generating something

Generated command documentation lacks information on return value in
several cases, e.g. query-tpm.

The obvious fix would be to require

qapi: Fix undocumented return values by generating something

Generated command documentation lacks information on return value in
several cases, e.g. query-tpm.

The obvious fix would be to require a Returns: section when a command
returns something.

However, note that many existing Returns: sections are pretty useless:
the description is basically the return type, which then gets rendered
like "Return: <Type> – <basically the return type>". This suggests
that a description is often not really necessary, and requiring one
isn't useful.

Instead, generate the obvious minimal thing when Returns: is absent:
"Return: <Type>".

This auto-generated Return documentation is placed is as follows:

1. If we have arguments, return goes right after them.
2. Else if we have errors, return goes right before them.
3. Else if we have features, return goes right before them.
4. Else return goes right after the intro

To facilitate this algorithm, a "TODO:" hack line is used to separate
the intro from the remainder of the documentation block in cases where
there are no other sections to separate the intro from e.g. examples and
additional detail meant to appear below the key sections of interest.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250711051045.51110-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[_insert_near_kind() code replaced by something simpler, commit
message amended to explain why we're doing this]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: add return-nodesc

This form is used to annotate a return type without an accompanying
description, for when there is no "Returns:" information in the source
doc, but we have a retu

docs/qapi-domain: add return-nodesc

This form is used to annotate a return type without an accompanying
description, for when there is no "Returns:" information in the source
doc, but we have a return type we want to generate a cross-reference to.

The syntax is:

:return-nodesc: TypeName

It's primarily necessary because Sphinx always expects both a type and a
description for the prior form and will format it accordingly. To have a
reasonable rendering when the body is missing, we need to use a
different info field list entirely.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250711051045.51110-2-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Long line wrapped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/sphinx: remove special parsing for freeform sections

Remove the QAPI doc section heading syntax, use plain rST section
headings instead.

Tests and documentation are updated to match.

Interest

docs/sphinx: remove special parsing for freeform sections

Remove the QAPI doc section heading syntax, use plain rST section
headings instead.

Tests and documentation are updated to match.

Interestingly, Plain rST headings work fine before this patch, except
for over- and underlining with '=', which the doc parser rejected as
invalid QAPI doc section heading in free-form comments.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Add more detail to commit message]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/sphinx: remove legacy QAPI manual generator

Thanks for your service!

Remove the old qapidoc and the option to enable the transmogrifier,
leaving the "transmogrifier" as the ONLY qapi doc gener

docs/sphinx: remove legacy QAPI manual generator

Thanks for your service!

Remove the old qapidoc and the option to enable the transmogrifier,
leaving the "transmogrifier" as the ONLY qapi doc generator. This in
effect also converts the QAPI test to use the new documentation
generator, too.

Update doc-good.txt output to match the new doc generator, which I
should've done exactly when we switched over to the transmogrifier, but,
uhh, oops!

Notes on the new format:
1. per-member IFCOND documentation is missing. Known issue.
2. Freeform documentation without a header is now copied through into
the output. This is a bug fix.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-4-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Fixes: b61a4eb3f32 (docs/qapidoc: support header-less freeform sections)
[Tweak commit message to say it's a bug fix, add Fixes:]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/sphinx: parse @references in freeform text

Oversight in the new qapidoc transmogrifier: @references in freeform
documentation blocks were not being transformed to literals. This fixes
that, and

docs/sphinx: parse @references in freeform text

Oversight in the new qapidoc transmogrifier: @references in freeform
documentation blocks were not being transformed to literals. This fixes
that, and the next patch ensures that we're testing for this O:-)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-3-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/sphinx: adjust qapidoc to cope with same-line error sections

Without this, the line the new QAPI doc generator chokes on

# Errors: some

in doc-good.json. We still use the old doc generat

docs/sphinx: adjust qapidoc to cope with same-line error sections

Without this, the line the new QAPI doc generator chokes on

# Errors: some

in doc-good.json. We still use the old doc generator for the tests,
but we're about to correct that.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250618165353.1980365-2-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Fixes: e9fbf1a0c6c2 (docs/qapidoc: add visit_errors() method)
[Amend commit message to point to reproducer, and add Fixes:]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: linting fixes

This restores the linting baseline in qapidoc. The order of some imports
change slightly here due to configuring isort a little better:
previously, isort was having diffi

docs/qapidoc: linting fixes

This restores the linting baseline in qapidoc. The order of some imports
change slightly here due to configuring isort a little better:
previously, isort was having difficulty understanding that "compat" and
"qapidoc_legacy" were local modules because docs/sphinx "isn't a python
package". Configuring this manually, isort chooses a different import
ordering, which _is_ intentional here.

Also: extra ignores are added for pylint. The most recent versions of
pylint don't require these ignores, but the oldest versions we support
do, so in the extra ignores go.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20250604200354.459501-3-jsnow@redhat.com

show more ...

docs/sphinx/qmp_lexer: Highlight elisions like comments, not prompts

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-8-armbru@redhat.com>
Reviewed-by: Eric B

docs/sphinx/qmp_lexer: Highlight elisions like comments, not prompts

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

docs/sphinx/qmp_lexer: Generalize elision syntax

Accept "... lorem ipsum ..." in addition to "...".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-7-armbr

docs/sphinx/qmp_lexer: Generalize elision syntax

Accept "... lorem ipsum ..." in addition to "...".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-7-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

docs/qapi-domain: add namespaced index support

Generate an index-per-namespace for the QAPI domain. Due to a limitation
with Sphinx's architecture, these indices must be defined during setup
time an

docs/qapi-domain: add namespaced index support

Generate an index-per-namespace for the QAPI domain. Due to a limitation
with Sphinx's architecture, these indices must be defined during setup
time and cannot be dynamically created on-demand when a namespace
directive is encountered.

Owing to that limitation, add a configuration value to conf.py that
specifies which QAPI namespaces we'll generate indices for.

Indices will be named after their namespace, e.g. the "QMP" namespace
will generate to "qapi-qmp-index.html" and can be referenced using
`qapi-qmp-index`.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-9-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi_domain: add namespace support to cross-references

This patch does three things:

1. Record the current namespace context in pending_xrefs so it can be
used for link resolution later,
2.

docs/qapi_domain: add namespace support to cross-references

This patch does three things:

1. Record the current namespace context in pending_xrefs so it can be
used for link resolution later,
2. Pass that recorded namespace context to find_obj() when resolving a
reference, and
3. Wildly and completely rewrite find_obj().

cross-reference support is expanded to tolerate the presence or absence
of either namespace or module, and to cope with the presence or absence
of contextual information for either.

References now work like this:

1. If the explicit reference target is recorded in the domain's object
registry, we link to that target and stop looking. We do this lookup
regardless of how fully qualified the target is, which allows direct
references to modules (which don't have a module component to their
names) or direct references to definitions that may or may not belong
to a namespace or module.

2. If contextual information is available from qapi:namespace or
qapi:module directives, try using those components to find a direct
match to the implied target name.

3. If both prior lookups fail, generate a series of regular expressions
looking for wildcard matches in order from most to least
specific. Any explicitly provided components (namespace, module)
*must* match exactly, but both contextual and entirely omitted
components are allowed to differ from the search result. Note that if
more than one result is found, Sphinx will emit a warning (a build
error for QEMU) and list all of the candidate references.

The practical upshot is that in the large majority of cases, namespace
and module information is not required when creating simple `references`
to definitions from within the same context -- even when identical
definitions exist in other contexts.

Even when using simple `references` from elsewhere in the QEMU
documentation manual, explicit namespace info is not required if there
is only one definition by that name.

Disambiguation *will* be required from outside of the QAPI documentation
when referencing e.g. block-core definitions, which are shared between
QEMU QMP and the QEMU Storage Daemon. In that case, there are two
options:

A: References can be made partially or fully explicit,
e.g. `QMP:block-dirty-bitmap-add` will link to the QEMU version of
the definition, while `QSD:block-dirty-bitmap-add` would link to the
QSD version.

B: If all of the references in a document are intended to go to the same
place, you can insert a "qapi:namespace:: QMP" directive to influence
the fuzzy-searching for later references.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-8-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Commit message typo fixed]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add :namespace: option to qapi-doc directive

Add a :namespace: option to the qapi-doc directive, which inserts a
qapi:namespace directive into the start of the generated document. This

docs/qapidoc: add :namespace: option to qapi-doc directive

Add a :namespace: option to the qapi-doc directive, which inserts a
qapi:namespace directive into the start of the generated document. This,
in turn, associates all auto-generated definitions by this directive
with the specified namespace.

The source info for these generated lines are credited to the start of
the qapi-doc directive, which isn't precisely correct, but I wasn't sure
how to get it more accurate without some re-parsing shenanigans.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-7-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: add qapi:namespace directive

Add a new directive that marks the beginning of a QAPI "namespace", for
example; "QMP", "QGA" or "QSD". This directive will associate all
subsequent QA

docs/qapi-domain: add qapi:namespace directive

Add a new directive that marks the beginning of a QAPI "namespace", for
example; "QMP", "QGA" or "QSD". This directive will associate all
subsequent QAPI directives in a document with the specified
namespace. This does not change the visual display of any of the
definitions or index entries, but does change the "Fully Qualified Name"
inside the QAPI domain's object table. This allows for two different
"namespaces" to define entities with otherwise identical names -- which
will come in handy for documenting both QEMU QMP and the QEMU Storage
Daemon.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-6-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: add :namespace: override option

Akin to the :module: override option, the :namespace: options allows you
to forcibly override the contextual namespace associatied with a
definition

docs/qapi-domain: add :namespace: override option

Akin to the :module: override option, the :namespace: options allows you
to forcibly override the contextual namespace associatied with a
definition.

We don't necessarily actually need this, but I felt compelled to stick
close to how the Python domain works that offers context overrides.

As of this commit, it is possible to add e.g. ":namespace: QMP" to any
QAPI directive to forcibly associate that definition with a given
namespace.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-5-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi_domain: add namespace support to FQN

This patch adds a namespace component to the "Fully Qualified Name", in
the form of "domain:module.name". As there are no namespace directives
or optio

docs/qapi_domain: add namespace support to FQN

This patch adds a namespace component to the "Fully Qualified Name", in
the form of "domain:module.name". As there are no namespace directives
or options yet, this component will simply be empty as of this patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-4-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: always store fully qualified name in signode

Currently, only the definition name is stored in the tree metadata; but
the node property is confusingly called "fullname". Rectify thi

docs/qapi-domain: always store fully qualified name in signode

Currently, only the definition name is stored in the tree metadata; but
the node property is confusingly called "fullname". Rectify this by
always storing the FQN in the tree metadata.

... While we're here, re-organize the code in preparation for namespace
support to make it a bit easier to add additional components of the
FQN. With this change, there is now extremely little code left that's
taken directly from the Python domain :)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-3-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi_domain: isolate TYPE_CHECKING imports

When using the annotations feature, type hints do not need to be
imported at runtime, only at type check time. Move type-check-only
imports into a con

docs/qapi_domain: isolate TYPE_CHECKING imports

When using the annotations feature, type hints do not need to be
imported at runtime, only at type check time. Move type-check-only
imports into a conditional to reduce the number of imports needed at
runtime.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-2-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: generate entries for undocumented members

Presently, we never have any empty text entries for members. The next
patch will explicitly generate such sections, so enable support for it
i

docs/qapidoc: generate entries for undocumented members

Presently, we never have any empty text entries for members. The next
patch will explicitly generate such sections, so enable support for it
in advance.

The parser will generate placeholder sections to indicate undocumented
members, but it's the qapidoc generator that's responsible for deciding
what to do with that stub section.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-59-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Tweak the stub section text]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: Add "the members of" pointers

Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over th

docs/qapidoc: Add "the members of" pointers

Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over the old system :)

This patch is meant to be a temporary solution until we can review and
merge the inliner.

The implementation of this patch is a little bit of a hack: Sphinx is
not designed to allow you to mix fields of different "type"; i.e. mixing
member descriptions and free-form text under the same heading. To
accomplish this with a minimum of hackery, we technically document a
"dummy field" and then just strip off the documentation for that dummy
field in a post-processing step. We use the "q_dummy" variable for this
purpose, then strip it back out before final processing. If this
processing step should fail, you'll see warnings for a bad
cross-reference. (So if you don't see any, it must be working!)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-58-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add intermediate output debugger

Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated

docs/qapidoc: add intermediate output debugger

Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated rst file to disk to allow for easy debugging and verification
of the generated document.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-57-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: process @foo into ``foo``

Add support for the special QAPI doc syntax to process @references as
``preformatted text``. At the moment, there are no actual
cross-references for individua

docs/qapidoc: process @foo into ``foo``

Add support for the special QAPI doc syntax to process @references as
``preformatted text``. At the moment, there are no actual
cross-references for individual members, so there is nothing to link
against. For now, process it identically to how we did in the old
qapidoc system.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-56-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: implement transmogrify() method

This is the true top-level processor for the new transmogrifier;
responsible both for generating the intermediate rST and then running
the nested parse

docs/qapidoc: implement transmogrify() method

This is the true top-level processor for the new transmogrifier;
responsible both for generating the intermediate rST and then running
the nested parse on that generated document to produce the final
docutils tree that is then - very finally - postprocessed by sphinx for
final rendering to HTML &c.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-55-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Use the opportunity to move the __version__ assignment to where
PEP 8 wants it]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_entity()

Finally, the core entry method for a qapi entity.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-54-jsnow@redhat.com>
Acked-by: Marku

docs/qapidoc: add visit_entity()

Finally, the core entry method for a qapi entity.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-54-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_sections() method

Implement the actual main dispatch method that processes and handles the
list of doc sections for a given QAPI entity.

Process doc sections in strict sourc

docs/qapidoc: add visit_sections() method

Implement the actual main dispatch method that processes and handles the
list of doc sections for a given QAPI entity.

Process doc sections in strict source order. This is good; reordering
doc text is undesirable. Improvement over the old doc generator, which
can reorder doc comments that don't adhere to (largely unspoken)
conventions.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-53-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Commit message extended]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_member() method

This method is used for generating the "members" of a wide variety of
things, including structs, unions, enums, alternates, etc. The field
name it uses to do

docs/qapidoc: add visit_member() method

This method is used for generating the "members" of a wide variety of
things, including structs, unions, enums, alternates, etc. The field
name it uses to do so is dependent on the type of entity the "member"
belongs to.

Currently, IF conditionals for individual members are not handled or
rendered, a small regression from the prior documentation
generator. This will be fixed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-52-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...