docs/sphinx/qmp_lexer: Highlight elisions like comments, not prompts

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-8-armbru@redhat.com>
Reviewed-by: Eric B

docs/sphinx/qmp_lexer: Highlight elisions like comments, not prompts

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

docs/sphinx/qmp_lexer: Generalize elision syntax

Accept "... lorem ipsum ..." in addition to "...".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-7-armbr

docs/sphinx/qmp_lexer: Generalize elision syntax

Accept "... lorem ipsum ..." in addition to "...".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250404121413.1743790-7-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

docs/qapi-domain: add namespaced index support

Generate an index-per-namespace for the QAPI domain. Due to a limitation
with Sphinx's architecture, these indices must be defined during setup
time an

docs/qapi-domain: add namespaced index support

Generate an index-per-namespace for the QAPI domain. Due to a limitation
with Sphinx's architecture, these indices must be defined during setup
time and cannot be dynamically created on-demand when a namespace
directive is encountered.

Owing to that limitation, add a configuration value to conf.py that
specifies which QAPI namespaces we'll generate indices for.

Indices will be named after their namespace, e.g. the "QMP" namespace
will generate to "qapi-qmp-index.html" and can be referenced using
`qapi-qmp-index`.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-9-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi_domain: add namespace support to cross-references

This patch does three things:

1. Record the current namespace context in pending_xrefs so it can be
used for link resolution later,
2.

docs/qapi_domain: add namespace support to cross-references

This patch does three things:

1. Record the current namespace context in pending_xrefs so it can be
used for link resolution later,
2. Pass that recorded namespace context to find_obj() when resolving a
reference, and
3. Wildly and completely rewrite find_obj().

cross-reference support is expanded to tolerate the presence or absence
of either namespace or module, and to cope with the presence or absence
of contextual information for either.

References now work like this:

1. If the explicit reference target is recorded in the domain's object
registry, we link to that target and stop looking. We do this lookup
regardless of how fully qualified the target is, which allows direct
references to modules (which don't have a module component to their
names) or direct references to definitions that may or may not belong
to a namespace or module.

2. If contextual information is available from qapi:namespace or
qapi:module directives, try using those components to find a direct
match to the implied target name.

3. If both prior lookups fail, generate a series of regular expressions
looking for wildcard matches in order from most to least
specific. Any explicitly provided components (namespace, module)
*must* match exactly, but both contextual and entirely omitted
components are allowed to differ from the search result. Note that if
more than one result is found, Sphinx will emit a warning (a build
error for QEMU) and list all of the candidate references.

The practical upshot is that in the large majority of cases, namespace
and module information is not required when creating simple `references`
to definitions from within the same context -- even when identical
definitions exist in other contexts.

Even when using simple `references` from elsewhere in the QEMU
documentation manual, explicit namespace info is not required if there
is only one definition by that name.

Disambiguation *will* be required from outside of the QAPI documentation
when referencing e.g. block-core definitions, which are shared between
QEMU QMP and the QEMU Storage Daemon. In that case, there are two
options:

A: References can be made partially or fully explicit,
e.g. `QMP:block-dirty-bitmap-add` will link to the QEMU version of
the definition, while `QSD:block-dirty-bitmap-add` would link to the
QSD version.

B: If all of the references in a document are intended to go to the same
place, you can insert a "qapi:namespace:: QMP" directive to influence
the fuzzy-searching for later references.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-8-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Commit message typo fixed]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add :namespace: option to qapi-doc directive

Add a :namespace: option to the qapi-doc directive, which inserts a
qapi:namespace directive into the start of the generated document. This

docs/qapidoc: add :namespace: option to qapi-doc directive

Add a :namespace: option to the qapi-doc directive, which inserts a
qapi:namespace directive into the start of the generated document. This,
in turn, associates all auto-generated definitions by this directive
with the specified namespace.

The source info for these generated lines are credited to the start of
the qapi-doc directive, which isn't precisely correct, but I wasn't sure
how to get it more accurate without some re-parsing shenanigans.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-7-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: add qapi:namespace directive

Add a new directive that marks the beginning of a QAPI "namespace", for
example; "QMP", "QGA" or "QSD". This directive will associate all
subsequent QA

docs/qapi-domain: add qapi:namespace directive

Add a new directive that marks the beginning of a QAPI "namespace", for
example; "QMP", "QGA" or "QSD". This directive will associate all
subsequent QAPI directives in a document with the specified
namespace. This does not change the visual display of any of the
definitions or index entries, but does change the "Fully Qualified Name"
inside the QAPI domain's object table. This allows for two different
"namespaces" to define entities with otherwise identical names -- which
will come in handy for documenting both QEMU QMP and the QEMU Storage
Daemon.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-6-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: add :namespace: override option

Akin to the :module: override option, the :namespace: options allows you
to forcibly override the contextual namespace associatied with a
definition

docs/qapi-domain: add :namespace: override option

Akin to the :module: override option, the :namespace: options allows you
to forcibly override the contextual namespace associatied with a
definition.

We don't necessarily actually need this, but I felt compelled to stick
close to how the Python domain works that offers context overrides.

As of this commit, it is possible to add e.g. ":namespace: QMP" to any
QAPI directive to forcibly associate that definition with a given
namespace.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-5-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi_domain: add namespace support to FQN

This patch adds a namespace component to the "Fully Qualified Name", in
the form of "domain:module.name". As there are no namespace directives
or optio

docs/qapi_domain: add namespace support to FQN

This patch adds a namespace component to the "Fully Qualified Name", in
the form of "domain:module.name". As there are no namespace directives
or options yet, this component will simply be empty as of this patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-4-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi-domain: always store fully qualified name in signode

Currently, only the definition name is stored in the tree metadata; but
the node property is confusingly called "fullname". Rectify thi

docs/qapi-domain: always store fully qualified name in signode

Currently, only the definition name is stored in the tree metadata; but
the node property is confusingly called "fullname". Rectify this by
always storing the FQN in the tree metadata.

... While we're here, re-organize the code in preparation for namespace
support to make it a bit easier to add additional components of the
FQN. With this change, there is now extremely little code left that's
taken directly from the Python domain :)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-3-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapi_domain: isolate TYPE_CHECKING imports

When using the annotations feature, type hints do not need to be
imported at runtime, only at type check time. Move type-check-only
imports into a con

docs/qapi_domain: isolate TYPE_CHECKING imports

When using the annotations feature, type hints do not need to be
imported at runtime, only at type check time. Move type-check-only
imports into a conditional to reduce the number of imports needed at
runtime.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250313044312.189276-2-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: generate entries for undocumented members

Presently, we never have any empty text entries for members. The next
patch will explicitly generate such sections, so enable support for it
i

docs/qapidoc: generate entries for undocumented members

Presently, we never have any empty text entries for members. The next
patch will explicitly generate such sections, so enable support for it
in advance.

The parser will generate placeholder sections to indicate undocumented
members, but it's the qapidoc generator that's responsible for deciding
what to do with that stub section.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-59-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Tweak the stub section text]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: Add "the members of" pointers

Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over th

docs/qapidoc: Add "the members of" pointers

Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over the old system :)

This patch is meant to be a temporary solution until we can review and
merge the inliner.

The implementation of this patch is a little bit of a hack: Sphinx is
not designed to allow you to mix fields of different "type"; i.e. mixing
member descriptions and free-form text under the same heading. To
accomplish this with a minimum of hackery, we technically document a
"dummy field" and then just strip off the documentation for that dummy
field in a post-processing step. We use the "q_dummy" variable for this
purpose, then strip it back out before final processing. If this
processing step should fail, you'll see warnings for a bad
cross-reference. (So if you don't see any, it must be working!)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-58-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add intermediate output debugger

Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated

docs/qapidoc: add intermediate output debugger

Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated rst file to disk to allow for easy debugging and verification
of the generated document.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-57-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: process @foo into ``foo``

Add support for the special QAPI doc syntax to process @references as
``preformatted text``. At the moment, there are no actual
cross-references for individua

docs/qapidoc: process @foo into ``foo``

Add support for the special QAPI doc syntax to process @references as
``preformatted text``. At the moment, there are no actual
cross-references for individual members, so there is nothing to link
against. For now, process it identically to how we did in the old
qapidoc system.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-56-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: implement transmogrify() method

This is the true top-level processor for the new transmogrifier;
responsible both for generating the intermediate rST and then running
the nested parse

docs/qapidoc: implement transmogrify() method

This is the true top-level processor for the new transmogrifier;
responsible both for generating the intermediate rST and then running
the nested parse on that generated document to produce the final
docutils tree that is then - very finally - postprocessed by sphinx for
final rendering to HTML &c.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-55-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Use the opportunity to move the __version__ assignment to where
PEP 8 wants it]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_entity()

Finally, the core entry method for a qapi entity.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-54-jsnow@redhat.com>
Acked-by: Marku

docs/qapidoc: add visit_entity()

Finally, the core entry method for a qapi entity.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-54-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_sections() method

Implement the actual main dispatch method that processes and handles the
list of doc sections for a given QAPI entity.

Process doc sections in strict sourc

docs/qapidoc: add visit_sections() method

Implement the actual main dispatch method that processes and handles the
list of doc sections for a given QAPI entity.

Process doc sections in strict source order. This is good; reordering
doc text is undesirable. Improvement over the old doc generator, which
can reorder doc comments that don't adhere to (largely unspoken)
conventions.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-53-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Commit message extended]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_member() method

This method is used for generating the "members" of a wide variety of
things, including structs, unions, enums, alternates, etc. The field
name it uses to do

docs/qapidoc: add visit_member() method

This method is used for generating the "members" of a wide variety of
things, including structs, unions, enums, alternates, etc. The field
name it uses to do so is dependent on the type of entity the "member"
belongs to.

Currently, IF conditionals for individual members are not handled or
rendered, a small regression from the prior documentation
generator. This will be fixed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-52-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_returns() method

Generates :return: fields for explicit returns statements. Note that
this does not presently handle undocumented returns, which is handled in
a later commit.

docs/qapidoc: add visit_returns() method

Generates :return: fields for explicit returns statements. Note that
this does not presently handle undocumented returns, which is handled in
a later commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-51-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: prepare to record entity being transmogrified

Prepare to keep a record of which entity we're working on documenting
for the purposes of being able to change certain generative features

docs/qapidoc: prepare to record entity being transmogrified

Prepare to keep a record of which entity we're working on documenting
for the purposes of being able to change certain generative features
conditionally and create stronger assertions.

If you find yourself asking: "Wait, but where does the current entity
actually get recorded?!", you're right! That part comes with the
visit_entity() implementation, which gets added later.

This patch is front-loaded for the sake of type checking in the
forthcoming commits before visit_entity() is ready to be added.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-50-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_feature() method

This adds a simple ":feat name: lorem ipsum ..." line to the generated
rST document, so at the moment it's only for "top level" features.

Features not attac

docs/qapidoc: add visit_feature() method

This adds a simple ":feat name: lorem ipsum ..." line to the generated
rST document, so at the moment it's only for "top level" features.

Features not attached directly to a QAPI definition are not currently
handled! This is a small regression over the prior documentation
generator that will be addressed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-49-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add add_field() and generate_field() helper methods

These are simple rST generation methods that assist in getting the types
and formatting correct for a field list entry. add_field()

docs/qapidoc: add add_field() and generate_field() helper methods

These are simple rST generation methods that assist in getting the types
and formatting correct for a field list entry. add_field() is a more
raw, direct call while generate_field() is intended to be used for
generating the correct field from a member object.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-48-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add format_type() method

This method is responsible for generating a type name for a given member
with the correct annotations for the QAPI domain. Features and enums do
not *have* typ

docs/qapidoc: add format_type() method

This method is responsible for generating a type name for a given member
with the correct annotations for the QAPI domain. Features and enums do
not *have* types, so they return None. Everything else returns the type
name with a "?" suffix if that type is optional, and ensconced in
[brackets] if it's an array type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-47-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_errors() method

Notably, this method does not currently address the formatting issues
present with the "errors" section in QAPIDoc and just vomits the text
verbatim into the

docs/qapidoc: add visit_errors() method

Notably, this method does not currently address the formatting issues
present with the "errors" section in QAPIDoc and just vomits the text
verbatim into the rST doc, with somewhat inconsistent results.

To be addressed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-46-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

docs/qapidoc: add visit_paragraph() method

This transforms "formerly known as untagged sections" into our pure
intermediate rST format. These sections are already pure rST, so this
method doesn't do

docs/qapidoc: add visit_paragraph() method

This transforms "formerly known as untagged sections" into our pure
intermediate rST format. These sections are already pure rST, so this
method doesn't do a whole lot except ensure appropriate newlines.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-45-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...