tests/qapi-schema: Drop temporary 'prefix'

Recent commit "qapi: Smarter camel_to_upper() to reduce need for
'prefix'" added a temporary 'prefix' to delay changing the generated
code.

Revert it. Th

tests/qapi-schema: Drop temporary 'prefix'

Recent commit "qapi: Smarter camel_to_upper() to reduce need for
'prefix'" added a temporary 'prefix' to delay changing the generated
code.

Revert it. This changes TestUnionEnumA's generated enumeration
constant prefix from TEST_UNION_ENUMA to TEST_UNION_ENUM_A.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240904111836.3273842-3-armbru@redhat.com>

show more ...

qapi: Smarter camel_to_upper() to reduce need for 'prefix'

camel_to_upper() converts its argument from camel case to upper case
with '_' between words. Used for generated enumeration constant
prefi

qapi: Smarter camel_to_upper() to reduce need for 'prefix'

camel_to_upper() converts its argument from camel case to upper case
with '_' between words. Used for generated enumeration constant
prefixes.

When some of the words are spelled all caps, where exactly to insert
'_' is guesswork. camel_to_upper()'s guesses are bad enough in places
to make people override them with a 'prefix' in the schema.

Rewrite it to guess better:

1. Insert '_' after a non-upper case character followed by an upper
case character:

OneTwo -> ONE_TWO
One2Three -> ONE2_THREE

2. Insert '_' before the last upper case character followed by a
non-upper case character:

ACRONYMWord -> ACRONYM_Word

Except at the beginning (as in OneTwo above), or when there is
already one:

AbCd -> AB_CD

This changes the default enumeration constant prefix for a number of
enums. Generated enumeration constants change only where the default
is not overridden with 'prefix'.

The following enumerations without a 'prefix' change:

enum old camel_to_upper()
new camel_to_upper()
------------------------------------------------------------------
DisplayGLMode DISPLAYGL_MODE
DISPLAY_GL_MODE
EbpfProgramID EBPF_PROGRAMID
EBPF_PROGRAM_ID
HmatLBDataType HMATLB_DATA_TYPE
HMAT_LB_DATA_TYPE
HmatLBMemoryHierarchy HMATLB_MEMORY_HIERARCHY
HMAT_LB_MEMORY_HIERARCHY
MultiFDCompression MULTIFD_COMPRESSION
MULTI_FD_COMPRESSION
OffAutoPCIBAR OFF_AUTOPCIBAR
OFF_AUTO_PCIBAR
QCryptoBlockFormat Q_CRYPTO_BLOCK_FORMAT
QCRYPTO_BLOCK_FORMAT
QCryptoBlockLUKSKeyslotState Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE
QCRYPTO_BLOCK_LUKS_KEYSLOT_STATE
QKeyCode Q_KEY_CODE
QKEY_CODE
XDbgBlockGraphNodeType X_DBG_BLOCK_GRAPH_NODE_TYPE
XDBG_BLOCK_GRAPH_NODE_TYPE
TestUnionEnumA TEST_UNION_ENUMA
TEST_UNION_ENUM_A

Add a 'prefix' so generated code doesn't change now. Subsequent
commits will remove most of them again. Two will remain:
MULTIFD_COMPRESSION, because migration code generally spells "multifd"
that way, and Q_KEY_CODE, because that one is baked into
subprojects/keycodemapdb/tools/keymap-gen.

The following enumerations with a 'prefix' change so that the prefix
is now superfluous:

enum old camel_to_upper()
new camel_to_upper() [equal to prefix]
------------------------------------------------------------------
BlkdebugIOType BLKDEBUGIO_TYPE
BLKDEBUG_IO_TYPE
QCryptoTLSCredsEndpoint Q_CRYPTOTLS_CREDS_ENDPOINT
QCRYPTO_TLS_CREDS_ENDPOINT
QCryptoSecretFormat Q_CRYPTO_SECRET_FORMAT
QCRYPTO_SECRET_FORMAT
QCryptoCipherMode Q_CRYPTO_CIPHER_MODE
QCRYPTO_CIPHER_MODE
QCryptodevBackendType Q_CRYPTODEV_BACKEND_TYPE
QCRYPTODEV_BACKEND_TYPE
QType [builtin] Q_TYPE
QTYPE

Drop these prefixes.

The following enumerations with a 'prefix' change without making the
'prefix' superfluous:

enum old camel_to_upper()
new camel_to_upper() [equal to prefix]
prefix
------------------------------------------------------------------
CpuS390Entitlement CPUS390_ENTITLEMENT
CPU_S390_ENTITLEMENT
S390_CPU_ENTITLEMENT
CpuS390Polarization CPUS390_POLARIZATION
CPU_S390_POLARIZATION
S390_CPU_POLARIZATION
CpuS390State CPUS390_STATE
CPU_S390_STATE
S390_CPU_STATE
QAuthZListFormat Q_AUTHZ_LIST_FORMAT
QAUTH_Z_LIST_FORMAT
QAUTHZ_LIST_FORMAT
QAuthZListPolicy Q_AUTHZ_LIST_POLICY
QAUTH_Z_LIST_POLICY
QAUTHZ_LIST_POLICY
QCryptoAkCipherAlgorithm Q_CRYPTO_AK_CIPHER_ALGORITHM
QCRYPTO_AK_CIPHER_ALGORITHM
QCRYPTO_AKCIPHER_ALG
QCryptoAkCipherKeyType Q_CRYPTO_AK_CIPHER_KEY_TYPE
QCRYPTO_AK_CIPHER_KEY_TYPE
QCRYPTO_AKCIPHER_KEY_TYPE
QCryptoCipherAlgorithm Q_CRYPTO_CIPHER_ALGORITHM
QCRYPTO_CIPHER_ALGORITHM
QCRYPTO_CIPHER_ALG
QCryptoHashAlgorithm Q_CRYPTO_HASH_ALGORITHM
QCRYPTO_HASH_ALGORITHM
QCRYPTO_HASH_ALG
QCryptoIVGenAlgorithm Q_CRYPTOIV_GEN_ALGORITHM
QCRYPTO_IV_GEN_ALGORITHM
QCRYPTO_IVGEN_ALG
QCryptoRSAPaddingAlgorithm Q_CRYPTORSA_PADDING_ALGORITHM
QCRYPTO_RSA_PADDING_ALGORITHM
QCRYPTO_RSA_PADDING_ALG
QCryptodevBackendAlgType Q_CRYPTODEV_BACKEND_ALG_TYPE
QCRYPTODEV_BACKEND_ALG_TYPE
QCRYPTODEV_BACKEND_ALG
QCryptodevBackendServiceType Q_CRYPTODEV_BACKEND_SERVICE_TYPE
QCRYPTODEV_BACKEND_SERVICE_TYPE
QCRYPTODEV_BACKEND_SERVICE

Subsequent commits will tweak things to remove most of these prefixes.
Only QAUTHZ_LIST_FORMAT and QAUTHZ_LIST_POLICY will remain.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240904111836.3273842-2-armbru@redhat.com>

show more ...

qapi: remove "Example" doc section

Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Upd

qapi: remove "Example" doc section

Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.

Migrating to the new syntax
---------------------------

The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.

('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)

Arbitrary rST for Examples is now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.

Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.

Why?
----

This patch has several benefits:

1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.

2. Example sections can now use fully arbitrary rST.

3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.

(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)

4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.

(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)

NOTE: As with the "Notes" conversion (d461c279737), this patch (and the
three preceding) may change the rendering order for Examples in
the current generator. The forthcoming qapidoc rewrite will fix
this by always generating documentation in source order.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20240717021312.606116-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: don't parse rST markup as section headers

The double-colon synax is rST formatting that precedes a literal code
block. We do not want to capture these as QAPI-specific sections.

Coerce

qapi/parser: don't parse rST markup as section headers

The double-colon synax is rST formatting that precedes a literal code
block. We do not want to capture these as QAPI-specific sections.

Coerce blocks that start with e.g. "Example::" to be parsed as untagged
paragraphs instead of special tagged sections.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20240626222128.406106-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Indentation tweaked for consistency]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: convert "Note" sections to plain rST

We do not need a dedicated section for notes. By eliminating a specially
parsed section, these notes can be treated as normal rST paragraphs in
the new QMP

qapi: convert "Note" sections to plain rST

We do not need a dedicated section for notes. By eliminating a specially
parsed section, these notes can be treated as normal rST paragraphs in
the new QMP reference manual, and can be placed and styled much more
flexibly.

Convert all existing "Note" and "Notes" sections to pure rST. As part of
the conversion, capitalize the first letter of each sentence and add
trailing punctuation where appropriate to ensure notes look sensible and
consistent in rendered HTML documentation. Markup is also re-aligned to
the de-facto standard of 3 spaces for directives.

Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and
update the QAPI parser to prohibit "Note" sections while suggesting a
new syntax. The exact formatting to use is a matter of taste, but a good
candidate is simply:

.. note:: lorem ipsum ...
... dolor sit amet ...
... consectetur adipiscing elit ...

... but there are other choices, too. The Sphinx readthedocs theme
offers theming for the following forms (capitalization unimportant); all
are adorned with a (!) symbol () in the title bar for rendered HTML
docs.

See
https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions
for examples of each directive/admonition in use.

These are rendered in orange:

.. Attention:: ...
.. Caution:: ...
.. WARNING:: ...

These are rendered in red:

.. DANGER:: ...
.. Error:: ...

These are rendered in green:

.. Hint:: ...
.. Important:: ...
.. Tip:: ...

These are rendered in blue:

.. Note:: ...
.. admonition:: custom title

admonition body text

This patch uses ".. note::" almost everywhere, with just two "caution"
directives. Several instances of "Notes:" have been converted to
merely ".. note::", or multiple ".. note::" where appropriate.
".. admonition:: notes" is used in a few places where we had an
ordered list of multiple notes that would not make sense as
standalone/separate admonitions. Two "Note:" following "Example:"
have been turned into ordinary paragraphs within the example.

NOTE: Because qapidoc.py does not attempt to preserve source ordering of
sections, the conversion of Notes from a "tagged section" to an
"untagged section" means that rendering order for some notes *may
change* as a result of this patch. The forthcoming qapidoc.py rewrite
strictly preserves source ordering in the rendered documentation, so
this issue will be rectified in the new generator.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com> [for block*.json]
Message-ID: <20240626222128.406106-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message clarified slightly, period added to one more note]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: fix comment parsing immediately following a doc block

If a comment immediately follows a doc block, the parser doesn't ignore
that token appropriately. Fix that.

e.g.

> ##
> # = Hello

qapi/parser: fix comment parsing immediately following a doc block

If a comment immediately follows a doc block, the parser doesn't ignore
that token appropriately. Fix that.

e.g.

> ##
> # = Hello World!
> ##
>
> # I'm a comment!

will break the parser, because it does not properly ignore the comment
token if it immediately follows a doc block.

Fixes: 3d035cd2cca6 (qapi: Rewrite doc comment parser)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-7-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: preserve indentation in QAPIDoc sections

Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that

qapi/parser: preserve indentation in QAPIDoc sections

Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that
removes indentation the same way get_doc_indented() did.

This is being done for the benefit of a new qapidoc generator which
requires that indentation in argument and features sections are
preserved.

Prior to this patch, a section like this:

```
@name: lorem ipsum
dolor sit amet
consectetur adipiscing elit
```

would have its body text be parsed into:

```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```

We want to preserve the indentation for even the first body line so that
the entire block can be parsed directly as rST. This patch would now
parse that segment into:

```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```

This is helpful for formatting arguments and features as field lists in
rST, where the new generator will format this information as:

```
:arg type name: lorem ipsum
dolor sit amet
consectetur apidiscing elit
```

...and can be formed by the simple concatenation of the field list
construct and the body text. The indents help preserve the continuation
of a block-level element, and further allow the use of additional rST
block-level constructs such as code blocks, lists, and other such
markup.

This understandably breaks the existing qapidoc.py; so a new function is
added there to dedent the text for compatibility. Once the new generator
is merged, this function will not be needed any longer and can be
dropped.

I verified this patch changes absolutely nothing by comparing the
md5sums of the QMP ref html pages both before and after the change, so
it's certified inert. QAPI test output has been updated to reflect the
new strategy of preserving indents for rST.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-6-jsnow@redhat.com>
[Lost commit message paragraph restored]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename visitor parameter @variants to @alternatives

A previous commit narrowed the type of .visit_alternate_type()
parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives.
Rename

qapi: Rename visitor parameter @variants to @alternatives

A previous commit narrowed the type of .visit_alternate_type()
parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives.
Rename it to @alternatives.

One of them passes @alternatives to helper function
gen_visit_alternate(). Rename its @variants parameter to
@alternatives as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename visitor parameter @variants to @branches

The previous commit narrowed the type of .visit_object_type()
parameter @variants from QAPISchemaVariants to QAPISchemaBranches.
Rename it to @b

qapi: Rename visitor parameter @variants to @branches

The previous commit narrowed the type of .visit_object_type()
parameter @variants from QAPISchemaVariants to QAPISchemaBranches.
Rename it to @branches.

Same for .visit_object_type_flat().

A few of these pass @branches to helper functions:
QAPISchemaGenRSTVisitor.visit_object_type() to ._nodes_for_members()
and ._nodes_for_variant_when(), and
QAPISchemaGenVisitVisitor.visit_object_type() to
gen_visit_object_members(). Rename the helpers' @variants parameters
to @branches as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Reject "Returns" section when command doesn't return anything

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-14-armbru@redhat.com>

qapi: New documentation section tag "Errors"

We use section "Returns" for documenting both success and error
response of commands.

I intend to generate better command success response documentation

qapi: New documentation section tag "Errors"

We use section "Returns" for documenting both success and error
response of commands.

I intend to generate better command success response documentation.
Easier when "Returns" documents just he success response.

Create new section tag "Errors". The next two commits will move error
response documentation from "Returns" sections to "Errors" sections.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-4-armbru@redhat.com>

show more ...

qapi: Slightly clearer error message for invalid "Returns" section

Change "'Returns:' is only valid for commands" to "'Returns' section
is only valid for commands".

Signed-off-by: Markus Armbruster

qapi: Slightly clearer error message for invalid "Returns" section

Change "'Returns:' is only valid for commands" to "'Returns' section
is only valid for commands".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-3-armbru@redhat.com>

show more ...