qapi: Smarter camel_to_upper() to reduce need for 'prefix'

camel_to_upper() converts its argument from camel case to upper case
with '_' between words. Used for generated enumeration constant
prefi

qapi: Smarter camel_to_upper() to reduce need for 'prefix'

camel_to_upper() converts its argument from camel case to upper case
with '_' between words. Used for generated enumeration constant
prefixes.

When some of the words are spelled all caps, where exactly to insert
'_' is guesswork. camel_to_upper()'s guesses are bad enough in places
to make people override them with a 'prefix' in the schema.

Rewrite it to guess better:

1. Insert '_' after a non-upper case character followed by an upper
case character:

OneTwo -> ONE_TWO
One2Three -> ONE2_THREE

2. Insert '_' before the last upper case character followed by a
non-upper case character:

ACRONYMWord -> ACRONYM_Word

Except at the beginning (as in OneTwo above), or when there is
already one:

AbCd -> AB_CD

This changes the default enumeration constant prefix for a number of
enums. Generated enumeration constants change only where the default
is not overridden with 'prefix'.

The following enumerations without a 'prefix' change:

enum old camel_to_upper()
new camel_to_upper()
------------------------------------------------------------------
DisplayGLMode DISPLAYGL_MODE
DISPLAY_GL_MODE
EbpfProgramID EBPF_PROGRAMID
EBPF_PROGRAM_ID
HmatLBDataType HMATLB_DATA_TYPE
HMAT_LB_DATA_TYPE
HmatLBMemoryHierarchy HMATLB_MEMORY_HIERARCHY
HMAT_LB_MEMORY_HIERARCHY
MultiFDCompression MULTIFD_COMPRESSION
MULTI_FD_COMPRESSION
OffAutoPCIBAR OFF_AUTOPCIBAR
OFF_AUTO_PCIBAR
QCryptoBlockFormat Q_CRYPTO_BLOCK_FORMAT
QCRYPTO_BLOCK_FORMAT
QCryptoBlockLUKSKeyslotState Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE
QCRYPTO_BLOCK_LUKS_KEYSLOT_STATE
QKeyCode Q_KEY_CODE
QKEY_CODE
XDbgBlockGraphNodeType X_DBG_BLOCK_GRAPH_NODE_TYPE
XDBG_BLOCK_GRAPH_NODE_TYPE
TestUnionEnumA TEST_UNION_ENUMA
TEST_UNION_ENUM_A

Add a 'prefix' so generated code doesn't change now. Subsequent
commits will remove most of them again. Two will remain:
MULTIFD_COMPRESSION, because migration code generally spells "multifd"
that way, and Q_KEY_CODE, because that one is baked into
subprojects/keycodemapdb/tools/keymap-gen.

The following enumerations with a 'prefix' change so that the prefix
is now superfluous:

enum old camel_to_upper()
new camel_to_upper() [equal to prefix]
------------------------------------------------------------------
BlkdebugIOType BLKDEBUGIO_TYPE
BLKDEBUG_IO_TYPE
QCryptoTLSCredsEndpoint Q_CRYPTOTLS_CREDS_ENDPOINT
QCRYPTO_TLS_CREDS_ENDPOINT
QCryptoSecretFormat Q_CRYPTO_SECRET_FORMAT
QCRYPTO_SECRET_FORMAT
QCryptoCipherMode Q_CRYPTO_CIPHER_MODE
QCRYPTO_CIPHER_MODE
QCryptodevBackendType Q_CRYPTODEV_BACKEND_TYPE
QCRYPTODEV_BACKEND_TYPE
QType [builtin] Q_TYPE
QTYPE

Drop these prefixes.

The following enumerations with a 'prefix' change without making the
'prefix' superfluous:

enum old camel_to_upper()
new camel_to_upper() [equal to prefix]
prefix
------------------------------------------------------------------
CpuS390Entitlement CPUS390_ENTITLEMENT
CPU_S390_ENTITLEMENT
S390_CPU_ENTITLEMENT
CpuS390Polarization CPUS390_POLARIZATION
CPU_S390_POLARIZATION
S390_CPU_POLARIZATION
CpuS390State CPUS390_STATE
CPU_S390_STATE
S390_CPU_STATE
QAuthZListFormat Q_AUTHZ_LIST_FORMAT
QAUTH_Z_LIST_FORMAT
QAUTHZ_LIST_FORMAT
QAuthZListPolicy Q_AUTHZ_LIST_POLICY
QAUTH_Z_LIST_POLICY
QAUTHZ_LIST_POLICY
QCryptoAkCipherAlgorithm Q_CRYPTO_AK_CIPHER_ALGORITHM
QCRYPTO_AK_CIPHER_ALGORITHM
QCRYPTO_AKCIPHER_ALG
QCryptoAkCipherKeyType Q_CRYPTO_AK_CIPHER_KEY_TYPE
QCRYPTO_AK_CIPHER_KEY_TYPE
QCRYPTO_AKCIPHER_KEY_TYPE
QCryptoCipherAlgorithm Q_CRYPTO_CIPHER_ALGORITHM
QCRYPTO_CIPHER_ALGORITHM
QCRYPTO_CIPHER_ALG
QCryptoHashAlgorithm Q_CRYPTO_HASH_ALGORITHM
QCRYPTO_HASH_ALGORITHM
QCRYPTO_HASH_ALG
QCryptoIVGenAlgorithm Q_CRYPTOIV_GEN_ALGORITHM
QCRYPTO_IV_GEN_ALGORITHM
QCRYPTO_IVGEN_ALG
QCryptoRSAPaddingAlgorithm Q_CRYPTORSA_PADDING_ALGORITHM
QCRYPTO_RSA_PADDING_ALGORITHM
QCRYPTO_RSA_PADDING_ALG
QCryptodevBackendAlgType Q_CRYPTODEV_BACKEND_ALG_TYPE
QCRYPTODEV_BACKEND_ALG_TYPE
QCRYPTODEV_BACKEND_ALG
QCryptodevBackendServiceType Q_CRYPTODEV_BACKEND_SERVICE_TYPE
QCRYPTODEV_BACKEND_SERVICE_TYPE
QCRYPTODEV_BACKEND_SERVICE

Subsequent commits will tweak things to remove most of these prefixes.
Only QAUTHZ_LIST_FORMAT and QAUTHZ_LIST_POLICY will remain.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240904111836.3273842-2-armbru@redhat.com>

show more ...

qapi: remove "Example" doc section

Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Upd

qapi: remove "Example" doc section

Fully eliminate the "Example" sections in QAPI doc blocks now that they
have all been converted to arbitrary rST syntax using the
".. qmp-example::" directive. Update tests to match.

Migrating to the new syntax
---------------------------

The old "Example:" or "Examples:" section syntax is now caught as an
error, but "Example::" is stil permitted as explicit rST syntax for an
un-lexed, generic preformatted text block.

('Example' is not special in this case, any sentence that ends with "::"
will start an indented code block in rST.)

Arbitrary rST for Examples is now possible, but it's strongly
recommended that documentation authors use the ".. qmp-example::"
directive for consistent visual formatting in rendered HTML docs. The
":title:" directive option may be used to add extra information into the
title bar for the example. The ":annotated:" option can be used to write
arbitrary rST instead, with nested "::" blocks applying QMP formatting
where desired.

Other choices available are ".. code-block:: QMP" which will not create
an "Example:" box, or the short-form "::" code-block syntax which will
not apply QMP highlighting when used outside of the qmp-example
directive.

Why?
----

This patch has several benefits:

1. Example sections can now be written more arbitrarily, mixing
explanatory paragraphs and code blocks however desired.

2. Example sections can now use fully arbitrary rST.

3. All code blocks are now lexed and validated as QMP; increasing
usability of the docs and ensuring validity of example snippets.

(To some extent - This patch only gaurantees it lexes correctly, not
that it's valid under the JSON or QMP grammars. It will catch most
small mistakes, however.)

4. Each qmp-example can be titled or annotated independently without
bypassing the QMP lexer/validator.

(i.e. code blocks are now for *code* only, so we don't have to
sacrifice exposition for having lexically valid examples.)

NOTE: As with the "Notes" conversion (d461c279737), this patch (and the
three preceding) may change the rendering order for Examples in
the current generator. The forthcoming qapidoc rewrite will fix
this by always generating documentation in source order.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20240717021312.606116-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: don't parse rST markup as section headers

The double-colon synax is rST formatting that precedes a literal code
block. We do not want to capture these as QAPI-specific sections.

Coerce

qapi/parser: don't parse rST markup as section headers

The double-colon synax is rST formatting that precedes a literal code
block. We do not want to capture these as QAPI-specific sections.

Coerce blocks that start with e.g. "Example::" to be parsed as untagged
paragraphs instead of special tagged sections.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20240626222128.406106-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Indentation tweaked for consistency]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: convert "Note" sections to plain rST

We do not need a dedicated section for notes. By eliminating a specially
parsed section, these notes can be treated as normal rST paragraphs in
the new QMP

qapi: convert "Note" sections to plain rST

We do not need a dedicated section for notes. By eliminating a specially
parsed section, these notes can be treated as normal rST paragraphs in
the new QMP reference manual, and can be placed and styled much more
flexibly.

Convert all existing "Note" and "Notes" sections to pure rST. As part of
the conversion, capitalize the first letter of each sentence and add
trailing punctuation where appropriate to ensure notes look sensible and
consistent in rendered HTML documentation. Markup is also re-aligned to
the de-facto standard of 3 spaces for directives.

Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and
update the QAPI parser to prohibit "Note" sections while suggesting a
new syntax. The exact formatting to use is a matter of taste, but a good
candidate is simply:

.. note:: lorem ipsum ...
... dolor sit amet ...
... consectetur adipiscing elit ...

... but there are other choices, too. The Sphinx readthedocs theme
offers theming for the following forms (capitalization unimportant); all
are adorned with a (!) symbol () in the title bar for rendered HTML
docs.

See
https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions
for examples of each directive/admonition in use.

These are rendered in orange:

.. Attention:: ...
.. Caution:: ...
.. WARNING:: ...

These are rendered in red:

.. DANGER:: ...
.. Error:: ...

These are rendered in green:

.. Hint:: ...
.. Important:: ...
.. Tip:: ...

These are rendered in blue:

.. Note:: ...
.. admonition:: custom title

admonition body text

This patch uses ".. note::" almost everywhere, with just two "caution"
directives. Several instances of "Notes:" have been converted to
merely ".. note::", or multiple ".. note::" where appropriate.
".. admonition:: notes" is used in a few places where we had an
ordered list of multiple notes that would not make sense as
standalone/separate admonitions. Two "Note:" following "Example:"
have been turned into ordinary paragraphs within the example.

NOTE: Because qapidoc.py does not attempt to preserve source ordering of
sections, the conversion of Notes from a "tagged section" to an
"untagged section" means that rendering order for some notes *may
change* as a result of this patch. The forthcoming qapidoc.py rewrite
strictly preserves source ordering in the rendered documentation, so
this issue will be rectified in the new generator.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com> [for block*.json]
Message-ID: <20240626222128.406106-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message clarified slightly, period added to one more note]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: fix comment parsing immediately following a doc block

If a comment immediately follows a doc block, the parser doesn't ignore
that token appropriately. Fix that.

e.g.

> ##
> # = Hello

qapi/parser: fix comment parsing immediately following a doc block

If a comment immediately follows a doc block, the parser doesn't ignore
that token appropriately. Fix that.

e.g.

> ##
> # = Hello World!
> ##
>
> # I'm a comment!

will break the parser, because it does not properly ignore the comment
token if it immediately follows a doc block.

Fixes: 3d035cd2cca6 (qapi: Rewrite doc comment parser)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-7-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/parser: preserve indentation in QAPIDoc sections

Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that

qapi/parser: preserve indentation in QAPIDoc sections

Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that
removes indentation the same way get_doc_indented() did.

This is being done for the benefit of a new qapidoc generator which
requires that indentation in argument and features sections are
preserved.

Prior to this patch, a section like this:

```
@name: lorem ipsum
dolor sit amet
consectetur adipiscing elit
```

would have its body text be parsed into:

```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```

We want to preserve the indentation for even the first body line so that
the entire block can be parsed directly as rST. This patch would now
parse that segment into:

```
lorem ipsum
dolor sit amet
consectetur adipiscing elit
```

This is helpful for formatting arguments and features as field lists in
rST, where the new generator will format this information as:

```
:arg type name: lorem ipsum
dolor sit amet
consectetur apidiscing elit
```

...and can be formed by the simple concatenation of the field list
construct and the body text. The indents help preserve the continuation
of a block-level element, and further allow the use of additional rST
block-level constructs such as code blocks, lists, and other such
markup.

This understandably breaks the existing qapidoc.py; so a new function is
added there to dedent the text for compatibility. Once the new generator
is merged, this function will not be needed any longer and can be
dropped.

I verified this patch changes absolutely nothing by comparing the
md5sums of the QMP ref html pages both before and after the change, so
it's certified inert. QAPI test output has been updated to reflect the
new strategy of preserving indents for rST.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-6-jsnow@redhat.com>
[Lost commit message paragraph restored]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: linter fixups

Fix minor irritants to pylint/flake8 et al.

(Yes, these need to be guarded by the Python tests. That's a work in
progress, a series that's quite likely to follow once I finish t

qapi: linter fixups

Fix minor irritants to pylint/flake8 et al.

(Yes, these need to be guarded by the Python tests. That's a work in
progress, a series that's quite likely to follow once I finish this
Sphinx project. Please pardon the temporary irritation.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-3-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Simplify QAPISchemaVariants @tag_member

For union types, the tag member is known only after .check().

We used to code this in a simple way: QAPISchemaVariants attribute
.tag_member was None f

qapi: Simplify QAPISchemaVariants @tag_member

For union types, the tag member is known only after .check().

We used to code this in a simple way: QAPISchemaVariants attribute
.tag_member was None for union types until .check().

Since this complicated typing, recent commit "qapi/schema: fix typing
for QAPISchemaVariants.tag_member" hid it behind a property.

The previous commit lets us treat .tag_member just like the other
attributes that become known only in .check(): declare, but don't
initialize it in .__init__().

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Move conditional code from QAPISchemaVariants to its subtypes

QAPISchemaVariants.check()'s code is almost entirely conditional on
union vs. alternate type.

Move the conditional code to QAPISc

qapi: Move conditional code from QAPISchemaVariants to its subtypes

QAPISchemaVariants.check()'s code is almost entirely conditional on
union vs. alternate type.

Move the conditional code to QAPISchemaBranches.check() and
QAPISchemaAlternatives.check(), where the conditions are always
satisfied.

Attribute QAPISchemaVariants.tag_name is now only used by
QAPISchemaBranches. Move it there.

Refactor the three types' .__init__() to make them a bit simpler.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename QAPISchemaAlternateType.variants to .alternatives

A previous commit narrowed the type of
QAPISchemaAlternateType.variants from QAPISchemaVariants to
QAPISchemaAlternatives. Rename it t

qapi: Rename QAPISchemaAlternateType.variants to .alternatives

A previous commit narrowed the type of
QAPISchemaAlternateType.variants from QAPISchemaVariants to
QAPISchemaAlternatives. Rename it to .alternatives.

Same for .__init__() parameter @variants.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename QAPISchemaObjectType.variants to .branches

A previous commit narrowed the type of QAPISchemaObjectType.variants
from QAPISchemaVariants to QAPISchemaBranches. Rename it to
.branches.

qapi: Rename QAPISchemaObjectType.variants to .branches

A previous commit narrowed the type of QAPISchemaObjectType.variants
from QAPISchemaVariants to QAPISchemaBranches. Rename it to
.branches.

Same for .__init__() parameter @variants.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename visitor parameter @variants to @alternatives

A previous commit narrowed the type of .visit_alternate_type()
parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives.
Rename

qapi: Rename visitor parameter @variants to @alternatives

A previous commit narrowed the type of .visit_alternate_type()
parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives.
Rename it to @alternatives.

One of them passes @alternatives to helper function
gen_visit_alternate(). Rename its @variants parameter to
@alternatives as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename visitor parameter @variants to @branches

The previous commit narrowed the type of .visit_object_type()
parameter @variants from QAPISchemaVariants to QAPISchemaBranches.
Rename it to @b

qapi: Rename visitor parameter @variants to @branches

The previous commit narrowed the type of .visit_object_type()
parameter @variants from QAPISchemaVariants to QAPISchemaBranches.
Rename it to @branches.

Same for .visit_object_type_flat().

A few of these pass @branches to helper functions:
QAPISchemaGenRSTVisitor.visit_object_type() to ._nodes_for_members()
and ._nodes_for_variant_when(), and
QAPISchemaGenVisitVisitor.visit_object_type() to
gen_visit_object_members(). Rename the helpers' @variants parameters
to @branches as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: New QAPISchemaBranches, QAPISchemaAlternatives

QAPISchemaVariants represents either a union type's branches, or an
alternate type's alternatives. Much of its code is conditional on
which one

qapi: New QAPISchemaBranches, QAPISchemaAlternatives

QAPISchemaVariants represents either a union type's branches, or an
alternate type's alternatives. Much of its code is conditional on
which one it actually is.

Create QAPISchemaBranches for branches, and QAPISchemaAlternatives for
alternatives, both subtypes of QAPISchemaVariants.

Replace QAPISchemaVariants by one of them where possible. Keep it
only where we actually deal with either of them.

QAPISchemaVariants.__init__() takes @tag_name and @tag_member, where
exactly one must be None: @tag_name for alternatives, @tag_member for
branches. Let QAPISchemaBranches.__init__() take just @tag_name, and
QAPISchemaAlternatives.__init__() take just @tag_member.

A later patch will move the conditional code to the subtypes.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

Merge tag 'pull-qapi-2024-04-24' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2024-04-24

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFA

Merge tag 'pull-qapi-2024-04-24' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2024-04-24

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmYovX4SHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZT0RQP/1R1oOSdfWmiSR5gdW+IDE88VrjNY+Md
# lZJ7dTtbIDbwABP6s7/wGIxlc4bf6lwIwhWNnfOa+y71jJ/u9aX2Q6D8wHY5lQnu
# b8jhzoP3UyB+LQV5TTX1nTqTaO72Bj0pxw/0/DeKCzg7kgjys06a1Ln9P0oyWs6x
# li/Cs133wPtZ5rISqif5yOmssber0h2D584k5MN2VK7eaGidLioQQIRmSDikPE6Z
# TpnOEqySInIFhPJmm77il19ZDpCrgdCoD7lXoqX1C6XScYz7dU+m/TaToFk1lECw
# VstR2SJT39TzbOLdis1O5/vsLP0QfciMRQbUktSrf4jQHumrkSa/OE6xKwJC3x82
# axJWc+BygcosylKc5CYVzwSlHugHw6Lf39qui//yzi5CXakzO6owKX7Q8AcH3PPy
# 6thncy0dvw1ggq/BZGYhjyG+6MXBCWipPGVXFp9Gf2cAayTALhwyNtPvDNX57fzT
# UZA/fa+/Wc9xZv1YAnxLaKyo4o65YWXVnq1eHXV17ny08BzZNYOC2ZuXim+rMThM
# lxzcrTkrMLgsGXetp59uhZw9JRnVvaxLqNXfC2bwpoRzXzuifvnPithl6nkSm1YB
# TvHGZZdO3B498kOW6947KrMVFh3t4aNWkdbOyetAMECy71H3Q4CTHYGFa5TzXT9O
# Rjz31NYdVNBE
# =uyfg
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 24 Apr 2024 01:06:22 AM PDT
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [undefined]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2024-04-24' of https://repo.or.cz/qemu/armbru: (25 commits)
qapi: Dumb down QAPISchema.lookup_entity()
qapi: Tighten check whether implicit object type already exists
qapi/schema: remove unnecessary asserts
qapi/schema: turn on mypy strictness
qapi/schema: add type hints
qapi/parser.py: assert member.info is present in connect_member
qapi/parser: demote QAPIExpression to Dict[str, Any]
qapi/schema: assert inner type of QAPISchemaVariants in check_clash()
qapi/schema: fix typing for QAPISchemaVariants.tag_member
qapi/schema: Don't initialize "members" with `None`
qapi/schema: add _check_complete flag
qapi/schema: assert info is present when necessary
qapi/schema: fix QAPISchemaArrayType.check's call to resolve_type
qapi: Assert built-in types exist
qapi/schema: assert resolve_type has 'info' and 'what' args on error
qapi/schema: add type narrowing to lookup_type()
qapi/schema: adjust type narrowing for mypy's benefit
qapi/schema: make c_type() and json_type() abstract methods
qapi/schema: declare type for QAPISchemaArrayType.element_type
qapi/schema: declare type for QAPISchemaObjectTypeMember.type
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

qapi: Dumb down QAPISchema.lookup_entity()

QAPISchema.lookup_entity() takes an optional type argument, a subtype
of QAPISchemaDefinition, and returns that type or None. Callers can
use this to save

qapi: Dumb down QAPISchema.lookup_entity()

QAPISchema.lookup_entity() takes an optional type argument, a subtype
of QAPISchemaDefinition, and returns that type or None. Callers can
use this to save themselves an isinstance() test.

The only remaining user of this convenience feature is .lookup_type().
But we don't actually save anything anymore there: we still need the
isinstance() to help mypy over the hump.

Drop the .lookup_entity() argument, and adjust .lookup_type().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-26-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
[Commit message typo fixed]

show more ...

qapi: Tighten check whether implicit object type already exists

Entities with names starting with q_obj_ are implicit object types.
Therefore, QAPISchema._make_implicit_object_type()'s .lookup_entit

qapi: Tighten check whether implicit object type already exists

Entities with names starting with q_obj_ are implicit object types.
Therefore, QAPISchema._make_implicit_object_type()'s .lookup_entity()
can only return a QAPISchemaObjectType. Assert that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-25-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>

show more ...

qapi/schema: remove unnecessary asserts

With strict typing enabled, these runtime statements aren't necessary
anymore; we can prove them statically.

Signed-off-by: John Snow <jsnow@redhat.com>
Revi

qapi/schema: remove unnecessary asserts

With strict typing enabled, these runtime statements aren't necessary
anymore; we can prove them statically.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-24-armbru@redhat.com>

show more ...

qapi/schema: turn on mypy strictness

This patch can be rolled in with the previous one once the series is
ready for merge, but for work-in-progress' sake, it's separate here.

Signed-off-by: John Sn

qapi/schema: turn on mypy strictness

This patch can be rolled in with the previous one once the series is
ready for merge, but for work-in-progress' sake, it's separate here.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-23-armbru@redhat.com>

show more ...

qapi/schema: add type hints

This patch only adds type hints, which aren't utilized at runtime and
don't change the behavior of this module in any way.

In a scant few locations, type hints are remov

qapi/schema: add type hints

This patch only adds type hints, which aren't utilized at runtime and
don't change the behavior of this module in any way.

In a scant few locations, type hints are removed where no longer
necessary due to inference power from typing all of the rest of
creation; and any type hints that no longer need string quotes are
changed.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-22-armbru@redhat.com>

show more ...

qapi/parser.py: assert member.info is present in connect_member

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbr

qapi/parser.py: assert member.info is present in connect_member

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-21-armbru@redhat.com>

show more ...

qapi/parser: demote QAPIExpression to Dict[str, Any]

Dict[str, object] is a stricter type, but with the way that code is
currently arranged, it is infeasible to enforce this strictness.

In particul

qapi/parser: demote QAPIExpression to Dict[str, Any]

Dict[str, object] is a stricter type, but with the way that code is
currently arranged, it is infeasible to enforce this strictness.

In particular, although expr.py's entire raison d'être is normalization
and type-checking of QAPI Expressions, that type information is not
"remembered" in any meaningful way by mypy because each individual
expression is not downcast to a specific expression type that holds all
the details of each expression's unique form.

As a result, all of the code in schema.py that deals with actually
creating type-safe specialized structures has no guarantee (myopically)
that the data it is being passed is correct.

There are two ways to solve this:

(1) Re-assert that the incoming data is in the shape we expect it to be, or
(2) Disable type checking for this data.

(1) is appealing to my sense of strictness, but I gotta concede that it
is asinine to re-check the shape of a QAPIExpression in schema.py when
expr.py has just completed that work at length. The duplication of code
and the nightmare thought of needing to update both locations if and
when we change the shape of these structures makes me extremely
reluctant to go down this route.

(2) allows us the chance to miss updating types in the case that types
are updated in expr.py, but it *is* an awful lot simpler and,
importantly, gets us closer to type checking schema.py *at
all*. Something is better than nothing, I'd argue.

So, do the simpler dumber thing and worry about future strictness
improvements later.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-20-armbru@redhat.com>

show more ...

qapi/schema: assert inner type of QAPISchemaVariants in check_clash()

QAPISchemaVariant's "variants" field is typed as
List[QAPISchemaVariant], where the typing for QAPISchemaVariant allows
its type

qapi/schema: assert inner type of QAPISchemaVariants in check_clash()

QAPISchemaVariant's "variants" field is typed as
List[QAPISchemaVariant], where the typing for QAPISchemaVariant allows
its type field to be any QAPISchemaType.

However, QAPISchemaVariant expects that all of its variants contain the
narrower QAPISchemaObjectType. This relationship is enforced at runtime
in QAPISchemaVariants.check(). This relationship is not embedded in the
type system though, so QAPISchemaVariants.check_clash() needs to
re-assert this property in order to call
QAPISchemaVariant.type.check_clash().

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-19-armbru@redhat.com>

show more ...

qapi/schema: fix typing for QAPISchemaVariants.tag_member

There are two related changes here:

(1) We need to perform type narrowing for resolving the type of
tag_member during check(), and

(2)

qapi/schema: fix typing for QAPISchemaVariants.tag_member

There are two related changes here:

(1) We need to perform type narrowing for resolving the type of
tag_member during check(), and

(2) tag_member is a delayed initialization field, but we can hide it
behind a property that raises an Exception if it's called too
early. This simplifies the typing in quite a few places and avoids
needing to assert that the "tag_member is not None" at a dozen
callsites, which can be confusing and suggest the wrong thing to a
drive-by contributor.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-18-armbru@redhat.com>

show more ...

qapi/schema: Don't initialize "members" with `None`

Declare, but don't initialize the "members" field with type
List[QAPISchemaObjectTypeMember].

This simplifies the typing from what would otherwis

qapi/schema: Don't initialize "members" with `None`

Declare, but don't initialize the "members" field with type
List[QAPISchemaObjectTypeMember].

This simplifies the typing from what would otherwise be
Optional[List[T]] to merely List[T]. This removes the need to add
assertions to several callsites that this value is not None - which it
never will be after the delayed initialization in check() anyway.

The type declaration without initialization trick will cause accidental
uses of this field prior to full initialization to raise an
AttributeError.

(Note that it is valid to have an empty members list, see the internal
q_empty object as an example. For this reason, we cannot use the empty
list as a replacement test for full initialization and instead rely on
the _checked/_check_complete fields.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-17-armbru@redhat.com>

show more ...