qapi/schema: fix QAPISchemaArrayType.check's call to resolve_type

Adjust the expression at the callsite to work around mypy's weak type
introspection that believes this expression can resolve to
QAP

qapi/schema: fix QAPISchemaArrayType.check's call to resolve_type

Adjust the expression at the callsite to work around mypy's weak type
introspection that believes this expression can resolve to
QAPISourceInfo; it cannot.

(Fundamentally: self.info only resolves to false in a boolean expression
when it is None; therefore this expression may only ever produce
Optional[str]. mypy does not know that 'info', when it is a
QAPISourceInfo object, cannot ever be false.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-14-armbru@redhat.com>

show more ...

qapi: Assert built-in types exist

QAPISchema.lookup_type('FOO') returns a QAPISchemaType when type 'FOO'
exists, else None. It won't return None for built-in types like
'int'.

Since mypy can't see

qapi: Assert built-in types exist

QAPISchema.lookup_type('FOO') returns a QAPISchemaType when type 'FOO'
exists, else None. It won't return None for built-in types like
'int'.

Since mypy can't see that, it'll complain that we assign the
Optional[QAPISchemaType] returned by .lookup_type() to QAPISchemaType
variables.

Add assertions to help it over the hump.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-13-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>

show more ...

qapi/schema: assert resolve_type has 'info' and 'what' args on error

resolve_type() is generally used to resolve configuration-provided type
names into type objects, and generally requires valid 'in

qapi/schema: assert resolve_type has 'info' and 'what' args on error

resolve_type() is generally used to resolve configuration-provided type
names into type objects, and generally requires valid 'info' and 'what'
parameters.

In some cases, such as with QAPISchemaArrayType.check(), resolve_type
may be used to resolve built-in types and as such will not have an
'info' argument, but also must not fail in this scenario.

Use an assertion to sate mypy that we will indeed have 'info' and 'what'
parameters for the error pathway in resolve_type.

Note: there are only three callsites to resolve_type at present where
"info" is perceived by mypy to be possibly None:

1) QAPISchemaArrayType.check()
2) QAPISchemaObjectTypeMember.check()
3) QAPISchemaEvent.check()

Of those three, only the first actually ever passes None; the other two
are limited by their base class initializers which accept info=None, but
neither subclass actually use a None value in practice, currently.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-12-armbru@redhat.com>

show more ...

qapi/schema: add type narrowing to lookup_type()

This function is a bit hard to type as-is; mypy needs some assertions to
assist with the type narrowing.

Signed-off-by: John Snow <jsnow@redhat.com>

qapi/schema: add type narrowing to lookup_type()

This function is a bit hard to type as-is; mypy needs some assertions to
assist with the type narrowing.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-11-armbru@redhat.com>

show more ...

qapi/schema: adjust type narrowing for mypy's benefit

We already take care to perform some type narrowing for arg_type and
ret_type, but not in a way where mypy can utilize the result once we add
ty

qapi/schema: adjust type narrowing for mypy's benefit

We already take care to perform some type narrowing for arg_type and
ret_type, but not in a way where mypy can utilize the result once we add
type hints, e.g.:

qapi/schema.py:833: error: Incompatible types in assignment (expression
has type "QAPISchemaType", variable has type
"Optional[QAPISchemaObjectType]") [assignment]

qapi/schema.py:893: error: Incompatible types in assignment (expression
has type "QAPISchemaType", variable has type
"Optional[QAPISchemaObjectType]") [assignment]

A simple change to use a temporary variable helps the medicine go down.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-10-armbru@redhat.com>

show more ...

qapi/schema: make c_type() and json_type() abstract methods

These methods should always return a str, it's only the default abstract
implementation that doesn't. They can be marked "abstract", which

qapi/schema: make c_type() and json_type() abstract methods

These methods should always return a str, it's only the default abstract
implementation that doesn't. They can be marked "abstract", which
requires subclasses to override the method with the proper return type.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-9-armbru@redhat.com>

show more ...

qapi/schema: declare type for QAPISchemaArrayType.element_type

A QAPISchemaArrayType's element type gets resolved only during .check().
We have QAPISchemaArrayType.__init__() initialize self.element

qapi/schema: declare type for QAPISchemaArrayType.element_type

A QAPISchemaArrayType's element type gets resolved only during .check().
We have QAPISchemaArrayType.__init__() initialize self.element_type =
None, and .check() assign the actual type. Using .element_type before
.check() is wrong, and hopefully crashes due to the value being None.
Works.

However, it makes for awkward typing. With .element_type:
Optional[QAPISchemaType], mypy is of course unable to see that it's None
before .check(), and a QAPISchemaType after. To help it over the hump,
we'd have to assert self.element_type is not None before all the (valid)
uses. The assertion catches invalid uses, but only at run time; mypy
can't flag them.

Instead, declare .element_type in .__init__() as QAPISchemaType
*without* initializing it. Using .element_type before .check() now
certainly crashes, which is an improvement. Mypy still can't flag
invalid uses, but that's okay.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-8-armbru@redhat.com>

show more ...

qapi/schema: declare type for QAPISchemaObjectTypeMember.type

A QAPISchemaObjectTypeMember's type gets resolved only during .check().
We have QAPISchemaObjectTypeMember.__init__() initialize self.ty

qapi/schema: declare type for QAPISchemaObjectTypeMember.type

A QAPISchemaObjectTypeMember's type gets resolved only during .check().
We have QAPISchemaObjectTypeMember.__init__() initialize self.type =
None, and .check() assign the actual type. Using .type before .check()
is wrong, and hopefully crashes due to the value being None. Works.

However, it makes for awkward typing. With .type:
Optional[QAPISchemaType], mypy is of course unable to see that it's None
before .check(), and a QAPISchemaType after. To help it over the hump,
we'd have to assert self.type is not None before all the (valid) uses.
The assertion catches invalid uses, but only at run time; mypy can't
flag them.

Instead, declare .type in .__init__() as QAPISchemaType *without*
initializing it. Using .type before .check() now certainly crashes,
which is an improvement. Mypy still can't flag invalid uses, but that's
okay.

Addresses typing errors such as these:

qapi/schema.py:657: error: "None" has no attribute "alternate_qtype" [attr-defined]
qapi/schema.py:662: error: "None" has no attribute "describe" [attr-defined]

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-7-armbru@redhat.com>

show more ...

qapi: create QAPISchemaDefinition

Include entities don't have names, but we generally expect "entities" to
have names. Reclassify all entities with names as *definitions*, leaving
the nameless inclu

qapi: create QAPISchemaDefinition

Include entities don't have names, but we generally expect "entities" to
have names. Reclassify all entities with names as *definitions*, leaving
the nameless include entities as QAPISchemaEntity instances.

This is primarily to help simplify typing around expectations of what
callers expect for properties of an "entity".

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-6-armbru@redhat.com>

show more ...

qapi/schema: add pylint suppressions

With this patch, pylint is happy with the file, so enable it in the
configuration.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <a

qapi/schema: add pylint suppressions

With this patch, pylint is happy with the file, so enable it in the
configuration.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-5-armbru@redhat.com>

show more ...

qapi: sort pylint suppressions

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marku

qapi: sort pylint suppressions

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-4-armbru@redhat.com>

show more ...

qapi/parser: shush up pylint

Shhh!

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <2

qapi/parser: shush up pylint

Shhh!

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-3-armbru@redhat.com>

show more ...

qapi/parser: fix typo - self.returns.info => self.errors.info

Small copy-pasto. The correct info field to use in this conditional
block is self.errors.info.

Fixes: 3a025d3d1ffa
Signed-off-by: John

qapi/parser: fix typo - self.returns.info => self.errors.info

Small copy-pasto. The correct info field to use in this conditional
block is self.errors.info.

Fixes: 3a025d3d1ffa
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240315152301.3621858-2-armbru@redhat.com>

show more ...

qapi: Inline and remove QERR_INVALID_PARAMETER_TYPE definition

Address the comment added in commit 4629ed1e98
("qerror: Finally unused, clean up"), from 2015:

/*
* These macros will go away, p

qapi: Inline and remove QERR_INVALID_PARAMETER_TYPE definition

Address the comment added in commit 4629ed1e98
("qerror: Finally unused, clean up"), from 2015:

/*
* These macros will go away, please don't use
* in new code, and do not add new ones!
*/

Manual changes (escaping the format in qapi/visit.py).

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240312141343.3168265-8-armbru@redhat.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

show more ...

qapi: Reject "Returns" section when command doesn't return anything

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-14-armbru@redhat.com>

qapi: New documentation section tag "Errors"

We use section "Returns" for documenting both success and error
response of commands.

I intend to generate better command success response documentation

qapi: New documentation section tag "Errors"

We use section "Returns" for documenting both success and error
response of commands.

I intend to generate better command success response documentation.
Easier when "Returns" documents just he success response.

Create new section tag "Errors". The next two commits will move error
response documentation from "Returns" sections to "Errors" sections.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-4-armbru@redhat.com>

show more ...

qapi: Slightly clearer error message for invalid "Returns" section

Change "'Returns:' is only valid for commands" to "'Returns' section
is only valid for commands".

Signed-off-by: Markus Armbruster

qapi: Slightly clearer error message for invalid "Returns" section

Change "'Returns:' is only valid for commands" to "'Returns' section
is only valid for commands".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-3-armbru@redhat.com>

show more ...

qapi: Memorize since & returns sections

This is chiefly to make code that looks up these sections easier to
read.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.23

qapi: Memorize since & returns sections

This is chiefly to make code that looks up these sections easier to
read.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-2-armbru@redhat.com>

show more ...

qapi: Divorce QAPIDoc from QAPIParseError

QAPIDoc stores a reference to QAPIParser just to pass it to
QAPIParseError. The resulting error position depends on the state of
the parser. It happens to

qapi: Divorce QAPIDoc from QAPIParseError

QAPIDoc stores a reference to QAPIParser just to pass it to
QAPIParseError. The resulting error position depends on the state of
the parser. It happens to be the current comment line. Servicable,
but action at a distance.

The commit before previous moved most uses of QAPIParseError from
QAPIDoc to QAPIParser. There are just three left. Convert them to
QAPISemError. This involves passing info to a few methods. Then drop
the reference to QAPIParser.

The three errors lose the column number. Not really interesting here:
it's the comment line's indentation.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-17-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qapi: Reject multiple and empty feature descriptions

The parser recognizes only the first "Features:" line. Any subsequent
ones are treated as ordinary text, as visible in test case
doc-duplicate-f

qapi: Reject multiple and empty feature descriptions

The parser recognizes only the first "Features:" line. Any subsequent
ones are treated as ordinary text, as visible in test case
doc-duplicate-features. Recognize "Features:" lines anywhere. A
second one is an error.

A 'Features:' line without any features is useless, but not an error.
Make it an error. This makes detecting a second "Features:" line
easier.

qapi/run-state.json actually has an instance of this since commit
fe17522d854 (qapi: Remove deprecated 'singlestep' member of
StatusInfo). Clean it up.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-16-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qapi: Rewrite doc comment parser

QAPISchemaParser is a conventional recursive descent parser. Except
QAPISchemaParser.get_doc() delegates most of the doc comment parsing
work to a state machine in

qapi: Rewrite doc comment parser

QAPISchemaParser is a conventional recursive descent parser. Except
QAPISchemaParser.get_doc() delegates most of the doc comment parsing
work to a state machine in QAPIDoc. The state machine doesn't get
tokens like a recursive descent parser, it is fed tokens.

I find this state machine rather opaque and hard to maintain.

Replace it by a conventional parser, all in QAPISchemaParser. Less
code, and (at least in my opinion) easier to understand.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-15-armbru@redhat.com>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qapi: Merge adjacent untagged sections

The parser mostly doesn't create adjacent untagged sections, and
merging the ones it does create is hardly worth the bother. I'm doing
it to avoid behavioral

qapi: Merge adjacent untagged sections

The parser mostly doesn't create adjacent untagged sections, and
merging the ones it does create is hardly worth the bother. I'm doing
it to avoid behavioral change in the next commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-14-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qapi: Call QAPIDoc.check() always

We currently call QAPIDoc.check() only for definition documentation.
Calling it for free-form documentation as well is simpler. No change,
because it doesn't actua

qapi: Call QAPIDoc.check() always

We currently call QAPIDoc.check() only for definition documentation.
Calling it for free-form documentation as well is simpler. No change,
because it doesn't actually do anything there.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-13-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qapi: Recognize section tags and 'Features:' only after blank line

Putting a blank line before section tags and 'Features:' is good,
existing practice. Enforce it.

Signed-off-by: Markus Armbruster

qapi: Recognize section tags and 'Features:' only after blank line

Putting a blank line before section tags and 'Features:' is good,
existing practice. Enforce it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-12-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qapi: Require descriptions and tagged sections to be indented

By convention, we indent the second and subsequent lines of
descriptions and tagged sections, except for examples.

Turn this into a har

qapi: Require descriptions and tagged sections to be indented

By convention, we indent the second and subsequent lines of
descriptions and tagged sections, except for examples.

Turn this into a hard rule, and apply it to examples, too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240216145841.2099240-11-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[Straightforward conflicts in qapi/migration.json resolved]

show more ...