Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-10-22-v3' into staging

QAPI patches for 2019-10-22

# gpg: Signature made Tue 22 Oct 2019 15:56:36 BST
# gpg: using RS

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-10-22-v3' into staging

QAPI patches for 2019-10-22

# gpg: Signature made Tue 22 Oct 2019 15:56:36 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-10-22-v3:
qapi: Allow introspecting fix for savevm's cooperation with blockdev
tests/qapi-schema: Cover feature documentation comments
tests: qapi: Test 'features' of commands
qapi: Add feature flags to commands
tests/qapi-schema: Tidy up test output indentation
qapi: Clear scripts/qapi/doc.py executable bits again
qapi: Split up scripts/qapi/common.py
qapi: Move gen_enum(), gen_enum_lookup() back to qapi/types.py
qapi: Speed up frontend tests
qapi: Eliminate accidental global frontend state
qapi: Store pragma state in QAPISourceInfo, not global state
qapi: Don't suppress doc generation without pragma doc-required

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Speed up frontend tests

"make check-qapi-schema" takes around 10s user + system time for me.
With -j, it takes a bit over 3s real time. We have worse tests. It's
still annoying when you work

qapi: Speed up frontend tests

"make check-qapi-schema" takes around 10s user + system time for me.
With -j, it takes a bit over 3s real time. We have worse tests. It's
still annoying when you work on the QAPI generator.

Some 1.4s user + system time is consumed by make figuring out what to
do, measured by making a target that does nothing. There's nothing I
can do about that right now. But let's see what we can do about the
other 8s.

Almost 7s are spent running test-qapi.py for every test case, the rest
normalizing and diffing test-qapi.py output. We have 190 test cases.

If I downgrade to python2, it's 4.5s, but python2 is a goner.

Hacking up test-qapi.py to exit(0) without doing anything makes it
only marginally faster. The problem is Python startup overhead.

Our configure puts -B into $(PYTHON). Running without -B is faster:
4.4s.

We could improve the Makefile to run test cases only when the test
case or the generator changed. But I'm after improvement in the case
where the generator changed.

test-qapi.py is designed to be the simplest possible building block
for a shell script to do the complete job (it's actually a Makefile,
not a shell script; no real difference). Python is just not meant for
that. It's for bigger blocks.

Move the post-processing and diffing into test-qapi.py, and make it
capable of testing multiple schema files. Set executable bits while
there.

Running it once per test case now takes slightly longer than 8s. But
running it once for all of them takes under 0.2s.

Messing with the Makefile to run it only on the tests that need
retesting is clearly not worth the bother.

Expected error output changes because the new normalization strips off
$(SRCDIR)/tests/qapi-schema/ instead of just $(SRCDIR)/.

The .exit files go away, because there is no exit status to test
anymore.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-5-armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-28' into staging

QAPI patches for 2019-09-28

# gpg: Signature made Sat 28 Sep 2019 16:18:13 BST
# gpg: using RSA k

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-28' into staging

QAPI patches for 2019-09-28

# gpg: Signature made Sat 28 Sep 2019 16:18:13 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-09-28: (27 commits)
qapi: Improve source file read error handling
qapi: Improve reporting of redefinition
qapi: Improve reporting of missing documentation comment
qapi: Eliminate check_keys(), rename check_known_keys()
qapi: Improve reporting of invalid 'if' further
qapi: Avoid redundant definition references in error messages
qapi: Improve reporting of missing / unknown definition keys
qapi: Improve reporting of invalid flags
qapi: Improve reporting of invalid 'if' errors
qapi: Move context-free checking to the proper place
qapi: Move context-sensitive checking to the proper place
qapi: Inline check_name() into check_union()
qapi: Plumb info to the QAPISchemaMember
qapi: Make check_type()'s array case a bit more obvious
qapi: Move check for reserved names out of add_name()
qapi: Report invalid '*' prefix like any other invalid name
qapi: Use check_name_str() where it suffices
qapi: Improve reporting of invalid name errors
qapi: Reorder check_FOO() parameters for consistency
qapi: Improve reporting of member name clashes
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Change frontend error messages to start with lower case

Starting error messages with a capital letter complicates things when
text can get interpolated both at the beginning and in the middle

qapi: Change frontend error messages to start with lower case

Starting error messages with a capital letter complicates things when
text can get interpolated both at the beginning and in the middle of
an error message. The next patch will do that. Switch to lower case
to keep it simpler.

For what it's worth, the GNU Coding Standards advise the message
"should not begin with a capital letter when it follows a program name
and/or file name, because that isn’t the beginning of a sentence. (The
sentence conceptually starts at the beginning of the line.)"

While there, avoid breaking lines containing multiple arguments in the
middle of an argument.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-7-armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-24' into staging

QAPI patches for 2019-09-24

# gpg: Signature made Tue 24 Sep 2019 13:10:36 BST
# gpg: using RSA k

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-24' into staging

QAPI patches for 2019-09-24

# gpg: Signature made Tue 24 Sep 2019 13:10:36 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-09-24: (37 commits)
qapi: Assert .visit() and .check_clash() run only after .check()
qapi: Fix excessive QAPISchemaEntity.check() recursion
qapi: Fix to .check() empty structs just once
qapi: Delete useless check_exprs() code for simple union kind
qapi: Clean up around check_known_keys()
qapi: Simplify check_keys()
qapi: Normalize 'if' in check_exprs(), like other sugar
qapi: Fix missing 'if' checks in struct, union, alternate 'data'
qapi: Reject blank 'if' conditions in addition to empty ones
qapi: Fix broken discriminator error messages
qapi: Remove null from schema language
qapi: Improve reporting of lexical errors
qapi: Use quotes more consistently in frontend error messages
tests/qapi-schema: Demonstrate suboptimal lexical errors
tests/qapi-schema: Demonstrate insufficient 'if' checking
tests/qapi-schema: Demonstrate broken discriminator errors
tests/qapi-schema: Demonstrate misleading optional tag error
tests/qapi-schema: Delete two redundant tests
tests/qapi-schema: Cover unknown pragma
qapi: Tweak code to match docs/devel/qapi-code-gen.txt
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Use quotes more consistently in frontend error messages

Consistently enclose error messages in double quotes. Use single
quotes within, except for one case of "'".

Signed-off-by: Markus Armb

qapi: Use quotes more consistently in frontend error messages

Consistently enclose error messages in double quotes. Use single
quotes within, except for one case of "'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2015-10-12' into staging

QAPI patches

# gpg: Signature made Mon 12 Oct 2015 18:56:35 BST using RSA key ID EB918653
# gpg: Good signature

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2015-10-12' into staging

QAPI patches

# gpg: Signature made Mon 12 Oct 2015 18:56:35 BST using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>"

* remotes/armbru/tags/pull-qapi-2015-10-12:
qapi: Simplify gen_visit_fields() error handling
qapi: Share gen_visit_fields()
qapi: Share gen_err_check()
qapi: Consistent generated code: minimize push_indent() usage
qapi: Consistent generated code: prefer common indentation
qapi: Consistent generated code: prefer common labels
qapi: Consistent generated code: prefer visitor 'v'
qapi: Consistent generated code: prefer error 'err'
qapi: Reuse code for flat union base validation
qapi: Test use of 'number' within alternates
qapi: Add tests for empty unions
qapi: Avoid assertion failure on union 'type' collision
qapi: Test for various name collisions
qapi: Clean up qapi.py per pep8
qapi: Invoke exception superclass initializer
qapi: Improve 'include' error message
qapi: Sort qapi-schema tests
MAINTAINERS: Specify QAPI include and test files
MAINTAINERS: Specify QObject include and test files
docs: Move files from docs/qmp/ to docs/

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Test for various name collisions

Expose some weaknesses in the generator: we don't always forbid
the generation of structs that contain multiple members that map
to the same C or QMP name. Th

qapi: Test for various name collisions

Expose some weaknesses in the generator: we don't always forbid
the generation of structs that contain multiple members that map
to the same C or QMP name. This has already been marked FIXME in
qapi.py in commit d90675f, but having more tests will make sure
future patches produce desired behavior; and updating existing
patches to better document things doesn't hurt, either. Some of
these collisions are already caught in the old-style parser
checks, but ultimately we want all collisions to be caught in the
new-style QAPISchema*.check() methods.

This patch focuses on C struct members, and does not consider
collisions between commands and events (affecting C function
names), or even collisions between generated C type names with
user type names (for things like automatic FOOList struct
representing array types or FOOKind for an implicit enum).

There are two types of struct collisions we want to catch:
1) Collision between two keys in a JSON object. qapi.py prevents
that within a single struct (see test duplicate-key), but it is
possible to have collisions between a type's members and its
base type's members (existing tests struct-base-clash,
struct-base-clash-deep), and its flat union variant members
(renamed test flat-union-clash-member).
2) Collision between two members of the C struct that is generated
for a given QAPI type:
a) Multiple QAPI names map to the same C name (new test
args-name-clash)
b) A QAPI name maps to a C name that is used for another purpose
(new tests flat-union-clash-branch, struct-base-clash-base,
union-clash-data). We already fixed some such cases in commit
0f61af3e and 1e6c1616, but more remain.
c) Two C names generated for other purposes clash
(updated test alternate-clash, new test union-clash-branches,
union-clash-type, flat-union-clash-type)

Ultimately, if we need to have a flat union where a tag value
clashes with a base member name, we could change the generator to
name the union (using 'foo.u.value' rather than 'foo.value') or
otherwise munge the C name corresponding to tag values. But
unless such a need arises, it will probably be easier to just
forbid these collisions.

Some of these negative tests will be deleted later, and positive
tests added to qapi-schema-test.json in their place, when the
generator code is reworked to avoid particular code generation
collisions in class 2).

[Note that viewing this patch with git rename detection enabled
may see some confusion due to renaming some tests while adding
others, but where the content is similar enough that git picks
the wrong pre- and post-patch files to associate]

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-6-git-send-email-eblake@redhat.com>
[Improve commit message and comments a bit, drop an unrelated test]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/qmp-unstable/queue/qmp' into staging

* remotes/qmp-unstable/queue/qmp: (38 commits)
Revert "qapi: Clean up superfluous null check in qapi_dealloc_type_str()"

Merge remote-tracking branch 'remotes/qmp-unstable/queue/qmp' into staging

* remotes/qmp-unstable/queue/qmp: (38 commits)
Revert "qapi: Clean up superfluous null check in qapi_dealloc_type_str()"
qapi: Document optional arguments' backwards compatibility
qmp: use valid JSON in transaction example
qmp: Don't use error_is_set() to suppress additional errors
dump: Drop pointless error_is_set(), DumpState member errp
qemu-option: Clean up fragile use of error_is_set()
qga: Drop superfluous error_is_set()
qga: Clean up fragile use of error_is_set()
qapi: Clean up fragile use of error_is_set()
tests/qapi-schema: Drop superfluous error_is_set()
qapi: Drop redundant, unclean error_is_set()
hmp: Guard against misuse of hmp_handle_error()
qga: Use return values instead of error_is_set(errp)
error: Consistently name Error ** objects errp, and not err
qmp: Consistently name Error ** objects errp, and not err
qga: Consistently name Error ** objects errp, and not err
qmp hmp: Consistently name Error * objects err, and not errp
pci-assign: assigned_initfn(): set monitor error in common error handler
pci-assign: propagate errors from assign_intx()
pci-assign: propagate errors from assign_device()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Use an explicit input file

Use an explicit input file on the command-line instead of reading from standard
input.

It also outputs the proper file name when there's an error.

Signed-off-by: L

qapi: Use an explicit input file

Use an explicit input file on the command-line instead of reading from standard
input.

It also outputs the proper file name when there's an error.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/qmp-unstable/queue/qmp' into staging

* remotes/qmp-unstable/queue/qmp:
tests: test-qmp-commands: Fix double free
qapi script: do not add "_" for every capit

Merge remote-tracking branch 'remotes/qmp-unstable/queue/qmp' into staging

* remotes/qmp-unstable/queue/qmp:
tests: test-qmp-commands: Fix double free
qapi script: do not add "_" for every capitalized char in enum
qapi script: do not allow string discriminator
qapi: convert BlockdevOptions to use enum discriminator
qapi script: support enum type as discriminator in union
qapi script: use same function to generate enum string
qapi script: code move for generate_enum_name()
qapi script: check correctness of union
qapi script: remember line number in schema parsing
qapi script: add check for duplicated key
qapi script: remember explicitly defined enum values

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi script: add check for duplicated key

It is bad that same key was specified twice, especially when a union has
two branches with same condition. This patch can prevent it.

Signed-off-by: Wencha

qapi script: add check for duplicated key

It is bad that same key was specified twice, especially when a union has
two branches with same condition. This patch can prevent it.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>

show more ...