| 86cdf9ec | 23-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
json: Clean up headers
The JSON parser has three public headers, json-lexer.h, json-parser.h,
json-streamer.h. They all contain stuff that is of no interest
outside qobject/json-*.c.
Collect the p
json: Clean up headers
The JSON parser has three public headers, json-lexer.h, json-parser.h,
json-streamer.h. They all contain stuff that is of no interest
outside qobject/json-*.c.
Collect the public interface in include/qapi/qmp/json-parser.h, and
everything else in qobject/json-parser-int.h.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-54-armbru@redhat.com>
show more ...
|
| f9277915 | 23-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
json: Fix streamer not to ignore trailing unterminated structures
json_message_process_token() accumulates tokens until it got the
sequence of tokens that comprise a single JSON value (it counts cur
json: Fix streamer not to ignore trailing unterminated structures
json_message_process_token() accumulates tokens until it got the
sequence of tokens that comprise a single JSON value (it counts curly
braces and square brackets to decide). It feeds those token sequences
to json_parser_parse(). If a non-empty sequence of tokens remains at
the end of the parse, it's silently ignored. check-qjson.c cases
unterminated_array(), unterminated_array_comma(), unterminated_dict(),
unterminated_dict_comma() demonstrate this bug.
Fix as follows. Introduce a JSON_END_OF_INPUT token. When the
streamer receives it, it feeds the accumulated tokens to
json_parser_parse().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-46-armbru@redhat.com>
show more ...
|
| 84a56f38 | 23-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
json: Pass lexical errors and limit violations to callback
The callback to consume JSON values takes QObject *json, Error *err.
If both are null, the callback is supposed to make up an error by
itse
json: Pass lexical errors and limit violations to callback
The callback to consume JSON values takes QObject *json, Error *err.
If both are null, the callback is supposed to make up an error by
itself. This sucks.
qjson.c's consume_json() neglects to do so, which makes
qobject_from_json() null instead of failing. I consider that a bug.
The culprit is json_message_process_token(): it passes two null
pointers when it runs into a lexical error or a limit violation. Fix
it to pass a proper Error object then. Update the callbacks:
* monitor.c's handle_qmp_command(): the code to make up an error is
now dead, drop it.
* qga/main.c's process_event(): lumps the "both null" case together
with the "not a JSON object" case. The former is now gone. The
error message "Invalid JSON syntax" is misleading for the latter.
Improve it to "Input must be a JSON object".
* qobject/qjson.c's consume_json(): no update; check-qjson
demonstrates qobject_from_json() now sets an error on lexical
errors, but still doesn't on some other errors.
* tests/libqtest.c's qmp_response(): the Error object is now reliable,
so use it to improve the error message.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-40-armbru@redhat.com>
show more ...
|
| 2cbd15aa | 23-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
json: Treat unwanted interpolation as lexical error
The JSON parser optionally supports interpolation. The lexer
recognizes interpolation tokens unconditionally. The parser rejects
them when inter
json: Treat unwanted interpolation as lexical error
The JSON parser optionally supports interpolation. The lexer
recognizes interpolation tokens unconditionally. The parser rejects
them when interpolation is disabled, in parse_interpolation().
However, it neglects to set an error then, which can make
json_parser_parse() fail without setting an error.
Move the check for unwanted interpolation from the parser's
parse_interpolation() into the lexer's finite state machine. When
interpolation is disabled, '%' is now handled like any other
unexpected character.
The next commit will improve how such lexical errors are handled.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-39-armbru@redhat.com>
show more ...
|
| 61030280 | 23-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
json: Rename token JSON_ESCAPE & friends to JSON_INTERP
The JSON parser optionally supports interpolation. The code calls it
"escape". Awkward, because it uses the same term for escape sequences
w
json: Rename token JSON_ESCAPE & friends to JSON_INTERP
The JSON parser optionally supports interpolation. The code calls it
"escape". Awkward, because it uses the same term for escape sequences
within strings. The latter usage is consistent with RFC 8259 "The
JavaScript Object Notation (JSON) Data Interchange Format" and ISO C.
Call the former "interpolation" instead.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-38-armbru@redhat.com>
show more ...
|
| 62815d85 | 23-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
json: Redesign the callback to consume JSON values
The classical way to structure parser and lexer is to have the client
call the parser to get an abstract syntax tree, the parser call the
lexer to
json: Redesign the callback to consume JSON values
The classical way to structure parser and lexer is to have the client
call the parser to get an abstract syntax tree, the parser call the
lexer to get the next token, and the lexer call some function to get
input characters.
Another way to structure them would be to have the client feed
characters to the lexer, the lexer feed tokens to the parser, and the
parser feed abstract syntax trees to some callback provided by the
client. This way is more easily integrated into an event loop that
dispatches input characters as they arrive.
Our JSON parser is kind of between the two. The lexer feeds tokens to
a "streamer" instead of a real parser. The streamer accumulates
tokens until it got the sequence of tokens that comprise a single JSON
value (it counts curly braces and square brackets to decide). It
feeds those token sequences to a callback provided by the client. The
callback passes each token sequence to the parser, and gets back an
abstract syntax tree.
I figure it was done that way to make a straightforward recursive
descent parser possible. "Get next token" becomes "pop the first
token off the token sequence". Drawback: we need to store a complete
token sequence. Each token eats 13 + input characters + malloc
overhead bytes.
Observations:
1. This is not the only way to use recursive descent. If we replaced
"get next token" by a coroutine yield, we could do without a
streamer.
2. The lexer reports errors by passing a JSON_ERROR token to the
streamer. This communicates the offending input characters and
their location, but no more.
3. The streamer reports errors by passing a null token sequence to the
callback. The (already poor) lexical error information is thrown
away.
4. Having the callback receive a token sequence duplicates the code to
convert token sequence to abstract syntax tree in every callback.
5. Known bug: the streamer silently drops incomplete token sequences.
This commit rectifies 4. by lifting the call of the parser from the
callbacks into the streamer. Later commits will address 3. and 5.
The lifting removes a bug from qjson.c's parse_json(): it passed a
pointer to a non-null Error * in certain cases, as demonstrated by
check-qjson.c.
json_parser_parse() is now unused. It's a stupid wrapper around
json_parser_parse_err(). Drop it, and rename json_parser_parse_err()
to json_parser_parse().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-35-armbru@redhat.com>
show more ...
|
| 2d36e843 | 06-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
qobject: qobject_from_jsonv() is dangerous, hide it away
qobject_from_jsonv() takes ownership of %p arguments. On failure, we
can't generally know whether we failed before or after %p, so
ownership
qobject: qobject_from_jsonv() is dangerous, hide it away
qobject_from_jsonv() takes ownership of %p arguments. On failure, we
can't generally know whether we failed before or after %p, so
ownership becomes indeterminate. To avoid leaks, callers passing %p
must terminate on error, e.g. by passing &error_abort. Trap for the
unwary; document and give the function internal linkage.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180806065344.7103-11-armbru@redhat.com>
show more ...
|
| 4ff18468 | 06-Aug-2018 |
Markus Armbruster <armbru@redhat.com> |
qobject: New qobject_from_vjsonf_nofail(), qdict_from_vjsonf_nofail()
Every printf()-like function sooner or later needs its vprintf()-like
buddy. The next commit will need qobject_from_jsonf_nofai
qobject: New qobject_from_vjsonf_nofail(), qdict_from_vjsonf_nofail()
Every printf()-like function sooner or later needs its vprintf()-like
buddy. The next commit will need qobject_from_jsonf_nofail()'s buddy,
and qdict_from_jsonf_nofail()'s buddy will be used later in this
series. Add both.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180806065344.7103-8-armbru@redhat.com>
show more ...
|
| d43b1694 | 03-Jul-2018 |
Markus Armbruster <armbru@redhat.com> |
qmp: Use QDict * instead of QObject * for response objects
By using the more specific type, we get fewer downcasts. The
downcasts are safe, but not obviously so, at least not locally.
Signed-off-b
qmp: Use QDict * instead of QObject * for response objects
By using the more specific type, we get fewer downcasts. The
downcasts are safe, but not obviously so, at least not locally.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-24-armbru@redhat.com>
show more ...
|
| cee32796 | 03-Jul-2018 |
Markus Armbruster <armbru@redhat.com> |
qmp: De-duplicate error response building
All callers of qmp_build_error_object() duplicate the code to wrap it
in a response object. Replace it by qmp_error_response() that
captures the duplicated
qmp: De-duplicate error response building
All callers of qmp_build_error_object() duplicate the code to wrap it
in a response object. Replace it by qmp_error_response() that
captures the duplicated code, including error_free().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-23-armbru@redhat.com>
show more ...
|
| a193352f | 03-Jul-2018 |
Markus Armbruster <armbru@redhat.com> |
qobject: New qdict_from_jsonf_nofail()
Many uses of qobject_from_jsonf() convert JSON objects. Create new
convenience function qdict_from_jsonf_nofail() that includes the
conversion to QDict. The
qobject: New qdict_from_jsonf_nofail()
Many uses of qobject_from_jsonf() convert JSON objects. Create new
convenience function qdict_from_jsonf_nofail() that includes the
conversion to QDict. The next few commits will put it to use.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-22-armbru@redhat.com>
show more ...
|
| 69240fe6 | 03-Jul-2018 |
Markus Armbruster <armbru@redhat.com> |
qmp: Don't let malformed in-band commands jump the queue
handle_qmp_command() reports certain errors right away. This is wrong
when OOB is enabled, because the errors can "jump the queue" then, as
qmp: Don't let malformed in-band commands jump the queue
handle_qmp_command() reports certain errors right away. This is wrong
when OOB is enabled, because the errors can "jump the queue" then, as
the previous commit demonstrates.
To fix, we need to delay errors until dispatch. Do that for semantic
errors, mostly by reverting ill-advised parts of commit cf869d53172
"qmp: support out-of-band (oob) execution". Bonus: doesn't run
qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
again in do_qmp_dispatch(). That's also due to commit cf869d53172.
The next commit will fix queue jumping for syntax errors.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-18-armbru@redhat.com>
show more ...
|