Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iscsi: Cap block count from GET LBA STATUS (CVE-2020-1711)
- AioContext fixes in QMP commands for

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iscsi: Cap block count from GET LBA STATUS (CVE-2020-1711)
- AioContext fixes in QMP commands for backup and bitmaps
- iotests fixes

# gpg: Signature made Mon 27 Jan 2020 17:49:58 GMT
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
iscsi: Don't access non-existent scsi_lba_status_descriptor
iscsi: Cap block count from GET LBA STATUS (CVE-2020-1711)
block/backup: fix memory leak in bdrv_backup_top_append()
iotests: Test handling of AioContexts with some blockdev actions
blockdev: Return bs to the proper context on snapshot abort
blockdev: Acquire AioContext on dirty bitmap functions
block/backup-top: Don't acquire context while dropping top
blockdev: honor bdrv_try_set_aio_context() context requirements
blockdev: unify qmp_blockdev_backup and blockdev-backup transaction paths
blockdev: unify qmp_drive_backup and drive-backup transaction paths
blockdev: fix coding style issues in drive_backup_prepare
iotests: Add more "skip_if_unsupported" statements to the python tests
iotests.py: Let wait_migration wait even more

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

blockdev: unify qmp_drive_backup and drive-backup transaction paths

Issuing a drive-backup from qmp_drive_backup takes a slightly
different path than when it's issued from a transaction. In the code

blockdev: unify qmp_drive_backup and drive-backup transaction paths

Issuing a drive-backup from qmp_drive_backup takes a slightly
different path than when it's issued from a transaction. In the code,
this is manifested as some redundancy between do_drive_backup() and
drive_backup_prepare().

This change unifies both paths, merging do_drive_backup() and
drive_backup_prepare(), and changing qmp_drive_backup() to create a
transaction instead of calling do_backup_common() direcly.

As a side-effect, now qmp_drive_backup() is executed inside a drained
section, as it happens when creating a drive-backup transaction. This
change is visible from the user's perspective, as the job gets paused
and immediately resumed before starting the actual work.

Also fix tests 141, 185 and 219 to cope with the extra
JOB_STATUS_CHANGE lines.

Signed-off-by: Sergio Lopez <slp@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
(make check-acceptance) (Cleber Ro

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
(make check-acceptance) (Cleber Rosa)
* Make iotests compatible with Python 3
(Max Reitz)
* device-crash-test whitelist update (Thomas Huth)
* Misc cleanups (Cleber Rosa)

# gpg: Signature made Wed 31 Oct 2018 00:28:39 GMT
# gpg: using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
scripts/qemu.py: use a more consistent docstring style
scripts/decodetree.py: fix reference to attributes
Travis support for the acceptance tests
Acceptance tests: add make rule for running them
Bootstrap Python venv for tests
iotests: Unify log outputs between Python 2 and 3
iotests: Modify imports for Python 3
iotests: 'new' module replacement in 169
iotests: Explicitly bequeath FDs in Python
iotests: Different iterator behavior in Python 3
iotests: Use // for Python integer division
iotests: Use Python byte strings where appropriate
iotests: Flush in iotests.py's QemuIoInteractive
iotests: Make nbd-fault-injector flush
scripts/device-crash-test: Remove devices that are not user_creatable anymore

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2
(they are

iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2
(they are no longer in 3, because unicode strings are the default
there). Second, the order of keys in dicts may differ. Third,
especially long numbers are longs in Python 2 and thus get an 'L'
suffix, which does not happen in Python 3.

We can get around all of these differences by dumping objects (lists and
dicts) in a language-independent format, namely JSON. The JSON
generator even allows emitting dicts with their keys sorted
alphabetically.

This changes the output of all tests that use these logging functions
(dict keys are ordered now, strings in dicts are now enclosed in double
quotes instead of single quotes, the 'L' suffix of large integers is
dropped, and "true" and "false" are now in lower case).
The quote change necessitates a small change to a filter used in test
207.

Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20181022135307.14398-10-mreitz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2018-06-11' into staging

Block patches:
- Various bug fixes
- Removal of qemu-img convert's deprecated -s option
- qemu-io now exits wi

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2018-06-11' into staging

Block patches:
- Various bug fixes
- Removal of qemu-img convert's deprecated -s option
- qemu-io now exits with an error when a command failed

# gpg: Signature made Mon 11 Jun 2018 15:23:42 BST
# gpg: using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2018-06-11: (29 commits)
iotests: Add case for a corrupted inactive image
qcow2: Do not mark inactive images corrupt
block: Make bdrv_is_writable() public
throttle: Fix crash on reopen
block/qcow2-bitmap: fix free_bitmap_clusters
qemu-img: Remove deprecated -s snapshot_id_or_name option
iotests: Fix 219's timing
iotests: improve pause_job
iotests: Test post-backing convert target behavior
qemu-img: Special post-backing convert handling
iotests: Add test for rebasing with relative paths
qemu-img: Resolve relative backing paths in rebase
iotests: Let 216 make use of qemu-io's exit code
iotests.py: Add qemu_io_silent
qemu-io: Exit with error when a command failed
qemu-io: Let command functions return error code
qemu-io: Drop command functions' return values
iotests: Repairing error during snapshot deletion
qcow2: Repair OFLAG_COPIED when fixing leaks
iotests: Rework 113
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Fix 219's timing

219 has two issues that may lead to sporadic failure, both of which are
the result of issuing query-jobs too early after a job has been
modified. This can then lead to dif

iotests: Fix 219's timing

219 has two issues that may lead to sporadic failure, both of which are
the result of issuing query-jobs too early after a job has been
modified. This can then lead to different results based on whether the
modification has taken effect already or not.

First, query-jobs is issued right after the job has been created.
Besides its current progress possibly being in any random state (which
has already been taken care of), its total progress too is basically
arbitrary, because the job may not yet have been able to determine it.
This patch addresses this by just filtering the total progress, like
what has been done for the current progress already. However, for more
clarity, the filtering is changed to replace the values by a string
'FILTERED' instead of deleting them.

Secondly, query-jobs is issued right after a job has been resumed. The
job may or may not yet have had the time to actually perform any I/O,
and thus its current progress may or may not have advanced. To make
sure it has indeed advanced (which is what the reference output already
assumes), keep querying it until it has.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180606190628.8170-1-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Generic background jobs
- qemu-iotests fixes for NFS and the 'migration' group
- sheepdog: Minor c

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Generic background jobs
- qemu-iotests fixes for NFS and the 'migration' group
- sheepdog: Minor code simplification

# gpg: Signature made Wed 23 May 2018 13:33:49 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (46 commits)
qemu-iotests: Test job-* with block jobs
iotests: Move qmp_to_opts() to VM
blockjob: Remove BlockJob.driver
job: Add query-jobs QMP command
job: Add lifecycle QMP commands
job: Add JOB_STATUS_CHANGE QMP event
job: Introduce qapi/job.json
job: Move progress fields to Job
job: Add job_transition_to_ready()
job: Add job_is_ready()
job: Add job_dismiss()
job: Add job_yield()
block: Cancel job in bdrv_close_all() callers
job: Move completion and cancellation to Job
job: Move transactions to Job
job: Switch transactions to JobTxn
job: Move job_finish_sync() to Job
job: Move .complete callback to Job
job: Add job_drain()
job: Convert block_job_cancel_async() to Job
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Test job-* with block jobs

This adds a test case that tests the new job-* QMP commands with
mirror and backup block jobs.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>