qemu-img: map: report compressed data blocks

Right now "qemu-img map" reports compressed blocks as containing data
but having no host offset. This is not very informative. Instead,
let's add anoth

qemu-img: map: report compressed data blocks

Right now "qemu-img map" reports compressed blocks as containing data
but having no host offset. This is not very informative. Instead,
let's add another boolean field named "compressed" in case JSON output
mode is specified. This is achieved by utilizing new allocation status
flag BDRV_BLOCK_COMPRESSED for bdrv_block_status().

Also update the expected qemu-iotests outputs to contain the new field.

Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Message-ID: <20230907210226.953821-3-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/mdroth/tags/qga-pull-2021-07-13-tag' into staging

qemu-ga patch queue for soft-freeze

* add support for Windows Server 2022 in get-osinfo comma

Merge remote-tracking branch 'remotes/mdroth/tags/qga-pull-2021-07-13-tag' into staging

qemu-ga patch queue for soft-freeze

* add support for Windows Server 2022 in get-osinfo command

# gpg: Signature made Tue 13 Jul 2021 19:10:05 BST
# gpg: using RSA key CEACC9E15534EBABB82D3FA03353C9CEF108B584
# gpg: Good signature from "Michael Roth <flukshun@gmail.com>" [full]
# gpg: aka "Michael Roth <mdroth@utexas.edu>" [full]
# gpg: aka "Michael Roth <mdroth@linux.vnet.ibm.com>" [full]
# Primary key fingerprint: CEAC C9E1 5534 EBAB B82D 3FA0 3353 C9CE F108 B584

* remotes/mdroth/tags/qga-pull-2021-07-13-tag:
qga-win: Add support of Windows Server 2022 in get-osinfo command

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ehabkost-gl/tags/x86-next-pull-request' into staging

x86 queue, 2021-07-13

Bug fixes:
* numa: Parse initiator= attribute before cpus= attri

Merge remote-tracking branch 'remotes/ehabkost-gl/tags/x86-next-pull-request' into staging

x86 queue, 2021-07-13

Bug fixes:
* numa: Parse initiator= attribute before cpus= attribute
(Michal Privoznik)
* Fix CPUID level for AMD (Zhenwei Pi)
* Suppress CPUID leaves not defined by the CPU vendor
(Michael Roth)

Cleanup:
* Hyper-V feature handling cleanup (Vitaly Kuznetsov)

# gpg: Signature made Tue 13 Jul 2021 17:09:01 BST
# gpg: using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg: issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost-gl/tags/x86-next-pull-request:
numa: Parse initiator= attribute before cpus= attribute
numa: Report expected initiator
target/i386: Fix cpuid level for AMD
target/i386: suppress CPUID leaves not defined by the CPU vendor
i386: Hyper-V SynIC requires POST_MESSAGES/SIGNAL_EVENTS privileges
i386: HV_HYPERCALL_AVAILABLE privilege bit is always needed
i386: kill off hv_cpuid_check_and_set()
i386: expand Hyper-V features during CPU feature expansion time
i386: make hyperv_expand_features() return bool
i386: hardcode supported eVMCS version to '1'
i386: clarify 'hv-passthrough' behavior

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.1-pull-request' into staging

Linux-user pull request 20210713

Update headers to linux v5.13
cleanup errno tar

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.1-pull-request' into staging

Linux-user pull request 20210713

Update headers to linux v5.13
cleanup errno target headers
Fix race condition on fd translation table

# gpg: Signature made Tue 13 Jul 2021 14:41:25 BST
# gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg: issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-6.1-pull-request:
linux-user: update syscall.tbl to Linux v5.13
linux-user, mips: update syscall-args-o32.c.inc to Linux v5.13
linux-user: update syscall_nr.h to Linux v5.13
fd-trans: Fix race condition on reallocation of the translation table.
linux-user/syscall: Remove ERRNO_TABLE_SIZE check
linux-user: Simplify host <-> target errno conversion using macros
linux-user/mips: Move errno definitions to 'target_errno_defs.h'
linux-user/hppa: Move errno definitions to 'target_errno_defs.h'
linux-user/alpha: Move errno definitions to 'target_errno_defs.h'
linux-user: Extract target errno to 'target_errno_defs.h'
linux-user/sparc: Rename target_errno.h -> target_errno_defs.h
linux-user/syscall: Fix RF-kill errno (typo in ERFKILL)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/dg-gitlab/tags/ppc-for-6.1-20210713' into staging

ppc patch queue 2021-07-13

I thought I'd sent the last PR before the 6.1 soft freeze, but

Merge remote-tracking branch 'remotes/dg-gitlab/tags/ppc-for-6.1-20210713' into staging

ppc patch queue 2021-07-13

I thought I'd sent the last PR before the 6.1 soft freeze, but
unfortunately I need one more. This last minute one puts in a SLOF
update, along with a couple of bugfixes.

# gpg: Signature made Tue 13 Jul 2021 03:07:20 BST
# gpg: using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392
# gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" [full]
# gpg: aka "David Gibson (Red Hat) <dgibson@redhat.com>" [full]
# gpg: aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" [full]
# gpg: aka "David Gibson (kernel.org) <dwg@kernel.org>" [unknown]
# Primary key fingerprint: 75F4 6586 AE61 A66C C44E 87DC 6C38 CACA 20D9 B392

* remotes/dg-gitlab/tags/ppc-for-6.1-20210713:
mv64361: Remove extra break from a switch case
pseries: Update SLOF firmware image
ppc/pegasos2: Allow setprop in VOF

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-07-09-v2' into staging

nbd patches for 2021-07-09

- enhance 'qemu-img map --output=json' to make it easier to duplicat

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-07-09-v2' into staging

nbd patches for 2021-07-09

- enhance 'qemu-img map --output=json' to make it easier to duplicate
backing chain allocation patterns
- fix a race in the 'yank' QMP command in relation to NBD requests

# gpg: Signature made Mon 12 Jul 2021 18:43:37 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-07-09-v2:
nbd: register yank function earlier
qemu-img: Reword 'qemu-img map --output=json' docs
qemu-img: Make unallocated part of backing chain obvious in map
iotests: Improve and rename test 309 to nbd-qemu-allocation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-img: Make unallocated part of backing chain obvious in map

The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that d

qemu-img: Make unallocated part of backing chain obvious in map

The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).

The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.

An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Support for FUSE exports
- Fix deadlock in bdrv_co_yield_to_drain()
- Use

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Support for FUSE exports
- Fix deadlock in bdrv_co_yield_to_drain()
- Use lock guard macros
- Some preparational patches for 64 bit block layer
- file-posix: Fix request extension to INT64_MAX in raw_do_pwrite_zeroes()

# gpg: Signature made Fri 11 Dec 2020 17:06:19 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (34 commits)
block: Fix deadlock in bdrv_co_yield_to_drain()
block: Fix locking in qmp_block_resize()
block: Simplify qmp_block_resize() error paths
block: introduce BDRV_MAX_LENGTH
block/io: bdrv_check_byte_request(): drop bdrv_is_inserted()
block/io: bdrv_refresh_limits(): use ERRP_GUARD
block/file-posix: fix workaround in raw_do_pwrite_zeroes()
can-host: Fix crash when 'canbus' property is not set
iotests/221: Discard image before qemu-img map
file-posix: check the use_lock before setting the file lock
iotests/308: Add test for FUSE exports
iotests: Enable fuse for many tests
iotests: Allow testing FUSE exports
iotests: Give access to the qemu-storage-daemon
storage-daemon: Call bdrv_close_all() on exit
iotests/287: Clean up subshell test image
iotests: Let _make_test_img guess $TEST_IMG_FILE
iotests: Restrict some Python tests to file
iotests/091: Use _cleanup_qemu instad of "wait"
iotests: Derive image names from $TEST_IMG
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests/221: Discard image before qemu-img map

See the new comment for why this should be done.

I do not have a reproducer on master, but when using FUSE block exports,
this tes

iotests/221: Discard image before qemu-img map

See the new comment for why this should be done.

I do not have a reproducer on master, but when using FUSE block exports,
this test breaks depending on the underlying filesystem (for me, it
works on tmpfs, but fails on xfs, because the block allocated by
file-posix has 16 kB there instead of 4 kB).

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201207152245.66987-1-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Wed 11 Sep 2019 15:36:02 BST
# gpg: using

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Wed 11 Sep 2019 15:36:02 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
virtio-blk: Cancel the pending BH when the dataplane is reset

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio,vhost: fixes, features, cleanups.

FLR support.
Misc fixes, cleanups.

Signed-off-by: Mic

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio,vhost: fixes, features, cleanups.

FLR support.
Misc fixes, cleanups.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Wed 04 Sep 2019 12:53:35 BST
# gpg: using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg: aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
libvhost-user: introduce and use vu_has_protocol_feature()
libvhost-user: fix SLAVE_SEND_FD handling
virtio-pci: Add Function Level Reset support
virtio-rng: change default backend to rng-builtin
virtio-rng: Keep the default backend out of VirtIORNGConf
rng-builtin: add an RNG backend that uses qemu_guest_getrandom()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20190903' into staging

Allow page table bit to swap endianness.
Reorganize watchpoints out of i/o path.
Return host address fr

Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20190903' into staging

Allow page table bit to swap endianness.
Reorganize watchpoints out of i/o path.
Return host address from probe_write / probe_access.

# gpg: Signature made Tue 03 Sep 2019 16:47:50 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-tcg-20190903: (36 commits)
tcg: Factor out probe_write() logic into probe_access()
tcg: Make probe_write() return a pointer to the host page
s390x/tcg: Pass a size to probe_write() in do_csst()
hppa/tcg: Call probe_write() also for CONFIG_USER_ONLY
mips/tcg: Call probe_write() for CONFIG_USER_ONLY as well
tcg: Enforce single page access in probe_write()
tcg: Factor out CONFIG_USER_ONLY probe_write() from s390x code
s390x/tcg: Fix length calculation in probe_write_access()
s390x/tcg: Use guest_addr_valid() instead of h2g_valid() in probe_write_access()
tcg: Check for watchpoints in probe_write()
cputlb: Handle watchpoints via TLB_WATCHPOINT
cputlb: Remove double-alignment in store_helper
cputlb: Fix size operand for tlb_fill on unaligned store
exec: Factor out cpu_watchpoint_address_matches
cputlb: Fold TLB_RECHECK into TLB_INVALID_MASK
exec: Factor out core logic of check_watchpoint()
exec: Move user-only watchpoint stubs inline
target/sparc: sun4u Invert Endian TTE bit
target/sparc: Add TLB entry with attributes
cputlb: Byte swap memory transaction attribute
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/juanquintela/tags/fail-pull-request' into staging

Fix multifd with big number of channels

# gpg: Signature made Wed 04 Sep 2019 07:28:31 BST

Merge remote-tracking branch 'remotes/juanquintela/tags/fail-pull-request' into staging

Fix multifd with big number of channels

# gpg: Signature made Wed 04 Sep 2019 07:28:31 BST
# gpg: using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [full]
# gpg: aka "Juan Quintela <quintela@trasno.org>" [full]
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/fail-pull-request:
multifd: Use number of channels as listen backlog
socket: Add num connections to qio_net_listener_open_sync()
socket: Add num connections to qio_channel_socket_async()
socket: Add num connections to qio_channel_socket_sync()
socket: Add backlog parameter to socket_listen

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine + x86 queue, 2019-09-03

Bug fixes:
* Fix die-id validation regression (Eduard

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine + x86 queue, 2019-09-03

Bug fixes:
* Fix die-id validation regression (Eduardo Habkost)
* vmmouse: Properly reset state (Jan Kiszka)
* hostmem-file: fix pmem file size check (Stefan Hajnoczi)
* Keep query-hotpluggable-cpus output compatible with older QEMU
if '-smp dies' is not set (Igor Mammedov)
* migration: Do not re-read the clock on pre_save in case of paused guest
(Maxiwell S. Garcia)

Cleanups:
* NUMA code cleanups (Tao Xu)
* Remove stale externs from includes (Alex Bennée)

Features:
* qapi: report the default CPU type for each machine (Daniel P. Berrangé)

# gpg: Signature made Tue 03 Sep 2019 21:57:37 BST
# gpg: using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg: issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
migration: Do not re-read the clock on pre_save in case of paused guest
x86: do not advertise die-id in query-hotpluggbale-cpus if '-smp dies' is not set
i386/vmmouse: Properly reset state
hostmem-file: fix pmem file size check
qapi: report the default CPU type for each machine
pc: Don't make die-id mandatory unless necessary
pc: Improve error message when die-id is omitted
pc: Fix error message on die-id validation
numa: move numa global variable numa_info into MachineState
numa: move numa global variable have_numa_distance into MachineState
numa: move numa global variable nb_numa_nodes into MachineState
hw/arm: simplify arm_load_dtb
includes: remove stale [smp|max]_cpus externs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190903' into staging

target-arm queue:
* Revert and correctly fix refactoring of unallocated_encoding()
* Take

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190903' into staging

target-arm queue:
* Revert and correctly fix refactoring of unallocated_encoding()
* Take exceptions on ATS instructions when needed
* aspeed/timer: Provide back-pressure information for short periods
* memory: Remove unused memory_region_iommu_replay_all()
* hw/arm/smmuv3: Log a guest error when decoding an invalid STE
* hw/arm/smmuv3: Remove spurious error messages on IOVA invalidations
* target/arm: Fix SMMLS argument order
* hw/arm: Use ARM_CPU_TYPE_NAME() macro when appropriate
* hw/arm: Correct reference counting for creation of various objects
* includes: remove stale [smp|max]_cpus externs
* tcg/README: fix typo
* atomic_template: fix indentation in GEN_ATOMIC_HELPER
* include/exec/cpu-defs.h: fix typo
* target/arm: Free TCG temps in trans_VMOV_64_sp()
* target/arm: Don't abort on M-profile exception return in linux-user mode

# gpg: Signature made Tue 03 Sep 2019 16:35:19 BST
# gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg: issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg: aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20190903: (21 commits)
target/arm: Don't abort on M-profile exception return in linux-user mode
target/arm: Free TCG temps in trans_VMOV_64_sp()
include/exec/cpu-defs.h: fix typo
atomic_template: fix indentation in GEN_ATOMIC_HELPER
tcg/README: fix typo s/afterwise/afterwards/
includes: remove stale [smp|max]_cpus externs
hw/net/xilinx_axi: Use object_initialize_child for correct ref. counting
hw/dma/xilinx_axi: Use object_initialize_child for correct ref. counting
hw/arm/fsl-imx: Add the cpu as child of the SoC object
hw/arm: Use sysbus_init_child_obj for correct reference counting
hw/arm: Use object_initialize_child for correct reference counting
hw/arm: Use ARM_CPU_TYPE_NAME() macro when appropriate
target/arm: Fix SMMLS argument order
hw/arm/smmuv3: Remove spurious error messages on IOVA invalidations
hw/arm/smmuv3: Log a guest error when decoding an invalid STE
memory: Remove unused memory_region_iommu_replay_all()
aspeed/timer: Provide back-pressure information for short periods
target/arm: Take exceptions on ATS instructions when needed
target/arm: Allow ARMCPRegInfo read/write functions to throw exceptions
target/arm: Factor out unallocated_encoding for aarch32
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-03' into staging

Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw fil

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-03' into staging

Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw files have their first block allocated so we can probe
the O_DIRECT alignment if necessary
- Various fixes

# gpg: Signature made Tue 03 Sep 2019 13:58:57 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-09-03:
iotests: Unify cache mode quoting
tests/check-block: Skip iotests when sanitizers are enabled
iotests: Check for enabled drivers before testing them
iotests: Add -display none to the qemu options
file-posix: fix request_alignment typo
iotests: Disable 126 for flat vmdk subformats
iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
iotests: Disable broken streamOptimized tests
vmdk: Reject invalid compressed writes
iotests: Keep testing broken relative extent paths
vmdk: Use bdrv_dirname() for relative extent paths
iotests: Fix _filter_img_create()
iotests: Test allocate_first_block() with O_DIRECT
block: posix: Always allocate the first block
block: fix permission update in bdrv_replace_node
qemu-io: add pattern file for write command

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: posix: Always allocate the first block

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster

block: posix: Always allocate the first block

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster
storage backed by XFS filesystem, reading this block using direct I/O
succeeds regardless of request length, fooling alignment detection.

In this case we fallback to a safe value (4096) instead of the optimal
value (512), which may lead to unneeded data copying when aligning
requests. Allocating the first block avoids the fallback.

Since we allocate the first block even with preallocation=off, we no
longer create images with zero disk size:

$ ./qemu-img create -f raw test.raw 1g
Formatting 'test.raw', fmt=raw size=1073741824

$ ls -lhs test.raw
4.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw

And converting the image requires additional cluster:

$ ./qemu-img measure -f raw -O qcow2 test.raw
required size: 458752
fully allocated size: 1074135040

When using format like vmdk with multiple files per image, we allocate
one block per file:

$ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

$ ls -lhs test*.vmdk
4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
4.0K -rw-r--r--. 1 nsoffer nsoffer 353 Aug 27 03:23 test.vmdk

I did quick performance test for copying disks with qemu-img convert to
new raw target image to Gluster storage with sector size of 512 bytes:

for i in $(seq 10); do
rm -f dst.raw
sleep 10
time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
done

Here is a table comparing the total time spent:

Type Before(s) After(s) Diff(%)
---------------------------------------
real 530.028 469.123 -11.4
user 17.204 10.768 -37.4
sys 17.881 7.011 -60.7

We can see very clear improvement in CPU usage.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827010528.8818-2-nsoffer@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge tag 's390-ccw-bios-2019-05-08' into s390-next-staging

Skip unsupported bootmap signature entries instead of aborting the boot process

# gpg: Signature made Wed 08 May 2019 11:

Merge tag 's390-ccw-bios-2019-05-08' into s390-next-staging

Skip unsupported bootmap signature entries instead of aborting the boot process

# gpg: Signature made Wed 08 May 2019 11:42:24 AM CEST
# gpg: using RSA key 2ED9D774FE702DB5
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [undefined]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [undefined]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]

* tag 's390-ccw-bios-2019-05-08':
pc-bios/s390: Update firmware image with "Skip bootmap signature entries" fix
s390-bios: Skip bootmap signature entries
pc-bios/s390-ccw: Clean up harmless misuse of isdigit()

show more ...

Merge remote-tracking branch 'remotes/stsquad/tags/pull-demacro-softmmu-100519-1' into staging

Demacrofy the SoftMMU

- the demacro itself
- refactor TLB_RECHECK and fix bug

Merge remote-tracking branch 'remotes/stsquad/tags/pull-demacro-softmmu-100519-1' into staging

Demacrofy the SoftMMU

- the demacro itself
- refactor TLB_RECHECK and fix bug
- move unaligned handler out

# gpg: Signature made Fri 10 May 2019 20:34:14 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-demacro-softmmu-100519-1:
cputlb: Do unaligned store recursion to outermost function
cputlb: Do unaligned load recursion to outermost function
cputlb: Drop attribute flatten
cputlb: Move TLB_RECHECK handling into load/store_helper
accel/tcg: demacro cputlb

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-05-07' into staging

Block patches:
- Fixes to qcow2's implementation of qemu-img check
- Our SSH driver now suppor

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-05-07' into staging

Block patches:
- Fixes to qcow2's implementation of qemu-img check
- Our SSH driver now supports bdrv_refresh_filename()
- Miscellaneous fixes

# gpg: Signature made Tue 07 May 2019 16:16:58 BST
# gpg: using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-05-07:
iotests: Fix iotests 110 and 126
commit: Use bdrv_append() in commit_start()
block: Assert that drv->bdrv_child_perm is set in bdrv_child_perm()
block/ssh: Implement .bdrv_dirname()
block/ssh: Implement .bdrv_refresh_filename()
qcow2: discard bitmap when removed
qcow2-refcount: don't mask corruptions under internal errors
qcow2-refcount: check_refcounts_l2: don't count fixed cluster as allocated
qcow2-refcount: check_refcounts_l2: reduce ignored overlaps
qcow2-refcount: avoid eating RAM
qcow2-refcount: fix check_oflag_copied

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-05-09' into staging

- Fix "make check" problem that occurred with LANG=C and Python 3.5 / 3.6
- Get rid of some m

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-05-09' into staging

- Fix "make check" problem that occurred with LANG=C and Python 3.5 / 3.6
- Get rid of some more dependencies on the global_qtest variable in the qtests
- Some other small test clean-ups
- Some copyright statement clarifications
- Mark TARGET_FMT_lu as poisoned

# gpg: Signature made Thu 09 May 2019 08:45:47 BST
# gpg: using RSA key 2ED9D774FE702DB5
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2019-05-09:
include/exec/poison: Mark TARGET_FMT_lu as poisoned, too
target/sh4: Fix LGPL information in the file headers
target/openrisc: Fix LGPL information in the file headers
hw/i2c/smbus_ich9: Fix the confusing contributions-after-2012 statement
tests: qpci_unplug_acpi_device_test() should not rely on global_qtest
tests/drive_del-test: Use qtest_init() instead of qtest_start()
tests/Makefile: Remove unused test-obj-y variable
tests/tpm-tests: Use g_test_skip() to mark skipped tests
tests/ide-test: Make test independent of global_qtest
tests/test-hmp: Use qtest_init() instead of qtest_start()
tests/qmp-cmd-test: Use qtest_init() instead of qtest_start()
tests/megasas: Make test independent of global_qtest
tests/tco: Make test independent of global_qtest
tests: Force Python I/O encoding for check-qapi-schema

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

Pull request trivial branch 2019-05-03

# gpg: Signature made Fri 03 May 2019 12:26:34 BS

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

Pull request trivial branch 2019-05-03

# gpg: Signature made Fri 03 May 2019 12:26:34 BST
# gpg: using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
sockets: avoid string truncation warnings when copying UNIX path
hw/sparc/leon3: Allow load of uImage firmwares
Makefile: Let the 'clean' rule remove qemu-ga.exe on Windows hosts
net: Print output of "-net nic, model=help" to stdout instead of stderr
Header cleanups
Update configure
configure: fix pam test warning
qom: use object_new_with_type in object_new_with_propv
doc: fix the configuration path
CODING_STYLE: indent example code as all others
CODING_STYLE: specify the indent rule for multiline code
hw/net/pcnet: Use qemu_log_mask(GUEST_ERROR) instead of printf

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-05-07' into staging

nbd patches for 2019-05-07

- iotest improvements

# gpg: Signature made Tue 07 May 2019 16

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-05-07' into staging

nbd patches for 2019-05-07

- iotest improvements

# gpg: Signature made Tue 07 May 2019 16:00:18 BST
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-05-07:
iotests: Make 182 do without device_add
iotests: Tweak 221 sizing for different hole granularities
tests/qemu-iotests: Fix more reference output files due to recent qemu-io change
iotests: Fix 233 for ports other than 10809
qemu-nbd: Look up flag names in array

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Tweak 221 sizing for different hole granularities

For some particular configurations of ext4, sizing an image to 84
sectors + 1 byte causes test failures when the size of the ho

iotests: Tweak 221 sizing for different hole granularities

For some particular configurations of ext4, sizing an image to 84
sectors + 1 byte causes test failures when the size of the hole is
rounded to a 4k alignment. Let's instead size things to 128 sectors +
1 byte, as the 64k boundary is more likely to work with various hole
granularities.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190506172111.31594-1-eblake@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>

show more ...

Merge tag 'tags/s390x-2018-06-18' into staging

Add support for pxelinux.cfg-style network booting to the s390x firmware

# gpg: Signature made Mon 18 Jun 2018 03:59:06 PM CEST
#

Merge tag 'tags/s390x-2018-06-18' into staging

Add support for pxelinux.cfg-style network booting to the s390x firmware

# gpg: Signature made Mon 18 Jun 2018 03:59:06 PM CEST
# gpg: using RSA key 2ED9D774FE702DB5
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [undefined]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [undefined]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]

* tag 'tags/s390x-2018-06-18':
pc-bios/s390-ccw: Update the s390-netboot.img binary
pc-bios/s390-ccw: Optimize the s390-netboot.img for size
pc-bios/s390-ccw/net: Try to load pxelinux.cfg file accoring to the UUID
pc-bios/s390-ccw/net: Add support for pxelinux-style config files
pc-bios/s390-ccw/net: Update code for the latest changes in SLOF
roms: Update SLOF submodule to current status
pc-bios/s390-ccw: define loadparm length

Signed-off-by: Cornelia Huck <cohuck@redhat.com>

show more ...