Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Wed 11 Sep 2019 15:36:02 BST
# gpg: using

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Wed 11 Sep 2019 15:36:02 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
virtio-blk: Cancel the pending BH when the dataplane is reset

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio,vhost: fixes, features, cleanups.

FLR support.
Misc fixes, cleanups.

Signed-off-by: Mic

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio,vhost: fixes, features, cleanups.

FLR support.
Misc fixes, cleanups.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Wed 04 Sep 2019 12:53:35 BST
# gpg: using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg: aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
libvhost-user: introduce and use vu_has_protocol_feature()
libvhost-user: fix SLAVE_SEND_FD handling
virtio-pci: Add Function Level Reset support
virtio-rng: change default backend to rng-builtin
virtio-rng: Keep the default backend out of VirtIORNGConf
rng-builtin: add an RNG backend that uses qemu_guest_getrandom()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20190903' into staging

Allow page table bit to swap endianness.
Reorganize watchpoints out of i/o path.
Return host address fr

Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20190903' into staging

Allow page table bit to swap endianness.
Reorganize watchpoints out of i/o path.
Return host address from probe_write / probe_access.

# gpg: Signature made Tue 03 Sep 2019 16:47:50 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-tcg-20190903: (36 commits)
tcg: Factor out probe_write() logic into probe_access()
tcg: Make probe_write() return a pointer to the host page
s390x/tcg: Pass a size to probe_write() in do_csst()
hppa/tcg: Call probe_write() also for CONFIG_USER_ONLY
mips/tcg: Call probe_write() for CONFIG_USER_ONLY as well
tcg: Enforce single page access in probe_write()
tcg: Factor out CONFIG_USER_ONLY probe_write() from s390x code
s390x/tcg: Fix length calculation in probe_write_access()
s390x/tcg: Use guest_addr_valid() instead of h2g_valid() in probe_write_access()
tcg: Check for watchpoints in probe_write()
cputlb: Handle watchpoints via TLB_WATCHPOINT
cputlb: Remove double-alignment in store_helper
cputlb: Fix size operand for tlb_fill on unaligned store
exec: Factor out cpu_watchpoint_address_matches
cputlb: Fold TLB_RECHECK into TLB_INVALID_MASK
exec: Factor out core logic of check_watchpoint()
exec: Move user-only watchpoint stubs inline
target/sparc: sun4u Invert Endian TTE bit
target/sparc: Add TLB entry with attributes
cputlb: Byte swap memory transaction attribute
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/juanquintela/tags/fail-pull-request' into staging

Fix multifd with big number of channels

# gpg: Signature made Wed 04 Sep 2019 07:28:31 BST

Merge remote-tracking branch 'remotes/juanquintela/tags/fail-pull-request' into staging

Fix multifd with big number of channels

# gpg: Signature made Wed 04 Sep 2019 07:28:31 BST
# gpg: using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [full]
# gpg: aka "Juan Quintela <quintela@trasno.org>" [full]
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/fail-pull-request:
multifd: Use number of channels as listen backlog
socket: Add num connections to qio_net_listener_open_sync()
socket: Add num connections to qio_channel_socket_async()
socket: Add num connections to qio_channel_socket_sync()
socket: Add backlog parameter to socket_listen

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine + x86 queue, 2019-09-03

Bug fixes:
* Fix die-id validation regression (Eduard

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine + x86 queue, 2019-09-03

Bug fixes:
* Fix die-id validation regression (Eduardo Habkost)
* vmmouse: Properly reset state (Jan Kiszka)
* hostmem-file: fix pmem file size check (Stefan Hajnoczi)
* Keep query-hotpluggable-cpus output compatible with older QEMU
if '-smp dies' is not set (Igor Mammedov)
* migration: Do not re-read the clock on pre_save in case of paused guest
(Maxiwell S. Garcia)

Cleanups:
* NUMA code cleanups (Tao Xu)
* Remove stale externs from includes (Alex Bennée)

Features:
* qapi: report the default CPU type for each machine (Daniel P. Berrangé)

# gpg: Signature made Tue 03 Sep 2019 21:57:37 BST
# gpg: using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg: issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
migration: Do not re-read the clock on pre_save in case of paused guest
x86: do not advertise die-id in query-hotpluggbale-cpus if '-smp dies' is not set
i386/vmmouse: Properly reset state
hostmem-file: fix pmem file size check
qapi: report the default CPU type for each machine
pc: Don't make die-id mandatory unless necessary
pc: Improve error message when die-id is omitted
pc: Fix error message on die-id validation
numa: move numa global variable numa_info into MachineState
numa: move numa global variable have_numa_distance into MachineState
numa: move numa global variable nb_numa_nodes into MachineState
hw/arm: simplify arm_load_dtb
includes: remove stale [smp|max]_cpus externs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190903' into staging

target-arm queue:
* Revert and correctly fix refactoring of unallocated_encoding()
* Take

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190903' into staging

target-arm queue:
* Revert and correctly fix refactoring of unallocated_encoding()
* Take exceptions on ATS instructions when needed
* aspeed/timer: Provide back-pressure information for short periods
* memory: Remove unused memory_region_iommu_replay_all()
* hw/arm/smmuv3: Log a guest error when decoding an invalid STE
* hw/arm/smmuv3: Remove spurious error messages on IOVA invalidations
* target/arm: Fix SMMLS argument order
* hw/arm: Use ARM_CPU_TYPE_NAME() macro when appropriate
* hw/arm: Correct reference counting for creation of various objects
* includes: remove stale [smp|max]_cpus externs
* tcg/README: fix typo
* atomic_template: fix indentation in GEN_ATOMIC_HELPER
* include/exec/cpu-defs.h: fix typo
* target/arm: Free TCG temps in trans_VMOV_64_sp()
* target/arm: Don't abort on M-profile exception return in linux-user mode

# gpg: Signature made Tue 03 Sep 2019 16:35:19 BST
# gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg: issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg: aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20190903: (21 commits)
target/arm: Don't abort on M-profile exception return in linux-user mode
target/arm: Free TCG temps in trans_VMOV_64_sp()
include/exec/cpu-defs.h: fix typo
atomic_template: fix indentation in GEN_ATOMIC_HELPER
tcg/README: fix typo s/afterwise/afterwards/
includes: remove stale [smp|max]_cpus externs
hw/net/xilinx_axi: Use object_initialize_child for correct ref. counting
hw/dma/xilinx_axi: Use object_initialize_child for correct ref. counting
hw/arm/fsl-imx: Add the cpu as child of the SoC object
hw/arm: Use sysbus_init_child_obj for correct reference counting
hw/arm: Use object_initialize_child for correct reference counting
hw/arm: Use ARM_CPU_TYPE_NAME() macro when appropriate
target/arm: Fix SMMLS argument order
hw/arm/smmuv3: Remove spurious error messages on IOVA invalidations
hw/arm/smmuv3: Log a guest error when decoding an invalid STE
memory: Remove unused memory_region_iommu_replay_all()
aspeed/timer: Provide back-pressure information for short periods
target/arm: Take exceptions on ATS instructions when needed
target/arm: Allow ARMCPRegInfo read/write functions to throw exceptions
target/arm: Factor out unallocated_encoding for aarch32
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-03' into staging

Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw fil

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-03' into staging

Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw files have their first block allocated so we can probe
the O_DIRECT alignment if necessary
- Various fixes

# gpg: Signature made Tue 03 Sep 2019 13:58:57 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-09-03:
iotests: Unify cache mode quoting
tests/check-block: Skip iotests when sanitizers are enabled
iotests: Check for enabled drivers before testing them
iotests: Add -display none to the qemu options
file-posix: fix request_alignment typo
iotests: Disable 126 for flat vmdk subformats
iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
iotests: Disable broken streamOptimized tests
vmdk: Reject invalid compressed writes
iotests: Keep testing broken relative extent paths
vmdk: Use bdrv_dirname() for relative extent paths
iotests: Fix _filter_img_create()
iotests: Test allocate_first_block() with O_DIRECT
block: posix: Always allocate the first block
block: fix permission update in bdrv_replace_node
qemu-io: add pattern file for write command

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: posix: Always allocate the first block

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster

block: posix: Always allocate the first block

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster
storage backed by XFS filesystem, reading this block using direct I/O
succeeds regardless of request length, fooling alignment detection.

In this case we fallback to a safe value (4096) instead of the optimal
value (512), which may lead to unneeded data copying when aligning
requests. Allocating the first block avoids the fallback.

Since we allocate the first block even with preallocation=off, we no
longer create images with zero disk size:

$ ./qemu-img create -f raw test.raw 1g
Formatting 'test.raw', fmt=raw size=1073741824

$ ls -lhs test.raw
4.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw

And converting the image requires additional cluster:

$ ./qemu-img measure -f raw -O qcow2 test.raw
required size: 458752
fully allocated size: 1074135040

When using format like vmdk with multiple files per image, we allocate
one block per file:

$ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

$ ls -lhs test*.vmdk
4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
4.0K -rw-r--r--. 1 nsoffer nsoffer 353 Aug 27 03:23 test.vmdk

I did quick performance test for copying disks with qemu-img convert to
new raw target image to Gluster storage with sector size of 512 bytes:

for i in $(seq 10); do
rm -f dst.raw
sleep 10
time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
done

Here is a table comparing the total time spent:

Type Before(s) After(s) Diff(%)
---------------------------------------
real 530.028 469.123 -11.4
user 17.204 10.768 -37.4
sys 17.881 7.011 -60.7

We can see very clear improvement in CPU usage.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827010528.8818-2-nsoffer@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...