Merge tag 'pull-qapi-2024-09-10' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2024-09-10

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFA

Merge tag 'pull-qapi-2024-09-10' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2024-09-10

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmbgS0ASHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTm+sP/1kEuGdbjjVLQC7U+gCClhALpYUtIMHQ
# qNZWmnxFyW9Jx6Lk8+FZBRJegNlmA9AJ2ZzD/2I/WoNtcrNiE53rIRG71Lcy8I6U
# CRjekLPn0jhFuYhmRfwz6ilcLkyMJ15Yi7iG6ssFsO1joBtU5Q1qpzQmFUl9UbV9
# 9iBe7mE99VzNwi3hMELEM9YuSgAs8st4Itnn8SiHkJ3s6hlmF8N4HwNKfQrseGfb
# FuJNN4p8Gcu0aF/dkGE8ADJBvtgzkFnavXt0GQPC7SjGOF7rUXxnUQlszNZM0OcK
# UAfWU06sIyiOholQQt8H8jawT+rGCLomfgPUBVq5K8WByd21IeQeS0upSPeTjxS+
# d/C8wzpcbhdNOECwC/wDtgZHvCC95cHNrxuC4+4/Q1KIVVr+1cWqe8hQzFvm3TIg
# dduEFTQDIxhpE0GO1ZWNf90upzBYBWwIsh4bqsMZS7SpaYIZ6QV45yViZ1w2WfbH
# m3/F34Z3yqgFuZQrZv4OPo7xHK0+y7uZ60RrhSJtE0X080syRJKBbGmNBRidoVyu
# JOONWH44I/XN5enZV8StJnqJr9MCV0DBZUCi9ZhP/kAhBjLc5cQ6NByEa9/rebYX
# 1bLTiA1JDLzDVIan+A8dz0riWmHBvTyBwhLnEXvXb9lcB3ozgHTb7axE5RnZSYLI
# YQgBOBMFuQKM
# =YO/Q
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 10 Sep 2024 14:36:00 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2024-09-10' of https://repo.or.cz/qemu/armbru:
qapi/vfio: Rename VfioMigrationState to Qapi*, and drop prefix
qapi/cryptodev: Rename QCryptodevBackendAlgType to *Algo, and drop prefix
qapi/cryptodev: Drop unwanted 'prefix'
qapi/crypto: Rename QCryptoAFAlg to QCryptoAFAlgo
qapi/crypto: Rename QCryptoRSAPaddingAlgorithm to *Algo, and drop prefix
qapi/crypto: Rename QCryptoAkCipherAlgorithm to *Algo, and drop prefix
qapi/crypto: Rename QCryptoIVGenAlgorithm to *Algo, and drop prefix
qapi/crypto: Rename QCryptoCipherAlgorithm to *Algo, and drop prefix
qapi/crypto: Rename QCryptoHashAlgorithm to *Algo, and drop prefix
qapi/crypto: Drop unwanted 'prefix'
qapi/machine: Rename CpuS390* to S390Cpu*, and drop 'prefix'
qapi/ui: Drop temporary 'prefix'
qapi/machine: Drop temporary 'prefix'
qapi/ebpf: Drop temporary 'prefix'
qapi/crypto: Drop temporary 'prefix'
qapi/common: Drop temporary 'prefix'
qapi/block-core: Drop temporary 'prefix'
tests/qapi-schema: Drop temporary 'prefix'
qapi: Smarter camel_to_upper() to reduce need for 'prefix'

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi/crypto: Rename QCryptoHashAlgorithm to *Algo, and drop prefix

QAPI's 'prefix' feature can make the connection between enumeration
type and its constants less than obvious. It's best used with

qapi/crypto: Rename QCryptoHashAlgorithm to *Algo, and drop prefix

QAPI's 'prefix' feature can make the connection between enumeration
type and its constants less than obvious. It's best used with
restraint.

QCryptoHashAlgorithm has a 'prefix' that overrides the generated
enumeration constants' prefix to QCRYPTO_HASH_ALG.

We could simply drop 'prefix', but then the prefix becomes
QCRYPTO_HASH_ALGORITHM, which is rather long.

We could additionally rename the type to QCryptoHashAlg, but I think
the abbreviation "alg" is less than clear.

Rename the type to QCryptoHashAlgo instead. The prefix becomes to
QCRYPTO_HASH_ALGO.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Acked-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240904111836.3273842-12-armbru@redhat.com>
[Conflicts with merge commit 7bbadc60b58b resolved]

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- configure: Enable -Wthread-safety if present
- no_co_wrapper to fix bdrv_open*() calls from coroutine co

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- configure: Enable -Wthread-safety if present
- no_co_wrapper to fix bdrv_open*() calls from coroutine context
- curl fixes, including enablement of newer libcurl versions
- MAINTAINERS: drop Vladimir from parallels block driver
- hbitmap: fix hbitmap_status() return value for first dirty bit case
- file-posix: Fix assertion failure in write_zeroes after moving
bdrv_getlength() to co_wrapper

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmPvgm0RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9ZxQg//ZWwwh/s/P1PnKAjInNZZNklAWKThNEbZ
# cF1S94w26IhEQqM0i6MflqcDsPU5t4xZtBUOizx++9M4G8amWnomJSdczUcKULla
# Az9yweFC1Gu6ENdw+ql5VOzCfpdH5Bn9Jkly5fxuI4vmnBz1PH1Dnd3P4wuLq2sL
# xna5dijVEhRc5mTKWjbp4nFfvQhucuEBPSNjgnZwEPbhciWxTMmB1GmyRvTxZy8v
# UY8PcoTlxdKeVQ6DTmkOirphpGj7HeNCEQnZppWs7vHys2oGi9kmR5qTKUNZxGrY
# 8yWiCiVDqbb50fhEC1srhph79bCij87QC1N33Bm+NuGjnjG4bKVx2B9DC8+6S/JS
# e3x6u+r0dd6/t0rjKnt1+inYqmM+i5lBJ7+R0yhWUQ+DYkvttNf5yiotD8qvccWJ
# Kcx14lfjPLK7siAMEY5K0bNMimhN4RR9oCLoPTOHei+vlxdfiMm2XPN61NNht5gD
# lYZ8JMBsEF/o2ebqTgsJrIHS+Q/8MqcwSunBc54fcXZoF+eiza3W2ArXLNfAEfGE
# U4JowNK2PrTIrpEjD+Vs0RsBBSmN5PcYIAz04ioODpDnYMq73/t3x9MKdVoxOT64
# AM7w58fSyWu8iwvkeA0d3XeVtSHFqZ49PqqIem4IegtnC/AXMUNrJ/VT99xHjeJY
# oLhOJz7LUg0=
# =FtaA
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 17 Feb 2023 13:34:37 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (22 commits)
hbitmap: fix hbitmap_status() return value for first dirty bit case
block/file-posix: don't use functions calling AIO_WAIT_WHILE in worker threads
MAINTAINERS: drop Vladimir from parallels block driver
block: temporarily hold the new AioContext of bs_top in bdrv_append()
block: Handle curl 7.55.0, 7.85.0 version changes
block: Assert non-coroutine context for bdrv_open_inherit()
block: Fix bdrv_co_create_opts_simple() to open images with no_co_wrapper
vpc: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
vmdk: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
vhdx: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
vdi: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
qed: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
qcow2: Fix open/create to open images with no_co_wrapper
qcow: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
parallels: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
luks: Fix .bdrv_co_create(_opts) to open images with no_co_wrapper
block: Create no_co_wrappers for open functions
block-coroutine-wrapper: Introduce no_co_wrapper
curl: Fix error path in curl_open()
configure: Enable -Wthread-safety if present
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

hbitmap: fix hbitmap_status() return value for first dirty bit case

The last return statement should return true, as we already evaluated that
start == next_dirty

Also, fix hbitmap_status() descrip

hbitmap: fix hbitmap_status() return value for first dirty bit case

The last return statement should return true, as we already evaluated that
start == next_dirty

Also, fix hbitmap_status() description in header

Cc: qemu-stable@nongnu.org
Fixes: a6426475a75 ("block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()")
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
Message-Id: <20230202181523.423131-1-andrey.zhadchenko@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of git://repo.or.cz/qemu/kevin into staging

Block layer patches

- Add vduse-blk export
- Dirty bitmaps: Fix and improve bitmap merge
- gluster: correctly set max_pdiscard
-

Merge tag 'for-upstream' of git://repo.or.cz/qemu/kevin into staging

Block layer patches

- Add vduse-blk export
- Dirty bitmaps: Fix and improve bitmap merge
- gluster: correctly set max_pdiscard
- rbd: report a better error when namespace does not exist
- aio_wait_kick: add missing memory barrier
- Code cleanups

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmK10yURHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9YnXg/+J9PKsGl7hbJUFVIb4RZ3pMQ0o4xC1TbS
# OjA63RRwdJWt1gGsgLNH6kdsgz04FXiXk7XNDTe8MYn21bvznCYGFsI2EFJiit+x
# HkOYXJwAGsrfKBViq8QznosBQbwf9fOrDUlOdbnixFH/dx6DXUZM2F9ud5shiuiC
# vrP76zjUgSlHhPwL+0xsJSNxxNISv845uN9SYNpiGbZDaYLBtNbEc8NA2GMOYW45
# sCbxJDxNSFNrTT5gaZyo4KGFH9393qJSmYHJ+q53elbhje5dpmcn2c/AkIG7/UM4
# HsEWd7Gj1YbLQV6kvkefJzF4BOuZmN/b8sG+LRDjrEMHLeOw5rh2ETmkQfIdy2H3
# 2U52mHqw9bMGWtds0ocOJFKw3vPZuOCJaA+ql/A0aWGO9Fjivx1cQcOVXn3jKgj5
# 7riSxigK9TCTd92Qhi3j4do86mJi9NpjgawbQ5oOvKqUiq29moOwcv1hit0WVzwB
# hJp2qsrM6w00wFseyp+10vHigW7ZMNEIh4TKYLfZ15Gvy6YIG/m/oyCYF4oVM5y5
# knbsnJBRpTjLnggag86Pk8Ji+Lxi0cbv7zrbn++/S9OLQ5VMV5OpxKXHmyX2HyXa
# RCV0rMaTC3s4hy8gElDSwSeor6NBaJ9TECcit67uJ/kw1eK+omoa4hKMCgKf6qYC
# GY+NRPELJGA=
# =OuiN
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 24 Jun 2022 08:07:17 AM PDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of git://repo.or.cz/qemu/kevin:
vduse-blk: Add name option
vduse-blk: Add serial option
nbd: Drop dead code spotted by Coverity
aio_wait_kick: add missing memory barrier
block/gluster: correctly set max_pdiscard
block/rbd: report a better error when namespace does not exist
qsd: document vduse-blk exports
libvduse: Add support for reconnecting
vduse-blk: Add vduse-blk resize support
vduse-blk: Implement vduse-blk export
libvduse: Add VDUSE (vDPA Device in Userspace) library
linux-headers: Add vduse.h
block/export: Abstract out the logic of virtio-blk I/O process
block/export: Fix incorrect length passed to vu_queue_push()
block: Support passing NULL ops to blk_set_dev_ops()
block: simplify handling of try to merge different sized bitmaps
block: improve block_dirty_bitmap_merge(): don't allocate extra bitmap
block: block_dirty_bitmap_merge(): fix error path
block: get rid of blk->guest_block_size
block: drop unused bdrv_co_drain() API

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block: simplify handling of try to merge different sized bitmaps

We have too much logic to simply check that bitmaps are of the same
size. Let's just define that hbitmap_merge() and
bdrv_dirty_bitma

block: simplify handling of try to merge different sized bitmaps

We have too much logic to simply check that bitmaps are of the same
size. Let's just define that hbitmap_merge() and
bdrv_dirty_bitmap_merge_internal() require their argument bitmaps be of
same size, this simplifies things.

Let's look through the callers:

For backup_init_bcs_bitmap() we already assert that merge can't fail.

In bdrv_reclaim_dirty_bitmap_locked() we gracefully handle the error
that can't happen: successor always has same size as its parent, drop
this logic.

In bdrv_merge_dirty_bitmap() we already has assertion and separate
check. Make the check explicit and improve error message.

Signed-off-by: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220517111206.23585-4-v.sementsov-og@mail.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'pull-misc-2022-03-21' of git://repo.or.cz/qemu/armbru into staging

Miscellaneous patches patches for 2022-03-21

# gpg: Signature made Mon 21 Mar 2022 14:48:16 GMT
# gpg: u

Merge tag 'pull-misc-2022-03-21' of git://repo.or.cz/qemu/armbru into staging

Miscellaneous patches patches for 2022-03-21

# gpg: Signature made Mon 21 Mar 2022 14:48:16 GMT
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-misc-2022-03-21' of git://repo.or.cz/qemu/armbru:
Use g_new() & friends where that makes obvious sense
9pfs: Use g_new() & friends where that makes obvious sense
scripts/coccinelle: New use-g_new-etc.cocci
block-qdict: Fix -Werror=maybe-uninitialized build failure

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Use g_new() & friends where that makes obvious sense

g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer,
for two reasons. One, it catches multiplication overflowing size_t.
Two, i

Use g_new() & friends where that makes obvious sense

g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer,
for two reasons. One, it catches multiplication overflowing size_t.
Two, it returns T * rather than void *, which lets the compiler catch
more type errors.

This commit only touches allocations with size arguments of the form
sizeof(T).

Patch created mechanically with:

$ spatch --in-place --sp-file scripts/coccinelle/use-g_new-etc.cocci \
--macro-file scripts/cocci-macro-file.h FILES...

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20220315144156.1595462-4-armbru@redhat.com>
Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>

show more ...

Merge remote-tracking branch 'remotes/hreitz-gitlab/tags/pull-block-2022-03-07' into staging

Block patches for 7.0-rc0:
- New fleecing backup scheme
- iotest fixes
- Fixes for the curl block driver

Merge remote-tracking branch 'remotes/hreitz-gitlab/tags/pull-block-2022-03-07' into staging

Block patches for 7.0-rc0:
- New fleecing backup scheme
- iotest fixes
- Fixes for the curl block driver
- Fix for the preallocate block driver
- IDE fix for zero-length TRIM requests

# gpg: Signature made Mon 07 Mar 2022 10:33:31 GMT
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF

* remotes/hreitz-gitlab/tags/pull-block-2022-03-07: (23 commits)
iotests/image-fleecing: test push backup with fleecing
iotests/image-fleecing: add test case with bitmap
iotests.py: add qemu_io_pipe_and_status()
iotests/image-fleecing: add test-case for fleecing format node
block: copy-before-write: realize snapshot-access API
block: introduce snapshot-access block driver
block/io: introduce block driver snapshot-access API
block/reqlist: add reqlist_wait_all()
block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
block/reqlist: reqlist_find_conflict(): use ranges_overlap()
block: intoduce reqlist
block/block-copy: add block_copy_reset()
block/copy-before-write: add bitmap open parameter
block/block-copy: block_copy_state_new(): add bitmap parameter
block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
block/block-copy: move copy_bitmap initialization to block_copy_state_new()
iotests: Write test output to TEST_DIR
tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode
tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
block: fix preallocate filter: don't do unaligned preallocate requests
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()

Add a convenient function similar with bdrv_block_status() to get
status of dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vseme

block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()

Add a convenient function similar with bdrv_block_status() to get
status of dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-9-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

# gpg: Signature made Wed 18 Mar 2020 20:23:28 GMT
# gpg: using RSA key F9B7ABDBBCAC

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

# gpg: Signature made Wed 18 Mar 2020 20:23:28 GMT
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/bitmaps-pull-request:
block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty
nbd/server: use bdrv_dirty_bitmap_next_dirty_area
nbd/server: introduce NBDExtentArray
block/dirty-bitmap: improve _next_dirty_area API
block/dirty-bitmap: add _next_dirty API
block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t
hbitmap: drop meta bitmaps as they are unused
hbitmap: unpublish hbitmap_iter_skip_words
hbitmap: move hbitmap_iter_next_word to hbitmap.c
hbitmap: assert that we don't create bitmap larger than INT64_MAX
build: Silence clang warning on older glib autoptr usage

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/dirty-bitmap: improve _next_dirty_area API

Firstly, _next_dirty_area is for scenarios when we may contiguously
search for next dirty area inside some limited region, so it is more
comfortable

block/dirty-bitmap: improve _next_dirty_area API

Firstly, _next_dirty_area is for scenarios when we may contiguously
search for next dirty area inside some limited region, so it is more
comfortable to specify "end" which should not be recalculated on each
iteration.

Secondly, let's add a possibility to limit resulting area size, not
limiting searching area. This will be used in NBD code in further
commit. (Note that now bdrv_dirty_bitmap_next_dirty_area is unused)

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-8-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

block/dirty-bitmap: add _next_dirty API

We have bdrv_dirty_bitmap_next_zero, let's add corresponding
bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than
bitmap iterators in some case

block/dirty-bitmap: add _next_dirty API

We have bdrv_dirty_bitmap_next_zero, let's add corresponding
bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than
bitmap iterators in some cases.

For test modify test_hbitmap_next_zero_check_range to check both
next_zero and next_dirty and add some new checks.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-7-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t

We are going to introduce bdrv_dirty_bitmap_next_dirty so that same
variable may be used to store its return value and to be its

block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t

We are going to introduce bdrv_dirty_bitmap_next_dirty so that same
variable may be used to store its return value and to be its parameter,
so it would int64_t.

Similarly, we are going to refactor hbitmap_next_dirty_area to use
hbitmap_next_dirty together with hbitmap_next_zero, therefore we want
hbitmap_next_zero parameter type to be int64_t too.

So, for convenience update all parameters of *_next_zero and
*_next_dirty_area to be int64_t.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-6-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

hbitmap: drop meta bitmaps as they are unused

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.c

hbitmap: drop meta bitmaps as they are unused

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-5-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

hbitmap: unpublish hbitmap_iter_skip_words

Function is internal and even commented as internal. Drop its
definition from .h file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.c

hbitmap: unpublish hbitmap_iter_skip_words

Function is internal and even commented as internal. Drop its
definition from .h file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-4-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

hbitmap: move hbitmap_iter_next_word to hbitmap.c

The function is definitely internal (it's not used by third party and
it has complicated interface). Move it to .c file.

Signed-off-by: Vladimir Se

hbitmap: move hbitmap_iter_next_word to hbitmap.c

The function is definitely internal (it's not used by third party and
it has complicated interface). Move it to .c file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-3-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

hbitmap: assert that we don't create bitmap larger than INT64_MAX

We have APIs which returns signed int64_t, to be able to return error.
Therefore we can't handle bitmaps with absolute size larger t

hbitmap: assert that we don't create bitmap larger than INT64_MAX

We have APIs which returns signed int64_t, to be able to return error.
Therefore we can't handle bitmaps with absolute size larger than
(INT64_MAX+1). Still, keep maximum to be INT64_MAX which is a bit
safer.

Note, that bitmaps are used to represent disk images, which can't
exceed INT64_MAX anyway.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-2-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' into staging

Block patches for softfreeze:
- iotest patches
- Improve performance of the mirror block job in write-blocking

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' into staging

Block patches for softfreeze:
- iotest patches
- Improve performance of the mirror block job in write-blocking mode
- Limit memory usage for the backup block job
- Add discard and write-zeroes support to the NVMe host block driver
- Fix a bug in the mirror job
- Prevent the qcow2 driver from creating technically non-compliant qcow2
v3 images (where there is not enough extra data for snapshot table
entries)
- Allow callers of bdrv_truncate() (etc.) to determine whether the file
must be resized to the exact given size or whether it is OK for block
devices not to shrink

# gpg: Signature made Mon 28 Oct 2019 12:13:53 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-10-28: (69 commits)
qemu-iotests: restrict 264 to qcow2 only
Revert "qemu-img: Check post-truncation size"
block: Pass truncate exact=true where reasonable
block: Let format drivers pass @exact
block: Evaluate @exact in protocol drivers
block: Add @exact parameter to bdrv_co_truncate()
block: Do not truncate file node when formatting
block/cor: Drop cor_co_truncate()
block: Handle filter truncation like native impl.
iotests: Test qcow2's snapshot table handling
iotests: Add peek_file* functions
qcow2: Fix v3 snapshot table entry compliancy
qcow2: Repair snapshot table with too many entries
qcow2: Fix overly long snapshot tables
qcow2: Keep track of the snapshot table length
qcow2: Fix broken snapshot table entries
qcow2: Add qcow2_check_fix_snapshot_table()
qcow2: Separate qcow2_check_read_snapshot_table()
qcow2: Write v3-compliant snapshot list on upgrade
qcow2: Put qcow2_upgrade() into its own function
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

hbitmap: handle set/reset with zero length

Passing zero length to these functions leads to unpredicted results.
Zero-length set/reset may occur in active-mirror, on zero-length write
(which is unlik

hbitmap: handle set/reset with zero length

Passing zero length to these functions leads to unpredicted results.
Zero-length set/reset may occur in active-mirror, on zero-length write
(which is unlikely, but not guaranteed to never happen).

Let's just do nothing on zero-length request.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191011090711.19940-2-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

pull request

# gpg: Signature made Thu 17 Oct 2019 22:54:14 BST
# gpg: using RSA key F9B7ABDBBCAC

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

pull request

# gpg: Signature made Thu 17 Oct 2019 22:54:14 BST
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/bitmaps-pull-request:
dirty-bitmaps: remove deprecated autoload parameter
MAINTAINERS: Add Vladimir as a reviewer for bitmaps
qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit
block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw
iotests: add test 260 to check bitmap life after snapshot + commit
block/qcow2-bitmap: do not remove bitmaps on reopen-ro
block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint()
block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps
iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW
block: reverse order for reopen commits
block: switch reopen queue from QSIMPLEQ to QTAILQ
block/dirty-bitmap: refactor bdrv_dirty_bitmap_next
block/dirty-bitmap: drop BdrvDirtyBitmap.mutex
block/dirty-bitmap: add bs link
block/dirty-bitmap: drop meta
block/qcow2: proper locking on bitmap add/remove paths
block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap
block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c
util/hbitmap: strict hbitmap_reset

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

util/hbitmap: strict hbitmap_reset

hbitmap_reset has an unobvious property: it rounds requested region up.
It may provoke bugs, like in recently fixed write-blocking mode of
mirror: user calls reset

util/hbitmap: strict hbitmap_reset

hbitmap_reset has an unobvious property: it rounds requested region up.
It may provoke bugs, like in recently fixed write-blocking mode of
mirror: user calls reset on unaligned region, not keeping in mind that
there are possible unrelated dirty bytes, covered by rounded-up region
and information of this unrelated "dirtiness" will be lost.

Make hbitmap_reset strict: assert that arguments are aligned, allowing
only one exception when @start + @count == hb->orig_size. It's needed
to comfort users of hbitmap_next_dirty_area, which cares about
hb->orig_size.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20190806152611.280389-1-vsementsov@virtuozzo.com>
[Maintainer edit: Max's suggestions from on-list. --js]
[Maintainer edit: Eric's suggestion for aligned macro. --js]
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

Rebase notes:

011/36:[0003] [FC] 'block/backup: upgrade copy_bitmap to BdrvDirtyBitmap'
016/36:[--

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

Rebase notes:

011/36:[0003] [FC] 'block/backup: upgrade copy_bitmap to BdrvDirtyBitmap'
016/36:[----] [-C] 'iotests: Add virtio-scsi device helper'
017/36:[0002] [FC] 'iotests: add test 257 for bitmap-mode backups'
030/36:[0011] [FC] 'block/backup: teach TOP to never copy unallocated regions'
032/36:[0018] [FC] 'iotests/257: test traditional sync modes'

11: A new hbitmap call was added late in 4.1, changed to
bdrv_dirty_bitmap_next_zero.
16: Context-only (self.has_quit is new context in 040)
17: Removed 'auto' to follow upstream trends in iotest fashion
30: Handled explicitly on-list with R-B from Max.
32: Fix capitalization in test, as mentioned on-list.

# gpg: Signature made Sat 17 Aug 2019 00:12:13 BST
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/bitmaps-pull-request: (36 commits)
tests/test-hbitmap: test next_zero and _next_dirty_area after truncate
block/backup: refactor write_flags
block/backup: deal with zero detection
qapi: add dirty-bitmaps to query-named-block-nodes result
iotests/257: test traditional sync modes
block/backup: support bitmap sync modes for non-bitmap backups
block/backup: teach TOP to never copy unallocated regions
block/backup: add backup_is_cluster_allocated
block/backup: centralize copy_bitmap initialization
block/backup: improve sync=bitmap work estimates
iotests/257: test API failures
block/backup: hoist bitmap check into QMP interface
iotests/257: Refactor backup helpers
iotests/257: add EmulatedBitmap class
iotests/257: add Pattern class
iotests: test bitmap moving inside 254
qapi: implement block-dirty-bitmap-remove transaction action
blockdev: reduce aio_context locked sections in bitmap add/remove
block/backup: loosen restriction on readonly bitmaps
iotests: add test 257 for bitmap-mode backups
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

hbitmap: enable merging across granularities

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-9-jsnow@redhat.com
Signed-off-by:

hbitmap: enable merging across granularities

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

hbitmap: Fix merge when b is empty, and result is not an alias of a

Nobody calls the function like this currently, but we neither prohibit
or cope with this behavior. I decided to make the function

hbitmap: Fix merge when b is empty, and result is not an alias of a

Nobody calls the function like this currently, but we neither prohibit
or cope with this behavior. I decided to make the function cope with it.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...