Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
re

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
``-vnc acl`` (since 4.0.0)
``-mon ...,control=readline,pretty=on|off`` (since 4.1)
``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
``query-cpus`` (since 2.12.0)
``query-cpus-fast`` ``arch`` output member (since 3.0.0)
``query-events`` (since 4.0)
chardev client socket with ``wait`` option (since 4.0)
``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
``ide-drive`` (since 4.2)
``scsi-disk`` (since 4.2)

# gpg: Signature made Thu 18 Mar 2021 09:23:39 GMT
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/dep-many-pull-request:
block: remove support for using "file" driver with block/char devices
block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
block: remove dirty bitmaps 'status' field
block: remove 'encryption_key_missing' flag from QAPI
hw/scsi: remove 'scsi-disk' device
hw/ide: remove 'ide-drive' device
chardev: reject use of 'wait' flag for socket client chardevs
machine: remove 'arch' field from 'query-cpus-fast' QMP command
machine: remove 'query-cpus' QMP command
migrate: remove QMP/HMP commands for speed, downtime and cache size
monitor: remove 'query-events' QMP command
monitor: raise error when 'pretty' option is used with HMP
ui, monitor: remove deprecated VNC ACL option and HMP commands

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

migrate: remove QMP/HMP commands for speed, downtime and cache size

The generic 'migrate_set_parameters' command handle all types of param.

Only the QMP commands were documented in the deprecations

migrate: remove QMP/HMP commands for speed, downtime and cache size

The generic 'migrate_set_parameters' command handle all types of param.

Only the QMP commands were documented in the deprecations page, but the
rationale for deprecating applies equally to HMP, and the replacements
exist. Furthermore the HMP commands are just shims to the QMP commands,
so removing the latter breaks the former unless they get re-implemented.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200617a' into staging

Migration (and HMP and virtiofs) pull 2020-06-17

Migration:
HMP/migration and test changes from Mao Zho

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200617a' into staging

Migration (and HMP and virtiofs) pull 2020-06-17

Migration:
HMP/migration and test changes from Mao Zhongyi
multifd fix from Laurent Vivier
HMP
qom-set partial reversion/change from David Hildenbrand
now you need -j to pass json format, but it's regained the
old 100M type format.
Memory leak fix from Pan Nengyuan

Virtiofs
fchmod seccomp fix from Max Reitz

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

# gpg: Signature made Wed 17 Jun 2020 19:34:58 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20200617a:
migration: fix multifd_send_pages() next channel
docs/xbzrle: update 'cache miss rate' and 'encoding rate' to docs
monitor/hmp-cmds: improvements for the 'info migrate'
monitor/hmp-cmds: add 'goto end' to reduce duplicate code.
monitor/hmp-cmds: delete redundant Error check before invoke hmp_handle_error()
monitor/hmp-cmds: don't silently output when running 'migrate_set_downtime' fails
monitor/hmp-cmds: add units for migrate_parameters
tests/migration: fix unreachable path in stress test
tests/migration: mem leak fix
hmp: Make json format optional for qom-set
qom-hmp-cmds: fix a memleak in hmp_qom_get
virtiofsd: Whitelist fchmod

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

docs/xbzrle: update 'cache miss rate' and 'encoding rate' to docs

Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id: <20200603080904.997083-9-maozhongyi@cmss.chinamobile.com>
R

docs/xbzrle: update 'cache miss rate' and 'encoding rate' to docs

Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id: <20200603080904.997083-9-maozhongyi@cmss.chinamobile.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

show more ...

monitor/hmp-cmds: improvements for the 'info migrate'

When running:

(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
...
xbzrle transferred: 640892 kbytes
xbzrle pages: 1664

monitor/hmp-cmds: improvements for the 'info migrate'

When running:

(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
...
xbzrle transferred: 640892 kbytes
xbzrle pages: 16645936 pages
xbzrle cache miss: 1525426
xbzrle cache miss rate: 0.09
xbzrle encoding rate: 91.42
xbzrle overflow: 40896
...
compression pages: 377710 pages
compression busy: 0
compression busy rate: 0.00
compressed size: 463169457
compression rate: 3.33

Add units for 'xbzrle cache miss' and 'compressed size',
make it easier to read.

Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id: <20200603080904.997083-8-maozhongyi@cmss.chinamobile.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200325b' into staging

Combo Migration/HMP/virtiofs pull

Small fixes all around.
Ones that are noticeable:
a) Igor's migration

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200325b' into staging

Combo Migration/HMP/virtiofs pull

Small fixes all around.
Ones that are noticeable:
a) Igor's migration compatibility fix affecting older machine types
has been seen in the wild
b) Philippe's autconverge fix should fix an intermittently
failing migration test.
c) Mao's makes a small change to the output of 'info
migrate_parameters' for tls-authz.

# gpg: Signature made Wed 25 Mar 2020 13:14:48 GMT
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20200325b:
migration: use "" instead of (null) for tls-authz
migration/ram: fix use after free of local_err
migration/colo: fix use after free of local_err
vl.c: fix migration failure for 3.1 and older machine types
tools/virtiofsd/passthrough_ll: Fix double close()
hmp/vnc: Fix info vnc list leak
tests/migration: Reduce autoconverge initial bandwidth
xbzrle: update xbzrle doc
hmp-cmd: fix a missing_break warning

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

xbzrle: update xbzrle doc

Add new parameter description, also:
1. Remove unsociable space.
2. Nit picking: s/two/2 in report

Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id:

xbzrle: update xbzrle doc

Add new parameter description, also:
1. Remove unsociable space.
2. Nit picking: s/two/2 in report

Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id: <20200320143216.423374-1-maozhongyi@cmss.chinamobile.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20161014' into staging

migration/next for 20161014

# gpg: Signature made Fri 14 Oct 2016 16:24:13 BST
# gpg: using R

Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20161014' into staging

migration/next for 20161014

# gpg: Signature made Fri 14 Oct 2016 16:24:13 BST
# gpg: using RSA key 0xF487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg: aka "Juan Quintela <quintela@trasno.org>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration/20161014:
docs/xbzrle: correction
migrate: move max-bandwidth and downtime-limit to migrate_set_parameter
migration: Fix seg with missing port
migration/postcopy: Explicitly disallow huge pages
RAMBlocks: Store page size
Postcopy vs xbzrle: Don't send xbzrle pages once in postcopy [for 2.8]
migrate: Fix bounds check for migration parameters in migration.c
migrate: Use boxed qapi for migrate-set-parameters
migrate: Share common MigrationParameters struct
migrate: Fix cpu-throttle-increment regression in HMP
migration/rdma: Don't flag an error when we've been told about one
migration: Make failed migration load set file error
migration/rdma: Pass qemu_file errors across link
migration: Report values for comparisons
migration: report an error giving the failed field

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

docs/xbzrle: correction

1. Default cache size is 64MB.
2. Semantics correction.

Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Jua

docs/xbzrle: correction

1. Default cache size is 64MB.
2. Semantics correction.

Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/amit-migration/tags/mig-2.3-1' into staging

A set of patches collected over the holidays. Mix of optimizations and
fixes.

# gpg: Signature made Fri 16 Jan 201

Merge remote-tracking branch 'remotes/amit-migration/tags/mig-2.3-1' into staging

A set of patches collected over the holidays. Mix of optimizations and
fixes.

# gpg: Signature made Fri 16 Jan 2015 07:42:00 GMT using RSA key ID 854083B6
# gpg: Good signature from "Amit Shah <amit@amitshah.net>"
# gpg: aka "Amit Shah <amit@kernel.org>"
# gpg: aka "Amit Shah <amitshah@gmx.net>"

* remotes/amit-migration/tags/mig-2.3-1:
vmstate: type-check sub-arrays
migration_cancel: shutdown migration socket
Handle bi-directional communication for fd migration
socket shutdown
Tests: QEMUSizedBuffer/QEMUBuffer
QEMUSizedBuffer: only free qsb that qemu_bufopen allocated
xbzrle: rebuild the cache_is_cached function
xbzrle: optimize XBZRLE to decrease the cache misses

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

xbzrle: optimize XBZRLE to decrease the cache misses

Avoid hot pages being replaced by others to remarkably decrease cache
misses

Sample results with the test program which quote from xbzrle.txt ra

xbzrle: optimize XBZRLE to decrease the cache misses

Avoid hot pages being replaced by others to remarkably decrease cache
misses

Sample results with the test program which quote from xbzrle.txt ran in
vm:(migrate bandwidth:1GE and xbzrle cache size 8MB)

the test program:

include <stdlib.h>
include <stdio.h>
int main()
{
char *buf = (char *) calloc(4096, 4096);
while (1) {
int i;
for (i = 0; i < 4096 * 4; i++) {
buf[i * 4096 / 4]++;
}
printf(".");
}
}

before this patch:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":1020,"xbzrle-cache":{"bytes":1108284,
"cache-size":8388608,"cache-miss-rate":0.987013,"pages":18297,"overflow":8,
"cache-miss":1228737},"status":"active","setup-time":10,"total-time":52398,
"ram":{"total":12466991104,"remaining":1695744,"mbps":935.559472,
"transferred":5780760580,"dirty-sync-counter":271,"duplicate":2878530,
"dirty-pages-rate":29130,"skipped":0,"normal-bytes":5748592640,
"normal":1403465}},"id":"libvirt-706"}

18k pages sent compressed in 52 seconds.
cache-miss-rate is 98.7%, totally miss.

after optimizing:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":2054,"xbzrle-cache":{"bytes":5066763,
"cache-size":8388608,"cache-miss-rate":0.485924,"pages":194823,"overflow":0,
"cache-miss":210653},"status":"active","setup-time":11,"total-time":18729,
"ram":{"total":12466991104,"remaining":3895296,"mbps":937.663549,
"transferred":1615042219,"dirty-sync-counter":98,"duplicate":2869840,
"dirty-pages-rate":58781,"skipped":0,"normal-bytes":1588404224,
"normal":387794}},"id":"libvirt-266"}

194k pages sent compressed in 18 seconds.
The value of cache-miss-rate decrease to 48.59%.

Signed-off-by: ChenLiang <chenliang88@huawei.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>

show more ...

Merge remote-tracking branch 'quintela/migration-next-20120808' into staging

* quintela/migration-next-20120808:
Restart optimization on stage3 update version
Add XBZRLE statistics
Add migrati

Merge remote-tracking branch 'quintela/migration-next-20120808' into staging

* quintela/migration-next-20120808:
Restart optimization on stage3 update version
Add XBZRLE statistics
Add migration accounting for normal and duplicate pages
Change total_time to total-time in MigrationStats
Add migrate_set_cache_size command
Add XBZRLE to ram_save_block and ram_save_live
Add xbzrle_encode_buffer and xbzrle_decode_buffer functions
Add uleb encoding/decoding functions
Add cache handling functions
Add XBZRLE documentation
Add migrate-set-capabilities
Add migration capabilities

show more ...

Add XBZRLE documentation

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>