Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* KVM: synic support, split irqchip support
* memory: cleanups, optimizations, ioeventfd emulation
* SCSI: small fixes,

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* KVM: synic support, split irqchip support
* memory: cleanups, optimizations, ioeventfd emulation
* SCSI: small fixes, vmw_pvscsi compatibility improvements
* qemu_log cleanups
* Coverity model improvements

# gpg: Signature made Thu 17 Dec 2015 16:35:21 GMT using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (45 commits)
coverity: Model g_memdup()
coverity: Model g_poll()
scsi: always call notifier on async cancellation
scsi: use scsi_req_cancel_async when purging requests
target-i386: kvm: clear unusable segments' flags in migration
rcu: optimize rcu_read_lock
memory: try to inline constant-length reads
memory: inline a few small accessors
memory: extract first iteration of address_space_read and address_space_write
memory: split address_space_read and address_space_write
memory: avoid unnecessary object_ref/unref
memory: reorder MemoryRegion fields
exec: make qemu_ram_ptr_length more similar to qemu_get_ram_ptr
exec: always call qemu_get_ram_ptr within rcu_read_lock
linux-user: convert DEBUG_SIGNAL logging to tracepoints
linux-user: avoid "naked" qemu_log
user: introduce "-d page"
xtensa: avoid "naked" qemu_log
tricore: avoid "naked" qemu_log
ppc: cleanup logging
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

linux-user: convert DEBUG_SIGNAL logging to tracepoints

"Unimplemented" messages go to stderr, everything else goes to tracepoints

Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Pa

linux-user: convert DEBUG_SIGNAL logging to tracepoints

"Unimplemented" messages go to stderr, everything else goes to tracepoints

Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

alpha: convert "naked" qemu_log to tracepoint

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Merge remote-tracking branch 'remotes/kraxel/tags/pull-fw-cfg-20151217-1' into staging

fw_cfg: doc updates, various optimizations.

# gpg: Signature made Thu 17 Dec 2015 08:59:32 GMT using RSA key I

Merge remote-tracking branch 'remotes/kraxel/tags/pull-fw-cfg-20151217-1' into staging

fw_cfg: doc updates, various optimizations.

# gpg: Signature made Thu 17 Dec 2015 08:59:32 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-fw-cfg-20151217-1:
fw_cfg: replace ioport data read with generic method
fw_cfg: add generic non-DMA read method
fw_cfg: avoid calculating invalid current entry pointer
fw_cfg: remove offset argument from callback prototype
fw_cfg: amend callback behavior spec to once per select
fw_cfg: move internal function call docs to header file

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/kraxel/tags/pull-usb-20151215-1' into staging

usb: ehci idt fix, event support for mtp

# gpg: Signature made Tue 15 Dec 2015 09:54:22 GMT using RSA key ID D3E8

Merge remote-tracking branch 'remotes/kraxel/tags/pull-usb-20151215-1' into staging

usb: ehci idt fix, event support for mtp

# gpg: Signature made Tue 15 Dec 2015 09:54:22 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-usb-20151215-1:
ehci: make idt processing more robust
usb-mtp: add support for basic mtp events
usb-mtp: Add support for inotify based file monitoring
usb-mtp: free objects on a mtp reset
usb-mtp: use a list for keeping track of children

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

fw_cfg: add generic non-DMA read method

Introduce fw_cfg_data_read(), a generic read method which works
on all access widths (1 through 8 bytes, inclusive), and can be
used during both IOPort and MM

fw_cfg: add generic non-DMA read method

Introduce fw_cfg_data_read(), a generic read method which works
on all access widths (1 through 8 bytes, inclusive), and can be
used during both IOPort and MMIO read accesses.

To maintain legibility, only fw_cfg_data_mem_read() (the MMIO
data read method) is replaced by this patch. The new method
essentially unwinds the fw_cfg_data_mem_read() + fw_cfg_read()
combo, but without unnecessarily repeating all the validity
checks performed by the latter on each byte being read.

This patch also modifies the trace_fw_cfg_read prototype to
accept a 64-bit value argument, allowing it to work properly
with the new read method, but also remain backward compatible
with existing call sites.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Marc Marí <markmb@redhat.com>
Signed-off-by: Gabriel Somlo <somlo@cmu.edu>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-id: 1446733972-1602-6-git-send-email-somlo@cmu.edu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

show more ...

usb-mtp: Add support for inotify based file monitoring

For now, we use inotify watches to track only a small number of
events, namely, add, delete and modify. Note that for delete, the kernel
alread

usb-mtp: Add support for inotify based file monitoring

For now, we use inotify watches to track only a small number of
events, namely, add, delete and modify. Note that for delete, the kernel
already deactivates the watch for us and we just need to
take care of modifying our internal state.

inotify is a linux only mechanism.

Suggested-by: Gerd Hoffman <kraxel@redhat.com>
Signed-off-by: Bandan Das <bsd@redhat.com>
Message-id: 1448314625-3855-4-git-send-email-bsd@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

show more ...

usb-mtp: use a list for keeping track of children

To support adding/removal of objects, we will need to update
the object cache hierarchy we have built internally. Convert
to using a Qlist for easie

usb-mtp: use a list for keeping track of children

To support adding/removal of objects, we will need to update
the object cache hierarchy we have built internally. Convert
to using a Qlist for easier management.

Signed-off-by: Bandan Das <bsd@redhat.com>
Message-id: 1448314625-3855-2-git-send-email-bsd@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Thu 03 Dec 2015 04:59:48 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan H

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Thu 03 Dec 2015 04:59:48 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
iotests: Add regresion test case for write notifier assertion failure
iotests: Add "add_drive_raw" method
block: Don't wait serialising for non-COR read requests
iothread: include id in thread name

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Don't wait serialising for non-COR read requests

The assertion problem was noticed in 06c3916b35a, but it wasn't
completely fixed, because even though the req is not marked as
serialising, it

block: Don't wait serialising for non-COR read requests

The assertion problem was noticed in 06c3916b35a, but it wasn't
completely fixed, because even though the req is not marked as
serialising, it still gets serialised by wait_serialising_requests
against other serialising requests, which could lead to the same
assertion failure.

Fix it by even more explicitly skipping the serialising for this
specific case.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1448962590-2842-2-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kraxel/tags/pull-vnc-20151116-1' into staging

vnc: buffer code improvements, bugfixes.

# gpg: Signature made Mon 16 Nov 2015 17:20:02 GMT using RSA key ID D3E8

Merge remote-tracking branch 'remotes/kraxel/tags/pull-vnc-20151116-1' into staging

vnc: buffer code improvements, bugfixes.

# gpg: Signature made Mon 16 Nov 2015 17:20:02 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-vnc-20151116-1:
vnc: fix mismerge
buffer: allow a buffer to shrink gracefully
buffer: factor out buffer_adj_size
buffer: factor out buffer_req_size
vnc: recycle empty vs->output buffer
vnc: fix local state init
vnc: only alloc server surface with clients connected
vnc: use vnc_{width,height} in vnc_set_area_dirty
vnc: factor out vnc_update_server_surface
vnc: add vnc_width+vnc_height helpers
vnc: zap dead code
vnc-jobs: move buffer reset, use new buffer move
vnc: kill jobs queue buffer
vnc: attach names to buffers
buffer: add tracing
buffer: add buffer_shrink
buffer: add buffer_move
buffer: add buffer_move_empty
buffer: add buffer_init
buffer: make the Buffer capacity increase in powers of two

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20151110' into staging

migration/next for 20151110

# gpg: Signature made Tue 10 Nov 2015 14:23:26 GMT using RSA key ID 5872D723
# g

Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20151110' into staging

migration/next for 20151110

# gpg: Signature made Tue 10 Nov 2015 14:23:26 GMT using RSA key ID 5872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg: aka "Juan Quintela <quintela@trasno.org>"

* remotes/juanquintela/tags/migration/20151110: (57 commits)
migration: qemu_savevm_state_cleanup becomes mandatory operation
Inhibit ballooning during postcopy
Disable mlock around incoming postcopy
End of migration for postcopy
Postcopy: Mark nohugepage before discard
postcopy: Wire up loadvm_postcopy_handle_ commands
Start up a postcopy/listener thread ready for incoming page data
Postcopy; Handle userfault requests
Round up RAMBlock sizes to host page sizes
Host page!=target page: Cleanup bitmaps
Don't iterate on precopy-only devices during postcopy
Don't sync dirty bitmaps in postcopy
postcopy: Check order of received target pages
Postcopy: Use helpers to map pages during migration
postcopy_ram.c: place_page and helpers
Page request: Consume pages off the post-copy queue
Page request: Process incoming page request
Page request: Add MIG_RP_MSG_REQ_PAGES reverse command
Postcopy: End of iteration
Postcopy: Postcopy startup in migration thread
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

End of migration for postcopy

Tweak the end of migration cleanup; we don't want to close stuff down
at the end of the main stream, since the postcopy is still sending pages
on the other thread.

Sig

End of migration for postcopy

Tweak the end of migration cleanup; we don't want to close stuff down
at the end of the main stream, since the postcopy is still sending pages
on the other thread.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Postcopy: Mark nohugepage before discard

Prior to servicing userfault requests we must ensure we've not got
huge pages in the area that might include non-transferred memory,
since a hugepage could i

Postcopy: Mark nohugepage before discard

Prior to servicing userfault requests we must ensure we've not got
huge pages in the area that might include non-transferred memory,
since a hugepage could incorrectly mark the whole huge page as present.

We mark the area as non-huge page (nhp) just before we perform
discards; the discard code now tells us to discard any areas
that haven't been sent (as well as any that are redirtied);
any already formed transparent-huge-pages get fragmented
by this discard process if they cotnain any discards.

Transparent huge pages that have been entirely transferred
and don't contain any discards are not broken by this mechanism;
they stay as huge pages.

By starting postcopy after a full precopy pass, many of the pages
then stay as huge pages; this is important for maintaining performance
after the end of the migration.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

postcopy: Wire up loadvm_postcopy_handle_ commands

Wire up more of the handlers for the commands on the destination side,
in particular loadvm_postcopy_handle_run now has enough to start the
guest r

postcopy: Wire up loadvm_postcopy_handle_ commands

Wire up more of the handlers for the commands on the destination side,
in particular loadvm_postcopy_handle_run now has enough to start the
guest running.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Start up a postcopy/listener thread ready for incoming page data

The loading of a device state (during postcopy) may access guest
memory that's still on the source machine and thus might need
a page

Start up a postcopy/listener thread ready for incoming page data

The loading of a device state (during postcopy) may access guest
memory that's still on the source machine and thus might need
a page fill; split off a separate thread that handles the incoming
page data so that the original incoming migration code can finish
off the device data.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Postcopy; Handle userfault requests

userfaultfd is a Linux syscall that gives an fd that receives a stream
of notifications of accesses to pages registered with it and allows
the program to acknowle

Postcopy; Handle userfault requests

userfaultfd is a Linux syscall that gives an fd that receives a stream
of notifications of accesses to pages registered with it and allows
the program to acknowledge those stalls and tell the accessing
thread to carry on.

We convert the requests from the kernel into messages back to the
source asking for the pages.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Postcopy: Use helpers to map pages during migration

In postcopy, the destination guest is running at the same time
as it's receiving pages; as we receive new pages we must put
them into the guests a

Postcopy: Use helpers to map pages during migration

In postcopy, the destination guest is running at the same time
as it's receiving pages; as we receive new pages we must put
them into the guests address space atomically to avoid a running
CPU accessing a partially written page.

Use the helpers in postcopy-ram.c to map these pages.

qemu_get_buffer_in_place is used to avoid a copy out of qemu_file
in the case that postcopy is going to do a copy anyway.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

postcopy_ram.c: place_page and helpers

postcopy_place_page (etc) provide a way for postcopy to place a page
into guests memory atomically (using the copy ioctl on the ufd).

Signed-off-by: Dr. David

postcopy_ram.c: place_page and helpers

postcopy_place_page (etc) provide a way for postcopy to place a page
into guests memory atomically (using the copy ioctl on the ufd).

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Page request: Consume pages off the post-copy queue

When transmitting RAM pages, consume pages that have been queued by
MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning.

Note

Page request: Consume pages off the post-copy queue

When transmitting RAM pages, consume pages that have been queued by
MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning.

Note:
a) After a queued page the linear walk carries on from after the
unqueued page; there is a reasonable chance that the destination
was about to ask for other closeby pages anyway.

b) We have to be careful of any assumptions that the page walking
code makes, in particular it does some short cuts on its first linear
walk that break as soon as we do a queued page.

c) We have to be careful to not break up host-page size chunks, since
this makes it harder to place the pages on the destination.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Page request: Process incoming page request

On receiving MIG_RPCOMM_REQ_PAGES look up the address and
queue the page.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Amit S

Page request: Process incoming page request

On receiving MIG_RPCOMM_REQ_PAGES look up the address and
queue the page.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Page request: Add MIG_RP_MSG_REQ_PAGES reverse command

Add MIG_RP_MSG_REQ_PAGES command on Return path for the postcopy
destination to request a page from the source.

Two versions exist:
MIG_RP_

Page request: Add MIG_RP_MSG_REQ_PAGES reverse command

Add MIG_RP_MSG_REQ_PAGES command on Return path for the postcopy
destination to request a page from the source.

Two versions exist:
MIG_RP_MSG_REQ_PAGES_ID that includes a RAMBlock name and start/len
MIG_RP_MSG_REQ_PAGES that just has start/len for use with the same
RAMBlock as a previous MIG_RP_MSG_REQ_PAGES_ID

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Postcopy: End of iteration

The end of migration in postcopy is a bit different since some of
the things normally done at the end of migration have already been
done on the transition to postcopy.

S

Postcopy: End of iteration

The end of migration in postcopy is a bit different since some of
the things normally done at the end of migration have already been
done on the transition to postcopy.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Postcopy: Postcopy startup in migration thread

Rework the migration thread to setup and start postcopy.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Amit Shah <amit.shah

Postcopy: Postcopy startup in migration thread

Rework the migration thread to setup and start postcopy.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

postcopy: Incoming initialisation

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Re

postcopy: Incoming initialisation

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...