virt/hyperv/vmbus.rst

1 .. SPDX-License-Identifier: GPL-2.0
5 VMbus is a software construct provided by Hyper-V to guest VMs.  It
7 devices that Hyper-V presents to guest VMs.   The control path is
11 and the synthetic device implementation that is part of Hyper-V, and
12 signaling primitives to allow Hyper-V and the guest to interrupt
17 establishes the VMbus control path with the Hyper-V host, then
21 Most synthetic devices offered by Hyper-V have a corresponding Linux
29 * PCI device pass-thru
34 * Key/Value Pair (KVP) exchange with Hyper-V
35 * Hyper-V online backup (a.k.a. VSS)
38 controller, synthetic NIC, and PCI pass-thru devices.  Other
41 Hyper-V that are used only by Windows guests and for which Linux
44 Hyper-V uses the terms "VSP" and "VSC" in describing synthetic
45 devices.  "VSP" refers to the Hyper-V code that implements a
53 --------------
55 between the VSP and the VSC.  Channels are bi-directional and used
56 for passing messages.   Most synthetic devices use a single channel,
60 Each channel consists of two ring buffers.  These are classic ring
64 The "in" ring buffer is for messages from the Hyper-V host to the
66 the Hyper-V host.  In Linux, the "in" and "out" designations are as
75 making up the ring is communicated to the Hyper-V host over the
84 ring buffer need not be concerned with ring buffer wrap-around.
89 directly in the ring without handling wrap-around.
92 passed to Hyper-V as a 4 Kbyte area.  But the memory for the actual
96 Hyper-V.  This case is handled by vmbus_establish_gpadl().
98 Hyper-V enforces a limit on the aggregate amount of guest memory
106 --------------
120 responses.  Some synthetic devices allow multiple requests to be in-
122 sending a request.  Hyper-V sends back the same transactionID in the
128 between the guest and the Hyper-V host, the actual data to be
130 specified as a separate data buffer that the Hyper-V host will
134 Hyper-V host to the guest contain the actual time value.  When the
142 1. vmbus_sendpacket():  Control-only messages and messages with
143    embedded data -- no GPAs
153 Historically, Linux guests have trusted Hyper-V to send well-formed
157 guest to not trust the hypervisor (AMD SNP-SEV, Intel TDX), trusting
158 the Hyper-V host is no longer a valid assumption.  The drivers for
160 values read from memory that is shared with Hyper-V, which includes
163 temporary buffer that is not shared with Hyper-V.  Validation is
164 performed in this temporary buffer without the risk of Hyper-V
168 VMbus interrupts
169 ----------------
173 ring buffer transitions from empty to non-empty.  If the guest sends
174 interrupts at other times, the host deems such interrupts to be
176 interrupts, the host may throttle that guest by suspending its
177 execution for a few seconds to prevent a denial-of-service attack.
180 message on the VMbus control path, or when a VMbus channel "in" ring
181 buffer transitions from empty to non-empty.  Each CPU in the guest
182 may receive VMbus interrupts, so they are best modeled as per-CPU
183 interrupts in Linux.  This model works well on arm64 where a single
184 per-CPU IRQ is allocated for VMbus.  Since x86/x64 lacks support for
185 per-CPU IRQs, an x86 interrupt vector is statically allocated (see
187 call the VMbus interrupt service routine.  These interrupts are
188 visible in /proc/interrupts on the "HYP" line.
190 The guest CPU that a VMbus channel will interrupt is selected by the
191 guest when the channel is created, and the host is informed of that
194 1. "Slow" devices that need only one VMbus channel.  The devices
196    relatively few interrupts.  Their VMbus channels are all
203    channels interrupts are assigned to CPUs that are spread out
204    among the available CPUs in the VM so that interrupts on
207 The assignment of VMbus channel interrupts to CPUs is done in the
209 normal Linux interrupt affinity mechanism, so the interrupts are
210 neither "unmanaged" nor "managed" interrupts.
212 The CPU that a VMbus channel will interrupt can be seen in
214 When running on later versions of Hyper-V, the CPU can be changed
218 VMbus channel interrupts.
221 VMbus channel interrupts assigned to it.  Any such channel
222 interrupts must first be manually reassigned to another CPU as
223 described above.  When no channel interrupts are assigned to the
228 channel interrupts by calling vmbus_chan_sched(), which looks at a
230 interrupts on this CPU.  If multiple channels have pending
231 interrupts for this CPU, they are processed sequentially.  When all
232 channel interrupts have been processed, vmbus_isr() checks for and
235 The VMbus channel interrupt handling code is designed to work
237 CPU assigned to the channel.  Specifically, the code does not use
238 CPU-based exclusion for correctness.  In normal operation, Hyper-V
240 channel is being changed via sysfs, the guest doesn't know exactly
241 when Hyper-V will make the transition.  The code must work correctly
242 even if there is a time lag before Hyper-V starts interrupting the
246 ------------------------------
247 Hyper-V and the Linux guest have a separate message-passing path
249 path does not use a VMbus channel.  See vmbus_post_msg() and
253 Hyper-V VMbus mechanism.  As part of establishing this connection,
254 the guest and Hyper-V agree on a VMbus protocol version they will
256 Hyper-V versions, and vice versa.
258 The guest then tells Hyper-V to "send offers".  Hyper-V sends an
262 identified by a GUID. The offer message from Hyper-V contains
266 class ID. The ordering of offer messages can vary from boot-to-boot
269 because Hyper-V supports adding devices, such as synthetic NICs,
278 The device driver probe function opens the primary VMbus channel to
279 the corresponding VSP. It allocates guest memory for the channel
280 ring buffers and shares the ring buffer with the Hyper-V host by
285 setup messages via the primary channel.  These messages may include
287 VSC and the VSP on the Hyper-V host.  The setup messages may also
289 mis-named as "sub-channels" since they are functionally
290 equivalent to the primary channel once they are created.
295 The Hyper-V host can send a "rescind" message to the guest to
300 rescinded, neither Hyper-V nor Linux retains any state about
301 its previous existence. Such a device might be re-added later,