H A D | spapr.c | 72fdd4de Tue Feb 27 09:22:58 CST 2018 Greg Kurz <groug@kaod.org> spapr: register dummy ICPs later Some older machine types create more ICPs than needed. We hence need to register up to xics_max_server_number() dummy ICPs to accomodate the migration of these machine types. Recent VSMT rework changed xics_max_server_number() to return DIV_ROUND_UP(max_cpus * spapr->vsmt, smp_threads) instead of DIV_ROUND_UP(max_cpus * kvmppc_smt_threads(), smp_threads); The change is okay but it requires spapr->vsmt to be set, which isn't the case with the current code. This causes the formula to return zero and we don't create dummy ICPs. This breaks migration of older guests as reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1549087 The dummy ICP workaround doesn't really have a dependency on XICS itself. But it does depend on proper VCPU id numbering and it must be applied before creating vCPUs (ie, creating real ICPs). So this patch moves the workaround to spapr_init_cpus(), which already assumes VSMT to be set. Fixes: 72194664c8a1 ("spapr: use spapr->vsmt to compute VCPU ids") Reported-by: Lukas Doktor <ldoktor@redhat.com> Signed-off-by: Greg Kurz <groug@kaod.org> Signed-off-by: David Gibson <david@gibson.dropbear.id.au> 72194664 Wed Feb 14 13:40:26 CST 2018 Greg Kurz <groug@kaod.org> spapr: use spapr->vsmt to compute VCPU ids Since the introduction of VSMT in 2.11, the spacing of VCPU ids between cores is controllable through a machine property instead of being only dictated by the SMT mode of the host: cpu->vcpu_id = (cc->core_id * spapr->vsmt / smp_threads) + i Until recently, the machine code would try to change the SMT mode of the host to be equal to VSMT or exit. This allowed the rest of the code to assume that kvmppc_smt_threads() == spapr->vsmt is always true. Recent commit "8904e5a75005 spapr: Adjust default VSMT value for better migration compatibility" relaxed the rule. If the VSMT mode cannot be set in KVM for some reasons, but the requested CPU topology is compatible with the current SMT mode, then we let the guest run with kvmppc_smt_threads() != spapr->vsmt. This breaks quite a few places in the code, in particular when calculating DRC indexes. This is what happens on a POWER host with subcores-per-core=2 (ie, supports up to SMT4) when passing the following topology: -smp threads=4,maxcpus=16 \ -device host-spapr-cpu-core,core-id=4,id=core1 \ -device host-spapr-cpu-core,core-id=8,id=core2 qemu-system-ppc64: warning: Failed to set KVM's VSMT mode to 8 (errno -22) This is expected since KVM is limited to SMT4, but the guest is started anyway because this topology can run on SMT4 even with a VSMT8 spacing. But when we look at the DT, things get nastier: cpus { ... ibm,drc-indexes = <0x4 0x10000000 0x10000004 0x10000008 0x1000000c>; This means that we have the following association: CPU core device | DRC | VCPU id -----------------+------------+--------- boot core | 0x10000000 | 0 core1 | 0x10000004 | 4 core2 | 0x10000008 | 8 core3 | 0x1000000c | 12 But since the spacing of VCPU ids is 8, the DRC for core1 points to a VCPU that doesn't exist, the DRC for core2 points to the first VCPU of core1 and and so on... ... PowerPC,POWER8@0 { ... ibm,my-drc-index = <0x10000000>; ... }; PowerPC,POWER8@8 { ... ibm,my-drc-index = <0x10000008>; ... }; PowerPC,POWER8@10 { ... No ibm,my-drc-index property for this core since 0x10000010 doesn't exist in ibm,drc-indexes above. ... }; }; ... interrupt-controller { ... ibm,interrupt-server-ranges = <0x0 0x10>; With a spacing of 8, the highest VCPU id for the given topology should be: 16 * 8 / 4 = 32 and not 16 ... linux,phandle = <0x7e7323b8>; interrupt-controller; }; And CPU hot-plug/unplug is broken: (qemu) device_del core1 pseries-hotplug-cpu: Cannot find CPU (drc index 10000004) to remove (qemu) device_del core2 cpu 4 (hwid 8) Ready to die... cpu 5 (hwid 9) Ready to die... cpu 6 (hwid 10) Ready to die... cpu 7 (hwid 11) Ready to die... These are the VCPU ids of core1 actually (qemu) device_add host-spapr-cpu-core,core-id=12,id=core3 (qemu) device_del core3 pseries-hotplug-cpu: Cannot find CPU (drc index 1000000c) to remove This patches all the code in hw/ppc/spapr.c to assume the VSMT spacing when manipulating VCPU ids. Fixes: 8904e5a75005 Signed-off-by: Greg Kurz <groug@kaod.org> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
|