Projects
home:dongzhihai
qemu
_service:tar_scm:arm-virt-kvm-Pre-create-disabl...
Sign Up
Log In
Username
Password
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File _service:tar_scm:arm-virt-kvm-Pre-create-disabled-possible-vCPUs-mach.patch of Package qemu
From 2669fd26cbc36e24ebfc844c240b45ad831701cc Mon Sep 17 00:00:00 2001 From: Salil Mehta <salil.mehta@huawei.com> Date: Tue, 5 May 2020 18:44:59 +0100 Subject: [PATCH] arm/virt,kvm: Pre-create disabled possible vCPUs @machine init In ARMv8 architecture, GIC needs all the vCPUs to be created and present when it is initialized. This is because: 1. GICC and MPIDR association must be fixed at the VM initialization time. This is represented by register GIC_TYPER(mp_afffinity, proc_num) 2. GICC(cpu interfaces), GICR(redistributors) etc all must be initialized at the boot time as well. 3. Memory regions associated with GICR etc. cannot be changed(add/del/mod) after VM has inited. This patch adds the support to pre-create all such possible vCPUs within the host using the KVM interface as part of the virt machine initialization. These vCPUs could later be attached to QOM/ACPI while they are actually hot plugged and made present. Co-developed-by: Salil Mehta <salil.mehta@huawei.com> Signed-off-by: Salil Mehta <salil.mehta@huawei.com> Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com> Reported-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com> [VP: Identified CPU stall issue & suggested probable fix] Signed-off-by: Salil Mehta <salil.mehta@huawei.com> --- hw/arm/virt.c | 53 +++++++++++++++++++++++++++++++++++++++++-- include/hw/core/cpu.h | 1 + target/arm/cpu64.c | 1 + target/arm/kvm.c | 32 ++++++++++++++++++++++++++ target/arm/kvm64.c | 9 +++++++- target/arm/kvm_arm.h | 11 +++++++++ 6 files changed, 104 insertions(+), 3 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 2f04bc7666..f10d75366b 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -2389,8 +2389,10 @@ static void machvirt_init(MachineState *machine) assert(possible_cpus->len == max_cpus); for (n = 0; n < possible_cpus->len; n++) { Object *cpuobj; + CPUState *cs; cpuobj = object_new(possible_cpus->cpus[n].type); + cs = CPU(cpuobj); aarch64 &= object_property_get_bool(cpuobj, "aarch64", NULL); object_property_set_int(cpuobj, "socket-id", @@ -2402,8 +2404,55 @@ static void machvirt_init(MachineState *machine) object_property_set_int(cpuobj, "thread-id", virt_get_thread_id(machine, n), NULL); - qdev_realize(DEVICE(cpuobj), NULL, &error_fatal); - object_unref(cpuobj); + if (n < smp_cpus) { + qdev_realize(DEVICE(cpuobj), NULL, &error_fatal); + object_unref(cpuobj); + } else { + CPUArchId *cpu_slot; + + /* handling for vcpus which are yet to be hot-plugged */ + cs->cpu_index = n; + cpu_slot = virt_find_cpu_slot(machine, cs->cpu_index); + + /* + * ARM host vCPU features need to be fixed at the boot time. But as + * per current approach this CPU object will be destroyed during + * cpu_post_init(). During hotplug of vCPUs these properties are + * initialized again. + */ + virt_cpu_set_properties(cpuobj, cpu_slot, &error_fatal); + + /* + * For KVM, we shall be pre-creating the now disabled/un-plugged + * possbile host vcpus and park them till the time they are + * actually hot plugged. This is required to pre-size the host + * GICC and GICR with the all possible vcpus for this VM. + */ + if (kvm_enabled()) { + kvm_arm_create_host_vcpu(ARM_CPU(cs)); + } + /* + * Add disabled vCPU to CPU slot during the init phase of the virt + * machine + * 1. We need this ARMCPU object during the GIC init. This object + * will facilitate in pre-realizing the GIC. Any info like + * mp-affinity(required to derive gicr_type) etc. could still be + * fetched while preserving QOM abstraction akin to realized + * vCPUs. + * 2. Now, after initialization of the virt machine is complete we + * could use two approaches to deal with this ARMCPU object: + * (i) re-use this ARMCPU object during hotplug of this vCPU. + * OR + * (ii) defer release this ARMCPU object after gic has been + * initialized or during pre-plug phase when a vCPU is + * hotplugged. + * + * We will use the (ii) approach and release the ARMCPU objects + * after GIC and machine has been fully initialized during + * machine_init_done() phase. + */ + cpu_slot->cpu = OBJECT(cs); + } } fdt_add_timer_nodes(vms); fdt_add_cpu_nodes(vms); diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index c30636a936..fdfb952259 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -528,6 +528,7 @@ struct CPUState { uint32_t kvm_fetch_index; uint64_t dirty_pages; int kvm_vcpu_stats_fd; + VMChangeStateEntry *vmcse; /* Use by accel-block: CPU is executing an ioctl() */ QemuLockCnt in_ioctl_lock; diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index e226b60b72..5d28838175 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -859,6 +859,7 @@ static void aarch64_cpu_initfn(Object *obj) * enabled explicitly */ cs->disabled = true; + cs->thread_id = 0; } static void aarch64_cpu_finalizefn(Object *obj) diff --git a/target/arm/kvm.c b/target/arm/kvm.c index f59f4f81b2..70cf15b550 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -659,6 +659,38 @@ void kvm_arm_reset_vcpu(ARMCPU *cpu) write_list_to_cpustate(cpu); } +void kvm_arm_create_host_vcpu(ARMCPU *cpu) +{ + CPUState *cs = CPU(cpu); + unsigned long vcpu_id = cs->cpu_index; + int ret; + + ret = kvm_create_vcpu(cs); + if (ret < 0) { + error_report("Failed to create host vcpu %ld", vcpu_id); + abort(); + } + + /* + * Initialize the vCPU in the host. This will reset the sys regs + * for this vCPU and related registers like MPIDR_EL1 etc. also + * gets programmed during this call to host. These are referred + * later while setting device attributes of the GICR during GICv3 + * reset + */ + ret = kvm_arch_init_vcpu(cs); + if (ret < 0) { + error_report("Failed to initialize host vcpu %ld", vcpu_id); + abort(); + } + + /* + * park the created vCPU. shall be used during kvm_get_vcpu() when + * threads are created during realization of ARM vCPUs. + */ + kvm_park_vcpu(cs); +} + /* * Update KVM's MP_STATE based on what QEMU thinks it is */ diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 3c175c93a7..03ce1e7525 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -562,7 +562,14 @@ int kvm_arch_init_vcpu(CPUState *cs) return -EINVAL; } - qemu_add_vm_change_state_handler(kvm_arm_vm_state_change, cs); + /* + * Install VM change handler only when vCPU thread has been spawned + * i.e. vCPU is being realized + */ + if (cs->thread_id) { + cs->vmcse = qemu_add_vm_change_state_handler(kvm_arm_vm_state_change, + cs); + } /* Determine init features for this CPU */ memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features)); diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h index 051a0da41c..31408499b3 100644 --- a/target/arm/kvm_arm.h +++ b/target/arm/kvm_arm.h @@ -163,6 +163,17 @@ void kvm_arm_cpu_post_load(ARMCPU *cpu); */ void kvm_arm_reset_vcpu(ARMCPU *cpu); +/** + * kvm_arm_create_host_vcpu: + * @cpu: ARMCPU + * + * Called at to pre create all possible kvm vCPUs within the the host at the + * virt machine init time. This will also init this pre-created vCPU and + * hence result in vCPU reset at host. These pre created and inited vCPUs + * shall be parked for use when ARM vCPUs are actually realized. + */ +void kvm_arm_create_host_vcpu(ARMCPU *cpu); + /** * kvm_arm_init_serror_injection: * @cs: CPUState -- 2.27.0
Locations
Projects
Search
Status Monitor
Help
Open Build Service
OBS Manuals
API Documentation
OBS Portal
Reporting a Bug
Contact
Mailing List
Forums
Chat (IRC)
Twitter
Open Build Service (OBS)
is an
openSUSE project
.
浙ICP备2022010568号-2