topology panic when using cpu generation newer than Penryn

[Leoyzen]

If you passthrough your cpu model and defined topology, you will see a kernel panic in MacOS.
The problem is divided to two part:
Fisrt is bootloader support(see disscusion here). This fixed in latest opencore.
The next is Apple MacOS support.
Apple use an undocumented Intel msr 0x35(see xen pr) to indentify topology when CPU Model newer than Penryn(Apple get the cpu topology from acpi when using Penryn, that's why Penryn works).
The xnu code related:

if (0 != (info_p->cpuid_features & CPUID_FEATURE_VMM) &&
	    PE_parse_boot_argn("-nomsr35h", NULL, 0)) {
		info_p->core_count = 1;
		info_p->thread_count = 1;
		cpuid_set_cache_info(info_p);
	} else {
		switch (info_p->cpuid_cpufamily) {
		case CPUFAMILY_INTEL_PENRYN:
			cpuid_set_cache_info(info_p);
			info_p->core_count   = info_p->cpuid_cores_per_package;
			info_p->thread_count = info_p->cpuid_logical_per_package;
			break;
		case CPUFAMILY_INTEL_WESTMERE: {
			uint64_t msr = rdmsr64(MSR_CORE_THREAD_COUNT);
			if (0 == msr) {
				/* Provide a non-zero default for some VMMs */
				msr = (1 << 16) | 1;
			}
			info_p->core_count   = bitfield32((uint32_t)msr, 19, 16);
			info_p->thread_count = bitfield32((uint32_t)msr, 15, 0);
			cpuid_set_cache_info(info_p);
			break;
		}
		default: {
			uint64_t msr = rdmsr64(MSR_CORE_THREAD_COUNT);
			if (0 == msr) {
				/* Provide a non-zero default for some VMMs */
				msr = (1 << 16) | 1;
			}
			info_p->core_count   = bitfield32((uint32_t)msr, 31, 16);
			info_p->thread_count = bitfield32((uint32_t)msr, 15, 0);
			cpuid_set_cache_info(info_p);
			break;
		}
		}
	}

Of course not only the VM user faces the problem, but also AMD Vanilla users. So we definately can use the way of AMD Vanilla did(using patches), but we need some survey which patches works.

[Leoyzen]

seems adding two patches below we can use newer cpu generation than Penryn with topology defined.But still no luck with odd number cores.

• algrey - cpu_topology_sort -disable _x86_validate_topology
• algrey - cpuid_set_cpufamily - force CPUFAMILY_INTEL_PENRYN

When setting odd cores and topology, the MacOS won't boot, and gives out nothing(after opencore bootloader it just hanged with no panic and no log).Still digging, but already a big progress.

[Leoyzen]

Opencore support is added by acidanthera/OcSupportPkg#17 after release 0.5.2.

[Leoyzen]

When emulating a CPU model (like Skylake-Server in QEMU), the cpu name under system info may be unknown.
We can just inject ProcessorType under SMBIOS of config.plist to a specific one, like 0x0F01(3841 in decimal), means 8/10/14/18-cores Xeon-W. Then the cpu name is normal(of course you should use the same core as Xeon-W defination).

It still not work when using 10/14/18-cores topology even with 0x0F01 specified. So it may not related to something Apple Hard check, may be cached related things?(QEMU may not handle cpu cache defination well when using odd cores topology.

[Leoyzen]

Finnally find some clues about this.
When booting with 1/3/2(socket/cores/threads) topology,opencore recognized it two ways in log:

00:008 00:004 OCCPU: MP services threads 6 (enabled 6) - Success
00:011 00:003 OCCPU: MP services Pkg 1 Cores 3 Threads 2 - Success
......
00:042 00:003 OCCPU: Pkg 1 Cores 4 Threads 6

So lookup the code, the second one (which is recognized as 4 cores) is using leaf 4 information:

if (Cpu->MaxId >= CPUID_CACHE_PARAMS && (Cpu->Model <= CPU_MODEL_PENRYN || Cpu->Hypervisor)) {
    AsmCpuidEx (CPUID_CACHE_PARAMS, 0, &CpuidCacheEax.Uint32, &CpuidCacheEbx.Uint32, NULL, NULL);
    if (CpuidCacheEax.Bits.CacheType != CPUID_CACHE_PARAMS_CACHE_TYPE_NULL) {
      CoreCount = (UINT16)GetPowerOfTwo32 (CpuidCacheEax.Bits.MaximumAddressableIdsForProcessorCores + 1);
      if (CoreCount < CpuidCacheEax.Bits.MaximumAddressableIdsForProcessorCores + 1) {
        CoreCount *= 2;
      }
      Cpu->CoreCount   = CoreCount;
      //
      // We should not be blindly relying on Cpu->Features & CPUID_FEATURE_HTT.
      // On Penryn CPUs it is set even without Hyper Threading.
      //
      if (Cpu->ThreadCount < Cpu->CoreCount) {
        Cpu->ThreadCount = Cpu->CoreCount;
      }
    }
  }

So it definately something wrong with cache params setting by QEMU.

Apple have this code too:

		reg[eax] = 4;           /* cpuid request 4 */
		reg[ecx] = index;       /* index starting at 0 */
		cpuid(reg);
		DBG("cpuid(4) index=%d eax=0x%x\n", index, reg[eax]);
		cache_type = bitfield32(reg[eax], 4, 0);
		if (cache_type == 0) {
			break;          /* no more caches */
		}
		cache_level             = bitfield32(reg[eax], 7, 5);
		cache_sharing           = bitfield32(reg[eax], 25, 14) + 1;
		info_p->cpuid_cores_per_package
		        = bitfield32(reg[eax], 31, 26) + 1;
		cache_linesize          = bitfield32(reg[ebx], 11, 0) + 1;
		cache_partitions        = bitfield32(reg[ebx], 21, 12) + 1;
		cache_associativity     = bitfield32(reg[ebx], 31, 22) + 1;
		cache_sets              = bitfield32(reg[ecx], 31, 0) + 1;

So there are two ways to bypass this logic:

1. setting disable cores in libvirt topology, like 8-cores when 1-core/2-threads disabled
2. there are patches in AMD-Vanilla to bypass the logic, like algrey - cpuid_set_cache_info - don't set cpuid_cores_per_package

The first one works and the seconds doesn't yet.But we made a big progress now right? We can set odd number cores now.

[Leoyzen]

3-cores/6-threads

<vcpu placement='static' current='6'>8</vcpu>
  <vcpus>
    <vcpu id='0' enabled='yes' hotpluggable='no' order='1'/>
    <vcpu id='1' enabled='yes' hotpluggable='yes' order='2'/>
    <vcpu id='2' enabled='yes' hotpluggable='yes' order='3'/>
    <vcpu id='3' enabled='yes' hotpluggable='yes' order='4'/>
    <vcpu id='4' enabled='yes' hotpluggable='yes' order='5'/>
    <vcpu id='5' enabled='yes' hotpluggable='yes' order='6'/>
    <vcpu id='6' enabled='no' hotpluggable='yes'/>
    <vcpu id='7' enabled='no' hotpluggable='yes'/>
  </vcpus>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='14'/>
    <vcpupin vcpu='2' cpuset='7'/>
    <vcpupin vcpu='3' cpuset='15'/>
    <vcpupin vcpu='4' cpuset='5'/>
    <vcpupin vcpu='5' cpuset='13'/>
  </cputune>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>

5-cores/5-threads

<vcpu placement='static' current='5'>8</vcpu>
  <vcpus>
    <vcpu id='0' enabled='yes' hotpluggable='no' order='1'/>
    <vcpu id='1' enabled='yes' hotpluggable='yes' order='2'/>
    <vcpu id='2' enabled='yes' hotpluggable='yes' order='3'/>
    <vcpu id='3' enabled='yes' hotpluggable='yes' order='4'/>
    <vcpu id='4' enabled='yes' hotpluggable='yes' order='5'/>
    <vcpu id='5' enabled='no' hotpluggable='yes'/>
    <vcpu id='6' enabled='no' hotpluggable='yes'/>
    <vcpu id='7' enabled='no' hotpluggable='yes'/>
  </vcpus>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='14'/>
    <vcpupin vcpu='2' cpuset='7'/>
    <vcpupin vcpu='3' cpuset='15'/>
    <vcpupin vcpu='4' cpuset='8'/>
  </cputune>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='8' threads='1'/>
  </cpu>

[heccoder]

Thanks for sharing the solution. I am not using odd cores, I am passing 8 cores and 2 threads each. With the topology skip patch I was able to have macOS properly recognize the cores/threads, however the CPU name shown is: Single-Core Intel Xeon.

My CPU is an i7-6950X, if I try to passthrough my original CPU with host in the qemu arguments, the VM gets stuck at Opencore, it doesn't even boot macOS. So I am emulating the Skylake-Server, however, I keep getting the following in the VM logs:

2020-03-30T21:35:05.019712Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.clflushopt [bit 23]
2020-03-30T21:35:05.019715Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.clwb [bit 24]
2020-03-30T21:35:05.019718Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.avx512cd [bit 28]
2020-03-30T21:35:05.019720Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.avx512bw [bit 30]
2020-03-30T21:35:05.019723Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.avx512vl [bit 31]
2020-03-30T21:35:05.019726Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:ECX.pku [bit 3]
2020-03-30T21:35:05.019732Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.0DH:EAX.xsavec [bit 1]
2020-03-30T21:35:05.019736Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.0DH:EAX.xgetbv1 [bit 2]
2020-03-30T21:35:05.019739Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.0DH:EAX [bit 5]
2020-03-30T21:35:05.019742Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.0DH:EAX [bit 6]
2020-03-30T21:35:05.019746Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.0DH:EAX [bit 7]
2020-03-30T21:35:05.019749Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.0DH:EAX [bit 9]
2020-03-30T21:35:05.020568Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.avx512f [bit 16]
2020-03-30T21:35:05.020576Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.avx512dq [bit 17]

[Leoyzen]

@sonoseco This is not error, only warning.Ingore it.
passthrough should work.If not, post your opencore log.

[heccoder]

@Leoyzen I've emulating the CPU for quite some time, however, I have a lot a audio issues and it seems that can be resolved by passing through the CPU. I still have the same issue as above, where Opencore hangs when using <qemu:arg value='host,vendor=GenuineIntel,+hypervisor,+invtsc,kvm=on,+fma,+avx,+avx2,+aes,+ssse3,+sse4_2,+popcnt,+sse4a,+bmi1,+bmi2'/>

This is the opencore log:

Edit: fixed the issue by using cpu and qemu:arg (adjusted to my config) from this XML: https://github.com/Pavo-IM/Hackinabox/blob/master/Catalina.xml

[RonjaPonja]

QEMU patch regarding MSR_CORE_THREAD_COUNT:
https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg04906.html

[RonjaPonja]

That actually only works for hvf, not kvm. Also see https://lkml.org/lkml/2020/9/16/977