Sandy Bridge-E 6 core/HT (12 processors) is being configured in logical flat mode. From dmesg: [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x00] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x00] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x00] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x00] disabled) Once booted i checked the local apic's: cpu 0 logical 01000000 physical 00000000 cpu 1 logical 02000000 physical 01000000 cpu 2 logical 04000000 physical 02000000 cpu 3 logical 08000000 physical 03000000 cpu 4 logical 10000000 physical 04000000 cpu 5 logical 20000000 physical 05000000 cpu 6 logical 40000000 physical 06000000 cpu 7 logical 80000000 physical 07000000 cpu 8 logical 00000000 physical 08000000 cpu 9 logical 00000000 physical 09000000 cpu 10 logical 00000000 physical 0a000000 cpu 11 logical 00000000 physical 0b000000 I would have expected cluster mode with logical id's (in hex): 01,02,04,08,11,12,14,18,21,22,24,28 x0 isnt a valid cluster id as the lower nibble is a mask meaning only 1,2,4,8 are selectable. Flat mode is only capable of supporting 8 processors or less.
Can you please attach dmesg with apic=debug added to the kernel command line? Thanks.
Created attachment 106271 [details] dmesg log with apic=debug added to kernel cmd line I have attached the requested info.
From your dmesg, the apic is in physical flat mode, which supports more than 8 cpus: [ 0.020000] Switched APIC routing to physical flat. Hi Len, Do you know if this is the expected setup here? Thanks.
I saw other systems will use x2apic_cluster mode, perhaps your CPU doesn't support that? [ 0.033861] Enabled IRQ remapping in x2apic mode [ 0.033864] Enabling x2apic [ 0.033867] Enabled x2apic [ 0.033877] Switched APIC routing to cluster x2apic.
I agree with your assessment of physical flat mode. I guess as long as you only use physical destination IDs its ok. The logical id's ought to be filled in properly tho. I have BIOS (intel bios) enabled for Virtualization and VTd (why 2 opts? don't know) there is no specific option there to enable X2Apic. The kernel is configured to support x2apic ( CONFIG_X86_X2APIC=y ). After boot I checked for x2apic support, "cpuid 1" returns ecx[21]=1 supporting x2apic, and msr 0x1b bit 10 = 0 which shows x2 mode is disabled. The DMAR table is there in acpi and has 2 DRHD structures defined that look right, I can "talk" to either apic defined in these structures. So, whats keeping x2mode from being setup? I see no mention of it in the logs at all. I'll see if I can determine what the kernel is thinking, I have the source.
Browsing the kernel source I found that if x2apic isn't enabled by BIOS then the kernel won't enable it. So x2apic not working is due to BIOS not handing off to the kernel in x2apic mode. I wish i could file a "bios bug/missing feature" on this. It would be nice to have it working
(In reply to comment #6) > Browsing the kernel source I found that if x2apic isn't enabled by BIOS then > the kernel won't enable it. So x2apic not working is due to BIOS not handing > off to the kernel in x2apic mode. I wish i could file a "bios bug/missing > feature" on this. It would be nice to have it working It may be due to BIOS doesn't think this system should support x2apic. I'm not sure of the optimal setting of APIC for your system, I think you can ask such questions in linux kernel mailing list, people may answer your questions there.
Most likely, the DMAR table in your system doesn't claim support of interrupt remapping. Can you please attach acpidump here? Thanks.
Created attachment 106815 [details] dmar acpi table attached requested dmar acpi table
The DMAR table doesn't claim support of interrupt remapping(flags is 0x00 instead of 0x01) so Linux will not force enable x2apic mode. I don't see Linux does anything wrong with APIC config for your system, what do you think?
I agree, this isn't a linux issue. This issue is due to the bios not setting up acpi correctly. I doubt it will get fixed. This can be closed.
BIOS doesn't set interrupt remapping flag in DMAR table.