Bug 12236

Please make sure you file a copy of this bug report with vmware as if its a bug in the vmware side of things there isn't anything we can do about it.

Does the same kernel boot fine outside of vmware on that system ?

Is already fixed in Linus's tree, and will be resolved in the 2.6.27.10 release tomorrow as well.

This is actually a different bug, the bug I fixed was for VMI, which is 32-bit only, and this is a 64-bit only bug... not sure how to change back to ASSIGNED, but feel free to assign it to me.

Thanks Zach!

Oops, sorry about that.

Zach, is there any other information we can provide to help here? Have you had a chance to attempt to reproduce it?

Thanks, I think I have everything I need.  I'm a bit slow on the uptake right now because I'm still recovering from wisdom tooth surgery.

You didn't select CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC, did you?

Can you attach the .config used to generate this kernel?  I can't find it either here or in the Gentoo bug report and so far I'm having a bit of trouble reproducing this panic.

I'll attach the log on monday when I'm at work again. In the meantime, an easy way to reproduce the problem (at least on our ESX setup) was simply booting the SystemRescueCd beta5 iso offered on http://www.sysresccd.org/ under the beta section (direct download link: http://beta.sysresccd.org/systemrescuecd-x86-1.1.4-beta6.iso ) It boots a 64bit 2.6.27.x kernel, when booted with "rescue64" and also produces the kernel PANIC on our system.

(In reply to comment #9)
> I'll attach the log on monday when I'm at work again.

Timo, you've been asked to attach the kernel .config
and not a (dmesg) log. :-)

This doesn't reproduce for me on VMware workstation; it could be a difference in configuration or a change in VMware.  I will try it on 3.5.

Interestingly, the crash is in dmi_string_nosave  ... which doesn't do much in the way of safety or validation.  The other bugs referenced in the Gentoo bug report appear to be potentially different hardware related crashes.

this doesn't reproduce on ESX 3.5, with the same system rescue CD.

Can I get a copy of the .vmx file used to boot this VM?

I suspect it is dependent on number of cpus or memory size.  At this point, I am doubtful this is a VMware bug, and suspect something to do with DMI..

Thanks for the headup Zachary, I'll try to provide the vmx file and more information about our hardware on monday, I'll also try to physically boot one of our blade servers from the SystemRescueCd and see how well this works.

Created attachment 19393 [details]
successful boot on VMware ESX

successful boot on VMware ESX

To test we've just upgraded one of our Blades (HP ProLiant BL465c G1) which failed before with VMware ESX Server version=3.5.0 build=build-98103 to VMware ESX Server version=3.5.0 build=build-130756 (and all available updates) and things magically started to work. Booting the SystemRescueCd (rescue64) as well as upgrading and booting an installed Gentoo kernel 2.6.27.9 (64bit) works without producing any PANIC now.

Sounds very plausible to me.. I was testing ESX 3.5U2, which is build 110268.  Since it's already fixed, I don't really see any need to open a bug on our end or binary search it down to a root cause... it's likely the DMI data in our BIOS was badly terminated and caused a panic here.  I'm going to go ahead and close it then.  If it recurs, feel free to re-open.

Zach - can you fill in more info about bad DMI data if you had it in the VMware BIOS. Our DMI code should be robust against such things and if it isn't I'd like to see it fixed.

(In reply to comment #17)
> Zach - can you fill in more info about bad DMI data if you had it in the
> VMware
> BIOS. Our DMI code should be robust against such things and if it isn't I'd
> like to see it fixed.

To be honest it could be many things that caused this bug and lack of DMI robustness is just a convenient suspect.  Perhaps it was an MMU bug that was fixed?  In any case, the same .iso image boots fine on one version of VMware and not on a slightly bugfixed version off the same branch.. pointing to a VMware bug as the real culprit.

However, inspecting the DMI code I do see a reliance on proper NULL termination and length setting of strings in several places:

static const char * __init dmi_string_nosave(const struct dmi_header *dm, u8 s)
{
        const u8 *bp = ((u8 *) dm) + dm->length;

        if (s) {
                s--;
                while (s > 0 && *bp) {
                        bp += strlen(bp) + 1;    // (1)
                        s--;
                }

                if (*bp != 0) {
                        size_t len = strlen(bp)+1;   // (2)
                        size_t cmp_len = len > 8 ? 8 : len;

                        if (!memcmp(bp, dmi_empty_string, cmp_len))
                                return dmi_empty_string;
                        return bp;
                }

(1) strlen isn't really the best choice here, there should be a bounded dmi_strlen function which takes as an argument the DMI header pointer and imposes sanity bounds on the maximum length

(2) in the second case, then length is irrelevant if it is longer than 8... so a bounded strncmp would be a better choice.