Bug 20242

Besides: "Hello!" to everybody.

Is there any other information needed?

On the architecture side x86 and x86_64 have been tested with kernels up to 2.6.33, showing the same behavior.

Created attachment 33872 [details]
custom _TMP method

please
1. mount the debugfs
2. apply this custom _TMP method by running "cat _TMP.aml > /sys/kernel/debug/acpi/custom_method"
3. enable the ACPI debug output by running "echo 1 > /sys/module/acpi/parameters/aml_debug_output"
4. dmesg -c
5. read the temperature by running "grep . /sys/class/thermal/thermal_zone*/*"
6. dmesg > dmesg.log
and then attach the dmesg.log here.

please do the same tests both when the temperature is normal and when it drops to 0C.

BTW, you need to do this test in a kernel newer than 2.6.35.

Thanks for your commitment.

The test actually gives no (measurable) result: dmesg.log is empty (0 bytes) in both cases. Used kernel was 2.6.36-rc6 (version string for git kernel: 2.6.36-rc6-00137-g7c6d45e).
(Of course I never got any message there in normal setup, too.)
dmesg -c pulls out the last lines containing
    [ACPI Debug]  String [0x15] "_Q80 : Temperature Up"
But the reading of temperature via grep gives no messages via dmesg, but the following values instead on stdout:
    /sys/class/thermal/thermal_zone0/cdev0_trip_point:2
    /sys/class/thermal/thermal_zone0/cdev1_trip_point:2
    /sys/class/thermal/thermal_zone0/mode:enabled
    /sys/class/thermal/thermal_zone0/temp:0
    /sys/class/thermal/thermal_zone0/trip_point_0_temp:105000
    /sys/class/thermal/thermal_zone0/trip_point_0_type:critical
    /sys/class/thermal/thermal_zone0/trip_point_1_temp:104000
    /sys/class/thermal/thermal_zone0/trip_point_1_type:hot
    /sys/class/thermal/thermal_zone0/trip_point_2_temp:96000
    /sys/class/thermal/thermal_zone0/trip_point_2_type:passive
    /sys/class/thermal/thermal_zone0/type:acpitz

Actually, I have made both runs consecutive without shutdown. If this is a problem, I will (of course) repeat the last run. 0°C were reached after 45 minutes of uptime.

Additional Infos:
The BIOS of the Laptop is an infamous InsydeH2O.

The above mentioned commit was first bad in a git bisect run with the previous commits running good for roughly a week. Where roughly means: 5 to seven days during workhours (not consecutive), as I can't trigger the problem, this is just a guess. But I don't feel like this result really helps, as this commit is basically empty. So I am testing the previous ones again.

If you would have advise on how I could help narrowing down the problem, I would be grateful.

Thanks,
Jan

In the working 2.6.31, what do you see in /proc/acpi/thermal_zone/*/* ?

Hi! Welcome on this bug. :-)

The output of "ls /proc/acpi/thermal_zone/*/*" is
    /proc/acpi/thermal_zone/TZ01/cooling_mode
    /proc/acpi/thermal_zone/TZ01/polling_frequency
    /proc/acpi/thermal_zone/TZ01/state
    /proc/acpi/thermal_zone/TZ01/temperature
    /proc/acpi/thermal_zone/TZ01/trip_points
using the working 2.6.31.14 kernel. With cat on 
    cooling_mod giving "<setting not supported>"
    polling_frequency giving "<polling disabled>"
    state giving "state: ok"
    temperature giving an expectable temperature in °C
    trip_points "critical (S5):           105 C
                    hot (S4):                104 C
                    passive:                 96 C: tc1=2 tc2=3 tsp=50 devices=CPU0 CPU1"

I try triggering the bug using the stress utility or doing multiple compiles with "make -j3". The Laptop has a core 2 duo processor. The system fairly never reaches 84°C using these procedures. But the behaviour triggers even with idle system load so I am unsure if this generated load really helps.

In my attempt to do a "git bisect" I found a false good marker and I am on my way back through another branch. I am now between commits
    bf25400e889dac3f9a3d5a5b77e8ec4c170a5006 (bad) and
    7a92d803227a523a9a5546e4e0dce1325a4b5926 (good)
with the above stated retentions.

Hope this helps!
Jan

(In reply to comment #8)
> Hi! Welcome on this bug. :-)
> 
> The output of "ls /proc/acpi/thermal_zone/*/*" is
>     /proc/acpi/thermal_zone/TZ01/cooling_mode
>     /proc/acpi/thermal_zone/TZ01/polling_frequency
>     /proc/acpi/thermal_zone/TZ01/state
>     /proc/acpi/thermal_zone/TZ01/temperature
>     /proc/acpi/thermal_zone/TZ01/trip_points

what's the output of "grep . /proc/acpi/thermal_zone/*/*"?

Output of "grep . /proc/acpi/thermal_zone/*/*"
/proc/acpi/thermal_zone/TZ01/cooling_mode:<setting not supported>
/proc/acpi/thermal_zone/TZ01/polling_frequency:<polling disabled>
/proc/acpi/thermal_zone/TZ01/state:state:                   ok
/proc/acpi/thermal_zone/TZ01/temperature:temperature:             49 C
/proc/acpi/thermal_zone/TZ01/trip_points:critical (S5):           105 C
/proc/acpi/thermal_zone/TZ01/trip_points:hot (S4):                104 C
/proc/acpi/thermal_zone/TZ01/trip_points:passive:                 96 C: tc1=2 tc2=3 tsp=50 devices=CPU0 CPU1

Hm, although my remaining git bisect line just holds 16 commits I don't know if there is anything useful within it. May someone please be so kind to check the commits and tell me if it is worth trying mostly good commits for weeks?

Any hints on how I could track that thing down?

Cheers,
    Jan

Okay, I got another bad commit 
f25752e67d9d9ee7562ae9944314dd8c057d3fa2

Hm, text got through to the server to early…

What is this ACPI EC thingie and could it be connected? I am now testing a commit having to do with a IRQ/Polling descision in this context. Actually I will have to test another thing, as the coretemp module for the Core 2 Duo is not updating its temperature properly (I might have mentioned this earlier, I know, sorry if it's useful, but I am sitting on this bug quite some time, now).

Actually the coretemp module reports a temperature which seems identical to the thermal zone temperature at system boot (or resume from suspend to disk in that matter). But afterwards it stays constant. The thermal zone is still updated up to the point it freezes at 0°C.

Hope this helps you to help me. ^^

Cheers,
    Jan

Latest results from git bisect testing. First bad commit seems to be:
    2a84cb9852f52c0cd1c48bca41a8792d44ad06cc
The previous commit is versioned 2.6.31-rc8, which I am going to test, just to be sure, but which should be okay. So this seems to be the commit this bug was introduced for my system.

I really hope this helps. :-)

Created attachment 35782 [details]
dmesg for 2.6.31-rc8 with DEBUG defined in ec.c

In drivers/acpi/ec.c defined DEBUG.
This is the output for the (working) 2.6.31-rc8 kernel.

Created attachment 35792 [details]
dmesg for 2.6.36 with DEBUG defined in ec.c

In drivers/acpi/ec.c defined DEBUG.
This is the output for the 2.6.36 kernel.
This kernel fails in the sense of the bug description.

In comparison to the 2.6.31 debug output you often see
    ACPI: EC: ---> status = 0x00

Addition to the last comment: Better take a look at the end of the file as the fan and thermal modules have been loaded later (in the running system).
Then it were only approx. 5 min to the bug occurance.

can you attach the dmesg for the working 2.6.31
with CONFIG_PRINTK_TIME=y and DEBUG _disabled_ in ec.c?
would like to see it for a significant uptime, maybe
with some change in thermal load, maybe some polling of the
temperature to stress the EC.  We are looking to see
if some old workarounds are being triggered on this box.

I will try this, but it might take some time. In the mean time a friend of mine and me have found a workaround by setting EC_FLAGS_MSI in the module, after finding the following line in the logs
    ACPI: EC: missing confirmations, switch off interrupt mode.

In the current 2.6 series hard wiring a call to ec_flag_msi worked, too.

This might be far away from optimal, but is working since Friday.

please attach the dmidecode output of your laptop

Created attachment 38022 [details]
dmidecode output

Sorry for the long delay, I hadn't very few time in the last weeks.

@Len Brown: Shall I still do the tests with the 2.6.31 kernel? Actually it is no problem for me to do, but if nobody is interested in this, I would skip it. :-)

Concerning the dmidecode output: Actually I have been trying different Bios versions first. This now is the latest Bios version availabe for this laptop, but previous versions failed, too. At least those I have tried.

A question for advise: To whom should I turn, if after a necessary Bios-Update some hotkeys are not working any longer?

Created attachment 43032 [details]
dmidecode output for the Quanta TW9

I've been having the same problem on a Quanta TW9, and the workaround in comment #19 works for me. Since Jan's laptop is a Quanta TW8, I would guess that this problem (and the workaround) would apply to all laptops in the TW series.

@Len Brown

Now I have finally done some testing on the vanilla 2.6.31.14 version of the Linux kernel (again).

Actually nothing of interest seems to happen at all. I am not even seeing the message from comment #19
     ACPI: EC: missing confirmations, switch off interrupt mode.
with the desired configuration.

Here the greps "ACPI: EC:" on three runs (the full logs are available):
(1) uptime: 1 day, 12 min
[    0.101636] ACPI: EC: Look up EC in DSDT
[    0.105002] ACPI: EC: non-query interrupt received, switching to interrupt mode
[    0.121609] ACPI: EC: GPE = 0x17, I/O: command/status = 0x66, data = 0x62
[    0.121611] ACPI: EC: driver started in interrupt mode

(2) uptime: 1 day,  6:23
[    0.101489] ACPI: EC: Look up EC in DSDT
[    0.105002] ACPI: EC: non-query interrupt received, switching to interrupt mode
[    0.122509] ACPI: EC: GPE = 0x17, I/O: command/status = 0x66, data = 0x62
[    0.122511] ACPI: EC: driver started in interrupt mode

(3) uptime: 2 days, 22:34
[    0.100722] ACPI: EC: Look up EC in DSDT
[    0.104002] ACPI: EC: non-query interrupt received, switching to interrupt mode
[    0.120891] ACPI: EC: GPE = 0x17, I/O: command/status = 0x66, data = 0x62
[    0.120893] ACPI: EC: driver started in interrupt mode

And then nothing else. I have recompiled the whole system (on gentoo) now while from time to time doing EC queries with 
    (a) for i in `seq 1000`; do sensors ; done
and
    (b) for i in `seq 10000`; do 
         cat/proc/acpi/thermal_zone/TZ01/temperature ; done

with no influence on the logs at all. The uptimes are optimistic, as I had some suspend time inbetween, but the real uptime of run 3 (from dmesg) is beyond 159028 seconds (44 h).

Anything else I could test or try?

Cheers,

Jan

Hi, Jan,

can you reproduce the problem in the latest upstream kernel?
If no, can you please verify if the patch attached below (DMI quirks) helps?

Created attachment 51682 [details]
patch: enable MSI workaround for Quanta laptops

please apply this patch on 2.6.38 kernel and see if it helps.

First I want you to thank you for your continuing support.

1. Vanilla 2.6.38 problem reproduction
To be honest I am experiencing strange problems with disk access using vanilla 2.6.38. Using a patched kernel these problems are vanishing. I had no time to do further checks on these disk problems, so I can't tell if theese two behaviours are related.

2. The patch
The patch seems to work fine. From dmesg

    ACPI: EC: Detected MSI hardware, enabling workarounds.
    ACPI: EC: Look up EC in DSDT
    ACPI: Executed 1 blocks of module-level executable AML code
    [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored

so I wouldn't expect any misbehaviour, but I will continue the test over the following week. (And I will try to come back to the vanilla 2.6.38 kernel.)

Thanks,

Jan

okay. Bug closed.
please feel free to re-open it if the problem can be reproduced in the latest upstream kernel, after applying the patch.

Okay with me. Thanks to all!

Jan

A patch referencing this bug report has been merged in v3.0-rc1:

commit 534bc4e3d27096e2f3fc00c14a20efd597837a4f
Author: Zhang Rui <rui.zhang@intel.com>
Date:   Tue Apr 26 16:30:02 2011 +0800

    ACPI EC: enable MSI workaround for Quanta laptops

Hi,

Can anyone here try the upstream kernel without this quirk.
We are about to remove the quirk as it is covered by the following commit:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=9e295ac

Thanks in advance.

Best regards
-Lv

Hi,

I am sorry, but I no longer have access to the device in question.

Thanks for keeping this issue in mind!

Jan