Created attachment 274161 [details] dmesg output We've encountered a strange issue with a piece of Python code that occurs on a single customer client: python -c ' import glob print glob.glob("/sys/firmware/acpi/tables/*") ' This prints the following error message: Traceback (most recent call last): File "<string>", line 3, in <module> File "/usr/lib/python2.6/glob.py", line 16, in glob return list(iglob(pathname)) File "/usr/lib/python2.6/glob.py", line 42, in iglob for name in glob_in_dir(dirname, basename): File "/usr/lib/python2.6/glob.py", line 60, in glob1 names = filter(lambda x: x[0] != '.', names) File "/usr/lib/python2.6/glob.py", line 60, in <lambda> names = filter(lambda x: x[0] != '.', names) IndexError: string index out of range This seems to be caused by a folder without a name in /sys/firmware/acpi/tables: root@client:~# command ls -lFa /sys/firmware/acpi/tables/ total 0 drwxr-xr-x 3 root root 0 2018-02-14 12:35 / drwxr-xr-x 3 root root 0 2018-02-14 12:35 ./ drwxr-xr-x 5 root root 0 2018-02-14 12:35 ../ -r-------- 1 root root 104 2018-02-14 12:35 APIC -r-------- 1 root root 40 2018-02-14 12:35 BOOT -r-------- 1 root root 34035 2018-02-14 12:35 DSDT drwxr-xr-x 2 root root 0 2018-02-14 12:35 dynamic/ -r-------- 1 root root 244 2018-02-14 12:35 FACP -r-------- 1 root root 64 2018-02-14 12:35 FACS1 -r-------- 1 root root 64 2018-02-14 12:35 FACS2 -r-------- 1 root root 56 2018-02-14 12:35 HPET -r-------- 1 root root 60 2018-02-14 12:35 MCFG -r-------- 1 root root 692 2018-02-14 12:35 SSDT1 -r-------- 1 root root 1263 2018-02-14 12:35 SSDT2 -r-------- 1 root root 458 2018-02-14 12:35 SSDT3 -r-------- 1 root root 4329 2018-02-14 12:35 SSDT4 -r-------- 1 root root 1095 2018-02-14 12:35 SSDT5 Bug 11539 seems to describe this issue, but that report is closed since 2008 / Linux 2.6.26. We're running: root@client:~# cat /proc/version Linux version 4.10.0-28-generic (buildd@lgw01-11) (gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2) ) #32-Ubuntu SMP Fri Jun 30 05:32:18 UTC 2017
python -c ' > import glob > print glob.glob("/sys/firmware/acpi/tables/*") > ' ['/sys/firmware/acpi/tables/SSDT', '/sys/firmware/acpi/tables/MCFG', '/sys/firmware/acpi/tables/APIC', '/sys/firmware/acpi/tables/dynamic', '/sys/firmware/acpi/tables/DSDT', '/sys/firmware/acpi/tables/FACP', '/sys/firmware/acpi/tables/data', '/sys/firmware/acpi/tables/HPET', '/sys/firmware/acpi/tables/FACS'] Seems you don' have FACS, but i'm not an expert on this, @Rui, @Erik, any ideas?
(In reply to Martin von Wittich from comment #0) > Created attachment 274161 [details] > dmesg output > > We've encountered a strange issue with a piece of Python code that occurs on > a single customer client: > > python -c ' > import glob > print glob.glob("/sys/firmware/acpi/tables/*") > ' > > This prints the following error message: > > Traceback (most recent call last): > File "<string>", line 3, in <module> > File "/usr/lib/python2.6/glob.py", line 16, in glob > return list(iglob(pathname)) > File "/usr/lib/python2.6/glob.py", line 42, in iglob > for name in glob_in_dir(dirname, basename): > File "/usr/lib/python2.6/glob.py", line 60, in glob1 > names = filter(lambda x: x[0] != '.', names) > File "/usr/lib/python2.6/glob.py", line 60, in <lambda> > names = filter(lambda x: x[0] != '.', names) > IndexError: string index out of range > > > This seems to be caused by a folder without a name in > /sys/firmware/acpi/tables: > > root@client:~# command ls -lFa /sys/firmware/acpi/tables/ > total 0 > drwxr-xr-x 3 root root 0 2018-02-14 12:35 / Could it be this ^ root directory that it's having issues with? I'm assuming that this is a symbolic link. You might try removing this link and invoke the python code again... > drwxr-xr-x 3 root root 0 2018-02-14 12:35 ./ > drwxr-xr-x 5 root root 0 2018-02-14 12:35 ../ > -r-------- 1 root root 104 2018-02-14 12:35 APIC > -r-------- 1 root root 40 2018-02-14 12:35 BOOT > -r-------- 1 root root 34035 2018-02-14 12:35 DSDT > drwxr-xr-x 2 root root 0 2018-02-14 12:35 dynamic/ > -r-------- 1 root root 244 2018-02-14 12:35 FACP > -r-------- 1 root root 64 2018-02-14 12:35 FACS1 > -r-------- 1 root root 64 2018-02-14 12:35 FACS2 > -r-------- 1 root root 56 2018-02-14 12:35 HPET > -r-------- 1 root root 60 2018-02-14 12:35 MCFG > -r-------- 1 root root 692 2018-02-14 12:35 SSDT1 > -r-------- 1 root root 1263 2018-02-14 12:35 SSDT2 > -r-------- 1 root root 458 2018-02-14 12:35 SSDT3 > -r-------- 1 root root 4329 2018-02-14 12:35 SSDT4 > -r-------- 1 root root 1095 2018-02-14 12:35 SSDT5 > > Bug 11539 seems to describe this issue, but that report is closed since 2008 > / Linux 2.6.26. We're running: > > root@client:~# cat /proc/version > Linux version 4.10.0-28-generic (buildd@lgw01-11) (gcc version 6.3.0 > 20170406 (Ubuntu 6.3.0-12ubuntu2) ) #32-Ubuntu SMP Fri Jun 30 05:32:18 UTC > 2017
I don't know what the python error means. what's the output of following bash code? for table in $(ls /sys/firmware/acpi/tables) do if [ -d $table ] then echo found directory: $table continue else echo found file: $table fi done you may also run similar code to dump the file names under /sys/firmware/acpi/tables/data/ and /sys/firmware/acpi/tables/dynamic/
(In reply to Erik Schmauss from comment #2) > Could it be this ^ root directory that it's having issues with? I'm assuming > that this is a symbolic link. You might try removing this link and invoke > the python code again... You're right, the issue is caused by this entry, although I wouldn't call it a "root" directory. It is an unnamed directory, that is, the name of that directory is "". The trailing / is produced by `ls -F` to indicate that the entry is a directory: -F, --classify append indicator (one of */=>@|) to entries The fact that this entry doesn't have a name in turn breaks the Python glob function, because it can't deal with nameless entries. I'm pretty certain that it isn't a symlink, because `ls -l` would display symlinks as `symlink => target`.
(In reply to Zhang Rui from comment #3) > I don't know what the python error means. Sorry for not clarifying this; I'll try to explain. The code crashes at this point: ``` File "/usr/lib/python2.6/glob.py", line 60, in <lambda> names = filter(lambda x: x[0] != '.', names) ``` Here you can see line 60 of glob.py of python2.6: https://hg.python.org/cpython/file/2.6/Lib/glob.py#l60 The code essentially loops through the array "names", which contains all filenames that have been generated by `os.listdir("/sys/firmware/acpi/tables")` (l. 56). It checks if the current filename begins with "." (which would indicate a hidden file). If the glob pattern begins with a ".", then it will only return filenames beginning with ".", and if the pattern doesn't, it will only return filenames not beginning with ".". Standard glob behaviour so far :) Unfortunately, the `names` array contains an empty string, ie it looks like this: ``` names = ["", "APIC", "BOOT", ... ] ``` The code crashes as soon as it tries to extract the first character of "" (the `x[0]` in the code), because the string doesn't contain any characters. As far as I know, empty filenames aren't allowed on Linux (can't find any documentation on this though, maybe I'm wrong?), so I'd argue that this is a kernel bug and not a Python bug. > what's the output of following bash code? I don't currently have access to the machine because it is a notebook on a remote customer site; I would have to ask the customer to boot it for me, because I can't wake it up remotely :/ > for table in $(ls /sys/firmware/acpi/tables) > do > if [ -d $table ] > then > echo found directory: $table > continue > else > echo found file: $table > fi > done > > > you may also run similar code to dump the file names under > /sys/firmware/acpi/tables/data/ and /sys/firmware/acpi/tables/dynamic/ I'm sorry, but I do not see how this code would be any help in the current situation. `ls` will just print an empty string for the unnamed folder that is causing this issue, and the word splitting of the shell would then eat that; the `for` loop will only ever see the valid entries of `/sys/firmware/acpi/tables`. The `ls` output in my original posting contains all the information that this code could gather. (Also, it would work only after `cd`'ing to `/sys/firmware/acpi/tables`, and parsing `ls` is always a mistake ;) )
I've got access to an affected machine again, so I've collected some further information: * `ls | cat` output - you can see the unnamed folder in the form of an empty line that shouldn't be there (`|cat` to force ls to print each filename on a line of its own; `command` builtin to circumvent `ls` aliases): root@notebooksak02:/sys/firmware/acpi/tables# command ls | cat APIC BOOT DSDT dynamic FACP FACS1 FACS2 HPET MCFG SSDT1 SSDT2 SSDT3 SSDT4 SSDT5 * `ls -l` output: root@notebooksak02:~# command ls -l /sys/firmware/acpi/tables/ total 0 drwxr-xr-x 3 root root 0 2018-04-03 10:35 -r-------- 1 root root 104 2018-04-03 10:35 APIC -r-------- 1 root root 40 2018-04-03 10:35 BOOT -r-------- 1 root root 34035 2018-04-03 10:35 DSDT drwxr-xr-x 2 root root 0 2018-04-03 10:35 dynamic -r-------- 1 root root 244 2018-04-03 10:35 FACP -r-------- 1 root root 64 2018-04-03 10:35 FACS1 -r-------- 1 root root 64 2018-04-03 10:35 FACS2 -r-------- 1 root root 56 2018-04-03 10:35 HPET -r-------- 1 root root 60 2018-04-03 10:35 MCFG -r-------- 1 root root 692 2018-04-03 10:35 SSDT1 -r-------- 1 root root 1263 2018-04-03 10:35 SSDT2 -r-------- 1 root root 458 2018-04-03 10:35 SSDT3 -r-------- 1 root root 4329 2018-04-03 10:35 SSDT4 -r-------- 1 root root 1095 2018-04-03 10:35 SSDT5 * For some reason I don't understand, the `ls` output changes (it prints lots of ??? and also an error message) when I `cd` to the folder: root@notebooksak02:~# cd /sys/firmware/acpi/tables/ root@notebooksak02:/sys/firmware/acpi/tables# command ls -l ls: cannot access : No such file or directory total 0 -????????? ? ? ? ? ? -r-------- 1 root root 104 2018-04-03 10:35 APIC -r-------- 1 root root 40 2018-04-03 10:35 BOOT -r-------- 1 root root 34035 2018-04-03 10:35 DSDT drwxr-xr-x 2 root root 0 2018-04-03 10:35 dynamic -r-------- 1 root root 244 2018-04-03 10:35 FACP -r-------- 1 root root 64 2018-04-03 10:35 FACS1 -r-------- 1 root root 64 2018-04-03 10:35 FACS2 -r-------- 1 root root 56 2018-04-03 10:35 HPET -r-------- 1 root root 60 2018-04-03 10:35 MCFG -r-------- 1 root root 692 2018-04-03 10:35 SSDT1 -r-------- 1 root root 1263 2018-04-03 10:35 SSDT2 -r-------- 1 root root 458 2018-04-03 10:35 SSDT3 -r-------- 1 root root 4329 2018-04-03 10:35 SSDT4 -r-------- 1 root root 1095 2018-04-03 10:35 SSDT5 * You can see the folder without a name in the second line of the `find` output, but it's impossible to differentiate it from the parent folder because both lines seem to end with `/`: root@notebooksak02:/sys/firmware/acpi/tables# cd root@notebooksak02:~# find /sys/firmware/acpi/tables/ /sys/firmware/acpi/tables/ /sys/firmware/acpi/tables/ /sys/firmware/acpi/tables/MCFG /sys/firmware/acpi/tables/FACS1 /sys/firmware/acpi/tables/SSDT4 /sys/firmware/acpi/tables/APIC /sys/firmware/acpi/tables/SSDT2 /sys/firmware/acpi/tables/BOOT /sys/firmware/acpi/tables/dynamic /sys/firmware/acpi/tables/dynamic/SSDT6 /sys/firmware/acpi/tables/dynamic/SSDT9 /sys/firmware/acpi/tables/dynamic/SSDT7 /sys/firmware/acpi/tables/dynamic/SSDT8 /sys/firmware/acpi/tables/DSDT /sys/firmware/acpi/tables/FACS2 /sys/firmware/acpi/tables/SSDT5 /sys/firmware/acpi/tables/FACP /sys/firmware/acpi/tables/SSDT3 /sys/firmware/acpi/tables/SSDT1 /sys/firmware/acpi/tables/HPET * `find` in the dir also looks a bit different, now you can differentiate the parent folder and the unnamed folder: root@notebooksak02:~# cd - /sys/firmware/acpi/tables root@notebooksak02:/sys/firmware/acpi/tables# find . ./ ./MCFG ./FACS1 ./SSDT4 ./APIC ./SSDT2 ./BOOT ./dynamic ./dynamic/SSDT6 ./dynamic/SSDT9 ./dynamic/SSDT7 ./dynamic/SSDT8 ./DSDT ./FACS2 ./SSDT5 ./FACP ./SSDT3 ./SSDT1 ./HPET * `find -ls` illustrates the problem better because `-ls` causes `find` to print an error message when it encounters the affected folder. I don't know for certain what `find` is trying to do when it encounters the error and I don't have `strace` on the machine, but I believe it is trying to `chdir()` into the unnamed folder: root@notebooksak02:/sys/firmware/acpi/tables# find -ls 1474 0 drwxr-xr-x 3 root root 0 Apr 3 10:35 . find: `./': No such file or directory 1482 0 -r-------- 1 root root 60 Apr 3 10:35 ./MCFG 1479 0 -r-------- 1 root root 64 Apr 3 10:35 ./FACS1 1485 0 -r-------- 1 root root 4329 Apr 3 10:35 ./SSDT4 1487 0 -r-------- 1 root root 104 Apr 3 10:35 ./APIC 1483 0 -r-------- 1 root root 1263 Apr 3 10:35 ./SSDT2 1488 0 -r-------- 1 root root 40 Apr 3 10:35 ./BOOT 1475 0 drwxr-xr-x 2 root root 0 Apr 3 10:35 ./dynamic 1493 0 -r-------- 1 root root 634 Apr 3 10:40 ./dynamic/SSDT6 1496 0 -r-------- 1 root root 71 Apr 3 10:40 ./dynamic/SSDT9 1494 0 -r-------- 1 root root 1575 Apr 3 10:40 ./dynamic/SSDT7 1495 0 -r-------- 1 root root 184 Apr 3 10:40 ./dynamic/SSDT8 1478 0 -r-------- 1 root root 34035 Apr 3 10:35 ./DSDT 1480 0 -r-------- 1 root root 64 Apr 3 10:35 ./FACS2 1486 0 -r-------- 1 root root 1095 Apr 3 10:35 ./SSDT5 1477 0 -r-------- 1 root root 244 Apr 3 10:35 ./FACP 1484 0 -r-------- 1 root root 458 Apr 3 10:35 ./SSDT3 1476 0 -r-------- 1 root root 692 Apr 3 10:35 ./SSDT1 1481 0 -r-------- 1 root root 56 Apr 3 10:35 ./HPET * Not sure if `dmidecode` is helpful at all, but it probably won't hurt to have as much information about the hardware as possible :) [see attached file] root@notebooksak02:~# dmidecode > dmidecode.txt * `os.listdir` is a Python function that lists the folder contents. This is used by `glob.glob` internally; as I explained above, this function returns an array, but the first element is an empty string (due to the unnamed folder) which causes `glob.glob` to fail. I've added the output as proof for my explanation above: root@notebooksak02:/sys/firmware/acpi/tables# cd root@notebooksak02:~# python -c ' import os print os.listdir("/sys/firmware/acpi/tables/") ' ['', 'MCFG', 'FACS1', 'SSDT4', 'APIC', 'SSDT2', 'BOOT', 'dynamic', 'DSDT', 'FACS2', 'SSDT5', 'FACP', 'SSDT3', 'SSDT1', 'HPET'] * I hope `acpidump` output might be useful. `acpidump -s` (summary): root@notebooksak02:~# ./acpidump-acpica -s ACPI: RSDP 0x000F67C0 000024 (v02 FUJ ) ACPI: RSDT 0x7F6D4944 00004C (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: XSDT 0x7F6D4990 000074 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: DSDT 0x7F6D4A04 0084F3 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: FACS 0x7F6E2FC0 000040 ACPI: SSDT 0x7F6DCEF7 0002B4 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: FACP 0x7F6DD1AB 0000F4 (v03 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: HPET 0x7F6DD29F 000038 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: MCFG 0x7F6DD2D7 00003C (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x7F6DD313 0004EF (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x7F6DD802 0001CA (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x7F6DD9CC 0010E9 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x7F6DEAB5 000447 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: APIC 0x7F6DEEFC 000068 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: BOOT 0x7F6DEF64 000028 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x00000000 00027A (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x00000000 000047 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x00000000 000627 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) ACPI: SSDT 0x00000000 0000B8 (v01 FUJ FJNB1CE 01140000 FUJ 00000100) * `acpidump` complete plain text dump: [see attached file] root@notebooksak02:~# ./acpidump-acpica | gzip -c > acpidump.txt.gz * `acpidump -b` complete binary dump: [see attached file] root@notebooksak02:~# mkdir dump root@notebooksak02:~# cd dump/ root@notebooksak02:~/dump# ../acpidump-acpica -b root@notebooksak02:~# tar cfz acpi.bin.tar.gz dump * Last but not least the output of Zhang Rui's script. I was wrong with my assumption "Also, it would work only after cd'ing...", but I think I was right by suspecting that the output wouldn't be of any help because `ls` prints an empty line for the unnamed folder, which the `for` loop will of course ignore :) (I had to remove the leading > inserted by the shell when pasting the multi-line command because they wreak havoc with bugzillas markdown parser) root@notebooksak02:~# for table in $(ls /sys/firmware/acpi/tables) do if [ -d $table ] then echo found directory: $table continue else echo found file: $table fi done found file: APIC found file: BOOT found file: DSDT found file: dynamic found file: FACP found file: FACS1 found file: FACS2 found file: HPET found file: MCFG found file: SSDT1 found file: SSDT2 found file: SSDT3 found file: SSDT4 found file: SSDT5 I can't think of any more things to collect now.
Created attachment 275065 [details] complete binary acpidump (acpidump -b)
Created attachment 275067 [details] complete plain text output of acpidump
Created attachment 275069 [details] complete dmidecode output
is there any change that you can try the latest upstream kernel on this machine, and check if there is any difference?
Hi, Martin, any new findings? does the problem still exist in the latest upstream kernel?
(In reply to Zhang Rui from comment #11) > Hi, Martin, > > any new findings? does the problem still exist in the latest upstream kernel? Sorry for not replying earlier, I forgot :( I currently don't have any access to an affected machine, so can't really try to reproduce the issue with the latest kernel. If you want you can close the bug; I'll report back when I encounter it again. Thanks!
then let's close it for now, and we can reopen it whenever you can access the machine and the problem can be reproduced in the latest upstream kernel.