Hi team, latest git commit as of 2019-07-19 08:45 CDT
A user of the Arch package reported this error in general usage when the package is compiled using Python 3:
$ trace-cmd report > trace.log
cound not load plugin '/usr/lib/trace-cmd/plugins/plugin_python.so'
/usr/lib/trace-cmd/plugins/plugin_python.so: undefined symbol: PyString_FromString
This is found in the python/ctracecmd.i file, we're building the package using these options:
make BUILD_TYPE=Release PYTHON_VERS=python3 \
prefix="/usr" DESTDIR="$pkgdir" all doc gui
A Google tells me `PyString_FromString` is py2 only and needs replaced (PyUnicode_FromString?) but I imagine there are others, this is just the one that popped out of the woodwork for the user. I'm not a programmer by trade. :)
I've reverted our package to name PYTHON_VERS=python2 and it's functional as expected. Python 2 goes EOL/fully deprecated on 1 Jan 2020 (less than 6 months). Thanks!
Created attachment 283841 [details]
Add support for Python 3
Does the attached patch fix it for you. Note, I was not the one that added this code. I just did some googling to see Python 3 replacements for the functions that appeared to have issues.
Created attachment 283843 [details]
Add support for Python 3 v2
I had some extra changes in that patch. This is the patch that I plan on adding (it will be three patches).
> Add support for Python 3 v2
tl;dr: patch seems to work, here's how I tested it:
I replaced all 4 `python2` occurrences in latest PKGBUILD
with `python`, then applied patch from Comment 2
and this one(fixes typo `cound` => `could`):
recompiled kernel with tracing on (so that this would work `$ sudo trace-cmd record -e compaction:* sleep 10`):
$ diffkconfigs .config.lastgoodNOTRACE .config
--- /dev/fd/63 2019-07-19 23:50:47.046989084 +0200
+++ /dev/fd/62 2019-07-19 23:50:47.046989084 +0200
rebooted into new kernel
`$ cd /tmp`
(started some craziness on another terminal: `$ while true; do time stress -m 220 --vm-bytes 10000000000 --timeout 10; done`)
$ sudo trace-cmd record -e compaction:* sleep 20
[sudo] password for user:
CPU0 data recorded at offset=0x5bc000
311296 bytes in size
CPU1 data recorded at offset=0x608000
299008 bytes in size
CPU2 data recorded at offset=0x651000
335872 bytes in size
CPU3 data recorded at offset=0x6a3000
544768 bytes in size
CPU4 data recorded at offset=0x728000
286720 bytes in size
CPU5 data recorded at offset=0x76e000
417792 bytes in size
CPU6 data recorded at offset=0x7d4000
188416 bytes in size
CPU7 data recorded at offset=0x802000
253952 bytes in size
CPU8 data recorded at offset=0x840000
167936 bytes in size
CPU9 data recorded at offset=0x869000
327680 bytes in size
CPU10 data recorded at offset=0x8b9000
618496 bytes in size
CPU11 data recorded at offset=0x950000
516096 bytes in size
$ sudo trace-cmd report > trace.log
$ l trace*
-rw-r--r-- 1 root root 10280960 19.07.2019 23:54 trace.dat
-rw-r--r-- 1 user user 15610338 19.07.2019 23:55 trace.log
As you can see, no errors were outputted(also no 'py' in trace.log, case insensitive search - I went overboard just in case). So patch must've worked for my particular test case here, unless I'm missing something(hopefully it did use python 3 instead of 2).
Previously(w/o this patch, that is) the error(s) would be like this: https://aur.archlinux.org/packages/trace-cmd-git/#comment-700805
Thanks for this patch, Steven. I'll keep using it for now.
PS: kernel used 5.2.1-g527a3db363a3 with compaction patch from here: https://bugzilla.kernel.org/show_bug.cgi?id=204165#c24 otherwise trace.log could've been like 4GiB.
Thanks for verifying. And I just released 2.8.2, where I should have waited. Oh well, now I'll add a 2.8.3 ;-)
Hi Steven - howaboutsynergy is said user who reported this of course, their positive results are deeper/better than I could have tested. :) Ship it!
for completion/correction purposes, and this is a different issue, but I've noticed something that slipped my test(s): if trace-cmd-git isn't already installed on my system(ArchLinux), then compilation will fail. I used to workaround this before by not using `gui` and `install_gui` in the PKGBUILD's `make` invocations.
(already filed: https://bugzilla.kernel.org/show_bug.cgi?id=204259 for this)
to give you an idea:
1. first remove trace-cmd from system:
`$ sudo pacman -Rs trace-cmd-git ; sudo pacman -Rs trace-cmd`
2. try to compile it, using latest PKGBUILD https://aur.archlinux.org/cgit/aur.git/commit/?h=trace-cmd-git&id=d0913fea414ed795fdcc1c5c86a00fb030ec024d
-- Detecting CXX compile features - done
project: Kernel Shark: (version: 0.9.8)
CMake Error at build/FindTraceCmd.cmake:58 (MESSAGE):
Could not find trace-cmd!
Call Stack (most recent call first):
-- Configuring incomplete, errors occurred!
See also "/home/user/build/1packages/4used/trace-cmd/makepkg_pacman/trace-cmd-git/src/trace-cmd/kernel-shark/build/CMakeFiles/CMakeOutput.log".
See also "/home/user/build/1packages/4used/trace-cmd/makepkg_pacman/trace-cmd-git/src/trace-cmd/kernel-shark/build/CMakeFiles/CMakeError.log".
make: *** [Makefile:259: /home/user/build/1packages/4used/trace-cmd/makepkg_pacman/trace-cmd-git/src/trace-cmd/kernel-shark/build/Makefile] Error 1
make: *** Waiting for unfinished jobs....
@howaboutsynergy - yes, this is a current artifact of our current trace-cmd-git packaging and testing, the very latest git changes are now searching the path for the `trace-cmd` binary instead of the previous methods just awhile back (where it would use the local copy just compiled). It's basically this commit right here which I have not caught up with in the PKGBUILD yet: https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/commit/kernel-shark/build/FindTraceCmd.cmake?id=1b25ee0b18702ae8a6afff2db234bceab52bdd3c
We're at the tail end of KS 1.0 dev, now that it's basically ready for a release this is going to become two packages in Arch (`trace-cmd` and `kernelshark`) linked with standard dependencies. Steven sent an email a few days ago, he's hoping to tag KS 1.0 this week which will be our signal to move forward and take the next steps in splitting up the packaging.
Fixed by commits:
ff3c2189ddbd ("trace-cmd: Replace PySting_FromString() with PyUnicode_FromString()")
dbddc64834a7 ("trace-cmd: Use PyMemoryView_FromMemory() for Python 3")
2091de2221e9 ("trace-cmd: Use PyLong_AsLong() for Python 3")
I'm kinda lazy to make another bug for the "`cound`" typo mentioned in Comment 3
Any chance someone could just apply it anyway?
I'm only asking because it doesn't seem present in master already: https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/log/