Hi.. Please watch minute 5:40 in my YouTube video about compiling KernelShark on 96Boards Rock960c single-board computer. In the video, you can see the issue with visualizing the per-CPU trace data in KernelShark. This may be some OpenGL rendering issue. The Rock960c has an ARM Mali T860MP4 GPU. YouTube video: https://www.youtube.com/watch?v=VDgeDSIUWz0&t=60s Rock960c Information: https://www.96boards.org/product/rock960c/ Regards, Alan Mikhak
I have also observed the same issue on NVIDIA Jetson TX2 Developer Kit. The TX2 DevKit has an NVIDIA GPU, not Mali. TX2 DevKit Information: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-tx2/ $ uname -a Linux tegra3 4.9.140-tegra #1 SMP PREEMPT Wed Mar 13 00:30:11 PDT 2019 aarch64 aarch64 aarch64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic
I also observed the same issue on ODROID-N2 running Ubuntu 18.04 with Mate desktop running on an ARM Mali-G52 GPU. https://www.hardkernel.com/shop/odroid-n2-with-4gbyte-ram/ $ uname -a Linux odroid6 4.9.177-28 #1 SMP PREEMPT Thu May 16 23:10:54 -03 2019 aarch64 aarch64 aarch64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic
Also observed on NVIDIA Jetson Nano DevKit which has NVIDIA GPU, not Mali. https://developer.nvidia.com/embedded/buy/jetson-nano-devkit $ uname -a Linux tegra4 4.9.140-tegra #1 SMP PREEMPT Wed Mar 13 00:32:22 PDT 2019 aarch64 aarch64 aarch64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic Changed the title since observed on other GPU brand, not just Mali, and all platforms reported so far have aarch64 architecture. Also, software rasterizer may be ruled out. KernelShark uses hardware acceleration on Jetson Nano whereas it uses the software rasterizer swrast_dri.so on ODROID-N2 and Rock960c as seen by the following example from Rock960c: $ kernelshark & [1] 1203 libEGL warning: DRI2: failed to authenticate Loading "trace.dat" $ ps -ax | grep kernelshark 1203 pts/0 Sl 0:08 kernelshark $ fgrep swrast /proc/1203/maps 7f85e5e000-7f868ad000 r-xp 00000000 b3:05 11351 /usr/lib/aarch64-linux-gnu/dri/swrast_dri.so 7f868ad000-7f868bd000 ---p 00a4f000 b3:05 11351 /usr/lib/aarch64-linux-gnu/dri/swrast_dri.so 7f868bd000-7f86924000 r--p 00a4f000 b3:05 11351 /usr/lib/aarch64-linux-gnu/dri/swrast_dri.so 7f86924000-7f8692e000 rw-p 00ab6000 b3:05 11351 /usr/lib/aarch64-linux-gnu/dri/swrast_dri.so
KernelShark example program 'dplot' running on Jetson Nano DevKit did not exhibit this issue. Ruled out OpenGL since 'dplot' uses OpenGL without going through Qt5, Resolved this issue on my Nvidia Jetson Nano DevKit by building Qt5 from sources then rebuilding trace-cmd and KernelShark in order to link with newly built Qt5. KernelShark reported finding a newer Qt5Widgets version 5.12.3 during its build process. Version of Qt5Widgets installed by 'sudo apt install qtbase5-dev' was 5.9.5. Reference: Building Qt 5 from Git: https://wiki.qt.io/Building_Qt_5_from_Git Notes on building Qt5 on Nvidia Jetson Nano DevKit: $ sudo apt build-dep qt5-default $ sudo apt install libxcb-xinerama0-dev flex bison gperf libicu-dev libxslt-dev ruby libssl-dev libxcursor-dev libxcomposite-dev libxdamage-dev libxrandr-dev libdbus-1-dev libfontconfig1-dev libcap-dev libxtst-dev libpulse-dev libudev-dev libpci-dev libnss3-dev libasound2-dev libxss-dev libegl1-mesa-dev gperf bison libasound2-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev $ sudo apt install '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev $ mkdir -p ~/src/qt.io $ cd ~/src/qt.io $ git clone https://code.qt.io/qt/qt5.git $ cd qt5 $ git submodule update --init $ mkdir qt-build $ cd qt-build $ ../configure -opensource -confirm-license -release -nomake examples -openssl-runtime -prefix /usr/local/qt5 $ make $ sudo make install Added the following to ~/.bashrc to find Qt5 version 5.12.3 in /usr/local/qt5 directory: $ vi ~/.bashrc export LD_LIBRARY_PATH=/usr/local/qt5/lib:$LD_LIBRARY_PATH export PATH=/usr/local/qt5/bin:$PATH
Also resolved this issue on my ODROID-N2 and Nvidia Jetson TX2 by building Qt5 from sources then rebuilding trace-cmd and KernelShark to link with newly built Qt5.
Closing as resolved.
Hi Yordan, I installed trace-cmd and kernelshark on my 96Boards Rock960c single-board computer using the following command: $ sudo apt install trace-cmd kernelshark The version of kernelshark installed by 'apt install' runs smoothly on my 96Boards Rock960c single-board and doesn't exhibit the visualization issue I reported here about the version I built from sources. It also doesn't require Qt5 binaries that I built from sources. Regards, Alan
Per your response elsewhere, kernelshark installed by apt command is based on GTK whereas the one built from git repo sources is based on Qt5.
I have observed this issue on the following 32-bit armv7l systems as well as one more aarch64 system. I changed the title to reflect that by replacing the word 'aarch64' with 'ARM'. Raspberry Pi 3 Model B+: $ uname -a Linux rpi6 4.19.57-v7+ #1244 SMP Thu Jul 4 18:45:25 BST 2019 armv7l GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Raspbian Description: Raspbian GNU/Linux 9.9 (stretch) Release: 9.9 Codename: stretch Raspberry Pi 4 Model B: $ uname -a Linux rpi8 4.19.57-v7l+ #1244 SMP Thu Jul 4 18:48:07 BST 2019 armv7l GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Raspbian Description: Raspbian GNU/Linux 10 (buster) Release: 10 Codename: buster Nano Pi M4: $ uname -a Linux nanopi1 4.4.167 #1 SMP Sun Jun 2 18:38:17 CST 2019 aarch64 aarch64 aarch64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic
Hi Yordan, I reopened this ticket to consider a code fix for this issue. I can resolve this issue by compiling Qt5 from sources before compiling kernelshark. It seems that linking with libQt5Widgets.so version 5.12.3 or higher resolves this issue. However, compiling Qt5 from sources takes a good bit of time and effort. The Qt5 build process runs overnight on some of these aarch64 and armv7l systems. This effort is a big barrier to adopting the latest version versus the older version based on GTK. It also requires enabling a large swap file and takes up quite a bit of disk space. To gauge the effort, if you like, please consider watching another video I uploaded to YouTube which shows the process of compiling Qt5 on a Raspberry Pi 3 as needed to resolve this issue: Raspberry Pi 3: Compiled Qt 5 On Debian 9.9 (stretch) https://www.youtube.com/watch?v=NxFvJD96Yr4&t=44s I noticed that the latest Raspbian 10 (buster) release included a newer Qt5 version. Unfortunately, it is still not up to version 5.12.3. Perhaps, one solution is to wait for future Debian and Ubuntu releases to package Qt 5.12.3 or higher. In the meantime, keeping the ticket open may help keep track of the issue. Regards, Alan
Linking kernelshark with Qt5Widgets version 5.9.5 on x86_64 Ubuntu 18.04 doesn't exhibit this issue. It seems linking with Qt5Widgets version 5.12.3 or higher is required only for ARM platforms to resolve this issue.
Hi Alan, Thanks for the report! I'm guessing that you want this BZ open as documenting the issue? That is, the only real solution we have for this is to wait for Qt5Widgets version 5.12.3 to be released on the affected platforms. Is this correct? I'm fine with keeping it open for that purpose. -- Steve
Hi Steven, It would help to find it easier if it remained open until QtWidgets version 5.12.3+ is released. Purpose is document and suggest a way to resolve. It doesn't show up in the default search if closed. Regards, Alan
Looks like QtWidgets is at 5.14. Can we close this bug?
Created attachment 286373 [details] attachment-17965-0.html I recently compiled KernelShark about a month ago on the new Raspberry Pi 4 model B. I ran into the same issue because the Raspbian 10 (buster) OS didn't have QtWidgets 5.3.12 required for native compilation of KernelShark. I was able to resolve the issues the same way by compiling Qt 5.3.12 from sources. I posted two videos about how I compiled Qt 5.3.12 and KernelShark 1.1.0 on Raspbian (10) buster.KernelShark 1.1.0 video: (33 views)https://youtu.be/Svj1x6Dq3SMQt 5.3.12 video: (132 views)https://youtu.be/UtbOZq7dyvMKeeping the issue open might make it easier to find. Please close if it needs to be.------ Original message------From: bugzilla-daemon@bugzilla.kernel.orgDate: Thu, Dec 19, 2019 3:47 PMTo: amikhak@wirelessfabric.com;Cc: Subject:[Bug 203871] KernelShark per-CPU trace data visualization issue on ARMhttps://bugzilla.kernel.org/show_bug.cgi?id=203871 --- Comment #14 from Steven Rostedt (rostedt@goodmis.org) --- Looks like QtWidgets is at 5.14. Can we close this bug?
If it is still an issue, we can keep it open for making it easy to find a solution for. Hopefully Raspberry Pi will update to a more current QtWidgets soon. ;-)
Is this bug still an issue?
I just compiled KernelShark on a fresh Raspberry Pi 4. This remains an issue because the default qtbase5-dev on Raspbian 10 (buster) is still 5.11.3. However, it seems ok to close this ticket. I saw two new warnings about libtraceevent and libtracefs missing today: ** NOTICE: libtraceevent not found on system ** ** Building obsolete local version of libtraceevent ** Consider installing the latest libtraceevent Those packages are not available for Raspbian 10 (buster) on armv7l. There was also an error when doing 'sudo make install_doc': ::: make -C libtracecmd install ASCIIDOC libtracecmd-peer.3.xsl XSLTPROC libtracecmd-peer.3 INSTALL libtracecmd-peer.3 to /usr/local/share/man/man3 install: cannot stat '/home/pi/src/kernel.org/trace-cmd/Documentation/libtracecmd/libtracecmd-peer.3': No such file or directory make[2]: *** [Makefile:25: /home/pi/src/kernel.org/trace-cmd/Documentation/libtracecmd/libtracecmd-peer.3.install] Error 1 rm /home/pi/src/kernel.org/trace-cmd/Documentation/libtracecmd/libtracecmd-peer.3.xsl make[1]: *** [Makefile:24: libtracecmd] Error 2 make: *** [Makefile:494: install_doc] Error 2 These new warnings and error seem unrelated to this issue.
On Wed, 10 Mar 2021 23:52:27 +0000 bugzilla-daemon@bugzilla.kernel.org wrote: > I saw two new warnings about libtraceevent and libtracefs missing today: > > ** NOTICE: libtraceevent not found on system > ** > ** Building obsolete local version of libtraceevent > ** Consider installing the latest libtraceevent > > Those packages are not available for Raspbian 10 (buster) on armv7l. Yes, we added those warnings now that we pulled those libraries out into their own repositories. https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/ https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git/ Just download the git repositories and install them before trace-cmd. Note, libtracefs depends on libtraceevent, so you need to install libtraceevent first.
On Wed, 10 Mar 2021 23:52:27 +0000 bugzilla-daemon@bugzilla.kernel.org wrote: > There was also an error when doing 'sudo make install_doc': > ::: > make -C libtracecmd install > ASCIIDOC libtracecmd-peer.3.xsl > XSLTPROC libtracecmd-peer.3 > INSTALL libtracecmd-peer.3 to > /usr/local/share/man/man3 > install: cannot stat > > '/home/pi/src/kernel.org/trace-cmd/Documentation/libtracecmd/libtracecmd-peer.3': > No such file or directory > make[2]: *** [Makefile:25: > > /home/pi/src/kernel.org/trace-cmd/Documentation/libtracecmd/libtracecmd-peer.3.install] > Error 1 > rm > > /home/pi/src/kernel.org/trace-cmd/Documentation/libtracecmd/libtracecmd-peer.3.xsl > make[1]: *** [Makefile:24: libtracecmd] Error 2 > make: *** [Makefile:494: install_doc] Error 2 > > These new warnings and error seem unrelated to this issue. Thanks for this report. I'll look into it.
I was able to compile and install libtraceevent. I saw one warning when doing 'make': event-parse.c: In function ‘print_arg_pointer’: event-parse.c:5243:29: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] trace_seq_printf(s, "%p", (void *)val); ^ I was also able to compile and install libtracefs. I saw one error doing 'sudo make install': make[1]: '/home/pi/src/kernel.org/libtracefs/lib/tracefs/libtracefs.a' is up to date. make[1]: Nothing to be done for 'libtracefs.so'. INSTALL /home/pi/src/kernel.org/libtracefs/libtracefs.pc to /usr/local/lib/arm-linux-gnueabihf/pkgconfig INSTALL /home/pi/src/kernel.org/libtracefs/lib/tracefs/libtracefs.so.1.0.2to /usr/local/lib INSTALL /home/pi/src/kernel.org/libtracefs/include/tracefs.h to /usr/local/include/tracefs INSTALL trace.conf to /etc/ld.so.conf.d/ /etc/ld.so.conf.d//libc.conf:/usr/local/lib /etc/ld.so.conf.d//trace.conf:/usr/local/lib /etc/ld.so.conf.d//trace.conf:/usr/local/lib /bin/sh: 1: /home/pi/src/kernel.org/libtracefs/test: not found I was then able to clean and build trace-cmd and KernelShark 1.2.0. The warnings about missing libtraceevent and libtracefs were resolved. However, I noticed one more warning when doing 'make' for trace-cmd: swig -Wall -python -noproxy -I/home/pi/src/kernel.org/trace-cmd/include/traceevent -I/home/pi/src/kernel.org/trace-cmd/include/trace-cmd ctracecmd.i /home/pi/src/kernel.org/trace-cmd/include/traceevent/event-parse.h:64: Warning 451: Setting a const char * variable may leak memory.
Hi Alan, Thanks to reporting this compilation warning. It turns to be a known issue, already fixed in the kernel tree. Just backported the fix in the libtracevent repo. https://lore.kernel.org/linux-trace-devel/20200902103121.345626-1-tz.stoyanov@gmail.com/ Tzvetomir Stoyanov
On Thu, 11 Mar 2021 01:36:39 +0000 bugzilla-daemon@bugzilla.kernel.org wrote: > https://bugzilla.kernel.org/show_bug.cgi?id=203871 > > --- Comment #21 from Alan Mikhak (amikhak@wirelessfabric.com) --- > I was able to compile and install libtraceevent. I saw one warning when doing > 'make': > > event-parse.c: In function ‘print_arg_pointer’: > event-parse.c:5243:29: warning: cast to pointer from integer of different > size > [-Wint-to-pointer-cast] > trace_seq_printf(s, "%p", (void *)val); > ^ > Thanks for the report. Yeah that's probably because val is of unsigned long long, and on 32 bit it would give you that warning. It's mostly harmless, and we should probably add some logic there for when this compiled on 32 bit, because 32 bit machines should still be able to parse 64 bit trace data. > I was also able to compile and install libtracefs. I saw one error doing > 'sudo > make install': > > make[1]: '/home/pi/src/kernel.org/libtracefs/lib/tracefs/libtracefs.a' is up > to > date. > make[1]: Nothing to be done for 'libtracefs.so'. > INSTALL /home/pi/src/kernel.org/libtracefs/libtracefs.pc to > /usr/local/lib/arm-linux-gnueabihf/pkgconfig > INSTALL > /home/pi/src/kernel.org/libtracefs/lib/tracefs/libtracefs.so.1.0.2to > /usr/local/lib > INSTALL /home/pi/src/kernel.org/libtracefs/include/tracefs.h to > /usr/local/include/tracefs > INSTALL trace.conf to /etc/ld.so.conf.d/ > /etc/ld.so.conf.d//libc.conf:/usr/local/lib > /etc/ld.so.conf.d//trace.conf:/usr/local/lib > /etc/ld.so.conf.d//trace.conf:/usr/local/lib > /bin/sh: 1: /home/pi/src/kernel.org/libtracefs/test: not found I noticed this too. It's part of a hack I put in to see if "ldconfig" should be executed (and in this case the answer is no). But it shouldn't warn. I'll get around to fixing that. > > I was then able to clean and build trace-cmd and KernelShark 1.2.0. The > warnings about missing libtraceevent and libtracefs were resolved. However, I > noticed one more warning when doing 'make' for trace-cmd: > > swig -Wall -python -noproxy > -I/home/pi/src/kernel.org/trace-cmd/include/traceevent > -I/home/pi/src/kernel.org/trace-cmd/include/trace-cmd ctracecmd.i > /home/pi/src/kernel.org/trace-cmd/include/traceevent/event-parse.h:64: > Warning > 451: Setting a const char * variable may leak memory. > This is one of those cases that you simply need to ignore. swig is a way to make C code into python, and it doesn't like the fact that one of the structures has a "const char *" field. But it's necessary for the C code. We may be able to fix that, but it's low on the priority scale. Thanks for all the reports!
I just compiled KernelShark on a Jetson Nano Developer Kit. This is an ARM aarch64 platform. It has a fresh installation of Nvidia JetPack 4.5.1 which includes Nvidia L4T R32.5.1 on Linux kernel 4.9.201. I still observe the original issue I reported here but resolved that by building Qt 5.13.2 from sources. I then observed the following new error when issuing the command 'make gui'. Please see attachment for more details. ::: -- Found trace-cmd: /home/ubuntu/src/kernel.org/trace-cmd/lib/trace-cmd/libtracecmd.a CMake Error at build/FindTraceCmd.cmake:76 (MESSAGE): Could not find tracefs! Call Stack (most recent call first): CMakeLists.txt:24 (include) -- Configuring incomplete, errors occurred! ::: I was able to resolve the error by modifying kernel-shark/build/FindTraceCmd.cmake: $ cd ~/src/kernel.org/trace-cmd $ git diff diff --git a/kernel-shark/build/FindTraceCmd.cmake b/kernel-shark/build/FindTraceCmd.cmake index a40f70e..6fafb68 100644 --- a/kernel-shark/build/FindTraceCmd.cmake +++ b/kernel-shark/build/FindTraceCmd.cmake @@ -73,7 +73,9 @@ IF (TRACEFS_FOUND) ELSE (TRACEFS_FOUND) - MESSAGE(FATAL_ERROR "\nCould not find tracefs!\n") + SET(TRACEFS_INCLUDE_DIR /usr/local/include) + SET(TRACEFS_LIBRARY /usr/local/lib64/libtracefs.a) + MESSAGE(STATUS "\nCould not find tracefs!\n") ENDIF (TRACEFS_FOUND) @@ -90,6 +92,8 @@ IF (TRACEEVENT_FOUND) ELSE (TRACEEVENT_FOUND) - MESSAGE(FATAL_ERROR "\nCould not find libtraceevent!\n") + SET(TRACEEVENT_INCLUDE_DIR /usr/local/include) + SET(TRACEEVENT_LIBRARY /usr/local/lib64/libtraceevent.a) + MESSAGE(STATUS "\nCould not find libtraceevent!\n") ENDIF (TRACEEVENT_FOUND)
Created attachment 296479 [details] KernelShark 1.3.0 error on 'make gui'
Note, we are about to remove KernelShark completely from the trace-cmd.git repo. After installing libtraceevent, libtracefs, and libtracecmd, can you try installing from its new home? https://git.kernel.org/pub/scm/utils/trace-cmd/kernel-shark.git/ $ git clone git://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git $ cd libtraceevent $ make $ sudo make install $ cd .. $ git clone git://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git $ cd libtracefs $ make $ sudo make install $ cd .. $ cd trace-cmd $ make libs $ sudo make install_libs $ cd .. $ git clone git://git.kernel.org/pub/scm/utils/trace-cmd/kernel-shark.git $ cd kernel-shark $ cmake . $ make $ sudo make install We are about to officially release this new repository, and after we do, the code in trace-cmd.git will be removed. Thanks!
Thanks. The new procedure worked without error on the same Jetson Nano. I also tried the new steps on my Raspberry Pi 4 Model B running on a 32-bit Raspbian 10 (buster). It produced the following error: [ 51%] Building CXX object src/CMakeFiles/kshark-gui.dir/KsSession.cpp.o /home/pi/src/kernel.org/kernel-shark/src/KsSession.cpp: In member function ‘void KsSession::loadDualMarker(KsDualMarkerSM*, KsTraceGraph*)’: /home/pi/src/kernel.org/kernel-shark/src/KsSession.cpp:594:30: error: no matching function for call to ‘KsSession::_getMarker(const char [6], uint64_t*)’ if (_getMarker("markA", &pos)) { ^ In file included from /home/pi/src/kernel.org/kernel-shark/src/KsSession.cpp:15: /home/pi/src/kernel.org/kernel-shark/src/KsSession.hpp:108:7: note: candidate: ‘bool KsSession::_getMarker(const char*, size_t*)’ bool _getMarker(const char* name, size_t *pos); ^~~~~~~~~~ /home/pi/src/kernel.org/kernel-shark/src/KsSession.hpp:108:7: note: no known conversion for argument 2 from ‘uint64_t*’ {aka ‘long long unsigned int*’} to ‘size_t*’ {aka ‘unsigned int*’} /home/pi/src/kernel.org/kernel-shark/src/KsSession.cpp:601:30: error: no matching function for call to ‘KsSession::_getMarker(const char [6], uint64_t*)’ if (_getMarker("markB", &pos)) { ^ In file included from /home/pi/src/kernel.org/kernel-shark/src/KsSession.cpp:15: /home/pi/src/kernel.org/kernel-shark/src/KsSession.hpp:108:7: note: candidate: ‘bool KsSession::_getMarker(const char*, size_t*)’ bool _getMarker(const char* name, size_t *pos); ^~~~~~~~~~ /home/pi/src/kernel.org/kernel-shark/src/KsSession.hpp:108:7: note: no known conversion for argument 2 from ‘uint64_t*’ {aka ‘long long unsigned int*’} to ‘size_t*’ {aka ‘unsigned int*’} make[2]: *** [src/CMakeFiles/kshark-gui.dir/build.make:312: src/CMakeFiles/kshark-gui.dir/KsSession.cpp.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:233: src/CMakeFiles/kshark-gui.dir/all] Error 2 make: *** [Makefile:149: all] Error 2 I also observed the following warning before the error: [ 6%] Building C object src/CMakeFiles/kshark.dir/libkshark-tepdata.c.o /home/pi/src/kernel.org/kernel-shark/src/libkshark-tepdata.c: In function ‘tepdata_dump_entry’: /home/pi/src/kernel.org/kernel-shark/src/libkshark-tepdata.c:1028:17: warning: format ‘%li’ expects argument of type ‘long int’, but argument 4 has type ‘int64_t’ {aka ‘const long long int’} [-Wformat=] "%i; %li; [UNKNOWN TASK]-%i; CPU %i; ; [UNKNOWN EVENT]; [NO INFO]; 0x%x", ~~^ %lli /home/pi/src/kernel.org/kernel-shark/src/libkshark-tepdata.c:1030:10: entry->ts, ~~~~~~~~~ [ 7%] Building C object src/CMakeFiles/kshark.dir/libkshark-configio.c.o /home/pi/src/kernel.org/kernel-shark/src/libkshark-configio.c: In function ‘kshark_trace_file_from_json’: /home/pi/src/kernel.org/kernel-shark/src/libkshark-configio.c:570:43: warning: format ‘%li’ expects argument of type ‘long int’, but argument 3 has type ‘int64_t’ {aka ‘long long int’} [-Wformat=] fprintf(stderr, "Timestamp mismatch! (%li!=%li)\nFile %s\n", ~~^ %lli time, st.st_mtime, file_str); ~~~~ [ 9%] Building C object src/CMakeFiles/kshark.dir/libkshark-collection.c.o [ 10%] Linking C shared library ../lib/libkshark.so [ 10%] Built target kshark
Created attachment 296491 [details] patch Hi Alan, Thanks a lot for reporting those problems! Please try the attached patch and tell us if it fixes all problems on your Raspberry Pi 4. Thanks again! best, Yordan
Hi Yordan, Thanks for the patch. I applied it and observed that it resolves the compiler errors and warnings I reported on 32bit armv7l Raspbian 10 (buster). I also applied the patch on my Jetson Nano and observed no compiler errors or warnings on 64bit aarch64 Ubuntu 18.04.5 LTS (bionic). Regards, Alan
Created attachment 299539 [details] KernelShark on Raspbian 11 (bullseye) for 32bit armv7l Hi Steven, I built KernelShark on Raspbian 11 (bullseye) which was released recently for 32bit armv7l architecture on Raspberry Pi 4 Model B. The per-CPU visualization issue remains unresolved even though cmake reports finding Qt5Widgets 5.15.2 on Raspbian 11 (bullseye). Please see the attached file showing the output of the commands I issued to build. Regards, Alan
Created attachment 299541 [details] Screenshot of KernelShark on Raspbian 11 (bullseye) for 32bit armv7l Screenshot of KernelShark on Raspbian 11 (bullseye) showing no per-CPU visualization
Hi Alan, Can you attach a compressed version of the trace.dat file you used? Thanks! -- Steve
Created attachment 299543 [details] Compressed trace.dat Please see attached compressed trace.data collected with the following command: $ sudo trace-cmd record -e sched ls -ltr /usr > /dev/null Regards, Alan
Hi Alan, Thanks for reporting this! This seems to be a problem with the OpenGL widget. I guess some dependencies are still missing. Please do the following cd kernel-shark/build/ ./cmake_clean.sh cmake .. make and post the printout. Yordan
I have noticed that these kind of drawing problems sometimes happen when a specific font is missing from the system. Kernel Shark searches for FreeSans.ttf font at compile time, using fc-list command. If there is no such command, or no such font with the exactly same name - there is a warning, and usually after that there are these drawing problems. Please, check the compile log file for any strange warnings. Thanks! Tzvetomir Stoyanov
Created attachment 299553 [details] cmake_clean.sh and rebuilt kernelshark Hi Yordan and Tzvetomir, Please see attached output of cmake_clean.sh followed by rebuild of kernelshark. It shows cmake found FreeSans.ttf but per-CPU visualization was still missing. Regards, Alan
Hi Alan, I think you are failing to clean the Cmake's cache. Looking into the log you attaches I see: '-- Build files have been written to: /home/pi/src/kernel.org/kernel-shark' while all build files must go in '/home/pi/src/kernel.org/kernel-shark/build' Most probably you accidentally executed 'cmake .' inside '/home/pi/src/kernel.org/kernel-shark' and now all cache files of Cmake are there. This is why ./cmake_clean.sh fails to remove this files. Please remove all Cmake staff from '/home/pi/src/kernel.org/kernel-shark' by hand and run the installation (a clean one) again. Thanks! Yordan
Created attachment 299579 [details] re-clone and build kernelshark in build directory Hi Yordan, I re-cloned kernelshark to get a fresh copy. I executed 'cmake ../' command from the build directory. It produced the same result. I also executed the './cmake_clean.sh' command and rebuilt with the same result. Separately, please note an issue with python-dev and some compiler warnings being reported when building trace-cmd. Regards, Alan
Created attachment 299581 [details] screenshot after re-clone and build kernelshark in build directory
Created attachment 299583 [details] python-dev issue with trace-cmd
Hi Alan, Thanks for reporting these trace-cmd compile warnings. I've submitted a small patch to fix them. However, unfortunately that does not fix the Kernel Shark drawing problem, it needs further analysis. The python-dev is not mandatory, that functionality is not used by Kernel Shark. Tzvetomir Stoyanov
Created attachment 299597 [details] trace-cmd: undefined reference to 'gettid' on Jetson Nano Hi Tzvetomir, Thanks for addressing the issues from Raspbian 11 (bullseye). I also tried building kernelshark 2.0.2 on Jetson Nano running Ubuntu 18.04 for aarch64. Please see the attached issue reporting undefined reference to 'gettid' when linking trace-cmd. It was preceded by a warning about 'gettid()' when compiling tracefs-hist.c for libtracefs. Regards, Alan
Hi Alan, Unfortunately, I see nothing unusual in your installation. All external dependencies seem to be properly discovered, including OpenGL and the font. I tested building KS using the same versions of gcc and Qt but I am not able to reproduce the problem on my machine. If you are keen of digging into the code to find the problem I can give you some ideas where to start looking. Thanks! Yordan
Hi Yordan, I am keen of digging into the code. It will be good to take in your ideas about where to start looking. Regards, Alan
Hi Alan, Thanks for reporting that issue. Please, can you check if this patch resolves it: https://lore.kernel.org/linux-trace-devel/20211118165727.75994-1-tz.stoyanov@gmail.com/ I do not know if getpid() is available on that platform, but at least it is more common API. We highly appreciate these kind of reports!
Hi Tzvetomir, I verified that the patch should resolve the libtracefs compiler warning about gettid() and the trace-cmd linker error. I didn't apply the patch directly. I just edited the file and replaced gettid with getpid. It should be the same effect. I verified the above for KernelShark 2.0.2 on three ARM platforms: 1. Raspberry Pi 4 Model B running 32-bit Raspbian 11 (bullseye) for armv7l 2. Nvidia Jetson Nano DevKit running 64-bit Ubuntu 18.04 for aarch64 3. FrienlyElec NanoPi M4 running 64-bit Ubuntu 18.04 for aarch64 I didn't observe the per-CPU visualization issue with KernelShark 2.0.2 running on Jetson Nano and NanoPi M4 with Qt 5.13.2 which I built from sources to upgrade Qt as a workaround for this issue. Please see the attached screenshots. The default Qt has been upgraded to 5.15.2 in Raspbian 11 (bullseye). I have not downgraded Qt to 5.13.2 on bullseye to see if it resolves the per-CPU issue as it did on Raspbian 10 (buster). I am avoiding the downgrade since the expectation was newer versions of Qt would resolve the issue the same as 5.13.2. Regards, Alan
Created attachment 299637 [details] run kernelshark on Jetson Nano aarch64 Ubuntu 18.04
Created attachment 299639 [details] run kernelshark 2.0.2 on NanoPi M4 aarch64 Ubuntu 18.04
Created attachment 299643 [details] run kernelshark 2.0.2 built with Qt 5.13.2 on raspbian-11-bullseye-32bit-armv7l Hi Yordan, I built Qt 5.13.2 natively from sources on Raspbian 11 (bullseye) 32-bit armv7l. I then rebuilt KernelShark 2.0.2 with Qt 5.13.2. As before, the per-CPU visualization issue gets resolved with Qt 5.13.2. Regards, Alan
Hi Alan, This may be some sort of integration issue between Qt and OpenGL packages. I guess that when you build Qt from source it finds the OpenGL package and links properly to it, while the original Qt package is doing something wrong. Anyway, thank you very much for investigating the problem! Best, Yordan