Bug 196551 - hwlat tracer do not work in kernel commandline
Summary: hwlat tracer do not work in kernel commandline
Status: RESOLVED CODE_FIX
Alias: None
Product: Tracing/Profiling
Classification: Unclassified
Component: Ftrace (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Steven Rostedt
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-08-01 09:00 UTC by Ziqian SUN (Zamir)
Modified: 2017-08-25 01:10 UTC (History)
0 users

See Also:
Kernel Version: 4.11.9-300.fc26.x86_64
Tree: Mainline
Regression: No


Attachments

Description Ziqian SUN (Zamir) 2017-08-01 09:00:48 UTC
Based on https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt , if I set ftrace=[tracer] and reboot, system should bootup with ftrace enabled.

I tried on Fedora 26 with 4.11.9-300.fc26.x86_64, if I set ftrace=hwlat, it did not enable after bootup.

$ sudo grubby --update-kernel /boot/vmlinuz-4.11.9-300.fc26.x86_64 --args "ftrace=hwlat" ; reboot
After my laptop boot up, it shows nop in /sys/kernel/debug/tracing/current_tracer.

While the same step works as expected for other tracers except mmiotrace (System hang when boot with ftrace=mmiotrace, I will file a separate bug for this after I have more details)

I think this either should be documented or be fixed in kernel.
Comment 1 Steven Rostedt 2017-08-01 15:58:53 UTC
Yeah, that's a simple fix. It's because the hwlat tracer is registered at "lateinit" and so is the "clear_boot_tracer" which frees the saved command line, to keep it from being checked after boot up. Since they are both registered to the initcall logic at the same level, it's up to the linker to decide which one runs first. If the clear function goes first, then you have this issue. If hwlat gets called first, then it will work.

The fix is to change the clear_boot_tracer to be lateinit_sync() that is called after all the lateinit functions.
Comment 2 Ziqian SUN (Zamir) 2017-08-02 10:33:57 UTC
(In reply to Steven Rostedt from comment #1)
> The fix is to change the clear_boot_tracer to be lateinit_sync() that is
> called after all the lateinit functions.

Thanks for the explain. I've proposed my patch based on your suggestions, and I will build and test today. If it passed my test, shall I send the patch to you or you will fix it yourself?
Comment 3 Steven Rostedt 2017-08-02 11:59:05 UTC
I've already fixed it. I'll be sending the fix to Linus later today.

Thanks!
Comment 4 Ziqian SUN (Zamir) 2017-08-02 13:15:37 UTC
(In reply to Steven Rostedt from comment #3)
> I've already fixed it. I'll be sending the fix to Linus later today.
> 
> Thanks!

Ah great! Thanks for the quick fix!
Comment 5 Steven Rostedt 2017-08-25 01:10:12 UTC
Actually, due to some failed testing (and then going on vacation), the fix just got in today.

It is:
  Commit 4bb0f0e73c8c30917d169c4a0f1ac083690c545b
  tracing: Call clear_boot_tracer() at lateinit_sync

Note You need to log in before you can comment on or make changes to this bug.