Bug 196551

Summary:	hwlat tracer do not work in kernel commandline
Product:	Tracing/Profiling	Reporter:	Ziqian SUN (Zamir) (sztsian)
Component:	Ftrace	Assignee:	Steven Rostedt (rostedt)
Status:	RESOLVED CODE_FIX
Severity:	normal
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	4.11.9-300.fc26.x86_64	Subsystem:
Regression:	No	Bisected commit-id:

Description Ziqian SUN (Zamir) 2017-08-01 09:00:48 UTC

Based on https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt , if I set ftrace=[tracer] and reboot, system should bootup with ftrace enabled.

I tried on Fedora 26 with 4.11.9-300.fc26.x86_64, if I set ftrace=hwlat, it did not enable after bootup.

$ sudo grubby --update-kernel /boot/vmlinuz-4.11.9-300.fc26.x86_64 --args "ftrace=hwlat" ; reboot
After my laptop boot up, it shows nop in /sys/kernel/debug/tracing/current_tracer.

While the same step works as expected for other tracers except mmiotrace (System hang when boot with ftrace=mmiotrace, I will file a separate bug for this after I have more details)

I think this either should be documented or be fixed in kernel.

Comment 1 Steven Rostedt 2017-08-01 15:58:53 UTC

Yeah, that's a simple fix. It's because the hwlat tracer is registered at "lateinit" and so is the "clear_boot_tracer" which frees the saved command line, to keep it from being checked after boot up. Since they are both registered to the initcall logic at the same level, it's up to the linker to decide which one runs first. If the clear function goes first, then you have this issue. If hwlat gets called first, then it will work.

The fix is to change the clear_boot_tracer to be lateinit_sync() that is called after all the lateinit functions.

Comment 2 Ziqian SUN (Zamir) 2017-08-02 10:33:57 UTC

(In reply to Steven Rostedt from comment #1)
> The fix is to change the clear_boot_tracer to be lateinit_sync() that is
> called after all the lateinit functions.

Thanks for the explain. I've proposed my patch based on your suggestions, and I will build and test today. If it passed my test, shall I send the patch to you or you will fix it yourself?

Comment 3 Steven Rostedt 2017-08-02 11:59:05 UTC

I've already fixed it. I'll be sending the fix to Linus later today.

Thanks!

Comment 4 Ziqian SUN (Zamir) 2017-08-02 13:15:37 UTC

(In reply to Steven Rostedt from comment #3)
> I've already fixed it. I'll be sending the fix to Linus later today.
> 
> Thanks!

Ah great! Thanks for the quick fix!

Comment 5 Steven Rostedt 2017-08-25 01:10:12 UTC

Actually, due to some failed testing (and then going on vacation), the fix just got in today.

It is:
  Commit 4bb0f0e73c8c30917d169c4a0f1ac083690c545b
  tracing: Call clear_boot_tracer() at lateinit_sync