Bug 11505

Summary:	oltp ~10% regression with 2.6.27-rc5 on stoakley machine
Product:	Process Management	Reporter:	Rafael J. Wysocki (rjw)
Component:	Scheduler	Assignee:	Ingo Molnar (mingo)
Status:	CLOSED CODE_FIX
Severity:	normal	CC:	a.p.zijlstra, ming.m.lin, yanmin_zhang
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	2.6.27-rc5	Subsystem:
Regression:	Yes	Bisected commit-id:
Bug Depends on:
Bug Blocks:	11167

Description Rafael J. Wysocki 2008-09-04 15:14:54 UTC

Subject    : oltp ~10% regression with 2.6.27-rc5 on stoakley machine
Submitter  : Lin Ming <ming.m.lin@intel.com>
Date       : 2008-09-04 7:06
References : http://marc.info/?l=linux-kernel&m=122051202202373&w=4
Handled-By : Peter Zijlstra <a.p.zijlstra@chello.nl>
Handled-By : Gregory Haskins <ghaskins@novell.com>
Handled-By : Ingo Molnar <mingo@elte.hu>

This entry is being used for tracking a regression from 2.6.26.  Please don't
close it until the problem is fixed in the mainline.

Comment 1 Rafael J. Wysocki 2008-09-12 11:08:21 UTC

References : http://marc.info/?t=122089704700005&r=1&w=4

Comment 2 Peter Zijlstra 2008-09-12 11:31:32 UTC

Does it regress against .26? I can't seem to reproduce this with MySQL
+sysbench on my quad, if anything .27-rc6 is faster than .26

Linux twins.programming.kicks-ass.net 2.6.26 #122 SMP PREEMPT Thu Sep 11 13:31:43 CEST 2008 x86_64 x86_64 x86_64 GNU/Linux
  1:     transactions:                        52618  (876.97 per sec.)
  2:     transactions:                        99682  (1661.36 per sec.)
  4:     transactions:                        190681 (3177.96 per sec.)
  8:     transactions:                        186881 (3114.58 per sec.)
 16:     transactions:                        180133 (3002.02 per sec.)
 32:     transactions:                        173474 (2890.97 per sec.)
 64:     transactions:                        166760 (2778.86 per sec.)

Linux twins.programming.kicks-ass.net 2.6.27-rc6 #123 SMP PREEMPT Thu Sep 11 14:02:45 CEST 2008 x86_64 x86_64 x86_64 GNU/Linux
  1:     transactions:                        53475  (891.25 per sec.)
  2:     transactions:                        100953 (1682.52 per sec.)
  4:     transactions:                        192381 (3206.31 per sec.)
  8:     transactions:                        187336 (3122.18 per sec.)
 16:     transactions:                        180992 (3016.38 per sec.)
 32:     transactions:                        172133 (2868.60 per sec.)
 64:     transactions:                        168367 (2805.68 per sec.)

Comment 3 Lin Ming 2008-09-15 19:12:03 UTC

Yes, ~14% regression with 2.6.27-rc5 compared with 2.6.26

Comment 4 Lin Ming 2008-09-17 17:55:37 UTC

sched_switch trace of oltp in 2.6.27-rc4
http://myfreefilehosting.com/f/fc6c8eaacf_1.31MB

sched_switch trace of oltp in 2.6.27-rc5
http://myfreefilehosting.com/f/a2f9aea1b0_0.42MB

Compared the 2 trace files, you can find that (at most time),
with 2.6.27-rc5, mysql switches to sysbench when it's going to sleep
          mysqld-3791  [07]   151.421836:   3791:120:R   +  3803:120:S
          mysqld-3791  [07]   151.421876:   3791:120:S ==>  3803:120:R
        sysbench-3803  [07]   151.421878:   3803:120:R   +  3791:120:S
        sysbench-3803  [07]   151.421887:   3803:120:S ==>  3791:120:R

with 2.6.27-rc4, mysql switches to sysbench when it's still running
          mysqld-3674  [07]    95.960220:   3674:120:R   +  3687:120:S
          mysqld-3674  [07]    95.960220:   3674:120:R ==>  3687:120:R
        sysbench-3687  [07]    95.960220:   3687:120:S ==>  3674:120:R

So with 2.6.27-rc5, sysbench wakes up mysql and then switches to it (mysql is in sleep state)

With 2.6.27-rc4, sysbench switches to mysql, no need to wake up it (mysql is still in running state)

Comment 5 Rafael J. Wysocki 2008-09-21 12:25:44 UTC

Patch : http://marc.info/?l=linux-kernel&m=122194673932703&w=4

Comment 6 Lin Ming 2008-09-21 19:27:21 UTC

kernel                  peformance
-----------------------------------
27-rc4			100%
27-rc5			89.6%
27-rc5+peter's patch	94.5%

Comment 7 Rafael J. Wysocki 2008-11-16 10:54:14 UTC

This issue appears to have been fixed.  Closing.