tick: Add tick skew boot option

Let the user decide whether power consumption or jitter is the more important consideration for their machines. Quoting removal commit af5ab277ded04bd9bc6b048c5a2f0e7d70ef0867: "Historically, Linux has tried to make the regular timer tick on the various CPUs not happen at the same time, to avoid contention on xtime_lock. Nowadays, with the tickless kernel, this contention no longer happens since time keeping and updating are done differently. In addition, this skew is actually hurting power consumption in a measurable way on many-core systems." Problems: - Contrary to the above, systems do encounter contention on both xtime_lock and RCU structure locks when the tick is synchronized. - Moderate sized RT systems suffer intolerable jitter due to the tick being synchronized. - SGI reports the same for their large systems. - Fully utilized systems reap no power saving benefit from skew removal, but do suffer from resulting induced lock contention. - 0209f649 rcu: limit rcu_node leaf-level fanout This patch was born to combat lock contention which testing showed to have been _induced by_ skew removal. Skew the tick, contention disappeared virtually completely. Signed-off-by: Mike Galbraith <mgalbraith@suse.de> Link: http://lkml.kernel.org/r/1336472458.21924.78.camel@marge.simpson.net Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

tick: Add tick skew boot option
Let the user decide whether power consumption or jitter is the more important consideration for their machines. Quoting removal commit af5ab277ded04bd9bc6b048c5a2f0e7d70ef0867: "Historically, Linux has tried to make the regular timer tick on the various CPUs not happen at the same time, to avoid contention on xtime_lock. Nowadays, with the tickless kernel, this contention no longer happens since time keeping and updating are done differently. In addition, this skew is actually hurting power consumption in a measurable way on many-core systems." Problems: - Contrary to the above, systems do encounter contention on both xtime_lock and RCU structure locks when the tick is synchronized. - Moderate sized RT systems suffer intolerable jitter due to the tick being synchronized. - SGI reports the same for their large systems. - Fully utilized systems reap no power saving benefit from skew removal, but do suffer from resulting induced lock contention. - 0209f649 rcu: limit rcu_node leaf-level fanout This patch was born to combat lock contention which testing showed to have been _induced by_ skew removal. Skew the tick, contention disappeared virtually completely. Signed-off-by: Mike Galbraith <mgalbraith@suse.de> Link: http://lkml.kernel.org/r/1336472458.21924.78.camel@marge.simpson.net Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Mike Galbraith · Thomas Gleixner
1 parent ce004178be
Showing 2 changed files with 27 additions and 0 deletions Side-by-side Diff
Documentation/kernel-parameters.txt
kernel/time/tick-sched.c
@@ -2532,6 +2532,15 @@
  
 	sched_debug	[KNL] Enables verbose scheduler debug messages.
  
+	skew_tick=	[KNL] Offset the periodic timer tick per cpu to mitigate
+			xtime_lock contention on larger systems, and/or RCU lock
+			contention on all systems with CONFIG_MAXSMP set.
+			Format: { "0" | "1" }
+			0 -- disable. (may be 1 via CONFIG_CMDLINE="skew_tick=1"
+			1 -- enable.
+			Note: increases power consumption, thus should only be
+			enabled if running jitter sensitive (HPC/RT) workloads.
+
 	security=	[SECURITY] Choose a security module to enable at boot.
 			If this boot parameter is not specified, only the first
 			security module asking for security registration will be
@@ -814,6 +814,8 @@
 	return HRTIMER_RESTART;
 }
  
+static int sched_skew_tick;
+
 /**
  * tick_setup_sched_timer - setup the tick emulation timer
  */
@@ -831,6 +833,14 @@
 	/* Get the next period (per cpu) */
 	hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update());
  
+	/* Offset the tick to avert xtime_lock contention. */
+	if (sched_skew_tick) {
+		u64 offset = ktime_to_ns(tick_period) >> 1;
+		do_div(offset, num_possible_cpus());
+		offset *= smp_processor_id();
+		hrtimer_add_expires_ns(&ts->sched_timer, offset);
+	}
+
 	for (;;) {
 		hrtimer_forward(&ts->sched_timer, now, tick_period);
 		hrtimer_start_expires(&ts->sched_timer,
@@ -910,4 +920,12 @@
 	tick_nohz_switch_to_nohz();
 	return 0;
 }
+
+static int __init skew_tick(char *str)
+{
+	get_option(&str, &sched_skew_tick);
+
+	return 0;
+}
+early_param("skew_tick", skew_tick);
...	...	@@ -2532,6 +2532,15 @@
2532	2532
2533	2533	sched_debug [KNL] Enables verbose scheduler debug messages.
2534	2534
	2535	+ skew_tick= [KNL] Offset the periodic timer tick per cpu to mitigate
	2536	+ xtime_lock contention on larger systems, and/or RCU lock
	2537	+ contention on all systems with CONFIG_MAXSMP set.
	2538	+ Format: { "0" \| "1" }
	2539	+ 0 -- disable. (may be 1 via CONFIG_CMDLINE="skew_tick=1"
	2540	+ 1 -- enable.
	2541	+ Note: increases power consumption, thus should only be
	2542	+ enabled if running jitter sensitive (HPC/RT) workloads.
	2543	+
2535	2544	security= [SECURITY] Choose a security module to enable at boot.
2536	2545	If this boot parameter is not specified, only the first
2537	2546	security module asking for security registration will be
...	...	@@ -814,6 +814,8 @@
814	814	return HRTIMER_RESTART;
815	815	}
816	816
	817	+static int sched_skew_tick;
	818	+
817	819	/**
818	820	* tick_setup_sched_timer - setup the tick emulation timer
819	821	*/
...	...	@@ -831,6 +833,14 @@
831	833	/* Get the next period (per cpu) */
832	834	hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update());
833	835
	836	+ /* Offset the tick to avert xtime_lock contention. */
	837	+ if (sched_skew_tick) {
	838	+ u64 offset = ktime_to_ns(tick_period) >> 1;
	839	+ do_div(offset, num_possible_cpus());
	840	+ offset *= smp_processor_id();
	841	+ hrtimer_add_expires_ns(&ts->sched_timer, offset);
	842	+ }
	843	+
834	844	for (;;) {
835	845	hrtimer_forward(&ts->sched_timer, now, tick_period);
836	846	hrtimer_start_expires(&ts->sched_timer,
...	...	@@ -910,4 +920,12 @@
910	920	tick_nohz_switch_to_nohz();
911	921	return 0;
912	922	}
	923	+
	924	+static int __init skew_tick(char *str)
	925	+{
	926	+ get_option(&str, &sched_skew_tick);
	927	+
	928	+ return 0;
	929	+}
	930	+early_param("skew_tick", skew_tick);