Doc/cpu-hotplug: Specify race-free way to register CPU hotplug callbacks

Recommend the usage of the new CPU hotplug callback registration APIs (__register_cpu_notifier() etc), when subsystems need to also perform initialization for already online CPUs. Provide examples of correct and race-free ways of achieving this, and point out the kinds of code that are error-prone. Cc: Rob Landley <rob@landley.net> Cc: Ingo Molnar <mingo@kernel.org> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Doc/cpu-hotplug: Specify race-free way to register CPU hotplug callbacks
Recommend the usage of the new CPU hotplug callback registration APIs (__register_cpu_notifier() etc), when subsystems need to also perform initialization for already online CPUs. Provide examples of correct and race-free ways of achieving this, and point out the kinds of code that are error-prone. Cc: Rob Landley <rob@landley.net> Cc: Ingo Molnar <mingo@kernel.org> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Srivatsa S. Bhat · Rafael J. Wysocki
1 parent 93ae4f978c
Showing 1 changed file with 45 additions and 0 deletions Side-by-side Diff
Documentation/cpu-hotplug.txt
@@ -312,11 +312,56 @@
 Q: I don't see my action being called for all CPUs already up and running?
 A: Yes, CPU notifiers are called only when new CPUs are on-lined or offlined.
    If you need to perform some action for each cpu already in the system, then
+   do this:
  
 	for_each_online_cpu(i) {
 		foobar_cpu_callback(&foobar_cpu_notifier, CPU_UP_PREPARE, i);
 		foobar_cpu_callback(&foobar_cpu_notifier, CPU_ONLINE, i);
 	}
+
+   However, if you want to register a hotplug callback, as well as perform
+   some initialization for CPUs that are already online, then do this:
+
+   Version 1: (Correct)
+   ---------
+
+   	cpu_notifier_register_begin();
+
+		for_each_online_cpu(i) {
+			foobar_cpu_callback(&foobar_cpu_notifier,
+					    CPU_UP_PREPARE, i);
+			foobar_cpu_callback(&foobar_cpu_notifier,
+					    CPU_ONLINE, i);
+		}
+
+	/* Note the use of the double underscored version of the API */
+	__register_cpu_notifier(&foobar_cpu_notifier);
+
+	cpu_notifier_register_done();
+
+   Note that the following code is *NOT* the right way to achieve this,
+   because it is prone to an ABBA deadlock between the cpu_add_remove_lock
+   and the cpu_hotplug.lock.
+
+   Version 2: (Wrong!)
+   ---------
+
+	get_online_cpus();
+
+		for_each_online_cpu(i) {
+			foobar_cpu_callback(&foobar_cpu_notifier,
+					    CPU_UP_PREPARE, i);
+			foobar_cpu_callback(&foobar_cpu_notifier,
+					    CPU_ONLINE, i);
+		}
+
+	register_cpu_notifier(&foobar_cpu_notifier);
+
+	put_online_cpus();
+
+    So always use the first version shown above when you want to register
+    callbacks as well as initialize the already online CPUs.
+
  
 Q: If i would like to develop cpu hotplug support for a new architecture,
    what do i need at a minimum?
...	...	@@ -312,11 +312,56 @@
312	312	Q: I don't see my action being called for all CPUs already up and running?
313	313	A: Yes, CPU notifiers are called only when new CPUs are on-lined or offlined.
314	314	If you need to perform some action for each cpu already in the system, then
	315	+ do this:
315	316
316	317	for_each_online_cpu(i) {
317	318	foobar_cpu_callback(&foobar_cpu_notifier, CPU_UP_PREPARE, i);
318	319	foobar_cpu_callback(&foobar_cpu_notifier, CPU_ONLINE, i);
319	320	}
	321	+
	322	+ However, if you want to register a hotplug callback, as well as perform
	323	+ some initialization for CPUs that are already online, then do this:
	324	+
	325	+ Version 1: (Correct)
	326	+ ---------
	327	+
	328	+ cpu_notifier_register_begin();
	329	+
	330	+ for_each_online_cpu(i) {
	331	+ foobar_cpu_callback(&foobar_cpu_notifier,
	332	+ CPU_UP_PREPARE, i);
	333	+ foobar_cpu_callback(&foobar_cpu_notifier,
	334	+ CPU_ONLINE, i);
	335	+ }
	336	+
	337	+ /* Note the use of the double underscored version of the API */
	338	+ __register_cpu_notifier(&foobar_cpu_notifier);
	339	+
	340	+ cpu_notifier_register_done();
	341	+
	342	+ Note that the following code is NOT the right way to achieve this,
	343	+ because it is prone to an ABBA deadlock between the cpu_add_remove_lock
	344	+ and the cpu_hotplug.lock.
	345	+
	346	+ Version 2: (Wrong!)
	347	+ ---------
	348	+
	349	+ get_online_cpus();
	350	+
	351	+ for_each_online_cpu(i) {
	352	+ foobar_cpu_callback(&foobar_cpu_notifier,
	353	+ CPU_UP_PREPARE, i);
	354	+ foobar_cpu_callback(&foobar_cpu_notifier,
	355	+ CPU_ONLINE, i);
	356	+ }
	357	+
	358	+ register_cpu_notifier(&foobar_cpu_notifier);
	359	+
	360	+ put_online_cpus();
	361	+
	362	+ So always use the first version shown above when you want to register
	363	+ callbacks as well as initialize the already online CPUs.
	364	+
320	365
321	366	Q: If i would like to develop cpu hotplug support for a new architecture,
322	367	what do i need at a minimum?