Doug / smarc-fsl-linux-kernel

Download as
Browse Code ยป

Commit 5baefd6d84163443215f4a99f6a20f054ef11236

Authored by John Stultz
Committed by Thomas Gleixner
1 parent f6c06abfb3
Exists in smarc-l5.0.0_1.0.0-ga and in 5 other branches imx_3.10.17_1.0.1_ga, imx_3.10.53_1.1.0_ga, imx_3.14.28_1.0.0_ga, smarc-imx_3.10.53_1.1.0_ga, smarc-imx_3.14.28_1.0.0_ga

hrtimer: Update hrtimer base offsets each hrtimer_interrupt 

The update of the hrtimer base offsets on all cpus cannot be made
atomically from the timekeeper.lock held and interrupt disabled region
as smp function calls are not allowed there.

clock_was_set(), which enforces the update on all cpus, is called
either from preemptible process context in case of do_settimeofday()
or from the softirq context when the offset modification happened in
the timer interrupt itself due to a leap second.

In both cases there is a race window for an hrtimer interrupt between
dropping timekeeper lock, enabling interrupts and clock_was_set()
issuing the updates. Any interrupt which arrives in that window will
see the new time but operate on stale offsets.

So we need to make sure that an hrtimer interrupt always sees a
consistent state of time and offsets.

ktime_get_update_offsets() allows us to get the current monotonic time
and update the per cpu hrtimer base offsets from hrtimer_interrupt()
to capture a consistent state of monotonic time and the offsets. The
function replaces the existing ktime_get() calls in hrtimer_interrupt().

The overhead of the new function vs. ktime_get() is minimal as it just
adds two store operations.

This ensures that any changes to realtime or boottime offsets are
noticed and stored into the per-cpu hrtimer base structures, prior to
any hrtimer expiration and guarantees that timers are not expired early.

Signed-off-by: John Stultz <johnstul@us.ibm.com>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Prarit Bhargava <prarit@redhat.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/1341960205-56738-8-git-send-email-johnstul@us.ibm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

Showing 1 changed file with 14 additions and 14 deletions Side-by-side Diff

... ... @@ -657,6 +657,14 @@
657 657 return 0;
658 658 }
659 659  
  660 +static inline ktime_t hrtimer_update_base(struct hrtimer_cpu_base *base)
  661 +{
  662 + ktime_t *offs_real = &base->clock_base[HRTIMER_BASE_REALTIME].offset;
  663 + ktime_t *offs_boot = &base->clock_base[HRTIMER_BASE_BOOTTIME].offset;
  664 +
  665 + return ktime_get_update_offsets(offs_real, offs_boot);
  666 +}
  667 +
660 668 /*
661 669 * Retrigger next event is called after clock was set
662 670 *
663 671  
664 672  
... ... @@ -665,22 +673,12 @@
665 673 static void retrigger_next_event(void *arg)
666 674 {
667 675 struct hrtimer_cpu_base *base = &__get_cpu_var(hrtimer_bases);
668   - struct timespec realtime_offset, xtim, wtm, sleep;
669 676  
670 677 if (!hrtimer_hres_active())
671 678 return;
672 679  
673   - /* Optimized out for !HIGH_RES */
674   - get_xtime_and_monotonic_and_sleep_offset(&xtim, &wtm, &sleep);
675   - set_normalized_timespec(&realtime_offset, -wtm.tv_sec, -wtm.tv_nsec);
676   -
677   - /* Adjust CLOCK_REALTIME offset */
678 680 raw_spin_lock(&base->lock);
679   - base->clock_base[HRTIMER_BASE_REALTIME].offset =
680   - timespec_to_ktime(realtime_offset);
681   - base->clock_base[HRTIMER_BASE_BOOTTIME].offset =
682   - timespec_to_ktime(sleep);
683   -
  681 + hrtimer_update_base(base);
684 682 hrtimer_force_reprogram(base, 0);
685 683 raw_spin_unlock(&base->lock);
686 684 }
... ... @@ -710,7 +708,6 @@
710 708 base->clock_base[i].resolution = KTIME_HIGH_RES;
711 709  
712 710 tick_setup_sched_timer();
713   -
714 711 /* "Retrigger" the interrupt to get things going */
715 712 retrigger_next_event(NULL);
716 713 local_irq_restore(flags);
... ... @@ -1264,7 +1261,7 @@
1264 1261 dev->next_event.tv64 = KTIME_MAX;
1265 1262  
1266 1263 raw_spin_lock(&cpu_base->lock);
1267   - entry_time = now = ktime_get();
  1264 + entry_time = now = hrtimer_update_base(cpu_base);
1268 1265 retry:
1269 1266 expires_next.tv64 = KTIME_MAX;
1270 1267 /*
1271 1268  
... ... @@ -1342,9 +1339,12 @@
1342 1339 * We need to prevent that we loop forever in the hrtimer
1343 1340 * interrupt routine. We give it 3 attempts to avoid
1344 1341 * overreacting on some spurious event.
  1342 + *
  1343 + * Acquire base lock for updating the offsets and retrieving
  1344 + * the current time.
1345 1345 */
1346 1346 raw_spin_lock(&cpu_base->lock);
1347   - now = ktime_get();
  1347 + now = hrtimer_update_base(cpu_base);
1348 1348 cpu_base->nr_retries++;
1349 1349 if (++retries < 3)
1350 1350 goto retry;