29 Oct, 2019

5 commits

  • guest_enter() doesn't call context_tracking_enabled() before calling
    context_tracking_enabled_this_cpu(). Therefore the guest code doesn't
    benefit from the static key on the fast path.

    Just make sure that context_tracking_enabled_*cpu() functions check
    the static key by themselves to propagate the optimization.

    Reported-by: Peter Zijlstra
    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Jacek Anaszewski
    Cc: Linus Torvalds
    Cc: Pavel Machek
    Cc: Rafael J . Wysocki
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Viresh Kumar
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191016025700.31277-11-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • This allows us to check if a remote CPU runs vtime accounting
    (ie: is nohz_full). We'll need that to reliably support reading kcpustat
    on nohz_full CPUs.

    Also simplify a bit the condition in the local flavoured function while
    at it.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Jacek Anaszewski
    Cc: Linus Torvalds
    Cc: Pavel Machek
    Cc: Peter Zijlstra
    Cc: Rafael J . Wysocki
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Viresh Kumar
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191016025700.31277-10-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • Standardize the naming on top of the vtime_accounting_enabled_*() base.
    Also make it clear we are checking the vtime state of the
    *current* CPU with this function. We'll need to add an API to check that
    state on remote CPUs as well, so we must disambiguate the naming.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Jacek Anaszewski
    Cc: Linus Torvalds
    Cc: Pavel Machek
    Cc: Peter Zijlstra
    Cc: Rafael J . Wysocki
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Viresh Kumar
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191016025700.31277-9-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • Standardize the naming on top of the context_tracking_enabled_*() base.
    Also make it clear we are checking the context tracking state of the
    *current* CPU with this function. We'll need to add an API to check that
    state on remote CPUs as well, so we must disambiguate the naming.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Jacek Anaszewski
    Cc: Linus Torvalds
    Cc: Pavel Machek
    Cc: Peter Zijlstra
    Cc: Rafael J . Wysocki
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Viresh Kumar
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191016025700.31277-7-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • Remove the superfluous "is" in the middle of the name. We want to
    standardize the naming so that it can be expanded through suffixes:

    context_tracking_enabled()
    context_tracking_enabled_cpu()
    context_tracking_enabled_this_cpu()

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Jacek Anaszewski
    Cc: Linus Torvalds
    Cc: Pavel Machek
    Cc: Peter Zijlstra
    Cc: Rafael J . Wysocki
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Viresh Kumar
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191016025700.31277-6-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     

09 Oct, 2019

2 commits

  • On context switch we are locking the vtime seqcount of the scheduling-out
    task twice:

    * On vtime_task_switch_common(), when we flush the pending vtime through
    vtime_account_system()

    * On arch_vtime_task_switch() to reset the vtime state.

    This is pointless as these actions can be performed without the need
    to unlock/lock in the middle. The reason these steps are separated is to
    consolidate a very small amount of common code between
    CONFIG_VIRT_CPU_ACCOUNTING_GEN and CONFIG_VIRT_CPU_ACCOUNTING_NATIVE.

    Performance in this fast path is definitely a priority over artificial
    code factorization so split the task switch code between GEN and
    NATIVE and mutualize the parts than can run under a single seqcount
    locked block.

    As a side effect, vtime_account_idle() becomes included in the seqcount
    protection. This happens to be a welcome preparation in order to
    properly support kcpustat under vtime in the future and fetch
    CPUTIME_IDLE without race.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Linus Torvalds
    Cc: Peter Zijlstra
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191003161745.28464-3-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • vtime_account_system() decides if we need to account the time to the
    system (__vtime_account_system()) or to the guest (vtime_account_guest()).

    So this function is a misnomer as we are on a higher level than
    "system". All we know when we call that function is that we are
    accounting kernel cputime. Whether it belongs to guest or system time
    is a lower level detail.

    Rename this function to vtime_account_kernel(). This will clarify things
    and avoid too many underscored vtime_account_system() versions.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Linus Torvalds
    Cc: Peter Zijlstra
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Wanpeng Li
    Cc: Yauheni Kaliuta
    Link: https://lkml.kernel.org/r/20191003161745.28464-2-frederic@kernel.org
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     

02 Nov, 2017

1 commit

  • Many source files in the tree are missing licensing information, which
    makes it harder for compliance tools to determine the correct license.

    By default all files without license information are under the default
    license of the kernel, which is GPL version 2.

    Update the files which contain no license information with the 'GPL-2.0'
    SPDX license identifier. The SPDX identifier is a legally binding
    shorthand, which can be used instead of the full boiler plate text.

    This patch is based on work done by Thomas Gleixner and Kate Stewart and
    Philippe Ombredanne.

    How this work was done:

    Patches were generated and checked against linux-4.14-rc6 for a subset of
    the use cases:
    - file had no licensing information it it.
    - file was a */uapi/* one with no licensing information in it,
    - file was a */uapi/* one with existing licensing information,

    Further patches will be generated in subsequent months to fix up cases
    where non-standard license headers were used, and references to license
    had to be inferred by heuristics based on keywords.

    The analysis to determine which SPDX License Identifier to be applied to
    a file was done in a spreadsheet of side by side results from of the
    output of two independent scanners (ScanCode & Windriver) producing SPDX
    tag:value files created by Philippe Ombredanne. Philippe prepared the
    base worksheet, and did an initial spot review of a few 1000 files.

    The 4.13 kernel was the starting point of the analysis with 60,537 files
    assessed. Kate Stewart did a file by file comparison of the scanner
    results in the spreadsheet to determine which SPDX license identifier(s)
    to be applied to the file. She confirmed any determination that was not
    immediately clear with lawyers working with the Linux Foundation.

    Criteria used to select files for SPDX license identifier tagging was:
    - Files considered eligible had to be source code files.
    - Make and config files were included as candidates if they contained >5
    lines of source
    - File already had some variant of a license header in it (even if
    Reviewed-by: Philippe Ombredanne
    Reviewed-by: Thomas Gleixner
    Signed-off-by: Greg Kroah-Hartman

    Greg Kroah-Hartman
     

05 Jul, 2017

1 commit

  • It's an unnecessary function between vtime_user_exit() and
    account_user_time().

    Tested-by: Luiz Capitulino
    Signed-off-by: Frederic Weisbecker
    Reviewed-by: Thomas Gleixner
    Acked-by: Rik van Riel
    Cc: Linus Torvalds
    Cc: Peter Zijlstra
    Cc: Wanpeng Li
    Link: http://lkml.kernel.org/r/1498756511-11714-2-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     

14 Jan, 2017

1 commit

  • CONFIG_VIRT_CPU_ACCOUNTING_NATIVE=y used to accumulate user time and
    account it on ticks and context switches only through the
    vtime_account_user() function.

    Now this model has been generalized on the 3 archs for all kind of
    cputime (system, irq, ...) and all the cputime flushing happens under
    vtime_account_user().

    So let's rename this function to better reflect its new role.

    Signed-off-by: Frederic Weisbecker
    Acked-by: Thomas Gleixner
    Acked-by: Martin Schwidefsky
    Cc: Benjamin Herrenschmidt
    Cc: Christian Borntraeger
    Cc: Fenghua Yu
    Cc: Heiko Carstens
    Cc: Linus Torvalds
    Cc: Michael Ellerman
    Cc: Paul Mackerras
    Cc: Peter Zijlstra
    Cc: Rik van Riel
    Cc: Stanislaw Gruszka
    Cc: Tony Luck
    Cc: Wanpeng Li
    Link: http://lkml.kernel.org/r/1483636310-6557-11-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     

14 Jul, 2016

3 commits

  • The vtime irqtime accounting headers are very scattered and convoluted
    right now. Reorganize them such that it is obvious that only
    CONFIG_VIRT_CPU_ACCOUNTING_NATIVE does use it.

    Signed-off-by: Frederic Weisbecker
    Cc: Linus Torvalds
    Cc: Mike Galbraith
    Cc: Paolo Bonzini
    Cc: Peter Zijlstra
    Cc: Radim Krcmar
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Wanpeng Li
    Link: http://lkml.kernel.org/r/1468421405-20056-5-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • Vtime generic irqtime accounting has been removed but there are a few
    remnants to clean up:

    * The vtime_accounting_cpu_enabled() check in irq entry was only used
    by CONFIG_VIRT_CPU_ACCOUNTING_GEN. We can safely remove it.

    * Without the vtime_accounting_cpu_enabled(), we no longer need to
    have a vtime_common_account_irq_enter() indirect function.

    * Move vtime_account_irq_enter() implementation under
    CONFIG_VIRT_CPU_ACCOUNTING_NATIVE which is the last user.

    * The vtime_account_user() call was only used on irq entry for
    CONFIG_VIRT_CPU_ACCOUNTING_GEN. We can remove that too.

    Signed-off-by: Frederic Weisbecker
    Cc: Linus Torvalds
    Cc: Mike Galbraith
    Cc: Paolo Bonzini
    Cc: Peter Zijlstra
    Cc: Radim Krcmar
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Cc: Wanpeng Li
    Link: http://lkml.kernel.org/r/1468421405-20056-4-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • The CONFIG_VIRT_CPU_ACCOUNTING_GEN irq time tracking code does not
    appear to currently work right.

    On CPUs without nohz_full=, only tick based irq time sampling is
    done, which breaks down when dealing with a nohz_idle CPU.

    On firewalls and similar systems, no ticks may happen on a CPU for a
    while, and the irq time spent may never get accounted properly. This
    can cause issues with capacity planning and power saving, which use
    the CPU statistics as inputs in decision making.

    Remove the VTIME_GEN vtime irq time code, and replace it with the
    IRQ_TIME_ACCOUNTING code, when selected as a config option by the user.

    Signed-off-by: Rik van Riel
    Signed-off-by: Frederic Weisbecker
    Cc: Linus Torvalds
    Cc: Mike Galbraith
    Cc: Paolo Bonzini
    Cc: Peter Zijlstra
    Cc: Radim Krcmar
    Cc: Thomas Gleixner
    Cc: Wanpeng Li
    Link: http://lkml.kernel.org/r/1468421405-20056-3-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Rik van Riel
     

04 Dec, 2015

2 commits

  • Readers need to know if vtime runs at all on some CPU somewhere, this
    is a fast-path check to determine if we need to check further the need
    to add up any tickless cputime delta.

    This fast path check uses context tracking state because vtime is tied
    to context tracking as of now. This check appears to be confusing though
    so lets use a vtime function that deals with context tracking details
    in vtime implementation instead.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Chris Metcalf
    Cc: Christoph Lameter
    Cc: Hiroshi Shimamoto
    Cc: Linus Torvalds
    Cc: Luiz Capitulino
    Cc: Mike Galbraith
    Cc: Paul E . McKenney
    Cc: Paul E. McKenney
    Cc: Peter Zijlstra
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Link: http://lkml.kernel.org/r/1447948054-28668-7-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     
  • vtime_accounting_enabled() checks if vtime is running on the current CPU
    and is as such a misnomer. Lets rename it to a function that reflect its
    locality. We are going to need the current name for a function that tells
    if vtime runs at all on some CPU.

    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Peter Zijlstra (Intel)
    Cc: Chris Metcalf
    Cc: Christoph Lameter
    Cc: Hiroshi Shimamoto
    Cc: Linus Torvalds
    Cc: Luiz Capitulino
    Cc: Mike Galbraith
    Cc: Paul E . McKenney
    Cc: Paul E. McKenney
    Cc: Peter Zijlstra
    Cc: Rik van Riel
    Cc: Thomas Gleixner
    Link: http://lkml.kernel.org/r/1447948054-28668-6-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     

03 Dec, 2013

2 commits


14 Aug, 2013

2 commits

  • If no CPU is in the full dynticks range, we can avoid the full
    dynticks cputime accounting through generic vtime along with its
    overhead and use the traditional tick based accounting instead.

    Let's do this and nope the off case with static keys.

    Signed-off-by: Frederic Weisbecker
    Cc: Steven Rostedt
    Cc: Paul E. McKenney
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Peter Zijlstra
    Cc: Borislav Petkov
    Cc: Li Zhong
    Cc: Mike Galbraith
    Cc: Kevin Hilman

    Frederic Weisbecker
     
  • If the arch overrides some generic vtime APIs, let it describe
    these on a dedicated and standalone header. This way it becomes
    convenient to include it in vtime generic headers without irrelevant
    stuff in such a low level header.

    Signed-off-by: Frederic Weisbecker
    Cc: Steven Rostedt
    Cc: Paul E. McKenney
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Peter Zijlstra
    Cc: Borislav Petkov
    Cc: Li Zhong
    Cc: Mike Galbraith
    Cc: Kevin Hilman
    Cc: Martin Schwidefsky
    Cc: Heiko Carstens

    Frederic Weisbecker
     

31 May, 2013

1 commit

  • While computing the cputime delta of dynticks CPUs,
    we are mixing up clocks of differents natures:

    * local_clock() which takes care of unstable clock
    sources and fix these if needed.

    * sched_clock() which is the weaker version of
    local_clock(). It doesn't compute any fixup in case
    of unstable source.

    If the clock source is stable, those two clocks are the
    same and we can safely compute the difference against
    two random points.

    Otherwise it results in random deltas as sched_clock()
    can randomly drift away, back or forward, from local_clock().

    As a consequence, some strange behaviour with unstable tsc
    has been observed such as non progressing constant zero cputime.
    (The 'top' command showing no load).

    Fix this by only using local_clock(), or its irq safe/remote
    equivalent, in vtime code.

    Reported-by: Mike Galbraith
    Suggested-by: Mike Galbraith
    Cc: Steven Rostedt
    Cc: Paul E. McKenney
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Peter Zijlstra
    Cc: Borislav Petkov
    Cc: Li Zhong
    Cc: Mike Galbraith
    Signed-off-by: Frederic Weisbecker
    Signed-off-by: Ingo Molnar

    Frederic Weisbecker
     

28 Jan, 2013

4 commits

  • While remotely reading the cputime of a task running in a
    full dynticks CPU, the values stored in utime/stime fields
    of struct task_struct may be stale. Its values may be those
    of the last kernel user transition time snapshot and
    we need to add the tickless time spent since this snapshot.

    To fix this, flush the cputime of the dynticks CPUs on
    kernel user transition and record the time / context
    where we did this. Then on top of this snapshot and the current
    time, perform the fixup on the reader side from task_times()
    accessors.

    Signed-off-by: Frederic Weisbecker
    Cc: Andrew Morton
    Cc: Ingo Molnar
    Cc: Li Zhong
    Cc: Namhyung Kim
    Cc: Paul E. McKenney
    Cc: Paul Gortmaker
    Cc: Peter Zijlstra
    Cc: Steven Rostedt
    Cc: Thomas Gleixner
    [fixed kvm module related build errors]
    Signed-off-by: Sedat Dilek

    Frederic Weisbecker
     
  • Do some ground preparatory work before adding guest_enter()
    and guest_exit() context tracking callbacks. Those will
    be later used to read the guest cputime safely when we
    run in full dynticks mode.

    Signed-off-by: Frederic Weisbecker
    Cc: Andrew Morton
    Cc: Gleb Natapov
    Cc: Ingo Molnar
    Cc: Li Zhong
    Cc: Marcelo Tosatti
    Cc: Namhyung Kim
    Cc: Paul E. McKenney
    Cc: Paul Gortmaker
    Cc: Peter Zijlstra
    Cc: Steven Rostedt
    Cc: Thomas Gleixner

    Frederic Weisbecker
     
  • Allow to dynamically switch between tick and virtual based
    cputime accounting. This way we can provide a kind of "on-demand"
    virtual based cputime accounting. In this mode, the kernel relies
    on the context tracking subsystem to dynamically probe on kernel
    boundaries.

    This is in preparation for being able to stop the timer tick in
    more places than just the idle state. Doing so will depend on
    CONFIG_VIRT_CPU_ACCOUNTING_GEN which makes it possible to account
    the cputime without the tick by hooking on kernel/user boundaries.

    Depending whether the tick is stopped or not, we can switch between
    tick and vtime based accounting anytime in order to minimize the
    overhead associated to user hooks.

    Signed-off-by: Frederic Weisbecker
    Cc: Andrew Morton
    Cc: Ingo Molnar
    Cc: Li Zhong
    Cc: Namhyung Kim
    Cc: Paul E. McKenney
    Cc: Paul Gortmaker
    Cc: Peter Zijlstra
    Cc: Steven Rostedt
    Cc: Thomas Gleixner

    Frederic Weisbecker
     
  • If we want to stop the tick further idle, we need to be
    able to account the cputime without using the tick.

    Virtual based cputime accounting solves that problem by
    hooking into kernel/user boundaries.

    However implementing CONFIG_VIRT_CPU_ACCOUNTING require
    low level hooks and involves more overhead. But we already
    have a generic context tracking subsystem that is required
    for RCU needs by archs which plan to shut down the tick
    outside idle.

    This patch implements a generic virtual based cputime
    accounting that relies on these generic kernel/user hooks.

    There are some upsides of doing this:

    - This requires no arch code to implement CONFIG_VIRT_CPU_ACCOUNTING
    if context tracking is already built (already necessary for RCU in full
    tickless mode).

    - We can rely on the generic context tracking subsystem to dynamically
    (de)activate the hooks, so that we can switch anytime between virtual
    and tick based accounting. This way we don't have the overhead
    of the virtual accounting when the tick is running periodically.

    And one downside:

    - There is probably more overhead than a native virtual based cputime
    accounting. But this relies on hooks that are already set anyway.

    Signed-off-by: Frederic Weisbecker
    Cc: Andrew Morton
    Cc: Ingo Molnar
    Cc: Li Zhong
    Cc: Namhyung Kim
    Cc: Paul E. McKenney
    Cc: Paul Gortmaker
    Cc: Peter Zijlstra
    Cc: Steven Rostedt
    Cc: Thomas Gleixner

    Frederic Weisbecker
     

19 Nov, 2012

2 commits

  • All vtime implementations just flush the user time on process
    tick. Consolidate that in generic code by calling a user time
    accounting helper. This avoids an indirect call in ia64 and
    prepare to also consolidate vtime context switch code.

    Signed-off-by: Frederic Weisbecker
    Reviewed-by: Steven Rostedt
    Cc: Peter Zijlstra
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Steven Rostedt
    Cc: Paul Gortmaker
    Cc: Tony Luck
    Cc: Fenghua Yu
    Cc: Benjamin Herrenschmidt
    Cc: Paul Mackerras
    Cc: Martin Schwidefsky
    Cc: Heiko Carstens

    Frederic Weisbecker
     
  • Prepending irq-unsafe vtime APIs with underscores was actually
    a bad idea as the result is a big mess in the API namespace that
    is even waiting to be further extended. Also these helpers
    are always called from irq safe callers except kvm. Just
    provide a vtime_account_system_irqsafe() for this specific
    case so that we can remove the underscore prefix on other
    vtime functions.

    Signed-off-by: Frederic Weisbecker
    Reviewed-by: Steven Rostedt
    Cc: Peter Zijlstra
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Steven Rostedt
    Cc: Paul Gortmaker
    Cc: Tony Luck
    Cc: Fenghua Yu
    Cc: Benjamin Herrenschmidt
    Cc: Paul Mackerras
    Cc: Martin Schwidefsky
    Cc: Heiko Carstens

    Frederic Weisbecker
     

30 Oct, 2012

4 commits

  • vtime_account() doesn't have the same role in
    CONFIG_VIRT_CPU_ACCOUNTING and CONFIG_IRQ_TIME_ACCOUNTING.

    In the first case it handles time accounting in any context. In
    the second case it only handles irq time accounting.

    So when vtime_account() is called from outside vtime_account_irq_*()
    this call is pointless to CONFIG_IRQ_TIME_ACCOUNTING.

    To fix the confusion, change vtime_account() to irqtime_account_irq()
    in CONFIG_IRQ_TIME_ACCOUNTING. This way we ensure future account_vtime()
    calls won't waste useless cycles in the irqtime APIs.

    Signed-off-by: Frederic Weisbecker
    Cc: Peter Zijlstra
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Steven Rostedt
    Cc: Paul Gortmaker

    Frederic Weisbecker
     
  • With CONFIG_VIRT_CPU_ACCOUNTING, when vtime_account()
    is called in irq entry/exit, we perform a check on the
    context: if we are interrupting the idle task we
    account the pending cputime to idle, otherwise account
    to system time or its sub-areas: tsk->stime, hardirq time,
    softirq time, ...

    However this check for idle only concerns the hardirq entry
    and softirq entry:

    * Hardirq may directly interrupt the idle task, in which case
    we need to flush the pending CPU time to idle.

    * The idle task may be directly interrupted by a softirq if
    it calls local_bh_enable(). There is probably no such call
    in any idle task but we need to cover every case. Ksoftirqd
    is not concerned because the idle time is flushed on context
    switch and softirq in the end of hardirq have the idle time
    already flushed from the hardirq entry.

    In the other cases we always account to system/irq time:

    * On hardirq exit we account the time to hardirq time.
    * On softirq exit we account the time to softirq time.

    To optimize this and avoid the indirect call to vtime_account()
    and the checks it performs, specialize the vtime irq APIs and
    only perform the check on irq entry. Irq exit can directly call
    vtime_account_system().

    CONFIG_IRQ_TIME_ACCOUNTING behaviour doesn't change and directly
    maps to its own vtime_account() implementation. One may want
    to take benefits from the new APIs to optimize irq time accounting
    as well in the future.

    Signed-off-by: Frederic Weisbecker
    Cc: Peter Zijlstra
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Steven Rostedt
    Cc: Paul Gortmaker

    Frederic Weisbecker
     
  • vtime_account_system() currently has only one caller with
    vtime_account() which is irq safe.

    Now we are going to call it from other places like kvm where
    irqs are not always disabled by the time we account the cputime.

    So let's make it irqsafe. The arch implementation part is now
    prefixed with "__".

    vtime_account_idle() arch implementation is prefixed accordingly
    to stay consistent.

    Signed-off-by: Frederic Weisbecker
    Cc: Peter Zijlstra
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Steven Rostedt
    Cc: Paul Gortmaker
    Cc: Tony Luck
    Cc: Fenghua Yu
    Cc: Benjamin Herrenschmidt
    Cc: Paul Mackerras
    Cc: Martin Schwidefsky
    Cc: Heiko Carstens

    Frederic Weisbecker
     
  • These APIs are scattered around and are going to expand a bit.
    Let's create a dedicated header file for sanity.

    Signed-off-by: Frederic Weisbecker
    Cc: Peter Zijlstra
    Cc: Ingo Molnar
    Cc: Thomas Gleixner
    Cc: Steven Rostedt
    Cc: Paul Gortmaker

    Frederic Weisbecker