06 Aug, 2020
2 commits
-
The initial assumption that all VDSO related data can be completely generic
does not hold. S390 needs architecture specific storage to access the clock
steering information.Add struct arch_vdso_data to the vdso data struct. For architectures which
do not need extra data this defaults to an empty struct. Architectures
which require it, enable CONFIG_ARCH_HAS_VDSO_DATA and provide their
specific struct in asm/vdso/data.h.Signed-off-by: Sven Schnelle
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200804150124.41692-2-svens@linux.ibm.com -
Architectures can have the requirement to add additional architecture
specific data to the VDSO data page which needs to be updated independent
of the timekeeper updates.To protect these updates vs. concurrent readers and a conflicting update
through timekeeping, provide helper functions to make such updates safe.vdso_update_begin() takes the timekeeper_lock to protect against a
potential update from timekeeper code and increments the VDSO sequence
count to signal data inconsistency to concurrent readers. vdso_update_end()
makes the sequence count even again to signal data consistency and drops
the timekeeper lock.[ Sven: Add interrupt disable handling to the functions ]
Signed-off-by: Thomas Gleixner
Signed-off-by: Sven Schnelle
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200804150124.41692-3-svens@linux.ibm.com
24 Jul, 2020
1 commit
-
Allocate the time namespace page among VVAR pages. Provide
__arch_get_timens_vdso_data() helper for VDSO code to get the
code-relative position of VVARs on that special page.If a task belongs to a time namespace then the VVAR page which contains
the system wide VDSO data is replaced with a namespace specific page
which has the same layout as the VVAR page. That page has vdso_data->seq
set to 1 to enforce the slow path and vdso_data->clock_mode set to
VCLOCK_TIMENS to enforce the time namespace handling path.The extra check in the case that vdso_data->seq is odd, e.g. a concurrent
update of the VDSO data is in progress, is not really affecting regular
tasks which are not part of a time namespace as the task is spin waiting
for the update to finish and vdso_data->seq to become even again.If a time namespace task hits that code path, it invokes the corresponding
time getter function which retrieves the real VVAR page, reads host time
and then adds the offset for the requested clock which is stored in the
special VVAR page.The time-namespace page isn't allocated on !CONFIG_TIME_NAMESPACE, but
vma is the same size, which simplifies criu/vdso migration between
different kernel configs.Signed-off-by: Andrei Vagin
Reviewed-by: Vincenzo Frascino
Reviewed-by: Dmitry Safonov
Cc: Mark Rutland
Link: https://lore.kernel.org/r/20200624083321.144975-4-avagin@gmail.com
Signed-off-by: Catalin Marinas
21 Apr, 2020
1 commit
-
While the explanation for time namespace vdso interactions is very
helpful it uses the wrong name in the comment when describing the clock
mode making grepping a bit annoying.This seems like an accidental oversight when moving from VCLOCK_TIMENS
to VDSO_CLOCKMODE_TIMENS. It seems that
660fd04f9317 ("lib/vdso: Prepare for time namespace support") misspelled
VCLOCK_TIMENS as VLOCK_TIMENS which explains why it got missed when
VCLOCK_TIMENS became VDSO_CLOCKMODE_TIMENS in
2d6b01bd88cc ("lib/vdso: Move VCLOCK_TIMENS to vdso_clock_modes").Update the comment to use VDSO_CLOCKMODE_TIMENS.
Fixes: 660fd04f9317 ("lib/vdso: Prepare for time namespace support")
Fixes: 2d6b01bd88cc ("lib/vdso: Move VCLOCK_TIMENS to vdso_clock_modes")
Signed-off-by: Christian Brauner
Signed-off-by: Thomas Gleixner
Acked-by: Andrei Vagin
Acked-by: Vincenzo Frascino
Link: https://lkml.kernel.org/r/20200420100615.1549804-1-christian.brauner@ubuntu.com
24 Mar, 2020
1 commit
-
CONFIG_GENERIC_GETTIMEOFDAY is a sufficient condition to verify if an
architecture implements asm/vdso/clocksource.h or not. The current
implementation wrongly assumes that the same is true for the config
option CONFIG_ARCH_CLOCKSOURCE_DATA.This results in a series of build errors on ia64/sparc/sparc64 like this:
In file included from ./include/linux/clocksource.h:31,
from ./include/linux/clockchips.h:14,
from ./include/linux/tick.h:8,
from fs/proc/stat.c:15:
./include/vdso/clocksource.h:9:10: fatal error: asm/vdso/clocksource.h:
No such file or directory
9 | #include
| ^~~~~~~~~~~~~~~~~~~~~~~~Fix the issue removing the unneeded config condition.
Fixes: 14ee2ac618e4 ("linux/clocksource.h: Extract common header for vDSO")
Reported-by: Thomas Gleixner
Signed-off-by: Vincenzo Frascino
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20200323133920.46546-1-vincenzo.frascino@arm.com
21 Mar, 2020
12 commits
-
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Refactor the unified vdso code to use the common headers.
Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-26-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Introduce processor.h to contain all the processor specific functions
that are suitable for vDSO inclusion.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-16-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split ktime.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-15-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split jiffies.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-14-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split time64.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-13-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split time32.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-12-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split time.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-11-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split math64.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-10-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split clocksource.h into linux and common headers to make the latter
suitable for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-9-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split limits.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-4-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split bits.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-3-vincenzo.frascino@arm.com -
The vDSO library should only include the necessary headers required for
a userspace library (UAPI and a minimal set of kernel headers). To make
this possible it is necessary to isolate from the kernel headers the
common parts that are strictly necessary to build the library.Split const.h into linux and common headers to make the latter suitable
for inclusion in the vDSO library.Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200320145351.32292-2-vincenzo.frascino@arm.com
18 Feb, 2020
1 commit
-
Move the time namespace indicator clock mode to the other ones for
consistency sake.Signed-off-by: Thomas Gleixner
Reviewed-by: Vincenzo Frascino
Link: https://lkml.kernel.org/r/20200207124403.656097274@linutronix.de
14 Jan, 2020
2 commits
-
To support time namespaces in the vdso with a minimal impact on regular non
time namespace affected tasks, the namespace handling needs to be hidden in
a slow path.The most obvious place is vdso_seq_begin(). If a task belongs to a time
namespace then the VVAR page which contains the system wide vdso data is
replaced with a namespace specific page which has the same layout as the
VVAR page. That page has vdso_data->seq set to 1 to enforce the slow path
and vdso_data->clock_mode set to VCLOCK_TIMENS to enforce the time
namespace handling path.The extra check in the case that vdso_data->seq is odd, e.g. a concurrent
update of the vdso data is in progress, is not really affecting regular
tasks which are not part of a time namespace as the task is spin waiting
for the update to finish and vdso_data->seq to become even again.If a time namespace task hits that code path, it invokes the corresponding
time getter function which retrieves the real VVAR page, reads host time
and then adds the offset for the requested clock which is stored in the
special VVAR page.If VDSO time namespace support is disabled the whole magic is compiled out.
Initial testing shows that the disabled case is almost identical to the
host case which does not take the slow timens path. With the special timens
page installed the performance hit is constant time and in the range of
5-7%.For the vdso functions which are not using the sequence count an
unconditional check for vdso_data->clock_mode is added which switches to
the real vdso when the clock_mode is VCLOCK_TIMENS.[avagin: Make do_hres_timens() work with raw clocks too: choose vdso_data
pointer by CS_RAW offset.]Suggested-by: Andy Lutomirski
Signed-off-by: Thomas Gleixner
Signed-off-by: Andrei Vagin
Signed-off-by: Dmitry Safonov
Signed-off-by: Thomas Gleixner
Link: https://lore.kernel.org/r/20191112012724.250792-21-dima@arista.com -
Place the branch with no concurrent write before the contended case.
Performance numbers for Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
(more clock_gettime() cycles - the better):
| before | after
-----------------------------------
| 150252214 | 153242367
| 150301112 | 153324800
| 150392773 | 153125401
| 150373957 | 153399355
| 150303157 | 153489417
| 150365237 | 153494270
-----------------------------------
avg | 150331408 | 153345935
diff % | 2 | 0
-----------------------------------
stdev % | 0.3 | 0.1Co-developed-by: Dmitry Safonov
Signed-off-by: Andrei Vagin
Signed-off-by: Dmitry Safonov
Signed-off-by: Thomas Gleixner
Tested-by: Vincenzo Frascino
Reviewed-by: Vincenzo Frascino
Link: https://lore.kernel.org/r/20191112012724.250792-2-dima@arista.com
26 Jun, 2019
1 commit
-
With the move to UAPI headers, such #ifdefs are no longer necessary.
Fixes: 361f8aee9b09 ("vdso: Define standardized vdso_datapage")
Signed-off-by: Catalin Marinas
Signed-off-by: Thomas Gleixner
Cc: Vincenzo Frascino
Cc: linux-arch@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-mips@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
Cc: Will Deacon
Cc: Arnd Bergmann
Cc: Russell King
Cc: Ralf Baechle
Cc: Paul Burton
Cc: Daniel Lezcano
Cc: Mark Salyzyn
Cc: Peter Collingbourne
Cc: Shuah Khan
Cc: Dmitry Safonov
Cc: Rasmus Villemoes
Cc: Huw Davies
Cc: Shijith Thotton
Cc: Andre Przywara
Link: https://lkml.kernel.org/r/20190624135624.GB29120@arrakis.emea.arm.com
23 Jun, 2019
3 commits
-
The new generic VDSO library allows to unify the update_vsyscall[_tz]()
implementations.Provide a generic implementation based on the x86 code and the bindings
which need to be implemented in architecture specific code.[ tglx: Moved it into kernel/time where it belongs. Removed the pointless
line breaks in the stub functions. Massaged changelog ]Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Tested-by: Shijith Thotton
Tested-by: Andre Przywara
Cc: linux-arch@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-mips@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
Cc: Catalin Marinas
Cc: Will Deacon
Cc: Arnd Bergmann
Cc: Russell King
Cc: Ralf Baechle
Cc: Paul Burton
Cc: Daniel Lezcano
Cc: Mark Salyzyn
Cc: Peter Collingbourne
Cc: Shuah Khan
Cc: Dmitry Safonov
Cc: Rasmus Villemoes
Cc: Huw Davies
Link: https://lkml.kernel.org/r/20190621095252.32307-4-vincenzo.frascino@arm.com -
In the last few years the kernel gained quite some architecture specific
vdso implementations which contain very similar code.Introduce a generic VDSO implementation of gettimeofday() which will be
shareable between architectures once they are converted over.The implementation is based on the current x86 VDSO code.
[ tglx: Massaged changelog and made the kernel doc tabular ]
Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Tested-by: Shijith Thotton
Tested-by: Andre Przywara
Cc: linux-arch@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-mips@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
Cc: Catalin Marinas
Cc: Will Deacon
Cc: Arnd Bergmann
Cc: Russell King
Cc: Ralf Baechle
Cc: Paul Burton
Cc: Daniel Lezcano
Cc: Mark Salyzyn
Cc: Peter Collingbourne
Cc: Shuah Khan
Cc: Dmitry Safonov
Cc: Rasmus Villemoes
Cc: Huw Davies
Link: https://lkml.kernel.org/r/20190621095252.32307-3-vincenzo.frascino@arm.com -
Define a common formet for the vdso datapage as a preparation for sharing
the VDSO implementation as a generic library.The datastructures are based on the current x86 layout.
[ tglx: Massaged changelog ]
Signed-off-by: Vincenzo Frascino
Signed-off-by: Thomas Gleixner
Tested-by: Shijith Thotton
Tested-by: Andre Przywara
Cc: linux-arch@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-mips@vger.kernel.org
Cc: linux-kselftest@vger.kernel.org
Cc: Catalin Marinas
Cc: Will Deacon
Cc: Arnd Bergmann
Cc: Russell King
Cc: Ralf Baechle
Cc: Paul Burton
Cc: Daniel Lezcano
Cc: Mark Salyzyn
Cc: Peter Collingbourne
Cc: Shuah Khan
Cc: Dmitry Safonov
Cc: Rasmus Villemoes
Cc: Huw Davies
Link: https://lkml.kernel.org/r/20190621095252.32307-2-vincenzo.frascino@arm.com