From a736ccf49716dde80660d93f32358b455662d1e1 Mon Sep 17 00:00:00 2001 From: Stephan Mueller Date: Tue, 25 Apr 2023 22:22:44 +0200 Subject: [PATCH 01/25] LRNG: Entropy Source and DRNG Manager The kernel crypto API contains deterministic random number generators (DRNG) which a caller must seed and reseed. The task of seeding a DRNG is a non-trivial task requiring the consideration of a significant number of aspects. The Entropy Source and DRNG Manager (ESDM) fills that gap to transparently seed and reseed DRNGs. A user of the ESDM obtains random numbers from an appropriately seeded and initialized DRNG. Further, the ESDM controls various entropy sources guaranteeing that they are properly initialized and managed. The ESDM consists of two main parts: - The entropy source (ES) manager implemented in lrng_es_mgr.c controls the available entropy sources including pulling appropritate amount of data from them for the DRNG manager. - The DRNG manager provided with lrng_drng_mgr.c controls the DRNG(s) and ensures proper seeding and reseeding. The entropy source manager controls the entropy sources registered in the lrng_es array. The entropy sources provide a function pointer data structure that is used to obtain the services from it. The ES manager triggers the initial seeding of the DRNGs during boot time in three stages: 1. The DRNG is seeded from the entropy sources if all entropy sources collectively have at least 32 bits of entropy available. The goal of this step is to ensure that the DRNG receive some initial entropy as early as possible. 2. The DRNG is reseeded from the entropy sources if all entropy sources collectively have at least 128 bits of entropy available. 3. The DRNG is reseeded from the entropy sources if all entropy sources collectively have at least 256 bits of entropy available. At the time of the reseeding steps, the DRNG requests as much entropy as is available in order to skip certain steps and reach the seeding level of 256 bits. This may imply that one or more of the aforementioned steps are skipped. In all listed steps, the DRNG is (re)seeded with a number of random bytes from the entropy pool that is at most the amount of entropy present in the entropy pool. This means that for example when the entropy pool contains 128 or more bits of entropy, the DRNG is seeded with that amount of entropy as well. Entropy sources (ES) inform the ES manager when new entropy has been collected using the lrng_es_add_entropy() function. That function schedules a DRNG (re)seed with the DRNG manager. When the DRNG manager requests entropy data, the function lrng_fill_seed_buffer fills the seed buffer by iterating through all available ES. The output of all entropy sources is concatenated with each other. Further, the seed buffer contains the amount of entropy each entropy credits its data. Finally a time stamp is added. The ES trigger such (re)seeding events only as long as not all DRNGs are fully seeded with 256 bits of entropy. Once that seeding level is reached, the triggers are not further processed. The DRNG manager initializes the initial DRNG instance during late stage of the kernel boot process before user space is triggered. The DRNG is seeded at the following occasions: - when the DRNG is initialized, the available amount of entropy is used, - during boot time until the DRNG is fully initialized, the reaching of the aforementioned seeding steps of 32/128/256 bits of entropy trigger a reseed of the DRNG. - at runtime after the elapse of 600 seconds since the last seeding, the DRNG reseeding flag is raised - at runtime when more than 2^20 generate operations were performed by the given DRNG since last reseeding, the reseeding flag is raised Raising the reseeding flag implies that the DRNG is seeded in process context the next time a caller requests random numbers. At runtime, the DRNG manager requires at least 128 bits of entropy from the entropy sources (or 256 bits when the FIPS mode is active to be SP800-90C compliant). It may be possible that the entropy sources may not deliver that amount. The DRNG is reseeded with the available amount of entropy and continues to operate. Yet, when after 2^30 generate requests since the last seeding with 128 bits (or 256 bits in FIPS mode) the DRNG cannot be seeded with 128 bits (or 256 bits), the DRNG becomes unseeded which means it will not produce random numbers until it is fully reseeded again. To support the DRNG manager, a DRNG implementation is provided with lrng_drng_kcapi.c. It uses the kernel crypto API RNG framework and allows the specification of the used DRNG with the kernel command line option of lrng_drng_kcapi.drng_name. If no reference is given, the default is the SP800-90A DRBG. In case the chosen DRNG requires the seed to have a certain length, a hash is used bring the entropy buffer into the proper size. In addition, the DRNG manager controls the message digest implementation offered to entropy sources when they want to perform a conditioning operation. As entropy sources may require the conditioning operation at any time, the default is a SHA-256 software hash implementation that neither sleeps nor does it need any memory allocation operation. Therefore, this hash is available even for the earliest kernel operations. The initial drop of the ESDM includes the entropy source of the "auxiliary" pool. This entropy source must always be present. It is an entropy pool that is based on the state of a message digest. Every insertion of data is a hash update operation. In order to obtain data, a hash final operation is performed. The purpose of this auxiliary pool is twofold: - Provide a general interface to inject an arbitrary amount of data from any external source. When providing such data, the caller may specify the amount of entropy it contains. - The auxiliary pool also provides the backtracking resistance for all entropy sources. Once a seed buffer is filled from all entropy sources it is re-inserted into the auxiliary pool at the same time it is used for seeding the DRNG. Naturally, the insertion of the seed buffer into the auxiliary pool is not credited with any entropy. If enabled during compile time with the boot option of fips=1, the entropy source oversampling is activated. The oversampling pulls 128 more bits of entropy than originally requested. This implies that when 256 bits of entropy are requested for a (re)seed of a DRNG, the ES are queried for 384 bits. This oversampling complies with SP800-90C. This patch set contains a number of header files for subsequent additions, but with the current Kconfig settings, these additional settings will be folded to noops. Signed-off-by: Stephan Mueller --- drivers/char/Kconfig | 2 + drivers/char/Makefile | 2 + drivers/char/lrng/Kconfig | 1017 +++++++++++++++++ drivers/char/lrng/Makefile | 11 + drivers/char/lrng/lrng_definitions.h | 163 +++ drivers/char/lrng/lrng_drng_atomic.h | 23 + drivers/char/lrng/lrng_drng_chacha20.c | 195 ++++ drivers/char/lrng/lrng_drng_chacha20.h | 42 + drivers/char/lrng/lrng_drng_drbg.h | 13 + drivers/char/lrng/lrng_drng_kcapi.h | 13 + drivers/char/lrng/lrng_drng_mgr.c | 742 ++++++++++++ drivers/char/lrng/lrng_drng_mgr.h | 86 ++ drivers/char/lrng/lrng_es_aux.c | 335 ++++++ drivers/char/lrng/lrng_es_aux.h | 44 + drivers/char/lrng/lrng_es_cpu.h | 17 + drivers/char/lrng/lrng_es_irq.h | 24 + drivers/char/lrng/lrng_es_jent.h | 17 + drivers/char/lrng/lrng_es_krng.h | 17 + drivers/char/lrng/lrng_es_mgr.c | 506 ++++++++ drivers/char/lrng/lrng_es_mgr.h | 56 + drivers/char/lrng/lrng_es_mgr_cb.h | 87 ++ drivers/char/lrng/lrng_es_sched.h | 20 + drivers/char/lrng/lrng_es_timer_common.h | 83 ++ drivers/char/lrng/lrng_interface_dev_common.h | 51 + .../char/lrng/lrng_interface_random_kernel.h | 17 + drivers/char/lrng/lrng_numa.h | 11 + drivers/char/lrng/lrng_sha.h | 14 + drivers/char/lrng/lrng_sha1.c | 88 ++ drivers/char/lrng/lrng_sha256.c | 72 ++ drivers/char/lrng/lrng_sysctl.h | 15 + include/linux/lrng.h | 251 ++++ 31 files changed, 4034 insertions(+) create mode 100644 drivers/char/lrng/Kconfig create mode 100644 drivers/char/lrng/Makefile create mode 100644 drivers/char/lrng/lrng_definitions.h create mode 100644 drivers/char/lrng/lrng_drng_atomic.h create mode 100644 drivers/char/lrng/lrng_drng_chacha20.c create mode 100644 drivers/char/lrng/lrng_drng_chacha20.h create mode 100644 drivers/char/lrng/lrng_drng_drbg.h create mode 100644 drivers/char/lrng/lrng_drng_kcapi.h create mode 100644 drivers/char/lrng/lrng_drng_mgr.c create mode 100644 drivers/char/lrng/lrng_drng_mgr.h create mode 100644 drivers/char/lrng/lrng_es_aux.c create mode 100644 drivers/char/lrng/lrng_es_aux.h create mode 100644 drivers/char/lrng/lrng_es_cpu.h create mode 100644 drivers/char/lrng/lrng_es_irq.h create mode 100644 drivers/char/lrng/lrng_es_jent.h create mode 100644 drivers/char/lrng/lrng_es_krng.h create mode 100644 drivers/char/lrng/lrng_es_mgr.c create mode 100644 drivers/char/lrng/lrng_es_mgr.h create mode 100644 drivers/char/lrng/lrng_es_mgr_cb.h create mode 100644 drivers/char/lrng/lrng_es_sched.h create mode 100644 drivers/char/lrng/lrng_es_timer_common.h create mode 100644 drivers/char/lrng/lrng_interface_dev_common.h create mode 100644 drivers/char/lrng/lrng_interface_random_kernel.h create mode 100644 drivers/char/lrng/lrng_numa.h create mode 100644 drivers/char/lrng/lrng_sha.h create mode 100644 drivers/char/lrng/lrng_sha1.c create mode 100644 drivers/char/lrng/lrng_sha256.c create mode 100644 drivers/char/lrng/lrng_sysctl.h create mode 100644 include/linux/lrng.h --- a/drivers/char/Kconfig +++ b/drivers/char/Kconfig @@ -422,4 +422,6 @@ config ADI and SSM (Silicon Secured Memory). Intended consumers of this driver include crash and makedumpfile. +source "drivers/char/lrng/Kconfig" + endmenu --- a/drivers/char/Makefile +++ b/drivers/char/Makefile @@ -43,3 +43,5 @@ obj-$(CONFIG_PS3_FLASH) += ps3flash.o obj-$(CONFIG_XILLYBUS_CLASS) += xillybus/ obj-$(CONFIG_POWERNV_OP_PANEL) += powernv-op-panel.o obj-$(CONFIG_ADI) += adi.o + +obj-$(CONFIG_LRNG) += lrng/ --- /dev/null +++ b/drivers/char/lrng/Kconfig @@ -0,0 +1,1017 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Linux Random Number Generator configuration +# + +config RANDOM_DEFAULT_IMPL + bool "Kernel RNG Default Implementation" + default y + help + The default random number generator as provided with + drivers/char/random.c is selected with this option. + +config LRNG_AUTO_SELECTED + bool + default y if !RANDOM_DEFAULT_IMPL + default n if RANDOM_DEFAULT_IMPL + select LRNG + +config LRNG + bool "Linux Random Number Generator" + default n + select CRYPTO_LIB_SHA256 if CRYPTO + help + The Linux Random Number Generator (LRNG) generates entropy + from different entropy sources. Each entropy source can + be enabled and configured independently. The interrupt + entropy source can be configured to be SP800-90B compliant. + The entire LRNG can be configured to be SP800-90C compliant. + Runtime-switchable cryptographic support is available. + The LRNG delivers significant entropy during boot. + + The LRNG also provides compliance to SP800-90A/B/C. + +menu "Linux Random Number Generator Configuration" + depends on LRNG + +if LRNG + +config LRNG_SHA256 + bool + default y if CRYPTO_LIB_SHA256 + +config LRNG_SHA1 + bool + default y if !CRYPTO_LIB_SHA256 + +config LRNG_COMMON_DEV_IF + bool + +config LRNG_DRNG_ATOMIC + bool + select LRNG_DRNG_CHACHA20 + +config LRNG_SYSCTL + bool + depends on SYSCTL + +config LRNG_RANDOM_IF + bool + default n if RANDOM_DEFAULT_IMPL + default y if !RANDOM_DEFAULT_IMPL + select LRNG_COMMON_DEV_IF + select LRNG_DRNG_ATOMIC + select LRNG_SYSCTL + +menu "Specific DRNG seeding strategies" + +config LRNG_AIS2031_NTG1_SEEDING_STRATEGY + bool "AIS 20/31 NTG.1 seeding strategy" + default n + help + When enabling this option, two entropy sources must + deliver 220 bits of entropy each to consider a DRNG + as fully seeded. Any two entropy sources can be used + to fulfill this requirement. If specific entropy sources + shall not be capable of contributing to this seeding + strategy, the respective entropy source must be configured + to provide less than 220 bits of entropy. + + The strategy is consistent with the requirements for + NTG.1 compliance in German AIS 20/31 draft from 2022 + and is only enforced with lrng_es_mgr.ntg1=1. + + Compliance with German AIS 20/31 from 2011 is always + present when using /dev/random with the flag O_SYNC or + getrandom(2) with GRND_RANDOM. + + If unsure, say N. + +endmenu # "Specific DRNG seeding strategies" + +# menu "LRNG Interfaces" +# +# config LRNG_KCAPI_IF +# tristate "Interface with Kernel Crypto API" +# depends on CRYPTO_RNG +# help +# The LRNG can be registered with the kernel crypto API's +# random number generator framework. This offers a random +# number generator with the name "lrng" and a priority that +# is intended to be higher than the existing RNG +# implementations. +# +# config LRNG_HWRAND_IF +# tristate "Interface with Hardware Random Number Generator Framework" +# depends on HW_RANDOM +# select LRNG_DRNG_ATOMIC +# help +# The LRNG can be registered with the hardware random number +# generator framework. This offers a random number generator +# with the name "lrng" that is accessible via the framework. +# For example it allows pulling data from the LRNG via the +# /dev/hwrng file. +# +# config LRNG_DEV_IF +# bool "Character device file interface" +# select LRNG_COMMON_DEV_IF +# help +# The LRNG can create a character device file that operates +# identically to /dev/random including IOCTL, read and write +# operations. +# +# endmenu # "LRNG Interfaces" + +# menu "Entropy Source Configuration" +# +# config LRNG_RUNTIME_ES_CONFIG +# bool "Enable runtime configuration of entropy sources" +# help +# When enabling this option, the LRNG provides the mechanism +# allowing to alter the entropy rate of each entropy source +# during boot time and runtime. +# +# Each entropy source allows its entropy rate changed with +# a kernel command line option. When not providing any +# option, the default specified during kernel compilation +# is applied. +# +# comment "Common Timer-based Entropy Source Configuration" +# +# config LRNG_IRQ_DFLT_TIMER_ES +# bool +# +# config LRNG_SCHED_DFLT_TIMER_ES +# bool +# +# config LRNG_TIMER_COMMON +# bool +# +# choice +# prompt "Default Timer-based Entropy Source" +# default LRNG_IRQ_DFLT_TIMER_ES +# depends on LRNG_TIMER_COMMON +# help +# Select the timer-based entropy source that is credited +# with entropy. The other timer-based entropy sources may +# be operational and provide data, but are credited with no +# entropy. +# +# config LRNG_IRQ_DFLT_TIMER_ES +# bool "Interrupt Entropy Source" +# depends on LRNG_IRQ +# help +# The interrupt entropy source is selected as a timer-based +# entropy source to provide entropy. +# +# config LRNG_SCHED_DFLT_TIMER_ES +# bool "Scheduler Entropy Source" +# depends on LRNG_SCHED +# help +# The scheduler entropy source is selected as timer-based +# entropy source to provide entropy. +# endchoice +# +# choice +# prompt "LRNG Entropy Collection Pool Size" +# default LRNG_COLLECTION_SIZE_1024 +# depends on LRNG_TIMER_COMMON +# help +# Select the size of the LRNG entropy collection pool +# storing data for the interrupt as well as the scheduler +# entropy sources without performing a compression +# operation. The larger the collection size is, the faster +# the average interrupt handling will be. The collection +# size represents the number of bytes of the per-CPU memory +# used to batch up entropy event data. +# +# The default value is good for regular operations. Choose +# larger sizes for servers that have no memory limitations. +# If runtime memory is precious, choose a smaller size. +# +# The collection size is unrelated to the entropy rate +# or the amount of entropy the LRNG can process. +# +# config LRNG_COLLECTION_SIZE_32 +# depends on LRNG_CONTINUOUS_COMPRESSION_ENABLED +# depends on !LRNG_SWITCHABLE_CONTINUOUS_COMPRESSION +# depends on !CRYPTO_FIPS +# bool "32 interrupt events" +# +# config LRNG_COLLECTION_SIZE_256 +# depends on !CRYPTO_FIPS +# bool "256 interrupt events" +# +# config LRNG_COLLECTION_SIZE_512 +# bool "512 interrupt events" +# +# config LRNG_COLLECTION_SIZE_1024 +# bool "1024 interrupt events (default)" +# +# config LRNG_COLLECTION_SIZE_2048 +# bool "2048 interrupt events" +# +# config LRNG_COLLECTION_SIZE_4096 +# bool "4096 interrupt events" +# +# config LRNG_COLLECTION_SIZE_8192 +# bool "8192 interrupt events" +# +# endchoice +# +# config LRNG_COLLECTION_SIZE +# int +# default 32 if LRNG_COLLECTION_SIZE_32 +# default 256 if LRNG_COLLECTION_SIZE_256 +# default 512 if LRNG_COLLECTION_SIZE_512 +# default 1024 if LRNG_COLLECTION_SIZE_1024 +# default 2048 if LRNG_COLLECTION_SIZE_2048 +# default 4096 if LRNG_COLLECTION_SIZE_4096 +# default 8192 if LRNG_COLLECTION_SIZE_8192 +# +# config LRNG_HEALTH_TESTS +# bool "Enable internal entropy source online health tests" +# depends on LRNG_TIMER_COMMON +# help +# The online health tests applied to the interrupt entropy +# source and to the scheduler entropy source to validate +# the noise source at runtime for fatal errors. These tests +# include SP800-90B compliant tests which are invoked if +# the system is booted with fips=1. In case of fatal errors +# during active SP800-90B tests, the issue is logged and +# the noise data is discarded. These tests are required for +# full compliance of the interrupt entropy source with +# SP800-90B. +# +# If both, the scheduler and the interrupt entropy sources, +# are enabled, the health tests for both are applied +# independent of each other. +# +# If unsure, say Y. +# +# config LRNG_RCT_BROKEN +# bool "SP800-90B RCT with dangerous low cutoff value" +# depends on LRNG_HEALTH_TESTS +# depends on BROKEN +# default n +# help +# This option enables a dangerously low SP800-90B repetitive +# count test (RCT) cutoff value which makes it very likely +# that the RCT is triggered to raise a self test failure. +# +# This option is ONLY intended for developers wanting to +# test the effectiveness of the SP800-90B RCT health test. +# +# If unsure, say N. +# +# config LRNG_APT_BROKEN +# bool "SP800-90B APT with dangerous low cutoff value" +# depends on LRNG_HEALTH_TESTS +# depends on BROKEN +# default n +# help +# This option enables a dangerously low SP800-90B adaptive +# proportion test (APT) cutoff value which makes it very +# likely that the APT is triggered to raise a self test +# failure. +# +# This option is ONLY intended for developers wanting to +# test the effectiveness of the SP800-90B APT health test. +# +# If unsure, say N. +# +# Default taken from SP800-90B sec 4.4.1 - significance level 2^-30 +# config LRNG_RCT_CUTOFF +# int +# default 31 if !LRNG_RCT_BROKEN +# default 1 if LRNG_RCT_BROKEN +# +# Default taken from SP800-90B sec 4.4.1 - significance level 2^-80 +# config LRNG_RCT_CUTOFF_PERMANENT +# int +# default 81 if !LRNG_RCT_BROKEN +# default 2 if LRNG_RCT_BROKEN +# +# Default taken from SP800-90B sec 4.4.2 - significance level 2^-30 +# config LRNG_APT_CUTOFF +# int +# default 325 if !LRNG_APT_BROKEN +# default 32 if LRNG_APT_BROKEN +# +# Default taken from SP800-90B sec 4.4.2 - significance level 2^-80 +# config LRNG_APT_CUTOFF_PERMANENT +# int +# default 371 if !LRNG_APT_BROKEN +# default 33 if LRNG_APT_BROKEN +# +# comment "Interrupt Entropy Source" +# +# config LRNG_IRQ +# bool "Enable Interrupt Entropy Source as LRNG Seed Source" +# default y +# depends on !RANDOM_DEFAULT_IMPL +# select LRNG_TIMER_COMMON +# help +# The LRNG models an entropy source based on the timing of the +# occurrence of interrupts. Enable this option to enable this +# IRQ entropy source. +# +# The IRQ entropy source is triggered every time an interrupt +# arrives and thus causes the interrupt handler to execute +# slightly longer. Disabling the IRQ entropy source implies +# that the performance penalty on the interrupt handler added +# by the LRNG is eliminated. Yet, this entropy source is +# considered to be an internal entropy source of the LRNG. +# Thus, only disable it if you ensured that other entropy +# sources are available that supply the LRNG with entropy. +# +# If you disable the IRQ entropy source, you MUST ensure +# one or more entropy sources collectively have the +# capability to deliver sufficient entropy with one invocation +# at a rate compliant to the security strength of the DRNG +# (usually 256 bits of entropy). In addition, if those +# entropy sources do not deliver sufficient entropy during +# first request, the reseed must be triggered from user +# space or kernel space when sufficient entropy is considered +# to be present. +# +# If unsure, say Y. +# +# choice +# prompt "Continuous entropy compression boot time setting" +# default LRNG_CONTINUOUS_COMPRESSION_ENABLED +# depends on LRNG_IRQ +# help +# Select the default behavior of the interrupt entropy source +# continuous compression operation. +# +# The LRNG IRQ ES collects entropy data during each interrupt. +# For performance reasons, a amount of entropy data defined by +# the LRNG entropy collection pool size is concatenated into +# an array. When that array is filled up, a hash is calculated +# to compress the entropy. That hash is calculated in +# interrupt context. +# +# In case such hash calculation in interrupt context is deemed +# too time-consuming, the continuous compression operation +# can be disabled. If disabled, the collection of entropy will +# not trigger a hash compression operation in interrupt context. +# The compression happens only when the DRNG is reseeded which is +# in process context. This implies that old entropy data +# collected after the last DRNG-reseed is overwritten with newer +# entropy data once the collection pool is full instead of +# retaining its entropy with the compression operation. +# +# config LRNG_CONTINUOUS_COMPRESSION_ENABLED +# bool "Enable continuous compression (default)" +# +# config LRNG_CONTINUOUS_COMPRESSION_DISABLED +# bool "Disable continuous compression" +# +# endchoice +# +# config LRNG_ENABLE_CONTINUOUS_COMPRESSION +# bool +# default y if LRNG_CONTINUOUS_COMPRESSION_ENABLED +# default n if LRNG_CONTINUOUS_COMPRESSION_DISABLED +# +# config LRNG_SWITCHABLE_CONTINUOUS_COMPRESSION +# bool "Runtime-switchable continuous entropy compression" +# depends on LRNG_IRQ +# help +# Per default, the interrupt entropy source continuous +# compression operation behavior is hard-wired into the kernel. +# Enable this option to allow it to be configurable at boot time. +# +# To modify the default behavior of the continuous +# compression operation, use the kernel command line option +# of lrng_sw_noise.lrng_pcpu_continuous_compression. +# +# If unsure, say N. +# +# config LRNG_IRQ_ENTROPY_RATE +# int "Interrupt Entropy Source Entropy Rate" +# depends on LRNG_IRQ +# range 256 4294967295 if LRNG_IRQ_DFLT_TIMER_ES +# range 4294967295 4294967295 if !LRNG_IRQ_DFLT_TIMER_ES +# default 256 if LRNG_IRQ_DFLT_TIMER_ES +# default 4294967295 if !LRNG_IRQ_DFLT_TIMER_ES +# help +# The LRNG will collect the configured number of interrupts to +# obtain 256 bits of entropy. This value can be set to any between +# 256 and 4294967295. The LRNG guarantees that this value is not +# lower than 256. This lower limit implies that one interrupt event +# is credited with one bit of entropy. This value is subject to the +# increase by the oversampling factor, if no high-resolution timer +# is found. +# +# In order to effectively disable the interrupt entropy source, +# the option has to be set to 4294967295. In this case, the +# interrupt entropy source will still deliver data but without +# being credited with entropy. +# +# comment "Jitter RNG Entropy Source" +# +# config LRNG_JENT +# bool "Enable Jitter RNG as LRNG Seed Source" +# depends on CRYPTO +# select CRYPTO_JITTERENTROPY +# help +# The LRNG may use the Jitter RNG as entropy source. Enabling +# this option enables the use of the Jitter RNG. Its default +# entropy level is 16 bits of entropy per 256 data bits delivered +# by the Jitter RNG. This entropy level can be changed at boot +# time or at runtime with the lrng_base.jitterrng configuration +# variable. +# +#choice +# prompt "Jitter RNG Async Block Number" +# default LRNG_JENT_ENTROPY_BLOCKS_NO_128 +# depends on LRNG_JENT +# help +# Select the number of Jitter RNG entropy blocks the asynchronous +# collection operation will fill. A caller for Jitter RNG entropy +# will be given data from the pre-filled blocks if available to +# prevent the Jitter RNG from utilizing the CPU too much in a +# possible hot code path. +# +# The number specifies the number of 256/384 bit blocks that will +# be held in memory and asynchronously filled with Jitter RNG data. +# +# The asynchronous entropy collection can also be disabled at +# kernel startup time when setting the command line option of +# lrng_es_jent.jent_async_enabled=0. Also, setting this option at +# runtime is allowed via the corresponding SysFS interface. This +# option is only available with the options SysFS and +# CONFIG_LRNG_RUNTIME_ES_CONFIG enabled. +# +# config LRNG_JENT_ENTROPY_BLOCKS_DISABLED +# bool "Async collection disabled" +# +# # Any block number is allowed, provided it is a power of 2 and +# # equal or larger than 4 (4 is due to the division in +# # lrng_jent_async_get when deciding to wake up the monitor). +# config LRNG_JENT_ENTROPY_BLOCKS_NO_32 +# bool "32 blocks" +# +# config LRNG_JENT_ENTROPY_BLOCKS_NO_64 +# bool "64 blocks" +# +# config LRNG_JENT_ENTROPY_BLOCKS_NO_128 +# bool "128 blocks (default)" +# +# config LRNG_JENT_ENTROPY_BLOCKS_NO_256 +# bool "256 blocks" +# +# config LRNG_JENT_ENTROPY_BLOCKS_NO_512 +# bool "512 blocks" +# +# config LRNG_JENT_ENTROPY_BLOCKS_NO_1024 +# bool "1024 blocks" +# +#endchoice +# +#config LRNG_JENT_ENTROPY_BLOCKS +# int +# default 0 if LRNG_JENT_ENTROPY_BLOCKS_DISABLED +# default 32 if LRNG_JENT_ENTROPY_BLOCKS_NO_32 +# default 64 if LRNG_JENT_ENTROPY_BLOCKS_NO_64 +# default 128 if LRNG_JENT_ENTROPY_BLOCKS_NO_128 +# default 256 if LRNG_JENT_ENTROPY_BLOCKS_NO_256 +# default 512 if LRNG_JENT_ENTROPY_BLOCKS_NO_512 +# default 1024 if LRNG_JENT_ENTROPY_BLOCKS_NO_1024 +# +# config LRNG_JENT_ENTROPY_RATE +# int "Jitter RNG Entropy Source Entropy Rate" +# depends on LRNG_JENT +# range 0 256 +# default 16 +# help +# The option defines the amount of entropy the LRNG applies to 256 +# bits of data obtained from the Jitter RNG entropy source. The +# LRNG enforces the limit that this value must be in the range +# between 0 and 256. +# +# When configuring this value to 0, the Jitter RNG entropy source +# will provide 256 bits of data without being credited to contain +# entropy. +# +# comment "CPU Entropy Source" +# +# config LRNG_CPU +# bool "Enable CPU Entropy Source as LRNG Seed Source" +# default y +# help +# Current CPUs commonly contain entropy sources which can be +# used to seed the LRNG. For example, the Intel RDSEED +# instruction, or the POWER DARN instruction will be sourced +# to seed the LRNG if this option is enabled. +# +# Note, if this option is enabled and the underlying CPU +# does not offer such entropy source, the LRNG will automatically +# detect this and ignore the hardware. +# +# config LRNG_CPU_FULL_ENT_MULTIPLIER +# int +# default 1 if !LRNG_TEST_CPU_ES_COMPRESSION +# default 123 if LRNG_TEST_CPU_ES_COMPRESSION +# +# config LRNG_CPU_ENTROPY_RATE +# int "CPU Entropy Source Entropy Rate" +# depends on LRNG_CPU +# range 0 256 +# default 8 +# help +# The option defines the amount of entropy the LRNG applies to 256 +# bits of data obtained from the CPU entropy source. The LRNG +# enforces the limit that this value must be in the range between +# 0 and 256. +# +# When configuring this value to 0, the CPU entropy source will +# provide 256 bits of data without being credited to contain +# entropy. +# +# Note, this option is overwritten when the option +# CONFIG_RANDOM_TRUST_CPU is set. +# +# comment "Scheduler Entropy Source" +# +# config LRNG_SCHED +# bool "Enable Scheduer Entropy Source as LRNG Seed Source" +# select LRNG_TIMER_COMMON +# help +# The LRNG models an entropy source based on the timing of the +# occurrence of scheduler-triggered context switches. Enable +# this option to enable this scheduler entropy source. +# +# The scheduler entropy source is triggered every time a +# context switch is triggered thus causes the scheduler to +# execute slightly longer. Disabling the scheduler entropy +# source implies that the performance penalty on the scheduler +# added by the LRNG is eliminated. Yet, this entropy source is +# considered to be an internal entropy source of the LRNG. +# Thus, only disable it if you ensured that other entropy +# sources are available that supply the LRNG with entropy. +# +# If you disable the scheduler entropy source, you MUST +# ensure one or more entropy sources collectively have the +# capability to deliver sufficient entropy with one invocation +# at a rate compliant to the security strength of the DRNG +# (usually 256 bits of entropy). In addition, if those +# entropy sources do not deliver sufficient entropy during +# first request, the reseed must be triggered from user +# space or kernel space when sufficient entropy is considered +# to be present. +# +# If unsure, say Y. +# +# config LRNG_SCHED_ENTROPY_RATE +# int "Scheduler Entropy Source Entropy Rate" +# depends on LRNG_SCHED +# range 256 4294967295 if LRNG_SCHED_DFLT_TIMER_ES +# range 4294967295 4294967295 if !LRNG_SCHED_DFLT_TIMER_ES +# default 256 if LRNG_SCHED_DFLT_TIMER_ES +# default 4294967295 if !LRNG_SCHED_DFLT_TIMER_ES +# help +# The LRNG will collect the configured number of context switches +# triggered by the scheduler to obtain 256 bits of entropy. This +# value can be set to any between 256 and 4294967295. The LRNG +# guarantees that this value is not lower than 256. This lower +# limit implies that one interrupt event is credited with one bit +# of entropy. This value is subject to the increase by the +# oversampling factor, if no high-resolution timer is found. +# +# In order to effectively disable the scheduler entropy source, +# the option has to be set to 4294967295. In this case, the +# scheduler entropy source will still deliver data but without +# being credited with entropy. +# +# comment "Kernel RNG Entropy Source" +# +# config LRNG_KERNEL_RNG +# bool "Enable Kernel RNG as LRNG Seed Source" +# depends on RANDOM_DEFAULT_IMPL +# help +# The LRNG may use the kernel RNG (random.c) as entropy +# source. +# +# config LRNG_KERNEL_RNG_ENTROPY_RATE +# int "Kernel RNG Entropy Source Entropy Rate" +# depends on LRNG_KERNEL_RNG +# range 0 256 +# default 256 +# help +# The option defines the amount of entropy the LRNG applies to 256 +# bits of data obtained from the kernel RNG entropy source. The +# LRNG enforces the limit that this value must be in the range +# between 0 and 256. +# +# When configuring this value to 0, the kernel RNG entropy source +# will provide 256 bits of data without being credited to contain +# entropy. +# +# Note: This value is set to 0 automatically when booting the +# kernel in FIPS mode (with fips=1 kernel command line option). +# This is due to the fact that random.c is not SP800-90B +# compliant. +# +# endmenu # "Entropy Source Configuration" + +config LRNG_DRNG_CHACHA20 + tristate + +# config LRNG_DRBG +# tristate +# depends on CRYPTO +# select CRYPTO_DRBG_MENU +# +# config LRNG_DRNG_KCAPI +# tristate +# depends on CRYPTO +# select CRYPTO_RNG +# +# config LRNG_SWITCH +# bool +# +# menuconfig LRNG_SWITCH_HASH +# bool "Support conditioning hash runtime switching" +# select LRNG_SWITCH +# help +# The LRNG uses a default message digest. With this +# configuration option other message digests can be selected +# and loaded at runtime. +# +# if LRNG_SWITCH_HASH +# +# config LRNG_HASH_KCAPI +# tristate "Kernel crypto API hashing support for LRNG" +# select CRYPTO_HASH +# select CRYPTO_SHA512 +# help +# Enable the kernel crypto API support for entropy compression +# and conditioning functions. +# +# endif # LRNG_SWITCH_HASH +# +# menuconfig LRNG_SWITCH_DRNG +# bool "Support DRNG runtime switching" +# select LRNG_SWITCH +# help +# The LRNG uses a default DRNG With this configuration +# option other DRNGs or message digests can be selected and +# loaded at runtime. +# +# if LRNG_SWITCH_DRNG +# +# config LRNG_SWITCH_DRNG_CHACHA20 +# tristate "ChaCha20-based DRNG support for LRNG" +# depends on !LRNG_DFLT_DRNG_CHACHA20 +# select LRNG_DRNG_CHACHA20 +# help +# Enable the ChaCha20-based DRNG. This DRNG implementation +# does not depend on the kernel crypto API presence. +# +# config LRNG_SWITCH_DRBG +# tristate "SP800-90A support for the LRNG" +# depends on !LRNG_DFLT_DRNG_DRBG +# select LRNG_DRBG +# help +# Enable the SP800-90A DRBG support for the LRNG. Once the +# module is loaded, output from /dev/random, /dev/urandom, +# getrandom(2), or get_random_bytes_full is provided by a DRBG. +# +# config LRNG_SWITCH_DRNG_KCAPI +# tristate "Kernel Crypto API support for the LRNG" +# depends on !LRNG_DFLT_DRNG_KCAPI +# depends on !LRNG_SWITCH_DRBG +# select LRNG_DRNG_KCAPI +# help +# Enable the support for generic pseudo-random number +# generators offered by the kernel crypto API with the +# LRNG. Once the module is loaded, output from /dev/random, +# /dev/urandom, getrandom(2), or get_random_bytes is +# provided by the selected kernel crypto API RNG. +# +# endif # LRNG_SWITCH_DRNG + +choice + prompt "LRNG Default DRNG" + default LRNG_DFLT_DRNG_CHACHA20 + help + Select the default deterministic random number generator + that is used by the LRNG. When enabling the switchable + cryptographic mechanism support, this DRNG can be + replaced at runtime. + + config LRNG_DFLT_DRNG_CHACHA20 + bool "ChaCha20-based DRNG" + select LRNG_DRNG_CHACHA20 + +# config LRNG_DFLT_DRNG_DRBG +# depends on RANDOM_DEFAULT_IMPL +# bool "SP800-90A DRBG" +# select LRNG_DRBG +# +# config LRNG_DFLT_DRNG_KCAPI +# depends on RANDOM_DEFAULT_IMPL +# bool "Kernel Crypto API DRNG" +# select LRNG_DRNG_KCAPI +endchoice + +# menuconfig LRNG_TESTING_MENU +# bool "LRNG testing interfaces" +# depends on DEBUG_FS +# help +# Enable one or more of the following test interfaces. +# +# If unsure, say N. +# +# if LRNG_TESTING_MENU +# +# config LRNG_TESTING +# bool +# +# config LRNG_TESTING_RECORDING +# bool +# +# comment "Interrupt Entropy Source Test Interfaces" +# +# config LRNG_RAW_HIRES_ENTROPY +# bool "Interface to obtain raw unprocessed IRQ noise source data" +# default y +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned high resolution time stamp noise that +# is collected by the LRNG for statistical analysis. Extracted +# noise data is not used to seed the LRNG. +# +# The raw noise data can be obtained using the lrng_raw_hires +# debugfs file. Using the option lrng_testing.boot_raw_hires_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_RAW_JIFFIES_ENTROPY +# bool "Entropy test interface to Jiffies of IRQ noise source" +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned Jiffies that is collected by +# the LRNG for statistical analysis. This data is used for +# seeding the LRNG if a high-resolution time stamp is not +# available. If a high-resolution time stamp is detected, +# the Jiffies value is not collected by the LRNG and no +# data is provided via the test interface. Extracted noise +# data is not used to seed the random number generator. +# +# The raw noise data can be obtained using the lrng_raw_jiffies +# debugfs file. Using the option lrng_testing.boot_raw_jiffies_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_RAW_IRQ_ENTROPY +# bool "Entropy test interface to IRQ number noise source" +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned interrupt number that is collected by +# the LRNG for statistical analysis. Extracted noise data is +# not used to seed the random number generator. +# +# The raw noise data can be obtained using the lrng_raw_irq +# debugfs file. Using the option lrng_testing.boot_raw_irq_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_RAW_RETIP_ENTROPY +# bool "Entropy test interface to RETIP value of IRQ noise source" +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned return instruction pointer value +# that is collected by the LRNG for statistical analysis. +# Extracted noise data is not used to seed the random number +# generator. +# +# The raw noise data can be obtained using the lrng_raw_retip +# debugfs file. Using the option lrng_testing.boot_raw_retip_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_RAW_REGS_ENTROPY +# bool "Entropy test interface to IRQ register value noise source" +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned interrupt register value that is +# collected by the LRNG for statistical analysis. Extracted noise +# data is not used to seed the random number generator. +# +# The raw noise data can be obtained using the lrng_raw_regs +# debugfs file. Using the option lrng_testing.boot_raw_regs_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_RAW_ARRAY +# bool "Test interface to LRNG raw entropy IRQ storage array" +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw noise data that is collected by the LRNG +# in the per-CPU array for statistical analysis. The purpose +# of this interface is to verify that the array handling code +# truly only concatenates data and provides the same entropy +# rate as the raw unconditioned noise source when assessing +# the collected data byte-wise. +# +# The data can be obtained using the lrng_raw_array debugfs +# file. Using the option lrng_testing.boot_raw_array=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_IRQ_PERF +# bool "LRNG interrupt entropy source performance monitor" +# depends on LRNG_IRQ +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# With this option, the performance monitor of the LRNG +# interrupt handling code is enabled. The file provides +# the execution time of the interrupt handler in +# cycles. +# +# The interrupt performance data can be obtained using +# the lrng_irq_perf debugfs file. Using the option +# lrng_testing.boot_irq_perf=1 the performance data of +# the first 1000 entropy events since boot can be sampled. +# +# comment "Scheduler Entropy Source Test Interfaces" +# +# config LRNG_RAW_SCHED_HIRES_ENTROPY +# bool "Interface to obtain raw unprocessed scheduler noise source data" +# depends on LRNG_SCHED +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned high resolution time stamp noise that +# is collected by the LRNG for the Scheduler-based noise source +# for statistical analysis. Extracted noise data is not used to +# seed the LRNG. +# +# The raw noise data can be obtained using the lrng_raw_sched_hires +# debugfs file. Using the option +# lrng_testing.boot_raw_sched_hires_test=1 the raw noise of the +# first 1000 entropy events since boot can be sampled. +# +# config LRNG_RAW_SCHED_PID_ENTROPY +# bool "Entropy test interface to PID value" +# depends on LRNG_SCHED +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned PID value that is collected by the +# LRNG for statistical analysis. Extracted noise +# data is not used to seed the random number generator. +# +# The raw noise data can be obtained using the +# lrng_raw_sched_pid debugfs file. Using the option +# lrng_testing.boot_raw_sched_pid_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_RAW_SCHED_START_TIME_ENTROPY +# bool "Entropy test interface to task start time value" +# depends on LRNG_SCHED +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned task start time value that is collected +# by the LRNG for statistical analysis. Extracted noise +# data is not used to seed the random number generator. +# +# The raw noise data can be obtained using the +# lrng_raw_sched_starttime debugfs file. Using the option +# lrng_testing.boot_raw_sched_starttime_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# +# config LRNG_RAW_SCHED_NVCSW_ENTROPY +# bool "Entropy test interface to task context switch numbers" +# depends on LRNG_SCHED +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# The test interface allows a privileged process to capture +# the raw unconditioned task numbers of context switches that +# are collected by the LRNG for statistical analysis. Extracted +# noise data is not used to seed the random number generator. +# +# The raw noise data can be obtained using the +# lrng_raw_sched_nvcsw debugfs file. Using the option +# lrng_testing.boot_raw_sched_nvcsw_test=1 +# the raw noise of the first 1000 entropy events since boot +# can be sampled. +# +# config LRNG_SCHED_PERF +# bool "LRNG scheduler entropy source performance monitor" +# depends on LRNG_SCHED +# select LRNG_TESTING +# select LRNG_TESTING_RECORDING +# help +# With this option, the performance monitor of the LRNG +# scheduler event handling code is enabled. The file provides +# the execution time of the interrupt handler in cycles. +# +# The scheduler performance data can be obtained using +# the lrng_sched_perf debugfs file. Using the option +# lrng_testing.boot_sched_perf=1 the performance data of +# the first 1000 entropy events since boot can be sampled. +# +# comment "Auxiliary Test Interfaces" +# +# config LRNG_ACVT_HASH +# bool "Enable LRNG ACVT Hash interface" +# select LRNG_TESTING +# help +# With this option, the LRNG built-in hash function used for +# auxiliary pool management and prior to switching the +# cryptographic backends is made available for ACVT. The +# interface allows writing of the data to be hashed +# into the interface. The read operation triggers the hash +# operation to generate message digest. +# +# The ACVT interface is available with the lrng_acvt_hash +# debugfs file. +# +# config LRNG_RUNTIME_MAX_WO_RESEED_CONFIG +# bool "Enable runtime configuration of max reseed threshold" +# help +# When enabling this option, the LRNG provides an interface +# allowing the setting of the maximum number of DRNG generate +# operations without a reseed that has full entropy. The +# interface is lrng_drng.max_wo_reseed. +# +#config LRNG_RUNTIME_FORCE_SEEDING_DISABLE +# bool "Enable runtime configuration of force seeding" +# help +# When enabling this option, the LRNG provides an interface +# allowing the disabling of the force seeding when the DRNG +# is not fully seeded but entropy is available. +# +# config LRNG_TEST_CPU_ES_COMPRESSION +# bool "Force CPU ES compression operation" +# help +# When enabling this option, the CPU ES compression operation +# is forced by setting an arbitrary value > 1 for the data +# multiplier even when the CPU ES would deliver full entropy. +# This allows testing of the compression operation. It +# therefore forces to pull more data from the CPU ES +# than what may be required. +# +# endif #LRNG_TESTING_MENU +# +# config LRNG_SELFTEST +# bool "Enable power-on and on-demand self-tests" +# help +# The power-on self-tests are executed during boot time +# covering the ChaCha20 DRNG, the hash operation used for +# processing the entropy pools and the auxiliary pool, and +# the time stamp management of the LRNG. +# +# The on-demand self-tests are triggered by writing any +# value into the SysFS file selftest_status. At the same +# time, when reading this file, the test status is +# returned. A zero indicates that all tests were executed +# successfully. +# +# If unsure, say Y. +# +# if LRNG_SELFTEST +# +# config LRNG_SELFTEST_PANIC +# bool "Panic the kernel upon self-test failure" +# help +# If the option is enabled, the kernel is terminated if an +# LRNG power-on self-test failure is detected. +# +# endif # LRNG_SELFTEST + +endif # LRNG + +endmenu # LRNG --- /dev/null +++ b/drivers/char/lrng/Makefile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for the Entropy Source and DRNG Manager. +# + +obj-y += lrng_es_mgr.o lrng_drng_mgr.o \ + lrng_es_aux.o +obj-$(CONFIG_LRNG_SHA256) += lrng_sha256.o +obj-$(CONFIG_LRNG_SHA1) += lrng_sha1.o + +obj-$(CONFIG_LRNG_DRNG_CHACHA20) += lrng_drng_chacha20.o --- /dev/null +++ b/drivers/char/lrng/lrng_definitions.h @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022 - 2023, Stephan Mueller + */ + +#ifndef _LRNG_DEFINITIONS_H +#define _LRNG_DEFINITIONS_H + +#include +#include +#include + +/*************************** General LRNG parameter ***************************/ + +/* + * Specific settings for different use cases + */ +#ifdef CONFIG_CRYPTO_FIPS +# define LRNG_OVERSAMPLE_ES_BITS 64 +# define LRNG_SEED_BUFFER_INIT_ADD_BITS 128 +#else /* CONFIG_CRYPTO_FIPS */ +# define LRNG_OVERSAMPLE_ES_BITS 0 +# define LRNG_SEED_BUFFER_INIT_ADD_BITS 0 +#endif /* CONFIG_CRYPTO_FIPS */ + +/* Security strength of LRNG -- this must match DRNG security strength */ +#define LRNG_DRNG_SECURITY_STRENGTH_BYTES 32 +#define LRNG_DRNG_SECURITY_STRENGTH_BITS (LRNG_DRNG_SECURITY_STRENGTH_BYTES * 8) +#define LRNG_DRNG_INIT_SEED_SIZE_BITS \ + (LRNG_DRNG_SECURITY_STRENGTH_BITS + LRNG_SEED_BUFFER_INIT_ADD_BITS) +#define LRNG_DRNG_INIT_SEED_SIZE_BYTES (LRNG_DRNG_INIT_SEED_SIZE_BITS >> 3) + +/* + * SP800-90A defines a maximum request size of 1<<16 bytes. The given value is + * considered a safer margin. + * + * This value is allowed to be changed. + */ +#define LRNG_DRNG_MAX_REQSIZE (1<<12) + +/* + * SP800-90A defines a maximum number of requests between reseeds of 2^48. + * The given value is considered a much safer margin, balancing requests for + * frequent reseeds with the need to conserve entropy. This value MUST NOT be + * larger than INT_MAX because it is used in an atomic_t. + * + * This value is allowed to be changed. + */ +#define LRNG_DRNG_RESEED_THRESH (1<<20) + +/* + * Maximum DRNG generation operations without reseed having full entropy + * This value defines the absolute maximum value of DRNG generation operations + * without a reseed holding full entropy. LRNG_DRNG_RESEED_THRESH is the + * threshold when a new reseed is attempted. But it is possible that this fails + * to deliver full entropy. In this case the DRNG will continue to provide data + * even though it was not reseeded with full entropy. To avoid in the extreme + * case that no reseed is performed for too long, this threshold is enforced. + * If that absolute low value is reached, the LRNG is marked as not operational. + * + * This value is allowed to be changed. + */ +#define LRNG_DRNG_MAX_WITHOUT_RESEED (1<<30) + +/* + * Min required seed entropy is 128 bits covering the minimum entropy + * requirement of SP800-131A and the German BSI's TR02102. + * + * This value is allowed to be changed. + */ +#define LRNG_FULL_SEED_ENTROPY_BITS LRNG_DRNG_SECURITY_STRENGTH_BITS +#define LRNG_MIN_SEED_ENTROPY_BITS 128 +#define LRNG_INIT_ENTROPY_BITS 32 + +/* AIS20/31: NTG.1.4 minimum entropy rate for one entropy source*/ +#define LRNG_AIS2031_NPTRNG_MIN_ENTROPY 220 + +/* + * Wakeup value + * + * This value is allowed to be changed but must not be larger than the + * digest size of the hash operation used update the aux_pool. + */ +#ifdef CONFIG_LRNG_SHA256 +# define LRNG_ATOMIC_DIGEST_SIZE SHA256_DIGEST_SIZE +#else +# define LRNG_ATOMIC_DIGEST_SIZE SHA1_DIGEST_SIZE +#endif +#define LRNG_WRITE_WAKEUP_ENTROPY LRNG_ATOMIC_DIGEST_SIZE + +/* + * If the switching support is configured, we must provide support up to + * the largest digest size. Without switching support, we know it is only + * the built-in digest size. + */ +#ifdef CONFIG_LRNG_SWITCH +# define LRNG_MAX_DIGESTSIZE 64 +#else +# define LRNG_MAX_DIGESTSIZE LRNG_ATOMIC_DIGEST_SIZE +#endif + +/* + * Oversampling factor of timer-based events to obtain + * LRNG_DRNG_SECURITY_STRENGTH_BYTES. This factor is used when a + * high-resolution time stamp is not available. In this case, jiffies and + * register contents are used to fill the entropy pool. These noise sources + * are much less entropic than the high-resolution timer. The entropy content + * is the entropy content assumed with LRNG_[IRQ|SCHED]_ENTROPY_BITS divided by + * LRNG_ES_OVERSAMPLING_FACTOR. + * + * This value is allowed to be changed. + */ +#define LRNG_ES_OVERSAMPLING_FACTOR 10 + +/* Alignmask that is intended to be identical to CRYPTO_MINALIGN */ +#define LRNG_KCAPI_ALIGN ARCH_KMALLOC_MINALIGN + +/* + * This definition must provide a buffer that is equal to SHASH_DESC_ON_STACK + * as it will be casted into a struct shash_desc. + */ +#define LRNG_POOL_SIZE (sizeof(struct shash_desc) + HASH_MAX_DESCSIZE) + +/* + * Identification of a permanent health falure. + * + * Allow the given number of back-to-back health failures until incuring a + * permanent health failure. The chosen value implies an alpha of 2^-60 + * considering that the alpha of one health failure is 2^-30 + */ +#define LRNG_PERMANENT_HEALTH_FAILURES 2 + +/****************************** Helper code ***********************************/ + +static inline u32 lrng_fast_noise_entropylevel(u32 ent_bits, u32 requested_bits) +{ + /* Obtain entropy statement */ + ent_bits = ent_bits * requested_bits / LRNG_DRNG_SECURITY_STRENGTH_BITS; + /* Cap entropy to buffer size in bits */ + ent_bits = min_t(u32, ent_bits, requested_bits); + return ent_bits; +} + +/* Convert entropy in bits into nr. of events with the same entropy content. */ +static inline u32 lrng_entropy_to_data(u32 entropy_bits, u32 entropy_rate) +{ + return ((entropy_bits * entropy_rate) / + LRNG_DRNG_SECURITY_STRENGTH_BITS); +} + +/* Convert number of events into entropy value. */ +static inline u32 lrng_data_to_entropy(u32 num, u32 entropy_rate) +{ + return ((num * LRNG_DRNG_SECURITY_STRENGTH_BITS) / + entropy_rate); +} + +static inline u32 atomic_read_u32(atomic_t *v) +{ + return (u32)atomic_read(v); +} + +#endif /* _LRNG_DEFINITIONS_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_drng_atomic.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_DRNG_ATOMIC_H +#define _LRNG_DRNG_ATOMIC_H + +#include "lrng_drng_mgr.h" + +#ifdef CONFIG_LRNG_DRNG_ATOMIC +void lrng_drng_atomic_reset(void); +void lrng_drng_atomic_seed_drng(struct lrng_drng *drng); +void lrng_drng_atomic_force_reseed(void); +struct lrng_drng *lrng_get_atomic(void); +#else /* CONFIG_LRNG_DRNG_ATOMIC */ +static inline void lrng_drng_atomic_reset(void) { } +static inline void lrng_drng_atomic_seed_drng(struct lrng_drng *drng) { } +static inline void lrng_drng_atomic_force_reseed(void) { } +static inline struct lrng_drng *lrng_get_atomic(void) { return NULL; } +#endif /* CONFIG_LRNG_DRNG_ATOMIC */ + +#endif /* _LRNG_DRNG_ATOMIC_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_drng_chacha20.c @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* + * Backend for the LRNG providing the cryptographic primitives using + * ChaCha20 cipher implementations. + * + * Copyright (C) 2022, Stephan Mueller + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include + +#include "lrng_drng_chacha20.h" + +/******************************* ChaCha20 DRNG *******************************/ + +#define CHACHA_BLOCK_WORDS (CHACHA_BLOCK_SIZE / sizeof(u32)) + +/* + * Update of the ChaCha20 state by either using an unused buffer part or by + * generating one ChaCha20 block which is half of the state of the ChaCha20. + * The block is XORed into the key part of the state. This shall ensure + * backtracking resistance as well as a proper mix of the ChaCha20 state once + * the key is injected. + */ +static void lrng_chacha20_update(struct chacha20_state *chacha20_state, + __le32 *buf, u32 used_words) +{ + struct chacha20_block *chacha20 = &chacha20_state->block; + u32 i; + __le32 tmp[CHACHA_BLOCK_WORDS]; + + BUILD_BUG_ON(sizeof(struct chacha20_block) != CHACHA_BLOCK_SIZE); + BUILD_BUG_ON(CHACHA_BLOCK_SIZE != 2 * CHACHA_KEY_SIZE); + + if (used_words > CHACHA_KEY_SIZE_WORDS) { + chacha20_block(&chacha20->constants[0], (u8 *)tmp); + for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++) + chacha20->key.u[i] ^= le32_to_cpu(tmp[i]); + memzero_explicit(tmp, sizeof(tmp)); + } else { + for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++) + chacha20->key.u[i] ^= le32_to_cpu(buf[i + used_words]); + } + + /* Deterministic increment of nonce as required in RFC 7539 chapter 4 */ + chacha20->nonce[0]++; + if (chacha20->nonce[0] == 0) { + chacha20->nonce[1]++; + if (chacha20->nonce[1] == 0) + chacha20->nonce[2]++; + } + + /* Leave counter untouched as it is start value is undefined in RFC */ +} + +/* + * Seed the ChaCha20 DRNG by injecting the input data into the key part of + * the ChaCha20 state. If the input data is longer than the ChaCha20 key size, + * perform a ChaCha20 operation after processing of key size input data. + * This operation shall spread out the entropy into the ChaCha20 state before + * new entropy is injected into the key part. + */ +static int lrng_cc20_drng_seed_helper(void *drng, const u8 *inbuf, u32 inbuflen) +{ + struct chacha20_state *chacha20_state = (struct chacha20_state *)drng; + struct chacha20_block *chacha20 = &chacha20_state->block; + + while (inbuflen) { + u32 i, todo = min_t(u32, inbuflen, CHACHA_KEY_SIZE); + + for (i = 0; i < todo; i++) + chacha20->key.b[i] ^= inbuf[i]; + + /* Break potential dependencies between the inbuf key blocks */ + lrng_chacha20_update(chacha20_state, NULL, + CHACHA_BLOCK_WORDS); + inbuf += todo; + inbuflen -= todo; + } + + return 0; +} + +/* + * Chacha20 DRNG generation of random numbers: the stream output of ChaCha20 + * is the random number. After the completion of the generation of the + * stream, the entire ChaCha20 state is updated. + * + * Note, as the ChaCha20 implements a 32 bit counter, we must ensure + * that this function is only invoked for at most 2^32 - 1 ChaCha20 blocks + * before a reseed or an update happens. This is ensured by the variable + * outbuflen which is a 32 bit integer defining the number of bytes to be + * generated by the ChaCha20 DRNG. At the end of this function, an update + * operation is invoked which implies that the 32 bit counter will never be + * overflown in this implementation. + */ +static int lrng_cc20_drng_generate_helper(void *drng, u8 *outbuf, u32 outbuflen) +{ + struct chacha20_state *chacha20_state = (struct chacha20_state *)drng; + struct chacha20_block *chacha20 = &chacha20_state->block; + __le32 aligned_buf[CHACHA_BLOCK_WORDS]; + u32 ret = outbuflen, used = CHACHA_BLOCK_WORDS; + int zeroize_buf = 0; + + while (outbuflen >= CHACHA_BLOCK_SIZE) { + chacha20_block(&chacha20->constants[0], outbuf); + outbuf += CHACHA_BLOCK_SIZE; + outbuflen -= CHACHA_BLOCK_SIZE; + } + + if (outbuflen) { + chacha20_block(&chacha20->constants[0], (u8 *)aligned_buf); + memcpy(outbuf, aligned_buf, outbuflen); + used = ((outbuflen + sizeof(aligned_buf[0]) - 1) / + sizeof(aligned_buf[0])); + zeroize_buf = 1; + } + + lrng_chacha20_update(chacha20_state, aligned_buf, used); + + if (zeroize_buf) + memzero_explicit(aligned_buf, sizeof(aligned_buf)); + + return ret; +} + +/* + * Allocation of the DRNG state + */ +static void *lrng_cc20_drng_alloc(u32 sec_strength) +{ + struct chacha20_state *state = NULL; + + if (sec_strength > CHACHA_KEY_SIZE) { + pr_err("Security strength of ChaCha20 DRNG (%u bits) lower than requested by LRNG (%u bits)\n", + CHACHA_KEY_SIZE * 8, sec_strength * 8); + return ERR_PTR(-EINVAL); + } + if (sec_strength < CHACHA_KEY_SIZE) + pr_warn("Security strength of ChaCha20 DRNG (%u bits) higher than requested by LRNG (%u bits)\n", + CHACHA_KEY_SIZE * 8, sec_strength * 8); + + state = kmalloc(sizeof(struct chacha20_state), GFP_KERNEL); + if (!state) + return ERR_PTR(-ENOMEM); + pr_debug("memory for ChaCha20 core allocated\n"); + + lrng_cc20_init_rfc7539(&state->block); + + return state; +} + +static void lrng_cc20_drng_dealloc(void *drng) +{ + struct chacha20_state *chacha20_state = (struct chacha20_state *)drng; + + pr_debug("ChaCha20 core zeroized and freed\n"); + kfree_sensitive(chacha20_state); +} + +static const char *lrng_cc20_drng_name(void) +{ + return "ChaCha20 DRNG"; +} + +const struct lrng_drng_cb lrng_cc20_drng_cb = { + .drng_name = lrng_cc20_drng_name, + .drng_alloc = lrng_cc20_drng_alloc, + .drng_dealloc = lrng_cc20_drng_dealloc, + .drng_seed = lrng_cc20_drng_seed_helper, + .drng_generate = lrng_cc20_drng_generate_helper, +}; + +#if !defined(CONFIG_LRNG_DFLT_DRNG_CHACHA20) && \ + !defined(CONFIG_LRNG_DRNG_ATOMIC) +static int __init lrng_cc20_drng_init(void) +{ + return lrng_set_drng_cb(&lrng_cc20_drng_cb); +} + +static void __exit lrng_cc20_drng_exit(void) +{ + lrng_set_drng_cb(NULL); +} + +late_initcall(lrng_cc20_drng_init); +module_exit(lrng_cc20_drng_exit); +MODULE_LICENSE("Dual BSD/GPL"); +MODULE_AUTHOR("Stephan Mueller "); +MODULE_DESCRIPTION("Entropy Source and DRNG Manager - ChaCha20-based DRNG backend"); +#endif /* CONFIG_LRNG_DFLT_DRNG_CHACHA20 */ --- /dev/null +++ b/drivers/char/lrng/lrng_drng_chacha20.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * LRNG ChaCha20 definitions + * + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_CHACHA20_H +#define _LRNG_CHACHA20_H + +#include + +/* State according to RFC 7539 section 2.3 */ +struct chacha20_block { + u32 constants[4]; + union { +#define CHACHA_KEY_SIZE_WORDS (CHACHA_KEY_SIZE / sizeof(u32)) + u32 u[CHACHA_KEY_SIZE_WORDS]; + u8 b[CHACHA_KEY_SIZE]; + } key; + u32 counter; + u32 nonce[3]; +}; + +struct chacha20_state { + struct chacha20_block block; +}; + +static inline void lrng_cc20_init_rfc7539(struct chacha20_block *chacha20) +{ + chacha_init_consts(chacha20->constants); +} + +#define LRNG_CC20_INIT_RFC7539(x) \ + x.constants[0] = 0x61707865, \ + x.constants[1] = 0x3320646e, \ + x.constants[2] = 0x79622d32, \ + x.constants[3] = 0x6b206574, + +extern const struct lrng_drng_cb lrng_cc20_drng_cb; + +#endif /* _LRNG_CHACHA20_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_drng_drbg.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * LRNG SP800-90A definitions + * + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_DRBG_H +#define _LRNG_DRBG_H + +extern const struct lrng_drng_cb lrng_drbg_cb; + +#endif /* _LRNG_DRBG_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_drng_kcapi.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * LRNG kernel crypto API DRNG definition + * + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_KCAPI_DRNG_H +#define _LRNG_KCAPI_DRNG_H + +extern const struct lrng_drng_cb lrng_kcapi_drng_cb; + +#endif /* _LRNG_KCAPI_DRNG_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_drng_mgr.c @@ -0,0 +1,742 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* + * LRNG DRNG management + * + * Copyright (C) 2022, Stephan Mueller + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include + +#include "lrng_drng_atomic.h" +#include "lrng_drng_chacha20.h" +#include "lrng_drng_drbg.h" +#include "lrng_drng_kcapi.h" +#include "lrng_drng_mgr.h" +#include "lrng_es_aux.h" +#include "lrng_es_mgr.h" +#include "lrng_interface_random_kernel.h" +#include "lrng_numa.h" +#include "lrng_sha.h" + +/* + * Maximum number of seconds between DRNG reseed intervals of the DRNG. Note, + * this is enforced with the next request of random numbers from the + * DRNG. Setting this value to zero implies a reseeding attempt before every + * generated random number. + */ +int lrng_drng_reseed_max_time = 600; + +/* + * Is LRNG for general-purpose use (i.e. is at least the lrng_drng_init + * fully allocated)? + */ +static atomic_t lrng_avail = ATOMIC_INIT(0); + +/* Guard protecting all crypto callback update operation of all DRNGs. */ +DEFINE_MUTEX(lrng_crypto_cb_update); + +/* + * Default hash callback that provides the crypto primitive right from the + * kernel start. It must not perform any memory allocation operation, but + * simply perform the hash calculation. + */ +const struct lrng_hash_cb *lrng_default_hash_cb = &lrng_sha_hash_cb; + +/* + * Default DRNG callback that provides the crypto primitive which is + * allocated either during late kernel boot stage. So, it is permissible for + * the callback to perform memory allocation operations. + */ +const struct lrng_drng_cb *lrng_default_drng_cb = +#if defined(CONFIG_LRNG_DFLT_DRNG_CHACHA20) + &lrng_cc20_drng_cb; +#elif defined(CONFIG_LRNG_DFLT_DRNG_DRBG) + &lrng_drbg_cb; +#elif defined(CONFIG_LRNG_DFLT_DRNG_KCAPI) + &lrng_kcapi_drng_cb; +#else +#error "Unknown default DRNG selected" +#endif + +/* DRNG for non-atomic use cases */ +static struct lrng_drng lrng_drng_init = { + LRNG_DRNG_STATE_INIT(lrng_drng_init, NULL, NULL, NULL, + &lrng_sha_hash_cb), + .lock = __MUTEX_INITIALIZER(lrng_drng_init.lock), +}; + +/* Prediction-resistance DRNG: only deliver as much data as received entropy */ +static struct lrng_drng lrng_drng_pr = { + LRNG_DRNG_STATE_INIT(lrng_drng_pr, NULL, NULL, NULL, + &lrng_sha_hash_cb), + .lock = __MUTEX_INITIALIZER(lrng_drng_pr.lock), +}; + +static u32 max_wo_reseed = LRNG_DRNG_MAX_WITHOUT_RESEED; +#ifdef CONFIG_LRNG_RUNTIME_MAX_WO_RESEED_CONFIG +module_param(max_wo_reseed, uint, 0444); +MODULE_PARM_DESC(max_wo_reseed, + "Maximum number of DRNG generate operation without full reseed\n"); +#endif + +static bool force_seeding = true; +#ifdef CONFIG_LRNG_RUNTIME_FORCE_SEEDING_DISABLE +module_param(force_seeding, bool, 0444); +MODULE_PARM_DESC(force_seeding, + "Allow disabling of the forced seeding when insufficient entropy is available\n"); +#endif + +/* Wait queue to wait until the LRNG is initialized - can freely be used */ +DECLARE_WAIT_QUEUE_HEAD(lrng_init_wait); + +/********************************** Helper ************************************/ + +bool lrng_get_available(void) +{ + return likely(atomic_read(&lrng_avail)); +} + +struct lrng_drng *lrng_drng_init_instance(void) +{ + return &lrng_drng_init; +} + +struct lrng_drng *lrng_drng_pr_instance(void) +{ + return &lrng_drng_pr; +} + +struct lrng_drng *lrng_drng_node_instance(void) +{ + struct lrng_drng **lrng_drng = lrng_drng_instances(); + int node = numa_node_id(); + + if (lrng_drng && lrng_drng[node]) + return lrng_drng[node]; + + return lrng_drng_init_instance(); +} + +void lrng_drng_reset(struct lrng_drng *drng) +{ + /* Ensure reseed during next call */ + atomic_set(&drng->requests, 1); + atomic_set(&drng->requests_since_fully_seeded, 0); + drng->last_seeded = jiffies; + drng->fully_seeded = false; + /* Do not set force, as this flag is used for the emergency reseeding */ + drng->force_reseed = false; + pr_debug("reset DRNG\n"); +} + +/* Initialize the DRNG, except the mutex lock */ +int lrng_drng_alloc_common(struct lrng_drng *drng, + const struct lrng_drng_cb *drng_cb) +{ + if (!drng || !drng_cb) + return -EINVAL; + if (!IS_ERR_OR_NULL(drng->drng)) + return 0; + + drng->drng_cb = drng_cb; + drng->drng = drng_cb->drng_alloc(LRNG_DRNG_SECURITY_STRENGTH_BYTES); + if (IS_ERR(drng->drng)) + return -PTR_ERR(drng->drng); + + lrng_drng_reset(drng); + return 0; +} + +/* Initialize the default DRNG during boot and perform its seeding */ +int lrng_drng_initalize(void) +{ + int ret; + + if (lrng_get_available()) + return 0; + + /* Catch programming error */ + WARN_ON(lrng_drng_init.hash_cb != lrng_default_hash_cb); + + mutex_lock(&lrng_drng_init.lock); + if (lrng_get_available()) { + mutex_unlock(&lrng_drng_init.lock); + return 0; + } + + /* Initialize the PR DRNG inside init lock as it guards lrng_avail. */ + mutex_lock(&lrng_drng_pr.lock); + ret = lrng_drng_alloc_common(&lrng_drng_pr, lrng_default_drng_cb); + mutex_unlock(&lrng_drng_pr.lock); + + if (!ret) { + ret = lrng_drng_alloc_common(&lrng_drng_init, + lrng_default_drng_cb); + if (!ret) + atomic_set(&lrng_avail, 1); + } + mutex_unlock(&lrng_drng_init.lock); + if (ret) + return ret; + + pr_debug("LRNG for general use is available\n"); + + /* Seed the DRNG with any entropy available */ + if (lrng_pool_trylock()) { + pr_info("Initial DRNG initialized triggering first seeding\n"); + lrng_drng_seed_work(NULL); + } else { + pr_info("Initial DRNG initialized without seeding\n"); + } + + return 0; +} + +static int __init lrng_drng_make_available(void) +{ + return lrng_drng_initalize(); +} +late_initcall(lrng_drng_make_available); + +bool lrng_sp80090c_compliant(void) +{ + /* SP800-90C compliant oversampling is only requested in FIPS mode */ + return fips_enabled; +} + +/************************* Random Number Generation ***************************/ + +/* Inject a data buffer into the DRNG - caller must hold its lock */ +void lrng_drng_inject(struct lrng_drng *drng, const u8 *inbuf, u32 inbuflen, + bool fully_seeded, const char *drng_type) +{ + BUILD_BUG_ON(LRNG_DRNG_RESEED_THRESH > INT_MAX); + pr_debug("seeding %s DRNG with %u bytes\n", drng_type, inbuflen); + if (drng->drng_cb->drng_seed(drng->drng, inbuf, inbuflen) < 0) { + pr_warn("seeding of %s DRNG failed\n", drng_type); + drng->force_reseed = true; + } else { + int gc = LRNG_DRNG_RESEED_THRESH - atomic_read(&drng->requests); + + pr_debug("%s DRNG stats since last seeding: %lu secs; generate calls: %d\n", + drng_type, + (time_after(jiffies, drng->last_seeded) ? + (jiffies - drng->last_seeded) : 0) / HZ, gc); + + /* Count the numbers of generate ops since last fully seeded */ + if (fully_seeded) + atomic_set(&drng->requests_since_fully_seeded, 0); + else + atomic_add(gc, &drng->requests_since_fully_seeded); + + drng->last_seeded = jiffies; + atomic_set(&drng->requests, LRNG_DRNG_RESEED_THRESH); + drng->force_reseed = false; + + if (!drng->fully_seeded) { + drng->fully_seeded = fully_seeded; + if (drng->fully_seeded) + pr_debug("%s DRNG fully seeded\n", drng_type); + } + } +} + +/* + * Perform the seeding of the DRNG with data from entropy source. + * The function returns the entropy injected into the DRNG in bits. + */ +static u32 lrng_drng_seed_es_nolock(struct lrng_drng *drng, bool init_ops, + const char *drng_type) +{ + struct entropy_buf seedbuf __aligned(LRNG_KCAPI_ALIGN), + collected_seedbuf; + u32 collected_entropy = 0; + unsigned int i, num_es_delivered = 0; + bool forced = drng->force_reseed; + + for_each_lrng_es(i) + collected_seedbuf.e_bits[i] = 0; + + do { + /* Count the number of ES which delivered entropy */ + num_es_delivered = 0; + + if (collected_entropy) + pr_debug("Force fully seeding level for %s DRNG by repeatedly pull entropy from available entropy sources\n", + drng_type); + + lrng_fill_seed_buffer(&seedbuf, + lrng_get_seed_entropy_osr(drng->fully_seeded), + forced && !drng->fully_seeded); + + collected_entropy += lrng_entropy_rate_eb(&seedbuf); + + /* Sum iterations up. */ + for_each_lrng_es(i) { + collected_seedbuf.e_bits[i] += seedbuf.e_bits[i]; + num_es_delivered += !!seedbuf.e_bits[i]; + } + + lrng_drng_inject(drng, (u8 *)&seedbuf, sizeof(seedbuf), + lrng_fully_seeded(drng->fully_seeded, + collected_entropy, + &collected_seedbuf), + drng_type); + + /* + * Set the seeding state of the LRNG + * + * Do not call lrng_init_ops(seedbuf) here as the atomic DRNG + * does not serve common users. + */ + if (init_ops) + lrng_init_ops(&collected_seedbuf); + + /* + * Emergency reseeding: If we reached the min seed threshold now + * multiple times but never reached fully seeded level and we collect + * entropy, keep doing it until we reached fully seeded level for + * at least one DRNG. This operation is not continued if the + * ES do not deliver entropy such that we cannot reach the fully seeded + * level. + * + * The emergency reseeding implies that the consecutively injected + * entropy can be added up. This is applicable due to the fact that + * the entire operation is atomic which means that the DRNG is not + * producing data while this is ongoing. + */ + } while (force_seeding && forced && !drng->fully_seeded && + num_es_delivered >= (lrng_ntg1_2022_compliant() ? 2 : 1)); + + memzero_explicit(&seedbuf, sizeof(seedbuf)); + + return collected_entropy; +} + +static void lrng_drng_seed_es(struct lrng_drng *drng) +{ + mutex_lock(&drng->lock); + lrng_drng_seed_es_nolock(drng, true, "regular"); + mutex_unlock(&drng->lock); +} + +static void lrng_drng_seed(struct lrng_drng *drng) +{ + BUILD_BUG_ON(LRNG_MIN_SEED_ENTROPY_BITS > + LRNG_DRNG_SECURITY_STRENGTH_BITS); + + /* (Re-)Seed DRNG */ + lrng_drng_seed_es(drng); + /* (Re-)Seed atomic DRNG from regular DRNG */ + lrng_drng_atomic_seed_drng(drng); +} + +static void lrng_drng_seed_work_one(struct lrng_drng *drng, u32 node) +{ + pr_debug("reseed triggered by system events for DRNG on NUMA node %d\n", + node); + lrng_drng_seed(drng); + if (drng->fully_seeded) { + /* Prevent reseed storm */ + drng->last_seeded += node * 100 * HZ; + } +} + +/* + * DRNG reseed trigger: Kernel thread handler triggered by the schedule_work() + */ +static void __lrng_drng_seed_work(bool force) +{ + struct lrng_drng **lrng_drng; + u32 node; + + /* + * If the DRNG is not yet initialized, let us try to seed the atomic + * DRNG. + */ + if (!lrng_get_available()) { + struct lrng_drng *atomic; + unsigned long flags; + + if (wq_has_sleeper(&lrng_init_wait)) { + lrng_init_ops(NULL); + return; + } + atomic = lrng_get_atomic(); + if (!atomic || atomic->fully_seeded) + return; + + atomic->force_reseed |= force; + spin_lock_irqsave(&atomic->spin_lock, flags); + lrng_drng_seed_es_nolock(atomic, false, "atomic"); + spin_unlock_irqrestore(&atomic->spin_lock, flags); + + return; + } + + lrng_drng = lrng_drng_instances(); + if (lrng_drng) { + for_each_online_node(node) { + struct lrng_drng *drng = lrng_drng[node]; + + if (drng && !drng->fully_seeded) { + drng->force_reseed |= force; + lrng_drng_seed_work_one(drng, node); + return; + } + } + } else { + if (!lrng_drng_init.fully_seeded) { + lrng_drng_init.force_reseed |= force; + lrng_drng_seed_work_one(&lrng_drng_init, 0); + return; + } + } + + if (!lrng_drng_pr.fully_seeded) { + lrng_drng_pr.force_reseed |= force; + lrng_drng_seed_work_one(&lrng_drng_pr, 0); + return; + } + + lrng_pool_all_numa_nodes_seeded(true); +} + +void lrng_drng_seed_work(struct work_struct *dummy) +{ + __lrng_drng_seed_work(false); + + /* Allow the seeding operation to be called again */ + lrng_pool_unlock(); +} + +/* Force all DRNGs to reseed before next generation */ +void lrng_drng_force_reseed(void) +{ + struct lrng_drng **lrng_drng = lrng_drng_instances(); + u32 node; + + /* + * If the initial DRNG is over the reseed threshold, allow a forced + * reseed only for the initial DRNG as this is the fallback for all. It + * must be kept seeded before all others to keep the LRNG operational. + */ + if (!lrng_drng || + (atomic_read_u32(&lrng_drng_init.requests_since_fully_seeded) > + LRNG_DRNG_RESEED_THRESH)) { + lrng_drng_init.force_reseed = lrng_drng_init.fully_seeded; + pr_debug("force reseed of initial DRNG\n"); + return; + } + for_each_online_node(node) { + struct lrng_drng *drng = lrng_drng[node]; + + if (!drng) + continue; + + drng->force_reseed = drng->fully_seeded; + pr_debug("force reseed of DRNG on node %u\n", node); + } + lrng_drng_atomic_force_reseed(); +} +EXPORT_SYMBOL(lrng_drng_force_reseed); + +static bool lrng_drng_must_reseed(struct lrng_drng *drng) +{ + return (atomic_dec_and_test(&drng->requests) || + drng->force_reseed || + time_after(jiffies, + drng->last_seeded + lrng_drng_reseed_max_time * HZ)); +} + +/* + * lrng_drng_get() - Get random data out of the DRNG which is reseeded + * frequently. + * + * @drng: DRNG instance + * @outbuf: buffer for storing random data + * @outbuflen: length of outbuf + * + * Return: + * * < 0 in error case (DRNG generation or update failed) + * * >=0 returning the returned number of bytes + */ +int lrng_drng_get(struct lrng_drng *drng, u8 *outbuf, u32 outbuflen) +{ + u32 processed = 0; + bool pr = (drng == &lrng_drng_pr) ? true : false; + + if (!outbuf || !outbuflen) + return 0; + + if (!lrng_get_available()) + return -EOPNOTSUPP; + + outbuflen = min_t(size_t, outbuflen, INT_MAX); + + /* If DRNG operated without proper reseed for too long, block LRNG */ + BUILD_BUG_ON(LRNG_DRNG_MAX_WITHOUT_RESEED < LRNG_DRNG_RESEED_THRESH); + if (atomic_read_u32(&drng->requests_since_fully_seeded) > max_wo_reseed) + lrng_unset_fully_seeded(drng); + + while (outbuflen) { + u32 todo = min_t(u32, outbuflen, LRNG_DRNG_MAX_REQSIZE); + int ret; + + /* In normal operation, check whether to reseed */ + if (!pr && lrng_drng_must_reseed(drng)) { + if (!lrng_pool_trylock()) { + drng->force_reseed = true; + } else { + lrng_drng_seed(drng); + lrng_pool_unlock(); + } + } + + mutex_lock(&drng->lock); + + if (pr) { + /* If async reseed did not deliver entropy, try now */ + if (!drng->fully_seeded) { + u32 coll_ent_bits; + + /* If we cannot get the pool lock, try again. */ + if (!lrng_pool_trylock()) { + mutex_unlock(&drng->lock); + continue; + } + + coll_ent_bits = lrng_drng_seed_es_nolock( + drng, true, "regular"); + + lrng_pool_unlock(); + + /* If no new entropy was received, stop now. */ + if (!coll_ent_bits) { + mutex_unlock(&drng->lock); + goto out; + } + + /* Produce no more data than received entropy */ + todo = min_t(u32, todo, coll_ent_bits >> 3); + } + + /* Do not produce more than DRNG security strength */ + todo = min_t(u32, todo, lrng_security_strength() >> 3); + } + ret = drng->drng_cb->drng_generate(drng->drng, + outbuf + processed, todo); + + mutex_unlock(&drng->lock); + if (ret <= 0) { + pr_warn("getting random data from DRNG failed (%d)\n", + ret); + return -EFAULT; + } + processed += ret; + outbuflen -= ret; + + if (pr) { + /* Force the async reseed for PR DRNG */ + lrng_unset_fully_seeded(drng); + if (outbuflen) + cond_resched(); + } + } + +out: + return processed; +} + +int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen, bool pr) +{ + struct lrng_drng **lrng_drng = lrng_drng_instances(); + struct lrng_drng *drng = &lrng_drng_init; + int ret, node = numa_node_id(); + + might_sleep(); + + if (pr) + drng = &lrng_drng_pr; + else if (lrng_drng && lrng_drng[node] && lrng_drng[node]->fully_seeded) + drng = lrng_drng[node]; + + ret = lrng_drng_initalize(); + if (ret) + return ret; + + return lrng_drng_get(drng, outbuf, outbuflen); +} + +/* Reset LRNG such that all existing entropy is gone */ +static void _lrng_reset(struct work_struct *work) +{ + struct lrng_drng **lrng_drng = lrng_drng_instances(); + + if (!lrng_drng) { + mutex_lock(&lrng_drng_init.lock); + lrng_drng_reset(&lrng_drng_init); + mutex_unlock(&lrng_drng_init.lock); + } else { + u32 node; + + for_each_online_node(node) { + struct lrng_drng *drng = lrng_drng[node]; + + if (!drng) + continue; + mutex_lock(&drng->lock); + lrng_drng_reset(drng); + mutex_unlock(&drng->lock); + } + } + + mutex_lock(&lrng_drng_pr.lock); + lrng_drng_reset(&lrng_drng_pr); + mutex_unlock(&lrng_drng_pr.lock); + + lrng_drng_atomic_reset(); + lrng_set_entropy_thresh(LRNG_INIT_ENTROPY_BITS); + + lrng_reset_state(); +} + +static DECLARE_WORK(lrng_reset_work, _lrng_reset); + +void lrng_reset(void) +{ + schedule_work(&lrng_reset_work); +} + +/******************* Generic LRNG kernel output interfaces ********************/ + +void lrng_force_fully_seeded(void) +{ + if (lrng_pool_all_numa_nodes_seeded_get()) + return; + + lrng_pool_lock(); + __lrng_drng_seed_work(true); + lrng_pool_unlock(); +} + +static int lrng_drng_sleep_while_not_all_nodes_seeded(unsigned int nonblock) +{ + lrng_force_fully_seeded(); + if (lrng_pool_all_numa_nodes_seeded_get()) + return 0; + if (nonblock) + return -EAGAIN; + wait_event_interruptible(lrng_init_wait, + lrng_pool_all_numa_nodes_seeded_get()); + return 0; +} + +int lrng_drng_sleep_while_nonoperational(int nonblock) +{ + lrng_force_fully_seeded(); + if (likely(lrng_state_operational())) + return 0; + if (nonblock) + return -EAGAIN; + return wait_event_interruptible(lrng_init_wait, + lrng_state_operational()); +} + +int lrng_drng_sleep_while_non_min_seeded(void) +{ + lrng_force_fully_seeded(); + if (likely(lrng_state_min_seeded())) + return 0; + return wait_event_interruptible(lrng_init_wait, + lrng_state_min_seeded()); +} + +ssize_t lrng_get_seed(u64 *buf, size_t nbytes, unsigned int flags) +{ + struct entropy_buf *eb = (struct entropy_buf *)(buf + 2); + u64 buflen = sizeof(struct entropy_buf) + 2 * sizeof(u64); + u64 collected_bits = 0; + int ret; + + /* Ensure buffer is aligned as required */ + BUILD_BUG_ON(sizeof(buflen) > LRNG_KCAPI_ALIGN); + if (nbytes < sizeof(buflen)) + return -EINVAL; + + /* Write buffer size into first word */ + buf[0] = buflen; + if (nbytes < buflen) + return -EMSGSIZE; + + ret = lrng_drng_sleep_while_not_all_nodes_seeded( + flags & LRNG_GET_SEED_NONBLOCK); + if (ret) + return ret; + + /* Try to get the pool lock and sleep on it to get it. */ + lrng_pool_lock(); + + /* If an LRNG DRNG becomes unseeded, give this DRNG precedence. */ + if (!lrng_pool_all_numa_nodes_seeded_get()) { + lrng_pool_unlock(); + return 0; + } + + /* + * Try to get seed data - a rarely used busyloop is cheaper than a wait + * queue that is constantly woken up by the hot code path of + * lrng_init_ops. + */ + for (;;) { + lrng_fill_seed_buffer(eb, + lrng_get_seed_entropy_osr(flags & + LRNG_GET_SEED_FULLY_SEEDED), + false); + collected_bits = lrng_entropy_rate_eb(eb); + + /* Break the collection loop if we got entropy, ... */ + if (collected_bits || + /* ... a DRNG becomes unseeded, give DRNG precedence, ... */ + !lrng_pool_all_numa_nodes_seeded_get() || + /* ... if the caller does not want a blocking behavior. */ + (flags & LRNG_GET_SEED_NONBLOCK)) + break; + + schedule(); + } + + lrng_pool_unlock(); + + /* Write collected entropy size into second word */ + buf[1] = collected_bits; + + return (ssize_t)buflen; +} + +void lrng_get_random_bytes_full(void *buf, int nbytes) +{ + lrng_drng_sleep_while_nonoperational(0); + lrng_drng_get_sleep((u8 *)buf, (u32)nbytes, false); +} +EXPORT_SYMBOL(lrng_get_random_bytes_full); + +void lrng_get_random_bytes_min(void *buf, int nbytes) +{ + lrng_drng_sleep_while_non_min_seeded(); + lrng_drng_get_sleep((u8 *)buf, (u32)nbytes, false); +} +EXPORT_SYMBOL(lrng_get_random_bytes_min); + +int lrng_get_random_bytes_pr(void *buf, int nbytes) +{ + lrng_drng_sleep_while_nonoperational(0); + return lrng_drng_get_sleep((u8 *)buf, (u32)nbytes, true); +} +EXPORT_SYMBOL(lrng_get_random_bytes_pr); --- /dev/null +++ b/drivers/char/lrng/lrng_drng_mgr.h @@ -0,0 +1,86 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_DRNG_H +#define _LRNG_DRNG_H + +#include +#include +#include + +#include "lrng_definitions.h" + +extern struct wait_queue_head lrng_init_wait; +extern int lrng_drng_reseed_max_time; +extern struct mutex lrng_crypto_cb_update; +extern const struct lrng_drng_cb *lrng_default_drng_cb; +extern const struct lrng_hash_cb *lrng_default_hash_cb; + +/* DRNG state handle */ +struct lrng_drng { + void *drng; /* DRNG handle */ + void *hash; /* Hash handle */ + const struct lrng_drng_cb *drng_cb; /* DRNG callbacks */ + const struct lrng_hash_cb *hash_cb; /* Hash callbacks */ + atomic_t requests; /* Number of DRNG requests */ + atomic_t requests_since_fully_seeded; /* Number DRNG requests since + * last fully seeded + */ + unsigned long last_seeded; /* Last time it was seeded */ + bool fully_seeded; /* Is DRNG fully seeded? */ + bool force_reseed; /* Force a reseed */ + + rwlock_t hash_lock; /* Lock hash_cb replacement */ + /* Lock write operations on DRNG state, DRNG replacement of drng_cb */ + struct mutex lock; /* Non-atomic DRNG operation */ + spinlock_t spin_lock; /* Atomic DRNG operation */ +}; + +#define LRNG_DRNG_STATE_INIT(x, d, h, d_cb, h_cb) \ + .drng = d, \ + .hash = h, \ + .drng_cb = d_cb, \ + .hash_cb = h_cb, \ + .requests = ATOMIC_INIT(LRNG_DRNG_RESEED_THRESH),\ + .requests_since_fully_seeded = ATOMIC_INIT(0), \ + .last_seeded = 0, \ + .fully_seeded = false, \ + .force_reseed = true, \ + .hash_lock = __RW_LOCK_UNLOCKED(x.hash_lock) + +struct lrng_drng *lrng_drng_init_instance(void); +struct lrng_drng *lrng_drng_pr_instance(void); +struct lrng_drng *lrng_drng_node_instance(void); + +void lrng_reset(void); +int lrng_drng_alloc_common(struct lrng_drng *drng, + const struct lrng_drng_cb *crypto_cb); +int lrng_drng_initalize(void); +bool lrng_sp80090c_compliant(void); +bool lrng_get_available(void); +void lrng_drng_reset(struct lrng_drng *drng); +void lrng_drng_inject(struct lrng_drng *drng, const u8 *inbuf, u32 inbuflen, + bool fully_seeded, const char *drng_type); +int lrng_drng_get(struct lrng_drng *drng, u8 *outbuf, u32 outbuflen); +int lrng_drng_sleep_while_nonoperational(int nonblock); +int lrng_drng_sleep_while_non_min_seeded(void); +int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen, bool pr); +void lrng_drng_seed_work(struct work_struct *dummy); +void lrng_drng_force_reseed(void); +void lrng_force_fully_seeded(void); + +static inline u32 lrng_compress_osr(void) +{ + return lrng_sp80090c_compliant() ? LRNG_OVERSAMPLE_ES_BITS : 0; +} + +static inline u32 lrng_reduce_by_osr(u32 entropy_bits) +{ + u32 osr_bits = lrng_compress_osr(); + + return (entropy_bits >= osr_bits) ? (entropy_bits - osr_bits) : 0; +} + +#endif /* _LRNG_DRNG_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_aux.c @@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* + * LRNG Slow Entropy Source: Auxiliary entropy pool + * + * Copyright (C) 2022, Stephan Mueller + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include + +#include "lrng_es_aux.h" +#include "lrng_es_mgr.h" +#include "lrng_sysctl.h" + +/* + * This is the auxiliary pool + * + * The aux pool array is aligned to 8 bytes to comfort the kernel crypto API + * cipher implementations of the hash functions used to read the pool: for some + * accelerated implementations, we need an alignment to avoid a realignment + * which involves memcpy(). The alignment to 8 bytes should satisfy all crypto + * implementations. + */ +struct lrng_pool { + u8 aux_pool[LRNG_POOL_SIZE]; /* Aux pool: digest state */ + atomic_t aux_entropy_bits; + atomic_t digestsize; /* Digest size of used hash */ + bool initialized; /* Aux pool initialized? */ + + /* Serialize read of entropy pool and update of aux pool */ + spinlock_t lock; +}; + +static struct lrng_pool lrng_pool __aligned(LRNG_KCAPI_ALIGN) = { + .aux_entropy_bits = ATOMIC_INIT(0), + .digestsize = ATOMIC_INIT(LRNG_ATOMIC_DIGEST_SIZE), + .initialized = false, + .lock = __SPIN_LOCK_UNLOCKED(lrng_pool.lock) +}; + +/********************************** Helper ***********************************/ + +/* Entropy in bits present in aux pool */ +static u32 lrng_aux_avail_entropy(u32 __unused) +{ + /* Cap available entropy with max entropy */ + u32 avail_bits = min_t(u32, lrng_get_digestsize(), + atomic_read_u32(&lrng_pool.aux_entropy_bits)); + + /* Consider oversampling rate due to aux pool conditioning */ + return lrng_reduce_by_osr(avail_bits); +} + +/* Set the digest size of the used hash in bytes */ +static void lrng_set_digestsize(u32 digestsize) +{ + struct lrng_pool *pool = &lrng_pool; + u32 ent_bits = atomic_xchg_relaxed(&pool->aux_entropy_bits, 0), + old_digestsize = lrng_get_digestsize(); + + atomic_set(&lrng_pool.digestsize, digestsize); + + /* + * Update the write wakeup threshold which must not be larger + * than the digest size of the current conditioning hash. + */ + digestsize = lrng_reduce_by_osr(digestsize << 3); + lrng_sysctl_update_max_write_thresh(digestsize); + lrng_write_wakeup_bits = digestsize; + + /* + * In case the new digest is larger than the old one, cap the available + * entropy to the old message digest used to process the existing data. + */ + ent_bits = min_t(u32, ent_bits, old_digestsize); + atomic_add(ent_bits, &pool->aux_entropy_bits); +} + +static int __init lrng_init_wakeup_bits(void) +{ + u32 digestsize = lrng_reduce_by_osr(lrng_get_digestsize()); + + lrng_sysctl_update_max_write_thresh(digestsize); + lrng_write_wakeup_bits = digestsize; + return 0; +} +core_initcall(lrng_init_wakeup_bits); + +/* Obtain the digest size provided by the used hash in bits */ +u32 lrng_get_digestsize(void) +{ + return atomic_read_u32(&lrng_pool.digestsize) << 3; +} + +/* Set entropy content in user-space controllable aux pool */ +void lrng_pool_set_entropy(u32 entropy_bits) +{ + atomic_set(&lrng_pool.aux_entropy_bits, entropy_bits); +} + +static void lrng_aux_reset(void) +{ + lrng_pool_set_entropy(0); +} + +/* + * Replace old with new hash for auxiliary pool handling + * + * Assumption: the caller must guarantee that the new_cb is available during the + * entire operation (e.g. it must hold the write lock against pointer updating). + */ +static int +lrng_aux_switch_hash(struct lrng_drng *drng, int __unused, + const struct lrng_hash_cb *new_cb, void *new_hash, + const struct lrng_hash_cb *old_cb) +{ + struct lrng_drng *init_drng = lrng_drng_init_instance(); + struct lrng_pool *pool = &lrng_pool; + struct shash_desc *shash = (struct shash_desc *)pool->aux_pool; + u8 digest[LRNG_MAX_DIGESTSIZE]; + int ret; + + if (!IS_ENABLED(CONFIG_LRNG_SWITCH)) + return -EOPNOTSUPP; + + if (unlikely(!pool->initialized)) + return 0; + + /* We only switch if the processed DRNG is the initial DRNG. */ + if (init_drng != drng) + return 0; + + /* Get the aux pool hash with old digest ... */ + ret = old_cb->hash_final(shash, digest) ?: + /* ... re-initialize the hash with the new digest ... */ + new_cb->hash_init(shash, new_hash) ?: + /* + * ... feed the old hash into the new state. We may feed + * uninitialized memory into the new state, but this is + * considered no issue and even good as we have some more + * uncertainty here. + */ + new_cb->hash_update(shash, digest, sizeof(digest)); + if (!ret) { + lrng_set_digestsize(new_cb->hash_digestsize(new_hash)); + pr_debug("Re-initialize aux entropy pool with hash %s\n", + new_cb->hash_name()); + } + + memzero_explicit(digest, sizeof(digest)); + return ret; +} + +/* Insert data into auxiliary pool by using the hash update function. */ +static int +lrng_aux_pool_insert_locked(const u8 *inbuf, u32 inbuflen, u32 entropy_bits) +{ + struct lrng_pool *pool = &lrng_pool; + struct shash_desc *shash = (struct shash_desc *)pool->aux_pool; + struct lrng_drng *drng = lrng_drng_init_instance(); + const struct lrng_hash_cb *hash_cb; + unsigned long flags; + void *hash; + int ret; + + entropy_bits = min_t(u32, entropy_bits, inbuflen << 3); + + read_lock_irqsave(&drng->hash_lock, flags); + hash_cb = drng->hash_cb; + hash = drng->hash; + + if (unlikely(!pool->initialized)) { + ret = hash_cb->hash_init(shash, hash); + if (ret) + goto out; + pool->initialized = true; + } + + ret = hash_cb->hash_update(shash, inbuf, inbuflen); + if (ret) + goto out; + + /* + * Cap the available entropy to the hash output size compliant to + * SP800-90B section 3.1.5.1 table 1. + */ + entropy_bits += atomic_read_u32(&pool->aux_entropy_bits); + atomic_set(&pool->aux_entropy_bits, + min_t(u32, entropy_bits, + hash_cb->hash_digestsize(hash) << 3)); + +out: + read_unlock_irqrestore(&drng->hash_lock, flags); + return ret; +} + +int lrng_pool_insert_aux(const u8 *inbuf, u32 inbuflen, u32 entropy_bits) +{ + struct lrng_pool *pool = &lrng_pool; + unsigned long flags; + int ret; + + spin_lock_irqsave(&pool->lock, flags); + ret = lrng_aux_pool_insert_locked(inbuf, inbuflen, entropy_bits); + spin_unlock_irqrestore(&pool->lock, flags); + + lrng_es_add_entropy(); + + return ret; +} +EXPORT_SYMBOL(lrng_pool_insert_aux); + +/************************* Get data from entropy pool *************************/ + +/* + * Get auxiliary entropy pool and its entropy content for seed buffer. + * Caller must hold lrng_pool.pool->lock. + * @outbuf: buffer to store data in with size requested_bits + * @requested_bits: Requested amount of entropy + * @return: amount of entropy in outbuf in bits. + */ +static u32 lrng_aux_get_pool(u8 *outbuf, u32 requested_bits) +{ + struct lrng_pool *pool = &lrng_pool; + struct shash_desc *shash = (struct shash_desc *)pool->aux_pool; + struct lrng_drng *drng = lrng_drng_init_instance(); + const struct lrng_hash_cb *hash_cb; + unsigned long flags; + void *hash; + u32 collected_ent_bits, returned_ent_bits, unused_bits = 0, + digestsize, digestsize_bits, requested_bits_osr; + u8 aux_output[LRNG_MAX_DIGESTSIZE]; + + if (unlikely(!pool->initialized)) + return 0; + + read_lock_irqsave(&drng->hash_lock, flags); + + hash_cb = drng->hash_cb; + hash = drng->hash; + digestsize = hash_cb->hash_digestsize(hash); + digestsize_bits = digestsize << 3; + + /* Cap to maximum entropy that can ever be generated with given hash */ + lrng_cap_requested(digestsize_bits, requested_bits); + + /* Ensure that no more than the size of aux_pool can be requested */ + requested_bits = min_t(u32, requested_bits, (LRNG_MAX_DIGESTSIZE << 3)); + requested_bits_osr = requested_bits + lrng_compress_osr(); + + /* Cap entropy with entropy counter from aux pool and the used digest */ + collected_ent_bits = min_t(u32, digestsize_bits, + atomic_xchg_relaxed(&pool->aux_entropy_bits, 0)); + + /* We collected too much entropy and put the overflow back */ + if (collected_ent_bits > requested_bits_osr) { + /* Amount of bits we collected too much */ + unused_bits = collected_ent_bits - requested_bits_osr; + /* Put entropy back */ + atomic_add(unused_bits, &pool->aux_entropy_bits); + /* Fix collected entropy */ + collected_ent_bits = requested_bits_osr; + } + + /* Apply oversampling: discount requested oversampling rate */ + returned_ent_bits = lrng_reduce_by_osr(collected_ent_bits); + + pr_debug("obtained %u bits by collecting %u bits of entropy from aux pool, %u bits of entropy remaining\n", + returned_ent_bits, collected_ent_bits, unused_bits); + + /* Get the digest for the aux pool to be returned to the caller ... */ + if (hash_cb->hash_final(shash, aux_output) || + /* + * ... and re-initialize the aux state. Do not add the aux pool + * digest for backward secrecy as it will be added with the + * insertion of the complete seed buffer after it has been filled. + */ + hash_cb->hash_init(shash, hash)) { + returned_ent_bits = 0; + } else { + /* + * Do not truncate the output size exactly to collected_ent_bits + * as the aux pool may contain data that is not credited with + * entropy, but we want to use them to stir the DRNG state. + */ + memcpy(outbuf, aux_output, requested_bits >> 3); + } + + read_unlock_irqrestore(&drng->hash_lock, flags); + memzero_explicit(aux_output, digestsize); + return returned_ent_bits; +} + +static void lrng_aux_get_backtrack(struct entropy_buf *eb, u32 requested_bits, + bool __unused) +{ + struct lrng_pool *pool = &lrng_pool; + unsigned long flags; + + /* Ensure aux pool extraction and backtracking op are atomic */ + spin_lock_irqsave(&pool->lock, flags); + + eb->e_bits[lrng_ext_es_aux] = lrng_aux_get_pool(eb->e[lrng_ext_es_aux], + requested_bits); + + /* Mix the extracted data back into pool for backtracking resistance */ + if (lrng_aux_pool_insert_locked((u8 *)eb, + sizeof(struct entropy_buf), 0)) + pr_warn("Backtracking resistance operation failed\n"); + + spin_unlock_irqrestore(&pool->lock, flags); +} + +static void lrng_aux_es_state(unsigned char *buf, size_t buflen) +{ + const struct lrng_drng *lrng_drng_init = lrng_drng_init_instance(); + + /* Assume the lrng_drng_init lock is taken by caller */ + snprintf(buf, buflen, + " Hash for operating entropy pool: %s\n" + " Available entropy: %u\n", + lrng_drng_init->hash_cb->hash_name(), + lrng_aux_avail_entropy(0)); +} + +struct lrng_es_cb lrng_es_aux = { + .name = "Auxiliary", + .get_ent = lrng_aux_get_backtrack, + .curr_entropy = lrng_aux_avail_entropy, + .max_entropy = lrng_get_digestsize, + .state = lrng_aux_es_state, + .reset = lrng_aux_reset, + .switch_hash = lrng_aux_switch_hash, +}; --- /dev/null +++ b/drivers/char/lrng/lrng_es_aux.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_AUX_H +#define _LRNG_ES_AUX_H + +#include "lrng_drng_mgr.h" +#include "lrng_es_mgr_cb.h" + +u32 lrng_get_digestsize(void); +void lrng_pool_set_entropy(u32 entropy_bits); +int lrng_pool_insert_aux(const u8 *inbuf, u32 inbuflen, u32 entropy_bits); + +extern struct lrng_es_cb lrng_es_aux; + +/****************************** Helper code ***********************************/ + +/* Obtain the security strength of the LRNG in bits */ +static inline u32 lrng_security_strength(void) +{ + /* + * We use a hash to read the entropy in the entropy pool. According to + * SP800-90B table 1, the entropy can be at most the digest size. + * Considering this together with the last sentence in section 3.1.5.1.2 + * the security strength of a (approved) hash is equal to its output + * size. On the other hand the entropy cannot be larger than the + * security strength of the used DRBG. + */ + return min_t(u32, LRNG_FULL_SEED_ENTROPY_BITS, lrng_get_digestsize()); +} + +static inline u32 lrng_get_seed_entropy_osr(bool fully_seeded) +{ + u32 requested_bits = lrng_security_strength(); + + /* Apply oversampling during initialization according to SP800-90C */ + if (lrng_sp80090c_compliant() && !fully_seeded) + requested_bits += LRNG_SEED_BUFFER_INIT_ADD_BITS; + return requested_bits; +} + +#endif /* _LRNG_ES_AUX_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_cpu.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_CPU_H +#define _LRNG_ES_CPU_H + +#include "lrng_es_mgr_cb.h" + +#ifdef CONFIG_LRNG_CPU + +extern struct lrng_es_cb lrng_es_cpu; + +#endif /* CONFIG_LRNG_CPU */ + +#endif /* _LRNG_ES_CPU_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_irq.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_IRQ_H +#define _LRNG_ES_IRQ_H + +#include + +#include "lrng_es_mgr_cb.h" + +#ifdef CONFIG_LRNG_IRQ +void lrng_irq_es_init(bool highres_timer); +void lrng_irq_array_add_u32(u32 data); + +extern struct lrng_es_cb lrng_es_irq; + +#else /* CONFIG_LRNG_IRQ */ +static inline void lrng_irq_es_init(bool highres_timer) { } +static inline void lrng_irq_array_add_u32(u32 data) { } +#endif /* CONFIG_LRNG_IRQ */ + +#endif /* _LRNG_ES_IRQ_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_jent.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_JENT_H +#define _LRNG_ES_JENT_H + +#include "lrng_es_mgr_cb.h" + +#ifdef CONFIG_LRNG_JENT + +extern struct lrng_es_cb lrng_es_jent; + +#endif /* CONFIG_LRNG_JENT */ + +#endif /* _LRNG_ES_JENT_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_krng.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_RANDOM_H +#define _LRNG_ES_RANDOM_H + +#include "lrng_es_mgr_cb.h" + +#ifdef CONFIG_LRNG_KERNEL_RNG + +extern struct lrng_es_cb lrng_es_krng; + +#endif /* CONFIG_LRNG_KERNEL_RNG */ + +#endif /* _LRNG_ES_RANDOM_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_mgr.c @@ -0,0 +1,506 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* + * LRNG Entropy sources management + * + * Copyright (C) 2022 - 2023, Stephan Mueller + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include + +#include "lrng_drng_atomic.h" +#include "lrng_drng_mgr.h" +#include "lrng_es_aux.h" +#include "lrng_es_cpu.h" +#include "lrng_es_irq.h" +#include "lrng_es_jent.h" +#include "lrng_es_krng.h" +#include "lrng_es_mgr.h" +#include "lrng_es_sched.h" +#include "lrng_interface_dev_common.h" +#include "lrng_interface_random_kernel.h" + +struct lrng_state { + bool can_invalidate; /* Can invalidate batched entropy? */ + bool perform_seedwork; /* Can seed work be performed? */ + bool lrng_operational; /* Is DRNG operational? */ + bool lrng_fully_seeded; /* Is DRNG fully seeded? */ + bool lrng_min_seeded; /* Is DRNG minimally seeded? */ + bool all_online_numa_node_seeded;/* All NUMA DRNGs seeded? */ + + /* + * To ensure that external entropy providers cannot dominate the + * internal noise sources but yet cannot be dominated by internal + * noise sources, the following booleans are intended to allow + * external to provide seed once when a DRNG reseed occurs. This + * triggering of external noise source is performed even when the + * entropy pool has sufficient entropy. + */ + + atomic_t boot_entropy_thresh; /* Reseed threshold */ + struct mutex reseed_in_progress; /* Flag for on executing reseed */ + struct work_struct lrng_seed_work; /* (re)seed work queue */ +}; + +static struct lrng_state lrng_state = { + false, false, false, false, false, false, + .boot_entropy_thresh = ATOMIC_INIT(LRNG_INIT_ENTROPY_BITS), + .reseed_in_progress = + __MUTEX_INITIALIZER(lrng_state.reseed_in_progress), +}; + +/* + * If the entropy count falls under this number of bits, then we + * should wake up processes which are selecting or polling on write + * access to /dev/random. + */ +u32 lrng_write_wakeup_bits = (LRNG_WRITE_WAKEUP_ENTROPY << 3); + +/* + * The entries must be in the same order as defined by enum lrng_internal_es and + * enum lrng_external_es + */ +struct lrng_es_cb *lrng_es[] = { +#ifdef CONFIG_LRNG_IRQ + &lrng_es_irq, +#endif +#ifdef CONFIG_LRNG_SCHED + &lrng_es_sched, +#endif +#ifdef CONFIG_LRNG_JENT + &lrng_es_jent, +#endif +#ifdef CONFIG_LRNG_CPU + &lrng_es_cpu, +#endif +#ifdef CONFIG_LRNG_KERNEL_RNG + &lrng_es_krng, +#endif + &lrng_es_aux +}; + +static bool ntg1 = false; +#ifdef CONFIG_LRNG_AIS2031_NTG1_SEEDING_STRATEGY +module_param(ntg1, bool, 0444); +MODULE_PARM_DESC(ntg1, "Enable AIS20/31 NTG.1 compliant seeding strategy\n"); +#endif + +/* Only panic the kernel on permanent health failure if this variable is true */ +static bool lrng_panic_on_permanent_health_failure = false; +module_param(lrng_panic_on_permanent_health_failure, bool, 0444); +MODULE_PARM_DESC(lrng_panic_on_permanent_health_failure, "Panic on reaching permanent health failure - only required if LRNG is part of a FIPS 140-3 module\n"); + +/********************************** Helper ***********************************/ + +bool lrng_enforce_panic_on_permanent_health_failure(void) +{ + return lrng_panic_on_permanent_health_failure; +} + +bool lrng_ntg1_2022_compliant(void) +{ + /* Implies use of /dev/random w/ O_SYNC / getrandom w/ GRND_RANDOM */ + return ntg1; +} + +void lrng_debug_report_seedlevel(const char *name) +{ +#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM + static void *previous = NULL; + void *caller = (void *) _RET_IP_; + struct lrng_drng *atomic = lrng_get_atomic(); + + if (READ_ONCE(previous) == caller) + return; + + if (atomic && !atomic->fully_seeded) + pr_notice("%pS %s called without reaching minimally seeded level (available entropy %u)\n", + caller, name, lrng_avail_entropy()); + + WRITE_ONCE(previous, caller); +#endif +} + +/* + * Reading of the LRNG pool is only allowed by one caller. The reading is + * only performed to (re)seed DRNGs. Thus, if this "lock" is already taken, + * the reseeding operation is in progress. The caller is not intended to wait + * but continue with its other operation. + */ +int lrng_pool_trylock(void) +{ + return mutex_trylock(&lrng_state.reseed_in_progress); +} + +void lrng_pool_lock(void) +{ + mutex_lock(&lrng_state.reseed_in_progress); +} + +void lrng_pool_unlock(void) +{ + mutex_unlock(&lrng_state.reseed_in_progress); +} + +/* Set new entropy threshold for reseeding during boot */ +void lrng_set_entropy_thresh(u32 new_entropy_bits) +{ + atomic_set(&lrng_state.boot_entropy_thresh, new_entropy_bits); +} + +/* + * Reset LRNG state - the entropy counters are reset, but the data that may + * or may not have entropy remains in the pools as this data will not hurt. + */ +void lrng_reset_state(void) +{ + u32 i; + + for_each_lrng_es(i) { + if (lrng_es[i]->reset) + lrng_es[i]->reset(); + } + lrng_state.lrng_operational = false; + lrng_state.lrng_fully_seeded = false; + lrng_state.lrng_min_seeded = false; + lrng_state.all_online_numa_node_seeded = false; + pr_debug("reset LRNG\n"); +} + +/* Set flag that all DRNGs are fully seeded */ +void lrng_pool_all_numa_nodes_seeded(bool set) +{ + lrng_state.all_online_numa_node_seeded = set; + if (set) + wake_up_all(&lrng_init_wait); +} + +bool lrng_pool_all_numa_nodes_seeded_get(void) +{ + return lrng_state.all_online_numa_node_seeded; +} + +/* Return boolean whether LRNG reached minimally seed level */ +bool lrng_state_min_seeded(void) +{ + return lrng_state.lrng_min_seeded; +} + +/* Return boolean whether LRNG reached fully seed level */ +bool lrng_state_fully_seeded(void) +{ + return lrng_state.lrng_fully_seeded; +} + +/* Return boolean whether LRNG is considered fully operational */ +bool lrng_state_operational(void) +{ + return lrng_state.lrng_operational; +} + +static void lrng_init_wakeup(void) +{ + wake_up_all(&lrng_init_wait); + lrng_init_wakeup_dev(); + lrng_kick_random_ready(); +} + +static u32 lrng_avail_entropy_thresh(void) +{ + u32 ent_thresh = lrng_security_strength(); + + /* + * Apply oversampling during initialization according to SP800-90C as + * we request a larger buffer from the ES. + */ + if (lrng_sp80090c_compliant() && + !lrng_state.all_online_numa_node_seeded) + ent_thresh += LRNG_SEED_BUFFER_INIT_ADD_BITS; + + return ent_thresh; +} + +bool lrng_fully_seeded(bool fully_seeded, u32 collected_entropy, + struct entropy_buf *eb) +{ + /* AIS20/31 NTG.1: two entropy sources with each delivering 220 bits */ + if (ntg1) { + u32 i, result = 0, ent_thresh = lrng_avail_entropy_thresh(); + + for_each_lrng_es(i) { + result += (eb ? eb->e_bits[i] : + lrng_es[i]->curr_entropy(ent_thresh)) >= + LRNG_AIS2031_NPTRNG_MIN_ENTROPY; + } + + return (result >= 2); + } + + return (collected_entropy >= lrng_get_seed_entropy_osr(fully_seeded)); +} + +u32 lrng_entropy_rate_eb(struct entropy_buf *eb) +{ + u32 i, collected_entropy = 0; + + for_each_lrng_es(i) + collected_entropy += eb->e_bits[i]; + + return collected_entropy; +} + +/* Mark one DRNG as not fully seeded */ +void lrng_unset_fully_seeded(struct lrng_drng *drng) +{ + drng->fully_seeded = false; + lrng_pool_all_numa_nodes_seeded(false); + + /* + * The init DRNG instance must always be fully seeded as this instance + * is the fall-back if any of the per-NUMA node DRNG instances is + * insufficiently seeded. Thus, we mark the entire LRNG as + * non-operational if the initial DRNG becomes not fully seeded. + */ + if (drng == lrng_drng_init_instance() && lrng_state_operational()) { + pr_debug("LRNG set to non-operational\n"); + lrng_state.lrng_operational = false; + lrng_state.lrng_fully_seeded = false; + + /* If sufficient entropy is available, reseed now. */ + lrng_es_add_entropy(); + } +} + +/* Policy to enable LRNG operational mode */ +static void lrng_set_operational(void) +{ + /* + * LRNG is operational if the initial DRNG is fully seeded. This state + * can only occur if either the external entropy sources provided + * sufficient entropy, or the SP800-90B startup test completed for + * the internal ES to supply also entropy data. + */ + if (lrng_state.lrng_fully_seeded) { + lrng_state.lrng_operational = true; + lrng_init_wakeup(); + pr_info("LRNG fully operational\n"); + } +} + +/* Available entropy in the entire LRNG considering all entropy sources */ +u32 lrng_avail_entropy(void) +{ + u32 i, ent = 0, ent_thresh = lrng_avail_entropy_thresh(); + + BUILD_BUG_ON(ARRAY_SIZE(lrng_es) != lrng_ext_es_last); + for_each_lrng_es(i) + ent += lrng_es[i]->curr_entropy(ent_thresh); + return ent; +} + +u32 lrng_avail_entropy_aux(void) +{ + u32 ent_thresh = lrng_avail_entropy_thresh(); + + return lrng_es[lrng_ext_es_aux]->curr_entropy(ent_thresh); +} + +/* + * lrng_init_ops() - Set seed stages of LRNG + * + * Set the slow noise source reseed trigger threshold. The initial threshold + * is set to the minimum data size that can be read from the pool: a word. Upon + * reaching this value, the next seed threshold of 128 bits is set followed + * by 256 bits. + * + * @eb: buffer containing the size of entropy currently injected into DRNG - if + * NULL, the function obtains the available entropy from the ES. + */ +void lrng_init_ops(struct entropy_buf *eb) +{ + struct lrng_state *state = &lrng_state; + u32 i, requested_bits, seed_bits = 0; + + if (state->lrng_operational) + return; + + requested_bits = ntg1 ? + /* Approximation so that two ES should deliver 220 bits each */ + (lrng_avail_entropy() + LRNG_AIS2031_NPTRNG_MIN_ENTROPY) : + /* Apply SP800-90C oversampling if applicable */ + lrng_get_seed_entropy_osr(state->all_online_numa_node_seeded); + + if (eb) { + seed_bits = lrng_entropy_rate_eb(eb); + } else { + u32 ent_thresh = lrng_avail_entropy_thresh(); + + for_each_lrng_es(i) + seed_bits += lrng_es[i]->curr_entropy(ent_thresh); + } + + /* DRNG is seeded with full security strength */ + if (state->lrng_fully_seeded) { + lrng_set_operational(); + lrng_set_entropy_thresh(requested_bits); + } else if (lrng_fully_seeded(state->all_online_numa_node_seeded, + seed_bits, eb)) { + if (state->can_invalidate) + invalidate_batched_entropy(); + + state->lrng_fully_seeded = true; + lrng_set_operational(); + state->lrng_min_seeded = true; + pr_info("LRNG fully seeded with %u bits of entropy\n", + seed_bits); + lrng_set_entropy_thresh(requested_bits); + } else if (!state->lrng_min_seeded) { + + /* DRNG is seeded with at least 128 bits of entropy */ + if (seed_bits >= LRNG_MIN_SEED_ENTROPY_BITS) { + if (state->can_invalidate) + invalidate_batched_entropy(); + + state->lrng_min_seeded = true; + pr_info("LRNG minimally seeded with %u bits of entropy\n", + seed_bits); + lrng_set_entropy_thresh(requested_bits); + lrng_init_wakeup(); + + /* DRNG is seeded with at least LRNG_INIT_ENTROPY_BITS bits */ + } else if (seed_bits >= LRNG_INIT_ENTROPY_BITS) { + pr_info("LRNG initial entropy level %u bits of entropy\n", + seed_bits); + lrng_set_entropy_thresh(LRNG_MIN_SEED_ENTROPY_BITS); + } + } +} + +void __init lrng_rand_initialize_early(void) +{ + struct seed { + unsigned long data[((LRNG_MAX_DIGESTSIZE + + sizeof(unsigned long) - 1) / + sizeof(unsigned long))]; + struct new_utsname utsname; + } seed __aligned(LRNG_KCAPI_ALIGN); + size_t longs = 0; + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(seed.data); i += longs) { + longs = arch_get_random_seed_longs(seed.data + i, + ARRAY_SIZE(seed.data) - i); + if (longs) + continue; + longs = arch_get_random_longs(seed.data + i, + ARRAY_SIZE(seed.data) - i); + if (longs) + continue; + longs = 1; + } + memcpy(&seed.utsname, init_utsname(), sizeof(*(init_utsname()))); + + lrng_pool_insert_aux((u8 *)&seed, sizeof(seed), 0); + memzero_explicit(&seed, sizeof(seed)); + + lrng_force_fully_seeded(); +} + +void __init lrng_rand_initialize(void) +{ + unsigned long entropy = random_get_entropy(); + ktime_t time = ktime_get_real(); + + lrng_pool_insert_aux((u8 *)&entropy, sizeof(entropy), 0); + lrng_pool_insert_aux((u8 *)&time, sizeof(time), 0); + + /* Initialize the seed work queue */ + INIT_WORK(&lrng_state.lrng_seed_work, lrng_drng_seed_work); + lrng_state.perform_seedwork = true; + + invalidate_batched_entropy(); + + lrng_state.can_invalidate = true; +} + +#ifndef CONFIG_LRNG_RANDOM_IF +static int __init lrng_rand_initialize_call(void) +{ + lrng_rand_initialize_early(); + lrng_rand_initialize(); + return 0; +} + +early_initcall(lrng_rand_initialize_call); +#endif + +/* Interface requesting a reseed of the DRNG */ +void lrng_es_add_entropy(void) +{ + /* + * Once all DRNGs are fully seeded, the system-triggered arrival of + * entropy will not cause any reseeding any more. + */ + if (likely(lrng_state.all_online_numa_node_seeded)) + return; + + /* Only trigger the DRNG reseed if we have collected entropy. */ + if (lrng_avail_entropy() < + atomic_read_u32(&lrng_state.boot_entropy_thresh)) + return; + + /* Ensure that the seeding only occurs once at any given time. */ + if (!lrng_pool_trylock()) + return; + + /* Seed the DRNG with any available noise. */ + if (lrng_state.perform_seedwork) + schedule_work(&lrng_state.lrng_seed_work); + else + lrng_drng_seed_work(NULL); +} + +/* Fill the seed buffer with data from the noise sources */ +void lrng_fill_seed_buffer(struct entropy_buf *eb, u32 requested_bits, + bool force) +{ + struct lrng_state *state = &lrng_state; + u32 i, req_ent = lrng_sp80090c_compliant() ? + lrng_security_strength() : LRNG_MIN_SEED_ENTROPY_BITS; + + /* Guarantee that requested bits is a multiple of bytes */ + BUILD_BUG_ON(LRNG_DRNG_SECURITY_STRENGTH_BITS % 8); + + /* always reseed the DRNG with the current time stamp */ + eb->now = random_get_entropy(); + + /* + * Require at least 128 bits of entropy for any reseed. If the LRNG is + * operated SP800-90C compliant we want to comply with SP800-90A section + * 9.2 mandating that DRNG is reseeded with the security strength. + */ + if (!force && + state->lrng_fully_seeded && (lrng_avail_entropy() < req_ent)) { + for_each_lrng_es(i) + eb->e_bits[i] = 0; + + goto wakeup; + } + + /* Concatenate the output of the entropy sources. */ + for_each_lrng_es(i) { + lrng_es[i]->get_ent(eb, requested_bits, + state->lrng_fully_seeded); + } + + /* allow external entropy provider to provide seed */ + lrng_state_exseed_allow_all(); + +wakeup: + lrng_writer_wakeup(); +} --- /dev/null +++ b/drivers/char/lrng/lrng_es_mgr.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_MGR_H +#define _LRNG_ES_MGR_H + +#include "lrng_es_mgr_cb.h" + +/*************************** General LRNG parameter ***************************/ + +#define LRNG_DRNG_BLOCKSIZE 64 /* Maximum of DRNG block sizes */ + +/* Helper to concatenate a macro with an integer type */ +#define LRNG_PASTER(x, y) x ## y +#define LRNG_UINT32_C(x) LRNG_PASTER(x, U) + +/************************* Entropy sources management *************************/ + +extern struct lrng_es_cb *lrng_es[]; + +#define for_each_lrng_es(ctr) \ + for ((ctr) = 0; (ctr) < lrng_ext_es_last; (ctr)++) + +bool lrng_enforce_panic_on_permanent_health_failure(void); +bool lrng_ntg1_2022_compliant(void); +bool lrng_pool_all_numa_nodes_seeded_get(void); +bool lrng_state_min_seeded(void); +void lrng_debug_report_seedlevel(const char *name); +void lrng_rand_initialize_early(void); +void lrng_rand_initialize(void); +bool lrng_state_operational(void); + +extern u32 lrng_write_wakeup_bits; +void lrng_set_entropy_thresh(u32 new); +u32 lrng_avail_entropy(void); +u32 lrng_avail_entropy_aux(void); +void lrng_reset_state(void); + +bool lrng_state_fully_seeded(void); + +int lrng_pool_trylock(void); +void lrng_pool_lock(void); +void lrng_pool_unlock(void); +void lrng_pool_all_numa_nodes_seeded(bool set); + +bool lrng_fully_seeded(bool fully_seeded, u32 collected_entropy, + struct entropy_buf *eb); +u32 lrng_entropy_rate_eb(struct entropy_buf *eb); +void lrng_unset_fully_seeded(struct lrng_drng *drng); +void lrng_fill_seed_buffer(struct entropy_buf *eb, u32 requested_bits, + bool force); +void lrng_init_ops(struct entropy_buf *eb); + +#endif /* _LRNG_ES_MGR_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_mgr_cb.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + * + * Definition of an entropy source. + */ + +#ifndef _LRNG_ES_MGR_CB_H +#define _LRNG_ES_MGR_CB_H + +#include + +#include "lrng_definitions.h" +#include "lrng_drng_mgr.h" + +enum lrng_internal_es { +#ifdef CONFIG_LRNG_IRQ + lrng_int_es_irq, /* IRQ-based entropy source */ +#endif +#ifdef CONFIG_LRNG_SCHED + lrng_int_es_sched, /* Scheduler entropy source */ +#endif + lrng_int_es_last, /* MUST be the last entry */ +}; + +enum lrng_external_es { + lrng_ext_link = lrng_int_es_last - 1, /* Link entry */ +#ifdef CONFIG_LRNG_JENT + lrng_ext_es_jitter, /* Jitter RNG */ +#endif +#ifdef CONFIG_LRNG_CPU + lrng_ext_es_cpu, /* CPU-based, e.g. RDSEED */ +#endif +#ifdef CONFIG_LRNG_KERNEL_RNG + lrng_ext_es_krng, /* random.c */ +#endif + lrng_ext_es_aux, /* MUST BE LAST ES! */ + lrng_ext_es_last /* MUST be the last entry */ +}; + +struct entropy_buf { + u8 e[lrng_ext_es_last][LRNG_DRNG_INIT_SEED_SIZE_BYTES]; + u32 now, e_bits[lrng_ext_es_last]; +}; + +/* + * struct lrng_es_cb - callback defining an entropy source + * @name: Name of the entropy source. + * @get_ent: Fetch entropy into the entropy_buf. The ES shall only deliver + * data if its internal initialization is complete, including any + * SP800-90B startup testing or similar. + * @curr_entropy: Return amount of currently available entropy. + * @max_entropy: Maximum amount of entropy the entropy source is able to + * maintain. + * @state: Buffer with human-readable ES state. + * @reset: Reset entropy source (drop all entropy and reinitialize). + * This callback may be NULL. + * @switch_hash: callback to switch from an old hash callback definition to + * a new one. This callback may be NULL. + */ +struct lrng_es_cb { + const char *name; + void (*get_ent)(struct entropy_buf *eb, u32 requested_bits, + bool fully_seeded); + u32 (*curr_entropy)(u32 requested_bits); + u32 (*max_entropy)(void); + void (*state)(unsigned char *buf, size_t buflen); + void (*reset)(void); + int (*switch_hash)(struct lrng_drng *drng, int node, + const struct lrng_hash_cb *new_cb, void *new_hash, + const struct lrng_hash_cb *old_cb); +}; + +/* Allow entropy sources to tell the ES manager that new entropy is there */ +void lrng_es_add_entropy(void); + +/* Cap to maximum entropy that can ever be generated with given hash */ +#define lrng_cap_requested(__digestsize_bits, __requested_bits) \ + do { \ + if (__digestsize_bits < __requested_bits) { \ + pr_debug("Cannot satisfy requested entropy %u due to insufficient hash size %u\n",\ + __requested_bits, __digestsize_bits); \ + __requested_bits = __digestsize_bits; \ + } \ + } while (0) + +#endif /* _LRNG_ES_MGR_CB_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_sched.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_SCHED_H +#define _LRNG_ES_SCHED_H + +#include "lrng_es_mgr_cb.h" + +#ifdef CONFIG_LRNG_SCHED +void lrng_sched_es_init(bool highres_timer); + +extern struct lrng_es_cb lrng_es_sched; + +#else /* CONFIG_LRNG_SCHED */ +static inline void lrng_sched_es_init(bool highres_timer) { } +#endif /* CONFIG_LRNG_SCHED */ + +#endif /* _LRNG_ES_SCHED_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_es_timer_common.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * LRNG Slow Noise Source: Time stamp array handling + * + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_ES_TIMER_COMMON_H +#define _LRNG_ES_TIMER_COMMON_H + +bool lrng_gcd_tested(void); +void lrng_gcd_set(u32 running_gcd); +u32 lrng_gcd_get(void); +u32 lrng_gcd_analyze(u32 *history, size_t nelem); +void lrng_gcd_add_value(u32 time); +bool lrng_highres_timer(void); + +/* + * To limit the impact on the interrupt handling, the LRNG concatenates + * entropic LSB parts of the time stamps in a per-CPU array and only + * injects them into the entropy pool when the array is full. + */ + +/* Store multiple integers in one u32 */ +#define LRNG_DATA_SLOTSIZE_BITS (8) +#define LRNG_DATA_SLOTSIZE_MASK ((1 << LRNG_DATA_SLOTSIZE_BITS) - 1) +#define LRNG_DATA_ARRAY_MEMBER_BITS (4 << 3) /* ((sizeof(u32)) << 3) */ +#define LRNG_DATA_SLOTS_PER_UINT (LRNG_DATA_ARRAY_MEMBER_BITS / \ + LRNG_DATA_SLOTSIZE_BITS) + +/* + * Number of time values to store in the array - in small environments + * only one atomic_t variable per CPU is used. + */ +#define LRNG_DATA_NUM_VALUES (CONFIG_LRNG_COLLECTION_SIZE) +/* Mask of LSB of time stamp to store */ +#define LRNG_DATA_WORD_MASK (LRNG_DATA_NUM_VALUES - 1) + +#define LRNG_DATA_SLOTS_MASK (LRNG_DATA_SLOTS_PER_UINT - 1) +#define LRNG_DATA_ARRAY_SIZE (LRNG_DATA_NUM_VALUES / \ + LRNG_DATA_SLOTS_PER_UINT) + +/* Starting bit index of slot */ +static inline unsigned int lrng_data_slot2bitindex(unsigned int slot) +{ + return (LRNG_DATA_SLOTSIZE_BITS * slot); +} + +/* Convert index into the array index */ +static inline unsigned int lrng_data_idx2array(unsigned int idx) +{ + return idx / LRNG_DATA_SLOTS_PER_UINT; +} + +/* Convert index into the slot of a given array index */ +static inline unsigned int lrng_data_idx2slot(unsigned int idx) +{ + return idx & LRNG_DATA_SLOTS_MASK; +} + +/* Convert value into slot value */ +static inline unsigned int lrng_data_slot_val(unsigned int val, + unsigned int slot) +{ + return val << lrng_data_slot2bitindex(slot); +} + +/* + * Return the pointers for the previous and current units to inject a u32 into. + * Also return the mask which the u32 word is to be processed. + */ +static inline void lrng_data_split_u32(u32 *ptr, u32 *pre_ptr, u32 *mask) +{ + /* ptr to previous unit */ + *pre_ptr = (*ptr - LRNG_DATA_SLOTS_PER_UINT) & LRNG_DATA_WORD_MASK; + *ptr &= LRNG_DATA_WORD_MASK; + + /* mask to split data into the two parts for the two units */ + *mask = ((1 << (*pre_ptr & (LRNG_DATA_SLOTS_PER_UINT - 1)) * + LRNG_DATA_SLOTSIZE_BITS)) - 1; +} + +#endif /* _LRNG_ES_TIMER_COMMON_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_interface_dev_common.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_INTERFACE_DEV_COMMON_H +#define _LRNG_INTERFACE_DEV_COMMON_H + +#include +#include + +/******************* Upstream functions hooked into the LRNG ******************/ +enum lrng_external_noise_source { + lrng_noise_source_hw, + lrng_noise_source_user +}; + +#ifdef CONFIG_LRNG_COMMON_DEV_IF +void lrng_writer_wakeup(void); +void lrng_init_wakeup_dev(void); +void lrng_state_exseed_set(enum lrng_external_noise_source source, bool type); +void lrng_state_exseed_allow_all(void); +#else /* CONFIG_LRNG_COMMON_DEV_IF */ +static inline void lrng_writer_wakeup(void) { } +static inline void lrng_init_wakeup_dev(void) { } +static inline void +lrng_state_exseed_set(enum lrng_external_noise_source source, bool type) { } +static inline void lrng_state_exseed_allow_all(void) { } +#endif /* CONFIG_LRNG_COMMON_DEV_IF */ + +/****** Downstream service functions to actual interface implementations ******/ + +bool lrng_state_exseed_allow(enum lrng_external_noise_source source); +int lrng_fasync(int fd, struct file *filp, int on); +long lrng_ioctl(struct file *f, unsigned int cmd, unsigned long arg); +ssize_t lrng_drng_write(struct file *file, const char __user *buffer, + size_t count, loff_t *ppos); +ssize_t lrng_drng_write_common(const char __user *buffer, size_t count, + u32 entropy_bits); +__poll_t lrng_random_poll(struct file *file, poll_table *wait); +ssize_t lrng_read_common_block(int nonblock, int pr, + char __user *buf, size_t nbytes); +ssize_t lrng_drng_read_block(struct file *file, char __user *buf, size_t nbytes, + loff_t *ppos); +ssize_t lrng_read_seed(char __user *buf, size_t nbytes, unsigned int flags); +ssize_t lrng_read_common(char __user *buf, size_t nbytes, bool pr); +bool lrng_need_entropy(void); + +extern struct wait_queue_head lrng_write_wait; + +#endif /* _LRNG_INTERFACE_DEV_COMMON_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_interface_random_kernel.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_INTERFACE_RANDOM_H +#define _LRNG_INTERFACE_RANDOM_H + +#ifdef CONFIG_LRNG_RANDOM_IF +void invalidate_batched_entropy(void); +void lrng_kick_random_ready(void); +#else /* CONFIG_LRNG_RANDOM_IF */ +static inline void invalidate_batched_entropy(void) { } +static inline void lrng_kick_random_ready(void) { } +#endif /* CONFIG_LRNG_RANDOM_IF */ + +#endif /* _LRNG_INTERFACE_RANDOM_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_numa.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_NUMA_H +#define _LRNG_NUMA_H + +static inline struct lrng_drng **lrng_drng_instances(void) { return NULL; } + +#endif /* _LRNG_NUMA_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_sha.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * LRNG SHA definition usable in atomic contexts right from the start of the + * kernel. + * + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_SHA_H +#define _LRNG_SHA_H + +extern const struct lrng_hash_cb lrng_sha_hash_cb; + +#endif /* _LRNG_SHA_H */ --- /dev/null +++ b/drivers/char/lrng/lrng_sha1.c @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* + * Backend for the LRNG providing the SHA-1 implementation that can be used + * without the kernel crypto API available including during early boot and in + * atomic contexts. + * + * Copyright (C) 2022, Stephan Mueller + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include "lrng_sha.h" + +/* + * If the SHA-256 support is not compiled, we fall back to SHA-1 that is always + * compiled and present in the kernel. + */ +static u32 lrng_sha1_hash_digestsize(void *hash) +{ + return SHA1_DIGEST_SIZE; +} + +static void lrng_sha1_block_fn(struct sha1_state *sctx, const u8 *src, + int blocks) +{ + u32 temp[SHA1_WORKSPACE_WORDS]; + + while (blocks--) { + sha1_transform(sctx->state, src, temp); + src += SHA1_BLOCK_SIZE; + } + memzero_explicit(temp, sizeof(temp)); +} + +static int lrng_sha1_hash_init(struct shash_desc *shash, void *hash) +{ + /* + * We do not need a TFM - we only need sufficient space for + * struct sha1_state on the stack. + */ + sha1_base_init(shash); + return 0; +} + +static int lrng_sha1_hash_update(struct shash_desc *shash, + const u8 *inbuf, u32 inbuflen) +{ + return sha1_base_do_update(shash, inbuf, inbuflen, lrng_sha1_block_fn); +} + +static int lrng_sha1_hash_final(struct shash_desc *shash, u8 *digest) +{ + return sha1_base_do_finalize(shash, lrng_sha1_block_fn) ?: + sha1_base_finish(shash, digest); +} + +static const char *lrng_sha1_hash_name(void) +{ + return "SHA-1"; +} + +static void lrng_sha1_hash_desc_zero(struct shash_desc *shash) +{ + memzero_explicit(shash_desc_ctx(shash), sizeof(struct sha1_state)); +} + +static void *lrng_sha1_hash_alloc(void) +{ + pr_info("Hash %s allocated\n", lrng_sha1_hash_name()); + return NULL; +} + +static void lrng_sha1_hash_dealloc(void *hash) { } + +const struct lrng_hash_cb lrng_sha_hash_cb = { + .hash_name = lrng_sha1_hash_name, + .hash_alloc = lrng_sha1_hash_alloc, + .hash_dealloc = lrng_sha1_hash_dealloc, + .hash_digestsize = lrng_sha1_hash_digestsize, + .hash_init = lrng_sha1_hash_init, + .hash_update = lrng_sha1_hash_update, + .hash_final = lrng_sha1_hash_final, + .hash_desc_zero = lrng_sha1_hash_desc_zero, +}; --- /dev/null +++ b/drivers/char/lrng/lrng_sha256.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause +/* + * Backend for the LRNG providing the SHA-256 implementation that can be used + * without the kernel crypto API available including during early boot and in + * atomic contexts. + * + * Copyright (C) 2022, Stephan Mueller + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include + +#include "lrng_sha.h" + +static u32 lrng_sha256_hash_digestsize(void *hash) +{ + return SHA256_DIGEST_SIZE; +} + +static int lrng_sha256_hash_init(struct shash_desc *shash, void *hash) +{ + /* + * We do not need a TFM - we only need sufficient space for + * struct sha256_state on the stack. + */ + sha256_init(shash_desc_ctx(shash)); + return 0; +} + +static int lrng_sha256_hash_update(struct shash_desc *shash, + const u8 *inbuf, u32 inbuflen) +{ + sha256_update(shash_desc_ctx(shash), inbuf, inbuflen); + return 0; +} + +static int lrng_sha256_hash_final(struct shash_desc *shash, u8 *digest) +{ + sha256_final(shash_desc_ctx(shash), digest); + return 0; +} + +static const char *lrng_sha256_hash_name(void) +{ + return "SHA-256"; +} + +static void lrng_sha256_hash_desc_zero(struct shash_desc *shash) +{ + memzero_explicit(shash_desc_ctx(shash), sizeof(struct sha256_state)); +} + +static void *lrng_sha256_hash_alloc(void) +{ + pr_info("Hash %s allocated\n", lrng_sha256_hash_name()); + return NULL; +} + +static void lrng_sha256_hash_dealloc(void *hash) { } + +const struct lrng_hash_cb lrng_sha_hash_cb = { + .hash_name = lrng_sha256_hash_name, + .hash_alloc = lrng_sha256_hash_alloc, + .hash_dealloc = lrng_sha256_hash_dealloc, + .hash_digestsize = lrng_sha256_hash_digestsize, + .hash_init = lrng_sha256_hash_init, + .hash_update = lrng_sha256_hash_update, + .hash_final = lrng_sha256_hash_final, + .hash_desc_zero = lrng_sha256_hash_desc_zero, +}; --- /dev/null +++ b/drivers/char/lrng/lrng_sysctl.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_SYSCTL_H +#define _LRNG_SYSCTL_H + +#ifdef CONFIG_LRNG_SYSCTL +void lrng_sysctl_update_max_write_thresh(u32 new_digestsize); +#else +static inline void lrng_sysctl_update_max_write_thresh(u32 new_digestsize) { } +#endif + +#endif /* _LRNG_SYSCTL_H */ --- /dev/null +++ b/include/linux/lrng.h @@ -0,0 +1,251 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ +/* + * Copyright (C) 2022, Stephan Mueller + */ + +#ifndef _LRNG_H +#define _LRNG_H + +#include +#include +#include +#include + +/* + * struct lrng_drng_cb - cryptographic callback functions defining a DRNG + * @drng_name Name of DRNG + * @drng_alloc: Allocate DRNG -- the provided integer should be used for + * sanity checks. + * return: allocated data structure or PTR_ERR on error + * @drng_dealloc: Deallocate DRNG + * @drng_seed: Seed the DRNG with data of arbitrary length drng: is + * pointer to data structure allocated with drng_alloc + * return: >= 0 on success, < 0 on error + * @drng_generate: Generate random numbers from the DRNG with arbitrary + * length + */ +struct lrng_drng_cb { + const char *(*drng_name)(void); + void *(*drng_alloc)(u32 sec_strength); + void (*drng_dealloc)(void *drng); + int (*drng_seed)(void *drng, const u8 *inbuf, u32 inbuflen); + int (*drng_generate)(void *drng, u8 *outbuf, u32 outbuflen); +}; + +/* + * struct lrng_hash_cb - cryptographic callback functions defining a hash + * @hash_name Name of Hash used for reading entropy pool arbitrary + * length + * @hash_alloc: Allocate the hash for reading the entropy pool + * return: allocated data structure (NULL is success too) + * or ERR_PTR on error + * @hash_dealloc: Deallocate Hash + * @hash_digestsize: Return the digestsize for the used hash to read out + * entropy pool + * hash: is pointer to data structure allocated with + * hash_alloc + * return: size of digest of hash in bytes + * @hash_init: Initialize hash + * hash: is pointer to data structure allocated with + * hash_alloc + * return: 0 on success, < 0 on error + * @hash_update: Update hash operation + * hash: is pointer to data structure allocated with + * hash_alloc + * return: 0 on success, < 0 on error + * @hash_final Final hash operation + * hash: is pointer to data structure allocated with + * hash_alloc + * return: 0 on success, < 0 on error + * @hash_desc_zero Zeroization of hash state buffer + * + * Assumptions: + * + * 1. Hash operation will not sleep + * 2. The hash' volatile state information is provided with *shash by caller. + */ +struct lrng_hash_cb { + const char *(*hash_name)(void); + void *(*hash_alloc)(void); + void (*hash_dealloc)(void *hash); + u32 (*hash_digestsize)(void *hash); + int (*hash_init)(struct shash_desc *shash, void *hash); + int (*hash_update)(struct shash_desc *shash, const u8 *inbuf, + u32 inbuflen); + int (*hash_final)(struct shash_desc *shash, u8 *digest); + void (*hash_desc_zero)(struct shash_desc *shash); +}; + +/* Register cryptographic backend */ +#ifdef CONFIG_LRNG_SWITCH +int lrng_set_drng_cb(const struct lrng_drng_cb *cb); +int lrng_set_hash_cb(const struct lrng_hash_cb *cb); +#else /* CONFIG_LRNG_SWITCH */ +static inline int +lrng_set_drng_cb(const struct lrng_drng_cb *cb) { return -EOPNOTSUPP; } +static inline int +lrng_set_hash_cb(const struct lrng_hash_cb *cb) { return -EOPNOTSUPP; } +#endif /* CONFIG_LRNG_SWITCH */ + +/* Callback to feed events to the scheduler entropy source */ +#ifdef CONFIG_LRNG_SCHED +extern void add_sched_randomness(const struct task_struct *p, int cpu); +#else +static inline void +add_sched_randomness(const struct task_struct *p, int cpu) { } +#endif + +/* + * lrng_get_random_bytes() - Provider of cryptographic strong random numbers + * for kernel-internal usage. + * + * This function is appropriate for in-kernel use cases operating in atomic + * contexts. It will always use the ChaCha20 DRNG and it may be the case that + * it is not fully seeded when being used. + * + * @buf: buffer to store the random bytes + * @nbytes: size of the buffer + */ +#ifdef CONFIG_LRNG_DRNG_ATOMIC +void lrng_get_random_bytes(void *buf, int nbytes); +#endif + +/* + * lrng_get_random_bytes_full() - Provider of cryptographic strong + * random numbers for kernel-internal usage from a fully initialized LRNG. + * + * This function will always return random numbers from a fully seeded and + * fully initialized LRNG. + * + * This function is appropriate only for non-atomic use cases as this + * function may sleep. It provides access to the full functionality of LRNG + * including the switchable DRNG support, that may support other DRNGs such + * as the SP800-90A DRBG. + * + * @buf: buffer to store the random bytes + * @nbytes: size of the buffer + */ +#ifdef CONFIG_LRNG +void lrng_get_random_bytes_full(void *buf, int nbytes); +#endif + +/* + * lrng_get_random_bytes_min() - Provider of cryptographic strong + * random numbers for kernel-internal usage from at least a minimally seeded + * LRNG, which is not necessarily fully initialized yet (e.g. SP800-90C + * oversampling applied in FIPS mode is not applied yet). + * + * This function is appropriate only for non-atomic use cases as this + * function may sleep. It provides access to the full functionality of LRNG + * including the switchable DRNG support, that may support other DRNGs such + * as the SP800-90A DRBG. + * + * @buf: buffer to store the random bytes + * @nbytes: size of the buffer + */ +#ifdef CONFIG_LRNG +void lrng_get_random_bytes_min(void *buf, int nbytes); +#endif + +/* + * lrng_get_random_bytes_pr() - Provider of cryptographic strong + * random numbers for kernel-internal usage from a fully initialized LRNG and + * requiring a reseed from the entropy sources before. + * + * This function will always return random numbers from a fully seeded and + * fully initialized LRNG. + * + * This function is appropriate only for non-atomic use cases as this + * function may sleep. It provides access to the full functionality of LRNG + * including the switchable DRNG support, that may support other DRNGs such + * as the SP800-90A DRBG. + * + * This call only returns no more data than entropy was pulled from the + * entropy sources. Thus, it is likely that this call returns less data + * than requested by the caller. Also, the caller MUST be prepared that this + * call returns 0 bytes, i.e. it did not generate data. + * + * @buf: buffer to store the random bytes + * @nbytes: size of the buffer + * + * @return: positive number indicates amount of generated bytes, < 0 on error + */ +#ifdef CONFIG_LRNG +int lrng_get_random_bytes_pr(void *buf, int nbytes); +#endif + +/* + * lrng_get_seed() - Fill buffer with data from entropy sources + * + * This call allows accessing the entropy sources directly and fill the buffer + * with data from all available entropy sources. This filled buffer is + * identical to the temporary seed buffer used by the LRNG to seed its DRNGs. + * + * The call is to allows users to seed their DRNG directly from the entropy + * sources in case the caller does not want to use the LRNG's DRNGs. This + * buffer can be directly used to seed the caller's DRNG from. + * + * The call blocks as long as one LRNG DRNG is not yet fully seeded. If + * LRNG_GET_SEED_NONBLOCK is specified, it does not block in this case, but + * returns with an error. + * + * Considering SP800-90C, there is a differentiation between the seeding + * requirements during instantiating a DRNG and at runtime of the DRNG. When + * specifying LRNG_GET_SEED_FULLY_SEEDED the caller indicates the DRNG was + * already fully seeded and the regular amount of entropy is requested. + * Otherwise, the LRNG will obtain the entropy rate required for initial + * seeding. The following minimum entropy rates will be obtained: + * + * * FIPS mode: + * * Initial seeding: 384 bits of entropy + * * Runtime seeding: 256 bits of entropy + * * Non-FIPS mode: + * * 128 bits of entropy in any case + * + * Albeit these are minimum entropy rates, the LRNG tries to request the + * given amount of entropy from each entropy source individually. If the + * minimum amount of entropy cannot be obtained collectively by all entropy + * sources, the LRNG will not fill the buffer. + * + * The return data in buf is structurally equivalent to the following + * definition: + * + * struct { + * u64 seedlen; + * u64 entropy_rate; + * struct entropy_buf seed; + * } __attribute((__packed__)); + * + * As struct entropy_buf is not known outsize of the LRNG, the LRNG fills + * seedlen first with the size of struct entropy_buf. If the caller-provided + * buffer buf is smaller than u64, then -EINVAL is returned + * and buf is not touched. If it is u64 or larger but smaller + * than the size of the structure above, -EMSGSIZE is returned and seedlen + * is filled with the size of the buffer. Finally, if buf is large + * enough to hold all data, it is filled with the seed data and the seedlen + * is set to sizeof(struct entropy_buf). The entropy rate is returned with + * the variable entropy_rate and provides the value in bits. + * + * The seed buffer is the data that should be handed to the caller's DRNG as + * seed data. + * + * @buf [out] Buffer to be filled with data from the entropy sources - note, the + * buffer is marked as u64 to ensure it is aligned to 64 bits. + * @nbytes [in] Size of the buffer allocated by the caller - this value + * provides size of @param buf in bytes. + * @flags [in] Flags field to adjust the behavior + * + * @return -EINVAL or -EMSGSIZE indicating the buffer is too small, -EAGAIN when + * the call would block, but NONBLOCK is specified, > 0 the size of + * the filled buffer. + */ +#ifdef CONFIG_LRNG +enum lrng_get_seed_flags { + LRNG_GET_SEED_NONBLOCK = 0x0001, /**< Do not block the call */ + LRNG_GET_SEED_FULLY_SEEDED = 0x0002, /**< DRNG is fully seeded */ +}; + +ssize_t lrng_get_seed(u64 *buf, size_t nbytes, unsigned int flags); +#endif + +#endif /* _LRNG_H */