4.24. Context Management Library
This document provides an overview of the Context Management library implementation in Trusted Firmware-A (TF-A). It enumerates and describes the APIs implemented and their accessibility from other components at EL3.
4.24.1. Overview
Arm TrustZone architecture facilitates hardware-enforced isolation between software running in various security states (Secure/Non-Secure/Realm). The general-purpose registers, most of the system registers and vector registers are not banked per world. When moving between the security states it is the responsibility of the secure monitor software (BL31(AArch64) / BL32(Aarch32)) in TF-A, not the hardware, to save and restore register state. Refer to Trustzone for AArch64 for more details.
EL3 Runtime Firmware, also termed as secure monitor firmware, is integrated with a context management library to handle the context of the CPU, managing the saving and restoring of register states across the worlds.
4.24.2. TF-A Context
In TF-A, the context is represented as a data structure used by the EL3 firmware to preserve the state of the CPU at the next lower exception level (EL) in a given security state and save enough EL3 metadata to be able to return to that exception level and security state. The memory for the context data structures are allocated in BSS section of EL3 firmware.
In a trusted system at any instance, a given CPU could be executing in one of the security states (Non-Secure, Secure, Realm). Each world must have its configuration of system registers independent of other security states to access and execute any of the architectural features.
If the CPU switches across security states (for example: from Non-secure to Secure or vice versa), the register contents, especially the ones that are not banked (EL2/EL1, vector, general-purpose registers), will be overwritten, as the software running in either state has the privileges to access them. Additionally, some of the architectural features enabled in the former security state will be unconditionally accessible in the latter security state as well. This can be a major concern when dealing with security-specific bits, as they need to be explicitly enabled or disabled in each state to prevent data leakage across the worlds.
In general, an ideal trusted system should have Secure world-specific configurations that are not influenced by Normal World operations. Therefore, for each CPU, we need to maintain world-specific context to ensure that register entries from one world do not leak or impact the execution of the CPU in other worlds. This will help ensure the integrity and security of the system, preventing any unauthorized access or data corruption between the different security states.
4.24.3. Design
The Context Management library in TF-A is designed to cover all the requirements for maintaining world-specific context essential for a trusted system. This includes implementing CPU context initialization and management routines, as well as other helper APIs that are required by dispatcher components in EL3 firmware, which are collectively referred to as CPU Context Management. The APIs and their usecases are listed in detail under the Library APIs section.
Originally, the Context Management library in TF-A was designed to cater for a two-world system, comprising of Non-Secure and Secure Worlds. In this case, the EL3 Firmware is assumed to be running in Secure World. With introduction of Realm Management Extension (RME), from Armv9.2 a system can have four distinct worlds (Non-Secure, Secure, Realm, Root). RME isolates EL3 from all other Security states and moves it into its own security state called root. EL3 firmware now runs at Root World and thereby is trusted from software in Non-secure, Secure, and Realm states. Refer to Security States with RME for more details.
Key principles followed in designing the context management library :
EL3 should only initialize immediate used lower EL
Context Management library running at EL3 should only initialize and monitor the immediate used lower EL. This implies that, when S-EL2 is present in the system, EL3 should initialise and monitor S-EL2 registers only. S-EL1 registers should not be the concern of EL3 while S-EL2 is in place. In systems where S-EL2 is absent, S-EL1 registers should be initialised from EL3.
Decentralized model for context management
Each world (Non-Secure, Secure, and Realm) should have their separate component in EL3 responsible for their respective world context management. Both the Secure and Realm world have associated dispatcher components in EL3 firmware to allow management of the respective worlds. For the Non-Secure world, PSCI Library (BL31)/context management library provides routines to help initialize the Non-Secure world context.
Flexibility for Dispatchers to select desired feature set to save and restore
Each feature is supported with a helper function is_feature_supported(void)
,
to detect its presence at runtime. This helps dispatchers to select the desired
feature set, and thereby save and restore the configuration associated with them.
Dynamic discovery of Feature enablement by EL3
TF-A supports four states for feature enablement at EL3, to make them available for lower exception levels.
#define FEAT_STATE_DISABLED 0
#define FEAT_STATE_ENABLED 1
#define FEAT_STATE_CHECK 2
#define FEAT_STATE_CHECK_ASYMMETRIC 3
A pattern is established for feature enablement behavior. Each feature must support the 3 possible values with rigid semantics.
FEAT_STATE_DISABLED - all code relating to this feature is always skipped. Firmware is unaware of this feature.
FEAT_STATE_ALWAYS - all code relating to this feature is always executed. Firmware expects this feature to be present in hardware.
FEAT_STATE_CHECK - same as
FEAT_STATE_ALWAYS
except that the feature’s existence will be checked at runtime. Default on dynamic platforms (example: FVP).FEAT_STATE_CHECK_ASYMMETRIC - same as
FEAT_STATE_CHECK
except that the feature’s existence is asymmetric across cores, which requires the feature existence is checked during warmboot path also. Note that only limited number of features can be asymmetric.
Note
Only limited number of features can be
FEAT_STATE_CHECK_ASYMMETRIC
this is due to the fact that Operating systems are designed for SMP systems. There are no clear guidelines what kind of mismatch is allowed but following pointers can help making a decision
All mandatory features must be symmetric.
Any feature that impacts the generation of page tables must be symmetric.
Any feature access which does not trap to EL3 should be symmetric.
Features related with profiling, debug and trace could be asymmetric
Migration of vCPU/tasks between CPUs should not cause an error
Whenever there is asymmetric feature support is added for a feature TF-A need to add feature specific code in context management code.
Note
FEAT_RAS
is an exception here, as it impacts the execution of EL3 and it is essential to know its presence at compile time. Refer toENABLE_FEAT
macro under Build Options section for more details.
4.24.4. Code Structure
lib/el3_runtime/(aarch32/aarch64) - Context library code directory.
4.24.4.1. Source Files
context_mgmt.c
: consists of core functions that setup, save and restore context for different security states alongside high level feature enablement APIs for individual worlds.cpu_data_array.c
: contains per_cpu_data structure instantiation.context.S
: consists of functions that save and restore some of the context structure members in assembly code.cpu_data.S
: consists of helper functions to initialise per_cpu_data pointers.el3_common_macros.S
: consists of macros to facilitate actions to be performed during cold and warmboot and el3 registers initialisation in assembly code.
4.24.4.2. Header Files
context_mgmt.h
: contains the public interface to Context Management Library.context.h
: contains the helper macros and definitions for context entries.cpu_data.h
: contains the public interface to Per CPU data structure.context_debug.h
: contains public interface to report context memory utilisation across the security states.context_el2.h
: internal header consisting of helper macros to access EL2 context entries. Used bycontext.h
.
Apart from these files, we have some context related source files under BL1
and BL31
directory. bl1_context_mgmt.c
bl31_context_mgmt.c
4.24.5. Bootloader Images utilizing Context Management Library
Bootloader |
Context Management Library |
BL1 |
Yes |
BL2 |
No |
BL31 (Aarch64- EL3runtime firmware) |
Yes |
BL32 (Aarch32- EL3runtime firmware) |
Yes |
4.24.6. CPU Data Structure
For a given system, depending on the CPU count, the platform statically allocates memory for the CPU data structure.
/* The per_cpu_ptr_cache_t space allocation */
cpu_data_t percpu_data[PLATFORM_CORE_COUNT];
This CPU data structure has a member element with an array of pointers to hold the Non-Secure, Realm and Secure security state context structures as listed below.
typedef struct cpu_data {
#ifdef __aarch64__
void *cpu_context[CPU_DATA_CONTEXT_NUM];
#endif
....
....
}cpu_data_t;
At runtime, cpu_context[CPU_DATA_CONTEXT_NUM]
array will be intitialised with
the Secure, Non-Secure and Realm context structure addresses to ensure proper
handling of the register state.
See Library APIs section for more details.
4.24.7. CPU Context and Memory allocation
4.24.7.1. CPU Context
The members of the context structure used by the EL3 firmware to preserve the state of CPU across exception levels for a given security state are listed below.
typedef struct cpu_context {
gp_regs_t gpregs_ctx;
el3_state_t el3state_ctx;
el1_sysregs_t el1_sysregs_ctx;
#if CTX_INCLUDE_EL2_REGS
el2_sysregs_t el2_sysregs_ctx;
#endif
#if CTX_INCLUDE_FPREGS
fp_regs_t fpregs_ctx;
#endif
cve_2018_3639_t cve_2018_3639_ctx;
#if CTX_INCLUDE_PAUTH_REGS
pauth_t pauth_ctx;
#endif
#if CTX_INCLUDE_MPAM_REGS
mpam_t mpam_ctx;
#endif
} cpu_context_t;
4.24.7.2. Context Memory Allocation
CPUs maintain their context per world. The individual context memory allocation for each CPU per world is allocated by the world-specific dispatcher components at compile time as shown below.
4.24.7.3. NS-Context Memory
It’s important to note that the Normal world doesn’t possess the dispatcher
component found in the Secure and Realm worlds. Instead, the PSCI library at EL3
handles memory allocation for Non-Secure
world context for all CPUs.
static cpu_context_t psci_ns_context[PLATFORM_CORE_COUNT];
4.24.7.4. Secure-Context Memory
Secure World dispatcher (such as SPMD) at EL3 allocates the memory for Secure
world context of all CPUs.
static spmd_spm_core_context_t spm_core_context[PLATFORM_CORE_COUNT];
4.24.7.5. Realm-Context Memory
Realm World dispatcher (RMMD) at EL3 allocates the memory for Realm
world
context of all CPUs.
rmmd_rmm_context_t rmm_context[PLATFORM_CORE_COUNT];
To summarize, the world-specific context structures are synchronized with per-CPU data structures, which means that each CPU will have an array of pointers to individual worlds. The figure below illustrates the same.
4.24.8. Context Setup/Initialization
The CPU has been assigned context structures for every security state, which include Non-Secure, Secure and Realm. It is crucial to initialize each of these structures during the bootup of every CPU before they enter any security state for the first time. This section explains the specifics of how the initialization of every CPU context takes place during both cold and warm boot paths.
4.24.8.1. Context Setup during Cold boot
The cold boot path is mainly executed by the primary CPU, other than essential CPU initialization executed by all CPUs. After executing BL1 and BL2, the Primary CPU jumps to the BL31 image for runtime services initialization. During this process, the per_cpu_data structure gets initialized with statically allocated world-specific context memory.
Later in the cold boot sequence, the BL31 image at EL3 checks for the presence of a Secure world image at S-EL2. If detected, it invokes the secure context initialization sequence under SPMD. Additionally, based on RME enablement, the Realm context gets initialized from the RMMD at EL3. Finally, before exiting to the normal world, the Non-Secure context gets initialized via the context management library. At this stage, all Primary CPU contexts are initialized and the CPU exits EL3 to enter the Normal world.
Note
The figure above illustrates a scenario on FVP for one of the build configurations with TFTF component at NS-EL2.
4.24.8.2. Context Setup during Warmboot
During a warm boot sequence, the primary CPU is responsible for powering on the secondary CPUs. Refer to CPU Reset and Firmware Design sections for more details on the warm boot.
The primary CPU initializes the Non-Secure context for the secondary CPU while
restoring re-entry information for the Non-Secure world.
It initialises via cm_init_context_by_index(target_idx, ep )
.
psci_warmboot_entrypoint()
is the warm boot entrypoint procedure.
During the warm bootup process, secondary CPUs have their secure context
initialized through SPMD at EL3. Upon successful SP initialization, the SPD
power management operations become shared with the PSCI library. During this
process, the SPMD duly registers its handlers with the PSCI library.
file: psci_common.c
const spd_pm_ops_t *psci_spd_pm;
file: spmd_pm.c
const spd_pm_ops_t spmd_pm = {
.svc_on_finish = spmd_cpu_on_finish_handler,
.svc_off = spmd_cpu_off_handler
}
Secondary CPUs during their bootup in the psci_cpu_on_finish()
routine get
their secure context initialised via the registered SPMD handler
spmd_cpu_on_finish_handler()
at EL3.
The figure above illustrates the same with reference of Primary CPU running at
NS-EL2.
4.24.9. Library APIs
The public APIs and types can be found in include/lib/el3_runtime/context_management.h
and this section is intended to provide additional details and clarifications.
4.24.9.1. Context Initialization for Individual Worlds
The library implements high level APIs for the CPUs in setting up their individual context for each world (Non-Secure, Secure and Realm).
-
static void setup_context_common(cpu_context_t *ctx, const entry_point_info_t *ep);
This function is responsible for the general context initialization that applies to all worlds. It will be invoked first, before calling the individual world-specific context setup APIs.
-
static void setup_ns_context(cpu_context_t *ctx, const struct entry_point_info *ep);
-
static void setup_realm_context(cpu_context_t *ctx, const struct entry_point_info *ep);
-
static void setup_secure_context(cpu_context_t *ctx, const struct entry_point_info *ep);
Depending on the security state that the CPU needs to enter, the respective world-specific context setup handlers listed above will be invoked once per-CPU to set up the context for their execution.
-
void cm_manage_extensions_el3(void)
This function initializes all EL3 registers whose values do not change during the
lifetime of EL3 runtime firmware. It is invoked from each CPU via the cold boot
path bl31_main()
and in the WarmBoot entry path void psci_warmboot_entrypoint()
.
4.24.9.2. Runtime Save and Restore of Registers
4.24.9.2.1. EL1 Registers
-
void cm_el1_sysregs_context_save(uint32_t security_state);
-
void cm_el1_sysregs_context_restore(uint32_t security_state);
These functions are utilized by the world-specific dispatcher components running at EL3 to facilitate the saving and restoration of the EL1 system registers during a world switch.
4.24.9.2.2. EL2 Registers
-
void cm_el2_sysregs_context_save(uint32_t security_state);
-
void cm_el2_sysregs_context_restore(uint32_t security_state);
These functions are utilized by the world-specific dispatcher components running at EL3 to facilitate the saving and restoration of the EL2 system registers during a world switch.
4.24.9.2.3. Pauth Registers
Pointer Authentication feature is enabled by default for Non-Secure world and
disabled for Secure and Realm worlds. In this case, we don’t need to explicitly
save and restore the Pauth registers during world switch.
However, CTX_INCLUDE_PAUTH_REGS
flag is explicitly used to enable Pauth for
lower exception levels of Secure and Realm worlds. In this scenario, we save the
general purpose and Pauth registers while we enter EL3 from lower ELs via
prepare_el3_entry
and restore them back while we exit EL3 to lower ELs
via el3_exit
.
.macro save_gp_pmcr_pauth_regs
func restore_gp_pmcr_pauth_regs
4.24.9.3. Feature Enablement for Individual Worlds
-
static void manage_extensions_nonsecure(cpu_context_t *ctx);
-
static void manage_extensions_secure(cpu_context_t *ctx);
-
static void manage_extensions_realm(cpu_context_t *ctx)
Functions that allow the enabling and disabling of architectural features for each security state. These functions are invoked from the top-level setup APIs during context initialization.
Further, a pattern is established for feature enablement code (AArch64).
Each feature implements following APIs as applicable:
Note: (xxx
is the name of the feature in the APIs)
is_feat_xxx_supported()
andis_feat_xxx_present()
- mandatory for all features.xxx_enable(cpu_context * )
andxxx_disable(cpu_context * )
- optional functions to enable the feature for the passed context only. To be called in the respective world’s setup_context to select behaviour.xxx_init_el3()
- optional function to enable the feature in-place in any EL3 registers that are never context switched. The values they write must never change, otherwise the functions mentioned in previous point should be used. Invoked fromcm_manage_extensions_el3()
.xxx_init_el2_unused()
- optional function to enable the feature in-place in any EL2 registers that are necessary for execution in EL1 with no EL2 present.
The above mentioned rules, followed for FEAT_SME
is shown below:
void sme_enable(cpu_context_t *context);
void sme_init_el3(void);
void sme_init_el2_unused(void);
void sme_disable(cpu_context_t *context);
4.24.10. Per-world Context
Apart from the CPU context structure, we have another structure to manage some
of the EL3 system registers whose values are identical across all the CPUs
referred to as per_world_context_t
.
The Per-world context structure is intended for managing EL3 system registers with
identical values across all CPUs, requiring only a singular context entry for each
individual world. This structure operates independently of the CPU context
structure and is intended to manage specific EL3 registers.
typedef struct per_world_context {
uint64_t ctx_cptr_el3;
uint64_t ctx_zcr_el3;
uint64_t ctx_mpam3_el3;
} per_world_context_t;
These functions facilitate the activation of architectural extensions that possess identical values across all cores for the individual Non-secure, Secure, and Realm worlds.
Copyright (c) 2024, Arm Limited and Contributors. All rights reserved.