7.23. QEMU virt Armv8-A

Trusted Firmware-A (TF-A) implements the EL3 firmware layer for QEMU virt Armv8-A. BL1 is used as the BootROM, supplied with the -bios argument. When QEMU starts all CPUs are released simultaneously, BL1 selects a primary CPU to handle the boot and the secondaries are placed in a polling loop to be released by normal world via PSCI.

BL2 edits the Flattened Device Tree, FDT, generated by QEMU at run-time to add a node describing PSCI and also enable methods for the CPUs.

If ARM_LINUX_KERNEL_AS_BL33 is set to 1 then this FDT will be passed to BL33 via register x0, as expected by a Linux kernel. This allows a Linux kernel image to be booted directly as BL33 rather than using a bootloader.

An ARM64 defconfig v5.5 Linux kernel is known to boot, FDT doesn’t need to be provided as it’s generated by QEMU.

Current limitations:

  • Only cold boot is supported

7.23.1. Getting non-TF images

QEMU_EFI.fd can be downloaded from http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-KERNEL-AARCH64/RELEASE_GCC5/QEMU_EFI.fd

or, can be built as follows:

git clone https://github.com/tianocore/edk2.git
cd edk2
git submodule update --init
make -C BaseTools
source edksetup.sh
export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
build -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemuKernel.dsc

Then, you will get Build/ArmVirtQemuKernel-AARCH64/DEBUG_GCC5/FV/QEMU_EFI.fd

Please note you do not need to use GCC 5 in spite of the environment variable GCC5_AARCH64_PREFIX.

The rootfs can be built by using Buildroot as follows:

git clone git://git.buildroot.net/buildroot.git
cd buildroot
make qemu_aarch64_virt_defconfig
utils/config -e BR2_TARGET_ROOTFS_CPIO
utils/config -e BR2_TARGET_ROOTFS_CPIO_GZIP
make olddefconfig
make

Then, you will get output/images/rootfs.cpio.gz.

7.23.2. Booting via semi-hosting option

Boot binaries, except BL1, are primarily loaded via semi-hosting so all binaries has to reside in the same directory as QEMU is started from. This is conveniently achieved with symlinks the local names as:

  • bl2.bin -> BL2

  • bl31.bin -> BL31

  • bl33.bin -> BL33 (QEMU_EFI.fd)

  • Image -> linux/arch/arm64/boot/Image

To build:

make CROSS_COMPILE=aarch64-none-elf- PLAT=qemu

To start (QEMU v5.0.0):

qemu-system-aarch64 -nographic -machine virt,secure=on -cpu cortex-a57  \
    -kernel Image                           \
    -append "console=ttyAMA0,38400 keep_bootcon"   \
    -initrd rootfs.cpio.gz -smp 2 -m 1024 -bios bl1.bin   \
    -d unimp -semihosting-config enable,target=native

7.23.3. Booting via flash based firmware

An alternate approach to deploy a full system stack on QEMU is to load the firmware via a secure flash device. This involves concatenating bl1.bin and fip.bin to create a boot ROM that is flashed onto secure FLASH0 with the -bios option.

For example, to test the following firmware stack:

  • BL32 - bl32.bin -> tee-header_v2.bin

  • BL32 Extra1 - bl32_extra1.bin -> tee-pager_v2.bin

  • BL32 Extra2 - bl32_extra2.bin -> tee-pageable_v2.bin

  • BL33 - bl33.bin -> QEMU_EFI.fd (EDK II)

  • Image -> linux/arch/arm64/boot/Image

  1. Compile TF-A

make CROSS_COMPILE=aarch64-linux-gnu- PLAT=qemu BL32=bl32.bin \
    BL32_EXTRA1=bl32_extra1.bin BL32_EXTRA2=bl32_extra2.bin \
    BL33=bl33.bin BL32_RAM_LOCATION=tdram SPD=opteed all fip

Or, alternatively, to build with TBBR enabled, as well as, BL31 and BL32 encrypted with test key:

make CROSS_COMPILE=aarch64-linux-gnu- PLAT=qemu BL32=bl32.bin \
    BL32_EXTRA1=bl32_extra1.bin BL32_EXTRA2=bl32_extra2.bin \
    BL33=bl33.bin BL32_RAM_LOCATION=tdram SPD=opteed all fip \
    MBEDTLS_DIR=<path-to-mbedtls-repo> TRUSTED_BOARD_BOOT=1 \
    GENERATE_COT=1 DECRYPTION_SUPPORT=aes_gcm FW_ENC_STATUS=0 \
    ENCRYPT_BL31=1 ENCRYPT_BL32=1

  1. Concatenate bl1.bin and fip.bin to create the boot ROM

dd if=build/qemu/release/bl1.bin of=flash.bin bs=4096 conv=notrunc
dd if=build/qemu/release/fip.bin of=flash.bin seek=64 bs=4096 conv=notrunc

  1. Launch QEMU

qemu-system-aarch64 -nographic -machine virt,secure=on
    -cpu cortex-a57  -kernel Image   \
    -append 'console=ttyAMA0,38400 keep_bootcon'  \
    -initrd rootfs.cpio.gz -smp 2 -m 1024 -bios flash.bin   \
    -d unimp

The -bios option abstracts the loading of raw bare metal binaries into flash or ROM memory. QEMU loads the binary into the region corresponding to the hardware’s entrypoint, from which the binary is executed upon a platform “reset”. In addition to this, it places the information about the kernel provided with option -kernel, and the RamDisk provided with -initrd, into the firmware configuration fw_cfg. In this setup, EDK II is responsible for extracting and launching these from fw_cfg.

Note

QEMU may be launched with or without ACPI (-acpi/-no-acpi). In either case, ensure that the kernel build options are aligned with the parameters passed to QEMU.

7.23.4. Running QEMU in OpenCI

Linaro’s continuous integration platform OpenCI supports running emulated tests on QEMU. The tests are kicked off on Jenkins and deployed through the Linaro Automation and Validation Architecture LAVA.

There are a set of Linux boot tests provided in OpenCI. They rely on prebuilt binaries for UEFI, the kernel, root file system, as well as, any other TF-A dependencies, and are run as part of the OpenCI TF-A daily job. To run them manually, a builder job may be triggered with the test configuration qemu-boot-tests.

You may see the following warning repeated several times in the boot logs:

pflash_write: Write to buffer emulation is flawed

Please ignore this as it is an unresolved issue in QEMU, it is an internal QEMU warning that logs flawed use of “write to buffer”.

Note

For more information on how to trigger jobs in OpenCI, please refer to Linaro’s CI documentation, which explains how to trigger a manual job.