Support for Arm CCA guest firmware v4 (RMM-v1.0-rel0)

[samimujawar]

Description

This v4 series enables the Arm Confidential Compute Architecture (CCA)
support for the Kvmtool guest firmware and is aligned with the ARM CCA
RMM 1.0-rel0 specification.

In addition to the RMM 1.0-rel0 specification updated this series has
been rebased with the latest edk2 & edk2-platforms codebase and the
intention is to integrate the Arm CCA support in ArmVirtPkg and enable
the guest firmware support for Realms.

Summary of updates in this v4 Series:

1. This series is a rebased version of the v3 series at:
   Support for Arm CCA guest firmware v3 (RMM-v1.0-rel0) edk2#6480
2. The patch "[PATCH v3 03/57] ArmPkg: Extend number of
   parameter registers in SMC" has been dropped as it is no
   longer required.
3. The feedback raised for the v3 series is not yet addressed and
   the plan is to discuss and address those as part of this pull request.

Summary of updates in the previous v3 Series:

1. Updates to align with RMM 1.0-rel0 specification.
2. Add support to parse the Kvmtool guest VM Device Tree
   and populate the memory map accordingly.
3. Check if the MMIO range is for a protected Realm device
   and if so, skip configuring the shared attribute for the
   MMIO region.
4. Add check to conditionally install the IORT table if
   a GIVc3-ITS is present.

Summary of updates in the previous v2 Series:

1. Variable emulation support patches that we part of v1 series
   are already merged, hence dropped from this series.
2. SetMemoryRegionAttributes() was dropped in the upstream
   code. Therefore, introduced SetMemoryProtectionAttribute()
   to configure the top bit of the Realm IPA space which is
   used as the protection bit
3. The patch to add the APRIORI Dxe ArmCcaDxe has been dropped.
4. Dropped patch that configured PcdMonitorConduitHvc as a
   dynamic PCD, and introduced ArmVirtMonitorLib, a new
   instance of the ArmMonitorLib that reads the conduit to
   be used from the FDT.
5. Bug fixes to correct the size of IMM field in RSI Host
   Call arguments, and to correct the RSI Version mask
6. Patches 32 to 43 include updates to the firmware support
   to RMM specification v1.0-EAC5.
7. Minor optimisations, e.g. to cache the current world value.

Introduction

Arm Confidential Compute Architecture (CCA)

Arm CCA is a reference software architecture and implementation that
builds on the Realm Management Extension (RME), enabling the execution
of Virtual machines (VMs), while preventing access by more privileged
software, such as hypervisor. Arm CCA allows the hypervisor to control
the VM, but removes the right for access to the code, register state or
data used by VM.

More information on the architecture is available here [1].

        Realm World     ||    Normal World   ||  Secure World  ||
                        ||        |          ||                ||
 EL0 x---------x        || x----x | x------x ||                ||
     | Realm   |        || |    | | |      | ||                ||
     |  VM*    |        || | VM | | |      | ||                ||
     |x-------x|        || |    | | |      | ||                ||
     ||       ||        || |    | | |  H   | ||                ||
     || Guest ||        || |    | | |      | ||                ||
 ----||  OS   ||--------||-|    |---|  o   |-||----------------||
     ||       ||        || |    | | |      | ||                ||
     |x-------x|        || |    | | |  s   | ||                ||
     |    ^    |        || |    | | |      | ||                ||
     |    |    |        || |    | | |  t   | ||                ||
     |+-------+|        || |    | | |      | ||                ||
     || REALM ||        || |    | | |      | ||                ||
     || GUEST ||        || |    | | |  O   | ||                ||
     || UEFI  ||        || |    | | |      | ||                ||
     |+-------+|        || |    | | |  S   | ||                ||
 EL1 x---------x        || x----x | |      | ||                ||
          ^             ||        | |      | ||                ||
          |             ||        | |      | ||                ||
 -------- R*------------||----------|      |-||----------------||
          S             ||          |      | ||                ||
          I             ||      x-->|      | ||                ||
          |             ||      |   |      | ||                ||
          |             ||      |   x------x ||                ||
          |             ||      |       ^    ||                ||
          v             ||     SMC      |    ||                ||
      x-------x         ||      |   x------x ||                ||
      |  RMM* |         ||      |   | HOST | ||                ||
      x-------x         ||      |   | UEFI | ||                ||
          ^             ||      |   x------x ||                ||
 EL2      |             ||      |            ||                ||
          |             ||      |            ||                ||
 =========|=====================|================================
          |                     |
          x------- *RMI* -------x

 EL3                   Root World
                       EL3 Firmware
 ===============================================================

Where:
RMM - Realm Management Monitor
RMI - Realm Management Interface
RSI - Realm Service Interface
SMC - Secure Monitor Call

RME introduces two added additional worlds, "Realm world" and "Root
World" in addition to the traditional Secure world and Normal world.
The Arm CCA defines a new component, Realm Management Monitor (RMM)
that runs at R-EL2. This is a standard piece of firmware, verified,
installed and loaded by the EL3 firmware (e.g., TF-A), at system boot.

The RMM provides a standard interface Realm Management Interface (RMI)
to the Normal world hypervisor to manage the VMs running in the Realm
world (also called Realms). These are exposed via SMC and are routed
through the EL3 firmware.

The RMM also provides certain services to the Realms via SMC, called
the Realm Service Interface (RSI). These include:

• Realm Guest Configuration
• Attestation & Measurement services
• Managing the state of an Intermediate Physical Address (IPA aka GPA)
  page
• Host Call service (Communication with the Normal world Hypervisor).

This patch series aligns with the RMM v1.0-rel0 specification, and
the latest version is available here [2].

The Trusted Firmware foundation has an implementation of the RMM -
TF-RMM - available here [4].

Implementation

This version of the Realm Guest UEFI firmware is intended to be
used with the Linux Kernel stack[7] which is also based on the
RMM specification v1.0-rel0[3].

This release includes the following features:
a) Boot a Linux Kernel in a Realm VM using the Realm Guest UEFI
firmware
b) Hardware description is provided using ACPI tables
c) Support for Virtio v1.0
d) All I/O are treated as non-secure/shared
e) Load the Linux Kernel and RootFS from a Virtio attached disk
using the Virtio-1.0 PCIe transport.

Overview of updates for enabling Arm CCA

The Arm CCA implementation is spread across a number of libraries
that provide required functionality during various phases of the
firmware boot.

The following libraries have been provided:
i. ArmCcaInitPeiLib - A library that implements the hook functions
in the PEI phase
ii. ArmCcaLib - A library that implements common functions like
checking if RME extension is implemented and to configure the
Protection attribute for the memory regions
iii. ArmCcaRsiLib - A library that implements the Realm Service
Interface functions.

A NULL implementation of the ArmCcaInitPeiLib and ArmCcaLib is also
provided for platforms that do not implement the RME extensions.

Additionally, the following DXE modules have been provided to implement
the required functionality in the DXE phase.
i. RealmApertureManagementProtocolDxe - A DXE that implements the
Realm Aperture Management Protocol, used to manage the sharing
of buffers in a Realm with the Host
ii. ArmCcaIoMmuDxe - A driver which implements the EDKII_IOMMU_PROTOCOL
that provides the necessary hooks so that DMA operations can be
performed by bouncing buffers using pages shared with the Host.

Arm CCA updates in PEI phase

For supporting Arm CCA two hooks have been added in the PrePi module:
i. An early hook to configure the System Memory as Protected RAM
ii. A second hook after the MMU is initialised to perform the
remaining CCA initialisations like reading the Realm Config
to determine the IPA width of the realm, configuring the
Protection attribute for the MMIO regions, etc.

These hook functions are implemented in ArmCcaInitPeiLib. A NULL
version of the library has also been provided for implementations
that do not have the RME extensions.

Additionally, the ArmVirtMemInfoLib has been updated to implement
a platform specific hook function ArmCcaConfigureMmio() that can
configure the protection attribute for the MMIO regions for the
platform.

   +=====+
   |PrePi|
   +=====+
      |
      _ModuleEntryPoint()
      ===================
              |
              DiscoverDramFromDt()
              |
              +--> ArmCcaInitPeiLib|ArmCcaConfigureSystemMemory()
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                             |      // configure System Memory
              ----------------      // as Protected RAM.
              |
             ...
              |
      --------
      |
      CEntryPoint()
      |
      PrePiMain()
      ===========
          |
         ...
          |
          ProcessLibraryConstructorList()
          |
          MemoryPeim()
          |
          ArmCcaInitPeiLib|ArmCcaInitialize()  // Perform Arm CCA
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  // initialisations,
                   |                           // like reading the
                   |                           // Realm Config, etc.
                   |
                   ArmVirtMemInfoLib|ArmCcaConfigureMmio()
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                          |   // Configure Protection attribute
                   --------   // for the MMIO region.
                   |
          ----------
          |
         ...
          |
        +===+
        |DXE|
        +===+

Building the UEFI firmware

a. Set up the development environment
Follow the steps as described in
https://github.com/tianocore/edk2-platforms/blob/master/Platform/ARM/Readme.md

b. The source code for the Host and Realm Guest firmware can
be downloaded from [12].

c. Building the Host UEFI firmware for FVP Base RevC AEM Model
Follow the instructions in
https://github.com/tianocore/edk2-platforms/blob/master/Platform/ARM/Readme.md
to "Build the firmware for Arm FVP Base AEMv8A-AEMv8A model
platform" based on your development environment configuration.

Note: The same firmware binary can be used for both the Arm FVP
Base AEMv8A-AEMv8A and the FVP Base RevC AEM Model.

d. Building the Realm Guest UEFI firmware for kvmtool:
To build the kvmtool guest firmware, run the following commands:

   $build -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtKvmTool.dsc -b DEBUG
   $build -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtKvmTool.dsc -b RELEASE

The Kvmtool guest firmware binaries are at the following location:

   $WORKSPACE/Build/ArmVirtKvmTool-AARCH64/<DEBUG|RELEASE>_GCC5/FV/KVMTOOL_EFI.fd

Running the stack

To run/test the stack, you would need the following components:

i. FVP Base AEM RevC model with FEAT_RME support [5]
ii. TF-A firmware for EL3 [6]
iii. TF-A RMM for R-EL2 [4]
iv. Linux Kernel [7]
v. kvmtool [8]
vi. UEFI Firmware for Arm CCA [12].

Instructions for building the remaining firmware components and
running the model are available here [10]. Once, the host kernel
has finished booting, a Realm can be launched by invoking the
lkvm command as follows:

 $ lkvm run --realm \
   --restricted_mem \
   --measurement-algo=["sha256", "sha512"] \
   --firmware KVMTOOL_EFI.fd \
   -m 512 \
   --irqchip=gicv3-its \
   --force-pci \
   --disk <Disk image containing the Guest Kernel & RootFS>
   <normal-vm-options>

Where:

• --measurement-algo (Optional) specifies the algorithm selected for
  creating the initial measurements by the RMM for this Realm (defaults
  to sha256)
• GICv3 is mandatory for the Realms
• --force-pci is required as only Virtio-v1.0 PCIe transport is
  supported.

Links

[1] Arm CCA Landing page (See Key Resources section for various documentations)
https://www.arm.com/armcca

[2] RMM Specification Latest
https://developer.arm.com/documentation/den0137/latest

[3] RMM v1.0-rel0 specification
https://developer.arm.com/documentation/den0137/1-0rel0

[4] Trusted Firmware RMM - TF-RMM
https://www.trustedfirmware.org/projects/tf-rmm/
GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
Branch: main
SHA: 5f9ddf18bf5295c705c0e79fe1b60e34aba11198

[5] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms

[6] Trusted Firmware for A class
https://www.trustedfirmware.org/projects/tf-a/
Branch: master
SHA: 61cdb4545b7de576d8ba18e746033d2c89e4f6c7

[7] Linux kernel support for Arm-CCA
https://gitlab.arm.com/linux-arm/linux-cca
Full stack branch: cca/cca-full/latest
SHA: 78b23c56de79838e09b5594b5281c93562e0f976

[8] kvmtool support for Arm CCA
https://gitlab.arm.com/linux-arm/kvmtool-cca
Branch: cca/latest
SHA: 0310eb6aa59f48786f38bfb12f2fa537ef33a890

[9] kvm-unit-tests support for Arm CCA
https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca
Branch: cca/v2

[10] Instructions for Building Firmware components and running the model, see
section 4.19.2 "Building and running TF-A with RME"
https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme

[11] RFC series posted previously for adding support for Arm CCA guest firmware:
v3: tianocore/edk2#6480
v2: https://edk2.groups.io/g/devel/message/117716
v1: https://edk2.groups.io/g/devel/message/103581

[12] UEFI Firmware support for Arm CCA
Host & Guest Support:
- Repo:
edk2: https://gitlab.arm.com/linux-arm/edk2-cca
edk2-platforms: https://gitlab.arm.com/linux-arm/edk2-platforms-cca
- Branch: 3223_arm_cca_v4

---

• Breaking change?
  No
• Impacts security?
  No
• Includes tests?
  See ‘Running the stack’ section above.

How This Was Tested

See ‘Running the stack’ section above.

Integration Instructions

N/A

[samimujawar]

I have marked in all the review feedback from the v3 series PR i.e. tianocore/edk2#6480

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

tianocore/edk2#6480 (comment)

[samimujawar]

tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

Ref: tianocore/edk2#6480 (comment)

[samimujawar]

I have addressed all the feedback except tianocore/edk2#6480 (comment)
which I think will need some alignment.