Merge branch 'cvw' of https://github.com/riscv-non-isa/riscv-arch-test into cvw

Author: Marina-Bellido

ctp/src/coverage.adoc

@@ -36,27 +36,34 @@ As compared to the base RVVI, Extended RVVI adds the following signals:
 |virt_adr_d|XLEN|Virtual address of data accessed by instruction
 |phys_adr_i|PA_BITS|Physical address of instruction
 |phys_adr_d|PA_BITS|Physical address of data accessed by instruction
-|pte_i|XLEN|Instruction page table entry
-|pte_d|XLEN|Data page table entry
-|ppn_i|PPN_BITS|Instruction physical page number
-|ppn_d|PPN_BITS|Data physical page number
-|page_type_i|2|Instruction page type ***explain
-|page_type_d|2|Data page type
-|read_access|1|Instruction reads data (loads, AMOs, ***CMO)
-|write_access|1|Instruction writes data (stores, AMO, ***cbo.zero)
-|execute_access|1|***define - is this jumps?  when asserted, and does it only affect data accesses?
+|g_phys_adr_i|PA_BITS|Guest physical address of instruction (when Hypervisor is active, 0 when inactive)
+|g_phys_adr_d|PA_BITS|Guest physical address of data accessed by instruction (when Hypervisor is active, 0 when inactive)
+|pte_i|XLEN|Instruction page table entry (VS-stage when Hypervisor is active)
+|pte_d|XLEN|Data page table entry (VS-stage when Hypervisor is active)
+|g_pte_i|XLEN|Instruction page table entry (G-stage when Hypervisor is active, 0 when inactive)
+|g_pte_d|XLEN|Data page table entry (G-stage when Hypervisor is active, 0 when inactive)
+|page_type_i|3|Instruction page type (VS-stage when hypervisor is active))
+|page_type_d|3|Data page type (VS-stage when hypervisor is active))
+|g_page_type_i|3|Instruction page type (G-stage when hypervisor is inactive, 0 when inactive)
+|g_page_type_d|3|Data page type (G-stage when hypervisor is inactive, 0 when inactive)
 |===
 
-*** PA_BITS
-PPN_BITS = PA_BITS - 12
-*** why do we need ppn when we have phys_addr?  Why not derive locally?
-
-*** do we need to add guest physical address?  Such as
-
-* gpa_adr_i
-* gpa_adr_d
+[[t-pagetypes]]
+.Page Types
+[options=header]
+[cols="1, 4" options=header]
+[%AUTOWIDTH]
+|===
+|Code|Type
+|000|kilopage
+|001|megapage
+|010|gigapage
+|011|terapage
+|100|petapage
+|other|reserved
+|===
 
-maybe guest page table entries?
+Some accesses touch more than one physical page.  Examples include misaligned accesses wrapping a page boundary, and some vector accesses.  Extended RVVI provideds the signals corresponding to the base address of the access. For vector, they correspond to the base address of the initial element access.
 
 === riscvISACOV Signals
 

ctp/src/ctp.adoc

@@ -62,6 +62,7 @@ include::intro.adoc[]
 include::cert.adoc[]
 include::profiles.adoc[]
 include::strategy.adoc[]
+include::req.adoc[]
 include::unpriv.adoc[]
 include::priv.adoc[]
 include::nonisa.adoc[]

ctp/src/exceptions.adoc

@@ -47,17 +47,41 @@ These additional exception tests are run on configurations with floating-point (
 ,===
 https://docs.google.com/spreadsheets/d/1W95I4jPbuQBnXzDdIZtOG4kae8vi7djpKtJXWCXPxMU/edit?gid=1241522932#gid=1241522932[Google Sheet Testplan]
 
-==== ExceptionsV Vector Exceptions
+==== ExceptionsVx Vector Integer Exceptions
 
-These additional exception tests are run on configurations with vector (V extension or subsets thereof that rely on mstatus.VS).
+These additional exception tests are run on configurations with vector (V extension or subsets thereof that rely on mstatus.VS) related to vector integer instructions.
 
-[[t-ExceptionsV-coverpoints]]
-.ExceptionsV Coverpoints
+[[t-ExceptionsVx-coverpoints]]
+.ExceptionsVx Coverpoints
 ,===
-//include::{testplansdir}/ExceptionsV.adoc[]
+//include::{testplansdir}/ExceptionsVx.adoc[]
 ,===
 
-*** add link
+https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=1139310135#gid=1139310135[Google Sheet Testplan]
+
+[[t-ExceptionsVf-coverpoints]]
+.ExceptionsVf Coverpoints
+
+These additional exception tests are run on configurations with vector (V extension or subsets thereof that rely on mstatus.VS) related to vector floating-point instructions.
+
+
+,===
+//include::{testplansdir}/ExceptionsVf.adoc[]
+,===
+
+https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=1839120623#gid=1839120623[Google Sheet Testplan]
+
+
+[[t-ExceptionsVls-coverpoints]]
+.ExceptionsVls Coverpoints
+
+These additional exception tests are run on configurations with vector (V extension or subsets thereof that rely on mstatus.VS) related to vector load/store instructions.
+
+,===
+//include::{testplansdir}/ExceptionsVls.adoc[]
+,===
+
+https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=1814229778#gid=1814229778[Google Sheet Testplan]
 
 ==== ExceptionsZalrsc Atomic LR/SC Exceptions
 

ctp/src/intro.adoc

@@ -3,6 +3,20 @@
 
 This Certification Test Plan describes the coverpoints and tests to certify RISC-V profiles.  It summarizes the profiles intended to be supported and the test suites that need to run on each profile.  It also summarizes the contents of each test suite.
 
+=== Certificates and Certification
+
+A RISC-V _Certificate_ is a named set of requirements for a RISC-V implementation, typically defined by a ratified profile and a set of additional requirements. For example, the RVA23 Certificate requires meeting the RVA23S64 profile and having a conforming machine mode implementation.***
+
+_Certification_ is the process of testing that a RISC-V implementation meets the requirements of a RISC-V certificate.  The goal of certification is to deliver software interoperability across implementations.
+
+There is a fuzzy line between certification and verification.  Certification involves testing that an implementation meets all of the normative rules in ratified specifications relevant to the certificate, but does not involve testing that requires detailed knowledge of the microarchitecture. Specifically, certification does not check timing (such as the arrival of interrupts relative to the start, middle, or end of multicycle instructions), pipeline hazards, power management features, etc.
+
+Certification assumes the ratified specification is read and interpreted with knowledge of computer architecture and reasonable judgement, because not every detail is spelled out in normative rules.  For example, certification that "ADD performs the addition of _rs1_ and _rs2_" involves testing ADD with a reasonable set of values.  The test plan author exercises judgement selecting these values because they are not spelled out.  For example, the values might involve every combination of {0, 1, -1, maximum positive integer, maximum negative integer, intermediate value} for the two source register values. The intermediate value involves an arbitrary choice.  A reasonable author might pick some other sensible edge values, such as some values one more or less than those in the set above.  However, certification does not attempt to prove every bit of a carry chain works for every possible input, or that all stuck-at faults would be detected. Similarly, the test plan author may exercise all 32 registers as sources or destinations for each instruction, which is implicitly required but not spelled out in a normative rule.
+
+When in doubt, it is better to test slightly more than strictly necessary for normative rules, rather than slightly less. Over-testing is acceptable, as long as it does not cause false failures or excessive test time. It is often easier to test cross-products of all relevant inputs, even if not all combinations are explicitly named by normative rules. It also reduces the risk of human error missing a required combination.
+
+An implementation that passes certification tests is said to be certified for the relevant certificate. The process of certification is described in ***.
+
 === Coverpoints and Tests
 
 Coverpoints are the key to certification.  They are the features of the RISC-V architecture that need to be tested in order to certify a profile.  Each coverpoint has a set of tests that exercise it, and each test has a set of coverpoints that it hits.

ctp/src/profiles.adoc

@@ -2,17 +2,19 @@
 
 == Profiles & Test Suites
 
-This test plan addresses Phase 0, Phase 1, and Phase 2 RISC-V certification objectives, including the following ratified and unratified profiles:
+This test plan addresses Phase 0, Phase 1, and Phase 2 RISC-V certification objectives, including the following ratified and unratified profiles.
 
 * https://drive.google.com/file/d/1Kg7Ner5ZlxFDclf92-9Tz88JvmZWt5Wb/view[RVI20 Profile] (Ratified): Phase 0
 ** RV{32/64}IMAFDC_Zifencei_Zicntr_Zihpm with machine mode, no PMP
-* Microcontroller Profile (Not Ratified, but market demand): Phase 1
-** RV{32/64}IMZca_Zcb_Zifencei_Zicsr_Zicntr with machine mode, user mode, interrupts, PMP16 footnote:[Tentative; may need updating based on the MRD Market Requirements Document]
+* MC100 Microcontroller Profile (https://github.com/riscv-software-src/riscv-unified-db/releases/tag/v0.1.0[MC100-32-CRD.pdf]): Phase 1
+** RV{32/64}IMZca_Zcb_Zifencei_Zicsr_Zicntr with machine mode, user mode, interrupts, PMP16 footnote:[Tentative; may need updating based on the MRD Market Requirements Document] ***update
 * https://drive.google.com/file/d/1Kg7Ner5ZlxFDclf92-9Tz88JvmZWt5Wb/view[RVA22S64 Profile] (Ratified): Phase 1.5, stepping stone to phase 2
 ** Application processor without vector, hypervisor, and other newer extensions
 * https://github.com/riscv/riscv-profiles/blob/main/src/rvb23-profile.adoc#rvb23s64-profile[RVB23S64 Profile] (Ratified): Phase 1.8, stepping stone to phase 2
 ** RVA23 less vector and hypervisor
-* https://github.com/riscv/riscv-profiles/blob/main/src/rva23-profile.adoc#rva23s64-profile[RVA23S64 Profile] (Ratified): Phase 2
+* RVA23 Profile (https://riscv-software-src.github.io/riscv-unified-db/pdfs/AC200-CRD.pdf[AC200 CRD]): Phase 2
+
+Where a CRD is available, it lists the mandatory and optional extensions and IN/OUT-OF_SCOPE parameters.
 
 === Profile Coverage Matrix
 
@@ -28,7 +30,7 @@ All privileged suites require at least version 1.12 of the privileged specificat
 [cols="1, 4, 1, 1, 1, 1, 1, 1" options=header]
 [%AUTOWIDTH]
 |===
-|Coverage File|Description|Test Plan|RVI20|uController|RVA22S64|RVB23S64|RVA23S64
+|Coverage File|Description|Test Plan|RVI20|MC100|RVA22S64|RVB23S64|RVA23S64
 |XLEN|||32/64|32|64|64|64
 8+^|Unprivileged
 |I|Integer|<<t-I-coverpoints>>|x|x|x|x|x
@@ -123,6 +125,7 @@ All privileged suites require at least version 1.12 of the privileged specificat
 |VMPMPZicbo|VM + PMP + CBOM/CBOZ|<<t-VMPMPZicbo-coverpoints>>|||x|x|x
 |Svade|Page Table A/D Exceptions|<<t-Svade-coverpoints>>|||x|x|x
 |Svadu|Page Table Update|<<t-Svadu-coverpoints>>||||o|o
+|SvaduPMP|Page Table Update + PMP|<<t-SvaduPMP-coverpoints>>||||o|o
 |Svpbmt|Page-based Memory Types|<<t-Svpbmt-coverpoints>>|||x|x|x
 |Svnapot|Naturally Aligned Pages|<<t-Svnapot-coverpoints>>|||o|x|x
 |Sscofpmf|Counter Filtering|<<t-Sscofpmf-coverpoints>>|||o|x|x
@@ -165,8 +168,11 @@ All privileged suites require at least version 1.12 of the privileged specificat
 |Zvknha32|Vector Hash 32-bit|<<t-Zvknh-coverpoints>>|||||
 |Zvknhb64|Vector Hash 64-bit|<<t-Zvknh-coverpoints>>||||o|o
 8+^|Vector Privileged
-|ExceptionsV|Vector Exceptions|<<t-ExceptionsV-coverpoints>>|||o|o|x
+|ExceptionsVx|Vector Integer Exceptions|<<t-ExceptionsVx-coverpoints>>|||o|o|x
+|ExceptionsVf|Vector Float Exceptions|<<t-ExceptionsVf-coverpoints>>|||o|o|x
+|ExceptionsVls|Vector Load/Store Exceptions|<<t-ExceptionsVls-coverpoints>>|||o|o|x
 |ZicsrV|Vector CSRs|<<t-ZicsrV-coverpoints>>|||o|o|x
+|ZicsrVF|Vector FP CSRs|<<t-ZicsrVF-coverpoints>>|||o|o|x
 |ZicsrUV|User Vector CSRs|<<t-ZicsrUV-coverpoints>>|||o|o|x
 |SsstrictV|Vector Strict|<<t-SsstrictV-coverpoints>>|||o|o|o
 8+^|Hypervisor

ctp/src/req.adoc

@@ -0,0 +1,10 @@
+== Test Suite Requirements
+
+The test suites may be provided by multiple suppliers, but must meet a common set of requirements to interoperate transparently for the user.
+
+The following requirements appear to be non-controversial.
+
+*
+The following requirements need analysis
+
+*** Update profile for MC100, remove U requirements

ctp/src/ssstrict.adoc

@@ -44,4 +44,4 @@ Test that all behaviors called out as "reserved" in the vector specification cau
 //include::{testplansdir}/SsstrictV.adoc[]
 ,===
 
-*** need link
+https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=743730639#gid=743730639[Google Sheet Testplan]

ctp/src/strategy.adoc

@@ -175,7 +175,7 @@ _rs2_.#[#norm:inst:sub:operation]#SUB performs the subtraction of _rs2_ from _rs
 
 === Coverpoints
 
-Coverpoints are written with one file that covers both RV32 and RV64, to reduce the duplication and risk of becoming out of sync.  When a coverage file contains coverpoints that apply only to one XLEN or the other (e.g. 32 or 64-bit edge values), they are separated by ``ifdef XLEN32` or `XLEN64` directives. Similarly, when a coverpoint applies only to a certain parameter value (PMP NA4 regions are not supported for granularity coarser than 4 bytes), they uses ``ifdef`` to exclude tests based on the parameter value.
+Coverpoints are written with one file that covers both RV32 and RV64, to reduce the duplication and risk of becoming out of sync.  When a coverage file contains coverpoints that apply only to one XLEN or the other (e.g. 32 or 64-bit edge values), they are separated by ```ifdef XLEN32`` or ``XLEN64`` directives. Similarly, when a coverpoint applies only to a certain parameter value (PMP NA4 regions are not supported for granularity coarser than 4 bytes), they uses ```ifdef`` to exclude tests based on the parameter value.
 
 === Tests
 

ctp/src/trickbox.adoc

@@ -6,8 +6,8 @@ Each DUT requires a trick box to perform the following non-ISA functions:
 * Run DUT-specific boot code
 * Deliver each supported type of interrupt to the core (Machine/Supervisor External/Timer/Software + guest external interrupts)
 * Send a character to a terminal to log a success message or test failure information
-* Terminate a test
-* If a debug module is supported, send commands to the DM
+* Terminate a test with a pass or fail indication
+* If a debug module is supported, send abstract commands to the DM and receive responses
 
 === Trick Box Macros
 
@@ -21,10 +21,11 @@ The trick box is implemented with DUT-specific macros in given in <<t-trickbox>>
 |===
 |Macro|Description
 |RVMODEL_BOOT|Perform boot operations, such as turning on a phase-locked loop or DRAM controller
-|RVMODEL_HALT|Terminate test.  When the test is run in simulation, this should end the simulation.
+|RVMODEL_HALT_PASS|Terminate test with a pass indication.  When the test is run in simulation, this should end the simulation.
+|RVMODEL_HALT_FAIL|Terminate test with a fail indication.  When the test is run in simulation, this should end the simulation.
 |RVMODEL_IO_INIT|Initialization steps needed prior to writing to the console
-|RVMODEL_IO_WRITE_STR(_R, _STR)|Write a null-terminated ASCII string to the console, where _R *** and _STR ***
-|RVMODE_SET_MEXT_INT| Sets the Machine External Interrupt (mip.MEIP = 1, if supported)
+|RVMODEL_IO_WRITE_STR(_R, _STR)|Write a null-terminated ASCII string to the console, where _R is a temporary register that may be trashed by the macro and _STR is a register containing the address of the first byte of the string
+|RVMODEL_SET_MEXT_INT| Sets the Machine External Interrupt (mip.MEIP = 1, if supported)
 |RVMODEL_CLR_MEXT_INT|Clears the Machine External Interrupt (mip.MEIP = 0)
 |RVMODEL_SET_MTIMER_INT|Sets the Machine Timer Interrupt (mip.MTIP = 1, if supported)
 |RVMODEL_CLR_MTIMER_INT|Clears the Machine Timer Interrupt (mip.MTIP = 0)
@@ -34,18 +35,17 @@ The trick box is implemented with DUT-specific macros in given in <<t-trickbox>>
 |RVMODEL_SET_SEXT_INT|Sets the Supervisor External Interrupt (sip.SEIP = 1, if supported)
 |RVMODEL_CLR_SEXT_INT|Clears the Supervisor External Interrupt (sip.SEIP = 0)
 |RVMODEL_SET_STIMER_INT|Sets the Supervisor Timer Interrupt (sip.STIP = 1, if supported)
-|RVMODEL_CLR_STIMER_INT|Clears the Supervisor Timer Interrupt (sip.STIP = 0)\
+|RVMODEL_CLR_STIMER_INT|Clears the Supervisor Timer Interrupt (sip.STIP = 0)
 |RVMODEL_SET_STIMER_INT_SOON|Cause the Supervisor Timer Interrupt to rise soon, but not immediately.
 |RVMODEL_SET_SSW_INT|Sets the Supervisor Software Interrupt (mip.SSIP = 1, if supported)
 |RVMODEL_CLR_SSW_INT|Clears the Supervisor Software Interrupt (mip.SSIP = 0)
 |RVMODEL_WRITE_GEIP(_R)|Write the value in _R to the Guext External Interrupt Pending (hgeip) register, if supported.  This is used to test guest external interrupts. Only the bottom GEILEN bits are written.
 |ACCESS_FAULT_ADDRESS|An address that causes an access fault when read or written. This is used to test exceptions.
 |===
 
-*** way to send success/fail code when terminating sim
-*** is STIMER_INT_SOON necessary?
+*** give a parameter defining delay for RVMODEL_SET_MTIMER_INT_SOON
 
-The macros are defined in ***. *** shows sample implementations of these macros compatible with Spike.
+The macros are defined in model_test.h. sail_test.h shows sample implementations of these macros compatible with Sail and Spike***.
 
 
 === Linker Script

ctp/src/v.adoc

@@ -179,8 +179,7 @@ As with other unprivileged testplans, an x in the spreadsheet indicates to use t
 * ls_e{8/16/32/64}: load/store edges with EEW=8,...,64.  Used to convert strided loads from elements to bytes.
 * ls: load/store edges instead of integer edges: vs2={0, random < 2*VLMAX}.  rs2= {1, 2, -1, -2, 0}
 * f: floating-point edges instead of integer edges: ***
-
-*** discuss special cases
+* cp_custom_*: custom coverpoint defined in the coverage file.  See definitions for https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=389766666#gid=389766666[CustomVx], https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=704208158#gid=704208158[CustomVf], https://docs.google.com/spreadsheets/d/11VGlIA__UYr7H47ZNqXtyYnGH5Fb41E8P9h53rN72ok/edit?gid=704208158#gid=704208158[CustomVls].
 
 The coverpoints for each vector instruction are given in the following sections.