Skip to content

Add dts profile diff #999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 75 commits into
base: develop
Choose a base branch
from
Open

Add dts profile diff #999

wants to merge 75 commits into from

Conversation

m-iwanicki
Copy link
Contributor

@m-iwanicki m-iwanicki commented Aug 11, 2025

Related to Dasharo/dts-scripts#97 (needs those changes to work).
UEFI Update profile will need to be generated again because of Dasharo/dts-scripts@171d8de

This PR adds basic support for profile checking along with implementing those checks for ODROID H4 workflows.

Comparing profiles is done in TRY block:

in test teardown only if the profile for the test exists.
Maybe adding separate variable in platform config to decide whether profile has to be checked for that workflow would be good idea just in case file or test is renamed. In the current state test would succeed, because it would skip profile checking if it couldn't find reference file.

@m-iwanicki m-iwanicki requested a review from DaniilKl August 11, 2025 16:20
@m-iwanicki m-iwanicki self-assigned this Aug 11, 2025
Signed-off-by: Michał Iwanicki <[email protected]>
It's done so generated profiles are identical to reference ones

Signed-off-by: Michał Iwanicki <[email protected]>
Used by 'Get File From DUT' keyword

Signed-off-by: Michał Iwanicki <[email protected]>
@m-iwanicki m-iwanicki marked this pull request as ready for review August 12, 2025 16:31
m-iwanicki and others added 6 commits August 21, 2025 15:27
@m-iwanicki
Copy link
Contributor Author

I cherry-picked commit from this PR: #647 to fix Dasharo/dasharo-issues#1526

Copy link
Contributor

@DaniilKl DaniilKl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally add the variable that signals that the platform should have the profile.

m-iwanicki and others added 8 commits August 22, 2025 16:10
Signed-off-by: Daniil Klimuk <[email protected]>

dts: profile: optiplex-[79]010: add Initial deployment profiles

Signed-off-by: Daniil Klimuk <[email protected]>

dts: profiles: add OptiPlex UEFI Update profiles

Signed-off-by: Daniil Klimuk <[email protected]>

platform-configs: optiplex-common: fix pre-commit

Signed-off-by: Daniil Klimuk <[email protected]>

platform-configs: optiplex: delete DTS TEST_*_MIGRATED vars

Signed-off-by: Daniil Klimuk <[email protected]>

platform-configs: optiplex: delete TEST_IS_COREBOOT

Signed-off-by: Daniil Klimuk <[email protected]>
Either transition didn't work or couldn't be verified

Signed-off-by: Michał Iwanicki <[email protected]>
Signed-off-by: Michał Iwanicki <[email protected]>
@DaniilKl
Copy link
Contributor

Looks ok. Lets check the tests results.

@m-iwanicki
Copy link
Contributor Author

m-iwanicki commented Aug 22, 2025

@DaniilKl partial results:

Needs to be fixed:

  • odroid - enable profile verification, fixed: 436831c
  • msi and novacustom heads transition - heads transition needs to use
    Wait For ME Warning Or Reboot
    or similar to deal with unflashable ME
  • msi uefi update - most likely some platform-config tweaking.
==============================================================================
Dts-E2E
==============================================================================
Create tests                                                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E001: novacustom-ns50mu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E002: novacustom-ns50mu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E003: novacustom-ns50pu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E004: novacustom-ns50pu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E005: novacustom-ns70mu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E006: novacustom-ns70mu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E007: novacustom-ns70pu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E008: novacustom-ns70pu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E009: novacustom-nv41mb Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E010: novacustom-nv41mb UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E011: novacustom-nv41mz Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E012: novacustom-nv41mz UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E013: novacustom-v540tnd Initial Deployment - DCR                   | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E014: novacustom-v540tnd UEFI Update - DCR                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E015: novacustom-v560tnd Initial Deployment - DCR                   | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E016: novacustom-v560tnd UEFI Update - DCR                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E017: novacustom-v560tne Initial Deployment - DCR                   | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E018: novacustom-v560tne UEFI Update - DCR                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E019: odroid-h4-plus Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E020: odroid-h4-plus UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E021: odroid-h4-plus Dasharo (coreboot+UEFI) to Dasharo (Slim Bo... | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E022: odroid-h4-plus Dasharo (Slim Bootloader+UEFI) Initial Depl... | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E023: pcengines-apu2 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E024: pcengines-apu2 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E025: pcengines-apu2 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E026: pcengines-apu3 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E027: pcengines-apu3 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E028: pcengines-apu3 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E029: pcengines-apu4 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E030: pcengines-apu4 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E031: pcengines-apu4 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E032: pcengines-apu6 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E033: pcengines-apu6 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E034: pcengines-apu6 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E035: msi-pro-z690-a-ddr5 Initial Deployment - DCR                  | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E036: msi-pro-z690-a-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E037: msi-pro-z690-a-ddr5 UEFI Update - DCR                         | PASS |
------------------------------------------------------------------------------
E2E038: msi-pro-z690-a-ddr5 UEFI Update - DPP                         | PASS |
------------------------------------------------------------------------------
E2E039: msi-pro-z690-a-ddr5 UEFI->Heads Transition - DPP              | FAIL |
Teardown failed:
Profiles are not identical!: 1 != 0
------------------------------------------------------------------------------
E2E040: msi-pro-z690-a-wifi-ddr4 Initial Deployment - DCR             | PASS |
------------------------------------------------------------------------------
E2E041: msi-pro-z690-a-wifi-ddr4 Initial Deployment - DPP             | PASS |
------------------------------------------------------------------------------
E2E042: msi-pro-z690-a-wifi-ddr4 UEFI Update - DCR                    | PASS |
------------------------------------------------------------------------------
E2E043: msi-pro-z690-a-wifi-ddr4 UEFI Update - DPP                    | PASS |
------------------------------------------------------------------------------
E2E044: msi-pro-z690-a-wifi-ddr4 UEFI->Heads Transition - DPP         | FAIL |
No match found for 'Successfully switched to Dasharo Heads firmware' in 5 minutes
Output:

Updating Dasharo firmware...
This may take several minutes. Please be patient and do not
power off your computer or touch the keyboard!
Backing up firmware configuration... Failed! Default settings will be used.
Done.
Can not determine if ME is disabled, assuming enabled.
Checking flash layout.
The firmware binary contains Management Engine (ME), but ME is not disabled!

Flashing ME when not in disabled state may cause unexpected power management issues.
Recovering from such state may require removal of AC power supply and resetting CMOS battery.
Keeping an older version of ME may cause a CPU to perform less efficient, e.g. if upgraded the CPU to a newer generation.
You have been warned.

Skip ME flashing and proceed with BIOS/firmware flashing/updating? (Y|n) .
------------------------------------------------------------------------------
E2E045: msi-pro-z790-p-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
E2E046: msi-pro-z790-p-ddr5 UEFI Update - DPP                         | FAIL |
Teardown failed:
Profiles are not identical!: 1 != 0
------------------------------------------------------------------------------
E2E047: msi-pro-z790-p-ddr5 UEFI->Heads Transition - DPP              | FAIL |
Teardown failed:
Profiles are not identical!: 1 != 0
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E048: novacustom-nv41pz Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E049: novacustom-nv41pz UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------

@m-iwanicki m-iwanicki force-pushed the add-dts-profile-diff branch from 107a205 to 9c51db6 Compare August 25, 2025 12:28
@m-iwanicki
Copy link
Contributor Author

@DaniilKl results:

61 tests, 57 passed, 4 failed

==============================================================================
Dts-E2E
==============================================================================
Create tests                                                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E001: novacustom-ns50mu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E002: novacustom-ns50mu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E003: novacustom-ns50pu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E004: novacustom-ns50pu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E005: novacustom-ns70mu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E006: novacustom-ns70mu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E007: novacustom-ns70pu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E008: novacustom-ns70pu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E009: novacustom-nv41mb Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E010: novacustom-nv41mb UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E011: novacustom-nv41mz Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E012: novacustom-nv41mz UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E013: novacustom-v540tnd Initial Deployment - DCR                   | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E014: novacustom-v540tnd UEFI Update - DCR                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E015: novacustom-v560tnd Initial Deployment - DCR                   | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E016: novacustom-v560tnd UEFI Update - DCR                          | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E017: novacustom-v560tne Initial Deployment - DCR                   | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E018: novacustom-v560tne UEFI Update - DCR                          | PASS |
------------------------------------------------------------------------------
E2E019: odroid-h4-plus Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E020: odroid-h4-plus UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E021: odroid-h4-plus Dasharo (coreboot+UEFI) to Dasharo (Slim Bo... | PASS |
------------------------------------------------------------------------------
E2E022: odroid-h4-plus Dasharo (Slim Bootloader+UEFI) Initial Depl... | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E023: pcengines-apu2 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E024: pcengines-apu2 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E025: pcengines-apu2 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E026: pcengines-apu3 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E027: pcengines-apu3 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E028: pcengines-apu3 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E029: pcengines-apu4 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E030: pcengines-apu4 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E031: pcengines-apu4 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E032: pcengines-apu6 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E033: pcengines-apu6 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E034: pcengines-apu6 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E035: msi-pro-z690-a-ddr5 Initial Deployment - DCR                  | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E036: msi-pro-z690-a-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E037: msi-pro-z690-a-ddr5 UEFI Update - DCR                         | PASS |
------------------------------------------------------------------------------
E2E038: msi-pro-z690-a-ddr5 UEFI Update - DPP                         | PASS |
------------------------------------------------------------------------------
E2E039: msi-pro-z690-a-ddr5 UEFI->Heads Transition - DPP              | PASS |
------------------------------------------------------------------------------
E2E040: msi-pro-z690-a-wifi-ddr4 Initial Deployment - DCR             | PASS |
------------------------------------------------------------------------------
E2E041: msi-pro-z690-a-wifi-ddr4 Initial Deployment - DPP             | PASS |
------------------------------------------------------------------------------
E2E042: msi-pro-z690-a-wifi-ddr4 UEFI Update - DCR                    | PASS |
------------------------------------------------------------------------------
E2E043: msi-pro-z690-a-wifi-ddr4 UEFI Update - DPP                    | PASS |
------------------------------------------------------------------------------
E2E044: msi-pro-z690-a-wifi-ddr4 UEFI->Heads Transition - DPP         | FAIL |
Teardown failed:
Profiles are not identical!: 1 != 0
------------------------------------------------------------------------------
E2E045: msi-pro-z790-p-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
E2E046: msi-pro-z790-p-ddr5 UEFI Update - DPP                         | PASS |
------------------------------------------------------------------------------
E2E047: msi-pro-z790-p-ddr5 UEFI->Heads Transition - DPP              | FAIL |
Teardown failed:
Profiles are not identical!: 1 != 0
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E048: novacustom-nv41pz Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
E2E049: novacustom-nv41pz UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
E2E050: novacustom-nv41pz UEFI->Heads Transition - DPP                | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E051: novacustom-v540tu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E052: novacustom-v540tu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
E2E053: novacustom-v540tu UEFI->Heads Transition - DPP                | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E054: novacustom-v560tu Initial Deployment - DCR                    | PASS |
------------------------------------------------------------------------------
[ WARN ] Workflow isn't configured for profile verification.
E2E055: novacustom-v560tu UEFI Update - DCR                           | PASS |
------------------------------------------------------------------------------
E2E056: novacustom-v560tu UEFI->Heads Transition - DPP                | PASS |
------------------------------------------------------------------------------
E2E057: optiplex-7010 Initial Deployment - DPP                        | FAIL |
Teardown failed:
Several failures occurred:

1) File '/home/miwanicki/projects/open-source-firmware-validation/dts/profiles/optiplex-7010 Initial Deployment - DPP.profile' does not exist.

2) Profiles are not identical!: 2 != 0
------------------------------------------------------------------------------
E2E058: optiplex-7010 UEFI Update - DPP                               | PASS |
------------------------------------------------------------------------------
E2E059: optiplex-9010 Initial Deployment - DPP                        | FAIL |
Teardown failed:
Several failures occurred:

1) File '/home/miwanicki/projects/open-source-firmware-validation/dts/profiles/optiplex-9010 Initial Deployment - DPP.profile' does not exist.

2) Profiles are not identical!: 2 != 0
------------------------------------------------------------------------------
E2E060: optiplex-9010 UEFI Update - DPP                               | PASS |
------------------------------------------------------------------------------
Dts-E2E                                                               | FAIL |
61 tests, 57 passed, 4 failed
==============================================================================

2 failures are expected (1 non-working heads transition on Z790 and 1 unverified heads transition).
2 other failures are due to missing profiles, or rather, wrong profile filename (it's Initial Deployment not Initial deployment), fixed here: 9a3edd8
Even after that profile for optiplex-7010 is different, fixed in 9c51db6.
I'll rerun tests one more time to make sure everything works and then squash most of commits in this PR:

------------------------------------------------------------------------------
E2E001: optiplex-7010 Initial Deployment - DPP                        | PASS |
------------------------------------------------------------------------------
E2E002: optiplex-9010 Initial Deployment - DPP                        | PASS |
------------------------------------------------------------------------------

@m-iwanicki
Copy link
Contributor Author

Every test passed except those 2 expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DTS extension failed to install after entering DPP keys
2 participants