[build] Make package metadata optional #21081

pjcollins · 2024-08-20T17:02:26Z

Looking at the package explorer for recent .NET 9 preview 7 builds I
noticed that other packages do not include sha metadata in the version
element. Compare the following by clicking "View Metadata source":

https://nuget.info/packages/Microsoft.iOS.Ref.net9.0_17.5/17.5.9231-net9-p7
https://nuget.info/packages/Microsoft.NETCore.App.Runtime.Mono.ios-arm64/9.0.0-preview.7.24405.7
https://nuget.info/packages/Microsoft.Maui.Sdk/9.0.0-preview.7.24407.4

The sha metadata inclusion in the version element but not the .nupkg
name also causes conflicts with the latest version of the
arcade publishing tooling which expects these to be in sync.

While waiting for a fix in arcade, we can remove this metadata to
restore dependency flow. It can be restored in the future by setting
the following:

NUGET_BUILD_METADATA=+sha.$(CURRENT_HASH)

The icon and other package metadata have been updated to better match
other .NET packages.

Looking at the package explorer for recent .NET 9 preview 7 builds I noticed that other packages do not include sha metadata in the version element. Compare the following by clicking "View Metadata source": https://nuget.info/packages/Microsoft.iOS.Ref.net9.0_17.5/17.5.9231-net9-p7 https://nuget.info/packages/Microsoft.NETCore.App.Runtime.Mono.ios-arm64/9.0.0-preview.7.24405.7 https://nuget.info/packages/Microsoft.Maui.Sdk/9.0.0-preview.7.24407.4 I believe the sha metadata was added when we were initially figuring out a package versioning schema that would produce unique package versions, however our usage of commit distance in both preview and stable versions should address this concern. The sha metadata inclusion in the version element but not the .nupkg name also causes conflicts with the latest version of the [arcade publishing tooling][0] which expects these to be in sync. The short hash in the metadata can be be replaced by the `repository` element which contains the repo url, branch, and full commit hash. The icon and other package metadata have been updated to better match other .NET packages. [0]: https://github.com/dotnet/arcade/blob/b4f4d40741f161e2c0d96c19c51a4013850ef65f/src/Microsoft.DotNet.Build.Tasks.Feed/src/PushToBuildStorage.cs

rolfbjarne

I believe the sha metadata was added when we were initially figuring out
a package versioning schema that would produce unique package versions,
however our usage of commit distance in both preview and stable versions
should address this concern.

I added it to make it very easy to figure out which commit produced any given package (as stated above the semver spec is very clear that metadata should be ignored, so uniqueness didn't factor into this decision).

The sha metadata inclusion in the version element but not the .nupkg
name also causes conflicts with the latest version of the
arcade publishing tooling which expects these to be in sync.

I believe this to be a bug in arcade. The semver spec doesn't explicitly say that metadata should be ignored when comparing versions, but it's very clear that metadata should be ignored when comparing version precedence: "Build metadata MUST be ignored when determining version precedence."

I believe arcade should ignore metadata when comparing versions - so is there any way this could be fixed in arcade instead?

dotnet/VERSIONS.md

@@ -4,7 +4,7 @@

 Our NuGet packages are versioned using [Semver 2.0.0][2].

-This is the scheme: `OsMajor.OsMinor.InternalRelease[-prereleaseX]+sha.1b2c3d4`.
+This is the scheme: `OsMajor.OsMinor.InternalRelease[-prereleaseX].CommitDistance`.


pjcollins · 2024-08-21T17:45:56Z

I believe the sha metadata was added when we were initially figuring out
a package versioning schema that would produce unique package versions,
however our usage of commit distance in both preview and stable versions
should address this concern.

I added it to make it very easy to figure out which commit produced any given package (as stated above the semver spec is very clear that metadata should be ignored, so uniqueness didn't factor into this decision).

The sha metadata inclusion in the version element but not the .nupkg
name also causes conflicts with the latest version of the
arcade publishing tooling which expects these to be in sync.

I believe this to be a bug in arcade. The semver spec doesn't explicitly say that metadata should be ignored when comparing versions, but it's very clear that metadata should be ignored when comparing version precedence: "Build metadata MUST be ignored when determining version precedence."

I believe arcade should ignore metadata when comparing versions - so is there any way this could be fixed in arcade instead?

Thanks for the extra context here. I started a thread to discuss potential changes here in the arcade side, but I am not sure how quickly that will move. We may still want to try to do something on our end to unblock our dependency flow.

dotnet/VERSIONS.md

rolfbjarne · 2024-08-22T08:59:34Z

I am not sure how quickly that will move.

I created a PR to hopefully make this happen: dotnet/arcade#15023

Co-authored-by: Rolf Bjarne Kvinge <[email protected]>

pjcollins · 2024-08-22T13:08:54Z

Thanks for the pending arcade fix! I've updated the PR and description to capture the feedback and note that the metadata is now optional. We can merge this to remove it for now, and restore it once we have an update to arcade if that sounds like a reasonable plan. Note that the + portion of the metadata string will need to be moved to the variable declaration when restored, which should hopefully not be problematic.

vs-mobiletools-engineering-service2 · 2024-08-22T15:41:10Z

✅ API diff for current PR / commit

Legacy Xamarin (No breaking changes)

~~iOS~~ (no change detected)
~~tvOS~~ (no change detected)
~~watchOS~~ (no change detected)
~~macOS~~ (no change detected)

NET (empty diffs)

~~iOS~~: (empty diff detected)
~~tvOS~~: (empty diff detected)
~~MacCatalyst~~: (empty diff detected)
~~macOS~~: (empty diff detected)

✅ API diff vs stable

Legacy Xamarin (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
watchOS: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)

.NET (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
MacCatalyst: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)
Microsoft.iOS vs Microsoft.MacCatalyst: vsdrops gist

Legacy Xamarin (stable) vs .NET

iOS: vsdrops gist
tvOS: vsdrops gist
macOS: vsdrops gist

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-22T15:44:53Z

📚 [CI Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2024-08-22T15:46:50Z

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

✅ All tests on macOS M1 - Mac Ventura (13) passed.

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-22T15:50:46Z

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

✅ All tests on macOS X64 - Mac Sonoma (14) passed.

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-22T15:52:08Z

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

✅ All tests on macOS M1 - Mac Monterey (12) passed.

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-22T15:59:40Z

💻 [CI Build] Tests on macOS M1 - Mac Big Sur (11) passed 💻

✅ All tests on macOS M1 - Mac Big Sur (11) passed.

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-22T17:28:08Z

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: test results.

🎉 All 168 tests passed 🎉

Tests counts

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 6 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 7 tests passed. Html Report (VSDrops) Download
✅ generator: All 2 tests passed. Html Report (VSDrops) Download
✅ install-source: All 1 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 7 tests passed. Html Report (VSDrops) Download
✅ introspection: All 8 tests passed. Html Report (VSDrops) Download
✅ linker: All 65 tests passed. Html Report (VSDrops) Download
✅ mac-binding-project: All 1 tests passed. Html Report (VSDrops) Download
✅ mmp: All 2 tests passed. Html Report (VSDrops) Download
✅ mononative: All 6 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 11 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 7 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 8 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 11 tests passed. Html Report (VSDrops) Download
✅ monotouch (watchOS): All 4 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ mtouch: All 1 tests passed. Html Report (VSDrops) Download
✅ xammac: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 8 tests passed. Html Report (VSDrops) Download
✅ xtro: All 2 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-23T11:17:49Z

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-23T13:51:23Z

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

vs-mobiletools-engineering-service2 · 2024-08-23T14:54:13Z

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: 2851cf71b5ccbdd5a262929cf19b7494f88b2fd3 [PR build]

dalexsoto · 2024-08-23T14:55:20Z

/sudo backport release/8.0.1xx-xcode15.4

vs-mobiletools-engineering-service2 · 2024-08-23T14:56:27Z

Backport Job to branch release/8.0.1xx-xcode15.4 Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2024-08-23T14:58:55Z

Hooray! Backport succeeded! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10097850 for more details.

pjcollins · 2024-08-23T16:21:37Z

/sudo backport release/9.0.1xx-rc1

vs-mobiletools-engineering-service2 · 2024-08-23T16:22:31Z

Backport Job to branch release/9.0.1xx-rc1 Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2024-08-23T16:26:08Z

Hooray! Backport succeeded! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10098330 for more details.

Looking at the package explorer for recent .NET 9 preview 7 builds I noticed that other packages do not include sha metadata in the version element. Compare the following by clicking "View Metadata source": https://nuget.info/packages/Microsoft.iOS.Ref.net9.0_17.5/17.5.9231-net9-p7 https://nuget.info/packages/Microsoft.NETCore.App.Runtime.Mono.ios-arm64/9.0.0-preview.7.24405.7 https://nuget.info/packages/Microsoft.Maui.Sdk/9.0.0-preview.7.24407.4 The sha metadata inclusion in the version element but not the .nupkg name also causes conflicts with the latest version of the [arcade publishing tooling][0] which expects these to be in sync. While waiting for a fix in arcade, we can remove this metadata to restore dependency flow. It can be restored in the future by setting the following: NUGET_BUILD_METADATA=+sha.$(CURRENT_HASH) The icon and other package metadata have been updated to better match other .NET packages. [0]: https://github.com/dotnet/arcade/blob/b4f4d40741f161e2c0d96c19c51a4013850ef65f/src/Microsoft.DotNet.Build.Tasks.Feed/src/PushToBuildStorage.cs Backport of #21081 --------- Co-authored-by: Peter Collins <[email protected]> Co-authored-by: Rolf Bjarne Kvinge <[email protected]>

…21105) Looking at the package explorer for recent .NET 9 preview 7 builds I noticed that other packages do not include sha metadata in the version element. Compare the following by clicking "View Metadata source": https://nuget.info/packages/Microsoft.iOS.Ref.net9.0_17.5/17.5.9231-net9-p7 https://nuget.info/packages/Microsoft.NETCore.App.Runtime.Mono.ios-arm64/9.0.0-preview.7.24405.7 https://nuget.info/packages/Microsoft.Maui.Sdk/9.0.0-preview.7.24407.4 The sha metadata inclusion in the version element but not the .nupkg name also causes conflicts with the latest version of the [arcade publishing tooling][0] which expects these to be in sync. While waiting for a fix in arcade, we can remove this metadata to restore dependency flow. It can be restored in the future by setting the following: NUGET_BUILD_METADATA=+sha.$(CURRENT_HASH) The icon and other package metadata have been updated to better match other .NET packages. [0]: https://github.com/dotnet/arcade/blob/b4f4d40741f161e2c0d96c19c51a4013850ef65f/src/Microsoft.DotNet.Build.Tasks.Feed/src/PushToBuildStorage.cs Backport of #21081 --------- Co-authored-by: Peter Collins <[email protected]> Co-authored-by: Rolf Bjarne Kvinge <[email protected]>

This comment has been minimized.

Sign in to view

Allow $(NUGET_BUILD_METADATA) to be empty

38deb04

This comment has been minimized.

Sign in to view

Update icon path

5093ac5

This comment has been minimized.

Sign in to view

pjcollins marked this pull request as ready for review August 20, 2024 23:39

pjcollins requested a review from rolfbjarne as a code owner August 20, 2024 23:39

This comment has been minimized.

Sign in to view

rolfbjarne requested changes Aug 21, 2024

View reviewed changes

rolfbjarne requested changes Aug 22, 2024

View reviewed changes

dotnet/VERSIONS.md Outdated Show resolved Hide resolved

dotnet/VERSIONS.md Show resolved Hide resolved

dotnet/VERSIONS.md Show resolved Hide resolved

pjcollins and others added 3 commits August 22, 2024 09:01

Update dotnet/VERSIONS.md

fe1ed12

Co-authored-by: Rolf Bjarne Kvinge <[email protected]>

Feedback

2dcc82c

Merge remote-tracking branch 'origin/main' into dev/pjc/pack-repo-meta

2851cf7

pjcollins requested a review from rolfbjarne August 22, 2024 13:11

pjcollins changed the title ~~[build] Update package metadata~~ [build] Make package metadata optional Aug 22, 2024

rolfbjarne approved these changes Aug 22, 2024

View reviewed changes

dalexsoto approved these changes Aug 23, 2024

View reviewed changes

vs-mobiletools-engineering-service2 mentioned this pull request Aug 23, 2024

[release/8.0.1xx-xcode15.4] [build] Make package metadata optional #21105

Merged

pjcollins merged commit 92092f6 into main Aug 23, 2024
29 checks passed

pjcollins deleted the dev/pjc/pack-repo-meta branch August 23, 2024 16:19

vs-mobiletools-engineering-service2 mentioned this pull request Aug 23, 2024

[release/9.0.1xx-rc1] [build] Make package metadata optional #21109

Merged

[build] Make package metadata optional #21081

[build] Make package metadata optional #21081

Uh oh!

Conversation

pjcollins commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rolfbjarne left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

pjcollins commented Aug 21, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rolfbjarne commented Aug 22, 2024

Uh oh!

pjcollins commented Aug 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

✅ API diff for current PR / commit

✅ API diff vs stable

ℹ️ Generator diff

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

📚 [CI Build] Artifacts 📚

Packages generated

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

💻 [CI Build] Tests on macOS M1 - Mac Big Sur (11) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 22, 2024

🚀 [CI Build] Test results 🚀

Test results

Tests counts

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 23, 2024

❌ [CI Build] Windows Integration Tests failed ❌

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 23, 2024

❌ [CI Build] Windows Integration Tests failed ❌

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 23, 2024

❌ [CI Build] Windows Integration Tests failed ❌

Uh oh!

dalexsoto commented Aug 23, 2024

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 23, 2024

Uh oh!

vs-mobiletools-engineering-service2 commented Aug 23, 2024

Uh oh!

Uh oh!

pjcollins commented Aug 23, 2024

Uh oh!

pjcollins commented Aug 20, 2024 •

edited

Loading

pjcollins commented Aug 22, 2024 •

edited

Loading