Port System.Net.Http to OpenBSD

[am11]

Contributes to #124911.

[dotnet-policy-service]

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[am11]

cc @sethjackson no test failure:

/home/am11/th/artifacts/bin/testhost/net11.0-openbsd-Debug-x64/dotnet exec --runtimeconfig System.Net.Http.Unit.Tests.runtimeconfig.json --depsfile System.Net.Http.Unit.Tests.deps.json xunit.console.dll System.Net.Http.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================
~/system.nethttp ~/system.nethttp
  Discovering: System.Net.Http.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Http.Unit.Tests (found 964 test cases)
  Starting:    System.Net.Http.Unit.Tests (parallel test collections = on [5 threads], stop on fail = off)
    System.Net.Http.Tests.CacheControlHeaderValueTest.ToString_NegativeValues_UsesMinusSignRegardlessOfCurrentCulture [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.ParseInt32EnvironmentVariableValue_WhenNotSet_DefaultIsUsed [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingInt32_InvalidValue_FallbackToDefault [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingSwitch_AppContextHasPriority [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.ParseDoubleEnvironmentVariableValue_ValidValue [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.ParseInt32EnvironmentVariableValue_InvalidValue_FallbackToDefault [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingInt32_EnvironmentVariable [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingSwitch_InvalidValue_FallbackToDefault [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingInt32_AppContextHasPriority [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.ParseDoubleEnvironmentVariableValue_InvalidValue_FallbackToDefault [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingSwitch_EnvironmentVariable [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingSwitch_WhenNotSet_DefaultIsUsed [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.ParseInt32EnvironmentVariableValue_ValidValue [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.QueryRuntimeSettingInt32_WhenNotSet_DefaultIsUsed [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.RuntimeSettingParserTest.ParseDoubleEnvironmentVariableValue_WhenNotSet_DefaultIsUsed [SKIP]
      Condition(s) not met: "SupportsRemoteExecutor (TargetInvocationException)"
    System.Net.Http.Tests.DiagnosticsHelperTest.GetRedactedUriString_DisableUriRedaction_DoesNotRedactUri [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.SystemProxyInfoTest.Ctor_ProxyEnvironmentVariableSet_IsHttpEnvironmentProxy [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.SystemProxyInfoTest.Ctor_NoEnvironmentVariables_NotHttpEnvironmentProxy [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_CredentialParsing_DefaultCredentials [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_TryCreateAndPossibleCgi_HttpProxyUpperCaseDisabledInCgi [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_TryCreate_CaseInsensitiveVariables [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_Uri_Parsing [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_EnvironmentProxy_Loaded [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_CredentialParsing_Basic [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpEnvironmentProxyTest.HttpProxy_Exceptions_Match [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HeaderEncodingTest.GetHeaderValue_RoundTrips_ReplacesDangerousCharacters(input: "😃", encodingName: null) [SKIP]
      The test case is invalid for the default encoding.
    System.Net.Http.Tests.HeaderEncodingTest.GetHeaderValue_RoundTrips_ReplacesDangerousCharacters(input: "\xffff", encodingName: null) [SKIP]
      The test case is invalid for the default encoding.
    System.Net.Http.Tests.HeaderEncodingTest.GetHeaderValue_RoundTrips_ReplacesDangerousCharacters(input: "�", encodingName: null) [SKIP]
      The test case is invalid for the default encoding.
    System.Net.Http.Tests.HeaderEncodingTest.GetHeaderValue_RoundTrips_ReplacesDangerousCharacters(input: "🙈🙉🙊", encodingName: null) [SKIP]
      The test case is invalid for the default encoding.
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_Multi_ConcurrentUse_Success [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_Local_Parsing [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_WindowsProxy_Loaded [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.GetProxy_ManualSettingsOnly_ManualSettingsUsed [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_WindowsProxy_Manual_Loaded [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_Multi_Success [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.IsBypassed_ReturnsFalse [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.GetProxy_BothAutoDetectAndManualSettingsButFailedAutoDetect_ManualSettingsUsed [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_WindowsProxy_PAC_Loaded [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
    System.Net.Http.Tests.HttpWindowsProxyTest.HttpProxy_Local_Bypassed [SKIP]
      Condition(s) not met: "IsSupported (TargetInvocationException)"
  Finished:    System.Net.Http.Unit.Tests
=== TEST EXECUTION SUMMARY ===
   System.Net.Http.Unit.Tests  Total: 2589, Errors: 0, Failed: 0, Skipped: 39, Time: 32.408s
~/system.nethttp

[sethjackson]

Nice! Did you not need the target fx change in System.Net.Security in conjunction so the managed TLS stuff works? Eg the csproj change in #129379?

The native SSL library lookup here is much better than what I hacked up previously. 😅

[am11]

@sethjackson, ops I haven't seen that PR. OpenSSL is what .NET officially support and LibreSSL at some point used to work few years ago. But these days, it doesn't work as it is missing some required symbols like CRYPTO_clear_free we depend on. So it's much better to declare openssl as required.

This PR is adding lookup probe fro eopenssl<N>/libssl.so.<X>.<Y> and picks up the highest version on OpenBSD.

I suggest we close the other PR as this one is fixing it based on succeeding unit tests.

[sethjackson]

Oh no worries at all! This one is better anyways. :)

Yeah understood on OpenSSL. I think that System.Net.Security needs opted in on OpenBSD and I thought the System.Net.Http stuff needed the managed TLS layer to work but it appears not. 💯

[am11]

Opened #129479 for System.Net.Security. Note that I have these installed:

$ pkg_info | grep heimdal
heimdal-7.8.0       Kerberos 5 implementation
heimdal-libs-7.8.0  Heimdal libraries and headers

[sethjackson]

Awesome! Yeah I installed heimdal too but was still getting some issues. But I see you worked your magic over in the other PR. 💯

[sethjackson]

Oh BTW did you have to set LD_LIBRARY_PATH to /usr/local/lib/eopenssl35 to get this to work? I did with my changes but not sure if you had to with the updated shim code. I'm assuming so since it's the same DlOpen call at the end but wasn't sure. Only reason I ask is so that I can add it to the package README if needed for the eventual port submission.

[am11]

Oh BTW did you have to set LD_LIBRARY_PATH to /usr/local/lib/eopenssl35 to get this to work? I did with my changes but not sure if you had to with the updated shim code. I'm assuming so since it's the same DlOpen call at the end but wasn't sure. Only reason I ask is so that I can add it to the package README if needed for the eventual port submission.

I had to set it initially, but then I added a probe which looks for /usr/local/lib/eopenssl prefix and finds /usr/local/lib/eopenssl35 or whichever "highest" number is available. Then picks up libssl.so.X.Y from there. Now we don't need to set LD_LIBRARY_PATH manually.

[am11]

I just pushed versioning.targets change in these PRs, so order of merge is independent while the CI is happy.

[sethjackson]

I had to set it initially, but then I added a probe which looks for /usr/local/lib/eopenssl prefix and finds /usr/local/lib/eopenssl35 or whichever "highest" number is available. Then picks up libssl.so.X.Y from there. Now we don't need to set LD_LIBRARY_PATH manually.

Oh right. I skimmed the shim changes and failed to realize it looks in the /usr/local/lib/eopenssl prefix.