Skip to content

Increase Fidelity of the sbt.testing.Framework Implementation #1107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

dubinsky
Copy link

@dubinsky dubinsky commented Apr 4, 2025

  • when Selectors supplied include only TestSelectors and TestWildcardSelectors, filter properties to run by matching their names against the Selectors;
  • added a test that actually runs ScalaCheck via the SBT Test Interface and demonstrates the (now correct) treatment of the Selectors;
  • test also demonstrates two unfixable infidelities in the treatment of the nested properties;
  • thankfully, ScalaCheck's implementation of sbt.testing.Framework is, unlike in some other test frameworks, shared between the platforms (JVM, Scala.js, Scala Native), so the fixes do not have to be replicated for each platform, but:
  • the test needs to supply a testClassLoader: ClassLoader parameter when calling sbt.testing.Framework.runner(); on platforms other than JVM, getClass.getClassLoader is not available, so Platform.getClassLoader: ClassLoader method was added to every Platform; on platforms other than JVM, it returns null - which is fine, since on those platforms sbt.testing.Framework.runner() ignores the testClassLoader parameter anyway.

fixes #1105

@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 05bd678 to 7ce3e5f Compare April 4, 2025 20:24
@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 7ce3e5f to da9cb92 Compare April 28, 2025 05:04
@dubinsky
Copy link
Author

@satorg any thoughts?

@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from da9cb92 to 4a468a9 Compare April 28, 2025 08:44
@dubinsky
Copy link
Author

@satorg I am starting to suspect that the plan here is to let this fix wither on the wine like the previous contribution in this area (#1031); if this is true, and my contribution is for some reason not wanted, please say so; otherwise - what do I need to do to get this merged?

Thank you!

@satorg
Copy link
Contributor

satorg commented Jun 15, 2025

@dubinsky thank you for the PR and sorry for the long reply!

Personally, I'd love to get both PRs (this and #1031) merged ultimately. I just don't feel very confident about this ScalaCheckFramework implementation, because I personally don't use it and mostly rely on other test framework runners.

Do you think your PR complements #1031 or supersedes it completely?

…dcardSelector`s, filter properties to run by matching their names against the `Selector`s;

- added a test that actually runs ScalaCheck via the `SBT Test Interface` and demonstrates the (now correct) treatment of the `Selector`s;
- test also demonstrates two unfixable infidelities in the treatment of the nested properties;
- thankfully, ScalaCheck's implementation of `sbt.testing.Framework` is, unlike in some other test frameworks, shared between the platforms (JVM, Scala.js, Scala Native), so the fixes do not have to be replicated for each platform, but:
- the test needs to supply a `testClassLoader: ClassLoader` parameter when calling `sbt.testing.Framework.runner()`; on platforms other than JVM, `getClass.getClassLoader` is not available, so `Platform.getClassLoader: ClassLoader` method was added to every `Platform`; on platforms other than the JVM, it returns `null`, which is fine since on those platforms `sbt.testing.Framework.runner()` ignores the `testClassLoader` parameter anyway.

fixes typelevel#1105
@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 4a468a9 to 2e33740 Compare June 16, 2025 05:00
@dubinsky
Copy link
Author

Personally, I'd love to get both PRs (this and #1031) merged ultimately. I just don't feel very confident about this ScalaCheckFramework implementation, because I personally don't use it and mostly rely on other test framework runners.

Code changes have comments, tests are added that verify the change, and issue #1105 has detailed explanations. Is there anything else I can do? Are you going to involve other contributors/maintainers that are more familiar with this area or do you have specific questions?

Do you think your PR complements #1031 or supersedes it completely?

My pull request completely supersedes #1031: it handles both TestSelector and TestWildcardSelector, whereas #1031 handles just TestSelector.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sbt ScalaCheckRunner: loss of test selection fidelity
2 participants