Skip to content

Fix: Update dependency versions and add spark-testing-base to share the same spark context accross the tests #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

n-raymond
Copy link

Hello !

I recently started to learn Spark and found your amazing exercises base. Since the dependency were not updated for a long time, I tried to do it myself and finally get the test working with the latest spark version.

I would like to share this modification with you if you are interested.

Fixes:

  • Update dependencies versions to get an up-to-date project
  • Add spark-testing-base to share the same spark context across the tests (since a unique spark context can only be instantiated at runtime) and refactor the test and classes to be compatible with this implementation.
  • Replace also each occurence of null by ??? to get a NotImplementedException which is more accurate.

Thank you for your tutorials and have a really nice day !

Nicolas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant