Parallelbuild

[allanbrondum]

Purpose

Declare to build modules in parallel. Will have effect in development environment and in CI

Changes

_Describe the changes that were needed.

Checklist

• My code follows the style of this project.
• The code compiles without warnings.
• I have performed a self-review of the changes.
• I have documented my code, in particular the intent of the
  hard-to-understand areas.
• (If necessary) I have updated the CHANGELOG.

[td202]

I'm not sure I like putting -j in the stack.yaml.

[allanbrondum]

I'm not sure I like putting -j in the stack.yaml.

What are your cons and pros (or principle for not doing it)?

What I like about it is that it

• creates consistency between CI and development environment in a declarative way, so you don't have to specify so many command line arguments to get same behaviour as in CI
• developers don't have to enable parallel module builds "manually" on their machine

[td202]

I just don't think -j is appropriate in all circumstances, so I don't think we should configure it like this. For instance, it may be better not to use -j when building dependencies, since stack already parallelizes at the package level.

[allanbrondum]

I just don't think -j is appropriate in all circumstances, so I don't think we should configure it like this. For instance, it may be better not to use -j when building dependencies, since stack already parallelizes at the package level.

I though think that enabling parallel module build by default should be good (not sure why it isn't default in GHC)? Seems a bit hypothetical to not do it?

Though, I tried timing building dependencies for the node (deleting ~/.stack-work/snapshots before each build), and I'm not sure what to conclude about how the different parameters affect compilation:

stack build --only-dependencies  696.41s user 128.75s system 428% cpu 3:12.57 total (plt branch)
stack build --only-dependencies --ghc-options -j  699.76s user 126.84s system 430% cpu 3:12.23 total (plt branch)
stack build --only-dependencies --ghc-options -j4  698.49s user 126.90s system 427% cpu 3:13.09 total (plt branch)
stack build --only-dependencies  811.14s user 142.65s system 583% cpu 2:43.57 total (plt branch, j4 ghc options in ~/.stack/config.yaml)
stack build --only-dependencies  865.71s user 237.69s system 625% cpu 2:56.54 total (this branch)
stack build --only-dependencies  869.05s user 236.49s system 616% cpu 2:59.44 total (this branch, j4 ghc options in ~/.stack/config.yaml)

It seems like the command line -j option has no effect. Specifying the option in package stack.yaml has some effect. And specifying it in ~/.stack/config.yaml has the largest effect.