[feature request] Run multiple commands in parallel

[Bessonov]

I didn't found any feature request or documentation about that. I'm experimenting with hasura and I want to run multiple commands in parallel. For example I have following definition:

	"scripts": {
		"start": "$_ run --stream --parallel dev",
		"dev": "tsc --build --watch packages/tsconfig.json",
		"console": "$_ run hasura --project hasura console --address 0.0.0.0 --console-port 8081 --no-browser --log-level DEBUG --api-port 8097",

With start I bootstrap the compiler, but also backend and frontend in other packages. Now, for hasura, I must start the console, which is a permanent running process, like other three. It would be nice if pnpm supports the workflow above. I can imagine something like:

	"scripts": {
		"start": "$_ run --stream --parallel dev:*",
		"dev:compiler": "tsc --build --watch packages/tsconfig.json",
		"dev:console": "$_ run hasura --project hasura console --address 0.0.0.0 --console-port 8081 --no-browser --log-level DEBUG --api-port 8097",

or:

		"start": "$_ run --stream --parallel dev,console",

[zkochan]

I think on Linux you can do something like

"scripts: {
  "start": "$_ run dev:compiler & $_ run dev:console",
  "dev:compiler": "tsc --build --watch packages/tsconfig.json",
  "dev:console": "$_ run hasura --project hasura console --address 0.0.0.0 --console-port 8081 --no-browser --log-level DEBUG --api-port 8097",
}

And if we implement #2621, then it will probably work on all platforms

[Bessonov]

Hey @zkochan thank you very much for your response. In past I've used concurrently. It worked, but not very satisfying, because it leads to some problems, at least in my setup. The --stream --parallel was a great replacement for that and works without glitches. Therefore I'm looking for pnpm native support to run multiple commands in the same package.json and not only in monorepo modules.

[zkochan]

Maybe we can add it. Not sure

I currently use npm-run-all for what you described

[gunta]

I found a case where this was needed:
When you need to bootstrap the project before installing any dependency. Or when you need to clean the project or remove all node_modules.
I was currently using npm-run-all but it doesn't work when there is nothing installed, so it becomes a chicken-egg problem.

It would be very useful to have cross-platform parallel processing functionality embedded in pnpm run, pnpm exec and pnpm dlx.

[robertherber]

I'm currently using concurrently for this. But would prefer to have it built into pnpm - especially since --parallel seems to already provide similar functionality.

[ftzi]

+1. Having to add a concurrently or npm-run-all feels unnecessary. pnpm could have this for a more modern usage.

[Bessonov]

After many trial and error I have discovered that the issue is not only "run tasks in parallel", but also get a reasonable output. ATM I am using https://github.com/folke/ultra-runner . This produces amazing output, but, unfortunately, it is not more maintained.

[Bessonov]

I wouldn't try it soon, but it seems like this one could be resolved by #6692

[zkochan]

no, not by that issue. As I said in that issue, the --parallel flag doesn't make any sense on single project.

But running multiple scripts is supported using regex

https://pnpm.io/cli/run#running-multiple-scripts

[KuSh]

My need falls under that request too and doesn't seem to be available at the moment:

I need to run multiple commands in parallel in multiple workspaces:

pnpm run --parallel --filter pkg1 --filter pkg2 --filter pkg3 cmd1 --filter pkg4 cmd2

I didn't found a way to do that unfortunately.
My actual workaround is to define cmd2 in pkg1 2 and 3 as an alias to cmd1

[zkochan]

As of today, it is possible to run multiple scripts in parallel using regex. For instance, run all scripts that start with watch::

pnpm run "/^watch:.*/"

[KuSh]

As of today, it is possible to run multiple scripts in parallel using regex. For instance, run all scripts that start with watch::

pnpm run "/^watch:.*/"

That's not exactly my use case as I want to run multiple different scripts in parallel (cmd1 and cmd2 are named differently)

[zkochan]

Can't you still write a regex that matches both command names? Something like:

pnpm run "/(cmd1|cmd2)/"

[KuSh]

See my previous comment, I need to run different scripts in different workspaces in parallel:

• run cmd1 in parallel in pkg1, pkg2 and pkg3 workspaces
• while also running cmd2 in pkg2 workspace

npm-run-all allow that by allowing to chain group of scripts, it even allow to chain parallel and sequential group of scripts, see https://github.com/mysticatea/npm-run-all/blob/master/docs/npm-run-all.md#run-a-mix-of-sequential-and-parallel-scripts

[zkochan]

So you can just create wrapper scripts and run those in parallel.

{
  "scripts": {
    "script1": "pnpm --filter pkg1 --filter pkg2 --filter pkg3 run cmd1",
    "script2": "pnpm --filter pkg2 run cmd2"
  }
}

[KuSh]

That's what I'm currently doing but:

• it is way much verbose
• it is way more difficult to maintain
• it pollutes shell autocomplete with uneeded entries
• output can't be clearly distinguished

[mnemitz]

+1 For this, would be great for pnpm to provide npm-run-all like functionality out of the box without regexes.

[zkochan]

What is the issue with regexes?