Bazel rule for generating injectors

[mohammadshamma]

It would be great if Wire would work out of the box with Bazel.

There could be a "go_wire_injector()" build rule that takes the source of the file containing the injector and generate a library that could be included as a dependency elsewhere. The generated file could be left out uncommitted in the source code repository (similar to how proto rules work).

[zombiezen]

Hey Mohammad. We've discussed this in the team in the past, and the conclusion we came to at the time is that Bazel's model of needing to know the precise set of dependencies for the generated code is at odds with Wire's model of stripping unneeded dependencies in the generated code. However, it would be nice to have a guide or documentation on how to use Wire correctly with a Bazel project.

(@jayconrod for visibility.)

[jayconrod]

I think the issue was that we don't really know at analysis time (before reading source files) what the imports will be in the generated go files. Is that roughly correct?

If we have an upper bound on that set of imports, for example, all transitive imports of a set of explicit dependencies, it could probably work.

[zombiezen]

I think the issue was that we don't really know at analysis time (before reading source files) what the imports will be in the generated go files. Is that roughly correct?

Correct.

If we have an upper bound on that set of imports, for example, all transitive imports of a set of explicit dependencies, it could probably work.

Yes, it is guaranteed that Wire's generated file will not import anything outside of its transitive imports. I think there may have been some additional concerns for inside-Google-Blaze, but I forget what they were.

[robfig]

We have an internal bazel rule for running wire, by setting up a GOPATH using the go_path rule and running the usual wire binary.
https://github.com/bazelbuild/rules_go/blob/master/go/private/tools/path.bzl#L155

It works, but it is really really really slow. Slow to the point of developers removing usage of wire because it extends the build by minutes.

Looking into it now, we could try go_path.mode = "link" instead of "copy" and disabling the bazel sandbox for it. Probably it would work acceptably using execution_requirements = {"local" = "1"}

[ah-quant]

I created an issue for this in bazel-gazelle, but closed it after Jay pointed me here. Is there progress or hope for progress in this issue? Because I'd love to use gocloud.dev with Bazel and that currently is not a good experience.

[zombiezen]

I don't know of anyone actively working on this issue, but this shouldn't block use of the Go CDK (gocloud.dev) inside Bazel. This issue tracks supporting Wire inside Bazel. If Bazel supports a package loader now, it might be easy enough to check in a file generated by Wire and use Gazelle, but I don't know where to look for that. @jayconrod?

[ah-quant]

Thank you for still reading and commenting here though you're no longer at Google - and thanks for gocloud.dev.
gocloud.dev is still usable, but this produces a lot of ugly output and superfuous dependency lists in our workflow.
gazelle somehow does not recognize tests. We convert go.mod to a deps.bzl file in our workflow, but gazelle sees packages from tests, too.
For these requires in go.mod, below is output we see on a fresh build:

require (
        github.com/golang/protobuf v1.4.2
        github.com/nats-io/nats.go v1.10.0
        github.com/pkg/sftp v1.11.0
        gocloud.dev v0.20.0
        gocloud.dev/pubsub/natspubsub v0.20.0
        golang.org/x/crypto v0.0.0-20200604202706-70a84ac30bf9
        google.golang.org/grpc v1.29.1
)

gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/baz: exit status 1: go: finding module for package example.com/baz
can't load package: cannot find module providing package example.com/baz: unrecognized import path "example.com/baz": reading https://example.com/baz?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/baz: exit status 1: go: finding module for package example.com/baz
can't load package: cannot find module providing package example.com/baz: unrecognized import path "example.com/baz": reading https://example.com/baz?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/baz: exit status 1: go: finding module for package example.com/baz
can't load package: cannot find module providing package example.com/baz: unrecognized import path "example.com/baz": reading https://example.com/baz?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/foo: exit status 1: go: finding module for package example.com/foo
can't load package: cannot find module providing package example.com/foo: unrecognized import path "example.com/foo": reading https://example.com/foo?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/anon1: exit status 1: go: finding module for package example.com/anon1
can't load package: cannot find module providing package example.com/anon1: unrecognized import path "example.com/anon1": reading https://example.com/anon1?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/anon2: exit status 1: go: finding module for package example.com/anon2
can't load package: cannot find module providing package example.com/anon2: unrecognized import path "example.com/anon2": reading https://example.com/anon2?go-get=1: 404 Not Found
gazelle: finding module path for import example.com/bar: exit status 1: go: finding module for package example.com/bar
can't load package: cannot find module providing package example.com/bar: unrecognized import path "example.com/bar": reading https://example.com/bar?go-get=1: 404 Not Found

NOTE this still misses a bit of info (gazelle conversion from go.mod to deps.bzl for example). I'll write a short guide on how to replicate this problem. Ignore it for now unless it already helps.

[zombiezen]

Ah, looks like you're hitting bazel-contrib/bazel-gazelle#610, not this issue. This issue covers generating your own Wire dependency injection using Bazel.

[ah-quant]

I think that was not all, just a very visible part - but it's been a while, I'll have to check again.
The other issue was very unwieldly and overly long dependency lists if I recall correctly.
I'll check doing an example project and report if I hit a snag or if everything is allright.
Thank you, though!

[commure-stabai]

Is there any hope for either a Bazel plugin for this or at least a best practices guide of how to use Wire with Bazel? It would be very helpful.