Add parameter to control extension output

[kgominyuk]

Hey, guys 👋

Code generation can significantly bloat the app size in large projects. The ability to fine-tune what is generated will help minimize impact while still allowing to enable features when they are needed.

This PR just shows the general direction, and I'd love to hear your thoughts so we're aligned 🙂

What has changed:

1. _MessageImplementationBase is replaced with _MessageEquatable and _MessageHashable
2. In generated code, one code block that implements all extensions is replaced with several blocks (one per extension).

Before:

extension Com_Example_Player: SwiftProtobuf.Message, SwiftProtobuf._MessageImplementationBase, SwiftProtobuf._ProtoNameProviding { ... }

After:

extension Com_Example_Player: SwiftProtobuf.Message { ... }
extension Com_Example_Player: SwiftProtobuf._MessageEquatable { ... }
extension Com_Example_Player: SwiftProtobuf._MessageHashable { ... }
extension Com_Example_Player: SwiftProtobuf._ProtoNameProviding { ... }

3. Add parameter to control extension output

For example we have proto:

message Player { ... }
message Board { ... }

Here are some examples of how GenerateExtensions changes code generation:

// Parameter is not set
GenerateExtensions=

extension Player: SwiftProtobuf.Message { ... }
extension Player: SwiftProtobuf._MessageEquatable { ... }
extension Player: SwiftProtobuf._MessageHashable { }
extension Player: SwiftProtobuf._ProtoNameProviding { ... }
extension Board: SwiftProtobuf.Message { ... }
extension Board: SwiftProtobuf._MessageEquatable { ... }
extension Board: SwiftProtobuf._MessageHashable { }
extension Board: SwiftProtobuf._ProtoNameProviding { ... }

// Does not generate extensions for all types.
GenerateExtensions=equatable=skip;hashable=skip;nameProviding=skip

extension Player: SwiftProtobuf.Message { ... }
extension Board: SwiftProtobuf.Message { ... }

// Generates only _MessageEquatable and _MessageHashable extensions in debug for all types.
GenerateExtensions=equatable=debug_only;hashable=debug_only;nameProviding=skip

extension Player: SwiftProtobuf.Message { ... }
#if DEBUG
extension Player: SwiftProtobuf._MessageEquatable { ... }
#endif
#if DEBUG
extension Player: SwiftProtobuf._MessageHashable { ... }
#endif
extension Board: SwiftProtobuf.Message { ... }
#if DEBUG
extension Board: SwiftProtobuf._MessageEquatable { ... }
#endif
#if DEBUG
extension Board: SwiftProtobuf._MessageHashable { ... }
#endif

// Generates only _ProtoNameProviding only for Player.
GenerateExtensions=equatable=skip;hashable=skip;nameProviding=skip;Player:nameProviding=normal 

extension Player: SwiftProtobuf.Message { ... }
extension Player: SwiftProtobuf._ProtoNameProviding { ... }
extension Board: SwiftProtobuf.Message { ... }

[allevato]

We're definitely interested in reducing the amount of codegen, but making it based on options is problematic: beyond just introducing more complexity, it also requires that all protos used throughout the entire system be generated with the same options to work correctly.

I think we're better off exploring ways of universally reducing codegen without removing features, such as the bytecode-based approach I've started looking at in #1789.

[thomasvl]

You might also want to read some of the past things that have come into play looking at binary size.

• Convert _MessageBase to a typealias #1498 Annotate marker protocols with @_marker #1497 as some choices result in size differences
• Look at how to factor out JSON/text support #18 Split text generation into different files #1240 to do different generation, hopefully so only the needed parts would be linked vs. needed custom generation options.

[kgominyuk]

Thanks for the feedback - I completely understand the hesitation about introducing new option-driven divergence in codegen. So by the default the generated code will be the same.

Universally reducing codegen is a great north star. The ultimate version of reducing the code - is to simply not include it to the binary. That's why I chose this direction.

100% agree that with great power comes great responsibility. But at a stage when developers care about the app size, they definitely can handle this power.

Could you explain in more details why generating code based on options is problematic?

[thomasvl]

Could you explain in more details why generating code based on options is problematic?

The problems can come when two different things both need the same set of protos. With fewer options, they can share that generation and thus the final binary only needs one copy. But if options change the generation, they can need things generated differently. This can result in them not being able to share a common generation and have to do module specific copies. But worse, if also means the generated proto types likely must be implementation details, i.e. - they can't put the messages in their public interfaces as that is more likely to result in the generation choices having problem if the protos are needed by something else.

[thomasvl]

btw - I haven't really looked at them, but SwiftPM Package Traits might be something to also look at why trying to think about this problem. I'm not sure that it would directly be usable, but since it seems like it might be a useful concept to reference on these topics.

[kgominyuk]

And when the generated code is slit into several files, two different things can grab whatever files they need. Is this correct?

[thomasvl]

And when the generated code is slit into several files, two different things can grab whatever files they need. Is this correct?

As long as the files aren't exclusive, yes.

This PRs ideas around "debug only" could still be a problem from that pov. If one module always needs something and another only wants it in debug, that doesn't seem like it could easily be resolved.

[kgominyuk]

This PRs ideas around "debug only" could still be a problem from that pov. If one module always needs something and another only wants it in debug, that doesn't seem like it could easily be resolved.

Let's forget about the debug only 🙂

If I have my.proto and the generated code will be split into my.pb.swift, my.pb.hashable.swift, my.pb.nameproviding.swift, is this the right direction?

Here is a draft with possible implementation - #1796

[thomasvl]

If I have my.proto and the generated code will be split into my.pb.swift, my.pb.hashable.swift, my.pb.nameproviding.swift, is this the right direction?

It could be. As with the referenced other ideas, I don't think anyone had found something we were sure was "right", vs. exploring options. We did know generation options did have some downsides when used in larger/complex proto graphs.