-
-
Notifications
You must be signed in to change notification settings - Fork 20.1k
nixos/netbird-relay: init module #559588
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
nixos/netbird-relay: init module #559588
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
92 changes: 92 additions & 0 deletions
92
nixos/modules/services/networking/netbird/netbird-relay.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # Netbird Relay {#module-services-netbird-relay} | ||
|
|
||
| The Relay service forwards encrypted WireGuard traffic between peers that cannot establish a direct P2P connection | ||
|
|
||
| For more information, check [external relays](https://docs.netbird.io/selfhosted/maintenance/scaling/set-up-external-relays) documentation. | ||
|
|
||
| ## Quickstart {#module-services-netbird-relay-quickstart} | ||
|
|
||
| You can run the relay directly, or behind a reverse proxy, like `traefik`. | ||
|
|
||
| To run it directly, you'll have to run it on HTTPS port (443) and it will need the TLS certificates. Which you can configure using the `acme` module. | ||
|
|
||
| In the following example, we can see the `traefik` route and the relay configuration. | ||
| The operator should configure the TLS for the domain used by the relay, whether it's via `traefik` itself or using the `acme` module. | ||
|
|
||
| ```nix | ||
| let | ||
| relayDomain = "relay.example.com"; | ||
| in | ||
| { | ||
| services.netbird.relay = { | ||
| enable = true; | ||
| settings = { | ||
| listen-address = ":33080"; | ||
| exposed-address = "rels://${relayDomain}:443"; | ||
| log-level = "info"; | ||
| enable-stun = true; | ||
| stun-ports = [ 3479 ]; | ||
| }; | ||
| openFirewall = true; | ||
| authSecretFile = "/run/auth_secret"; | ||
| }; | ||
|
|
||
| services.traefik = { | ||
| enable = true; | ||
| dynamicConfigOptions = { | ||
| http = { | ||
| routers = { | ||
| vpn-relay = { | ||
| rule = "Host(`${relayDomain}`)"; | ||
| entryPoints = [ "websecure" ]; | ||
| service = "vpn-relay-svc"; | ||
| tls = { }; | ||
| }; | ||
| }; | ||
| services = { | ||
| vpn-relay-svc = { | ||
| loadBalancer.servers = [ { url = "http://[::1]:33080"; } ]; | ||
| }; | ||
| }; | ||
| }; | ||
| }; | ||
| }; | ||
| } | ||
| ``` | ||
|
|
||
| Finally, you must let `netbird.server` know about the new relay: | ||
|
|
||
| ```nix | ||
| { }: | ||
| { | ||
| services.netbird.server = { | ||
| # ...other config... | ||
| management = { | ||
| # ...other config... | ||
| settings = { | ||
| # ...other config... | ||
| Relay = { | ||
| Addresses = [ "rels://${relayDomain}:443" ]; | ||
| CredentialsTTL = "24h0m0s"; | ||
| Secret = { | ||
| _secret = "/run/auth_secret"; | ||
| }; | ||
| }; | ||
| }; | ||
| }; | ||
| }; | ||
| } | ||
| ``` | ||
| Done! | ||
|
|
||
| ## TLS with Let's Encrypt {#module-services-netbird-relay-tls} | ||
|
|
||
| You can expose the netbird relay directly to the internet with TLS. Then you won't need a reverse proxy. | ||
|
|
||
| To use TLS, set `settings."letsencrypt-domains"`. | ||
| The relay gets and renews the certificates only when `settings."letsencrypt-domains"` is set. | ||
|
|
||
| Certificates are stored on disk, and the services runs as a systemd dynamic user. | ||
| The only writable location is therefore a `StateDirectory`, which can only be under `/var/lib`. | ||
|
|
||
| Open the ports `tcp/443` and `tcp/80` manually for the HTTP challenge. |
248 changes: 248 additions & 0 deletions
248
nixos/modules/services/networking/netbird/netbird-relay.nix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,248 @@ | ||
| { | ||
| config, | ||
| lib, | ||
| pkgs, | ||
| ... | ||
| }: | ||
|
|
||
| let | ||
| inherit (lib) | ||
| getExe' | ||
| types | ||
| ; | ||
|
|
||
| cfg = config.services.netbird.relay; | ||
| cmd = getExe' cfg.package "netbird-relay"; | ||
| derivedLEDataDir = lib.optionalAttrs ( | ||
| cfg.settings ? "letsencrypt-domain" && !cfg.settings ? "letsencrypt-data-dir" | ||
| ) { "letsencrypt-data-dir" = "/var/lib/netbird-relay"; }; | ||
|
|
||
| args = lib.cli.toCommandLineShellGNU { } (cfg.settings // derivedLEDataDir); | ||
|
|
||
| listenPortMatch = builtins.match ".*:([0-9]+)$" cfg.settings.listen-address; | ||
| listenPort = if listenPortMatch == null then null else lib.toInt (builtins.head listenPortMatch); | ||
|
|
||
| # Checks | ||
| authSecretSet = cfg.authSecretFile != null; | ||
| needsPrivPort = listenPort != null && listenPort < 1024; | ||
| letsencryptEnabled = cfg.settings ? "letsencrypt-domain"; | ||
| in | ||
| { | ||
| options.services.netbird.relay = { | ||
| enable = lib.mkEnableOption "Netbird's Relay Service"; | ||
| package = lib.mkPackageOption pkgs "netbird-relay" { }; | ||
|
|
||
| # as per RFC 0042: | ||
| settings = lib.mkOption { | ||
| type = types.submodule { | ||
| freeformType = | ||
| with lib.types; | ||
| nullOr (oneOf [ | ||
| bool | ||
| str | ||
| # For --letsencrypt-domains strings | ||
| (listOf (oneOf [ str ])) | ||
| ]); | ||
| options.listen-address = lib.mkOption { | ||
| type = types.str; | ||
| description = '' | ||
| The host address and port separated by colon where the relay will listen | ||
| ''; | ||
| default = ":33080"; | ||
| }; | ||
| options.exposed-address = lib.mkOption { | ||
| type = types.str; | ||
| description = '' | ||
| Exposed address for peers. Address told to the peers to connect to | ||
| ''; | ||
| example = "rels://relay.example.com:443"; | ||
| }; | ||
| options.enable-stun = lib.mkOption { | ||
| type = types.bool; | ||
| default = false; | ||
| description = "Enable embedded STUN server"; | ||
| }; | ||
| options.stun-ports = lib.mkOption { | ||
| type = types.listOf types.port; | ||
| default = [ 3478 ]; | ||
| description = "STUN server UDP ports"; | ||
| }; | ||
| options.log-level = lib.mkOption { | ||
| type = types.enum [ | ||
| "panic" | ||
| "fatal" | ||
| "error" | ||
| "warn" | ||
| "info" | ||
| "debug" | ||
| "trace" | ||
| ]; | ||
| default = "info"; | ||
| description = "Log level of the netbird relay service"; | ||
| }; | ||
| options.metrics-port = lib.mkOption { | ||
| type = types.port; | ||
| default = 9092; | ||
| description = "Metrics endpoint http port. Metrics are accessible under host:metrics-port/metrics"; | ||
| }; | ||
| options.health-listen-address = lib.mkOption { | ||
| type = types.str; | ||
| description = '' | ||
| Listen address of healthcheck server | ||
| ''; | ||
| default = ":9000"; | ||
| }; | ||
| }; | ||
| default = null; | ||
| description = '' | ||
| Settings to configure the netbird relay. | ||
| Converted automatically into cli args, check all the options with `netbird-relay --help` | ||
| ''; | ||
| }; | ||
|
|
||
| environmentFile = lib.mkOption { | ||
| type = types.nullOr types.externalPath; | ||
| default = null; | ||
| description = '' | ||
| Path (as string) to an environmentFile with the netbird relay environment variables. | ||
| You can find all the variables under [Relay runtime env variables](https://docs.netbird.io/selfhosted/environment-variables#runtime-variables-2). | ||
|
|
||
| WARNING: You must manually open any port configured via environmentFile. | ||
| If you use an Internet domain privileged port (e.g 443), add CAP_NET_BIND_SERVICE to systemd's CapabilityBoundingSet and AmbientCapabilities | ||
| ''; | ||
| example = "/run/netbird-relay.env"; | ||
| }; | ||
|
|
||
| openFirewall = lib.mkOption { | ||
| type = types.bool; | ||
| default = false; | ||
| description = '' | ||
| Open STUN ports in the firewall for the netbird relay. | ||
|
|
||
| WARNING: You must manually open listen-address port/tcp and port 80/tcp, | ||
| if you are exposing the relay directly to the internet. | ||
| Review [set-up-external-relays](https://docs.netbird.io/selfhosted/maintenance/scaling/set-up-external-relays) | ||
| ''; | ||
| }; | ||
|
|
||
| authSecretFile = lib.mkOption { | ||
|
woile marked this conversation as resolved.
|
||
| type = types.nullOr types.externalPath; | ||
| default = null; | ||
| description = '' | ||
| Path (as string) to a file containing the auth-secret used by netbird to connect to the relay server. | ||
| It will populate `NB_AUTH_SECRET` | ||
| ''; | ||
| example = "/run/auth_secret"; | ||
| }; | ||
|
|
||
| }; | ||
|
|
||
| config = lib.mkIf cfg.enable { | ||
| assertions = [ | ||
| { | ||
| assertion = authSecretSet || cfg.environmentFile != null; | ||
| message = "services.netbird.relay requires NB_AUTH_SECRET, either set authSecretFile or configure it in environmentFile"; | ||
| } | ||
| { | ||
| assertion = !(cfg.settings.auth-secret or null != null); | ||
| message = "settings.auth-secret puts the secret into the nix store and the cli args. Use relay.authSecretFile instead"; | ||
| } | ||
| { | ||
| assertion = | ||
| !letsencryptEnabled || lib.hasPrefix "/var/lib/" (cfg.settings."letsencrypt-data-dir" or ""); | ||
| message = "settings.letsencrypt-data-dir must be under /var/lib, the only writable location when using DynamidcUser in systemd"; | ||
| } | ||
| { | ||
| assertion = !(cfg.settings.enable-stun && cfg.settings.stun-ports == [ ]); | ||
| message = "Missing stun ports while enable-stun = true"; | ||
| } | ||
| ]; | ||
|
|
||
| systemd.services.netbird-relay = { | ||
| description = "Relay for Netbird, a wireguard VPN"; | ||
| documentation = [ "https://netbird.io/docs/" ]; | ||
| after = [ | ||
| "network.target" | ||
| ]; | ||
| wantedBy = [ "multi-user.target" ]; | ||
|
|
||
| script = '' | ||
| ${lib.optionalString authSecretSet '' | ||
| export NB_AUTH_SECRET="$(< "$CREDENTIALS_DIRECTORY/auth_secret")" | ||
| ''} | ||
| exec ${cmd} ${args} | ||
| ''; | ||
| unitConfig = { | ||
| # Spread the restart | ||
| StartLimitIntervalSec = 60; | ||
| StartLimitBurst = 10; | ||
| }; | ||
| serviceConfig = { | ||
| EnvironmentFile = lib.mkIf (cfg.environmentFile != null) [ cfg.environmentFile ]; | ||
|
|
||
| # Let systemd ingest the secret safely | ||
| LoadCredential = lib.mkIf authSecretSet "auth_secret:${cfg.authSecretFile}"; | ||
|
|
||
| Restart = "always"; | ||
| RestartSec = "5s"; | ||
|
|
||
| StateDirectory = lib.mkIf letsencryptEnabled ( | ||
| lib.removeSuffix "/" (lib.removePrefix "/var/lib/" cfg.settings."letsencrypt-data-dir") | ||
| ); | ||
|
|
||
| # Hardening::Start | ||
| # Dynamic user automatically sets: | ||
| # NoNewPrivileges = true; | ||
| # RemoveIPC = true; | ||
| # RestrictSUIDSGID = true; | ||
| # ProtectSystem = "strict"; | ||
| # ProtectHome = "read-only"; | ||
| # see https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#DynamicUser= | ||
| DynamicUser = true; | ||
|
|
||
| LockPersonality = true; | ||
| MemoryDenyWriteExecute = true; | ||
| # Relay is mostly stateless, no need to give access to any of this: | ||
| PrivateDevices = true; | ||
| PrivateMounts = true; | ||
| PrivateTmp = true; | ||
| ProtectClock = true; | ||
| ProtectControlGroups = true; | ||
| ProtectHostname = true; | ||
| ProtectKernelLogs = true; | ||
| ProtectKernelModules = true; | ||
| ProtectKernelTunables = true; | ||
| ProcSubset = "pid"; | ||
| ProtectProc = "invisible"; | ||
| RestrictNamespaces = true; | ||
| RestrictRealtime = true; | ||
| SystemCallFilter = [ "@system-service" ]; | ||
| SystemCallArchitectures = "native"; | ||
|
|
||
| # If a user wants to bind netbird to 443, we must allow binding to | ||
| # a socket to Internet domain privileged ports | ||
| CapabilityBoundingSet = lib.optionals needsPrivPort [ "CAP_NET_BIND_SERVICE" ]; | ||
| AmbientCapabilities = lib.optionals needsPrivPort [ "CAP_NET_BIND_SERVICE" ]; | ||
|
|
||
| # Use only IP, no socket needed | ||
| RestrictAddressFamilies = [ | ||
| "AF_INET" | ||
| "AF_INET6" | ||
| "AF_NETLINK" | ||
| ]; | ||
| UMask = "0077"; | ||
| # Hardening::End | ||
| }; | ||
| }; | ||
| networking.firewall.allowedUDPPorts = lib.mkIf ( | ||
| cfg.openFirewall && cfg.settings.enable-stun | ||
| ) cfg.settings.stun-ports; | ||
| }; | ||
|
|
||
| meta = { | ||
| doc = ./netbird-relay.md; | ||
| maintainers = [ | ||
| lib.maintainers.woile | ||
| ]; | ||
| }; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.