Skip to content

Conversation

@Jappie3
Copy link
Contributor

@Jappie3 Jappie3 commented Mar 9, 2025

This PR updates the Dovecot version to 2.4. Version 2.3 will be kept around for compatibility, as 2.4 has quite a few breaking changes (https://doc.dovecot.org/2.4.0/installation/upgrade/2.3-to-2.4.html). Updates for all Dovecot plugins present in nixpkgs are also included.

Besides that, the NixOS module for Dovecot is also updated to support the changes to the config format.

Resolves #388262

see #410912 for a followup regarding an RFC42-style settings option

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 9, 2025

just missed @2xsaiko 's comment on the issue:

This would probably a good place to add versioned packages, i.e. dovecot_2_3 and dovecot_2_4, at least for this upgrade. These changes are non-trivial and very likely people will need to change their config for it. Least we could do is not silently break it but have them explicitly change the version.

sounds like a good idea, I'm not an expert at this but I think we can lock the module changes behind system.stateVersion?

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 9, 2025

I would also like to explore whether or not we can use settings for all dovecot config instead of some options combined with extraConfig - i.e. NixOS/rfcs#42.

@github-actions github-actions bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 9, 2025
@dblsaiko
Copy link
Contributor

dblsaiko commented Mar 9, 2025

Ah, nice, I just started doing the same thing. Didn't get very far yet though -- just the by-name change below.

sounds like a good idea, I'm not an expert at this but I think we can lock the module changes behind system.stateVersion?

I would just set dovecot to continue being 2.3 for now, or set the module to use 2.3, and have it print a warning when used so people manually update to 2.4. Then in the next release cycle, 2.3 can be removed and hard error.

I'm not sure if this is exactly how this is usually done, but take a look at how it was done for Kanidm for example. #357734

Please also move the package to by-name while you're at it (see dblsaiko@push-mputwrznzqyv).

I would also like to explore whether or not we can use settings for all dovecot config instead of some options combined with extraConfig - i.e. NixOS/rfcs#42.

#286184 is a start for that.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 9, 2025

by-name change... thanks for reminding me!

I would just set dovecot to continue being 2.3 for now, or set the module to use 2.3, and have it print a warning when used so people manually update to 2.4. Then in the next release cycle, 2.3 can be removed and hard error.

we should probably introduce a services.dovecot.package then... set it to 2.3 for 25.05 & change it to throw when 25.11 rolls around nvm that - 2.3 is still supported & I can't immediately find an EOL date... so ig we can just leave it be as an 'older version' for the time being and deprecate it when upstream does

I'm not sure if this is exactly how this is usually done, but take a look at how it was done for Kanidm for example. #357734

ah thanks! The first thing that came to mind for my was the postgres module, but same idea!

https://github.com/NixOS/nixpkgs/blob/20755fa05115c84be00b04690630cb38f0a203ad/nixos/modules/services/databases/postgresql.nix#L529C1-L561C64

#286184 is a start for that.

completely missed this one... I'll do some experimenting to see what's possible while keeping the module backwards-compatible for 2.3!

@dblsaiko
Copy link
Contributor

dblsaiko commented Mar 9, 2025

we should probably introduce a services.dovecot.package then

Right, that would be part of that.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 11, 2025

pulled in your commit & added 2.4 alongside 2.3... not 100% sold on the commit message but good enough for now

both packages build on my machine, haven't checked any functionality (notably the two things mentioned in the original PR description abt the openssl patch and that one hunk)

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 11, 2025
@dblsaiko
Copy link
Contributor

We'll also have to deal with the plugin packages because they are dovecot version dependent. Here's a proof of concept for pigeonhole: dblsaiko@push-sntmrkppzolr

And a (as of right now very non working) change adding the package option back dblsaiko@push-truxwztylsmm

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 12, 2025

uhhh is it preferable to have no broken commits? i.e. stuff that wouldn't eval?
cuz to do so I have to add all the plugin packages together with dovecot_2_4 in a single commit

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 12, 2025
@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 12, 2025

fixed both fts-xapian & exporter
flatcurve is apparently included in dovecot now, going by the original commit message in which it was added to nixpkgs (20a503b) and upstream docs (https://doc.dovecot.org/main/core/plugins/fts_flatcurve.html)

dovecot_pigeonhole_2_4 has a build failure i'm too tired to troubleshoot now, dovecot_pigeonhole_2_3 works fine

probably gonna have to rebase all of this later, just wanted to get the code on here in case you or anyone else has a clue wrt that build failure

@Jappie3 Jappie3 force-pushed the dovecot branch 2 times, most recently from f6622ba to 53b2306 Compare March 12, 2025 22:29
@github-actions github-actions bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 12, 2025
@dblsaiko
Copy link
Contributor

dblsaiko commented Mar 12, 2025

uhhh is it preferable to have no broken commits? i.e. stuff that wouldn't eval? cuz to do so I have to add all the plugin packages together with dovecot_2_4 in a single commit

I'd keep them separate, ordered after the commit for dovecot. (also it would eval, just not build, right?)

dovecot_pigeonhole_2_4 has a build failure i'm too tired to troubleshoot now, dovecot_pigeonhole_2_3 works fine

Huh, does it? Both of them built for me. I'll have to compare your branch to mine.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 13, 2025

I'd keep them separate, ordered after the commit for dovecot. (also it would eval, just not build, right?)

Sounds good, and yeah I think you're right - it would still eval.

Huh, does it? Both of them built for me. I'll have to compare your branch to mine.

Had a typo in all-packages.nix which pointed dovecot_2_4 to 2_3.nix, woops 😅
I'll fix that later today

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 13, 2025

dovecot_fts_xapian gave me a build failure on 1.9... upstream released 1.9.1 to fix this, but that tried to install itself to /nix/store for some reason...

the error in question
Making install in src
make[1]: Entering directory '/build/source/src'
make[2]: Entering directory '/build/source/src'
make[2]: Nothing to be done for 'install-exec-am'.
 /nix/store/vz67zaqyfln2jcqwar682kdlrxfx9vq1-coreutils-9.6/bin/mkdir -p '/nix/store/9gp6nrcynrps8bkyp9ql298h560vqcr5-dovecot-2.4.0/lib/dovecot/modules'
 /nix/store/ki4if6b0w5bqv8dc5lrjp8xm7wjy9dlf-bash-5.2p37/bin/bash ../libtool   --mode=install /nix/store/vz67zaqyfln2jcqwar682kdlrxfx9vq1-coreutils-9.6/bin/install -c   lib21_fts_xapian_plugin.la '/nix/st>
libtool: install: /nix/store/vz67zaqyfln2jcqwar682kdlrxfx9vq1-coreutils-9.6/bin/install -c .libs/lib21_fts_xapian_plugin.so /nix/store/9gp6nrcynrps8bkyp9ql298h560vqcr5-dovecot-2.4.0/lib/dovecot/modules/li>
install: cannot create regular file '/nix/store/9gp6nrcynrps8bkyp9ql298h560vqcr5-dovecot-2.4.0/lib/dovecot/modules/lib21_fts_xapian_plugin.so': Permission denied
make[2]: *** [Makefile:520: install-dovecot_moduleLTLIBRARIES] Error 1
make[2]: Leaving directory '/build/source/src'
make[1]: *** [Makefile:755: install-am] Error 2
make[1]: Leaving directory '/build/source/src'
make: *** [Makefile:504: install-recursive] Error 1

I just committed a horrible hack to fix that problem by modifying the Makefile with sed before installing, see 9d4bdfb

The problem is that around line 527 in src/Makefile it tries to do the following:

echo " $(MKDIR_P) '$(DESTDIR)$(dovecot_moduledir)'"; \
$(MKDIR_P) "$(DESTDIR)$(dovecot_moduledir)" || exit 1; \
echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(dovecot_moduledir)'"; \
$(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(dovecot_moduledir)"; \

with dovecot_moduledir pointing to /nix/store/[...]-dovecot/lib/dovecot/modules/lib21_fts_xapian_plugin.so instead of $out/lib/dovecot/modules (where $out is /nix/store/[...]-dovecot-ftl-xapian-1.9.1).

Following the trail upwards, it's clear that dovecot_moduledir comes from --with-moduledir: reading this back, I'm not sure if dovecot_moduledir comes from moduledir...

src/Makefile.in
417:dovecot_moduledir = @dovecot_moduledir@


$> rg "with-moduledir" configure -A8 -B2
20865:# Check whether --with-moduledir was given.
20866-if test ${with_moduledir+y}
20867-then :
20868-  withval=$with_moduledir; moduledir="$withval"
20869-else case e in #(
20870-  e) moduledir="\$(libdir)/dovecot"
20871-	 ;;
20872-esac
20873-fi


$> rg "with-moduledir" m4/dovecot.m4 -B2 -A6
250-AC_DEFUN([DC_DOVECOT_MODULEDIR],[
251-	AC_ARG_WITH(moduledir,
252:	[  --with-moduledir=DIR    Base directory for dynamically loadable modules],
253-		[moduledir="$withval"],
254-		[moduledir="\$(libdir)/dovecot"]
255-	)
256-	AC_SUBST(moduledir)

which we (nixpkgs) correctly set to $out, not ${dovecot}, using configureFlags:

  configureFlags = [
    "--with-dovecot=${dovecot}/lib/dovecot"
    "--with-moduledir=${placeholder "out"}/lib/dovecot/modules"
  ];

so my guess is that upstream somehow borked this flag and it gets mixed up somewhere with the ${dovecot} path from /nix/store? If that's the case, then we should probably take this upstream to get this resolved.

I want to stress that I am *not* a Make, autotools or m4 expert, this is just my two cents

pinging @Julm and @symphorien as maintainers of this package, maybe you guys have a better understanding of what's going on

@dblsaiko
Copy link
Contributor

Had a typo in all-packages.nix which pointed dovecot_2_4 to 2_3.nix, woops 😅

Oh, oops. I saw that, didn't say anything because I didn't want to be annoying reviewing every commit you make 2 seconds after you push them if there's something wrong with one, then forgot about it when I replied.

(speaking of, please squash that format commit into the parent, I just forgot to format it. my pretty alejandra-style formatting vs evil nixfmt 😔)

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 13, 2025

doing a diff on src/Makefile of 1.8.4 and 1.9.1, it's clear that settingsdir was changed to dovecot_moduledir (moduledir with my hacky workaround)... I'll take this upstream as I don't think this is currently implemented as intended

	@$(NORMAL_INSTALL)																	@$(NORMAL_INSTALL)
	@list='$(settings_LTLIBRARIES)'; test -n "$(settingsdir)" || list=; \									     |		@list='$(module_LTLIBRARIES)'; test -n "$(moduledir)" || list=; \
	list2=; for p in $$list; do \																list2=; for p in $$list; do \
	  if test -f $$p; then \																  if test -f $$p; then \
	    list2="$$list2 $$p"; \																    list2="$$list2 $$p"; \
	  else :; fi; \																		  else :; fi; \
	done; \																			done; \
	test -z "$$list2" || { \																test -z "$$list2" || { \
	  echo " $(MKDIR_P) '$(DESTDIR)$(settingsdir)'"; \											     |		  echo " $(MKDIR_P) '$(DESTDIR)$(moduledir)'"; \
	  $(MKDIR_P) "$(DESTDIR)$(settingsdir)" || exit 1; \											     |		  $(MKDIR_P) "$(DESTDIR)$(moduledir)" || exit 1; \
	  echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(settingsdir)'";  |		  echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(moduledir)'"; \
	  $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(settingsdir)"; \	     |		  $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(moduledir)"; \
	}																			}

grosjo/fts-xapian#191

@grosjo
Copy link

grosjo commented Mar 15, 2025

Hi @Jappie3 ,
Can you test with latest git ( grosjo/fts-xapian#191 )

Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Oct 14, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Oct 14, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
@Jappie3
Copy link
Contributor Author

Jappie3 commented Oct 14, 2025

I had to manually add pigeonhole to environment.systemPackages and had to enable sieve for lmtp

@hesiod if I understand things correctly, adding dovecot_pigeonhole to environment.systemPackages fixed your issues with the option sieve_extensions not being recognized?

I still think having to add the plugin to systemPackages is the result of #387642, but you're correct that this should've surfaced earlier if you're on unstable...

anyways, it's difficult to conditionally add dovecot_pigeonhole systemPackages automatically because there's not one master toggle, sieve support can be enabled at several places, so I don't see an easy way to solve this in the dovecot module... open for suggestions of course

@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Oct 14, 2025
@mweinelt
Copy link
Member

mweinelt commented Oct 18, 2025

diff --git a/nixos/modules/services/mail/dovecot.nix b/nixos/modules/services/mail/dovecot.nix
index 0795561b3050..80f521634b35 100644
--- a/nixos/modules/services/mail/dovecot.nix
+++ b/nixos/modules/services/mail/dovecot.nix
@@ -38,6 +38,8 @@ let
 
   cfg = config.services.dovecot2;
   dovecotPkg = cfg.package;
+  pigeonHolePkg = if lib.versionAtLeast cfg.package.version "2.4" then pkgs.dovecot_pigeonhole_2_4 else pkgs.dovecot_pigeonhole_2_3;
+  sievec = lib.getExe' pigeonHolePkg "sievec";
 
   baseDir = "/run/dovecot2";
   stateDir = "/var/lib/dovecot";
@@ -935,7 +937,7 @@ in
             else
               cp -p '${from}' '${stateDir}/sieve/${to}'
             fi
-            ${pkgs.dovecot_pigeonhole}/bin/sievec '${stateDir}/sieve/${to}'
+            ${sievec} '${stateDir}/sieve/${to}'
           '') cfg.sieve.scripts
         )}
         chown -R '${cfg.mailUser}:${cfg.mailGroup}' '${stateDir}/sieve'
@@ -947,11 +949,11 @@ in
           el:
           optionalString (el.before != null) ''
             cp -p ${el.before} ${stateDir}/imapsieve/before/${baseNameOf el.before}
-            ${pkgs.dovecot_pigeonhole}/bin/sievec '${stateDir}/imapsieve/before/${baseNameOf el.before}'
+            ${sievec} '${stateDir}/imapsieve/before/${baseNameOf el.before}'
           ''
           + optionalString (el.after != null) ''
             cp -p ${el.after} ${stateDir}/imapsieve/after/${baseNameOf el.after}
-            ${pkgs.dovecot_pigeonhole}/bin/sievec '${stateDir}/imapsieve/after/${baseNameOf el.after}'
+            ${sievec} '${stateDir}/imapsieve/after/${baseNameOf el.after}'
           ''
         ) cfg.imapsieve.mailbox}

@Jappie3
Copy link
Contributor Author

Jappie3 commented Oct 22, 2025

ah yeah, good point
applied

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Oct 28, 2025
@hesiod
Copy link
Contributor

hesiod commented Oct 29, 2025

I think I've identified one of the config issues that I was facing earlier. I need to add an empty line to the beginning of extraConfig, otherwise the config will contain a line like this sieve_plugins = userdb static { (where userdb static { is the beginning of extraConfig in this example) and dovecot will error out. So in conclusion, a newline seems to be missing somewhere before extraConfig.

@mweinelt
Copy link
Member

https://www.openwall.com/lists/oss-security/2025/10/29/4

@Jappie3
Copy link
Contributor Author

Jappie3 commented Oct 30, 2025

a newline seems to be missing somewhere before extraConfig

thanks, should be fixed

<         (concatMapAttrsStringSep "\n" (key: value: "  ${key} = ${value}") cfg.pluginSettings)
---
>         (concatMapAttrsStringSep "\n" (key: value: ''
>           ${key} = ${value}
>         '') cfg.pluginSettings)

and thanks for the CVE notice, I'll update the package to 2.4.2 & fix the merge conflict while I'm at it

edit: building with pcre2 (new dependency according to release notes, see dovecot/core@49ae6e7) under buildInputs throws

make[3]: *** No rule to make target '-L/nix/store/wqlkv0cjacnm7arca156p3d3vh0kdbmk-pcre2-10.46/lib', needed by 'test-regex'.  Stop.

without pcre2 it builds fine (tests also succeeded) - will look into this later, no clue if it actually needs to be added to buildInputs

Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 21, 2025
Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 and others added 6 commits November 21, 2025 23:28
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Request: dovecot 2.3.21.1 → 2.4.0