Add SlaClip adaptive clipping optimizer for DP-SGD (algorithm of ICML 2026 spotlight paper)

[ZsyRock]

Summary

This PR adds SlaClip as a self-contained research prototype under
research/slaclip.

SlaClip uses the norm budget left unused by clipping to jointly release a
private Slack Indicator—a noisy, binned estimate of the gradient-norm CDF—
together with the standard DP-SGD gradient.

The extended per-sample vector preserves the original L2 norm bound. Under the
same sampling rule, normalization, clipping threshold, and noise multiplier,
the gradient and Slack Indicator are therefore released as one Gaussian
mechanism with the same per-step privacy-accounting parameters as vanilla
DP-SGD.

Usage

The implementation exposes two composable steps:

1. SlaClipDPOptimizer jointly releases the DP gradient and private Slack
   Indicator. Without a controller, the clipping threshold remains fixed.
2. SlaClipController consumes the released indicator and applies equations
   (28)-(30) from the paper to adapt the clipping threshold as post-processing.

Using both components implements the paper's SlaClip method. Users may also use
the first component independently to obtain private CDF information for other
DP-safe post-processing methods.

The recommended entry point is SlaClipPrivacyEngine, a research-scoped
subclass of Opacus PrivacyEngine. Users can call its inherited
make_private() or make_private_with_epsilon() APIs directly; model
wrapping, data-loader preparation, secure RNG selection, optimizer
construction, and privacy-accountant hooks are handled automatically. Full integration instructions, examples,
privacy assumptions, parameters, automatic K selection, and limitations are
provided in research/slaclip/README.md.

Scope

This PR:

• is self-contained under research/slaclip;
• does not modify PrivacyEngine or other Opacus core APIs;
• supports the standard non-distributed DPOptimizer path;
• does not add benchmark or end-to-end training scripts;
• is intended as an experimental research prototype.

Tests

python -m pytest -q research/slaclip

Result:

17 passed, 12 subtests passed

The tests cover:

• the SlaClipPrivacyEngine entry point;
• inherited make_private_with_epsilon() support;
• automatic accountant-hook attachment;
• explicit rejection of unsupported clipping modes;
• automatic K selection from equation (36);
• equations (7)-(8) and the extended-gradient norm bound;
• agreement of the first d coordinates with native DPOptimizer;
• indicator-only operation;
• the paper clipping controller and threshold bounds;
• empty Poisson batches;
• a single accountant event for the joint release.

Black, flake8, isort, and git diff --check also pass.

Reference

This implementation is based on:

SlaClip: Gradient Norm Slacks can be Indicator for Adaptive Clipping in DP-SGD, ICML 2026 Spotlight.

[meta-cla]

Hi @ZsyRock!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[ZsyRock]

Hi maintainers, thanks for taking a look. It seems the workflows are still awaiting approval and the PR has not been imported yet. Please let me know if there is anything I should do on my side.

[iden-kalemaj]

Hi @ZsyRock, for emerging research related to DP-SGD we recommend committing changes to the research folder. Once a method becomes established enough in the community, we can move it into the main folder. This prevents the core Opacus code from becoming difficult to maintain with the addition of new methods.

[meta-codesync]

This pull request has been imported. If you are a Meta employee, you can view this in D110207613. (Because this pull request was imported automatically, there will not be any future comments.)

[ZsyRock]

Thanks for the guidance. I revised the PR to move SlaClip into research/slaclip and removed the changes to the core PrivacyEngine and optimizer selection paths. The updated version is self-contained under the research folder, includes a README plus lightweight tests, and no longer modifies Opacus core APIs.

I verified the research tests locally with:

conda run -n opacus python -m pytest research/slaclip -q

which passes with 5 passed.

[iden-kalemaj]

Thank you for the contribution @ZsyRock, please consider also adding a module extending PrivacyEngine that people can use as an entrypoint for accessing your method, see example here. I left a few other comments on the code as well.

[ZsyRock]

Thanks for the review. I have updated the implementation and documentation
based on the feedback.

The PR is now a self-contained research prototype under research/slaclip
with no Opacus core changes. The README includes the methodology, privacy
argument, two composable usage modes, PrivacyEngine integration instructions,
automatic K selection, parameters, limitations, tests, and citation.

The branch has also been rebased onto the latest main, with one commit and
four changed files. The GitHub Actions workflows currently show
action_required; it appears they need approval before the jobs can start.

Could you please re-review the updated PR and approve the workflow runs when
convenient?

[ZsyRock]

Hi @ZsyRock, for emerging research related to DP-SGD we recommend committing changes to the research folder. Once a method becomes established enough in the community, we can move it into the main folder. This prevents the core Opacus code from becoming difficult to maintain with the addition of new methods.

Thanks for the suggestion. I added a research-scoped
SlaClipPrivacyEngine entry point in commit 170ccdd.

Users can now call the standard make_private() or
make_private_with_epsilon() APIs directly. The entry point automatically
constructs SlaClipDPOptimizer and preserves Opacus model wrapping,
data-loader preparation, secure RNG selection, and privacy-accountant hook
attachment.

Omitting the controller enables the indicator-only mode, while passing
SlaClipController enables the complete paper SlaClip method. I also updated
the README with both usage modes and added integration tests.

The implementation remains entirely under research/slaclip and does not
modify Opacus core APIs.

[ZsyRock]

Hi @iden-kalemaj, just a gentle follow-up on PR #824.
I addressed the previous review feedback in the current head 170ccdd: the implementation is now fully scoped under research/slaclip, the README has been expanded, and a research-scoped SlaClipPrivacyEngine entry point has been added. Both review threads are resolved, and the branch is up to date with main.
The CLA and Meta import checks have passed, but the GitHub Actions workflows have not run for the current commit. Could you please re-review the updated PR and approve the workflows when convenient? Please let me know if any further changes are needed. Thanks!