Silent corrupted LoRA weight updates with Opacus 1.5.4 + PEFT 0.18.x: training appears normal but models are unusable

[MN-NR]

Environment

• opacus: 1.5.4
• peft: 0.18.1
• torch: 2.11.0+cu128
• Python: 3.10

What Happened

I ran DP-LoRA fine-tuning across 6 runs (2 models × 3 epsilon values, ~45 GPU-hours
total). Training appeared completely normal throughout:

• Loss decreased as expected
• Epsilon accumulated correctly
• No errors or warnings at any point
• Checkpoints saved successfully

All 6 models were unusable at inference time.

Minimal Reproduction

from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM
from opacus import PrivacyEngine
import torch, torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

model = AutoModelForCausalLM.from_pretrained("gpt2", dtype=torch.float32)
config = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.0,
    target_modules=["c_attn"], task_type="CAUSAL_LM")
model = get_peft_model(model, config)
optimizer = optim.AdamW(
    [p for p in model.parameters() if p.requires_grad], lr=1e-4)

dummy = torch.randint(0, 1000, (4, 32))
dl = DataLoader(TensorDataset(dummy), batch_size=4)

pe = PrivacyEngine(accountant='rdp')
model, optimizer, dl = pe.make_private_with_epsilon(
    module=model, optimizer=optimizer, data_loader=dl,
    target_epsilon=8.0, target_delta=1e-5, epochs=1, max_grad_norm=1.0,
)

param_name = [n for n in dict(model._module.named_parameters()) 
              if 'lora_B' in n][0]
before = dict(model._module.named_parameters())[param_name].clone()

model.train()
optimizer.zero_grad()
batch = dummy.cuda() if torch.cuda.is_available() else dummy
out = model(input_ids=batch, labels=batch)
out.loss.backward()
optimizer.step()

after = dict(model._module.named_parameters())[param_name]
print(f"Weight changed: {not torch.allclose(before.cpu(), after.cpu())}")
print(f"Max delta: {(after.cpu() - before).abs().max():.8f}")

With PEFT 0.18.1:
Weight changed: False
Max delta: 0.00000000

With PEFT 0.13.2:
Weight changed: True
Max delta: 0.00010004

The only way to detect the failure was:

1. Noticing identical utility collapse across all epsilon values
   simultaneously (genuine epsilon-dependent collapse would scale with ε)
2. Manually comparing saved adapter weight norms against known
   initialization values
3. Running inference and observing broken output

Suspected Root Cause

PEFT 0.18.x changed LoRA parameter naming from lora_A.weight to
lora_A.default.weight by introducing a ModuleDict with a named
adapter key. Opacus 1.5.4 was written before this change and does not
correctly handle the new parameter structure when registering per-sample
gradient hooks via GradSampleModule.

The exact mechanism varies by architecture as some models show near-zero
updates, others show partial corrupted updates ,but all produce
unusable models.

Workaround

pip install peft==0.13.2

Confirmed working across full multi-epoch training runs on both 2 models.

Suggestion

Opacus's LoRA+PEFT tutorial should specify a compatible PEFT version,
or add a version compatibility check that warns users when an
incompatible PEFT version is detected.

[MN-NR]

I discovered this bug during a research project and spent considerable
time debugging it. I'm interested in contributing to the fix.
Happy to collaborate or take a different part of the work if useful
just let me know if there's anything I can help with.

[immu4989]

The reproducer is concrete — useful. One diligence note: I checked the LoRA layer source at both PEFT v0.13.2 (your working version) and v0.18.0, and lora_A has been a nn.ModuleDict({adapter_name: nn.Linear(...)}) in both. So the access path lora_A..weight predates 0.18, and the named hypothesis ("PEFT 0.18.x renamed lora_A.weight → lora_A.default.weight") probably isn't the root cause. Something else changed between 0.13.2 and 0.18 that breaks Opacus's hook registration.

A few PEFT 0.18.0 changes look like more plausible suspects:

• FIX: Wrong coupling between requires_grad and the active adapter huggingface/peft#2765 ("FIX: Wrong coupling between requires_grad and the active adapter") — decoupled requires_grad=True from set_adapter(). The reproducer filters [p for p in model.parameters() if p.requires_grad] when building the optimizer and apparently gets a non-empty set, so requires_grad is True at optimizer-construction time — but this change might still affect when it becomes True relative to Opacus's add_hooks() walk, which keys off has_trainable_params / trainable_modules.
• FIX Multiple issues with target_parameters huggingface/peft#2710 ("FIX Multiple issues with target_parameters") — changed which modules get wrapped; could break Opacus's module-iteration assumptions.
• Module structure changes from Transformers v5 compatibility work.

@HuanyuZhang — happy to do a PEFT-version bisect (0.13.2 → 0.14 → 0.15 → ... → 0.18) using @MN-Noor's reproducer to pin down the exact PEFT commit, then trace which Opacus assumption it violates. That'd scope the fix — could land as anything from a requires_grad re-check after PEFT activation, to a doc side version-compat warning, to something more structural. Worth me running it?

@MN-Noor — the epsilon-collapse-across-all-values heuristic is a clean detection signal and probably worth turning into a PrivacyEngine validation pass once we land a fix.

[iden-kalemaj]

thank you @MN-Noor for reporting this issue and @immu4989 for offering to root cause.

Let's first land a commit which specifies a working version of the peft library in the LoRA+Opacus tutorial. @MN-Noor would you like to submit such commit?

Then ideally we can root cause and figure out how we can make opacus compatible with the latest version of peft. @MN-Noor and @immu4989 maybe you could collaborate to pin down the root cause first?

[immu4989]

Thanks iden-kalemaj , sounds like a clean split.
@MN-Noor : happy to start the bisect whenever you're ready. Rough plan: run your reproducer against clean installs of 0.13.2 → 0.14 → ... → 0.18 to narrow to the breaking minor, then git bisect within that minor for the exact commit. I can take the first pass and post findings here, or we can split it however suits you. Let me know your preference.

[MN-NR]

@iden-kalemaj I'll open a PR to submit the tutorial commit.
@immu4989 the bisect plan sounds good. Happy for you to take the first
pass (0.13.2 → 0.18 across minor versions) using my reproducer, then
we can git bisect within the breaking minor together. I'll stay close
and can run anything locally on my end (RTX 5090, CUDA 12, Python 3.10)
if you need a second machine to confirm results. Let me know what you find.

[immu4989]

First pass bisect results — surprise:

PEFT	Weight changed	Max delta
0.13.2	True	0.00010000
0.14.0	True	0.00010000
0.15.0	True	0.00010000
0.16.0	True	0.00010000
0.17.0	True	0.00010000
0.18.0	True	0.00010000
0.18.1	True	0.00010000

Env: macOS, CPU only, Python 3.12, opacus 1.5.4, transformers 4.57.6, torch 2.12.0. Used your reproducer verbatim, only diff: a try/except wrapping from_pretrained(..., dtype=torch.float32) falling back to torch_dtype= for cross-version compat (the dtype= path was taken every iteration). All 7 versions match your working baseline (you reported 0.13.2 → max_delta=0.00010004). The bug doesn't fire on CPU.

Points to the bug being CUDA-path-specific rather than a pure logic regression in PEFT or Opacus. Two supporting observations:

1. Your reproducer moves only batch to CUDA via dummy.cuda() — the model stays on CPU as written. In your actual failing runs (the 45 GPU-hours), was the model also moved to CUDA (via explicit .cuda(), accelerate, or device_map)? That setup delta might be where the real variable lives.
2. max_delta = 0.00010000 is identical across every PEFT version on CPU — gradient path is intact. Whatever breaks on GPU is bypassed here.

@MN-Noor — if you can run the same sweep on your RTX 5090 with the exact CUDA setup that produced your original failures, we can either confirm CUDA specificity, identify which sub version is the breaking one, or discover that the device setup itself is the variable.

@iden-kalemaj — flagging since this changes the diagnosis. If reproduces only on CUDA, the eventual fix is probably narrower than a general PEFT-version compat layer.

[MN-NR]

@iden-kalemaj I have pushed the tutorial commit. Let me know if you need anything else from my side.

[iden-kalemaj]

@MN-NR following the analysis of @immu4989, would you mind checking if the error still appears with peft = 0.18 if you add model.cuda() before model.train()?

[immu4989]

Tested as you suggested. peft 0.18.1 works end to end with opacus when model.to(device) happens before get_peft_model() , the silent corruption pattern doesn't fire.

Side by side from a 50-step toy training run on Kaggle T4 (GPT-2-small + LoRA r=16, target_modules=['c_attn'], same code path):

Variant	transformers	peft	opacus	Max |Δlora_B| (50 steps)
Old pins	4.57.6	0.13.2	1.6.0	0.00319205
Latest	5.0.0	0.18.1	1.6.0	0.00319205

Same max|Δ| to 8 decimal places — both versions follow the identical computational path when the model is on CUDA before PEFT wrapping. With either pinning, weights move; the privacy engine attaches cleanly.

Looking at @MN-NR's original reproducer, the model is never explicitly moved to CUDA — only batch.cuda() is called inside the training loop. With PEFT 0.18+, accelerate-style lazy device handling can leave parts of the model on CPU when opacus walks add_hooks(), which appears to be what triggers the silent corruption. Calling model.to(device) before get_peft_model() settles all parameters on CUDA up front, so the hook walk finds them as expected.

Implication: the root cause is device placement ordering, not the peft version. A doc side fix (showing the correct ordering pattern in the LoRA tutorial) addresses the same surface as pinning peft, without locking users to an older PEFT release.

If that lines up with what you're seeing on your end, #827 can proceed with peft 0.18 (no version pin). PR #826's tutorial pin might be replaced or supplemented by a device ordering note, happy to leave that call to you and @MN-NR.

[MN-NR]

@iden-kalemaj Tested with peft==0.18.0 and model.cuda() added before
model.train() on RTX 5090 (CUDA 12.8, PyTorch 2.11.0, Python 3.10):

Weight changed: True
Max delta: 0.00010000

Bug does not reproduce. Consistent with @immu4989's CPU bisect results.

One warning worth flagging:
"Full backward hook is firing when gradients are computed with respect
to module outputs since no inputs require gradients."

Not sure if this is related to the original hook registration issue —
flagging in case it helps narrow the root cause.

[immu4989]

Thanks @MN-NR , three independent confirmations now (your RTX 5090 single step, my Mac CPU bisect across peft 0.13.2 → 0.18.1, my Kaggle T4 50 step run on peft 0.18.1). Same max|Δ| consistent with one optimizer step at lr=1e-4. Root cause settled as device-placement ordering.

On the Full backward hook warning , that fires in my Kaggle runs as well (both the peft 0.13.2 and 0.18.1 variants), including the v1 starter that produced a clearly working result. It's PyTorch's standard caveat about backward hooks attached to modules whose inputs don't require gradients (token IDs upstream of LoRA, plus the frozen base model). The hook is doing its job , that's informational, not the silent corruption signal. I think it's orthogonal to #820, but worth a separate triage if anyone wants to look at suppressing it.

[immu4989]

Now that the root cause is settled, one more piece worth considering before this closes: the failure mode is silent by construction. Loss decreases, the accountant ticks, checkpoints save. Nothing in the training loop tells the user their weights never moved. The docs fix in #830 helps people who read the tutorial, but it can't catch the next variant of this bug class (a different library interaction, optimizer built over stale params, and so on).

Proposal: an opt in check_weight_updates: bool = False kwarg on make_private / make_private_with_epsilon. When enabled, DPOptimizer.step snapshots lightweight fingerprints of trainable params (first ~1K elements per param) for the first few non-skipped steps, compares after original_optimizer.step(), and raises a UserWarning if no trainable parameter changed, pointing at the known causes (device-placement ordering, stale optimizer params, lr=0). After ~3 clean steps it self disables, so steady state overhead is zero.

A few design notes from reading the code:

• make_private already forwards **kwargs to the optimizer constructor, so plumbing is minimal
• attach_step_hook isn't usable for this: it's single-slot (already consumed by the accountant) and fires in pre_step, before weights move
• Only requires_grad=True params are checked, so LoRA-style models with frozen bases don't false-positive
• Poisson skipped steps don't consume a check
• Default False keeps it behavior-neutral; whether it should ever become default-on is your call

There's precedent for this category in the repo already: make_private raises at setup time when module params differ from optimizer params (the ModuleValidator.fix()-after-optimizer case). This extends the same protection to runtime, where setup-time identity checks can't see device-placement problems.

Rough size: ~40 LOC in optimizer.py, docstring updates, ~60 LOC of tests in privacy_engine_test.py following the existing test_validator_weight_update_check pattern (no warning on healthy training, warning at lr=0, no false positive on frozen-base models, skipped steps don't consume checks).

Happy to send the PR once #830 lands if this shape works for you. Also fine to adjust the API (standalone utility, different kwarg name, check count knob) if you'd prefer.