FEAT add crescendo persona scenario techniques (movie director, history lecture, journalist interview)#1677
Merged
rlundeen2 merged 1 commit intomicrosoft:mainfrom May 1, 2026
Conversation
…ry lecture, journalist interview) Builds on PR microsoft#1665 by adding three persona-driven YAML variants of single_turn_crescendo, wired through a new pyrit/setup/initializers/components/scenarios.py initializer per maintainer guidance on PR microsoft#1657. No new attack primitives. Refs microsoft#388 Signed-off-by: precognitivem0nk <rextedgorman@gmail.com>
rlundeen2
approved these changes
May 1, 2026
Contributor
|
This looks great, ty @precognitivem0nk! We're building some easier benchmarks soon, and I'm stoked to see how these all compare |
Contributor
Author
|
Thanks for the quick review and merge. Looking forward to the new benchmarks. Happy to add more persona variants when the comparison surfaces gaps. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds three persona-driven YAML variants of the
single_turn_crescendotechnique landed in #1665 (movie director, history lecture, journalist interview), plus a newpyrit/setup/initializers/components/scenarios.pyinitializer (ScenarioTechniqueInitializer, registry namescenario_technique, execution_order 3) that registers them.Why
Following @rlundeen2's guidance on the closed PR #1657: ship YAML variants on top of #1665 instead of a new attack primitive. The scenario-techniques pattern is more flexible and lets dataset diversity drive crescendo coverage without new code paths.
Refs #388.
Files
pyrit/datasets/executors/red_teaming/crescendo_movie_director.yaml(new, the persona @rlundeen2 specifically suggested)pyrit/datasets/executors/red_teaming/crescendo_history_lecture.yaml(new)pyrit/datasets/executors/red_teaming/crescendo_journalist_interview.yaml(new)pyrit/setup/initializers/components/scenarios.py(new, slots into the existing target/scorer/scenario_technique 1-2-3 chain)pyrit/setup/initializers/components/__init__.py(export)pyrit/setup/initializers/__init__.py(re-export)tests/unit/setup/test_scenarios_initializer.py(new, 26 tests covering YAML loading, schema, persona distinctness, registration, and the get_default_adversarial_target fallback)Schema
Each persona YAML uses the same schema as
crescendo_simulated.yamlfrom #1665: parameters[objective, max_turns],data_type: text, and avalueblock templated on{{ objective }}and{{ max_turns }}.num_turnsdefaults to 3, matchingSeedSimulatedConversation's canonical default.Opt-in by design
The personas live behind the new
ScenarioTechniqueInitializerrather than in the coreSCENARIO_TECHNIQUESlist, matching @rlundeen2's note on #1657 ("I might put them in an additional scenario initializer vs the core"). Users opt in by includingscenario_techniquein their initializer config; out of the box, no Scenario auto-loads them.crescendo_simulatedfrom #1665 remains the always-available default.Verification
pytest tests/unit/setup/: 207 passedpytest tests/unit/scenario/: 445 passedpytest tests/unit/: 7075 passed, 102 skipped (env-gated live targets), 0 failuresruff check,ruff format --check: clean on all new and modified filesSeedPrompt.from_yaml_file, renders cleanly withrender_template_value(objective=..., max_turns=3), no Jinja artifacts leftScenarioTechniqueInitializer.initialize_async()in isolation registers all 3 personas plus the core specsNotes
upstream/mainby 5 unrelated commits. Trial merge is clean and conflict-free. Happy to rebase if preferred.