Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .azuredevops/test-job-template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ jobs:
echo "Microsoft ODBC Driver 18 installed successfully."
displayName: 'Install ODBC Driver 18 for SQL Server'
- bash: uv sync --extra dev --extra all
- bash: uv sync --extra all
name: install_PyRIT
- bash: df -all -h
name: disk_space_check
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer_setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ if [ ! -f "$HASH_FILE" ] || [ "$(cat $HASH_FILE)" != "$CURRENT_HASH" ]; then

# Install dependencies
uv pip install ipykernel
uv pip install -e ".[dev,all]"
uv sync --extra all
# Register the kernel with Jupyter
python -m ipykernel install --user --name=pyrit-dev --display-name="Python (pyrit-dev)"

Expand Down
7 changes: 5 additions & 2 deletions .github/instructions/test.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ applyTo: '**/tests/**'

Readable, maintainable tests. Reuse helpers from `conftest.py` and `mocks.py` in each test tier.

## General Rules

- Do NOT add `@pytest.mark.asyncio` — `asyncio_mode = "auto"` is configured project-wide so all async tests are discovered automatically.
- Use `AsyncMock` for async methods, `MagicMock` for sync.

## Test Tiers

Most tests should be unit tests. Integration and end-to-end tests are for testing that systems work toegether.
Expand All @@ -20,7 +25,6 @@ Most tests should be unit tests. Integration and end-to-end tests are for testin
- File naming: `test_[component].py`
- Group tests in classes prefixed with `Test`
- Use `@pytest.mark.usefixtures("patch_central_database")` on classes touching Central Memory
- Use `@pytest.mark.asyncio` and `AsyncMock` for async methods
- Reuse `tests/unit/mocks.py` helpers: `MockPromptTarget`, `get_sample_conversations`, `get_mock_target_identifier`, `openai_chat_response_json_dict`
- Key fixtures from `tests/unit/conftest.py`: `patch_central_database`, `sqlite_instance`
- No network calls should ever happen in unit tests, but file access is okay
Expand All @@ -43,7 +47,6 @@ Most tests should be unit tests. Integration and end-to-end tests are for testin

- Use `unittest.mock.patch` / `patch.object` — not `monkeypatch`
- Prefer `patch.object(instance, "method", new_callable=AsyncMock)` over broad module-path patches
- Use `AsyncMock` directly for async methods, `MagicMock` for sync
- Use `spec=ClassName` on mocks when you need to constrain to a real interface
- Use `side_effect` for sequences or raising exceptions
- For environment variables: `patch.dict("os.environ", {...})`
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/build_and_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ jobs:
**/uv.lock

- name: Install dev extras
run: uv sync --extra dev --extra all
run: uv sync --extra all
Comment thread
rlundeen2 marked this conversation as resolved.

- name: disk space
shell: bash
Expand Down Expand Up @@ -113,14 +113,14 @@ jobs:

# Install PyRIT with optional extras
- name: Install PyRIT with uv
# If the matrix extras is 'dev_all', then we install '.[dev,all]'
# otherwise just install the literal extras from the matrix
# If the matrix extras is 'dev_all', then we install all extras
# otherwise just install the default dependencies
shell: bash
run: |
if [ "${{ matrix.package_extras }}" = "dev_all" ]; then
uv sync --extra dev --extra all
uv sync --extra all
else
uv sync --extra dev
uv sync
fi

- name: Run unit tests
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/diff_cover.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
**/uv.lock
- name: Install dev extras
run: uv sync --extra dev --extra all
run: uv sync --extra all

- name: Run unit tests with coverage (fail under 78%)
run: make unit-test-cov-xml
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
**/uv.lock

- name: Install PyRIT with uv
run: uv sync --extra dev --extra all
run: uv sync --extra all

# Build the book
- name: Build the book
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/frontend_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ jobs:
- name: Install Python dependencies
run: |
cd ..
uv sync --extra dev
uv sync

- name: Install frontend dependencies
run: npm ci
Expand Down
2 changes: 1 addition & 1 deletion doc/contributing/10_release_process.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ This should print

This step is crucial to ensure that the new package works out of the box.

Create a new environment with the equivalent of `uv venv --python 3.11`. You do not need to test with multiple versions of python or environments, but this manual process can detect issues with the package. Install the built wheel file `uv pip install dist/pyrit-x.y.z-py3-none-any.whl[all,dev]`.
Create a new environment with the equivalent of `uv venv --python 3.11`. You do not need to test with multiple versions of python or environments, but this manual process can detect issues with the package. Install the built wheel file `uv pip install dist/pyrit-x.y.z-py3-none-any.whl[all]`.

Once the package is successfully installed in the new environment, run `uv pip show pyrit`. Ensure that the version matches the release `vx.y.z` and that the package is found under the site-packages directory of the environment, like `..\venv\Lib\site-packages`.

Expand Down
4 changes: 2 additions & 2 deletions doc/contributing/4_running_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

Testing plays a crucial role in PyRIT development. Ensuring robust tests in PyRIT is crucial for verifying that functionalities are implemented correctly and for preventing unintended alterations to these functionalities when changes are made to PyRIT.

For running PyRIT tests, you need to have `pytest` package installed, but if you've already set up your development dependencies with the command
`pip install -e .[dev]`, `pytest` should be included in that setup.
For running PyRIT tests, you need to have `pytest` package installed, but if you've already set up your development environment with
`uv sync`, `pytest` should be included in that setup.


## Running PyRIT test files
Expand Down
2 changes: 1 addition & 1 deletion doc/getting_started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Pre-configured container with JupyterLab. Best if you want to get started immedi
:link: ./install_local
**Custom Setup**

Install with pip, uv, or conda. Best if you need to integrate PyRIT into existing Python workflows, prefer lighter-weight installations, or want direct access from your system Python.
Install with pip or uv. Best if you need to integrate PyRIT into existing Python workflows, prefer lighter-weight installations, or want direct access from your system Python.
::::

:::::
Expand Down
10 changes: 0 additions & 10 deletions doc/getting_started/install_local.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,6 @@ Or with uv:
uv pip install pyrit
```

## Install with Conda

If you prefer conda for managing environments:

```bash
conda create -y -n pyrit python=3.11
conda activate pyrit
pip install pyrit
```

## Matching Notebooks to Your Version

```{important}
Expand Down
76 changes: 9 additions & 67 deletions doc/getting_started/install_local_dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ Set up a PyRIT development environment on your local machine.
**Development Version:** Contributor installations use the **latest development code** from the `main` branch, not a stable release. The notebooks in your cloned repository will match your code version.
```

## Option 1: uv (Recommended)
## Setup with uv

[uv](https://github.com/astral-sh/uv) is a fast Python package installer and resolver. We recommend it for PyRIT development.
[uv](https://github.com/astral-sh/uv) is a fast Python package installer and resolver that we use for PyRIT development.
Comment thread
rlundeen2 marked this conversation as resolved.

**Why uv?**
- **Much faster** than pip (10-100x faster dependency resolution)
- **Simpler** than conda/mamba for pure Python projects
- **Simpler** environment management for pure Python projects
- **Native Windows support** — no WSL required, although if using a devcontainer, WSL is recommended
- **Automatic virtual environment management**
- **Compatible with existing pyproject.toml**
Expand Down Expand Up @@ -49,20 +49,17 @@ Set up a PyRIT development environment on your local machine.
2. The repository includes a `.python-version` file that pins Python 3.12. Run:

```bash
uv sync --extra dev
uv sync
```

This command will:
- Create a `.venv` directory with a virtual environment
- Install Python 3.12 if not already available
- Install PyRIT in editable mode; `uv sync` by default installs in editable mode so no extra flag is necessary
- Install all dependencies including dev tools (pytest, black, ruff, etc.)
- Install all dependencies including dev tools (pytest, ruff, etc.) via the `dev` dependency group
- Create a `uv.lock` file for reproducible builds


If you are having problems getting pip to install, try this link for details here: [this post](https://stackoverflow.com/questions/77134272/pip-install-dev-with-pyproject-toml-not-working) for more details.


3. Verify Installation

```bash
Expand All @@ -80,11 +77,7 @@ VS Code should automatically detect the `.venv` virtual environment. If not:
3. Choose `.venv\Scripts\python.exe`

#### Running Jupyter Notebooks
You can create a Jupyter kernel by first installing ipykernel:
```bash
uv add --dev ipykernel
```
then, create the kernel using:
You can create a Jupyter kernel using:
```bash
uv run ipython kernel install --user --env VIRTUAL_ENV $(pwd)/.venv --name=pyrit-dev
```
Expand Down Expand Up @@ -141,8 +134,8 @@ uv sync --extra huggingface
# For all extras
uv sync --extra all

# Multiple extras
uv sync --extra dev --extra playwright --extra gcg
# Multiple extras (dev dependencies are always included automatically)
uv sync --extra playwright --extra gcg
```

### Development Workflow
Expand All @@ -165,7 +158,7 @@ uv sync
#### Running Code Formatters

```bash
uv run black .
uv run ruff format .
uv run ruff check --fix .
```

Expand All @@ -182,57 +175,6 @@ uv run pre-commit install
uv run pre-commit run --all-files
```

## Option 2: Conda

If you prefer conda for environment management, you can use it to create a Python environment and install PyRIT for development.

### Prerequisites

1. **Conda or Miniconda**: Download from [https://docs.anaconda.com/free/miniconda/](https://docs.anaconda.com/free/miniconda/)

2. **Git**: Clone the repository:
```bash
git clone https://github.com/microsoft/PyRIT
```

3. **Node.js and npm**: Required for building the frontend. Download [Node.js](https://nodejs.org/) (version 18+).

### Installation

1. Create a conda environment with the correct Python version:

```bash
conda create -y -n pyrit-dev python=3.12
conda activate pyrit-dev
```

2. Navigate to the cloned PyRIT directory and install in editable mode with dev dependencies:

```bash
pip install -e .[dev]
```

3. Verify installation:

```bash
pip show pyrit
```

### Jupyter Kernel Setup

Create a Jupyter kernel for the conda environment:

```bash
pip install ipykernel
python -m ipykernel install --user --name=pyrit-dev --display-name "PyRIT Dev"
```

Then start Jupyter:

```bash
jupyter lab
```

## Next Step: Configure PyRIT

After installing, configure your AI endpoint credentials.
Expand Down
10 changes: 7 additions & 3 deletions doc/getting_started/troubleshooting/jupyter_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,19 @@ uv pip list
Then activate it using

```bash
conda activate <env_name>
# Windows
.\.venv\Scripts\Activate.ps1

# macOS/Linux
source .venv/bin/activate
```

Next, you need to install the IPython kernel in the virtual environment.

Note: Jupyter and ipykernel are no longer installed by default with the base package. If you need to use Jupyter notebooks with PyRIT, you'll need to install these dependencies using one of the following methods:

1. Install with development dependencies: `uv pip install --extra dev`
2. Install with all optional dependencies: `pip install --extra all`
1. Install with development dependencies: `uv sync`
2. Install with all optional dependencies: `uv sync --extra all`
3. Install just the notebook dependencies manually: `uv pip install jupyter ipykernel`

After installing these dependencies, you can proceed with the kernel setup steps below.
Expand Down
45 changes: 1 addition & 44 deletions doc/getting_started/troubleshooting/local_dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Try regenerating the lock file:

```powershell
Remove-Item uv.lock
uv sync --extra dev
uv sync
```

### Module not found errors
Expand All @@ -32,46 +32,3 @@ PyRIT is installed in editable mode, so changes to the source code are immediate
```bash
uv sync --reinstall-package pyrit
```

## Conda Issues

### Conda environment not activating

Make sure you're using the correct activation command for your shell:

```bash
conda activate pyrit-dev
```

If `conda activate` doesn't work, you may need to initialize conda for your shell first:

```bash
conda init powershell # Windows PowerShell
conda init bash # macOS/Linux
```

Then restart your terminal.

### Package conflicts with conda and pip

When using conda environments with `pip install -e .[dev]`, you may see dependency conflicts. To resolve:

```bash
conda deactivate
conda remove -n pyrit-dev --all
conda create -y -n pyrit-dev python=3.12
conda activate pyrit-dev
pip install -e .[dev]
```

### Jupyter kernel not finding PyRIT

If Jupyter can't find your PyRIT installation, make sure the kernel is registered from within the activated conda environment:

```bash
conda activate pyrit-dev
pip install ipykernel
python -m ipykernel install --user --name=pyrit-dev --display-name "PyRIT Dev"
```

Then select the "PyRIT Dev" kernel in Jupyter or VS Code.
2 changes: 1 addition & 1 deletion doc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ Get started immediately with a pre-configured environment:
**Custom Setup**

Install PyRIT directly on your machine:
- ✅ Pip, uv, or conda
- ✅ Pip or uv
- ✅ Full Python environment control
- ✅ Easy integration with existing workflows
::::
Expand Down
7 changes: 5 additions & 2 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ COPY --chown=vscode:vscode doc/ /app/doc/
# Note: We use 'uv pip' because the devcontainer creates venv with uv (no pip by default)
RUN if [ "$PYRIT_SOURCE" = "pypi" ]; then \
echo "Installing PyRIT from PyPI version: $PYRIT_VERSION"; \
uv pip install --python /opt/venv/bin/python pyrit[dev,speech,opencv,fairness_bias,fastapi,playwright]==$PYRIT_VERSION; \
uv pip install --python /opt/venv/bin/python pyrit[speech,opencv,fairness_bias,fastapi,playwright]==$PYRIT_VERSION; \
elif [ "$PYRIT_SOURCE" = "local" ]; then \
echo "Installing PyRIT from local source"; \
uv pip install --python /opt/venv/bin/python -e .[dev,speech,opencv,fairness_bias,fastapi,playwright]; \
uv pip install --python /opt/venv/bin/python -e .[speech,opencv,fairness_bias,fastapi,playwright]; \
echo "Building frontend..."; \
/opt/venv/bin/python build_scripts/prepare_package.py; \
fi && \
Expand All @@ -77,6 +77,9 @@ RUN mkdir -p /app/notebooks /app/data /app/assets && \
RUN mkdir -p /home/vscode/.pyrit && \
chown -R vscode:vscode /home/vscode/.pyrit

# ipykernel and jupyter are in the dev dependency-group (not pip extras), so install them explicitly
RUN uv pip install --python /opt/venv/bin/python ipykernel jupyter

# Register the Jupyter kernel for the venv
RUN /opt/venv/bin/python -m ipykernel install --user --name pyrit --display-name "PyRIT"

Expand Down
Loading
Loading