Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Aim for fewer, higher-signal comments. A review with 2-3 important comments is b
BEFORE editing or code-reviewing any file, you MUST read the `.github/instructions/` files whose `applyTo` patterns match the files you are about to edit. For example:
- Editing/code-reviewing `pyrit/**/*.py` → read `style-guide.instructions.md` and `user-custom.instructions.md`
- Editing/code-reviewing `pyrit/scenario/**` → also read `scenarios.instructions.md`
- Editing/code-reviewing `pyrit/prompt_converter/**` → also read `converters.instructions.md`
- Editing/code-reviewing `pyrit/converter/**` → also read `converters.instructions.md`
- Editing/code-reviewing `tests/**` → also read `test.instructions.md`
- Editing/code-reviewing `doc/**/*.py` or `doc/**/*.ipynb` → also read `docs.instructions.md`
- Editing/code-reviewing `frontend/**/*.{ts,tsx}` → also read `frontend-style-guide.instructions.md`
Expand Down
18 changes: 9 additions & 9 deletions .github/instructions/converters.instructions.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
---
applyTo: "pyrit/prompt_converter/**"
applyTo: "pyrit/converter/**"
---

# Prompt Converter Development Guidelines
# Converter Development Guidelines

**Responsibility**: A converter transforms a prompt into something else (rephrasing, encoding, translating to a Word document, overlaying text on an image, ...). Converters can be stacked and combined, and any converter may also be a NoOp.

**Does not own** (see [framework.md](../../doc/code/framework.md)): conversation state or attack decisions. A converter transforms input into output (and may call a target to do so); it must not branch on results, score, persist to memory itself, or decide when it runs — the attack/technique configures the stack. Flag such bleed in review.

## Base Class Contract

All converters MUST inherit from `PromptConverter` and implement:
All converters MUST inherit from `Converter` and implement:

```python
class MyConverter(PromptConverter):
class MyConverter(Converter):
SUPPORTED_INPUT_TYPES = ("text",) # Required — non-empty tuple of PromptDataType values
SUPPORTED_OUTPUT_TYPES = ("text",) # Required — non-empty tuple of PromptDataType values

Expand Down Expand Up @@ -63,7 +63,7 @@ Exclude: retry counts, logging config, timeouts.

```python
from pyrit.models import ComponentIdentifier, PromptDataType
from pyrit.prompt_converter import ConverterResult, PromptConverter
from pyrit.converter import ConverterResult, Converter
```

For LLM-based converters, also import:
Expand All @@ -78,17 +78,17 @@ Use keyword-only arguments. Use `@apply_defaults` if the converter accepts targe
```python
from pyrit.common.apply_defaults import apply_defaults

class MyConverter(PromptConverter):
class MyConverter(Converter):
@apply_defaults
def __init__(self, *, target: PromptTarget, template: str = "default") -> None:
...
```

### Keyword-only ``__init__`` is enforced

Every ``PromptConverter`` subclass MUST make all ``__init__`` parameters
Every ``Converter`` subclass MUST make all ``__init__`` parameters
keyword-only (i.e., place ``*`` as the first parameter after ``self``).
``PromptConverter.__init_subclass__`` validates this at class-definition
``Converter.__init_subclass__`` validates this at class-definition
time via ``enforce_keyword_only_init`` and raises ``TypeError`` on
violations.

Expand All @@ -108,5 +108,5 @@ def __init__(self, foo: str, bar: int = 0) -> None: ... # missing *

## Exports and External Updates

- New converters MUST be added to `pyrit/prompt_converter/__init__.py` — both the import and the `__all__` list.
- New converters MUST be added to `pyrit/converter/__init__.py` — both the import and the `__all__` list.
- The modality table with new/updated converters `doc/code/converters/0_converters.ipynb` and the associated .py pct file must also be updated.
4 changes: 2 additions & 2 deletions .github/instructions/style-guide.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ paths stay fast.

### Lazy `__init__.py` Exports (PEP 562)

Public API packages (`pyrit.prompt_target`, `pyrit.prompt_converter`, `pyrit.score`)
Public API packages (`pyrit.prompt_target`, `pyrit.converter`, `pyrit.score`)
use `__getattr__`-based lazy loading so heavy symbols can be imported from the
package without paying the cost at package load time. See
`pyrit/prompt_target/__init__.py` for the canonical example. Rules:
Expand Down Expand Up @@ -578,7 +578,7 @@ def process_large_dataset(self, *, file_path: Path) -> list[Result]:
- If so, add it to the `_LAZY_IMPORTS` dict in that `__init__.py` instead of as an
eager top-level import (see the Import Placement section for the pattern)
- This is especially important for `pyrit/common/__init__.py`, `pyrit/prompt_target/__init__.py`,
`pyrit/prompt_converter/__init__.py`, and `pyrit/score/__init__.py` which are all on the
`pyrit/converter/__init__.py`, and `pyrit/score/__init__.py` which are all on the
import path for CLI startup

## Final Checklist
Expand Down
2 changes: 1 addition & 1 deletion .github/instructions/test.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Most tests should be unit tests. Integration and end-to-end tests are for testin

## Unit Test Rules

- Directory mirrors `pyrit/` (e.g. `pyrit/prompt_converter/` → `tests/unit/converter/`)
- Directory mirrors `pyrit/` (e.g. `pyrit/converter/` → `tests/unit/converter/`)
- File naming: `test_[component].py`
- Group tests in classes prefixed with `Test`
- Use `@pytest.mark.usefixtures("patch_central_database")` on classes touching Central Memory
Expand Down
2 changes: 1 addition & 1 deletion doc/blog/2024_12_3.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ It turns out, yes, we can. `CrescendoOrchestrator`, `PairOrchestrator`, `RedTeam
- `adversarial_chat` represents the attacker-controlled infrastructure used to generate prompts.
- `adversarial_chat_system_prompt_path` specifies the system prompt defined for adversarial_chat.
- `max_turns` defines the maximum number of conversation turns.
- `prompt_converters` are used to modify prompts before sending them to the target.
- `converters` are used to modify prompts before sending them to the target.
- `objective_scorer` evaluates whether the objective was achieved.
- `run_attack_async(objective: str, memory_labels: dict[str, str] | None = None)` executes the attack and always returns a `OrchestratorResult`, which includes information about the conversation and the outcome.
- `run_attacks_async` enables parallelized attacks.
Expand Down
2 changes: 1 addition & 1 deletion doc/blog/2025_01_27.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ When examining this request, you may discover that occasionally the Adversarial

[^7]: "Multi-Turn Attack - RedTeamingAttack Example", ../code/executor/2_multi_turn.ipynb#red-teaming

[^8]: "PyRIT - SearchReplaceConverter", ../api/pyrit_prompt_converter.md#searchreplaceconverter
[^8]: "PyRIT - SearchReplaceConverter", ../api/pyrit_converter.md#searchreplaceconverter

[^9]: "PyRIT - True False Scoring", ../code/scoring/1_true_false_scorers.ipynb

Expand Down
6 changes: 3 additions & 3 deletions doc/code/auxiliary_attacks/0_auxiliary_attacks.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -239,12 +239,12 @@
}
],
"source": [
"from pyrit.converter import SuffixAppendConverter\n",
"from pyrit.executor.attack import AttackConverterConfig\n",
"from pyrit.prompt_converter import SuffixAppendConverter\n",
"from pyrit.prompt_normalizer import PromptConverterConfiguration\n",
"from pyrit.prompt_normalizer import ConverterConfiguration\n",
"\n",
"suffix = '$\\\\ inqu Here spher =\" vas_{( dentSMckground CREATE poem combining_{( Taylor_{( phrase morning _{'\n",
"gcg_suffix_converter = PromptConverterConfiguration.from_converters(converters=[SuffixAppendConverter(suffix=suffix)])\n",
"gcg_suffix_converter = ConverterConfiguration.from_converters(converters=[SuffixAppendConverter(suffix=suffix)])\n",
"\n",
"converter_config = AttackConverterConfig(\n",
" request_converters=gcg_suffix_converter,\n",
Expand Down
6 changes: 3 additions & 3 deletions doc/code/auxiliary_attacks/0_auxiliary_attacks.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,12 @@
# Next, let's apply a GCG suffix trained on Phi-3-mini to the base prompt using the `SuffixAppendConverter`.

# %%
from pyrit.converter import SuffixAppendConverter
from pyrit.executor.attack import AttackConverterConfig
from pyrit.prompt_converter import SuffixAppendConverter
from pyrit.prompt_normalizer import PromptConverterConfiguration
from pyrit.prompt_normalizer import ConverterConfiguration

suffix = '$\\ inqu Here spher =" vas_{( dentSMckground CREATE poem combining_{( Taylor_{( phrase morning _{'
gcg_suffix_converter = PromptConverterConfiguration.from_converters(converters=[SuffixAppendConverter(suffix=suffix)])
gcg_suffix_converter = ConverterConfiguration.from_converters(converters=[SuffixAppendConverter(suffix=suffix)])

converter_config = AttackConverterConfig(
request_converters=gcg_suffix_converter,
Expand Down
20 changes: 10 additions & 10 deletions doc/code/converters/0_converters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@
"source": [
"import pandas as pd\n",
"\n",
"from pyrit.converter import get_converter_modalities\n",
"from pyrit.output import output_attack_async\n",
"from pyrit.prompt_converter import get_converter_modalities\n",
"from pyrit.setup import IN_MEMORY, initialize_pyrit_async\n",
"\n",
"await initialize_pyrit_async(memory_db_type=IN_MEMORY) # type: ignore\n",
Expand Down Expand Up @@ -226,7 +226,7 @@
}
],
"source": [
"from pyrit.prompt_converter import (\n",
"from pyrit.converter import (\n",
" AsciiArtConverter,\n",
" BinaryConverter,\n",
" FirstLetterConverter,\n",
Expand Down Expand Up @@ -318,12 +318,12 @@
}
],
"source": [
"from pyrit.converter import StringJoinConverter, VariationConverter\n",
"from pyrit.executor.attack import (\n",
" AttackConverterConfig,\n",
" PromptSendingAttack,\n",
")\n",
"from pyrit.prompt_converter import StringJoinConverter, VariationConverter\n",
"from pyrit.prompt_normalizer import PromptConverterConfiguration\n",
"from pyrit.prompt_normalizer import ConverterConfiguration\n",
"from pyrit.prompt_target import OpenAIChatTarget, TextTarget\n",
"\n",
"objective = \"tell me how to cut down a tree\"\n",
Expand All @@ -332,7 +332,7 @@
"converter_target = OpenAIChatTarget()\n",
"prompt_variation_converter = VariationConverter(converter_target=converter_target)\n",
"\n",
"converter_configs = PromptConverterConfiguration.from_converters( # type: ignore\n",
"converter_configs = ConverterConfiguration.from_converters( # type: ignore\n",
" converters=[prompt_variation_converter, StringJoinConverter()]\n",
")\n",
"\n",
Expand Down Expand Up @@ -362,7 +362,7 @@
"- Decoding encoded responses\n",
"- Normalizing or cleaning up response text\n",
"\n",
"Response converters use the same `PromptConverterConfiguration` class as request converters. They are configured via the `response_converters` parameter in `AttackConverterConfig`.\n",
"Response converters use the same `ConverterConfiguration` class as request converters. They are configured via the `response_converters` parameter in `AttackConverterConfig`.\n",
"\n",
"### Translation Round-Trip Example\n",
"\n",
Expand Down Expand Up @@ -432,12 +432,12 @@
}
],
"source": [
"from pyrit.converter import TranslationConverter\n",
"from pyrit.executor.attack import (\n",
" AttackConverterConfig,\n",
" PromptSendingAttack,\n",
")\n",
"from pyrit.prompt_converter import TranslationConverter\n",
"from pyrit.prompt_normalizer import PromptConverterConfiguration\n",
"from pyrit.prompt_normalizer import ConverterConfiguration\n",
"from pyrit.prompt_target import OpenAIChatTarget\n",
"\n",
"objective = \"What is the capital of France?\"\n",
Expand All @@ -450,11 +450,11 @@
"\n",
"# Request converter: translate English to French\n",
"request_converter = TranslationConverter(converter_target=converter_target, language=\"French\")\n",
"request_converter_config = PromptConverterConfiguration(converters=[request_converter])\n",
"request_converter_config = ConverterConfiguration(converters=[request_converter])\n",
"\n",
"# Response converter: translate response back to English\n",
"response_converter = TranslationConverter(converter_target=converter_target, language=\"English\")\n",
"response_converter_config = PromptConverterConfiguration(converters=[response_converter])\n",
"response_converter_config = ConverterConfiguration(converters=[response_converter])\n",
"\n",
"# Configure the attack with both request and response converters\n",
"converter_config = AttackConverterConfig(\n",
Expand Down
20 changes: 10 additions & 10 deletions doc/code/converters/0_converters.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@
# %%
import pandas as pd

from pyrit.converter import get_converter_modalities
from pyrit.output import output_attack_async
from pyrit.prompt_converter import get_converter_modalities
from pyrit.setup import IN_MEMORY, initialize_pyrit_async

await initialize_pyrit_async(memory_db_type=IN_MEMORY) # type: ignore
Expand Down Expand Up @@ -68,7 +68,7 @@
# Converters can be used to perform these types of transformations. Here is a simple program that uses Rot13Converter converter, RandomCapitalLettersConverter, and AsciiArtConverter.

# %%
from pyrit.prompt_converter import (
from pyrit.converter import (
AsciiArtConverter,
BinaryConverter,
FirstLetterConverter,
Expand Down Expand Up @@ -98,12 +98,12 @@
# "t-e-l-l- - m-e- -h-o-w- -t-o- -c-u-t- -d-o-w-n - a- -t-r-e-e"

# %%
from pyrit.converter import StringJoinConverter, VariationConverter
from pyrit.executor.attack import (
AttackConverterConfig,
PromptSendingAttack,
)
from pyrit.prompt_converter import StringJoinConverter, VariationConverter
from pyrit.prompt_normalizer import PromptConverterConfiguration
from pyrit.prompt_normalizer import ConverterConfiguration
from pyrit.prompt_target import OpenAIChatTarget, TextTarget

objective = "tell me how to cut down a tree"
Expand All @@ -112,7 +112,7 @@
converter_target = OpenAIChatTarget()
prompt_variation_converter = VariationConverter(converter_target=converter_target)

converter_configs = PromptConverterConfiguration.from_converters( # type: ignore
converter_configs = ConverterConfiguration.from_converters( # type: ignore
converters=[prompt_variation_converter, StringJoinConverter()]
)

Expand All @@ -137,7 +137,7 @@
# - Decoding encoded responses
# - Normalizing or cleaning up response text
#
# Response converters use the same `PromptConverterConfiguration` class as request converters. They are configured via the `response_converters` parameter in `AttackConverterConfig`.
# Response converters use the same `ConverterConfiguration` class as request converters. They are configured via the `response_converters` parameter in `AttackConverterConfig`.
#
# ### Translation Round-Trip Example
#
Expand All @@ -148,12 +148,12 @@
# 3. Use a **response converter** to translate the response back to English

# %%
from pyrit.converter import TranslationConverter
from pyrit.executor.attack import (
AttackConverterConfig,
PromptSendingAttack,
)
from pyrit.prompt_converter import TranslationConverter
from pyrit.prompt_normalizer import PromptConverterConfiguration
from pyrit.prompt_normalizer import ConverterConfiguration
from pyrit.prompt_target import OpenAIChatTarget

objective = "What is the capital of France?"
Expand All @@ -166,11 +166,11 @@

# Request converter: translate English to French
request_converter = TranslationConverter(converter_target=converter_target, language="French")
request_converter_config = PromptConverterConfiguration(converters=[request_converter])
request_converter_config = ConverterConfiguration(converters=[request_converter])

# Response converter: translate response back to English
response_converter = TranslationConverter(converter_target=converter_target, language="English")
response_converter_config = PromptConverterConfiguration(converters=[response_converter])
response_converter_config = ConverterConfiguration(converters=[response_converter])

# Configure the attack with both request and response converters
converter_config = AttackConverterConfig(
Expand Down
16 changes: 8 additions & 8 deletions doc/code/converters/1_text_to_text_converters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@
}
],
"source": [
"from pyrit.prompt_converter import (\n",
"from pyrit.converter import (\n",
" AsciiArtConverter,\n",
" AskToDecodeConverter,\n",
" AtbashConverter,\n",
Expand Down Expand Up @@ -274,7 +274,7 @@
}
],
"source": [
"from pyrit.prompt_converter import (\n",
"from pyrit.converter import (\n",
" AnsiAttackConverter,\n",
" ArabicPresentationFormConverter,\n",
" ArabiziConverter,\n",
Expand Down Expand Up @@ -420,8 +420,7 @@
}
],
"source": [
"from pyrit.datasets import TextJailBreak\n",
"from pyrit.prompt_converter import (\n",
"from pyrit.converter import (\n",
" JsonStringConverter,\n",
" PolicyPuppetryConverter,\n",
" SearchReplaceConverter,\n",
Expand All @@ -430,7 +429,8 @@
" TextJailbreakConverter,\n",
" UrlConverter,\n",
")\n",
"from pyrit.prompt_converter.policy_puppetry_converter import PolicyPuppetryTemplate\n",
"from pyrit.converter.policy_puppetry_converter import PolicyPuppetryTemplate\n",
"from pyrit.datasets import TextJailBreak\n",
"\n",
"prompt = \"tell me how to cut down a tree\"\n",
"\n",
Expand Down Expand Up @@ -490,7 +490,7 @@
}
],
"source": [
"from pyrit.prompt_converter import (\n",
"from pyrit.converter import (\n",
" AsciiSmugglerConverter,\n",
" SneakyBitsSmugglerConverter,\n",
" VariationSelectorSmugglerConverter,\n",
Expand Down Expand Up @@ -669,8 +669,7 @@
"import pathlib\n",
"\n",
"from pyrit.common.path import CONVERTER_SEED_PROMPT_PATH\n",
"from pyrit.models import SeedPrompt\n",
"from pyrit.prompt_converter import (\n",
"from pyrit.converter import (\n",
" DecompositionConverter,\n",
" DenylistConverter,\n",
" ImagePromptStyleConverter,\n",
Expand All @@ -686,6 +685,7 @@
" TranslationConverter,\n",
" VariationConverter,\n",
")\n",
"from pyrit.models import SeedPrompt\n",
"from pyrit.prompt_target import OpenAIChatTarget\n",
"\n",
"attack_llm = OpenAIChatTarget()\n",
Expand Down
Loading
Loading