v0.8.0

What's Changed

Targets:

• HTTPTarget now supports rate limiting
• Some users encountered errors in Azure OpenAI when hitting content filter errors using error code 500. PyRIT now catches content filter responses with both error codes 400 (as before) and 500 (new) and returns a clean response record.

Datasets:

• fetch_babelscape_alert_dataset had a bug causing it to be limited to a single category even when users specified both. This is now fixed!
• added fetch_red_team_social_bias_dataset
• added fetch_darkbench_dataset
• added fetch_mlcommons_ailuminate_demo_dataset

Converters:

• added UnicodeReplacementConverter
• added sneaky_bits option to AsciiSmugglerConverter in the encoding_mode argument. The unicode_tags argument is now removed and replaced by more options in encoding_mode (i.e., unicode_tags, unicode_tags_control, and sneaky_bits).

Scanner: A basic version was introduced in v0.7.0 that supported only sending single-turn prompts. v0.8.0 expands on this with support for most multi-turn orchestrators (incl. adversarial chat targets and scorers) and memory. This feature is still considered experimental and may change considerably in the following versions.

Other:

• support for Python 3.13 in addition to 3.10-3.12.
• For single-piece responses, we now have a convenient get_value() method.
• PyRIT used to print warnings that torch isn't installed (unless the corresponding extra was installed). This was caused by transformers and is now turned off as it doesn't serve any purpose.
• In previous versions, PyRIT started supporting .env.local as an override to the .env file for endpoint secrets. However, when using this outside of the normal repository structure (e.g., when running PyRIT without cloning this repo) the code failed to discover .env.local in the current working directory. This is now fixed.

Full list of changes

• [DevContainer] Provide a uniform development environment by @bashirpartovi in #787
• FEAT: Add Rate Limit Support for HTTP Target by @nina-msft in #786
• DOC Updating contribution docs by @bashirpartovi in #788
• MAINT support python 3.13 by @AdrGav941 in #779
• FIX: fixed dev container permission issue by @bashirpartovi in #789
• FEAT: simplify extraction of converted values from responses by @paulinek13 in #783
• MAINT: improve organization of dataset fetch functions (refactoring) by @paulinek13 in #785
• FEAT: Added cross-platform compatibility and needed language support for toml and docker by @bashirpartovi in #797
• MAINT: Update release version to 0.7.1.dev0 by @jsong468 in #800
• FIX: prevent data overwrite in fetch_babelscape_alert_dataset by @paulinek13 in #799
• DOC contributor guide flowchart, small text updates, and add Roakey to README by @romanlutz in #798
• DOC: clarify OpenAITarget targets httpx_client_kwargs timeout settings by @clod81 in #801
• FIX: Add exception on response parsing when call to Openrouter.ai by @hagsmand in #796
• FIX make sure conversation IDs are not sent out as UUIDs to the database by @ayeganov in #723
• FEAT support adversarial_chat and scoring in scanner to enable automated multi-turn-orchestrators by @romanlutz in #706
• FIX move misplaced test file to tests/unit/converter by @romanlutz in #794
• FEAT: Added Red Team Social Bias dataset by @MoolmanM in #714
• DOC improve API reference for auth, cli, common, chat_message_normalizer by @romanlutz in #793
• FEAT: UnicodeReplacementConverter by @nina-msft in #803
• FIX: Updating pre-commit to fix build issues by @rlundeen2 in #810
• MAINT: Making test_connect more resilient by @rlundeen2 in #806
• [FIX] fix bad domain by @mgstate in #815
• [FIX] Integration test fixes: add hugging face token in notebook and fix test_fetch_datasets by @jsong468 in #819
• FEAT: Added memory config to scanner by @bashirpartovi in #808
• FEAT: add DarkBench dataset by @paulinek13 in #821
• MAINT: improving build/test time by @bashirpartovi in #820
• FIX handle Azure OpenAI content_filter errors with HTTP status code 500 by @romanlutz in #825
• FIX turn off transformers warning by @romanlutz in #829
• TEST: Adding integration test for content filters by @rlundeen2 in #830
• MAINT: Separating integration test local .env by @rlundeen2 in #817
• FEAT: add MLCommons AILuminate v1.0 DEMO Prompt Set by @paulinek13 in #828
• FIX find .env.local in current working directory by @romanlutz in #832
• BREAKING FEAT: Sneaky Bits - Advanced Data Smuggling Techniques by @KutalVolkan in #827
• FEAT add ps-fuzz prompts by @ryanjieh in #823

New Contributors

• @bashirpartovi made their first contribution in #787
• @clod81 made their first contribution in #801
• @hagsmand made their first contribution in #796
• @MoolmanM made their first contribution in #714
• @mgstate made their first contribution in #815
• @ryanjieh made their first contribution in #823

Full Changelog: v0.7.0...v0.8.0