Skip to content

No data under regulation folder #2

@prasoonvarshney

Description

@prasoonvarshney

The README notes regulation evaluation as

cd regulation

# Attack evaluation on regulatory compliance
python evaluate_attack.py --model_id OpenSafetyLab/MD-Judge-v0_2-internlm2_7b

# Conversation evaluation
python evaluate_conversation.py --model_id meta-llama/Llama-Guard-3-8B

# Query evaluation
python evaluate_query.py --model_id allenai/wildguard

The evaluate_attack.py file in turn tries to access dataset_path = f"{base_dir}/dataset/regulation/gdpr/query/unsafe.jsonl"

However, there is no such file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions