Paper -> CoT pipeline: Optimize prompt for CoT extraction from paper

[thehunmonkgroup]

Requirements:

• Optimize the TASK section (instructions on how to extract the question/chain/answer set)
• Optimize the CHAIN OF REASONING CRITERIA section (criteria for choosing the best question/chain/answer set from a paper)

Deliverable: the final prompt

[thehunmonkgroup]

Here is the full archive of artifacts (inference data for profiling and CoT extraction, JSONL training files) for a test run of 100 papers through the pipeline:

paper-cot-extraction-test-data.tar.gz

The CoT extraction logs are in results/inference with -cot-extraction.txt extension.