Team Members: Ronan Takizawa, Kaylie Stuteville, Willa Polman, Anna Vu.
AssessAI is a desktop software tool designed to help users evaluate and test the performance of large language models (LLMs) specifically in summarizing custom datasets. The tool allows clients to upload their datasets and assess how well various LLMs summarize their content. With this, users can determine how effectively these models could potentially serve their own projects.
Screen.Recording.2025-02-10.at.8.01.08.AM.mov
- Load Huggingface models
- Finetune models with summarization dataset
- Model Evaluation (ROUGE, BLEU, METEOR)
- Python 3.8 to 3.11
- git
- Minimum: 8GB RAM, Recommended: 16GB RAM
- Storage: 10GB+ free space
- CPU: Multi-core processor supporting AVX2 instructions (AVX2 instructions are used in llama.cpp to run mistral-7B)
- Text summarization datasets in “input_text” “target_text” format
- Clone this repository:
git clone https://github.com/ColoradoCollege-MathCS/assess.ai
cd assess.ai- Install dependencies:
pip install -r requirements.txt- Download the required model first (this is required before first run to do G-EVAL):
cd src
python download_mistral.py- Start the application:
python main.py