From e2fd1153c40664e03e4c3db726458ade0577ea6a Mon Sep 17 00:00:00 2001 From: Jeremy Vincent Date: Wed, 23 Oct 2024 19:01:41 +0200 Subject: [PATCH 1/2] Add CircleCI evals-orb --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 8806ba0..3901c2f 100644 --- a/README.md +++ b/README.md @@ -261,6 +261,7 @@ If you're interested in the field of LLM, you may find the above list of milesto - [Giskard](https://github.com/Giskard-AI/giskard) - Testing & evaluation library for LLM applications, in particular RAGs - [LangSmith](https://www.langchain.com/langsmith) - a unified platform from LangChain framework for: evaluation, collaboration HITL (Human In The Loop), logging and monitoring LLM applications. - [Ragas](https://github.com/explodinggradients/ragas) - a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. +- [CircleCI Evals Orb](https://circleci.com/developer/orbs/orb/circleci/evals) - a CircleCI orb for managing and executing evaluation jobs with reporting and testing capabilities. ## LLM Training Frameworks From 924b00dcc261a13649d9a755e33412089408479d Mon Sep 17 00:00:00 2001 From: Jeremy Vincent Date: Wed, 23 Oct 2024 20:37:05 +0200 Subject: [PATCH 2/2] Update README.md Co-authored-by: Ryan E. Hamilton <104376313+ryan-circleci@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3901c2f..720ff41 100644 --- a/README.md +++ b/README.md @@ -261,7 +261,7 @@ If you're interested in the field of LLM, you may find the above list of milesto - [Giskard](https://github.com/Giskard-AI/giskard) - Testing & evaluation library for LLM applications, in particular RAGs - [LangSmith](https://www.langchain.com/langsmith) - a unified platform from LangChain framework for: evaluation, collaboration HITL (Human In The Loop), logging and monitoring LLM applications. - [Ragas](https://github.com/explodinggradients/ragas) - a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. -- [CircleCI Evals Orb](https://circleci.com/developer/orbs/orb/circleci/evals) - a CircleCI orb for managing and executing evaluation jobs with reporting and testing capabilities. +- [CircleCI Evals Orb](https://circleci.com/developer/orbs/orb/circleci/evals) - a CircleCI orb for managing and executing evaluation jobs with reporting, testing, and pass/fail decision-making capabilities. ## LLM Training Frameworks