You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am getting the error "TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')" and context_recall as 0
#1796
Open
tvsathish opened this issue
Dec 26, 2024
· 0 comments
[ ] I checked the documentation and related resources and couldn't find an answer to my question.
Your Question
Why am I getting this error "TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')" and why is my context_recall value showing up as 0.000000
Code Examples
import pprint
import pandas as pd
from langchain_openai import AzureChatOpenAI
from langchain_openai import AzureOpenAIEmbeddings
from ragas import SingleTurnSample, EvaluationDataset
from ragas import evaluate
from ragas.embeddings import LangchainEmbeddingsWrapper
from ragas.llms import LangchainLLMWrapper
from ragas.metrics import LLMContextRecall, Faithfulness, FactualCorrectness, SemanticSimilarity
def empty_nan_value(cell_value):
return '' if pd.isna(cell_value) else cell_value
def create_turn_sample(row):
return SingleTurnSample(
user_input=row['user_input'],
retrieved_contexts=[empty_nan_value(row['context1']), empty_nan_value(row['context2']),
empty_nan_value(row['context3']),
empty_nan_value(row['context4'])],
response=row['response'],
reference=row['reference'])
df = pd.read_excel("<RESULTS_EXCEL_FILE>")
eval_dataset = EvaluationDataset([create_turn_sample(row) for index, row in df.iterrows()])
azure_config = {
"base_url": "<AZURE_CHAT_COMPLETIONS_URL>",
# your endpoint
"model_deployment": "<AZURE_DEPLOYMENT>", # your model deployment name
"model_name": "gpt-4o" # your model name
}
evaluator_llm = LangchainLLMWrapper(AzureChatOpenAI(
openai_api_version="2024-08-01-preview",
azure_endpoint=azure_config["base_url"],
azure_deployment=azure_config["model_deployment"],
model=azure_config["model_name"],
validate_base_url=False,
))
metrics = [
LLMContextRecall(llm=evaluator_llm),
FactualCorrectness(llm=evaluator_llm),
Faithfulness(llm=evaluator_llm)
]
results = evaluate(dataset=eval_dataset, metrics=metrics)
pprint.pprint(results)
df = results.to_pandas()
df.head()
`
AZURE_OPENAI_KEY is environment variable
Additional context
Output:
Evaluating: 14%|████████████████████████████ | 25/174 [01:55<02:04, 1.19it/s]Exception raised in Job[31]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 23%|████████████████████████████████████████████▊ | 40/174 [02:55<03:08, 1.41s/it]Exception raised in Job[34]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 25%|█████████████████████████████████████████████████▎ | 44/174 [02:58<01:56, 1.12it/s]Exception raised in Job[37]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 30%|██████████████████████████████████████████████████████████▎ | 52/174 [03:59<06:44, 3.31s/it]Exception raised in Job[58]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 34%|██████████████████████████████████████████████████████████████████ | 59/174 [04:17<06:32, 3.41s/it]Exception raised in Job[43]: TimeoutError()
Evaluating: 35%|████████████████████████████████████████████████████████████████████▎ | 61/174 [04:59<19:02, 10.11s/it]Exception raised in Job[46]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 41%|███████████████████████████████████████████████████████████████████████████████▌ | 71/174 [05:02<02:13, 1.29s/it]Exception raised in Job[70]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 41%|████████████████████████████████████████████████████████████████████████████████▋ | 72/174 [05:03<01:46, 1.05s/it]Exception raised in Job[73]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 44%|█████████████████████████████████████████████████████████████████████████████████████▏ | 76/174 [05:44<19:13, 11.77s/it]Exception raised in Job[61]: TimeoutError()
Evaluating: 56%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 98/174 [07:04<05:08, 4.06s/it]Exception raised in Job[91]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 58%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 101/174 [07:05<01:56, 1.59s/it]Exception raised in Job[100]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 67%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 116/174 [08:06<04:46, 4.93s/it]Exception raised in Job[109]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 67%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 117/174 [08:07<03:35, 3.78s/it]Exception raised in Job[115]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 75%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 131/174 [09:10<05:46, 8.06s/it]Exception raised in Job[136]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 79%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋ | 137/174 [09:11<01:04, 1.74s/it]Exception raised in Job[142]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 80%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 140/174 [09:11<00:30, 1.10it/s]Exception raised in Job[139]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 84%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 146/174 [09:27<02:06, 4.53s/it]Exception raised in Job[124]: TimeoutError()
Evaluating: 88%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 153/174 [10:13<00:58, 2.77s/it]Exception raised in Job[148]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 90%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 157/174 [10:14<00:16, 1.01it/s]Exception raised in Job[157]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 91%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 158/174 [10:15<00:16, 1.01s/it]Exception raised in Job[169]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 174/174 [12:19<00:00, 4.25s/it] {'context_recall': 0.0000, 'factual_correctness': 0.1174, 'faithfulness': 0.0996}
The text was updated successfully, but these errors were encountered:
[ ] I checked the documentation and related resources and couldn't find an answer to my question.
Your Question
Why am I getting this error "TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')" and why is my context_recall value showing up as 0.000000
Code Examples
`
AZURE_OPENAI_KEY is environment variable
Additional context
Output:
Evaluating: 14%|████████████████████████████ | 25/174 [01:55<02:04, 1.19it/s]Exception raised in Job[31]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 23%|████████████████████████████████████████████▊ | 40/174 [02:55<03:08, 1.41s/it]Exception raised in Job[34]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 25%|█████████████████████████████████████████████████▎ | 44/174 [02:58<01:56, 1.12it/s]Exception raised in Job[37]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 30%|██████████████████████████████████████████████████████████▎ | 52/174 [03:59<06:44, 3.31s/it]Exception raised in Job[58]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 34%|██████████████████████████████████████████████████████████████████ | 59/174 [04:17<06:32, 3.41s/it]Exception raised in Job[43]: TimeoutError()
Evaluating: 35%|████████████████████████████████████████████████████████████████████▎ | 61/174 [04:59<19:02, 10.11s/it]Exception raised in Job[46]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 41%|███████████████████████████████████████████████████████████████████████████████▌ | 71/174 [05:02<02:13, 1.29s/it]Exception raised in Job[70]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 41%|████████████████████████████████████████████████████████████████████████████████▋ | 72/174 [05:03<01:46, 1.05s/it]Exception raised in Job[73]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 44%|█████████████████████████████████████████████████████████████████████████████████████▏ | 76/174 [05:44<19:13, 11.77s/it]Exception raised in Job[61]: TimeoutError()
Evaluating: 56%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 98/174 [07:04<05:08, 4.06s/it]Exception raised in Job[91]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 58%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 101/174 [07:05<01:56, 1.59s/it]Exception raised in Job[100]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 67%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 116/174 [08:06<04:46, 4.93s/it]Exception raised in Job[109]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 67%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 117/174 [08:07<03:35, 3.78s/it]Exception raised in Job[115]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 75%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 131/174 [09:10<05:46, 8.06s/it]Exception raised in Job[136]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 79%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋ | 137/174 [09:11<01:04, 1.74s/it]Exception raised in Job[142]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 80%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 140/174 [09:11<00:30, 1.10it/s]Exception raised in Job[139]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 84%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 146/174 [09:27<02:06, 4.53s/it]Exception raised in Job[124]: TimeoutError()
Evaluating: 88%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 153/174 [10:13<00:58, 2.77s/it]Exception raised in Job[148]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 90%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 157/174 [10:14<00:16, 1.01it/s]Exception raised in Job[157]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 91%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 158/174 [10:15<00:16, 1.01s/it]Exception raised in Job[169]: TypeError(ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'')
Evaluating: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 174/174 [12:19<00:00, 4.25s/it]
{'context_recall': 0.0000, 'factual_correctness': 0.1174, 'faithfulness': 0.0996}
The text was updated successfully, but these errors were encountered: