Skip to content

Commit eb08648

Browse files
Add trace-based evaluation in agent evaluation metrics to get correct score (#478)
* Add trace-based evaluation in agent evaluation metrics to get correct score * check reason is provided * remove credentials * fix the failure due to dependency * fix the failure due to dependency * fix scorecardpy version to 0.1.9.6 * fix copyright
1 parent 509a0ef commit eb08648

21 files changed

Lines changed: 9335 additions & 9537 deletions

File tree

‎notebooks/code_sharing/operational_deposit/operational_deposit_poc.ipynb‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1132,6 +1132,7 @@
11321132
},
11331133
{
11341134
"cell_type": "markdown",
1135+
"id": "copyright-e2f10038a74449cba590e46511b2368c",
11351136
"metadata": {},
11361137
"source": [
11371138
"<!-- VALIDMIND COPYRIGHT -->\n",

‎notebooks/how_to/data_and_datasets/use_dataset_model_objects.ipynb‎

Lines changed: 982 additions & 981 deletions
Large diffs are not rendered by default.

‎notebooks/how_to/tests/explore_tests/explore_test_suites.ipynb‎

Lines changed: 789 additions & 788 deletions
Large diffs are not rendered by default.

‎notebooks/how_to/tests/explore_tests/explore_tests.ipynb‎

Lines changed: 4435 additions & 4434 deletions
Large diffs are not rendered by default.

‎notebooks/how_to/tests/run_tests/configure_tests/enable_pii_detection.ipynb‎

Lines changed: 675 additions & 675 deletions
Large diffs are not rendered by default.

‎notebooks/tutorials/model_development/4-finalize_testing_documentation.ipynb‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -960,6 +960,7 @@
960960
},
961961
{
962962
"cell_type": "markdown",
963+
"id": "copyright-75cfc55507924d27b0d37b140c473293",
963964
"metadata": {},
964965
"source": [
965966
"<!-- VALIDMIND COPYRIGHT -->\n",

‎notebooks/tutorials/model_validation/4-finalize_validation_reporting.ipynb‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1205,6 +1205,7 @@
12051205
},
12061206
{
12071207
"cell_type": "markdown",
1208+
"id": "copyright-b55920b2495443d1894125f60e582bb4",
12081209
"metadata": {},
12091210
"source": [
12101211
"<!-- VALIDMIND COPYRIGHT -->\n",

‎notebooks/use_cases/agents/banking_tools.py‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
from typing import Optional
22
from datetime import datetime
33
from langchain.tools import tool
4+
from deepeval.tracing import observe
45

56

67
def _score_dti_ratio(dti_ratio: float) -> int:
@@ -79,6 +80,7 @@ def _get_credit_description(credit_score: int) -> str:
7980

8081
# Credit Risk Analyzer Tool
8182
@tool
83+
@observe(type="tool")
8284
def credit_risk_analyzer(
8385
customer_income: float,
8486
customer_debt: float,
@@ -279,8 +281,8 @@ def _handle_recommend_product(customer):
279281

280282
def _handle_get_info(customer, customer_id):
281283
"""Handle get info action."""
282-
credit_tier = ('Excellent' if customer['credit_score'] >= 750 else
283-
'Good' if customer['credit_score'] >= 700 else
284+
credit_tier = ('Excellent' if customer['credit_score'] >= 750 else
285+
'Good' if customer['credit_score'] >= 700 else
284286
'Fair' if customer['credit_score'] >= 650 else 'Poor')
285287

286288
return f"""CUSTOMER ACCOUNT INFORMATION
@@ -308,6 +310,7 @@ def _handle_get_info(customer, customer_id):
308310

309311
# Customer Account Manager Tool
310312
@tool
313+
@observe(type="tool")
311314
def customer_account_manager(
312315
account_type: str,
313316
customer_id: str,
@@ -362,6 +365,7 @@ def customer_account_manager(
362365

363366
# Fraud Detection System Tool
364367
@tool
368+
@observe(type="tool")
365369
def fraud_detection_system(
366370
transaction_id: str,
367371
customer_id: str,

‎notebooks/use_cases/agents/document_agentic_ai.ipynb‎

Lines changed: 2176 additions & 2196 deletions
Large diffs are not rendered by default.

‎poetry.lock‎

Lines changed: 4 additions & 64 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)