Skip to content

Commit 1744b55

Browse files
committed
Rename row_ids parameter to filtered_row_ids in evaluation_test function for clarity
- Updated the parameter name from row_ids to filtered_row_ids to better reflect its purpose in filtering evaluation rows. - Adjusted related documentation to ensure consistency and clarity regarding the new parameter name.
1 parent 9a6fe7a commit 1744b55

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

eval_protocol/pytest/evaluation_test.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ def evaluation_test(
7979
aggregation_method: AggregationMethod = "mean",
8080
passed_threshold: EvaluationThreshold | float | EvaluationThresholdDict | None = None,
8181
num_runs: int = 1,
82-
row_ids: Sequence[str] | None = None,
82+
filtered_row_ids: Sequence[str] | None = None,
8383
max_dataset_rows: int | None = None,
8484
mcp_config_path: str | None = None,
8585
max_concurrent_rollouts: int = 8,
@@ -147,7 +147,7 @@ def evaluation_test(
147147
Success rate must be above success, and if set, standard error must be below standard_error.
148148
Success rate +/- one standard_error is equivalent to 68% confidence interval.
149149
num_runs: Number of times to repeat the rollout and evaluations.
150-
row_ids: List of row_ids to use filter for the evaluation. If provided, only the rows with the given row_ids will be evaluated.
150+
filtered_row_ids: List of row_ids to filter for the evaluation. If provided, only the rows with the given row_ids will be evaluated.
151151
max_dataset_rows: Limit dataset to the first N rows.
152152
mcp_config_path: Path to MCP config file that follows MCPMultiClientConfiguration schema
153153
max_concurrent_rollouts: Maximum number of concurrent rollouts to run in parallel.
@@ -288,8 +288,8 @@ def _log_eval_error(status: Status, rows: list[EvaluationRow] | None, passed: bo
288288
else:
289289
raise ValueError("No input dataset, input messages, or input rows provided")
290290

291-
if row_ids is not None:
292-
data = [row for row in data if row.input_metadata.row_id in row_ids]
291+
if filtered_row_ids is not None:
292+
data = [row for row in data if row.input_metadata.row_id in filtered_row_ids]
293293

294294
"""
295295
data_loaders handles preprocess_fn internally so we want

0 commit comments

Comments
 (0)