fix: improve handling of NaN values in evaluation metrics calculation!

[amindadgar]

• Updated the evaluation workflow to handle NaN values when calculating the mean of metrics, ensuring that only valid mean values are included in the results.
• This change enhances the robustness of the evaluation process by preventing NaN results from affecting the overall metrics.

Summary by CodeRabbit

• Chores
  • Improved evaluation workflow to compute metric averages more robustly: only includes metrics present, drops NaN values before averaging, and omits metrics with undefined results. Produces cleaner, more reliable averages in evaluation reports, reduces noise from incomplete datasets, and maintains existing behavior for missing metrics without impacting product functionality.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

The evaluation workflow in .github/workflows/evaluation.yml updates metric averaging logic: it iterates over defined metrics, computes means with NaNs dropped, and only records a metric if its mean is not NaN. Missing columns remain excluded.

Changes

Cohort / File(s)	Summary
GitHub Actions workflow: evaluation .github/workflows/evaluation.yml	Replaced dict comprehension with explicit loop to compute metric means using dropna; skip metrics whose computed mean is NaN; unchanged handling for absent columns.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Dev as Developer
  participant GH as GitHub Actions Runner
  participant Eval as Evaluation Step
  participant DF as DataFrame

  Dev->>GH: Push / PR triggers evaluation.yml
  GH->>Eval: Run evaluation script
  Eval->>DF: Load metrics data
  loop For each metric
    Eval->>DF: df[m].dropna().mean()
    alt Mean is numeric
      Eval->>Eval: Include metric in avgs
    else Mean is NaN
      Eval->>Eval: Skip metric
    end
  end
  Eval-->>GH: Output filtered averages

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

I hop through rows where NaNs once lay,
Nibbling means that won’t decay;
Skip the voids, keep numbers clean,
Carrot-bright averages, crisp and lean.
In YAML burrow, I compute and beam—
A tidy warren of the meanest mean.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/222-incorrenct-eval-scoring

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d864a43 and aee3b65.

📒 Files selected for processing (1)

• .github/workflows/evaluation.yml (1 hunks)

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}