Skip to content

Bug: "Open Source Contributions" resume section is silently dropped, but open_source is the highest-weighted scoring category (35/100) #409

Description

@ayushh0110

Summary

The open_source scoring category has the highest weight in the software_engineering_intern role (35/100 points), yet the pipeline has no mechanism to extract an "Open Source Contributions" section from the resume. This means resumes with dedicated OSS sections will always score poorly on the most impactful category, regardless of their actual contributions.

Steps to Reproduce

  1. Create a resume PDF with a clearly labeled "Open Source Contributions" section listing merged PRs to major repos (e.g., litellm, llama_index, instructor)
  2. Run the scoring pipeline:
    python score.py ./resume/sample.pdf --role software_engineering_intern
  3. Observe that open_source scores low with evidence stating "no direct contributions to external open source projects"
  4. Check the cached JSON (cache/resumecache_*.json) — no OSS data is present

Root Cause

The bug spans three layers:

1. Extraction — pdf.py line 269

The section extraction list is hardcoded to 6 sections:

sections = ["basics", "work", "education", "skills", "projects", "awards"]

There is no "open_source" or "contributions" section. Any resume content that doesn't map to one of these 6 sections is silently discarded.

2. Schema — models.py line 193

The JSONResume model has no field for open source contributions:

class JSONResume(BaseModel):
    basics, work, volunteer, education, awards, certificates,
    publications, skills, languages, interests, references, projects

Even if extraction was added, there's no schema field to store the data.

3. Serialization — transform.py line 728

convert_json_resume_to_text() only serializes existing schema fields. Since there's no OSS field, the evaluator LLM never sees this data in its prompt.

Evidence

Raw PDF extraction (pymupdf4llm.to_markdown()) correctly extracts the section:

Open Source Contributions

– litellm (BerriAI, 50k⋆): Fixed the MCP semantic tool filter silently dropping native (non-MCP) tools...
– llama_index (run-llama, 50k⋆): Fixed a Bedrock streaming bug...
– instructor (567-labs, 13k⋆): Added an on_event callback...

But the cached resumecache_*.json contains zero OSS data. The text exists in the PDF — the pipeline just doesn't capture it.

Impact

  • open_source is worth 35/100 points — the single highest-weighted category
  • Candidates with significant OSS contributions (merged PRs to major repos) are scored as if they have none
  • The only way to score well on open_source is through the GitHub API enrichment step, which itself depends on a working GITHUB_TOKEN and only checks the candidate's own repos — not their PRs to other projects

Suggested Fix

  1. Add an "open_source" section to the extraction pipeline in pdf.py
  2. Add a corresponding Jinja template in prompts/templates/
  3. Add an open_source_contributions field to JSONResume in models.py
  4. Serialize the new field in convert_json_resume_to_text() in transform.py

Alternatively, pass the raw PDF markdown text alongside the structured JSON to the evaluator, so non-schema sections aren't lost.

Environment

  • Python 3.12
  • Model: gemini-2.5-flash
  • OS: Windows 11

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions