Skip to content

@test_reference should fail when reference file is missing #127

Open
@omus

Description

@omus

Calling @test_reference on a non-existent file when JULIA_REFERENCETESTS_UPDATE is unset should cause a test failure instead of generating a new reference file:

julia> @test_reference "dne.txt" "hello"
┌ Info: Reference file for "dne.txt" did not exist. It has been created:- NEW CONTENT -----------------
│ hello
│ -------------------------------
└   new_reference = "/Users/cvogt/.julia/dev/TestReports/dne.txt"
[ Info: Please run the tests again for any changes to take effect

The rational behind this change is that if a user forgets to commit a reference file CI jobs will pass even though the reference test is broken. This happened to me in: JuliaTesting/TestReports.jl#99

Metadata

Metadata

Assignees

No one assigned

    Labels

    breakingChanging this would be a breaking featureenhancement

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions