Open
Description
Calling @test_reference
on a non-existent file when JULIA_REFERENCETESTS_UPDATE
is unset should cause a test failure instead of generating a new reference file:
julia> @test_reference "dne.txt" "hello"
┌ Info: Reference file for "dne.txt" did not exist. It has been created:
│ - NEW CONTENT -----------------
│ hello
│ -------------------------------
└ new_reference = "/Users/cvogt/.julia/dev/TestReports/dne.txt"
[ Info: Please run the tests again for any changes to take effect
The rational behind this change is that if a user forgets to commit a reference file CI jobs will pass even though the reference test is broken. This happened to me in: JuliaTesting/TestReports.jl#99