Replace test suite with spec conformance test suite

[johannschopplich]

I'd be great to use https://github.com/toon-format/spec as a submodule or similar and run the official conformance test suite instead of managing a custom one.

[s-celles]

Hi @johannschopplich

Not sure I understand correctly what you are looking for.
I didn't see much (fresh) information about fixtures in the Julia ecosystem

https://discourse.julialang.org/t/testing-fixture-library-for-julia/26703

Do you have so reference / pointers about this to recommand? (not specially in Julia)

I looked at
https://github.com/toon-format/spec/blob/main/tests/fixtures.schema.json
and noticed https://toon-format.org/schemas/test-fixture.json isn't published

Is that normal?

I'm not really specialist about JSON Schema.

What should such a Fixture submodule scope be?

• Download fixtures (how to have an index of all fixtures?)
• Ensure each fixture validate JSON schema (we have https://github.com/fredo-dedup/JSONSchema.jl )
• Run for each fixture JSON file tests

Maybe this should be part of a separate Julia package. But in such a case this should be quite general (ie not only about TOON encoding/decoding).

Any opinion?

I see for example in toon-python that fixtures have directly been integrated in toon-python repository (like in this repository)
https://github.com/toon-format/toon-python/tree/main/tests/fixtures/decode
I'm not sure that's the correct approach to use.
I prefer to use latest fixtures which have been downloaded recently to ensure that ToonFormat.jl encoding/decoding implementation is still correct.

In current ToonFormat.jl test implementation we have:

• download_fixtures.jl which is responsible of downloading latest fixtures
• test_spec_fixtures.jl wich is responsible of running tests against downloaded fixtures
• runtests.jl which is responsible of running all tests (both custom tests and fixtures)

[s-celles]

Fixtures could be include via Git submodule at test/spec/

[johannschopplich]

Downloading the latest fixtures on the fly is fine by me as well. Well done, thanks.