Making ci tests self contained

[aaishwarymishra]

Updated PR

Isolate and reduce generated-project integration tests

Problem

Generated projects are currently run with the test suite’s Python environment.
The tests also train every model/dataset combination for one epoch, making CI unnecessarily expensive.

Proposed solution

• Keep lightweight generation checks for all combinations.
• Run one representative end-to-end smoke test in an isolated uv environment.
• Install only dependencies declared by the generated project.
• Limit training to one batch or the smallest practical workload.
• Rely on the shared uv package cache to avoid repeated downloads.

Acceptance criteria

• Generated code runs in an isolated environment.
• Missing generated dependencies cause the test to fail.
• One representative project completes a minimal training run.
• All combinations remain covered by lightweight generation tests.

[TahaZahid05]

I think while there are obvious downsides of the tests taking more time, it is still better to make the test accurate so we don't merge broken code. Testing model training with the correct dependencies is critical so that Trainite works as intended.

[aaishwarymishra]

@vfdev-5 thoughts?

[aniketpandey05]

@aaishwarymishra @TahaZahid05 can i pick this up?

[aaishwarymishra]

Hi, sorry @aniketpandey05 this is for an event :)

[saraelsayed13]

@aaishwarymishra I’d like to work on this issue as part of the Catreloaded-2026 event

[vfdev-5]

@aaishwarymishra can you update the description to make it clear about what to do for this issue.
As it was discussed previously in our meetings, we do not need now slow integration tests running on each PR, instead we can run them nightly. If we do that we should propose what kind of tests we should run on GHA (cpu only)

[aaishwarymishra]

completed