Unexpected: Git diffing doesn't take into account unstaged files

[jamesbraza]

I am manually changing my promptfooconfig.yaml in CI. This means the change is not committed, it's a local change.

promptfoo-action v1's git diff here: https://github.com/promptfoo/promptfoo-action/blob/v1/src/main.ts#L101-L105

It doesn't take into account local changes as leading to changed files. Thus my promptfoo-action doesn't invoke promptfoo and instead emits:

No LLM prompt or config files were modified.

Can we make this diff account for local file changes as well?

[typpo]

Interesting. Wouldn't it be pretty unusual for files to be changed locally on a stateless ci runner?

…

On Thu, May 30, 2024, 11:22 AM James Braza ***@***.***> wrote: I am manually changing my promptfooconfig.yaml in CI. This means the change is not committed, it's a local change. promptfoo-action v1's git diff here: https://github.com/promptfoo/promptfoo-action/blob/v1/src/main.ts#L101-L105 It doesn't take into account local changes as leading to changed files. Thus my promptfoo-action doesn't invoke promptfoo and instead emits: No LLM prompt or config files were modified. Can we make this diff account for local file changes as well? — Reply to this email directly, view it on GitHub <#122>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACLYJVZLTGFDXTWR5RQGHLZE5U7VAVCNFSM6AAAAABIRNLXHWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDMMRYGY2DKOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jamesbraza]

Yeah I think in general you're right that it is unusual, so feel free to close this out.

To share, my use case is generating the promptfooconfig.yaml on the fly in CI: https://github.com/Future-House/LitQA/blob/ba2b3f9f9798b8c7f40632c9ec3d68b3c1a6c67f/.github/create_tests.py