Add author disclosure to PR template (incentivizing AI usage disclosure)

[AdriaandeJongh]

What problem(s) does this PR solve?

The current PR template asks for disclosing AI, but is so lenient about it that at least half of the obviously-AI PRs from the last month or so don’t even bother mentioning it.

It is important for reviewers to know when AI is used or not, and if it was used: to what extent. Not being sure comes to the detriment of first time reviewers as half the time of a reviewer is spent on investigating the PR and author to determine AI usage. This is increasingly demotivating to reviewers. PRs that do not disclose the use of generative AI waste reviewer time with code that makes no sense. Code added by an agent that doesn’t care, can’t take responsibility for it, or can’t continue to maintain it, should not be added to the codebase. It is important for the health of Godot that human made contributions can be reviewed with the attention they deserve. AI use disclose, or non-use disclosure, is part of the solution. This PR makes it more explicit.

Additional information

Please do suggest any improvements you can think of.

Author disclosure

No generative AI was used to make this PR.

[Ivorforce]

While i agree better info on this would really help, I'm concerned about the community effect:

• It could be seen as a signal that Godot has "embraced" AI into their workflow.
• The question as part of the template could be seen as mistrustful or defensive.

This is the reason we originally decided against this heavy handed approach.

I do agree that we already see enough low quality AI PRs that cast a bad rep onto new contributors as a whole, making review more difficult.
But if we go for this we should be aware of the downsides as well.

[Ivorforce]

A potential way to mitigate point 1 might be to frame it in terms of "AI free" instead of "AI use", though this slightly misrepresents our own rules and slightly exacerbates point two (imo).

[AdriaandeJongh]

Right! I’ll attempt to reword it with that in mind.

[scgm0]

I want to know what it means to use llm ai for code completion? In most cases, the completion results are similar to the previous smart completion.

[allenwp]

Thanks for exploring this! One consideration is the amount of lines and the amount of words in the PR template: the more text, the less likely people will read it and the more likely people will simply delete it (maybe??)

I also found when testing the original PR that too many lines of text caused the PR template text to overflow the visible text box when working in my web browser so that some of the text would appear “below the fold”, so to speak.

So anyway, all that to say, an alternative may be to simply move the existing single line of text to be under this new heading. It’s succinct and clear and the new heading would force people to take the moment of time to read it and think what they need to write. My gut tells me that people don’t report because they’re not forced to write something and putting a heading will force them to. I’m not sure more text will solve the problem, but also I really don’t know why people may contribute and also not want to do their best to support the practices of the project they’re contributing to. This is not my expertise…

Anyway, the original PR that I authored to change the template was highly scrutinized, but personally I’m not too picky on the details because this template isn’t like engine code that will ship to games with millions of players and difficult to change the behaviour of. We can change this easily and at the drop of a hat.

[Cykyrios]

Maybe this new section could link to the AI-assisted contributions section of the contributing docs? This would help make it clear that people who open AI-generated PRs without disclosure purposely ignored Godot's AI policy.

[EdwardChanCH]

I also found when testing the original PR that too many lines of text caused the PR template text to overflow the visible text box when working in my web browser so that some of the text would appear “below the fold”, so to speak.

As a new volunteer, I agree. I started to Ctrl + A Backspace after reading the original <!-- instructions --> a couple times, so I did not even notice #118855 until I saw all the new PRs having the same headings (I initially thought they were AI-generated spam PRs). Adding an AI usage section at the bottom may have little effect if people overlook it.

I think the <!-- instructions --> can skip the explanations, and instead redirect people to relevant doc pages. Using single-line <!-- --> could also save some space.

Maybe this new section could link to the AI-assisted contributions section of the contributing docs? This would help make it clear that people who open AI-generated PRs without disclosure purposely ignored Godot's AI policy.

Adding onto that, a checkbox, like in Godot Proposal, may be more straightforward:

AI disclosure

[ ] Yes, this pull request follows the AI-assisted contribution guidelines:
https://contributing.godotengine.org/en/latest/pull_requests/pull_request_guidelines.html#ai-assisted-contributions

... and feels less like defending my integrity with "No generative AI was used to make this PR". People who use AI and read the (new) guidelines will know to report their AI usage beneath the checkbox.

[Ivorforce]

I also found when testing the original PR that too many lines of text caused the PR template text to overflow the visible text box when working in my web browser so that some of the text would appear “below the fold”, so to speak.

As a new volunteer, I agree. I started to Ctrl + A Backspace after reading the original <!-- instructions --> a couple times, so I did not even notice #118855 until I saw all the new PRs having the same headings (I initially thought they were AI-generated spam PRs). Adding an AI usage section at the bottom may have little effect if people overlook

I think this is uniquely a 'contributed before' problem. Contributors who are used to delete the template will adjust fast enough. New contributors should honor the template. We'd be willing to close pull requests that ignore the template.

Maybe this new section could link to the AI-assisted contributions section of the contributing docs? This would help make it clear that people who open AI-generated PRs without disclosure purposely ignored Godot's AI policy.

Adding onto that, a checkbox, like in Godot Proposal, may be more straightforward:

AI disclosure

[ ] Yes, this pull request follows the AI-assisted contribution guidelines:
https://contributing.godotengine.org/en/latest/pull_requests/pull_request_guidelines.html#ai-assisted-contributions

This is a decent idea but imo, it strongly exacerbate my point (1) where this framing might be interpreted as "AI is welcome here" to contributors who don't read the guidelines.

[allenwp]

I should also mention that there is another existing PR that is similar to this one that's worth considering: #119242

I've added a comment to #118624 that links to this PR because that discussion explored the advantages and disadvantages of having more headings added to the template.

Personally, I've never been a part of discovering or reviewing a PR that contains AI-assisted content. Instead, my reviews are focused on those that are close to being merged. So I actually have no experience or expertise on the specific subject of AI-assisted contributions.

Besides my comment about how less text may have a higher likelihood of being read than more text, I expect I don't have anything else to contribute to this discussion due to my lack of experience or expertise on the subject of AI-assisted PRs. I'll let others chime in on this PR or #119242.

[charjr]

A potential way to mitigate point 1 might be to frame it in terms of "AI free" instead of "AI use"

Just throwing one in:

Disclosure of Ownership

• I made this PR independently, without use of generative AI in any capacity.
• I made most of this PR, with partial assistance by generative AI, detailed below:

By lacking the "I blindly shovelled slop in your direction" option, it's implicitly rejected.

Contributors that fail to tick either box or tick the second box without detailing the assistance would be politely prompted to add/fill this section in. Code review would be postponed until the PR is ready.

[AdriaandeJongh]

I updated this PR with a version in which the title is about author disclosure and the invisible comment is about AI use, thus avoiding the "AI is welcome here"-trap (I think).

I should also mention that there is another existing PR that is similar to this one that's worth considering: #119242

Ah yeah, I totally missed this PR. Slightly different approaches...

[Ivorforce]

I like the new framing! Better than my suggestion. It can also be used to disclose whether any code was copied, or another PR salvaged, which I think is appropriate since it's pretty relevant license wise.

[Ivorforce]

I think it's worth trying the effect of this.

We've been rolling with @allenwp's #118855 for a month now and I personally quite like not having to search for the solved problems anymore.
If this does the same for AI disclosures, it will probably be worth it.

Need 1 or 2 more approvals to consider this mergable i'd say.

[charjr]

Not a maintainer, but I approve. I appreciate that the wording focuses more on disclosing human authorship.

Will this section be considered mandatory?

[AdriaandeJongh]

Will this section be considered mandatory?

Good question! As a template, it can be removed by contributors so it is not technically mandatory. However, it can be made mandatory by triage teams and reviewers, in a way that any PR that fails to disclose authorship can be closed. But this also burdens maintainers, so... idk, I wouldn't force it, at least not for now. I think this added section can make it more appealing and attractive to reviewers to focus on and review PRs which indicate no AI usage, which is I think a more positive spin.

[AdriaandeJongh]

@allenwp do you approve of this PR? Any remaining thoughts and feedback?

@akien-mga with the upcoming new AI policy, would this PR be nice to get in? any feedback?

[allenwp]

@allenwp do you approve of this PR? Any remaining thoughts and feedback?

I truly know nothing about AI usage and have zero experience dealing with AI authored PRs, so I don't feel at all comfortable making a call on this. This PR has changes that are very similar to my first attempt at an updated PR template, which was overall poorly received, so I must leave it up to others who are more familiar with the AI subject to decide if it's better or worse than the current template. Maybe AThousandShips or the folks who are currently working on the new AI policy would be best to review this to ensure that it's well-supporting the newly refined policies and communications.