Experimenting with CodeRabbit AI generated reviews

[beutlich]

I enabled CodeRabbit (the [coderabbitai[bot]) for AI automated reviews. Within the 14 days of the trial period we can see proper review comments. Feel free to test till 2025-04-22 and give feedback. I did not touch any of the many configuration options of CodeRabbit so far.

After that trial period only the review summary is available as part of a free service, see https://www.coderabbit.ai/pricing for more. If no feedback is given I guess I will simply deactivate it again since the review summary is of less relevance.

[maltelenz]

Looks like it hasn't ingested enough Modelica in its training materials: #3406 (comment)

[AHaumer]

It's ok to test such a funny AI, but I doubt it's worth a pence.
I wouldn't feel comfortable with that rabbit doing reviews.
Modelica is a little bit special compared to other programming languages.

[HansOlsson]

I would say that there a number of different questions related to this: * Are the AI-reviews good enough to replace current human reviews? No (and currently seems far from it). * Are the AI-summaries good enough to be included at the top? (Given that ppl often don’t clearly summarize their Pull-Requests): No, as the problem is that they are too often misleading. * But could using added AI-summaries get people started on writing better summaries? Possibly. * Is it worth looking at the detailed comments? Likely/Possibly. Best Regards, Hans OLSSON Modelica Multi-Physics/Control Software Engineering Senior Manager Dassault Systemes AB | IDEON Gateway, Scheelevägen 27 | SE22363 Lund | Sweden [3DS Logo] From: Anton Haumer ***@***.***> Sent: Friday, April 11, 2025 10:05 To: modelica/ModelicaStandardLibrary ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [modelica/ModelicaStandardLibrary] Experimenting with CodeRabbit AI generated reviews (Discussion #4611) It's ok to test such a funny AI, but I doubt it's worth a pence. I wouldn't feel comfortable with that rabbit doing reviews. Modelica is a little bit special compared to other programming languages. — Reply to this email directly, view It's ok to test such a funny AI, but I doubt it's worth a pence. I wouldn't feel comfortable with that rabbit doing reviews. Modelica is a little bit special compared to other programming languages. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/modelica/ModelicaStandardLibrary/discussions/4611*discussioncomment-12800825__;Iw!!FbCVDoc3r24SyHFW!-YWhSbV3ZjDpXKFktuqAC-skGwkWegxcHlqM6oriEhBzDVlMAIPfql2TPguZjowGpW-p-z1u2m41BW9hQzdgHAvB$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ADE25FR527RQ7TUBBU64QZ32Y5ZRHAVCNFSM6AAAAAB2ZUPJH6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEOBQGA4DENI__;!!FbCVDoc3r24SyHFW!-YWhSbV3ZjDpXKFktuqAC-skGwkWegxcHlqM6oriEhBzDVlMAIPfql2TPguZjowGpW-p-z1u2m41BW9hQ-JajrzW$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email. Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer https://www.3ds.com/privacy-policy/contact/

[casella]

I agree with @HansOlsson that these chatbot-related reports are no replacement for human judgment (as all things AI) but could help us finding issues that we might have missed. The problem is, how good the signal-to-noise ratio is. To me, it still seems a bit too low.

[AHaumer]

For god's sake, keep that rabbit quiet! It's a useless pain in the ass.

[beutlich]

Thanks for the feedback. With the end of the two-weeks-trial period the experiment stopped by uninstalling the CodeRabbitAI App again.