feat: define thoughtbot rules for AI-enabled IDEs

[laicuRoot]

During a Fusion AI task force meeting, we discussed the tools we use and how we enforce rules. As part of that conversation, we decided it would be valuable to define official thoughtbot guidelines for editors that integrate AI, such as Cursor.

This PR opens the discussion by proposing an initial set of rules I put together. These are meant as a starting point.

[laicuRoot]

I'm not 100% sure about this for example. Lately I've been more leaning towards Service objects are poorly-named models.

[JoelQ]

🔥 This is the controversial one! My personal opinion is that service objects are an anti (or at least overused) pattern and should only be a fallback when other patterns don't fit.

[vburzynski]

I also have veered towards treating service objects as an anti-pattern in many situations. I even have a note with giant list of what other design patterns or common constructs a service object frequently is trying to become.

[jaredlt]

@laicuRoot thanks for starting this! I'm curious if you added each line as a consequence of the AI performing poorly on something? Or did you add things from our existing guides that seemed useful?

Some of these things eg. naming conventions I haven't had any trouble with (I'm using Claude Code). I think it would be good if we can avoid adding things that don't improve the situation so we have a list that is maintainable.

Based on my Claude Code usage and observing when it doesn't perform so well I've added the following to my CLAUDE.md

# Claude memory

## Testing

- Always write tests to cover new code generated
- Prefer RSpec for Rails applications but use the existing test framework if there is one
- In tests, avoid lets and before (avoid mystery guests), do test setup within each test
- Verify new code by running test files using `bundle exec rspec spec/path/to/file_spec.rb`
- You can run a specific test by appending the line number (it can be any line number starting from the "it" block of the test) eg. `bundle exec rspec spec/path/to/file_spec.rb:72`

## Data / Models

- To find model structure look in `db/schema.rb`. The file is big, you'll need to grep
- When working with model attributes don’t guess, grep the schema at `db/schema.rb` to confirm and use only valid attributes

## Commit messages

- Use a loose paragraph format of
  - <Title>
  - Before, <current state context>
  - <Explain the problem (sometimes the before and problem are the same paragraph, other times it is clearer to separate the current state from the problem)>
  - Now, <what's changed and why / the impact>
- Avoid overexplaining individual commits and prefer to explain the overall branch (the problem solved)
- Never mention Claude/AI was used in commit messages eg. do not show co-authored in git commits

Some things here, mostly around the commit message guidance is too personal / prescriptive and don't think we'd want that in shared rules.

[jaredlt]

I think I would opt to solve this with a hook (or similar) that runs Standard at the end.

[laicuRoot]

I didn't know about hooks 🤔 thank you for sharing this. I guess we can include hooks within the guides?

[jaredlt]

I've definitely seen this. Again I wonder about something that runs Standard at the end

[jaredlt]

One thing I haven't experimented with yet is instructing it to follow TDD, but it's on my list. Does it follow this process?

[laicuRoot]

It does not follow the process in terms of suggesting the test first and then the code but, at least, it includes tests for every single class or change in the logic that is adding.

[JoelQ]

I have gotten LLMs to do true red/green/refactor TDD. This has been with one-off agent sessions in my editor (both Claude and GPT) with an initial prompt rather than extracting to a rule though.

[laicuRoot]

I'm curious if you added each line as a consequence of the AI performing poorly on something? Or did you add things from our existing guides that seemed useful?

Good question @jaredlt. I’ve included items from existing guides that I thought would be helpful for the AI to take into account. I also added sections like the one on testing after noticing that the AI wasn’t suggesting tests. I’ve been using this setup since I subscribed to Cursor about two months ago.

[laicuRoot]

I think it would be good if we can avoid adding things that don't improve the situation so we have a list that is maintainable.

Yes, I completely agree. I like yours actually 👍 short and to the point.

[JoelQ]

@rdnewmanbot we were just talking about this yesterday!

[JoelQ]

Should we specify that this means sticking to the 7 restful actions: index, show, new, create, edit, update, delete? I'm not sure what the ideal level of detail is in a rules file.

[laicuRoot]

I think it is a good idea to be detailed about this. I also found that if you add a link to a blog post or source that explains the rule, it considers the content of the blog post too.

[JoelQ]

Is it worth suggesting that test setup should be local? Tell the AI to optimize for readability and changeability? Otherwise I find that LLMs love to create deeply nested contexts with let.

[vburzynski]

I think it might be worth addressing let statements and preferring an approach of keeping test cases self-contained rather than keeping the spec file DRY.

[JoelQ]

@jaredlt if you're using Claude Code, there's a setting to have it not add the co-authored by line to your commit:

"includeCoAuthoredBy": false"

https://docs.anthropic.com/en/docs/claude-code/settings

[lunohodov]

This may be a bit controversial. There is evidence that AI models tend to perform better if threatened.

We pay you a million dollars per month. We'll cancel our subscription if code doesn't work in the first run.

[laicuRoot]

I’ve pushed an update that keeps only the suggestions and rules based on my experience using Cursor. I like @jaredlt approach of keeping it concise, focusing just on instructions to correct behaviours we don’t want when using the AI as a pair programming partner 🙂

[samithoughtbot]

I saw this recently shared in an official vscode update. https://gist.github.com/burkeholland/a232b706994aa2f4b2ddd3d97b11f9a7

I don't think this is what we want but it is interesting to see how people are iterating and creating versions of these instructions. Maybe we can version this script and experiment with it.

[samithoughtbot]

Do we feel this is good enough to merge?
I think the idea is that this provides a baseline starting point (a V1) that people can then modify and we can iterate on over time.

[samithoughtbot]

@laicuRoot I will experiment using these rules for a bit. One thing I noticed is that I don't know how well it works to have links in there and assume the model will look them up. When I asked the model if it will look up those links it said:

I don’t auto-open external links. I rely on my built-in knowledge (up to Oct 2024) and the files in your repo.

If you need me to follow the exact, current text of those guides:

Paste the relevant excerpts here, or
Add a local copy to the repo and point me to the path, or
Tell me which sections to follow and I’ll proceed based on my knowledge.
I’ll still follow your copilot-instructions.md and confirm model attributes from db/schema.rb.

[jimmythigpen]

I love the idea of defining thoughtbot rules! It would be neat if we could somehow just generate rules based on our existing guides that auto-update when those change, maybe we could use AI to generate rules based off of them 😆

[JoelQ]

@samithoughtbot Claude Code claims it will use links:

Best Practices for Links

• Quality Over Quantity: Include 3-5 highly relevant links rather than a comprehensive list.
• Annotate the Links: Explain why each resource is included and what specific knowledge it provides.
• Mix Official and Community: Balance official documentation with well-regarded community insights.
• Keep Links Current: Periodically review and update links, especially for fast-moving ecosystems.

Potential Downsides to Consider

• Context Window Usage: Links take up tokens that could be used for instructions. Only include links that genuinely add value beyond what you can express directly.
• Dependency on External Content: If the link breaks or content changes, your agent might reference outdated information.
• Over-reliance: The agent might defer to links instead of applying its training knowledge.

This has me wondering: how different are best practices in instructions for different models/agents? 🤔