Contributing to the Guidelines

This is a rough draft of the proposal and review process for FAQs and exemplars, to be updated as we develop the process alongside our first set of FAQs and exemplars. Feedback and suggestions for refining this process are welcome!

We welcome contribution at all levels. To make it easy to "dive in", we have organized this document in increasing order of workload from the contributor's perspective. This document describes the mechanics of how to contribute the to guidelines. See the Style Guide for a description of what the guidelines should look like.

The Transparent Statistics Guidelines are organized into topics (e.g., "Effect Size"). Each topic contains an FAQ on that topic and one or more exemplars.

Commenting on or suggesting changes to an existing topic

We welcome any sort of feedback on the existing topics. The easiest way to "dive in" is to open an issue on Github, comment on an existing issue, or comment on a pull request. We value feedback, so if you are not sure where to send your feedback, open an issue!

Proposing minor changes to an existing topic

Minor changes are those that do not need broader review. These might be minor changes to style or formatting for clarity, but should not be substantial changes to content. They may also include adding a citation to an existing statement that does not change the content of the statement (merely gives it more support).

To propose minor changes to an existing topic:

1. Make those changes on your own branch or fork.

2. Make a pull request to merge those changes onto master.

3. Pull request is accepted by a core member (this may be the same person who is making the minor change).

Proposing major changes to an existing topic

Major changes are any that change meaning or add new content (e.g. a new exemplar in an existing topic).

1. Create initial version in RMarkdown/git. New content is created on the topic branch, or some branch/fork of the corresponding topic branch. If you are contributing to the topic and do not have write access to the main repo, you can create a pull request to have your content merged onto the topic branch in the main repo when you are ready.

2. Open pull request to track review. Authors create a pull request to merge the topic branch onto the master branch. The pull request should be titled something like New version of [topic] topic [with X, Y, Z] and have the review label, and should have two reviewers with subject matter expertise who were not involved in the creation of the new content assigned to it. This checklist should be the contents of the first comment in the pull request:

   - [ ] First review by another community member *with* subject matter expertise
       - [ ] Testing not necessary
   - [ ] Second review by another community member *with* subject matter expertise
       - [ ] Testing not necessary
   - [ ] Check & fix coding style
   - [ ] First test by another community member *without* subject matter expertise
   - [ ] Second test by another community member *without* subject matter expertise
   

   Coding style: see Style-Guide

3. Update pull request as review proceeds. As the above checklist is completed, the initial comment should be edited to tick those boxes like so:

   - [x] First review by another community member *with* subject matter expertise
       - [ ] Testing not necessary
   - [ ] Second review by another community member *with* subject matter expertise
       - [ ] Testing not necessary
   - [ ] Check & fix coding style
   - [ ] First test by another community member *without* subject matter expertise
   - [ ] Second test by another community member *without* subject matter expertise
   

   If the new content is of a type that does not require testing, during the first two steps of review, a reviewer can check their corresponding "Testing not necessary" box. If both such boxes are checked, the last two steps (testing) can be skipped. In general, all new exemplars should be tested, but other types of changes (e.g. new FAQ content) may not require testing. This is indicated by checking those boxes and striking out the test steps:

   - [x] First review by another community member *with* subject matter expertise
       - [x] Testing not necessary
   - [x] Second review by another community member *with* subject matter expertise
       - [x] Testing not necessary
   - [ ] Check & fix coding style
   - [ ] ~~First test by another community member *without* subject matter expertise~~
   - [ ] ~~Second test by another community member *without* subject matter expertise~~
   

   All comments and discussion about the topic should be made on this pull request so that everyone involved in the review is notified and so that relevant comments and changes are documented.

4. Accept request to merge topic onto master. Once the above reviews are complete and all boxes checked (or struck out), the pull request is accepted by a guidelines-core member who is not one of the primary topic authors.

Developing a new topic

To develop a new topic:

1. Create Google docs initial version. FAQ started in Google docs and developed among initial authors. Place each FAQ as one file in the FAQ folder on the shared Google Drive

2. Port to markdown/git. Once developed to a point where broader feedback is desired or once there are no more edits and comments for 5 days, FAQ is ported to Markdown in a separate branch on the main github repo named after the topic (henceforth the "topic branch"). Exemplar content is written into the same markdown file. All work on that topic will be conducted on that branch (not master) until it has passed review. Work on this guideline can also proceed on other branches and/or other repos, but all such work must eventually end up back on the topic branch in the main repo before review.

3. Open pull request to track review. Authors create a pull request to merge the topic branch onto the master branch. The pull request should be titled First version of [topic] topic and have the review label, and should have two reviewers with subject matter expertise who were not involved in the creation of the topic assigned to it. This checklist should be the contents of the first comment in the pull request:

   - [ ] First review by another community member *with* subject matter expertise
   - [ ] Second review by another community member *with* subject matter expertise
   - [ ] Check & fix coding style
   - [ ] First test by another community member *without* subject matter expertise
   - [ ] Second test by another community member *without* subject matter expertise
   

4. Update pull request as review proceeds. As the above checklist is completed, the initial comment should be edited to tick those boxes like so:

   - [x] First review by another community member *with* subject matter expertise
   - [ ] Second review by another community member *with* subject matter expertise
   - [ ] Check & fix coding style
   - [ ] First test by another community member *without* subject matter expertise
   - [ ] Second test by another community member *without* subject matter expertise
   

   Coding style: see Style-Guide

   All comments and discussion about the topic should be made on this pull request so that everyone involved in the review is notified and so that relevant comments and changes are documented.

5. Accept request to merge topic onto master. Once the above reviews are complete and all boxes checked, the pull request is accepted by a guidelines-core member who is not one of the primary topic authors.