Moderation of proposals?

[ChrisBarker-NOAA]

In the discussion of #148, the issue of a moderator was brought up:

#148 (comment)

Which refers to theRules for CF Conventions Changes:

http://cfconventions.org/rules.html

The question at hand was whether the "moderator" could be the proposer / a proponent of the proposal, or whether that is a conflict of interest.

I suggest that we should modify rules to make this more clear, an maybe make a change/addition:

In that doc the moderator:

"""
The moderator periodically summarises discussion on github, keeps it moving forward and tries to achieve a consensus. It is expected that everyone with an interest will contribute to the discussion and to achieving a consensus during this stage. During the discussion, if an objection is raised, answered and not reasserted, the moderator will assume the objection has been dropped. However, since consensus is the best outcome, it will be helpful if anyone who expresses an objection explicitly withdraws it on changing their mind or deciding to accept the majority view.

The moderator is encouraged to organize conference calls and/or webex-type interactions if this might help resolve an issue more quickly.
"""

This is a LOT of work I expect that we will be most likely to get someone to do it well if that have some "skin in the game" -- granted, lots of folks have skin in the game in the sense that they want CF to be as good as it can be, but I think someone that really wants the new feature is going to be more invested. And anyone that knows about and cares about CF will probably form an opinion anyway, so a truly "unbiased" moderator is kind of impossible.

So I suggest that the role of the moderator be divided (though it could still be one person, I suppose)

Role 1 (Propoonent?):

The moderator periodically summarises discussion on github, keeps it moving forward and tries to achieve a consensus.

The end result is a document that summarises both the proposal, and the discussion / rejected alternatives, objections, etc.

Role 2 (Moderator):

"attempt to move toward a decision on the proposal by summarising the discussion and indicating the outcome as consensus, near consensus, or not near consensus"

I guess what I'm suggesting is that the development of a final proposal and the guiding of a decion on that proposal be handled a bit differently.

I also think that the document that summarises both the proposal, and the discussion: e.g. rejected alternatives, objections, etc. be maintained in a single place -- could be the initial issue, or better, yet, somewhere else in the repo to be preserved for posterity. See #130 for more on that idea.

[martinjuckes]

Hi @ChrisBarker-NOAA ,

I agree that it should be possible for one person to be moderator and proponent, but there are risks to that approach. To avoid any confusion, could the guidelines suggest that "moderator" posts should be clearly separate from "proponent" or "contributor" posts, so that when a moderator/proponent switches roles it is clear.

Unfortunately, it is possible for discussion to evolve in a complex way, so that maintaining a record in one issue may not be practical in all cases, but I like the idea of using the first entry as some form of summary. Would it help if we asked that the first post be made in the "moderator" role, stating the scope but not proposing the solution, and the "proponent" role, even if it is the same person, should not be started until the 2nd post? We could then have the top issue describing the state of the discussion as seen by the moderator .. is there consensus etc, and the 2nd post could perhaps describe the proponents view of the current state of the proposal.

[ChrisBarker-NOAA]

It is certainly god to clearly define the perspective when someone is poting. However, I think that a challenge to the idea of an unbiased moderator is that anyone that cares enough about CF to be involved will probably form an opinion about a proposal, even if they weren't the original proponent.

Most import in all of this is that do have a single "document" that is updated as the discussion continues. The discussion can get pretty long (and sometimes contentious), and very hard to follow, particular for someone joining in the middle (or late). So the "moderator" or "proponent" or someone really needs to keep an up to date version of one document that lays out:

• The problem to be solved

• The proposed solution(s)

• The pros / cons of the options

• if / when a decision is made, a summary of the deciding factors.

This could be the initial entry in an issue -- I'm pretty sure that it can be constantly updated an edited.

If so, like Martin proposed, then it would probably be good for the initial entry to be a first draft of the above doc, and any real advocacy be kept for the second post on an issue.

Though I think it would be better to have a single place for proposals, see: #150 for my ideas on that.

But in any case, we really do need one place that folks can see a summary of the current state of the discussion.

[graybeal]

I like the idea very much of making the first issue both the statement of the problem, and the summary of the discussion. In GitHub issues any of the comments can be edited, including the first entry. If a template is created, it can be used for the first ticket and added to as discussion continues. For complex issues this format may be limiting, and you may want to allow instead an 'offline' summary that is pointed to by the ticket. I like #150 <#150> for complex issues, because having all those in one place can be quite illuminating. One caveat: Sometimes the problem definition evolves from what the original writer suggested. Of course Git has a record of the previous versions, but they can be awkward to get to. My suggestion is that if that is happening, someone save a copy of the original problem statement (for example, at the end of the first entry, or beginning of the second entry), so that people who are reading the thread much later can see what the early comments were responding to. The other place that the decision and deciding factors can go, which would be more 'traditional' for issue tracking systems, is in the comment where the ticket is closed. That's a common place to look for such info. John

…

On May 24, 2019, at 6:01 AM, Chris Barker ***@***.***> wrote: It is certainly god to clearly define the perspective when someone is poting. However, I think that a challenge to the idea of an unbiased moderator is that anyone that cares enough about CF to be involved will probably form an opinion about a proposal, even if they weren't the original proponent. Most import in all of this is that do have a single "document" that is updated as the discussion continues. The discussion can get pretty long (and sometimes contentious), and very hard to follow, particular for someone joining in the middle (or late). So the "moderator" or "proponent" or someone really needs to keep an up to date version of one document that lays out: The problem to be solved The proposed solution(s) The pros / cons of the options if / when a decision is made, a summary of the deciding factors. This could be the initial entry in an issue -- I'm pretty sure that it can be constantly updated an edited. If so, like Martin proposed, then it would probably be good for the initial entry to be a first draft of the above doc, and any real advocacy be kept for the second post on an issue. Though I think it would be better to have a single place for proposals, see: #150 <#150> for my ideas on that. But in any case, we really do need one place that folks can see a summary of the current state of the discussion. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#151?email_source=notifications&email_token=AAJVJUEJLUEBQKQFHP34WBTPW7RJPA5CNFSM4GDQ23NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWFIF5Q#issuecomment-495616758>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJVJUGPE4K3ZQIENKR5QL3PW7RJPANCNFSM4GDQ23NA>.

[JonathanGregory]

I would like to reinforce John's caveat above. I think the initial statement of the problem should remain at the head of the issue. The entire dialogue can be useful in tracing the evolution of the idea and the reasons for decisions, and it needs its starting point.

When ideas are being sketched, it seems to me that it's most convenient to do so in the issue. At that stage there probably is not a single current version. Once the principles have been decided, detailed wording will be debated, for which a pull request is helpful. Since there isn't an automatic link between these, it's important that if someone writes out their precise wording in a pull request, they make a link to it in the issue. I presume that enables the whole thing to be studied afterwards, doesn't it?

Jonathan

[ChrisBarker-NOAA]

My suggestion is that if that is happening, someone save a copy of the original problem statement (for example, at the end of the first entry, or beginning of the second entry)

I think it's REALLY important to have essentially one "document" that is a full summary of the propbel, the solution and the process that lead to that solution. That document should evolve as the discussion occurs, so that at any moment, some can join, and see the current status of the discussion, and when done, it serves as a record of the decision process.

Then, yes the actual text for the CF doc goes in a PR (with a link to the issue).

The other place that the decision and deciding factors can go, which would be more 'traditional' for issue tracking systems, is in the comment where the ticket is closed. That's a common place to look for such info.

yes, but that comment, be definition, can't be written or edited until the issue is ready to be closed, so not a practical place for a summary doc of the discussion-in-progess.

IF we want to keep the summary doc and the discussion in one place, then I think the only option is to have the first post be that doc -- and the actual discussion start in the first comment -- so if there is an "original problem statement" that should be preserved, it goes in the first comment, not the original post.

Again, I refer you to "Enhancement Proposals" as used by the Python development community:

https://www.python.org/dev/peps/

https://www.numpy.org/neps/index.html

https://matplotlib.org/devel/MEP/index.html

Having an archive of all of these is really helpful -- I think we should consider it -- but if not, at least a similar doc in the initial issue post.

[graybeal]

On May 29, 2019, at 2:50 PM, Chris Barker <notifications@github.com<mailto:notifications@github.com>> wrote: """My suggestion is that if that is happening, someone save a copy of the original problem statement (for example, at the end of the first entry, or beginning of the second entry), """ I think it's REALLY important to have essentially one "document" that is a full summary of the propbel, the solution and the process that lead to that solution. That document should evolve as the discussion occurs, so that at any moment, some can join, and see the current status of the discussion, and when done, it serves as a record of the decision process. For a community like Python where hundreds (or thousands or more) of not-so-technical people need to see a significant number of such summaries, I agree having all those things together and evolving is a nice way to present things. For a community of CF standard developers—where only a subset of the community access these requests, many of them only monitor or revisit a small subset of the requests, and *very* few of the community actually read all of them, especially once they are complete—for that group, I think it can be fine for the top ticket to contain the issue that the ticket (currently) addresses, and also the status of the ongoing discussion (updated only now and again); with the 'final answer' showing up at the end of that thread. In this 'lesser' model, all of the information can be discovered by the very few people who need to discover it in a detailed way (the ticket comments themselves will contain the entire process, so it doesn't have to be massively summarized); all of the core information can be quickly discovered by people who are newly joining the conversation; and ticket maintainers don't have to spend their time constantly updating the current status and process for those very few people who join late. It's really OK in my humble opinion for the late joiners to review the unsummarized part of the thread, that's how everything else in software maintenance works. And Jonathan or anyone can prompt a summary any time, as needed. Then, yes the actual text for the CF doc goes in a PR (with a link to the issue). "The other place that the decision and deciding factors can go, which would be more 'traditional' for issue tracking systems, is in the comment where the ticket is closed. That's a common place to look for such info." yes, but that comment, be definition, can't be written or edited until the issue is ready to be closed, so not a practical place for a summary doc of the discussion-in-progess. Yes, exactly why that summary comment *could* go at the end. (I wasn't trying to say the last comment should contain the summary-in-progress.) But I think either place, beginning or end, is OK. IF we want to keep the summary doc and the discussion in one place, then I think the only option is to have the first post be that doc -- and the actual discussion start in the first comment -- so if there is an "original problem statement" that should be preserved, it goes in the first comment, not the original post. Either way can be made to work, because the original statement could go at the end of the original post as an addendum. Again, I refer you to "Enhancement Proposals" as used by the Python development community: https://www.python.org/dev/peps/ https://www.numpy.org/neps/index.html https://matplotlib.org/devel/MEP/index.html Having an archive of all of these is really helpful -- I think we should consider it -- but if not, at least a similar doc in the initial issue post. I think having a nice template would help organize things in any case. (Maybe a bit too much for some of the simpler proposals, but we could try it.) Then if/when the discussion gets complicated, or we want more visibility for the discussion, the moderator could move the content into a more nicely formatted external document. (Which could even be on the GitHub wiki.) Those nicely formatted documents could form the corpus you want to see for the 'important stuff'—which I agree would be nice—while the nits and details that don't require much discussion could live on as (nicely formatted) issues. John — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#151?email_source=notifications&email_token=AAJVJUE7DIZYTEW3SVD5JXLPX33BNA5CNFSM4GDQ23NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWQXVCQ#issuecomment-497121930>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAJVJUFQVJSXPIGVLIZ24LDPX33BNANCNFSM4GDQ23NA>. ======================== John Graybeal Technical Program Manager Center for Expanded Data Annotation and Retrieval /+/ NCBO BioPortal Stanford Center for Biomedical Informatics Research 650-736-1632

[castelao]

By splitting into two roles, the proponent can be unrelated to the CF committee. Thus an important task for the moderator would be to guarantee that the code of conduct is followed during the discussions. It might be worth to be explicit about that in the new description.

[JonathanGregory]

Neither the proposer (proponent) nor the moderator needs to be a member of the CF committee. They should be different people if possible, because it's not so easy to be objective about your own proposal. The main problem with the rules is that people do not often volunteer to act as moderator. The rules are at http://cfconventions.org/rules.html

[martinjuckes]

Hi Jonathan: it is difficult to see how discussions are expected to start with a moderator volunteering ... the rules assume that every "new" proposal has a volunteer moderator, and don't give any advice on how to get to that stage. How are people expected to engage with the community to find a moderator if they need to have a moderator before using the community discussion platform. Perhaps there needs to be a separate area for issues seeking moderation, so that people who have fresh ideas can share them and get some feedback, and also use the platform to ask for moderators.
(edited by JBG)

[JonathanGregory]

Dear Martin
I don't think it's required for discussions to have a moderator in order to get going, but it helps to have one in place before lots has been said. The rules say, "A member of the conventions committee, or another suitably qualified person, volunteers to moderate the discussion. If no-one volunteers, the chairman of the committee will ask someone to do it." The first case happens sometimes; I don't know whether the second happens.
Best wishes
Jonathan

[castelao]

@JonathanGregory, you're right. With the split roles concept, I wrongly assumed that the moderator would be necessarily a member of the conventions committee, the first option in the rules for the CF Conventions.

With split roles, I imagined the moderator as someone very familiar with the procedures, that could oversee the proponent without necessarily doing the heavy lift of the proposal, i.e. promoting the discussions, editings, summaries, etc. Someone (the moderator) that would verify if the correct procedure for a proposal is followed, like the timeline; periodic summaries are adequate and all voices were considered; the code of conduct was respected; potentially catch conceptual flaws earlier, ... I'm assuming that the proponent is the most interested in the matter so will probably put the most energy on keeping it moving, while the moderator would guarantee that it is a proper and fair moving. In that case, if the moderator were not a member of the committee, it would be appointed by the committee, thus a trusted one.

I agree that it is not easy to be objective with your own proposal, but to help with that, doesn't need to be a moderator but anyone can join the discussion.

I also agree with @martinjuckes on the importance of assigning a moderator in the early stages of a proposal.

[martinjuckes]

@JonathanGregory : if user X raises an issue and at a later stage Y agrees to moderate it, is it possible for Y to edit the top level entry of the issue? Or is it expected that the moderator completes a review of the current state of the discussion and the proposer then copies that into the top entry? I can see room for confusion when several people are proposing different approaches to a complex problem. This may not be important, but I think we do need clarity about how people will find the latest post from the moderator.

@ChrisBarker-NOAA has raised the issue of the work load for the moderator. Asking someone to present a clear overview of the substance of a discussion which may involve misunderstandings arising from a range of people addressing a complex problem from multiple perspectives is a big ask. This could be simplified by restricting the required contribution of the moderator to a more procedural one: rather than "summarising the discussion", as stated in the current rules, I think it would be more reasonable to ask the moderator to summarise the state of the discussion (e.g. "X, Y and Z have raised objections to the proposal. The proposer has responded, but the objections have not been withdrawn. Y has made a modified proposal, but the proposer has outstanding objections to the revised proposal"). This would be a much easier task for the moderator, and leave the job of merging the original proposal with fresh ideas from the discussion with the proposer.

[JonathanGregory]

Dear Martin
Yes, I agree that summarising the content of the discussion is hard, and that may be one reason why few people volunteer to moderate. I feel that the moderator should be like the chair of a meeting, who would summarise the state of the discussion, as you say. If possible, the moderator should direct people's attention to things that have been raised but not settled.
I think we should keep the entire discussion in chronological order, including the moderator's summaries at the appropriate places. Does that make it too hard to find the latest summary? You can find it by starting at the bottom and working upwards.
We have to make the procedures as simple as possible, to maximise the chance that they will be followed, I feel.
Jonathan