DONOTMERGE a place to discuss my draft enrollment CIP #1559
Conversation
nfrisby
requested review from
jasagredo,
amesgen,
fraser-iohk,
dnadales and
geo2a
as code owners
nfrisby
force-pushed
the
nfrisby/enrollment-cip-discussion
branch
from
d6436bc to
2ddc629
Compare
|
@abailly I'd appreciate your thoughts on this, if any come to mind. Thanks. |
WIP-enrollment-CIP.md
Outdated
| We've simply had many less contributors and many fewer design iterations, each of which has been a massive investment. | ||
| My relevant suggestions are essentially just a few sentences. | ||
|
|
||
| > The authors of any CIP that explicitly affects the Cardano consensus protocol will need to communicate early and often with some combination of the IOG Researchers and the available experts on the existing implementation. |
There was a problem hiding this comment.
Have you considered adding the TSC too?
There was a problem hiding this comment.
Yeah, I didn't want to dilute the list. I think of the Researchers and the engineers as the end points of the spectrum of experts regarding the existing protocol, and in my mind the TSC is somewhere in the middle of that spectrum.
... but maybe that's naive, in particular "experts on usability/incentives/etc" might also need to be there. EG "product"? Hmm 🤔
There was a problem hiding this comment.
There are already discussions happening in the "Node diversity forum", see e.g https://forum.cardano.org/t/node-diversity-online-show-tell-session-2/146548/3 and we (my team) are working towards making available a language-agnostic testing environments (Antithesis and Tartarus) which could be leveraged to experiment and validate early proposals.
We should find a way to express a reasonably objective process to assess any submission under the Consensus CIP.
|
Regarding this Comment #1205 (comment) (but posting here)
Can you expand on this statement?
Note that in the future, there will be another node (Amaru), this project is making heavy investments in their knowledge of the consensus stack. So it is not far-fetched that other contributors to the consensus stack can be expected. A result of this will be suggestions on how to improve it as well. It makes sense that thus far only plutus and ledger were involved in such an "enrollment" CIP process, as both have direct users. With a second node, there is a direct user of consensus that can produce improvements. So, in this case, is the above addition you made enough to guide a CIP for consensus? I am not an expert, but I can imagine that a proposal for consensus might require some checks or basic requirements to call it a proper improvement (which might require significant effort from the author). Similarly, in the plutus "enrollment" CIP, it tries to express what things should be checked and thought about when dealing with plutus improvements, this so that the experts can easily verify that the CIP really improves things and also gives a solid argument for it. Given this, is such an CIP also not needed for consensus? |
| @@ -0,0 +1,9 @@ | |||
| In my current estimation, compared to the Plutus- and Ledger-enrolling CIPs above, any CIP enrolling Consensus would be trivial. | |||
There was a problem hiding this comment.
@perturbing said #1559 (comment):
any CIP enrolling Consensus would be trivial.
Can you expand on this statement?
By "it would be trivial" I mean it would just be the quoted paragraphs below, which is the slightest fraction of the Plutus and Ledger enrollment CIPs.
| @@ -0,0 +1,9 @@ | |||
| In my current estimation, compared to the Plutus- and Ledger-enrolling CIPs above, any CIP enrolling Consensus would be trivial. | |||
| We've simply had many less contributors and many fewer design iterations, each of which has been a massive investment. | |||
There was a problem hiding this comment.
@perturbing said #1559 (comment):
We've simply had many less contributors and many fewer design iterations, each of which has been a massive investment.
Note that in the future, there will be another node (Amaru), this project is making heavy investments in their knowledge of the consensus stack. So it is not far-fetched that other contributors to the consensus stack can be expected. A result of this will be suggestions on how to improve it as well.
It makes sense that thus far only plutus and ledger were involved in such an "enrollment" CIP process, as both have direct users. With a second node, there is a direct user of consensus that can produce improvements. So, in this case, is the above addition you made enough to guide a CIP for consensus?
Yes, it's my intention that "available experts on the existing implementation" included CF. I should reword it, eg to "experts on the existing implementationS".
I'm hoping @abailly et al hold similar Office Hours as we do, and so I could included that in the list of relevant meetings as well.
I am not an expert, but I can imagine that a proposal for consensus might require some checks or basic requirements to call it a proper improvement (which might require significant effort from the author). Similarly, in the plutus "enrollment" CIP, it tries to express what things should be checked and thought about when dealing with plutus improvements, this so that the experts can easily verify that the CIP really improves things and also gives a solid argument for it.
Given this, is such an CIP also not needed for consensus?
In my experience, the cognitive load of "all the things consensus needs to worry about" is just huge. An exhaustive list wouldn't help people---it'd be intimidating and people would bounce off it. My opinion is that inviting them to talk to us and trying to share a specific subset of our concerns (edit: relevant to their idea) with them to start with is much more likely to lead to useful consensus CIPs.
If people find that argument compelling, I could add a paragraph saying as much.
There was a problem hiding this comment.
In my experience, the cognitive load of "all the things consensus needs to worry about" is just huge. An exhaustive list wouldn't help people---it'd be intimidating and people would bounce off it. My opinion is that inviting them to talk to us and trying to share a specific subset of our concerns with them to start with is much more likely to lead to useful consensus CIPs.
I think this will be a wonderful addition that makes it explicit why no other checks are there.
Also tagging some of the other CIP editors for suggestions. @Ryun1 @rphair @Crypto2099
There was a problem hiding this comment.
We do not hold Office Hours yet, but are considering setting up a recurring meeting.
the cognitive load of "all the things consensus needs to worry about" is just huge
Is it true if you remove the "implementation details" of the existing consensus from the equation? In my modest experience with consensus, the level of complexity one has to deal with comes from a large part from having to deal with 6-7 years of accumulated and dense non-boring Haskell code. I am not discussing whether or not this is a good thing, just suggesting the conceptual aspects alone are probably much more manageable.
There was a problem hiding this comment.
I pushed up a commit with the edits implied by our discussion in this thread 👍 Thanks!
There was a problem hiding this comment.
the level of complexity one has to deal with comes from a large part from having to deal with 6-7 years of accumulated and dense non-boring Haskell code
The Hard Fork Combinator is one thing, but what I wrote above is not about the implementation. It's more about the ~10-15 people across multiple teams I ultimately have to talk when anyone wants to change any Consensus behavior: networking concerns, ledger concerns, incentive concerns, attack vectors the Researchers know about that I didn't yet, limits on the SPOs' patience for interface changes, dApps' patience, deployment burdens, etc, etc, etc.
There was a problem hiding this comment.
What you describe does not seem very specific to Cardano, it's the standard situation for any significantly used piece of distributed system: there are a lot of stakeholders whose needs and requirements one needs to address, and yes, that's complex like any social system.
Perhaps the problem is that this load is concentrated on the shoulders of a handful of persons, or even one person, you? I have worked for company running software comprising millions of line of codes and responsible for trading billions of dollars every day and one thing I have learned there is that no-one can know everything and the knowledge has to spread and codified in the form of documents, processes, etc.
WIP-enrollment-CIP.md
Outdated
| We've simply had many less contributors and many fewer design iterations, each of which has been a massive investment. | ||
| My relevant suggestions are essentially just a few sentences. | ||
|
|
||
| > The authors of any CIP that explicitly affects the Cardano consensus protocol will need to communicate early and often with some combination of the IOG Researchers and the available experts on the existing implementation. |
There was a problem hiding this comment.
There are already discussions happening in the "Node diversity forum", see e.g https://forum.cardano.org/t/node-diversity-online-show-tell-session-2/146548/3 and we (my team) are working towards making available a language-agnostic testing environments (Antithesis and Tartarus) which could be leveraged to experiment and validate early proposals.
We should find a way to express a reasonably objective process to assess any submission under the Consensus CIP.
| @@ -0,0 +1,9 @@ | |||
| In my current estimation, compared to the Plutus- and Ledger-enrolling CIPs above, any CIP enrolling Consensus would be trivial. | |||
| We've simply had many less contributors and many fewer design iterations, each of which has been a massive investment. | |||
There was a problem hiding this comment.
What you describe does not seem very specific to Cardano, it's the standard situation for any significantly used piece of distributed system: there are a lot of stakeholders whose needs and requirements one needs to address, and yes, that's complex like any social system.
Perhaps the problem is that this load is concentrated on the shoulders of a handful of persons, or even one person, you? I have worked for company running software comprising millions of line of codes and responsible for trading billions of dollars every day and one thing I have learned there is that no-one can know everything and the knowledge has to spread and codified in the form of documents, processes, etc.
|
|
||
| > The authors of any CIP that explicitly affects the Cardano consensus protocol will need to communicate early and often with some combination of the IOG Researchers and the available experts on the various existing implementations. | ||
| > We suggest beginning by scheduling a discussion on the agenda of a session of INTERSECT's Consensus Technical Working Group. | ||
| > The discussion there will likely involve scheduling additional discussions with the aforementioned Researchers and experts (eg the IOE Consensus Team holds weekly office hours for this sort of communication; Amaru does not hold Office Hours yet, but is planning to). |
There was a problem hiding this comment.
This CIP could be the seed from which to grow a proper process that would outlive the people actually implementing consensus now, which requires defining rules and process to change the rules.
What kind of discussion? Over what kind of artefacts? Who would be part of these discussions? What would be the requirements for a CIP on Consensus to even be discussed by experts? What kind of SLA would experts agree on? etc.
There was a problem hiding this comment.
To draw the analogy with research: you want to do some research, you publish a paper in some venue, the reviewers review your paper, possibly requesting changes and explanations, you update your paper, you get final approval or rejection.
There's a social contract and a clear process with some relatively well-known rules and SLAs (eg. you have to review papers throughly, and with deadlines).
In our case, a CIP could require some supporting code, perhaps a formal spec with some proofs about the implementation, conformance with previously known or provided test vectors, etc.
There was a problem hiding this comment.
responding to invitation in #1559 (comment):
I think this draft already has many of the points of what would be a complete CIP for the category... and I agree with all statements from other reviewers so far about why a CIP for the Consensus category would be welcome.
My only suggestion would be something helpful to the CIP process itself (though generally irrelevant to actual Consensus CIPs):
| > The only apparent risk to this plan is if there's an overwhelming influx of dead-end ideas. | ||
| > That doesn't seem likely at the moment, and we would adapt the scheme if it happens. |
There was a problem hiding this comment.
Over the years we've had some CIP submissions (and a CPS) intended to solve other perceived protocol or operations problems, but would in fact influence Consensus behaviour (e.g. adjusting parameters like block size / frequency or VRF function).
These won't come from Consensus subject matter experts and generally the authors won't be aware they have stepped into the "enormous" list of things that real Consensus changes would consider: in fact they'll likely be submitted in a different category altogether.
Therefore our editorial process (in this case, "screening" out inadmissible CIPs) would be helped if there were a section of the Consensus CIP which addressed the question "How to know if your proposed protocol change affects Consensus" or some such title.
Then editors could provide that deep link in the review process as a means of either impartially dropping the proposal or, maybe in some rare cases, pointing them to further resources to help them develop it into a real Consensus proposal.
DO NOT MERGE, this is just so we can have threaded conversations about this draft text
The initial content of this PR is just the content of this GitHub Issue Comment #1205 (comment)