From eb74510597765b49e3d71d20cbe53881867c44fc Mon Sep 17 00:00:00 2001 From: Gil Forcada Codinachs Date: Sun, 7 Jul 2024 19:26:36 +0200 Subject: [PATCH 1/3] feat: rewrite the continuous integration docs Explain Jenkins. --- .../core/continuous-integration.md | 107 ++++++------------ 1 file changed, 36 insertions(+), 71 deletions(-) diff --git a/docs/contributing/core/continuous-integration.md b/docs/contributing/core/continuous-integration.md index 9cf3e6fda..c53bcba25 100644 --- a/docs/contributing/core/continuous-integration.md +++ b/docs/contributing/core/continuous-integration.md @@ -7,105 +7,70 @@ myst: "keywords": "Plone, continuous integration, best practices" --- -# Essential continuous integration practices +# Continuous Integration -The {term}`CI` system at [jenkins.plone.org](https://jenkins.plone.org) is a shared resource for Plone core developers to notify them of regressions in Plone core code. +As a complex system build out of hundreds of small packages, Plone can be easily broken, inadvertently, with any change on any of its packages. -Build breakages are a normal and expected part of the development process. -The aim is to find errors and remove them as quickly as possible, without expecting perfection and zero errors. -However, there are some essential practices that you need follow to achieve a stable build: +For that a mix of a {term}`CI` system at [jenkins.plone.org](https://jenkins.plone.org) and GitHub Actions ensure that any change is thoroughly checked. +## Jenkins -## 1) Don't check in on a broken build +Jenkins is the primary source of truth to know if the complete test suite of any Plone versions in development is passing or not. -Do not make things more complicated for the developer who is responsible for breaking the build. +Jenkins is configured to react on changes done in any of the hundreds of repositories that all together make Plone. -If the build breaks, the developer has to identify the cause of the breakage as soon as possible and should fix it. -This strategy gives the developer the best option to find out what caused the breakage and fix it immediately. -Fixing the build is easier with a clear look at the problem. -Checking in further changes and triggering new builds will complicate matters and lead to more problems. +Whenever a change on any of those repositories happen, Jenkins starts a slew of testing jobs that ultimately answer the question of if those changes pass Plone's test suite. -If the build is broken over a longer period of time (more than a couple of hours) you should either: +### Triggering jobs (automatically) -- notify the developer who is responsible for the breakage -- fix the problem yourself -- revert the commit so you and others can continue to work +As running the Jenkins jobs is a bit expensive, in time, Jenkins does not run automatically on any new commit. -```{note} -There is one exception to this rule. -Sometimes there are changes or tests that depend on changes in other packages. -If this is the case, there is no way around breaking a single build for a certain period of time. -In this case, run the all tests locally with all the changes and commit them within a time frame of ten minutes. -``` - - -## 2) Always run all commit tests locally before committing - -Follow this practice so the build stays green, and other developers can continue to work without breaking the first rule. - -Remember that Plone development can happen all over the world, at all times. -Other developers may have checked in changes since your last synchronization. -These may interact with your work. - -Therefore it's essential that you merge changes from the main branch into your feature branch, then run the tests again, before you push your changes to GitHub. - -A common source of errors on check-in is to forget to add some files to the repository. -Use {command}`git status` to check and correct for this. -Also double-check to not check in files that should not be part of a package, such as editor configuration files and git submodules. +Rather, Jenkins waits for a special comment to be found in a pull request on a repository that is monitored by Jenkins. +```text +@jenkins-plone-org please run jobs +``` -## 3) Wait for commit tests to pass before moving on - -Always monitor the build's progress, and fix the problem right away if it fails. -If you introduced a regression, you have a far better chance of fixing the build sooner than later. -Also another developer might have committed in the meantime (by breaking rule 1), making things more complicated for you. - - -## 4) Never go home on a broken build - -Take into account the first rule of CI ("Don't check in on a broken build"). -Breaking the build essentially stops all other developers from working on it. -Therefore going home on a broken build—or even on a build that has not finished yet—is _not_ acceptable. -It will prevent all other developers from working, or they will need to fix the errors that you introduced. +Pasting the text above on a pull request, will trigger Jenkins to run all configured test suites related to the current pull request, **with the changes of the pull request itself**. +#### buildout.coredev special case -## 5) Always be prepared to revert to the previous revision +`buildout.coredev` is the special repository where our configuration and tooling to orchestrate Plone lies in. -For other developers to work on the build, you should always be prepared to revert to the previous passing revision. +Contrary to all the other repositories, whenever a commit is made on this repository, it automatically triggers Jenkins jobs to be run. +### Triggering jobs (manually) -## 6) Time-box fixing before reverting +Jenkins jobs can also be triggered manually by going to our Jenkins server, login in with your GitHub account, finding the right job, and triggering the `Run` button. -When the build breaks on check-in, try to fix it for ten minutes. -If, after ten minutes, you aren't finished with the solution, revert to the previous version from your version control system. -This way you will allow other developers to continue to work. +### Results +Once a jenkins job is finished, it will update its status icon on Jenkins UI itself, but also report back to GitHub, the result of the job. -## 7) Don't comment out failing tests +For pull requests, this means a green check mark or a red cross will be displayed at the end of the pull request, on pull requests listings, even on the favicon itself. -Once you begin to enforce the previous rule, the result is often that developers start commenting out failing tests in order to get the build passing again as quickly as possible. -While this impulse is understandable, it is _not acceptable_. +This gives the expected feedback to the pull request author, or any reviewer interested in knowing how a given pull request fared in Jenkins. Thus allowing pull request reviewers or authors to decide if the pull request can already be merged. -The tests were passing for a while and then start to fail. -This means that you either caused a regression, made assumptions that are no longer valid, or the application has changed the functionality being tested for a valid reason. +#### On failures -You should always either fix the code (if a regression has been found), modify the test (if one of the assumptions has changed), or delete it (if the functionality under test no longer exists). +If the Jenkins jobs reports failures, the statuses of the pull request will provide a `Details` link back to Jenkins. +That link leads back to the Jenkins job build for that pull request. -## 8) Take responsibility for all breakages that result from your changes +In that page, there is the list of test failures (if any) already listed. Clicking on them show the stacktrace that should help diagnose why there was an error on a given test. -If you commit a change and all the tests you wrote pass, but others break, the build is still broken. -This also applies to tests that fail in `buildout.coredev` and don't belong directly to the package you worked on. -This means that you have introduced a regression bug into the application. +On the side bar of the job build page in Jenkins there is also `Console Output` link. This leads to the **full job output**, note that is large, where it can be seen all the steps and debugging information that Jenkins generated running the job. -It is _your responsibility_ to fix all tests that do not pass because of your changes. +This can be helpful whenever the errors where not on a single test, but rather a general testing set up/tear down problem. -There are some tests in Plone that fail randomly, and the community is always working on fixing those. -If you think you hit such a test, try to fix it or re-run the Jenkins job to see if it passes again. +### Retrying -In any case, the developer who made the commit is responsible to make it pass. +If there were errors on the Jenkins jobs, and corrections have been made on the code, triggering the jenkins jobs again is a matter of giving the special comment on the GitHub pull request again: +```text +@jenkins-plone-org please run jobs +``` -## Further reading +Jenkins jobs can also be restarted within Jenkins UI itself. On the Jenkins job itself, and provided you are logged in, you can find a `Retry` link on the left toolbar. Clicking it will trigger a new Jenkins job with the same configuration as the one you are on. -These rules were taken from the excellent book "Continuous Delivery" by Jez Humble and David Farley (Addison Wesley), and have been adopted and rewritten for the Plone community. +Jenkins will update GitHub statuses accordingly. From 4605f6a780aed0e8ffd4a741d086ba904f6fd33d Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Tue, 20 Aug 2024 15:08:49 -0700 Subject: [PATCH 2/3] Remove unused term --- docs/glossary.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/glossary.md b/docs/glossary.md index 7776a1095..c2e0cdcda 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -107,10 +107,6 @@ Chef CloudFormation [AWS CloudFormation](https://aws.amazon.com/cloudformation/) gives developers and systems administrators an way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion. -Travis CI - Travis CI is a hosted, distributed continuous integration service used to build and test software projects hosted at GitHub. - Open source projects may be tested with limited runs via [travis-ci.com](https://www.travis-ci.com). - Solr [Solr](https://solr.apache.org/) is a popular, blazing-fast, open source enterprise search platform built on Apache Lucene. From e85e1aa0b30544af2bf7ddfa1aa832e62f7a9c3d Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Tue, 20 Aug 2024 16:44:18 -0700 Subject: [PATCH 3/3] Use @gforcada's contribution as a base to further enhance docs - Make statements more concise - Address the reader with "you can do X" instead of passive voice "X is done" - Add pretty inline icons and MyST markup magic - Remove italicization style of `guilabel` elements - Add to index, remove from exclusion --- .../contributing/core/icon-build-now.svg | 1 + docs/_static/contributing/core/icon-retry.svg | 1 + .../contributing/core/icon-terminal.svg | 1 + docs/_static/custom.css | 1 - docs/conf.py | 1 - .../core/continuous-integration.md | 81 +++++++++++-------- docs/contributing/core/index.md | 4 +- 7 files changed, 52 insertions(+), 38 deletions(-) create mode 100644 docs/_static/contributing/core/icon-build-now.svg create mode 100644 docs/_static/contributing/core/icon-retry.svg create mode 100644 docs/_static/contributing/core/icon-terminal.svg diff --git a/docs/_static/contributing/core/icon-build-now.svg b/docs/_static/contributing/core/icon-build-now.svg new file mode 100644 index 000000000..70b5d54b2 --- /dev/null +++ b/docs/_static/contributing/core/icon-build-now.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/_static/contributing/core/icon-retry.svg b/docs/_static/contributing/core/icon-retry.svg new file mode 100644 index 000000000..cd349c507 --- /dev/null +++ b/docs/_static/contributing/core/icon-retry.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/_static/contributing/core/icon-terminal.svg b/docs/_static/contributing/core/icon-terminal.svg new file mode 100644 index 000000000..9f15c271a --- /dev/null +++ b/docs/_static/contributing/core/icon-terminal.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/_static/custom.css b/docs/_static/custom.css index 41bdd4361..6242c425c 100644 --- a/docs/_static/custom.css +++ b/docs/_static/custom.css @@ -422,7 +422,6 @@ span.menuselection { padding: 4px 5px; font-size: 90%; font-weight: bold; - font-style: italic; white-space: nowrap; } diff --git a/docs/conf.py b/docs/conf.py index ffe125d00..13b6e95bf 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -125,7 +125,6 @@ "**/CONTRIBUTORS.rst", "**/LICENSE.rst", "**/README.rst", - "contributing/core/continuous-integration.md", "plone.restapi/.*", "plone.restapi/bin", "plone.restapi/docs/source/glossary.md", # There can be only one Glossary. diff --git a/docs/contributing/core/continuous-integration.md b/docs/contributing/core/continuous-integration.md index c53bcba25..70f754865 100644 --- a/docs/contributing/core/continuous-integration.md +++ b/docs/contributing/core/continuous-integration.md @@ -1,76 +1,89 @@ --- myst: html_meta: - "description": "Essential continuous integration practices" - "property=og:description": "Essential continuous integration practices" - "property=og:title": "Essential continuous integration practices" - "keywords": "Plone, continuous integration, best practices" + "description": "Working with continuous integration via Jenkins and GitHub workflows in Plone" + "property=og:description": "Working with continuous integration via Jenkins and GitHub workflows in Plone" + "property=og:title": "Continuous integration" + "keywords": "Plone, continuous integration, Jenkins, GitHub workflows" --- -# Continuous Integration +# Continuous integration -As a complex system build out of hundreds of small packages, Plone can be easily broken, inadvertently, with any change on any of its packages. +As a complex system built from hundreds of small packages, Plone can be easily and inadvertently broken from a change to any of its packages. +To help prevent breaking Plone, a mix of a {term}`continuous integration` (CI) systems of GitHub workflows and Jenkins thoroughly checks all changes. -For that a mix of a {term}`CI` system at [jenkins.plone.org](https://jenkins.plone.org) and GitHub Actions ensure that any change is thoroughly checked. +Do not use continuous integration as a replacement for running tests and other checks locally before pushing commits to a Plone repository. +See {ref}`test-and-code-quality-label` for an explanation, and {ref}`contributing-specific-contribution-policies-for-projects-label` for a project's specific contributing guidelines. -## Jenkins -Jenkins is the primary source of truth to know if the complete test suite of any Plone versions in development is passing or not. +## GitHub workflows + +Most Plone repositories use {term}`GitHub workflows` to run tests, check code quality, and perform other CI tasks on pushed commits to an open pull request. + +If a pull request is opened from a branch in the repository, then checks will automatically run. +Only members of certain Plone GitHub organization teams and users with write permission in the repository can do this. +Otherwise GitHub workflows do not run automatically, and require a member with write permission to run the checks. -Jenkins is configured to react on changes done in any of the hundreds of repositories that all together make Plone. +GitHub workflows should pass before running Jenkins checks and before merging an open pull request. -Whenever a change on any of those repositories happen, Jenkins starts a slew of testing jobs that ultimately answer the question of if those changes pass Plone's test suite. -### Triggering jobs (automatically) +## Jenkins + +[Jenkins](https://jenkins.plone.org) is the authoritative source to see whether the complete test suite passes for any Plone version under development. +Jenkins can be run on a change to any of Plone's hundreds of repositories that it monitors. -As running the Jenkins jobs is a bit expensive, in time, Jenkins does not run automatically on any new commit. -Rather, Jenkins waits for a special comment to be found in a pull request on a repository that is monitored by Jenkins. +### Run Jenkins + +Running the entire test suite on Jenkins takes at least 30 minutes, and consumes precious server resources. +For this reason, Jenkins is configured not to run automatically on each push to an open pull request. +Instead you should wait until you have completed pushing commits to an open pull request, and have seen that all GitHub workflows have passed. + +After all GitHub checks pass, and if Jenkins monitors the project, you can run Jenkins by adding a comment to the open pull request. ```text @jenkins-plone-org please run jobs ``` -Pasting the text above on a pull request, will trigger Jenkins to run all configured test suites related to the current pull request, **with the changes of the pull request itself**. +This will trigger Jenkins to run all configured test suites using the changes in the pull request. -#### buildout.coredev special case +You can also manually run Jenkins jobs by visiting our [Jenkins server](https://jenkins.plone.org/), logging in with your GitHub account, finding the right job, and clicking the Build now icon {guilabel}`Build now` button. -`buildout.coredev` is the special repository where our configuration and tooling to orchestrate Plone lies in. -Contrary to all the other repositories, whenever a commit is made on this repository, it automatically triggers Jenkins jobs to be run. +#### `buildout.coredev` special case -### Triggering jobs (manually) +`buildout.coredev` contains the configuration and tooling to orchestrate Plone. +Contrary to all the other repositories, whenever a push is made to this repository, it automatically triggers Jenkins to run its jobs. -Jenkins jobs can also be triggered manually by going to our Jenkins server, login in with your GitHub account, finding the right job, and triggering the `Run` button. ### Results -Once a jenkins job is finished, it will update its status icon on Jenkins UI itself, but also report back to GitHub, the result of the job. +Once a Jenkins job completes, it updates its status icon in the Jenkins UI, and reports the result of the job to the open pull request on GitHub. -For pull requests, this means a green check mark or a red cross will be displayed at the end of the pull request, on pull requests listings, even on the favicon itself. +In GitHub at the end of the pull request, a status indicator of either a green check mark {guilabel}`✅` or a red cross {guilabel}`❌` will be displayed for the job, followed by a {guilabel}`Details` link back to the Jenkins job build for that pull request. +The status indicator will also appear in the GitHub Octocat favicon in the web browser tab on the pull request, and on lists of pull requests. -This gives the expected feedback to the pull request author, or any reviewer interested in knowing how a given pull request fared in Jenkins. Thus allowing pull request reviewers or authors to decide if the pull request can already be merged. +You can use the results to decide whether to merge the pull request. -#### On failures -If the Jenkins jobs reports failures, the statuses of the pull request will provide a `Details` link back to Jenkins. +### Failure -That link leads back to the Jenkins job build for that pull request. +If the Jenkins job reports a failure, you can investigate its cause. +Click the {guilabel}`Details` link to show the stacktrace on Jenkins. -In that page, there is the list of test failures (if any) already listed. Clicking on them show the stacktrace that should help diagnose why there was an error on a given test. +In the side bar of the job build page in Jenkins, click the Console Output terminal icon {guilabel}` Console Output` link to show the full console output. +You can examine all the steps and debugging information that Jenkins generated while running the job. +This can be helpful whenever the errors were not in a single test, but rather a test set up or tear down or other problem. -On the side bar of the job build page in Jenkins there is also `Console Output` link. This leads to the **full job output**, note that is large, where it can be seen all the steps and debugging information that Jenkins generated running the job. -This can be helpful whenever the errors where not on a single test, but rather a general testing set up/tear down problem. +### Retry -### Retrying - -If there were errors on the Jenkins jobs, and corrections have been made on the code, triggering the jenkins jobs again is a matter of giving the special comment on the GitHub pull request again: +If there were errors on the Jenkins jobs, and you push additional commits to correct the errors, you can trigger the Jenkins jobs again by adding a comment to the open pull request in GitHub. ```text @jenkins-plone-org please run jobs ``` -Jenkins jobs can also be restarted within Jenkins UI itself. On the Jenkins job itself, and provided you are logged in, you can find a `Retry` link on the left toolbar. Clicking it will trigger a new Jenkins job with the same configuration as the one you are on. - +You can also restart Jenkins jobs in the Jenkins UI. +After logging in to Jenkins, and finding the appropriate job, you can click the Retry icon {guilabel}`Retry` link on the left toolbar to retry the failed job. Jenkins will update GitHub statuses accordingly. diff --git a/docs/contributing/core/index.md b/docs/contributing/core/index.md index 4b7773c0b..4bab9b6dd 100644 --- a/docs/contributing/core/index.md +++ b/docs/contributing/core/index.md @@ -299,7 +299,7 @@ Plone has a continuous integration ({term}`CI`) setup and follows CI rules. When you push a change to a Plone package, there may be GitHub workflows that run automatically. The CI package [mr.roboto](https://github.com/plone/mr.roboto) will perform some checks and suggest that you run Jenkins after all other CI runs. -%% See {doc}`continuous-integration` for more information. +See {doc}`continuous-integration` for more information. ## Additional material @@ -308,7 +308,7 @@ The CI package [mr.roboto](https://github.com/plone/mr.roboto) will perform some :maxdepth: 1 documentation -%% continuous-integration +continuous-integration mrdeveloper plips plip-review