generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 160
docs: Update overview page with comprehensive Prow introduction #553
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
deepsm007
wants to merge
1
commit into
kubernetes-sigs:main
Choose a base branch
from
deepsm007:docs/pr4-update-overview
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+104
−2
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,7 +7,9 @@ description: > | |
|
|
||
| #  | ||
|
|
||
| Prow is a Kubernetes based CI/CD system. Jobs can be triggered by various types of events and report their status to many different services. In addition to job execution, Prow provides GitHub automation in the form of policy enforcement, chat-ops via `/foo` style commands, and automatic PR merging. | ||
| Prow is a Kubernetes-based Continuous Integration and Continuous Deployment (CI/CD) system. It provides automated testing, code review automation, and project management features for Kubernetes and other open-source projects. Prow was originally developed as part of the Kubernetes project and has since become a widely-used CI/CD platform. | ||
|
|
||
| Jobs can be triggered by various types of events and report their status to many different services. In addition to job execution, Prow provides GitHub automation in the form of policy enforcement, chat-ops via `/foo` style commands, and automatic PR merging. | ||
|
|
||
| See the [GoDoc](https://pkg.go.dev/sigs.k8s.io/prow/pkg) for library docs. | ||
| Please note that these libraries are intended for use by prow only, and we do | ||
|
|
@@ -17,7 +19,107 @@ For a brief overview of how Prow runs jobs take a look at ["Life of a Prow Job"] | |
|
|
||
| To see common Prow usage and interactions flow, see the pull request interactions [sequence diagram](/images/pr-interactions-sequence.svg). | ||
|
|
||
| #### Functions and Features | ||
| ## What Problem Does It Solve? | ||
|
|
||
| Modern software development, especially in large open-source projects like Kubernetes, requires sophisticated CI/CD infrastructure to: | ||
|
|
||
| - **Automate Testing**: Run tests automatically on every pull request and commit | ||
| - **Code Review Automation**: Automate repetitive review tasks and enforce project policies | ||
| - **Project Management**: Automate issue management, labeling, and project workflows | ||
| - **Scalability**: Handle thousands of repositories and millions of test runs | ||
| - **Integration**: Integrate with GitHub, Gerrit, and other code hosting platforms | ||
| - **Resource Management**: Efficiently manage compute resources across multiple clusters | ||
|
|
||
| Prow solves these problems by providing a Kubernetes-native CI/CD platform that scales horizontally, integrates deeply with code hosting platforms, and offers extensive plugin-based automation. | ||
|
|
||
| ## Who Is It For? | ||
|
|
||
| ### Primary Users | ||
|
|
||
| 1. **Kubernetes SIG Testing**: The team responsible for maintaining Kubernetes CI infrastructure | ||
| 2. **Open Source Project Maintainers**: Teams managing large open-source projects requiring robust CI/CD | ||
| 3. **CI/CD Engineers**: Engineers building and maintaining CI/CD pipelines | ||
| 4. **Developers**: Contributors who interact with Prow through GitHub/Gerrit | ||
|
|
||
| ### Skill Level Requirements | ||
|
|
||
| - **Basic Users**: Need to understand YAML configuration and basic CI/CD concepts | ||
| - **Advanced Users**: Should be familiar with Kubernetes, Go programming, and CI/CD best practices | ||
| - **Contributors**: Need strong Go skills, Kubernetes API knowledge, and understanding of distributed systems | ||
|
|
||
| ## Key Features | ||
|
|
||
| ### 1. Job Execution | ||
| Prow executes CI jobs as Kubernetes Pods: | ||
| - **Presubmit Jobs**: Run on pull requests before merge | ||
| - **Postsubmit Jobs**: Run after code is merged | ||
| - **Periodic Jobs**: Run on a schedule (cron-based) | ||
| - **Batch Jobs**: Run multiple jobs in parallel | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not true, batch jobs test multiple PRs at once before merge to ensure they are compatible with each other |
||
|
|
||
| ### 2. Webhook Handling | ||
| Prow's `hook` component processes GitHub/Gerrit webhooks: | ||
| - Validates webhook signatures | ||
| - Triggers appropriate jobs based on events | ||
| - Executes plugins based on webhook content | ||
| - Manages job lifecycle | ||
|
|
||
| ### 3. Plugin System | ||
| Extensible plugin architecture for automation: | ||
| - **Label Management**: Automatically label PRs and issues | ||
| - **Milestone Management**: Track project milestones | ||
| - **Approval Automation**: `/approve`, `/lgtm` commands | ||
| - **Test Automation**: `/test`, `/retest` commands | ||
| - **Issue Management**: Auto-close, auto-assign, etc. | ||
| - **Custom Plugins**: Write your own plugins | ||
|
|
||
| ### 4. Tide - Automated Merging | ||
| Tide automatically merges pull requests when: | ||
| - All required tests pass | ||
| - Code is approved by required reviewers | ||
| - Branch protection rules are satisfied | ||
| - No merge conflicts exist | ||
|
|
||
| ### 5. Deck - Web UI | ||
| Deck provides a web interface for: | ||
| - Viewing job status and history | ||
| - Browsing test results and logs | ||
| - Managing Prow configuration | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we have this |
||
| - Viewing plugin help | ||
| - Spyglass integration for artifact viewing | ||
|
|
||
| ### 6. Spyglass - Artifact Viewer | ||
| Spyglass provides a unified interface for viewing: | ||
| - Build logs | ||
| - Test results (JUnit XML) | ||
| - Coverage reports | ||
| - Custom artifacts | ||
|
|
||
| ### 7. Status Reporting (Crier) | ||
| Crier reports job status back to: | ||
| - GitHub (commit status, PR comments) | ||
| - Gerrit (code review comments) | ||
| - Pub/Sub (for external integrations) | ||
| - Slack (notifications) | ||
|
|
||
| ### 8. Resource Management | ||
| - **Sinker**: Cleans up old ProwJobs and Pods | ||
| - **Scheduler**: Distributes jobs across clusters | ||
| - **Plank**: Manages job execution and Pod creation | ||
|
|
||
| ### 9. Multi-Cluster Support | ||
| Prow can distribute jobs across multiple Kubernetes clusters: | ||
| - Cluster selection based on job requirements | ||
| - Load balancing across clusters | ||
| - Cluster-specific configurations | ||
|
|
||
| ### 10. Integration Support | ||
| - **GitHub**: Full integration with GitHub API | ||
| - **Gerrit**: Support for Gerrit code review | ||
| - **Slack**: Notifications and chat-ops | ||
| - **Pub/Sub**: Event streaming | ||
| - **GCS**: Artifact storage | ||
|
|
||
| #### Additional Functions and Features | ||
|
|
||
| * Job execution for testing, batch processing, artifact publishing. | ||
| * GitHub events are used to trigger post-PR-merge (postsubmit) jobs and on-PR-update (presubmit) jobs. | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we have integration with anything else than GH and gerrit