Proposal for project management on Github

[vinothchandar]

Update: Where we finally landed after discussion is fairly different from original starting point below. Changes are being merged to site and Github

Today, we've moved JIRA to read-only mode. As we work on migrating relevant issues over from JIRA to GH, I want to start a discussion on how we organize dev tasks here..

Below is an outline consistent with RFC-85, as well as common standard practices adopted by GH projects.

Background

Previously, the Hudi community organized work in JIRA using:

• Epics → high-level initiatives
• Stories → user-facing requirements
• Subtasks → fine-grained technical tasks
• Issue types → Bug, Blocker, Improvement, Feature etc..
• Versions → for release tracking

Now that we have migrated to GitHub Issues, we need an equivalent process

Work Breakdown Structure

Epics

• Represented as GitHub Issues with type:epic
• Use a task list (- [ ]) inside the issue body to track linked stories or tasks.
• Epics can be grouped in GitHub Projects for visualization.

Stories

• Represented as GitHub Issues with type:story
• Should be linked to an Epic (via “parent relationship”).

Tasks

• Represented as GitHub Issues with type: task
• Fine-grained work items, often linked to a Story.
• Can be directly referenced in PRs using “Closes #ID”.

⸻

Issue Types & Labels

Additionally, we will rely on labels to replicate JIRA’s issue types:
• type: bug → Defects and regressions
• type: feature → New functionality
• type: improvement → Enhancements to existing functionality
• type: blocker → Critical issues that block releases

Additional dimensions:
• area: <component> (e.g., area: writer, area: compaction, area: metadata)
• priority: high | medium | low
• status: triaged | in-progress | blocked | done

⸻

Release Tracking

•	Use GitHub Milestones for release versions (e.g., 0.17.0, 0.18.0).
•	Issues are assigned to milestones to track inclusion in a release.
•	Each milestone will have:
•	A target release date
•	A changelog draft (via GitHub release notes generator)

⸻

Projects for Planning

•	Use GitHub Projects (v2) for community roadmaps and release tracking.
•	Columns: Backlog → In Progress → In Review → Done
•	Epics and Stories can be cards in Projects for visibility.

⸻

Contribution Workflow

1.	Contributor opens an issue (bug, feature, improvement, task).
2.	Triagers apply correct labels (type, area, priority, status).
3.	Issue may be linked to an Epic/Story if relevant.
4.	Issue assigned to a milestone if intended for a release.
5.	Work progresses via PRs, linked back to the issue.
6.	Upon merge, issue is automatically closed if PR references it.

Issue and PR titles

Instead of JIRA numbers, we can move towards denoting the nature of code change or issue easily in title.

Per https://www.conventionalcommits.org/en/v1.0.0/

We will use the following types

• (feat) → A new feature (e.g., (feat) Add support for async compaction)
• (fix) → A bug fix (e.g., (fix) Resolve NPE in HoodieTimeline)
• (docs) → Documentation changes only (e.g., (docs) Update quickstart for Spark 3.5)
• (style) → Code style, formatting, or lint-only changes (no logic impact)
• (refactor) → Code changes that neither fix a bug nor add a feature (e.g., cleanups, restructuring)
• (perf) → Performance improvements (e.g., (perf) Optimize bloom filter lookup)
• (test) → Adding or correcting tests
• (chore) → Tooling, build system, CI/CD, or maintenance tasks

Breaking changes are conveyed using a trailer to commit message and also a ! as deescribed e.g. feat!: or fix!:

⸻

Next Steps

We can use this discussion to update/improve the process and finalize.

• Community review of this proposal.
• Cleanup existing labels.
• Changes to PR linting scripts.
• Changes to Issue and GH templtes.
• Seed initial epics and stories.
• Update contributor guide with the new workflow.
• Migration and reorganization from JIRA.

[danny0405]

questions:

1. should we force an issue there for every PR, my preference is if people are not sure whether to fire the PR or how to fix, then an issue discussion is required, otherwise go fire the PR directly(the GH prompts are numerous daily and I preffer not to mess it up);
2. do we have templates for Epic/Story/Task, it would be very helpful to make everyone follow the criteria.

concerns:

1. the keyworkds and tags are so many, it's hard to form a criteria here, wondring how to automize it to ease the life of developers.

[vinothchandar]

I think this is a good callout. I suggest doing this, while adding a new section in pull request template
Describe the Issue this PR solves

The answer can

• Either be inline
• or fixes #

[vinothchandar]

Reviewers can suggest new issues to be filed as deemed appropriate..

[vinothchandar]

Updated to adopt https://www.conventionalcommits.org/en/v1.0.0/ for commit title standards.

[xushiyan]

+1. we can leverage tooling for CI validation https://github.com/conventional-changelog/commitlint?tab=readme-ov-file#getting-started

[vinothchandar]

cc @yihua whos already working on changing the linting..

[yihua]

The Github action is added on validating the PR title following the Conventional Commits spec.

[vinothchandar]

I will wait till monday EOD PST, to finalize these. and then proceed to making the changes.

please chime in if you have thoughts.

[xushiyan]

Looks good to me.

• labels with dimension prefix will be very helpful, making the labels self-explanatory. maintainers should not modify the label list randomly once labels are finalized with descriptions. Label modification needs to go through PMC, as labels will be critical info, being used for issue/pr filtering/searching
• every issue and PR targeted to a release should be added to the release milestone and have necessary labels, use auto labeler to label PR based on title, for e.g., a (feat): PR will be labeled with type:feature
• ideally a task issue should be closed by 1 PR. in case a task needs multiple PRs to close, maintainers make a call: either split the task so the PR closes one of the split task, or make sure PR does not use closing keywords (closes, fixes, etc) to link to the issue, only using issue number to link them.

[vinothchandar]

can we make a story/epic around all these devex improvement tasks..

[vinothchandar]

Moving forward with changes.. Tracking in #13906