Merge pull request #18 from manicminer/feature/skip-merge-requests-with-no-branch

Skip merge requests with no head/source branch

Author: manicminer

README.md

@@ -57,10 +57,21 @@ Written in Go, this is a cross-platform CLI utility that accepts the following r
   -rename-trunk-branch string
         specifies the new trunk branch name (incompatible with -rename-master-to-main)
   -report
-        report on primitives to be migrated instead of beginning migration```
+        report on primitives to be migrated instead of beginning migration
+  -skip-invalid-merge-requests
+        when true, will log and skip invalid merge requests instead of raising an error
+```
+
+## Authentication
+
+For authentication, the `GITLAB_TOKEN` and `GITHUB_TOKEN` environment variables must be populated. You cannot specify tokens as command-line arguments.
 
 Use the `-github-user` argument to specify the GitHub username for whom the authentication token was issued (mandatory). You can also specify this with the `GITHUB_USER` environment variable.
 
+Specify the location of a self-hosted instance of GitLab with the `-gitlab-domain` argument, or a GitHub Enterprise instance with the `-github-domain` argument.
+
+## Specify repositories
+
 You can specify an individual GitLab project with the `-gitlab-project` argument, along with the target GitHub repository with the `-github-repo` argument.
 
 Alternatively, you can supply the path to a CSV file with the `-projects-csv` argument, which should contain two columns:
@@ -69,18 +80,39 @@ Alternatively, you can supply the path to a CSV file with the `-projects-csv` ar
 gitlab-group/gitlab-project-name,github-org-or-user/github-repo-name
 ```
 
-For authentication, the `GITLAB_TOKEN` and `GITHUB_TOKEN` environment variables must be populated. You cannot specify tokens as command-line arguments.
+If the destination repository does not exist, this tool will attempt to create a private repository. If the destination repo already exists, it will be used unless you specify `-delete-existing-repos`
+
+> [!WARNING]  
+> To delete existing GitHub repos prior to migrating, pass the `-delete-existing-repos` argument. _This is potentially dangerous, you won't be asked for confirmation!_
+
+## Pull requests
 
 To enable migration of GitLab merge requests to GitHub pull requests (including closed/merged ones!), specify `-migrate-pull-requests`.
 
-To delete existing GitHub repos prior to migrating, pass the `-delete-existing-repos` argument. _This is potentially dangerous, you won't be asked for confirmation._
+Whilst the git repository itself will be migrated verbatim, the pull requests are managed using the GitHub API and typically will be authored by the person supplying the authentication token.
 
-Note: If the destination repository does not exist, this tool will attempt to create a private repository. If the destination repo already exists, it will be used unless you specify `-delete-existing-repos`
+Each pull request, along with every comment, will be prepended with a Markdown table showing the original author and some other metadata that is useful to know.  This is also used to map pull requests and their comments to their counterparts in GitLab and enables the tool to be idempotent.
 
-Specify the location of a self-hosted instance of GitLab with the `-gitlab-domain` argument, or a GitHub Enterprise instance with the `-github-domain` argument.
+As a bonus, if your GitLab users add the URL to their GitHub profile in the `Website` field of their GitLab profile, this tool will add a link to their GitHub profile in the markdown header of any PR or comment they originally authored.
+
+This tool also migrates merged/closed merge requests from your GitLab projects. It does this by reconstructing temporary branches in each repo, pushing them to GitHub, creating then closing the pull request, and lastly deleting the temporary branches. Once the tool has completed, you should not have any of these temporary branches in your repo - although GitHub will not garbage collect them immediately such that you can click the `Restore branch` button in any of these PRs.
+
+If you have a large number of merge requests, or projects with a long history spanning many GitLab upgrades, you may wish to specify the `-skip-invalid-merge-requests` argument. This will cause the tool to emit INFO messages for merge requests that it considers invalid, such as those that are still marked as Open but have no source/head branch, or where there is no diff for a closed merge request. Without this option, an error will be logged instead.
+
+_Example migrated pull request (open)_
+
+![example migrated open pull request](pr-example-open.jpeg)
+
+_Example migrated pull request (closed)_
+
+![example migrated closed pull request](pr-example-closed.jpeg)
+
+## Renaming the default/trunk branch
 
 As a bonus, this tool can transparently rename the trunk branch on your GitHub repository - enable with the `-rename-trunk-branch` argument. This will also work for any open merge requests as they are translated to pull requests.
 
+## Concurrency
+
 By default, 4 workers will be spawned to migrate up to 4 projects in parallel. You can increase or decrease this with the `-max-concurrency` argument. Note that due to GitHub API rate-limiting, you may not experience any significant speed-up. See [GitHub API docs](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api) for details.
 
 Specify `-loop` to continue migrating projects until canceled. This is useful for daemonizing the migration tool, or automatically restarting when migrating a large number of projects (or a small number of very large projects).
@@ -106,24 +138,6 @@ _Note that this tool performs a forced mirror push, so it's not recommended to r
 
 For pull requests and their comments, the corresponding IDs from GitLab are added to the Markdown header, this is parsed to enable idempotence (see next section).
 
-## Pull Requests
-
-Whilst the git repository will be migrated verbatim, the pull requests are managed using the GitHub API and typically will be authored by the person supplying the authentication token.
-
-Each pull request, along with every comment, will be prepended with a Markdown table showing the original author and some other metadata that is useful to know.  This is also used to map pull requests and their comments to their counterparts in GitLab and enables the tool to be idempotent.
-
-As a bonus, if your GitLab users add the URL to their GitHub profile in the `Website` field of their GitLab profile, this tool will add a link to their GitHub profile in the markdown header of any PR or comment they originally authored.
-
-This tool also migrates merged/closed merge requests from your GitLab projects. It does this by reconstructing temporary branches in each repo, pushing them to GitHub, creating then closing the pull request, and lastly deleting the temporary branches. Once the tool has completed, you should not have any of these temporary branches in your repo - although GitHub will not garbage collect them immediately such that you can click the `Restore branch` button in any of these PRs.
-
-_Example migrated pull request (open)_
-
-![example migrated open pull request](pr-example-open.jpeg)
-
-_Example migrated pull request (closed)_
-
-![example migrated closed pull request](pr-example-closed.jpeg)
-
 ## Contributing, reporting bugs etc...
 
 Please use GitHub issues & pull requests. This project is licensed under the MIT license.

helper.go

@@ -7,6 +7,7 @@ import (
 	"fmt"
 	"io"
 	"net/http"
+	"strings"
 	"time"
 
 	"github.com/google/go-github/v74/github"
@@ -108,6 +109,28 @@ func pointer[T any](v T) *T {
 	return &v
 }
 
+func parseProjectSlugs(slugs []string) ([]string, []string, error) {
+	if len(slugs) != 2 {
+		return nil, nil, fmt.Errorf("too many fields")
+	}
+
+	delimPosition := strings.LastIndex(slugs[0], "/")
+	gitlabPath := []string{
+		slugs[0][:delimPosition],
+		slugs[0][delimPosition+1:],
+	}
+	githubPath := strings.Split(slugs[1], "/")
+
+	if len(gitlabPath) != 2 {
+		return nil, nil, fmt.Errorf("invalid GitLab project: %s", slugs[0])
+	}
+	if len(githubPath) != 2 {
+		return nil, nil, fmt.Errorf("invalid GitHub project: %s", slugs[1])
+	}
+
+	return gitlabPath, githubPath, nil
+}
+
 func roundDuration(d, r time.Duration) time.Duration {
 	if r <= 0 {
 		return d

main.go

@@ -12,7 +12,6 @@ import (
 	"os/signal"
 	"regexp"
 	"strconv"
-	"strings"
 	"sync"
 	"time"
 
@@ -326,28 +325,6 @@ func main() {
 	}
 }
 
-func parseProjectSlugs(slugs []string) ([]string, []string, error) {
-	if len(slugs) != 2 {
-		return nil, nil, fmt.Errorf("too many fields")
-	}
-
-	delimPosition := strings.LastIndex(slugs[0], "/")
-	gitlabPath := []string{
-		slugs[0][:delimPosition],
-		slugs[0][delimPosition+1:],
-	}
-	githubPath := strings.Split(slugs[1], "/")
-
-	if len(gitlabPath) != 2 {
-		return nil, nil, fmt.Errorf("invalid GitLab project: %s", slugs[0])
-	}
-	if len(githubPath) != 2 {
-		return nil, nil, fmt.Errorf("invalid GitHub project: %s", slugs[1])
-	}
-
-	return gitlabPath, githubPath, nil
-}
-
 func printReport(ctx context.Context, projects []Project) {
 	logger.Debug("building report")
 

project.go

@@ -335,6 +335,17 @@ func (p *project) migrateMergeRequest(ctx context.Context, mergeRequest *gitlab.
 		}
 	}
 
+	if strings.EqualFold(mergeRequest.State, "opened") {
+		if _, err = p.repo.Branch(mergeRequest.SourceBranch); err != nil {
+			if errors.Is(err, git.ErrBranchNotFound) && skipInvalidMergeRequests {
+				logger.Info("skipping invalid merge request as source branch does not exist", "name", p.gitlabPath[1], "group", p.gitlabPath[0], "project_id", p.project.ID, "merge_request_id", mergeRequest.IID, "source_branch", mergeRequest.SourceBranch)
+				return false, nil
+			} else {
+				return false, fmt.Errorf("checking source branch for merge request: %v", err)
+			}
+		}
+	}
+
 	// Proceed to create temporary branches when migrating a merged/closed merge request that doesn't yet have a counterpart PR in GitHub (can't create one without a branch)
 	if pullRequest == nil && !strings.EqualFold(mergeRequest.State, "opened") {
 		logger.Trace("searching for existing branch for closed/merged merge request", "name", p.gitlabPath[1], "group", p.gitlabPath[0], "project_id", p.project.ID, "merge_request_id", mergeRequest.IID, "source_branch", mergeRequest.SourceBranch)