Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for migrating from Gitlab #9084

Merged
merged 52 commits into from
Apr 19, 2020
Merged
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
0a8bd8a
First stab at a Gitlab migrations interface.
aqtrans Nov 18, 2019
f58c065
Modify JS to show migration for Gitlab
aqtrans Nov 19, 2019
b10ed23
Properly strip out #gitlab tag from repo name
aqtrans Nov 19, 2019
823d181
Working Gitlab migrations!
aqtrans Nov 19, 2019
eaf52a4
Try #2 at trying to hide credentials.
aqtrans Nov 19, 2019
4efa640
Add go-gitlab dependency
aqtrans Nov 19, 2019
d651d11
Vendor go-gitlab
aqtrans Nov 19, 2019
9b38e86
Use gitlab.BasicAuthClient
aqtrans Nov 19, 2019
ee4cefa
Replaced repoPath with repoID globally.
aqtrans Nov 19, 2019
f61f9cd
Logging touchup
aqtrans Nov 19, 2019
1a47b38
Properly set private repo status.
aqtrans Nov 19, 2019
88b8918
Add go-gitlab vendor cache
aqtrans Nov 19, 2019
22a6080
Fix PR migrations:
aqtrans Nov 20, 2019
9f316b3
Ensure merged PRs are closed and set with the proper time
aqtrans Nov 20, 2019
c993d84
Remove copyright and some commented code
aqtrans Nov 20, 2019
0aa2da8
Rip out '#gitlab' based self-hosted Gitlab support
aqtrans Nov 20, 2019
356b5ac
Hide given credentials for migrated repos.
aqtrans Nov 19, 2019
0af580a
Use asset.URL directly, no point in parsing.
aqtrans Nov 25, 2019
936e3f0
Fix importing Milestones.
aqtrans Nov 27, 2019
9258610
Fix Gitlab Milestone migration if DueDate isn't set
aqtrans Dec 4, 2019
9befc70
Empty Milestone due dates properly return nil, not zero time
aqtrans Dec 4, 2019
61ad1a2
Add GITLAB_READ_TOKEN to drone unit-test step
aqtrans Dec 4, 2019
b12c8c2
Add working gitlab_test.go.
aqtrans Dec 4, 2019
476cb3b
Merge branch 'master' into gitlab-migrate
aqtrans Dec 5, 2019
40bc377
Fix linting issues
aqtrans Dec 5, 2019
52cfcde
Add modified JS files
aqtrans Dec 5, 2019
fe63bba
Remove pre-build JS files
aqtrans Dec 7, 2019
43eaf85
Merge branch 'master', remote-tracking branch 'upstream' into gitlab-…
aqtrans Dec 7, 2019
52b0021
Only merged PRs are marged as merged/closed
aqtrans Dec 8, 2019
7a66ced
Test topics
aqtrans Dec 8, 2019
e278651
Skip test if gitlab is inaccessible
aqtrans Dec 8, 2019
f9fe81f
Grab personal token from username, not password.
aqtrans Dec 9, 2019
97038d4
Merge branch 'master' of https://github.com/go-gitea/gitea into gitla…
aqtrans Jan 19, 2020
12bec11
Add SetContext() to GitlabDownloader.
aqtrans Jan 19, 2020
2a72164
Merge branch 'master' into gitlab-migrate
techknowlogick Feb 17, 2020
228a553
Merge branch 'master' of https://github.com/go-gitea/gitea into gitla…
aqtrans Mar 1, 2020
9a0d02c
Checking Updated field in Issues.
aqtrans Mar 1, 2020
0497f61
Merge branch 'gitlab-migrate' of github.com:aqtrans/gitea into gitlab…
aqtrans Mar 1, 2020
e731c2d
Actually fetch Issue Updated time from Gitlab
aqtrans Mar 1, 2020
958f2d7
Add Gitlab migration GetReviews() stub
aqtrans Mar 1, 2020
48efb1f
Fix Patch and Clone URLs
aqtrans Mar 1, 2020
4657268
Merge branch 'master' into gitlab-migrate
techknowlogick Mar 19, 2020
6edbb44
check Updated too
6543 Mar 31, 2020
b839aa4
fix mod
6543 Mar 31, 2020
a224698
Merge pull request #2 from 6543-forks/gitlab-m-fix
aqtrans Apr 8, 2020
00f8b3a
Merge branch 'master' into gitlab-migrate
aqtrans Apr 8, 2020
90b8f42
make vendor with go1.14
6543 Apr 10, 2020
7fec22e
Merge pull request #3 from 6543-forks/gitlab_migrate-fix
aqtrans Apr 11, 2020
29937a9
Merge branch 'master' into gitlab-migrate
lafriks Apr 17, 2020
4dd6e89
Merge branch 'master' into gitlab-migrate
6543 Apr 19, 2020
a651248
Merge branch 'master' into gitlab-migrate
lunny Apr 19, 2020
f5988e8
Merge branch 'master' into gitlab-migrate
lunny Apr 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add working gitlab_test.go.
A Personal Access Token, given in env variable GITLAB_READ_TOKEN
is required to run the test.
  • Loading branch information
aqtrans committed Dec 4, 2019
commit b12c8c20afdf78cbf73911a4980f38525d48a8b9
236 changes: 236 additions & 0 deletions modules/migrations/gitlab_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
// Copyright 2019 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package migrations

import (
"os"
"testing"
"time"

"code.gitea.io/gitea/modules/migrations/base"

"github.com/stretchr/testify/assert"
)

func TestGitlabDownloadRepo(t *testing.T) {
// Skip tests if Gitlab token is not found
gitlabPersonalAccessToken := os.Getenv("GITLAB_READ_TOKEN")
if gitlabPersonalAccessToken == "" {
t.Skip("skipped test because GITLAB_READ_TOKEN was not in the environment")
}

lafriks marked this conversation as resolved.
Show resolved Hide resolved
downloader := NewGitlabDownloader("https://gitlab.com", "gitea/test_repo", "", gitlabPersonalAccessToken)
if downloader == nil {
t.Fatal("NewGitlabDownloader is nil")
}
repo, err := downloader.GetRepoInfo()
assert.NoError(t, err)
// Repo Owner is blank in Gitlab Group repos
assert.EqualValues(t, &base.Repository{
Name: "test_repo",
Owner: "",
Description: "Test repository for testing migration from gitlab to gitea",
CloneURL: "https://gitlab.com/gitea/test_repo.git",
OriginalURL: "https://gitlab.com/gitea/test_repo",
}, repo)

topics, err := downloader.GetTopics()
lafriks marked this conversation as resolved.
Show resolved Hide resolved
assert.NoError(t, err)
assert.True(t, len(topics) == 0)

milestones, err := downloader.GetMilestones()
assert.NoError(t, err)
assert.True(t, len(milestones) >= 2)
t.Log(milestones)

for _, milestone := range milestones {
switch milestone.Title {
case "1.0":
assertMilestoneEqual(t, "", "1.0",
"",
"2019-11-28 08:42:30.301 +0000 UTC",
"2019-11-28 15:57:52.401 +0000 UTC",
"",
"closed", milestone)
case "1.1.0":
assertMilestoneEqual(t, "", "1.1.0",
"",
"2019-11-28 08:42:44.575 +0000 UTC",
"2019-11-28 08:42:44.575 +0000 UTC",
"",
"active", milestone)
}
}

labels, err := downloader.GetLabels()
assert.NoError(t, err)
assert.True(t, len(labels) >= 9)
for _, l := range labels {
switch l.Name {
case "bug":
assertLabelEqual(t, "bug", "d9534f", "", l)
case "documentation":
assertLabelEqual(t, "documentation", "f0ad4e", "", l)
case "confirmed":
assertLabelEqual(t, "confirmed", "d9534f", "", l)
case "enhancement":
assertLabelEqual(t, "enhancement", "5cb85c", "", l)
case "critical":
assertLabelEqual(t, "critical", "d9534f", "", l)
case "disucssion":
assertLabelEqual(t, "disucssion", "428bca", "", l)
case "suggestion":
assertLabelEqual(t, "suggestion", "428bca", "", l)
case "support":
assertLabelEqual(t, "support", "f0ad4e", "", l)
case "duplicate":
assertLabelEqual(t, "duplicate", "7F8C8D", "", l)
}
}

releases, err := downloader.GetReleases()
assert.NoError(t, err)
assert.EqualValues(t, []*base.Release{
{
TagName: "v0.9.99",
TargetCommitish: "0720a3ec57c1f843568298117b874319e7deee75",
Name: "First Release",
Body: "A test release",
Created: time.Date(2019, 11, 28, 9, 9, 48, 840000000, time.UTC),
PublisherID: 1241334,
PublisherName: "lafriks",
},
}, releases[len(releases)-1:])

// downloader.GetIssues()
issues, isEnd, err := downloader.GetIssues(1, 2)
assert.NoError(t, err)
assert.EqualValues(t, 2, len(issues))
assert.False(t, isEnd)

var (
closed1 = time.Date(2019, 11, 28, 8, 46, 23, 275000000, time.UTC)
closed2 = time.Date(2019, 11, 28, 8, 45, 44, 959000000, time.UTC)
)
assert.EqualValues(t, []*base.Issue{
{
Number: 1,
Title: "Please add an animated gif icon to the merge button",
Content: "I just want the merge button to hurt my eyes a little. :stuck_out_tongue_closed_eyes:",
Milestone: "1.0.0",
PosterID: 1241334,
PosterName: "lafriks",
State: "closed",
Created: time.Date(2019, 11, 28, 8, 43, 35, 459000000, time.UTC),
Labels: []*base.Label{
{
Name: "bug",
},
{
Name: "discussion",
},
},
Reactions: nil,
Closed: &closed1,
},
{
Number: 2,
Title: "Test issue",
Content: "This is test issue 2, do not touch!",
Milestone: "1.1.0",
PosterID: 1241334,
PosterName: "lafriks",
State: "closed",
Created: time.Date(2019, 11, 28, 8, 44, 46, 277000000, time.UTC),
Labels: []*base.Label{
{
Name: "duplicate",
},
},
Reactions: nil,
Closed: &closed2,
},
}, issues)

// downloader.GetComments()
comments, err := downloader.GetComments(2)
assert.NoError(t, err)
assert.EqualValues(t, 4, len(comments))
assert.EqualValues(t, []*base.Comment{
{
IssueIndex: 2,
PosterID: 1241334,
PosterName: "lafriks",
Created: time.Date(2019, 11, 28, 8, 44, 52, 501000000, time.UTC),
Content: "This is a comment",
Reactions: nil,
},
{
IssueIndex: 2,
PosterID: 1241334,
PosterName: "lafriks",
Created: time.Date(2019, 11, 28, 8, 45, 2, 329000000, time.UTC),
Content: "changed milestone to %2",
Reactions: nil,
},
{
IssueIndex: 2,
PosterID: 1241334,
PosterName: "lafriks",
Created: time.Date(2019, 11, 28, 8, 45, 45, 7000000, time.UTC),
Content: "closed",
Reactions: nil,
},
{
IssueIndex: 2,
PosterID: 1241334,
PosterName: "lafriks",
Created: time.Date(2019, 11, 28, 8, 45, 53, 501000000, time.UTC),
Content: "A second comment",
Reactions: nil,
},
}, comments[:4])

// downloader.GetPullRequests()
prs, err := downloader.GetPullRequests(1, 1)
assert.NoError(t, err)
assert.EqualValues(t, 1, len(prs))

assert.EqualValues(t, []*base.PullRequest{
{
Number: 4,
Title: "Test branch",
Content: "do not merge this PR",
Milestone: "1.0.0",
PosterID: 1241334,
PosterName: "lafriks",
State: "opened",
Created: time.Date(2019, 11, 28, 15, 56, 54, 104000000, time.UTC),
Labels: []*base.Label{
{
Name: "bug",
},
},
PatchURL: "https://gitlab.com/gitea/test_repo/merge_requests/2.patch",
Head: base.PullRequestBranch{
Ref: "feat/test",
CloneURL: "https://gitlab.com/gitea/test_repo/merge_requests/2",
SHA: "9f733b96b98a4175276edf6a2e1231489c3bdd23",
RepoName: "test_repo",
OwnerName: "lafriks",
},
Base: base.PullRequestBranch{
Ref: "master",
SHA: "",
OwnerName: "lafriks",
RepoName: "test_repo",
},
Closed: nil,
Merged: false,
MergedTime: nil,
MergeCommitSHA: "",
},
}, prs)
}