Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added test environment for mssql #4282

Merged
merged 43 commits into from
Dec 12, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
6570d21
Added test environment for m$sql
kolaente Jun 20, 2018
d365cd6
Added template for test environment for m$sql
kolaente Jun 20, 2018
2aa2f19
Fix password
kolaente Jun 20, 2018
38e8b9c
Fix password (again)
kolaente Jun 20, 2018
6e3e458
Fix password (again again)
kolaente Jun 20, 2018
f03fc16
Fix db
kolaente Jun 20, 2018
002b300
Ci trigger (Looking at you drone....)
kolaente Jun 20, 2018
fa731e2
Ci trigger (Looking at you drone....)
kolaente Jun 20, 2018
8b2e67e
Ci trigger (Looking at you drone....)
kolaente Jun 20, 2018
f5069eb
Ci trigger (Looking at you drone....)
kolaente Jun 20, 2018
3fc4bb3
Create master database for mssql integration tests
jonasfranz Jul 5, 2018
6ebc478
Merge remote-tracking branch 'upstream/master' into mssql-test
jonasfranz Jul 5, 2018
1c7c44f
Create database only if master do not exist
jonasfranz Jul 5, 2018
729a512
Fix mssql integration tests by using custom database "gitea"
jonasfranz Jul 5, 2018
8ba423a
Merge branch 'master' into mssql-test
jonasfranz Jul 5, 2018
d975ba3
Merge branch 'master' of https://github.com/go-gitea/gitea into mssql…
kolaente Jul 6, 2018
ed8008d
Moved defer
kolaente Jul 6, 2018
63681b7
Merge branch 'master' into mssql-test
lafriks Jul 12, 2018
d1907f9
Merge branch 'master' into mssql-test
techknowlogick Jul 13, 2018
c5e48e4
Merge branch 'master' into mssql-test
jonasfranz Jul 13, 2018
b889df5
Merge branch 'master' into mssql-test
jonasfranz Oct 8, 2018
b377bea
Merge branch 'master' into mssql-test
kolaente Oct 9, 2018
f104ce7
Merge branch 'master' into mssql-test
kolaente Oct 17, 2018
b07270d
Merge branch 'master' into mssql-test
kolaente Oct 21, 2018
f14225e
Merge branch 'master' into mssql-test
lunny Oct 22, 2018
f02366c
Merge branch 'master' into mssql-test
techknowlogick Oct 29, 2018
4f3508b
Merge branch 'master' into mssql-test
lunny Oct 29, 2018
98d0f22
Merge branch 'master' into mssql-test
lunny Oct 30, 2018
1d0c7dd
Merge branch 'master' into mssql-test
techknowlogick Oct 30, 2018
b2ac9c6
Merge branch 'master' into mssql-test
lafriks Oct 30, 2018
f429734
Merge branch 'master' into mssql-test
lunny Oct 31, 2018
5318157
Merge branch 'master' into mssql-test
lunny Oct 31, 2018
b3fa696
Merge branch 'master' into mssql-test
lunny Nov 2, 2018
e5efe98
Merge branch 'master' into mssql-test
techknowlogick Nov 4, 2018
ee146b0
Merge branch 'master' into mssql-test
techknowlogick Nov 20, 2018
543e997
Merge branch 'master' into mssql-test
techknowlogick Nov 29, 2018
15715d6
Merge branch 'master' into mssql-test
lunny Dec 11, 2018
46da16e
Merge branch 'master' into mssql-test
kolaente Dec 11, 2018
b2196d8
bump xorm
kolaente Dec 11, 2018
d6ad629
updated xorm
kolaente Dec 11, 2018
1a27388
Merge branch 'master' into mssql-test
kolaente Dec 11, 2018
6f5200c
Fixed build
kolaente Dec 11, 2018
96f7f86
Merge branch 'master' into mssql-test
techknowlogick Dec 11, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .drone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,20 @@ pipeline:
when:
event: [ push, tag, pull_request ]

test-mssql:
Copy link
Member

@techknowlogick techknowlogick Jul 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: I was being lazy, pay attention to @JonasFranzDEV as his solution is better than mine.
before the test steps (look at group test) you could have a step that creates the DB like:

mssql-create-db:
   image: microsoft/mssql-tools
   commands:
    - sqlcmd -S mssql -U sa -P MwantsaSecurePassword1 -Q "CREATE DATABASE master"
   when:
      event: [ push, tag, pull_request ]

I haven't tested the above (or indentation) but something like it should work

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The better option might be to create the database in code like we do for pgsql and mysql:
https://github.com/go-gitea/gitea/blob/master/integrations/integration_test.go#L110

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've implemented it and the test works now

image: golang:1.10
pull: true
group: test
environment:
TAGS: bindata
TEST_LDAP: "1"
commands:
- curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash
- apt-get install -y git-lfs
- make test-mssql
when:
event: [ push, tag, pull_request ]

generate-coverage:
image: golang:1.11
pull: true
Expand Down Expand Up @@ -347,6 +361,15 @@ services:
when:
event: [ push, tag, pull_request ]

mssql:
image: microsoft/mssql-server-linux:latest
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=MwantsaSecurePassword1
- MSSQL_PID=Standard
when:
event: [ push, tag, pull_request ]

ldap:
image: gitea/test-openldap:latest
when:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,14 @@ coverage.all
/integrations/gitea-integration-mysql
/integrations/gitea-integration-pgsql
/integrations/gitea-integration-sqlite
/integrations/gitea-integration-mssql
/integrations/indexers-mysql
/integrations/indexers-pgsql
/integrations/indexers-sqlite
/integrations/indexers-mssql
/integrations/mysql.ini
/integrations/pgsql.ini
/integrations/mssql.ini
/node_modules


Expand Down
4 changes: 2 additions & 2 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 1 addition & 2 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ ignored = ["google.golang.org/appengine*"]

[[override]]
name = "github.com/go-xorm/xorm"
#version = "0.6.5"
revision = "ad69f7d8f0861a29438154bb0a20b60501298480"
revision = "401f4ee8ff8cbc40a4754cb12192fbe4f02f3979"

[[override]]
name = "github.com/go-sql-driver/mysql"
Expand Down
19 changes: 16 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ TEST_PGSQL_HOST ?= pgsql:5432
TEST_PGSQL_DBNAME ?= testgitea
TEST_PGSQL_USERNAME ?= postgres
TEST_PGSQL_PASSWORD ?= postgres
TEST_MSSQL_HOST ?= mssql:1433
TEST_MSSQL_DBNAME ?= gitea
TEST_MSSQL_USERNAME ?= sa
TEST_MSSQL_PASSWORD ?= MwantsaSecurePassword1

ifeq ($(OS), Windows_NT)
EXECUTABLE := gitea.exe
Expand All @@ -74,9 +78,9 @@ clean:
$(GO) clean -i ./...
rm -rf $(EXECUTABLE) $(DIST) $(BINDATA) \
integrations*.test \
integrations/gitea-integration-pgsql/ integrations/gitea-integration-mysql/ integrations/gitea-integration-sqlite/ \
integrations/indexers-mysql/ integrations/indexers-pgsql integrations/indexers-sqlite \
integrations/mysql.ini integrations/pgsql.ini
integrations/gitea-integration-pgsql/ integrations/gitea-integration-mysql/ integrations/gitea-integration-sqlite/ integrations/gitea-integration-mssql/ \
integrations/indexers-mysql/ integrations/indexers-pgsql integrations/indexers-sqlite integrations/indexers-mssql \
integrations/mysql.ini integrations/pgsql.ini integrations/mssql.ini

.PHONY: fmt
fmt:
Expand Down Expand Up @@ -204,6 +208,11 @@ generate-ini:
-e 's|{{TEST_PGSQL_USERNAME}}|${TEST_PGSQL_USERNAME}|g' \
-e 's|{{TEST_PGSQL_PASSWORD}}|${TEST_PGSQL_PASSWORD}|g' \
integrations/pgsql.ini.tmpl > integrations/pgsql.ini
sed -e 's|{{TEST_MSSQL_HOST}}|${TEST_MSSQL_HOST}|g' \
-e 's|{{TEST_MSSQL_DBNAME}}|${TEST_MSSQL_DBNAME}|g' \
-e 's|{{TEST_MSSQL_USERNAME}}|${TEST_MSSQL_USERNAME}|g' \
-e 's|{{TEST_MSSQL_PASSWORD}}|${TEST_MSSQL_PASSWORD}|g' \
integrations/mssql.ini.tmpl > integrations/mssql.ini

.PHONY: test-mysql
test-mysql: integrations.test generate-ini
Expand All @@ -213,6 +222,10 @@ test-mysql: integrations.test generate-ini
test-pgsql: integrations.test generate-ini
GITEA_ROOT=${CURDIR} GITEA_CONF=integrations/pgsql.ini ./integrations.test

.PHONY: test-mssql
test-mssql: integrations.test generate-ini
GITEA_ROOT=${CURDIR} GITEA_CONF=integrations/mssql.ini ./integrations.test

.PHONY: bench-sqlite
bench-sqlite: integrations.sqlite.test
GITEA_ROOT=${CURDIR} GITEA_CONF=integrations/sqlite.ini ./integrations.sqlite.test -test.cpuprofile=cpu.out -test.run DontRunTests -test.bench .
Expand Down
13 changes: 13 additions & 0 deletions integrations/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ func TestMain(m *testing.M) {
helper = &testfixtures.PostgreSQL{}
} else if setting.UseSQLite3 {
helper = &testfixtures.SQLite{}
} else if setting.UseMSSQL {
helper = &testfixtures.SQLServer{}
} else {
fmt.Println("Unsupported RDBMS for integration tests")
os.Exit(1)
Expand Down Expand Up @@ -130,6 +132,17 @@ func initIntegrationTest() {
if _, err = db.Exec("CREATE DATABASE testgitea"); err != nil {
log.Fatalf("db.Exec: %v", err)
}
case setting.UseMSSQL:
host, port := models.ParseMSSQLHostPort(models.DbCfg.Host)
db, err := sql.Open("mssql", fmt.Sprintf("server=%s; port=%s; database=%s; user id=%s; password=%s;",
host, port, "master", models.DbCfg.User, models.DbCfg.Passwd))
if err != nil {
log.Fatalf("sql.Open: %v", err)
}
if _, err := db.Exec("If(db_id(N'gitea') IS NULL) BEGIN CREATE DATABASE gitea; END;"); err != nil {
log.Fatalf("db.Exec: %v", err)
}
defer db.Close()
}
routers.GlobalInit()
}
Expand Down
72 changes: 72 additions & 0 deletions integrations/mssql.ini.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod

[database]
DB_TYPE = mssql
HOST = {{TEST_MSSQL_HOST}}
NAME = {{TEST_MSSQL_DBNAME}}
USER = {{TEST_MSSQL_USERNAME}}
PASSWD = {{TEST_MSSQL_PASSWORD}}
SSL_MODE = disable

[indexer]
ISSUE_INDEXER_PATH = integrations/indexers-mssql/issues.bleve
REPO_INDEXER_ENABLED = true
REPO_INDEXER_PATH = integrations/indexers-mssql/repos.bleve

[repository]
ROOT = integrations/gitea-integration-mssql/gitea-repositories

[repository.local]
LOCAL_COPY_PATH = tmp/local-repo-mssql
LOCAL_WIKI_PATH = tmp/local-wiki-mssql

[server]
SSH_DOMAIN = localhost
HTTP_PORT = 3003
ROOT_URL = https://localhost:3003/
DISABLE_SSH = false
SSH_LISTEN_HOST = localhost
SSH_PORT = 2201
START_SSH_SERVER = true
LFS_START_SERVER = true
LFS_CONTENT_PATH = data/lfs-mssql
OFFLINE_MODE = false
LFS_JWT_SECRET = Tv_MjmZuHqpIY6GFl12ebgkRAMt4RlWt0v4EHKSXO0w
APP_DATA_PATH = integrations/gitea-integration-mssql/data

[mailer]
ENABLED = false

[service]
REGISTER_EMAIL_CONFIRM = false
ENABLE_NOTIFY_MAIL = false
DISABLE_REGISTRATION = false
ENABLE_CAPTCHA = false
REQUIRE_SIGNIN_VIEW = false
DEFAULT_KEEP_EMAIL_PRIVATE = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
NO_REPLY_ADDRESS = noreply.example.org

[picture]
DISABLE_GRAVATAR = false
ENABLE_FEDERATED_AVATAR = false

[session]
PROVIDER = file
PROVIDER_CONFIG = data/sessions-mssql

[log]
MODE = console,file
ROOT_PATH = mssql-log

[log.console]
LEVEL = Warn

[log.file]
LEVEL = Debug

[security]
INSTALL_LOCK = true
SECRET_KEY = 9pCviYTWSb
INTERNAL_TOKEN = eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYmYiOjE0OTU1NTE2MTh9.hhSVGOANkaKk3vfCd2jDOIww4pUk0xtg9JRde5UogyQ
7 changes: 4 additions & 3 deletions models/models.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ type Engine interface {
Count(...interface{}) (int64, error)
Decr(column string, arg ...interface{}) *xorm.Session
Delete(interface{}) (int64, error)
Exec(string, ...interface{}) (sql.Result, error)
kolaente marked this conversation as resolved.
Show resolved Hide resolved
Exec(...interface{}) (sql.Result, error)
Find(interface{}, ...interface{}) error
Get(interface{}) (bool, error)
ID(interface{}) *xorm.Session
Expand Down Expand Up @@ -200,7 +200,8 @@ func getPostgreSQLConnectionString(DBHost, DBUser, DBPasswd, DBName, DBParam, DB
return
}

func parseMSSQLHostPort(info string) (string, string) {
// ParseMSSQLHostPort splits the host into host and port
func ParseMSSQLHostPort(info string) (string, string) {
host, port := "127.0.0.1", "1433"
if strings.Contains(info, ":") {
host = strings.Split(info, ":")[0]
Expand Down Expand Up @@ -235,7 +236,7 @@ func getEngine() (*xorm.Engine, error) {
case "postgres":
connStr = getPostgreSQLConnectionString(DbCfg.Host, DbCfg.User, DbCfg.Passwd, DbCfg.Name, Param, DbCfg.SSLMode)
case "mssql":
host, port := parseMSSQLHostPort(DbCfg.Host)
host, port := ParseMSSQLHostPort(DbCfg.Host)
connStr = fmt.Sprintf("server=%s; port=%s; database=%s; user id=%s; password=%s;", host, port, DbCfg.Name, DbCfg.User, DbCfg.Passwd)
case "sqlite3":
if !EnableSQLite3 {
Expand Down
30 changes: 30 additions & 0 deletions vendor/github.com/go-xorm/xorm/context_cache.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion vendor/github.com/go-xorm/xorm/dialect_mssql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion vendor/github.com/go-xorm/xorm/dialect_mysql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion vendor/github.com/go-xorm/xorm/dialect_sqlite3.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 15 additions & 2 deletions vendor/github.com/go-xorm/xorm/engine.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions vendor/github.com/go-xorm/xorm/engine_group.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading