Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config diff #4619

Open
wants to merge 43 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
1fc038b
story(configDiffView) : open api spec
adi6859 Feb 1, 2024
8eda368
story(configDiffView) : open api spec updated
adi6859 Feb 1, 2024
2d00c6a
story(configDiffView) : open api spec updated for error state
adi6859 Feb 1, 2024
00d1e6c
story(configDiffView) : WIP
adi6859 Feb 1, 2024
4e3582b
story(configDiffView) : WIP "some code changed"
adi6859 Feb 1, 2024
8c9a4cc
story(configDiffView) : support for names added
adi6859 Feb 2, 2024
55a2c0c
story(configDiffView) : iota removed
adi6859 Feb 2, 2024
bcdfa26
story(configDiffView) : pg no rows handled
adi6859 Feb 2, 2024
197db9b
story(configDiffView) : spelling check
adi6859 Feb 2, 2024
7469461
story(configDiffView) : code review comment resolved
adi6859 Feb 5, 2024
c7a145a
story(configDiffView) : env id added
adi6859 Feb 5, 2024
4eaa138
story(configDiffView) : intersection added
adi6859 Feb 5, 2024
256e505
story(configDiffView) : comments removed
adi6859 Feb 5, 2024
47f798f
story(configDiffView) : code review comment resolved
adi6859 Feb 6, 2024
de50f02
story(configDiffView) : comment removed
adi6859 Feb 6, 2024
f2947e3
story(configDiffView) : CMCSNames DTO moved
adi6859 Feb 6, 2024
a3b3271
story(configDiffView) : main merge
adi6859 Feb 7, 2024
44862e9
story(configDiffView) : null case handled
adi6859 Feb 7, 2024
7733ea0
story(configDiffView) : logger added
adi6859 Feb 7, 2024
06e39fc
story(configDiffView) : main merge
adi6859 Feb 8, 2024
e4e39c6
story(configDiffView) : code refactored
adi6859 Feb 8, 2024
dce967a
story(configDiffView) : code refactored v2
adi6859 Feb 8, 2024
4174161
story(configDiffView) : spec updated
adi6859 Feb 8, 2024
186efa7
story(configDiffView) : main merge
adi6859 Feb 13, 2024
736047a
story(configDiffView) : code refactored
adi6859 Feb 14, 2024
16d07d3
story(configDiffView) : config names
adi6859 Feb 14, 2024
3a55c5b
Merge branch 'main' into config-diff
prakash100198 Feb 19, 2024
a4e9c4f
Merge branch 'main' into config-diff
prakash100198 Feb 20, 2024
706afc6
merged with main
vikramdevtron Jun 25, 2024
014e59d
Merge branch 'main' into config-diff
prakash100198 Jul 3, 2024
6196f31
main sync
prakash100198 Jul 3, 2024
e076783
overridden and global flag introduced in config diff autocomplete api
prakash100198 Jul 3, 2024
a1eba0f
Merge branch 'main' into config-diff
prakash100198 Jul 9, 2024
ffa3ba1
ent sync
prakash100198 Jul 9, 2024
4749c45
Merge branch 'main' into config-diff
prakash100198 Jul 16, 2024
0e319d2
Merge branch 'main' into config-diff
prakash100198 Jul 17, 2024
2c5c7c6
using a single key instead of global and overridden key in config/aut…
prakash100198 Jul 17, 2024
8696b08
ConfigState made string instead of int
prakash100198 Jul 17, 2024
a11549f
Merge branch 'main' into config-diff
prakash100198 Jul 23, 2024
f9d9c65
not sending inheriting in case base config
prakash100198 Jul 23, 2024
359022a
Merge branch 'main' into config-diff
prakash100198 Jul 30, 2024
5628d6c
code review comment incorporation
prakash100198 Jul 30, 2024
41784ca
Merge branch 'main' into config-diff
prakash100198 Jul 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
story(configDiffView) : WIP
  • Loading branch information
adi6859 committed Feb 1, 2024
commit 00d1e6cc36050271f2ddea65e830074d42439c84
8 changes: 8 additions & 0 deletions Wire.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ import (
"github.com/devtron-labs/devtron/pkg/chart"
chartRepoRepository "github.com/devtron-labs/devtron/pkg/chartRepo/repository"
"github.com/devtron-labs/devtron/pkg/commonService"
"github.com/devtron-labs/devtron/pkg/config"
delete2 "github.com/devtron-labs/devtron/pkg/delete"
"github.com/devtron-labs/devtron/pkg/deploymentGroup"
"github.com/devtron-labs/devtron/pkg/devtronResource"
Expand Down Expand Up @@ -671,6 +672,13 @@ func InitializeApp() (*App, error) {
scopedVariable.NewScopedVariableRestHandlerImpl,
wire.Bind(new(scopedVariable.ScopedVariableRestHandler), new(*scopedVariable.ScopedVariableRestHandlerImpl)),

router.NewDeploymentConfigurationRouter,
wire.Bind(new(router.DeploymentConfigurationRouter), new(*router.DeploymentConfigurationRouterImpl)),
restHandler.NewDeploymentConfigurationRestHandlerImpl,
wire.Bind(new(restHandler.DeploymentConfigurationRestHandler), new(*restHandler.DeploymentConfigurationRestHandlerImpl)),
config.NewDeploymentConfigurationServiceImpl,
wire.Bind(new(config.DeploymentConfigurationService), new(*config.DeploymentConfigurationServiceImpl)),

util.NewGitCliUtil,

router.NewTelemetryRouterImpl,
Expand Down
82 changes: 82 additions & 0 deletions api/restHandler/DeploymentConfigurationRestHandler.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package restHandler

import (
"fmt"
"github.com/devtron-labs/devtron/api/restHandler/common"
"github.com/devtron-labs/devtron/pkg/auth/authorisation/casbin"
"github.com/devtron-labs/devtron/pkg/auth/user"
"github.com/devtron-labs/devtron/pkg/config"
"github.com/devtron-labs/devtron/util/rbac"
"github.com/gorilla/mux"
"go.uber.org/zap"
"gopkg.in/go-playground/validator.v9"
"net/http"
"strconv"
)

type DeploymentConfigurationRestHandler interface {
ConfigAutoComplete(w http.ResponseWriter, r *http.Request)
}
type DeploymentConfigurationRestHandlerImpl struct {
logger *zap.SugaredLogger
userAuthService user.UserService
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
validator *validator.Validate
enforcerUtil rbac.EnforcerUtil
enforcer casbin.Enforcer
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
deploymentConfigurationService config.DeploymentConfigurationService
}

func NewDeploymentConfigurationRestHandlerImpl(logger *zap.SugaredLogger,
userAuthService user.UserService,
enforcerUtil rbac.EnforcerUtil,
enforcer casbin.Enforcer,
deploymentConfigurationService config.DeploymentConfigurationService,
) *DeploymentConfigurationRestHandlerImpl {
return &DeploymentConfigurationRestHandlerImpl{
logger: logger,
userAuthService: userAuthService,
enforcerUtil: enforcerUtil,
enforcer: enforcer,
deploymentConfigurationService: deploymentConfigurationService,
}
}

func (handler *DeploymentConfigurationRestHandlerImpl) ConfigAutoComplete(w http.ResponseWriter, r *http.Request) {
userId, err := handler.userAuthService.GetLoggedInUser(r)
if userId == 0 || err != nil {
common.WriteJsonResp(w, err, "Unauthorized User", http.StatusUnauthorized)
return
}
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
vars := mux.Vars(r)
//todo aditya get from query params
appId, err := strconv.Atoi(vars["appId"])
if err != nil {
handler.logger.Errorw("request err, CSEnvironmentFetch", "err", err, "appId", appId)
common.WriteJsonResp(w, err, nil, http.StatusBadRequest)
return
}
envId, err := strconv.Atoi(vars["envId"])
if err != nil {
handler.logger.Errorw("bad request", "err", err)
common.WriteJsonResp(w, err, nil, http.StatusBadRequest)
return
}

//RBAC START
token := r.Header.Get("token")
object := handler.enforcerUtil.GetAppRBACNameByAppId(appId)
ok := handler.enforcerUtil.CheckAppRbacForAppOrJob(token, object, casbin.ActionGet)
if !ok {
common.WriteJsonResp(w, fmt.Errorf("unauthorized user"), nil, http.StatusForbidden)
return
}
//RBAC END

res, err := handler.deploymentConfigurationService.ConfigAutoComplete(appId, envId)
if err != nil {
handler.logger.Errorw("service err, CSEnvironmentFetch", "err", err, "appId", appId)
common.WriteJsonResp(w, err, nil, http.StatusInternalServerError)
return
}
common.WriteJsonResp(w, err, res, http.StatusOK)
}
26 changes: 26 additions & 0 deletions api/router/DeploymentConfigRouter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package router

import (
"github.com/devtron-labs/devtron/api/restHandler"
"github.com/gorilla/mux"
)

type DeploymentConfigurationRouter interface {
initDeploymentConfigurationRouter(configRouter *mux.Router)
}

type DeploymentConfigurationRouterImpl struct {
deploymentGroupRestHandler restHandler.DeploymentConfigurationRestHandler
}

func NewDeploymentConfigurationRouter(deploymentGroupRestHandler restHandler.DeploymentConfigurationRestHandler) *DeploymentConfigurationRouterImpl {
router := &DeploymentConfigurationRouterImpl{
deploymentGroupRestHandler: deploymentGroupRestHandler,
}
return router
}

func (router DeploymentConfigurationRouterImpl) initDeploymentConfigurationRouter(configRouter *mux.Router) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subrouter for orchestrator/config, should add deployment in route. also service does not aligns with the name and route, have to discuss this

configRouter.Path("/autocomplete").
HandlerFunc(router.deploymentGroupRestHandler.ConfigAutoComplete).Methods("POST")
}
11 changes: 8 additions & 3 deletions api/router/router.go
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ type MuxRouter struct {
rbacRoleRouter user.RbacRoleRouter
scopedVariableRouter ScopedVariableRouter
ciTriggerCron cron.CiTriggerCron
deploymentConfigurationRouter DeploymentConfigurationRouter
}

func NewMuxRouter(logger *zap.SugaredLogger, HelmRouter PipelineTriggerRouter, PipelineConfigRouter PipelineConfigRouter,
Expand Down Expand Up @@ -154,7 +155,9 @@ func NewMuxRouter(logger *zap.SugaredLogger, HelmRouter PipelineTriggerRouter, P
rbacRoleRouter user.RbacRoleRouter,
scopedVariableRouter ScopedVariableRouter,
ciTriggerCron cron.CiTriggerCron,
proxyRouter proxy.ProxyRouter) *MuxRouter {
proxyRouter proxy.ProxyRouter,
deploymentConfigurationRouter DeploymentConfigurationRouter,
) *MuxRouter {
r := &MuxRouter{
Router: mux.NewRouter(),
HelmRouter: HelmRouter,
Expand Down Expand Up @@ -224,6 +227,7 @@ func NewMuxRouter(logger *zap.SugaredLogger, HelmRouter PipelineTriggerRouter, P
rbacRoleRouter: rbacRoleRouter,
scopedVariableRouter: scopedVariableRouter,
ciTriggerCron: ciTriggerCron,
deploymentConfigurationRouter: deploymentConfigurationRouter,
}
return r
}
Expand Down Expand Up @@ -309,8 +313,9 @@ func (r MuxRouter) Init() {
chartRefRouter := r.Router.PathPrefix("/orchestrator/chartref").Subrouter()
r.ChartRefRouter.initChartRefRouter(chartRefRouter)

configMapRouter := r.Router.PathPrefix("/orchestrator/config").Subrouter()
r.ConfigMapRouter.initConfigMapRouter(configMapRouter)
configRouter := r.Router.PathPrefix("/orchestrator/config/").Subrouter()
r.ConfigMapRouter.initConfigMapRouter(configRouter)
r.deploymentConfigurationRouter.initDeploymentConfigurationRouter(configRouter)

appStoreRouter := r.Router.PathPrefix("/orchestrator/app-store").Subrouter()
r.AppStoreRouter.Init(appStoreRouter)
Expand Down
25 changes: 25 additions & 0 deletions internal/sql/repository/chartConfig/ConfigMapRepository.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ type ConfigMapRepository interface {
GetByAppIdAppLevel(appId int) (*ConfigMapAppModel, error)
GetByAppIdAndEnvIdEnvLevel(appId int, envId int) (*ConfigMapEnvModel, error)
GetEnvLevelByAppId(appId int) ([]*ConfigMapEnvModel, error)
GetConfigNamesAppLevel(appId int) ([]CMCSNames, error)
GetConfigNamesEnvLevel(appId int, envId int) ([]CMCSNames, error)
}

type ConfigMapRepositoryImpl struct {
Expand All @@ -58,6 +60,29 @@ type ConfigMapAppModel struct {
SecretData string `sql:"secret_data"`
sql.AuditLog
}
type CMCSNames struct {
CMName string `json:"cm_name"`
CSName string `json:"cs_name"`
}

func (impl ConfigMapRepositoryImpl) GetConfigNamesAppLevel(appId int) ([]CMCSNames, error) {
var cMCSNames []CMCSNames
query := "SELECT json_array_elements(config_map_data::json->'maps')->>'name' AS cm_name, " +
"json_array_elements(secret_data::json->'secrets')->>'name' AS cs_name " +
"FROM config_map_app_level WHERE app_id = ?;"
adi6859 marked this conversation as resolved.
Show resolved Hide resolved

_, err := impl.dbConnection.Query(cMCSNames, query, appId)
return cMCSNames, err
}
func (impl ConfigMapRepositoryImpl) GetConfigNamesEnvLevel(appId int, envId int) ([]CMCSNames, error) {
var cMCSNames []CMCSNames
query := "SELECT json_array_elements(config_map_data::json->'maps')->>'name' AS cm_name, " +
"json_array_elements(secret_data::json->'secrets')->>'name' AS cs_name " +
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
"FROM config_map_app_level WHERE app_id = ? AND env_id=?;"

_, err := impl.dbConnection.Query(cMCSNames, query, appId, envId)
return cMCSNames, err
}

func (impl ConfigMapRepositoryImpl) CreateAppLevel(model *ConfigMapAppModel) (*ConfigMapAppModel, error) {
err := impl.dbConnection.Insert(model)
Expand Down
46 changes: 46 additions & 0 deletions pkg/config/DeploymentConfigurationService.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package config

import (
"github.com/devtron-labs/devtron/internal/sql/repository/chartConfig"
"go.uber.org/zap"
)

type DeploymentConfigurationService interface {
ConfigAutoComplete(appId int, envId int) (*ConfigDataResponse, error)
}

type DeploymentConfigurationServiceImpl struct {
logger *zap.SugaredLogger
configMapRepository chartConfig.ConfigMapRepository
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
}

func NewDeploymentConfigurationServiceImpl(logger *zap.SugaredLogger, configMapRepository chartConfig.ConfigMapRepository) (*DeploymentConfigurationServiceImpl, error) {
deploymentConfigurationService := &DeploymentConfigurationServiceImpl{
logger: logger,
configMapRepository: configMapRepository,
}

return deploymentConfigurationService, nil
}

func (impl DeploymentConfigurationServiceImpl) ConfigAutoComplete(appId int, envId int) (*ConfigDataResponse, error) {
var configDataResponse ConfigDataResponse
var cMCSNamesEnvLevel []chartConfig.CMCSNames
cMCSNamesAppLevel, err := impl.configMapRepository.GetConfigNamesAppLevel(appId)
if err != nil {
return nil, err
}
if envId > 0 {
cMCSNamesEnvLevel, err = impl.configMapRepository.GetConfigNamesEnvLevel(appId, envId)
}
for i, name := range cMCSNamesAppLevel {
configDataResponse.ResourceConfig[i].Name = name.CMName
configDataResponse.ResourceConfig[i].Type = CM
}
for i, name := range cMCSNamesEnvLevel {
configDataResponse.ResourceConfig[i].Name = name.CMName
configDataResponse.ResourceConfig[i].Type = CM
}

return nil, nil
}
24 changes: 24 additions & 0 deletions pkg/config/bean.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package config

type ConfigState int

const (
PublishedConfigState ConfigState = iota + 2
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
)

type ResourceType string

const (
CM ResourceType = "ConfigMap"
CS ResourceType = "Secret"
DeploymentTemplate ResourceType = "Deployment Template"
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
)

type ConfigDefinition struct {
adi6859 marked this conversation as resolved.
Show resolved Hide resolved
Name string `json:"name"`
ConfigState ConfigState `json:"draftState"`
Type ResourceType `json:"type"`
}
type ConfigDataResponse struct {
ResourceConfig []ConfigDefinition `json:"resourceConfig"`
}
13 changes: 10 additions & 3 deletions wire_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.