* Disable sidecar for activator and autoscaler

* Fix tests

* updated to kfp 1.0.0-rc.3

* Adapt to kfp 1.0rc3 refactoring

* update test snapshots

* fix profile controller iam binding

* rename

adding expected test changes

…authorization (#1342)

* Add argo to stacks/generic

* Pull pipelines manifest from upstream

* Updated kfp

* Minio v3 manifests

* Rename minio configmap

* Add generic minio install

* Generate new test data

* Mysql kustomize v3 manifest - generic install

* Add mysql gcp pd install

* Generate test data

* Pipelines kustomize v3 manifests

* Add kfp ui virtual service

* Add metadata deployment to stacks/generic

* Use common cluster domain

* Deploy metadata writer

* Add kfp cache server

* Update test data

* Enable KFP multi user mode without istio security

* Fix persistence agent watch namespace

* Fix namespace env for some deployments

* Fix cluster roles and bindings

* fix rename

* Fix pipelines ui role

* Updated kfp to rc2

* simplify pipeline v3 manifest using updated kfp rc2 manifest

* Fix pipeline-install-config

* remove redundant configmap

* update tests

* updated to kfp 1.0.0-rc.3

* Adapt to kfp 1.0rc3 refactoring

* update test snapshots

* fix pull kfp script to detect empty dir

* fix example ref

* update snapshot

* fix gcp pd manifest

* Update stacks ref

* revert alice example to gcp stack

* update snapshot

* fix profile controller iam binding

* Update kfp profile controller can be configured to different images and
istio sidecar

* add missing viewer controller cluster roles

* Use python3 for sync.py

* Revert gcp stack back to use non multi user kfp

* revert unintended changes

* revert upstream changes

…n on IBM Cloud (#1336)

* kfctl_ibm_dex support

* add dex config

* fix

* update default config

* update dex-config.yaml

* add support for kustomize fns

* update docker image repo

* update oidc-authservice

* set default userid-header

* update tests

* Stable is still on 1.14 which is getting to be quite old.
  Tekton for example requires 1.15 at least.

Related to: GoogleCloudPlatform/kubeflow-distribution#67

…1368)

* Add argo to stacks/generic

* Pull pipelines manifest from upstream

* Updated kfp

* Minio v3 manifests

* Rename minio configmap

* Add generic minio install

* Generate new test data

* Mysql kustomize v3 manifest - generic install

* Add mysql gcp pd install

* Generate test data

* Pipelines kustomize v3 manifests

* Add kfp ui virtual service

* Add metadata deployment to stacks/generic

* Use common cluster domain

* Deploy metadata writer

* Add kfp cache server

* Update test data

* Enable KFP multi user mode without istio security

* Fix persistence agent watch namespace

* Fix namespace env for some deployments

* Fix cluster roles and bindings

* fix rename

* Fix pipelines ui role

* Updated kfp to rc2

* simplify pipeline v3 manifest using updated kfp rc2 manifest

* Fix pipeline-install-config

* remove redundant configmap

* update tests

* updated to kfp 1.0.0-rc.3

* Adapt to kfp 1.0rc3 refactoring

* update test snapshots

* fix pull kfp script to detect empty dir

* fix example ref

* update snapshot

* fix gcp pd manifest

* Update stacks ref

* revert alice example to gcp stack

* update snapshot

* fix profile controller iam binding

* Update kfp profile controller can be configured to different images and
istio sidecar

* add missing viewer controller cluster roles

* Use python3 for sync.py

* Revert gcp stack back to use non multi user kfp

* revert unintended changes

* revert upstream changes

* Secure kfp multi user mode with istio authorization

* patch minio to disable istio sidecar injection

* fix cache server istio authz

* enable istio sidecar for profiles deploy

* enable istio sidecar for centraldashboard

* Do not protect profile controller with istio

* Allow admission webhook traffic to cache-server

* revert gcp stack back to pipeline generic

* Reuse minio generic install as base for gcp-pd and ibm

* update snapshot

…rom the secret (#1372)

* refactor: pipelines profile controller should get minio access keys from the secret

* do not print secrets in log

* refactor: pipelines profile controller should get minio access keys from the secret

* do not print secrets in log

* use kfp multi user mode for gcp stacks

* update snapshot

* Related to: GoogleCloudPlatform/kubeflow-distribution#74

Fix common labels for knative and kfserving.

  * There were typos and they were including labels that by convention
    we currently don't want in common labels.

…a3 (#1345)

* image gcr.io/kubeflow-images-public/notebook-controller:vmaster-g101c77a3
* Image built from kubeflow/kubeflow@101c77a3

* Allow namespace install and update to 1.2.1

* remove duplicate vars

* update for 1.2.1 and automation

* update version

* Update tests and change validate to allows CRDs with status

* Add common labels to kfp components

* Add KFP application

* update snapshot

* Use json format for json patch, because yaml will look like a resource and fail tests

* Remove part of label

* update snapshots

* Fix profile controller deployment version

* update snapshot

* Fix userid-header for gcp

* update snapshot

* Fix b64encode exception

* update snapshot

* update snapshot

* related to GoogleCloudPlatform/kubeflow-distribution#74

* knative needs to be installed in its own namespace so we don't want
  to make it part of the kubeflow namespace kustomization package

* We will have to install it separately in the blueprint.

…#1387)

* Per GoogleCloudPlatform/kubeflow-distribution#89 we need to get rid of the legacy
  partial setters and move to using a KptFile and substitutions.

* In preparation for that we want to check in a set of test data
  that is the result of running our kpt cfg set with a given set of
  values

* This test data will be used to verify that the refactoring to use
  a KptFile doesn't change the output.

* After adding the KptFile we can simply regenerate the testdata
  and then look at the diff to ensure there are no unexpected changes.

* feat: update KFP to 1.0.0

* update test snapshot

* pull new 1.0.0

* fix and update test snapshot

* Migrate AWS manifest to v3 pattern

* Clean up tests files

* Add istio namespace to istio ingress

* Update KFP pipeline test case for aws stack

* Convert v1 to v2 setters & substituions in gcp

* The latest version of kpt started choking on gcp/v2 because we were
  still using the old style setters and substitutions.

* This PR creates a kptfile to use the new setter and substitutions.

* hack/create_kptfile.py contains a script to generate lot of the setters
  and substitutions.

* kf-vm-sa.yaml shouldn't specify the namespace; this will get set in an overlay

* Move workload identity bindings for kf-admin KSA from kubeflow/instance in gcp blueprints repo into this repository.

related to gcp-blueprints#89

* Fix image mirror substitution.

* Create a KptFile for stacks.

* Add conversion for stacks.

* Add KptFile for stacks/gcp

* add knative eventing

* add knative eventing tests

…1391)

…1406)

* image gcr.io/kubeflow-images-public/centraldashboard:vmaster-gd601b2d0
* Image built from kubeflow/kubeflow@d601b2d0

* image gcr.io/kubeflow-images-public/kfam:vmaster-g9f3bfd00
* Image built from kubeflow/kubeflow@9f3bfd00

* Changes needed for V3 change on AWS

* Adopt AWS ECR image for jupyter-web-app components and generate all tests against manifest V3 changes

* Adopt Cognito change against V3

* Adopt change for multi-user KFP

* Moving central dashboard links to configmap

* Updating tests for centraldashboard configmap

* chore: upgrade istio to 1.4.10-asm.15

* update snapshot

* Revert "update snapshot"

This reverts commit 9523ad4.

)

…RM. (#1432)

* Fix management blueprint kptfile and stop using namespace mode for CNRM.

* The management blueprint should have its own KptFile
  * Prior to this PR there was only a KptFile at gcp/
  * This doesn't work because for the management cluster we
    only pull the package gcp/v2/management

* Related to GoogleCloudPlatform/kubeflow-distribution#102
* Related to GoogleCloudPlatform/kubeflow-distribution#93

* For CNRM Switch to workload identity and stop using namespace mode for CNRM; GoogleCloudPlatform/kubeflow-distribution#13

  * Using namespace mode is just extra complexity because we have to install
    a separate copy of the CNRM controller for every project.
    * The only reason to do really do that is if you want to use different
      GCP service accounts to manage different projects. Typically that's
      not what we do.
    * With workload identity we have 1 namespace per project but they
      all use the same GCP SA so the GCP sa can just be authorized to
      access multiple projects or a folder as needed.

* Update the resources to the v1beta1 spec for use with AnthosCLI

  * It looks like anthoscli requires a NodePool resource
  * With the v1beta1 specs we need to add the annotation gke.cluster.io = "bootstrap:https://" so that anthoscli is able to probably group the resources.

* Move cnrm-install iam and services into kustomize packages
  * This way we can hydrate them like we do other manifests

* Fix the setters and substitutions for CNRM to make them unique per name
  * This way we could potentially have multiple management clusters per project
    which if nothing else will be useful for testing.

* Add workload identity pool to the management cluster.

* Management nodepool should set workloadMetadataConfig so that we run the workload identity servers.

* Fix.

Related to: GoogleCloudPlatform/kubeflow-distribution#102

* add multi-user kfdef configurations

* replace env with envs

* update multi-user

* update with namespaces

* different  ClusterRbacConfig for dex and non-dex

* add v1.1.0 ibm kfdef

* update generated tests

Need `userid-header` param to use proper header when handling
the API calls. Default header value: `x-goog-authenticated-user-email`
would be used if this param doesn't present. It may only work on GCP.
Therefore, we do need this param to be properly passed to kfam.

* Add a kustomize function to remove namespace

* This is needed to produce YAMLs that are compatible with ACM

related to

* GoogleCloudPlatform/kubeflow-distribution#27 kustomize function to remove namespace
* GoogleCloudPlatform/kubeflow-distribution#4 instructions for ACM

* Fix spec.

* Add the serviemanagement API because this is what cloudendpoints uses.

* cert-manager namespace needs label control-plane

* cert-manager namespace needs the control-plane label to prevent
  application of kfserving webhooks.

related to: #1450

* Fix error in YAML

#1455)

* Related to #1169 we should be using the gateway
  in the istio-system namespace and not the gateway in the kubeflow
  namespace

* kubeflow/gcl-blueprints#22 With ACM we want to use
  istio-system/ingressgateway and stop using the kubeflow-gateway.

* Add Jeffwan to OWNERS files

* Address code review feedbacks

#1274)

* image gcr.io/kubeflow-images-public/profile-controller:vmaster-ga49f658f
* Image built from kubeflow/kubeflow@a49f658f

* Migrate Istio and Dex to V3

* Roll back AWS change

…1475)

* image gcr.io/kubeflow-images-public/jupyter-web-app:vmaster-g845af298
* Image built from kubeflow/kubeflow@845af298

…98 (#1474)

* image gcr.io/kubeflow-images-public/notebook-controller:vmaster-g845af298
* Image built from kubeflow/kubeflow@845af298

* Remove tensorboard manifest

Please check #415 (comment) for more details.

* Remove Tensorboard references in kfdef and stacks

* fix: let KFP profile controller use KFP version from upstream

* update snapshots

* disable suffix hash to workaround a problem

* update snapshots

* stop using commonprefix to let configmapref work

* update snapshots

* v1.1 istio dex components for kubernetes installation

* Adds kfdef and creds correction

* Test cases generated

* v1.1.0 kfdef added

* v1.1 manifests for vanilla k8s

* adds k8s_istio v3 kfdef

* add tests

…nd. (#1513)

* Ref https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-xlb#disabling_http force all traffic from client to ingress to be https.

* Fix GoogleCloudPlatform/kubeflow-distribution#116

* Explicitly set the default backend for the GCP ingress.

  * https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#default_backend

  * This prevents a separate forwarding rule from being created which is
    confusing.

* Add missing files for k8s_istio.v1.1.0 KFDef

* Also removed tensorboard from the KFDef

* Generate tests

…e notebook-controller crash (#1510)

* added regex to container cpu and memory requests and limits

* added newly generated tests

* added tests generated with kustomize 3.2.1

* Add Myself into AWS OWNERS

* Update OWNERS

* ASM Mesh id should be proj-${PROJECTNUBMER}

* Fix GoogleCloudPlatform/kubeflow-distribution#117
* The pattern for the mesh id needs to be proj-${PROJECTNUMBER}
* Dedube the mesh substitutions; there were two different substitutions "mesh-id" and "asm-mesh-id" for the mesh id.

* Address comments.

* Remove isSet.

* Clear isSet.

…tion used in KfDef (#1582)

* add IBM stack components

* add IBM stack components

* revert seldon test expected data

* switch to base_v3 for istio-stack

Related to GoogleCloudPlatform/kubeflow-distribution#143

Co-authored-by: Jeremy Lewi <[email protected]>

…1592)

* image gcr.io/kubeflow-images-public/pytorch-operator:vmaster-g518f9c76
* Image built from kubeflow/kubeflow@518f9c76

…1585)

* image gcr.io/kubeflow-images-public/xgboost-operator:vmaster-g56c2c075
* Image built from kubeflow/kubeflow@56c2c075

* image gcr.io/kubeflow-images-public/tf_operator:vmaster-gda226016
* Image built from kubeflow/kubeflow@da226016

…l bug (#1604)

* feat(kfp): update kfp upstream manifests to 1.0.4

* update snapshots

* Change Katib manifests for v1beta1 version

* Change images to kubeflowkatib

* Change image tag to v1beta1-e294a90

…d0 (#1570)

* image gcr.io/kubeflow-images-public/notebook-controller:vmaster-g6eb007d0
* Image built from kubeflow/kubeflow@6eb007d0

* Add E2E Test on Kubeflow Shared Test-infra

* Use different branches

Add Arrikto folk to the OWNERS file, as part of forming the new WG
Manifests, in order to carry out the plan specified in the charter.

Signed-off-by: Yannis Zarkadas <[email protected]>

* Update kfserving manifests for v0.4.0

This follows the pattern used in pipelines/upstream where
'upstream' contains the installation manifests from the
source component repo. A script is provided to sync the upstream
directory with the source repo.

Patches are provided to get kfserving working in the kubeflow namespace.

* Generate  tests

* Update to  use  0.4.1

* Add updated 0.4.1 yaml

* Fix one remaining use in fluentd

* Fix #538

* Seldon 1.4.0 Upgrade

* update for 1.4.0

* remove status resource from CRD

* Support AppID

* Add service account for oidc-authservice

* Make it work with existing oidc-authservice

* New application oidc-authservice-for-appid

* Address code review comments

* Use secret to configure authservice

* Update doc for authservice configuration

* Add missing command to get namespace created

* Improves doc

* Multi-tenant Kubeflow by using IBM Cloud AppID

* Remap components to application for IBM stack

Remove reference to components
Regenerate tests

Move global variable to individual application

* rename app to kubeflow-apps for v3

* Adding owners files

* adding required openshift scc

* Adding istio

* Adding istio

* Fixing kustomize

* adding cert-manager

* Adding Argo

* Adding cert manager

* Adding jupyter web app

* adding profiles and nb controller

* Adding components

* Adding Seldon

* Adding kfdef for openshift

* Fixing the fsgrp issue with nb controller

* Adding a patch for jnakfour in metadatadb

* moving openshift-scc to top

* Adding namespace in profiles

* Disabling tls in destination rule for pipeline ui

* Adding route

* Moving profiles and pipeline to top kustomize

* Adding new line

* Adding custom profile image to disable istio injection

* Updating profile image and istio scc

* Switching pipeline to generic and pulling out to application

Co-authored-by: Juana Nakfour <nakfour>