Is there a way to limit or slice a given string to a length in compositions

[ashishvaishno]

Hello,

I am accepting a service name from my developer in the claim. This string is used to create the role. There are use cases where the string is longer than 64 char (string limit of a role name). I have an opa policy to invalidate such claims.

But if its possible to trim the string to 64 characters, I can use that to create the role.

Regards
Ashish

[jaylevin]

@ashishvaishno

One option would be to use the substr function from the Sprig library with the go-templating function:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1beta1
    kind: XR
  mode: Pipeline
  pipeline:
    - step: render-resources
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            {{ $roleTrimmed := substr 0 63 .observed.composite.resource.spec.roleName }}

            apiVersion: iam.aws.upbound.io/v1beta1
            kind: Role
            metadata:
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: role
            spec:
              forProvider:
                roleName: {{ $roleTrimmed | toYaml }}
    - step: automatically-detect-ready-composed-resources
      functionRef:
        name: function-auto-ready

[ashishvaishno]

@jaylevin let me try that :) So composite function pipeline's works in conjunction with existing resources described in the composition?

[ashishvaishno]

Seems it doesn't 😢

When in mode: Pipeline the resources and patchSets fields of a Composition's spec are ignored. Put otherwise, a Composition must specify P&T-style resources, or a pipeline. It cannot specify both. The new mode field must be optional. The default value is mode: Resources, which honors (only) the P&T resources as do GA Compositions today.

[bobh66]

You can run the crossplane beta convert pipeline-composition tool on the existing P&T composition to turn it into a Composition Function-based P&T composition, which puts the P&T logic into the pipeline for you and then you could add the go-templating step to the pipeline if you need it.

[ashishvaishno]

@bobh66 @jaylevin My use case was creating s3 with pod identity. I wanted to use the composite function for the role resource and keep everything else as is. Based on the documentation we cannot have mode as both Pipeline and Resources (unless i missed something). It can only be one. I wanted to adjust my existing composition to accomodate the below composite function.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: test.xs3withpodidentity.crossplane.xyz.com
  labels:
    environment: test
spec:
  compositeTypeRef:
    apiVersion: crossplane.xyz.com/v1alpha1
    kind: XS3WithPodIdentity
  mode: Pipeline
  pipeline:
    - step: render-resources
  - functionRef:
      name: go-templating
    input:
      apiVersion: gotemplating.fn.crossplane.io/v1beta1
      inline:
        template: |
          {{ $roleTrimmed := substr 0 63 .observed.composite.resource.spec.resourceConfig.serviceAccountName }}
          {{ $roleTrimmed = cat "k8s-dev-" $roleTrimmed }}
          {{ $roleTrimmed = nospace $roleTrimmed }}
          {{ $serviceAccountName := .observed.composite.resource.spec.resourceConfig.serviceAccountName }}
          {{ $bucketName := .observed.composite.resource.spec.resourceConfig.bucketName }}
          apiVersion: iam.aws.upbound.io/v1beta1
          kind: Role
          metadata:
            annotations:
              gotemplating.fn.crossplane.io/composition-resource-name: role
              crossplane.io/external-name: {{ $roleTrimmed | toYaml }}
          spec:
            providerConfigRef:
              name: aws-provider-config-dev
            deletionPolicy: Delete
            forProvider:
              tags:
                serviceAccountName: {{ $serviceAccountName }}
                environment: "dev"
              assumeRolePolicy: |
                {
                  "Version": "2012-10-17",
                  "Statement": [
                    {
                      "Effect": "Allow",
                      "Principal": {
                        "Service": "pods.eks.amazonaws.com"
                      },
                      "Action": [
                        "sts:TagSession",
                        "sts:AssumeRole"
                      ]
                    }
                  ]
                }
      kind: GoTemplate
      source: Inline

So I guess i will have to migrate the below composition to make use of composition function. My existing composition (without composition function) is below

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: dev.xs3withpodidentity.crossplane.xyz.com
  labels:
    environment: dev
spec:
  compositeTypeRef:
    apiVersion: crossplane.xyz.com/v1alpha1
    kind: XS3WithPodIdentity
  patchSets:
    - name: deletion-policy-field
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.deletionPolicy
          toFieldPath: spec.deletionPolicy
    - name: common-fields
      patches:
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
            strategy: string
            string:
              fmt: "aws-provider-config-%s"
          toFieldPath: spec.providerConfigRef.name
    - name: tags
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
          toFieldPath: spec.forProvider.tags.Environment
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceAccountName
          toFieldPath: spec.forProvider.tags.serviceAccountName
    - name: region
      patches:          
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
          toFieldPath: spec.forProvider.region
          transforms:
            - type: map
              map:
                dev: eu-central-1
                stage: eu-west-1
                prod: eu-west-1
    - name: s3-extra-tag
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.awsBackup
          toFieldPath: spec.forProvider.tags.AWSBackup
  resources:
    - name: s3-bucket
      base:
        apiVersion: s3.aws.upbound.io/v1beta1
        kind: Bucket
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: deletion-policy-field
        - type: PatchSet
          patchSetName: tags
        - type: PatchSet
          patchSetName: region
        - type: PatchSet
          patchSetName: s3-extra-tag
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketName
          toFieldPath: metadata.annotations[crossplane.io/external-name]
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketName
          toFieldPath: spec.forProvider.tags.Name
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.arn
          toFieldPath: status.bucketArn
    - name: s3-bucketpublicaccessblock
      base:
        apiVersion: s3.aws.upbound.io/v1beta1
        kind: BucketPublicAccessBlock
        spec:
          forProvider:
            blockPublicAcls: true
            blockPublicPolicy: true
            ignorePublicAcls: true
            restrictPublicBuckets: true
            bucketSelector:
              matchControllerRef: true
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: deletion-policy-field
        - type: PatchSet
          patchSetName: region
    - name: s3-bucketserversideencryptionconfiguration
      base:
        apiVersion: s3.aws.upbound.io/v1beta1
        kind: BucketServerSideEncryptionConfiguration
        spec:
          forProvider:
            bucketSelector:
              matchControllerRef: true
            rule:
              - applyServerSideEncryptionByDefault:
                  - sseAlgorithm: AES256
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: deletion-policy-field
        - type: PatchSet
          patchSetName: region
    - name: s3-bucketversioning
      base:
        apiVersion: s3.aws.upbound.io/v1beta1
        kind: BucketVersioning
        spec:
          forProvider:
            bucketSelector:
              matchControllerRef: true
            versioningConfiguration:
              - status: ""
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: deletion-policy-field
        - type: PatchSet
          patchSetName: region
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketVersioningStatus
          toFieldPath: spec.forProvider.versioningConfiguration[0].status
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.versioningConfiguration[0].status
          toFieldPath: status.bucketVersioningStatus
    - name: s3-bucketlifecycleconfiguration
      base:
        apiVersion: s3.aws.upbound.io/v1beta1
        kind: BucketLifecycleConfiguration
        spec:
          forProvider:
            bucketSelector:
              matchControllerRef: true
            rule:
              - id: "Cleanup of Non Current Versions"
                noncurrentVersionExpiration:
                  - noncurrentDays: 
                status: ""
              - id: "Cleanup of Old Objects"
                expiration:
                  - days: 
                status: ""
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: deletion-policy-field
        - type: PatchSet
          patchSetName: region
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketVersioningStatus
          toFieldPath: spec.forProvider.rule[0].status
          transforms:
            - type: map
              map:
                Disabled: Disabled
                Suspended: Enabled
                Enabled: Enabled
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketNonCurrentVersionExpirationDays
          toFieldPath: spec.forProvider.rule[0].noncurrentVersionExpiration[0].noncurrentDays
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketCleanupOldObjectStatus
          toFieldPath: spec.forProvider.rule[1].status
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.bucketCleanupOldObjectExpirationDays
          toFieldPath: spec.forProvider.rule[1].expiration[0].days
        - type: ToCompositeFieldPath
          fromFieldPath: spec.forProvider.rule[0].status
          toFieldPath: status.bucketVersioningStatus
        - type: ToCompositeFieldPath
          fromFieldPath: spec.forProvider.rule[0].noncurrentVersionExpiration[0].noncurrentDays
          toFieldPath: status.bucketNonCurrentVersionExpirationDays
        - type: ToCompositeFieldPath
          fromFieldPath: spec.forProvider.rule[1].status
          toFieldPath: status.bucketCleanupOldObjectStatus
        - type: ToCompositeFieldPath
          fromFieldPath: spec.forProvider.rule[1].expiration[0].days
          toFieldPath: status.bucketCleanupOldObjectExpirationDays
    - name: iamrole
      base:
        apiVersion: iam.aws.upbound.io/v1beta1
        kind: Role
        spec:
          deletionPolicy: Delete
          forProvider:
            assumeRolePolicy: |
              {
                "Version": "2012-10-17",
                "Statement": [
                  {
                    "Effect": "Allow",
                    "Principal": {
                      "Service": "pods.eks.amazonaws.com"
                    },
                    "Action": [
                      "sts:TagSession",
                      "sts:AssumeRole"
                    ]
                  }
                ]
              }
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: tags
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
            - fromFieldPath: spec.resourceConfig.serviceAccountName
            strategy: string
            string:
              fmt: "k8s-%s-%s"
          toFieldPath: metadata.annotations[crossplane.io/external-name]
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.arn
          toFieldPath: status.roleArn
    - name: iam-policy
      base: 
        apiVersion: iam.aws.upbound.io/v1beta1
        kind: Policy
        spec:
          deletionPolicy: Delete
          forProvider:
            description: ""
            policy: ""
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: PatchSet
          patchSetName: tags
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
            - fromFieldPath: spec.resourceConfig.serviceAccountName
            strategy: string
            string:
              fmt: "k8s-%s-%s"
          toFieldPath: metadata.annotations[crossplane.io/external-name]
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.arn
          toFieldPath: status.policyArn
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: spec.resourceConfig.bucketName
            strategy: string
            string:
              fmt: "IAM Policy for bucket %s created by crossplane"
          toFieldPath: spec.forProvider.description
        - type: CombineFromComposite
          toFieldPath: spec.forProvider.policy
          policy:
            fromFieldPath: Required
          combine:
            variables:
            - fromFieldPath: spec.resourceConfig.bucketName
            - fromFieldPath: spec.resourceConfig.serviceAccountName
            - fromFieldPath: spec.resourceConfig.bucketName
            - fromFieldPath: spec.resourceConfig.serviceAccountName
            strategy: string
            string:
              fmt: |
                { 
                  "Version": "2012-10-17",
                  "Statement": [
                    {
                      "Action": "s3:ListBucket",
                      "Effect": "Allow",
                      "Resource": [
                          "arn:aws:s3:::%s"
                      ],
                      "Condition": {
                        "StringLike": {
                          "aws:PrincipalTag/kubernetes-pod-name": "%s*"
                        }
                      }
                    },
                    {
                      "Action": [
                        "s3:ListObjects",
                        "s3:PutObject",
                        "s3:GetObject",
                        "s3:DeleteObject"
                      ],
                      "Effect": "Allow",
                      "Resource": [
                          "arn:aws:s3:::%s/*"
                      ],
                      "Condition": {
                        "StringLike": {
                          "aws:PrincipalTag/kubernetes-pod-name": "%s*"
                        }
                      }
                    }
                  ]
                }
    - name: policy-attachment-1
      base:
        apiVersion: iam.aws.upbound.io/v1beta1
        kind: RolePolicyAttachment
        spec:
          deletionPolicy: Delete
          forProvider:
            policyArnSelector:
              matchControllerRef: true
            roleSelector:
              matchControllerRef: true
      patches:
        - type: PatchSet
          patchSetName: common-fields
    - name: pod-identity-association
      base:
        apiVersion: eks.aws.upbound.io/v1beta1
        kind: PodIdentityAssociation
        spec:
          deletionPolicy: Delete
          forProvider:
            clusterName: "go-main-dev"
            region: "eu-central-1"
            namespace: ""
            serviceAccount: ""
            roleArnSelector:
              matchControllerRef: true
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceNamespace
          toFieldPath: spec.forProvider.namespace
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceAccountName
          toFieldPath: spec.forProvider.serviceAccount
    - name: service-account
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        spec:
          deletionPolicy: Delete
          forProvider:
            manifest:
              apiVersion: v1
              kind: ServiceAccount
              metadata:
                name: ""
                namespace: ""
              imagePullSecrets:
                - name: registrypullsecret
      patches:
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
            strategy: string
            string:
              fmt: "kubernetes-provider-config-%s"
          toFieldPath: spec.providerConfigRef.name
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceNamespace
          toFieldPath: spec.forProvider.manifest.metadata.namespace
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceAccountName
          toFieldPath: spec.forProvider.manifest.metadata.name
    - name: remote-configmap
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        spec:
          deletionPolicy: Delete
          forProvider:
            manifest:
              apiVersion: v1
              kind: ConfigMap
              metadata:
                name: ""
                namespace: ""
              data:
                roleArn: ""
                policyArn: ""
                bucketArn: ""
                serviceAccountName: ""
                region: ""
                bucketVersioningStatus: ""
      patches:
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
            strategy: string
            string:
              fmt: "kubernetes-provider-config-%s"
          toFieldPath: spec.providerConfigRef.name
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceNamespace
          toFieldPath: spec.forProvider.manifest.metadata.namespace
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceAccountName
          toFieldPath: spec.forProvider.manifest.metadata.name
        - type: FromCompositeFieldPath
          fromFieldPath: status.roleArn
          toFieldPath: spec.forProvider.manifest.data.roleArn
        - type: FromCompositeFieldPath
          fromFieldPath: status.policyArn
          toFieldPath: spec.forProvider.manifest.data.policyArn
        - type: FromCompositeFieldPath
          fromFieldPath: status.bucketArn
          toFieldPath: spec.forProvider.manifest.data.bucketArn
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.serviceAccountName
          toFieldPath: spec.forProvider.manifest.data.serviceAccountName
        - type: FromCompositeFieldPath
          fromFieldPath: status.bucketVersioningStatus
          toFieldPath: spec.forProvider.manifest.data.bucketVersioningStatus
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
          toFieldPath: spec.forProvider.manifest.data.region
          transforms:
            - type: map
              map:
                dev: eu-central-1
                stage: eu-west-1
                prod: eu-west-1

[bobh66]

Based on the documentation we cannot have mode as both Pipeline and Resources (unless i missed something). It can only be one. I wanted to adjust my existing composition to accomodate the below composite function.

As described above: #5492 (comment) convert your existing composition to function patch&transform and then add your go-templating function to the pipeline

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[bobh66]

I have not tested it but I wonder if the regex transform would work - something like:

patches:
  - type: FromCompositeFieldPath
    fromFieldPath: spec.resourceConfig.serviceAccountName
    toFieldPath: metadata.annotations["crossplane.io/external-name"]
    transforms:
      - type: string
        string:
          type: Regexp
          regexp:
            match: '.{63}'
            group: 1

I think that says to match the first 63 characters of the serviceAccountName field, but again I have not tested it.