Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set or skip an object property based on a condition #387

Open
majastrz opened this issue Aug 29, 2020 · 87 comments
Open

Set or skip an object property based on a condition #387

majastrz opened this issue Aug 29, 2020 · 87 comments
Assignees
Labels
discussion This is a discussion issue and not a change proposal. enhancement New feature or request revisit

Comments

@majastrz
Copy link
Member

Is your feature request related to a problem? Please describe.
When conditionally enabling ipv6 in a template, there are cases where some properties need to be either set or omitted from a resource declaration. Currently in Bicep, this requires creating two separate object literals and choosing between them via the ternary operator.

Describe the solution you'd like
Can we create a simpler way of doing this that doesn't involve declaring two separate objects? Should we consider introducing a concept of undefined that is separate from null?

@majastrz majastrz added enhancement New feature or request discussion This is a discussion issue and not a change proposal. labels Aug 29, 2020
@ghost ghost added the Needs: Triage 🔍 label Aug 29, 2020
@alex-frankel
Copy link
Collaborator

@marcre - we're pretty sure null with the ternary operator would fulfill this, so wondering if we are missing something.

@marcre
Copy link
Collaborator

marcre commented Sep 4, 2020

Yes and no - depends on how we handle NULL.. Does null mean we omit the value when we send to the RP, or do we send the null on. If we don't send the null, then we don't need undefined. However, with PUTCH RPs like compute you may need to be able to explicitly set something to null to make sure it is cleared if previously set.

@majastrz
Copy link
Member Author

majastrz commented Sep 4, 2020

Agree with that. We would like the distinction between null and undefined/omitted properties not to exist, but the reality disagrees.

@slavizh
Copy link
Contributor

slavizh commented Sep 4, 2020

agree as well. Not all RPs react on null equally.

@alex-frankel alex-frankel added this to the Committed Backlog milestone Sep 24, 2020
@pacodelacruz
Copy link

I have had to rely on the createObject function nested in a compex comparison function inside a copy for an array property and it is very painful. Particularly when error messages from the RPs do not help.

Happy to share a sample of this!

I'd say that having an easy way of defining if you want a property to be omitted, set to null, or set to an object would be very useful.

@khowling
Copy link

+1, this would be super useful in so many use-cases. for example, like in Javascript

param autoscaller bool
param agentCount int = 3

// define the regular node profile
var nodepool1Profile = {
  name: 'nodepool1'
  count: agentCount
  enableAutoScaling: autoscaller
}

// add the 'maxCount' property to agentPoolProfiles  if autoscaller is true
let agentPoolProfiles =  autoscaller? {...nodepool1Profile, maxCount: agentCount+10} : nodepool1Profile 

@yuanyi2121
Copy link

yuanyi2121 commented Mar 3, 2021

+1 really like this to be supported. My scenarios is - I have 3 different set of nsg rules, which contains 90% same rules. I'd like to be able to define all rules in one array and use bool parameters to control which rules to be included when instantiated.
I think if makes more sense than ternary in this case:
myarray: [
if (includeA) {
itemA
}
{
ItemB
}
]

@onionhammer
Copy link

onionhammer commented May 12, 2021

Would love if this existed. I'm trying to define the 'analytical TTL' on a Cosmos container and it only accepts an 'int' but in the .NET SDK it's a nullable int, so 'undefined' would be the only way I could conditionally set an analytical storage ttl.

Or it could be done w/ spread ( #1560 )

Or per the ugly way:

image

@khowling
Copy link

Looks like AppGateway uses a number of resource specific top-level properties (zones, identity etc). I cannot see any way to conditionally include these (for my use-case, the identity object). Looks like you cannot use a variable for the entire resource body resource appgw 'Microsoft....@2020' = varname , so cannot use the union method. Any suggestions?

image

@alex-frankel
Copy link
Collaborator

@khowling does using the ternary operator work?

var shouldUseIdentity = true

resource appGw 'Microsoft.Network/applicationGateways@2021-02-01' = {
  name: 'foo'
  identity: shouldUseIdentity ? {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${userAssignedMI}': {}
    }
  } : null
}

@khowling
Copy link

@alex-frankel Unfortunately not, that produces:

Deployment template parse failed: 'Required property 'type' not found in JSON. Path

if I replace {} with null, I get

Error BCP036: The property "identity" expected a value of type "ManagedServiceIdentity" but the provided value is of type "null | object".

@daniellittledev
Copy link

I ran into this issue trying to create a bicep file that creates an Azure function with a consumption plan in the test environment and a standard plan (for always on) in production.

Cannot update the site 'test-func' because it uses AlwaysOn feature which is not allowed in the target compute mode.

It fails the value for AlwaysOn is null or false, so the property needs to be omitted entirely.

@ehrnst
Copy link
Contributor

ehrnst commented Sep 9, 2021

I encountered this today when trying to conditionally add vnet rules to a storage account depending on a boolean input. ARM will not accept an empty virtualNetworkRules property.

@onionhammer
Copy link

onionhammer commented Sep 9, 2021

Maybe undefined or omit should be a literal value thats assignable to properties.

@reidcurry
Copy link

reidcurry commented Sep 17, 2021

@khowling does using the ternary operator work?

var shouldUseIdentity = true

resource appGw 'Microsoft.Network/applicationGateways@2021-02-01' = {
  name: 'foo'
  identity: shouldUseIdentity ? {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${userAssignedMI}': {}
    }
  } : null
}

Thanks Alex. I'm new to Bicep and your little snip helped me out. Only thing I changed was the null for {}

Here is what I ended up using when one probe used statusCodes, but the second probe did not need "Use probe matching conditions" checked:

probes: [for probe in appGateway.probes : {
      name: probe.name
      properties: {
        protocol: probe.protocol
        path: probe.path
        interval: probe.interval
        timeout: probe.timeout
        unhealthyThreshold: probe.unhealthyThreshold
        pickHostNameFromBackendHttpSettings: probe.pickHostNameFromBackendHttpSettings
        minServers: probe.minServers
        match: probe.shouldShowMatchStatusCodes ? {
          statusCodes: [
            probe.matchStatusCodes
          ]
        } : {}
      }
    }]

@jrobins-tfa
Copy link

Another use case where this is causing trouble: I'm making a template for an App Gateway. In the backendHttpSettingsCollection, there are two attributes, hostname and pickHostNameFromBackendAddress. If pickHostNameFromBackendAddress is true, you don't need hostname, otherwise you do. I tried to do this conditionally based on my parameters:

backendHttpSettingsCollection: [for backendHttpSetting in backendHttpSettingsCollection : {
   name: backendHttpSetting.name
   properties: {
     port: (contains(backendHttpSetting.properties, 'port')) ? backendHttpSetting.properties.port : null
     protocol: (contains(backendHttpSetting.properties, 'protocol')) ? backendHttpSetting.properties.protocol : null
     hostName: (contains(backendHttpSetting.properties, 'hostname')) ? backendHttpSetting.properties.hostname : null
     pickHostNameFromBackendAddress: (contains(backendHttpSetting.properties, 'pickHostNameFromBackendAddress')) ? backendHttpSetting.properties.pickHostNameFromBackendAddress : false

The problem is that even having "hostname: null" in your deployment causes an error if pickHostNameFromBackendAddress is true:

ApplicationGatewayBackendHttpSettingsHostNameFieldConflict - HostName field may only be specified if pickHostNameFromBackendAddress is set to false in context '/subscriptions/......'

I have two backendHttpSettingsCollections, one with pickHostNameFromBackendAddress false, the other with it true. So I need to be able to conditionally include or exclude the hostname attribute, not just conditionally assign a value or a null to it.

@onionhammer
Copy link

We need an 'undefined' value that can be put in.. it's familiar to people who know any JavaScript

@erlandsen-tech
Copy link

What we do now as a workaround (awaiting a proper alternative) is using modules and conditional deployment. We have several VM images that needs to deploy, but only some of the have plans. We now need one module for VmWithPlan, and one VmWithoutPlan. It works, but It means we need to repeat our code in two files which we don't want. Please keep me posted regarding a fix.

@AlexanderSehr
Copy link

We have a similar issue. We're trying to structure our templates with dedicated modules for child-resources. However, the user does not need to provide all values in the template's parameter file.
The problem is that the child-resource template (e.g. container for storage account) comes with its own set of parameters, default values & validation - and not all parameters are mandatory.
The problem is, the invoking template (e.g. storage account) will need an option to pass all parameters to this child-template the user specified, but not more. However, there is no way of achieving this and e.g. hand over null so that the child-template would fall back tot he default:

For example:
image
will result into: Error: The value of deployment parameter 'cors' is null. Please specify the value or use the parameter reference

The only workaround here is to not pass null but instead duplicate the default value we have already specified in the child-template parameter:
image

With some sort of undefined type value we could solve this issue and have the child-template just fall back to the default.

@shenglol
Copy link
Contributor

I'd like to point out that skip property would also be useful wen conditionally creating element in an array.
The propName[condition]: propValue syntax would not cover array elements. the special keyword undefined would, but perhaps we can adjust the syntax a bit.

@mimq If there is a requirement to omit array items, I think we should not rely on a special syntax to achieve that. Using null and a filter function might be the way to go.

@miqm
Copy link
Collaborator

miqm commented Nov 18, 2022

I'd like to point out that skip property would also be useful wen conditionally creating element in an array.
The propName[condition]: propValue syntax would not cover array elements. the special keyword undefined would, but perhaps we can adjust the syntax a bit.

@mimq If there is a requirement to omit array items, I think we should not rely on a special syntax to achieve that. Using null and a filter function might be the way to go.

This is the same problem as with properties - why we need to omit or introudce undefined if we can just have null? :) Perhaps there might be cases when we want to have null in array but conditionally put. In fact, the if would be just a nicer way of writing union

@jeskew
Copy link
Contributor

jeskew commented Nov 18, 2022

Lambdas open up a lot of options that weren't available when this issue was opened. Conditional properties on objects can also be handled with reduce (or with filter and toObject once #8982 lands).

I do wonder though if lambdas are any more approachable for a non-developer audience than undefined would be, especially if (as @miqm points out) a property or array element with an explicit value of null is semantically differentiated from an omitted property or array element, and the user has to select a different sentinel value to assign and then filter out.

@miqm
Copy link
Collaborator

miqm commented Nov 19, 2022

I think we should also consider case with upcoming strong typing for parameters. When we develop modules we expect
params to have some object type. However there are lots of situations that some parameters of passed object we'd like to have optional. But when declaring object that will be passed we'd need to explicitly specify that particular parameter is null which will lead to quite noisy parameters. In C# if we have a class we can define default value for a property. Would it be possible in bicep as well? It'd be handy that's for sure.

Other question with conditional properties would be whether we show them in object completions or not. And if we do (I think we should) we should make user to handle the case when property is not set due to the condition.

@D-Bissell
Copy link

I know Ansible/Jinja have the omit filter to ignore properties. Something like that perhaps?

  • name: touch files with an optional mode
    file:
    dest: "{{ item.path }}"
    state: touch
    mode: "{{ item.mode | default(omit) }}"
    loop:
    • path: /tmp/foo
    • path: /tmp/bar
    • path: /tmp/baz
      mode: "0444"

That is, if 'mode' is set, then use it, but otherwise the property is ignored as though it were not being defined.

@Candelit
Copy link

Maybe this is just obvious for everyone in this thread, but for those who don't get that (Like me) and have a problem to solve, I can report that passing json('null') works to omit a value for a property.
In my case to an existing VirtualNetwork with a loop to add the subnets, now I have to add a AzureFirewall subnet.
Azure Firewall requires a subnet named AzureFirewallSubnet and it cannot have an NSG defined. If you try, it fails the entire deployment.
This line saved me after hours and hours of testing and research.

networkSecurityGroup: subnet.properties.subnetName == 'AzureFirewallSubnet' ? json('null') : {id: resourceId('Microsoft.Network/networkSecurityGroups', 'nsg-vn-iac-${envclass}-${subnet.properties.subnetName}')}

In this case, null, '', {} in direct form or via a variable/parameter did not work. But json('null') did.

Hope this helps anyone

@jikuja
Copy link

jikuja commented Dec 30, 2022

Maybe this is just obvious for everyone in this thread, but for those who don't get that (Like me) and have a problem to solve, I can report that passing json('null') works to omit a value for a property.

It does not work with all resource types.

@slavizh
Copy link
Contributor

slavizh commented Jun 15, 2023

Maybe this is just obvious for everyone in this thread, but for those who don't get that (Like me) and have a problem to solve, I can report that passing json('null') works to omit a value for a property.

It does not work with all resource types.

Yep, agree with @jikuja . You are stating a single case but putting null on property acts different not only on per RP bases but sometimes on per property base for the same resource.

@leelax22
Copy link

rules: [for i in range(0, length(appObject.ruleArray)): {
          name: appObject.ruleArray[i].name
          description: appObject.ruleArray[i].description
          ruleType: 'ApplicationRule'
          destinationAddresses: contains(appObject.ruleArray[i], 'destinationAddresses') ? [appObject.ruleArray[i].destinationAddresses] : 
          fqdnTags: contains(appObject.ruleArray[i], 'fqdnTags') ? [appObject.ruleArray[i].fqdnTags] : json('null')
          httpHeadersToInsert: contains(appObject.ruleArray[i], 'httpHeadersToInsert') ? [appObject.ruleArray[i].httpHeadersToInsert] : json('null')
          protocols: contains(appObject.ruleArray[i], 'protocols') ? [appObject.ruleArray[i].protocols] : json('null')
          sourceAddresses: contains(appObject.ruleArray[i], 'sourceAddresses') ? [appObject.ruleArray[i].sourceAddresses] : json('null')
          sourceIpGroups: contains(appObject.ruleArray[i], 'sourceIpGroups') ? [appObject.ruleArray[i].sourceIpGroups] : json('null')
          targetFqdns: contains(appObject.ruleArray[i], 'targetFqdns') ? [appObject.ruleArray[i].targetFqdns] : json('null')
          targetUrls: contains(appObject.ruleArray[i], 'targetUrls') ? [appObject.ruleArray[i].targetUrls] : json('null')
          //terminateTLS: contains(appObject.ruleArray[i], 'terminateTLS') ? [appObject.ruleArray[i].terminateTLS]
          webCategories: contains(appObject.ruleArray[i], 'fqdnTags') ? [appObject.ruleArray[i].fqdnTags] : json('null')
        }]

I'm having a similar problem too. I am trying to enter the rules of the firewall, but there are cases where there are items that do not need to be entered among each parameter for each rule. It seems that there is no error when the template is distributed without even entering the key value, but if null or [''] values are entered, an error is output because there is no value for the key value. I wish there was a workaround.

@milamber9
Copy link

I would love this feature to exist but fwiw I used the following workaround in case it helps someone.

Context: Conditional site config for function apps.

Added an extra param to my module:

@description('Optional. Add additional site config properties to function app.')
param addSiteConfig object = {}

Then added the following to my resource call:

...
siteConfig: union(addSiteConfig, {
      appSettings: ...
})

@onionhammer
Copy link

This does break intellisense on the properties, but yes this is definitely a thing

@peter-de-wit
Copy link

peter-de-wit commented Jan 5, 2024

Another use case for this. I like re-deployable templates for our customers. It is now not possible to redeploy templates that have VMs and customdata property on initial deployment due the fact that you cannot pass null to the customdata when initial deployment == false.

example:

// VM deployment bicep script
// Other params are removed for simplicity
param isInitialDeployment bool = true 

// Create vm 
resource vm {
  properties: {
    osProfile: {
    computerName: 'myVM'
    adminUsername: 'install'
    adminPassword: adminPassword
    // When re-deploying vm, you get 'Changing property customdata is not allowed'. exception due the fact that null is not the same as do not set.
    customData: isInitialDeployment ? base64(customData) : null // this should be fixable by providing undefined / other syntax
  }
}

@spoelstraethan
Copy link

Another use case for this. I like re-deployable templates for our customers. It is now not possible to redepeloy templates that have VMs and customdata property on initial deployment due the fact that you cannot pass null to the customdata when initial deployment == false.

example:

// VM deployment bicep script
// Other params are removed for simplicity
param isInitialDeployment bool = true 

// Create vm 
resource vm {
  properties: {
    osProfile: {
    computerName: 'myVM'
    adminUsername: 'install'
    adminPassword: adminPassword
    // When re-deploying vm, you get 'Changing property customdata is not allowed'. exception due the fact that null is not the same as do not set.
    customData: isInitialDeployment ? base64(customData) : null // this should be fixable by providing undefined / other syntax
  }
}

The workaround we ended up using for this is setting custom data to an include for cloud init for Linux in a remote file, either on a web server or blob storage. The custom data in the template won't ever change while the web server can present specific data based on properties of the accessing client or pull down a script to determine what to do based on other properties of the client like registering it with a configuration management tool like Ansible.

If you need to have a no-op just make the default contents of the script exit early.

For windows you just use an additional run command or startup script to parse and execute the remote web file, and this is one of the places where AWS has done it better than Azure because the user data flow is the same for Windows and Linux.

@onionhammer
Copy link

So many hacky workarounds for this.. undefined would make things so much cleaner in a ton of cases.

@carlosharrycrf
Copy link

Another example for this is with Microsoft.DBforPostgreSQL/serverGroupsv2. Setting administratorLoginPassword to null throws and error when deploying.

@onionhammer
Copy link

onionhammer commented Mar 7, 2024

Yet another great usecase for this:

image

enablePurgeProtection cannot be set to false, or it will error ( " The property "enablePurgeProtection" cannot be set to false. Enabling the purge protection for a vault is an irreversible action." "), but it can be omitted.

Additionally, you cannot for expressions inside of a union, so in my scenario I just cant conditionally set this.

@brumbrum05
Copy link

yes would like to have undifined for virtualNetworkSubnetId in Web/sites too, because null does not seem to work.

@xInfinitYz
Copy link

This should have been resolved long time ago

@lansalot
Copy link

Would be handy indeed - in my case, I have to pass an empty array in my parameter object. I tried using empty() but unless I specify that [] for delegations in my first parameter, it fails as the property is missing. Template:

    subnets: [for subnet in subnets: {
      name: subnet.name
      properties:{
        addressPrefix: subnet.addressPrefix
        delegations: empty(subnet.delegations) ? [] : subnet.delegations
        routeTable: empty(subnet.delegations) ? {
          id: routeTableID
        } : null
        networkSecurityGroup: {
          id: nsg.id
        }
      }
    }]

parameters file:

    "vNetSubnets": {
      "value": [
        {
          "name": "pe-vnet",
          "addressPrefix": "10.200.0.0/28",
          "delegations" : []   <<< here
        },
        {
          "name": "powerplatformaccess-vnet",
          "addressPrefix": "10.200.0.16/28",
          "delegations": [
            {
              "name": "Microsoft.PowerPlatform/vnetaccesslinks",
              "type": "Microsoft.Network/virtualNetworks/subnets/delegations",
              "properties": {
                "serviceName": "Microsoft.PowerPlatform/vnetaccesslinks"
              }
            }
          ]
        }
      ]
    },

@jrobins-tfa
Copy link

Would be handy indeed - in my case, I have to pass an empty array in my parameter object. I tried using empty() but unless I specify that [] for delegations in my first parameter, it fails as the property is missing. Template:

    subnets: [for subnet in subnets: {
      name: subnet.name
      properties:{
        addressPrefix: subnet.addressPrefix
        delegations: empty(subnet.delegations) ? [] : subnet.delegations
        routeTable: empty(subnet.delegations) ? {
          id: routeTableID
        } : null
        networkSecurityGroup: {
          id: nsg.id
        }
      }
    }]

parameters file:

    "vNetSubnets": {
      "value": [
        {
          "name": "pe-vnet",
          "addressPrefix": "10.200.0.0/28",
          "delegations" : []   <<< here
        },
        {
          "name": "powerplatformaccess-vnet",
          "addressPrefix": "10.200.0.16/28",
          "delegations": [
            {
              "name": "Microsoft.PowerPlatform/vnetaccesslinks",
              "type": "Microsoft.Network/virtualNetworks/subnets/delegations",
              "properties": {
                "serviceName": "Microsoft.PowerPlatform/vnetaccesslinks"
              }
            }
          ]
        }
      ]
    },

@lansalot - There are two better options for you in this particular scenario. The older approach would be to use contains() instead of empty():

delegations: !contains(subnet, 'delegations') ? [] : subnet.delegations
But with the addition of safe dereferences to the language (which I guess isn't so new any more), you can use that to get your value or null if it isn't there.

delegations: subnet.?delegations
This solves your particular use case, but there are still plenty of others on this thread that don't have a good solution. I hope they implement this feature some time soon!

@lansalot
Copy link

Oh fantastic, didn't know about subnet.?delegations, that's a sweet option :)

anthony-c-martin added a commit that referenced this issue Apr 23, 2024
… on lambdas (#13658)

Adds the spread operator `...` as well as various new functions +
indexes on lambdas:
1. Spread operator - usage is as follows:
    * In an object:
        ```bicep
        var objA = { bar: 'bar' }
var objB = { foo: 'foo', ...objA } // equivalent to { foo: 'foo', bar:
'bar' }
        ```
    * In an array:
        ```bicep
        var arrA = [ 2, 3 ]
        var arrB = [ 1, ...arrA, 4 ] // equivalent to [ 1, 2, 3, 4 ]
        ```
1. New functions + usage:
    * `objectKeys`: Returns the keys of an object parameter:
        ```bicep
        var example = objectKeys({ a: 1, b: 2 }) // returns [ 'a', 'b' ]
        ```
* `mapValues`: Create an object from an input object, using a custom
lambda to map values:
        ```bicep
var example = mapValues({ foo: 'foo' }, val => toUpper(val)) // returns
{ foo: 'FOO' }
        ```
* `groupBy`: Create an object with array values from an array, using a
grouping condition:
        ```bicep
var example = groupBy(['foo', 'bar', 'baz'], x => substring(x, 0, 1)) //
returns { f: [ 'foo' ], b: [ 'bar', 'baz' ]
        ```
* `shallowMerge`: Perform a shallow merge of input object parameters:
        ```bicep
var example = shallowMerge([{ foo: 'foo' }, { bar: 'bar' }]) // returns
{ foo: 'foo', bar: 'bar' }
        ```
1. Optional indices on lambdas + usage:
    * `map`:
        ```bicep
var example = map(['a', 'b'], (x, i) => { index: i, val: x }) // returns
[ { index: 0, val: 'a' }, { index: 1 val: 'b' } ]
        ```
    * `reduce`:
        ```bicep
var example = reduce([ 2, 3, 7 ], (cur, next, i) => (i % 2 == 0) ? cur +
next : cur) // returns 9
        ```
    * `filter`:
        ```bicep
var example = filter([ 'foo', 'bar', 'baz' ], (val, i) => i < 2 &&
substring(val, 0, 1) == 'b') // returns [ 'bar' ]
        ```
Closes #13560
Closes #9244
Closes #1560
Addresses some of the issues described under the following: #2082,
#1853, #387

###### Microsoft Reviewers: [Open in
CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/Azure/bicep/pull/13658)
@slavizh
Copy link
Contributor

slavizh commented May 3, 2024

@anthony-c-martin as mentioned during the call it might be good to explore that you have the undefined syntax available in Bicep only and when compiled to ARM spread syntax to be used for it. I did not log the other feedback around intellisense when you use spread as I saw that you already logged that. I would definitely use spread syntax instead of union especially if intelisense is available. If undefined is also available will just make that case easier to use and a little bit more readable the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion This is a discussion issue and not a change proposal. enhancement New feature or request revisit
Projects
None yet
Development

No branches or pull requests