Feature: Interfaces

[WhitWaldo]

Is your feature request related to a problem? Please describe.
Large deployments can span sets of resources deployed to all number of different scopes. Given a tenant, a management group and a subscription, I may deploy resources at a global scope (those intended to be created only once) and deploy others on a regional basis (either one per region or sets of common individual regional deployments).

As the deployment is intended to span all the resources at a given scope, it introduces an enormous pile of parameters to keep track of, leading to a "root" file that dishes out a subset of these parameters to child modules that deploy the actual and dependent resources. There's an excellent example of this in @brwilkinson 's AzureDeploymentFramework repository, where you have a directory containing all the resources for any scope of deployment. The author's stated intent is that one can deploy everything as a greenfield deployment or incrementally push to only a single "spoke" in a given region.

Parameters are then supplied on a scoping basis (not only ARM scopes, but also including values like tenant name and deployment ID (keeping track of how many deployments exist in a region) and passed into the pipeline at that root, where they trickle down to each resource based on the configuration.

Unfortunately, in an effort to create a simplified version of this for my own purposes, I've found that this approach leaves room for some complexity - all the types and names of values are hidden away in these scoped parameter files and Bicep simply trusts that it'll find them in the blobbed 'object' types specified in the markup.

In an effort to ensure that each resource can be independently deployed, it becomes necessary in this example to pass whole untyped objects down from the root to the children in an identical fashion so that sub-properties can be accessed and consumed as necessary.

The shapes of modules are fairly clearly described. You have a number of input parameters which are generally clearly typed and that informs the developer about how to treat it. If I indicate that I have a string param, I can prescribe it allowed values and treat it as an enum in the rest of the module:

@allowed([
  'apple'
  'banana'
  'carrot'
])
param myOptions string 

This helps me as the developer to understand what values it may or may not contain as I build out the module. Similarly, I can indicate a numerical input and put constraints on the range or a boolean and describes its default value. These all help the developer.

However, if I simply pass an untyped object or an untyped array, I can infer that there are subproperties of the former and that there are zero or more elements of the latter, but that's as far as I can see into the shape of the argument. Today, Bicep apparently takes a standoff-ish approach to handling these object properties as it just assumes they're accurate and doesn't share any feedback in VS Code when specifying them, because it has no source of truth from which to discern what the properties should or shouldn't be.

Describe the solution you'd like
I would like to propose the notion of an interface as used in TypeScript (for whatever reason, TypeScript's actual docs have deprecated their own interface page) to describe the shape of parameter objects strictly at design-time and without any impact on the build output.

Put simply, in any given module, I would propose a new language construct, the interface to be defined and provide a parameter-like approach for defining (exclusively for Bicep) the shapes of objects and the objects passed in arrays into other modules. For example, I'm picturing something like this:

//Module A
interface globalType = {
  prefix: 'az'
  environment: 'g'
  deploymentId: 2
}

Today, as long as the definition stays inside of a single module, I could just define this as a variable object and Bicep can infer both the names and types of the object for same-module definitions. But as soon as I pass this into another module, that's entirely lost because I can only pass it in as another untyped object parameter. I'd rather be able to do something like:

//Module B
param globalOptions globalType
param listOfOptions array<globalType>

Today, at best I can do:

//Module B
param globalOptions object
param listOfOptions array

And I lose all the clarity of what exactly is in globalOptions or listOfOptions, trusting both that I passed the appropriate object into whatever is calling this module and that the properties I'm calling on it are actually defined somewhere (and not just a runtime error waiting to happen).

To repeat, like TypeScript, this should not at all change what's actually produced by the compiler. At build time, Bicep would still just treat these parameters in Module B as object and an array as it does today, but it would allow the developer some peace of mind to catch typos, utilize Intellisense to know the properties of objects without leaving sets of other windows open and enable optional benefits of type safety without any impact on the dynamic nature of Bicep for anyone that wants to opt out.

Thanks for taking the time to read this proposal.

[brwilkinson]

There are currently some investigations for how to achieve some of these asks.

Also the ability to define a custom object/types has been part of the same conversations.

I will add links here as I find them for reference.

#4158

[alex-frankel]

@WhitWaldo - can you confirm that #4158 satisfies the requirements you've outlined here? If anything is missing, I'd recommend adding it there. Going to close this in the meantime.

[WhitWaldo]

I'd suggest that #4158 is similar in that we're both looking for strongly typed objects, but I'd say that my approach is an extension of his last proposal (Type inference via target-typing). Namely, their proposal looks like it's generally intended to contain properties common to a single given resource. I'm ideally looking for something more broadly applicable - namely, the ability to say, pass in an object describing a subnet and NSGs, but have the downstream usage of that object be aware of the typed properties of that object and constraint to the expected types accordingly.

The example used in #4158 instead looks as though the type would be inferred by the usage of the type, but I'd instead assert that in my approach you'd know the type of each property and the shape of it when the parameter itself was defined (much like variables).

[alex-frankel]

pass in an object describing a subnet and NSGs, but have the downstream usage of that object be aware of the typed properties of that object and constraint to the expected types accordingly.

We've had a few discussions since #4158 was created, but I think this is more or less the goal. Enable stricter types for objects that the language service is aware of and can validate. cc @rynowak

[WhitWaldo]

I guess I can't re-open this myself, but I wanted to follow-up with a recent real-world example to go with this that's more relevant to my proposal here than to #4158 @alex-frankel @brwilkinson

I'm working on building out the infrastructure to deploy Azure Service Fabric applications via ARM and am starting with a generic module for the deployment.

Skipping over most of the project structure as it's just not relevant here, there are two kinds of services one can deploy to an SF cluster: stateless and stateful services. In ARM, they are both represented by a single type "Microsoft.ServiceFabric/clusters/applications/services" but each uses quite different properties of that type.

Ideally, with the proposal I outlined above, I could indicate something like (a number of different ideas illustrated in the first one):

// This one could be illustrated using #4158 since it's contained as a separate type altogether in the ARM markup
var serviceLoadMetrics interface = {
  defaultLoad: int = 10 //Default value, absence of it indicates it's a required value not unlike params in Bicep files, ideally infer type from it too without me explicitly indicating as much as in primaryDefaultLoad below
  name: string
  primaryDefaultLoad = 10
  secondaryDefaultLoad: int? //I don't specify a default value, but the ? indicates it's optional nonetheless
  weight: ['High', 'Low', 'Medium', 'Zero'] //Similar to using @allowed, but inline in the interface
}

// And from here on, we're in a world that I think #4158 doesn't suffice for
var serviceFabricService interface = {
  name: string
  partitionScheme: partitionScheme //Into which I can pass a namedPartitionScheme, a singletonPartitionScheme or a uniformIntRangePartitionScheme
}

var sfStatelessService interface as serviceFabricService = {
  instanceCount: int
  serviceLoadMetrics: serviceLoadMetrics[] //Indicates an array of the above-defined type
}

var partitionScheme interface = {
}

//Demonstrate interface inheritance
var namedPartitionScheme interface as partitionScheme = {
  names: string[]
}

var singletonPartitionScheme interface as partitionScheme = {
}

var uniformIntRangePartitionScheme interface as partitionScheme = {
  count: int
  highKey: int
  lowKey: int
}

var sfStatefulService interface = {
  targetReplicaSetSize = 3
  minReplicaSetSize: int = 2 //Again, ideally just infer the type if I don't specify it
  replicaRestartWaitDuration: string = '00:01:00.0'
  quorumLossWaitDuration: '00:02:00.0'
  hasPersistedState: bool = false
}

Now, in a separate file, I need only do something like the following:

var serviceA './my-modules/ServiceFabricApplicationDeployment.bicep': sfStatefulService = { //Like a resource type, this illustrates where to find the interface
  name: 'ServiceA'
  //minReplicaSetSize: Don't need to specify as it already has a default value of 2
  replicaRestartWaitDuration: '00:05:00.0' //But I can specify it if I want to override the default
  partitionScheme: { //Automatically type matches to a uniformIntRangePartitionScheme even without explicitly indicating it
    count: 5
    lowKey: 0
    highKey: 100
  }
}

module SFApp './my-modules/ServiceFabricApplicationDeployment.bicep'  = {
  services: [serviceA] //Configured to accept serviceFabricService[], meaning either a stateful or stateless service
  //Other properties
}

And then per the markup above, I get type-checking on any objects I pass around and know that even if I have several dozen files that all point into this module, I can be certain that they contain the necessarily typed properties to properly populate the parameters necessary.

Additional thoughts - for your benefit, it might be worth swapping var with interface when defining the interfaces themselves since these aren't actually variables that would be populated and exposed in the ARM template itself - thinking in a similar vein as TypeScript, these exist purely at development time and do not otherwise leak into the deployed files.

[WhitWaldo]

Wanted to jump in with another example relevant to today's task. I'm working on simplifying my various modules so I can both publish them for public consumption and simplify everything.

As is not uncommon when working with Azure resources, child resources tend to require more than one value to be set and they each require these values be set alongside each other. Bicep has neither an optional nor required keyword today, rather allowing all parameters to be set by a caller and allowing default values to be set in the module. This is generally fine, but doesn't assist module development for more complex scenarios like this.

Let's say I've got an Event Hub resource and I would like to optionally allow a caller to add some number of authorization rules to. Each of these authorization rules needs to specify a name and an array of "rights", spanning 1 to 3 of the following values: 'Send', 'Manage' and/or 'Listen'. I can attach these in a module like the following:

@description('The name of the authorization rule')
param Name string

@minLength(1)
@maxLength(3)
@allowed([
  'Send'
  'Manage'
  'Listen'
])
param Rights array

@description('The name of the Event Hub resource')
param EventHubName string

@description('The name of the Key Vault to save the ')
param KeyVaultName string = ''

resource EventHub 'Microsoft.EventHub/namespaces/eventhubs@2022-01-01-preview' existing = {
  name: EventHubName
}

resource AuthorizationRule 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2022-01-01-preview' = {
  name: Name
  parent: EventHub
  properties: {
    rights: Rights
  }
}

As it's not intended that the authorization-rule module be accessed externally, I need to make these name and rights properties visible on the event-hub.bicep entry module then, but this is where I run into an issue without something like my interfaces concept.

I could do something like the following:

@param EventHubName string
@param AuthorizationRuleNames array = []
@param AuthorizationRuleRights array = []

//...

module EventHubAuthorizationRule './authorization-rule.bicep' = [for (authorizationRuleName, i) in AuthorizationRuleNames: {
  name: 'eh-${EventHubName}-AuthRule-${authorizationRuleName}'
  params: {
    Name: authorizationRuleName
    Rights: AuthorizationRuleRights[i]
  }
}]

But the problem with this is that as the module author, I'm limited only in describing in the description metadata that authorization rule rights really need to be an array of arrays and I lose out on being able to put the validation constraints on what the passed values actually are. Further, I have no way of dictating that the length of AuthorizationRuleRights should be the same length as AuthorizationRuleNames.

So I could just make it a single array and expect that the author provides me with an appropriately formatted JSON object, but I lose all validation capability again and will only know whether it works or not at runtime (where I can count on the error message being less than stellar per #4848).

@param EventHubName string
@param AuthorizationRules array

Going back to my proposal here, this could be quite easily remedied with some sort of strongly typed interface that I can decorate with the existing validation options:

interface authorizationRule = {
  name: string
  @minlength(1)
  @minlength(3)
  @allowed([
    'Listen'
    'Send'
    'Manage'
  ])
  rights: array
}

Now in my module, I can easily indicate that I'll accept zero or more of this interface:

@param EventHubName string
@param AuthorizationRules authorizationRule[] = [] //Indicates that members of the array should be typed with the interface - alternatively specified as array<authorizationRule>

//...

module EventHubAuthorizationRule './authorization-rule.bicep' = [for (authorizationRules, i) in AuthorizationRules: {
  name: 'eh-${EventHubName}-AuthRule-${authorizationRule.name}'
  params: {
    Name: authorizationRule.name
    Rights: AuthorizationRule.rights
  }
}]

And when compiled, the interface drops away. Like that of a TypeScript interface, it provides only for the shape of the expected values and doesn't actually have any sort of runtime presence.

This would easily allow me, as the module developer, to clearly articulate the shape of complex objects without reliance on the consumer reading and understanding verbose in-lined documentation, trial and error through runtime failures or mismatching of sets of arrays that I'm enumerating through by index. Rather, I can define the shape of the expected arguments between modules (ideally both in via parameters and out via outputs) and make that available at development time via Intellisense, at build-time via the compiler and leave the runtime entirely oblivious to it all.

[alex-frankel]

cc @jeskew as FYI

I'm not reopening only because I don't want multiple issues speaking to the same problem, not because this issue is not valuable. We will take a look at these examples and try to factor that into our design (which there will be ample opportunity for feedback on).

We are taking a close look at how to define custom types and perform operations on those types as you'd expect in other languages. You should be able to use custom types in other type definitions, and we should be able to validate that deeply, etc.

[WhitWaldo]

@alex-frankel It's fine not re-opening it, so long as it's visible as a possible solution to the problem. Encountered this again today in the example above so it's just top of mind for me. Reviewing the feedback in #4158 , I still find that proposal far too strictly bound to specific resources and not nearly flexible enough for larger, more modular deployments that span dozens of potentially inter-configurable resources (e.g. optionally create 1 or more authorization rules in an event hub, but optionally save each of their connection strings to a specified key vault).

The above is doable with today's typing, but only in a way that's not easily transferrable outside of a single developer's domain knowledge about the larger Bicep deployment philosophy, much less a public repository of such modules.

Thanks for the continued consideration!