-
Notifications
You must be signed in to change notification settings - Fork 730
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: versioning of types for Azure #6641
Comments
Options for import syntaxWe want feedback on the best syntax for imports and versions. The solution we choose needs to meet the following requirements:
InputsWe want imports to be able to specify all of the following:
As an added detail, sometimes some of this information needs to be design-time constant, meaning the compiler needs to be able to compute the value to provide the right tooling experience. The version of a provider or the OptionsHere's some of the options we talked through while writing this proposal. More suggestions are welcome! Option 1
This proposal uses a string to specify both the provider and version. We'd still be able to provide completion here for the version like we can do with API versions on resources. What I like about this proposal is that it uses existing syntax ( Option 2
This proposal introduces a new contextual keyword What I like about this proposal is that it solves the string/symbol confusion. What I dislike about this proposal is that the ordering of Option 3
This option makes the configuration block mandatory and used to specify the version. What I like about this proposal is that it doesn't introduce any new constructs, and lets the configuration block do the work. We know that we'll need a configuration block for some use cases. If we do this one we might want to do single-line-objects as well to streamline it. Based on that option we also mapped out a few variants: 3A
3B
3C
|
Suggestion: add support for |
Suggestion: while you're adding the |
While I like the idea of removing complexity, I believe that this change can introduce undesired complexities. Undesired complexity 1: Undesired complexity 2: Unsure if making this import optional would make things better or not. |
I like this proposal and the decoupling of Bicep compiler releases from the Azure type updates. My preference is Option 3:
This particular syntax feels the most native to Bicep AND sets up for future extensibility. ConcernHaving said that, while removing the need to include API versions with the declared resource is a benefit, I do agree with the concerns raised by the community members in the April Community Call. Many customers, especially those in regulated industries (Financial Services, Healthcare, etc), will need/want control over the versions of the types that are deployed (especially if the service version is public preview). To do this properly, these customers would author Azure Policy to deny the deployment of unapproved resources or versions of resources. However, this can lead to a poor authoring/deployment experience given the following scenario:
SuggestionEnable defining allowed/disallowed resource types and versions in an external configuration file. This file could be managed by a central team and pushed to Bicep module registry when changes occur. Benefits
|
As far as I understand this will be optional in addition to the current method we use. If that is the case I do not have anything against this being implemented but I think it would be useful only if it works properly and the following things are present:
From proposed syntax option 3 seems the best from all proposed. About the breaking change for targetScope. Would be nice if there are a few versions that allows you to have the old way and the new way at the same time. That way you can migrate over time without having to be stuck at particular version. |
We also need to consider how pulling in types from private registries will work. Having that capability will help with:
|
Some notes from Wednesday's discussion:
import az {
targetScope: 'subscription'
}
|
Adding a comment here to ensure there is some symmetry in style between the following:
|
We are finally closed on the provider version syntax: import 'az@v1'
import 'kubernetes@v2' We also revisited the syntax for setting target scope type, and made some updates since last time (#6641 (comment)). The new syntax looks like this: import 'az@v1' with {
scopeType: 'subscription'
} The main difference is that a Another change is that now // main.bicep
import 'az@v1' with {
scopeType: ['resourceGroup', 'subscription']
}
resource roleAssignment 'Microsoft.Authorization/roleAssignment@2020-01-01' = { ... } We also explored some alternative options for supporting multiple target scopes, but they each have some major drawbacks. For transparency, I would like to share the options: Union importimport 'az/resourceGroup@v1' | 'az/subscription@v1' Import decorator@[sys.]config({
scopeType: ['resourceGroup', 'subscription]
})
import 'az@v1' Exact importimport 'az/resourceGroupOrSubscription@v1' Implicit scope typeimport 'az@v1' // scope types(s) is inferred by checking what resource types are declared in the module. If you find some of the options particularly interesting and wonder why we did not choose it, feel free to comment and I'd be happy to explain the reason in detail 🙂. |
I still would like to see importing version by specific provider. For example I would only want to specify the version for Virtual Machines provider rather for the whole Az. Array of scopes is good. |
One thing I'm not sure about this approach is that API versions are bound to resource types, not Azure resource providers. For example, virtual machine has an API version 2021-01-01, but virtual machine extension may not. |
@shenglol that is true but also az@v1 is not API version like 2021-01-01. So let's say if you define az.compute/virtualMachines@v1, v1 will contain different API versions for each resource under microsoft.compute/virtualMachines. |
Got it, this makes more sense to me now. I thought the proposal was to use an API version as the import version, like import 'az@v1/Microsoft.Compute'
resource vm 'virtualMachines@2021-01-01' = { ... } However, we'll need to understand the implication on Setting import 'az@v1/Microsoft.Compute' with {
scopeType: 'managementGroup'
} Selecting import 'az@v1/Microsoft.Authorization' with {
scopeType: 'subscription'
}
import 'az@v1/Microsoft.Blueprint' with {
scopeType: 'subscription'
} , but having to override the Another thing we might want to think about is the influence on import aliases. For example, if there are multiple namespaces imported in a module, it might not make sense to set aliases for both imports: import 'az@v1/Microsoft.Compute' as az
import 'az@v1/Microsoft.Network' as az // ??? |
Some things I'm pleased with and some concerned with. The @, in general, is the most confusing aspect of Bicep to new developers and abstracting that away would be great. However, as @pacodelacruz and others have pointed out, you've now created an artificial tie of the 100's of APIs to a single bicep AZ version - with no easy way to understand that mapping. I'm going to have to somehow know that And again, for a new developer, this is a nightmare. My main concern behind this proposal is the existence of I can see the desire to have a specific version - But please, please don't make the Import AZ required. This is a monstrous breaking change that I can't even fathom why you think this is a good idea. One of the basic premises from the start of Bicep has been that Bicep deploys Azure resources. ("Bicep is a Domain Specific Language (DSL) for deploying Azure resources declaratively.") Adding other providers for import exposes the need for addressing the There was mention in the September CC that this could be added to a bicepconfig.json to work through this but this then breaks the concept of bicep files/modules being distributable - you now would have to have a X.bicep file and a bicepconfig.json file that makes it work. Code sharing is now effectively broken and you have a huge "Works on my machine" problem. Not to mention now that a module from a repo may distribute a bicepconfig.json file which can have a potential impact to other resources in your deploy. Please keep with the mindset that Bicep deploys Azure FIRST and don't break my code! |
@shenglol yeah my proposal was very fast example that was just to show what I mean rather propose actual syntax. I have suggested granularity per resource provider for versioning since it the versioning was introduced to us but may be somehow got slip upon implementation. I think many have agreed that this is needed. As you show than scope becomes a problem. I do not think that defining scope per resource provider is good approach. May be an option like this would be better:
That way you define scope once. You import az without specific version but you import Authorization RP with specific version. So for every resource that you define for Authorization RP API version will be optional only if you want to change it from the API version defined in v1 and for other RPs you will have to specify version. I am not good of creating syntaxes so the above is just to demonstrate the most flexible way to define things without having to write a bunch of unneeded code. I am better of picking a syntax when multiple such are proposed :) Overall I understand the desired change to have one version for Az, although for me at this point I prefer defining API version per resource. For me there are a few things that this syntax should allow:
|
@ChristopherGLewis The rationale behind the import story is to decouple resource providers and the Bicep compiler to support extensibility. Ideally, users should be able to pull providers and their resource types from a registry, including
This matches our design. The default
In Bicep, there is a built-in configuration file that is always loaded by the Bicep compiler. The file is invisible to users, and all custom Bicep configuration files are essentially overriding values in the built-in configuration file. We are actually thinking about adding |
Re-listening to the community call, I would also like to advocate for a solution where "no import statement" means "automatic import of az", while "any import statement" means "import only what is specified". |
May be this can be solved like this:
This way all existing bicep files will not require change as they already use specific version when they define a resource and for them it will not matter what is the default az version. Of course as you want to make a change on how scope is defined that will be a breaking change for existing templates where the target scope is explicitly defined. Even with this proposal my previous proposal of being able to define version per RP (az.compute for example) and defining multiple scopes in an easy as well stays. |
Hi rynowak, this issue has been marked as stale because it was labeled as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. Thanks for contributing to bicep! 😄 🦾 |
Versioning of types for Azure
This is a propose to revamp the versioning of Azure's types as part of Bicep and the use of API versions in code. Have you thought much about resource types in Bicep? The definitions of Azure's types change over time as:
These type definitions are used to power the compiler's validation and the completion/documentation features of the editor. Right now these type definitions are shipped as as part of the compiler, so you only get updates to Azure's type definitions when the team updates the compiler/language.
As Bicep matures further we're planning to:
Decoupling the versioning of Azure's types from the compiler presents an opportunity to streamline the experience of working with Azure API versions for users. The goal of this proposal is to simplify the authoring and maintenance of Bicep files by reducing the number of places API versions need to be configured, while also introducing the flexibility we need to ship the compiler and extensibility features.
While developing the proposal for how to package and version Azure's types, it occurred to us that most users could rely on the versioning of the types package and reduce the number of places they have to choose a version. Azure API guidelines should provide enough stability that we feel comfortable simplifying the guidance and default behavior, while still supporting explicit API versions for exact control.
This proposal should result in the following simplified guidance for users:
az
types when you author the file. Feel free to update this on your own cadence with confidence that the upgrades are safe.Proposal
As we're adding extensibility, we're thinking about a provider model whether each provider has its own versioning. It's possible to think about Azure's types using this framework as the
az
provider.This proposal changes the
az
provider from automatically-imported to opt-in - and will automatically assign the API versions of Azure resources based on the version of the provider. This should result in animport az ...
statement in most files, but overall will reduce the concept count and number of places that versions appear.This proposal makes the following language changes:
import az ...
declaration will be required to access Azure resource types.import az ...
declaration will be used to configuration the version ofaz
provider.az
provider.import az ...
configuration block will be used to configure the target scope. This replaces the dedicatedtargetScope
statement.Example Before/After
This sample deploys a storage account, server farm, and website. This is taken from the following documentation here. This sample is a resource-group scoped deployment and uses the implicit default of
targetScope = 'resourceGroup'
.Before
Note that version numbers appear as part of every type declaration. The user had to select a version of each resource type to type out this file, whether they have a strong opinion about which version to use or not.
We've been looking for a long time for ways to simplify interactions with API versions when authoring Bicep files. While API versioning is important as a mechanism, making a specific version choice often feels arbitrary.
After
Note that are there is one and only one version number in the file - the version of the
az
provider. The user does not need to choose a separate version of each resource type - they get a recommended version by default.The after version of the sample should result in the same compilation output as the before version - assuming a hypothetical
0.6.1
version of the Bicepaz
provider.Details of the proposal
Import az
The
import 'az@<version>'
declaration becomes required to resolve Azure's resource types. We'll be able to provide a version selection/completion experience in the editor so users don't need to memorize version numbers.Without the
import 'az@<version>'
declaration, Azure types would produce a diagnostic with a compiler fix to add the provider. This is similar to the experience in a language like C# when you type in a reference to a type without its using statement.API Versions
import 'az@<version>'
will include a version as required configuration. The code-fix described above would automatically choose the latest available version.Since the version of the
az
provider is pinned for the file by the import declaration, we are able to simplify the resource declarations. Theaz
provider will designate a stable API version for each resource type. A resource declaration may specify an API version explicitly; otherwise the corresponding stable version will be used. Users are free to specify the version if they want to choose something different - in particular if they want to choose a preview versionHovering over a declaration in VS Code will include the API version in the tooltip. The API versions are still there, you just don't need to spend time on them unless you have a reason.
This change removes a concept from the process of adding a resource to the file. Having already declared the provider version you no longer need to choose an API Version when adding a resource.
Versioning of the Provider
The
az
provider (and all providers) will follow a versioning scheme similar to semantic versioning. Azure's API guidelines prevent teams from making breaking changes to their APIs - even when they introduce new API versions - so major version releases will be rare.Upgrading the version of the
az
provider will upgrade the effective API version of resources in the file - and we do not consider this a breaking-change. A live at HEAD philosophy for versioning makes it easy for users to discover and use new features without having to guess at the right API version. We expect these upgrades to be safe, based on Azure's policies for API governance.Target Scope
The selection of target scope will also be part of the configuration of the
az
provider instead of a language keyword/statement.This simplifies the language by removing a statement. As part of the work to introduce extensibility we're adding the ability for a provider to specify strongly-typed configuration.
targetScope
is better suited for this feature rather than being a feature built into the language.resourceGroup
will still be the default, so most files will not need this configuration.We will make this change at the same time we introduce
import az
to make the migration easier for users.Integration Process
This will be a breaking change to Bicep as we're removing a language keyword and also changing the type resolution rules. If we make this change, will roll it out gradually over several releases and driven by our tools to reduce the cost for users to migrate.
We would first introduce a warning when Azure types are used without the import or when the
targetScope
statement is used. This will come with a code fix to add the import statement using the latest types package and the existing value oftargetScope
. We will ship Bicep in this state for a 1-2 releases and gather feedback on the new design. During this period the old design and new design will exist side-by-side and it is up to users to update at their own pace. Since this is just a warning, users will have the ability to suppress or ignore the diagnostic.Once we're confident that the new design is having the desired impact we will remove support for the old design in a future release. The mandatory change for users is to add
import 'az@<version>'
and replace thetargetScope
statement. It's up to users to choose whether they want to un-pin their resource declarations - both styles will be valid.FAQ
Q: Why not use a configuration file for this? We can support this in the future via a config file if that is desired. It seems more actionable for us to do this first via Bicep because we have the ability to do code-fixes and display warnings easily. We don't want to make a configuration file required to use Bicep so code is the logical starting point.
Q: Why are you using semver for provider versioning? We wanted to avoid using dates for versioning of the provider because they feel arbitrary. Users are more familiar with semver as a set of principles.
Q: Will you require an exact version to be specified or will you allow floating versions? Right now we plan to require explicit versions to be specified. We want to avoid any situation where deployments of Bicep code are non-repeatable - your files and configuration should result in the same behavior 6 months from now. This is similar to why we don't allow the API versions of resources to float today in Bicep code.
It's likely that if we introduced floating versions we'd also need to introduce a lock file - similar to NPM's
package-lock.json
to make things repeatable. This adds a lot of complexity for new users, and we want to make sure a single-file works well for Bicep.If we roll out this proposal this is definitely an area where we will want feedback.
Q: What happens when I upgrade/downgrade the az Provider? Changing the version of the
az
provider may change the versions of the resources declared in the file - but this is always the result of a conscious choice from the user to make a change. If users prefer the old approach they are free to pin the version of each resource.Q: How does this interact with modules? Each module is understood separately by the compiler and can version independently. There's no problem with using many different versions of
az
across modules.Q: How will you distribute versions of the Azure types (or other extensibility)? We plan to host these using OCI artifacts like the module gallery.
Q: What will happen if I have no internet connection? The tools will be able to use cached versions of the
az
types but will require an internet connection to download new versions. It's likely we'll implement a fallback code path that behaves reasonably in the event that you cannot resolve the right version. We don't want the tools to fail completely in this case.The text was updated successfully, but these errors were encountered: