Module Path Substitution

[JFolberth]

Is your feature request related to a problem? Please describe.
Yes. When hosting bicep modules in a registry would like to either a.) substitute the registry URL per environment or b.) substitute the tag in the module location for the specific environment/version.

This is an issue with teams who may have different module definitions for dev/test/prod or have the desire to run things like a beta module in their dev environment. Currently the module reference requires a path that cannot be interpopulated. Other IaC providers get around this with the source property in the module block.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Something like:
module appService '${parameters.registryURL}:${parameters.registryTag}'

[ahelland]

Yeah, this is kind of a pain point.

As for allowing parameters in the module definition I guess the problem is not being able to restore the module and as such not providing intellisense/parameter validation, etc.

The registry part can be "fake-parametrized" by using aliases in the bicepconfig.json file, but since you can't have multiple bicepconfig files in the same directory that has limitations as well. (I haven't experimented building a folder hierarchy with config files.)

I currently work around the registry part by using a placeholder value like "registry": "${registry}.azurecr.io" and doing a substitution with sed in the pipeline for the actual ACR name. Feels a bit hackish though. Still doesn't solve the problem of controlling the module version across environments either.

[majastrz]

@ahelland and @JFolberth One of our goals with Bicep is to always ensure that we provide accurate completions and validation even when dealing with modules sourced from an external registry. Completely generic parameterization of the module sources makes providing that rather difficult.

I'm wondering if there is a way to simplify the problem and still provide sufficient completion and validation. Could answer the following questions about this scenario?

1. If we allowed placeholders in bicep config or module paths, are the placeholder values completely freeform in your scenarios or are they restricted to a known set?
2. If the set of placeholder values is limited, would you be open to declaring all of them in the bicepconfig.json?

Let's say you configured your registry as follows and {registry} has three values "one" and "two":

"registry": "${registry}.azurecr.io"

4. What would you expect to happen if your Bicep file declared a module from {registry} but the contents of the registry were completely different and incompatible?
5. Would a user in your environment have access to all the registries involved here or only a subset? If only a subset, would you expect compilation to fail or succeed if we can only restore from a subset of the registries?

[ahelland]

Here are my initial thoughts @majastrz :

1. I can certainly imagine freeform scenarios if I want, but I would say the "v1 use cases" could probably be served by a known set of values.
2. Ideally I would want separation like a bicepconfig_dev,json, bicepconfig_test.json, etc. split. (Based on "the dev environment has no knowledge of prod" and so forth. But realistically it's not a secret that the dev ACR is named contoso-dev.azurecr.io is we work within our own org.) Could be in different subscriptions though and be logically separated. Multiple bicepconfig files would obviously need other considerations aside registry settings.
3. I would except Bicep to return an error if it isn't able to figure out things out :) But of course one needs to accept some level of responsibility - if I have contoso-dev.azurecr.io/module-one:v1 and contoso-prod.azurecr.io/module-one:v1 it would be bad design on my part if one of the modules create a VM while the other creates a service bus. (Bad naming and provider referencing as well of course.) I could structure things so I refer to module-one-${environment} , but I don't know what guessing matrix that would create for Bicep in the background.
4. I would expect some restrictions. That could be both that developers only have access to contoso-dev.azurecr.io while a DevOps engineer might have access to contoso-prod.azurecr.io, or on a network level that an NSG locks down contoso-prod.azurecr.io so only build agents can access it. I would assume 401/403 is returned when attempting to pull from a registry where you don't have access. NSG lockdown possibly a timeout or 502? I get errors already when I refer to modules that don't exist in a given registry so it's something I need to be ready to handle anyways.

[majastrz]

@ahelland Makes sense. I've got some additional questions:

1. In your scenario, would DevOps engineers and developers both use VS code or would one of them focus mostly on CLI/pipelines?
2. When you move modules from one registry to another, do you use OCI tools to copy OCI artifacts around or do you simply do bicep publish again using the second or third registry?

[ahelland]

1. I'd say there's a healthy mix here depending on the team (as a consultant I've seen more than one team in action). IaC will be done by one or more devs using VS Code - usually towards a sandbox or dev subscription. (VS Code seems to be the standard tool for "proper" IaC like Bicep.) CLI or PowerShell used for plumbing purposes. Pipelines for pushing to other environments. (MS agents to bootstrap the lower levels of infra and then self-hosted if using private endpoints - which in turn affect the usage of ACR.)
2. The cli commands for copying from one ACR to another feel a bit clunky but is workable if you need to move specific modules. If I have ten updated modules bicep publish feels subjectively better. I've written VS Code tasks to automate the publish part so it's just a few clicks to loop through. I'm aware of ORAS, but haven't used it on the projects I'm working on.

[JFolberth]

Posting to keep the thread alive as I mostly agree with @ahelland responses.

I think at the end of the day the substitution needs the ability to occur at CLI level. This would ensure the ability to automate the deployments w/ the appropriate configurations.

For publishing the modules I feel this is out of scope of the question. Modules being published would adhere to there own lifecycle and really don't want to open this up outside of the ask of how to accommodate leveraging a single bicep file for deployment across various environments which may contain different registry values.

[majastrz]

I agree with the point about publish since bicep publish takes the exact "path" where to publish the module. I also agree about having ability to override the placeholder values via CLI. It would provide quite a lot of flexibility especially since we're not quite ready to take the leap to supporting multiple bicepconfig.json files.

I'm wondering if this could be a good starting point:

• Allow placeholders to be defined in bicepconfig.json.
• Require default values for the placeholders to be set in bicepconfig to allow VS Code to function correctly.
• Command line switches in bicep CLI to override placeholder values. Ideally, we should design these so we can perform some basic validation to catch as many errors as early as we can.
• Expose the command line switches in Az PowerShell and CLI

Thoughts?

[ahelland]

Sounds like a plan @majastrz

We're talking about exposing the registry specifically as a command line switch like --registry and not everything in bicepconfig.json right?

Definitely need a working default for the Intellisense.

Out of curiosity - what takes so long when doing the bicep publish? For publishing a single module locally it's not an issue. But when doing the publishing in a pipeline I have a couple options:

• Attempting to publish a module that hasn't been updated will produce an error/warning since it already exists. I can use the --force parameter but that potenially breaks versioning so don't want to do that.
• I could possibly implement logic to only attempt publishing new modules which makes the looping script more complex.
• Not sure if it's an option to check if the module exists and then do a conditonal based on the result.

The easiest script-wise is to just loop through the modules sub-directories and attempt all of them. Either they give the warning the module exists or a new version is published. Works fine, but adds time to the pipeline.

Oh well, not a technical problem per se I guess :)

[majastrz]

Yes, this would be specific to the registry sections, but we could consider expanding it in the future. For the issue of publish taking a long time, can you open a separate issue?

[majastrz]

I took off the labels so the bot doesn't close this.