Publish with sources part 1

[StephenWeatherford]

Microsoft Reviewers: Open in CodeFlow

Draft design doc: https://gist.github.com/StephenWeatherford/723d1a8ac530cfd4ba734d6aef0c545a

CHANGES:

lang server

new temporary feature flag "publishSource"

• To publish a module with sources, publishSource must be set to true in a bicepconfig.json file accessible from the module being published.
• a lot of the changes were just related to getting the publish sources feature flag through to code and tests

Added "bicepSources" stream parameter to PublishArtifact paths (thinking of changing this to a SourceArchive reference, since that's what the stream is) and module dispatcher.

AzureContainerRegistryManager - moved creation of layers to callers (documentationUri/description concepts are better handled by the module-specific code)

Added TryGetSource to ArtifactRegistry classes

OciModuleRegistry.cs

• extracted MediaTypeComparer to common code
• a few renames
• create layers here before calling AzureContainerRegistryManager
• write source archive file to registry cache on disk
• added a separate bug Consider Write out OCI layers as binary files instead of main.json/source.zip/etc - may require versioning the module cache #11900 to deal with writing binary layers instead of this

new SourceArchive - Contains the individual source code files for a Bicep file and all of its dependencies.

• retrieves all the sources from a compilation group (thus representing a bicep file and all its dependencies in JSON form)
• dealing with showing bicep for module dependencies will happen later
• needs more work, will be done in later PR

ISourceFile - added ability to return its exact source in text so that SourceArchive can retrieve it from a compilation group

vscode/lang server (these changes apply even if feature flag not on):

• Changed the interface between lang server and vscode extension in showing source code, in order to handle displaying .bicep source instead of just .json. The URI path has to end in ".bicep" in order for vscode to display .bicep files as Bicep. That wasn't previously the case.
• note: there is no easy way to determine the publishSource feature flag in this scenario, so if we find the source.tar.gz file, we always display bicep source instead of JSON
• note: Right now, we only display the main .bicep file, you can't traverse into deep dependencies

Testing

• New CachedModules/CachedModuleExtensions.cs to allow asserting against the current state of the on-disk registry cache (supersedes FeatureProviderAssertions.cs), and other testing additions

Example of registry cache with restored source:

A look inside the source.tar.gz file:

An example __metadata file (there's work to be done, including removing sensitive path information from files)

{
    "metadataVersion": 0,
    "entryPoint": "my complicated module.bicep",
    "sourceFiles": [
        {
            "path": "my complicated module.bicep",
            "archivePath": "my complicated module.bicep",
            "kind": "bicep"
        },
        {
            "path": "modules/main.bicep",
            "archivePath": "modules/main.bicep",
            "kind": "bicep"
        },
        {
            "path": "../../../../../.bicep/ts/e5ef2b13-6478-4887-ad57-1aa6b9475040/sawbicep/storagespec/1.0a/main.json",
            "archivePath": "../../../../../.bicep/ts/e5ef2b13-6478-4887-ad57-1aa6b9475040/sawbicep/storagespec/1.0a/main.json",
            "kind": "templateSpec"
        },
        {
            "path": "../simpleModule/storageAccount.bicep",
            "archivePath": "../simpleModule/storageAccount.bicep",
            "kind": "bicep"
        },
        {
            "path": "../../../../../.bicep/br/mcr.microsoft.com/bicep$app$dapr-containerapps-environment/1.2.2$/main.json",
            "archivePath": "../../../../../.bicep/br/mcr.microsoft.com/bicep$app$dapr-containerapps-environment/1.2.2$/main.json",
            "kind": "armTemplate"
        },
        {
            "path": "../../../../../.bicep/br/mcr.microsoft.com/bicep$samples$hello-world/1.0.2$/main.json",
            "archivePath": "../../../../../.bicep/br/mcr.microsoft.com/bicep$samples$hello-world/1.0.2$/main.json",
            "kind": "armTemplate"
        },
        {
            "path": "../../../../../.bicep/ts/e5ef2b13-6478-4887-ad57-1aa6b9475040/sawbicep/storagespec/2.0a/main.json",
            "archivePath": "../../../../../.bicep/ts/e5ef2b13-6478-4887-ad57-1aa6b9475040/sawbicep/storagespec/2.0a/main.json",
            "kind": "templateSpec"
        },
        {
            "path": "../../../../../.bicep/br/mcr.microsoft.com/bicep$app$app-configuration/1.0.1$/main.json",
            "archivePath": "../../../../../.bicep/br/mcr.microsoft.com/bicep$app$app-configuration/1.0.1$/main.json",
            "kind": "armTemplate"
        }
    ]
}

[github-actions]

Test this change out locally with the following install scripts (Action run 6331540643)

VSCode

• Mac/Linux

  bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 6331540643

• Windows

  iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 6331540643"

Azure CLI

• Mac/Linux

  bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 6331540643

• Windows

  iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 6331540643"

[github-actions]

Test Results

     132 files  ±    0       132 suites  ±0   3h 18m 36s ⏱️ +3s
10 653 tests +100  10 653 ✔️ +100  0 💤 ±0  0 ❌ ±0 
51 479 runs  +716  51 479 ✔️ +716  0 💤 ±0  0 ❌ ±0 

Results for commit 6916646. ± Comparison against base commit 1abffea.

♻️ This comment has been updated with latest results.

[majastrz]

  "properties": {

PR description stated that publishSource is a new feature. Don't we need to add it here?

---

Refers to: src/vscode-bicep/schemas/bicepconfig.schema.json:625 in 8901b7b. [](commit_id = 8901b7b, deletion_comment = False)

[StephenWeatherford]

  "properties": {

PR description stated that publishSource is a new feature. Don't we need to add it here?

Refers to: src/vscode-bicep/schemas/bicepconfig.schema.json:625 in 8901b7b. [](commit_id = 8901b7b, deletion_comment = False)

Fixed in 6916646

[majastrz]

  "properties": {

I saw the other comment. Makes sense if the feature is temporary.

---

In reply to: 1735977506

---

Refers to: src/vscode-bicep/schemas/bicepconfig.schema.json:625 in 8901b7b. [](commit_id = 8901b7b, deletion_comment = False)

[majastrz]

[StephenWeatherford]

  "properties": {

I saw the other comment. Makes sense if the feature is temporary.

In reply to: 1735977506

Refers to: src/vscode-bicep/schemas/bicepconfig.schema.json:625 in 8901b7b. [](commit_id = 8901b7b, deletion_comment = False)

I thought all experimental features were temporary by definition. No?