Service Principal creation lags behind final validation (ServicePrincipalNotFound)

[stvhwrd]

What happened:

When creating a Kubernetes cluster in the Azure Portal (GUI) and creating a new service principal with defaults, the final validation fails:

{
    "code": "InvalidTemplateDeployment",
    "message": "The template deployment 'microsoft.aks-20190822092943' is not valid according to the validation procedure. The tracking id is '81f94590-473d-41f8-b415-91d9c5adcf6a'. See inner errors for details. Please see https://aka.ms/arm-deploy for usage details.",
    "details":
    [
        {
            "code": "ServicePrincipalNotFound",
            "message": "Provisioning of resource(s) for container service stehowa-cluster in resource group stehowa failed. Message: {\n \"code\": \"ServicePrincipalNotFound\",\n \"message\": \"Internal server error\"\n }. Details: "
        }
    ]
}

After waiting a few minutes and re-running the validation (with no other changes), it passes successfully.

What you expected to happen:

Final validation to succeed on first run.

How to reproduce it (as minimally and precisely as possible):

Create a new Kubernetes cluster via the Azure Portal GUI and use default settings for service principal.

Anything else we need to know?:

Nope!

Environment:

• Kubernetes version (use kubectl version): 1.13.10 (default)
• Size of cluster (how many worker nodes are in the cluster?) 1
• General description of workloads in the cluster (e.g. HTTP microservices, Java app, Ruby on Rails, machine learning, etc.) N/A
• Others:

[ghost]

Hi:

I have the same problem when creating the cluster,

Greetings

[sarmadjari]

I have the same issue when creating AKS cluster through the portal, did few tests with one of Azure Support Engineers and we confirm this issue.

[stvhwrd]

I reported the bug from within the Azure portal and received this response from Azure IaaS:

Unfortunately a known issue – there’s a replication delay in AAD ☹ We’re working with both the AKS RP and AAD graph teams to try to address this issue but we don’t have any quick fix we can do.

[Ricciolo]

As a workaround, when you get the error, go back to Authentication section, click configure on service principal and select use existing. Id and secret should be already set from previous review process. Go to the of the wizard and try again

[stvhwrd]

Thanks @Ricciolo, that is definitely one way to do it — but ultimately it’s just about waiting for that service principal to be created. You don’t actually need to change any configuration to make it work, you just need to wait a couple of minutes and then force a page refresh. The original report mentions this.

[ahelwer]

Even worse is that this issue is intermittent - You can run Test-AzResourceGroupDeployment until it stops spitting out the ServicePrincipalNotFound error, but then you run New-AzResourceGroupDeployment and it fails with ServicePrincipalNotFound. Please just add a large number of retries in the validation before saying the service principal does not exist!

This issue also occurs if you create the service principal yourself just prior to running the deployment.

This is related to issue #1206

[jluk]

Hey folks, I'm going to close this in favor of issue #1206 as this looks like a duplicate. Please bring all comments into that issue to help consolidate. If you feel like it should be re-opened as a separate issue just comment and I'll revisit.

[stvhwrd]

Thanks @jluk 🤙