[aws-eks] Unable to use subnets of imported VPC #10341

moserda · 2020-09-14T11:59:39Z

I'm experiencing the exact same issue as described in #6115.
If cdk.context.json is nonexistent/empty and an imported VPC is used in eks.Cluster, CDK fails with Vpc must contain private subnets to configure private endpoint access.

Reproduction Steps

const vpc = ec2.Vpc.fromLookup(this, "myvpc", {
  vpcId: "vpcId"
});

const cluster = new eks.Cluster(this, "mycluster", {
  clusterName: "mycluster",
  version: eks.KubernetesVersion.V1_17,
  defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
  defaultCapacity: 2,
  defaultCapacityInstance: new ec2.InstanceType(defaultCapacityInstance),
  endpointAccess: eks.EndpointAccess.PRIVATE,
  vpc: vpc,
  vpcSubnets: [
    {subnets: vpc.privateSubnets.slice(0, 1)}
  ]
});

What did you expect to happen?

What actually happened?

Environment

CLI Version : 1.62.0 (build 8c2d7fc)
Framework Version: 1.62.0 (build 8c2d7fc)
Node.js Version: 12.18.3
OS :
Language (Version): Typescript

Other

Stack trace

Error: Vpc must contain private subnets to configure private endpoint access
    at new Cluster (C:\Users\redacted\node_modules\@aws-cdk\aws-eks\lib\cluster.ts:931:15)
    at new PefEksStack (C:\Users\redacted\lib\pef_eks-stack.ts:123:25)
    at Object.<anonymous> (C:\Users\redacted\bin\pef_eks.ts:11:1)
    at Module._compile (internal/modules/cjs/loader.js:1137:30)
    at Module.m._compile (C:\Users\redacted\node_modules\ts-node\src\index.ts:858:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
    at Object.require.extensions.<computed> [as .ts] (C:\Users\redacted\node_modules\ts-node\src\index.ts:861:12)
    at Module.load (internal/modules/cjs/loader.js:985:32)
    at Function.Module._load (internal/modules/cjs/loader.js:878:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)

This is 🐛 Bug Report

The text was updated successfully, but these errors were encountered:

iliapolo · 2020-09-19T18:27:50Z

@dmoser04

Haven't been able to reproduce this. Are you sure your VPC actually contains private subnets? Mind sharing your cdk.context.json?

moserda · 2020-09-21T08:07:29Z

Hi @iliapolo,

I believe you're right and it was an error on our side. So what we are actually doing is something like

const vpc = ec2.Vpc.fromLookup(this, "myvpc", {
  vpcId: "vpcId"
});

const subnetsToSelect = ["subnet-03b480529ed1ddcf6", "subnet-0b551f64601296295", "subnet-082440ba51cdfe9b0"]
const selectedSubnets = vpc.privateSubnets.filter(vpcSubnet => subnetsToSelect.includes(vpcSubnet.subnetId))

const cluster = new eks.Cluster(this, "mycluster", {
  clusterName: "mycluster",
  version: eks.KubernetesVersion.V1_17,
  defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
  defaultCapacity: 2,
  defaultCapacityInstance: new ec2.InstanceType("t3.large"),
  endpointAccess: eks.EndpointAccess.PRIVATE,
  vpc: vpc,
  vpcSubnets: [
    {subnets: selectedSubnets }
  ]
});

So when CDK passes over the code the first time, it detects that the context is missing some info and refreshes it. However, selectedSubnets is empty and eks.Cluster complains with the abovementioned error.
Therefore, as a workaround, we probably need to check if the vpcId == 'vpc-12345' (i.e. the dummy vpc) and only filter to our subnets if that is not the case. Is there probably are more elegant way to detect whether CDK needs to refresh its context?

Please accept my apologies for this invalid bug report. Feel free to close it.

iliapolo · 2020-09-21T21:55:42Z

@dmoser04 I'm still a little vague as to what exactly is happening.

You mentioned:

However, selectedSubnets is empty and eks.Cluster complains with the abovementioned error.

Why is selectedSubnets empty?

Also, is subnetsToSelect an actual fixed well known id list? if so, you could do:

vpcSubnets: [
    { subnets: subnetsToSelect.map((id, index) => ec2.Subnet.fromSubnetId(this, `Subnet${index}`, id)) }
  ]

Will that do the trick?

moserda · 2020-09-22T07:13:12Z

Hi @iliapolo ,

Why is selectedSubnets empty?

When you run this code with a non-existent or empty cdk.context.json

    const vpc = ec2.Vpc.fromLookup(this, "MyVpc", { vpcId: "vpc-3ccd3554" });

    const subnetsToSelect = ["subnet-03b480529ed1ddcf6", "subnet-0b551f64601296295", "subnet-082440ba51cdfe9b0"]

    let selectedSubnets = vpc.privateSubnets.filter(vpcSubnet => subnetsToSelect.includes(vpcSubnet.subnetId))

    // Check if the correct subnets are resolved after CDK has refreshed its context (i.e. after 
    // the vpcId is no longer the dummy vpc id)
    // if (vpc.vpcId != 'vpc-12345') {
    //   const a = subnetsToSelect.sort()
    //   const b = selectedSubnets.map(sn => sn.subnetId).sort()
    //   for(let i=0; i<a.length; i++) {
    //     assert(a[i] === b[i])
    //   }      
    // } else {
    //   // Otherwise eks.Cluster would complain
    //   selectedSubnets = vpc.privateSubnets
    // }

    console.log("----------------- VPC Info -----------------")
    console.log(vpc)
    console.log("----------------- /VPC Info -----------------")

    console.log("----------------- Subnet Info -----------------")
    console.log(selectedSubnets)
    console.log("----------------- /Subnet Info -----------------")

    const cluster = new eks.Cluster(this, "mycluster", {
      clusterName: "mycluster",
      version: eks.KubernetesVersion.V1_17,
      defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
      defaultCapacity: 2,
      defaultCapacityInstance: new ec2.InstanceType("t3.large"),
      endpointAccess: eks.EndpointAccess.PRIVATE,
      vpc: vpc,
      vpcSubnets: [
        { subnets: selectedSubnets }
      ]
    });

it will fail with the error shown above. From the log you can see, that CDK is using a dummy VPC (vpc-12345) with dummy subnets (s-12345, s-67890, p-12345, and p-67890). Therefore selectedSubnets is an empty list.

PS C:\Users\REDACTED\eksclusterissue> .\node_modules\.bin\cdk ls -v --profile eksissue
CDK toolkit version: 1.63.0 (build 7a68125)
Command line arguments: {
  _: [ 'ls' ],
  v: 1,
  verbose: 1,
  profile: 'eksissue',
  'ignore-errors': false,
  ignoreErrors: false,
  json: false,
  j: false,
  ec2creds: undefined,
  i: undefined,
  'version-reporting': undefined,
  versionReporting: undefined,
  'path-metadata': true,
  pathMetadata: true,
  'asset-metadata': true,
  assetMetadata: true,
  'role-arn': undefined,
  r: undefined,
  roleArn: undefined,
  staging: true,
  'no-color': false,
  noColor: false,
  fail: false,
  long: false,
  l: false,
  '$0': 'node_modules\\aws-cdk\\bin\\cdk'
}
cdk.json: {
  "app": "npx ts-node bin/eksclusterissue.ts",
  "context": {
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true",
    "@aws-cdk/core:stackRelativeExports": "true"
  }
}
merged settings: {
  versionReporting: true,
  pathMetadata: true,
  output: 'cdk.out',
  app: 'npx ts-node bin/eksclusterissue.ts',
  context: {
    '@aws-cdk/core:enableStackNameDuplicates': 'true',
    'aws-cdk:enableDiffNoFail': 'true',
    '@aws-cdk/core:stackRelativeExports': 'true'
  },
  assetMetadata: true,
  profile: 'eksissue',
  toolkitBucket: {},
  staging: true
}
Determining if we're on an EC2 instance.
Does not look like an EC2 instance.
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to eu-central-1
Resolving default credentials
Retrieved account ID REDACTED from disk cache
Setting "CDK_DEFAULT_ACCOUNT" environment variable to REDACTED
context: {
  '@aws-cdk/core:enableStackNameDuplicates': 'true',
  'aws-cdk:enableDiffNoFail': 'true',
  '@aws-cdk/core:stackRelativeExports': 'true',
  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true
}
outdir: cdk.out
env: {
  CDK_DEFAULT_REGION: 'eu-central-1',
  CDK_DEFAULT_ACCOUNT: 'REDACTED',
  CDK_CONTEXT_JSON: '{"@aws-cdk/core:enableStackNameDuplicates":"true","aws-cdk:enableDiffNoFail":"true","@aws-cdk/core:stackRelativeExports":"true","aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '5.0.0',
  CDK_CLI_VERSION: '1.63.0'
}
----------------- VPC Info -----------------
LookedUpVpc {
  node: ConstructNode {
    host: [Circular],
    _actualNode: Node {
      host: [Circular],
      _locked: false,
      _aspects: [],
      _children: [Object],
      _context: {},
      _metadata: [],
      _dependencies: Set {},
      invokedAspects: [],
      id: 'MyVpc',
      scope: [EksclusterissueStack]
    }
  },
  stack: EksclusterissueStack {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    _missingContext: [ [Object] ],
    _stackDependencies: {},
    templateOptions: {},
    _logicalIds: LogicalIDs { renames: {}, reverse: {} },
    account: 'REDACTED',
    region: 'eu-central-1',
    environment: 'aws:https://REDACTED/eu-central-1',
    terminationProtection: undefined,
    _stackName: 'EksclusterissueStack',
    tags: TagManager {
      tags: Map {},
      priorities: Map {},
      initialTagPriority: 50,
      resourceTypeName: 'aws:cdk:stack',
      tagFormatter: KeyValueFormatter {},
      tagPropertyName: 'tags'
    },
    artifactId: 'EksclusterissueStack',
    templateFile: 'EksclusterissueStack.template.json',
    synthesizer: LegacyStackSynthesizer {
      cycle: false,
      addedImageAssets: Set {},
      stack: [Circular]
    },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  },
  env: { account: 'REDACTED', region: 'eu-central-1' },
  _physicalName: undefined,
  _allowCrossEnvironment: false,
  physicalName: '${Token[TOKEN.52]}',
  natDependencies: [],
  incompleteSubnetDefinition: true,
  internetConnectivityEstablished: ConcreteDependable {
    _dependencyRoots: [],
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Getter] }
  },
  vpcId: 'vpc-12345',
  cidr: '1.2.3.4/5',
  _vpnGatewayId: undefined,
  availabilityZones: [ 'dummy1a', 'dummy1b' ],
  publicSubnets: [
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.53]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1a',
      subnetId: 's-12345',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.54]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1b',
      subnetId: 's-67890',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    }
  ],
  privateSubnets: [
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.55]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1a',
      subnetId: 'p-12345',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.56]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1b',
      subnetId: 'p-67890',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    }
  ],
  isolatedSubnets: [],
  [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [ [Circular] ] }
}
----------------- /VPC Info -----------------
----------------- Subnet Info -----------------
[]
----------------- /Subnet Info -----------------

C:\Users\REDACTED\eksclusterissue\node_modules\@aws-cdk\aws-eks\lib\cluster.ts:875
      throw new Error('Vpc must contain private subnets when public endpoint access is disabled');
            ^
Error: Vpc must contain private subnets when public endpoint access is disabled
    at new Cluster (C:\Users\REDACTED\eksclusterissue\node_modules\@aws-cdk\aws-eks\lib\cluster.ts:875:13)
    at new EksclusterissueStack (C:\Users\REDACTED\eksclusterissue\lib\eksclusterissue-stack.ts:37:21)
    at Object.<anonymous> (C:\Users\REDACTED\eksclusterissue\bin\eksclusterissue.ts:7:1)
    at Module._compile (internal/modules/cjs/loader.js:1137:30)
    at Module.m._compile (C:\Users\REDACTED\eksclusterissue\node_modules\ts-node\src\index.ts:858:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
    at Object.require.extensions.<computed> [as .ts] (C:\Users\REDACTED\eksclusterissue\node_modules\ts-node\src\index.ts:861:12)
    at Module.load (internal/modules/cjs/loader.js:985:32)
    at Function.Module._load (internal/modules/cjs/loader.js:878:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
Subprocess exited with error 1
Error: Subprocess exited with error 1
    at ChildProcess.<anonymous> (C:\Users\REDACTED\eksclusterissue\node_modules\aws-cdk\lib\api\cxapp\exec.ts:118:23)
    at ChildProcess.emit (events.js:315:20)
    at ChildProcess.EventEmitter.emit (domain.js:483:12)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)

However, if we include the commented code as well, it works because CDK seems to execute everything twice:

Execute the code and detect which context info is missing. Because we only have the dummy vpc at this point, we use the dummy subnets as well because otherwise eks.Cluster would fail
Fetch the required context info and execute code again. This time the VPC is correctly resolved and selectedSubnets is filled.

See the log

PS C:\Users\REDACTED\eksclusterissue> .\node_modules\.bin\cdk ls -v --profile eksissue
CDK toolkit version: 1.63.0 (build 7a68125)
Command line arguments: {
  _: [ 'ls' ],
  v: 1,
  verbose: 1,
  profile: 'eksissue',
  'ignore-errors': false,
  ignoreErrors: false,
  json: false,
  j: false,
  ec2creds: undefined,
  i: undefined,
  'version-reporting': undefined,
  versionReporting: undefined,
  'path-metadata': true,
  pathMetadata: true,
  'asset-metadata': true,
  assetMetadata: true,
  'role-arn': undefined,
  r: undefined,
  roleArn: undefined,
  staging: true,
  'no-color': false,
  noColor: false,
  fail: false,
  long: false,
  l: false,
  '$0': 'node_modules\\aws-cdk\\bin\\cdk'
}
cdk.json: {
  "app": "npx ts-node bin/eksclusterissue.ts",
  "context": {
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true",
    "@aws-cdk/core:stackRelativeExports": "true"
  }
}
merged settings: {
  versionReporting: true,
  pathMetadata: true,
  output: 'cdk.out',
  app: 'npx ts-node bin/eksclusterissue.ts',
  context: {
    '@aws-cdk/core:enableStackNameDuplicates': 'true',
    'aws-cdk:enableDiffNoFail': 'true',
    '@aws-cdk/core:stackRelativeExports': 'true'
  },
  assetMetadata: true,
  profile: 'eksissue',
  toolkitBucket: {},
  staging: true
}
Determining if we're on an EC2 instance.
Does not look like an EC2 instance.
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to eu-central-1
Resolving default credentials
Retrieved account ID REDACTED from disk cache
Setting "CDK_DEFAULT_ACCOUNT" environment variable to REDACTED
context: {
  '@aws-cdk/core:enableStackNameDuplicates': 'true',
  'aws-cdk:enableDiffNoFail': 'true',
  '@aws-cdk/core:stackRelativeExports': 'true',
  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true
}
outdir: cdk.out
env: {
  CDK_DEFAULT_REGION: 'eu-central-1',
  CDK_DEFAULT_ACCOUNT: 'REDACTED',
  CDK_CONTEXT_JSON: '{"@aws-cdk/core:enableStackNameDuplicates":"true","aws-cdk:enableDiffNoFail":"true","@aws-cdk/core:stackRelativeExports":"true","aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '5.0.0',
  CDK_CLI_VERSION: '1.63.0'
}
----------------- VPC Info -----------------
LookedUpVpc {
  node: ConstructNode {
    host: [Circular],
    _actualNode: Node {
      host: [Circular],
      _locked: false,
      _aspects: [],
      _children: [Object],
      _context: {},
      _metadata: [],
      _dependencies: Set {},
      invokedAspects: [],
      id: 'MyVpc',
      scope: [EksclusterissueStack]
    }
  },
  stack: EksclusterissueStack {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    _missingContext: [ [Object] ],
    _stackDependencies: {},
    templateOptions: {},
    _logicalIds: LogicalIDs { renames: {}, reverse: {} },
    account: 'REDACTED',
    region: 'eu-central-1',
    environment: 'aws:https://REDACTED/eu-central-1',
    terminationProtection: undefined,
    _stackName: 'EksclusterissueStack',
    tags: TagManager {
      tags: Map {},
      priorities: Map {},
      initialTagPriority: 50,
      resourceTypeName: 'aws:cdk:stack',
      tagFormatter: KeyValueFormatter {},
      tagPropertyName: 'tags'
    },
    artifactId: 'EksclusterissueStack',
    templateFile: 'EksclusterissueStack.template.json',
    synthesizer: LegacyStackSynthesizer {
      cycle: false,
      addedImageAssets: Set {},
      stack: [Circular]
    },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  },
  env: { account: 'REDACTED', region: 'eu-central-1' },
  _physicalName: undefined,
  _allowCrossEnvironment: false,
  physicalName: '${Token[TOKEN.52]}',
  natDependencies: [],
  incompleteSubnetDefinition: true,
  internetConnectivityEstablished: ConcreteDependable {
    _dependencyRoots: [],
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Getter] }
  },
  vpcId: 'vpc-12345',
  cidr: '1.2.3.4/5',
  _vpnGatewayId: undefined,
  availabilityZones: [ 'dummy1a', 'dummy1b' ],
  publicSubnets: [
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.53]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1a',
      subnetId: 's-12345',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.54]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1b',
      subnetId: 's-67890',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    }
  ],
  privateSubnets: [
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.55]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1a',
      subnetId: 'p-12345',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.56]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'dummy1b',
      subnetId: 'p-67890',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    }
  ],
  isolatedSubnets: [],
  [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [ [Circular] ] }
}
----------------- /VPC Info -----------------
----------------- Subnet Info -----------------
[
  ImportedSubnet {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    stack: EksclusterissueStack {
      node: [ConstructNode],
      _missingContext: [Array],
      _stackDependencies: {},
      templateOptions: {},
      _logicalIds: [LogicalIDs],
      account: 'REDACTED',
      region: 'eu-central-1',
      environment: 'aws:https://REDACTED/eu-central-1',
      terminationProtection: undefined,
      _stackName: 'EksclusterissueStack',
      tags: [TagManager],
      artifactId: 'EksclusterissueStack',
      templateFile: 'EksclusterissueStack.template.json',
      synthesizer: [LegacyStackSynthesizer],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    env: { account: 'REDACTED', region: 'eu-central-1' },
    _physicalName: undefined,
    _allowCrossEnvironment: false,
    physicalName: '${Token[TOKEN.55]}',
    internetConnectivityEstablished: ConcreteDependable {
      _dependencyRoots: [],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    _availabilityZone: 'dummy1a',
    subnetId: 'p-12345',
    routeTable: { routeTableId: 'rtb-12345p' },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  },
  ImportedSubnet {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    stack: EksclusterissueStack {
      node: [ConstructNode],
      _missingContext: [Array],
      _stackDependencies: {},
      templateOptions: {},
      _logicalIds: [LogicalIDs],
      account: 'REDACTED',
      region: 'eu-central-1',
      environment: 'aws:https://REDACTED/eu-central-1',
      terminationProtection: undefined,
      _stackName: 'EksclusterissueStack',
      tags: [TagManager],
      artifactId: 'EksclusterissueStack',
      templateFile: 'EksclusterissueStack.template.json',
      synthesizer: [LegacyStackSynthesizer],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    env: { account: 'REDACTED', region: 'eu-central-1' },
    _physicalName: undefined,
    _allowCrossEnvironment: false,
    physicalName: '${Token[TOKEN.56]}',
    internetConnectivityEstablished: ConcreteDependable {
      _dependencyRoots: [],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    _availabilityZone: 'dummy1b',
    subnetId: 'p-67890',
    routeTable: { routeTableId: 'rtb-57890p' },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  }
]
----------------- /Subnet Info -----------------
Some context information is missing. Fetching...
Listing VPCs in REDACTED:eu-central-1
Describing VPC vpc-3ccd3554
Setting "vpc-provider:account=REDACTED:filter.vpc-id=vpc-3ccd3554:region=eu-central-1:returnAsymmetricSubnets=true" context to {"vpcId":"vpc-3ccd3554","vpcCidrBlock":"172.31.0.0/16","availabilityZones":[],"subnetGroups":[{"name":"Public","type":"Public","subnets":[{"subnetId":"subnet-09279861","cidr":"172.31.0.0/20","availabilityZone":"eu-central-1a","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-7c408206","cidr":"172.31.16.0/20","availabilityZone":"eu-central-1b","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-c3e9d689","cidr":"172.31.32.0/20","availabilityZone":"eu-central-1c","routeTableId":"rtb-29e74241"}]},{"name":"Private","type":"Private","subnets":[{"subnetId":"subnet-082440ba51cdfe9b0","cidr":"172.31.80.0/20","availabilityZone":"eu-central-1b","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-0b551f64601296295","cidr":"172.31.64.0/20","availabilityZone":"eu-central-1c","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-03b480529ed1ddcf6","cidr":"172.31.48.0/20","availabilityZone":"eu-central-1c","routeTableId":"rtb-29e74241"}]}]}
Setting "CDK_DEFAULT_REGION" environment variable to eu-central-1
Setting "CDK_DEFAULT_ACCOUNT" environment variable to REDACTED
context: {
  'vpc-provider:account=REDACTED:filter.vpc-id=vpc-3ccd3554:region=eu-central-1:returnAsymmetricSubnets=true': {
    vpcId: 'vpc-3ccd3554',
    vpcCidrBlock: '172.31.0.0/16',
    availabilityZones: [],
    subnetGroups: [ [Object], [Object] ]
  },
  '@aws-cdk/core:enableStackNameDuplicates': 'true',
  'aws-cdk:enableDiffNoFail': 'true',
  '@aws-cdk/core:stackRelativeExports': 'true',
  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true
}
outdir: cdk.out
env: {
  CDK_DEFAULT_REGION: 'eu-central-1',
  CDK_DEFAULT_ACCOUNT: 'REDACTED',
  CDK_CONTEXT_JSON: '{"vpc-provider:account=REDACTED:filter.vpc-id=vpc-3ccd3554:region=eu-central-1:returnAsymmetricSubnets=true":{"vpcId":"vpc-3ccd3554","vpcCidrBlock":"172.31.0.0/16","availabilityZones":[],"subnetGroups":[{"name":"Public","type":"Public","subnets":[{"subnetId":"subnet-09279861","cidr":"172.31.0.0/20","availabilityZone":"eu-central-1a","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-7c408206","cidr":"172.31.16.0/20","availabilityZone":"eu-central-1b","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-c3e9d689","cidr":"172.31.32.0/20","availabilityZone":"eu-central-1c","routeTableId":"rtb-29e74241"}]},{"name":"Private","type":"Private","subnets":[{"subnetId":"subnet-082440ba51cdfe9b0","cidr":"172.31.80.0/20","availabilityZone":"eu-central-1b","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-0b551f64601296295","cidr":"172.31.64.0/20","availabilityZone":"eu-central-1c","routeTableId":"rtb-29e74241"},{"subnetId":"subnet-03b480529ed1ddcf6","cidr":"172.31.48.0/20","availabilityZone":"eu-central-1c","routeTableId":"rtb-29e74241"}]}]},"@aws-cdk/core:enableStackNameDuplicates":"true","aws-cdk:enableDiffNoFail":"true","@aws-cdk/core:stackRelativeExports":"true","aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '5.0.0',
  CDK_CLI_VERSION: '1.63.0'
}
----------------- VPC Info -----------------
LookedUpVpc {
  node: ConstructNode {
    host: [Circular],
    _actualNode: Node {
      host: [Circular],
      _locked: false,
      _aspects: [],
      _children: [Object],
      _context: {},
      _metadata: [],
      _dependencies: Set {},
      invokedAspects: [],
      id: 'MyVpc',
      scope: [EksclusterissueStack]
    }
  },
  stack: EksclusterissueStack {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    _missingContext: [],
    _stackDependencies: {},
    templateOptions: {},
    _logicalIds: LogicalIDs { renames: {}, reverse: {} },
    account: 'REDACTED',
    region: 'eu-central-1',
    environment: 'aws:https://REDACTED/eu-central-1',
    terminationProtection: undefined,
    _stackName: 'EksclusterissueStack',
    tags: TagManager {
      tags: Map {},
      priorities: Map {},
      initialTagPriority: 50,
      resourceTypeName: 'aws:cdk:stack',
      tagFormatter: KeyValueFormatter {},
      tagPropertyName: 'tags'
    },
    artifactId: 'EksclusterissueStack',
    templateFile: 'EksclusterissueStack.template.json',
    synthesizer: LegacyStackSynthesizer {
      cycle: false,
      addedImageAssets: Set {},
      stack: [Circular]
    },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  },
  env: { account: 'REDACTED', region: 'eu-central-1' },
  _physicalName: undefined,
  _allowCrossEnvironment: false,
  physicalName: '${Token[TOKEN.52]}',
  natDependencies: [],
  incompleteSubnetDefinition: false,
  internetConnectivityEstablished: ConcreteDependable {
    _dependencyRoots: [],
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Getter] }
  },
  vpcId: 'vpc-3ccd3554',
  cidr: '172.31.0.0/16',
  _vpnGatewayId: undefined,
  availabilityZones: [ 'eu-central-1a', 'eu-central-1b', 'eu-central-1c' ],
  publicSubnets: [
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.53]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'eu-central-1a',
      subnetId: 'subnet-09279861',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.54]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'eu-central-1b',
      subnetId: 'subnet-7c408206',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.55]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'eu-central-1c',
      subnetId: 'subnet-c3e9d689',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    }
  ],
  privateSubnets: [
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.56]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'eu-central-1b',
      subnetId: 'subnet-082440ba51cdfe9b0',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.57]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'eu-central-1c',
      subnetId: 'subnet-0b551f64601296295',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    ImportedSubnet {
      node: [ConstructNode],
      stack: [EksclusterissueStack],
      env: [Object],
      _physicalName: undefined,
      _allowCrossEnvironment: false,
      physicalName: '${Token[TOKEN.58]}',
      internetConnectivityEstablished: [ConcreteDependable],
      _availabilityZone: 'eu-central-1c',
      subnetId: 'subnet-03b480529ed1ddcf6',
      routeTable: [Object],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    }
  ],
  isolatedSubnets: [],
  [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [ [Circular] ] }
}
----------------- /VPC Info -----------------
----------------- Subnet Info -----------------
[
  ImportedSubnet {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    stack: EksclusterissueStack {
      node: [ConstructNode],
      _missingContext: [],
      _stackDependencies: {},
      templateOptions: {},
      _logicalIds: [LogicalIDs],
      account: 'REDACTED',
      region: 'eu-central-1',
      environment: 'aws:https://REDACTED/eu-central-1',
      terminationProtection: undefined,
      _stackName: 'EksclusterissueStack',
      tags: [TagManager],
      artifactId: 'EksclusterissueStack',
      templateFile: 'EksclusterissueStack.template.json',
      synthesizer: [LegacyStackSynthesizer],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    env: { account: 'REDACTED', region: 'eu-central-1' },
    _physicalName: undefined,
    _allowCrossEnvironment: false,
    physicalName: '${Token[TOKEN.56]}',
    internetConnectivityEstablished: ConcreteDependable {
      _dependencyRoots: [],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    _availabilityZone: 'eu-central-1b',
    subnetId: 'subnet-082440ba51cdfe9b0',
    routeTable: { routeTableId: 'rtb-29e74241' },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  },
  ImportedSubnet {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    stack: EksclusterissueStack {
      node: [ConstructNode],
      _missingContext: [],
      _stackDependencies: {},
      templateOptions: {},
      _logicalIds: [LogicalIDs],
      account: 'REDACTED',
      region: 'eu-central-1',
      environment: 'aws:https://REDACTED/eu-central-1',
      terminationProtection: undefined,
      _stackName: 'EksclusterissueStack',
      tags: [TagManager],
      artifactId: 'EksclusterissueStack',
      templateFile: 'EksclusterissueStack.template.json',
      synthesizer: [LegacyStackSynthesizer],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    env: { account: 'REDACTED', region: 'eu-central-1' },
    _physicalName: undefined,
    _allowCrossEnvironment: false,
    physicalName: '${Token[TOKEN.57]}',
    internetConnectivityEstablished: ConcreteDependable {
      _dependencyRoots: [],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    _availabilityZone: 'eu-central-1c',
    subnetId: 'subnet-0b551f64601296295',
    routeTable: { routeTableId: 'rtb-29e74241' },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  },
  ImportedSubnet {
    node: ConstructNode { host: [Circular], _actualNode: [Node] },
    stack: EksclusterissueStack {
      node: [ConstructNode],
      _missingContext: [],
      _stackDependencies: {},
      templateOptions: {},
      _logicalIds: [LogicalIDs],
      account: 'REDACTED',
      region: 'eu-central-1',
      environment: 'aws:https://REDACTED/eu-central-1',
      terminationProtection: undefined,
      _stackName: 'EksclusterissueStack',
      tags: [TagManager],
      artifactId: 'EksclusterissueStack',
      templateFile: 'EksclusterissueStack.template.json',
      synthesizer: [LegacyStackSynthesizer],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    env: { account: 'REDACTED', region: 'eu-central-1' },
    _physicalName: undefined,
    _allowCrossEnvironment: false,
    physicalName: '${Token[TOKEN.58]}',
    internetConnectivityEstablished: ConcreteDependable {
      _dependencyRoots: [],
      [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
    },
    _availabilityZone: 'eu-central-1c',
    subnetId: 'subnet-03b480529ed1ddcf6',
    routeTable: { routeTableId: 'rtb-29e74241' },
    [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
  }
]
----------------- /Subnet Info -----------------
EksclusterissueStack

Also, is subnetsToSelect an actual fixed well known id list? if so, you could do:

Yes, the VPC and subnet ids are fixed and injected via the build pipeline.

vpcSubnets: [
    { subnets: subnetsToSelect.map((id, index) => ec2.Subnet.fromSubnetId(this, `Subnet${index}`, id)) }
  ]

Will that do the trick?

Works like a charm! Many thank for that tip

iliapolo · 2020-09-22T11:27:55Z

@dmoser04 Thanks for the details. I'll see if I can make sense of it cause something still feels fishy.

In any case, glad it helped 👍

moserda added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 14, 2020

github-actions bot assigned iliapolo Sep 14, 2020

github-actions bot added the @aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service label Sep 14, 2020

iliapolo added effort/small Small work item – less than a day of effort p1 and removed needs-triage This issue or PR still needs to be triaged. labels Sep 14, 2020

iliapolo added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed effort/small Small work item – less than a day of effort p1 labels Sep 19, 2020

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Sep 22, 2020

iliapolo closed this as completed Sep 22, 2020

iliapolo added guidance Question that needs advice or information. and removed bug This issue is a bug. labels Sep 22, 2020

codeJack mentioned this issue Mar 16, 2022

(aws-eks) Dummy VPC injected during lookup pass cause dependent constructs to eventually fail #19425

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[aws-eks] Unable to use subnets of imported VPC #10341

[aws-eks] Unable to use subnets of imported VPC #10341

moserda commented Sep 14, 2020

iliapolo commented Sep 19, 2020

moserda commented Sep 21, 2020

iliapolo commented Sep 21, 2020

moserda commented Sep 22, 2020

iliapolo commented Sep 22, 2020

[aws-eks] Unable to use subnets of imported VPC #10341

[aws-eks] Unable to use subnets of imported VPC #10341

Comments

moserda commented Sep 14, 2020

Reproduction Steps

What did you expect to happen?

What actually happened?

Environment

Other

iliapolo commented Sep 19, 2020

moserda commented Sep 21, 2020

iliapolo commented Sep 21, 2020

moserda commented Sep 22, 2020

iliapolo commented Sep 22, 2020