perf: avoid building module graph if dynamic specifier already in graph

[dsherret]

Real fix instead of #24020

Benchmark 1: ../../deno/deno_old run -A repro.ts
  Time (mean ± σ):     272.0 ms ±   5.4 ms    [User: 267.9 ms, System: 95.2 ms]
  Range (min … max):   267.1 ms … 287.0 ms    11 runs
 
Benchmark 2: ../../deno/target/release/deno run -A repro.ts
  Time (mean ± σ):     237.2 ms ±   2.0 ms    [User: 236.0 ms, System: 93.1 ms]
  Range (min … max):   235.6 ms … 243.0 ms    12 runs
 
Summary
  ../../deno/target/release/deno run -A repro.ts ran
    1.15 ± 0.02 times faster than ../../deno/deno_old run -A repro.ts

Benchmark 1: ../../deno/deno_old run -A repro2.ts
  Time (mean ± σ):     202.2 ms ±   3.4 ms    [User: 233.0 ms, System: 61.3 ms]
  Range (min … max):   199.1 ms … 211.7 ms    14 runs
 
Benchmark 2: ../../deno/target/release/deno run -A repro2.ts
  Time (mean ± σ):     175.8 ms ±   1.8 ms    [User: 207.7 ms, System: 59.7 ms]
  Range (min … max):   173.8 ms … 179.9 ms    16 runs
 
Summary
  ../../deno/target/release/deno run -A repro2.ts ran
    1.15 ± 0.02 times faster than ../../deno/deno_old run -A repro2.ts