deps: upgrade npm beta to [email protected]

[zkat]

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• documentation is changed or added
• commit message follows commit guidelines

Affected core subsystem(s)

• deps

Description of Changes

Ahahaha it's May 9th! Check it out!

This is the first PR for the long-awaited npm@5. This is beta software, but we intend to have it production-ready by May 30th. This release has all the really big changes going in.

The biggest thing to note about this release is that it involves a rewrite of significant subsystems. Most notably, the entire caching and package-fetching layer have been replaced with new code, right down to the http client being used.

Beyond that, this release includes a number of significant ux-related changes: --save happens by default, a new package-lock.json is generated by default. This file is similar to npm-shrinkwrap.json, and both of the files have been overhauled to play nicer with cross-platform setups and file dependencies.

The release cooked up for this PR is mirroring the current beta releases, but with the binary still named npm, instead of npm5. It also omits the beta notice that happens every time you run the "main" beta client. You can expect this particular release to have a number of issues (tracked in the beta tracking issue for npm@5. There's a number of tests we've excluded from the main test suite because they need to have a few updates before they pass with npm5.

Edit: I know this PR is a bit last minute, but we already talked about how tight a timeline this would be for us. We're working really hard on this release to get it stable and you should see bugfixes and perf improvements happen regularly. We can talk about how often you want us to submit new beta versions to the node repo -- npm i -g npm5 is still the preferred method of testing the beta out, for us.

Changelog

• npm@5 ongoing changelog

[refack]

@zkat I know it's not the place, but we need npm/npm#16484 in.

[addaleax]

@jasnell Jaaaaaaaames… please tell me it’s not too late to have this in v8.x? :)

(Honestly, I think having this in there would be good because the perf improvements alone should reflect pretty well on Node v8.0.0 itself. :3)

[zkat]

ahahaha sorry. That fell through the cracks in our dev branches even though I already landed it in the main release-next-5 branch. I'll roll up a new release later? I'm about to head out (or I might ping @iarna to do it)

[zkat]

There we go: I've patched in [email protected] with the exact patch that'll appear in the next release (just so I don't have to run a whole release just for this)

[jasnell]

It's not too late but it won't make it into the RC.0 build I'll be doing later today. We should try to get this landed this week. It just needs to be clear that (a) we won't push out the May 30th release date any further and (b) if this does not go stable in time it'll get pulled before the release, so this really needs eyes on it.

[zkat]

That sgtm, James! Thanks for considering this PR. We'll be working hard to make sure we have a great release ready ASAP. ✌🏼

[Fishrock123]

Getting some failures:

test/tap/git-prepare.js ............................... 3/6 3s
  install from git repo with prepare script
  not ok exited successfully
    +++ found
    --- wanted
    -0
    +1
    compare: ===
    at:
      line: 104
      column: 7
      file: test/tap/git-prepare.js
      type: global
    stack: |
      test/tap/git-prepare.js:104:7
      f (node_modules/once/once.js:25:25)
      ChildProcess.<anonymous> (test/common-tap.js:76:5)
    source: |
      t.equal(code, 0, 'exited successfully')

  install from git repo with prepare script
  not ok no actual output on stderr
    +++ found
    --- wanted
    +npm ERR! prepare-git-dep npm ERR! code ELIFECYCLE
    +npm ERR! prepare-git-dep npm ERR! errno 123
    +npm ERR! prepare-git-dep npm ERR! [email protected] prepublish: `exit 123`
    +npm ERR! prepare-git-dep npm ERR! Exit status 123
    +npm ERR! prepare-git-dep npm ERR!
    +npm ERR! prepare-git-dep npm ERR! Failed at the [email protected] prepublish script.
    +npm ERR! prepare-git-dep npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
    +npm ERR! prepare-git-dep
    +npm ERR! prepare-git-dep npm ERR! A complete log of this run can be found in:
    +npm ERR! prepare-git-dep npm ERR!     /Users/Jeremiah/Documents/node/test-npm/test/npm_cache/_logs/2017-05-09T23_14_30_875Z-debug.log
    +npm ERR! prepare-git-dep
    +npm ERR! code 123
    +npm ERR! null: npm exited with code 123 while attempting to build undefined. Clone the repository manually and run 'npm install' in it for more information.
    +
    +npm ERR! A complete log of this run can be found in:
    +npm ERR!     /Users/Jeremiah/Documents/node/test-npm/test/tap/git-prepare/cache/_logs/2017-05-09T23_14_30_920Z-debug.log
    compare: ===
    at:
      line: 105
      column: 7
      file: test/tap/git-prepare.js
      type: global
    stack: |
      test/tap/git-prepare.js:105:7
      f (node_modules/once/once.js:25:25)
      ChildProcess.<anonymous> (test/common-tap.js:76:5)
    source: |
      t.equal(stderr, '', 'no actual output on stderr')

  install from git repo with prepare script
  not ok ENOENT: no such file or directory, open '/Users/Jeremiah/Documents/node/test-npm/test/tap/git-prepare/deps/parent/node_modules/child/build-artifact'
    errno: -2
    code: ENOENT
    syscall: open
    path: >-
      /Users/Jeremiah/Documents/node/test-npm/test/tap/git-prepare/deps/parent/node_modules/child/build-artifact
    test: install from git repo with prepare script

test/tap/lifecycle-signal.js ......................... 8/10 3s
  lifecycle wait for async child process exit
  not ok expect truthy value
    at:
      line: 102
      column: 7
      file: test/tap/lifecycle-signal.js
      type: ChildProcess
    stack: |
      ChildProcess.<anonymous> (test/tap/lifecycle-signal.js:102:7)

  lifecycle force kill using multiple SIGINT signals
  not ok expect truthy value
    at:
      line: 126
      column: 7
      file: test/tap/lifecycle-signal.js
      type: ChildProcess
    stack: |
      ChildProcess.<anonymous> (test/tap/lifecycle-signal.js:126:7)

I'm also not certain if this means anything:

test/tap/spec-local-specifiers.js ................... 61/71 22s
  ~ test ambiguity for github repos
  ~ specifically test that output is valid
  ~ verify that env is as expected
  ~ to package.json and npm-shrinkwrap.json w/ drive abs
  ~ should remove the symlink
  ~ should not remove the transitive deps if it was not a `link:` type specifier.
  ~ should not remove transitive deps if it's outside the package and --preserver-symlinks isn't set
  ~ listing: should look right, not include version
  ~ outdated: show LOCAL for wanted / latest
  ~ update: if specifier exists, do nothing. otherwise as if `npm install`.

[addaleax]

@Fishrock123 Can you check whether applying npm/npm#16474 helps for some of these?

[gibfahn]

@Fishrock123 I'm only seeing the three git-prepare failures (also testing on macOS Sierra):

test/tap/git-prepare.js ............................... 3/6 2s
test/tap/lifecycle-signal.js ........................ 10/10 11s

[Fishrock123]

As a note, npm5 tests take only 540s to run, compared to 680s on npm4.

[zkat]

There's some disabled tests right now that most likely account for the difference. The biggest perf boosts in npm5, the cache/networking changes, aren't touched very much by the test suite itself (which largely uses local specifiers for tests). The biggest bottleneck in test suite performance is generally the child process spawn time, since a big chunk of it is integration tests that launch a full npm process.

Not to rain on any parades or anything 😂

[zkat]

Hey all, I've pushed beta.51 into this branch. I don't know if it'll fix the specific issues y'all were having, but we're fixing as fast as we can ^_^

[Fishrock123]

Less errors.

test/tap/00-verify-bundle-deps.js ................... 84/85
  all deps are bundled deps or dev deps
  not ok marked-man is in bundledDependencies
    at:
      line: 9
      column: 7
      file: test/tap/00-verify-bundle-deps.js
      type: global
    stack: |
      test/tap/00-verify-bundle-deps.js:9:7
      Array.forEach (native)
      Test.<anonymous> (test/tap/00-verify-bundle-deps.js:8:8)
      Object.<anonymous> (test/tap/00-verify-bundle-deps.js:7:1)
    source: |
      t.assert(

And the last two from last time (3 total) progress!

[zkat]

@Fishrock123 I'm not sure why this particular test would be failing. Have there been any significant changes in node8 as far as things like process.pid, exit signals, etc?

[zkat]

p.s. the verify-bundle-deps test should be fixed, and the git-prepare.js test from before should not be failing (since it's not there right now)

[zkat]

The beta has been updated to beta.56. The test suite passes on CI (https://travis-ci.org/npm/npm/builds/233098396) and this release includes some improvements to lockfile behavior and some fields from the spec that weren't there before.

We looked into the lifecycle-signal.js failure from before and it turns out it was only happening on node8, when run with coverage turned on. Considering we've made no changes to lifecycle signal handling with this release, the conclusion was that something fishy's going on in nyc with child processes. I've let Ben know about it and disabled the test itself in the meantime. You can run it manually with node test/need-npm5-update/lifecycle-signal.js to confirm the test passes without whatever it is nyc is doing.

[addaleax]

We looked into the lifecycle-signal.js failure from before and it turns out it was only happening on node8, when run with coverage turned on.

@zkat It does fail outside of CI on Linux boxes, even with Node 7, see npm/npm#16474 again :)

[zkat]

あ！
Thanks, Anna. We basically haven't been looking at the issue tracker for a few months now and I missed this I guess 😁

[addaleax]

@zkat No problem :) I currently get 11 failing tests when running npm test at the current release-beta-5 – I’ll try to see if I can do any about those, too.

[zkat]

@addaleax release-beta-5 isn't meant to pass tests -- it has a couple of patches intended for pushing the code up to https://npm.im/npm5 as part of the beta specifically. The beta printout itself will likely break a bunch of tests. The code I'm pushing with this release is off release-beta-5-node, which I just pushed.

[zkat]

I can push up an update to the latest beta, too. I think it's in pretty good condition right now. You can check the known issues thread and see if there's anything that scares you too much, but none of that stuff scares me enough to hold anything back. I've been keeping unstable patches out of the releases.

[gibfahn]

+1 on landing this before the next rc, that way people have a chance to test it out as early as possible.

[addaleax]

Landed in 9946173, c0d858f 🎉

[gibfahn]

Speaking of npm issues, I think this is one: npm/npm#16734 (seems to be a win-x64 issue, so if anyone can reproduce it that'd be useful, cc/ @nodejs/platform-windows ).