fix(performance): reduce number of npm install instances

[imsnif]

Before this PR, when isolating environments we would run an instance of npm install once per component. When we have several components as dependencies, this could become quite costly (dozens of instances of npm running at the same time).

This process is not needed because when isolating components we do not need the devDependencies.
This reduces isolation time significantly (times here are for packages already cached by npm):

Before:

$ time bit isolate -d /tmp/my-isolated-env grommet.grommet/data-table --use-capsule -i

66.55user 6.87system 0:21.02elapsed 349%CPU (0avgtext+0avgdata 178288maxresident)k
0inputs+9056outputs (0major+858461minor)pagefaults 0swaps

After:

$ time bit isolate -d /tmp/my-isolated-env grommet.grommet/data-table --use-capsule -i

5.08user 0.58system 0:03.97elapsed 142%CPU (0avgtext+0avgdata 198936maxresident)k
0inputs+600outputs (0major+101991minor)pagefaults 0swaps

---

This change is 

[AppVeyorBot]

❌ Build bit 1.0.7032 failed (commit 0afab98779 by @imsnif)

[imsnif]

Hey @davidfirst & @GiladShoham - this is ready for review now.

I added support for components containing dependencies. This mainly involved addressing a few issues:

1. Writing the dependent components as file: links inside the root package.json of the capsule. I tried to re-use as much code as possible here. Would love to hear if there's a more straightforward way to do this. :)
2. When we persist component files to the HD, we write a version "latest" in their package.json. This version string is not a valid semver, and so npm refuses to install (both for the root component and for all of its file: dependencies. This was addressed by either persisting an existing version (if the component is not modified), using a prerelease patch version if it is (incrementing one if it exists), or using '0.0.1-0' if we don't have a version (using the latest string).
3. Changing the writing of symlinks inside the fs-capsule to be atomic. This caused issues when persisting symlinks to the capsule that have already created by npm-install. It was the easiest and quickest-win solution, since we need some of those symlinks (eg. binding files outside node_modules).
4. I added an e2e test case for this (a component with a dependency-component that has a package requirement).

Please let me know if I missed anything or if you'd like anything changed. :)

[AppVeyorBot]

❌ Build bit 1.0.7063 failed (commit 754b727b74 by @imsnif)

[davidfirst]

@imsnif , great work with the re-use! :)

Regarding # 3) I'm not sure it'll work for all cases. e.g. when configuring the dist dir to be outside the components dir, the symlinks Bit generates point to the dist directory whereas Yarn points to the source.
It's probably not relevant here because I don't see a reason to configure the capsule this way. However, I'd suggest making sure that NPM & Yarn both on the same page here, generating the same symlinks as we do. Otherwise, we should probably delete the current symlink and create ours.

[imsnif]

@davidfirst - cool, thanks for the review! I also thought about the symlinks issue, but didn't have a concrete example in mind. I don't think we have a really big IO issue here, so maybe just delete it if it's there? Do you think that's a good default behaviour?

[AppVeyorBot]

❌ Build bit 1.0.7068 failed (commit 0111079a80 by @imsnif)

[AppVeyorBot]

❌ Build bit 1.0.7074 failed (commit 6de5504bed by @davidfirst)

[davidfirst]

@imsnif , I fixed the e2e failing tests of "publishComponent unable to find the following file".
However, as you can see we've left with one failure of the test written in this PR: "new components with package dependencies (untagged)".
This test passed on my local (and probably on yours). The reason why it fails on CircleCI is due to the NPM version is using: v3.x.
Since version 5, when a package has a path value, it installs the dependencies in node_modules of that path. Before version 5, it installs them in the dependent node_module. See the tree output below I got after SSH into Circle. In this case, it installed pad-left inside bar/foo instead of utils/is-type, as a result, is-type doesn't find the pad-left.

In the past, we tried this strategy when importing components and ended up checking whether it uses NPM >= 5. This is the PR: #1358 it might help you.

Once fixed, it'd be great if you could test it on NPM < 5, NPM > 5 and Yarn. It won't be easy to write e2e-tests for all these versions :(

Tree output when Npm is v3.10.10.

├── app.js
└── components
    ├── .dependencies
    │   └── utils
    │       ├── is-string
    │       │   ├── index.js
    │       │   ├── package.json
    │       │   └── utils
    │       │       ├── is-string.js
    │       │       └── is-type.js
    │       └── is-type
    │           ├── index.js
    │           ├── package.json
    │           └── utils
    │               └── is-type.js
    └── bar
        └── foo
            ├── bar
            │   └── foo.js
            ├── node_modules
            │   ├── @bit
            │   │   ├── utils.is-string
            │   │   │   ├── package.json
            │   │   │   └── utils
            │   │   │       └── is-string.js
            │   │   └── utils.is-type
            │   │       ├── package.json
            │   │       └── utils
            │   │           └── is-type.js
            │   └── left-pad
            │       ├── .travis.yml
            │       ├── COPYING
            │       ├── README.md
            │       ├── index.d.ts
            │       ├── index.js
            │       ├── package.json
            │       ├── perf
            │       │   ├── O(n).js
            │       │   ├── es6Repeat.js
            │       │   └── perf.js
            │       └── test.js
            ├── package.json
            └── utils
                └── is-string.js

[AppVeyorBot]

❌ Build bit 1.0.7201 failed (commit 9d669e99dd by @imsnif)

[AppVeyorBot]

❌ Build bit 1.0.7203 failed (commit 9a2b7fe95b by @imsnif)

[AppVeyorBot]

❌ Build bit 1.0.7218 failed (commit 4189842b94 by @davidfirst)

[AppVeyorBot]

❌ Build bit 1.0.7227 failed (commit a2c85efb2a by @GiladShoham)

[AppVeyorBot]

❌ Build bit 1.0.7228 failed (commit 92e8ca9882 by @GiladShoham)

[AppVeyorBot]

✅ Build bit 1.0.7238 completed (commit ee99bd58c9 by @davidfirst)

[AppVeyorBot]

❌ Build bit 1.0.7240 failed (commit 3393a7bcd3 by @davidfirst)