Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Would you recommend this for webpack-dev-server / webpack --watch? #35

Closed
hrgui opened this issue Apr 2, 2018 · 6 comments
Closed

Would you recommend this for webpack-dev-server / webpack --watch? #35

hrgui opened this issue Apr 2, 2018 · 6 comments

Comments

@hrgui
Copy link

hrgui commented Apr 2, 2018

I notice with speed-measure-webpack-plugin on, there is a memory leak over time:

Note I have about 2000 modules roughly (file is about 5 mbs)
Test i did was the following on a file:

console.log('sdsdssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss')

Memory starts at 977mb, rises all the way to 1.6 gbs after a bunch of saves (so every s above there is a save^), then gives the following error:

<--- Last few GCs --->

[36905:0x102801600]   326069 ms: Mark-sweep 1421.1 (1555.1) -> 1413.8 (1555.1) MB, 446.3 / 0.0 ms  allocation failure GC in old space requested
[36905:0x102801600]   326622 ms: Mark-sweep 1413.8 (1555.1) -> 1413.7 (1509.6) MB, 553.1 / 0.0 ms  last resort
[36905:0x102801600]   327136 ms: Mark-sweep 1413.7 (1509.6) -> 1413.7 (1499.1) MB, 513.5 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x23ce3779cca1 <JSObject>
    2: fromString(aka fromString) [buffer.js:308] [bytecode=0x10cb92da4d61 offset=202](this=0x23ce37782241 <undefined>,string=0x1860f1eefe9 <Very long string[5010042]>,encoding=0x23ce377fe311 <String[4]: utf8>)
    4: from [buffer.js:173] [bytecode=0x10cb92da4851 offset=11](this=0xc2a75ec4ae1 <JSFunction Buffer (sfi = 0x23b0d14b8981)>,value=0x1860f1eefe9 <Very long string[5010042]>,encodingOrOff...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 5: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 6: v8::String::WriteUtf8(char*, int, int*, int) const [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 7: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 8: node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
 9: node::Buffer::(anonymous namespace)::CreateFromString(v8::FunctionCallbackInfo<v8::Value> const&) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
11: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
12: v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [~/hrgui.nvm/versions/node/v8.3.0/bin/node]
13: 0x30c8e12040dd
@hrgui
Copy link
Author

hrgui commented Apr 2, 2018

Here's a snippet of code I was running in order to make webpack crash:
the counter went to 75, then it crashed.

const fs = require('fs');

function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

async function loop() {
  let i = 0;
  while(true) {
    await sleep(3000);
    console.log(i);
    fs.writeFileSync('sandbox.tpl.html', ++i);
  }
}

loop();

@stephencookdev
Copy link
Owner

stephencookdev commented Apr 2, 2018

Thanks for the code-sample, @hrgui :) If this doesn't reproduce without SMP running, then yes - there's probably a memory leak somewhere 🤔

I'll open up the inspector and try and work out where it's leaking from

But to clarify - yes, the intended usage is that this plugin should work without issue in watch mode. This is a bug. I've just never tested it over so many saves.

@stephencookdev
Copy link
Owner

Cool, so I've managed to reproduce this.

A large part of the issue is that wrappedHooks and wrappedObjs never get cleared - which means lots of old build objects are persisting between partial builds.

It should be easy enough to wipe these at the end of each partial build - and I can raise a PR for this tonight.

I just want to properly look through the memory profiler again, once those 2 arrays are cleared, to ensure there are no other possible leaks we're exposing

@stephencookdev
Copy link
Owner

So I've raised #38 to hopefully fix this.

I can't see any additional memory leaks, but if you could test it with your set-up and confirm that there isn't any other leak that would be appreciated :)

@stephencookdev
Copy link
Owner

Okay, this is now published under v1.2.2 - please re-open if you're still experiencing any issue with this!

@Cody0913
Copy link

Cody0913 commented Apr 9, 2019

Not sure if related, although I ran into a similar issue with using webpack encore using their yarn encore dev --watch which usually works with no issues, however when I wrap it with smp I only get the below issue after a file change is made.

 WAIT  Compiling...                                                                                                                  12:14:07 PM

TypeError: Cannot read property 'call' of undefined
    at ManifestPlugin.<anonymous> (/home/user/projects/drm/drm-web/node_modules/webpack-manifest-plugin/lib/plugin.js:191:53)
    at args (/home/user/projects/drm/drm-web/node_modules/speed-measure-webpack-plugin/WrappedPlugin/index.js:52:19)
    at _err1 (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:19:1)
    at func.apply.argsButLast.map.concat.callbackArgs (/home/user/projects/drm/drm-web/node_modules/speed-measure-webpack-plugin/WrappedPlugin/index.js:44:9)

    at Proxy.emit (/home/user/projects/drm/drm-web/node_modules/@symfony/webpack-encore/lib/friendly-errors/asset-output-display-plugin.js:27:9)
    at args (/home/user/projects/drm/drm-web/node_modules/speed-measure-webpack-plugin/WrappedPlugin/index.js:40:16)
    at _err0 (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:12:1)
    at func.apply.argsButLast.map.concat.callbackArgs (/home/user/projects/drm/drm-web/node_modules/speed-measure-webpack-plugin/WrappedPlugin/index.js:44:9)

    at Proxy.emit (/home/user/projects/drm/drm-web/node_modules/@symfony/webpack-encore/lib/webpack/delete-unused-entries-js-plugin.js:46:9)
    at args (/home/user/projects/drm/drm-web/node_modules/speed-measure-webpack-plugin/WrappedPlugin/index.js:40:16)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:7:1)

    at Compiler.emitAssets (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compiler.js:441:19)
    at onCompiled (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Watching.js:50:19)
    at hooks.afterCompile.callAsync.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compiler.js:630:14)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:15:1)

    at compilation.seal.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compiler.js:627:30)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)

    at AsyncSeriesHook.lazyCompileHook (/home/user/projects/drm/drm-web/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeAssets.callAsync.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1325:35)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)

    at AsyncSeriesHook.lazyCompileHook (/home/user/projects/drm/drm-web/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeChunkAssets.callAsync.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1316:32)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)

    at AsyncSeriesHook.lazyCompileHook (/home/user/projects/drm/drm-web/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.additionalAssets.callAsync.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1311:36)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)

    at AsyncSeriesHook.lazyCompileHook (/home/user/projects/drm/drm-web/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeTree.callAsync.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1307:32)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)

    at AsyncSeriesHook.lazyCompileHook (/home/user/projects/drm/drm-web/node_modules/tapable/lib/Hook.js:154:20)
    at Compilation.seal (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1244:27)
    at hooks.make.callAsync.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compiler.js:624:17)
    at _done (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:9:1)
    at _err20 (eval at create (/home/user/projects/drm/drm-web/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:260:22)
    at _addModuleChain (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1095:12)
    at processModuleDependencies.err (/home/user/projects/drm/drm-web/node_modules/webpack/lib/Compilation.js:1007:9)
error Command failed with exit code 1.

I wrapped it in the way instructed in a previous issue raised #77

I am using

Node v8.11.2
Yarn 1.13.0
Webpack Encore v0.26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants