Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak #1089

Closed
dissy123 opened this issue Aug 15, 2024 · 25 comments
Closed

Memory Leak #1089

dissy123 opened this issue Aug 15, 2024 · 25 comments

Comments

@dissy123
Copy link

Describe the bug

I am installing nuxt-booster in a npm package in a monorepository, when i do that i get a memory leak
it is difficult to share a reproduction cause of the complex setup (a lot of custom source code), maybe we can make a call so i can show it to you?

To Reproduce
Steps to reproduce the behavior:
install nuxt-booster

Screenshots
image
image
image

Desktop (please complete the following information):

  • OS: mac, linux
  • Browser chrome
  • Version latest
@ThornWalli
Copy link
Contributor

Hello @dissy123,

Before we talk on the phone ;)

Question If this only occurs during installation, it could be due to a dependency that is also installed.

{
    "@nuxt/image": "0.7.1",
    "browserslist-useragent-regexp": "4.1.1",
    "deepmerge": "4.3.1",
    "defu": "6.1.4",
    "dynamic-class-list": "2.0.2",
    "pathe": "1.1.2",
    "probe-image-size": "7.2.3",
    "serialize-to-js": "3.1.2",
    "sort-css-media-queries": "2.2.0",
    "vue-lazy-hydration": "2.0.0-beta.4"
  }

Can you perhaps recreate this behavior with one of the specified dependencies?

Basically, we only deliver code, and problems may arise with the dependencies in the monorepo.

If necessary, even switch from NPM to PNPM, where the monorepo support is better.

@dissy123
Copy link
Author

Hello @ThornWalli ,

So this is how i setup the moduls in my module.ts

image
image

I checked all the dependencies,

i uninstalled all the dependencies, like @nuxt/image and defu which i installed on my own.
So this are the dependencies from nuxt-booster, which will now only get installed via booster
image

i am using "nuxt-booster": "^3.1.3" and upgraded nuxt to 3.12.4

The thing is the memory leak only is happening when i build the project for production so with: "pnpm build"

image

When i run with pnpm dev then it is not that big

image

I also tried to test if it works when i install @nuxt/image on my own and deactivate it in nuxt-booster.
I tried all settings in nuxt-booster to switch on or off, but the memory leek only disappers if i disable nuxt-booster completly.

@dissy123
Copy link
Author

for me it looks like that on each request that is made to the site it is adding around 10 MB to the Node / BackingStore, maybe the payload is always saved for each request?

@ThornWalli
Copy link
Contributor

How big are your rendered index files?

Do you write a lot to the store during generation?

@dissy123
Copy link
Author

dissy123 commented Aug 16, 2024

yes in my store there is the whole website as a json, cause i am building a json to html renderer kind of, so there are really big json files in the store.
One other thing is that when i prerender the site the error disappears, it only happens when the page is in SSR mode.

@ThornWalli
Copy link
Contributor

Can you set optimizeSSR to false in the module settings?

https://basics.github.io/nuxt-booster/guide/options.html#optimizessr

And look again?

@dissy123
Copy link
Author

image

image

yes it it also happening with optimizeSSR: false

@dissy123
Copy link
Author

image
maybe the memory leak could happen here? i will investigate it further next week!
wish you a happy weekend!

@ThornWalli
Copy link
Contributor

ThornWalli commented Aug 16, 2024

This is just a preparation of a very small video, as base64. But this is then in the main entry.

You could reduce the module setup.

First comment on everything once and then uncomment on it step by step.

./node_modules/nuxt-booster/dist/module.mjs

const module = defineNuxtModule({
  meta: {
    name: "nuxt-booster",
    configKey: "booster",
    compatibility: {
      nuxt: "^3.0.x"
    }
  },
  defaults: getDefaultOptions(),
  async setup(moduleOptions, nuxt) {
    // const runtimeDir = resolver.resolve("./runtime");
    // nuxt.options.alias["#booster"] = runtimeDir;
    // nuxt.options.build.transpile.push(runtimeDir);
    // deprecationsNotification(moduleOptions);
    // await addModules(nuxt, moduleOptions);
    // setPublicRuntimeConfig(nuxt, moduleOptions);
    // if (moduleOptions.detection.performance && nuxt.options.ssr) {
    //   if (isWebpackBuild(nuxt)) {
    //     nuxt.hook(
    //       "webpack:config",
    //       registerAppEntry$1(
    //         resolve(nuxt.options.buildDir, MODULE_NAME, "entry")
    //       )
    //     );
    //   } else {
    //     nuxt.hook(
    //       "vite:extend",
    //       registerAppEntry(
    //         resolve(nuxt.options.buildDir, MODULE_NAME, "entry.js")
    //       )
    //     );
    //   }
    // } else {
    //   logger.warn(
    //     "Module functionality is limited without ssr and performance check"
    //   );
    // }
    // if (moduleOptions.optimizeSSR) {
    //   optimizeSSR(moduleOptions, nuxt);
    // } else {
    //   logger.warn(
    //     "Preload optimization is disabled by module option `optimizeSSR`."
    //   );
    // }
    // await addBuildTemplates(nuxt, moduleOptions);
    // addImportsDir(resolve(runtimeDir, "composables"));
  }
});

There must be a reason somewhere...

Have a nice weekend 🙂

@dissy123
Copy link
Author

It happens when importing the BoosterImage Component. When just installing the plugin, then the memory leak is also not there. But if I import the image Component it Leaks.

But when i comment this out it works ;)

image

It has todo something with the useHead!

I found a few old Articles about a Memory Leak in useHead.
nuxt/nuxt#15095
nuxt/nuxt#15093
nuxt/nuxt#13397

But they didn't really help, maybe you have an idea? =)

@ThornWalli
Copy link
Contributor

Iiiiihhhhhhh 🤮

Let's think about removing the reactive. In theory, loading the meta in the setup is enough. nothing changes in the head afterwards.

Must think about it 🙂

@dissy123
Copy link
Author

thank you! for helping me out! ;)

@dissy123
Copy link
Author

image

it is just this line, but i had to comment out the className in the return and the "this.className" in the concat below.

image

@ThornWalli
Copy link
Contributor

A. Can you set the width and height to 0 here

const { width, height } = await $booster.getImageSize(source.src);
?
And comment the method.
It actually only executes useHead() once when the meta has been set. According to console.log.

Question is if maybe the query of the image sizes causes problems.


B. Alternatively the test:

children: minify(new Source(meta.value).style)

The meta.value to meta

export function getImageStyleDescription(meta, className) {
  return {
    key: className,
    type: 'text/css',
    children: minify(new Source(meta).style)
  };
}

And replace the setup from Base.vue:


  async setup(props) {
    const $img = useImage();
    const $booster = useNuxtApp().$booster;
    const { isCritical } = useBoosterCritical();

    const resolvedCrossorigin = computed(() => {
      return getCrossorigin(props.crossorigin || $booster.crossorigin);
    });

    if (props.source) {
      const source = new Source(props.source);
      const config = $img.getSizes(source.src, {
        sizes: source.sizes,
        modifiers: source.getModifiers(),
        ...source.getOptions($booster)
      });

      useHead(() => {
        return headData.value;
      });

      let meta;

      try {
        meta = await source.getMeta(config.src, $booster);
      } catch (error) {
        console.error(error);
      }

      const headData = ref(null);

      headData.value = (({ meta, config, isCritical, resolvedCrossorigin }) => {
        if (meta) {
          return {
            style: [
              meta && getImageStyleDescription(meta, meta.className)
            ].filter(Boolean),
            link: [
              !(!config || !isCritical) &&
                (import.meta.server || process.env.prerender) &&
                new Source(source).getPreload(
                  config.srcset,
                  config.sizes,
                  resolvedCrossorigin
                )
            ].filter(Boolean),
            noscript: [
              (import.meta.server || process.env.prerender) && {
                key: 'img-nojs',
                children: `<style>img { content-visibility: unset !important; }</style>`
              }
            ].filter(Boolean)
          };
        }
        return {};
      })({
        meta,
        config,
        isCritical: isCritical.value,
        resolvedCrossorigin: resolvedCrossorigin.value
      });

      return {
        isCritical,
        config,
        meta,
        className: meta.className,
        resolvedCrossorigin
      };
    }
    return {
      isCritical,
      resolvedCrossorigin
    };
  }

Can't set the useHead after the await... that must not wait for async, then the NuxtApp won't find it...

I have a suspicion about this position:

async function getImageSize (src) {
`;
if (options.mode === 'client') {
code += ` const { width, height } = await new Promise((resolve) => {
const img = new global.Image();
img.onload = () => resolve({width: img.naturalWidth, height: img.naturalHeight});
img.src = src;
});
return {width, height};`;
} else {
code += `
const isNitroPrerender = 'x-nitro-prerender' in useRequestHeaders()
try {
// Nur im Generate!
let url = src;
if (isNitroPrerender) {
url = url.replace(useRequestURL().origin, '');
}
if (!isNitroPrerender || !(url in dimensionCache)) {
const blob = await useRequestFetch()(url);
const { imageMeta } = await import('image-meta').then(
module => module.default || module
);
const data = await fetchRetry(URL.createObjectURL(blob), undefined, 3).then(async res =>
Buffer.from(await res.arrayBuffer())
);
const dimension = await imageMeta(data);
if (isNitroPrerender) {
dimensionCache[url] = dimension;
} else {
return dimension;
}
}
return dimensionCache[url];
} catch (error) {
console.error('getImageSize: ' + src, error);
return { width: 0, height: 0 };
}
`;
}
code += `
}

@dissy123
Copy link
Author

A:

image
Better but not perfect!

image

A + B:

image

LOOKS GREAT!! :-)

@ThornWalli
Copy link
Contributor

Top problem found... 🙂

I'm preparing a customization 😉

@dissy123
Copy link
Author

thank you ! =)
finally found it after 2 weeks! hahahaha

@ThornWalli
Copy link
Contributor

Normal, everything is basically very complex 🤪

@ThornWalli
Copy link
Contributor

ThornWalli commented Aug 17, 2024

@dissy123 You could try npm i nuxt-booster@next

For now, adjust the cache when retrieving the dimensions.

@dissy123
Copy link
Author

dissy123 commented Aug 18, 2024

There where 2 imports missing in Base.vue

 useAttrs,
 markRaw

image

As it is:
image

With the meta line commented out:
image
image

With the useHead and meta line commented out:
image
image

somehow it is not good to set that meta.value property :/

@ThornWalli
Copy link
Contributor

Good morning,

  1. have you installed version npm i [email protected], the missing imports should actually be fixed there...

  2. the meta.value unfortunately has to be written. Doesn't really do anything serious, except to query the dimensions from the respective main src...
    This is necessary because we need the respective dimensions for CSS…
    The question now is whether the asnychone call that appears with getMeta can also be output in a middleware during generation. You have to think about it...

@dissy123
Copy link
Author

dissy123 commented Aug 18, 2024

yes great it works now with next.2 ;)

image

the backing store stays the same! no object duplication :-)

the rest is from another memory leak somewhere else ;)

Thanks a lot for fixing! and providing this great module! =)

@ThornWalli
Copy link
Contributor

ThornWalli commented Aug 18, 2024

@dissy123 I have added another update.

Destroy the requested image with the reuse of URL.revokeObjectURL(blob).

const objectUrl = URL.createObjectURL(blob);
const data = await fetch(objectUrl).then(async res =>
Buffer.from(await res.arrayBuffer())
);
const dimension = await imageMeta(data);
URL.revokeObjectURL(objectUrl);

@ThornWalli ThornWalli mentioned this issue Aug 18, 2024
@dissy123
Copy link
Author

wow updating to next4 removed a lot of RAM again! :-) kind of 10 mb in initial ram usage!

@ThornWalli
Copy link
Contributor

Close the issue then 🙂

Is now also in the main.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants