vrview - Runaway GPU memory leak when displaying a sequence of VR Images

[garyjamessilva]

Hi there,

I am encountering a runaway GPU memory leak when displaying a sequence of VR Images in Chrome Browser.

VR Image sequence ( 4096x2048 pixels per image ) is advanced periodically using vrView.setContent() within a single vrview window.

Using Chrome browser Version 57.0.2987.110 (64-bit) on Apple Mac.

Chrome Task Manager, shows GPU is using 2.7GB allocated after displaying (only) 46 VR images, GPU memory is only growing not releasing until system is Out Of Memory.

Please investigate & fix :-)

Thanks

[garyjamessilva]

[fix2015]

same pb, do you have any idea how it can be fixed?

[fix2015]

maybe you have some methold clear all event and buffer, or maybe some to do re-init

[fix2015]

try this one #142

[garyjamessilva]

Hi fix2015,

Tried your patches but still having same issue -

so not resolved.

GPU still using 4.2GB memory after display 120+ VR images.

Thanks for trying...

[fix2015]

can you show your code, i will try maybe will fix

[fix2015]

we can put in render this code and it will work i hope, but need to see your code

[garyjamessilva]

Hi there,

The VR View code I used was from your own link where you said 'try this one #142', that is I specifically used all of the code from this revision when I retested -
vrview-b4977952dddd49c7e48b7088e26394fdcdbcbaf7.zip

My own javascript code for driving my VR Image 'slideshow' is generated dynamically by my server ( in the context of what folder a user is currently looking at ) following is an example -

InJeCtEd.js.txt

Thanks for your support.

[fix2015]

try this one 9c696ba it should help

[garyjamessilva]

Hi again,
Ok thanks, I also tried - 'try this one 9c696ba it should help' - but that was no good at all (!), on loading a single VR Image many many javascript console log messages are immediately emitted, the defined HotSpots are not showing at all and the Google Chrome Helper App Process immediately rose to using 18GB memory (!) -

and

What did you Test with your code fix ???

[fix2015]

i tested vrview example, and it was good, so it can be not pb in vrview it can be pb with your code, so can you create repository and i will try on my comp

[garyjamessilva]

Hi,

Yes the issue might be in my own javascript code but, I did not change my own code between testing of -

vrview-9c696ba1c076ed98d703570204681954a7237d1e.zip

and

vrview-b4977952dddd49c7e48b7088e26394fdcdbcbaf7.zip

I used the following zip file based on - 'try this one 9c696ba it should help' -

vrview.zip

• for the last tests I did, that did not work as mentioned ( when Chrome Helper was using 18GB memory )

When I reverted back to the old version from vrview -b4977952dddd49c7e48b7088e26394fdcdbcbaf7.zip - my code was running as before, HotSpots showing ok, but GPU memory gradually rising and not be deallocated, as per the original issue I reported (#135)

Thanks,

[fix2015]

how i can run your app?

[garyjamessilva]

Hi,

Hmm, not easily, the vrview.zip file I posted in the last message is served by an embedded HTTP Web server that is hosted inside an Android mobile app I write. Even if I give you a copy of the Android app you'd still need to generate or import some VR images using the app before you can then use Chrome Browser running in a PC to access & view the VR images in a VR Web View... I will probably need to work out how to get the vrview.zip file working in a more suitable / generic Web server container like Tomcat.

Alternatively, to reproduce the issue, just cut / paste & edit some of the javascript code from the InJeCtEd.js.txt file posted in a previous message (above) into a HTML file, then provide and reference some suitable VR JPG Images and run it as you would with you own tests - similar to https://googlevr.github.io/vrview/examples/hotspots/index.html - but use a timer to switch between the images ( using vrView.setContent(..) ) in an endless loop, this is when you start to see the GPU memory issue.

I am suspicious that my 'sldeshow' code in InJeCtEd.js.txt may be causing the memory leak by using a timer, see - timer = setInterval( function() { slideShow(); }, sleep ); - so references to memory are held & not being freed by each call to vrView.setContent(..), I should perhaps try using 'promises' instead of a timer eg: - https://developers.google.com/web/fundamentals/getting-started/primers/promises in the hope that the memory does gets freed... But I see the issue as more likely being with vrView.setContent(..) not freeing it own memory each time a new image content is loaded & set.

The GPU memory issue is not readily apparent until you have displayed 50 or more images using vrView.setContent(..), this happens regardless of whether you cycle through the same small set of images eg 3 or 4 images or lots of different images, I need to be able to display hundreds of VR images or more and not have the user run out of memory...

(I am on vacation for the next 2 weeks, so can't work on this until I return.)

[mandelbox]

I have the same issue. 9c696ba does fix the memory leak however there appear to be a few new bugs introduced in this version.

I create the texture I want to use in the vrview in a canvas. In order to load these textures into the VR view I create a data url with the canvas contents and load that using setContent. This works well using a build from the primary tree master branch (aside from the memory leak, but this occurs in the primary branch even if I load regular images from my server) but does not work with 9c696ba. Is there a better way for me to load data from a canvas?

I also have an issue with 9c696ba when using non power of two textures. Using such a textures causes a loop where the texture is reloaded endlessly.

[NPTPolynomial]

I have the same issue on both my laptop (Mac + safari), and my iPhone6S Plus (Safari and Chrome). On the phone, it overloads the browser (both Safari and Chrome) and forces the browser to close on its own after navigation 4 - 6 hotspots.

On the laptop, I can see through the Activity monitor ram usage spike up significantly as I go from one hotspot to another.

[NPTPolynomial]

I made a gif of the problem. It is probably too large to upload (sorry, if it takes too long to see it. 4mb).

I made a small sample of the project in my blog. Open your Activity Monitor and monitor your ram as you go back and forth between the hotspots... maybe 10-15 times to effectively see the memory leak. On mobile, maybe 4-5 times will cause your browser to reload.

https://aidanco.com/vrview-master/examples/hotspots_insikt/

[tommytee]

The memory leak can be fixed by not creating the spheres more than once. Instead dispose and recreate the geometry and material of each.
https://github.com/tommytee/vrview/blob/gaze/src/embed/sphere-renderer.js#L177

This branch (named gaze) also has fixes to make "gaze to click" work. Explained here: #145

Modules are updated including three.js r85

This repo also a branch named testSwitch which has these edits plus a testing function.

[tommytee]

Here is demo of the testSwitch branch: https://vrview.io/branch/test-switch/examples/hotspots/index.html

the switch on the top left will continuously load content

[garyjamessilva]

I tried the branch name 'gaze', it seems to partly fix the memory leak when displaying multiple images, but not when images are internally resized due to not being a power of 2 -
THREE.WebGLRenderer: image is not power of two (5660x2830). Resized to 4096x2048

• when this image resizing happens memory is not being released so same memory leak error still persists.

'testSwitch' works ok, memory does not leak as images are already power of 2 so are not resized. You could test this further by changing one of the sample images eg: dolphins.jpg so that it is not power of 2, eg resize to 5000 x 5000 pixels and run your test again.

Thanks for your help.

[tommytee]

I resized the images to 5000 x 5000 and i get the memory leak. Its not as bad but still there.

It appears that the makePowerOfTwo function is causing this. https://github.com/mrdoob/three.js/blob/a4922d637762f2abf0ac7282b2bea8538bf1ee3e/src/renderers/webgl/WebGLTextures.js#L46

This may relate: mrdoob/three.js#11378

Can you resize your images in advance? That would be the best option, if possible.

[garyjamessilva]

Yes, I will resize images so they are power of 2, this has long been required/recommended with OpenGL, better to do it once beforehand anyway rather than every time when rendering.
Thanks for confirming the ongoing memory leak, seems deallocation/garbage collection is not managed very well in Javascript ?

[tommytee]

Quick fix for the three.js bug:

add var POTCanvas line here:

https://github.com/googlevr/vrview/blob/master/build/three.js#L17521

Change the makePowerOfTwo function to reuse the canvas ( a few lines down )

https://github.com/googlevr/vrview/blob/master/build/three.js#L17561

function makePowerOfTwo( image ) {

      if ( image instanceof HTMLImageElement || image instanceof HTMLCanvasElement ) {

	  if ( ! POTCanvas )
		POTCanvas = document.createElementNS( 'https://www.w3.org/1999/xhtml', 'canvas' );

	  POTCanvas.width = _Math.nearestPowerOfTwo( image.width );
	  POTCanvas.height = _Math.nearestPowerOfTwo( image.height );

	  var context = POTCanvas.getContext( '2d' );
	  context.drawImage( image, 0, 0, POTCanvas.width, POTCanvas.height );

	  console.warn( 'THREE.WebGLRenderer: image is not power of two (' + image.width + 'x' + image.height + '). Resized to ' + POTCanvas.width + 'x' + POTCanvas.height, image );

	  return POTCanvas;

	}

	return image;

}

[tommytee]

three.js fork with the fix https://github.com/tommytee/three.js/blob/POTCanvas/src/renderers/webgl/WebGLTextures.js

branch is named POTCanvas

[soulreaver156]

Hello,

I appear to be having a very similar problem where the memory increases with each hotspot click, resulting in an eventual crash. With your sphere-renderer.js fix, Do I simply go into the vrview>src>embed>sphere-renderer.js folder and copy and paste the new code above into that file? I looks like making changes to the src folder isn't affecting the experience. Is there something that needs to be done in the build folder as well?

Thank you!

[lincolnfrog]

@soulreaver156 you need to make the change to sphere-renderer.js and then you need to build the project in order to get the changes into the embed.js or embed.min.js file.

Check out the README.md for instructions. I believe the command is:
$ npm run build

[soulreaver156]

@lincolnfrog Thank you - that did the trick!