Memory management

[thomasnordquist]

There is the issue of quantity of topics, message history size, message parsing and preview

30.000 topics x 50kb messages => 1.5GB Ram used

Known "bugs"

• Diffing does not work with large messages. The diffing capability is limited by the message history ring-buffer size. (~20k)
• Large messages are truncated (~10k)

React Fiber

React fibers memory management is still a little bit mysterious when all topics of a 30.000 topic tree are expanded, there are about 400.000 dom nodes, reacht fiber seems to hold a reference to old nodes when they no longer exist due to React Fiber's "memoization". It is unclear when these references are cleared. When they have finally cleared it appears electron can reclaim almost all of the allocated memory.
In this extreme scenario, it is possible to run out of memory.
We maybe should prohibit actions like "expand all", when there are too many topics in a tree.
It is unlikely one could even fathom this amount of data.

Long term stability

Currently, all topics are stored, the amount of stored messages per topic is limited. However incoming topics names may be generated and may flood the tree over time.
This may cause long-term memory issues. It might be wise to forget very old data when a specific threshold is reached.

Detect and handle out-of-memory

Currently, the app would most likely crash or freeze.
We could clear the message history ring buffers and drop old topics.

[thomasnordquist]

Memo: research WeakMap and React Fiber, the history Ringbuffer may provoke that react keeps all memoized nodes since the references still exist in the ringbuffer.
Idea: Immutables? Clone instances before presenting them. The tree updates the most, "leaks" the most. (Leak in parenthesis because it is not leaking permanently)

[thomasnordquist]

I added the base framework for automated leak tests, based on existing UI-Tests.