Memory leak during exception logging

[vgrytsun]

When using logging.exception sentry does not release exception instance, which results in the memory leak. Exception could be pretty heavy, so depending on the app, it could even cause an OOM.

import logging
import weakref
import time
import sentry_sdk

weak_exception = None
class CustomException(Exception):
    pass

def callback(ref):
    print("Exception was garbage collected!")

sentry_sdk.init(<init params>)
try:
    raise CustomException('Custom Exception')
except Exception as e:
    logging.exception("Exception logging")
    weak_exception = weakref.ref(e, callback)
    sentry_sdk.flush()

while True:
    time.sleep(1)
    print(weak_exception())

In the code above, with sentry OFF, exception is garbage collected almost immediately, with sentry ON it stays around even after sentry is forced to flush all the events.

It seems like pretty severe issue, so curious if there is any workaround.

Version 2.5

[sentrivana]

Hey @vgrytsun, thanks for bringing this up. The open reference is held by the DedupeIntegration here. The integration remembers the last thing that happened so that it can filter out any duplicates coming in afterwards. It always just holds the last exception, so as soon as you raise and capture another exception, the first CustomException gets garbage collected (while the new exception is kept around instead). It shouldn't accumulate over time.

[vgrytsun]

@sentrivana thank you for a quick response! So the workaround is to disable DedupeIntegration (the only downside i can see is that each client will be reporting more events) ?
Also, i am curious whether instead of last exception, sentry should hold a reference to the last event, which should be much lighter object (it's pretty much nested dictionary of strings)

[sentrivana]

@vgrytsun You could disable the DedupeIntegration, yes, with exactly the downside you mentioned; you might be getting duplicate events.

TBH I'm not sure if the event is in general a lighter object than an exception, my gut feeling is that it's usually the other way around. What kind of exceptions are you seeing that are so big?

[vgrytsun]

Exception has a reference to traceback instance, which holds stack frames and their local variables. And those local variables could be arbitrary sizes depending on the application. So for the memory intensive app it could cause perf degradation or out of memory crashes.

[sentrivana]

We can definitely investigate whether we need to keep the whole exception or if the event or just some part of the exception would be enough. 👍🏻