Live: Source cleanup to free memory for multiple streams.

[szhuangzan]

SrsSource* source = SrsSource::fetch(req);
if (!source) {
    if ((ret = SrsSource::create(req, server, server, &source)) != ERROR_SUCCESS) {
        return ret;
    }
}

If a server is continuously requested with rtmp:https://xxx/live/(1-100000), the number of objects keeps increasing, and no release logic is found.

[winlinvip]

That's right, this overhead can be calculated.

Note: There is a workaround, Gracefully Quit, which can be considered as a more mature solution.

[willkk]

Same question, if SRS keeps running, the number of sources will keep increasing, when will they be released?

[winlinvip]

9900 streams only occupy 65MB of memory.

Note: There is a workaround, Gracefully Quit, which can be considered as a more mature solution.

[winlinvip]

The situation of Source cleanup is as follows:

• In SRS2, an attempt was made to clean up the Source (reference: c7b97aa1c3), but there were too many problems, so this feature was later disabled: srs occasional crash caused by cleaning up unused SrsSource mechanism #713, SOURCE: The SRS server crashed after stopping the stream. #671
• When supporting K8S, SRS supports Gracefully Quit, which can safely exit when there is no traffic and then be pulled up by the watchdog (there are related Linux solutions). This solution can temporarily avoid this problem.

Therefore, I still need to restart this Issue to support a more comprehensive Source cleanup capability.

[winlinvip]

Source cleanup will be resolved in SRS 5.0, mainly in monitoring and RTC scenarios, where there will be a large number of streams (Sources).

Since the Source pointer is widely referenced, after a long period of consideration, the best solution is to refer to C++11's smart pointers to avoid crashes after cleanup.

Note: There is a workaround, Gracefully Quit, which can be considered as a more mature solution.

[winlinvip]

The problem of Source cleanup is essentially due to the direct reference to Source, which makes it impossible to count and accurately clean up. A better approach is to make Source an internal type and expose a wrapped type externally. Each time it is used, a new wrapped type is created, similar to a smart pointer.

void* fetch_source(url) // The fetched void* avoids being converted to Source.

Add a wrapped type:

WrapSource WrapSource::wrap(void*) // Provides Source functions.

Also avoid creating pointers, always use instances, so you can accurately know how many references there are.

Then, combined with a simple GC for delayed release, this can most simply solve the Source cleanup problem.

It's a bit like a smart pointer, but much simpler, relying on specifications and type hiding to implement. It will be a bit more complicated, but overall it's quite simple.

This can also be used to solve other objects that need to be cleaned up but are widely referenced.

[winlinvip]

Pushing a stream every 5 seconds, the memory will increase to 4.2GB in about three days. In scenarios with a large number of push-pull streams, such as stress testing, monitoring, and calling, this problem is quite serious.

Some optimizations are planned for 5.0, without deleting Source, to do some cleanup and slow down the memory growth. The plan is to delete Source in 6.0 and completely solve this problem.

[winlinvip]

Synchronize the progress: It is confirmed that WebRTC's own implementation of ShraredPtr and WeakPtr smart pointers will be referenced, but a simpler version will be implemented and used only in Source objects.

It will be done in two steps to avoid affecting stability: it will be implemented and partially cleaned up in 5.0, and it is expected to be completely resolved in 6.0.

[jinleileiking]

Another approach is to refactor with Rust, but the implementation might be challenging.