Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why socket message processed so long? #1127

Open
vaniakurd opened this issue Jul 27, 2022 · 4 comments
Open

Why socket message processed so long? #1127

vaniakurd opened this issue Jul 27, 2022 · 4 comments

Comments

@vaniakurd
Copy link

Sokcet options: LogLevel = Microsoft.Extensions.Logging.LogLevel.Trace

i just wanna ask about message processed. I not sure but it seems to me that 3 ms for parse response is a lot. This logs from VS outpoot window.
logs.txt

@vaniakurd
Copy link
Author

Thanks for answer. I will test BinanceAPI for socket soon. Can u help me with other issue(#1145 ) with Ticker socket ?

@Hulkstance
Copy link

@vaniakurd, the data handlers are synchronous, i.e. you code inside the data handler await _socketClient.SpotStreams.SubscribeToKlineUpdatesAsync(.., dataHandler => { ... }) is called synchronously as soon as the message comes from the websocket.

@JKorf used Action<T> in opposed to Func<T, ValueTask> for the data handlers, so you should handle it yourself. I personally have in-memory representations / cached versions of the structures I need, e.g. local orderbook, local tradebook, positions, wallets, etc. and I'm keeping the state up to date which is not affected by backpressure. The immutable memory snapshots can be accessed on demand.

In plain English, you should be off-loading the data handlers and process the messages in a separate context (thread). In your case you're causing backpressure. Backpressure is when the websocket produces more messages than you could handle (consume). Think about it for a second, the websocket produces 1 message per second and your data handler takes 5 seconds to execute. Your consumer is always going to be late. If you run your application for a day, you'll end up working with old data. This is usually solved by the pull-based or push-based programming model. Refer to: https://www.infoq.com/articles/Async-Streams/. IAsyncEnumerable is Microsoft's way of trying to implement backpressure-able streams. Some people would rather use System.Reactive (Rx.NET) which is using the push programming model:

client
    .Streams
    .TradesStream
    .ObserveOn(TaskPoolScheduler.Default)
    .Subscribe(trade => { code1 });

@vaniakurd
Copy link
Author

@Hulkstance, Maybe i don't understand something, but even if i write code with empty dataHendler i will see the same situation.
SpotStreams.SubscribeToTickerUpdatesAsync( symb, msg =>{return;})

This code from CryptoExchange.Net:

protected virtual void HandleMessage( string data )
        {
            var timestamp = DateTime.UtcNow;
            ...
            var messageEvent = new MessageEvent( this, tokenData, ApiClient.Options.OutputOriginalData ? data : null, timestamp );
            var (handled, userProcessTime, subscription) = HandleData( messageEvent );
            ....
            var total = DateTime.UtcNow - timestamp;`
            _log.Write( LogLevel.Trace, $"Socket {SocketId}{( subscription == null ? "" : " subscription " + subscription!.Id )} message 
                     processed in { (int) total.TotalMilliseconds}
            ms, ({ (int) userProcessTime.TotalMilliseconds}
            ms user code)");
        }

I dont understand what work so slow, with emty dataHandler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@vaniakurd @Hulkstance and others