Websocket support #57

dparlevliet · 2017-09-17T19:07:06Z

For some reason someone asked me on the nodejs package how to do websockets in python, they provided some code to get started and I was able to fix it so I thought you guys might be interested in using it dparlevliet/node.bittrex.api#24

skyl · 2017-09-25T01:17:47Z

Is there anywhere I can read up on corehub = connection.register_hub('coreHub')? Where can I get a listing of possible values for eg 'updateSummaryState', 'updateExchangeState'? Is this stuff documented?

dparlevliet · 2017-09-25T01:35:37Z

No, it's not documented yet, the Websocket API is still in development by Bittrex. There are other methods, but they require authentication and there's no current way to authenticate yet. Bittrex does this on the website when you login.

So, coreHub and those states are the only entry types available to us at this time.

skyl · 2017-09-25T01:44:49Z

The code works. I adapted it to a little class:

from requests import Session
from signalr import Connection


class WebsocketAPI(object):

    url = 'http:https://socket.bittrex.com/signalr'

    def __init__(self, market, timeout=120000):
        with Session() as session:
            self.connection = Connection(self.url, session)
        self.corehub = self.connection.register_hub('coreHub')
        self.connection.received += self.debug
        self.connection.error += self.error
        self.connection.start()
        self.corehub.server.invoke('SubscribeToExchangeDeltas', market)
        self.corehub.client.on('updateSummaryState', self.ticker)
        self.corehub.client.on('updateExchangeState', self.market)
        self.connection.wait(timeout)

    def error(self, error):
        print(error)

    def debug(self, *args, **kwargs):
        print('debug', args, kwargs)

    def ticker(self, *args, **kwargs):
        print('ticker', args, kwargs)

    def market(self, *args, **kwargs):
        print('market', args, kwargs)

Useful. Thanks for pointing this out. This is a nice firehose for me. Much appreciated.

dparlevliet · 2017-09-25T01:45:33Z

No problem, happy to help.

skyl · 2017-09-25T01:49:07Z

I think this issue should stay open; we should get support for this into this library one way or the other. Maybe @ericsomdahl wants to weigh in. This is the future though ;P

dparlevliet · 2017-09-25T02:24:19Z

Ah, my mistake. I didn't realise you weren't the maintainer.

slazarov · 2017-09-26T15:19:24Z

@dparlevliet and @skyl thank you for the code! Are you able to clarify what the different 'Type' are? E.g 'Type': 0 or 'Type': 1.

[Edit]
I was able to find the answer. Thanks to ilyacherevkov in this thread: n0mad01/node.bittrex.api#23

0 – new order entries at matching price, you need to add it to the orderbook
1 – cancelled / filled order entries at matching price, you need to delete it from the orderbook
2 – changed order entries at matching price (partial fills, cancellations), you need to edit it in the orderbook

dparlevliet · 2017-09-27T23:31:25Z

Sorry. Yep, that's correct.

tundeanderson · 2017-10-27T10:48:41Z

Hi, almost finished my implementation.... now just working on getting a snapshot of the market to then add the deltas to, for this I'm doing hub.server.invoke('QueryExchangeState', market). I get the snapshot back successfully, but it always comes back with an empty MarketName, so matching these up with the correct deltas is difficult. Has anyone else experienced this?

slazarov · 2017-10-27T21:06:31Z

Get an order book snapshot from the API and compare one of the sides (Bid/Ask) to that of the Websocket snapshot.

skyl · 2017-11-12T21:35:23Z

Is anyone else experiencing this?

HTTPError: 503 Server Error: Service Temporarily Unavailable for url: http:https://socket.bittrex.com/signalr/negotiate?connectionData=%5B%7B%22name%22%3A+%22coreHub%22%7D%5D&clientProtocol=1.5

I don't think I've changed anything. I'm thinking maybe Bittrex is under heavy load in the last couple of days and shut it down. Is it just me?

skyl · 2017-11-12T21:46:22Z

I didn't notice the connection token parameter before. This is what gets sent from the website now:

wss:https://socket.bittrex.com/signalr/connect?transport=webSockets&clientProtocol=1.5&connectionToken=28%2BotNt6hw6w%2BaNTgvve4ME4aTHBf5Mk2niqkjoyMXRA9c7KU088N8q%2F3DYcoOlNdE4W8tuqy1vR8ssLLaMQ0YKfUkCJ%2B0UsiI%2B8A6b&connectionData=%5B%7B%22name%22%3A%22corehub%22%7D%5D&tid=1

dparlevliet · 2017-11-12T21:51:48Z

I will reference it here, too, for continuity n0mad01/node.bittrex.api#67 (comment)

skyl · 2017-11-12T22:09:22Z

@dparlevliet is my hero.

So, altogether, I have working, as of right now, for ticker data for example:

class BittrexSignalr(object):

    url = 'http:https://socket.bittrex.com/signalr'

    def __init__(self):
        # important! imports go inside so that gevent can patch first. hrm.
        # https://github.com/gevent/gevent/issues/941#issuecomment-281497659
        import cfscrape
        from signalr import Connection

        with cfscrape.create_scraper() as session:
            self.connection = Connection(self.url, session)
        self.corehub = self.connection.register_hub('coreHub')

    def start(self, timeout=120000):
        self.connection.start()
        self.connection.wait(timeout)

    def listen(self):
        raise NotImplementedError()


class BittrexTickerFeed(BittrexSignalr):
    """
    Subclass and override update method to handle deltas feed.
    """

    def listen(self):
        self.corehub.client.on('updateSummaryState', self.update)

    def update(self, ticker):
        """
        print(ticker['Deltas'][0])
        {
            'MarketName': 'BTC-ADX',
            'High': 0.00026361,
            'Low': 0.00018862,
            'Volume': 4721661.42599791,
            'Last': 0.0001916,
            'BaseVolume': 1047.95908808,
            'TimeStamp': '2017-10-09T04:07:15.64',
            'Bid': 0.00018966,
            'Ask': 0.00019157,
            'OpenBuyOrders': 666,
            'OpenSellOrders': 7204,
            'PrevDay': 0.00026087,
            'Created': '2017-07-11T22:42:17.78'
        }
        """
        print(ticker['Deltas'])

Then, I subclass BittrexTickerFeed to do something real with the ticker deltas on update.

# call listen before start
myfeed = MyBittrexTickerFeed()
myfeed.listen()
myfeed.start()

p0ntsNL · 2017-11-19T13:16:40Z

python script is not working anymore as of today, something wrong after changes they've made on their website this morning.

https://twitter.com/BittrexExchange/status/932173635447431168

tundeanderson · 2017-11-19T13:26:25Z

Just checked mine, and mine still works

p0ntsNL · 2017-11-19T13:32:16Z

any chance you can leave your code here?
Because the above code, is not working (copy paste, no changes)

p0ntsNL · 2017-11-19T13:49:24Z

Did some more testing, it seems SubscribeToExchangeDeltas does work but updateSummaryState does not, do you have updateSummaryState working with python atm? I am very interested in this please

tundeanderson · 2017-11-19T15:33:42Z

Ah, I'm not using updateSummaryState, only updateExchangeState, SubscribeToExchangeDeltas and QueryExchangeState. Seems like they might have changed updateSummaryState then :(

p0ntsNL · 2017-11-19T17:31:48Z

yeah thought so :( it is not working anymore damnit.

Anyone found a solution yet?

skyl · 2017-11-19T19:22:25Z

Hrm. Yeah, confirmed that I'm not getting any data anymore. The connection seems to happen just fine; but, self.corehub.client.on('updateSummaryState', self.update) event never fires :/

skyl · 2017-11-19T19:26:25Z

Paging Dr. @dparlevliet ... any idea where 'updateSummaryState' got off to?

p0ntsNL · 2017-11-19T21:18:39Z

Ty so much for confirming skyl, I have been looking for around 3 hours but I can not find anything. Lets hope dparlevliet has an idea.

developerwok · 2017-11-20T05:33:28Z

the last time i received something from updateSummaryState seems to be around "2017-11-19T07:38:35.93Z"

p0ntsNL · 2017-11-20T07:12:31Z

Yeah that is exactly when mine stopped as well, also this might be related to the Bittrex tweet around that time.

https://twitter.com/BittrexExchange/status/932173635447431168

So I guess they've changed the endpoint to something else, because there must be something that they use themselfs to get volume, high, low etc etc

p0ntsNL · 2017-11-20T10:43:58Z

taken from the refered post:

Thanks. It seems Bittrex killed "updateSummaryState" 👎
#57

Edit: looking at the source log from the referenced issue, "updateSummaryState" works if you're using "https://socket-stage.bittrex.com" as socket feed Uri.

To mimic what this does in python, probably you need to make a simple HTTP get request to "https://www.bittrex.com". Get the returned headers and cookies from that request and open the feed to the "socket-stage" uri, using those headers / cookies. By simple I mean using the cloudflare package, of course.

p0ntsNL · 2017-11-20T11:16:00Z

anybody able to translate this to python:

https://github.com/tompetko/SignalR.WebSocketSharpTransport/blob/master/SignalR.WebSocketSharpTransport.Demo/Program.cs

check from line 270

p0ntsNL · 2017-11-20T17:48:45Z

no one yet?

slazarov · 2017-11-24T15:02:26Z

@p0nt I've created a basic websocket client. You can check it here python-bittrex-websocket.
@ericsomdahl, perhaps we can merge them at certain point.

nanvel · 2017-11-24T17:21:27Z

import asyncio
import json
import time

import aiohttp


SOCKET_URL = 'https://socket.bittrex.com/signalr/'
SOCKET_HUB = 'corehub'


async def socket(tickers):
    """
    Uses signalr protocol: https://github.com/TargetProcess/signalr-client-py
    https://github.com/slazarov/python-bittrex-websocket/blob/master/bittrex_websocket/websocket_client.py
    """
    conn_data = json.dumps([{'name': SOCKET_HUB}])
    async with aiohttp.ClientSession() as session:
        url = SOCKET_URL + 'negotiate' + '?' + urlencode({
            'clientProtocol': '1.5',
            'connectionData': conn_data,
            '_': round(time.time() * 1000)
        })
        async with session.get(url) as r:
            socket_conf = await r.json()

        socket_url = SOCKET_URL.replace('https', 'wss') + 'connect' + '?' + urlencode({
            'transport': 'webSockets',
            'clientProtocol': socket_conf['ProtocolVersion'],
            'connectionToken': socket_conf['ConnectionToken'],
            'connectionData': conn_data,
            'tid': 3
        })
        async with session.ws_connect(socket_url) as ws:
            for n, ticker in enumerate(tickers, start=1):
                message = {
                    'H': SOCKET_HUB,
                    'M': 'SubscribeToExchangeDeltas',
                    'A': [ticker],
                    'I': n
                }
                await ws.send_str(json.dumps(message))
            async for msg in ws:
                print(msg.type, msg.data)


if __name__ == '__main__':
    ioloop = asyncio.get_event_loop()
    ioloop.run_until_complete(socket(['BTC-ETH', 'BTC-NEO']))

skyl · 2017-11-24T19:47:27Z

Is there any updateSummaryState equivalent?

slazarov · 2017-11-24T21:52:59Z

@skyl can you test updateSummaryState with this code:
https://github.com/slazarov/python-bittrex-websocket/blob/master/bittrex_websocket/summary_state.py

skyl · 2017-11-25T06:56:23Z

@slazarov works for me.

p0ntsNL · 2017-11-29T17:50:39Z

awesome man looks good, so this means you can not use "updateSummaryState" anymore without mentioning tickers, which previously just updated all bittrex tickets instead of chosen one?

slazarov · 2017-11-29T17:54:14Z

@p0nt I was testing it yesterday. Trying to invoke 'SubscribeToSummaryDeltas' with the specific tickers results in an error. In another words, you will receive the summary state deltas for all the updated tickers on the exchange. Bittrex seems to constantly change it.

p0ntsNL · 2017-12-03T05:51:31Z

Ah thats great! I am looking for that to just get all tickers summary info instead of just the one I subscribe to.

Thank yuu!

hippylover · 2017-12-08T04:58:25Z

For QueryExchangeState, is the solution really to get the orderbook from rest api and compare the colums?

I suppose i need to do a QueryExchangeState and not just a rest call in order to get the right nonces?

A weird thing too is that when i do a QueryExchangeState i also receive updateExchangeState stuff as if i had called it...(i guess i don't have to call it then, if i can get queryexchangestate to start telling me anyway and i can only figure out which market it is...)

hippylover · 2017-12-08T05:22:13Z

Also this was the only way i managed to be notified of what QueryExchangeState spits out:

connection.received += getOrderBooks
#corehub.client.on('queryExchangeState', getOrderBooks) this doesnt work

Any ideas?

slazarov · 2017-12-08T07:54:15Z

@hippylover there are two solutions. Firstly, you can get the order book from the REST API and compare it. Secondly, you can track the QueryExchangeState invokes and know which order book snapshot to expect next. Both have pros and cons.

REST API

Pro: for larger ticker subscriptions you will sync the nounces faster because you can request asynchronously
Con: you are using the rest API, so it's not a native method

Websocket Invoke

Pro: you are doing it natively
Con: the invokes are FIFO and that's how you have to sync them. You must make sure that your order nounces match or are lower than the order book snapshot nounce for the sync to work.

In any case, if you scroll up you would see that I have posted a link to a websocket library which you can test.

@ericsomdahl I don't want to hijack the thread by referring people to my library, not my point really. My idea is to get people to test the library which is solely written for that, iron out bugs/issues and combine at some point.

ericsomdahl · 2017-12-08T12:44:31Z

@slazarov no worries. I hope for your lib to merge here eventually... Or at least become the "officially recognized" python implementation for websocket support, depending on your preference.

skyl · 2017-12-09T19:59:33Z

@slazarov this code was working for a while but doesn't work anymore:

from bittrex_websocket.websocket_client import BittrexSocket


class BittrexTickerSender(BittrexSocket):
    exchange = Bittrex()

    def on_debug(self, **kwargs):
        pass

    def on_open(self):
        self.client_callbacks = ['updateSummaryState']

    def on_message(self, ticker):
        # Bittrex keeps breaking me :/
        print('UPDATe', ticker)

# ....
# ....
if __name__ == `__main__`:
        bts = BittrexTickerSender()
        bts.run()
        while True:
            print('ok..')
            time.sleep(5)

I'm guess Bittrex changed something again? When I print/log in on_debug, I only see 'updateExchangeState' and no updateSummaryState

slazarov · 2017-12-09T20:21:06Z

@skyl I have released v0.0.2, please check the repo, reinstall through pip, read the new documentation and tell me if you are experiencing the issue.

skyl · 2017-12-10T21:43:51Z

@slazarov I have something working again. I'm using master / 0.0.3. It seems like if I try to subscribe to all the markets I get a lot of:

Traceback (most recent call last):
  File "/.../env/lib/python3.6/site-packages/gevent/hub.py", line 863, in switch
    assert getcurrent() is self.hub, "Can only use Waiter.switch method from the Hub greenlet"
AssertionError: Can only use Waiter.switch method from the Hub greenlet
Sun Dec 10 21:40:05 2017 <io at 0x110d1e7b8 fd=30 events=READ active callback=<bound method Waiter.switch of <gevent.hub.Waiter object at 0x110d40ea0>> args=(<object object at 0x10e8adc60>,)> failed with AssertionError

Do you know what this is referring to?

If I just subscribe to a couple of markets this doesn't seem to happen ... I guess we could take this conversation up in your repo ...

slazarov · 2017-12-10T22:02:20Z

@skyl Move the issue to the correct repo and provide the code you are using so I could replicate it (it's important to know which subscribe method you are using). The error might be caused by a lot of factors, conflicts with gevent or maybe something outdate within the SignalR library which uses gevent.

halcyonjuly7 · 2017-12-20T09:29:44Z

@nanvel have you gotten your async code to work still? it seems that right now the DDOS protection page is screwing it up.

nanvel · 2017-12-20T09:39:06Z

@halcyonjuly7 it was working before. I don't use this API anymore. DDOS protection page should have no relation to the API IMHO.

nanvel · 2017-12-20T09:43:11Z

@halcyonjuly7 I see "503 Service Temporarily Unavailable" in response. So, probably, will be available soon.

halcyonjuly7 · 2017-12-20T10:00:56Z

@nanvel one would think that. but using the code you provided a month ago.

you called await r.json() it was throwing an error because it wasn't returning json anymore so I checked the text instead await r.text() and I got the html saying "blah blah this happens automatically please wait up to 5 seconds"

so what endpoint are you using now?

nanvel · 2017-12-20T12:33:38Z

@halcyonjuly7 I see the same page "Checking your browser before accessing bittrex.com." now.
Have no good idea how to deal with it. Worst case you can try selenium, but there must be better solutions. I don't use the socket, only these: https://bittrex.com/home/api

skyl · 2017-12-21T19:53:12Z

@slazarov Have you found a workaround for this? I'm now seeing:

DEBUG:requests.packages.urllib3.connectionpool:https://socket-stage.bittrex.com:443 "GET /signalr/negotiate?connectionData=%5B%7B%22name%22%3A+%22coreHub%22%7D%5D&clientProtocol=1.5 HTTP/1.1" 503 None

We had something going with cfscrape before but that got broken too ...

slazarov · 2017-12-23T20:27:09Z

@skyl it should be fixed now. Update to v0.0.5.1
Issue: slazarov/python-bittrex-websocket#12

zaycker · 2017-12-23T22:10:20Z

Guys I used ws feature with https://github.com/zolzolwik/node.bittrex.api for a few days and there was a moment when it was died :( And they use cloudscraper to bypass cloudflare protection and it didn't help. I don't remember http status code but I made a decision that good scheduler + get_market_summaries more reliable than ws. Or you have to use it as a fallback to hedge your bets :)

panamantis · 2018-01-17T11:03:00Z

I've confirmed that a websocket connection can be make to an authenticated/private call (ie. chat.server.invoke('QueryBalanceState'))
Transfer cookies + agent name from ie/ logged in selenium session -> cfscrape sesh. I'll elaborate later.

Though making this headless is a bit of a challenge re: recaptcha + google authenticator. Cookies seem to last 2-3 hours and could possibly be refreshed though.

MoroZvlg · 2018-02-27T17:16:12Z

Sorry, i don't really understand what should i do with connection.wait() if i want listen sockets forever. Can anybody explain?

MohammedAiyub · 2020-07-22T06:13:15Z

I didn't notice the connection token parameter before. This is what gets sent from the website now:

wss:https://socket.bittrex.com/signalr/connect?transport=webSockets&clientProtocol=1.5&connectionToken=28%2BotNt6hw6w%2BaNTgvve4ME4aTHBf5Mk2niqkjoyMXRA9c7KU088N8q%2F3DYcoOlNdE4W8tuqy1vR8ssLLaMQ0YKfUkCJ%2B0UsiI%2B8A6b&connectionData=%5B%7B%22name%22%3A%22corehub%22%7D%5D&tid=1

Hi, can you please how to get the connectionToken?
your above link support V3 version? if not can you please give the url for V3?

dparlevliet closed this as completed Sep 25, 2017

dparlevliet reopened this Sep 25, 2017

dparlevliet mentioned this issue Oct 20, 2017

Making bittrex websocket api work in python dparlevliet/node.bittrex.api#24

Closed

skyl mentioned this issue Nov 12, 2017

Error 503 trying to get candles from Bittrex dparlevliet/node.bittrex.api#61

Open

p0ntsNL mentioned this issue Nov 14, 2017

bittrex websocket BitcoinExchangeFH/BitcoinExchangeFH#90

Closed

Websocket support #57

Websocket support #57

Comments

dparlevliet commented Sep 17, 2017

skyl commented Sep 25, 2017

dparlevliet commented Sep 25, 2017

skyl commented Sep 25, 2017 • edited Loading

dparlevliet commented Sep 25, 2017

skyl commented Sep 25, 2017

dparlevliet commented Sep 25, 2017

slazarov commented Sep 26, 2017 • edited Loading

dparlevliet commented Sep 27, 2017

tundeanderson commented Oct 27, 2017

slazarov commented Oct 27, 2017

skyl commented Nov 12, 2017

skyl commented Nov 12, 2017

dparlevliet commented Nov 12, 2017

skyl commented Nov 12, 2017 • edited Loading

p0ntsNL commented Nov 19, 2017

tundeanderson commented Nov 19, 2017

p0ntsNL commented Nov 19, 2017 • edited Loading

p0ntsNL commented Nov 19, 2017 • edited Loading

tundeanderson commented Nov 19, 2017

p0ntsNL commented Nov 19, 2017

skyl commented Nov 19, 2017

skyl commented Nov 19, 2017

p0ntsNL commented Nov 19, 2017

developerwok commented Nov 20, 2017

p0ntsNL commented Nov 20, 2017

p0ntsNL commented Nov 20, 2017 • edited Loading

p0ntsNL commented Nov 20, 2017

p0ntsNL commented Nov 20, 2017

slazarov commented Nov 24, 2017

nanvel commented Nov 24, 2017 • edited Loading

skyl commented Nov 24, 2017

slazarov commented Nov 24, 2017

skyl commented Nov 25, 2017

p0ntsNL commented Nov 29, 2017

slazarov commented Nov 29, 2017

p0ntsNL commented Dec 3, 2017

hippylover commented Dec 8, 2017 • edited Loading

hippylover commented Dec 8, 2017

slazarov commented Dec 8, 2017

ericsomdahl commented Dec 8, 2017

skyl commented Dec 9, 2017 • edited Loading

slazarov commented Dec 9, 2017

skyl commented Dec 10, 2017

slazarov commented Dec 10, 2017

halcyonjuly7 commented Dec 20, 2017 • edited Loading

nanvel commented Dec 20, 2017

nanvel commented Dec 20, 2017

halcyonjuly7 commented Dec 20, 2017

nanvel commented Dec 20, 2017

skyl commented Dec 21, 2017

slazarov commented Dec 23, 2017 • edited Loading

zaycker commented Dec 23, 2017 • edited Loading

panamantis commented Jan 17, 2018

MoroZvlg commented Feb 27, 2018

MohammedAiyub commented Jul 22, 2020

skyl commented Sep 25, 2017 •

edited

Loading

slazarov commented Sep 26, 2017 •

edited

Loading

skyl commented Nov 12, 2017 •

edited

Loading

p0ntsNL commented Nov 19, 2017 •

edited

Loading

p0ntsNL commented Nov 19, 2017 •

edited

Loading

p0ntsNL commented Nov 20, 2017 •

edited

Loading

nanvel commented Nov 24, 2017 •

edited

Loading

hippylover commented Dec 8, 2017 •

edited

Loading

skyl commented Dec 9, 2017 •

edited

Loading

halcyonjuly7 commented Dec 20, 2017 •

edited

Loading

slazarov commented Dec 23, 2017 •

edited

Loading

zaycker commented Dec 23, 2017 •

edited

Loading