Can you publish a simple example on how to use this?

[firepol]

Hi, I know it's one of your newest projects, but I was wondering if you would be so kind to publish an example on how to use this, e.g. make a folder examples containing example_bitstamp_order_book.py

Something simple that prints in the command line e.g. the bid/ask price in real time.

Thanks & cheers

[ajeep8]

@nlsdfnbch , what is the Hermes? How to install it?
I "pip install hermes", it seem not your hermes.
And I find https://github.com/cossacklabs/hermes-core, it seem a cryptography framework.

[deepbrook]

@ajeep8, it's a library in the crypto toolbox project. Try installing with pip install hermes-zmq.

I'll try and add examples this week!
Thanks you two for contributing!

[firepol]

Hei, I found an interesting tutorial showing how to deal with websockets. I'm quite new to websockets and also in async programming, thus, good examples would be awesome.

https://mmquant.net/replicating-orderbooks-from-websocket-stream-with-python-and-asyncio/

This tutorial (I haven't tried it yet, but I went through it) seems to explain how to get order books for multiple pairs on the same exchange. In the tutorial bitfinex is the exchange.

It would be cool an example like this, with thoth.

Also another example, how to deal with multiple websockets with different exchanges, e.g. my use case is to compare the price of different pairs between different exchanges.

[deepbrook]

Thoth doesn't use asyncio - I havent workd with it yet. Instead, it uses ZeroMQ to handle interprocess communication. Since Thoth was designed to support multiple connections for each exchange node, this was the logical decision.

As for the examples:

Thoth is in a very early stage of its development. The current implementation is roughly equivalent to bitex's api module - thus it doesn't offer much convenience methods, but much rather acts as a proxy for the websocket connections it provides.

To understand how it works, you need to understand hermes-zmq, as it's based on its layout.

I'll give you a quick summary:
Hermes supplies 3 essential classes: Publishers, Receivers, and Nodes.

Publishers send data from an internal queue via a customizable zmq socket. Receivers receive this data and put it on an internal queue.
Nodes pull the two together - they may consist of one or more receivers and publishers, process data from the receiver and push it to the publishers.
Publishers and receivers do not necessarily need to be hermes classes - anything that implements a recv() (Receiver) or send(Publisher) can be added to a Node.

Now back to Thoth.

Currently, Thoth only implements Connectors for Pusher and Websocket apis (Receivers), and the DataNode class has been written (which is, unsurprisingly, a subclass of hermes.Node). That means, for now we receive data via the connectors and store it in the Nodes.
This data is NOT touched in a significant way - we merely extract topics, if available. Otherwise, this data is provided as is, or raw.
The next step is to provide a publisher, as the DataNode was initially planned as a standalone object - since it provides zmq sockets, which support thousands of connections, it makes little sense to initialize a DataNode per process. Instead, it should be started up once, and processes that require data from it can simply subscribe via zmq sockets.
And ultimately, I plan on adding a tiny client class that takes care of the zmq stuff as well, resulting in a simple, socket-like object that allows users to just call "recv" on it, and access the data.

Until then, you can, of course, still use the Connector classes, which are subclasses of the Thread class.

from thoth.connectors.bitfinex import BitfinexConnector
import zmq

ctx = zmq.Context()
sock = zmq.socket(zmq.PULL)

b = BitfinexConnector()
b.start()
payload = {'event': 'subscribe', 'channel': 'ticker', 'symbol': 'tBTCUSD'}
b.send(**payload)
sock.bind('/tmp/btfxwss')
while True:
    print(sock.recv())

[firepol]

Since I'm new to multi-threading and related stuff I'm watching a tutorial about it... but I was thinking about a more advanced example not just while True print something.

I mean a real world example would be:

• subscribe to the bitfinex ticker for btc-usd
• subscribe to the bitstamp ticker for btc-usd

Perform something on an event, e.g. when a channel is updated, call a method that does the following:

• save the received ticker in a variable
• parse what we need, e.g. the bid price
• compare the bid price of the ticker we just received with the other one and say which one is higher e.g. if bitstamp.bid > bitfinex.bid print('bitstamp is higher')

Or save the ticker in a db (e.g. using sql alchemy, and saving to an sql lite db), to keep it simple update all the time the same record, so to have the most up-to-date value in a db, which you can monitor separately.

Or simply update the bid/ask price in a db, to allow to create a websocket for a website like this one: https://cryptowat.ch/

If you have the values of several exchanges you can make a tutorial that shows how to create a simple page (no charts, just just update a number to keep it simple) updated in real time...

[deepbrook]

Well, you did say "simple example" ;)
The issue with a more advanced example is that it would be very specific to thoths version and the implemented exchange at this moment.
Additionally, providing order books or saving data in a database isn't in the scope of this project. Like bitex, we only provide data, but do not process it, aside from basic formatting.

[firepol]

Well, "simple" is up to interpretation ;)

One thing is to have the very simple "hello world" example that doesn't really help anybody, another would be an example that actually does something...

I mean of course I can read books, watch hours of tutorials, videos etc. (which I started doing) and maybe in the end I'll get an idea on how to use this. But if you, the core maintainer, the expert about this, provide a good non-trivial example, this can help even non-veterans to get up and started quickly.

Without a proper example doing something more useful than just print something, it's difficult to start using a library and also to understand it's full potential. I guess as you released it open source, you would be happy if people use it. If there is no time to document properly, an example with comments should be more than enough. Then of course questions will follow-up and help to document more in depth.

I'll actually follow some tutorials about websockets and how to process them, if I found something useful I'll share it here.

[deepbrook]

Sure, the project would much benefit from such examples - I do not disagree with that. However, as I have said before: the current version isn't at a point where examples could be written without having to update them constantly to prevent them from brewking. The project isn't stable as it is; if I'm being generous I'd say it's in the early stages of alpha.

So you will have to be either patient and come back some time later and see if the examples you wish for have been added, or add them yourself, should you feel like doing so.

[firepol]

Sure, sorry if I sounded too pushy, didn't want to be, but yes I'm kinda excited to try new things. Maybe I'll learn the basics of websockets etc. and when I know more I'll try to make a basic example using thoth, even if it's alpha, and probably ask you some support. No rush. Thanks again and looking forward to try it.

[ajeep8]

How https://cryptowat.ch/ gather data?

[deepbrook]

@firepol , no worries!

[deepbrook]

@ajeep8 cryptowatch isn't supported by thoth.
Should they have a websocket API, you can of course submit a feature request.

[ajeep8]

I mean how cryptowatch gather data from so many exchanges, they had a framework like thoth? of course they did not opensource.