Store Binance Socket data to mongoDB

SQL is fine and for larger groups of people who interact with db probably a better option as Mongo. Just once I went from SQL to NoSQL db I was hooked. For me is much less work to set proper calls and for any new data to be stored there is not bunch of preparation with like datatypes, etc., just simply import as we will see in this article.

Photo by Campaign Creators on Unsplash

First, you need to install mongo or just use the online version (free 512MB to5GB). To download it, please use the official web page. Once you made everything ready, go to python and set class to interact properly with it.

After successfully setting up mongo part is time to open socket as I described here.

From Binance we get a raw data market with letters like t, T, S… For better handling and less mistakes at my import all data are properly renamed as “Open”, “High”, “Low”, “Close”. It’s super great to have those named properly in case that you are planning to use data for candlestick charts so then OHLC data is already prepared properly. Such example is amCharts. Simple export data from mongo to .json and chart at wordpress web page is live really fast and simple as seen at bellow picture from my webpage TokensQuant.com:

So let’s insert data from the socket to Mongo with the code below and be ready for later data handling or drawing charts:

DBConnection().createTable("testBinance" + "_1mkline", sckt["data"]["s"]).update_one(
    {"_id": sckt["data"]["E"]}, {
        "$set": {"symbol": sckt["data"]["s"],
                 "Open": float(sckt["data"]["k"]["o"]),
                 "Close": float(sckt["data"]["k"]["c"]),
                 "High": float(sckt["data"]["k"]["h"]),
                 "Low": float(sckt["data"]["k"]["l"]),
                 "TotalTrades": float(sckt["data"]["k"]["n"]),
                 }}, upsert=True)

For UTC like timestamp parsing use: timestamp = datetime.utcfromtimestamp(int(ts)).strftime(‘%Y-%m-%d %H:%M’)
timestamp = parser.parse(timestamp) and change as: {“_id”: timestamp}

Cool thing with mongo is Compass as quick and easy control, modification’s can be made. For our example insert looks like as seen below:

Conclusion

When combining more feed’s and tons of data processed each second Mongo shows its strength as fast and reliable DB. Must say that am really fond of it. Am looking forward to show you how to make charts at matplotlib or even with amCharts at WordPress from data we gathered today. That’s why you are more than welcome to follow me here, just please not in real life :)!