Using Historical Data Websocket for Real-Time OHLC Data

[jackroc97]

I am trying to stream 1 min OHLC data in real-time. I've tried using the market history example as a starting point for this. I am a little confused on how to interpret the data. The first response is OHLC for the 1 min bar for the previous minute (for example, if I run the script at 17:06, it would be the bar for 17:05). A sample response is shown below:

QueueAccessor(key=IbkrWsKey.MARKET_HISTORY, size=1) {'serverId': '588547', 'symbol': 'AAPL', 'text': 'APPLE INC', 'priceFactor': 100, 'startTime': '20241223-17:05:00', 'high': '25495/56.04/1', 'low': '25482/245.49/0', 'timePeriod': '60s', 'barLength': 60, 'mdAvailability': 'S', 'mktDataDelay': 0, 'outsideRth': False, 'volumeFactor': 100, 'priceDisplayRule': 1, 'priceDisplayValue': '2', 'negativeCapable': False, 'messageVersion': 2, 'data': [{'o': 254.91, 'c': 254.91, 'h': 254.94, 'l': 254.82}, {'o': 254.9, 'c': 254.9, 'h': 254.95, 'l': 254.9}], 'points': 1, 'travelTime': 9, 'topic': 'smh+265598', 'payload': '{"period": "1min", "bar": "1min", "outsideRTH": true, "source": "trades", "format": "%o/%c/%h/%l"}'}

The subsequent responses however only contain either just "open" or "open" and "close" data, with no time associated. For example:

QueueAccessor(key=IbkrWsKey.MARKET_HISTORY, size=2) {'serverId': '588547', 'tickNum': 36, 'negativeCapable': False, 'data': [{'o': 254.8}], 'points': 0, 'travelTime': 44668, 'topic': 'smh+265598', 'payload': '{"period": "1min", "bar": "1min", "outsideRTH": true, "source": "trades", "format": "%o/%c/%h/%l"}'}

Is it possible to use this websocket endpoint to get a real-time stream of OHLC bars, or would that be a misuse of it? It appears to return data in realtime, but seems to be incomplete... or perhaps I am misinterpreting the response.

[Voyz]

hey @jackroc97 That's right the Market History is not quite what you'd expect it to be. In fact, the documentation states:

https://www.interactivebrokers.com/campus/ibkr-api-page/cpapi-v1/#ws-sub-historical-data

NOTE: Historical data will only respond once, though customers will still need to unsubscribe from the endpoint.

---

To get live data, you want to use the MARKET_DATA channel as shown in ws_02 example:

https://github.com/Voyz/ibind/blob/master/examples/ws_02_intermediate.py

You can find the list of fields you may want to use here:

https://github.com/Voyz/ibind/blob/master/ibind/client/ibkr_definitions.py

Or here:

https://www.interactivebrokers.com/campus/ibkr-api-page/cpapi-v1/#market-data-fields

Let me know if it makes sense 👍

[jackroc97]

Thanks for the response @Voyz! So it seems that there is no "native" way to stream live OHLC values through the Web API, similar to how you can using the TWS API? I had previously been using the TWS API, but I have been frustrated with it recently and decided to try switching over to the Web API.

My goal is to construct a live feed of OHLCV candles on the minute timescale. My initial approach was to use the MARKET_DATA channel as you mentioned, but I had some confusion about that as well. According to IBKR's own documentation (and the documentation you posted), the Open, High, Low, Close, and Volume fields in MARKET_DATA are all on the daily time-frame. It seems like it would be possible to construct an OHLC series using a MARKET_DATA subscription that reportslast_price andlast_size. However, when I create a subscription for that, some of the responses have either last_price or last_size as empty. For example:

_updated, last_price, last_size
1734971603246,254.68,200
1734971604747,254.65,
1734971606247,,300
1734971651976,,
1734971652476,254.70,100
1734971653977,254.69,

I'm not really sure what to do with the data that is missing one of the two fields. How can there be a price with no size, or a size with no price?

[Voyz]

Understood, thanks for pointing out that MARKET_DATA's ohlc are daily, not live.

Did you try talking to IBKR support and see what they say on the topic?

I'm not really sure what to do with the data that is missing one of the two fields. How can there be a price with no size, or a size with no price?

I've seen this before. IBKR support told me that these are incomplete data points and that I can collect or ignore these until a full datapoint comes out.

---

As for your application - can I suggest calling the market history REST endpoint on a minutely interval? Would that work for you?

[jackroc97]

Thanks again for the response.

Did you try talking to IBKR support and see what they say on the topic?

Not yet, but I may reach out to them and see what their recommendations are.

As for your application - can I suggest calling the market history REST endpoint on a minutely interval? Would that work for you?

I think that would probably be an acceptable solution for me, thanks!

[bitter-geek]

Regarding live data, isn't that what the marketdata/snapshot endpoint is for? Granted, I'm new to most of this stuff so I could be way off base.

When I use snapshot with the client gateway, it appears to work (albeit delayed a bit, because I think truly live data is paywalled).

However, when I try to use (for example) prices = client.live_marketdata_snapshot('72539702','31,70,71,7295') in IBind, I get a 400 error: "Bad Request: no bridge." On the client gateway, that would mean I just needed to query iserver/accounts, but I don't know how to do that in IBind.

Please forgive my ignorance; again, I'm new to most of this.

[Voyz]

@bitter-geek 400 bad bridge sounds like a different error with connectivity if I'm not mistaken. As for your request, you wanna pass the fields as a list of strings, not a comma-separated string, ie.:

prices = client.live_marketdata_snapshot('72539702',['31', '70', '71', '7295'])

[bitter-geek]

As for your request, you wanna pass the fields as a list of strings, not a comma-separated string

Ah, thank you for that!

I'm still getting the 400 error, though. How would I query iserver/accounts in IBind? Or does live_marketdata_snapshot handle that automatically? In the Web API reference, it says "A pre-flight request must be made prior to ever receiving data" when using marketdata/snapshot, but I have no idea what that means; I never did anything special when using the client gateway except query iserver/accounts when I got the 400 error.

A quick note: Thank you SO MUCH for making this library! It's a massive improvement over IBKR's implementation and documentation.

[Voyz]

To do that preflight thing, call this: ibkr_client.receive_brokerage_accounts. But you say you already call that prior?

A quick note: Thank you SO MUCH for making this library! It's a massive improvement over IBKR's implementation and documentation.

🥰 thank you! Happy that it is helpful! And I appreciate your contribution into this discussion 🙌

[bitter-geek]

Here's what I've got so far, just mostly borrowing some of the example code:

import os
import requests
import time

load_dotenv('oauth/ibind.env')

from ibind import IbkrClient, ibind_logs_initialize
from ibind.support.oauth import OAuthConfig

ibind_logs_initialize()

cacert = os.getenv('IBIND_CACERT', False)


client = IbkrClient(
    cacert=cacert,
    use_oauth=True,
)

print('\n#### live session token ####')
print(client.live_session_token)

print('\n#### get_accounts ####')
accounts = client.portfolio_accounts().data
client.account_id = accounts[0]['accountId']
print(accounts)

print('\n\n#### get_ledger ####')
ledger = client.get_ledger().data
for currency, subledger in ledger.items():
    print(f'\t Ledger currency: {currency}')
    print(f'\t cash balance: {subledger["cashbalance"]}')
    print(f'\t net liquidation value: {subledger["netliquidationvalue"]}')
    print(f'\t stock market value: {subledger["stockmarketvalue"]}')
    print()

import time

def get_live_marketdata(symbols):
    try:
        prices = client.live_marketdata_snapshot(symbols, ['31','70','71','7295'])
        return prices
    except Exception as e:
        print(f"Error fetching live market data: {e}")
        return None

client.receive_brokerage_accounts()

print("### PRICES ###")

# Try multiple times since it can take a bit to "warm up"
for i in range(3):
    prices = get_live_marketdata(['72539702'])
    if prices:
        print(f"Success: {prices}")
    time.sleep(5)

The ledger and account info print out just fine.

Without using receive_brokerage_accounts it fails on the snapshot. When I add receive_brokerage_accounts, it fails on that line with the same error 400:

  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/ibind/base/rest_client.py", line 218, in _process_response
    response.raise_for_status()
  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.ibkr.com/v1/api/iserver/accounts

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/ibind/base/rest_client.py", line 202, in request
    return self._process_response(response, result)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/ibind/base/rest_client.py", line 226, in _process_response
    raise ExternalBrokerError(f'{self}: response error {result} :: {response.status_code} :: {response.reason} :: {response.text}', status_code=response.status_code) from e
ibind.support.errors.ExternalBrokerError: IbkrClient: response error Result(data=None, request={'url': 'https://api.ibkr.com/v1/api/iserver/accounts'}) :: 400 :: Bad Request :: {"error":"Bad Request: no bridge","statusCode":400}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/ibind_test/venv/test.py", line 51, in <module>
    client.receive_brokerage_accounts()
  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/ibind/client/ibkr_client_mixins/accounts_mixin.py", line 78, in receive_brokerage_accounts
    return self.get('/iserver/accounts')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/ibind/base/rest_client.py", line 128, in get
    return self.request(method='GET', endpoint=path, base_url=base_url, extra_headers=extra_headers, params=params, log=log)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/ibind_test/venv/lib/python3.12/site-packages/ibind/base/rest_client.py", line 214, in request
    raise ExternalBrokerError(f'{self}: request error: {str(e)}') from e
ibind.support.errors.ExternalBrokerError: IbkrClient: request error: IbkrClient: response error Result(data=None, request={'url': 'https://api.ibkr.com/v1/api/iserver/accounts'}) :: 400 :: Bad Request :: {"error":"Bad Request: no bridge","statusCode":400}```

[Voyz]

I can see that you're using OAuth. I'm imagining that this could be a new error related to it. I'd suggest you speak to IBKR support about this error, as I can't recall ever seeing this error on non-oauth setup.

Additionally, could I ask you to open a new issue on this 400: Bad bridge error? That seems different to this issue we're in. Thanks!

[Voyz]

I'm going to close this issue due to inactivity. Thanks for your contribution and please feel free to request a reopen if you'd like to continue the discussion 👍

[kowkowhuang]

I want to expand this topic as I am investigating to use IBKR as a data source for non-US market. I couldn't find a definitive answer from the documentation or an existing workaround so I would like to ask help here :)

We're currently using a non-IBKR real-time API that provides accurate OHLC bars and spot prices, which we combine with your option_chain example to build order templates. This setup has been working well for us and we're very profitable. So thank you!

Problem Identified:
From our testing, we've observed that the history() endpoint has a ~2-3 minute latency. We have evidence from comparing session totals that confirms this lag.

Known Available IBKR Mechanisms:

REST History endpoint – Laggy (2-3 min delay observed)
REST Snapshot price – Real-time
WebSocket heavyweight – Real-time

Our Questions for the Community:

Question 1:
will ibind consider building a synthetic real-time OHLC bar builder API?

We're thinking of a hybrid workaround:

Use prior close as Open
Use real-time snapshot as Close
Use 5-second rolling min/max snapshots as temporary High/Low
Backfill with proper OHLC once the standard REST API history() data becomes available (after the 2-3 min lag)

Has anyone tried or implemented something similar? Any thoughts on this approach?

Question 2:
IbkrWsKey.MARKET_HISTORY WebSocket stream have definitively lower latency than the REST endpoint?
If the WebSocket stream provides real-time or near-real-time market history data, we could potentially use it instead of the laggy REST endpoint. Has anyone benchmarked the latency difference between these two sources?

[Voyz]

Hey @kowkowhuang glad to hear you've found the examples useful 🙌

As for the lag - could it possibly be the fact that they are not publishing a new bar until it is completed? You haven't mentioned the period resolution you're working with, but for a 2-3 min bar you'd expect a 2-3 min lag, right? They only publish completed bars, not currently developing ones - just like the name "market data history" suggests.

In either case, if your system's edge comes from most up to date price data, then I'd imagine listening to the WebSockets would be the most reasonable solution.

As for your questions:

Question 1: will ibind consider building a synthetic real-time OHLC bar builder API?

If there's interest in the community and someone - for example you - submits a PR with this kind of API then in principle I wouldn't be against having this kind of high-level supporting module.

However, personally I do not have resources to contribute these kind of custom functionalities to IBind at this moment.

Has anyone tried or implemented something similar? Any thoughts on this approach?

I think the key viability question comes from what you've already indirectly mentioned: "Backfill with proper OHLC". Your idea sounds doable in theory, but if you were to trade on this information and your internal bar would differ from the proper OHLC you'd get from IBKR, which one would you use? If it's the same - then great, I can't imagine this would take too long to implement.

In either case, I'd suggest using the WebSocket, not polling the REST endpoint. It's a pretty standard solution, I've built systems for many clients using the WebSockets and the market data channel.

Question 2: IbkrWsKey.MARKET_HISTORY WebSocket stream have definitively lower latency than the REST endpoint?

I haven't tried it out. I find Market History WebSocket pretty much useless as it doesn't seem to continue feeding data after the initial response.

The correct way - as far as I can see right now - would be to get the live marketdata (smd channel) and process it accordingly.