How to access real-time data using Intrinio’s web socket clients

What is a web socket?

Our real-time equity feed gives you low latency real-time equity data with no exchange fees, reporting requirements, or paperwork – saving you thousands each month in end-user costs. This proprietary process outputs best-bid/best-offer quote data (from across the available exchanges) as well as data for every trade that has occurred on each of the available exchanges.

As opposed to our REST API, a web socket client does not utilize a request/response model for delivering data. Rather, with one of our web socket clients, equities data can be streamed, in real-time, directly into your application. This means that you will always receive the latest pricing data without the round-trip overhead or API usage limits.

Our web socket product allows you to dynamically “subscribe” and “unsubscribe” to data streams for individual equities (identified by ticker symbol, or list of ticker symbols), or to a “firehose” stream which will provide updates for every equity across all the included exchanges.

Regardless of which client SDK option you choose to use, the output will include top-of-book updates for asks and bids as well as all trades. If your use-case only includes the need for trade data, ask and bid updates can be excluded to lower bandwidth and CPU requirements.

All client SDK’s, linked to below, function similarly. For implementation-specific details, please see each client’s respective GitHub page.

Getting started with Web Socket API

To start streaming real-time equities data from our Multi-Exchange feed, use one of the following client SDK’s:

Each of the above SDK’s includes sample code that will allow you to run and test your streaming data connection. The sample code can also be used as a template or as the basis for your own project.

To begin streaming data, you must configure your client with your account credentials. In most cases (except when using JavaScript), this consists, exclusively, of your “Production” API Key, available in the “API Keys” section of your account page (here) . Each SDK will make it clear where to place your API key (usually in a JSON configuration file in the project root). And that’s it! That’s all that is needed to begin streaming data. Each sample application contains a preselected subset of equities data to use as a starting point. Instructions on streaming dataset selection is available in the “How to stream real-time equities data” section.

How to stream real time equities data

To stream real-time equities data (trades or quotes), you must “join” a channel. You can think of a channel as a specific feed. The channel(s) you join will determine what data you receive. Channels can be:

  • The ticker symbol of a security, such as AAPL, MSFT, or GOOG.
  • An array of ticker symbols.
  • The lobby (or “firehose”) channel, which streams ask, bid, and trade data for ALL securities available on any of the feed’s exchanges.
  • The trades-only lobby channel, which streams trade data (but not ask or bid data) for ALL securities available on any of the feed’s exchanges.

Note: the lobby (or “firehose”) channels require special access. Talk to your account manager or sales representative for more information.

Joining channels with SDK

Most SDK’s provide 2 ways to join channels:

  1. Static identification –

    This is the easiest method and is recommended when connecting to either of the lobby/firehose channels. With static identification, you identify the channels that you want to stream before the application starts. In most SDKs this is done by declaring them in the application’s JSON configuration file (usually under the parameter: “Symbols”).

  2. Dynamic identification –

    This method of symbol identification allows you to modify the data you receive, on-the-fly. For example, you can subscribe (and unsubscribe) from updates on certain tickers symbols using any custom logic or trigger event that you want. Since streaming data can be resource intensive, this allows you to receive only the data that you need, when you need it.

Regardless of SDK, joining channels is done using one of the available “join” methods/functions. See your specific SDK’s GitHub documentation for more details.

Leaving Channels with SDK

Most accounts are limited in the number of tickers symbols that they can be subscribed to, concurrently. If you reach this threshold (or if you wish to unsubscribe from a channel for any other reason), you can “leave” any/all currently-subscribed channels using one of the SDK’s available “leave” methods/functions. See your specific SDK’s GitHub documentation for more details.

Limits on streams

Depending on your subscription/contract, the following limits may apply to your WebSocket usage for streaming equities price data:

    Max Subscriptions (default: 500) –

    You may only listen for updates on this number of ticker symbols, simultaneously.

    Concurrent Connections (default: 2) –

    You may open this number of WebSocket connections, simultaneously.

    Firehose Enabled (default: no) –

    You may (or may not) be able to access the lobby/firehose feed.

    Trades Only (default: no) –

    Your usage is restricted to trades only (no ask or bid messages will be sent to any of your connected applications).

All client SDKs will generate warning messages when usage limits are reached. It is recommended that you record console output to determine if any of these limits are being hit. If the need arises, please speak to your account manager or sales representative about increasing these limits.

Output format of SDK

Regardless of which SDK is used, data is managed similarly. The raw data is transferred over the wire, in batches, using a proprietary binary message format. That data is then unpacked by the client SDK, translated into a native-language object, and handed off to you within a callback. Depending on which SDK is being used, the callback methods are registered with the top-level “client” object either at the point of client object creation, or just after (but before calling any of the “join” functions). A callback must be registered to handle “trades”. Optionally, if “quotes” (i.e., ask and bid) are important, a second callback can be registered to handle those as well.

As mentioned above, both quote and trade data will be represented as an object in the language being used. Regardless of language, the fields will be the same.

Trade

objects will have the following fields:

  • Symbol – the ticker symbol
  • Price – the price of the trade (in USD)
  • Size – the number of shares exchanged in the trade
  • Total Volume – the total number of shares traded so far, today (across all tracked exchanges)
  • Timestamp – a Unix timestamp representing the time of the trade

Quote (ask/bid)

objects will have the following fields:

  • Symbol – the ticker symbol
  • Type – a quote type indicator (either ask or bid)
  • Price – the price of the ask or bid (in USD)
  • Size – the number of shares offered as part of the quote
  • Timestamp – a Unix timestamp representing the time that the quote was placed

Best practices for real-time SDKs

When designing a system or product to work with one of our real-time SDK’s, there are a few things to note:

  1. Do not perform long-running tasks within the callback methods.
  2. The SDKs were designed for high throughput. If a callback takes too much time to process an update (ask, bid, or trade) subsequent update will be delayed. Ultimately, the queues used to hold the data temporarily will overflow and updates will be dropped.

  3. Carefully choose which SDK you’ll be using.
  4. All programming languages are not created equally; some are better at data processing than others. If you’ll be working with many symbols (using the “firehose”, for example), .Net would be a much better choice than JavaScript. In fact, some SDKs cannot handle the full volume of the “firehose” feed.

  5. Determine whether you will need quote data.
  6. You may need quote data, in addition to trade data and subscribe to the feed accordingly. Quotes (asks and bids) account for over 90% of the feed traffic so, if quote data is not needed, the performance of your system can be improved, drastically, by eliminating this data.

  7. Be careful when generating connections to the server.
  8. Most accounts come with the ability to open two connections simultaneously (one for production and one for development or testing purposes). The SDKs handle unplanned disconnects and failures automatically, so it is not necessary to roll your own code. One of the most common problems that we see in customer code is that, attempts to create new connections any time an issue is encountered. If the previous connection has not been properly closed, this will quickly exhaust your connection allotment and cascade into an unresolvable spiral of connection failures.

Don't see an SDK you're looking for? Have an issue with our web socket? Talk to our team.