SignalR Streaming

Introduction

This document provides an introduction to OpenAPI Streaming.

OpenAPI Streaming provides real time updates on quotes, positions, orders, balances etc. to client application without having to poll the OpenAPI at high frequencies. Streaming is feature where server push the content to connected clients instantly as it becomes available, so results in less network traffic and lower latency.  

Below steps are required for a client application to use OpenAPI Streaming:

1. Get a valid OpenAPI access token.
2. Establish connection to the streaming server.
3. Setup one or more subscriptions, to specify what you want to subscribe to.
4. OpenAPI Streaming server sends streaming updates to the client application.
5. Server send heartbeats to client in case no data is being transmitted for a certain period of time.

Get OpenAPI Access Token

All OpenAPI end points require client application to pass a valid OpenAPI access token. Client application can get access token by interacting with our security system. Depending on client application type, there are different authentication flows details of which can be found here: Security.

Establish Connection with Streaming Server

The OpenAPI Streaming feature depends on Microsoft's SignalR libraries, so a client application should have a reference to the latest stable version of Microsoft.AspNet.SignalR libraries depending on the client application type i.e. windows or web client.

Once we have dependencies in place , next steps are 

• Create connection proxy.
• Register event handlers with connection proxy. This refers to handling of wide range of connection lifetime events in OpenAPIs .
• Start Connection to connect with streaming server.

In Web based JavaScript application, a client can create the connection proxy by using $.connection method of SignalR.JS library as shown in code snippet below:

contextId = encodeURIComponent("C" + Date.now());
        
var queryString = "authorization=" + encodeURIComponent(accessToken) + "&context=" + contextId;
 
console.info("Initializing  Streaming Connection, Url: " + streamingConnectionUrl);
        
//Create connection proxy.
_connection = $.connection(streamingConnectionUrl, queryString, true);
        
//Register eventHandlers with connection proxy.
_connection.stateChanged(onStateChanged.bind(this));
_connection.received(onMessagesReceived.bind(this));
        
//Connect to streaming server.
_connection.start();

 Parameters explanation

• streamingConnectionUrl - The URL of the streaming server. For more information , please refer Environments
• QueryString -  Client have to send data to the server while making connection. Client must send followings 2 information's in query string parameter: 
• contextId : The streaming context id that this request is associated with.
  Within a user session the context id binds subscriptions to streaming connections. That makes it possible for a single user session to have several streaming connections open each with a distinct set of subscription, e.g. when opening multiple tabs in a web browser. The context id is client generated, making it possible to set up subscriptions and a streaming connection in parallel, and can be up to 50 characters long (a-z, A-Z, 0-9, dash, and underscore).
   

• authorization : Valid OpenAPI access token, you got in Step 1. Finally query string becomes like this -

  
  

• Logging - it is an optional parameter, with defaults to false. SignalR has a built-in logging function that you can enable on the client for troubleshooting. For example if you enable logging, logs tell you which of the following transport methods SignalR is using like this :

Setup Subscriptions to specify what you want to subscribe to

After making a connection, client application can set up subscription(s) with resources which support streaming and later can unsubscribe with resources to stop streaming or real time updates.

Each streaming resource supports following endpoints. You should call these endpoints as per your requirement.

• Subscribe              --> /openapi/{Service}/{Version}/{Some-Resource}/subscriptions/

• Unsubscribe (Single)   --> /openapi/{Service}/{Version}/{Some-Resource}/subscriptions/{contextId}/{referenceId}

• Unsubscribe (Multiple) --> /openapi/{Service}/{Version}/{Some-Resource}/subscriptions/{contextId}

Subscribe:  /openapi/{Service}/{Version}/{Some-Resource}/subscriptions/

Create endpoint is used to set up a subscription with a resource. Client must send POST request on this endpoint with required parameters.

POST /openapi/trade/v1/infoprices/subscriptions/

Request parameters of create endpoint

In request parameter, a serialized object whose parameters are as shown below is sent in the body of the request. 

Below is sample request with bare minimal required parameters only. For more information about concrete type of a resource please look at the Reference Documentation

Post /openapi/trade/v1/infoprices/subscriptions/
{
  "Arguments": {
    "AssetType": "FxSpot",
    "Uics": "21",
  },
  "ContextId": "29931122",
  "Format": "application/json",
  "ReferenceId": "0f8fad5b-d9cb-469f-a165-70867728950e",
  "RefreshRate": 5,
  "Tag": "2345223"
} 

Response

As soon as you call this endpoint on the resource, it makes a subscription with the streaming server and returns a data snapshot. The data snapshot is returned as the response of the endpoint. Immediately there after the streaming server starts sending  data to the client on connection made earlier and client receives it in the connection.received event handler.

Structure of data snapshot

Unsubscribe (Single): /openapi/{Service}/{Version}/{Some-Resource}/subscriptions/{contextId}/{referenceId}

The Unsubscribe(Single ) endpoint is used to delete a subscription from current session identified by the reference id.

Example structure of DELETE Request to Unsubscribe (Single):

Delete /openapi/trade/v1/infoprices/subscriptions/{contextId}/{referenceId}

Request parameters of remove (single) endpoint

Unsubscribe (Multiple): /openapi/{Service}/{Version}/{Some-Resource}/subscriptions/{contextId}

Unsubscribe(Multiple) endpoint is used to delete all subscriptions and free all resources on the server.

Example structure of DELETE Request to Unsubscribe (Multiple):

Delete /openapi/trade/v1/infoprices/subscriptions/{contextId}

Streaming Updates

All streaming messages are sent as incremental updates to the initial snapshot. The client must therefore maintain a representation of the snapshot and when an update is received apply that update to the snapshot.

Structure of streaming updates 

When an update is large, it may be split into a number of partitions each sent as an incremental update instruction. For example, if an update contains updates to 100 elements in a list, it may be sent as two separate updates each containing 50 list elements. Such updates will typically be sent immediately after each other but there may be other messages in between. In order for a client to determine whether an incremental update is a partition, it should inspect the __pn and __pc properties. __pc indicates the total number of partitions in the message and __pn indicates the number of the partition, starting from 0. When __pn=__pc-1 the last partition has been received.

Once again just for the reminder, here is where you will be receiving the streaming updates:

connection.received(function (data) {
       //This method receives streaming updates from server.
        onMessagesReceived(data,true);
   });

It is important to understand that messages over the streaming channel includes both data and control messages, and the client application must be able to handle both types of messages correctly.

Handling streaming data

How interpretation of the streaming data should be done and what is the structure of the data can be determined by looking at the ReferenceId property of the data. Following table informs us about the various kind of instructions we can get from the server:

The following subsections describe how the Data property should be interpreted depending on the data type.

Object Updates

Object updates contain only the properties whose values have changed. If a property is of a structured type (object or list), the property value is an also update and must be applied as such. Otherwise, the value in the snapshot can be replaced directly. In the following example, the object has a simple Age value and a structured Address property where the Street field is changed.

The initial snapshot:

{
   "Name": "Mister Green",
   "Age": 42,
   "Address": {
     "Street": "Green Boulevard",
     "City": "Green Town"
   }
}

The incremental update that is sent to the client to reflect a new Age and Street value:

{
   "Age": 43,                   // Simple: Value in snapshot must be replaced
   "Address": {                 // Structured: Update must be applied recursively
     "Street": "Red Boulevard"  // Simple: Value in snapshot must be replaced
   }
}

The snapshot after updating the Age and the Street fields:

{
   "Name": "Mister Green",
   "Age": 43,
   "Address": {
     "Street": "Red Boulevard",
     "City": "Green Town"
   }
}

 Array Updates Without Key Properties

An update for an array where the array elements have no key always contains the fully specified array value. Clients should replace the snapshot directly with the update value. This is true for both simple and structured types.

Array Updates With Key Properties

Updates to arrays of objects with key properties are sent as arrays that can contain three different types of elements: add elements, update elements, and delete elements. All the array elements in the update contains the key properties so that the corresponding element in the client's snapshot can be found. 

For each of the elements in the update array the client must apply the update to the snapshot in the following way:

• If the snapshot does not contain an element with the same key, the update is a fully specified object that should be added to the snapshot.
• If the snapshot contains an element with the same key, the update can either be a deletion or an update to that element:
• If the update has a property called "__meta_deleted", the element must be deleted from the client's snapshot
• Otherwise the element in the client's snapshot must be updated (not replaced) with the new value

In the example below, objects in the list are identified by the Name property. 

The snapshot contains two element:

[
   {
     "Name": "Mister Red",
     "Age": 42,
     "Address": {
       "Street": "Red Boulevard",
       "City": "Red Town"
    },
    {
     "Name": "Mister Green",
     "Age": 42,
     "Address": {
       "Street": "Green Boulevard",
       "City": "Green Town"
    }
]

The update contains one element update (Mister Red), one delete (Mister Green), and one add (Mister Blue).

[
   // Updated element contains only updated properties
   {
     "Name": "Mister Red",
     "Age": 43
   },
   // Deleted element contains a "__meta_deleted" property and the key properties
   {
     "Name": "Mister Green",
     "__meta_deleted": true
   },
   // New element is fully specified
   {
     "Name": "Mister Blue",
     "Age": 42,
     "Address": {
       "Street": "Blue Boulevard",
       "City": "Blue Town"
    }
]

The result of applying the update is:

[
   {
     "Name": "Mister Red",
     "Age": 43,
     "Address": {
       "Street": "Red Boulevard",
       "City": "Red Town"
    },
    {
     "Name": "Mister Blue",
     "Age": 42,
     "Address": {
       "Street": "Blue Boulevard",
       "City": "Blue Town"
    }
]

Heartbeat

When no data is sent for a while on a subscription, streaming server starts sending out heartbeats to let the client know that it is still active.

Heartbeat instructions have following format:

Like other streaming data, client receives it too in connection.received event handler. A Heartbeat is an object with the following properties: 

When no data or heartbeat has been received for the amount of time specified in the subscription response (InactivityTimeout), the client should reset the subscription using a new reference id.

[
   {
     {
       "ReferenceId": "_heartbeat",
       "Timestamp": "...",
       "Heartbeats": [
         {
           "OriginatingReferenceId": "MyReferenceId1",
           "Reason": "SubscriptionTemporarilyDisabled"
         },
         {
            "OriginatingReferenceId": "MyReferenceId2",
            "Reason": "NoNewData"
          },
       ]
   }
]

Subscription reset instructions

In some error scenarios, the streaming server may send subscription reset instructions to instruct the client to reset a given subset of its subscriptions. In this case, value of ReferenceID in the stream data will be "_resetsubscriptions" and reference ids of the subscriptions which should be reset is sent in parameter TargetReferenceIds.

Resetting a subscription means that the subscription should be deleted and another one created in its place using a new reference id. Any update received for a subscription after a reset instruction must be ignored.

Reset instruction have the following format :

Refers to a known subscription

In this case Reference id of the streaming instruction will be equal to one of the reference Id's of the made subscriptions. In this case apply incremental update to the snapshot of the subscription with the given reference id. Structure of instruction is what is defined above in streaming updates section.

Subscription disconnected instructions

It is possible that a subscription will be closed by the server. This can happen if a user changes their account password. When this happens a "_disconnect" instruction will be sent out on the stream and after that, no more messages will be sent. The client cannot create a new subscription after receiving a disconnect message. All subsequent requests will be rejected with a 401 - Unauthorized response from the server until the user is logged out and logs back in. The message will usually be sent to all open subscriptions at the same time.

Disconnecting a subscription Means that the entire user session should be closed and any messages received after this message should be ignored. The user should be logged out and asked to log in again.

Disconnect instructions have the following format:

References

How to set up a developer machine

Security

Environments

Reference Documentation