The Need for Greater Trade Visibility
Back office analysts typically use periodic intraday or end-of-day summaries of the day’s trades to run their analysis for risk assessment and regulatory compliance. Real-time systems generally are not used in the back office (like they are in the front office) due to high costs of maintaining such a system. This means there are significant time gaps in which analysts are unable to get a complete and detailed view on the firm’s trading activities. As overall trading activity grows, the risk due to this limited visibility will only get worse.
The missing piece is the real-time (or near-real-time) summaries of trades that lets analysts quickly view running totals for the day. Such aggregations are easily done in a batch process, but again, the latency associated with batch processing leads to time windows where analysts do not have an up-to-date view of trading status.
Solution: Continuous Query with Drilldown
One way to deliver greater trade visibility to the back office without the high expense of a real-time system is deploying a near-real-time, on-demand trade monitoring solution. In such a solution, when an analyst runs a query, the system quickly scans all trade data for the day, indexes the relevant data for the given query, then delivers an answer to the query. This is all done in seconds to give a fast response to users without the overhead of a fully real-time system. The indexes are stored in-memory to quickly answer additional queries from the analyst. Since all indexes are generated on-demand, there is no extra repository of data to maintain.
In the reference implementation, Hazelcast Jet uses a set of pre-built connectors to connect to a trade data source, typically Apache Kafka. It quickly ingests all trade data since the beginning of the day and indexes the relevant data for the given query. The indexed data is stored in Hazelcast IMDG, where the analyst can run additional queries using a browser connected to a web applications interface.
More information is available in the technical white papers below in the Resources section.
Continuous Query with Drill Down Trade Monitoring Demo
This explainer video provides a quick overview of the continuous query with drilldown solution that lets back office analysts get a near-real-time summarized view of the day’s trades. The solution leverages Hazelcast Jet and Hazelcast IMDG in the backend to quickly load and index the trade data.