Hazelcast IMDG Integrates with Apache Cassandra to Deliver Fast, Scalable IoT Data Platform for Future Grid
Future Grid works with several Australian utility companies to automate the processing of sensor and smart meter data which crosses energy networks. Their customers are collecting approximately 3 billion data points per day. In terms of daily post processing, this equates to 20 billion records as each record has multiple, individual data points –a massive scaling challenge. To make the most of this information, utility organizations need a real-time data aggregation and processing solution which enables them to make complex real-time decisions.
When Future Grid first tried to solve this problem, it used traditional relational databases. However, it soon became apparent traditional databases couldn’t cope with huge volumes of data in real-time, main issue being that they can’t execute algorithms against incoming data fast enough. Future Grid then decided to build its own solution combining Hazelcast IMDG with Apache Cassandra’s persistence data store capabilities.
This case study tells the story of how Future Grid built its data platform and the primary use cases of their customers including:
- Power quality, interval and event derivations: clean de-duplicate five minute power quality data and daily per device “rollup” that includes pre-calculations to make further analysis faster and more accurate.
- Loss of Neutral Detection: using machine learning and fast data processing to monitor and predict safety issues, reducing shock instances significantly.
- Phase based substation aggregation: transformer modelling using aggregate meter interval data to provide better visibility per phase substation usage. Used for long term asset planning, phase balancing and alerting of exceeding designed rating.
- Customer Phase Cross referencing: using machine learning to investigate data correctness of meter to substation mappings including a responsive, real-time visualization solution.