Get Hazelcast

Hazelcast Jet Now Available on Pivotal Cloud Foundry

July 06, 2017

I am very excited to announce that after months of engineering work and assistance from Pivotal, Hazelcast Jet is now available as a Tile on Pivotal Cloud Foundry (PCF) – the first general purpose data processing platform available on PCF. As a member of the Cloud Foundry community we are determined to enhance the services available on the platform by equipping developers with an open distributed processing engine for big data streams driven by a mature and stable tribe of contributors. Deploying Hazelcast in the cloud has never been easier.

Hazelcast Jet for Pivotal Cloud Foundry is a lightweight on-demand service broker that dynamically creates and scales Hazelcast Jet clusters without pre-provisioning blocks of VMs (virtual machines), Available as a Tile via Pivotal Network, this integration provides a frictionless user experience for Pivotal Cloud Foundry users, requiring no additional IT Administrator intervention to support on-demand services. With Hazelcast’s IMDG providing storage functionality, Hazelcast Jet is an Apache 2 licensed open source project that performs parallel execution to enable data-intensive applications to operate in near real-time.

Hazelcast Jet Diagram

Using directed acyclic graphs (DAG) to model relationships between individual steps in the data processing pipeline, Hazelcast Jet is simple to deploy and can execute both batch and stream-based data processing applications. It is appropriate for applications that require a near real-time experience such as sensor updates in IoT architectures (house thermostats, lighting systems), in-store e-commerce systems and social media platforms.

IMap<String, Integer> counts - source
    .map(e -> e.getValue().toLowerCase())
    .flatMap(line ->
    .collect(toIMap(m -> m, m -> 1,
        (left, right) -> left + right));

Cloud Foundry users are now able to provision and scale new Hazelcast Jet clusters, and add and remove nodes seamlessly as required. After downloading the Hazelcast tile from PivNet, the PaaS Operator simply uploads the tile into Pivotal Ops Manager, enters any required configuration into the GUI, and clicks ‘Install’. After installation, the PaaS Operator need have no further interaction with the Tile.

Users have the ability to choose from a variety of VM types, allowing them to increase the amount of RAM available to each node. Each Hazelcast Jet cluster is provisioned on exclusive VMs, ensuring privacy of data and high availability. In addition, developers are able to use Hazelcast Spring Cloud connectors to automatically have a ClientConfig object available in their applications.

Please read the Pivotal documentation to understand how to install and configure Hazelcast Jet for PCF (sample code is provided).

Hazelcast Jet is built on top of a one-record-per-time architecture (sometimes known as continuous operators). This means that it processes incoming records as soon as possible, opposed to accumulating records into micro-batches, consequently lowering latency for applications. Jet ingests data at high-velocity (via socket, file, HDFS or Kafka interfaces), and processes the business logic or complex computation on incoming data. A pure in-memory approach, Jet is 20x faster than Hadoop, enabling users to meet service-level requirements. It is extremely simple to program – in particular Jet can be fully embedded for OEMs and Microservices – making it is easier for manufacturers to build and maintain next generation systems.

About the Author

About the Author

Greg Luck

Greg Luck

Chief Technical Officer

Greg Luck is a leading technology entrepreneur with more than 15 years of experience in high-performance in-memory computing. He is the founder and inventor of Ehcache, a widely used open source Java distributed cache that was acquired by Software AG (Terracotta) in 2009, where he served as CTO. Prior to that, Greg was the Chief Architect at Australian start-up that went public on the Australian Stock Exchange (ASX:WTF) in 2006. Greg is a current member of the Java Community Process (JCP) Executive Committee, and since 2007 has been the Specification Lead for JSR 107 (Java Specification Requests) JCACHE. Greg has a master's degree in Information Technology from Queensland University of Technology and a Bachelor of Commerce from the University of Queensland.

Latest Blogs

Hazelcast IMDG 4.0 is Released

Hazelcast IMDG 4.0 is Released

Announcing the Hazelcast Community License

Redis Load Handling vs Data Integrity: Tradeoffs in Distributed Data Store Design

View all blogs by the author

Subscribe to the blog

Follow us on: