Hazelcast Jet Now Available on Pivotal Cloud Foundry

I am very excited to announce that after months of engineering work and assistance from Pivotal, Hazelcast Jet is now available as a Tile on Pivotal Cloud Foundry (PCF) – the first general purpose data processing platform available on PCF. As a member of the Cloud Foundry community we are determined to enhance the services available on the platform by equipping developers with an open distributed processing engine for big data streams driven by a mature and stable tribe of contributors. Deploying Hazelcast in the cloud has never been easier.

Hazelcast Jet for Pivotal Cloud Foundry is a lightweight on-demand service broker that dynamically creates and scales Hazelcast Jet clusters without pre-provisioning blocks of VMs (virtual machines), Available as a Tile via Pivotal Network, this integration provides a frictionless user experience for Pivotal Cloud Foundry users, requiring no additional IT Administrator intervention to support on-demand services. With Hazelcast’s IMDG providing storage functionality, Hazelcast Jet is an Apache 2 licensed open source project that performs parallel execution to enable data-intensive applications to operate in near real-time.

Using directed acyclic graphs (DAG) to model relationships between individual steps in the data processing pipeline, Hazelcast Jet is simple to deploy and can execute both batch and stream-based data processing applications. It is appropriate for applications that require a near real-time experience such as sensor updates in IoT architectures (house thermostats, lighting systems), in-store e-commerce systems and social media platforms.

IMap<String, Integer> counts - source
    .stream()
    .map(e -> e.getValue().toLowerCase())
    .flatMap(line -> Arrays.stream(PATTERN.split(line)))
    .collect(toIMap(m -> m, m -> 1,
        (left, right) -> left + right));

Cloud Foundry users are now able to provision and scale new Hazelcast Jet clusters, and add and remove nodes seamlessly as required. After downloading the Hazelcast tile from PivNet, the PaaS Operator simply uploads the tile into Pivotal Ops Manager, enters any required configuration into the GUI, and clicks ‘Install’. After installation, the PaaS Operator need have no further interaction with the Tile.

Users have the ability to choose from a variety of VM types, allowing them to increase the amount of RAM available to each node. Each Hazelcast Jet cluster is provisioned on exclusive VMs, ensuring privacy of data and high availability. In addition, developers are able to use Hazelcast Spring Cloud connectors to automatically have a ClientConfig object available in their applications.

Please read the Pivotal documentation to understand how to install and configure Hazelcast Jet for PCF (sample code is provided).

Hazelcast Jet is built on top of a one-record-per-time architecture (sometimes known as continuous operators). This means that it processes incoming records as soon as possible, opposed to accumulating records into micro-batches, consequently lowering latency for applications. Jet ingests data at high-velocity (via socket, file, HDFS or Kafka interfaces), and processes the business logic or complex computation on incoming data. A pure in-memory approach, Jet is 20x faster than Hadoop, enabling users to meet service-level requirements. It is extremely simple to program – in particular Jet can be fully embedded for OEMs and Microservices – making it is easier for manufacturers to build and maintain next generation systems.