Announcing Hazelcast Jet 0.7

We are happy to announce the release of Hazelcast Jet 0.7. This release brings lots of new features as well as the launch of our Jet Management Center product.

Job Elasticity and Graceful Shutdown

In previous versions of Jet, it was possible to automatically restart a job from where it left off using snapshots if one of the nodes failed. This feature is now enhanced with full support for job elasticity. A job may scale up and down automatically when nodes are added or removed from the cluster.

Additionally when a node is shutdown all the running jobs can be stopped gracefully, and restarted automatically after the node shutdown is complete. To read more about this feature, please see the reference manual.

Suspend and Resume Jobs

It is also possible to suspend and resume streaming jobs manually. When a job is suspended, a snapshot will be saved in its current state and when resumed the job will be started from that state. A code sample illustration the feature can be found at the code samples repository

Source Builder API

Previous to this release, the only way to create custom sources was to create a custom Processor using the Core API. It is now possible to create a custom source using the new source builder API which makes it easy to connect Jet to new sources.

A simple file source can be built as below:

BatchSource<String> fileSource = SourceBuilder
    .batch("file-source", x ->                               
            new BufferedReader(new FileReader("input.txt")))
    .<String>fillBufferFn((in, buf) -> {                          
        String line = in.readLine();
        if (line != null) {
            buf.add(line);
        } else {
            buf.close();                                     
        }
    })
    .destroyFn(BufferedReader::close)
    .build();

Jet Management Center

This version also marks the release of the first version of the Jet Management Center. Jet Management Center is a web application which gives an overview of the cluster and makes it possible to visualize the DAG (directed acyclic graph) and the data flow.

Jet Management Center is free for one node use and can be downloaded here. It is also available in Enterprise and Professional.

Pipeline API additions

There are several new additions to the pipeline API:

  • Merge operation allows merging two streams of the same type into one stream.
  • Distinct operation allows filtering of distinct items
  • Rolling aggregations can be used to a compute an aggregation where the aggregated value is emitted for every new incoming item.
  • Co-aggregations have been simplified so that they can be composed from aggregate operations with single inputs.
  • Keyed mapUsingContext which can be used for enrichment and lookups.
  • Support for enriching a stream directly from IMap or ReplicatedMap.

JMS and JDBC connectors

There are new sink and source connectors which make it possible to use JMS and JDBC as sources or sinks.

For full examples, please see the code samples repository.

Support for reading from and writing to Avro files

We have also added support for directly reading from or writing to Avro files directly. It can be used in a similar fashion to the file reader and writer:

BatchStage<Forecast> source = p.drawFrom(AvroSources.files("forecasts", Forecast.class)                                    
                               .setName("Read Forecast Files");

Security Suite

It is now possible to use Jet with TLS to have all member to member and member to client communication encrypted when using Jet Enterprise version.

For a full list of changes, have a look at the release notes.