CloverETL 4.2

Go to Downloads (4.2.1) Latest update August 17, 2016

CloverETL 4.2 improves (not only) team productivity and brings performance boost for parallel processing.

We have redesigned the internals of how Designer interacts with Server so that you can now put Server remote projects under version control. For Server Corporate, we're bringing “data partitioning” – a feature formerly available only in Cluster. It lets you run multiple parallel instances of a component at once – giving you more punch for operations that can utilize multiple CPU cores or high latency operations, such as API calls.

New features in 4.2

Version control (Git, SVN, ...) now works with Server projects

Designer now mirrors remote Server projects to your local disk, allowing you to easily put the local copy under version control.

With this you can manage remote Server projects under version control directly. You no longer need to keep a separate versioned local copy of the Server sandbox.

Version control operations such as pull, push, commit, merge, etc. work directly with the files in the local copy of the project, so you can use any standard version control tool, including built-in Eclipse plugins for Git and SVN. Designer then transparently synchronizes all changes to the Server sandbox.

 

Remote Server files available locally

As an effect of the previous feature, you can open and save files originating from a Server sandbox with any tool you need.

Convert between local and Server projects

You can easily deploy a project to Server from your workspace by using Convert To Server Project shortcut. Conversely, you can disconnect a Server project to keep just the local copy or connect and sync it with another Server sandbox.

Data Partitioning: Run components in parallel

Run parts of a transformation (individual components or sequences of steps) in parallel, optimizing workflows for multiple processor cores on a single Server box.

We've made Cluster capabilities available to single Server (available in Corporate only). You can now run multiple instances of a component to increase throughput of throttled Web Service APIs, to parallelize CPU intensive operations that would otherwise run only in a single thread, or simply partition and process data on multiple disks to maximize I/O performance.

 

Toggle details on edge

Press D to cycle between levels of detail when running data partitioning on a single Server (Corporate) or mulitple nodes (Cluster). You can check numbers of records being processed by and transfered between individual workers.

Source editor Syntax highlighting

The Source tab in the main editor now supports syntax highlighting.

Amazon S3 auto-select endpoint for region

Amazon recommends using correspoding endpoint URLs when accessing S3 buckets hosted in specific regions. You can now select the Amazon region and CloverETL will automatically generate the right URL.

Draw edge from input

You can now draw new edge from an input port to create a new upstream component. Just click an input port, drag to a blank area and then use the Add Component dialog to create the component.

Release Compatibility/Upgrade notes, Features & Fixes Published Download

Improvements

Create sandbox directory on import of Server configuration
Improvement CLO-8982 4.2.1
Possibility to delete files from disk when deleting sandboxes on Server
Improvement CLO-8333 4.2.1
Deleting project from disk outside of Designer should not remove sandbox files
Improvement CLO-8901 4.2.1

Fixes

Bugs in visualization of clustered sandboxes in error states
Fix CLO-9078 4.2.1
Export to CloverETL Server sandbox doesn't work
Fix CLO-9029 4.2.1
Designer freezes on disconnected projects
Fix CLO-8880 4.2.1
Checkconfig caused Designer connection timeout for big archives and wildcards
Fix CLO-9154 4.2.1
Synchronization fails on filenames that differ in letter case only
Fix CLO-8958 4.2.1
CopyFiles: Wrong escaping of plus sign in SFTP password
Fix CLO-8885 4.2.1
August 17, 2016 Download CloverETL 4.2 4.2.1

Productivity / Version Control for Server Projects

Placeholders for large files in Designer
New Feature CLO-7899 4.2.0-M1
Resolving Conflicts between local and server copy
New Feature CLO-7836 4.2.0-M1
Ignore files in server project synchronization
New Feature CLO-7604 4.2.0-M1
Unify various CloverETL project natures
Improvement CLO-7741 4.2.0-M1
Convert local project to Server project (and back)
New Feature CLO-8290 4.2.0-M1
Handling of connection outages
Improvement CLO-7613 CLO-8207 4.2.0-M1
Graph header (.grf/.jbf XML files) is now not modified when not needed
Improvement CLO-8192 4.2.0-M1

Platform Support

Hadoop integration now supports versions CDH 4.3, 5.4
Improvement CLO-7254 4.2.0-M1
Dropped support for Eclipse 3.X
Improvement CLO-6003 4.2.0-M1

Packaging / Versions

Designer is now bundled with Java 8
Improvement CLO-8558 4.2.0-M1
Designer now comes with Java sources in the bundled JDK
Improvement CLO-7042 4.2.0-M1
Server bundle upgraded to Tomcat 8
Improvement CLO-7764 4.2.0-M1

Engine Improvements

Amazon S3 support: Replaced JetS3t with Amazon SDK
Improvement CLO-7170 4.2.0-M1
Validator email address rule has new option "Allow addresses with no TLD". Formerly, this was a default behaviour, now you have to explicitly turn it on.
Compatibility New Features CLO-6916 4.2.0-M1
FlatFileReader and FlatFileWriter aliases for UniversalDataReader and UniversalDataWriter
New Feature CLO-8266 4.2.0-M1
Improved temp file management in Spreadsheet components
Improvement CLO-7118 4.2.0-M1
JSONWriter and XMLWriter: New attribute "Write raw value" enables you to take raw snippets of JSON/XML and send them to its output without escaping it.
Improvement CLO-6786 4.2.0-M1
New second output port for unmatched data in ExtMergeJoin, ExtHashJoin
Improvement CLO-6588 4.2.0-M1
New metadata templates for port writing outputs in XMLWriter, UniversalDataWriter, CloverDataWriter, SpreadsheetDataWriter, StructuredDataWriter, JSONWriter
Improvement CLO-4728 4.2.0-M1

Designer Improvements

Syntax highlighting in graph source editor
Improvement CLO-6465 4.2.0-M1
Improved searching for dependencies of graph elements
Improvement CLO-5588 4.2.0-M1
Removed legacy options from CloverETL Preferences page
Improvement CLO-7962 4.2.0-M1
iif() and lookup() CTL functions were missing in function list in Transform and Expression editors
Improvement CLO-7533 4.2.0-M1
Draging & dropping a job into jobflow now names the newly created component based on the name of the job
Improvement CLO-6972 4.2.0-M1
Drawing an edge from input port: You can now click an input port, draw a new edge and then pick the source component form the Add Component dialog
New Feature CLO-6050 4.2.0-M1
S3: Added region selection option to Edit URL dialog
Improvement CLO-8588 4.2.0

Server Improvements

Data partitioning (run component in parallel) in CloverETL Server Corporate
New Feature CLO-8874 4.2.0
Users can now pick JNDI names from JNDI tree whenever JNDI configuration option is present
New Feature CLO-7160 CLO-7159 4.2.0-M1
New option to create a heap dump directly from Server Console
New Feature CLO-7151 4.2.0-M1

Fixes

Shift+Space to insert a subgraph should use ${SUBGRAPH_DIR} instead of sandbox://
Fix CLO-7683 4.2.0-M1
SimpleGather is blocked in fast propagate mode
Fix CLO-7211 4.2.0-M1
Opening file though File menu is throwing a null pointer exception
Fix CLO-7297 4.2.0-M1
SpreadSheetWriter: mapping by order does not work with hyperlinks
Fix CLO-7104 4.2.0-M1
Transform Editor does not open at all when metadata missing
Fix CLO-3549 4.2.0-M1

Compatibility

S3 connectivity: Replaced JetS3t with Amazon SDK
Compatibility Improvement CLO-7170 4.2.0-M1
Designer as eclipse plugin: Dropped support of Eclipse 3.x
Compatibility CLO-6003 4.2.0-M1
XMLExtract: wrong attribute value is parsed
Compatibility Fix CLO-7126 4.2.0-M1
DBJoin should propagate metadata
Compatibility Improvement CLO-8477 4.2.0-M1
Rollup - preExecute() is never called
Compatibility Fix CLO-7202 4.2.0-M1
SpreadSheetWriter: mapping by order does not work with hyperlinks
Compatibility Fix CLO-7104 4.2.0-M1
Sequences used in COMPILED CTL no longer skip values
Compatibility Fix CLO-5762 4.2.0-M1
Validator email address rule has new option "Allow addresses with no TLD". Formerly, this was a default behaviour, now you have to explicitly turn it on.
Compatibility Fix CLO-6916 4.2.0-M1
CloverETL now supports IBM InfoSphere MDM version 11.5
Compatibility CLO-8790 4.2.0

Under the hood

Metadata propagation algorithm moved from Designer to Server
Improvement CLO-6437 4.2.0-M1
Allow more hyperlinks in graph run log
Improvement CLO-7023 4.2.0-M1
Custom password storage for remembering Server project credentials
Fix CLO-8806 4.2.0
June 08, 2016 Download CloverETL 4.2 4.2.0
  • ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)
  • Administrators (This item is an improvement or feature that will help setup, install, administer and manage the application)
  • Support (This item helps staff supporting the production operation to identify potential problems or avoid such)