Customer Portal

Releases

CloverETL 4.2

Go to Downloads (4.2.1) Latest update August 17, 2016

CloverETL 4.2 improves (not only) team productivity and brings performance boost for parallel processing.

We have redesigned the internals of how Designer interacts with Server so that you can now put Server remote projects under version control. For Server Corporate, we're bringing “data partitioning” – a feature formerly available only in Cluster. It lets you run multiple parallel instances of a component at once – giving you more punch for operations that can utilize multiple CPU cores or high latency operations, such as API calls.

New features in 4.2

Version control (Git, SVN, ...) now works with Server projects

Designer now mirrors remote Server projects to your local disk, allowing you to easily put the local copy under version control.

With this you can manage remote Server projects under version control directly. You no longer need to keep a separate versioned local copy of the Server sandbox.

Version control operations such as pull, push, commit, merge, etc. work directly with the files in the local copy of the project, so you can use any standard version control tool, including built-in Eclipse plugins for Git and SVN. Designer then transparently synchronizes all changes to the Server sandbox.

 

Remote Server files available locally

As an effect of the previous feature, you can open and save files originating from a Server sandbox with any tool you need.

Convert between local and Server projects

You can easily deploy a project to Server from your workspace by using Convert To Server Project shortcut. Conversely, you can disconnect a Server project to keep just the local copy or connect and sync it with another Server sandbox.

Data Partitioning: Run components in parallel

Run parts of a transformation (individual components or sequences of steps) in parallel, optimizing workflows for multiple processor cores on a single Server box.

We've made Cluster capabilities available to single Server (available in Corporate only). You can now run multiple instances of a component to increase throughput of throttled Web Service APIs, to parallelize CPU intensive operations that would otherwise run only in a single thread, or simply partition and process data on multiple disks to maximize I/O performance.

 

Toggle details on edge

Press D to cycle between levels of detail when running data partitioning on a single Server (Corporate) or mulitple nodes (Cluster). You can check numbers of records being processed by and transfered between individual workers.

Source editor Syntax highlighting

The Source tab in the main editor now supports syntax highlighting.

Amazon S3 auto-select endpoint for region

Amazon recommends using correspoding endpoint URLs when accessing S3 buckets hosted in specific regions. You can now select the Amazon region and CloverETL will automatically generate the right URL.

Draw edge from input

You can now draw new edge from an input port to create a new upstream component. Just click an input port, drag to a blank area and then use the Add Component dialog to create the component.

Release Compatibility/Upgrade notes, Features & Fixes Published Download

Improvements

Create sandbox directory on import of Server configuration 		Improvement CLO-8982 4.2.1
Possibility to delete files from disk when deleting sandboxes on Server 		Improvement CLO-8333 4.2.1
Deleting project from disk outside of Designer should not remove sandbox files 		Improvement CLO-8901 4.2.1

Fixes

Bugs in visualization of clustered sandboxes in error states 		Fix CLO-9078 4.2.1
Export to CloverETL Server sandbox doesn't work 		Fix CLO-9029 4.2.1
Designer freezes on disconnected projects 		Fix CLO-8880 4.2.1
Checkconfig caused Designer connection timeout for big archives and wildcards 		Fix CLO-9154 4.2.1
Synchronization fails on filenames that differ in letter case only 		Fix CLO-8958 4.2.1
CopyFiles: Wrong escaping of plus sign in SFTP password 		Fix CLO-8885 4.2.1
August 17, 2016 Download CloverETL 4.2 4.2.1

Productivity / Version Control for Server Projects

Placeholders for large files in Designer 		New Feature CLO-7899 4.2.0-M1
Resolving Conflicts between local and server copy 		New Feature CLO-7836 4.2.0-M1
Ignore files in server project synchronization 		New Feature CLO-7604 4.2.0-M1
Unify various CloverETL project natures 		Improvement CLO-7741 4.2.0-M1
Convert local project to Server project (and back) 		New Feature CLO-8290 4.2.0-M1
Handling of connection outages 		Improvement CLO-7613 CLO-8207 4.2.0-M1
Graph header (.grf/.jbf XML files) is now not modified when not needed 		Improvement CLO-8192 4.2.0-M1

Platform Support

Hadoop integration now supports versions CDH 4.3, 5.4 		Improvement CLO-7254 4.2.0-M1
Dropped support for Eclipse 3.X 		Improvement CLO-6003 4.2.0-M1

Packaging / Versions

Designer is now bundled with Java 8 		Improvement CLO-8558 4.2.0-M1
Designer now comes with Java sources in the bundled JDK 		Improvement CLO-7042 4.2.0-M1
Server bundle upgraded to Tomcat 8 		Improvement CLO-7764 4.2.0-M1

Engine Improvements

Amazon S3 support: Replaced JetS3t with Amazon SDK 		Improvement CLO-7170 4.2.0-M1
Validator email address rule has new option "Allow addresses with no TLD". Formerly, this was a default behaviour, now you have to explicitly turn it on. 		Compatibility New Features CLO-6916 4.2.0-M1
FlatFileReader and FlatFileWriter aliases for UniversalDataReader and UniversalDataWriter 		New Feature CLO-8266 4.2.0-M1
Improved temp file management in Spreadsheet components 		Improvement CLO-7118 4.2.0-M1
JSONWriter and XMLWriter: New attribute "Write raw value" enables you to take raw snippets of JSON/XML and send them to its output without escaping it. 		Improvement CLO-6786 4.2.0-M1
New second output port for unmatched data in ExtMergeJoin, ExtHashJoin 		Improvement CLO-6588 4.2.0-M1
New metadata templates for port writing outputs in XMLWriter, UniversalDataWriter, CloverDataWriter, SpreadsheetDataWriter, StructuredDataWriter, JSONWriter 		Improvement CLO-4728 4.2.0-M1

Designer Improvements

Syntax highlighting in graph source editor 		Improvement CLO-6465 4.2.0-M1
Improved searching for dependencies of graph elements 		Improvement CLO-5588 4.2.0-M1
Removed legacy options from CloverETL Preferences page 		Improvement CLO-7962 4.2.0-M1
iif() and lookup() CTL functions were missing in function list in Transform and Expression editors 		Improvement CLO-7533 4.2.0-M1
Draging & dropping a job into jobflow now names the newly created component based on the name of the job 		Improvement CLO-6972 4.2.0-M1
Drawing an edge from input port: You can now click an input port, draw a new edge and then pick the source component form the Add Component dialog 		New Feature CLO-6050 4.2.0-M1
S3: Added region selection option to Edit URL dialog 		Improvement CLO-8588 4.2.0

Server Improvements

Data partitioning (run component in parallel) in CloverETL Server Corporate 		New Feature CLO-8874 4.2.0
Users can now pick JNDI names from JNDI tree whenever JNDI configuration option is present 		New Feature CLO-7160 CLO-7159 4.2.0-M1
New option to create a heap dump directly from Server Console 		New Feature CLO-7151 4.2.0-M1

Fixes

Shift+Space to insert a subgraph should use ${SUBGRAPH_DIR} instead of sandbox:// 		Fix CLO-7683 4.2.0-M1
SimpleGather is blocked in fast propagate mode 		Fix CLO-7211 4.2.0-M1
Opening file though File menu is throwing a null pointer exception 		Fix CLO-7297 4.2.0-M1
SpreadSheetWriter: mapping by order does not work with hyperlinks 		Fix CLO-7104 4.2.0-M1
Transform Editor does not open at all when metadata missing 		Fix CLO-3549 4.2.0-M1

Compatibility

S3 connectivity: Replaced JetS3t with Amazon SDK 		Compatibility Improvement CLO-7170 4.2.0-M1
Designer as eclipse plugin: Dropped support of Eclipse 3.x 		Compatibility CLO-6003 4.2.0-M1
XMLExtract: wrong attribute value is parsed 		Compatibility Fix CLO-7126 4.2.0-M1
DBJoin should propagate metadata 		Compatibility Improvement CLO-8477 4.2.0-M1
Rollup - preExecute() is never called 		Compatibility Fix CLO-7202 4.2.0-M1
SpreadSheetWriter: mapping by order does not work with hyperlinks 		Compatibility Fix CLO-7104 4.2.0-M1
Sequences used in COMPILED CTL no longer skip values 		Compatibility Fix CLO-5762 4.2.0-M1
Validator email address rule has new option "Allow addresses with no TLD". Formerly, this was a default behaviour, now you have to explicitly turn it on. 		Compatibility Fix CLO-6916 4.2.0-M1
CloverETL now supports IBM InfoSphere MDM version 11.5 		Compatibility CLO-8790 4.2.0

Under the hood

Metadata propagation algorithm moved from Designer to Server 		Improvement CLO-6437 4.2.0-M1
Allow more hyperlinks in graph run log 		Improvement CLO-7023 4.2.0-M1
Custom password storage for remembering Server project credentials 		Fix CLO-8806 4.2.0
June 08, 2016 Download CloverETL 4.2 4.2.0
  • For Developers (Improvements most useful for developers bringing new functionality or optimizations in data transformation and orchestration)
  • For Administrators (Improvements or features that will help setup, install, administer and manage the platform)
  • For Support (Helps staff supporting the production environment to identify and escalate potential problems or avoid such)
  • For Security (Improvements and changes relevant to security focused staff – sys admins and developers alike)