CloverDX 5.3

Go to Downloads (5.3.1) Latest update August 22, 2019 description

CloverDX 5.3 focuses on reusability and productivity during development and deployment. We’ve introduced a pre-configured Docker image for the Server, a new user interface for SQL-driven dynamic metadata, full control over metadata propagation in Custom Java components and improvements to Cluster performance and scalability.

New features in 5.3

Dynamic metadata driven by a SQL Query

With this feature, you can configure CloverDX to dynamically generate metadata from a predefined SQL query. This enables you to create reusable, data-agnostic transformations and subgraphs that automatically adapt to changing structures in your database.

Dynamic metadata has been available to skilled developers (in the underlying graph XML code) for a long time. With 5.3 we’ve added a user interface to make the feature easily accessible to all users. Metadata defined in this new UI is backwards compatible — if you had used dynamic metadata in the past, you won’t need to make any changes.

Since CloverDX 5.3, SQL query metadata can be easily created and edited directly in the Designer. We’ve also added additional configuration options so that you’re not limited just to the default metadata properties (for example, you can set delimiters now).


 

Metadata propagation from CustomJava components

Here’s another feature geared towards reusability of code. Custom components implemented in Java (using CustomJavaReader, CustomJavaWriter, etc.) can now define how they propagate (or create their own) metadata on input and output ports.

This way you can better define the component’s interface and behavior. Development in Designer is also easier as your custom components behave more predictably. You have full control over the propagation in the component’s code — you can reuse existing metadata, take metadata from other ports or create your own.


 

Improved CloverDX Cluster load balancer

Optimizing the utilization of system resources and scaling of data integration workloads has been another key area we’ve focused on.

In 5.3 we’ve modified the Cluster load balancing algorithm to spread the workload much more evenly across available nodes. This leads to a better utilization of available resources, increased throughput, shorter processing times per job, and reduces the likelihood of losing nodes due to system overload. Additionally we’ve specifically focused on preventing nodes from crashing due to low system memory. 

Our extensive internal testing has shown significantly better results over previous versions, especially when processing large numbers of jobs that start at the same time. We’ve been able to reach up to 40% faster data processing.


 

CloverDX Server console changes

On the Server Monitoring screen, we’ve revamped the performance charts for better readability (and they’re prettier too).

Color coding of different Server instances is another improvement for better and safer management of multiple environments (dev, test, UAT, prod). You can now add a little colored tag to your Server instance to indicate the environment you’re currently working with. No more accidental mistakes on prod!

Additionally, we’ve made minor improvements to the Server UI; improved the lifecycle of info messages displayed in the Server Console and introduced better control over auto refresh in the Execution History.


 

Docker

With CloverDX 5.3 we’ve added a Docker file as another option to get and deploy the Server (previous options are still available without change). The Docker file contains all the necessary settings and configuration to easily run production-grade CloverDX Server in a container, greatly shortening the time it takes to configure a production Server. 

This allows to create a modern data processing architecture easily using CloverDX. Just deploy containerized Server and mount additional volumes. No more configuration, just focus on code and run your data jobs anywhere where Docker lives.

For more information and details how to use containerised CloverDX, see our github repository at https://github.com/cloverdx/cloverdx-server-docker

Other changes

  • Support for Pivotal tcServer 3.2.14

  • Java in CloverDX Designer bundle has been updated to Oracle JDK 8 b202

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
  • Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.9 to 5.0). If you're on our maintenance program, we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
  • Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.9 to 5.0) and we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license key from there.

 

Release Compatibility/Upgrade notes, Features & Fixes Published Download

Fixes

Backport "Graph fails with NoHttpResponseException" to 5.3.1
CLO-17217 Fix
Graph analysis error found in checkconfig shows and error dialog
CLO-17106 Fix
Server simple HTTP API documentation link not working
CLO-17005 Fix
Move Extract Metadata action back to the first level of context menu
CLO-16972 Fix
August 22, 2019 Download CloverDX 5.3 5.3.1

Features & Improvements

Designer support for SQL Query Metadata
CLO-16381 Improvement
Improved cluster load balancer
CLO-16345 Improvement
Introduce support for standalone CloverDX Server in Docker container
CLO-16928 Improvement
New server monitoring charts
CLO-15753 Improvement
Custom metadata propagation for CustomJavaComponent
CLO-5905 Improvement
Remove online activation from Designer
CLO-16859 Compatibility
Update Java in Designer bundle to 1.8.0_202
CLO-16438
Improve discoverability of database related components
CLO-16408 Improvement
Configuration of poll interval at Execution History
CLO-16395 Improvement
Allow to use SQL Query Metadata as a type in CTL
CLO-16383 Improvement
Record level attributes for SQL Query Metadata
CLO-16382 Improvement
Make the component preview in Data Inspector toggleable
CLO-16184 Improvement
Keep last message available in Server Console UI after refresh
CLO-14879 Improvement
Introduce server color coding in Server Console UI
CLO-14388 Improvement
Display metadata field names in data inspector
CLO-12393 Improvement
Support Salesforce API 46 (Summer '19)
CLO-16855 Improvement

Fixes

LDAPReader and LDAPWriter cannot use LDAP attributes containing a dash in their name
CLO-16366 Fix
DB query keeps running in the DB when graph abort is attempted
CLO-16273 Fix
July 04, 2019 Download CloverDX 5.3 5.3.0
  • ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)
  • Administrators (This item is an improvement or feature that will help setup, install, administer and manage the application)
  • Support (This item helps staff supporting the production operation to identify potential problems or avoid such)