CloverDX 5.9

Go to Downloads (5.9.0) Latest update December 15, 2020 Documentation

In CloverDX 5.9 we continued our work on the operations side of the product by improving Operations Dashboard. You can now create multiple dashboards so that different users can have their own customized views of the environment. 

To allow our users to work directly with Apache Kafka, we are introducing native Kafka connections and components that allow you to read and write Kafka topics. 

About this release: This is a production release. We recommend upgrading to this version from all previous versions. Please check Installation/Upgrade section.

New features in 5.9

Operations Dashboard Improvements

We introduced Operations Dashboard in CloverDX 5.8. In this release we’re improving usability and flexibility of the dashboard. The largest change is the introduction of multiple dashboards. You can now create different views of your environment to suit different teams or users.

We’ve also implemented “past alerts” functionality that will guarantee that your team will never miss any alert. Monitors will now remember that an alert has been raised and will show you a badge that will help you see which monitors had failures in the past even though they may be running properly again. 

CloverDX Operations Dashboard showing multiple dashboards as well as a monitor with past alerts badge.

CloverDX Incubation

As we always strive to bring you the latest and greatest features in our products, we are introducing a new way that will allow us to deliver features to you earlier that usually – CloverDX Incubation

With Incubation, we are aiming at providing production-quality code (i.e., fully tested) which you can play with and provide feedback to us about its usability and your use cases. While the code delivered in Incubation is fully tested, we do not recommend using these features in production since we expect to make larger (possibly not backwards compatible) changes based on the feedback we receive from you.

Once the API or interface of Incubation feature is stable (which usually will be the next release), we will move the feature out of Incubation, and it will then live in its proper place. We’ll then maintain backwards compatibility just like we do for any other part of CloverDX.

CloverDX 5.9.0 Incubation category in Component Palette in the Designer.

Kafka Connections and Components

We are releasing native support for Kafka as part of our Incubation pack in CloverDX 5.9. With new Kafka connections you can easily connect directly to your Kafka cluster. You can connect to Kafka clusters with SSL/TLS, Kerberos or Plain authentication.

When working with Kafka, you can use three new components: 

  • KafkaReader: allows you to consume messages from Kafka topics. We support string as well as binary data and you can configure many different options (including auto-commit, inactivity periods and much more). 

  • KafkaWriter: allows you to write messages to Kafka topic. You can work with text as well as binary data and just as with KafkaReader, we support full range of options to help you determine how to write your data. 

  • KafkaCommit: allows you to manage your offsets within topic partitions manually to make sure you only move the offset once your data has been safely consumed and processed.

Graph that consumes JSON-formatted data from Kafka topic, writes it into a database table and commits offset only if the messages were successfully written.

Since these components are all part of Incubation, we’d like to ask you for feedback – how you imagine you can use the components, whether they fit into your use cases and of course any usability issues.

Microsoft SQL Server 2019 Support

CloverDX Server now supports SQL Server 2019 as its backend database. We still support SQL Server 2014 so you do not need to migrate although we recommend you update the driver to the latest one (8.4 at the time of writing).

Other Highlights

Besides the above, we’ve made many more changes in CloverDX. Following is a selection of the more interesting changes: 

  • Performance improvement when listing files in sandboxes – the files are now streamed as they are read instead of waiting for all files to be discovered. 

  • CloverDX Designer installer now ships with AdoptOpenJDK 11 instead of Oracle JDK 8. Oracle JDK 8 is still supported if you need it. 

Feature Removal Notice

CloverDX 5.9 is the last release that will support the following: 

  • Launch Services. Launch Services have been replaced by Data Services and have been deprecated since CloverETL 4.9 (released in July 2018). Launch Services users will be required to migrate their code to use Data Services instead. More details in CLO-20307

  • RSE projects. RSE is a framework that was used to synchronize Designer and Server projects up until CloverETL 4.2 (June 2016) where we implemented more powerful “sync projects”. Users will be required to re-link their projects as “sync projects”. More details in CLO-20308

  • Deprecated components XLSDataReader and XLSDataWriter. Both components are deprecated and have better replacement (SpreadsheetDataReader and SpreadsheetDataWriter). More details in CLO-20309

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
  • Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.9 to 5.0). If you're on our maintenance program, we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
  • Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.9 to 5.0) and we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license key from there.

 

Release Compatibility/Upgrade notes, Features & Fixes Published Download

Features & Improvements

Add dashboard name to monitor log
CLO-20263 Improvement
Loosen restrictions on supported instance types in AWS Marketplace
CLO-20221 Improvement
Server shows "time difference is too long" warning
CLO-20084 Improvement
KafkaCommit component
CLO-20035 Improvement
Kafka connection
CLO-20028 Improvement
KafkaWriter component
CLO-20027 Improvement
KafkaReader component
CLO-20026 Improvement
Add warnings to monitor detail
CLO-19962 Improvement
Multiple Dashboards
CLO-19938 Improvement
Show History action for triggers
CLO-19936 Improvement
Bundle AdoptOpenJDK11 with Designer
CLO-19918 Improvement
Require db_owner only when patching the database
CLO-19917 Improvement
History of failures in monitoring
CLO-19843 Improvement
Ability to configure Worker to change log layout to JSON via log4j2 config
CLO-19531 Improvement
Improve header of Running jobs in Monitoring page
CLO-19099 Improvement
Clicking on the new line in FMT editor should add a new field
CLO-18599 Improvement
Support MS SQLServer 2019 as Server database
CLO-18318 Improvement
Streaming directory listing for sandboxes
CLO-18036 Improvement Compatibility
Improvements and small fixes in CTL editor
CLO-16402 Improvement
Add item type to Monitor detail
CLO-19790 Improvement
Add action to reset all items in a monitor
CLO-19785 Improvement
Sandbox upload progress bar
CLO-18867 Improvement
Wording of MS SQL Server JDBC specific is misleading
CLO-18844 Improvement

Fixes

Cannot Use Copy/MoveFiles on File Larger than 5GB on S3
CLO-20231 Fix
Metadata editor wrong fonts
CLO-20072 Fix
Zip Bomb Detected
CLO-20034 Fix
Support MySQL replication for Server backend DB
CLO-19613 Fix
SQL statement MERGE cause graphs failures with MS SQL
CLO-14427 Fix
December 15, 2020 Documentation Download CloverDX 5.9 5.9.0
  • ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)
  • Administrators (This item is an improvement or feature that will help setup, install, administer and manage the application)
  • Support (This item helps staff supporting the production operation to identify potential problems or avoid such)