In CloverDX 6.3 we are introducing anonymization functionality to help our users deal with sensitive data in their jobs. We are also adding an ability to work with Excel files directly in Wrangler and many other improvements across the board.
The new data anonymization functionality will allow users to anonymize sensitive data in their jobs regardless of how they use CloverDX. For Wrangler users, we are introducing new anonymization steps while the engineers using Designer can take advantage of the new Anonymization library as well as new built-in CTL functions to build more complex anonymization processes.
We are also introducing a new Excel data source for Wrangler. This source will allow you to simply drop an Excel file to Wrangler and start working with your data right away. At the same time, our built-in data targets for CSV and Excel are now able to append to existing files, making reconciliation reports and continuous data update scenarios much easier to accomplish directly in CloverDX Wrangler.
We’ve updated CloverDX Designer to be compatible with Java 17. With this upgrade, we recommend you use Java 17 for all CloverDX products for best performance and security.
In this release we added an ability to anonymize sensitive data to ensure that it can be safely used, for example, when sharing your data with 3rd parties or when test data is needed. CloverDX now supports:
For Wrangler users, we are introducing a new step category – Anonymization. These are the steps that help with basic anonymization data:
Data engineers creating their processes in Designer can now leverage a new CloverDX Marketplace library – AnonymizationLib.
This library allows you to create shared configuration with anonymization rules and simply anonymize each data set by passing it through the Anonymization component. Anonymization rules are automatically applied based on the field names, so you don’t have to do that every time you need to anonymize a similar set of data.
We’ve introduced a new built-in data source in Wrangler – a Microsoft Excel data source allowing you to work with Excel files directly, significantly simplifying the data extraction process.
Just like the CSV source, the Excel source will analyze the layout of the spreadsheet file to determine the columns in the data. The data source allows you to work with your files right away – just drop an Excel file to Wrangler and start working with your data.
We’ve extended both built-in data targets – Excel and CSV – to allow you to append to a previously written output file. This will allow you to, for example, run a report repeatedly and append new data to existing output file, so that you do not have to merge the files manually later.
We are continuing to expand the functionality that allows you to manipulate your data in CloverDX Wrangler. In this release we added several new steps:
We improved the tooltip shown when hovering over the column header. It now shows more details about the data quality in the column and allows you to easily copy technical column name to clipboard to help you write formulas faster:
We've finished unifying the visual theme of the Server Console with the modern look & feel introduced with CloverDX 6.0. This is only a cosmetic update with minor improvements and does not require re-training.
We are continuing updates of CloverDX stack and with CloverDX 6.3 we’ve upgraded CloverDX Designer to use Java 17. The Designer will now install Eclipse Temurin JDK 17 by default.
The recommended deployment stack for CloverDX is therefore Apache Tomcat 9.x with Eclipse Temurin JDK 17 and CloverDX Designer with Temurin JDK 17.
Java 17 is an LTS (Long Term Support) version with Eclipse Foundation providing support at least until October 2027 compared to Java 11 with support ending in October 2024.
If you with to use CloverDX with Java 11, you can do so on the Server. CloverDX Designer will no longer run with Java 11, but it can be set to Java 11 compatibility mode if required to work with CloverDX Server deployed on Java 11.
New CTL functions for data anonymization
We’ve added several variants of CTL function addNoise to help you mimic the behavior of the anonymization steps from Wrangler in your CTL code.
List archive content in ListFiles components
ListFiles component can now look inside archive files (zip, gzip, tar, tgz, tar.gzip) and list files that are compressed in the archive. The component supports full set of functionality with regards to archives on other protocols (e.g., you can list content of zip archive in an s3 bucket without having to download the archive first) as well as nested archives (zip within zip etc.).
Dynamically populate Data App enumeration parameters on App start
User management improvements
We’ve updated the design of the user and group management modules in CloverDX Server Console. The user interface has been improved to work better with larger amounts of users and groups and to be more consistent and easier to use.
Execution properties for Wrangler workspaces
We’ve introduced a concept of Execution properties to Wrangler workspaces. You can now easily configure that your Wrangler jobs should have debugging enabled (so that you can see data on edges in Job Inspector) and you can also change the Log level.
Both settings will be useful mainly to CloverDX Server administrators and connector developers who will be able to easily see what is going on within Wrangler job in more detail compared to previous CloverDX versions.
Better integration of Data Catalog into source and target settings
Data Catalog now opens inside the data source and data target settings when you wish to select a different source or target instead of the current one. This helps you preserve the context of the job you are working on instead of switching out to a separate Data Catalog page.
HL7Reader no longer in Incubation
We’ve graduated the HL7Reader component from Incubation category in Designer. It can now be found in Readers category in the Component Palette.
At the same time, we’ve fixed composite naming when parsing HL7 files. This change may be incompatible with existing jobs that rely on previously produced composite names. Please review your jobs to ensure that they still work as expected.
To help you install or upgrade to this version, we've prepared a simple checklist:
Before You Upgrade
|Compatibility/Upgrade notes, Features & Fixes
Excel Data Source Improvements
Wrangler UI Improvements
|January 23, 2024
|Documentation Download CloverDX 6.3 6.3.1
Reading Excel in Wrangler
Server UI Redesign
User and Groups Management
|December 12, 2023
|Documentation Download CloverDX 6.3 6.3.0