Releases
CloverDX 6.5 is bringing numerous “quality of life” improvements across all corners of our platform. We’re also introducing an incubation version of a major new feature that will be coming in full force in the upcoming 6.6 and later releases – the Data Manager. You will be able to support business processes that involve direct manual intervention into data coming through your data pipelines, as well as use the Data Manager to store and manage your shared static (reference) data (lookup tables).
We are introducing a new more powerful mapping in the Aggregate component. The new mapping allows you to configure conditional aggregations reducing the need for coding and allowing you to build your jobs much faster.
We made several improvements to CloverDX Wrangler. These include the ability to automatically perform data type conversions in your output mapping, ability to preview data sent to your data target and improvements in the Delete columns step.
We are also introducing a new CloverDX Home app which simplifies the navigation in CloverDX platform.
We’ve updated the visual design of Data Apps. New Data Apps are easier to navigate and use and align better with the overall visual design of CloverDX.
With this release we improved the auto-configuration of CloverDX to make it easier to build cluster of CloverDX Servers. This feature can be effectively used to build CloverDX instances in Kubernetes or in other infrastructure as code platforms. We’ve published two new examples which show you how to run a single CloverDX Server instance of a cluster of multiple CloverDX Servers in Kubernetes.
And as with every release, we made many smaller changes that will help you when using CloverDX to manage your data. This includes new CTL functions, better Component Palette in Designer, LookupTableReaderWriter improvements and more.
New features in 6.5
More powerful Aggregate component
Our improved Aggregation mapping allows you to use Aggregator to conditionally aggregate your data. This will allow you to use Aggregator component in situations that would have required usage of Denormalizer component in previous versions of CloverDX. This in turn reduces the need for coding and allows even less experienced users to create more complex aggregations without overcomplicating their graphs.
The updated component also allows you to create metadata as you go along so that you no longer need metadata defined before you start working on your aggregation. This helps you build your mappings more effectively without having to go back and forth between multiple different dialogs.
Wrangler improvements
Automatic type conversions in mapping
Mapping to target in your Wrangler jobs will now automatically apply conversions between some data types. Automatic conversions are only applied in cases where the conversion is safe and no data is lost. This includes conversions of various data types to string or conversion from integer to decimal.
Preview of data sent to target
You can now click on the data target "step" in your Wrangler jobs in the Step list panel to see the data exactly as it is sent to the target – including the mapping if any was present.
This allows you to better see and understand what kind of data your target is getting, which is especially nice when automatic data conversions are applied as described above.
Delete columns step improvements
Delete columns step has an additional parameter – Mode of operation – which allows you to select whether to keep or remove the selected columns. The “keep selected” mode that was added in CloverDX 6.5 is especially useful in cases where you have large number of input fields and only need to retain small number of them – instead of selecting each of the many fields to remove, you can select the fields you wish to keep.
Improved platform navigation
With the addition of the Data Manager (see below), CloverDX now includes four apps that help non-technical users to work with their data. To help you navigate the platform, we are introducing CloverDX Home – a new page that allows you to see which apps you can use. The new Home page is available via https://your.server.com/clover/home.
To help you navigate in various apps, we added a new App menu which allows you to quickly jump to other apps in CloverDX platform.
Visual Refresh of Data Apps
We’ve improved the design and usability of Data Apps. Data Apps now have more modern look that fits better into CloverDX platform.
Kubernetes deployments
We are introducing new examples that show how you can run CloverDX Server or a cluster of CloverDX Server instances in Kubernetes. The deployment examples have been configured with reasonable defaults and can be used directly or as templates for your own deployments.
To get the examples, please visit our GitHub repository.
Incubation: Data Manager
We are introducing a preview of CloverDX Data Manager – a new part of CloverDX platform that helps users to collaboratively improve data quality in CloverDX platform.
Data Manager implements a workflow that allows technical users to submit data for review to domain experts. Domain experts can use Data Manager web app to review, edit and approve changes in the data. Once that is done, the approved records can be picked up by CloverDX graphs and further processed before being sent to downstream systems.
This collaborative workflow allows each user to contribute in their area of expertise – whether it is technical users who build the “backend” processes to ingest data and feed it to Data Manager or non-technical users who know the data very well and can manually fix the issues that cannot be resolved automatically.
Data Manager is an Incubation feature. If you’d like to try it, please contact your CloverDX account manager or [email protected].
Smaller changes
New Component Palette in Designer
We improved Component Palette in CloverDX Designer. New Palette works much better with a larger number of components and categories. This is especially helpful for users who have a lot of libraries deployed in their CloverDX Server.
LookupTableReaderWriter component propagates metadata
When working with lookups, the LookupTableReaderWriter component will now automatically propagate metadata corresponding to the lookup table on its input and output ports. This will help you build your jobs more effectively and reduce issues coming from misaligned metadata between your graphs and lookups.
Improved export/import of sandbox configuration
The export of sandbox configuration will now export name of the sandbox as well similarly like is done for “global” configuration export. This will allow the sandbox-specific configuration to be imported on another Server easily as it will allow the receiving Server to create a copy of the sandbox right aways rather than requiring the sandbox to be present before importing the configuration.
Simplified cluster configuration
We’ve improved the configuration management for CloverDX Cluster. It is no longer necessary to provide a complete jGroups configuration on each node. Nodes in the cluster will apply reasonable default settings and share them across all nodes.
This is especially useful for initial_hosts property which will now be configured automatically by the Server allowing you to easily automate the deployment since it is no longer necessary to know the host names in advance.
It is still possible to fine-tune the configuration if needed – for example to change ports or other settings.
New CTL functions
We’ve added four new CTL functions in CloverDX 6.5:
- getDayOfWeek
- validateCreditCard
- validateEmail
- validatePhoneNumber
The last three functions replicate the functionality of Wrangler validation steps allowing you to get the same powerful data validation in your CTL code without having to build a Wrangler job.
New REST API to query Server users
New REST API endpoint /users will allow you to list user accounts configured on the Server.
To help you install or upgrade to this version, we've prepared a simple checklist:
Before You Upgrade
- Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
- Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
- There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
- Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.
Designer Upgrade
- Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
- Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
- Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.9 to 5.0). If you're on our maintenance program, we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.
Server Upgrade
- Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
- Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
- Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
- Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.9 to 5.0) and we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license key from there.
| Release | Compatibility/Upgrade notes, Features & Fixes | Published | Download | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| August 13, 2024 | Documentation Download CloverDX 6.5 6.5.1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Wrangler Improvements
Kubernetes
Aggregate Component
API Improvements
Miscellaneous
Security
Fixes
Compatibility
|
June 25, 2024 | Documentation Download CloverDX 6.5 6.5.0 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- For Developers (Improvements most useful for developers bringing new functionality or optimizations in data transformation and orchestration)
- For Administrators (Improvements or features that will help setup, install, administer and manage the platform)
- For Support (Helps staff supporting the production environment to identify and escalate potential problems or avoid such)
- For Security (Improvements and changes relevant to security focused staff – sys admins and developers alike)