Customer Portal

In this release we are continuing our theme of CloverDX Libraries and making significant improvement by publishing CloverDX Marketplace. CloverDX Marketplace is a curated collection of CloverDX Libraries that implement various functionality – connectors, utilities and more. We are starting with 13 libraries and will be expanding the Marketplace with more libraries over time to help you use your CloverDX more effectively.

We are introducing official CloverDX RPM repository which will allow you to deploy your CloverDX instances with a single command on compatible operating systems (launching with Red Hat Enterprise Linux and CentOS Linux).

In this release we are also adding new component to help you work with HL7® v2 data – HL7Reader. This component can be used in data validation, mapping and reporting use cases around HL7® data. The component has been designed to be robust and is capable of handling malformed or non-standard files to allowing you to process as much data as possible.

To further help with your CloverDX deployments, we are introducing a new configuration file properties allowing you to set master password programmatically. This will allow you to improve your DevOps processes and make your deployments easier and fully automated.

We've made many smaller updates on backend – we now support PostgreSQL 14, deployments to IBM Open Liberty 22 and Salesforce components can now use new API v55.2.0 and more.

And as usually, we upgraded several internal dependencies to fix various security vulnerabilities that have been discovered by our automated scans or reported to us.

New features in 5.16

CloverDX Marketplace

CloverDX Marketplace is a public repository of high-quality libraries you can easily use in your solutions to save development time by focusing on your core use case rather than building various connectors or other commonly used subgraphs.

You can easily access the Marketplace in two different ways – via its website https://marketplace.cloverdx.com/ or directly as a Repository in CloverDX Server.

The web app allows you to quickly browse and explore Marketplace content without having to use CloverDX Server Console. You can also download all libraries and install them directly in your Server or examine their source if you'd like to know how they are implemented.

When working with CloverDX Server, you will be able to access a new built-in Repository hosted in Azure Blob. You can then install libraries from the Marketplace with just few clicks:

All libraries in the Marketplace are free to use and provided on “as is” basis. To see how they work or to improve them, you can easily access their source code. This will also help you learn how to use various CloverDX  features that were used when building the libraries.

We will be adding more libraries over time, so keep checking our Tech blog for news about library releases in the Marketplace.

 

CloverDX Libraries

We continue improving Library functionality to allow you to build more powerful and useful libraries. In this release, we are adding ability to include additional objects in libraries in addition to subgraphs.

You can now package graphs, jobflows, metadata or even Data Services in your libraries. These improvements unlock interesting use cases especially when combined with the ability to create your own Repository for your organization.

For example, you can publish a set of Data Service endpoints that provide statistics of jobs running on CloverDX Server, cleverly combining internal data for its consumers like DataDog or Splunk. It is now possible to quickly install and deploy such Data Services on the Server using just few clicks.

Ability to package jobflows can be used in situations where larger frameworks are bundled within a library. For example, library may implement a Unit testing framework for CloverDX projects which you can just schedule via standard CloverDX Server automation features (Schedulers, Event Listeners) and have then run automatically, making CI/CD pipelines a smooth sailing.

Packaging metadata on the other hand opens possibility to version control data structures provided or required by your source and target applications. Metadata libraries may be a leap forward to best practice in Data Governance use cases.

Each library can now also carry its own documentation in Markdown format. When you click on such library in Server's Library module, you will be able to view the documentation giving you details of what is included in the library.

CloverDX RPM repository

Installation of CloverDX Server is now whole lot simpler thanks to our new official RPM packages. These packages are compatible with Red Hat Enterprise Linux and CentOS Linux distributions. CloverDX Server can now be installed with default (non-production) settings using simple one-liner. Our RPM repository contains all necessary dependencies to be able to run CloverDX on an open-source stack with Apache Tomcat 9 and Eclipse Temurin JDK.

To get more information about how to install CloverDX from the RPM repository, visit our Linux packaging documentation.

 

Support for parsing of HL7® medical standard data

We are continuing to expand our connector library and introducing a new incubation component – HL7® Reader – that will allow you to parse HL7® v2 documents. The component allows you to ingest HL7® data in many different versions – starting with v2.1 up to and including v2.9 – and convert into CloverDX variant data type (which is essentially a JSON document).

The component has been designed to be resilient and handle even malformed data since it is quite common for the files to use customized formats and deviate from the standard. Mapping HL7 data to variant will then allow you to pick the message apart and process only the parts that you need. You can implement advanced validation, data conversions, reporting and more by using functionality built into CloverDX platform.

 

Other updates

Salesforce components use new API

We've upgraded the API that that is used by SalesforceReader and SalesforceWriter components. For the reader this means that you can, for example, use SELECT FIELDS(ALL), SELECT FIELDS(CUSTOM) and SELECT FIELDS(STANDARD) notation instead of explicitly listing fields in your SOQL.

New cloud releases

We've upgraded our AWS Marketplace and Azure Marketplace offers to PostgreSQL 14. On both platforms this change requires upgrade of the underlying template. This means that if you are running your CloverDX instances from either marketplace, you will not be able to directly upgrade. Instead, you will have to perform additional steps – detailed instructions available for each platform:

 

 

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
  • Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.9 to 5.0). If you're on our maintenance program, we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
  • Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.9 to 5.0) and we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license key from there.

 

Release Compatibility/Upgrade notes, Features & Fixes Published Download

SpreadSheetReader Improvement

Configure POI buffer limit
CLO-25107 Improvement

Security

Server audit log contains master password / secure params in plaintext
CLO-23822 Security
Server audit log contains user console login password in plaintext
CLO-25724 Security
Plaintext secure strings in server’s Overview tab
CLO-25056 Security
May 02, 2023 Documentation Download CloverDX 5.16 5.16.2

Security Fixes

CVE FIX - Upgrade commons-text to v1.10.0
CLO-24743 Security
CVE FIX - Upgrade kafka-clients to v2.8.2
CLO-24604 Security
CVE FIX - Upgrade groovy-all to v2.5.18
CLO-24601 Security
CVE FIX - Upgrade woodstox-core to v5.3.0
CLO-24600 Security
CVE FIX - Upgrade hadoop-common to v3.3.4
CLO-24599 Security
CVE FIX - Upgrade json-smart to v2.4.8
CLO-24598 Security
CVE FIX - Update snakeyaml to v1.33
CLO-24442 Security
CVE FIX - Upgrade jsoup to v1.15.3
CLO-24413 Security
Security Audit of Data Apps/Dashboard - update uri-template-lite to 22.9.0
CLO-24504 Security

Fixes

DatabaseWriter in batch mode loses last rejected record
CLO-24649 Fix
Data Service HTTPS Connector - client certificate is not required
CLO-24779 Fix
Syntax error in Excel spreadsheet causes SpreadsheetDataWriter to fail
CLO-24724 Fix
Worker doesn't start if it isn't possible to detect name of the application container
CLO-24638 Fix
Slow log4j logging in designer
CLO-24570 Fix
CloverDX Marketplace repository fails to load when user uses Environment Variables
CLO-24585 Fix
CloverDX Marketplace repository fails to load when user uses Azure CLI
CLO-24584 Fix
CloverDX Marketplace repository fails to load - ClientAuthenticationException: DefaultAzureCredential authentication failed
CLO-24575 Fix
October 24, 2022 Documentation Download CloverDX 5.16 5.16.1

Public Marketplace and Repositories Improvements

Merge "Upload Library" and "Libraries repository" together
CLO-23413 Improvement
Sort libraries in libraries view
CLO-23697 Improvement
Extend Libraries to allow additional entities - Graphs, Jobflows
CLO-23947 Improvement
Extend Libraries to allow additional entities - DataServices
CLO-24009 Improvement
Extend Libraries to allow additional entities - Metadata
CLO-24010 Improvement
Add CloverDX public library repository to Server
CLO-24151 Improvement
Add ability to browse CloverDX repository in Designer
CLO-24152 Improvement

HL7Reader Component

HL7 reader
CLO-23835 Improvement
HL7Reader - allow schema override
CLO-24421 Improvement

Miscellaneous

Increase record and field size limit
CLO-23523 Improvement
HTTPConnector - add a delay between retries
CLO-19987 Improvement
Master password auto import
CLO-22301 Improvement
Update non-bulk Salesforce components to make use of the new Salesforce API version
CLO-23032 Improvement
Increase max limit size for Support package
CLO-24014 Improvement
Use full version of POI
CLO-24153 Improvement
Implement shared filesystem availability monitoring for cluster nodes
CLO-24272 Improvement

Deployment Stack

Upgrade supported version of PostgreSQL server database to 14
CLO-23677 Improvement
Upgrade to postgre 14 in azure offer
CLO-24015 Improvement
Azure marketplace - use Postgres Flexible Server
CLO-24107 Improvement
AWS marketplace - use Postgres 14
CLO-24436 Improvement
Upgrade supported version of Open Liberty 22
CLO-23689 Improvement
Remove Jetty and use Embedded Tomcat - in Server
CLO-23496 Improvement
Remove Jetty and use Embedded Tomcat - in Designer
CLO-23797 Improvement

Security

CVE FIX - Upgrade poi to v5.2.2
CLO-23157 Security
CVE FIX - Upgrade google-http-client-gson to v1.42.0
CLO-23522 Security
CVE FIX - Upgrade aws-java-sdk to v1.12.279
CLO-24141 Security
CVE FIX - Upgrade postgresql to v42.4.1
CLO-24254 Security
DataApp login screen can be seen as malicious by BitDefender
CLO-23319 Security

Fixes

We do not comply all dependencies for Salesforce's connector library
CLO-16878 Fix
Log level execution property is overridden when launching graph from Designer
CLO-18685 Fix
ExecuteScript - enviroment variables values cleaned/lost when editing input mapping
CLO-22921 Fix
Remove remote persistence
CLO-23147 Fix
File not found Error while selecting the subgraph from library in Execution History
CLO-23161 Fix
Metadata extraction from SQL: SQL cannot start with a comment
CLO-23343 Fix
Remove jakarta.faces-api dependency from server
CLO-23489 Fix
JobInspector wrong visualization of too width components
CLO-23640 Fix
Project conversion can lose data
CLO-24106 Fix
Unhandled exception when adding ":" to the path when exporting library
CLO-24168 Fix
Performance issues in Designer 5.15 when opening graphs
CLO-24283 Fix

Compatibility

Azure marketplace - use Postgres Flexible Server
CLO-24107 Compatibility
Drop configuration property rest.api.absoluteUrls
CLO-23628 Compatibility
Remove remote persistence
CLO-23147 Compatibility
September 20, 2022 Documentation Download CloverDX 5.16 5.16.0
  • For Developers (Improvements most useful for developers bringing new functionality or optimizations in data transformation and orchestration)
  • For Administrators (Improvements or features that will help setup, install, administer and manage the platform)
  • For Support (Helps staff supporting the production environment to identify and escalate potential problems or avoid such)
  • For Security (Improvements and changes relevant to security focused staff – sys admins and developers alike)