Customer Portal

Releases

CloverDX 5.7

Go to Downloads (5.7.1) Latest update April 08, 2021 Documentation

CloverDX 5.7 brings a job queue to better automatically manage load on the Server. The job queue protects the Server from CPU and memory overloading. Additionaly, new user's experience of Server is improved by a guided tour of the Server Console, with the help of a new Data Warehouse example. The installation and statup of the Server is more robust with the help of database update confirmation, better behavior in cluster and better progress reporting. The variant CTL type is integrated deeper into CloverDX.

New features in 5.7

Job Queue

The new job queue handles requests for job execution, and determines when to actually start each job. It manages the overall load on the Server to prevent CPU or memory overloading. This helps the Server especially in case of high production load and in cases of spikes of data processing.

In case of high CPU load or heap memory consumption, jobs are enqueued and wait to be started later. The job queue automatically starts the jobs later when the overall load decreases.

The Server Console monitoring and logging capabilities were extended to provide information about enqueued jobs.  APIs and components were also extended. The job queue manages executions on each cluster node, and the cluster load balancer is aware of job queues of each node to better spread the load.

Job queue in Monitoring

Server Evaluation Improvements

Use experience and learning curve of new users of Server is simplified by a new guided tour of the Server Console. The tour takes the users trough major parts of the Server and explains the main concepts.

To support the guided tour there is a new example. The Data Warehouse example shows a more complex scenario that uses a wider range of Server's functionality - jobs, schedules, listeners, data services etc.

CloverDX Server guided tour

Server Installation Robustness

We have improved the installation and upgrade process of Server to prevent some common issues and simplify the whole process.

User approval before changing database schema

Upgrades to a new Server version can require updates to the Server's database schema. We have modified this process to required user's approval in the Server Console. This helps to guide the users to create backups of the Server's database and to understand that this process might take some time.

For automation purposes, it is possible to disable this manual approval.

Parallel start of cluster nodes when upgrading

When upgrading CloverDX Cluster to a new version, it is now possible to start all nodes in parallel and only one of them will perform changes to the Server's database. So now it's not necessary to manually start the nodes in sequence to prevent database issues.

Reporting of progress during startup

The Server Console shows more detailed progress information during startup of the Server. You can see the where in the startup process the Server currently is, and you can see the Server's logs in the browser.

Facilitate SAML debugging

If single sign-on using SAML is enabled, it can be complex to analyze issues in the login process - communication problems, configuration issues etc. We've improved logging of SAML integration and we provide a way to enable more detailed logging.

CloverDX Server in cloud marketplaces

The AWS and Azure Marketplace offerings with CloverDX Server were improved:

  • it is possible to enable centralized collection of Server logs in a cloud logging service 
  • security was improved by enabling disk encryption by default and implementing recommended security updates
  • it is possible to enable JMX monitoring

Variant improvements

The CTL type "variant" was more integrated into the rest of the CloverDX stack:

  • variant can be used in dictionary
  • ability to use variant in additional container functions, foreach statement and addtional utility functions

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
  • Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.9 to 5.0). If you're on our maintenance program, we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
  • Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.9 to 5.0) and we automatically renew the keys for you. Just go to the License Keys again and copy/paste the license key from there.

 

Release Compatibility/Upgrade notes, Features & Fixes Published Download

Platform Updates & Security

Update JSF to version 2.2.16 		CLO-20786 Security
Remove the XSS vulnerability from the error page 		CLO-20814 Security
Update embedded tomcat to 9.0.41 and 8.5.61 		CLO-20360 Security
Update quartz to 2.3.2 		CLO-20359 Security
Update jackson-databind to 2.12.1 		CLO-20356 Security
Update dom4j to 2.1.3 		CLO-20355 Security
Update derby to 10.14.2.0 		CLO-20354 Security
April 08, 2021 Documentation Download CloverDX 5.7 5.7.1

Job Queue

Job Queue 		CLO-19409 Improvement
Submit time should be provided by jobflow components 		CLO-19112 Improvement
Performance log should contain size of Job Queue 		CLO-18952 Improvement
Simple HTTP API operation "graph_status" provides new Submit time instead of Started time 		CLO-18765 Improvement Compatibility
Updated cluster load balancing algorithm in compliance with Job Queue 		CLO-18714 Improvement
Add 'Submit time' to RunRecord 		CLO-18698 Improvement
CloverDX Server suspend operation should allow to finish enqueued jobs 		CLO-19266 Improvement

Server Evaluation

Welcome page for evaluators at Server UI 		CLO-19217 Improvement
Distribute data warehouse example (DWHExample) with server 		CLO-19161 Improvement

Server Installation

Require user approval before applying system database patches 		CLO-19125 Improvement Compatibility
Prevent corruption of system database during parallel start of multiple cluster nodes 		CLO-19026 Improvement
Reporting of progress during server startup 		CLO-19024 Improvement
Facilitate SAML debugging 		CLO-18910 Improvement
Persist suspended state of Server and sandboxes 		CLO-19031 Improvement

Cloud Marketplaces

Enable VM disk encryption 		CLO-18958 Improvement
Centralized log collection in Cloud 		CLO-18776 Improvement
CloudFormation stack should be finished when CloverDX Server is ready 		CLO-18760 Improvement
Enable JMX for monitoring in Cloud offerings 		CLO-18544 Improvement

Variant

Add variant dictionary entry type to engine 		CLO-18286 Improvement Compatibility
Variant record2map and findAllValues functions 		CLO-18504 Improvement
Variant foreach and in statements 		CLO-18503 Improvement
Dynamic field access functions for variant 		CLO-18360 Improvement
Dictionary type Variant 		CLO-18286 Improvement
Additional container functions for variant 		CLO-18050 Improvement
Allow automatic upcasting in CTL cast function 		CLO-18916 Improvement

Additional Improvements

Ability to specify a certificate using keystore name and/or certificate alias 		CLO-19009 Improvement
Server Console should allow to specify Execution Properties for Data Service jobs 		CLO-19337 Improvement
Update bundled PostgreSQL JDBC driver 		CLO-18860 Improvement
Support PostgreSQL 11.7 as server DB 		CLO-18833 Improvement
Streaming directory listing for local files 		CLO-18845 Improvement
Inherit user.timezone and locale properties from Core to Worker 		CLO-18794 Improvement
Upgrade POI to 4.1.2 version 		CLO-18682 Improvement Compatibility
Easier reading of optional form parameters in Data Service 		CLO-18413 Improvement
GarbageManJob should not check remote job execution - performance issue 		CLO-18331 Improvement
Option to enable CORS filter in CloverDX Server 		CLO-18172 Improvement
DefaultReadOnly property in Worker JNDI 		CLO-17808 Improvement
Return the number of affected rows from DBExecute 		CLO-13531 Improvement
Worker has redundant property worker.ssl.enabled on command line 		CLO-17590 Improvement

Fixes

Spreadsheet components high memory usage with in-memory mode 		CLO-18639
Server on tcServer doesn't start because of scanning of spring-web*.jar 		CLO-19145
Extract metadata from Hive DB fails with 'Method not supported' error 		CLO-18973
JSONExtract Unable to Create Schema from Valid JSON 		CLO-18753
Spring framework 5.1.2.RELEASE Security Vulnerability 		CLO-18736
Nothing happens after attempting to rename externalized metadata 		CLO-18414
NPE in DBExecute when trying to assign multiple IN OUT params 		CLO-18236
Each data service invocation causes unneeded round trips to DB 		CLO-18128
mail.smtp.starttls.enable parameter isn't working for Clover Server e-mail settings 		CLO-9369
CTL: fix parsing of floating point numbers 		CLO-18918 Compatibility
Broken "See Execution Properties" link in file properties in Sandboxes module on Server 		CLO-18809
Change MySQL run record datetime precision to support milliseconds 		CLO-17495
Accept certificate dialog does not work with certificate chains 		CLO-15269
JDBC connections are opened sequentially 		CLO-13377
July 21, 2020 Documentation Download CloverDX 5.7 5.7.0
  • For Developers (Improvements most useful for developers bringing new functionality or optimizations in data transformation and orchestration)
  • For Administrators (Improvements or features that will help setup, install, administer and manage the platform)
  • For Support (Helps staff supporting the production environment to identify and escalate potential problems or avoid such)
  • For Security (Improvements and changes relevant to security focused staff – sys admins and developers alike)