New features in 6.6

Data Apps

We’ve made a few smaller improvements in Data Apps that together allow you to better customize how the apps work. The first improvement allows you to redirect an app to a different URL (or to another app) when the app runs. The second improvement allows you to hide apps from the App Catalog while still allowing the users to run them and finally, you now have a bit more control over the app output display.

Combined, these changes allow you to essentially create wizard apps with multiple pages thus enhancing the overall usability of the Data Apps.

Documentation

We’ve reorganized the documentation in CloverDX 6.6 to help users find what they are looking for more effectively. The new documentation is built around user roles rather than products.

Security

It is now possible to enable encryption of temporary files that are created during job execution. This will make it easier to satisfy various requirements for data security and protection. Temporary file encryption can be enabled by setting execution property encrypt_temp_files on a sandbox or on a job.

To allow you to encrypt even the files that you create manually, you can enable encryption on file produced by CloverDataWriter component. CloverDataReader component can then read such encrypted files.

Additionally, we’ve added support for dynamic secure parameters. Secure parameters allow you to store various secrets like password, access keys or similar in an encrypted parameter that has its value hidden from various views that display job parameters.

You can now combine this with the ability to have dynamic parameters helping you in cases where you need to dynamically determine the secret – for example based on environment configuration or other job parameters.

Data Manager – batching

Batching in Data Manager allows you to split your data sets into smaller groups of records – batches – based on the data the rows contain. For example, you can configure your data set with customer to be batched based on the customer state, invoices data set based on month or any other value in your data or even batch based on the incoming file name and so on.

The batch membership is computed in real-time based on row data, so you can even change the batch row is in if you change the value of the batching key column in that row.

Data Manager tracks the batch status in a simple way giving you a quick overview of how far your work on given batch is.

Additionally, when processing batched data after the Data Manager, you can opt for processing complete batches only. This will push Data Manager data editors and approvers to complete batches as they work on them rather than starting batches that are not completely processed. This setting is part of DataSetReader component configuration.

Data Manager – lookups

Lookups in the Data Manager are an extension of the list functionality from CloverDX 6.5. While lists allowed you to restrict the values in specific columns to a set of values, lookups extend this to allow you to add additional data to each value. These additional columns in your lookup are shown in the Data Manager when you edit a value in a cell. You can then use this data to help you pick the correct value of the lookup since in many use cases just the name of the item may not be enough to figure out the correct value.

As with lists, the lookups are managed by the Server and work in a very similar way to lookups. It is therefore very easy to extend an existing list to a lookup with additional columns.

Data Manager – usability and performance

We’ve made many changes in the Data Manager to improve the overall user experience when using the app. Among the bigger changes visible right away is the ability to filter and search within the data set when editing data:

We’ve added support for schema evolution in Data Manager’s data sets. This means that you can change the data set layout even if the data set contains data. This will allow you to update your data sets as the requirements for your data process evolve.

Other

Wrangler Fill down step

We’ve added a new Wrangler step called Fill down. This step allows you to fill missing values in a column based on the previous value. This is useful especially when dealing with Excel files that use merged cells as this step allows you to populate the rows which do not have value in the cell that is part of the merged cell in Excel.

Improved support for different screen scaling factors in Designer

Many modern monitors are high-resolution and require increased scaling for UI elements to make sure they are not too small – e.g., everything is scaled to 150% or even more. We’ve updated the Designer to work much better when graphs are moved between machines with different display scales – you should no longer see components that overlap or that are too far apart. Instead, the text within components will scale better to keep the visual layout as you’d expect.

Platform support

In this release we are continuing to improve our support for Java 17 deployment. We’ve upgraded the JBoss stack to that you can now run CloverDX on Red Hat JBoss Web Server 5.4 with Red Hat Open JDK 17. Note that Java 11 is not supported on JBoss anymore.

Currently supported combinations of Java and application servers are the following:

We will be continuing the full transition to Java 17 and Tomcat 10.1 in the next release (CloverDX 7.0). This means that CloverDX 6.6 is the last release with support for VMware tc Server stack (this stack will not be supported anymore in CloverDX 7.0) and Red Hat JBoss Web Server 5.4 stack (you will have to upgrade to JBoss 6 in CloverDX 7.0).

We are adding support for Oracle 23 for CloverDX Server deployments. Both Oracle 21 and Oracle 23 are currently supported.

Compatibility

This is the last release that supports deprecated components SystemExecute and RunGraph. Both components will be removed in CloverDX 7.0. Both have powerful, safer and easier to use replacements – ExecuteGraph and ExecuteScript.

Additionally, we will also be removing several other components which have been deprecated previously. This will affect the following components: DelimitedDataReader & Writer, Filter (old version), FixLenDataReader & Writer, HashJoin, InfobrightBulkWriter, JavaExecute, KeyGenerator, MergeJoin, Sort, SpeedLimiter, XMLWriter (old).

All of these components have more modern and better replacements.