New features in 6.4

New output types for Data Apps – html and images

In this release, we extended our Data Apps to allow you to return images and html files directly. Both will be displayed in the Data App result right away. This will allow you to easily implement more interactive apps that provide additional controls to the user or to deliver quick and visual reports directly in the app without requiring users to download a file to display in 3rd party app.

Step groups and step conditions in Wrangler

You can now group your steps in your Wrangler jobs into step groups. Groups allow you to organize your jobs for better readability by putting related steps together making. This makes the job easier to navigate and understand. Groups do not change the flow of the transformation – it is still a linear sequence of steps.

Another feature we added is the ability to add step conditions to your steps or groups. A step condition allows you to selectively enable or disable a step or group based on data being processed by Wrangler.

This allows you to represent some transformations more naturally without having to resort to more complex formulas with nested IF statements.

REST API for library management

In this release we expanded the CloverDX REST API with several endpoints that will allow you to query and manipulate libraries installed on the Server. This kind of API is very useful when following DevOps practices for deployment automation – whether it is deployment of the CloverDX Server instances or of the new or updated projects to the Server.

With the new API you will have fuller control over the environment, and it will allow you to implement more complex automations or processes.

Docker image in Docker Hub

Docker is an easy way of installing CloverDX Server while ensuring that it is configured in reasonable way. Starting with CloverDX 6.4, we are publishing official Docker images of CloverDX Server in our Docker Hub repository: CloverDX on Docker Hub.

Using the official image will make it easier for you to install and upgrade CloverDX Server. It will help especially if you are following DevOps practices and wish to automate your CloverDX deployment.

Native builds for Apple Silicon

In this release we are adding new download artefacts suitable for all Mac users who have newer machines with Apple CPUs (M1, M2 and M3 CPUs). You can now download a special build that does not require Rosetta 2 compatibility layer when running on the Apple CPUs. This in turns leads to lower memory footprint and better performance.

We will still be providing builds for Intel CPUs at least until Q3 2024. After that point we will no longer provide new Intel-compatible builds. The Intel builds of all releases before CloverDX 6.7 will be provided and maintained as per our support policy.

Smaller updates

Formula editor usability

We’ve improved the Wrangler’s formula editor – the content assist will now also offer list of functions and their parameters. The editor will also provide better feedback about errors when you are editing the formula – it will underline the incorrect syntax to allow you to quickly spot and fix it.

Dark Mode in Designer

We listened to frequent requests from our customers and worked on improving and polishing the Dark Mode in CloverDX Designer. With this release, you can select Dark Mode either for CloverDX Designer or have it follow your OS settings. Dark Mode works well in Windows as well as on macOS.

Support for PostgreSQL 15 and for Microsoft SQL Server 2022

Starting with this release, you can run CloverDX Server with its system database hosted on a PostgreSQL 15 or MS SQL Server 2022.

With this change, we also updated the default configuration for our AWS Marketplace and Azure Marketplace offers – they will both use PostgreSQL 15 as the system database.

New CTL functions and CTL multi-line strings

We’ve added many new CTL functions which will make it easier for you to build your transformations:

• evalExpression: to allow you to evaluate a string as a CTL expression.
• escapeXML and unescapeXML: to allow you to work with XML fragments that may contain XML escape sequences like &lt; for “<” (less than) sign etc.
• formatMessage and formatMessageWithLocale: to help you compose string using a string template with arguments and formatting hints.
• appendAll: to append map entries from one map to another map.
• currentTimeMillis: to return current epoch in milliseconds (i.e., same as date2long(today())).
• string2timeunit: to convert textual name of a time unit to its CTL “enum” value.
• printLog: to print into custom logs defined in your logging configuration.
• randomDecimal: to generate random decimal numbers.
• md5HexString, sha256HexString, sha1HexString: new hashing functions to produce hex representation of the hash directly instead of byte values.

At the same time, we added support for multi-line string blocks in CTL. These start and end with """ (three consecutive double quote characters). A multi-line string allows you to write complex strings with simpler and cleaner syntax. See our documentation for more details and examples.

RunGraph, SystemExecute, and TableauWriter components deprecated

All three components are now deprecated. While they remain part of the product with unchanged functionality, we recommend you update your to not use these components anymore (see below). These components will be removed in a future release.

If you already have a job that uses RunGraph, please switch to jobflow components ExecuteGraph or ExecuteJobflow. Both offer better interface which allows for better control of child job parameters and its execution while also providing much better ability to handle errors in child jobs.

Similarly, if you are already using SystemExecute, please switch to jobflow component ExecuteScript which offers a better interface for running your scripts.

TableauWriter component does not have any direct replacement. If you need to use this, component, please let us know (e.g., by contacting your account rep or via a support ticket).

Simple HTTP API deprecated

We’ve deprecated CloverDX Server’s Simple HTTP API and disabled it by default. This is a legacy API that provided access to various management endpoints for the Server – running jobs, getting their statuses, working with sandboxes and more.

The API has been completely superseded by CloverDX Server REST API which provides more robust and powerful endpoints compared to the Simple HTTP API.

If needed, the API can be enabled using http.api.enabled in your CloverDX Server configuration. Please keep in mind that the API will be removed in a future release.

Logging improvements in CloverDX Server

We’ve added a new configuration property for CloverDX Server that allows you to change the location of CloverDX logs with one simple step instead of modifying log4j configuration directly. You can use clover.logging.dir to change the target location of all log files produced by CloverDX.

We also improved default settings with regards to log rolling and retention. Log files will now roll daily instead of after 5 MB and will have a date in their name to help you find the correct log file.