New features in 6.2

Write to any target in Wrangler

With CloverDX 6.2 we are introducing another major functionality update – CloverDX Wrangler can now write to any target system. You can create Data targets and publish them in the Data Catalog.

Once a target is available in the Data Catalog, you can write to it from your Wrangler jobs instead of the default CSV or Excel targets.

Data targets work just like Data sources. They are built in CloverDX Designer so that you can create targets that implement connectivity to any system – whether it is a cloud application like Salesforce, on-prem or cloud database, or even an API.

To make it easier for you to run your jobs in Wrangler while making sure you do not push incomplete data to your target system, we’ve added Test run functionality. Writing to target is disabled during the Test run so you can focus on getting your data right before enabling the writes to populate your target system.

Visual data mapping in Wrangler

Data targets can request data to match a certain structure by prescribing a set of required or optional columns. When using such a target in your Wrangler job, you will have access to a new Mapping mode which allows you to easily map your data to the layout your target requires.

Mapping is interactive and validated as you are working on it. This means you are getting immediate feedback about whether your mapping is complete and whether all required columns are mapped or not. At the same time, mapping does not use any code – it is configured using drag & drop which makes it easy to use.

To make mapping of large number of columns easy, we offer an Auto-mapping functionality. Auto-mapping can suggest column mapping automatically leaving only subset of columns for manual mapping.

Collaboration between business and IT

In CloverDX 6.2 we’ve extended the ability to export your Wrangler jobs and import them into your Designer projects. Jobs imported in this way appear as subgraphs in Designer project. These subgraphs can then be used as any other subgraph as part of a more complex process.

A typical use cases for this are data ingestion or customer data onboarding. In both cases you often need to ingest data in many different formats (for example, from different clients). Building ingestion jobs to read all input formats in Designer can be very time consuming and require a lot of effort from the IT team.

A better solution is to use CloverDX Wrangler in which domain experts create mappings from the source data layouts – they know the data well and can often do this faster than IT-led implementation. Once their Wrangler mappings are ready, they can be exported and shared with the IT team who can integrate them into a larger framework built in Designer.

In this way, it is possible for non-technical domain experts to directly design transformations that are part of a larger process that is implemented in Designer and automated on a Server. This can save significant amount of time during processes such customer onboarding.

New Wrangler steps

We are continuing to expand the functionality that allows you to manipulate your data in CloverDX Wrangler. In this release we added several new steps:

• Five new validation steps to make it easier to verify that your data matches your expectations: Validate if not empty, Validate value range, Validate pattern match, Validate text length, and Validate against list.
• Split column step that allows you to split a single column into multiple columns based on a delimiter.

At the same time, we’ve improved usability of many other Wrangler steps. In particular, we’ve added support for content assist and real-time validation of formula into all formula steps (Calculate formula, Filter rows based on formula, Validate with formula).

Health check for CloverDX libraries

CloverDX libraries can now designate a special Health Check job in their configuration. The health check job is used by the CloverDX Server to verify connectivity and status of the library – the Server runs the job periodically and depending on its status can mark the library “unhealthy”.

The job can be used, for example, for testing whether APIs that the library needs to work are still accessible etc. It is possible to create Monitors in Operations Dashboard to track the status of library health. In this way you can easily monitor the overall health of the connectors that are deployed on your CloverDX Server instance.

CloverDX on Java 17

We are continuing updates of CloverDX stack and with CloverDX 6.2 you can now run CloverDX Server on Eclipse Temurin JDK 17 when using Apache Tomcat 9 application server.

Java 17 is an LTS (Long Term Support) version with Eclipse Foundation providing support at least until October 2027 compared to Java 11 with support ending in October 2024.

At the same time, you can expect better performance when running CloverDX Server on Java 17. Based on our tests, you should see lower overall memory consumption and 5% to 20% improvements in processing time depending on the use case.

We’ve updated our Docker configuration so that it will now use Java 17 by default. This has one huge benefit – Java 17 properly reports performance metrics when running inside the container which allows us to enable the Job queue by default. This will increase the overall stability of CloverDX Server, especially under heavy load.

Smaller updates

Snowflake driver update

We’ve updated bundled Snowflake driver to the latest version. This update allows us to use a newer and simpler layout for JDBC URL when creating Snowflake connections.

SMB (Samba) access change

We’ve updated the smbj library which provides SMB connectivity in CloverDX. The new version introduces stricter controls for anonymous and guest user access to comply with SMB specification. These changes mean that it is no longer possible to use anonymous access to log-in to SMB. As a workaround, SMB administrators can create user “guest” without a password.

Support package and logs

We’ve added the ability for Server Console users to download support packages from all nodes of a cluster with a single click. This can save a lot of time when working on larger cluster deployments.

Additionally, it is now also possible to download logs for complete execution tree from Executions History in Server Console. Again, this can help when solving issues when running more complex job hierarchies.

New REST APIs

We’ve introduced several new API endpoints:

• To abort a running job: /executions/abort and /executions/{runId}/abort
• To download a support package from a single Server node: /server/support-package
• To download a combined support package from all currently connected cluster nodes: /cluster/support-package