Cluster

Improved handling of "running" jobs on cluster node that was lost

Improvement CLO-5721 3.5.5

Fixes

Fixed: FastSort was losing records for extremely small run size	Fix CLO-5672 3.5.5
Fixed: URL dialog did not work for SFTP connection with certificate-based authentication	Fix CLO-3577 3.5.5
Fixed: Readers were showing checkConfig warning for port protocol	Fix CLO-5532 3.5.5
Fixed: Writers did not decode escape sequences in sandbox URLs	Fix Compatibility CLO-5660 3.5.5

Features & Improvements

Improved http multipart entities support in HTTPConnector component. Extended multipart entities mapping options.

CLO-5037 3.5.4

Fixes

Fixed: syntax error while using multiple quote characters in DBOutputTable query	Fix CLO-5574 3.5.4
Fixed: cannot delete value for integer type attributes in component editor	Fix CLO-5558 3.5.4
Fixed: problem with simultaneous run of two Success components	Fix CLO-5463 3.5.4
Fixed: multipart entities attribute mapping does not work	Fix CLO-5036 3.5.4

Features & Improvements

Improved error reporting in case of usage of unsupported jdbc.dialect in the CloverETL Server configuration

CLO-4860 3.5.3

Fixes

Fixed: deletion of whole sandbox in case of empty fileURL in DeleteFiles component	Compatibility Fix CLO-5114 3.5.3
Fixed: output dictionary was not working correctly in clustered graph	Fix CLO-5364 3.5.3
Fixed: first line in table was not visible and mapping lines were positioned incorrectly on OSX Yosemite	Mac OSX Fix CLO-5292 3.5.3
Fixed: missing files from root of project when exporting project to the sandbox	Fix CLO-5219 3.5.3
Fixed: designer was not starting in some cases on OSX	Mac OSX Fix CLO-4681 3.5.3
Fixed: strings trimmed for fixed-length metadata	Fix CLO-5225 3.5.3
Fixed: UniversalDataWriter was not creating directories in case of fileURL specified using dictionary	Fix CLO-2551 3.5.3
Fixed: problem with unregistering JDBC driver on JBoss	Server Fix CLO-4807 3.5.3
Fixed: minor issues with UniversalDataReader/UniversalDataWriter and fixed length metadata	Fix CLO-5083 CLO-3955 3.5.3
Fixed: SpreadSheetReader - problems with default date format	Compatibility Fix CLO-4866 3.5.3
Fixed: XMLWriter was not correctly handling zero input records	Fix CLO-4851 3.5.3
Fixed: ComplexDataReader selector rules order is not kept as entered	Fix CLO-5118 3.5.3
Fixed: Parameter override values are not passed to worker graphs	Fix CLO-5362 3.5.3

IBM Infosphere Virtual MDM integration

Integrated IBM Virtual MDM components (former Initiate components). Delivered as a separate update site and ZIP file with engine plugin for server.

New CLO-3769 3.5.2

Features & Improvements

Improved implementation of cluster asynchronous reliable messaging	Cluster Improvement CLO-4529 3.5.2
Updated JDK bundled with Designer installer to JDK version 1.7.0_51	Designer Compatibility CLO-4504 3.5.2
Started bundling JDK in Designer package for OS X (JDK version 1.7.0_51)	Max OSX Compatibility CLO-4004 3.5.2
DBOutputTable support for “RETURNING” in PostgreSQL (as of 8.2)	Improvement CLO-2887 3.5.2

Fixes

Fixed Archivator task deleting server instance run history incorrectly	Server Fix CLO-4471 3.5.2
Fixed: The word "where" in an SQL comment used in Dynamic metadata query was incorrectly interpreted as part of the query	Fix CLO-4238 3.5.2
Fixed: SystemExecute was closing error file descriptor during execution	Fix CLO-4085 3.5.2
Fixed: Launch Service couldn't use Compiled mode unless “Save run record” was enabled	Server Fix CLO-3899 3.5.2
Fixed: DBLookupTable had incorrectly synchronized activeLookups variable	Fix CLO-3895 3.5.2
Fixed: AddressDoctor was missing “RangesToExpand” and “FlexibleRangeExpansion” attributes	Fix CLO-3894 3.5.2
Fixed: dash ("-") was passing through isNumber function as a number	Compatibility Fix CLO-3856 3.5.2
Fixed: Designer was freezing when parameter files were linked in “wrong” order – now it works for any order	Fix CLO-3697 3.5.2
Fixed SpreadsheetDataWriter dictionary streaming mode was not working	Fix CLO-3278 3.5.2
Fixed: attempt to connect to Microsoft Access (via Designer) killed the whole server	Server Fix CLO-2707 3.5.2
Fixed: StackOverflowException causing EULA not being displayed in Help>License Manager	Fix CLO-3843 3.5.2

AddressDoctor integration

Added support for CAMEO data in AddressDoctor	News CLO-3105 3.5.1
Added "MaxItemCount" for address elements standardization in AddressDoctor	New CLO-3108 3.5.1
Additional options for AddressDoctor configuration: EnrichmentGeoCodingType , MaxResultCount, ElementAbbreviation and StandardizeInvalidAddresses settings have been added, PRESERVE_INPUT option also added to PreferredLanguage and PreferredScript.	New CLO-3377 3.5.1
AddressDoctor missing attributes on output	Fix CLO-3163 3.5.1
Removed some of the irrelevant AddressDoctor options in the Parameters config	Improvement CLO-3162 3.5.1

Features & Improvements

View data dialog: 'Truncate strings and byte arrays' setting is now remembered	Improvement CLO-3379 3.5.1
When parsing deprecated format of parameter file (.prm from versions prior to 3.5), invalid lines are now ignored	Compatibility Fix CLO-3338 3.5.1
Inline CTL code (`some code`) can now be used inside DB password field	Compatibility New CLO-3454 3.5.1

Fixes

Fixed Database Browser leaving JDBC connections open	Fix CLO-3596 3.5.1
Fixed performance of SimpleGather	Fix CLO-3538 3.5.1
Fixed executeGraph not propagating the nodeID attribute properly, so the load-balancer decision was applied for the execution	Cluster Fix CLO-3461 3.5.1
Fixed NullPointerException when attempting to run a graph that uses a secured parameter inside an unsecured parameter	Fix CLO-3455 3.5.1
Fixed sometimes incorrect navigation from Execution History to Sandboxes using the sandbox icon	Server Fix CLO-3378 3.5.1
Parameters in obsolete format (created prior to CloverETL 3.5) are now converted every time such parameters/or graph are saved	Compatibility New CLO-3257 3.5.1
Server users with an underscore in their names could not create Server projects from Designer	Server Fix CLO-3240 3.5.1
Fixed debug files being always created when running graph via RunGraph	Fix CLO-3171 3.5.1
Better error message when XMLWriter mapping dialog can't be opened (e.g. because of a missing field)	Improvement CLO-3037 3.5.1
Fix: DB Connection dialog - after selecting a different connection, values remain the same.	Mac OS Fix CLO-3024 3.5.1
Passwords are masked in component tooltips showing component's parameters	Improvement CLO-3011 3.5.1
Launch Services test page: fixed Test buttons and back navigation	Server Fix CLO-3009 3.5.1
Leftover temp files are now properly removed when uploading large files via designer-server integration	Designer-Server Fix CLO-2996 3.5.1
SpreadsheetDataReader: Fixed handling of null rows when using horizontal mapping and in-memory read mode. (component ending in an infinite loop)	Fix CLO-2983 3.5.1
XMLWriter: Fixed NullPointerException when more sorting keys are specified than actually used	Fix CLO-2924 3.5.1
Fixed invalid placeholders in "graph finished" email template	Server Fix CLO-2846 3.5.1
Fixed Pivot component producing wrong data for string keys	Fix CLO-2798 3.5.1
Fixed hidden menus contributing to an Invalid Menu Extension when opening graph	Fix CLO-1561 3.5.1
Fixed "Export graph to HTML" when complete project is select - images being created mutliple times, errors, etc.	Fix CLO-3611 3.5.1
Fixed HP Vertica DB connection failing to execute meta-function	Fix CLO-3287 3.5.1
Fixed DBExecute not reporting errors properly when ErrCode and ErrText are present with auto-filling.	Fix CLO-3075 3.5.1
Fixed how server determines "user.home" property: e.g. Weblogic on Windows	Server Fix CLO-2851 3.5.1
CTL2: function removeNonPrintable() incorrectly removes also all non-ASCII characters (unicode) on Java 6 - see details for more info	Fix CLO-1814 3.5.1

Cluster

Cluster status monitoring	Cluster New 3.5.0-M1
Database connection robustness	Cluster New 3.5.0-M1
JGroups port discovery and negotiation	Cluster New 3.5.0-M1

Server/Cluster Administration

Configuration migration	Server New 3.5.0-M1
Monitoring UI	Server New 3.5.0-M1

Connectors

Mongo DB	Engine New 3.5.0-M1
Exasol connectivity	Engine New 3.5.0-M2
HTTP Connector - streaming and binary data support	Engine Improvement 3.5.0-M2
JSONExtract	Engine Improvement 3.5.0-M2
XMLWriter: Support for CDATA	Engine New 3.5.0
XMLReader: Mapping editor recognizes schema from xsi:xmlSchemaLocation	Engine New 3.5.0
HP Vertica connectivity	Engine New 3.5.0

Security

Audit log	Server New 3.5.0-M1
SFTP key-based authentication	Engine Improvement 3.5.0-M1
Secure parameters	Server New 3.5.0-M2
Configurable value encryption (BouncyCastle)	Server New 3.5.0-M2
Password encryption in Server configuration files	Server New 3.5.0

Data Quality

Data Quality Package + Validator	DataQuality MajorFeature 3.5.0-M2
Validator: If-then-else conditional validation	DataQuality New 3.5.0
Validator: Output phone number formatting in PhoneNumber rule	DataQuality New 3.5.0
Validator: Rule descriptions	DataQuality New 3.5.0
Validator: CTL return codes	DataQuality New 3.5.0

Parameters

Parameter files use XML	Engine New Compatibility 3.5.0-M2
Parameters have description	Engine New Compatibility 3.5.0-M2

BigData

Tested CDH 4.4.0 & 4.5.0	Engine 3.5.0
Tested Impala (JDBC)	Engine 3.5.0

CTL

CTL error reporting	Engine Improvement 3.5.0
getRawParamValue()	Engine 3.5.0

Known Issues & Compatibility

Changed behavior of string functions (e.g. isNumber) failing on empty or null strings.	CLO-945 Compatibility 3.5.0-M1
Changed behavior of base64byte() function	CLO-884 Compatibility 3.5.0-M1
Changed record count field to "long" in the Profiler - might produce incompatible integer/long in metadata coming from ProfilerProbe and ExecuteProfileJob.	CLO-884 Compatibility 3.5.0-M1
CloverDataReader/Writer data files not compatible between version 3.4 and 3.5 when these components are used in a mix of jobflows and transformation graphs. Read comments in related issue.	Jobflows CLO-1382 Compatibility 3.5.0-M2
Functions ceil() and floor() return decimal instead of number for parameters of type decimal	CTL2 CLO-2005 Compatibility 3.5.0-M2
Some conversion functions return NULL instead of throwing an exception	CTL2 CLO-1586 Compatibility 3.5.0-M2
Some date functions return NULL instead of throwing an exception	CTL2 CLO-1584 Compatibility 3.5.0-M2
Null 'case' values in switch() are now allowed	CTL2 CLO-737 Compatibility 3.5.0-M2
Function get() for lookups now always returns NULL for keys not found or NULL key, exception if unknown field is requested. Before, there was a difference between compiled and interpreted mode - returning null or exception respectively	CTL2 CLO-1582 Compatibility 3.5.0-M2
Fixlen data reading changed in regards to automatic trimming. Read the related issue	CLO-1405 Compatibility 3.5.0-M2
Built-in MySQL JDBC driver updated to version 5.1.26 (formerly 5.1.22). The new version is optimized for MySQL 5.6.	CLO-1886 Compatibility 3.5.0-M2
FastSort defaults have been changed. Maximum open files is now 1000 by default, Number of sorting threads is 1	CLO-1775 Compatibility 3.5.0-M2
Old version of Designer on Windows is automatically uninstalled when new version is installed (only versions 3.5.0-M2 and newer are uninstalled)	Designer CLO-1028 Compatibility 3.5.0-M2
Handling of null/empty values in Validator	Data Quality CLO-1162 Compatibility 3.5.0
ParallelReader error metadata no longer uses integer for record number	Data Quality CLO-1162 Compatibility 3.5.0
ParallelReader error metadata no longer uses integer for record number	CLO-2509 Compatibility 3.5.0
EmailReader: Fixed inconsistent behavior on POP3 vs. IMAP	CLO-2501 Compatibility 3.5.0
Validate names of graph parameters	CLO-2235 Compatibility 3.5.0
XMLExtract: explicit mapping to have priority over implicit automap by name	CLO-2029 Compatibility 3.5.0
RecordFilter interface declares two isValid() methods	CLO-2668 Compatibility 3.5.0
str2bits does not check validity of its input string	CLO-2022 Compatibility 3.5.0