How to write custom aggregate function

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

madan_clover3
Posts: 25
Joined: Tue Jun 10, 2014 10:39 am

How to write custom aggregate function

Postby madan_clover3 » Tue Dec 03, 2019 11:04 am

Hi,

I am trying to implement custom aggregation logic, something like "concatinate string values of a field from all the records". I did not find a provision to write custom aggregate functions. AggregateProcessor has FunctionRegistry, it registers aggregate functions supported by clover but no option to register custom functions. Is it intentionally blocked or missed ? Please descirbe what are the problems foreseen with custom functions.

I can use Denormalizer to achieve my functionality (string concatination from multiple rows). But I am already using aggregate for other aggregation requirements. If I can add custom function with existing functions I can avoid extra sorting and extra components in the graph. Write now the only option I see is copy aggregate classes and change its behaviour to accept custom functions.

I am using clover.engine version 4.6.0. Checked in version 5.1.1 as well. Aggregate behaviour is same in both versions.

Please suggest.

cholastal
Posts: 137
Joined: Tue Sep 01, 2015 1:22 pm

Re: How to write custom aggregate function

Postby cholastal » Mon Dec 09, 2019 4:27 pm

Hi,

First of all, the open source engine is officially not supported and also discontinued, version 5.1.1 is the latest ever to be released. However, here are some hints.

1. Denormalizer is definitely a cleaner way in terms of not altering the source code.

2. If you need to re-use the Denormalizer along with certain other "servicing" components, you can use subgraphs (v. 5.1.1) to make the whole logic reusable instead of creating the same pattern over and over.

3. If you still intend to rather code the function for the aggregator, you should be able to achieve that by creating your own class (the function) that extends AggregateFunction and register it in the FunctionRegistry constructor. This way your function will look like every other aggregator function to the Aggregator.

The reason why there is no easier way to create custom aggregation functions is that the Aggregator simply wasn't designed to be extendable with custom functions.

Please note that altering the source code is without any warranty and we cannot support any issues related to the open source engine and/or the custom modifications.

Best regards.

---
Lukas Cholasta
CloverCARE Support
CloverDX

Visit us online at http://www.cloverdx.com


cron