Hi, I’m very new to CloverDX platform and I am currently playing around with the components to figure out how to conduct a complex transformation for large datasets. I want to create custom logs to the log each failure and success in each component I use in during validation and transformation and gather those logs into two files.
Is there a component I can use to help gather these logs into a UniversalDataWriter? Currently I am only able to log failures from each component separately using a CustomJavaTransformer to only flag out the recordID to log. Will really appreciate some help!
Hi Crystal,
That’s a very interesting question—and it actually raises a few more.
To begin with, there are no dedicated components in CloverDX specifically designed for creating custom loggers within jobs. That said, every job execution is automatically logged, and the output includes much of the information you’re likely looking for. This is referred to as the run log, and it’s accessible in Designer at the bottom under the Console tab:
If you’re aiming to capture success/failure at the component level, parsing the run log might actually be a suitable solution.
Additionally, many components in CloverDX support an error (reject) port. You can configure these components to route failed or invalid records through that port, allowing the job to continue rather than failing entirely. You can then process or log those rejected records as needed—commonly using a FlatFileWriter/UniversalDataWriter, Map, or subgraph dedicated to error handling.
If you’re able to share more details about your use case—especially why you’d like to log each component separately—I may be able to suggest a cleaner or more tailored solution. There’s often a more centralized or scalable approach available, especially when working with large datasets.
Kind regards,
Lukas
