Unfortunately, CloverETL does not support emojis by default. No CTL function can help you directly with emojis. Also, I am not sure how the emojis are represented in your files, what encoding is used etc.
Do you have any sample file containing emojis, please? Could you post it here or send via email?
How big your input files are?
Do the input files contain just text or also some structured data?
Are they binary files or plain text files?
There is a java library that could be useful if you decide to write your own java transformation.
You can also use find function of CTL and look for regular expressions
Do you have any sample file containing emojis, please? Could you post it here or send via email? I will attach here.
How big your input files are? Input files can range from KB to a few MB
Do the input files contain just text or also some structured data? The files are structured, typically pipe delimited. The emoji characters have been found in various fields
Other emoji characters have been found in other fields in other files, and our goal is to just raise an error when any such character is found and return the file to the client to be corrected as they see fit, sent back to us, and reprocessed.
Are they binary files or plain text files? Delimited text files
Thanks for the responses. Do you really want to focus only on emojis? Are other non-ascii characters allowed?
If you want to check for all non-ascii characters, we have CTL functions like isAscii(string arg) or even removeNonAscii(string arg) that could be useful.
If other non-ascii characters are allowed and you want to remove only emojis, I am affraid that the suggested java library or the CTL functions like find(string arg, string regex) or replace(string arg, string regex, string replacement) with regular expressions are the only reasonable options that come to my mind.