I would like to read the filename(s) of input file(s) used in my data reader component and pass the filename through as an additional field using a subsequent reformat component - could anyone provide me with a simple example of how this can be done? Example of the filename structure would be FILENAME_20130823.txt
This can be easily done using the metadata connected to reader’s output port. All you need to do is to edit the metadata, add new field and set its Autofilling property to “source_name” (please see attached screenshot). This will extract the source name (the File URL from your reader) and place it to a particular record. I have created a short sample for you, so please find it attached below.
Is there a way to reformat the file_name value to remove the file directory details from the field so that you are left with only the file name and not the whole route path plus the file name?
Unfortunately, we do not provide this option directly. You can either split the value by “/” character (or “\” in case of OS Windows) and take only the value after the last slash or (if you have CloverETL Server and jobflows) you can use ListFiles component which provides information about both filename and filename + path.
I’m sorry but my clover skills aren’t that good so I don’t know how to strip out the file path using the “/” or “\” values?
If an example path and filename were “C:\NH\KPI2\Requests\Reactive\New\Filename123.csv” could you provide an example of what the function would be to strip out the “\” values and get to the last part?
Hello, I tried to use the example and it worked really well. I would like to ask you about some explanation of usage of the last part - ‘([\\]+)$’)[0]
I understand the first part but I’m not able to reuse this type of regular expression and I was not able to find deeper detail in manual. Thanks s lot.
And its semantic is: take one or more non-slash and non-backslash chars from end of string. So from string “/my/path/to/file.txt” or “c:\my\other\path\file.txt” is extracted “file.txt”.
[^/\\] = all chars except / or \ (so the ^ inverts meaning)
Missing ^ is probably issue in your regex. Its semantics actually is: Take one or more \ from end of string and that is not what you are looking for, I guess. Try to rewrite it as
Please help me
I used to always use source_name and sheet_name in metadata for excel files, but past source codes are unreacheable, it was 2 years ago.
Now I’ve changed direct file name setting from ${data_in}\1.xls to port:$0.Field1:source and source_name autofilling stopped working, while sheet_name continues doing it.
As I remember, source_name always worked regardless of filename setting way. Am I wrong and what should I do?
And on other hand, getting file names through port fails on one of three XLSreaders in my graph with very scanty message
ERROR [WatchDog] - Graph execution finished with error
ERROR [WatchDog] - Node XLS_READER2 finished with status: Component has finished and input port 0 still contains some unread records.
ERROR [WatchDog] - Node XLS_READER2 error details:
INFO [exNode_0_1390551613757_LEVEL1_LINES] - Number of commited records: 0
Unfortunately, there is an issue in your version of CloverETL related to this functionality, please see the issue report here: https://bug.javlin.eu/browse/CL-2626
It is fixed since CloverETL version 3.3.2, so if you upgrade your CloverETL instalation, it should work as you expect.