Using the Json writer to create a Json for each record

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

Bea95
Posts: 6
Joined: Mon Mar 15, 2021 4:44 pm

Using the Json writer to create a Json for each record

Postby Bea95 » Mon Mar 15, 2021 5:03 pm

Hi there,

Is there a way to loop the Json Writer so it creates a json structure for each input record that is passed to it? I want to create multiple Jsons as an input to the HTTP connector so I can send a request for each each record. So far I have been able to do it by manually creating the Json structure as a string using the reformat component and concatenating the the inputs but was wondering if it was possible to do the same with the Json writer.

For example , I would like to produce for each record

{
"request_id" : "3f6d7499-3134-48fd-a020-62f7f9f2ceb6",
"name" : {
"first_name" : "John",
"middle_name" : "Eric",
"surname":"Doe"
}
}

I have also tried split out the json structure using the streaming option as opposed to the discrete option but was unsure how to do split it in a way the produces a valid json for each part


Thanks in advance!

jandikovae
Posts: 64
Joined: Fri Nov 04, 2016 8:51 am

Re: Using the Json writer to create a Json for each record

Postby jandikovae » Tue Mar 16, 2021 2:34 pm

Hi there,

I believe that you can indeed achieve your goal by using the JSONWriter component. The component, however, assumes that it is intended to create one JSON file from all records (from the whole data flow). In order to tell the component that you need one result for each row, you might want to set the "Records per file" attribute to 1. This way, the JSONWriter creates a separate JSON file for each record.

If you then use the output port writing function (you can use the discrete option now), the component will send a list of JSON strings to the output port, where each row corresponds with each original record. The result on the edge looks like this:

CaptureJson.PNG
Result
CaptureJson.PNG (19.41 KiB) Viewed 584 times

I have created a graph for your review (I am including input data that the graph uses). Please take a look. I hope this meets your needs, but if not, please let me know.

Best Regards,
Eva
Attachments
jsoninput.txt
(100 Bytes) Downloaded 41 times
JsonWriter.grf
Graph
(2.09 KiB) Downloaded 41 times
---
Eva Jandikova
CloverCARE Support
CloverDX

Visit us online at http://www.cloverdx.com

Bea95
Posts: 6
Joined: Mon Mar 15, 2021 4:44 pm

Re: Using the Json writer to create a Json for each record

Postby Bea95 » Tue Mar 16, 2021 3:40 pm

jandikovae wrote:Hi there,

I believe that you can indeed achieve your goal by using the JSONWriter component. The component, however, assumes that it is intended to create one JSON file from all records (from the whole data flow). In order to tell the component that you need one result for each row, you might want to set the "Records per file" attribute to 1. This way, the JSONWriter creates a separate JSON file for each record.

If you then use the output port writing function (you can use the discrete option now), the component will send a list of JSON strings to the output port, where each row corresponds with each original record. The result on the edge looks like this:

CaptureJson.PNG
I have created a graph for your review (I am including input data that the graph uses). Please take a look. I hope this meets your needs, but if not, please let me know.

Best Regards,
Eva



Thank you Eva,

That was exactly what I was looking for!


cron