JSON Nested Elements

CloverDX Platform
1

I’m attempting to create a graph that processes an HTTP JSON response and outputs only the data needed while retaining the orignial data structure.

I’m fine until I get to nested elements.

Sample structure:


{
"tickets":[
{
            "url": "https://xxxxx.zendesk.com/api/v2/tickets/#####.json",
            "id": ########,
            "external_id": null,
            "custom_fields":[
                {
                    "id": ###########,
                    "value": "Foo"
                },
                {
                    "id": ###########,
                    "value": "Bar"
                }
            ],
            ....
}
]
}

How, can read the http resposne and output a “tickets” object with the following structure?


{
"tickets":[
{
            "id": ########,
            "custom_fields":[
                {
                    "id": ###########,
                    "value": "Foo"
                }
            ],
            ....
}
]
}

My goal for this process is to retain only one or two specific custom_field values. The same would go for other elements where I only need to retain certain values in in the data set.

The ultimate goal is to only provide the data required for the use case as much of the data contains PII which we need to exclude.

I’m able to generate a JSON file that contains anything within the first level of the tickets array, but I’m struggling with how to handle nested arrays such as “custom_fields”.

Thanks in advance!

2

Hello Mybludrunsorange,
please find the attached solution based on your example. The main idea is to extract the data to 2 output ports by using JSONExtract and outputting them with JSONWriter by using the parent key concept. Below is a more detailed explanation of the key setup:

  • I used the equivalent XML Schema (.XSD file) that Clover extracted automatically

  • I mapped the ‘tickets’ and the ‘custom_fields’ elements to flow into separate ports with different metadata while retaining the ‘ticketID’ field in both of them as the key that preserves the mutual link between the 2 ports

  • In JSONWriter, I added bindings to the respective elements (ticketIDmetadata on Port 0 to the ‘tickets’ element and customFieldsMetadata on Port 1 to the ‘custom_fields’ element) and mapped the fields accordingly

  • On the ‘custom_fields’ binding, I defined the key/parent key relation in order to recreate the mutual link between the 2 data flows

  • In the Advanced properties of JSONWriter, I have also set the sorting input to TRUE and defined sort keys (‘ticketID’ on both ports)

  • From your example, I assumed that you need to omit also some of the custom fields based on their id. If my assumption is correct, this can be achieved by a Filter component as demonstrated in the attached project

Regards,