MongoDBReader formatting issues

CloverDX Platform
1

Hi,

I am trying to read some data from MongoDB. An example data is below:


{
	_id: "XXXXXXXXXXXX",
	name: "Foo",
	description: "bar",
	metadata: {
		info: "Hello World"
	}
}

If I read a document as shown above from MongoDB the document is returned as JSON Object which is a map field.
But the metadata field is returned as string.

From the above example, is there a way to get the metadata field as a map field as well? If not, how do I convert the string field into map field?

Sincerely,
Shubha

2

Hi Shubha,
unfortunately, extracting nested elements as json objects (map[string,string]) is currently not supported in CloverETL. We do plan to incorporate such feature in the future but due to its complexity and resource demandingness, it has not been developed yet.
However, there are a couple of workaround options of how to deal with this issue. Attached is a project where I demonstrate 2 useful approaches to nested elements:

  • If you need to extract the ‘metadata’ nested element as json, you would have to deconstruct the original json into individual fields and then reconstruct the ‘metadata’ part again. In my example, I am using JSONExtract to parse the original json into individual fields and the JSONWriter component to build up the nested element part. Note: due to the way how JSONWriter works, the output json is a simple string field (not a map).

  • If you do insist on having the output json as a map[string,string] field, review the Reformat component in my example. The source code will show you have to construct a map from the given individual fields.

Kind regards,