Why JSONExtract fails on OpenAI output (and how to debug + fix it)?

Resolved Cases
1

I’m using OpenAIClient in my CloverDX graph to auto-tag incoming tickets. I’m being very explicit in the prompt and asking the model to return JSON that matches a fixed structure (including allowed values), something like:
{
“ticket”: “string”,
“tag1”: “Designer | Server | Both | Other”,
“tag2”: “Upgrade | Compatibility | Performance | Database | Cloud | Security | Java | Docker | Memory | Graph | ThirdParty | System | AppServer | API | “””,
“confidence”: “High | Medium | Low”,
“explanation”: “string (may be “”)”
}

The OpenAI response looks like the JSON I am looking for, but when I try to generate a tree/schema in JSONExtract mapping, assuming this very same structure, it fails with an error.

What’s the right way to make this robust in a CloverDX transformation?

2

You’re running into a very common “AI-meets-ETL” issue: a prompt can strongly encourage a shape, but it can’t guarantee the model will follow your exact schema 100% of the time—especially if your downstream component expects strict JSON typing and structure. Even a tiny deviation (missing field, extra field, extra parent element, different casing, array vs. string, stray text before {, etc.) can break schema-based mapping.

A couple of tips for you:

  1. Be as specific as possible in your prompt. In practice, there is a difference between:
    • “Please return JSON like this…” (prompt-only, not guaranteed)
    vs.
    • “Return JSON that must validate against this schema.” (schema-bound)

  2. Debug it the CloverDX way: capture the raw AI output first
    Before you fight JSONExtract, log what you really receive: temporarily write the OpenAI response to a FlatFileWriter. More specifically, for a single graph run, you can replace the JSONExtract with the FlatFileWriter, which outputs the OpenAI response directly to a file. Once you have the real sample, you can review it and use the raw JSON schema in the JSONExtract component.

3

There is another “angle” to approach this.

Directly in AIClient (or in some connected Map component) parse the text produced by LLM using CTL’s parseJson(). If the JSON is at least well-formed then you receive CTL’s variant data structure which you can start querying.
If it is not even well-formed (parsing fails) you can enclose the parseJson() in try-catch, get the exact error message and send it back to LLM to fix the output.

This is quite common approach as LLM occasionally diverts from the instructions dictating the output structure. Then mechanism, as described, allows such random errors to be recovered from or at least detected & such problematic result discarded.

Below is a snippet of handling the LLM response in AIClient as mentioned:

function integer processResponse(list[ChatMessage] chatContext, integer iterationIndex, string assistantResponse) {
  $out.0.* = $in.0.*; //whatever needs to just propagate
 
   variant response;
   try{
        response=parseJson(assistantResponse);
   }catch(CTLExecption ex){
        /*here we just ignore, but could halt processing 
         by returning STOP or assemble message for LLM to ask to correct the output*/
        return SKIP;  
   }
   //we have parsed response, let's fill some output fields
 
   $out.0.result1 = toString(nvl(response["result1"], ""));
   $out.0.result2 = response["arrayOfProperties"] !=null ?  toString(response["arrayOfProperties"][0]["propA"]) : "no result produced";
  
   return OK;

}