Is there any way to add a new reader for a DataRecordMetadata class? The output of my transformed data can be validated against a schema, and that schema contains enough information to create the output Metadata. In the past, I’ve been creating the output metadata XML file by pretty much copy and pasting from our schema file. I’d like to just read in our schema file and generate the metadata file from that, it would save me a tedious step in the process.
Hello,
I’m not sure if I understand your question well, but If you want to compare clover metadata with xsd metadata you can use XsdMetadata.java type for creating xsd file from clover metadata or MetadataXsd.java type for creating clover metadata from xsd file. Thees import/export operations are built-in in CloverGUI, so if you use CloverGUI you can import/export xsd/clover file to/from clover/xsd with Wizard (see Metadata from XSD).
I don’t use clover GUI, and the schema file I’d like to use as metadata is not XSD, it’s an OWL ontology, which as I said contains enough information for me to build a metadata object.
I have code that will create a DataRecordMetadata instance from the ontology file, but I have no way to make clover use it. I tried creating a dummy metadata file, and then overwriteing it in the transformation graph with my generated metadata object of the same name, but during the transformation, clover still uses the dummy metadata. (i think this is due to how records are copied over edges, I tried overriding the duplicate method on the metadata record and get erratic results)
What i’d like is to add a new metadata tag, or add a hook into the factory that creates the metadata that would call my code for creating a metadata object instead of the default of creating it from an XML file.
I’ve been looking through the clover code and this doesn’t seem possible without editing the source, which I can easily do, but I was hoping to not have a custom version of clover to do this.
i’ve found the reason for why my workaround of trying to override a dummy metadata record does not work. the metadata map is created in the read method of TransformationGraphXMLReaderWriter, then assigned to the graph. I’m overriding the dummy metadata record when nodes are created during the call to instantiatePhases. Since that happens before the edges are created, I thought that when the edges are finally built, they’d use the new generated metadata record, not the dummy one specified on input.
However, the local metadata map is passed to the instantiateEdges function rather than the copy with the transformation graph. So the edges use the dummy data, but the transformation graph has the correct metadata for the id, and no longer has the dummy data (which was the intent). Thus, I end up with edges that are malformed, and I get errors during transformation.
I suspect that if I pass the record metadata map from the transformation graph to instantiateEdges instead of the local copy from read, my workaround will work.
But I’m changing the clover code, which I was trying to avoid, and even if that’s ok, I still have a rather ugly workaround.
Are there plans on making the data record metadata stuff a little more extendable?
Here’s the code that makes my workaround feasible, this goes between instantiatePhases and instantiateEdges in the TransformationGraphXMLReaderWriter. I’d still like to have a better method for adding my own metadata format than this workaround of overriding a dummy record from a Node constructor.
// Updates the metadata map with any changes that were effected during phase initialization before
// passing it to the instantiateEdges method where it will be used. This ensures that the edges are
// created using the same metadata as contained in the graph.
metadata.clear();
for (Iterator<String> it = graph.getDataRecordMetadata(); it.hasNext();) {
String aName = it.next();
metadata.put(aName, graph.getDataRecordMetadata(aName));
}
We are not currently considering to introduce any hook to write your own metadata builder from your custom xml format.
You said you have a piece of code which build up our DataRecordMetadata from your proprietary format. Than you can use DataRecordMetadataXMLReaderWriter class to backward generation of standard metadata xml. What about this simple solution?
PS: Unfortunately the method DataRecordMetadataXMLReaderWriter.write() is not maintained code. Probably you need to do slight update and if you want to integrate your change to official release, please, send us the diff. Thanks a lot.
It’s not a custom XML format, nor is it proprietary. It’s an OWL ontology, which is a w3c standard.
I just don’t want to have to write XML metadata record files in clover’s format which have the exact same information that our ontology contains, it’s a maintenance nightmare. Anytime I change one, I have to make an analogous change to the other.
And writing out the xml file from the generated metaobject, while workable, isn’t much better than my workaround.
i’m basically extending clover’s framework to support conversions from “normal” data, CSV, XML, RDBMS, etc. to semantic-web friendly data in OWL or RDF format.
It would be nice if things like adding new metadata formats were a little more friendly.
If there are no plans to add the ability to add your own metadata format’s to clover, what are the chances of my patch getting applied to TransformationGraphXMLReaderWriter?
Hello,
if you write type implementing org.jetel.database.IConnection, TransformationGraphXMLReaderWriter is able to instantiate your metadata. TransformationGraphXMLReaderWriter can create metadata from internal or external definition or from a connection. You can see implementation of createMetadata(Properties) method in DBConnection class.
yes, implementing IConnection also seems to work, and I prefer this than having a patched copy of clover. Thanks for pointing that out to me.