EXT_MERGE_JOIN cannot perform Full Outer Join?

CloverDX Platform
1

graph file looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<Graph name="Test" description="POC" revision="1.0">
<Global>
  <Metadata fileURL="output.fmt" id="OutMetadata"/>
  <Metadata fileURL="data1.fmt" id="InMetadata1"/>
  <Metadata fileURL="data2.fmt" id="InMetadata2"/>
</Global>
<Phase number="0">
  <Node id="INPUT1" type="DATA_READER" fileURL="data1" quoteStrings="true"/>
  <Node id="INPUT2" type="DATA_READER" fileURL="data2" quoteStrings="true"/>
  <Node id="SORT_INPUT1" type="EXT_SORT" sortKey="REFERENCE"/>
  <Node id="SORT_INPUT2" type="EXT_SORT" sortKey="REFERENCE"/>
  <Node id="JOIN" type="EXT_MERGE_JOIN" joinKey="REFERENCE" joinType="fullOuter" transformClass="myPackage.ReformatJoinTest"/>
  <Node id="OUTPUT" type="DELIMITED_DATA_WRITER" append="false" fileURL="output.txt"/>

  <Edge id="INEDGE1" fromNode="INPUT1:0" toNode="SORT_INPUT1:0" metadata="InMetadata1"/>
  <Edge id="INEDGE2" fromNode="INPUT2:0" toNode="SORT_INPUT2:0" metadata="InMetadata2"/>
  <Edge fromNode="SORT_INPUT1:0" id="INNEREDGE1" metadata="InMetadata1" toNode="JOIN:0"/>
  <Edge fromNode="SORT_INPUT2:0" id="INNEREDGE2" metadata="InMetadata2" toNode="JOIN:1"/>
  <Edge fromNode="JOIN:0" id="OUTEDGE" metadata="OutMetadata" toNode="OUTPUT:0"/>
</Phase>
</Graph>

data1 file:
"000000010272 ",“WASILLA”,“AK”,99654
"000000010273 ",“WASILLA”,“AK”,99654

data2 file:
"000000010274 ",“address1”
"000000010273 ",“address2”

output.txt result:
"000000010272 ",“WASILLA”,“AK”,99654,
"000000010273 ",“WASILLA”,“AK”,99654,“address2”

looks like it’s doing a left-outer join instead of performing a full-outer join??

any help is greatly appreciated :slight_smile:

Thanks!

al

2

Hello,
it’s really bug. I’ve found where is the problem and I’m fixing it.

3

Hi Agata,

Do you have an estimated date of when this bug will be fixed? Please let me know what I need to do after you fix it…

Thanks,
al

4

Hi,
fix is available in SVN branch 2.4 (svn://svn.berlios.de/svnroot/repos/clov … .component) and will be in nearest patch release.

5

Hi Agata,

I could not built (using ant) from the SVN you gave me… Could you send me the JAR file instead?

Do you have a date on the next patch release? if it’s in the near future, I can wait for that too…

Thanks so much for your help :slight_smile:

al

6

Hi,
patch will be released today or tomorrow :slight_smile: , so I’ll not send you the jar file.

7

Hi Agata,

After downloading CloverETL rel 2.4.3, I am still not getting the correct result for full-outer join…

Using the same example,

data1 file:
"000000010272 ",“WASILLA”,“AK”,99654
"000000010273 ",“SACRAMENTO”,“CA”,95825

data2 file:
"000000010274 ",“address1”
"000000010273 ",“address2”

the output.txt is:
"000000010273 ",“SACRAMENTO”,“CA”,95825,“address2”
"000000010274 ",“SACRAMENTO”,“CA”,95825,“address1”

However, for full-outer join, the correct result should be:

"000000010272 ",“WASILLA”,“AK”,99654,
"000000010273 ",“SACRAMENTO”,“CA”,95825,“address2”
"000000010274 ",“address1”

right?? Please advice…

Thanks,
albert