Reader and Writer component for Cassandra

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

Ypoint Solutions
Posts: 5
Joined: Fri Jul 24, 2015 12:46 pm

Reader and Writer component for Cassandra

Postby Ypoint Solutions » Fri Jul 24, 2015 12:58 pm

Hi,

I have used JDBC driver to Read and write to Cassandra Database,However the performance is very bad using external JDBC Driver.
I found PenthoETL(open source) has separate big data components for reading and writing to Cassandra database and its performance is pretty faster than using JDBC driver in CloverETL.

Is there any other way to read and write to Cassandra through CloverETL.
I have gone through following thread, but couldn't help me much.

viewtopic.php?f=4&t=7159

Is Clover team working on building Cassandra reader and writer component?




Thanks.
Bhavin
Data Integration Architect
Ypoint Solutions

dpavlis
Posts: 182
Joined: Sat Mar 10, 2007 8:12 pm

Re: Reader and Writer component for Cassandra

Postby dpavlis » Fri Jul 24, 2015 3:24 pm

If you are skilled in Java, you might try to wrap a piece of Java code from following Datastax/Cassandra blog.
http://www.datastax.com/dev/blog/bulk-loading

A Reformat component or the new 4.1 version CustomJavaWriter component
might be used to create the solution.
David Pavlis
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

kame10
Posts: 1
Joined: Tue Feb 12, 2019 9:04 am

Re: Reader and Writer component for Cassandra

Postby kame10 » Tue Feb 12, 2019 9:27 am

Ypoint Solutions wrote:Hi,

I have used JDBC driver to Read and write to Cassandra Database,However the performance is very bad using external JDBC Driver.
I found PenthoETL(open source) has separate big data components for reading and writing to Cassandra database and its performance is pretty faster than using JDBC driver in CloverETL.

Is there any other way to read and write to Cassandra through CloverETL.
I have gone through following thread, but couldn't help me much.

http://forum.cloveretl.com/viewtopic.php?f=4&t=7159

Is Clover team working on building Cassandra reader and writer component?




Thanks.
Bhavin
Data Integration Architect
Ypoint Solutions


Hi,
Can you please tell me which JDBC driver did you use? There are many drivers available. Few drivers like Thrift are bad in performance and should be avoided.

bartonv
Posts: 120
Joined: Wed May 03, 2017 12:10 pm

Re: Reader and Writer component for Cassandra

Postby bartonv » Fri Feb 22, 2019 4:14 pm

Hello,
there are 3 options of how to connect to Cassandra from CloverDX:

  1. Using a dedicated JDBC driver
  2. Using the Datastax Java library
  3. By calling a script

The first option is not recommended as it appears that no one is actively maintaining or supporting JDBC connectors for Cassandra. The Google archive version we found is a couple of years old and is not being actively developed.
The second option is viable, yet significantly more complex. More information can be found in this Forum article.
In CloverDX Designer, the third option gets as simple as using an ExecuteScript component and feeding it with the proper script.

I have attached 2 projects to this ticket one of which shows the ExecuteScript approach (option #3) and one of which demonstrates the Datastax driver and custom Java code approach (#2).

Kind regards,
Attachments
Cassandra-datastax.zip
(9.23 MiB) Downloaded 28 times
Cassandra.zip
(385.36 KiB) Downloaded 29 times
---
Vladimir Barton
CloverCARE Support
CloverDX

Visit us online at http://www.cloverdx.com


cron