I have URL generator which feeds URL into the HTTP connector which then push it to trash.
The Normalizer is able to generate the valid URL but HTTP connector throws following error.
org.apache.http.conn.HttpHostConnectException: Connection to http://search.twitter.com refused.
any advice?
Thanks
Ashwani
Hi, ashwani,
Are you using some proxy or firewall? If so, please try it again without proxy or with firewall disabled.
If you can not leave proxy out, connecting via a proxy server is available too, in a way like:
http:(proxy://proxyHost:proxyPort)//www.domain.com
If this does not help, post your graph here, please, and we will think up something else.
Best regards,
Hello,
I am using the ‘HTTP CONNECTOR’ component to send a request to a SSL enabled web-service of Endeca-Server. Unfortunately cant find a way to configure the component for a SSL-connection. any help is very appreciated.
Regards,
Farnaz
Hi, Farnaz,
What exactly would you like to configure? HTTPConnector should be ready to connect to HTTPS immediately. But if you want to configure your key store and trust store, nice description can be found here http://stackoverflow.com/questions/1277 … in-eclipse and here http://stilius.net/java/java_ssl.php
By the way, we have also WebServiceClient component in commercial version of Designer. It is more suitable for communication with web services.
Best regards,
Thanks imriskal,
I guess there is something wrong with the SSL configuration I am using then. Thanks for your help.
regards, Farnaz
I am using HTTPConnector . My Endeca server is SSL enabled. I have added configuration for SSL keystore and password in Integrator IDE e.g. Default VM arguments, changing .ini file.
I am able to run graphs with SSL enabled configuration. One graph that I am using to retrieve Endeca views definition by calling Endecas web services has HTTPConnector with https protocol in URL. After running this graph I am getting javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated exception. Where this graph works well for NON-SSL environment.
I can see there is no flag just like “Disable SSL Certificate Validation” in this component.
Please suggest some solution or how we can use WEbServiceClient here.
Regards,
Bhushan
Dear Bhushan,
Is the certificate of your server self-signed? If so, you should set truststore and truststore password as well.
What do you mean by “I am able to run graphs with SSL enabled configuration.”? Does it mean that this is the only graph throwing this exception? And what exception is it exactly? Could you please post also your graph run log? What makes the graph different than the other graphs?
I do not know what is your goal so I unfortunately can not give you any advice. Could you please describe your task in more detials? Documentation page for WebServiceClient is here: http://doc.cloveretl.com/documentation/ … lient.html
And regarding the Disable SSL Certificate Validation property in HTTPClient, we already have this issue in our bug tracking system, see https://bug.javlin.eu/browse/CLO-2308 The fix is tentatively planned for version 3.6.0-M1.
Thank you in advance for your answers.
Kind regards,
Hi Imriskal,
Yes, This is the only graph that I am unable to run. I have set -Djavax.net.ssl.keyStore, -Djavax.net.ssl.keyStorePassword, -Djavax.net.ssl.trustStore, -Djavax.net.ssl.trustStorePassword in CLoverETL IDE (integrator)
Error Log Is :
INFO [main] - [Clover] phase: 0 initialized successfully.
INFO [WatchDog] - Starting up all nodes in phase [0]
INFO [WatchDog] - Successfully started all nodes in phase!
ERROR [HTTP_CONNECTOR0_0] - javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated.
ERROR [WatchDog] - Graph execution finished with error
ERROR [WatchDog] - Node HTTP_CONNECTOR0 finished with status: ERROR
ERROR [WatchDog] - Node HTTP_CONNECTOR0 error details:
INFO [WatchDog] - [Clover] Post-execute phase finalization: 0
INFO [WatchDog] - [Clover] phase: 0 post-execute finalization successfully.
INFO [WatchDog] - Execution of phase [0] finished with error - elapsed time(sec): 0
ERROR [WatchDog] - !!! Phase finished with error - stopping graph run !!!
INFO [WatchDog] - -----------------------** Summary of Phases execution **---------------------
INFO [WatchDog] - Phase# Finished Status RunTime(sec) MemoryAllocation(KB)
INFO [WatchDog] - 0 ERROR 0 22395
INFO [WatchDog] - ------------------------------** End of Summary **---------------------------
INFO [WatchDog] - WatchDog thread finished - total execution time: 0 (sec)
INFO [main] - Freeing graph resources.
ERROR [main] - Execution of graph failed !
Have you tried your keystore and truststore outside CloverETL? Did they work? Also, please try to display Eclipse error log (Window → Show View → Other… → General → Error log) after the graph fails and search for the whole stack trace of the error.
If this is the only graph failing, what does the graph contain in opposed to the other graph? Did you use some functions not used in the other graphs?
Kind regards,
Imriskal,
Yes I am using keystore and passwords outside CloverETL too and those modules are working absolutely fine. I ran the graph again but Error Log not showing any error but console is showing above mentioned error.
Bhushan C.
Hi Bhushan,
Could I ask you where exactly have you set the keystore and trustore parameters?
CloverETL uses one Java Virtaul Machine for running Designer and another Java Virtual Machine to run graphs. If you want to set some additional arguments for running graphs (keystore and trustore parameters), you can set them for the desired graph via menu “Run → Run Configurations”, the “Arguments” tab. This is the correct place where you should put additional VM arguments, which are related to the graph.
If you want to be sure which Java attributes are used when you running your graph, you can print them by the CTL function getJavaProperties(). I prepared for you sample graph which prints all used Java properties (it prints them via Trash component to Console). Please try to use this function in your graph (e.g. in the first phase of graph) to be sure you are using correct Java attributes (especially keystore and trustore attributes).
Hi Jiri,
I have set keystore and trustore parameters in VM arguments of Installed JRE (Windows > Prferences> Installed JRE). Also I set these parameters in “Run → Run Configurations”, the “Arguments” tab too still its not working.
I copied the pipeline in PrintJavaProperties.grf that you provided into my graph and changed the phases to 0 and 1. In logs I can see keystore and trustore parameters javax.net.ssl.trustStorePassword, javax.net.ssl.trustStore, javax.net.ssl.keyStore, javax.net.ssl.keyStorePassword set.
Still not able to run.
Hi Bhushan,
Please set Log level to DEBUG or ALL, it should show us some more details about the error. You can set the log level via Run → Run configurations… → Log level.
Are you sure your keystore and truststore for both client and server are set properly and contain all necessary keys? Could you please check them in a tool like http://keystore-explorer.sourceforge.net/ ? And this is the proper way to set them: http://doc.cloveretl.com/documentation/ … icate.html
Kind regards,