Monet Bulk load does not work from Java after upgrade to Oct2014
Hello, I hate to be a bugger again. :( We use Pentaho PDI (KETTLE) for loading data from our Oracle DWH to MonetDB. After upgrade from Oct2013-SP6 to Oct2014 bulk loaders stopped working. Any attempt to use ends with Server connection lost message. Here is log message from KETTLE: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - COPY 1 RECORDS INTO drmaxtst.typy_pohybu_stage FROM STDIN USING DELIMITERS '|',' 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ','"' NULL AS 'null'; 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : Error in step, asking everyone to stop because of: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : org.pentaho.di.core.exception.KettleException: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - An error occurred writing data to the MonetDB API (MAPI) process 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - Connection to server lost! 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeBufferToMonetDB(MonetDBBulkLoader.java:506) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeRowToMonetDB(MonetDBBulkLoader.java:230) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.processRow(MonetDBBulkLoader.java:207) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:60) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at java.lang.Thread.run(Thread.java:744) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - Caused by: java.io.IOException: Connection to server lost! 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:197) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeBufferToMonetDB(MonetDBBulkLoader.java:494) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ... 4 more To be honest the problem first showed up after upgrade from Jan2014SP1 to Jan2014SP2. I was hoping it might be temporary which seems it is not. Thank you for any help, Radovan
Hi Radovan, The issues is due to a fix in MonetDB, which directly impacted how PDI's bulk loader uses the MonetDb JDBC connection. It is tracked in the PDI Issue tracking system here: http://jira.pentaho.com/browse/PDI-12278 More recently there was a blog post how to work around it or fix it (if you are willing to re-compile the PDI source): http://ubiquis.co.uk/pdi/monetdb-bulk-loader-and-pdi/ I hope this helps. Best regards, Dimitar
On 2014-Nov-13, at 11:30 , Radovan Bičiště <radovan.biciste@ceosdata.com> wrote:
Hello, I hate to be a bugger again. :( We use Pentaho PDI (KETTLE) for loading data from our Oracle DWH to MonetDB. After upgrade from Oct2013-SP6 to Oct2014 bulk loaders stopped working. Any attempt to use ends with Server connection lost message. Here is log message from KETTLE: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - COPY 1 RECORDS INTO drmaxtst.typy_pohybu_stage FROM STDIN USING DELIMITERS '|',' 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ','"' NULL AS 'null'; 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : Error in step, asking everyone to stop because of: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : org.pentaho.di.core.exception.KettleException: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - An error occurred writing data to the MonetDB API (MAPI) process 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - Connection to server lost! 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeBufferToMonetDB(MonetDBBulkLoader.java:506) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeRowToMonetDB(MonetDBBulkLoader.java:230) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.processRow(MonetDBBulkLoader.java:207) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:60) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at java.lang.Thread.run(Thread.java:744) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - Caused by: java.io.IOException: Connection to server lost! 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:197) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeBufferToMonetDB(MonetDBBulkLoader.java:494) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ... 4 more
To be honest the problem first showed up after upgrade from Jan2014SP1 to Jan2014SP2. I was hoping it might be temporary which seems it is not. Thank you for any help, Radovan _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Dimitar, Thank you. I really appreciate it!!! I'm going to hold for a bit and wait for the merge to PDI stable release most likely. Then upgrade for both PDI and our MonetDBs. Radovan On 11/13/2014 11:39 AM, Dimitar Nedev wrote:
Hi Radovan,
The issues is due to a fix in MonetDB, which directly impacted how PDI's bulk loader uses the MonetDb JDBC connection. It is tracked in the PDI Issue tracking system here: http://jira.pentaho.com/browse/PDI-12278
More recently there was a blog post how to work around it or fix it (if you are willing to re-compile the PDI source): http://ubiquis.co.uk/pdi/monetdb-bulk-loader-and-pdi/
I hope this helps.
Best regards, Dimitar
On 2014-Nov-13, at 11:30 , Radovan Bičiště <radovan.biciste@ceosdata.com> wrote:
Hello, I hate to be a bugger again. :( We use Pentaho PDI (KETTLE) for loading data from our Oracle DWH to MonetDB. After upgrade from Oct2013-SP6 to Oct2014 bulk loaders stopped working. Any attempt to use ends with Server connection lost message. Here is log message from KETTLE: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - COPY 1 RECORDS INTO drmaxtst.typy_pohybu_stage FROM STDIN USING DELIMITERS '|',' 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ','"' NULL AS 'null'; 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : Error in step, asking everyone to stop because of: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : org.pentaho.di.core.exception.KettleException: 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - An error occurred writing data to the MonetDB API (MAPI) process 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - Connection to server lost! 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeBufferToMonetDB(MonetDBBulkLoader.java:506) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeRowToMonetDB(MonetDBBulkLoader.java:230) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.processRow(MonetDBBulkLoader.java:207) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:60) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at java.lang.Thread.run(Thread.java:744) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - Caused by: java.io.IOException: Connection to server lost! 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:197) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader.writeBufferToMonetDB(MonetDBBulkLoader.java:494) 2014/11/13 11:13:02 - MonetDB Bulk Loader.0 - ... 4 more
To be honest the problem first showed up after upgrade from Jan2014SP1 to Jan2014SP2. I was hoping it might be temporary which seems it is not. Thank you for any help, Radovan _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Dimitar Nedev -
Radovan Bičiště