- users-list - monetdb.org

MonetDB Cross Partition Joins
by Wolf Rödiger 17 Jan '13

17 Jan '13

Hello, According to its man page MonetDB is able to work as a cluster. However, I found no documentation on cross partition joins. Is there a way to partition a relation in MonetDB across servers and perform a cross partition join? Regards Wolf _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

JDBC error: 'protocol violation: unexpected line in data block: .'
by Luis Neves 11 Jan '13

11 Jan '13

Hello all, Is this a known bug? Is there any workaround? * Stack Trace java.sql.SQLException: 'protocol violation: unexpected line in data block: .' at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2535) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2284) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.execute(MonetPreparedStatement.java:247) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.executeQuery(MonetPreparedStatement.java:265) * Table definition sql>\d sys.wa_blogs_audience CREATE TABLE "sys"."wa_blogs_audience" ( "tid" CHAR(32) NOT NULL, "ts" BIGINT, "ts_hour" BIGINT, "ts_day" BIGINT, "ts_month" BIGINT, "s_site_name" VARCHAR(32), "s_hostname" VARCHAR(256), "n_pageviews" DOUBLE, "n_visits" DOUBLE, "n_visitors" DOUBLE, "h_visitors_register" VARCHAR(1881), CONSTRAINT "wa_blogs_audience_tid_pkey" PRIMARY KEY ("tid") ); * SQL SELECT ts, hllagg(h_visitors_register) FROM wa_blogs_audience WHERE ts>=1355172393000 AND ts<=1357764393000 GROUP BY ts; * Output from mclient $ mclient -p9701 kpi_v1 -f raw -s "SELECT ts, hllagg(h_visitors_register) FROM wa_blogs_audience WHERE ts>=1355172393000 AND ts<=1357764393000 GROUP BY ts;" at: <https://dl.dropbox.com/u/279023/misc/mclient_output.txt> * JDBC debug log at: <https://dl.dropbox.com/u/279023/misc/monet_1357773164928.log> Thanks in advance! -- Luis Neves _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 6

Perf. benchmark for monetdb
by nitin sharma 10 Jan '13

10 Jan '13

Hi there, I am evaluating monetdb and was wondering if there are performance bench mark numbers like the one published in C-store paper: http://people.csail.mit.edu/tdanford/6830papers/stonebraker-cstore.pdf I am looking for perf. number for 100 millions rows in fact table (500 GB) joined with couple of dimension tables. Does moentdb supports horizontal partitioning of data and joins across machine (distributed joins) ? Thanks!Nitin _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 3

Weird concurrency issue with JDBC getConnection
by Percy Wegmann 10 Jan '13

10 Jan '13

Dear Monet Team, I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang. I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this. Cheers, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

What is normally done to recover from 'crash'ed databases.
by Brandon Jackson 08 Jan '13

08 Jan '13

Hello, I have a toy database on my laptop that I've been using while testing development of a Pentaho Data Integration plugin for bulk loading. One day I shutdown my machine without stopping monetdb. When I turned on the computer and started monetdbd on the database everything seems to work fine. 'monetdb status' outputs the following: brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb status name state uptime health demo crashed 0%, 0s crash 2013-01-05 00:29:20 brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb start demo starting database 'demo'... done brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb status name state uptime health demo running 5s 50%, 0s crash 2013-01-05 00:29:20 brandon-jacksons-macbook-pro:bin bjackson$ I cannot really tell if there is any real damage. What do people normally do to check the integrity of their databases and repair damage if it exists? Does a set of actions restore the health of the database? If I missed some documentation please feel free to refer me to some links. Thanks for the information. Brandon _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 3

Fwd: Re: c3p0 ConnectionTester socket read stuck after SQLException
by Hannes Mühleisen 08 Jan '13

08 Jan '13

Forwarding this to the list, please keep the discussion there... -------- Original Message -------- Subject: Re: c3p0 ConnectionTester socket read stuck after SQLException Date: Tue, 8 Jan 2013 08:17:35 -0800 (PST) From: Tapomay Dey <tapomay(a)yahoo.com> Reply-To: Tapomay Dey <tapomay(a)yahoo.com> To: Hannes.Muehleisen(a)cwi.nl <Hannes.Muehleisen(a)cwi.nl> My operations are as follows: 1. Create connections with auto-commit=false 2. Read queries from an external module. 3. Execute queries on the connection acquired in step 1 4. Update external module status as per result of queries. 5. If all is well, commit the connection. 6. If commit fails(optimistic locking is the culprit here), revert changes to the external module. All of these steps are implemented in a command chain pattern code. Thus I need to be able to pass around the connection. Thanks and Regards, Tapomay. ------------------------------------------------------------------------ *From:* Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl> *To:* Communication channel for MonetDB users <users-list(a)monetdb.org> *Sent:* Thursday, January 3, 2013 4:19 PM *Subject:* Re: c3p0 ConnectionTester socket read stuck after SQLException Hello Tapomay, after looking at your code, I was wondering why you chose the double management of connections in your Main/JdbcMonetdbTestClient classes. C3P0 provides a way to define "connection customizers" (com.mchange.v2.c3p0.ComboPooledDataSource.setConnectionCustomizerClassName(String)), which you can use to disable autocommit on new connections. This should simplify your example considerably. Nevertheless, I will try to provoke the exceptions you have mentioned and see how the JDBC driver handles them together with C3P0. Best, Hannes On 20/12/12 15:41, Tapomay Dey wrote: > As the source code has many unrelated dependencies I tried to extract > the relevant code. > Please find attached an eclipse project for the same. > However I haven't been able to reproduce the issue in this test project. > It should give you a fair idea of the sequence of calls made. Plz note > that a single insert query used in the live project is much much larger > (10K approx.) than the one in the test. > I reconfirmed that the issue is still there in the larger project. > > PS: I had sent this code long back and it got moderated just now(due to > some 40KB limit). Thus I have removed the dependencies > monetdb-jdbc-2.6.jar and c3p0-0.9.1.2.jar from lib. Plz add them to lib > folder and everything should work(after u set the correct DB names in > the code).(Anyways this code doesn't reproduce the issue I am facing so > I guess just a manual overview should suffice) > > Regards, > Tapomay. > > ------------------------------------------------------------------------ > *From:* Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl <mailto:Hannes.Muehleisen@cwi.nl>> > *To:* Communication channel for MonetDB users <users-list(a)monetdb.org <mailto:users-list@monetdb.org>> > *Sent:* Thursday, December 20, 2012 2:17 PM > *Subject:* Re: c3p0 ConnectionTester socket read stuck after SQLException > > Hello Tapomay, > > I will have a look later today, could you please also provide the source > code that you have used to provoke the issue? > > Thanks, > > Hannes > > On 20/12/12 08:26, Tapomay Dey wrote: > > Please find the thread dump below. > > Running MonetDB Database Server v1.6 (Oct2012-SP1). Compiled > > from MonetDB-11.13.5 src distribution. > > C3P0 version: 0.9.1.2. Also tried with 0.9.2. > > Occurs only after SQLException sometimes: INSERT INTO xyz : table xyz > > doesn't exist. > > Otherwise it works great. > > Also tried setting c3p0's preferredQuery for ConnectionTester to "select > > 1;". Problem persists. > > > > Name: pool-15-thread-2 > > State: RUNNABLE > > Total blocked: 0 Total waited: 1 > > > > Stack trace: > > java.net.SocketInputStream.socketRead0(Native Method) > > java.net.SocketInputStream.read(SocketInputStream.java:150) > > java.net.SocketInputStream.read(SocketInputStream.java:121) > > java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > > java.io.BufferedInputStream.read1(BufferedInputStream.java:275) > > java.io.BufferedInputStream.read(BufferedInputStream.java:334) > > - locked java.io.BufferedInputStream@5cec5796 > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream._read(MapiSocket.java:846) > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.readBlock(MapiSocket.java:895) > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.read(MapiSocket.java:965) > > sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) > > sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) > > sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) > > - locked java.io.InputStreamReader@206a6883 > > java.io.InputStreamReader.read(InputStreamReader.java:184) > > java.io.BufferedReader.fill(BufferedReader.java:154) > > java.io.BufferedReader.readLine(BufferedReader.java:317) > > - locked java.io.InputStreamReader@206a6883 > > java.io.BufferedReader.readLine(BufferedReader.java:382) > > > nl.cwi.monetdb.mcl.io.BufferedMCLReader.readLine(BufferedMCLReader.java:119) > > > nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:196) > > - locked nl.cwi.monetdb.mcl.io.BufferedMCLReader@29f38115 > > > nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2296) > > - locked nl.cwi.monetdb.mcl.net.MapiSocket@319f9f37 > > > nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2274) > > > nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) > > nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) > > nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:525) > > > com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:73) > > > com.mchange.v2.c3p0.impl.DefaultConnectionTester.statusOnException(DefaultConnectionTester.java:143) > > > com.mchange.v2.c3p0.AbstractConnectionTester.statusOnException(AbstractConnectionTester.java:82) > > > com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPooledConnection.java:437) > > - locked com.mchange.v2.c3p0.impl.NewPooledConnection@3340302c > > > com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:78) > > > com.sokrati.moneta.mclient.JdbcMonetdbClient.executeUpdate(JdbcMonetdbClient.java:154) > > > com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:46) > > > com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:1) > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > java.util.concurrent.FutureTask.run(FutureTask.java:166) > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > java.lang.Thread.run(Thread.java:722) > > > > Regards, > > Tapomay. > > > > > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > > http://mail.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > http://mail.monetdb.org/mailman/listinfo/users-list > > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org <mailto:users-list@monetdb.org> > http://mail.monetdb.org/mailman/listinfo/users-list > _______________________________________________ users-list mailing list users-list(a)monetdb.org <mailto:users-list@monetdb.org> http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

the dbname option disappeared
by Stefan de Konink 04 Jan '13

04 Jan '13

I have justed checked out the Feb2013 release and noticed that the dbname option is gone, is there a substitute? Stefan _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

5 8

c3p0 ConnectionTester socket read stuck after SQLException
by Tapomay Dey 03 Jan '13

03 Jan '13

Please find the thread dump below. Running MonetDB Database Server v1.6 (Oct2012-SP1). Compiled from MonetDB-11.13.5 src distribution. C3P0 version: 0.9.1.2. Also tried with 0.9.2. Occurs only after SQLException sometimes: INSERT INTO xyz : table xyz doesn't exist. Otherwise it works great. Also tried setting c3p0's preferredQuery for ConnectionTester to "select 1;". Problem persists. Name: pool-15-thread-2 State: RUNNABLE Total blocked: 0 Total waited: 1 Stack trace: java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:150) java.net.SocketInputStream.read(SocketInputStream.java:121) java.io.BufferedInputStream.fill(BufferedInputStream.java:235) java.io.BufferedInputStream.read1(BufferedInputStream.java:275) java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked java.io.BufferedInputStream@5cec5796 nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream._read(MapiSocket.java:846) nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.readBlock(MapiSocket.java:895) nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.read(MapiSocket.java:965) sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked java.io.InputStreamReader@206a6883 java.io.InputStreamReader.read(InputStreamReader.java:184) java.io.BufferedReader.fill(BufferedReader.java:154) java.io.BufferedReader.readLine(BufferedReader.java:317) - locked java.io.InputStreamReader@206a6883 java.io.BufferedReader.readLine(BufferedReader.java:382) nl.cwi.monetdb.mcl.io.BufferedMCLReader.readLine(BufferedMCLReader.java:119) nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:196) - locked nl.cwi.monetdb.mcl.io.BufferedMCLReader@29f38115 nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2296) - locked nl.cwi.monetdb.mcl.net.MapiSocket@319f9f37 nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2274) nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:525) com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:73) com.mchange.v2.c3p0.impl.DefaultConnectionTester.statusOnException(DefaultConnectionTester.java:143) com.mchange.v2.c3p0.AbstractConnectionTester.statusOnException(AbstractConnectionTester.java:82) com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPooledConnection.java:437) - locked com.mchange.v2.c3p0.impl.NewPooledConnection@3340302c com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:78) com.sokrati.moneta.mclient.JdbcMonetdbClient.executeUpdate(JdbcMonetdbClient.java:154) com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:46) com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:1) java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) java.util.concurrent.FutureTask.run(FutureTask.java:166) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) java.lang.Thread.run(Thread.java:722) Regards, Tapomay. _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 6

Re: users-list Digest, Vol 5, Issue 1
by swapnil joshi 03 Jan '13

03 Jan '13

Hi nitin, # Serving database 'demo', using 12 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 23.998 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information !IOException:mal_mapi.listen: operation failed: bind to stream socket port 50000 failed: solution 1 = It's an just port conflict error the port had assign to monetdb is already used by any other application please change the port of monetdb or find your another application which using same port and sopt it's service, then it will work. Solution 2 = If you are an student and if you are using your own system / personal computer then just shut down the system and restart it and first start monetdb server. If any only if you are studying monetdb on your personal computer . (solution 2 is foolish solution :) ) On Thu, Jan 3, 2013 at 4:30 PM, <users-list-request(a)monetdb.org> wrote: > nitin -- Regards, Swapnil K. Joshi _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

Error while starting monetdb on windows server 2008 R2
by nitin sharma 03 Jan '13

03 Jan '13

Hi there, I am getting following error while running monetdb server >>>># MonetDB 5 server v11.13.7 "Oct2012-SP2" # Serving database 'demo', using 12 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 23.998 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information !IOException:mal_mapi.listen:operation failed: bind to stream socket port 50000 failed: No error > Any pointer what could be the issue? thx,Nitin _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

failed to fork mserver: database 'lwg' started up, but failed to open up a communication channel
by Przemyslaw Wroblewski 28 Dec '12

28 Dec '12

hello, I'm trying to evaluate monetdb but I've failed at the beginning of the tutorial: http://www.monetdb.org/Documentation/monetdbd When I'm trying ot start up database I've got following: $ monetdb start lwg starting database 'lwg'... FAILED start: starting 'lwg' failed: database 'lwg' started up, but failed to open up a communication channel and in merovingian.log: 2012-12-27 16:43:26 MSG merovingian[29552]: startup of database under maintenance 'lwg' forced 2012-12-27 16:43:26 MSG merovingian[29552]: starting database 'lwg', up min/avg/max: 1s/1s/1s, crash average: 0.00 0.00 0.00 (1-1=0) 2012-12-27 16:43:27 MSG merovingian[29552]: sending process 9489 (database 'lwg') the TERM signal 2012-12-27 16:43:27 MSG lwg[9489]: arguments: /usr/bin/mserver5 --set gdk_dbfarm=/var/lib/monetdb --dbname=lwg --set merovingian_uri=mapi:monetdb://przemtest1:50000/lwg --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/lib/monetdb/lwg/.mapi.sock --set monet_vault_key=/var/lib/monetdb/lwg/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2012-12-27 16:43:27 MSG lwg[9489]: # MonetDB 5 server v11.13.7 "Oct2012-SP2" 2012-12-27 16:43:27 MSG lwg[9489]: # Serving database 'lwg', using 8 threads 2012-12-27 16:43:27 MSG lwg[9489]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2012-12-27 16:43:27 MSG lwg[9489]: # Found 23.591 GiB available main-memory. 2012-12-27 16:43:27 MSG lwg[9489]: # Copyright (c) 1993-July 2008 CWI. 2012-12-27 16:43:27 MSG lwg[9489]: # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved 2012-12-27 16:43:27 MSG lwg[9489]: # Visit http://www.monetdb.org/ for further information 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:batcalc.rank_grp[3]:object code for command sql.rank_grp missing 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:batcalc.rank_grp[1]:object code for command sql.rank missing 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:batcalc.dense_rank_grp[3]:object code for command sql.dense_rank_grp missing 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:batcalc.dense_rank_grp[1]:object code for command sql.dense_rank missing 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:aggr.median[7]:object code for command sql.middle missing 2012-12-27 16:43:27 MSG lwg[9489]: #WARNING: LoaderException:loadLibrary:Loading error could not locate library lsst (from within file 'lsst') 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:user.main[1240]:object code for command sql.prelude missing 2012-12-27 16:43:27 MSG lwg[9489]: !TypeException:user.main[1240]:'sql.prelude' undefined in: _1240:void := sql.prelude() 2012-12-27 16:43:27 MSG merovingian[29552]: database 'lwg' (9489) has exited with exit status 0 2012-12-27 16:43:27 MSG merovingian[29552]: database 'lwg' has shut down 2012-12-27 16:43:27 ERR control[29552]: (local): failed to fork mserver: database 'lwg' started up, but failed to open up a communication channel I'm on 64bit debian 6.0.4 in vserver container. It looks like exactly as http://bugs.monetdb.org/show_bug.cgi?id=3134 but I've latest version which should this bug already fixed. best regards Przemek _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

Fwd: Enhance Performance
by Arjun Harish 27 Dec '12

27 Dec '12

I am bench-marking monetDB against some other databases. I have a system with intel i-7 processor, 32GB Ram and 2 TB Harddisk on a Ubuntu 12.10 system. I have a data work load of 700 GB in monetdb now. How can i enhance the performance and speed up the querying process. Any Suggestions would we welcome and please reply in detail. I am a newbie to Monetdb. Arjun Data Scientist -- Arjun Harish Nadh Data Scientist Flutura Business Solutions Pvt. Ltd. +919972736428 arjun.harish(a)fluturasolutions.com _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

MonetDB Update: some problems while updating
by Miguel Ping 26 Dec '12

26 Dec '12

Hi all, We tried to update monetdb to the latest version, but encountered some problems. The prev version is working just fine. We're using Debian, and installed from the debs. See the log below. lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 6.0.4 (squeeze) Release: 6.0.4 Codename: squeeze ... 2012-12-26 11:43:42 MSG kpi_v1[13255]: # MonetDB 5 server v11.13.7 "Oct2012-SP2" 2012-12-26 11:43:42 MSG kpi_v1[13255]: # Serving database 'kpi_v1', using 24 threads 2012-12-26 11:43:42 MSG kpi_v1[13255]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2012-12-26 11:43:42 MSG kpi_v1[13255]: # Found 23.587 GiB available main-memory. 2012-12-26 11:43:42 MSG kpi_v1[13255]: # Copyright (c) 1993-July 2008 CWI. 2012-12-26 11:43:42 MSG kpi_v1[13255]: # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved 2012-12-26 11:43:42 MSG kpi_v1[13255]: # Visithttp://www.monetdb.org/ for further information 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:batcalc.rank_grp[3]:object code for command sql.rank_grp missing 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:batcalc.rank_grp[1]:object code for command sql.rank missing 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:batcalc.dense_rank_grp[3]:object code for command sql.dense_rank_grp missing 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:batcalc.dense_rank_grp[1]:object code for command sql.dense_rank missing 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:aggr.median[7]:object code for command sql.middle missing 2012-12-26 11:43:42 MSG kpi_v1[13255]: #WARNING: LoaderException:loadLibrary:Loading error could not locate library lsst (from within file 'lsst') 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:user.main[1240]:object code for command sql.prelude missing 2012-12-26 11:43:42 MSG kpi_v1[13255]: !TypeException:user.main[1240]:'sql.prelude' undefined in: _1240:void := sql.prelude() 2012-12-26 11:43:42 MSG merovingian[13238]: database 'kpi_v1' (13255) has exited with exit status 0 2012-12-26 11:43:42 MSG merovingian[13238]: database 'kpi_v1' has shut down 2012-12-26 11:43:42 ERR control[13238]: (local): failed to fork mserver: database 'kpi_v1' started up, but failed to open up a communication channel Thanks _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

UDF compilation
by Miguel Ping 21 Dec '12

21 Dec '12

What are the steps required to include additional .c and .h files in compiling user-defined functions? It was a long time since I used autotools and such; it seems to me that I would need to change the monetdb configuration files in order to regenerate the Makefile.* files within the UDF dir. My problem is that I require alot of code for my UDF, and right now I'm copy/paste'ing code to udf.c and udf.h; it would be easier to include additional .h and .c files for UDF compilation. Thanks _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

Announcement: New Oct2012-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Dec '12

21 Dec '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP2 bugfix release SQL * Fixed crash when performing an INSERT on a table with string-like column defaulting to NULL and omitting that column from VALUES, bug #3168 Java Module * Implemented type map support of Connection to allow custom mapping of UDTs to Java classes. By default the INET and URL UDTs are now mapped to nl.cwi.monetdb.jdbc.types.{INET,URL}. Most notably, ResultSet.getObject() and PreparedStatement.setObject() deal with the type map. * Fixed a problem in PreparedStatement where the prepared statement's ResultSetMetaData (on its columns to be produced) incorrectly threw exceptions about non existing columns. Bug #3192 Bug Fixes * 2291: small doubles end up as NULL after arithmetic * 2579: SQL: incorrect cast from double to int * 2659: tuples INTERSECT vs count (... INTERSECT ...) differs * 2805: row_number doesn't work in complex query * 2886: warning: Catalog parameter '' ignored * 2977: timestamp minus date results in program error * 2978: now() + 1 results in illegal argument * 2999: Up-cast from SQL drops order information? * 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause * 3124: insert into table with two foreign keys crashes mserver5 * 3136: Memleak (in querycache?) while querying * 3143: Type resolution error in SQL procedural code * 3163: mserver5 crashes with large number of concurrent clients * 3168: crash when insert varchar/char/clob with default NULL * 3169: php driver uses call-time pass-by-reference * 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either * 3181: Join query returns zero results when there is an index on the tables * 3185: inconsistency between documentation and implementation of command line options * 3186: case evaluted to early * 3188: Storage of client history * 3189: Overflow in calculation * 3190: When subtracting two columns of type "timestamp", no results are returned * 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed. * 3192: "No such column with index" error if CASE in SELECT clause * 3199: Inconspicuous sequence of prepare, execute, commit and close produces error * 3205: Inet datatype handling inconsistent between int, cast( as inet) and string _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

OLTP Benchmark on Monetdb
by Hassan, Ahmad 19 Dec '12

19 Dec '12

Hi Team, I am trying to run TPC-C benchmark provided by opensource oltpbenchmark (http://oltpbenchmark.com) on MonetDB. But I am getting JDBC Exception while loading the data set: java.sql.BatchUpdateException: Error(s) occurred while executing the batch, see next SQLExceptions for details at nl.cwi.monetdb.jdbc.MonetStatement.executeBatch(MonetStatement.java:209) Exception in thread "main" java.lang.ArithmeticException: / by zero at com.oltpbenchmark.benchmarks.tpcc.TPCCLoader.load(TPCCLoader.java:1857) Please can you let me know any TPC-C benchmark implementation that is compatible with monetdb? Thanks. Kind Regards, Ahmad _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

MonetDB compiling with CLANG/LLVM
by Hassan, Ahmad 18 Dec '12

18 Dec '12

Hi, I want to ask if someone has tried compiling MonetDB with Clang/LLVM compiler? I intend to compile MonetDB with LLVM for analysing the query execution at runtime. Does someone know the changes in the build process required for building Monetdb with clang/llvm? Thanks. Cheers, Ahmad _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

Unable to execute prepared statements with a large number of parameters (say over 140 or so)
by Percy Wegmann 14 Dec '12

14 Dec '12

I couldn't find any existing bugs that cover this, so I logged a new one: http://bugs.monetdb.org/show_bug.cgi?id=3208 This is kind of a show-stopper for me right now, so I'm hoping that someone might have some idea where to start looking for a solution. I'm debugging sql_scenario.c right now, but it'll take me some time to get familiar with the code. Cheers, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

Still searching to find the Octopus :-)
by Masood Mortazavi 13 Dec '12

13 Dec '12

I'm still searching for user documentation that would explain MonetDB/octopus usage/feature. Does any exist? Thanks, Masood P.S. My earlier e-mails regarding the octopus documentation was somewhat rushed, it seems. The documentation in the man pages are actually for remtoe access and funnel-multiplex features, which seem entirely different. _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

Does anybody know the endian ness (little, big) of BAT file?
by swapnil joshi 10 Dec '12

10 Dec '12

Hi, Does anybody know the endian ness (little, big) of BAT file? -- Regards, Swapnil K. Joshi _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

Adding group_concat support
by Tapomay Dey 07 Dec '12

07 Dec '12

Just confirming. Can we yet define functions without having to recompile. Planning to try adding group_concat support. Regards, Tapomay. _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 2

Re: Failed to load 128GB CSV file using Oct2012
by Stefan Manegold 05 Dec '12

05 Dec '12

Jennie, just to be sure: are you sure there is sufficient free disk space on your machine, say, twice the expected database size before starting the load? How large is your database (and sql_logs) just after the failure while the server is still running? How much disk space is still free at that time? Stefan Ying Zhang <Y.Zhang(a)cwi.nl> wrote: Hai all, I try to load a CVS file of ~128GB in the ssdb branch (based on Oct2012), but after quite sometime, it failed with: #GDKrealloc(19598973840) fails, try to free up space [memory in use=22695528216,virtual memory in use=138875848472] #GDKrealloc(19598973840) result [mem=22695528216,vm=138875848472] In the last Sep., I was able to execute the same query with the same data. But now it takes much longer and ends in a crash. For INS1-ers, the command to start the server: /export/scratch2/zhang/monet-install/ssdb/debug/bin/mserver5 --set gdk_dbfarm=/export/scratch2/zhang/dbfarm --dbname=ssdb --set mapi_usock=/export/scratch2/zhang/dbfarm/ssdb/.mapi.sock --set monet_vault_key=/export/scratch2/zhang/dbfarm/ssdb/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set mapi_port=60000 The command to start the query: cd /ufs/zhang/papers/ssdb/monetdb ./run.sh loadimg small output Does anyone have an idea? If anymore information is needed, please let me know. I'm re-running the query to get a GDB stack trace. Thanks! Jennie _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 2

Query takes too much time
by Ivan Videla 05 Dec '12

05 Dec '12

Hi All, I have a table with 268 millions of records from supermarket transactions tran_id, prod_id, quantity, price, cost and i want to make a calculation like sum(quantity*(price-cost)). But this query takes too much time. I want to know if i can improve the results using an index. Or monetdb itself use an index on-the-fly. If not i would like to know how can i make an index to improve the execution time. Regards, Ivan _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 12

Failed to load 128GB CSV file using Oct2012
by Ying Zhang 05 Dec '12

05 Dec '12

Hai all, I try to load a CVS file of ~128GB in the ssdb branch (based on Oct2012), but after quite sometime, it failed with: #GDKrealloc(19598973840) fails, try to free up space [memory in use=22695528216,virtual memory in use=138875848472] #GDKrealloc(19598973840) result [mem=22695528216,vm=138875848472] In the last Sep., I was able to execute the same query with the same data. But now it takes much longer and ends in a crash. For INS1-ers, the command to start the server: /export/scratch2/zhang/monet-install/ssdb/debug/bin/mserver5 --set gdk_dbfarm=/export/scratch2/zhang/dbfarm --dbname=ssdb --set mapi_usock=/export/scratch2/zhang/dbfarm/ssdb/.mapi.sock --set monet_vault_key=/export/scratch2/zhang/dbfarm/ssdb/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set mapi_port=60000 The command to start the query: cd /ufs/zhang/papers/ssdb/monetdb ./run.sh loadimg small output Does anyone have an idea? If anymore information is needed, please let me know. I'm re-running the query to get a GDB stack trace. Thanks! Jennie _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

INET query/type conversion issues
by Percy Wegmann 04 Dec '12

04 Dec '12

Hi MonetDB Team, I encountered a strange problem related to querying against an INET column. To reproduce: percy@percy-sandbox:~$ uname -a Linux percy-sandbox 3.2.0-33-generic #52-*Ubuntu* SMP Thu Oct 18 16:29:15 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux percy@percy-sandbox:~$ mclient -u monetdb -d click password: Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2012) Database: *MonetDB v11.13.3 (Oct2012)*, 'mapi:monetdb://percy-sandbox:50000/click' Type \q to quit, \? for a list of available commands auto commit mode: on sql>create table test_table(col inet); operation successful (27.394ms) sql>insert into test_table values(*'127.0.0.1'*); 1 affected row (20.370ms) sql>insert into test_table values(*inet '127.0.0.1'*); 1 affected row (19.654ms) sql>insert into test_table values(*cast('127.0.0.1' as inet)*); 1 affected row (25.038ms) sql>select count(*) from test_table where col = '127.0.0.1'; +------+ | L1 | +======+ | *3* | +------+ 1 tuple (1.547ms) sql>select count(*) from test_table where col = inet '127.0.0.1'; +------+ | L1 | +======+ | *0* | +------+ 1 tuple (0.946ms) sql>select count(*) from test_table where col = cast('127.0.0.1' as inet); +------+ | L1 | +======+ | *2* | +------+ 1 tuple (0.982ms) sql>select count(*) from test_table where col = inet '127.0.0.1'; +------+ | L1 | +======+ | *2* | +------+ 1 tuple (0.937ms) sql>select count(*) from test_table where col = inet '127.0.0.1'; +------+ | L1 | +======+ | *0* | +------+ 1 tuple (0.801ms) Expected Result: All queries return a count of 3 Actual Result: Queries return different counts depending on how INET value was bound to query. Particularly interesting is that the output when using the inet '127.0.0.1' syntax is non-deterministic. Does this ring any bells? Should I log it? Cheers, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 1