- users-list - monetdb.org

[MonetDB-users] DBD::monetdb and database name
by Dave Howorth 14 Apr '10

14 Apr '10

I just started trying to use DBD::MonetDB and ran into a couple of issues. I'm not sure where you prefer to get reports so I thought I'd send it here first. First, although both the version on CPAN and the version on my machine, presumably from the MonetDB installation, both claim to be version 0.09, they are different. IMHO, this is a Bad Idea. Having the up-to-date version on CPAN is desirable since that's what Perl programmers expect. Having two versions with the same number is essential, since otherwise CPAN's and Perl's dependency mechanisms will fail and bug reporting becomes a real problem. Second, and what caused me to investigate more carefully, when I typed in the first example program for DBD::monetdb I saw: $ ./monetdb-test-1.pl DBI connect('','',...) failed: merovingian: no such database 'demo', please create it first That doesn't surprise me, because I didn't create a demo database. But I was surprised when I couldn't find any facility to name the database in the documentation or the code. Except when I actually read the installed code as opposed to the published version on CPAN I see that there is. So please add docs. I've suggested a patch below. Cheers, Dave PS The code will compile a little faster if you use __END__ --- /usr/lib/perl5/DBD/monetdb.pm 2010-03-22 16:22:51.000000000 +0000 +++ perl/DBD/monetdb.pm 2010-04-14 15:18:13.000000000 +0100 @@ -670,6 +670,8 @@ 1; +__END__ + =head1 NAME DBD::monetdb - MonetDB Driver for DBI @@ -783,6 +785,7 @@ $dsn = 'dbi:monetdb:'; $dsn = "dbi:monetdb:host=$host"; $dsn = "dbi:monetdb:host=$host;port=$port"; + $dsn = "dbi:monetdb:host=$host;database=$database"; $dbh = DBI->connect($dsn, $user, $password); @@ -794,7 +797,11 @@ =item port -The port where MonetDB daemon listens to. Default for MonetDB is 50000. +The port where MonetDB daemon listens. Default for MonetDB is 50000. + +=item database + +The name of the database to connect to. =back

2 1

[MonetDB-users] how to set the vm_maxsize and mem_maxsize
by HotMan 14 Apr '10

14 Apr '10

If i want to restrict monetdb (ie let it use less than the available memory) i don't konw how to set the vm_maxsize and mem_maxsize.could you help me? my os is centos 5.2 thank you ! --HotMan -- View this message in context: http://old.nabble.com/how-to-set-the-vm_maxsize-and-mem_maxsize-tp28220922p… Sent from the monetdb-users mailing list archive at Nabble.com.

3 3

[MonetDB-users] monetdb is a MMDB?when this db save the data into disk?
by zhliangzh 14 Apr '10

14 Apr '10

monetdb is a MMDB?i don't konw when this db save the data into disk? -- View this message in context: http://old.nabble.com/monetdb-is-a-MMDB-when-this-db-save-the-data-into-dis… Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

[MonetDB-users] error when connecting with iSQL-Viewer
by Dave Howorth 13 Apr '10

13 Apr '10

I've just installed iSQL-Viewer and used its wizard to set up a connection to the VOC database that I loaded into MonetDB. But the connection doesn't work; instead I see an error apparently from MonetDB: java.sql.SQLException: ResultSet is closed! at nl.cwi.monetdb.jdbc.MonetResultSet.absolute(MonetResultSet.java:181) at nl.cwi.monetdb.jdbc.MonetResultSet.relative(MonetResultSet.java:2014) at nl.cwi.monetdb.jdbc.MonetResultSet.next(MonetResultSet.java:1981) at org.isqlviewer.sql.ResultSetWrapper.next(ResultSetWrapper.java:1194) at org.isqlviewer.ui.JdbcWorkbench.buildSchemaMenu(JdbcWorkbench.java:1145) at org.isqlviewer.ui.JdbcWorkbench.changeConnection(JdbcWorkbench.java:695) at org.isqlviewer.ui.JdbcWorkbench.actionPerformed(JdbcWorkbench.java:290) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.isqlviewer.swing.action.SwingEventManager$EventLoop.delegateEvent(SwingEventManager.java:340) at org.isqlviewer.swing.action.SwingEventManager$EventLoop.run(SwingEventManager.java:319) at java.lang.Thread.run(Thread.java:636) I've googled the error message without finding anything relevant. I know the VOC database itself works because I managed to connect to it via both the command-line and using SQuirreL. The merovingian.log just says: 2010-04-13 14:06:14 MSG merovingian[7379]: proxying client pcx36.lmb.internal:52677 for database 'test' to mapi:monetdb://127.0.0.1:50002/test Can anybody give me a clue what might cause that error message? Thanks, Dave

2 4

[MonetDB-users] "Copy into ... from STDIN" end of file
by Matthieu Guamis 09 Apr '10

09 Apr '10

Hi, I'm trying to load data using copy command and STDIN source in command line but the process ends without waiting for records. I think this is first step to load data from a second process without using file source. Here is the command: mclient -l sql -d database -u user -P pwd -s "COPY INTO table FROM STDIN" It returns immediately with exit code 0. With postgresql, the "same" command (psql -U user -h host database -c "\copy table from STDIN") prompt for datas and wait until antislash-dot ("\.") input ends records so it allows a process to load datas using STDIN. Do you think this is possible with mclient sql copy command? Regards, Matthieu

2 3

[MonetDB-users] monetdb permission denied
by Dave Howorth 09 Apr '10

09 Apr '10

Hi, noob question ... I've just installed MonetDB on an Ubuntu Karmic machine. I've got merovingian running: monetdb 7379 1 0 Apr06 ? 00:00:01 /usr/bin/merovingian but I see permission denied when I try to use monetdb on the same machine: $ monetdb status permission denied As an experiment, I have added my user to the monetdb group and I have changed the permissions so that the monetdb group can access the socket: root:/var/MonetDB5/dbfarm# ls -la total 12 drwxrwx--- 2 monetdb monetdb 4096 Apr 6 17:13 . drwxrwx--- 3 monetdb monetdb 4096 Apr 6 17:13 .. srwxrwx--- 1 monetdb monetdb 0 Apr 6 17:13 .merovingian_control -rw------- 1 monetdb nogroup 0 Apr 6 17:13 .merovingian_lock -rw------- 1 monetdb nogroup 48 Apr 6 17:13 .merovingian_pass But I still see permission denied. What's the correct way to set the system up? A pointer to whichever piece of the docs that I've missed would be ideal :) Thanks, Dave

4 8

[MonetDB-users] Announcement: New Feb2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 08 Apr '10

08 Apr '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2010-SP1 bug fix release of the MonetDB suite of programs. More information on this release is available at <http://monetdb.cwi.nl/Development/Releases/Feb2010/>. -- Sjoerd Mullender

2 1

[MonetDB-users] Issue with large number of databases
by Steve Benz 05 Apr '10

05 Apr '10

Hi, We've been pilot testing monet with a small number of db's (~10) for a few months now with great success. We're getting ready to roll out databases for a larger number of databases, but we're experience some issues. We're hoping to run about 130 databases concurrently, however when we get above 55 the command `monetdb status` appears to experience issues: WARNING: failed to parse response from merovingian: IllegalArgumentException:sabaoth.deserialise:string does not contain path: /usr/local/MonetDB/var/Monet (it then successfully lists 54 of the databases) (we have our monetdb installed in /usr/local) The merovingian.log doesn't show any errors, just: 2010-04-05 09:55:58 MSG control[2260]: (local): served status list There also doesn't appear to be any odd characters in any of the db names. Has anyone successfully tested this number of db's running on the same server? Thanks for any help you can provide! Steve

2 1

[MonetDB-users] Superball build failure
by Matthew Jones 02 Apr '10

02 Apr '10

I recently downloaded the MonetDB-Feb2010-SuperBall-SP1.tar.bz2 file from the site in order to build MonetDB and I get this build failure: (...) config.status: creating mal_config.h config.status: executing depfiles commands config.status: executing libtool commands make all-recursive make[1]: Entering directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5' Making all in src make[2]: Entering directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5/src' make all-recursive make[3]: Entering directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5/src' Making all in mal make[4]: Entering directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5/src/mal' make[4]: Leaving directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5/src/mal' make[4]: *** No rule to make target `mal.c', needed by `all'. Stop. make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5/src' make[2]: Leaving directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5/src' make[1]: Leaving directory `/var/tmp/MonetDB-1270235519/MonetDB5-server/MonetDB5' Please refer to the full log at /var/tmp/MonetDB-1270235519/log If you believe this is an error in the script or the software, file a bug and attach the logfile. make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 making MonetDB5-server failed I actually also see this error if I build the server package standalone after building and installing the pre-reqs 73, Matthew W. Jones (KI4ZIB) http://matburt.net

2 4

Re: [MonetDB-users] [Monetdb-developers] dealing with 1G XML file in MonetDB/XQuery
by Martin Kersten 02 Apr '10

02 Apr '10

Mo Zhou wrote: > Hello, > > Could you please reply as soon as possible? From your email it is unclear what platform you are using. I guess it is a Windows 32bit version. The document you use may easily consume more memory when processed, exceeding the 32-bit addressing scheme. A 64-bit version is advised. (See earlier threads on this topic) regards, Martin > > > Thanks, > Mo > On Thu, Apr 1, 2010 at 4:49 PM, Mo Zhou <mozhou(a)umail.iu.edu> wrote: >> Hi, >> >> When I load a 1G XML file to MonetDB/XQuery, it throws exceptions as >> follows. I did not find related parameters in the monetDB config file >> to enlarge the heap size. I wonder how I can solve this problem? Thank >> you. >> >> #GDKmmap(1073741824) fails, try to free up space [memory in >> use=156511940,virtual memory in use=1309212672] >> #GDKmmap(1073741824) result [mem=156511940,vm=1309212672] >> MAPI = mozhou@localhost:40000 >> QUERY = pf:add-doc("/.../new1GL1Comp/xmldoc3.xml","1GL1doc3") >> ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. >> !ERROR: [shred_url]: first error was: >> !ERROR: HEAPextend: failed to extend to 1073741824 for 03/02/30243theap >> !ERROR: shredBAT_append_str: >> APPEND-STR[_prop_text](http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromVendor20/Offer31281/), >> BUNappend fails >> !ERROR: CMDshred_url: operation failed. >> >> -- >> Thanks, >> Mo >> > > >

2 1

Re: [MonetDB-users] [Monetdb-developers] the database and index size
by Stefan de Konink 01 Apr '10

01 Apr '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 01-04-10 17:33, Mo Zhou schreef: > Does monetdb compress data in the database? I load plain text file > (12M) into a database with only one table. I shut down the database > right after that and check the size of the database. The database size > is only 8M. Obviously the amount of storage required for storing integers larger than 4 digits in plain text is higher than the amount of data that is required to represent a 4byte integer. Regarding to strings best effort deduplication is choosen. Like with any fileformat there is some metadata overhead. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAku0xBsACgkQYH1+F2Rqwn2vWwCdFts9Z8wYtLEsrqslimq/I3Vi OWEAnjIArIUtzI8yf4mhdlHF/Fy1pS5x =aXYs -----END PGP SIGNATURE-----

1 0

[MonetDB-users] ubuntu server installation / php modules
by dariuszs 29 Mar '10

29 Mar '10

Hi, I've installed monetdb on *Ubuntu 9.04 Server* with apt-get. These are the contents of php client /. /usr /usr/share /usr/share/doc /usr/share/doc/php5-monetdb-client /usr/share/doc/php5-monetdb-client/changelog.Debian.gz /usr/share/doc/php5-monetdb-client/copyright I need the moduels, where are they? Thanks. Dariusz.

2 3

[MonetDB-users] can't log in to my own databases...
by Robert Dormer 28 Mar '10

28 Mar '10

Hello all, I've installed monetdb from the Feb 2010 source release on a Gentoo 10.1 box. I've created a database (finance), and have released it from administration mode, but every time I try to access it with mclient, I get a prompt for a user name and password, followed by an InvalidCredentialsException:checkCredentials error. When I created the database it didn't ask for any user name or password. How do I get in to my own database?

2 1

[MonetDB-users] doco on syntax for SQL create function?
by Michael Pheasant 27 Mar '10

27 Mar '10

Hi I'm trying to create an SQL function but struggling with thi syntax for the body of the function. Is there other more detailed doco than this page? http://monetdb.cwi.nl/SQL/Documentation/Functions.html#Functions Cheers Mike

2 1

[MonetDB-users] PHP Benchmarks of Native versus Cimpl?
by Steve Benz 25 Mar '10

25 Mar '10

Hi~ I've recently converted our monet install from the cimpl version of the php interface to the new native socket interface. I see the readme references some benchmarks at a url that does not appear to exist (http://www.ins.cwi.nl/~gm/bench/) Would someone be able to please point me to a version of these? Thanks so much! Steve

5 7

[MonetDB-users] Virtual Memory Requirement
by Hering Cheng 25 Mar '10

25 Mar '10

Hi, I am trying to understand the limitations of MonetDB with regard to memory. It seems that the amount of virtual memory (RAM plus swap space) available to MonetDB must be greater than the amount of data. Before I give up on MonetDB, can someone please confirm or correct my understanding? I did not understand the virtual memory requirement when I read the following in its documentation ( http://monetdb.cwi.nl/MonetDB/Documentation/Size-Limitations.html#Size-Limi… ): *The maximal database size supported by MonetBD depends on the underlying processing platform, e.g., a 32- or 64-bit processor, and storage device, e.g., the file system and disk raids. * For example, I started up merovingian (using the Feb 2010 release), limiting it and its child processes to 32 GB of virtual memory (via ulimit -v). I then try to load 115 GB of data into a single table. mclient reports the following: MAPI = chenher@myserver50000 COPY 472112651 RECORDS INTO mytable FROM 'mydata.txt' DELIMITERS '|', '\n'; ERROR = !SQLException:importTable:failed to import table !ERROR: HEAPextend: failed to extend to 1888450604 for 17/1734tail !ERROR: TABLETcreate_bats: Failed to create bat of size 472112651 I understand if MonetDB must be able to fit all its data within virtual memory. But if this is the case, then unfortunately I cannot use it, since I have terabytes of data and I do not have that much virtual memory on my hardware. Thank you. Hering

4 8

[MonetDB-users] How to run monetdb and create a database, with the geom module?
by Michael Pheasant 25 Mar '10

25 Mar '10

I cannot figure out how to do geom stuff with the Fedora 12 rpm binary install. If I run $ sudo mserver5 --dbinit="include geom; include sql;sql.start();" then in another shell try and create a test database i get an error than monetdb (merovingian) is not running: $ sudo monetdb create test monetdb: cannot perform: MonetDB Database Server (merovingian) is not running If I run merovingian, then it starts the MonetDB with --dbinit="include sql" I can then create a database, but I cant load the GEOM extensions. So I feel like I end up in a catch-22. Can someone explain how I can run monetdb, and create a database, with the geom module? Cheers Mike.

2 1

[MonetDB-users] Load a file.txt in bats
by Miguel Ángel Águila Lorente 24 Mar '10

24 Mar '10

Hello, I'm creating a database with MAL. I would need help for creating bats with the information of a file. May be It could be posible to do with tablet.load but I still not know how it works. The format of the file is quite simple: int;int;int; int;int;int; I expect you can help me. Thank you. Mike

2 3

[MonetDB-users] Java executeBatch() behavior
by sateesh 23 Mar '10

23 Mar '10

Hi All, When we try to execute a Set of SQL Statements using executeBatch() and if one of the SQL Query fails then MonetDB seems to be ignoring the rest. Is there a way we can configure MonetDB to execute the remaining SQL Queries in the Batch ?? Thanks Sateesh -- View this message in context: http://old.nabble.com/Java-executeBatch%28%29-behavior-tp27783879p27783879.… Sent from the monetdb-users mailing list archive at Nabble.com.

2 6

[MonetDB-users] Transaction Management
by אייל 22 Mar '10

22 Mar '10

hi, this is from the documentation of Transaction Management: "If at commit time it can be assured that the data prepared for update affects tables not changed in the mean time, the results are merged." dose it mean that two users cannot change DIFFERENT records of the same table at the same time? what isolation levels dose MonetDB supports? thanks.

1 0

Re: [MonetDB-users] multiple result sets
by אייל 22 Mar '10

22 Mar '10

Martin, thanks again for your response. lastly i made experiments with mysql (innoDB and innoDB-plugin), postgreSQL and MonetDB. MonetDB won even those queries that weren't the reason MonetDB invented for. yes it's an amazing platform. doing what you suggested with JDBC is possible, but not convenient. for me, without getting into the details of the project that already works against mssql, i would just say it's too much of an effort. i think this feature can help MonetDB become more like a real world database, than an amazing platform for academic research. thanks.

1 0

Re: [MonetDB-users] multiple result sets
by אייל 21 Mar '10

21 Mar '10

sending it again due to a mistake in the subject. sorry. Martin, thanks for your response. my actual application is now served by a mssql server, which has many stored-procedures, some of them are 45kb of TSQL-code, that may return about ten result-sets at once. these procedures are normally executed within about 50msec. having this in mind... for the sake of our discussion only, lets define : "optimization" : automatically turning queries into query templates. "compilation" : parsing the bulk of statements + deciding for each of the queries which "optimization" should be used (or built - if no suitable "optimization" found). "bandwidth": sending a large bulk of statements from the client to the server. now i would like ask: dose MonetDB stores "compilation" for each stored-procedure for reuse? if the answer is "no", then i understand: "optimization" i pay only once :) "compilation" i pay many times :( "bandwidth" i pay only once :) if the answer is "yes": "optimization" i pay only once :) "compilation" i pay only once :) "bandwidth" i pay only once :) the situation today is i cannot use stored-procedure (because they don't return result sets), so it actually dose not meter whether the answer is "yes" or "no": "optimization" i pay only once :) "compilation" i pay many times :( "bandwidth" i pay many times :( since MonetDB actually support returning multiple result-sets (without stored procedure), and it even support returning a single result set (using stored-function), it looks like adding this feature to stored-procedures as well, is not so much of a work (i'm thinking of contributing this work myself, having your kindly help and general directions). but if MonetDB dose NOT even store a "compilation" for each stored procedure, i guess we are too much far away... note: every time i said "stored-procedure", i actually meant "stored-function" as well. thanks again.

2 1

Re: [MonetDB-users] Virtual Memory Requirement
by אייל 21 Mar '10

21 Mar '10

Martin, thanks for your response. my actual application is now served by a mssql server, which has many stored-procedures, some of them are 45kb of TSQL-code, that may return about ten result-sets at once. these procedures are normally executed within about 50msec. having this in mind... for the sake of our discussion only, lets define : "optimization" : automatically turning queries into query templates. "compilation" : parsing the bulk of statements + deciding for each of the queries which "optimization" should be used (or built - if no suitable "optimization" found). "bandwidth": sending a large bulk of statements from the client to the server. now i would like ask: dose MonetDB stores "compilation" for each stored-procedure for reuse? if the answer is "no", then i understand: "optimization" i pay only once :) "compilation" i pay many times :( "bandwidth" i pay only once :) if the answer is "yes": "optimization" i pay only once :) "compilation" i pay only once :) "bandwidth" i pay only once :) the situation today is i cannot use stored-procedure (because they don't return result sets), so it actually dose not meter whether the answer is "yes" or "no": "optimization" i pay only once :) "compilation" i pay many times :( "bandwidth" i pay many times :( since MonetDB actually support returning multiple result-sets (without stored procedure), and it even support returning a single result set (using stored-function), it looks like adding this feature to stored-procedures as well, is not so much of a work (i'm thinking of contributing this work myself, having your kindly help and general directions). but if MonetDB dose NOT even store a "compilation" for each stored procedure, i guess we are too much far away... note: every time i said "stored-procedure", i actually meant "stored-function" as well. thanks again.

1 0

Re: [MonetDB-users] multiple result sets
by אייל 18 Mar '10

18 Mar '10

Fabian, thanks for your response. i did try it. as far as i can see, stored procedures cannot return result set at all... i hoped to get an official verification for my findings. and maybe any work around. about the network traffic: the result would be the same with or without stored procedure, but the request would be much smaller. sending a request like: "call sp1(x,y,z);" is much smaller than a very large bulk of statements. of course, the more critical point is to avoid unnecessary re-compilation of a large and complex bulk of statements, that should be able to execute many times during a single second. in mssql, and mysql, procedures do able to return multiple result sets. i don't know what is the standard sql2003 about it. do you think it worth a bug\feature report? thanks, again.

3 2

[MonetDB-users] multiple result sets
by אייל 18 Mar '10

18 Mar '10

hi, dose monetdb support returning multiple result sets by a stored procedure or a stored function? something equevalent to this (mssql code): create procedure p1 as begin select 'a' as f1; select 'b' as f1, 'c' as f2 end; any workaround?? i realized that if i do (in mclient terminal): sql>select 'a' as f1; select 'b' as f1 ,'c' as f2; i get: f1 -- a f1|f2 ----- b |c but this is not good enough for me. the actual procedures are long and complex, and i want to gain the benefit of avoiding the re-compilation and reduced network traffic (in mssql it makes big difference for me). i also realized that i can do: create function f1() returns table... but this is good for a single result only. BTW: dose the monetdb-jdbc-driver support multiple result sets (java.sql.Statement.getMoreResult() etc.)? thanks!

2 1

[MonetDB-users] newbie question about monetdb5 server installation
by Min Zhou 16 Mar '10

16 Mar '10

hi all, I failed in install moentdb beacause the configure script can't find the correct openssl path. here is my instruction ./monetdb-install.sh --prefix=$HOME/monetdb --enable-sql the failure hapened when compiling monetdb5 server with informations like below: configure: error: MonetDB 5 requires OpenSSL (you need a more recent version of OpenSSL) configuring MonetDB5-server failed Please refer to the full log at /var/tmp/MonetDB-1268650686/log If you believe this is an error in the script or the software, file a bug and attach the logfile. Terminated here is my openssl information: $which openssl /usr/local/ssl/bin/openssl $openssl version OpenSSL 0.9.8m 25 Feb 2010 How can i fix it? thanks, Min -- My research interests are distributed systems, parallel computing and bytecode based virtual machine. My profile: http://www.linkedin.com/in/coderplay My blog: http://coderplay.javaeye.com

4 8

[MonetDB-users] Cursor Alternative
by Nafees Ur Rehman 16 Mar '10

16 Mar '10

Hi, Its mentioned on your website (http://monetdb.cwi.nl/SQL/Documentation/SQL-Features.html#SQL-Features) that monetDB/SQL does not support cursors. My question is whether it has any alternative for cursors? I am working on integrating Data Mining techniques with Database systems. So I need to translate the DM algorithms and write them in monetDB. Regards Nafees Ur Rehman University of Konstanz.

2 1

[MonetDB-users] (no subject)
by Abernethy, Simon 15 Mar '10

15 Mar '10

Dear Sir I have been testing the lastest version of mserver5 under Linux on an embedded arm platform (Marvell PXA270 XScale 600Mhz) and a PC. MonetDB server v5.16.5 kernel v1.34 I have built it for gumstix (arm based platform) and for PC I have built it using the monetdb-install.sh with the following parameters export EXTRA_ECONF="-enable-optimize -disable-debug -enable-strict -with-gcc=arm-angstrom-linux-gnueabi-gcc -host=arm-angstrom-linux-gnueabi -with-openssl=$HOME/gumstix/openssl -without-perl -without-ruby -without-python -without-php -without-java -without-swig without-libxml2 -with-pcre=$HOME/gumstix/pcre . /monetdb-install.sh -nightly=stable -prefix=$HOME/gumstix/monet -build ==$HOME/gumstix/monetBuildGum -enable-sql Unfortunately using 2 instances of my test program (test1.cpp attached) writing to the database at the same time, I see the memory being consumed and not released - it is very noticeable on a gumstix and will use all the memory within a hour. If I stop the writer applications the memory usage remains constant. No further memory is released If I halt the database server and reload the same same database I get a considerable reduction in the memory i.e. mserver5 memory footprint on a gumstix (http://www.gumstix.com/ verdex-pro 11% memory usage at Start running 2 instances of the test application (test1) 35.8% memory running for 3mins then stopping the write (test1) applications 10.3% memory after stopping and restarting database 12.8% memory after reloading database The test programs were run for only a few minutes. I also have valgrind reports of the same programs being run a the pc if helpful Simon Abernethy Software Engineer MiPlaza Philips Research Europe WB6.014 (HTC34 6.14) 5656 AE Eindhoven, The Netherlands Tel +31 40 27 49566 Fax +31 40 27 42081 Email simon.abernethy(a)philips.com<mailto:simon.abernethy@philips.com> http://www.miplaza.com ________________________________ The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

1 0

[MonetDB-users] merovingian module loading
by Mitja Kleider 09 Mar '10

09 Mar '10

Hello, I would like to use merovingian and load the geom module as if I would be running > mserver5 --dbinit="include geom;include sql;sql.start();" Is this possible with merovingian? Regards Mitja

2 1

[MonetDB-users] sql query size
by dariuszs 06 Mar '10

06 Mar '10

Hi, Is there any limit on how big the sql query can be (in bytes)? Thanks. Dariusz.

3 3

[MonetDB-users] fetch_field errors.
by Guillaume Yziquel 04 Mar '10

04 Mar '10

Hello. I've been reading, in Mapi.mx, the following: > @item char **mapi_fetch_field_array(MapiHdl hdl) > > Return an array of string pointers to the individual fields. A zero > is returned upon encountering end of sequence or error. This can be > analyzed in using @code{mapi\_error()}. As I'm not a big fan of dynamic type checks, or dynamic error checks, I'd appreciate knowing: what kind of errors might this be? Any errors assuming anything might fail? Just to know... -- Guillaume Yziquel http://yziquel.homelinux.org/

2 3

[MonetDB-users] Cannot load libMapi with C++ after compiling and linking
by Alexander Koglin 03 Mar '10

03 Mar '10

Hi everybody, I tried to setup an Eclipse project by linking the libMapi against it and adding some entities to MonetDB SQL and always get the following error: *.../Project/Debug/Project: error while loading shared libraries: libMapi.so.1: cannot open shared object file: No such file or directory* After all linking and compiling goes alright. When Invoking the GCC C++ Compiler I call the following directive: *g++ -I/usr/local/Tarari/include -I/root/MonetDB/include/MonetDB -I/root/xerces-2.8/include -O0 -g3 -Wall -c -fmessage-length=0 `monetdb-clients-config --cflags --libs` -MMD -MP -MF"src/IndexHandler.d" -MT"src/IndexHandler.d" -o"src/IndexHandler.o" "../src/IndexHandler.cpp" g++ -I/usr/local/Tarari/include -I/root/MonetDB/include/MonetDB -I/root/xerces-2.8/include -O0 -g3 -Wall -c -fmessage-length=0 `monetdb-clients-config --cflags --libs` -MMD -MP -MF"src/MonetMapiUpdater.d" -MT"src/MonetMapiUpdater.d" -o"src/MonetMapiUpdater.o" "../src/MonetMapiUpdater.cpp" g++ -I/usr/local/Tarari/include -I/root/MonetDB/include/MonetDB -I/root/xerces-2.8/include -O0 -g3 -Wall -c -fmessage-length=0 `monetdb-clients-config --cflags --libs` -MMD -MP -MF"src/NodeFetcher.d" -MT"src/NodeFetcher.d" -o"src/NodeFetcher.o" "../src/NodeFetcher.cpp"* When invoking the GCC C++ Linker I call the following directive: *g++ -L/usr/local/Tarari/lib -L/root/MonetDB/lib -L/root/xerces-2.8/lib `monetdb-clients-config --cflags --libs` -o"NodeFetcherCPP" ./src/IndexHandler.o ./src/MonetMapiUpdater.o ./src/NodeFetcher.o -lMapi -ltararixpath -lxerces-c * What can be the reason for the error during execution? Thank you for any help, Alexander

3 2

[MonetDB-users] Cursor Alternative?
by Nafees Ur Rehman 03 Mar '10

03 Mar '10

Hi, Is there any alternative available in MonetDB to operate on records from within the function code? As cursors are not supported. Regards Nafees Ur Rehman Database & Information System Deptt of Computer Science University of Konstanz, Germany.

2 2

[MonetDB-users] Name of fields in query result?
by Guillaume Yziquel 02 Mar '10

02 Mar '10

Hello. Here are a few functions/methods from the postgresql-ocaml binding. > class type result = object > (** Main routines *) > > method nfields : int > (** [#nfields] @return the number of fields in a query result. *) > > method fname : int -> string > (** [#fname n] @return the name of the [n]th field. > > @raise Error if field out of range. > *) > > method fnumber : string -> int > (** [#fnumber field] @return the index of the field named [field]. > > @raise Not_found if no such named field. > *) I'm looking for equivalents of fname and fnumber methods in MAPI. Couldn't find an equivalent. So, is there a way, when you get the result of a query, to get the names of the fields in the result? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

4 8

[MonetDB-users] Announcement: New Feb2010 feature release of MonetDB suite
by Sjoerd Mullender 02 Mar '10

02 Mar '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Feb2010/>. -- Sjoerd Mullender

2 1

[MonetDB-users] Slowdowns after Delete
by Reeves, Matthew J. 01 Mar '10

01 Mar '10

Hello, I have a MonetDB database that contains a table of rolling events. There are 200 million records with 500,000 bulk loaded each day with the oldest 500,000 deleted at midnight. I'm finding the database gets slower as time goes on. It gets to the point of being almost unusable after a week. Is there something I should be doing different? Thanks, Matt

4 4

[MonetDB-users] Xquery Standoff Annotation Success Stories? GATE too?
by David Epstein 25 Feb '10

25 Feb '10

Hello, I am looking for information and experiences with monetdb's standoff annotation extension for Xquery. In particular, I'm looking for a way to more flexibly and conveniently search through documents I've manually annotated for semantic content using GATE (General Architecture for Text Engineering), which has its own standoff format. Has anyone had success in querying GATE documents using monetdb xquery with the standoff extension? If not GATE, what are people using to produce the standoff notation? thank you, -david David Epstein PhD Candidate Urban & Regional Planning University of Michigan USA

2 5

[MonetDB-users] path to XML collection using doc() ?
by David Epstein 25 Feb '10

25 Feb '10

Hi everyone, Using the GUI (on localhost) I loaded: Migdal_2001_State_in_Society_BOOK.xml into test_collection_01. Using the query below (from http://monetdb.cwi.nl/XQuery/QuickTour/Books/index.html) as a template but using my own setup and schema, I keep getting: ERROR = !ERROR: err:FODC0002, Error retrieving resource (no such document... I've tried just about every permutation in the doc() statement I can think of. What should the path be? <bib> { for $b in doc("http://monetdb.cwi.nl/XQuery/files/bib.xml")/bib/book where $b/publisher = "Addison-Wesley" and $b/@year > 1991 return <book year="{ $b/@year }"> { $b/title } </book> } </bib> thank you, -david

2 1

[MonetDB-users] Error on Queries 4.x of Star Schema Benchmark
by Rui Anastácio 23 Feb '10

23 Feb '10

Hello, I'm testing MonetDB with Star Schema Benchmark on a Linux Ubuntu box. I'm able to run all queries except 4.1, 4.2 and 4.3 I've tested with Scale Factor of 1, 5 and 10, and with 1, 3 and 7Gb RAM, and in all situations I gets this error: java.sql.SQLException: MALException:algebra.join:GDK reported error. ERROR: HEAPextend: failed to extend to 954941288 for 15/1560tail at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2208) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:1967) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:434) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:347) at nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:451) at test.Main.test(Main.java:188) at test.Main.main(Main.java:158) With 1 Gb of RAM it could be lack of memory but not with 7Gb and a SF of 1. Any idea why this occours? Thanks. Rui Anastácio

3 7

[MonetDB-users] Different query results after each execution
by Alex Bo. 20 Feb '10

20 Feb '10

Hi, I am getting weird behavior with a query that involves a LIKE comparison: This is the query: SELECT ("test2"."a1") AS "a1", (Count(DISTINCT CASE WHEN (Lower("test2"."a2") LIKE '%aaa%') THEN "test2"."a3" ELSE NULL END)) AS "m1" FROM "test"."test2" GROUP BY "test2"."a1" Whenever I run this query I get different results in my m1 field. If I remove the LIKE then it behaves fine. I can't seem to reproduce it on new tables with little data, but with the data I have (a table with 109402 rows) it does reproduce all the time. Can I upload the dbfarm and file this as a bug? -- View this message in context: http://old.nabble.com/Different-query-results-after-each-execution-tp276669… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Weird behaviour with namespaces in XQuery
by Ignacio Martín Oya 19 Feb '10

19 Feb '10

Hello: I'm using MonetDB for a project. I've observed some weird results when doing a XQuery on an inserted document. Steps to reproduce the bug are: xquery>pf:add-doc ('file://var/tmp/pres-rules.xml', 'pres-rules.xml', 'pres-rules', 1) more>xquery>declare namespace xs="http://www.w3.org/2001/XMLSchema"; more>declare namespace cr="urn:ietf:params:xml:ns:common-policy"; more>declare namespace pr="urn:ietf:params:xml:ns:pres-rules"; more>count (doc('pres-rules.xml')/cr:ruleset/cr:rule[@id="pres_whitelist"]/cr:conditions/cr:identity) more><?xml version="1.0" encoding="utf-8"?> <XQueryResult>0</XQueryResult> However, sometimes, if I restart the Mserver, the same query returns: more>xquery>declare namespace xs="http://www.w3.org/2001/XMLSchema"; more>declare namespace cr="urn:ietf:params:xml:ns:common-policy"; more>declare namespace pr="urn:ietf:params:xml:ns:pres-rules"; more>count (doc('pres-rules.xml')/cr:ruleset/cr:rule[@id="pres_whitelist"]/cr:conditions/cr:identity) more><?xml version="1.0" encoding="utf-8"?> <XQueryResult>1</XQueryResult> What is the expected result. I'm sure it's related with namespaces after some tests. Are you aware of this behaviour? There exists some workaround? Document to test is attached. I'm using latest version of MonetDB with XQuery support: # MonetDB Server v4.34.4 # based on GDK v1.34.4 # Copyright (c) 1993-July 2008, CWI. All rights reserved. # Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved. # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically linked. # Detected 0.979 GiB (1051709440 bytes) main-memory. # Visit http://monetdb.cwi.nl/ for further information. # PF/Tijah module v0.13.0 loaded. http://dbappl.cs.utwente.nl/pftijah # MonetDB/XQuery module v0.34.4 loaded (default back-end is 'algebra') # XRPC administrative console at http://127.0.0.1:50001/admin Cheers, Nacho. -- -- Ignacio Martín Oya Project Engineer http://www.azetti.com Important: This e-mail may contain information confidential to Azetti Networks. If you are not the intended recipient it may be unlawful for you to read, copy, distribute, disclose or otherwise use information contained in it. If this is the case, please contact us immediately and we will assist you. Tel: +34 91 658 69 53 Fax: +34 91 651 40 42 E-mail: Reply to sender

1 0

[MonetDB-users] Monetdb + unixODBC
by Alexander Barkov 18 Feb '10

18 Feb '10

Hello, I am a new in MonetDB. I'm testing MonetDB + unixODBC with an application which inserts long BLOB values, and I'm having some problems. Can you please help me? 1. When I use rc= SQLBindParameter(hstmt, position, SQL_PARAM_INPUT, SQL_C_DEFAULT, SQL_LONGVARBINARY, 0, 0, (char*) data, data_size, &BindBuf[position].size); I get error "Optional feature is not implemented". If I change SQL_LONGVARBINARY to SQL_VARCHAR, it works, however it wants the data in HEX notation, instead of raw binary buffer. That means I need 2*data_size extra memory to encode my binary data in hex, and then send the result send to SQLBindParameter. Is there a way to bind raw binary data, to avoid hex encoding? 2. When I insert blob data using rc= SQLExecDirect(hstms, "INSERT INTO t1 (blob_column)" "VALUES " "('<long hex-string>')", SQL_NTS); It works fine for short data, but it returns this error when hex-string is long enough: [SQLSTATE:S1000][unixODBC][MonetDB][ODBC Driver 1.0]current transaction is aborted (please ROLLBACK) I guess I hit some maximum possible query length limit, or maybe maximum possible transaction size. Is there a way to configure MonetDB to allow longer values? 3. Does the native MonetDB API have the same limitations with the bind types and maximum query(or transaction) size? Thank you very much!

2 9

[MonetDB-users] MAL BAT Types
by Gerrard_56 14 Feb '10

14 Feb '10

hi, I’m writing a theoretical paper about MonetDB and I have a question about the MAL. Does the MAL also use inverted lists and are the updated in the same way described for the MIL in the thesis of P. A. Boncz “Monet: A Next-Generation DBMS Kernel For Query-Intensive Applications” on page 137f. Thanks for any information gerrard -- View this message in context: http://old.nabble.com/MAL-BAT-Types-tp27583769p27583769.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Admin GUI is blank page
by jasonpriem 12 Feb '10

12 Feb '10

Hi, I've just installed MonetDB on my 32-bit Vista system. The server and client both start up fine; however, http://127.0.0.1:50001/admin gives me a blank page. I can't really think of much to try, save deactivating the firewall (which made no difference). Any suggestions? Thanks. -- View this message in context: http://old.nabble.com/Admin-GUI-is-blank-page-tp27559054p27559054.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] Missing element names when bulkloading XML files into MonetDB
by Alexander Koglin 12 Feb '10

12 Feb '10

I have buldloaded a collection of 10.000 XML files into MonetDB like this: for $d in doc("/monetdbfiles.xml")//doc return pf:add-doc(fn:concat($d/@path,$d/@name), fn:string($d/@name), "abap10k", 0) <> Now the XML files have missing element names and look like this in MonetDB: <AST> < type="compound" line="1" id="1"> < type="stmt" line="1" id="2"/> < type="stmt" line="5" id="3"> < type="clause" line="5" id="4"> < type="idf" line="5" id="5"/> </> What could be wrong? Thanks for your help.

2 1

[MonetDB-users] Pager in mclient?
by Mark Bucciarelli 11 Feb '10

11 Feb '10

>From man page, it looks like I should be able to pipe the output of a command through less. But I can't figure out the syntax. For example: sql>\|less \d mytable sql> Can I do this? If so, is there a way to set it up so this always happens? This is candidate release from November 2009. Thanks, Mark

2 1

[MonetDB-users] where is the data?
by Zoltan Tomory 09 Feb '10

09 Feb '10

Curiosity has gotten the best of me. Where is the datafile/s kept and what is it called? Thanks in advance, Zoltan Tomory

2 1

[MonetDB-users] Monetdb and Hardy Heron
by Radek Maciaszek 09 Feb '10

09 Feb '10

Hi, I noticed the release for Karmic Koala but none for Hardy Heron. Is it possible to install monetdb with hardy? Is there perhaps any wiki page in monetdb documentation which expplains steps? Looking forward to trying monetdb! Radek

3 3

[MonetDB-users] dump and restore---use merovingian database files?
by Mark Bucciarelli 09 Feb '10

09 Feb '10

I just had to restore a database dump, and I got nervous watching top. The box, which doesn't have much RAM, started to swap on the restore. At times, the monetdb process was using over 90% of available RAM. While I only have 500M RAM, the database only has 10m records. The gzipped dump is 142MB. Other than the backup/restore issue, Monet seems quite happy running with this much data and so little RAM. It looks like it could support much more data---but I'm not so sure the restore will. If I run $ monetdb stop -a can I safely tar the database files and use that as a backup? Or do I need to stop merovingian too? Or is this approach not safe? Thanks, Mark

3 3

[MonetDB-users] Ubuntu distribution is broken with latest release.
by Mike Mansell 08 Feb '10

08 Feb '10

Attempting to install latest release using Ubuntu packages but the package definition appears to still be referencing the older 1.34.2-20091210 versions while the actual libraries have been updated to the newer 1.34.4-20100112 version. Installation gets errors: W: Failed to fetch http://monetdb.cwi.nl/downloads/Ubuntu/pool/monetdb/m/monetdb/libmonetdb1_1… 404 Not Found W: Failed to fetch http://monetdb.cwi.nl/downloads/Ubuntu/pool/monetdb/m/monetdb-client/libmon… 404 Not Found W: Failed to fetch http://monetdb.cwi.nl/downloads/Ubuntu/pool/monetdb/m/monetdb5-server/libmo… 404 Not Found W: Failed to fetch http://monetdb.cwi.nl/downloads/Ubuntu/pool/monetdb/m/monetdb-sql/libmonetd… 404 Not Found W: Failed to fetch http://monetdb.cwi.nl/downloads/Ubuntu/pool/monetdb/m/monetdb5-server/monet… 404 Not Found W: Failed to fetch http://monetdb.cwi.nl/downloads/Ubuntu/pool/monetdb/m/monetdb-sql/monetdb5-… 404 Not Found

7 14

[MonetDB-users] MonetDB XQuery Bulk Load / Bulkload
by Alexander Koglin 07 Feb '10

07 Feb '10

I have currently installed a version of MonetDB XQuery for Linux and want to bulkload data like this: http://monetdb.cwi.nl/XQuery/Documentation/Bulk-Loading-a-Collection.html#B… For that purpose, I used the following command inside the Client: for $d in doc("/monetdbfiles.xml")//doc return pf:add-doc(fn:concat($d/@path,$d/@name), fn:string($d/@name), "abap10k", 0) <> It references the following, definitely existing XML file: <dir> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCDEF"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCIMP"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCMAC"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CI"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM001"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM002"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM003"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM004"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM005"> </dir> But unfortunately an error is thrown (I also tried it unter Windows and it was the same): MAPI = root@localhost:50000 QUERY = for $d in doc("/monetdbfiles.xml")//doc ERROR = !ERROR: err:FODC0002, Error retrieving resource (no such document "/monetdbfiles.xml"). What could be the reason for that?

1 1

[MonetDB-users] authorize! what's wrong?
by korisk 07 Feb '10

07 Feb '10

Hi! I've installed monetdb from sources. ./MonetDB5-server-5.16.4.tar.bz2 ./MonetDB-client-1.34.4.tar.bz2 ./MonetDB-SQL-2.34.4.tar.bz2 merovingan started database test created but mclient asks for password! and i don't know which one. Manual says that monetdb uses system authorization, but mclient answers: korisk@pс64:~$ mclient -l sql -d test user(korisk): password: InvalidCredentialsException:checkCredentials:invalid credentials for user 'korisk' i use Ubuntu 9.10 x86_64 What's wrong? Thank you.

2 1

[MonetDB-users] authorization! what's wrong?
by korisk 07 Feb '10

07 Feb '10

Hi! I've installed monetdb from sources. ./MonetDB5-server-5.16.4.tar.bz2 ./MonetDB-client-1.34.4.tar.bz2 ./MonetDB-SQL-2.34.4.tar.bz2 merovingan started database test created but mclient asks for password! and i don't know which one. Manual says that monetdb uses system authorization, but mclient answers: korisk@pс64:~$ mclient -l sql -d test user(korisk): password: InvalidCredentialsException:checkCredentials:invalid credentials for user 'korisk' i use Ubuntu 9.10 x86_64 What's wrong? Thank you.

2 2

[MonetDB-users] connecting streams to Mapi.
by Guillaume Yziquel 05 Feb '10

05 Feb '10

Hi. I'm investigating how to connect custom streams to a Mapi structure. I've been reading, for instance the start_mal_server() function of monetdb5-server-5.16.2/src/tools/embeddedclient.c.in. I now sort of understand how the stream library works (though not fully), and I'm trying to understand how the streams are effectively used by the client of the Mapi struct. I've therefore looking at the start_mal_server() function, and here's a piece of code whose semantics I do not understand: > in = ((stream **) arg)[0]; > out = ((stream **) arg)[1]; > free(arg); > snprintf(buf, sizeof(buf), > "in := \"" PTRFMT "\":streams;\n" > "out := \"" PTRFMT "\":streams;\n" > "mapi.malclient(in, out);\n", PTRFMTCAST in, PTRFMTCAST out); > callString(mal_clients, buf, 0); I'm specifically interested in the 'in' stream since it is the one I would like to write to. Could someone enlighten me here? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 4

[MonetDB-users] Feb2010 64 bit starts and immediately stops on Windows
by Alex Bo. 05 Feb '10

05 Feb '10

Hi dear MonetDB people, I built MonetDB 64 bit on Windows from the Feb2010 branch today. Everything ran fine, but when I start the server, after seeing the normal output, it immediately dissapears. Here's the output: # MonetDB server v5.18.0, based on kernel v1.36.0 # Serving database 'demo', using 1 thread # Compiled for i686-pc-win32/64bit with 32bit OIDs dynamically linked # Found 1.999 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\MonetCVS_base_Feb10\MonetCVS64_cleanBuild\Install\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.36.0 loaded Building MonetDB 32 bit works just fine. Thanks. -- View this message in context: http://old.nabble.com/Feb2010-64-bit-starts-and-immediately-stops-on-Window… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] MCinitClient() and user oid.
by Guillaume Yziquel 05 Feb '10

05 Feb '10

Hi. I've been trying to get access to the Client structures on the server side. I've therefore been looking at MCinitClient(). What I do not understand is how do you know which oid for the user you should pick, in order to pass this oid as an argument to MCinitClient. > http://yziquel.homelinux.org/topos/api/ocaml-monetdb5/MonetDB5.MAL.Client.h… > > Client MCinitClient(oid user, bstream *fin, stream *fout) Moreover, it seems to me that in an embedded setting, it doesn't seem to be a smart idea to use MCinitClient. It would perhaps be better to grab the already existing Client structure for the admin. Probably with MCfindClient. > http://yziquel.homelinux.org/topos/api/ocaml-monetdb5/MonetDB5.MAL.Client.h… > > Client MCfindClient(oid user, str scen, bstream *fin, stream *fout) fin and fout seem optional arguments (meaning you can pass NULL for sensible defaults). But as before, the oid of the user, and the sceanrio string are arguments whose proposed values are unclear to me. How do you get the admin's Client struct? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

3 7

[MonetDB-users] copy into with 'float' types crashes
by Alex Bo. 04 Feb '10

04 Feb '10

Hi, I am encountering something very strange. (Windows - NovSP1 from CVS) On some machines I encountered crashes in Monet when doing a copy into on tables with a float column. After trying to investigate, I narrowed it down to this: Run: 1. CREATE TABLE test (f1 int, f2 float); 2. COPY INTO test from 'filename.txt' using delimiters ',','\n','"' null as ''; the text file contains: 1,1 2,2 This sequence exactly caused a server crash (without any message) on some machines, while on others it didn't. The machines it happened on were: 1. 32 bit XP with 3 GB RAM. 2. 32 bit Vista with 4 GB RAM. In both cases, changing the column type from float to real worked. Could it be that float, being a 64 bit floating point, might cause crashes on 32 bit machines? Is there anything I can to help you get a clearer picture of this? Thanks. -- View this message in context: http://old.nabble.com/copy-into-with-%27float%27-types-crashes-tp27438598p2… Sent from the monetdb-users mailing list archive at Nabble.com.

3 4

[MonetDB-users] The maximum used memory option
by Stefan de Konink 04 Feb '10

04 Feb '10

It was discussed before that there should exist an option to limit the amount of 'detected' memory to a certain value. Is this option available, if so, what is its syntax? The other way around was to use ulimit on the process, does this mean MonetDB will get a sigkill if it grows out of boundaries or will it just use that value as upper memory limit? Stefan

2 1

[MonetDB-users] Problem using win 32 odbc driver
by klaas＠maxml.nl 04 Feb '10

04 Feb '10

Good morning, I'm using MonetDB v5.16.2 on windows xp 32. Connecting to the database with Java clients works fine, odbc (version 10.1.12) is not. Table DDL: CREATE TABLE _r_language( id smallint NOT NULL AUTO_INCREMENT, abbrev varchar(3) NOT NULL, name varchar(50) NOT NULL, primary key (id) ); For example in php: <?php $conn = odbc_connect('DSN=MonetDB;HOST=localhost;PORT=50000', 'monetdb', 'monetdb') or die('unable to connect'); $result = odbc_exec($conn, 'select * from _r_language;'); odbc_result_all($result); ?> results in: ? ?? ??? 1 English EN 2 German DE 3 French FR 4 Dutch NL 5 Polish PL The fieldnames are not returned. Using the same ODBC driver in MS Access, the table can be linked, column_names are shown, but retrieving data generates error message: ODBC - call failed [MonetDb][ODBC Driver 1.0]TODO: column names of level >= 3 (#0) What can you recommend? Thanks for your help, regards, Klaas

1 0

[MonetDB-users] Types oid, size_t.
by Guillaume Yziquel 03 Feb '10

03 Feb '10

Hi. I've been wanting to wrap up oids into OCaml, and I'm wondering a few things about the following statement in gdk.mx: > @item oid: > Unique @strong{long int} values uses as object identifier. Highest bit cleared always. > Thus, oids-s are 31-bit numbers on 32-bit systems, and 63-bit numbers on 64-bit systems. Moreover, the oid type is typedefed in the following way: > #ifdef MONET_OID32 > #define SIZEOF_OID SIZEOF_INT > typedef unsigned int oid; > #else > #define SIZEOF_OID SIZEOF_SIZE_T > typedef size_t oid; > #endif I understood that MONET_OID32 was the exception rather than the rule, so I'm assuming, for now, that oid is size_t. But then, I'm wondering why the highest bit is always cleared. Moreover, OCaml has a way to deal with their integers which is rather peculiar. They are long ints, but the lowest bit has a peculiar meaning. If the lowest bit is 0, then this long int is in fact a pointer to an OCaml value in the OCaml heap. If the lowest bit is 1, then shift the value one bit, and you have the Ocaml integer. This scheme takes advantage of the alignement of OCaml values in the OCaml heap. So ocaml integers are 31-bit on 32-bit systems, and 63-bit on 64 bit systems. The difference is that the lowest bit is set, whereas it's the highest bit that is cleared for oids. Therefore, to convert oids to ocaml ints and conversely, I wrote: > #define ocamlmonetdb5_Val_oid(x) ((value) ((((oid) x) << 1) + 1)) > #define ocamlmonetdb5_Oid_val(x) (((oid) x) >> 1) Two questions: -1- This seems to work, but I'd appreciate if you could double-check this. Just to check if I understood correctly the bit layout of oids. -2- I'm wondering why the highest bit of oids is always cleared. This doesn't seem to be implied by the fact that oids are typedefed size_t. So what's the rationale? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

1 0

[MonetDB-users] sql date
by dariuszs 02 Feb '10

02 Feb '10

Hi, Is there any documentation on how to manipulate dates (add/subtract months, days, years)? Running query: sql>select date '1900-01-01' + interval '100' year; Produces error: Incorrect interval (3>1)..... Thanks. Dariusz.

1 0

[MonetDB-users] size_t vs. var_t on Windows
by Alex Bo. 31 Jan '10

31 Jan '10

Hi, I've been trying to build the MonetDB5 module from the Feb2010 branch from CVS on Windows. However, I am getting this: .\..\..\..\..\src\modules\kernel\group.mx(1330) : warning C4267: '=' : conversion from 'size_t' to 'var_t', possible loss of data .\..\..\..\..\src\modules\kernel\group.mx(1330) : warning C4267: '=' : conversion from 'size_t' to 'var_t', possible loss of data This did not happen in the November releases on Windows. Any ideas on how to approach this? Do I need to use some special flags? Thanks. -- View this message in context: http://old.nabble.com/size_t-vs.-var_t-on-Windows-tp27396298p27396298.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] MonetDB/XQuery stability problems w/ shredding
by Xander Schrijen 29 Jan '10

29 Jan '10

Hi list, I'm using MonetDB/XQuery to run queries on sets of XML documents of all kinds (build files, process descriptions). These sets can range from a few dozen to a few thousand documents, so the speed of MonetDB can be a real boon. However, when shredding these documents there may be a few errors. Sometimes a namespace URI is reported as not well formed, sometimes shredding 'hangs' (esp. when using the batch-import method from the manual), sometimes I can't find a reason. After such an error, the database seems to be corrupted. Sometimes queries for which collections are in the database give errors, sometimes an XPath query doesn't finish. In most cases, the ERROR=.... from mclient doesn't give a proper error (ascii garbage). I reallize these reports are vague. However, even though this happens a lot, reproducing the exact sequence can be hard. (Importing a few 100 documents, waiting for the import error to occur, using the right query to trigger the error...) But as it is now, I don't even know if these things are known, have work-arounds, or require fixing the xml documents upfront. Are there people on the list with experience with this that are willing to help me out? Regards, Xander.

2 4

[MonetDB-users] Announcement: New Nov2009-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 28 Jan '10

28 Jan '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2009-SP2 bug fix release of the MonetDB suite of programs. More information on this release will be available at <http://monetdb.cwi.nl/Development/Releases/Nov2009/>. Fixes include: - No need to require bash for running the *-config scripts. (SF bug #2914563) - MonetDB5 development package now contains all development include files. (SF RFE #2929558) - Added support for compiling on OpenBSD (contributed, i.e. not tested by us). - General code improvements, partly inspired by running Coverity Scan. - mclient -D will exit with a non-zero exit code if dumping failed. (SF bug #2925674) - The Python interface now uses the same socket options as the C version, giving a boost in performance. (SF bug #2925750) - Fixed a bug in the Python interface to deal with fields with embedded newline characters. (SF bug #2917219) - Added initial support to the Python client interface for specifying the encoding. - Fixed a bug in execute argument parsing in Python interface. - Added missing .sql files to the Windows installer. - In MonetDB/SQL, fixed bug in not(invalidtable). (SF bug #2927174) -- Sjoerd Mullender

1 0

[MonetDB-users] OCaml-MonetDB is working.
by Guillaume Yziquel 22 Jan '10

22 Jan '10

Hello. I've managed to pull together, in my scarce free time, an OCaml binding for MonetDB. It is now working and I have written a very simple MAL client for an embedded database: > yziquel@seldon:~/git/ocaml-monetdb5$ utils/malclient.native > io.print(1); > 1 > io.print(2); > 2 I've got a small question concerning MAL. As I see it, MAL is currently a human-readable language. Is there somewhere in the MonetDB system a place where it is compiled to a more space-efficient representation? Or is interpreted as such? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] MonetDB XQuery Bulk Load / Bulkload
by Alexander Koglin 21 Jan '10

21 Jan '10

I have currently installed a version of MonetDB XQuery for Linux and want to bulkload data like this: http://monetdb.cwi.nl/XQuery/Documentation/Bulk-Loading-a-Collection.html#B… For that purpose, I used the following command inside the Client: for $d in doc("/monetdbfiles.xml")//doc return pf:add-doc(fn:concat($d/@path,$d/@name), fn:string($d/@name), "abap10k", 0) <> It references the following, definitely existing XML file: <dir> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCDEF"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCIMP"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCMAC"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CI"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM001"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM002"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM003"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM004"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM005"> </dir> But unfortunately an error is thrown (I also tried it unter Windows and it was the same): MAPI = root@localhost:50000 QUERY = for $d in doc("/monetdbfiles.xml")//doc ERROR = !ERROR: err:FODC0002, Error retrieving resource (no such document "/monetdbfiles.xml"). What could be the reason for that?

3 2

[MonetDB-users] compression
by Jason Kinzer 20 Jan '10

20 Jan '10

Hi, Are there any plans on the horizon to roll compression into Monet? The X-100 project looked really interesting in this regard, but as I understand it, that work has been transferred into VectorWise. If there are no plans, is this because it's completely antithetical to the monet architecture (from the papers it seems like X-100 was, to some degree at least, 'integrated' in), or more due to lack of resources? My motivating example here is OLAP: I frequently have 1 relatively large fact table and then many much smaller dimensional tables. If optional compression were available, it we be nice to compress all or some of the BATs for the fact table columns and then have the others work as usual. (Well, at least this sounds good, maybe it makes no sense). Another motivation is there seems to be a lot of anecdotal evidence for companies moving from larger big iron servers to more numerous, smaller machines - so it would be really nice to have this capability for more memory constrained settings. I understand on a basic level how compression conflicts with the relatively simple approach monet uses to load BATs (e.g. memory map), but, dwelling in ignorance, I blithely assume there could be some solution not as complex as X-100 if one were to accept a significant performance cost. For example: decompressing BAT data on the fly as part of a BATiterator. I probably don't have the skills to implement even a basic on-the-fly decompression approach like this, but just wondering aloud: how hard a problem is this? Thanks, Jason

2 3

[MonetDB-users] manually setting gdk_nr_threads to N?
by Nafees Ur Rehman 19 Jan '10

19 Jan '10

Hi, I am using Windows 2003 with two core processors configures, and the gdk_nr_threads does not show the 2nd processor. Can you tell me how can I set that manually? Regards Nafees Ur Rehman

2 1

[MonetDB-users] Standard Deviation tips and hints request
by Dennis Forbes 17 Jan '10

17 Jan '10

Hi all. While it's possible to calculate the stddev with an aggregated cross join, is there perhaps a better way? Any tips or hints on where I should look? Can this be done in MIL or some other tactic to provide a more straightforward, better performing experience? I searched the web before asking and all I've found are unanswered questions, however I would really like to solve this even if it includes extending the system. Dennis W. Forbes Vastardis Capital Services 41 Madison Avenue, 30th Floor New York, NY 10010 P: 646-230-0933 F: 212-243-5251

2 2

[MonetDB-users] mapi_query with MAL code?
by Guillaume Yziquel 17 Jan '10

17 Jan '10

Hello. I've been using an embedded SQL with moderate success with my OCaml binding. Essentially passing the SQL statement through the mapi_query() function call. I would now like to try out some MAL code in a similar setting: embedded MonetDB, code passed via mapi_query(). Could someone provide me with a simple "Hello world" MAL code to try? For now, all I've been able to achieve with MAL are segfaults... All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] native socket protocol vs mapi
by Andy Chambers 16 Jan '10

16 Jan '10

Hi, I want to be able to use monetdb from common lisp and was looking at the other language bindings for clues on how to write one for cl. Python and Ruby seem to communicate using a socket rather than use the C api. However, in common lisp it's relatively easy to use C bindings (though CFFI) so I'd rather use them if I thought they were going to continue to be supported. Are there any other factors I should consider in deciding which API to use? Many Thanks, Andy

4 9

[MonetDB-users] Star Schema Benchmark: Can't load SF 5 ou SELECT, LINE_READ Crash
by Rui Anastácio 12 Jan '10

12 Jan '10

Hello, I Have Ubunto 9.10 Karmik Koala installed. On top of this I have installed MonetDB latest version (same problem with previous one). I'm working with Star Schema Benchmark (SSBM). For Scale Factor 1 and 3 I can load the data. For SF 5 our above I get READ_LINE error when loading LINEORDER table. Also when running select Q4.1 I also get the same error and the server crashes. Anyone as a clue? Thanks Rui A.

2 3

Re: [MonetDB-users] monetdb_sql/mal() ?
by Guillaume Yziquel 12 Jan '10

12 Jan '10

Jason Kinzer a écrit : >> >> Besides, It would seem to overlap with Mauricio's work which already >> provides some parsing from his OCaml algebra to SQL. But the thing is, >> you've got the SQL string created at OCaml compile time. I may be wrong about this compile time assertion. It now seems to me that this is runtime. In which case, it would make sense to map Mauricio's relational algebra to MAL's relational algebra. >> What would be nice >> is to have the SQL -> MAL compilation also happen at the same time you >> compile the OCaml program. >> >> (Moreover, it seems to me that MAL comes in two flavour: a human-readable >> one and a machine readable one. Is that correct? If yes, I'm aiming at the >> machine readable one.) >> >> How easy would that be? Do you need a running embedded server for that, or >> simply a subset of the functionalities of the MonetDB shared libraries? > Ah, I see now, so you're talking about actually writing a SQL->MAL compiler > as part of that existing Ocaml SQL frontend (sort of cut out the > "middle-man"). Yes. > That sounds neat, but bear in mind that could be tantamount > to re-writing the existing Monet SQL frontend, depending on your goals In fact, I would like to do an OCaml/C binding to the SQL -> MAL compilation that is already done in MonetDB. This seems to me the least effort path. But I'd like to know where in the code is this compilation / conversion from SQL to MAL done. > - so I let you draw your own conclusions. Try doing an "explain" on various SQL > statements to see the generated MAL. Sorry, I'm quite new to MonetDB and still having a hard time with the documentation. What is this "explain" thing I should be doing? > I would say writing a very naive, > minimal SQL translation is not easy but certainly within raeach. Here > someone other than I should comment, but I think most of the complexity of > the existing frontend lies in the optimization passes, tx handling, > approaches to handle datasets that won't fit in memory, etc. Rewriting an SQL -> MAL translation is not my goal. Reusing an existing one would be one my goals. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] monetdb_sql/mal() ?
by Guillaume Yziquel 11 Jan '10

11 Jan '10

Hello. I've been looking at Debian packages, and the followig things struck me: -1- You have two shared libraries: libembeddedsql.so and libembeddedmal.so which are distributed, but there's only headers for the *sql.so one. You have to write your own for the *mal.so one. -2- In the source, there are two libembeddedclient.c.in files: one for sql, one for mal. The one for sql contains a monetdb_sql() function call. The one for mal does not. Is there a good reason for this? Or is it simply the way it currently is? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 4

[MonetDB-users] problems with the interpretation of the query history
by Gerrard_56 10 Jan '10

10 Jan '10

Hi. I’m a new MonetDB user and I use the MonetDB server v5.14.2 with the MonetDB/SQL module v2.32.2 on a windows xp platform. I have some problems with the interpretation of the queryhistory and the callhistory. My first question is: What is the meaning of the parse and the optimize attributes in the queryhistory. Is the optimize attribute used to store the time which the front-end needs to parse the sql statement into a mal plan respectively is the optimize attribute used to store the time used for the optimization done for the mal plan. I’m asking because couldn’t find an explanation in the sql manual. My next questions concern the xtime and the rtime attributes of the callhistory. Sometimes for some queries the xtime is 0 usec. Does this indicate that the query was already executed once before and therefore there isn’t a preparation time needed for the result set? Sometimes for some queries only the rtime is 0 usec but the the xtime is > 0 usec. What does this means? There is a preparation time for a result set but there is no time needed to construct and return the result set to the user. For me this makes no sense. Sometimes there is the xtime and the rtime 0 usec. Does this mean that the query never runs in the database engine? Does somebody know the reason for this behavior? Lust but not least in the sql manual you can read that all storage is in K/M/G bytes. But where can I find out if K bytes M bytes or G bytes are used? Thank you for any help Gerrard (I apologize for any grammar and spelling mistakes in this posting but my English is not the best.) -- View this message in context: http://old.nabble.com/problems-with-the-interpretation-of-the-query-history… Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] sql execution time
by Gerrard_56 09 Jan '10

09 Jan '10

Hi. I would like to compare the performance of MonetDB with the performance of an Oracle DB by using the TPC-H benchmark. I use the MonetDB server v5.14.2 with the MonetDB/SQL module v2.32.2 on a windows xp platform. On the MonetDB website (http://monetdb.cwi.nl/SQL/Benchmark/TPCH/) I read that they used the mclient with the –t argument in order to obtain the timings for the queries. I tried the same but the displayed time for queries which return several 100 rows is very high. For example the TPC-H query nr. 3 with returns 11620 row the time was 13796.174 msec. If I execute the same query in the db visualize it just needs 0.864 sec for the result. I think the problem with the mclient is that it needs a lot of time to display the result of the queries. Is there a better alternative to the –t argument of the mclient to get a useful time result? Thank you for any help Gerrard -- View this message in context: http://old.nabble.com/sql-execution-time-tp27088775p27088775.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Ebuilds for MonetDB
by W-Mark Kubacki 07 Jan '10

07 Jan '10

Hello. I'd like to thank you for MonetDB by providing fresh ebuilds for the suite. And yes, I did find Fabian Groffen's, but they were outdated and removed from main tree. So, find new ones in my Gentoo overlay [1] along with binaries for amd64 and arm on my binhosts [2,3]. Unfortunately Monet features a lot unused configure flags and deviates from the Filesystem Hierarchy Standard (FHS) 2.3 [4] (*), therefore I had to put an extra efford to sieve the first out and have files installed corretly. If you happen to find something I missed, don't hesitate to drop me a line or two. -- Beste Grüße W-Mark Kubacki http://mark.ossdl.de/ [1] http://git.ossdl.de/?p=ossdl-overlay.git;a=summary [2] http://binhost.ossdl.de/x86_64-pc-linux-gnu/ , http://binhost.ossdl.de/x86_64-pc-linux-gnu/Packages [3] http://binhost.ossdl.de/armv5tel-softfloat-linux-gnueabi/ , http://binhost.ossdl.de/armv5tel-softfloat-linux-gnueabi/Packages [4] http://www.pathname.com/fhs/

3 4

[MonetDB-users] vaultkey, embedded, and merovingian.
by Guillaume Yziquel 05 Jan '10

05 Jan '10

Hi. I've been creating a database from a call to monetdb_sql() from libembeddedsql5.so, from my OCaml binding. After that, I start up a merovingian process and I do the following: > yziquel@seldon:~/git/ocaml-monetdb5$ monetdb status > name state uptime health last crash > testdb stopped 100%, 49s - > yziquel@seldon:~/git/ocaml-monetdb5$ monetdb start testdb > starting database 'testdb'... FAILED > start: starting 'testdb' failed: cannot start database 'testdb': no .vaultkey found (did you create the database with `monetdb create testdb`?) I'm quite baffled by the error I just got. What is this .vaultkey problem? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

3 5

[MonetDB-users] Inserting '\' into varchar(1)
by Alex Bo. 04 Jan '10

04 Jan '10

Hi, I think I found a bug: I can't seem to insert the character '\' into Monet with COPY INTO when I use a single character column: 1. I have a text file that only contains this: '1,"\\"' 2. create table test1 (f1 int, f2 varchar(1)) 3. copy into test1 from 'C:\test1.txt' using delimiters ',','\r\n','"' null as ''; I get: SQLException:sql:value '\\' from line 1 field 2 not inserted, expecting type str If I change the field size from varchar(1) to varchar(2) it works. (Aug09 & Nov09 on Windows) Is this a bug? Thanks & happy new year. -- View this message in context: http://old.nabble.com/Inserting-%27%5C%27-into-varchar%281%29-tp27010393p27… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Nightly builds are still OK?
by Guillaume Yziquel 04 Jan '10

04 Jan '10

Hi. I've been trying to fetch nightly builds, but on the webpage http://monetdb.cwi.nl/testing/projects/monetdb/Current/.DailyBuilds./ it seems that the last nightly build was one week or so ago. Are nightly builds still working fine? Just to know... All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] Error on "rank() over (order by sum(.."
by Makoto YUI 04 Jan '10

04 Jan '10

Hi, The following query yields "SELECT: subquery result missing" error on Nov2009-SP1. select s_name, sum(score), rank() over (order by sum(score) desc) from score_table group by s_name -- preparation create table score_table ( s_name varchar(20), subject varchar(10), score int ); insert into score_table values('foo','english',70); insert into score_table values('foo','history',50); insert into score_table values('foo','math',60); insert into score_table values('bar','english',45); insert into score_table values('bar','history',75); It seems allowed to use aggregation functions in "over(..)" in other databases. http://www.google.com/search?q="rank()+over+(order+by+sum(" Thanks, Makoto

1 0

[MonetDB-users] Limitation in "create table .. as subquery"
by Makoto YUI 04 Jan '10

04 Jan '10

Hello, create table t1 (f1 int); insert into t1 values (1); insert into t1 values (2); create table t2 as (select * from t1 limit 1) with data; The above "create table" caused the following syntax error on the Limit clause. syntax error, unexpected LIMIT, expecting INTERSECT or EXCEPT or UNION or ')' in: "create table t2 as (select * from t1 limit" Is this an expected behavior? This "subquery" issue might be related. http://sourceforge.net/tracker/?func=detail&atid=482468&aid=2620437&group_i… I'm using Nov2009-SP1 on Linux. Thanks, Makoto

3 7

[MonetDB-users] Problem with BSS initialisation.
by Guillaume Yziquel 04 Jan '10

04 Jan '10

Hello. I've spent quite some time lately debugging my issue with OCaml binding of MonetDB libraries: http://www.mail-archive.com/caml-list@yquem.inria.fr/msg05449.html It seems that the BSS is not properly initialised to 0 when dynamically loaded. It seems that the code in mal_box.c relies on the fact that BSS is initialised to 0 when loaded. I'm still trying to figure out how exactly is the libmonetdb5.so library loaded. This is still an obscure point. ltrace did not provide useful information. Advice welcome. -- Guillaume Yziquel http://yziquel.homelinux.org/

2 2

[MonetDB-users] Multiple core usage on Windows
by Alex Bo. 03 Jan '10

03 Jan '10

Hi, Does MonetDB split a single query plan over multiple CPU cores when its possible? (maybe the DataFlow optimizer?) For example, the following query could be executed by all cores in parallel: SELECT field1,SUM(field2) FROM Table1 GROUP BY field1 When i tried to see if it does, i got some weird results: (I am running MonetDB on Windows, on a quad core machine) When I set '--set gdk_nr_threads=1', queries never seem to use more than 25%. (seems logical since only one CPU core is being deployed) When I set '--set gdk_nr_threads=2', queries use up to 50% CPU, but the query is ~15% slower. When I set '--set gdk_nr_threads=4', queries use up to 98% CPU, but the query is take twice as long as with 1 thread. So, my initial assumption that more CPU utilization (more cores over the same query) will result faster query results (where applicable) is wrong. Can it be something specific to the windows platform? or maybe single query execution over multiple cores doesn't work without a specific optimizer flag enabled?... or maybe is it just a bug... Thanks. -- View this message in context: http://old.nabble.com/Multiple-core-usage-on-Windows-tp27002955p27002955.ht… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

Re: [MonetDB-users] segfault with monetdb_sql().
by Guillaume Yziquel 02 Jan '10

02 Jan '10

Martin Kersten a écrit : >>> >>> Program received signal SIGSEGV, Segmentation fault. >>> [Switching to Thread 0x2aaaaf66d910 (LWP 4061)] >>> 0x00002aaaac676de3 in findBox () from /usr/lib/libmonetdb5.so.5 > > i miss here the line number, seems that is ubunti specific > you can do > print i > print box[i] > print name Providing debian packages of libraries with debugging symbols would be great. > i suspect box[i] is not properly initialized to 0 I'll try to check that. >>> (gdb) backtrace >>> #0 0x00002aaaac676de3 in findBox () from /usr/lib/libmonetdb5.so.5 > this gives one hint, it is likely passed without name causing idcmp to fail >>> #1 0x00002aaaac677086 in openBox () from /usr/lib/libmonetdb5.so.5 >>> #2 0x00002aaab7a51cbb in MTIMEprelude () from > > happens while loading the data/time module. It initializes the "time" > module Should I assume that the mtime.mal + lib_mtime.so is the first one to be loaded? Just to know if some dynamic linking successfully happened before or not. >>> /usr/lib/MonetDB5/lib/lib_mtime.so >>> #3 0x00002aaaac6826cf in ?? () from /usr/lib/libmonetdb5.so.5 >>> #4 0x00002aaaac68b2a0 in runMAL () from /usr/lib/libmonetdb5.so.5 >>> #5 0x00002aaaac67cac7 in MALengine () from /usr/lib/libmonetdb5.so.5 >>> #6 0x00002aaaac67d5f6 in malBootstrap () from /usr/lib/libmonetdb5.so.5 >>> #7 0x00002aaaac669f75 in mal_init () from /usr/lib/libmonetdb5.so.5 >>> #8 0x00002aaaab91ef60 in ?? () from /usr/lib/libembeddedsql5.so >>> #9 0x00002aaaab39b73a in start_thread () from /lib/libpthread.so.0 >>> #10 0x00002aaaab67c69d in clone () from /lib/libc.so.6 >>> #11 0x0000000000000000 in ?? () >>> (gdb) >> >> Please tell me what else could be useful. > > This was already useful, because i added two additional defense lines > to capture improperly set environments. OK. I'll check. >>> I am not an CAML expert, but i guess there may be potential issues >>> in location and dynamic loading of the necessary libraries. >>> Where does CAML get the libraries from? >> >> As a rather general statement, dynamic loading is supported within >> OCaml world. I'm not so sure how this works when it is C code that >> triggers dynamic loading, with OCaml in bytecode mode... > > I have added some extra code to make failing loads more visible. > In general, it seems that the modules are not found at the location I'll ask on the OCaml list for that. >> I would like to pinpoint the segfault more precisely in order to go >> back to the OCaml mailing list about potential dynamic loading issues. > I think it can not find your libraries > > set the variable gdk_debug=10 this will show you the environment > settings as seen upon start. Including the location where it expect > database and libraries to recide. Will do that. However, I believe it is not a problem of location, as this works out fine when the OCaml binding is compiled to native code. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 4

[MonetDB-users] (no subject)
by Zoltan Tomory 30 Dec '09

30 Dec '09

Hi everybody, Forgive the reference to a cartoon, but it seems strangely on topic ... >There's a wonderful comic strip on the web called xkcd; I don't know if you've heard of it. I have to share this one with you. It's called "Exploits of a mother": > >http://xkcd.com/327/ Happy New Year! Zoltan Tomory zoltan.tomory(a)mobot.org

1 0

Re: [MonetDB-users] segfault with monetdb_sql().
by Guillaume Yziquel 29 Dec '09

29 Dec '09

Hello. As some discussion had gone off-list, here's some relevant background: The OCaml binding works when the OCaml library is compiled to native code, and it segfaults when the OCaml library is compiled to bytecode. Martin Kersten a écrit : > > What is the gdb stack trace of the processes involved? segfaults are only > analysed that way. Quite new to gdb, so I do not know if this is what you've asked for: > gdb --args ocamlrun test/monetdb_sql.byte > GNU gdb (GDB) 7.0-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > <http://www.gnu.org/software/gdb/bugs/>... > Reading symbols from /usr/bin/ocamlrun...(no debugging symbols found)...done. > (gdb) run > Starting program: /usr/bin/ocamlrun test/monetdb_sql.byte > [Thread debugging using libthread_db enabled] > [New Thread 0x2aaaaf66d910 (LWP 4061)] > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x2aaaaf66d910 (LWP 4061)] > 0x00002aaaac676de3 in findBox () from /usr/lib/libmonetdb5.so.5 > (gdb) backtrace > #0 0x00002aaaac676de3 in findBox () from /usr/lib/libmonetdb5.so.5 > #1 0x00002aaaac677086 in openBox () from /usr/lib/libmonetdb5.so.5 > #2 0x00002aaab7a51cbb in MTIMEprelude () from /usr/lib/MonetDB5/lib/lib_mtime.so > #3 0x00002aaaac6826cf in ?? () from /usr/lib/libmonetdb5.so.5 > #4 0x00002aaaac68b2a0 in runMAL () from /usr/lib/libmonetdb5.so.5 > #5 0x00002aaaac67cac7 in MALengine () from /usr/lib/libmonetdb5.so.5 > #6 0x00002aaaac67d5f6 in malBootstrap () from /usr/lib/libmonetdb5.so.5 > #7 0x00002aaaac669f75 in mal_init () from /usr/lib/libmonetdb5.so.5 > #8 0x00002aaaab91ef60 in ?? () from /usr/lib/libembeddedsql5.so > #9 0x00002aaaab39b73a in start_thread () from /lib/libpthread.so.0 > #10 0x00002aaaab67c69d in clone () from /lib/libc.so.6 > #11 0x0000000000000000 in ?? () > (gdb) Please tell me what else could be useful. > I am not an CAML expert, but i guess there may be potential issues > in location and dynamic loading of the necessary libraries. > Where does CAML get the libraries from? As a rather general statement, dynamic loading is supported within OCaml world. I'm not so sure how this works when it is C code that triggers dynamic loading, with OCaml in bytecode mode... I would like to pinpoint the segfault more precisely in order to go back to the OCaml mailing list about potential dynamic loading issues. > We can help (on Xmas days) only with proper output. The level of > programming > calls for expertise in dealing with C debuggers, dynamic loaders, and > thread > management. Guessing does not bring us further. You have to use the > right tools. OK. This is always a good opportunity to learn more about C world... Thanks for pointing out the right tools. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

1 0

[MonetDB-users] Instalation prob. in Ubuntu
by Rui Anastácio 28 Dec '09

28 Dec '09

Hi, I'm trying to install monetdb in my Linux Ubuntu 9.10 Karmik and have followed the instructions but have this dependency problem: monetdb-geom-dev: Depends: libmonetdb5-geom0 not installable monetdb-testing: Depends: python-monetdb not installable Regards Rui Anastácio

1 0

[MonetDB-users] Updating a single column by means of a secondary table
by Stefan de Konink 27 Dec '09

27 Dec '09

Hi, I have the following issue; I would like to update a new column with respect to old data. In essence this would be the idea that I want to do: update anbikvk set intrekking = (select intrekking from anbi_update where anbi_update.naam = anbikvk.naam); The above way does not work, the only other way that I would know would involve a temporary table. Could anyone elaborate how this is possible using the update statement? Stefan

3 3

[MonetDB-users] segfault with monetdb_sql().
by Guillaume Yziquel 27 Dec '09

27 Dec '09

Hello. I've been having segfaults when using the following wrapper code. (Do not pay attention to the CAML* macros, and String_val is a macro that converts a string in OCaml to a C string, and Val_ptr is a function that wraps up a pointer into OCaml.) > CAMLprim value ml_monetdb_sql (value ml_dbfarm, value ml_dbname) { > CAMLparam2(ml_dbfarm, ml_dbname); > CAMLlocal1(result); > char * dbfarm; > char * dbname; > dbfarm = String_val(ml_dbfarm); > dbname = String_val(ml_dbname); > Mapi mapi = monetdb_sql(dbfarm, dbname); > result = Val_ptr(mapi); > CAMLreturn(result); > } I'm calling this code with dbfarm and dbname set to a place I can write to, and that does not contain folders, files, or data at the time the function is called from OCaml code. When executing the following piece of code from the OCaml toplevel > #use "topfind";; > #thread;; > #load "monetDB5.cma";; > > open MonetDB5;; > let mapi = monetdb_sql "/home/yziquel/sandbox/monetdb" "testdb";; I get a segfault arising from monetdb's shared libraries (I'm using the Debian packages on a 64 bits platform): > yziquel@seldon:~/git/ocaml-monetdb5$ ocaml -init ocamlinit > Objective Caml version 3.11.1 > > Findlib has been successfully loaded. Additional directives: > #require "package";; to load a package > #list;; to list the available packages > #camlp4o;; to load camlp4 (standard syntax) > #camlp4r;; to load camlp4 (revised syntax) > #predicates "p,q,...";; to set these predicates > Topfind.reset();; to force that packages will be reloaded > #thread;; to enable threads > > /usr/lib/ocaml/threads: added to search path > /usr/lib/ocaml/unix.cma: loaded > /usr/lib/ocaml/threads/threads.cma: loaded > Erreur de segmentation Here at the last line... Looking at the code in monetdb-sql-2.34.2/src/backends/monet5/embeddedclient.c.in I'm unable to determine if the segault comes from the invocation of pthread_create or from the invocation of mapi_start_talking. > Mapi > monetdb_sql(char *dbfarm, char *dbname) > { > Mapi mid; > pthread_t sqlthread; > stream **server; > > int len = mo_builtin_settings(&embedded_set); > > /* needed, to prevent the MonetDB config file to be used */ > len = mo_add_option(&embedded_set, len, opt_config, "prefix", MONETDB5_PREFIX); > len = mo_add_option(&embedded_set, len, opt_config, "config", MONETDB5_CONFFILE); > len = mo_add_option(&embedded_set, len, opt_config, "monet_mod_path", > "@QXlibdir@@QDIRSEP@MonetDB5@PATHSEP@@QXlibdir@@QDIRSEP@MonetDB5@QDIRSEP@lib@PATHSEP@@QXlibdir@@QDIRSEP@MonetDB5@QDIRSEP@bin"); > > embedded_len = mo_system_config(&embedded_set, len); > embedded_len = mo_add_option(&embedded_set, embedded_len, opt_cmdline, "gdk_dbfarm", dbfarm); > embedded_len = mo_add_option(&embedded_set, embedded_len, opt_cmdline, "gdk_dbname", dbname); > > server = mapi_embedded_init(&mid,"sql"); > > pthread_create(&sqlthread, NULL, start_sql_server, (void *) server); > > mapi_start_talking(mid); > > return mid; > } Could someone give me hints as to what is going wrong here? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

3 6

Re: [MonetDB-users] mserver5 crashed after 10, 000 times sql queries through JDBC interface
by xulizhong 26 Dec '09

26 Dec '09

> Could you provide the query and schema? CREATE TABLE STOCKINFO ( STOCK_ID INTEGER NOT NULL, STOCK_CODE VARCHAR(30), EX_CODE VARCHAR(30), SH_CODE VARCHAR(30), LONGNAME_CN VARCHAR(50), SHORTNAME_CN VARCHAR(30), EX_ID INTEGER NOT NULL, CSRC_CAT INTEGER DEFAULT NULL, SW_CAT INTEGER DEFAULT NULL, SH_CAT INTEGER DEFAULT NULL, HAS_DETAIL BOOLEAN DEFAULT NULL, BG_CODE VARCHAR(20), IPO_TIME DATE DEFAULT NULL, BOARD_CAT INTEGER DEFAULT NULL, IS_ST BOOLEAN DEFAULT NULL, HAS_OPTION BOOLEAN DEFAULT NULL, CURRENCY INTEGER DEFAULT NULL, CARRY INTEGER DEFAULT NULL, DECIMALS INTEGER DEFAULT NULL, MIN_PRICE_CHANGE DOUBLE DEFAULT NULL, STOCK_CAT INTEGER DEFAULT NULL, SURGED_LIMIT DOUBLE DEFAULT NULL, DECLINE_LIMIT DOUBLE DEFAULT NULL, HIGH_52WK INTEGER DEFAULT NULL, LOW_52WK INTEGER DEFAULT NULL ); query: SELECT STOCK_ID FROM STOCKINFO WHERE LOW_52WK<85 and HIGH_52WK>110 and EX_ID=15 AND IPO_TIME>'2000-1-1' AND STOCK_CODE LIKE '1%'; I write a simple java program using 4 threads to execute this query with a connection pool of 4 max active connections. The mserver5 process' memory increase steadily upto 2.5G, then the process crashes with segmentation fault:( I have noticed something special, when I run the query using : ResultSet result = conn.prepareStatement(sql).executeQuery(); the monetdb5 server will consume large amount of memory and crash definitely. However, when I query using: ResultSet result = conn.createStatement().executeQuery(sql); the server's memory usage is very stable. This may indicate there's something wrong with the prepared statement of the sql module of MonetDB5. Below is my test program: ////////////////////////////////////////////////////////// import java.sql.Connection; import java.sql.ResultSet; import java.sql.SQLException; import java.util.Date; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.TimeUnit; import org.apache.commons.dbcp.BasicDataSource; public class MonetBenchmark { /** * @param args */ public static void main(String[] args) { ExecutorService executor = Executors.newFixedThreadPool(4); final BasicDataSource dataSource = new BasicDataSource(); dataSource.setMaxActive(4); dataSource.setInitialSize(4); dataSource.setDriverClassName("nl.cwi.monetdb.jdbc.MonetDriver"); dataSource.setUrl("jdbc:monetdb://localhost/demo"); dataSource.setUsername("monetdb"); dataSource.setPassword("monetdb"); final String sql = "SELECT STOCK_ID FROM STOCKINFO WHERE LOW_52WK<85 and HIGH_52WK>110 and EX_ID=15 AND IPO_TIME>'2000-1-1' AND STOCK_CODE LIKE '1%';"; System.out.println("running ... " + new Date());; long start = System.currentTimeMillis(); int loop = 10000; for (int x = 0; x < loop; x++) { executor.submit(new Runnable() { public void run() { Connection conn = null; try { conn = dataSource.getConnection(); ResultSet rs = conn.createStatement().executeQuery(sql); //using prepared statement will cause the server crashed with segmentation fault and very big memory usage. // ResultSet rs = conn.prepareStatement(sql).executeQuery(); while(rs.next()) { //... } } catch (SQLException e) { e.printStackTrace(); } finally { try { conn.close(); } catch (Exception e) { } } } }); } try { executor.shutdown(); executor.awaitTermination(5, TimeUnit.MINUTES); } catch (InterruptedException e) { e.printStackTrace(); } long time = System.currentTimeMillis() - start; System.out.println(time + " ms for " + loop + " times query! " + (time / loop) + " ms/q"); } } And my last question, is this newest version of MonetDB suitable for production use if I always use normal jdbc statement to execute my query? Thanks alot.

3 3

[MonetDB-users] OCaml binding for embedded MonetDB5.
by Guillaume Yziquel 24 Dec '09

24 Dec '09

Hello. I would like to create a binding for the Objective Caml language of an embedded MonetDB5. My personal focus would be on embedded_mal, but I'm perfectly willing to get started on embedded_sql. I've been looking at some documentation, which I unfortunately find rather terse. For instance: http://monetdb.cwi.nl/MonetDB/Documentation/A-Simple-Example.html I'm, for example, wondering about the semantics of the embedded_sql function whose code from embeddedclient.c.in is reproduced below. > Mapi > embedded_sql(opt *set, int len) > { > Mapi mid; > pthread_t sqlthread; > stream **server; > > if (set) { > embedded_set = set; > embedded_len = len; > } > server = mapi_embedded_init(&mid,"sql"); > > pthread_create(&sqlthread, NULL, start_sql_server, (void *) server); > > mapi_start_talking(mid); > > return mid; > } I'd therefore like to know if there is some piece of documentation (aside the code itself) where I could gain a deeper understanding of what I want to wrap up and bind. Links appreciated. -- Guillaume Yziquel http://yziquel.homelinux.org/

1 1

[MonetDB-users] monetdb program in debian packages?
by Guillaume Yziquel 24 Dec '09

24 Dec '09

Hello. I've been trying to follow section 1.78.1 Start and Stop the Server of the manual for MonetDB 5, and I fail to find the monetdb program in the available debian packages. Which package should I install to be able to launch the monetdb program as in section 1.78.1? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

3 5

[MonetDB-users] mserver5 crashed after 10, 000 times sql queries through JDBC interface
by xulizhong 24 Dec '09

24 Dec '09

I'm using the latest MonetDB5: MonetDB-Nov2009-SuperBall-SP1 on Red Hat Enterprise Linux AS release 4 (Nahant Update 4) x86_64. dependencies: openssl-0.9.8l, pcre-8.00. The box has 8G physical memory, about half of them is available. building options: ./monetdb-install.sh --prefix=/opt/monetdb5 --enable-sql --enable-optimise After installation, I started the mserver5 process: /opt/monetdb5/bin/mserver5 --dbinit "include sql;", then I load one table exported from MySQL with about 1M rows and 20 columns. I noticed small memory increase when I run a simple query on the mclient console for some times. So I run this query through the JDBC interface for 10000, I can see a big memory consumption rise when the test program is running. and finally the mserver5 process crashed with segmentation fault. I can always reproduce this problem. What's wrong? Have I used some wrong build options? Help! Frank

2 1

[MonetDB-users] Fatal error: aborted too deep recursion
by Harm van Beek(DT) 24 Dec '09

24 Dec '09

During the development of our tool, using Xquery on MonetDB, we ran onto the following error: ERROR = !fatal error: aborted too deep recursion >From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. I tried several MonetDB releases (May2009-SP2, Aug2009-SP2, Nov2009-DB0) on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result. Does anyone have any directions on how to solve this? Can we reconfigure the compiler to fix this (or recompile it with options)? Thanks in advance! Harm van Beek Netherlands Forensic Institute

6 11

[MonetDB-users] man page for debian package.
by Guillaume Yziquel 23 Dec '09

23 Dec '09

Hi. Just a simple suggestion: I'm using the Debian package for MonetDB v5, and have just started tinkering with it. One thing I really like is a man page. It would be nice to have one. Not necessarily an exhaustive one, but one that would comment the mserver5 options a bit more clearly than mserver5 --help, and which would provide pointers to documentation. Finding documentation on monetdb wasn't easy. Writing good documentation is a painful task, but providing pointers to documentation within a man page is much easier... All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

1 0

[MonetDB-users] Memory Usage
by Rui Anastácio 23 Dec '09

23 Dec '09

Hi ! I'm runnig MonetDB under Linux Ububtu with 4Gb of RAM. I need to benchmark it running 1Gb, 2Gb, 3Gb and 4Gb. Todo so I need to limit the memory usage. How can I configure MonetDB to limit it's memory usage? and doing so am I able to run complex queries (with short memory) ? Regards Rui Anastácio

4 4

[MonetDB-users] MonetDB scalability
by Bojan Šmid 22 Dec '09

22 Dec '09

Hi, I am thinking of using MonetDB as a data storage for my application. The application will create lots of data (few GB of new data each day) while there will not be many queries (probably few thousands/day). Queries will be complex, but they should execute really fast. Since data will grow pretty fast, I must prepare for scaling, so I need to know is there a way to distribute MonetDB over multiple machines? Thanks and regards

4 5

Re: [MonetDB-users] MonetDB scalability
by OG 22 Dec '09

22 Dec '09

Hello, Somebody on this list asked about scalability the other day: http://markmail.org/message/2unmuwq2cfz7mrmm I was wondering the same thing. Is there a way to partition/shard MonetDB and still have joins work across partitions living on separate servers? If not, is there a recommended practice for what to do with MonetDB once the limit of a single server has been reached? Thanks, Otis -- Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch

2 1

[MonetDB-users] UTF-8 for table/column names
by Alex Bo. 22 Dec '09

22 Dec '09

Hi, Is there support for UTF8/Unicode for field/table names? It seems like i can only use ASCII characters (for example, a field name with specific Spanish letters wouldn't work) inside field/table names. I'm using the Nov09 on Windows. Thanks. -- View this message in context: http://old.nabble.com/UTF-8-for-table-column-names-tp26888868p26888868.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3