- users-list - monetdb.org

[MonetDB-users] (no subject)
by Richard Wesley 20 Oct '08

20 Oct '08

Hi All - I am trying to evaluate MonetDB on Windows by loading up some large data sets. I'm not having much luck and I'm hoping someone can help me out. The first set of problems revolve around the ODBC driver. It seems to not be able to report errors or even return the results of queries reliably. Most errors come back as a generic error message although we occasionally run into strange messages involving UTF-16 conversions when no string columns are involved (e.g. a count distinct of a numeric column). We are using the July 2008 version from the web site. Are these known issues and is there a more recent build we could try? This is not in production yet, we are simply trying to evaluate the system. I have managed to work around the ODBC problems very slowly and have got to a point where I can load small data sets. But when I then try to do a large insert using the COPY FROM syntax, my client got a "connection terminated" error. When I take the COPY FROM statement and paste it into the client, the MonetDB server crashes. The only log files I can find have a few (mostly unprintable) characters in them. Restarting the server and trying again produces the same problem. The insert query is COPY 6568253 RECORDS INTO "Airlines"."_2005_On_Time_Performance" FROM 'C:\\DOCUME~1\\hawkfish\\LOCALS~1\\Temp\\1\\7A.tmp\\table.tbl' USING DELIMITERS '\t', '\n', '\''; Is there some limit on the COPY INTO syntax? Is there some log file somewhere that could point me in the right direction? TIA, ________________________________________________________ Richard Wesley Senior Software Developer Tableau Software Visit: http://www.trytableau.com/now.html

4 13

[MonetDB-users] MonetDB with Pentaho and Kettle
by Tom Barber 17 Oct '08

17 Oct '08

Hi Guys, As of 3.1 Kettle now has a connector built in for Monet so I thought I'd give it another go after not getting too far earlier in the year. Anyway I have it up and running on a Quad Core Xeon box running Centos x86_64 and it seems pretty happy, but when I try to copy sizable amounts of data into it, kettle stops with an error: 2008/10/17 12:29:22 - Table output.0 - Prepared statement : INSERT INTO voc.d_budgets3 (entity_anl_code, trad_code, Cost, Sales, CostMTD, SalesMTD) VALUES ( ?, ?, ?, ?, ?, ?) 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : Because of an error, this step can't continue: 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : org.pentaho.di.core.exception.KettleException: 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : Error inserting row into table [d_budgets3] with values: [19], [A09], [0], [0], [0], [0] 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : Error inserting/updating row 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : current transaction is aborted (please ROLLBACK) But to my untrained eye the data looks fine(coming from a mysql database) and I don't know where to start debugging this or what settings I could change. The monet table is 2 Strings, 2 Int, 2 Big Num and it always inputs a load of data before stopping. What can I look at to resolve the issues? otherwise, rocking database and keep up the good work. Thanks Tom CONFIDENTIALITY NOTICE This e-mail is confidential and may also be privileged. Please notify us immediately if you are not the intended recipient. You should not copy it, forward it or use it for any purpose or disclose the contents to any person. If you have received this e-mail in error, please notify us as soon as possible and destroy your copy. Unless otherwise expressly stated, this e-mail is not intended to constitute a business letter, order form or other offer or invitation to you, nor does this e-mail form the basis of any contract. Contracts may not be concluded by e-mail and any contract or agreement attached is subject to contract and shall not and is not intended to create a legally binding relationship. This e-mail is sent by or on behalf of Encon Limited or the relevant subsidiary undertaking of Encon Limited with which you are dealing. Encon Limited is registered in England and Wales with company number 3411533 registered office at The Wolseley Center, Harrison Way, Leamington Spa, CV31 3HH, United Kingdom. VAT No 362 0233 93. Encon Ireland is registered in Ireland with company number 262605 registered office at Wolseley Ireland Moydrum Road, Athlone, Co Westmeath. Save a Tree ! - Please think before you print.

3 3

[MonetDB-users] Commercial Support
by Rick Spickelmier 17 Oct '08

17 Oct '08

Hi! Are there companies that are providing commercial support for MonetDB? The home page for MonetDB seems to imply that. If so, can somebody send me a pointer and does anybody have any experience with it? Thanks! Rick Rick Spickelmier | CTO | Success Metrics, Inc. | w (415) 738-2078 | ricks(a)successmetricsinc.com <mailto:ricks@successmetricsinc.com> | www.successmetricsinc.com | www.birst.com

3 2

[MonetDB-users] CREATE INDEX syntax?
by Richard Wesley 14 Oct '08

14 Oct '08

Hi Sjored - I have a lot of questions, so I thought I start a few threads if that is all right. What is the CREATE INDEX syntax? According to the manual it is CREATE [UNIQUE] INDEX <index_name> ON <table_name> ( <column_name>, ...) ; But whey I use this syntax, I get strange errors: sql>CREATE INDEX "_tidx_usernames_2a" ON "Performance"."usernames" ("username"); !CREATE INDEX: no such table 'usernames' sql>select count(*) from "Performance"."usernames"; +--------+ | count_ | +========+ | 5 | +--------+ sql> ________________________________________________________ Richard Wesley Senior Software Developer Tableau Software Visit: http://www.trytableau.com/now.html

1 0

[MonetDB-users] MonetDB on Windows
by bryan rasmussen 07 Oct '08

07 Oct '08

Hi, I haven't looked at monetdb for a while, but now for a new project I have I think it would be a really good solution. I downloaded the current windows installers yesterday (I want to develop this project on Windows for various reasons, specifically related to demos of another component part of the project I need to get up and running quickly that will need to run on Windows) anyway the installers did not install. I don't necessarily want to compile the source etc. since I didn't want to spend a lot of time on getting the database to work in the demo phase, so the question is: what is the last working Windows MSI version so I can download that version and worry about building a new one later? The main problem in the installed servers are a hard coded path to a config file, according to the following error message I get: Could not open file c:\Documents and Settings\sjoerd\My Documents\src\stable\vs32\NT32\etc\monetdb5.conf I guess I could create that path for the purposes of testing but I would rather get a working version. Thanks, Bryan Rasmussen

3 6

[MonetDB-users] Reusing index on another computer
by jz＠uva 07 Oct '08

07 Oct '08

Hi all, I have built an index on a server. I would like to reuse that index on my desktop for testing purposes. However, when I copy the demo directory (consisting of the index), and I restart MServer Mserver --dbinit='module(pathfinder);' --set xquery_backend=milprint_summer then I discover this error: !ERROR: database created with incompatible Mserver So my question is: can I, and if yes, how do I reuse/transfer an index from one computer to another one? Thanks, junte

6 13

[MonetDB-users] allocate timing and deallocate timing of memory
by gashi gashi 07 Oct '08

07 Oct '08

Hi, All I'm working with MonetDB Server v5.6.0, based on kernel 1.24.0 over RedHat Linux Enterprise 5.2. Would you tell me timing which in the memory is allocated and timing in which the memory is deallocated, or is there an instruction that deallocate the memory? Regards, gashi

2 1

[MonetDB-users] merovingian locking up - port issue?
by Ross Bates 07 Oct '08

07 Oct '08

I'm having an issue with merovingian locking up. This is the typical scenario. - Start mero and my db with ./merovingian sobi - Everything works great for anywhere between 50 & 100 queries, then mero stops responding (I'm using python, jdbc & php) - I can still connect to the db from the command line using ./mclient -d sobi -lsql -p 50001 - both merovingian and mserver5 processes continue to run, but mero will end up dying within 1-2 minutes by itself leaving the mserver5 process running - even when mero dies, it appears that all the ports are still in use. This is the result of netstat -a AFTER mero is dead: tcp 0 0 *:50000 *:* LISTEN tcp 0 0 *:50001 *:* LISTEN tcp 0 0 mayfair.local:50000 mayfair.local:48973 ESTABLISHED tcp 0 0 mayfair.local:48973 mayfair.local:50000 ESTABLISHED udp 0 0 *:50000 *:* - when I try to restart mero to reattach it to the database it fails with this error: ./merovingian: binding to stream socket port 50000 failed: Address already in use - stopping the db doesn't work becase mero isn't running. ./monetdb stop sobi warning: MonetDB Database Server is not running stop: cannot perform: MonetDB Database Server (merovingian) is not running - the only thing left to do is kill the mserver5 process and restart both mero and the db. Unfortunately the merovingian.log file doesn't tell me anything. Any other tips for helping troubleshoot this issue? The version is 5.7.0 and was built from the Aug 23rd nightly. Many Thanks, Ross

2 8

[MonetDB-users] Can I copy a local txt file into a remote MonetDB table
by Li Xiaohua 07 Oct '08

07 Oct '08

Hi: I want to copy a txt file into a remote MonetDB table. When I use the copy into command. what should I use after the "into"? using servername.database.schema.tablename? does it work?

1 0

[MonetDB-users] running the monetdb MSI installers on Windows
by bryan rasmussen 06 Oct '08

06 Oct '08

Hi, Is there anyone that has managed to run any particular Monet MSI installer with a resulting successful install of MonetDB (successful implying the database server will start afterwards)? If yes, which version did you use? The problems I have run into so far with the current stable release: Can not find DLLs when starting. Dlls do not seem to have proper entry points (when I try to run them) the server is expecting to find a config file under documents and Settings\Sjoerd\..... I tried uninstalling and installing one release earlier for the sql database, got the first problem again, didn't check if it had all the other problems. Best Regards, Bryan Rasmussen

1 0

Re: [MonetDB-users] best practices with connection pool
by gashi gashi 01 Oct '08

01 Oct '08

Hi,All I found this subject in the archives, and I am having the same problem. ---------------------------------------------------------------------- >> What is currently the best practice of connecting to MonetDB5/SQL? Is >> there much overhead in the connection phase (thus: cache the connection) >> or can a developer just reconnect each time the database connection is >> required without much overhead? >> >> The target platform is extremely demanding in requests and memory, so I >> would like to design it good from the start. >Then go for connection pooling. Logging in requires a handshake, which >is expensive on large scale. Beware that using >1 connection at the >same time to an SQL server is a challenge that is only minimally tested. >My practical experiences with that boils down to signal 11. ---------------------------------------------------------------------- Would you let me know more detail? Your meaning of "Beware that using >1 connection at the same time..." is which case? Case 1. : 2 or more process try to connect at the same. Case 2. : 1 process is connecting, and other process try to connect. I appreciate your kindly support. gashi

2 3

[MonetDB-users] billions rows ?
by eric Gtep 28 Sep '08

28 Sep '08

Hello, I'm testing for store large collections of genotypes on Monetdb. In few words, a genotype is a little piece of information ( about 3 or 4 char ) related to an individual and a dna marker. So a genotypes looks like contents of cell of sheet. Sometime we need to access them by individuals (cross marker), sometime by dna marker (cross individual). Today genotyping technologies provide to get 600, 000,0000 genotypes by run. Is MonetDB able to manage efficiently tables with several billions of rows ? have you any example of application with lot ( > 1 billion) of rows in only one table ? I' ve compiled and installed monetDB/SQL on a Dell PE2950 with 2 quadcores intel xeon 2.66 Ghz and 4 GB RAM with success I 've created a very basic table to store genotype : Create table genotypes ( ind char(10), mark char(10), alleles(char3) ) after I've populated this table with the "copy into table" statement About 370 millions rows have been loaded after 7 minutes. I haven't defined any index. >From mclient I sent the query below : select * from genotypes where alleles ="A A"; Immediatly the server became frozen and after about ten minutes a w unix command showed : load average 16 !!!! I stopped the query Could you explain to me what has appended ? Is this behaviour normal ? Is it possible to restrict the cpu ressources allowed to monetDB ? Thank you in advance for your advices and your help Eric

4 3

[MonetDB-users] How to declare a bug ?
by Pierre BOIZOT 27 Sep '08

27 Sep '08

Hi All, Somebody can explain me how i can submit a bug ? Regards. Pierre -- Pierre Boizot Chemin des Ecureuils, 8 CH-1009 PULLY Mobile : +41 795713660

2 1

[MonetDB-users] Perl DBD
by Pierre BOIZOT 26 Sep '08

26 Sep '08

Hi all, I install on a ubuntu monetdb , i began to test sql , insert , update and select . very nice performance. But when i try to use Perl DBD to access at the database . nothing work :( the CPAN is not update. the perl DBD in the source have no test no example :( what programming interface work with monetdb ? perl ? perhaps i jump something. python ? Where can i find small examples? Thank you for your help Regards. Pierre - Pierre Boizot Rue des philosophes , 13 1400 Yverdon les bains Mobile : +41 795713660

3 2

[MonetDB-users] Installation Problems
by Scott Stephens 25 Sep '08

25 Sep '08

I've been attempting to install MonetDB by compiling the Aug. 2008 superball. My system is 64-bit linux, specifically the Ubuntu Intrepid Ibex alpha . It appears that the installation is failing while compiling the JDBC driver, with the error message: BUILD FAILED java.lang.UnsupportedClassVersionError: Bad version number in .class file I'm using the Sun Java 1.5.0 JRE and JDK. You can look at my full installation log here: http://docs.google.com/Doc?id=dhqbmd8x_6fqdnn7c3 Can somebody help me out? I'm hoping I've just made an easily corrected newbie mistake, but I'm out of things to try. Thanks, Scott

3 8

[MonetDB-users] basic math error?
by Kirk Abbott 24 Sep '08

24 Sep '08

Hello All, sql>select 585.0 / 635.0; +----------------------+ | sql_div_single_value | +======================+ | 0 | +----------------------+ and sql>select cast(585.0 / 635.0 as numeric(10,4)); +----------------------+ | sql_div_single_value | +======================+ | 0.0000 | +----------------------+ Is not what I was expecting. Ideas? Kirk.

4 5

[MonetDB-users] Bug....
by Pierre BOIZOT 23 Sep '08

23 Sep '08

Hi all, a test with the gold data .... no errors on the load if i stop and restart the database errors is allways here. sql>select age,gender, count(*), avg(price) from car group by age, gender; +-----+-------+-------------+-----------------------+ | age |gender |count_gender |avg_price | +=====+=======+=============+=======================+ | 19 |f | 110025 | -1449.6788093615087 | | 41 |m | 97200 | -6411.6489300411522 | ........ 52 |f | 95940 | -6467.6865332499483 | +-----+-------+-------------+-----------------------+ Timer 2694.008 msec 92 rows sql>select count(*) from car where price = 0; +--------+ | count_ | +========+ | 0 | +--------+ Timer 2502.145 msec 1 rows sql>select count(*) from car where price < 0; +--------+ | count_ | +========+ | 0 | +--------+ Timer 148.724 msec 1 rows see the trace file . Regards Pierre -- Pierre Boizot Rue des philosophes , 13 1400 Yverdon les bains Mobile : +41 795713660

1 0

[MonetDB-users] Several newbie questions: Performance and plans
by Mark V 23 Sep '08

23 Sep '08

Hi, I am more of an end user, new to column store DB's in general and MonetDB in particular. I'm thinking of using MonetDB in a context where CRUD would be done using SQL statements (as much as possible). I have some questions that I'd appreciate some feedback on: It seems that in column stores single insert/update/delete are much slower than reads. In LucidDB this can be in the order of seconds. Q1) In MonetDB are single insert/update/delete in the order of seconds? Comparable to LucidDB? Q2) If so, when might this change/improve and by how much? On comparative performance. I've seen the Goggle tech talk (2008/01) on MonetDB/X100 indicating some incredible performance: Q3) When might those improvements be incorporated into the stable/released MonetDB? Q4) How does MonetDB compare to Kdb and Vertica? Finally, I'm toying with writing a MonetSQL adapter for Ruby's Sequel ORM, hence the reliance on SQL statements. I have seen the MonetDB ActiveRecord library, which uses SWIG to make the Ruby MonetDB-mapi library. Q5) Will MonetDB be implementing something like MySQL's Information_Schema? Q6) What is the relative performance of the JDBC and SWIG-MAPI interfaces? Thanks Mark

2 1

[MonetDB-users] Ubuntu packages and launchpad PPA for MonetDB
by Gijs Molenaar 12 Sep '08

12 Sep '08

Hello everyone, I'm working on precompiled Ubuntu packages for MonetDB, which can be found here: https://launchpad.net/~monetdb/+archive These packages are meant for Ubuntu users who don't want to compile monetdb on every Ubuntu system where they want to use this software. You can install the packages and automatically update to new released packages by adding the following to /etc/apt/sources.list: --- 8< cut here --- deb http://ppa.launchpad.net/monetdb/ubuntu hardy main deb-src http://ppa.launchpad.net/monetdb/ubuntu hardy main ------------------- These packages are quite 'fresh' and not fully tested and completed, but the basics should work fine. If you want to test them out or/and you have some comments, feel free to contact me. Note that these packages are not signed due to a limitation in launchpad. -- Gijs Molenaar http://gijs.pythonic.nl

3 5

Re: [MonetDB-users] I'm looking for a fast way to write data to the DB from a scripting language.
by Michalis Polakis 07 Sep '08

07 Sep '08

Good morning Jay, you can use the Ruby MAPI Wrapper => MAPI calls through Ruby. It's elegant and simple and was used to implement the Ruby on Rails adapter for MonetDB. Plus, from what I see in the SWIG input file, the insert and copy_from functions should be implemented in the Ruby MAPI Wrapper. If you want to go the Ruby way, you may find the files in the MonetDB clients package or at http://rubyforge.org/projects/monetdb-ror/ as part of the Ruby on Rails MonetDB adapter. Regards, Michalis ----------------------------------------------------------------------------------------------- Hi, I have 50 million rows and 15 columns that I'd like to put into one table. My first attempt to transfer them was through PHP using the monetdb_query method. This is causing a lot of disk thrashing and my throughput is about 100 rows/sec. I get an "undefined function" error for both monetdb_insert and monetdb_copy_from. I would like to find a fast method, but from inside a common scripting language. Thanks, Jay

3 3

[MonetDB-users] I'm looking for a fast way to write data to the DB from a scripting language.
by JJ Jakosky 07 Sep '08

07 Sep '08

Hi, I have 50 million rows and 15 columns that I'd like to put into one table. My first attempt to transfer them was through PHP using the monetdb_query method. This is causing a lot of disk thrashing and my throughput is about 100 rows/sec. I get an "undefined function" error for both monetdb_insert and monetdb_copy_from. I would like to find a fast method, but from inside a common scripting language. Thanks, Jay

1 0

[MonetDB-users] Dual instances of MonetDB for 1 dbfarm - is it possible?
by McKennirey.Matthew 04 Sep '08

04 Sep '08

3 4

[MonetDB-users] Monetdb - User defined functions
by Kirk Abbott 04 Sep '08

04 Sep '08

Hello, We are trying to move some data over to Monetdb and are finding that we are missing some of our favorite formatting functions (to_date, to_char etc.). Does Monetdb have a user defined functions extension framework? If so, we would try to mimic these functions there. Thanks, Kirk.

3 8

Re: [MonetDB-users] ERROR 404 /~monet/tars/latest/stable/MonetDB.tar.gz.md5sum
by McKennirey.Matthew 04 Sep '08

04 Sep '08

3 2

[MonetDB-users] Incorrect SQL update result
by Christophe Galerne 03 Sep '08

03 Sep '08

Hi, I am running: # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'clg' # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.24.0 loaded I am copying some sample SQL which illustrates what I believe is a bug: ################################################################################ CREATE TABLE SOURCE ( TOTO VARCHAR(5), TITI VARCHAR(5), VALUE DOUBLE ); INSERT INTO SOURCE VALUES('1', '1', 5.0) , ('1', '2', 4.0), ('2', '1', 5.0), ('2', '2', 5.0), ('2', '3', 5.0); INSERT INTO SOURCE VALUES('1', '1', 5.0) , ('1', '2', 4.0), ('2', '1', 5.0), ('2', '2', 5.0), ('2', '3', 5.0); CREATE TABLE TARGET ( TOTO VARCHAR(5), TITI VARCHAR(5), VALUE DOUBLE ); INSERT INTO TARGET VALUES('1', '1', 0) , ('1', '2', 0), ('2', '1', 0), ('2', '2', 0), ('2', '3', 0); UPDATE TARGET SET VALUE = ( SELECT SUM(VALUE) FROM SOURCE WHERE TOTO=SOURCE.TOTO and TITI=SOURCE.TITI); CREATE TABLE TARGETOS ( TATA VARCHAR(5), TUTU VARCHAR(5), VALUE DOUBLE ); INSERT INTO TARGETOS VALUES('1', '1', 0) , ('1', '2', 0), ('2', '1', 0), ('2', '2', 0), ('2', '3', 0); UPDATE TARGETOS SET VALUE = ( SELECT SUM(VALUE) FROM SOURCE WHERE SOURCE.TOTO=TATA and SOURCE.TITI=TUTU); ################################################################################ The expected results is for the tables TARGET and TARGETOS to contain the same result since they only differ by the name of their columns. It seems that monetdb gets confused by the fact that table SOURCE and table TARGET have the column names. Thanks, Christophe

2 2

[MonetDB-users] SyntaxException in "monetdb create demo" statement
by rmaadil 03 Sep '08

03 Sep '08

I have installed MnetDB by downloading the file "MonetDB-ODBC-Installer-i386-20080812.msi". Then I opened MonetDB SQL Server. The last message that I got on the command prompt was "MonetDB/SQL module v2.24.0 loaded". Then I tried the statement "monetdb create demo", but I got the following error SyntaxException:parseError:monetdb create demo SyntaxException:parseError ^';' expected Can anybody tell me that where I was going wrong. -- View this message in context: http://www.nabble.com/SyntaxException-in-%22monetdb-create-demo%22-statemen… Sent from the monetdb-users mailing list archive at Nabble.com.

2 5

[MonetDB-users] ERROR 404 /~monet/tars/latest/stable/MonetDB.tar.gz.md5sum
by McKennirey.Matthew 03 Sep '08

03 Sep '08

2 1

Re: [MonetDB-users] [ADT] bug report?
by Stefan Manegold 02 Sep '08

02 Sep '08

On Mon, Sep 01, 2008 at 11:22:10PM +0200, Ying Zhang wrote: > Hi, > > I have just updated my monet source from souceforge and did a from > scratch compilation on my leopard. > > Fabian's fix gets rid of the error "implicit declaration of..." (thanks!) > However, the other error "ISO C90 forbids.." is still there. With the > following patch, I was able to compile the module MonetDB (a complete > rebuild of monet modules is still ongoing). > > I simply moved the definition of len to the begin of the function: > > $ cvs diff src/gdk/gdk_system.mx > Index: src/gdk/gdk_system.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_system.mx,v > retrieving revision 1.115 > diff -r1.115 gdk_system.mx > 570a571,574 > > #if defined(__FreeBSD__) || defined(__NetBSD__) || > defined(__OpenBSD__) || defined(__MACH__) > > size_t len = sizeof(int); > > #endif > > > 578d581 > < size_t len = sizeof(int); Thanks. I just checked in (my version of ;-)) your patch. Stefan > Jennie > > > On Sep 01, 2008, at 22:35 , Hendrike Peetz wrote: > >> nope. still the same problem. I checked out from the svn right after I >> got your mail. >> >> On Sep 1, 2008, at 10:13 PM, Fabian Groffen (Project Account) wrote: >> >>> I tried to fix the issue. Please report if it works. >>> >>> On 01-09-2008 22:01:01 +0200, Fabian Groffen (Project Account) wrote: >>>> On 01-09-2008 21:57:13 +0200, Fabian Groffen wrote: >>>>>>> cc1: warnings being treated as errors >>>>>>> ../../../MonetDB/src/gdk/gdk_system.mx: In function >>>>>>> 'MT_check_nr_cores': >>>>>>> ../../../MonetDB/src/gdk/gdk_system.mx:575: warning: ISO C90 >>>>>>> forbids mixed declarations and code >>>>>>> ../../../MonetDB/src/gdk/gdk_system.mx:576: error: implicit >>>>>>> declaration of function 'sysctlbyname' >>>>> >>>>> No. The MonetDB sources aren't tested on Leopard on a daily >>>>> basis, and >>>>> this change/code in particular isn't at all. I tried it on >>>>> Tiger, but >>>>> didn't care to check on Leopard. >>>> >>>> Actually, the actual code was never tested, as it's just missing an >>>> include ;) >> > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[MonetDB-users] PF: (new) compilation problem on Mac OS 10.5
by Ying Zhang 02 Sep '08

02 Sep '08

Hello, On Mac OS 10.5, source code just checked out from sourceforge. rebuild (debootstrap, boostrap, configure, make, make install) monet and its modules ends with some errors in pathfinder: Putting files in AC_CONFIG_AUX_DIR, `conf'. /opt/local/share/aclocal/audiofile.m4:12: warning: underquoted definition of AM_PATH_AUDIOFILE /opt/local/share/aclocal/audiofile.m4:12: run info '(automake)Extending aclocal' /opt/local/share/aclocal/audiofile.m4:12: or see http://sources.redhat.com/automake/automake.html#Extending-aclocal configure.in:85: warning: macro `AM_MONETDB_JAVA' not found in library patching file aclocal.m4 Hunk #1 succeeded at 428 (offset -5958 lines). configure.in:183: installing `conf/compile' configure.in:37: installing `conf/install-sh' configure.in:37: installing `conf/missing' configure.in:85: error: possibly undefined macro: AM_MONETDB_JAVA If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. This problem does not occur on my cwi desktop. Does anyone have an idea? Thanks! Jennie

2 3

Re: [MonetDB-users] [ADT] bug report?
by Stefan Manegold 01 Sep '08

01 Sep '08

On Mon, Sep 01, 2008 at 10:08:09PM +0200, Hendrike Peetz wrote: > Ok, I assume I should have set compiler options to --disable-strict. well, for us, it should also work with --enable-strict (default in CVS version as that is meant for developers --- and as a reminder to us to produce clean code ... ;-)) --- but for "users", --disable-strict (default in tarballs) is indeed just fine. > But now I can still not check, because it says: > > $ make check > make check-recursive > Making check in src > make check-recursive > Making check in common > make check-am > make[5]: Nothing to be done for `check-am'. > Making check in gdk > make check-am > make[5]: Nothing to be done for `check-am'. > Making check in testing > make check-am > rm -f Mapprove.py > ln -s Mtest.py Mapprove.py > make[4]: Nothing to be done for `check-am'. > Making check in conf > make check-am > make[3]: Nothing to be done for `check-am'. for which part did you call `make check`? for "MonetDB" (Common) there are indeed not tests; neither for (MonetDB/)"java"; only for clients, MonetDB4, MonetDB5, sql, geom, pathfinder, template there are tests and `make check` should work --- be aware though, that some tests are known to fail on some platforms ... > Sorry for bothering you, but we were two people (who do know the basics > of working with unix) trying to get this running today and couldn't make > it work. No problem --- in fact, `make check` is not "mendatory" for getting it to work --- just `make && make install` (or use monetdb-install.sh) and then try to run `Mserver` (MonetDB4) or `mserver5` (MonetDB5) as a very basic "sanitycheck". Stefan > Hendrike > > > Begin forwarded message: > >> From: Hendrike Peetz <hendrike(a)peetz-intelligence.com> >> Date: September 1, 2008 9:27:37 PM GMT+02:00 >> To: Monet(a)cwi.nl >> Subject: [ADT] bug report? >> >> Dear people in charge, >> >> I have been spending a lot of time today trying to install Monet on >> Leopard 10.5.4, 64 bit, core 2 duo. >> >> Before I did anything with the cvs, I tried to work with the Super >> tarball version. Compiling was not a problem but the tests were not >> installed and I was not too sure wether everything is working. I thus >> wanted to install it properly: >> >> I checked out the latest version (friday I assume) from the cvs. >> bootstrap and configure worked fine, but I had problems with the >> compiling. >> >> I tried gcc4.1 (apple) and gcc4.2 (apple) (Yes, I recompiled my c- >> compiler!). Both of them returned the attached error. >> I am not sure wether this is my mistake or some weird code, but as I >> was able to compile the tarball version I assume its something in the >> code. >> >> Best regards, >> >> Hendrike >> >> >> >> >> >> > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [MonetDB-users] [ADT] bug report?
by Stefan Manegold 01 Sep '08

01 Sep '08

Dear Hendrike, thank you very much for your detailed report. I suppose you sucessfully compiled the latest released "Stable" "Aug2008" version of the Super tarball, right? Then, indeed the compilation problems seem to be triggered by the (our) code in the latest "cutting-edge" "Current" development version from the CVS heads. Although we do quite some testing (cf. http://monetdb.cwi.nl/Development/TestWeb/) we do not (yet?) have a "Leopard" machine in our testing pool (we actually also lost access to the "Tiger" machine we used to have...). We will look into the problem ASAP and try to fix it. Could you please also file your problem as bug report in the MonetDB bug tracker at SF: http://sourceforge.net/tracker/?atid=482468&group_id=56967&func=browse ? For the time being you might want to try the latest "Stable" version from the CVS branches as listed at http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Version4.24/ind… . What puzzles me, though, is your initial point that no tests were included in the tests were not included in the Super tarball. They should actaully be in there. I'll check again ... Stefan On Mon, Sep 01, 2008 at 09:27:37PM +0200, Hendrike Peetz wrote: > Dear people in charge, > > I have been spending a lot of time today trying to install Monet on > Leopard 10.5.4, 64 bit, core 2 duo. > > Before I did anything with the cvs, I tried to work with the Super > tarball version. Compiling was not a problem but the tests were not > installed and I was not too sure wether everything is working. I thus > wanted to install it properly: > > I checked out the latest version (friday I assume) from the cvs. > bootstrap and configure worked fine, but I had problems with the > compiling. > > I tried gcc4.1 (apple) and gcc4.2 (apple) (Yes, I recompiled my c- > compiler!). Both of them returned the attached error. > I am not sure wether this is my mistake or some weird code, but as I was > able to compile the tarball version I assume its something in the code. > > Best regards, > > Hendrike > > > $ make > make all-recursive > Making all in src > make all-recursive > Making all in common > make all-am > make[5]: Nothing to be done for `all-am'. > Making all in gdk > make all-am > /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../.. -I../../../MonetDB/src/gdk -I../../../MonetDB/src/gdk -I../common -I../../../MonetDB/src/gdk/../common -DLIBGDK -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -Wno-uninitialized -D_REENTRANT -c -o libbat_la-gdk_system.lo `test -f 'gdk_system.c' || echo '../../../MonetDB/src/gdk/'`gdk_system.c > gcc -DHAVE_CONFIG_H -I. -I../.. -I../../../MonetDB/src/gdk -I../../../MonetDB/src/gdk -I../common -I../../../MonetDB/src/gdk/../common -DLIBGDK -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -Wno-uninitialized -D_REENTRANT -c gdk_system.c -fno-common -DPIC -o .libs/libbat_la-gdk_system.o > cc1: warnings being treated as errors > ../../../MonetDB/src/gdk/gdk_system.mx: In function 'MT_check_nr_cores': > ../../../MonetDB/src/gdk/gdk_system.mx:575: warning: ISO C90 forbids mixed declarations and code > ../../../MonetDB/src/gdk/gdk_system.mx:576: error: implicit declaration of function 'sysctlbyname' > make[5]: *** [libbat_la-gdk_system.lo] Error 1 > make[4]: *** [all] Error 2 > make[3]: *** [all-recursive] Error 1 > make[2]: *** [all] Error 2 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 > > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 4

[MonetDB-users] batcalc.ifThenElse:requires bats of identical size
by Rick Spickelmier 28 Aug '08

28 Aug '08

Hi! I am running into this error: batcalc.ifThenElse:requires bats of identical size My server (running on Windows XP 64): # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'demo' # Compiled for i686-pc-win32/64bit with 32bit OIDs dynamically linked The following recreates the problem: CREATE TABLE test1 (col1 INTEGER); INSERT INTO test1 VALUES (null); INSERT INTO test1 VALUES (null); SELECT CASE WHEN col1=1 THEN 'ONE' END AS "pc1" FROM test1 GROUP BY "pc1"; Any ideas? Obviously the above isn't too interesting, but it a simplified test case from a more complex query on a much larger dataset. Seems to be related to GROUP BY on a CASE... Thanks! Rick +----------------------------------------------------------------------- ------+ | mal | +======================================================================= ======+ | function user.s0_3():void; | | _1:bat[:oid,:int]{rows=1:lng} := sql.bind("voc","test2","col1",0); | | _6:bat[:oid,:int]{rows=1:lng} := sql.bind("voc","test2","col1",1); | | _8{rows=2:lng} := algebra.kunion(_1,_6); | | _1:bat[:oid,:int]{rows=1:lng} := nil; | | _6:bat[:oid,:int]{rows=1:lng} := nil; | | _9:bat[:oid,:int]{rows=1:lng} := sql.bind("voc","test2","col1",2); | | _11{rows=2:lng} := algebra.kdifference(_8,_9); | | _8{rows=2:lng} := nil; | | _12{rows=3:lng} := algebra.kunion(_11,_9); | | _11{rows=2:lng} := nil; | | _9:bat[:oid,:int]{rows=1:lng} := nil; | | _13:bat[:oid,:oid]{rows=1:lng} := sql.bind_dbat("voc","test2",1); | | _14{rows=1:lng} := bat.reverse(_13); | | _13:bat[:oid,:oid]{rows=1:lng} := nil; | | _15{rows=3:lng} := algebra.kdifference(_12,_14); | | _14{rows=1:lng} := nil; | | _17{rows=3:lng,tlb>=0@0:oid,tub<1@0:oid} := algebra.markT(_15,0@0); | | _15{rows=3:lng} := nil; | | _18{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := bat.reverse(_17); | | _17{rows=3:lng,tlb>=0@0:oid,tub<1@0:oid} := nil; | | _19{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := algebra.join(_18,_12); | | _12{rows=3:lng} := nil; | | _20:bat[:oid,:bit]{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := | : batcalc.==(_19,1); | | _23{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := algebra.project(_18,nil:str); | | _24:bat[:oid,:str]{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := | : batcalc.ifthenelse(_20,"ONE",_23); | | _20:bat[:oid,:bit]{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := nil; | | _23{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := nil; | | (ext36,grp34):= group.new(_24); | | _24:bat[:oid,:str]{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := nil; | | _27 := bat.mirror(ext36); | | ext36 := nil; | | _28 := algebra.join(_27,_19); | | _27 := nil; | | _19{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := nil; | | _29:bat[:oid,:bit] := batcalc.==(_28,1); | | _28 := nil; | | _30{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := algebra.project(_18,nil:str); | | _18{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := nil; | | _31:bat[:oid,:str] := batcalc.ifthenelse(_29,"ONE",_30); | | _29:bat[:oid,:bit] := nil; | | _30{rows=3:lng,hlb>=0@0:oid,hub<1@0:oid} := nil; | | _32 := sql.resultSet(1,1,_31); | | sql.rsColumn(_32,"voc.","pc1","char",0,0,_31); | | _31:bat[:oid,:str] := nil; | | _37 := io.stdout(); | | sql.exportResult(_37,_32,""); | | end s0_3; | +----------------------------------------------------------------------- ------+ Rick Spickelmier | CTO | Success Metrics, Inc. | w (415) 738-2078 | ricks(a)successmetricsinc.com

3 3

[MonetDB-users] upgrading ...
by ewilde 24 Aug '08

24 Aug '08

hi, i have updated my monet installtion from "june 2008" to "current" on Aug 19, using YAM on fedora 9. i tried to run monet-server: mserver5 --dbinit="include sql;" --set gdk_vm_minsize=1000000000000 and i have got: # MonetDB server v5.7.0, based on kernel v1.25.0 # Serving database 'demo', using 2 threads # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ #WARNING: LoaderException:loadLibrary:sql: loader error lib_logger.so.0: cannot open shared object file: No such file or directory !MALException:MAL.getAddress:address of 'sql.SQLsession' not found !MALException:MAL.getAddress:address of 'sql.SQLsession2' not found !MALException:MAL.getAddress:address of 'sql.SQLinitEnvironment' not found !MALException:MAL.getAddress:address of 'sql.SQLcommit' not found !MALException:MAL.getAddress:address of 'sql.SQLrollback' not found !MALException:MAL.getAddress:address of 'sql.SQLstatement' not found !MALException:MAL.getAddress:address of 'sql.SQLstatementOut' not found !MALException:MAL.getAddress:address of 'sql.SQLassert' not found !MALException:MAL.getAddress:address of 'sql.SQLassertInt' not found !MALException:MAL.getAddress:address of 'sql.SQLassertLng' not found !MALException:MAL.getAddress:address of 'sql.setVariable' not found !MALException:MAL.getAddress:address of 'sql.getVariable' not found !MALException:MAL.getAddress:address of 'sql.mvc_logfile' not found !MALException:MAL.getAddress:address of 'sql.mvc_next_value' not found !MALException:MAL.getAddress:address of 'batsql.mvc_bat_next_value' not found !MALException:MAL.getAddress:address of 'sql.mvc_get_value' not found !MALException:MAL.getAddress:address of 'sql.mvc_restart_seq' not found !MALException:MAL.getAddress:address of 'sql.mvc_bind_dbat_wrap' not found !MALException:MAL.getAddress:address of 'sql.mvc_bind_idxbat_wrap' not found !MALException:MAL.getAddress:address of 'sql.mvc_bind_wrap' not found . . .[lines were removed because the message would not been sent otherwise] . . !MALException:MAL.getAddress:address of 'mat.mat_bind_idxbat_wrap' not found !MALException:MAL.getAddress:address of 'mat.inc_parts_wrap' not found !MALException:MAL.getAddress:address of 'mat.send_part_wrap' not found !MALException:MAL.getAddress:address of 'mat.get_part_wrap' not found !MALException:MAL.getAddress:address of 'mat.print_part_wrap' not found !MALException:MAL.getAddress:address of 'constraints.Cparts' not found !TypeException:user.main[1]:'sql.prelude' undefined in: _1:void := sql.prelude() > this file lib_logger.so.0 (which is actually a link to lib_logger.so.0.0.0) exists in: /usr/lib64/MonetDB5/lib thanks in advanced for any help, ewilde.

3 2

[MonetDB-users] test 2
by ewilde 24 Aug '08

24 Aug '08

1 0

[MonetDB-users] C# odbc connection
by Aviad Harell 21 Aug '08

21 Aug '08

Hi there, I'm playing here with MonetDB. I've downloaded the monetDb sql DB and the odbc driver. With the console I succeeded to connect and query the voc sample db, which is great. I also succeeded to import data to excel from the MonetDb via odbc as shown on the website. Now, I want to connect to the monetDB via C# and odbc driver. How can I do that? Thanks, Aviad

2 1

[MonetDB-users] C# odbc connection
by Aviad Harell 21 Aug '08

21 Aug '08

Hi there, I'm playing here with MonetDB. I've downloaded the monetDb sql DB and the odbc driver. With the console I succeeded to connect and query the voc sample db, which is great. I also succeeded to import data to excel from the MonetDb via odbc as shown on the website. Now, I want to connect to the monetDB via C# and odbc driver. How can I do that? Thanks, Aviad

1 0

[MonetDB-users] C# odbc connection
by Aviad 20 Aug '08

20 Aug '08

Hi there, I'm playing here with MonetDB. I’ve downloaded the monetDb sql DB and the odbc driver. With the console I succeeded to connect and query the voc sample db, which is great. I also succeeded to import data to excel from the MonetDb via odbc as shown on the website. Now, I want to connect to the monetDB via C# and odbc driver. How can I do that? Thanks, Aviad -- View this message in context: http://www.nabble.com/C--odbc-connection-tp19073114p19073114.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

Re: [MonetDB-users] Order of WHERE clause has significant performance impact
by Ross Bates 15 Aug '08

15 Aug '08

On Tue, Aug 12, 2008 at 08:31:22PM -0500, Ross Bates wrote: > Hi All - I have 2 queries which run very different based on how I > order the WHERE clause. > > When I put the larger filter first, it runs for 3.5 seconds. > > Query 1: > > select c5, c6, sum(sale) from sobi.sample > where > c5 in (5253,5110,5375,5268,5257,5266,5243,5256,5252,5361,5379,5363,5386,5367,5240,5241,5261,5249,5245,5260,5247,5264,5354,5250,5373,5366,5246,5356,5365,5255,5262,5259,5251,5369,5357,5269,5364,5380,5244,5248,5242,5254,5358,5360,5263,5362,5374,5378,5370,5294,5339,5272,5277,5275,5295,5340,5273,5276,5270,5061,5081,5079,5080,5066,5056,5073,5060,5070,5099,5101,5100,5067,5063,5077,5072,5062,5075,5083,5086,5071,5095,5058,5087,5090,5102,5084,5098,5076,5059,5074,5094,5091,5065,5069,5068,5104,5109,5107,5108,5013,5019,5022,5010,5009,5023,5011,5024,5012,5021,5007,5008,5025,5020,5055,5006,5036,5038,5054,5032,5033,5031,5039,5035,5026,5027,5030,5200,5043,5005,5041,5049,5042,5052,5040,5053,5045,5047,5048,5050,5051,5044,5046) > and c6 in (5182) > group by c5, c6; > > > When I reverse the order of the WHERE clause, it runs in about .4 seconds. > > > Query 2: > > c5, c6, sum(sale) from sobi.sample > where > c6 in (5182) > and > c5 in (5253,5110,5375,5268,5257,5266,5243,5256,5252,5361,5379,5363,5386,5367,5240,5241,5261,5249,5245,5260,5247,5264,5354,5250,5373,5366,5246,5356,5365,5255,5262,5259,5251,5369,5357,5269,5364,5380,5244,5248,5242,5254,5358,5360,5263,5362,5374,5378,5370,5294,5339,5272,5277,5275,5295,5340,5273,5276,5270,5061,5081,5079,5080,5066,5056,5073,5060,5070,5099,5101,5100,5067,5063,5077,5072,5062,5075,5083,5086,5071,5095,5058,5087,5090,5102,5084,5098,5076,5059,5074,5094,5091,5065,5069,5068,5104,5109,5107,5108,5013,5019,5022,5010,5009,5023,5011,5024,5012,5021,5007,5008,5025,5020,5055,5006,5036,5038,5054,5032,5033,5031,5039,5035,5026,5027,5030,5200,5043,5005,5041,5049,5042,5052,5040,5053,5045,5047,5048,5050,5051,5044,5046) > group by c5, c6; > > This in on MonetDB server v5.6.0, based on kernel v1.24.0 which I > built from the super tarball. > > I can attach the entire trace files for each query if appropriate for the list. > > While I am still learning to read the trace, it looks like most of the > time spent in the first query on a series of algebra.semijoin > operations. Here are the excerpted lines: > > Query 1: > > [2320345 usec # _461 := > algebra.semijoin(_167=<~tmp_2116>bat[:str,:oid]{8280020}, > _460=<~tmp_2016>bat[:str,:oid]{142}) ] > [776017 usec # _463 := > algebra.semijoin(_158=<tmp_2130>bat[:oid,:str]{8280020}, > _462=<~tmp_1572>bat[:oid,:str]{5135708}) ] > [253551 usec # _470 := > algebra.semijoin(_464=<~tmp_2116>bat[:str,:oid]{5135708}, > _469=<~tmp_2016>bat[:str,:oid]{1}) ] > > > Query 2: > > [412122 usec # _179 := > algebra.semijoin(_167=<~tmp_2076>bat[:str,:oid]{8280020}, > _178=<~tmp_2016>bat[:str,:oid]{1}) ] > [ 194 usec # _181 := > algebra.semijoin(_158=<tmp_2070>bat[:oid,:str]{8280020}, > _180=<~tmp_1572>bat[:oid,:str]{304}) ] > [ 0 usec # _470 := > algebra.semijoin(_182=<~tmp_2076>bat[:str,:oid]{304}, > _469=<~tmp_2016>bat[:str,:oid]{142}) ] > > > Any assistance would be greatly appreciated. Indeed the optimizer isn't smart enough to use the correct (most selective) in first. Could you send the create statement for the sobi.sample table? If possible could you try this on the 'current' and with mclient -lsql -G. That last -G enables the new implementation which first translates sql into a relational algebra and then in binary algebra. Niels > > -Ross Niels - thanks for the response. I tried using the -G switch and the performance time was the same for both versions of the query. The table is pretty simple (see below). I've tried the query against the table using both int and varchar for the columns being used in the WHERE clause. When c6 and C7 are declared as ints the queries run faster, but the order of the WHERE clause still creates a difference. 3.3 vs .4 when varchar 1.9 vs .2 when int CREATE TABLE "sobi"."sample" ( c1 char(1), c2 varchar(24), c3 varchar(12), c4 varchar(12), c5 varchar(12), c6 int, c7 int, c8 varchar(12), c9 varchar(12), sale int ) CREATE TABLE "sobi"."sample" ( c1 char(1), c2 varchar(24), c3 varchar(12), c4 varchar(12), c5 varchar(12), c6 varchar(12), c7 varchar(12), c8 varchar(12), c9 varchar(12), sale int ) Would it introduce any issues or risks to have the optimizer reorder the execution of each WHERE clause based on the number of predicates specified? I am still learning about the optimizer, but would changing any steps in the default SQL optimization sequence help in this situation? Thanks again, Ross

4 9

[MonetDB-users] Newbie question about XML / SQL capabilities
by Bill Walton 14 Aug '08

14 Aug '08

Greetings all, I just became aware of MonetDB via a post to the Ruby on Rails list about a new connection adapter for MonetDB (thanks Michalis). I'm intrigued by what looks like a very impressive capability. If I'm reading the site right, MonetDB can import an XML file using the XQuery client and then query that same data using the SQL client. Is that a correct reading? TIA, Bill

2 1

[MonetDB-users] Order of WHERE clause has significant performance impact
by Ross Bates 13 Aug '08

13 Aug '08

Hi All - I have 2 queries which run very different based on how I order the WHERE clause. When I put the larger filter first, it runs for 3.5 seconds. Query 1: select c5, c6, sum(sale) from sobi.sample where c5 in (5253,5110,5375,5268,5257,5266,5243,5256,5252,5361,5379,5363,5386,5367,5240,5241,5261,5249,5245,5260,5247,5264,5354,5250,5373,5366,5246,5356,5365,5255,5262,5259,5251,5369,5357,5269,5364,5380,5244,5248,5242,5254,5358,5360,5263,5362,5374,5378,5370,5294,5339,5272,5277,5275,5295,5340,5273,5276,5270,5061,5081,5079,5080,5066,5056,5073,5060,5070,5099,5101,5100,5067,5063,5077,5072,5062,5075,5083,5086,5071,5095,5058,5087,5090,5102,5084,5098,5076,5059,5074,5094,5091,5065,5069,5068,5104,5109,5107,5108,5013,5019,5022,5010,5009,5023,5011,5024,5012,5021,5007,5008,5025,5020,5055,5006,5036,5038,5054,5032,5033,5031,5039,5035,5026,5027,5030,5200,5043,5005,5041,5049,5042,5052,5040,5053,5045,5047,5048,5050,5051,5044,5046) and c6 in (5182) group by c5, c6; When I reverse the order of the WHERE clause, it runs in about .4 seconds. Query 2: c5, c6, sum(sale) from sobi.sample where c6 in (5182) and c5 in (5253,5110,5375,5268,5257,5266,5243,5256,5252,5361,5379,5363,5386,5367,5240,5241,5261,5249,5245,5260,5247,5264,5354,5250,5373,5366,5246,5356,5365,5255,5262,5259,5251,5369,5357,5269,5364,5380,5244,5248,5242,5254,5358,5360,5263,5362,5374,5378,5370,5294,5339,5272,5277,5275,5295,5340,5273,5276,5270,5061,5081,5079,5080,5066,5056,5073,5060,5070,5099,5101,5100,5067,5063,5077,5072,5062,5075,5083,5086,5071,5095,5058,5087,5090,5102,5084,5098,5076,5059,5074,5094,5091,5065,5069,5068,5104,5109,5107,5108,5013,5019,5022,5010,5009,5023,5011,5024,5012,5021,5007,5008,5025,5020,5055,5006,5036,5038,5054,5032,5033,5031,5039,5035,5026,5027,5030,5200,5043,5005,5041,5049,5042,5052,5040,5053,5045,5047,5048,5050,5051,5044,5046) group by c5, c6; This in on MonetDB server v5.6.0, based on kernel v1.24.0 which I built from the super tarball. I can attach the entire trace files for each query if appropriate for the list. While I am still learning to read the trace, it looks like most of the time spent in the first query on a series of algebra.semijoin operations. Here are the excerpted lines: Query 1: [2320345 usec # _461 := algebra.semijoin(_167=<~tmp_2116>bat[:str,:oid]{8280020}, _460=<~tmp_2016>bat[:str,:oid]{142}) ] [776017 usec # _463 := algebra.semijoin(_158=<tmp_2130>bat[:oid,:str]{8280020}, _462=<~tmp_1572>bat[:oid,:str]{5135708}) ] [253551 usec # _470 := algebra.semijoin(_464=<~tmp_2116>bat[:str,:oid]{5135708}, _469=<~tmp_2016>bat[:str,:oid]{1}) ] Query 2: [412122 usec # _179 := algebra.semijoin(_167=<~tmp_2076>bat[:str,:oid]{8280020}, _178=<~tmp_2016>bat[:str,:oid]{1}) ] [ 194 usec # _181 := algebra.semijoin(_158=<tmp_2070>bat[:oid,:str]{8280020}, _180=<~tmp_1572>bat[:oid,:str]{304}) ] [ 0 usec # _470 := algebra.semijoin(_182=<~tmp_2076>bat[:str,:oid]{304}, _469=<~tmp_2016>bat[:str,:oid]{142}) ] Any assistance would be greatly appreciated. -Ross

2 1

[MonetDB-users] Announcement: new minor release of MonetDB Clients package
by Sjoerd Mullender 12 Aug '08

12 Aug '08

The MonetDB team at CWI is pleased to announve a bug fix release for the MonetDB Clients package. A number of bugs in the ODBC driver have been fixed: - fixes for the handling of UTF-8 encoded queries and query parameters; - fixes for dealing with wide characters. There is also a fix for mclient: - the timer (-t/--time option) could overflow. There are new Windows installers for the ODBC driver, and there are new RPMs for Fedora 8 and 9. All of this for both 32 and 64 bit architectures. There is also a new Super Source Ball. -- Sjoerd Mullender

1 0

Re: [MonetDB-users] MonetDB rpm issues
by Stefan Manegold 11 Aug '08

11 Aug '08

On Mon, Aug 11, 2008 at 10:52:34AM +0100, John Baxter wrote: > Stefan, > > Sorry about the html format, it is just the default, i've changed it to > plain text now. Thanks! > cenos == centos, my mistake. > > CentOS is basically the same as Red Hat Enterprise server but rebranded. > I feel it is more stable than fedora core and as it is also open source > it is just as usable. > > I'm sorry, i didn't realise i would have to compile it first, i'll do > that today, thank you for the tips about what order to do it in. > > One last thing, i'm sure i read somewhere that the sql implementation is > the same as postgresql. Is this still correct, and is the implementation > exactly the same or are there some exceptions? Well, we do not follow or copy any other SQL implementation (in particular not their modification and/or extensions of the standard), but rather (try to) stick to teh standard. For details see http://monetdb.cwi.nl/SQL/Documentation/index.html and in particular http://monetdb.cwi.nl/SQL/Documentation/SQL-Reference.html Stefan > Thanks again for your assistance. > > Kind Regards > > John > > > Stefan Manegold wrote: >> Hi John, >> >> On Fri, Aug 08, 2008 at 04:26:09PM +0100, John Baxter wrote: >> >>> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> >>> <html> >>> <head> >>> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> >>> </head> >>> <body bgcolor="#ffffff" text="#000000"> >>> Stefan, >>> >>> Thanks very much for replying, i really appreciate it. >>> >>> These are the RPM's i have tried: >>> >>> MonetDB5-server-5.6.0-4.src.rpm >>> MonetDB-SQL-2.24.0-4.src.rpm >>> >>> I'm only installing this on a local centos install. I downloaded the >>> RPM's to the root of the filesystem and ran them from there. >>> I'm running cenos from it's GUI at the moment so it would be the GUI >>> >> ^^^^^ >> Typo (i.e., "cenos" == "centos") or is that the application/tools you use to >> install RPMs? >> I don't know CentOS and/or it's adminstrative tools at all. Sorry. >> >> However, the "platform independent" RPMs are *source* RPMs that cannot be >> installed directly using, e.g., `rpm -i`. Instead, they have to be compiled >> (into platform-specific binary RPMs, first, e.g., via `rpmbuild --rebuild` >> or `rpmbuild --recompile`. >> >> See your systems RPM documentation (e.g., `man rpm` / `man rpmbuild` and/or >> http://www.rpm.org/ for a detailed documentation of RPM. >> >> >> Before compiling/building and installing >> MonetDB5-server-5.6.0-4.src.rpm >> and MonetDB-SQL-2.24.0-4.src.rpm >> >> you first need to compile/build and install >> MonetDB-1.24.0-4.src.rpm >> and MonetDB-client-1.24.0-4.src.rpm >> >> >> Stefan >> >> >> ps: We would greatly apprechiate if you would not sedn HTML(-only) >> emails/postings --- plain text is more than sufficient (and much easier >> to handle) ... >> >> >> >>> that gave me the error messages. >>> >>> Thanks >>> >>> John >>> >>> Stefan Manegold wrote: >>> <blockquote cite="mid:20080808151929.GA10101@cwi.nl" type="cite"> >>> <pre wrap="">Hi John, >>> >>> which "platform independent RPM's" are you refering to? >>> >>> On our download page on SF we only provide Fedora 8 & 9 (binary) RPMs as >>> well as source-RPMs --- only the latter are platform independent, except >>> that the target OS need to obviously provide RPM support in general. >>> >>> How exactly do you try to install the (source?) RPMs? >>> >>> And who (i.e., which program) give the error messages? >>> >>> Stefan >>> >>> On Fri, Aug 08, 2008 at 03:49:45PM +0100, John Baxter wrote: >>> </pre> >>> <blockquote type="cite"> >>> <pre wrap="">Stefan, >>> >>> I hope you don't mind me e-mailing you. You were kind enough to help >>> me once before when i was having trouble installing MonetDB and i'm >>> having trouble again. >>> >>> I'm trying to install the platform independent RPM's on centos 5.2 >>> but it is saying that it "cannot install source packages" and then >>> "no packages were given for installation" >>> >>> Are the RPM's usable at the moment? >>> >>> If you can offer any advice that would be great. >>> >>> Thank you >>> >>> Kind Regards >>> >>> John Baxter >>> </pre> >>> </blockquote> >>> <pre wrap=""> >>> </pre> >>> </blockquote> >>> >>> <pre class="moz-signature" cols="72">-- Co: Baxter Designs >>> >>> Web: <a class="moz-txt-link-freetext" href="http://www.baxterdesigns.co.uk">http://www.baxterdesigns.co.uk</a> >>> >>> Mail: <a class="moz-txt-link-abbreviated" href="mailto:john@baxterdesigns.co.uk">john(a)baxterdesigns.co.uk</a> >>> >>> Tel: 07968754215</pre> >>> </body> >>> </html> >>> >>> >> >> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >> >> >> > > -- > Co: Baxter Designs > > Web: http://www.baxterdesigns.co.uk > > Mail: john(a)baxterdesigns.co.uk > > Tel: 07968754215 -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 5

[MonetDB-users] (no subject)
by Matt Magoffin 11 Aug '08

11 Aug '08

Hello, I am interested in evaluating MonetDB/XQuery. I have some XML collections for a project I work on that are of 1-2 million individual XML documents for which I use XQuery to access for OLAP style reporting operations. This is a Java-based project, and while working on this project I developed a thin XQuery-centric API so that I could evaluate/use many different XQuery implementations, which my company released as an open source project called Xaj (http://sourceforge.net/projects/xaj) Currently I am using a Berkeley DB XML implementation, but am interested in exploring other options. I'd like to implement a MonetDB version of this API now, but I am having some difficulty finding out how to effectively access MonetDB through Java. So far I was able to use the MonetDB JDBC driver to construct XQuery statements and get XML results, and also I was able to construct MonetDB pf:add-doc() statements to add documents, but I am wondering if this is the best approach (and if so, how to add documents to a remote MonetDB server using the pf:add-doc() mechanism). The API I am trying to implement you can view here, to give you an idea what I'm trying to do: http://xaj.svn.sourceforge.net/viewvc/xaj/xaj/src/net/sf/xaj/XmlDb.java?rev… It was somewhat modeled after the defunct XML:DB API, but focused just on add/store/XQuery operations. Any help/advise would be greatly appreciated! -- m@

2 3

Re: [MonetDB-users] MonetDB rpm issues
by Stefan Manegold 11 Aug '08

11 Aug '08

Hi John, which "platform independent RPM's" are you refering to? On our download page on SF we only provide Fedora 8 & 9 (binary) RPMs as well as source-RPMs --- only the latter are platform independent, except that the target OS need to obviously provide RPM support in general. How exactly do you try to install the (source?) RPMs? And who (i.e., which program) give the error messages? Stefan On Fri, Aug 08, 2008 at 03:49:45PM +0100, John Baxter wrote: > Stefan, > > I hope you don't mind me e-mailing you. You were kind enough to help me > once before when i was having trouble installing MonetDB and i'm having > trouble again. > > I'm trying to install the platform independent RPM's on centos 5.2 but > it is saying that it "cannot install source packages" and then "no > packages were given for installation" > > Are the RPM's usable at the moment? > > If you can offer any advice that would be great. > > Thank you > > Kind Regards > > John Baxter -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[MonetDB-users] Announcing MonetDB support for Ruby on Rails
by Michalis Polakis 10 Aug '08

10 Aug '08

Hello folks, I developed the MonetDB adapter for Rail's ActiveRecord. It will enable you to work with MonetDB databases in Ruby on Rails. You are more than welcome to try the code, give me some feedback, and maybe help me maintain it. To get the code visit : http://rubyforge.org/projects/monetdb-ror Thanks, Michalis Polakis

1 0

[MonetDB-users] MonetDB Indexes and Data Types
by Ross Bates 09 Aug '08

09 Aug '08

Hi All - I'm new to MonetDB and just getting my feet wet. Quick question for the experts. Does the data type of the column impact the performance of the indexes created on the fly by MonetDB? In other words, should I expect WHERE clauses on INT columns to perform faster than VARCHAR like I would in a tradtional row store? Many Thanks, Ross

2 1

[MonetDB-users] full text search?
by mark 09 Aug '08

09 Aug '08

hi, i am beginning with monetdb. is there any support for full text search on monet-db via plugin or natively? thanks a lot

2 1

[MonetDB-users] COPY INTO on Ubuntu 64bit is slow
by Cronje Fourie 08 Aug '08

08 Aug '08

Hi guys.... I've got to machines running Monet. 1. Quad core Intel with 4GB RAM, Ubuntu 8.04 32BIT (server ed) 2. AMD64 Quad Core, 4GB RAM, KUbuntu 8.04 64BIT (kde 3.5.9) On system 1 I can do a "copy into" of 6million records in no time at all :) On system 2 the same function, with the exact same data takes forever. Any thoughts on the reason? Is this a memory related issue? Cronje

7 12

[MonetDB-users] [XQuery] Communicate with Java?
by Chaojing Yang 08 Aug '08

08 Aug '08

Dear all, I'm a totally newbie and try to implements an LDAP-application over MonetDB/XQuery due to efficient search performance of column-store. Like the example XQueryLoad.java does i have succeed to import a XML-file and now come my questions: * How can i manipulate my XML-files with JAVA? Like doing searches and changes over XML-files. * How is the performance of Subtree-search? Like doing searchs over the whole tree or any subtrees. As once discussed in http://comments.gmane.org/gmane.comp.db.monetdb.user/1027, i'm in the same situation. Best regards, Chaojing Yang -- ------------------------------------------------------------------------------------------------------------------------------------------ Email: chaojing.yang(a)gmail.com Blog: http://goethe-ist-taoist.blogspot.com/ §1 Ich widerspreche der Nutzung oder Übermittlung meiner Daten für Werbezwecke oder sowie andere unbefugte Verwendungszwecke. (§ 28 Abs. 4 BDSG=Bundesdatenschutzgesetz). §2 Durch die Datenübertragung bzw. Kommunikation per E-Mail liegt ein Verstoß gegen das BDSG vor, da ihre Daten insoweit nicht ausreichend geschützt sind. Ich bitte Sie daher, mir schriftlich zu bestätigen, dass Sie mit dem Verstoß gegen das BDSG einverstanden sind und keine Schadensersatzanspräche bzw. Unterlassungsanspräche mir gegenüber geltend machen. §3 Der Inhalt dieser Nachricht ist vertraulich und nur für den oben angegebenen Empfänger bestimmt.

5 8

[MonetDB-users] Does MonetDB's JDBC Driver support CachedRowSet??
by sateesh 07 Aug '08

07 Aug '08

Hi All, I am wondering what JDBC Spec does the MonetDB's JDBC Driver conform to ? (is it JDBC3.0 ,4.0) We would like to use the pagination at the Database level and hence i want to know whether the JDBC Driver has any implementation of CachedRowSet.. Thanks Sateesh -- View this message in context: http://www.nabble.com/Does-MonetDB%27s--JDBC-Driver-support-CachedRowSet---… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] protocol violation: unexpected line in data block
by toshiki.shiga＠alsi.co.jp 07 Aug '08

07 Aug '08

Hi, All Please explain this message? "protocol violation: unexpected line in data block: #02 #GDKmemchk (memcur=16870976,memmax=1741950382) (vmcur=1047527424,vmmax=1610612736)" Regards, Shiga

3 5

[MonetDB-users] Update statement to update from other table
by Cronje Fourie 05 Aug '08

05 Aug '08

Hi all... I'm testing MonetDB and evaluating all functionality. One item I've not been able to find much info on is updating table A from table B. ie... UPDATE PERSON SET SALARY = sal.newval FROM PERSON P, SAL_DATA sal WHERE p.id = sal.p_id Or something to that effect :) Is it possible? Thanks Cronje

2 1

[MonetDB-users] Is there a recommended OS for MonetDB?
by McKennirey.Matthew 04 Aug '08

04 Aug '08

2 1

[MonetDB-users] monetdb copy command
by ashokguruswamy 01 Aug '08

01 Aug '08

my seasonal compliments.. this is ashok. i am using monetdb for my application with java. just i need to copy the monetdb table to a csv file.. in an sql manual i read the syntax for copying from table to csvfile is Syntax: COPY <subquery> INTO <file_name> [ [USING] DELIMITERS tuple_separator [’,’ record_separator [ ’,’ string_quote ]]] [ NULL AS null_string ]; Operating System: Windows XP My query: copy select * from table_name into 'D:/file_name.csv' using delimiters '|'; but if i execute this, am facing the following problem.. any suggestion? asap... Error: Syntax error, unexpected SELECT, expecting INTO in: “copy select”.. Also the query specified below which having --with null as 'null_String"-- is not working.. copy select * from table_name into 'D:/file_name.csv' using delimiters '|' with null as ' '; I want a way to copy monetdb tables to csv files.. -- View this message in context: http://www.nabble.com/monetdb-copy-command-tp18752294p18752294.html Sent from the monetdb-users mailing list archive at Nabble.com.

4 4

Re: [MonetDB-users] [Questions] Shredding 1GB XML File and Query Performance
by John 31 Jul '08

31 Jul '08

Thank you for your response! > Were you able to see any (error-) message in the server window? I could not see any message in the server window as it was directly closed. > On you 64-bit hardware, are you running 64-bit or 32-bit windows? > If 64-bit, did you install the 64-bit or 32-bit MonetDB/XQuery? I am using 32-bit windows. > The serialized size ("1 GB") does not say much about the actual complexity > of the XML document, e.g., how many nodes, attributes, etc. does the > document contain? > Do you know these statistics, or could you share your document with us for > testing? I tested using XBench DCSD. The total numbers of nodes and attributes were almost 22-23 millions and 1.5 millions, respectively. Thanks! John

1 0

[MonetDB-users] constant bulk inserts to monetdb
by uriel katz 30 Jul '08

30 Jul '08

i have a application where i have constant bulk inserts(about 50000 rows of 1Kbyte each) every 1 minute,i am wondering which is the fastest and most efficient way to insert data. i am using now the COPY command but it is kind of awkward since i need to dump my data as csv and then execute COPY to insert it,is there any streaming method or a special api for bulk inserts? also when i issue a copy command it uses a little bit of cpu aroud 2%(it is a quad core setup with windows so i guess this means 8% of one cpu) and keep loading and releasing memory(i have 8GB of ram) even that it doesn`t get near 2GB. is this ok,what it is actually doing? P.S.: are insert/selects multi threaded? thanks for this awsome peace of software! -Uriel

4 3

[MonetDB-users] constant bulk inserts to monetdb
by uriel katz 30 Jul '08

30 Jul '08

>>>> >>>> ok,but why it don`t utilize one core to the max,it seem to keep using 8% >>>> of >>>> the core and takes about 17 seconds to insert 200,000 record of 1Kb each >>>> (the actual file is 64MB). >>>> >>>> >>> >>> That depends on the total setup. Especially, when you are exceeding >>> memory then Linux >>> is not the best operating system. One of the issues is that dirty pages >>> are not properly flushed. >>> A BATsave in the new bulk loader at critical points helped to avoid this >>> case. >>> if you do a COPY into a clean bat, then the log overhead is neglectable. >>> >>>> >>>> can i some how skip the logs and make it insert right into the bat >>>> files? >>>> also how can i stream the tuples programaticlly from C mapi or ODBC? >>>> >>>> >>> >>> See the C Mapi library. Or you might look at the Stethoscope, which >>> streams tuples >>> from server to an application. >>> >> >> i am running MonetDB on windows with a 8GB of ram. >> > > That should be more then enough ; ) > >> what is Stethoscope? >> > > it is part of the source distribution and a Linux utility that picks up a > stream of tuples from the server. does it work on windows,and how it works,using a special mil/mal syntax or does it is a plugin in the server for fast loads? also how you guys load data into benchmark databases which are(according to papers i have seen) 1-100GB of data. > > The information you provide is hard to track down to a possible cause. > a COPY operation most likely reads the complete file into its buffers > before processing it. > > One pitfall you may have stumbled upon is the following. > Did you indicate the number of records that you are about to copy into the > table? > If not, then the system has to guess and will repeatedly adjust this guess, > which > involves quite some overhead. > Pleae use, COPY 50000 RECORDS INTO ..... even if i want to read all the file? >>> >>> Do you happen to be a Ruby-on-Rails expert? >>> >> >> i am not a ruby on rails expert,sorry i am python guy :) >> >>>> >>>> thanks again. >>>> >>>> Martin Kersten wrote: >>>> >>>> >>>>> >>>>> uriel katz wrote: >>>>> >>>>> >>>>>> >>>>>> i have a application where i have constant bulk inserts(about 50000 >>>>>> rows of 1Kbyte each) every 1 minute,i am wondering which is the fastest and >>>>>> most efficient way to insert data. >>>>>> >>>>>> >>>>> >>>>> COPY into is the fastest >>>>>

2 1

Re: [MonetDB-users] constant bulk inserts to monetdb
by Martin Kersten 30 Jul '08

30 Jul '08

uriel katz wrote: >>> ok,but why it don`t utilize one core to the max,it seem to keep using 8% of >>> the core and takes about 17 seconds to insert 200,000 record of 1Kb each >>> (the actual file is 64MB). >>> >>> >> That depends on the total setup. Especially, when you are exceeding memory then Linux >> is not the best operating system. One of the issues is that dirty pages are not properly flushed. >> A BATsave in the new bulk loader at critical points helped to avoid this case. >> if you do a COPY into a clean bat, then the log overhead is neglectable. >> >>> can i some how skip the logs and make it insert right into the bat files? >>> also how can i stream the tuples programaticlly from C mapi or ODBC? >>> >>> >> See the C Mapi library. Or you might look at the Stethoscope, which streams tuples >> from server to an application. >> > i am running MonetDB on windows with a 8GB of ram. > That should be more then enough ; ) > what is Stethoscope? > it is part of the source distribution and a Linux utility that picks up a stream of tuples from the server. The information you provide is hard to track down to a possible cause. a COPY operation most likely reads the complete file into its buffers before processing it. One pitfall you may have stumbled upon is the following. Did you indicate the number of records that you are about to copy into the table? If not, then the system has to guess and will repeatedly adjust this guess, which involves quite some overhead. Pleae use, COPY 50000 RECORDS INTO ..... >> Do you happen to be a Ruby-on-Rails expert? >> > i am not a ruby on rails expert,sorry i am python guy :) > >>> thanks again. >>> >>> Martin Kersten wrote: >>> >>> >>>> uriel katz wrote: >>>> >>>> >>>>> i have a application where i have constant bulk inserts(about 50000 rows of 1Kbyte each) every 1 minute,i am wondering which is the fastest and most efficient way to insert data. >>>>> >>>>> >>>> COPY into is the fastest >>>> >>>> >>>>> i am using now the COPY command but it is kind of awkward since i need to dump my data as csv and then execute COPY to insert it,is there any streaming method or a special api for bulk inserts? >>>>> >>>>> >>>>> >>>> I think it is possible to inline the tuples into the sql stream as well. The SQL developer will answer this one. >>>> >>>> >>>>> also when i issue a copy command it uses a little bit of cpu aroud 2%(it is a quad core setup with windows so i guess this means 8% of one cpu) and keep loading and releasing memory(i have 8GB of ram) even that it doesn`t get near 2GB. >>>>> is this ok,what it is actually doing? >>>>> >>>>> >>>> We have recently upgraded our bulk loader to utilize as many cores as possible >>>> This is not yet in the release. >>>> The preview of effects you can see in http://monetdb.cwi.nl/projects/monetdb//SQL/Benchmark/TPCH/ >>>> >>>> >>>>> P.S.: are insert/selects multi threaded? >>>>> >>>>> thanks for this awsome peace of software! >>>>> >>>>> -Uriel >>>>> >>>>> >>>

1 0

[MonetDB-users] [Questions] Shredding 1GB XML File and Query Performance
by John 30 Jul '08

30 Jul '08

I just installed MonetDB/XQuery 0.24.0 (released on 30 June 2008) and wanted to store some XML documents. When I tried to shred an XML document with almost 1GB filesize, I received the following error message. > MAPI = monetdb@localhost:50000 > ACTION= read_line > QUERY = pf:add-doc("C:/Data/TestData-10.xml","TestData-10.xml") > ERROR = Connection terminated After receiving this error on my mclient window, I noticed that the MonetDB XQuery Server and MClient were terminated. FYI, I used a Windows XP Pro SP3 PC with Intel Core2 Duo E6550 processor and 3.25GB of RAM. The size of my hard disk is 232 GB. May I know how I can shred this file? Another question is about to test the query performance. I tried to execute an XQuery using the following command. > mclient.bat -lxq -t -G XQuery-1.xq This command will print out the query result and return the "Timer". a. Can I hide the result? I tried to add "-f none" to the above command, but it did not work. b. What is the "Timer"? How can I get only the query time? Thanks John

3 3

[MonetDB-users] [Question]How to change the format of datetime?
by chenhao333333 28 Jul '08

28 Jul '08

Are there any functions like date_format in MonetDB, I want to do something like date_format(zncsrq, '%Y%m%d', that is Mysql's function, in MonetDB. Best regards!

2 1

[MonetDB-users] JESUS LOPEZ LOPEZ - F009939R está ausente de la oficina.
by jesus.lopez＠correo.aeat.es 28 Jul '08

28 Jul '08

Estaré ausente de la oficina desde el 28/07/2008 y no volveré hasta el 08/08/2008. Responderé a su mensaje cuando regrese.

1 0

[MonetDB-users] MonetDB/XQuery XQTS results available?
by Richard Eckart 28 Jul '08

28 Jul '08

Hello everybody, are there any results of the official XML Query Test Suite on MonetDB/ XQuery available or is there an easy way to generate them? Regards, Richard

1 0

[MonetDB-users] Is there Function likes TRUNCATE ?
by nori 25 Jul '08

25 Jul '08

Hi, All. Is there the function such as "TRUNCATE TABLE" in other DB? Regards, Nori. -- Nori tunohiko(a)gmail.com

4 4

[MonetDB-users] Cannot compile MonetDB_4-24
by Stefan Klinger 23 Jul '08

23 Jul '08

Thanks for your help, however I still cannot compile. Actually, I just need to work with most current versions of Pathfinder, and maybe try some other branches of that one as well, until I find out where the fixes end up, and which one works for me. Unortunately I was unable to find a detailed description of which branches of what modules are required to build which other branches of whatever modules I need anyway. If someone would be so kind as to show me how I can achieve this without touching CVS, I would be deeply grateful. Until then, I'll try to follow the CVS trail. On 17 July 2008, Stefan Manegold wrote with possible deletions: > In case you prefer to use the CVS sources, and you do not require the latest > cutting-edge (potentially instable) development version we recommend to > checkout the latest Stable branches. > I ust realize that the respective branch tags are not (yet?) listed on the > MonetDB web site, hence I'll share them here: > > buildtools: MonetDB_1-24 > MonetDB: MonetDB_1-24 > clients: Clients_1-24 > MonetDB4: MonetDB_4-24 > MonetDB5: MonetDB_5-6 > pathfinder: XQuery_0-24 > sql: SQL_2-24 > geom: Geom_0-4 Thank you for this information, but compilation fails for MonetDB4 on the 'make' command with the following error message: /home/sk/opt/MonetDB/include/MonetDB/gdk/gdk_system.h:117: error: expected ‘)’ before ‘*’ token A detailed description of what I did is in the attached 'report' file, which actually is a bash script. The resulting logfile [1] from report 2>&1 | tee logfile is available for download. Kind regards Stefan ____________________ [1] logfile, link valid for one week, 409565 Bytes https://depot.uni-konstanz.de/get/72fwr6 -- Stefan Klinger o/klettern /\/ bis zum send plaintext only - max size 32kB - no spam \ Abfallen http://stefan-klinger.de

4 10

[MonetDB-users] Problems With CREATE TEMPORARY TABLE
by James Leigh 21 Jul '08

21 Jul '08

Hi all, I am having trouble using MonetDB, with dynamic temporary table creation. I have included a simple unit test to demonstrate the problem. These tests work fine on other JDBC SQL servers, but fail with the below exceptions when using MonetDB. Any ideas on how to make new tables available to other connections or keep new tables from interrupting other jdbc statements? Thanks, James ---%<--- java.sql.SQLException: INSERT INTO: no such table 'paints_2' at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2058) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:1808) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:371) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:340) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.<init>(MonetPreparedStatement.java:102) at nl.cwi.monetdb.jdbc.MonetConnection.prepareStatement(MonetConnection.java:625) at nl.cwi.monetdb.jdbc.MonetConnection.prepareStatement(MonetConnection.java:586) at org.apache.commons.dbcp.DelegatingConnection.prepareStatement(DelegatingConnection.java:248) at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareStatement(PoolingDataSource.java:302) at MonetDBJdbcTest.testDualCreateTable(MonetDBJdbcTest.java:28) ---%<--- java.sql.SQLException: Query did not produce a result set at nl.cwi.monetdb.jdbc.MonetPreparedStatement.executeQuery(MonetPreparedStatement.java:228) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93) at MonetDBJdbcTest.testSelectWhileCreateTable(MonetDBJdbcTest.java:77) ---%<--- import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; import java.sql.SQLException; import java.sql.Statement; import junit.framework.TestCase; import org.apache.commons.dbcp.BasicDataSource; public class MonetDBJdbcTest extends TestCase { private BasicDataSource ds1; public void testDualCreateTable() throws Exception { Connection c4 = ds1.getConnection(); c4.setAutoCommit(true); Connection c76 = ds1.getConnection(); c76.setAutoCommit(false); Statement s93 = c76.createStatement(); s93.execute("CREATE TEMPORARY TABLE TRANSACTION_STATEMENTS (\n ctx INTEGER NOT NULL,\n subj INTEGER NOT NULL,\n pred INTEGER NOT NULL,\n obj INTEGER NOT NULL,\n expl BOOL NOT NULL\n)"); s93.close(); Statement s96 = c4.createStatement(); s96.execute("CREATE TABLE paints_2 (\n ctx INTEGER NOT NULL,\n subj INTEGER NOT NULL,\n obj INTEGER NOT NULL,\n expl BOOL NOT NULL\n)"); s96.close(); PreparedStatement ps102 = c76.prepareStatement("INSERT INTO paints_2\nSELECT DISTINCT ctx, subj, obj, expl FROM TRANSACTION_STATEMENTS tr\nWHERE NOT EXISTS (SELECT * FROM paints_2 st\nWHERE st.ctx = tr.ctx AND st.subj = tr.subj AND st.obj = tr.obj AND st.expl = tr.expl)"); ps102.close(); c76.close(); c4.close(); } public void testSelectWhileCreateTable() throws Exception { Connection c2 = ds1.getConnection(); c2.setAutoCommit(true); Statement s7 = c2.createStatement(); s7.execute("CREATE TABLE HASH_VALUES (\n id INTEGER NOT NULL,\n value BIGINT NOT NULL\n)"); s7.close(); PreparedStatement ps72 = c2.prepareStatement("SELECT id, value\nFROM HASH_VALUES\nWHERE value IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"); ps72.setLong(1, 157585763729791056l); ps72.setLong(2, 1105705122070497121l); ps72.setLong(3, 940939309403574879l); ps72.setLong(4, 44738479624713631l); ps72.setLong(5, 93693228172326007l); ps72.setLong(6, 909856949769965718l); ps72.setLong(7, 388219017765868455l); ps72.setLong(8, 498326321060763331l); ps72.setLong(9, 917168269028472831l); ps72.setLong(10, 560146884310966732l); ps72.setLong(11, 178986943741094305l); ps72.setNull(12, -5); ps72.setNull(13, -5); ps72.setNull(14, -5); ps72.setNull(15, -5); ResultSet rs73 = ps72.executeQuery(); rs73.next(); rs73.close(); Statement s76 = c2.createStatement(); s76.execute("CREATE TABLE TRIPLES (\n ctx INTEGER NOT NULL,\n subj INTEGER NOT NULL,\n pred INTEGER NOT NULL,\n obj INTEGER NOT NULL,\n expl BOOL NOT NULL\n)"); s76.close(); ps72.setLong(1, 157585763729791056l); ps72.setLong(2, 93693228172326007l); ps72.setLong(3, 44738479624713631l); ps72.setLong(4, 909856949769965718l); ps72.setLong(5, 388219017765868455l); ps72.setLong(6, 917168269028472831l); ps72.setNull(7, -5); ps72.setNull(8, -5); ps72.setNull(9, -5); ps72.setNull(10, -5); ps72.setNull(11, -5); ps72.setNull(12, -5); ps72.setNull(13, -5); ps72.setNull(14, -5); ps72.setNull(15, -5); ResultSet rs88 = ps72.executeQuery(); rs88.close(); ps72.close(); c2.close(); } @Override protected void setUp() throws Exception { Class.forName("nl.cwi.monetdb.jdbc.MonetDriver"); ds1 = new BasicDataSource(); ds1.setUrl("jdbc:monetdb://localhost/jdbc_test"); ds1.setUsername("monetdb"); ds1.setPassword("monetdb"); } @Override protected void tearDown() throws Exception { Connection c106 = ds1.getConnection(); Statement s107 = c106.createStatement(); try { s107.execute("DROP TABLE HASH_VALUES"); s107.execute("DROP TABLE TRIPLES"); } catch (SQLException exc) { // ignore } try { s107.execute("DROP TABLE TRANSACTION_STATEMENTS"); s107.execute("DROP TABLE paints_2"); } catch (SQLException exc) { // ignore } s107.close(); c106.close(); ds1.close(); } }

2 6

Re: [MonetDB-users] [Monetdb-developers] tables/databases simultaneous updates & extended-stored-procedures
by Martin Kersten 19 Jul '08

19 Jul '08

SOL ZADEH wrote: > > Thanks Martin; let me explain better: > > if table update are too costly (slow etc) then could try insertion > instead (having/including the timestamps as an index) > providing the server can cope with very frequent table insertions. > > Is there any benchmark tests in relation to table insertions? you can have a look at the TPCH loading figures. http://monetdb.cwi.nl/projects/monetdb//SQL/Benchmark/TPCH/ I have no hard figures in my head, but the DVD store benchmark we ran 2 years back came out directly after MySQL for high-volume business transactions. http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/DVDstore/index.html so it all depends what you consider costly/slow. > > Regarding : > >> Concurrent transactions can freely read a consistent copy of the > database. > does this mean "dirty reads" ? no, only consistent reads > Regards > > > > ----- Original Message ---- > From: Martin Kersten <Martin.Kersten(a)cwi.nl> > To: SOL ZADEH <solzadeh(a)yahoo.com> > Cc: Niels Nes <Niels.Nes(a)cwi.nl>; monetdb-developers(a)lists.sourceforge.net > Sent: Saturday, July 19, 2008 2:22:16 PM > Subject: Re: [Monetdb-developers] tables/databases simultaneous > updates & extended-stored-procedures > > SOL ZADEH wrote: > > > > Thanks Neils for your prompt reply. > > > > Is there a quantified limitation on "insertions" on scenarios I > > mentioned below (multiple tables/database)? > > I.e. Does very frequent insertions on 2 or more tables (on same > > database or other database on the same server) > > in any way effect the performance of the other queries > > (select,insert,update) on the "same" > > or "other" tables (i.e. is parallel processing possible & locking > > performance etc). > > Regards > Trying to interpret you question what it means. > MonetDB does not lock tables or databases during update. Concurrent > transactions can > freely read a consistent copy of the database. > All interference comes from sharing the same CPU, memory and IO channels. > > > > ----- Original Message ---- > > From: Niels Nes <Niels.Nes(a)cwi.nl <mailto:Niels.Nes@cwi.nl>> > > To: SOL ZADEH <solzadeh(a)yahoo.com <mailto:solzadeh@yahoo.com>> > > Cc: monetdb-developers(a)lists.sourceforge.net > <mailto:monetdb-developers@lists.sourceforge.net> > > Sent: Saturday, July 19, 2008 8:49:55 AM > > Subject: Re: [Monetdb-developers] tables/databases simultaneous > > updates & extended-stored-procedures > > > > On Fri, Jul 18, 2008 at 05:13:54PM -0700, SOL ZADEH wrote: > > > Hi > > > > > > > > > I have a couple of questions and really appreciate if any one > > > could answer them or instruct me to the relevant forum topics: > > > > > > > > > 1- what is the limitations (speed, threshold etc) on > > > simultaneous tables /databases update i.e. via multiple odbc > > > connection lets say (also any other faster API to interface > > > with the MonetDB ?) > > MonetDB research concentrated mostly in low update/high query loads. The > > update implementation is still quite capable, but no reall performance > > experiments on updates have been conducted. Next to odbc we have > > mapi and jdbc APIs available. > > > > > > > > > 2- Is there a way to have the triggers to generate > > > real-world (non-db) events i.e. similar to the Extended Stored > > > Procedure in SqlServer or the way Oracle allows you to run Java > > > code inside the stored procedures. > > MonetDB can be extended using C-code and that code could be run within > > the triggers. > > > > Niels > > > > > > Thanks > > > > > > > > ------------------------------------------------------------------------- > > > This SF.Net email is sponsored by the Moblin Your Move Developer's > > challenge > > > Build the coolest Linux based applications with Moblin SDK & win > > great prizes > > > Grand prize is a trip for two to an Open Source event anywhere in > > the world > > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > <http://moblin-contest.org/redirect.php?banner_id=100&url=/> > > <http://moblin-contest.org/redirect.php?banner_id=100&url=/ > <http://moblin-contest.org/redirect.php?banner_id=100&url=/>> > > > _______________________________________________ > > > Monetdb-developers mailing list > > > Monetdb-developers(a)lists.sourceforge.net > <mailto:Monetdb-developers@lists.sourceforge.net> > > <mailto:Monetdb-developers@lists.sourceforge.net > <mailto:Monetdb-developers@lists.sourceforge.net>> > > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > > > -- > > > > Niels Nes, Centre for Mathematics and Computer Science (CWI) > > Kruislaan 413, 1098 SJ Amsterdam, The Netherlands > > room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 > > url: http://www.cwi.nl/~niels <http://www.cwi.nl/%7Eniels> > <http://www.cwi.nl/%7Eniels> e-mail: > > Niels.Nes(a)cwi.nl <mailto:Niels.Nes@cwi.nl> <mailto:Niels.Nes@cwi.nl > <mailto:Niels.Nes@cwi.nl>> > > > > ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------- > > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > > Build the coolest Linux based applications with Moblin SDK & win > great prizes > > Grand prize is a trip for two to an Open Source event anywhere in > the world > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > <http://moblin-contest.org/redirect.php?banner_id=100&url=/> > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Monetdb-developers mailing list > > Monetdb-developers(a)lists.sourceforge.net > <mailto:Monetdb-developers@lists.sourceforge.net> > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > >

1 0

Re: [MonetDB-users] Range join question [MonetDB]
by Martin Kersten 18 Jul '08

18 Jul '08

Vasilis Vassalos wrote: > Hello again, > > for the student project I've told you about, we need to perform range > joins, so we tried some on MonetDB. > We're having trouble getting range joins on strings working properly. > > Michalis, my student, has looked into it and reports that > ALGrangejoin (algebra.c) calls BATrangejoin (gdk_rangejoin.c) that > performs the join and returns a BAT > as a result. BATrangejoin has a switch with a case for TYPE_chr that > compares strings You are really delving deep in the sources ; ) An area only meant for the brave at heart. > > When we try to perform a range join on a string attribute (char(n), > even for n=1, varchar(n)) we get > an exception: > !MALException:algebra.rangejoin:GDKerror > !ERROR: BATrangejoin: type not implemented > > The switch statement skips the TYPE_chr case and goes to the default. > For example, > >> sql>create table s1 (id int, v char(10)); >> sql>create table s2 (id int, v char(10)); >> sql>insert into s1 values (1,'s1'); >> sql>insert into s1 values (2,'s2'); >> sql>insert into s1 values (3,'s3'); >> sql>insert into s1 values (4,'s4'); >> sql>insert into s1 values (5,'s5'); >> sql>insert into s1 values (6,'s6'); >> sql>insert into s2 values (1,'s1'); >> sql>insert into s2 values (2,'s2'); >> sql>insert into s2 values (3,'s3'); >> sql>insert into s2 values (4,'s4'); >> sql>insert into s2 values (5,'s5'); >> sql>insert into s2 values (6,'s6'); >> sql>select s1.id from s1, s2 where s1.v between s2.v and s2.v||'c' ; >> !MALException:algebra.rangejoin:GDKerror >> !ERROR: BATrangejoin: type not implemented > This is an instance of the more general theta join. We will have a look into the issue.. It certainly should not have falled back to an error this low in the system. > whereas > >> sql>select s1.id from s1, s2 where s2.id=2 and s1.id between s2.id >> and s2.id+1 ; >> +----+ >> | id | >> +====+ >> | 2 | >> | 3 | >> +----+ > > Thanks! >

1 0

[MonetDB-users] Two questions with XQuery
by Andreas Meinl 17 Jul '08

17 Jul '08

Hi there, I'm currently porting some XQuery scripts to MonetDB/XQuery, and encountered two maybe simple obstacles. The first one is that I can't use fn:collection, but instead always have to apply pf:collection. The online documentation [1] states both alternatives should be practicable. For example in eXist and Sedna I can write: let $a := fn:collection("samples") return $a//id But this does not work for MonetDB/XQuery! Here I have to use: let $a := pf:collection("samples")/child::document-node() return $a//id My second question is that I cannot execute this kind of queries, which runs well in eXist and Sedna: <average> { fn:round(fn:avg( for $doc in fn:collection("samples") return $doc/example/@sum )) } </average> For MonetDB/XQuery I have to pull out pf:collection and add a second return statement: let $col := pf:collection("samples")/child::document-node() return <average> { fn:round(fn:avg( for $doc in $col return $doc/example/@sum )) } </average> So what am I doing wrong? Does MonetDB/XQuery by any chance comply more with any standards than eXist and Sedna do? Or ist fn:collection just not implemented in the current release of MonetDB/XQuery? I would be glad if someone could give me a hint! It's so confusing... Thank you! Regards, Andreas Meinl [1] http://monetdb.cwi.nl/projects/monetdb/XQuery/QuickTour/DOCMGT/ index.html#openall

4 10

[MonetDB-users] Compile: buildtools -> hard, MonetDB -> fails
by Stefan Klinger 17 Jul '08

17 Jul '08

Hello, it took me quite a while to compile the build tools, since the make process failed without any meaningful error messages, but I cannot build Monetdb. I'd expect the configure script to fail with a descriptive error message if required packages (in my case 'libtool', 'flex') are missing. > prefix="${HOME}/opt/MonetDB" > cvs -d:pserver:anonymous@monetdb.cvs.sourceforge.net:/cvsroot/monetdb checkout buildtools [...] > pushd buildtools > ./bootstrap [...] ./bootstrap: line 29: libtoolize: command not found [...] The message about a missing 'libtoolize' is easily overlooked, and configure was running fine anyway: > ./configure --prefix="${prefix}" [...] But the make fails > make [...] /tmp/sk/2/buildtools/mel/mel.yy:331: undefined reference to `mel_yyerror(char const*)' [...] make: *** [all] Error 2 In case you don't know that 'yyerror' hints at flex, you're lost. Well, I figured that out and went on: > make install > popd Then I tried to continue with MonetDB: > cvs -d:pserver:anonymous@monetdb.cvs.sourceforge.net:/cvsroot/monetdb checkout MonetDB > pushd MonetDB > ./bootstrap automake 1.10.1 is 1.7 or newer. Good. autoconf 2.61 is 2.57 or newer. Good. libtool 1.5.26Debian1.5.26-4 is 1.4 or newer. Good. Python 2.5.2 is 2.0.0 or newer. Good. Traceback (most recent call last): File "/home/sk/opt/MonetDB/bin/autogen.py", line 29, in <module> from autogen import am ImportError: No module named autogen Now this leaves me completely clueless... Any help wouldbe appreciated. Kind regards, Stefan -- Stefan Klinger o/klettern /\/ bis zum send plaintext only - max size 32kB - no spam \ Abfallen http://stefan-klinger.de

3 3

[MonetDB-users] mserver5 seg fault
by mark 14 Jul '08

14 Jul '08

i have over 6gb free space, but it crashes like this.. [mark@localhost ~]$ sudo mserver5 # MonetDB server v5.7.0, based on kernel v1.25.0 # Serving database 'demo' # Compiled for i686-redhat-linux-gnu/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /etc/monetdb5.conf) !ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), please restart. !ERROR: (One potential cause could be that your disk might be full...) [mark@localhost ~]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 19G 12G 6.7G 63% / /dev/sda3 194M 19M 165M 10% /boot tmpfs 2.0G 1.1M 2.0G 1% /dev/shm /dev/sda2 50G 40G 9.9G 80% /mnt/c gvfs-fuse-daemon 19G 12G 6.7G 63% /home/mark/.gvfs

3 4

[MonetDB-users] creating database - monetdb not found
by mark 14 Jul '08

14 Jul '08

i installed MonetDB using yum and these packages were installed Installed: MonetDB.i686 0:1.25.0-1_Fedora9.oid32 MonetDB-client.i686 0:1.25.0-1_Fedora9.oid32 MonetDB-client-python.i686 0:1.25.0-1_Fedora9.oid32 MonetDB-testing.i686 0:1.25.0-1_Fedora9.oid32 MonetDB5-server.i686 0:5.7.0-1_Fedora9.oid32 Dependency Installed: MonetDB-python.i686 0:1.25.0-1_Fedora9.oid32 now i want to create a database like in the documentation shell> monetdb create demo successfully created database 'demo' [mark@localhost ~]$ monetdb bash: monetdb: command not found what other packages do i need to install? thanks

2 2

[MonetDB-users] Repeated database corruption - v2.24.0
by Kirk Abbott 13 Jul '08

13 Jul '08

Hello, I am using Monetdb on Windows XP (see the server specs below). Unfortunately, I seem to corrupt the database very frequently involving a complete removal of the MonetDB5 directory (and all associated work). Corruption seems to occur if I hit a Ctrl-C (say on a long running query). Questions: 1) Is this a problem only on Windows? or is it a problem with this version of the server? 2) Is there a better way to kill a long running query? Clearly, corrupting the entire database (it won't restart!) just by killing a query, does not confidence build Any ideas? Kirk. # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'demo' # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\Program Files\CWI\MonetDB5\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.24.0 loaded >

2 2

[MonetDB-users] SQL Table partitioning in MonetDB
by Napoma 10 Jul '08

10 Jul '08

I have this problem... i have a table with 33.000.000 of records.. the table is named i_predicate with four columns as follows (oid, name, isubj, iobj)... i want to execute a physical partition on i-predicate respect to the values of name column.. in this way if i query i_predicate monetdb has to use only the respective partition with the respective value of name... does it exist a sql statement for creating partitions of i_predicate? does it exist a script for this partition on MonetDB? and if i create the partition... i have to define function to query correctly the partitions? Thanks -- View this message in context: http://www.nabble.com/SQL-Table-partitioning-in-MonetDB-tp18384206p18384206… Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

[MonetDB-users] PF: warning: query will always return an error
by Ying Zhang 10 Jul '08

10 Jul '08

Hello, In stable branch + milprint_summer, executing the following query (you need to have a "hello.xml" doc shredded in collection "hello"): let $path := 'backup/test/hello/' let $index := element documents { for $document at $pos in pf:documents('hello') let $uri := concat($path, string(10000 + (($pos div 10000)) cast as xs:integer), '/', string($pos) , '.xml') return (text { "
 " }, element document { attribute uri {$uri}, $document/text() }), text { "
" } } return (put($index, concat($path,'documents.xml')), for $document in $index/document return put(exactly-one(doc(string($document))), exactly-one($document/@uri))) Results in the first 'put' is done, while the second 'put' is not executed, actually, the MIL code generated by 'pf' contains only translation of (the first) one 'put'. Also, the query gives a warning: query will always return an error If I comment out the first 'put', the second 'put' in the for-loop get executed, *without* the warning. Executing a similar query: (put(<a/>, "a.xml"), for $n in ("b.xml", "c.xml") return put(exactly-one(doc("hello.xml")), $n) ) also results in the warning, however, the generated MIL code contains translation of both 'put' and three documents are put in dbfarm. I'd like to know why the first query fails, and what has triggered the warning. Thanks a lot! Kind regards, Jennie

6 11

Re: [MonetDB-users] MonetDB/XQuery - Bug 1899258
by Ying Zhang 10 Jul '08

10 Jul '08

On Wed, Jul 09, 2008 at 06:29:08PM +0200, Andreas Meinl wrote: > Hi Ying Zhang, > > regarding bug 1899258 "PF: Error message with Administrative Console": I > still have the same problem, even with the current release of > MonetDB/XQuery, which I downloaded just today. The package is called > MonetDB4-XQuery-Installer-i386-20080624.msi. You have closed this bug on 20 > June 2008. How can I check which version of libxml2 I have installed (or, > is bundled with this release of MonetDB/XQuery)? I've attached two > screenshots with similar error messages. Thanks a lot! Hello Andreas, Given the second screenshot, it indeed looks like you are still having this problem. Is it true that if you click on one of the buttons for the first time, it will first hang for about 30 seconds, before you get this error message? In the current release, the Windows installer contains (should contain) an older version of libxml2 (2.6.30) due to this problem. So, this error should actually not happen. You can find the libxml2 DLL file in the lib directory of your installation dir, which is typically 'program files\cwi'. So please check you have the correct version of libxml2 in your installation. Regards, Jennie > > Regards, > > Andreas Meinl > > > > > >

1 0

[MonetDB-users] ODBC + non-ascii
by Pawel Lewicki 08 Jul '08

08 Jul '08

Hi, I don't know if problem is related to used interface (ODBC) but I have a problem with inserting non-ascii data. All data I insert seems broken. I started with 'utf-8' but tested 'cp1250' and 'iso8859-2' as well. No luck. All data I take back has broken non-ascii characters. Any clues? (Windows 32-bit "Jun 2008" release) Pawel Lewicki

2 4

[MonetDB-users] Update using two (2) tables
by Kirk Abbott 04 Jul '08

04 Jul '08

Hello, I am having trouble with the syntax for doing an update using 2 tables. In postgres, one can do something like: update table1 set name = table2.name from table2 where table1.id = table2.id ; In oracle (it's something like) update table1 a set name = (select b.name from table2 b where a.id = b.id) ; What is the correct way to do this in MonetDb (as both of the above fail) Thanks, Kirk.

3 2

[MonetDB-users] XQ: inconsistency between fn:collection and pf:collection after a del-doc
by luc touraille 03 Jul '08

03 Jul '08

Hi, Currently working on a big project with MonetDB XQuery, I stumbled upon an awkward bug (unless it is a *very* perturbing feature): after deleting a document in a collection "myColl", using the function pf:del-doc, this document is still displayed when calling fn:collection("myColl"), and not when using pf:collection("myColl"). I assume the latter result is the expected one, so why is the first query displaying data that is not supposed to be in the database anymore ? Have I misunderstood something or is it a true bug (maybe related to the http://www.nabble.com/--monetdb-Bugs-1976341---XQ%3A-leftovers-after-deleti… bug #1976341 : leftovers after deleting a document ) ? N.B.: this problem occured with the Feb 2008 release; I tried to reproduce it with the Jun 2008 release, and this time it's the pf:collection() function that gave me the wrong results (the fn: one being unsupported yet). Greetings, Luc Touraille -- View this message in context: http://www.nabble.com/XQ%3A-inconsistency-between-fn%3Acollection-and-pf%3A… Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] New MonetDB Release
by Stefan Manegold 28 Jun '08

28 Jun '08

Dear MonetDB users and developers, The MonetDB team at CWI is pleased to announce the new "Jun2008" release of the MonetDB suite of programs. For MonetDB/XQuery, this release marks the roll forward from the old "milprint_summer"-based Pathfinder compiler to the new "Algebra"-based Pathfinder compiler. MonetDB/SQL has been extended with initial support for OpenGIS via the "Geom[etry]" extension module, aka. MonetDB/SQL/GIS. Additionally, the configure defaults for compilation from sources on Unix(-like) systems as well as the Windows installers have been cleaned-up. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ IMPORTANT NOTE: Before upgrading to the latest "Jun2008" release from any previous release of MonetDB, please make a dump of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ The new release consists of the following packages: MonetDB Common 1.24.0 MonetDB Clients 1.24.0 MonetDB4 Server 4.24.0 MonetDB4 XQuery 0.24.0 MonetDB5 Server 5.6.0 MonetDB5 SQL 2.24.0 MonetDB5 Geom 0.4.0 monetdb-1.8-jdbc.jar There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. Enjoy! The MonetDB development team. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[MonetDB-users] Python on Windows
by Kees Varekamp 27 Jun '08

27 Jun '08

Does anyone know how to get the Python libs running on Win XP? Tried it with latest trunk and release versions of Monet, and with Python 2.4 and 2.5 File "C:\Python25\lib\MonetDB\MapiLib.py", line 7, in <module> import _MapiLib ImportError: DLL load failed: The specified module could not be found. _MapiLib.pyd is there alright. Is this a version problem? Thanks, Kees

1 0

[MonetDB-users] MonetDB database size restrictions
by Sam Mason 19 Jun '08

19 Jun '08

Hi, I hope you don't get asked this question too often, I've had a look for FAQs but couldn't find anything obvious. I'm trying to load some data into MonetDB (built on a i386 Debian "etch" box from MonetDB-Feb2008-SuperBall.tar.bz2) and I'm getting strange problems. I've got a table with 11 columns (2 int, 5 varchar, 4 date) and about 38 million rows (the text file I want to pull in is 3.3GB). If I try COPYing the data in, Monet alternately stops making progress (the computer is reading from the disk slowly, so maybe it's just going to take a *long* time to finish) or crashes with the following message: !SQLException:importTable:failed to import table !ERROR: GDKload: cannot mmap(): name=24/2470, ext=tail.priv !OS: Cannot allocate memory !ERROR: GDKload failed: name=24/2470, ext=tail.priv If I try splitting the file into smaller pieces (about a gig each), things work and I end up with the data in the database. It would be nice to know why I needed to split the file up first. Now I've got the data in I tried running a few queries to make sure everything went OK. I'm getting some very strange results when doing aggregations. The following work: SELECT count(*) FROM source_livestock; +----------+ | count_ | +==========+ | 37898310 | +----------+ SELECT sex_code, count(*) FROM source_livestock GROUP BY sex_code; +----------+---------------+ | sex_code |count_sex_code | +==========+===============+ | M | 17505304 | | F | 20392997 | | null | 9 | +----------+---------------+ This last query came back amazingly quickly, about ten times faster than Postgres. Monet looks like a fairly amazing database if I can routinely expect queries to be this much faster. SELECT count(*); +--------------+ | single_value | +==============+ | 1 | +--------------+ but the following don't do what I'd expect: SELECT count(1); MAPI = monetdb@endemic:50001 ACTION= read_line QUERY = select count(1); ERROR = Connection terminated I'd expect to get a single row with a single value containing the number 1 from this query. And for the following query I'd expect to get the same, just with the result being zero. SELECT count(null); !syntax error, unexpected sqlNULL in: "select count(null" This is how I found the bug: SELECT sex_code, count(1) FROM source_livestock GROUP BY sex_code; !TypeException:user.s3_1[26]:'aggr.count_no_nil' undefined in: _36:bat[:any,:int] := aggr.count_no_nil(_35:bte, _31:bat[:oid,:oid], _32:bat[:oid,:oid]) !SQLException:sql.rsColumn:Cannot access descriptor !WARNING: BATdescriptor: range error 1 I've come to expect COUNT(*) and COUNT(1) as having the same semantics. But I may be missing something. SELECT sex_code, count(1+1) FROM source_livestock GROUP BY sex_code; !TypeException:user.s4_1[28]:'aggr.count_no_nil' undefined in: _39:bat[:any,:int] := aggr.count_no_nil(_38:bte, _32:bat[:oid,:oid], _33:bat[:oid,:oid]) +----------+---------------------+ | sex_code |sql_add_single_value | +==========+=====================+ | M | monetdb | +----------+---------------------+ This second one is particularly bad because I get an incorrect answer back. Any idea what I'm doing wrong? Thanks, Sam

2 13

[MonetDB-users] MonetDB Server dies on connect
by Robert Schumacher 17 Jun '08

17 Jun '08

Hello, i was evaluating MonetDB. Got the Server running finr on Windows XP x64. I created about 12 Tabled by using a JDBC Client, then the Server died instantly. Since then when I start the Server it dies asap when I either start the Windows Client or perform a JDBC connection. I get no error log, no message, nothing. Has anybody an idea how to get MonetDB to stop crashing on connect? Thanks, Robert Robert Schumacher, Senior Software Developer prevero Systems GmbH Landsberger Straße 154 80339 München Tel +49 89 455 77 - 0 Fax +49 89 455 77 - 200 Registergericht: Amtsgericht München, HRB 143809 -- Geschäftsführer: Matthias Thurner Bitte beachten Sie unsere Veranstaltungshinweise unter www.prevero.de

4 14

Re: [MonetDB-users] [Monetdb-developers] arrow keys not working in mclient in stable nightly
by Martin Kersten 16 Jun '08

16 Jun '08

Rt Ibmer wrote: > Hi - I use a SSH client to access my Fedora 8 core box. When I was using the RELEASE built I was able to use left/right/up/down/home/end keys within mclient to edit the line. For instance I could push up arrow to have it automatically cycle through the previous lines, and then use left arrow to move over to a part of the line and process to change it. > There have not been changes in that area. Running against the current also does not show the problem > That all worked fine under the RELEASE build. > > However now that I have built from source the nightly stable (as of a couple nights ago) I can no longer use these keys. For instance when I press UP arrow it simply adds "^[[A" to the line. Other arrow keys / home /end do something similar. > Nothing has changed in this area. I mostly use a Putty ssh client to access my machine and work with mclient. It works as expected. You may want to check the preference setting of the ssh client. > Not sure but I think something broke regarding this at some point between the released build and the stable nightly. Thanks. > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

2 1

Re: [MonetDB-users] replication in MonetDB
by Martin Kersten 15 Jun '08

15 Jun '08

Alexandru Toth wrote: > Hello, > > I'm the developer of Cubulus OLAP > (http://sourceforge.net/projects/cubulus/ ) . Hello Alexandru Welcome to the MonetDB user community. > > On the roadmap there is a hint about replication, coming in SQL 2.2 > (February 2008) release. Is it implemented ? If not, is it possible to use We have been experimenting with replication techniques. Some of the hooks have been included in the distribution. However, we are not yet satisfied and with the result. It remains on our R&D list to come up with a novel and efficient solution. > some existing software ex: http://dbreplicator.org/ Middleware solutions could potentially provide a short term solution. However, we don't know if they are powerful enough or require functionality currently not available in our JDBC driver. > > Or, is it possible to load data into "master", stop it, sync the > dbfarm to several slaves (into new directories), and re-start the > slaves with the new dbfarm path ? Of course, slaves are used as > read-only copies. ofcourse, provided you don't forget to copy the sql_logs as well. You can even instruct the server that the database is readonly. See the documentation : http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Storage-Modes.html regards, Martin > > -Alex > >

1 0

Re: [MonetDB-users] [Monetdb-developers] MonetDB Server dies on connect
by Martin Kersten 13 Jun '08

13 Jun '08

Sjoerd Mullender wrote: > Martin Kersten wrote: > >> Sjoerd, >> >> I was planning to try out the installers on my Vista machine. >> The previous version is v5.4.0 installed >> >> Uninstalling it gives the following error: >> There is a problem with this Windows Installer >> package. A program required for this install >> to complete could not be run. Contact your >> support personnel or package vendor. >> >> You want specific information, or should i manually >> remove the C:\Program Files\CWI/MonetDB5 >> > > Manually removing won't help unless you also edit the registry. > > My guess is that it complains about the uninstall of the ODBC driver. > Can you run > "C:\Program Files\CWI\MonetDB5\odbcinstall" /Uninstall > (I think that is the exact name) manually from a cmd window? > ran this program , which worked. However, uninstalling MonetDB still gives the same error.

2 1

[MonetDB-users] New default configure options for MonetDB
by Stefan Manegold 09 Jun '08

09 Jun '08

Dear MonetDB developer and users, For convenience of both developers and users as well as to comply even more with open source standards, we now set/use the following defaults for the configure options --enable-strict, --enable-assert, --enable-debug, --enable-optimize (Unless specified differently, the same defaults are used for both the "Stable" release branches and the "Current" development trunk.) When compiling from CVS sources (as mainly done by developers): strict=yes assert=yes debug=yes optimize=no (*) When compiling from packaged/distributed sources (i.e., tarballs) (as mainly done by users): strict=no assert=no debug=no optimize=no (*) For building binary distributions (RPMs, MSIs): strict=no assert=no debug=no optimize=yes With nightly testing for the "Stable" release branches: strict=yes assert=no debug=no optimize=yes With nightly testing for the "Current" development trunk: strict=yes assert=yes debug=no optimize=yes (*) IMPORTANT NOTE: Since "--enable-optimize=yes" is no longer the default for any case except binary packages, it is *strongly recommended* to (re)compile everything from scratch, *explicitly configured* with "--enable-debug=no --enable-optimize=yes", in case you want/need to run any performance experiments with MonetDB! (Please note: "--enable-X=yes" is equivalent to "--enable-X", and "--enable-X=no" is equivalent to "--disable-X".) Don't hesitate to ask in case you have any questions! Enjoy! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [MonetDB-users] [Monetdb-developers] Fastest way to calculate median?
by Martin Kersten 08 Jun '08

08 Jun '08

Rt Ibmer wrote: > Hi - I have an int column with several million rows of data. What is the fastest way to write a query to get the median value of the column? Apparently there is no median() function in SQL. A web search revealed lots of various talk about using stored procedures and fancy functions. None of this looked like it would be very efficient. I searched these archives but didn't see a single mention of median. Thank you! > True, the complexity of the median() in essence requires a partial sort of the complete list. It is a member of a group of statistical functions. A feature request has been added to the list. > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

1 0

[MonetDB-users] Moving records from SQL Server to MonetDB
by Rt Ibmer 07 Jun '08

07 Jun '08

Hi All - I'm new to monetdb and it looks fantastic! I have about 2 millions rows of data that I'd like to export from SQL Server and import them into Monetdb so I can do some preliminary benchmarking. What is the best way to do this? The schemas are the same for SQL Server and MonetDB. So basically I'm looking for the best command to run on SQL Server to export the data in a format best suited for importing into MonetDB, and what is the command to then load the data into MonetDB? Thanks!

1 0

[MonetDB-users] [Fwd: RE: PROBLEM WITH MONETDB 5.0]
by Martin Kersten 07 Jun '08

07 Jun '08

-------- Original Message -------- Subject: RE: PROBLEM WITH MONETDB 5.0 Date: Sat, 7 Jun 2008 23:16:13 +0800 From: Tran Quoc Trung <tqtrung(a)comp.nus.edu.sg> To: 'Martin Kersten' <Martin.Kersten(a)cwi.nl> References: <38563.137.132.80.35.1212469981.squirrel(a)mysoc.nus.edu.sg> <4844E431.2090403(a)cwi.nl> <3363.137.132.80.35.1212475247.squirrel(a)mysoc.nus.edu.sg> <4844F364.70601(a)cwi.nl> Hi Mr. Martin, I have installed MonetDB both by using monetdb-install.h and following the document for expert in the website on Ubuntu 2.6 64-bits. I can use the MonetDB server and client. However, when writing a program (in C) to connect to MonetDB, I encounter the error that libMapi.so require libc-2.7.so (The libc in my system is libc-2.5.so). Are there any solution to handle this situation (b/c I restrict not to upgrade the libc due to using the common server with other project). Could you please help me to take a look at this. Thank you very much Trung. -----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Tuesday, 3 June, 2008 3:32 PM To: tqtrung(a)comp.nus.edu.sg; Communication channel for MonetDB users Subject: Re: PROBLEM WITH MONETDB 5.0 tqtrung(a)comp.nus.edu.sg wrote: > Hi, > Thank you very much. I also guess the reason due to the size of the > operands. Currently, I am running on Linux Fedora Core 8, 32 bit. The > cardinality of the opearand s in the range 100,000 to 1,500,000 records. > the sizes themselves are not a problem, but the expected size of the join result could be. you could take a sample to see how it explodes (algebra.sample()) > Are there any other solutions if my machine could not play with those > queries. > Please help me to check it again. > > Thanks, > Trung > > > >> tqtrung(a)comp.nus.edu.sg wrote: >> >>> Dear all, >>> I am a PhD student at School of Computing, NUS, Singapore. Currently, I >>> used MonetDB as the DBMS to do some experiments. I issue a query with >>> about 10 joins, but the system responds with the following errors: >>> >>> >> Please inform us about the platform and distribution you are useing. >> >>> !MALException:algebra.join:GDKerror >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> >>> >> this typically is an error when the join algorithm can not allocated >> enough memory >> to hold the intermediate result. For example, on a 32-bit system this >> might cause >> a problem quickly. >> If the join result should fit in memory, then there might be a problem >> in the >> estimation of the intermediate size needed. For this we need more >> information, >> such as the cardinality of the operands and properties obtained by >> bat.info() >> >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=tail.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=44/4401, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=44/4401, ext=tail.priv >>> >>> I also try to change the parameter in monetdb5.conf about the memory: >>> gdk_maxsize and gdk_vm_minsize but it still does NOT work. >>> >>> >> please don't play with these settings. They only control a thread for >> flushing dirty pages to disk. >> >>> Could you please help me to figure out which parameters I could change >>> to >>> make it work. >>> >>> Thank you very much >>> Trung >>> >>> >>> >> > >

1 0

[MonetDB-users] [Fwd: Re: php extension]
by Martin Kersten 06 Jun '08

06 Jun '08

-------- Original Message -------- Subject: Re: php extension Date: Fri, 6 Jun 2008 11:20:39 +1200 From: Kees Varekamp <kees.varekamp(a)gmail.com> To: Sjoerd Mullender <sjoerd(a)acm.org> CC: Niels Nes <Niels.Nes(a)cwi.nl>, monet(a)cwi.nl, Sjoerd Mullender <Sjoerd.Mullender(a)cwi.nl> References: <d8981ee0806042218l13609c8an47065c5b9a8354e9(a)mail.gmail.com> <20080605054117.GA350(a)cwi.nl> <4847D4D4.8020608(a)acm.org> <d8981ee0806050543w19dd0028gbb9327c5378549b1(a)mail.gmail.com> <4847EEFD.8080102(a)acm.org> Tnx. Na een herstart doet ie het! Groeten, Kees 2008/6/6 Sjoerd Mullender <sjoerd(a)acm.org <mailto:sjoerd@acm.org>>: Kees Varekamp wrote: > Bedankt! Hij start nu, ik ben weer een stap verder. > > Krijg nu een *Warning*: monetdb_connect() [function.monetdb-connect > <http://localhost/monet/function.monetdb-connect>]: Unable to connect to > MonetDB server: gethostbyname failed > > Het voorbeeldje op de website > (http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/The-PHP-Library.ht…) > connect als volgt: > $db = monetdb_connect('sql', 'localhost', 50000, 'monetdb','monetdb') > > De synopsis eronder vermeldt echter: > monetdb_connect([string host [, string port [, string username [, string > password [, string language]]]]] > > Beide returnen bovenstaande foutmelding (zowel met localhost als met > 127.0.0.1 <http://127.0.0.1> <http://127.0.0.1>). Welke is juist? En klopt het dat ik in > principe de php_pgsql.dll functies kan vertalen naar Monet functies? > > Het is niet kritiek trouwens, if all else fails zet ik een Linux machine op. Het ziet ernaar uit dat de synopsis niet correct is en de volgorde van de argumenten inderdaad is zoals in dat voorbeeld. Ik heb dit ook net geprobeerd en het werkt. De hostnaam wordt met de C functie gethostbyname vertaald. Zowel de string 'localhost' als '127.0.0.1 <http://127.0.0.1>' zou moeten werken, maar het ligt aan je Windows installatie. Je kunt experimenteren met php -a (in een cmd window) en dan: php > print gethostbyname('localhost'); Dat zou 127.0.0.1 <http://127.0.0.1> moeten printen. Je kunt ook de file c:\windows\system32\drivers\etc\hosts bekijken om te zien of er een entry in staat voor localhost. -- Sjoerd Mullender

1 0

Re: [MonetDB-users] PROBLEM WITH MONETDB 5.0
by Martin Kersten 03 Jun '08

03 Jun '08

tqtrung(a)comp.nus.edu.sg wrote: > Hi, > Thank you very much. I also guess the reason due to the size of the > operands. Currently, I am running on Linux Fedora Core 8, 32 bit. The > cardinality of the opearand s in the range 100,000 to 1,500,000 records. > the sizes themselves are not a problem, but the expected size of the join result could be. you could take a sample to see how it explodes (algebra.sample()) > Are there any other solutions if my machine could not play with those > queries. > Please help me to check it again. > > Thanks, > Trung > > > >> tqtrung(a)comp.nus.edu.sg wrote: >> >>> Dear all, >>> I am a PhD student at School of Computing, NUS, Singapore. Currently, I >>> used MonetDB as the DBMS to do some experiments. I issue a query with >>> about 10 joins, but the system responds with the following errors: >>> >>> >> Please inform us about the platform and distribution you are useing. >> >>> !MALException:algebra.join:GDKerror >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> >>> >> this typically is an error when the join algorithm can not allocated >> enough memory >> to hold the intermediate result. For example, on a 32-bit system this >> might cause >> a problem quickly. >> If the join result should fit in memory, then there might be a problem >> in the >> estimation of the intermediate size needed. For this we need more >> information, >> such as the cardinality of the operands and properties obtained by >> bat.info() >> >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=tail.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=44/4401, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=44/4401, ext=tail.priv >>> >>> I also try to change the parameter in monetdb5.conf about the memory: >>> gdk_maxsize and gdk_vm_minsize but it still does NOT work. >>> >>> >> please don't play with these settings. They only control a thread for >> flushing dirty pages to disk. >> >>> Could you please help me to figure out which parameters I could change >>> to >>> make it work. >>> >>> Thank you very much >>> Trung >>> >>> >>> >> > >

1 0

Re: [MonetDB-users] PROBLEM WITH MONETDB 5.0
by Martin Kersten 03 Jun '08

03 Jun '08

tqtrung(a)comp.nus.edu.sg wrote: > Dear all, > I am a PhD student at School of Computing, NUS, Singapore. Currently, I > used MonetDB as the DBMS to do some experiments. I issue a query with > about 10 joins, but the system responds with the following errors: > Please inform us about the platform and distribution you are useing. > !MALException:algebra.join:GDKerror > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv > !OS: Cannot allocate memory > this typically is an error when the join algorithm can not allocated enough memory to hold the intermediate result. For example, on a 32-bit system this might cause a problem quickly. If the join result should fit in memory, then there might be a problem in the estimation of the intermediate size needed. For this we need more information, such as the cardinality of the operands and properties obtained by bat.info() > !ERROR: GDKload failed: name=45/4542, ext=head.priv > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=tail.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=45/4542, ext=tail.priv > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=45/4542, ext=head.priv > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=45/4542, ext=head.priv > !ERROR: GDKload: cannot mmap(): name=44/4401, ext=tail.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=44/4401, ext=tail.priv > > I also try to change the parameter in monetdb5.conf about the memory: > gdk_maxsize and gdk_vm_minsize but it still does NOT work. > please don't play with these settings. They only control a thread for flushing dirty pages to disk. > Could you please help me to figure out which parameters I could change to > make it work. > > Thank you very much > Trung > >

1 0

[MonetDB-users] Is it possible getting types IN params or result set?
by Marc Weber 29 May '08

29 May '08

Hi, I'd like to work on embedding query statements into the haskell language by using template haskell to automatically derive insert and select functions having proper types automatically. sqli for Java already does it. resulting in something as: (rows :: [ (Int, String) ]) = $( select "SELECT intV, stringV FROM table") $( ) means a function is automatically created returning a list of tules of type (Int, String). Because haskell is strongly typed the function select creating the the type safe function must get to know which type the query will return. When using INSERT statements it would be nice to also now which type of parameters have to be passed. Does MonetDB provide kind of query function WhichInOutTypesDoesQueryHave("SELECT ..") or WhichInOutTypesDoesQueryHave("INSERT INTO FOO (abc,def) VALUES (?,?)") This way syntax and semantics could be checked at compile time. Sincerly Marc Weber

3 5

[MonetDB-users] /bin/bash: ../Mx/Mx: No such file or directory
by Ying Zhang 28 May '08

28 May '08

Dear all, I'm trying to compile monet on my laptop But I already get a problem when I 'make' in buildtools: ../Mx/Mx -x C /home/jennie/monet/current/buildtools/mel/any_arg.mx /bin/bash: ../Mx/Mx: No such file or directory The directory Mx indeed contains too few files, but 'make' in Mx seems doing nothing: $ cd ../Mx $ ls config.log config.status Makefile mx_config.h stamp-h1 $ make make all-am make[1]: Entering directory `/home/jennie/scratch/monet-build/current/buildtools/build-debug/Mx' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/home/jennie/scratch/monet-build/current/buildtools/build-debug/Mx' $ ls config.log config.status Makefile mx_config.h stamp-h1 Sys. info: $ uname -a Linux 2.6.22-14-generic #1 SMP Tue Feb 12 07:42:25 UTC 2008 i686 GNU/Linux I did: bootstrap && configure && make && make install with the configure options: --prefix=$INSTALL_DIR --enable-strict --enable-assert --enable-debug --disable-optimize Does anyone have an idea why Mx is not generated by 'make'? Thanks! Jennie

2 1

Re: [MonetDB-users] (no subject)
by Martin Kersten 27 May '08

27 May '08

Yeo Yvonne wrote: > Dear Sir, > > I would like to find out more about MONETDB and the diffenerces > betwwen ORacle and MONETBD? Dear Yeo This is a question of comparing apples and oranges ;) There is no document available to compare the systems. Please have a look at the introductionary sections of the MonetDB manual. The rest is a study by itself. regards, Martin > > Thanks. > > With Regards, > Yvonne Yeo. >

3 2

[MonetDB-users] fedora core 9 repo or rpms?
by mobiledreamers＠gmail.com 23 May '08

23 May '08

Please share fc9 repos or how do we install monetdb in fc9 sudo yum -c /tmp/MonetDB-Stable-Fedora.repo resolvedep http://monetdb.cwi.nl/testing/projects/monetdb/Stable/.DailyBuilds./0_lates…: [Errno 14] HTTP Error 404: Not Found Trying other mirror. Error: Cannot retrieve repository metadata (repomd.xml) for repository: MonetDB-Stable. Please verify its path and try again *Forcing to use the 8 repo in 9 gives these errors* sudo yum -c /tmp/MonetDB-Release-Fedora.repo install MonetDB5-server MonetDB MonetDB-SQL-server5 Setting up Install Process Parsing package install arguments Resolving Dependencies --> Running transaction check ---> Package MonetDB.i386 0:1.22.0-1.fc8.oid32 set to be updated --> Processing Dependency: libssl.so.6 for package: MonetDB --> Processing Dependency: libcrypto.so.6 for package: MonetDB ---> Package MonetDB5-server.i386 0:5.4.0-1.fc8.oid32 set to be updated --> Processing Dependency: libssl.so.6 for package: MonetDB5-server --> Processing Dependency: MonetDB-client >= 1.22 for package: MonetDB5-server --> Processing Dependency: libcrypto.so.6 for package: MonetDB5-server --> Processing Dependency: libMapi.so.0 for package: MonetDB5-server ---> Package MonetDB-SQL-server5.i386 0:2.22.0-1.fc8.oid32 set to be updated --> Running transaction check ---> Package MonetDB-client.i386 0:1.22.0-1.fc8.oid32 set to be updated --> Processing Dependency: libssl.so.6 for package: MonetDB-client --> Processing Dependency: libcrypto.so.6 for package: MonetDB-client ---> Package MonetDB.i386 0:1.22.0-1.fc8.oid32 set to be updated --> Processing Dependency: libssl.so.6 for package: MonetDB --> Processing Dependency: libcrypto.so.6 for package: MonetDB ---> Package MonetDB5-server.i386 0:5.4.0-1.fc8.oid32 set to be updated --> Processing Dependency: libssl.so.6 for package: MonetDB5-server --> Processing Dependency: libcrypto.so.6 for package: MonetDB5-server --> Finished Dependency Resolution MonetDB-client-1.22.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libcrypto.so.6 is needed by package MonetDB-client-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB5-server-5.4.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libssl.so.6 is needed by package MonetDB5-server-5.4.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB5-server-5.4.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libcrypto.so.6 is needed by package MonetDB5-server-5.4.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB-1.22.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libssl.so.6 is needed by package MonetDB-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB-1.22.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libcrypto.so.6 is needed by package MonetDB-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB-1.22.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libcrypto.so.6 is needed by package MonetDB-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB5-server-5.4.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libssl.so.6 is needed by package MonetDB5-server-5.4.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB-client-1.22.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libssl.so.6 is needed by package MonetDB-client-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB5-server-5.4.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libcrypto.so.6 is needed by package MonetDB5-server-5.4.0-1.fc8.oid32.i386 (MonetDB-Release) MonetDB-1.22.0-1.fc8.oid32.i386 from MonetDB-Release has depsolving problems --> Missing Dependency: libssl.so.6 is needed by package MonetDB-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) Error: Missing Dependency: libcrypto.so.6 is needed by package MonetDB5-server-5.4.0-1.fc8.oid32.i386 (MonetDB-Release) Error: Missing Dependency: libssl.so.6 is needed by package MonetDB-client-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) Error: Missing Dependency: libssl.so.6 is needed by package MonetDB5-server-5.4.0-1.fc8.oid32.i386 (MonetDB-Release) Error: Missing Dependency: libcrypto.so.6 is needed by package MonetDB-client-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) Error: Missing Dependency: libcrypto.so.6 is needed by package MonetDB-1.22.0-1.fc8.oid32.i386 (MonetDB-Release) Error: Missing Dependency: libssl.so.6 is needed by package MonetDB-1.22.0-1.fc8.oid32.i386 (MonetDB-Release)

2 1

[MonetDB-users] MonetDB users
by Seth Grimes 23 May '08

23 May '08

Hello all, I've just joined the list to learn more about use of MonetDB. The DBMS looks quite interesting. Who's using it outside the research community at CWI? For what type and size of applications? Thanks, Seth -- Seth Grimes Alta Plana Corp, analytical computing & data management Intelligent Enterprise magazine (CMP), Contributing Editor grimes(a)altaplana.com http://altaplana.com 301-270-0795

1 0

[MonetDB-users] deep-equals function
by luc touraille 21 May '08

21 May '08

Hi everyone, I was wondering if the deep-equals function, described in the http://www.w3.org/TR/xpath-functions/#func-deep-equal W3C XQuery functions spec , was planned to be supported soon in MonetDB/XQuery. I saw on the CVS repository that the latest files of pathfinder included its signature, but I can't find its implementation. Can anyone enlighten me ? Thanks, Luc -- View this message in context: http://www.nabble.com/deep-equals-function-tp17364244p17364244.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Data Compression Algorithms
by Veneta Dobreva 20 May '08

20 May '08

Hello, I have the following question about MonetDB: does MonetDB implement some data compression algorithms? I would like to use an RLE (run-length encoding) algorithm for compression of my data. Is this already implemented in MonetDB? Thank you in advace for your help! Regards, Veneta Dobreva -- Dipl.-Inf. Veneta Dobreva Lehrstuhl Informatik III Tel.: ++49 (0)89 289-17288 Fakultät für Informatik Fax: ++49 (0)89 289-17263 TU München, Raum 02.11.041 MI veneta.dobreva(a)in.tum.de http://www-db.in.tum.de/~dobreva/

2 1

[MonetDB-users] MonetDB V 4.22 installation on Mac OS X 10.5.2 and Mac OS X 10.4.0.
by Irini Fundulaki 18 May '08

18 May '08

Hi, I have the following problems with MonetDB XQuery Release Feb 2008, V. 4.22 on Mac OS X 10.5.2 and Mac OS X 10.4.0. 1) Mac OS X 10.5.2: a) I was not able to install MonetDB from sources. I downloaded MonetDB-Feb2008-SuperBall.bz2 and followed the instructions in http://monetdb.cwi.nl/projects/monetdb//MonetDB/Documentation/Linux-Install…. I successfully installed MonetDB, clients and MonetDB4-server, but when installing MonetDB4-XQuery I had the following error: Undefined symbols: "_embedded_mil", referenced from: _main in prog.o ld: symbol(s) not found collect2: ld returned 1 exit status make[5]: *** [Mbeddedxq] Error 1 make[4]: *** [all] Error 2 make[3]: *** [all-recursive] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 b) by calling monetdb-install.sh. MonetDB was installed with no problems. Nevertheless, mclient with a simple query (e.g., <example>hello</example) stalled. Mserver stalled when executing xmldb_print(); 2) Mac OS X 10.4.0 a) I was able to install MonetDB XQuery Release Feb 2008, V. 4.22 using monetdb-install.sh. Mserver did not stall with xmldb_print(). mclient for the same query exhibited different behaviors: that is, sometimes it stalled, and others not (again, the same simple query as before). I had downloaded MonetDB on September 2007, and installed it on Mac OS X 10.4.0. I had no problem with that version. Thanks for looking into it, Irini

2 1