- users-list - monetdb.org

[MonetDB-users] Restore db on different platform / OID size
by Sander Kruseman 15 Apr '11

15 Apr '11

When attempting to restore a db dump, made on Windows 7 with 32 bits sized object ID's, on OSX with 64 bits sized object ID's, I run into errors. Like: CREATE FUNCTION: name 'env' already in use If I tweak the dump script by first dropping the functions that appear to exist, I get stuff like: ' from line 1 field 5 not inserted, expecting type real failed to import table Should what I try to do be possible anyway? Am I doing anything wrong? Thanks for your feedback. Sander Kruseman

2 1

[MonetDB-users] bulk upload data
by Nic Chidu 14 Apr '11

14 Apr '11

What way do you suggest for bulk uploading data? Thanks. Nic

3 2

[MonetDB-users] documentation on metadata statement
by Nic Chidu 13 Apr '11

13 Apr '11

Where can I find doc about metadata statements. Stuff like showing database, table, running environment. Thanks, Nic

2 1

[MonetDB-users] Series UPDATE statements will significantly decrease server performance
by Vladimir Vlach 13 Apr '11

13 Apr '11

Hi, We are experiencing performance issue with MonetDB when running UPDATE. We have a table with 1M rows and 3.000 columns. All columns except primary_id are FLOAT type. When I execute series of these statements the performance on first 30-35 statements is quite acceptable (less then 1s) but then suddenly it gets really slow. One UPDATE takes 30 seconds or more. After next 5-10 statements the performance decreases and one UPDATE takes 1 minute or longer. We run this with AutoCommit. The behavior is very persistent. UPDATE table SET column=column+1 WHERE primary_id='e1a7d7c7204c292e56e942dbdad118d3' ; Would you say MonetDB is not suitable for bulk UPDATE operations? Thanks, Vlad

1 0

[MonetDB-users] UPDATE INNER JOIN syntax
by Osmin Castillo 12 Apr '11

12 Apr '11

Hi guys: How can I to update a table field based on a inner join update? What's the syntax for MonetDB? thanks in advance Osmin Castillo

2 1

[MonetDB-users] How to recover Database after crash
by Vladimir Vlach 11 Apr '11

11 Apr '11

Hi, is there guide on how to recover DB after server crash? I get following error: root@monetdb:/var/monetdb5/dbfarm# monetdb release vlad_a-f taken database out of maintenance mode: vlad_a-f root@monetdb:/var/monetdb5/dbfarm# monetdb start vlad_a-f starting database 'vlad_a-f'... FAILED start: starting 'vlad_a-f' failed: database 'vlad_a-f' appears to shut itself down after starting, check merovingian's logfile for possible hints root@monetdb:/var/monetdb5/dbfarm# tail /var/log/monetdb/merovingian.log 2011-04-11 11:40:53 MSG merovingian[2477]: starting database 'vlad_a-f', up min/avg/max: 2s/1h/15h, crash average: 0.00 0.30 0.20 (18-12=6) 2011-04-11 11:40:53 MSG vlad_a-f[2783]: arguments: /usr/bin/mserver5 --config=/etc/monetdb5.conf --dbname=vlad_a-f --set merovingian_uri=mapi:monetdb://monetdb:50000/vlad_a-f --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/monetdb5/dbfarm/vlad_a-f/.mapi.sock --set monet_vault_key=/var/monetdb5/dbfarm/vlad_a-f/.vaultkey --set monet_daemon=yes 2011-04-11 11:40:53 ERR vlad_a-f[2783]: !FATAL: BBPinit: could not write bat/BBP.dir 2011-04-11 11:40:53 MSG merovingian[2477]: database 'vlad_a-f' (2783) has exited with exit status 1 2011-04-11 11:41:02 ERR control[2477]: (local): failed to fork mserver: database 'vlad_a-f' appears to shut itself down after starting, check merovingian's logfile for possible hints Is there a procedure how to recover this? Thanks, Vlad

3 3

[MonetDB-users] Temporary tables
by Charalampos Nikolaou 10 Apr '11

10 Apr '11

Hi, this page (http://monetdb.cwi.nl/SQL/Documentation/Table-Definition.html) states that only the LOCAL TEMPORARY scope of SQL is implemented. Considering the following interaction with in mclient, it does not seem that is implemented at all. Am I missing something here or it is a bug? sql>CREATE LOCAL TEMPORARY TABLE transaction_statements ( ctx INTEGER NOT NULL, subj INTEGER NOT NULL, pred INTEGER NOT NULL, obj INTEGER NOT NULL, expl BOOL NOT NULL ); operation successful sql>INSERT INTO transaction_statements (ctx, subj, pred, obj, expl) VALUES (0, 1, 2, 805306369, 1); 1 affected row (10.450ms) sql>SELECT * FROM transaction_statements; +-----+------+------+-----+------+ | ctx | subj | pred | obj | expl | +=====+======+======+ +-----+------+------+-----+------+ 0 tuples (1.490ms) sql>SELECT * FROM tmp.transaction_statements; +-----+------+------+-----+------+ | ctx | subj | pred | obj | expl | +=====+======+======+ +-----+------+------+-----+------+ 0 tuples (1.443ms) sql>SELECT * FROM tmp._tables; +------+------------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-------+------+-------+ | id | name | sche | query | type | syste | comm | reado | : : : ma_i : : : m : it_a : nly : : : : d : : : : ctio : : : : : : : : : n : : +======+========================+======================================================================+======+=======+======+=======+ | 5678 | transaction_statements | 2103 | null | 0 | false | 1 | false | +------+------------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-------+------+-------+ 1 tuple (2.013ms) Thanks, Babis

2 1

[MonetDB-users] build failed on undefined crypt references
by Sander Kruseman 10 Apr '11

10 Apr '11

Hi, Building from recent tar.bz2 (11.1.1) on Linux/x86_64 I get: libtool: link: gcc -g -O2 -Wno-declaration-after-statement -D_REENTRANT -o .libs/mclient mclient.o ReadlineTools.o -pthread -Wl,--export-dynamic ./.libs/libmcutil.a ../mapilib/.libs/libmapi.so /var/scratch/sander/MonetDB-11.1.1/common/stream/.libs/libstream.so ../../common/stream/.libs/libstream.so -lbz2 -lreadline -ltermcap -lcrypt -pthread -Wl,-rpath -Wl,/usr/local/lib ../mapilib/.libs/libmapi.so: undefined reference to `MD5' ../mapilib/.libs/libmapi.so: undefined reference to `SHA224' ../mapilib/.libs/libmapi.so: undefined reference to `RIPEMD160' ../mapilib/.libs/libmapi.so: undefined reference to `SHA512' ../mapilib/.libs/libmapi.so: undefined reference to `SHA384' ../mapilib/.libs/libmapi.so: undefined reference to `SHA256' ../mapilib/.libs/libmapi.so: undefined reference to `SHA1' What could be causing this? Any help is much appreciated. Sander Kruseman

2 1

[MonetDB-users] How to access MonetDB in Javascript without XRPC/XQuery?
by dsuh 09 Apr '11

09 Apr '11

Now that XRPC/XQuery is being removed from the MonetDB source, are there any options for client-side browser Javascript access to SQL queries? Would it end up looking like using the PHP interface in the web server side and passing the results to the client-side Javascript? Exactly how would this look like? I am interested in using HTML5/Javascript on the client-side to create a mashup of different data including GIS information from MonetDB. Can anyone suggest possible solutions? David Suh

2 1

[MonetDB-users] history support in mclient across sessions
by Charalampos Nikolaou 07 Apr '11

07 Apr '11

Hi, does mclient support saving of commands in a history file and accessing them again in a future mclient session? Currently, there is a -H option, but it doesn't seem to work (Mar2011 release). Thanks, Babis

2 1

[MonetDB-users] Killing long running queries
by Ramkumarlak 06 Apr '11

06 Apr '11

Is there any way to know the current running queries (like mysql's "show full processlist"). Also is there any way of killing long running queries. -- View this message in context: http://old.nabble.com/Killing-long-running-queries-tp31276295p31276295.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 5

Re: [MonetDB-users] [expath] Kudos on CXAN
by Christian Grün 05 Apr '11

05 Apr '11

…time for MonetDB to join up ;) ___________________________ On Tue, Apr 5, 2011 at 12:04 AM, Florent Georges <fgeorges(a)gmail.com> wrote: > On 3 April 2011 14:31, Chris Maloney wrote: > >> Florent, I suddenly realized this morning why you have the >> image that you do on the CXAN site -- it's a pronunciation >> hint! > > Good catch! > > For the others, the picture on the home page of CXAN > <http://cxan.org/beta/> is a painting of Paul Cezanne > <http://en.wikipedia.org/wiki/Paul_C%C3%A9zanne>, a French > painter. > > I hesitated with a self-portrait (this one is a portrait of > Victor Chocquet, dont ask me who he is), but I liked this one. > The hint is that the X is actually pronounced like a Z, to make > the name fluent (either a la French, with a C like in sailer, or > English style, with a C like in sea). > > Very good catch :-) > > -- > Florent Georges > http://fgeorges.org/ > > -- > You received this message because you are subscribed to the Google Groups "EXPath" group. > To post to this group, send email to expath(a)googlegroups.com. > To unsubscribe from this group, send email to expath+unsubscribe(a)googlegroups.com. > For more options, visit this group at http://groups.google.com/group/expath?hl=en.

1 0

[MonetDB-users] query dit not produce result set
by Bas Kaptijn 04 Apr '11

04 Apr '11

Hi, with the MAR2010 release (x64 windows through installer) (and also the release Feb2010-SP2) I get an exception running a particular query, whilst a slightly modified one with the expected result (except for the naming of a result column) does not. I narrowed the problem down to this: sql>CREATE TABLE dbg (a INT, b INT); operation successful sql>INSERT INTO dbg (a,b) VALUES (10,10); 1 affected row (32.410ms) sql>SELECT a as d, SUM(b), (2 * (SUM(b) / (SELECT 2))) as f FROM dbg GROUP BY d; +------+------+------+ | d | L451 | f | +======+======+======+ | 10 | 10 | 10 | +------+------+------+ 1 tuple (8.841ms) sql>SELECT a as d, SUM(b) as e, (2 * (SUM(b) / (SELECT 2))) as f FROM dbg GROUP BY d; sql> The second select query does not produce a result set (as stated in the SQLException I get when doing this through JDBC). The monetdb client just shows a new prompt without stating any warning/error. The query works when I remove the "as e" ... or the "select 2" but in the real case I get this problem this select does much more than just selecting the number 2. I think this is a bug. Or am I doing something that is not (yet) supported in MonetDB? Thanks Bas Kaptijn

2 1

[MonetDB-users] for help(software information is added)
by Xianmin Liu 31 Mar '11

31 Mar '11

Hi, Sorry for missing the software information in last letter. I used MonetDB4-XQuery-Installer-x86_64-20110324 version, it seems to be the latest. If you have read my last letter, pls ignore followings. I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following. ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB. !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy hand helms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec Can you help me? How can I fix it? Thanks a lot. -- Xianmin Liu

2 2

[MonetDB-users] DROP TABLE IF EXISTS <table>;
by Michael Sioutis 31 Mar '11

31 Mar '11

Hello, I want to prevent an SQLException from occurring for tables that do not exist in my database. For example: sql>DROP TABLE "locked"; SQLException:sql.catalog:DROP TABLE: no such table 'locked' It seems (and is documented) that there is no such thing like "IF EXISTS" in MonetDB. Is there a way to circumvent the SQLException? Mike

5 8

Re: [MonetDB-users] for help(software information is added)
by Peter Boncz 31 Mar '11

31 Mar '11

Xianmin, On Windows we only build with 32bits oids (even with 64bits pointers). When working with 32-bits offsets string heaps are restricted to 16GB. For large data, we recommend 64-bits linux. Then grab a oid64 rpm. Peter From: Xianmin Liu [mailto:xianmliu@gmail.com] Sent: donderdag 31 maart 2011 11:09 To: monetdb-users(a)lists.sourceforge.net; Peter Boncz Cc: Wenfei Fan; Gao Cong; Nan Tang; Anastasios (Tasos) Kementsietsidis Subject: for help(software information is added) Hi, Sorry for missing the software information in last letter. I used MonetDB4-XQuery-Installer-x86_64-20110324 version, it seems to be the latest. If you have read my last letter, pls ignore followings. I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following. ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB. !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy hand helms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec Can you help me? How can I fix it? Thanks a lot. -- Xianmin Liu

1 0

[MonetDB-users] for help
by Xianmin Liu 31 Mar '11

31 Mar '11

Hi, I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following. ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB. !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy hand helms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec Can you help me? How can I fix it? Thanks a lot. -- Xianmin Liu

2 1

[MonetDB-users] DROP TABLE IF EXISTS <table>;
by Michael Sioutis 30 Mar '11

30 Mar '11

Hello, I want to prevent an SQLException from occurring for tables that do not exist in my database. For example: sql>DROP TABLE "locked"; SQLException:sql.catalog:DROP TABLE: no such table 'locked' It seems (and is documented) that there is no such thing like "IF EXISTS" in MonetDB. Is there a way to circumvent the SQLException? Mike

1 0

[MonetDB-users] How to determine size of a table and its indexes
by Nekas 30 Mar '11

30 Mar '11

In Oracle and MySQL for example, we can obtain the space occupied in the DB by tables and indexes in the user space. Is this possible in MonetDB? If so, how can I do it? I often need it, so it would be quite helpful to me. Thanks you in advance, Ana T.

1 2

[MonetDB-users] Announcement: New Mar2011 feature release of MonetDB suite
by Sjoerd Mullender 28 Mar '11

28 Mar '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Mar2011 feature release of the MonetDB suite of programs. This release marks the last planned release of MonetDB4 and Pathfinder, and hence the last planned release of MonetDB/XQuery. Development of the Pathfinder compiler continues at the University of Tübingen. The next release, currently planned for April 2011, should be a smooth upgrade from this release, even if MonetDB/XQuery is installed (which will of course then not be upgraded). This release also marks the first release of the new way of building the MonetDB suite. For details on both these issues, and on the other features and bug fixes of this release, please see the release notes at <http://monetdb.cwi.nl/Development/Releases/Mar2011/>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] Recycler Optimizer
by Henry Addington 24 Mar '11

24 Mar '11

Hello: I understand some of the optimizers/pipelines are experimental as indicated in the monetdb5.conf file. Has anyone used the Recycle pipeline for Skyserver? Would you mind sharing your experience, in terms of performance improvement and whether it can bet used in a production environment? We are using the Oct2010-SP1 release for a db that is read-only. I would guess if there are issues, I would think the problems would likely show up in read/write systems. Any feedback would be appreciated.... It looks like the Recycle pipeline for Skyserver pipeline is based on the Nov-2009 pipeline. Has anyone tried the recycler optimizer with the Feb2010 default pipeline? I was going to add "recycle" between "deadcode" and "reduce" in the default pipeline and see if it would work. Thanks. Henry

3 4

[MonetDB-users] ask for help
by Xianmin Liu 23 Mar '11

23 Mar '11

Hi, It's me again. Recently, I was doing some experiments using MonetDB/XQuery. And I asked one question about errors when loading XML data some days ago, Peter solved my problem very kindly. Now, there is an error again, pls help me. Following is my setting, CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB System: Ubuntu 10.04 64bits MonetDB: setup by deb, serverV4.40.3 GDKv1.40.3, MonetDB/XQuery module 0.40.3 loaded. I used pf:add-doc to add data, the data size is 3GB(generated by xmark generator with factor 30), error returned: ERROR: !xquery> -- Xianmin Liu

2 1

[MonetDB-users] User-defined MAL functions
by Petros Tsialiamanis 22 Mar '11

22 Mar '11

Hi, I have some questions about using user-defined MAL functions I would like to create a function in MAL. 1) Is it possible that the function can take variable number of BATs as argument like in the case of algebra.joinPath(l:bat[:any,:any]... ) ? 2) If yes, how I can get the number of BATs passed as argument and 3) how I can get the n-th element of the argument? Thank you in advance Petros

2 1

[MonetDB-users] Some questions
by Henning Grosch 22 Mar '11

22 Mar '11

Hi everyone, I have a some questions about using monetDB. First thing I noticed when using extract on a timestamp field I can only cast the timestamp as year, month, day and so on. Is it somehow possible to extract the week of the year? Week starts with monday. Another thing is, can i use if statements in a query like i could in mysql? And the last one, i already searched in the archiv and found a post about win 32 odbc driver and column names which appear as questionmarks or other symbols like when using a wrong encoding, does anybody has the same problem or knows a solution? Regards Henning

1 0

Re: [MonetDB-users] ask for help
by Peter Boncz 21 Mar '11

21 Mar '11

Hi, You cannot load the 1G of data in the 32-bits windows version on MXQ. Simply won’t fit the virtual address space. Use the 64-bits version (best: use the 64-bits Linux version). Peter From: Xianmin Liu [mailto:xianmliu@gmail.com] Sent: Monday, March 21, 2011 2:11 PM To: monetdb-users(a)lists.sourceforge.net Cc: Nan Tang; Wenfei Fan Subject: [MonetDB-users] ask for help Hi, I got some errors when using MonetDB/XQuery, could you pls give me some help? I was running experiments over large XML data using MonetDB/XQuery. My hardware: CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB MonetDB: Server v4.32.0 based on GDK v1.32.0, Compiled for i686-pc-win32/32bit, MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra') When I try to load xmark1G.xml (generated by xmark generator with factor 10) by command 'pf:add-doc("H:\lxm\1G\xmark1000M.xml","xmark1G.xml")<>' Return errors: ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 278712278 for 03\373theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: CMDshred_url: operation failed. -- Xianmin Liu

1 0

[MonetDB-users] ask for help
by Xianmin Liu 21 Mar '11

21 Mar '11

Hi, I got some errors when using MonetDB/XQuery, could you pls give me some help? I was running experiments over large XML data using MonetDB/XQuery. My hardware: CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB MonetDB: Server v4.32.0 based on GDK v1.32.0, Compiled for i686-pc-win32/32bit, MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra') When I try to load xmark1G.xml (generated by xmark generator with factor 10) by command 'pf:add-doc("H:\lxm\1G\xmark1000M.xml","xmark1G.xml")<>' Return errors: ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 278712278 for 03\373theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: CMDshred_url: operation failed. -- Xianmin Liu

1 0

[MonetDB-users] EXTRACT(SECOND FROM timestamp) gives 0 seconds
by spamme.gs＠googlemail.com 18 Mar '11

18 Mar '11

Hello, I can't understand the following behaviour : Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2010-SP1) Database: MonetDB v5.22.3, 'tradedb' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select extract(second from timestamp '2011-3-10 9:30:42.246'); +---------------------+ | second_single_value | +=====================+ | 0.246 | +---------------------+ Whoops. Well maybe monetdb can EXPLAIN what's goin on sql>explain select extract(second from timestamp '2011-3-10 9:30:42.246'); | function user.s2_1{autoCommit=true}():void; | | sql.mvc(); | | _4 := mtime.milliseconds(2011-03-10 09:30:42.246:timestamp); | | sql.exportValue(1,".","second_single_value","decimal",9,3,8,_4,""); | | end s2_1; | Ok ... Looks like this function is declared here: (modules/atoms/mtime.mal) 413 command milliseconds(d:daytime) :int 414 address MTIMEdaytime_extract_milliseconds And digging into the source of _that_ function shows us that it's just returning the millisecond part. So, any chance of a quick fix here? It looks like I'm not the first to ask this question: http://sourceforge.net/mailarchive/message.php?msg_id=23238837 but that poor guy was completely ignored.

2 1

[MonetDB-users] LC_ALL=C in monetdb-install.sh breaks RDoc
by Lazaros Koromilas 18 Mar '11

18 Mar '11

Hello, all! I did the following on an Arch Linux yystem: MonetDB-Oct2010-SuperBall-SP1> ./monetdb-install.sh --enable-xquery ... make[8]: Entering directory `/var/tmp/MonetDB-1299754542/MonetDB-client/clients/ src/ruby/adapter' mkdir -p /home/myuser/MonetDB/lib/ruby/gems/1.9.1 gem install --local --install-dir /home/myuser/MonetDB/lib/ruby/gems/1.9.1 --forc e --rdoc activerecord-monetdb-adapter-0.1.gem ERROR: While executing gem ... (Encoding::UndefinedConversionError) U+2019 from UTF-8 to US-ASCII Successfully installed activerecord-monetdb-adapter-0.1 1 gem installed Installing ri documentation for activerecord-monetdb-adapter-0.1... Installing RDoc documentation for activerecord-monetdb-adapter-0.1... make[8]: *** [install-exec-local-activerecord-monetdb-adapter-0_1_gemspec] Error 1 ... Setting LC_ALL to UTF-8 fixes my problem. --- monetdb-install.sh.orig 2011-03-10 13:34:58.000000000 +0200 +++ monetdb-install.sh 2011-03-10 13:35:05.000000000 +0200 @@ -108,7 +108,7 @@ : ${FETCHCOMMAND:="wget --progress=dot:mega -t 5 --passive-ftp -O \"\${FILE}\" \"\${URI}\""} # make sure we get English error messages and so on -export LC_ALL=C +export LC_ALL="en_US.UTF-8" # parse command line options ORGARG=$*

2 3

[MonetDB-users] MonetDB/XQuery order by problem
by Rob Fynes 13 Mar '11

13 Mar '11

Hello, I'm trying to produce a sorted list of results with MonetDB/XQuery using 'order by', with a query that goes something like this: for $cname in pf:collections()/text() return if (local:is-date-between($cname, **start_date_here**, **end_date_here**)) then for $s in pf:collection($cname)//Dublin/stations[time/hour=13 and time/minute>=22 and time/minute<=25] order by $s/timeOfDay return $s else () ...the 'is-date-between' is just some custom function to check if the $cname (collection names are dates) is between the start and end dates supplied to the function. However, the problem is with the nested for loop. I want each $s that satisfies the XPath expression to be returned, but ordered by $s/timeOfDay. When I submit this query, I get the following error: ERROR = !type error: [err:XPTY0004] Orderspec requires at most one atomic item (given: string*). I've tried a number of variations on this, but the error is always the same. If I remove the order by clause, the results will return correctly, but in the unsorted order that they appear in the database. I'm still a beginner with XQuery, so I'm unsure where I'm going wrong, and documentation and tutorials have been unhelpful so far. Any help is appreciated! Regards, -Rob

2 3

[MonetDB-users] Copy records into table failed
by Tibor Barna 11 Mar '11

11 Mar '11

Hello, I am trying to import data from MySQL into MonetDB usig the bulk loader, the csv has ~11GB, 163Mil Rows. When running: sql>copy 163715240 records into sys.clicks from '/data/csv_monet_dump/clicks.tbl' using delimiters '|',' more>', '"' null as ''; I always get this error message: SQLException:importTable:failed to import table ERROR: HEAPextend: failed to extend to 654860960 for 32/3231tail ERROR: TABLETcreate_bats: Failed to create bat of size 163715240 merovingian.log: 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) fails, try to free up space [memory in use=41411528,virtual memory in use=12310675456] 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) result [mem=32087208,vm=12310675456] I am using 64bit openSUSE 11.3, 12GB RAM, 4GB Swap. Does anybody know how can i overcome this error? Any help would be most welcome :) Cherers, Tibor. -- View this message in context: http://old.nabble.com/Copy-records-into-table-failed-tp31115783p31115783.ht… Sent from the monetdb-users mailing list archive at Nabble.com.

3 7

[MonetDB-users] Get MAL from SQL
by Kusber, Michael 10 Mar '11

10 Mar '11

Hi, is there a possibility to see the MAL code generated from a specific SQL statement? Kind regards, Michael

3 2

[MonetDB-users] Are these bugs？
by tangray 09 Mar '11

09 Mar '11

hi all when i use c++ pragrm to insert some data into a table,the program crashes,the message is below: so i use gdb to debug the bug, i find that these places may be bugs： 1. out of bounds access file mapi.c function: mapi_param_store 2356 strncpy(hdl->query + k, p, q - p); 2357 k += q - p; 2358 hdl->query[k] = 0; q-p is equal or bigger than lim - k ,the strncpy will hava a out of bounds access. this may be the reason to crash. 2. out of bounds access file:mapi.c function: mapi_param_store 2411 case MAPI_DOUBLE: 2412 checkSpace(20); 2413 sprintf(hdl->query + k, "%.17g", *(double *) src); 2414 break; sprintf(hdl->query + k, "%.17g", *(double *) src) , sprintf function with "%.17g" format may produces a string longer than 20,eg： double d = 0.000200000000000001; double d = 0.000200000000000001; printf("%.17g",d); the result is : 0.00020000000000000101 so i think, in some paticular condition ,this will hava a out of bounds access too. I'm working with MonetDB Server v5.22.3, based on kernel 1.40.3 over fedora 12.

2 1

[MonetDB-users] lose some datas of a table's one field, how can this happen?
by tangray 08 Mar '11

08 Mar '11

Hi when execute sql : select * from td_ind_security_held where id = 1560000 in DbVisualizer 7.1.1,it appears "protocol violation: unexpected line in data block" td_ind_security_held has 30 columns(1 int,6 varchar, others decimal) and 7219618 rows. so i count field's records one by one , all fields's records are equal to 7219618 except the allgain's records, it is only 658337. it obviously lost some data. when program inserted the data into the table , the monet server carshed out some times, can this be the reson for the lose? I'm working with MonetDB Server v5.22.3, based on kernel 1.40.3 (downloaded msi from sourceforge) over Windows XP SP3.

2 1

Re: [MonetDB-users] MonetDB/XQuery JDBC result issue
by Wouter Alink 04 Mar '11

04 Mar '11

Hello Rob, Yes, the good thing about the JDBC interface is that the result is streamed to the client, so that there is no large memory-consumption on the client-side. However, be aware that, due to the way MonetDB works internally, the whole result is first computed and kept in memory at the server, before it is shipped to the client. This will for most use-cases not be a problem, but if you are calculating huge result sets it might. If you only fetch the first 10 rows of a result at the client, you may want to wrap a "subsequence(<your-query>, 1, 10)" around your query. Kind regards, Wouter On Fri, Mar 4, 2011 at 3:42 PM, Rob Fynes <monetdbmail(a)gmail.com> wrote: > Hi Wouter, > > Thanks for the response. You were right about that, it's working fine now. > I was thrown slightly when I saw the MonetDB documentation that said "The > result comes back as a single-row single-column multi-line string result". I > figured that the 'entire' set of results would be stuffed into one row and > one column in the result set object. In retrospect, this wouldn't make much > sense, since you could end up with a *really* big string! > > Thanks, > -Rob > > > On Wed, Mar 2, 2011 at 1:46 PM, Wouter Alink <wouter.alink(a)gmail.com>wrote: > >> Dear Rob Fynes, >> >> You have already provided the answer in your email. XQuery specifies that >> a result is actually a sequence of items. The "XQueryResult"-node has only >> been added to convert the actual result sequence into a valid XML fragment. >> Using XQuery over JDBC works like SQL; you can iterate over the results. >> There is no artificial XQueryResult-node added. >> >> For example: >> >> ResultSet rs = conn.executeQuery("...."); >> while(rs.next()) { >> .. do something with the result .. >> } >> >> With kind regards, >> Wouter Alink >> >> >> >> On Tue, Mar 1, 2011 at 10:13 PM, Rob Fynes <monetdbmail(a)gmail.com>wrote: >> >>> Hello, >>> >>> I've been having an issue with MonetDB/XQuery using the JDBC driver. >>> Essentially, when I supply an *XPath* query such as the following: >>> >>> pf:collection("collection_name")//stations[date/year>=2010] >>> >>> ... using the mclient utility, there's no problem. The result is an >>> <XQueryResult> root node containing all the relevant 'stations' nodes as >>> requested by the query. >>> However, if I issue the same query to the server from a Java application >>> using the JDBC driver for MonetDB, the string result that is returned merely >>> contains one 'stations' node (the first one that gets processed, I think). >>> >>> My guess is that the resultset (of 'stations' nodes) is not being wrapped >>> in a root node, as is the case when querying using the mclient utility. >>> But I'm unsure if this due to a bug, or if there's something that I'm not >>> doing correctly. >>> >>> I'm also aware that this issue can be resolved by modifying the query >>> (when using JDBC) to something such as: >>> >>> <result> { >>> pf:collection("collection_name")//stations[date/year>=2010] >>> } </result> >>> >>> ... but I have reasons for not wanting to do this. >>> Any help is much appreciated. If anyone needs more information, just let >>> me know (and if it makes any difference, I'm running a local server >>> instance). >>> >>> Cheers, >>> -Rob >>> >>> >>> ------------------------------------------------------------------------------ >>> Free Software Download: Index, Search & Analyze Logs and other IT data in >>> Real-Time with Splunk. Collect, index and harness all the fast moving IT >>> data >>> generated by your applications, servers and devices whether physical, >>> virtual >>> or in the cloud. Deliver compliance at lower cost and gain new business >>> insights. http://p.sf.net/sfu/splunk-dev2dev >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >>> >> >

1 0

[MonetDB-users] MonetDB/XQuery JDBC result issue
by Rob Fynes 02 Mar '11

02 Mar '11

Hello, I've been having an issue with MonetDB/XQuery using the JDBC driver. Essentially, when I supply an *XPath* query such as the following: pf:collection("collection_name")//stations[date/year>=2010] ... using the mclient utility, there's no problem. The result is an <XQueryResult> root node containing all the relevant 'stations' nodes as requested by the query. However, if I issue the same query to the server from a Java application using the JDBC driver for MonetDB, the string result that is returned merely contains one 'stations' node (the first one that gets processed, I think). My guess is that the resultset (of 'stations' nodes) is not being wrapped in a root node, as is the case when querying using the mclient utility. But I'm unsure if this due to a bug, or if there's something that I'm not doing correctly. I'm also aware that this issue can be resolved by modifying the query (when using JDBC) to something such as: <result> { pf:collection("collection_name")//stations[date/year>=2010] } </result> ... but I have reasons for not wanting to do this. Any help is much appreciated. If anyone needs more information, just let me know (and if it makes any difference, I'm running a local server instance). Cheers, -Rob

2 1

[MonetDB-users] draft message
by Martin Kersten 01 Mar '11

01 Mar '11

Dear MonetDB users and developers, The preparations for the March 2011 release are on target. This release will feature a much simplified installation and compilation process using a unified bootstrap/configure/make/install. Ofcourse, installers and RPMs are also provided. Furthermore, the March 2011 release will be the freeze of the MonetDB XQuery source tree. It will remain downloadable from the website, but no active development or maintenance is foreseen. In April 2011 a MonetDB/SQL feature release will mark the split of the two code bases. From that point onward, we will primarily be concerned with SQL and its enhancements. Saying goodbye to the very old mserver4 code base, also triggered a major code renovation activity. The MonetDB/SQL code base will be cleaned from legacy code, 1-time experimental code, and all that is not strictly needed for the intended functionality. We expect that this cleanup will simplify maintenance in the future. The results of the renovation step will become visible in the fall of 2011. We plan for a smooth transition step for existing SQL environments. However, under the hood, we will change the MAL and kernel library interfaces significantly. To assess the consequences in the community we would like to learn about projects that directly address the MAL or GDK layer. Please send us an email at info(a)monetdb.org with the project highlights. regards, Martin Kersten

1 0

[MonetDB-users] protocol violation: unexpected line in data block
by tangray 28 Feb '11

28 Feb '11

Hi when execute sql : select * from td_ind_security_held where id = 1560000 in DbVisualizer 7.1.1,it appears "protocol violation: unexpected line in data block" td_ind_security_held has 30 columns(1 int,6 varchar, others decimal) and about 7 million rows. I'm working with MonetDB Server v5.22.3, based on kernel 1.40.3 (downloaded msi from sourceforge) over Windows XP SP3.

2 3

Re: [MonetDB-users] Query time takes equally long on whole
by dsuh 24 Feb '11

24 Feb '11

We have experienced big improvements from Nov-2009SP2 to Oct-2010SP1 as well except for views. It appears that the mitosis optimizer has problems with views and queries on them run slower than without it. (~9 sec slower on 1 min queries on views and ~20 sec slower on 2 min queries on views using default_pipe rather than no_mitosis_pipe or nov2009_pipe) The mitosis optimizer was the big change in the optimizer pipeline between these versions. Is it possible that this is still from the bug?: http://bugs.monetdb.org/show_bug.cgi?id=2472 Can anyone tell me if there are any plans for optimization improvements (either mitosis, commonTerms, or otherwise) with queries on views? David Suh > > Hello Henry, > Thank you for your reply! See inline. > > On Thu, Feb 24, 2011 at 7:54 AM, Henry Addington <hs.addington(a)gmail.com > >wrote: > > > I am no technical expert on MonetDB. But, we have been using MonetDB for > > more than a year now. So, I can give you couple of suggestions > > (or workarounds) that might help based on our experience. We are using a > > third-party application that generates queries and sends them to MonetDB > > which has a transaction table with more than 1.6billion records and > growing > > (timeseries data like yours..) linked to dimensional tables. The kinds of > > queries that are being generated are almost exact type that you are > showing, > > some sort of an aggregation query using group by and filtered with where > > clause. > > > > One thing that we noticed with queries with joins is that inner join > seems > > to be significantly faster than left/right joins even if the results are > no > > different between the 3 types of joins. I know that might not be an > option > > for you. But, it works for our use case. > > > > I've changed my queries into two separate ones: one for the total > timescope, and one for (occasionally) getting the results spread over the > interval for one item. The new total timescope query is faster on less time > as scope, and as fast as before on the total 24 hour scope, so definitely > an > improvement for me. > > > > > > Another thing that we noticed is if you add constraints, ie primary and > > foreign keys, you will get better performance. I think you might already > > have them in your tables though.... > > > > I do have them, but i didn't add them in my create statement. I''m also > going to try if this adds extra speed. > > > > > > We are in the process of testing Oct-2010SP1 release so we can upgrade > > our Nov-2009SP2 environment. We are seeing significant performance > > improvement in most of our typical queries and much smaller memory > footprint > > with the Oct-2010SP1 release. But, we see some performance degradation > with > > queries based on views instead of tables. You might want to try the > Nov-2009 > > optimizer and see if that helps. You can do this by specifyng the > Nov-2009 > > optimizer in the monetdb5.conf file. > > > > This didn't show any difference on the original query, but thanks for > mentioning. I might need it in the future. > > Kind regards, > Rob Berentsen > > > > > On Wed, Feb 23, 2011 at 4:24 AM, Rob Berentsen <rhberentsen(a)gmail.com > >wrote: > > > >> Hello, > >> > >> I'm new to monetdb and installed it two weeks ago to do some tests. I > was > >> very much impressed with the speed on large tables, but in some of my > latest > >> tests regarding subparts of these tables based on a unix timestamp the > time > >> was the same as querying the whole table. Does anyone know what I can do > to > >> get higher speeds on queries in monetdb that only use part of the table, > >> like partition pruning, explicit foreign keys or unique statements, > >> indexes or some other way? > >> > >> I have the following two tables and number of records: > >> > >> CREATE TABLE table1 (T1vid INT NOT NULL, T1Field1 VARCHAR(20) NOT > >> NULL, T1Field2 TINYINT NOT NULL); > >> This table has about 400.000 records. The records are non time bounded, > >> it's just a list. The T1vid is an increased ID, but not generated in > >> momentdb at the moment, but in mysql > >> > >> CREATE TABLE table2 (T2mid INT NOT NULL, T2vid INT NOT NULL, T2timestamp > >> INT NOT NULL, T2Field1 INT NOT NULL, T2Field2 INT NOT NULL); > >> This table has about 1.000.000 records and a unix_timestamp field. It > get > >> a few new records every second and records older then 24 hours are > deleted. > >> The T2mid-field refers to some other table that's not in momentdb's > >> database, so just an integer. The T2vid-field however refers to the > >> T1vid-field in table1 (a foreign key, but i didn't define it that way). > >> > >> The following query takes about 3,5 seconds, it doesnt't have the > >> timestamp included in the WHERE-clause: > >> SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / > 3600) > >> as interval > >> FROM table2 AS t2 > >> LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid > >> WHERE t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique > >> v.T1vid ID's*) > >> GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY > interval > >> DESC; > >> Resulting is a list of the 15 v.T1vid ID's times the number of > interval's. > >> t2.T2timestamp is now devided by 3600 and grouped on this result, so > each > >> set of 15 v.T1vid ID's would reflect 1 hour, returning max 24x15 rows. > >> ** > >> The following query also takes about 3,5 seconds, but it does have the > >> timestamp included in the WHERE-clause. It only needs to access 1 hour > of > >> data instead of the whole 24 hours, and groups it into intervals of 5 > >> minutes (300 sec): > >> SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / > 300) > >> as interval > >> FROM table2 AS t2 > >> LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid > >> WHERE* t2.T2timestamp BETWEEN 1298360000 AND 1298363600* > >> AND t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique > v.T1vid > >> ID's*) > >> GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY > interval > >> DESC; > >> > >> t2.T2timestamp is now devided by 300, so each set of 15 v.T1vid ID's > would > >> reflect 5 minuntes, returning max 12x15 rows. > >> > >> I would very much appreciate any help or hints; please let me know any > >> question you might have. > >> > >> > >> Kind regards, > >> Rob Berentsen > >> > >> > >> > ------------------------------------------------------------------------------ > >> Free Software Download: Index, Search & Analyze Logs and other IT data > in > >> Real-Time with Splunk. Collect, index and harness all the fast moving IT > >> data > >> generated by your applications, servers and devices whether physical, > >> virtual > >> or in the cloud. Deliver compliance at lower cost and gain new business > >> insights. http://p.sf.net/sfu/splunk-dev2dev > >> _______________________________________________ > >> MonetDB-users mailing list > >> MonetDB-users(a)lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/monetdb-users > >> > >> > > > > > > > ------------------------------------------------------------------------------ > > Free Software Download: Index, Search & Analyze Logs and other IT data in > > Real-Time with Splunk. Collect, index and harness all the fast moving IT > > data > > generated by your applications, servers and devices whether physical, > > virtual > > or in the cloud. Deliver compliance at lower cost and gain new business > > insights. http://p.sf.net/sfu/splunk-dev2dev > > _______________________________________________ > > MonetDB-users mailing list > > MonetDB-users(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > > > >

1 0

[MonetDB-users] monetdb on ubuntu 10.10 - xquery results displaying only in single line!
by sagarjauhari 24 Feb '11

24 Feb '11

Hello, my xquery results are displaying only in a single line. The hello world example on http://monetdb.cwi.nl/XQuery/QuickTour/DOCMGT/ displays result correctly as: > doc("HelloWorld.xml") <?xml version="1.0" encoding="utf-8"?> <doc> <greet kind="informal">Hi </greet> <greet kind="casual">Hello </greet> <location kind="global">World</location> <location kind="local">Amsterdam</location> </doc> But this: xquery>for $a in (1 to 3) return <number>{$a}</number> outputs this: <XQueryResult><number>1</number><number>2</number><number>3</number></XQueryResult> and for the file :test.xq: (1, 42.0, "Hello World", <node attr="value">test</node>) the xquery mclient -lx -f xml test.xq outputs <?xml version="1.0" encoding="utf-8"?> <XQueryResult>1 42.000000 Hello World<node attr="value">test</node></XQueryResult> Any suggestions? Thanks in advance -- View this message in context: http://old.nabble.com/monetdb-on-ubuntu-10.10---xquery-results-displaying-o… Sent from the monetdb-users mailing list archive at Nabble.com.

2 4

[MonetDB-users] Stored Procedures in MonetDB
by Irini Fundulaki 24 Feb '11

24 Feb '11

Hi all, would it be possible to let us know whether MonetDB/SQL supports stored procedures and provide us with pointers to the features supported (e.g., recursion). Many Thanks Irini ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

[MonetDB-users] Query time takes equally long on whole and just small part of table
by Rob Berentsen 24 Feb '11

24 Feb '11

Hello, I'm new to monetdb and installed it two weeks ago to do some tests. I was very much impressed with the speed on large tables, but in some of my latest tests regarding subparts of these tables based on a unix timestamp the time was the same as querying the whole table. Does anyone know what I can do to get higher speeds on queries in monetdb that only use part of the table, like partition pruning, explicit foreign keys or unique statements, indexes or some other way? I have the following two tables and number of records: CREATE TABLE table1 (T1vid INT NOT NULL, T1Field1 VARCHAR(20) NOT NULL, T1Field2 TINYINT NOT NULL); This table has about 400.000 records. The records are non time bounded, it's just a list. The T1vid is an increased ID, but not generated in momentdb at the moment, but in mysql CREATE TABLE table2 (T2mid INT NOT NULL, T2vid INT NOT NULL, T2timestamp INT NOT NULL, T2Field1 INT NOT NULL, T2Field2 INT NOT NULL); This table has about 1.000.000 records and a unix_timestamp field. It get a few new records every second and records older then 24 hours are deleted. The T2mid-field refers to some other table that's not in momentdb's database, so just an integer. The T2vid-field however refers to the T1vid-field in table1 (a foreign key, but i didn't define it that way). The following query takes about 3,5 seconds, it doesnt't have the timestamp included in the WHERE-clause: SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / 3600) as interval FROM table2 AS t2 LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid WHERE t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique v.T1vid ID's*) GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY interval DESC; Resulting is a list of the 15 v.T1vid ID's times the number of interval's. t2.T2timestamp is now devided by 3600 and grouped on this result, so each set of 15 v.T1vid ID's would reflect 1 hour, returning max 24x15 rows. ** The following query also takes about 3,5 seconds, but it does have the timestamp included in the WHERE-clause. It only needs to access 1 hour of data instead of the whole 24 hours, and groups it into intervals of 5 minutes (300 sec): SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / 300) as interval FROM table2 AS t2 LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid WHERE* t2.T2timestamp BETWEEN 1298360000 AND 1298363600* AND t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique v.T1vid ID's*) GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY interval DESC; t2.T2timestamp is now devided by 300, so each set of 15 v.T1vid ID's would reflect 5 minuntes, returning max 12x15 rows. I would very much appreciate any help or hints; please let me know any question you might have. Kind regards, Rob Berentsen

2 2

[MonetDB-users] MAL changes in March2011
by Mike De-La-Columnar 22 Feb '11

22 Feb '11

Hi, It seems like the whole MAL language is being upgraded. (the use of columns instead of BATs) are those changes just cosmetic, or is there something else that will benefit from it? also, will the old MAL work in the new version? -- View this message in context: http://old.nabble.com/MAL-changes-in-March2011-tp30985134p30985134.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] rdf.sql missing from Mar2011 branch
by Babis Nikolaou 18 Feb '11

18 Feb '11

Hello, the rdf.sql file for creating the schema of an RDF database is missing from the branch of Mar2011. Could you add it please? Thanks, Babis

4 12

[MonetDB-users] Kernel / FS tuning
by Franck Routier 18 Feb '11

18 Feb '11

Hi, I was wondering if anyone has suggestions about what tweaks might best suit MonetDB's needs regarding : - (linux) kernel parameters : memory management, ... - FS (type and parameters) : block size, write barriers, ... Is there any kind of best practice or even some thoughts on this ? Thanks in advance for your inputs, Franck

1 0

[MonetDB-users] Problem starting monetdbd after installing MonetDB
by Babis Nikolaou 10 Feb '11

10 Feb '11

Hi all, i get the following error after installing MonetDB from CVS (tip) and running monetdb/d. $ monetdbd cannot open config file (null) unable to retrieve startup status $ monetdb cannot open config file (null) As opposed to the previous version i had (a version of CVS during November), now there is no /etc subdirectory inside the MonetDB installation directory, where the configuration files lie. I also searched for any configuration files inside the build directory, in case sth had gone wrong during installation, but nothing has been generated. I also noticed that the installation procedure when compiling from CVS has been changed. Do i miss something? I just run bootstrap, configure, make, and last make install. One last remark: in the manpage of monetdbd (and monetdb), the configuration file in question is referenced as @MONETDB5_CONFFILE@. Shouldn't had been mapped to a filename? Thanks, Babis

3 3

[MonetDB-users] Bug in EXTRACT?
by Alex Bo. 06 Feb '11

06 Feb '11

Hi, There seems to be a bug in the October SP. If you run this query: SELECT EXTRACT(YEAR FROM f1 .a1) as a2, EXTRACT(YEAR FROM f1 .a1) as a3 FROM (select '1996-02-01' as a2, '1996-07-01' as a1) f1 it returns: 1996, 1996-02-01 Instead of: 1996, 1996 The second column is returned as timestamp instead of int. Also, this does not happen if the first aliases (a2, a3) are changed to different names, nor does it happen if the two EXTRACTS are not identical. If one is YEAR and the second is MONTH it works, but if both are MONTH the same bug happens. I tried it also on the August build and it works fine there. Any ideas? -- View this message in context: http://old.nabble.com/Bug-in-EXTRACT--tp30855882p30855882.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

Re: [MonetDB-users] query slowdown with concurrent write
by Bennett Hiles 04 Feb '11

04 Feb '11

Hi Martin and Stefan, Thanks for the quick response, and sorry if this created another thread, I'm not familiar with how to use this mailing list. I am using: MonetDB v5.22.3 on ubuntu 9.04 64bit Queries and data loading use the php library. To further explain my use case, I am planning on having monetdb be updated about once a minute with 1000-10000 new rows of data. At the same time I will be having as many clients as possible making queries on the database. I have so far seen that 5 clients can be handled with satisfiable response times. Is this a suitable application for monetdb? The query times I have seen so far are great. My problems are handling the queries with concurrent writes and if possible allowing for more clients to make simultaneous queries. Re: Martin's questions: I tried using the recycle optimizer in the default pipe and my queries didn't slow down, which is awesome. But, my problem now is that the values aren't actually added to the dataset until an insert transaction occurs when I am not querying the database. Therefore, if I continuously query the database then the inserts are never seen, but if I stop querying for a second and another insert happens the database is updated. I am fine with some delay in the insert time, but is there anyway I could have it executed slowly alongside the queries, or am I too greedy :) The pipe I created is the same as the default pipe with the recycler optimizer added: bennett_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,recycle,reduce,dataflow,history,multiplex,garbageCollector My schema and and an example query are: CREATE TABLE my_test_table ( account_id bigint, dim1 bigint, dim2 bigint, dim3 bigint, dim5 varchar(50), dim6 varchar(50), dim7 varchar(256), dim8 bigint, dim9 varchar(256), dim10 varchar(256), unique_id bigint, my_time timestamp, summable int ); QUERY: select dim5, sum(summable), count(distinct unique_id) from my_test_table where account_id = 76 and my_time >= '2010-07-03 00:00:00' and my_time < '2010-07-10 00:00:00' group by dim5; Re Stefan's questions: The results from the trace are below. I apologize if any of the data looks incorrect, I scrubbed my table's field names to be more generic. I let the database cool down for 1 minute and 10 minutes, but the queries remained slow. If I restarted the database, the queries resumed their quick response times. FAST QUERY TIME: sql>trace select dim5, sum(summable) as summables, count(distinct unique_id) as unqiue_ids from my_test_table where account_id = 76 and my_time >= '2010-07-03 00:00:00' and my_time < '2010-07-10 00:00:00' group by dim5; +-----------------------+--------+--------+ | dim1 | summables | unique_ids | +=======================+========+========+ | test1 | 251 | 18 | | test2 | 48 | 11 | | test3 | 1 | 1 | | test4 | 2 | 1 | | test4 | 106 | 2 | +-----------------------+--------+--------+ 5 tuples (125.534ms) +-------+------------------------------------------------------------------------------------------+ | ticks | stmt | +=======+==========================================================================================+ | 3 | _5 := sql.mvc(); | | 18 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","account_id | : : ",2); : | 7 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","accou | : : nt_id",0); : | 4 | _15:bat[:oid,:lng] <tmp_1102>[0] := sql.bind(_5=21685368,"sys","my_test_table","account_id | : : ",1); : | 6 | _18:bat[:oid,:oid] <tmp_1357>[0] := sql.bind_dbat(_5=21685368,"sys","my_test_table",1); | | 21 | _13<tmp_1527>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat[: | : : oid,:lng][0]); : | 4 | _19<tmpr_1357>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 10 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=21685368,"sys","my_test_table" | : : ,"my_time",0); : | 27 | _14<tmp_1532>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 11 | _23 := calc.timestamp(A1="2010-07-03 00:00:00"); | | 12 | _17<tmp_1527>[8000000] := algebra.kunion(_14=nil,_15=nil:bat[:oid,:lng]); | | 4 | _24 := calc.timestamp(A2="2010-07-10 00:00:00"); | | 7 | _20<tmp_1532>[8000000] := algebra.kdifference(_17=nil,_19=<tmpr_1357>[0]); | | 4 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=21685368,"sys","my_test_table","my_t | : : ime",2); : | 29575 | _25<tmp_1531>[359554] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-07-03 00: | : : 00:00.000,_24=2010-07-10 00:00:00.000,true,false); : | 12 | _31:bat[:oid,:timestamp] <tmp_1132>[0] := sql.bind(_5=21685368,"sys","my_test_table","my_t | : : ime",1); : | 24 | _28<tmp_1527>[359554] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestamp] | : : [0]); : | 23 | _32<tmp_1536>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00:00 | : : .000,_24=2010-07-10 00:00:00.000,true,false); : | 35 | _29<tmp_1501>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00:00 | : : .000,_24=2010-07-10 00:00:00.000,true,false); : | 8 | _39:bat[:oid,:str] <tmp_1364>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","dim5",0); : | 12 | _30<tmp_1534>[359554] := algebra.kunion(_28=nil,_29=nil); | | 5 | _44:bat[:oid,:str] <tmp_1103>[0] := sql.bind(_5=21685368,"sys","my_test_table","dim5 | : : e",2); : | 23 | _33<tmp_1501>[359554] := algebra.kunion(_30=nil,_32=nil); | | 6 | _46<tmp_1536>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:str],_44=<tmp_1103>:bat[ | : : :oid,:str][0]); : | 8 | _34<tmp_1534>[359554] := algebra.kdifference(_33=nil,_19=nil); | | 9 | _47<tmp_1501>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:str]); | | 4 | _48:bat[:oid,:str] <tmp_1104>[0] := sql.bind(_5=21685368,"sys","my_test_table","dim5",1); : | 10 | _50<tmp_1527>[8000000] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:str]); | | 6 | _66:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",0); : | 3 | _68:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",2); : | 7 | _69<tmp_1501>[8000000] := algebra.kdifference(_66=nil:bat[:oid,:lng],_68=<tmp_1101>:bat[ | : : :oid,:lng][0]); : | 9 | _70<tmp_1541>[8000000] := algebra.kunion(_69=nil,_68=nil:bat[:oid,:lng]); | | 4 | _71:bat[:oid,:lng] <tmp_1102>[0] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",1); : | 8 | _74<tmp_1501>[8000000] := algebra.kunion(_70=nil,_71=nil:bat[:oid,:lng]); | | 4 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=21685368,"sys","my_test_table","summable",2) | : : ; : | 7 | _59<tmp_1541>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:bat[ | : : :oid,:int][0]); : | 8 | _60<tmp_1524>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 3 | _61:bat[:oid,:int] <tmp_1066>[0] := sql.bind(_5=21685368,"sys","my_test_table","summable",1) | : : ; : | 8 | _62<tmp_1541>[8000000] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 15525 | _35<tmp_1536>[359554] := algebra.semijoin(_20=nil,_34=nil); | | 2446 | _36<tmp_1534>[139] := algebra.uselect(_35=nil,A0=76:lng); | | 10 | _37<tmp_1536>[139] := algebra.markT(_36=nil,0@0:oid); | | 4 | _38<tmpr_1536>[139] := bat.reverse(_37=nil); | | 33 | _75<tmp_1477>[139] := algebra.leftjoin(_38=<tmpr_1536>[139],_74=nil); | | 47 | _51<tmp_1403>[139] := algebra.leftjoin(_38=<tmpr_1536>[139],_50=nil); | | 4 | _76<tmp_1477>[139] := algebra.selectNotNil(_75=nil); | | 47 | (ext61<tmp_1502>[5],grp59<tmp_1476>[139]) := group.done(_51=<tmp_1403>[139]); | | 19 | _63<tmp_1363>[139] := algebra.leftjoin(_38=nil,_62=nil); | | 8 | _54<tmp_1541>[5] := bat.mirror(ext61=<tmp_1502>[5]); | | 3 | _64<tmp_1363>[139] := algebra.selectNotNil(_63=nil); | | 47 | _55<tmp_1471>[5] := algebra.join(_54=<tmp_1541>[5],_51=nil); | | 43 | (_77<tmp_1344>[33],grp85) := group.derive(ext61=nil,grp59=<tmp_1476>[139],_76=<tmp_1477> | : : [139]); : | 6 | _79<tmp_1527>[33] := bat.mirror(_77=nil); | | 22 | _80<tmp_1506>[33] := algebra.join(_79=nil,_76=nil); | | 20 | _81:bat[:oid,:wrd] <tmp_1527>[5] := aggr.count(_80=nil,grp59=<tmp_1476>[139],_54=<tmp_15 | : : 41>[5]); : | 21 | _65:bat[:oid,:lng] <tmp_1536>[5] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 63484 | barrier _137 := language.dataflow(); | | 9 | _82 := sql.resultSet(3,1,_55=<tmp_1471>[5]); | | 7 | sql.rsColumn(_82=1,"sys.my_test_table","dim5","varchar",50,0,_55=nil); | | 5 | sql.rsColumn(_82=1,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 5 | sql.rsColumn(_82=1,"sys.my_test_table","unique_ids","wrd",64,0,_81=nil:bat[:oid,:wrd]); | | 3 | _99 := io.stdout(); | | 41 | sql.exportResult(_99=14711568,_82=1); | | 83716 | user.s4_1(76:lng,"2010-07-03 00:00:00","2010-07-10 00:00:00"); | +-------+------------------------------------------------------------------------------------------+ SLOW QUERY TIME: sql>trace select dim5, sum(summable) as summables, count(distinct unique_id) as unqiue_ids from my_test_table where account_id = 76 and my_time >= '2010-07-03 00:00:00' and my_time < '2010-07-10 00:00:00' group by dim5; +-----------------------+--------+--------+ | dim1 | summables | unique_ids | +=======================+========+========+ | test1 | 251 | 18 | | test2 | 48 | 11 | | test3 | 1 | 1 | | test4 | 2 | 1 | | test4 | 106 | 2 | +-----------------------+--------+--------+ 5 tuples (2.7s) +---------+----------------------------------------------------------------------------------------+ | ticks | stmt | +=========+========================================================================================+ | 3 | _5 := sql.mvc(); | | 20 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","account_ | : : id",2); : | 5 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","acc | : : ount_id",0); : | 25 | _13<tmp_1705>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat | : : [:oid,:lng][0]); : | 4 | _15:bat[:oid,:lng] <tmp_1664>[10] := sql.bind(_5=21685368,"sys","my_test_table","account | : : _id",1); : | 20 | _14<tmp_1706>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 4 | _18:bat[:oid,:oid] <tmp_1070>[0] := sql.bind_dbat(_5=21685368,"sys","my_test_table",1); | | 8 | _19<tmpr_1070>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 31627 | _51<tmp_1670>[0] := algebra.leftjoin(_38=<tmpr_1665>[0],_50=nil); | | 20 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","my_time",0); : | 10 | _23 := calc.timestamp(A1="2010-07-03 00:00:00"); | | 3 | _24 := calc.timestamp(A2="2010-07-10 00:00:00"); | | 7243 | (ext61<tmp_1710>[0],grp59<tmp_1661>[0]) := group.done(_51=<tmp_1670>[0]); | | 16 | _54<tmp_1665>[0] := bat.mirror(ext61=<tmp_1710>[0]); | | 47358 | _63<tmp_1356>[0] := algebra.leftjoin(_38=nil,_62=nil); | | 6 | _64<tmp_1356>[0] := algebra.selectNotNil(_63=nil); | | 13 | (_77<tmp_1710>[0],grp85) := group.derive(ext61=nil,grp59=<tmp_1661>[0],_76=<tmp_1651>[ | : : 0]); : | 16 | _79<tmp_1674>[0] := bat.mirror(_77=nil); | | 11 | _80<tmp_1710>[0] := algebra.join(_79=nil,_76=nil); | | 10 | _81:bat[:oid,:wrd] <tmp_1651>[0] := aggr.count(_80=nil,grp59=<tmp_1661>[0],_54=<tmp_16 | : : 65>[0]); : | 20 | _55<tmp_1711>[0] := algebra.join(_54=<tmp_1665>[0],_51=nil); | | 11 | _65:bat[:oid,:lng] <tmp_1674>[0] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 11 | _82 := sql.resultSet(3,1,_55=<tmp_1711>[0]); | | 9 | sql.rsColumn(_82=235,"sys.my_test_table","dim5","varchar",50,0,_55=nil); | | 5 | sql.rsColumn(_82=235,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 4 | sql.rsColumn(_82=235,"sys.my_test_table","unique_ids","wrd",64,0,_81=nil:bat[:oid,:wrd]); | | 2 | _99 := io.stdout(); | | 30 | sql.exportResult(_99=24880032,_82=235); | | 420626 | _17<tmp_1705>[8000010] := algebra.kunion(_14=nil,_15=nil:bat[:oid,:lng]); | | 17 | _20<tmp_1665>[8000010] := algebra.kdifference(_17=nil,_19=<tmpr_1070>[0]); | | 13 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=21685368,"sys","my_test_table","my | : : _time",2); : | 5 | _31:bat[:oid,:timestamp] <tmp_1704>[10] := sql.bind(_5=21685368,"sys","my_test_table","m | : : y_time",1); : | 13 | _32<tmp_1673>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00: | : : 00.000,_24=2010-07-10 00:00:00.000,true,false); : | 8 | _39:bat[:oid,:str] <tmp_1364>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","dim5",0); : | 4 | _44:bat[:oid,:str] <tmp_1103>[0] := sql.bind(_5=21685368,"sys","my_test_table","dim5",2); : | 7 | _46<tmp_1661>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:str],_44=<tmp_1103>:ba | : : t[:oid,:str][0]); : | 16 | _47<tmp_1706>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:str]); | | 3 | _5 := sql.mvc(); | | 4 | _48:bat[:oid,:str] <tmp_1631>[10] := sql.bind(_5=21685368,"sys","my_test_table","dim5",1); : | 21 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","account_ | : : id",2); : | 5 | _15:bat[:oid,:lng] <tmp_1637>[10] := sql.bind(_5=25090760,"sys","my_test_table","account | : : _id",1); : | 4 | _18:bat[:oid,:oid] <tmp_1070>[0] := sql.bind_dbat(_5=25090760,"sys","my_test_table",1); | | 3 | _19<tmpr_1070>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 5 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","my_time",0); : | 8 | _23 := calc.timestamp(A1="2010-07-27 00:00:00"); | | 3 | _24 := calc.timestamp(A2="2010-08-03 00:00:00"); | | 31883 | _25<tmp_1712>[391659] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-07-27 0 | : : 0:00:00.000,_24=2010-08-03 00:00:00.000,true,false); : | 11 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=25090760,"sys","my_test_table","my | : : _time",2); : | 21 | _28<tmp_1674>[391659] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestam | : : p][0]); : | 33 | _29<tmp_1714>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-07-27 00:00: | : : 00.000,_24=2010-08-03 00:00:00.000,true,false); : | 11 | _30<tmp_1713>[391659] := algebra.kunion(_28=nil,_29=nil); | | 4 | _31:bat[:oid,:timestamp] <tmp_1660>[10] := sql.bind(_5=25090760,"sys","my_test_table","m | : : y_time",1); : | 12 | _32<tmp_1714>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-07-27 00:00: | : : 00.000,_24=2010-08-03 00:00:00.000,true,false); : | 10 | _33<tmp_1674>[391659] := algebra.kunion(_30=nil,_32=nil); | | 7 | _39:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",0); : | 4 | _44:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","unqiue_id",2); : | 6 | _46<tmp_1713>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:lng],_44=<tmp_1101>:ba | : : t[:oid,:lng][0]); : | 12 | _47<tmp_1715>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:lng]); | | 4 | _48:bat[:oid,:lng] <tmp_1657>[10] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",1); : | 366305 | _50<tmp_1713>[8000010] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:lng]); | | 14 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=25090760,"sys","my_test_table","summable", | : : 2); : | 17 | _59<tmp_1715>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:ba | : : t[:oid,:int][0]); : | 16 | _60<tmp_1716>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 4 | _61:bat[:oid,:int] <tmp_1362>[10] := sql.bind(_5=25090760,"sys","my_test_table","summable" | : : ,1); : | 5 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","acc | : : ount_id",0); : | 10 | _13<tmp_1717>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat | : : [:oid,:lng][0]); : | 298001 | _62<tmp_1715>[8000010] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 21 | _14<tmp_1716>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 374628 | _17<tmp_1717>[8000010] := algebra.kunion(_14=nil,_15=nil:bat[:oid,:lng]); | | 20 | _20<tmp_1716>[8000010] := algebra.kdifference(_17=nil,_19=<tmpr_1070>[0]); | | 15 | _34<tmp_1720>[391659] := algebra.kdifference(_33=nil,_19=nil); | | 65082 | _35<tmp_1674>[391659] := algebra.semijoin(_20=nil,_34=nil); | | 2960 | _36<tmp_1714>[0] := algebra.uselect(_35=nil,A0=239:lng); | | 12 | _37<tmp_1674>[0] := algebra.markT(_36=nil,0@0:oid); | | 5 | _38<tmpr_1674>[0] := bat.reverse(_37=nil); | | 58011 | _51<tmp_1651>[0] := algebra.leftjoin(_38=<tmpr_1674>[0],_50=nil); | | 26 | (ext61<tmp_1710>[0],grp59<tmp_1713>[0]) := group.done(_51=<tmp_1651>[0]); | | 5 | _54<tmp_1674>[0] := bat.mirror(ext61=<tmp_1710>[0]); | | 27086 | _63<tmp_1356>[0] := algebra.leftjoin(_38=nil,_62=nil); | | 24 | _55<tmp_1715>[0] := algebra.join(_54=<tmp_1674>[0],_51=<tmp_1651>[0]); | | 3 | _66<tmp_1651>[0] := algebra.selectNotNil(_51=nil); | | 11 | (_67<tmp_1710>[0],grp75) := group.derive(ext61=nil,grp59=<tmp_1713>[0],_66=<tmp_1651>[ | : : 0]); : | 10 | _69<tmp_1712>[0] := bat.mirror(_67=nil); | | 5 | _70<tmp_1710>[0] := algebra.join(_69=nil,_66=nil); | | 9 | _71:bat[:oid,:wrd] <tmp_1651>[0] := aggr.count(_70=nil,grp59=<tmp_1713>[0],_54=<tmp_16 | : : 74>[0]); : | 3 | _64<tmp_1356>[0] := algebra.selectNotNil(_63=nil); | | 11 | _65:bat[:oid,:lng] <tmp_1712>[0] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 1226556 | barrier _119 := language.dataflow(); | | 9 | _72 := sql.resultSet(3,1,_55=<tmp_1715>[0]); | | 8 | sql.rsColumn(_72=236,"sys.my_test_table","unique_id","bigint",64,0,_55=nil); | | 4 | sql.rsColumn(_72=236,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 3 | sql.rsColumn(_72=236,"sys.my_test_table","unique_ids","wrd",64,0,_71=nil:bat[:oid,:wrd]); | | 3 | _85 := io.stdout(); | | 37 | sql.exportResult(_85=24880032,_72=236); | | 2 | _5 := sql.mvc(); | | 18 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","account_ | : : id",2); : | 4 | _15:bat[:oid,:lng] <tmp_1637>[10] := sql.bind(_5=25090760,"sys","my_test_table","account | : : _id",1); : | 3 | _18:bat[:oid,:oid] <tmp_1070>[0] := sql.bind_dbat(_5=25090760,"sys","my_test_table",1); | | 3 | _19<tmpr_1070>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 6 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","my_time",0); : | 8 | _23 := calc.timestamp(A1="2010-06-24 00:00:00"); | | 3 | _24 := calc.timestamp(A2="2010-07-01 00:00:00"); | | 18 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","acc | : : ount_id",0); : | 678 | _13<tmp_1713>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat | : : [:oid,:lng][0]); : | 22 | _14<tmp_1674>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 49964 | _25<tmp_1714>[372221] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-06-24 0 | : : 0:00:00.000,_24=2010-07-01 00:00:00.000,true,false); : | 14 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=25090760,"sys","my_test_table","my | : : _time",2); : | 21 | _28<tmp_1713>[372221] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestam | : : p][0]); : | 24 | _29<tmp_1720>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-06-24 00:00: | : : 00.000,_24=2010-07-01 00:00:00.000,true,false); : | 11 | _30<tmp_1712>[372221] := algebra.kunion(_28=nil,_29=nil); | | 4 | _31:bat[:oid,:timestamp] <tmp_1660>[10] := sql.bind(_5=25090760,"sys","my_test_table","m | : : y_time",1); : | 13 | _32<tmp_1720>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-06-24 00:00: | : : 00.000,_24=2010-07-01 00:00:00.000,true,false); : | 10 | _33<tmp_1713>[372221] := algebra.kunion(_30=nil,_32=nil); | | 5 | _39:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",0); : | 5 | _44:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",2); : | 6 | _46<tmp_1712>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:lng],_44=<tmp_1101>:ba | : : t[:oid,:lng][0]); : | 8 | _47<tmp_1717>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:lng]); | | 3 | _48:bat[:oid,:lng] <tmp_1657>[10] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",1); : | 1381664 | _50<tmp_1661>[8000010] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:str]); | | 17 | _66:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",0); : | 4 | _68:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",2); : | 19 | _69<tmp_1706>[8000000] := algebra.kdifference(_66=nil:bat[:oid,:lng],_68=<tmp_1101>:ba | : : t[:oid,:lng][0]); : | 17 | _70<tmp_1721>[8000000] := algebra.kunion(_69=nil,_68=nil:bat[:oid,:lng]); | | 4 | _71:bat[:oid,:lng] <tmp_1703>[10] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",1); : | 405606 | _50<tmp_1712>[8000010] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:lng]); | | 15 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=25090760,"sys","my_test_table","summable", | : : 2); : | 20 | _59<tmp_1717>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:ba | : : t[:oid,:int][0]); : | 17 | _60<tmp_1722>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 4 | _61:bat[:oid,:int] <tmp_1362>[10] := sql.bind(_5=25090760,"sys","my_test_table","summable" | : : ,1); : | 374062 | _74<tmp_1706>[8000010] := algebra.kunion(_70=nil,_71=nil:bat[:oid,:lng]); | | 16 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=21685368,"sys","my_test_table","summable", | : : 2); : | 18 | _59<tmp_1721>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:ba | : : t[:oid,:int][0]); : | 16 | _60<tmp_1723>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 4 | _61:bat[:oid,:int] <tmp_1361>[10] := sql.bind(_5=21685368,"sys","my_test_table","summable" | : : ,1); : | 1884678 | _25<tmp_1707>[359554] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-07-03 0 | : : 0:00:00.000,_24=2010-07-10 00:00:00.000,true,false); : | 28 | _28<tmp_1724>[359554] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestam | : : p][0]); : | 30 | _29<tmp_1726>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00: | : : 00.000,_24=2010-07-10 00:00:00.000,true,false); : | 11 | _30<tmp_1725>[359554] := algebra.kunion(_28=nil,_29=nil); | | 10 | _33<tmp_1726>[359554] := algebra.kunion(_30=nil,_32=nil); | | 7 | _34<tmp_1725>[359554] := algebra.kdifference(_33=nil,_19=nil); | | 63572 | _35<tmp_1726>[359554] := algebra.semijoin(_20=nil,_34=nil); | | 2812 | _36<tmp_1673>[139] := algebra.uselect(_35=nil,A0=76:lng); | | 10 | _37<tmp_1726>[139] := algebra.markT(_36=nil,0@0:oid); | | 5 | _38<tmpr_1726>[139] := bat.reverse(_37=nil); | | 58840 | _75<tmp_1715>[139] := algebra.leftjoin(_38=<tmpr_1726>[139],_74=nil); | | 6 | _76<tmp_1715>[139] := algebra.selectNotNil(_75=nil); | | 21903 | _51<tmp_1711>[139] := algebra.leftjoin(_38=<tmpr_1726>[139],_50=nil); | | 62 | (ext61<tmp_1344>[5],grp59<tmp_1651>[139]) := group.done(_51=<tmp_1711>[139]); | | 11 | _54<tmp_1661>[5] := bat.mirror(ext61=<tmp_1344>[5]); | | 46 | _55<tmp_1570>[5] := algebra.join(_54=<tmp_1661>[5],_51=nil); | | 34 | (_77<tmp_1500>[33],grp85) := group.derive(ext61=nil,grp59=<tmp_1651>[139],_76=<tmp_171 | : : 5>[139]); : | 6 | _79<tmp_1706>[33] := bat.mirror(_77=nil); | | 24 | _80<tmp_1502>[33] := algebra.join(_79=nil,_76=nil); | | 22 | _81:bat[:oid,:wrd] <tmp_1706>[5] := aggr.count(_80=nil,grp59=<tmp_1651>[139],_54=<tmp_ | : : 1661>[5]); : | 554759 | _62<tmp_1717>[8000010] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 541977 | _62<tmp_1721>[8000010] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 41224 | _63<tmp_1356>[139] := algebra.leftjoin(_38=nil,_62=nil); | | 10 | _64<tmp_1356>[139] := algebra.selectNotNil(_63=nil); | | 48 | _65:bat[:oid,:lng] <tmp_1721>[5] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 2761640 | barrier _137 := language.dataflow(); | | 12 | _82 := sql.resultSet(3,1,_55=<tmp_1570>[5]); | | 8 | sql.rsColumn(_82=3,"sys.my_test_table","dim5","varchar",50,0,_55=nil); | | 5 | sql.rsColumn(_82=3,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 6 | sql.rsColumn(_82=3,"sys.my_test_table","unique_ids","wrd",64,0,_81=nil:bat[:oid,:wrd]); | | 3 | _99 := io.stdout(); | | 45 | sql.exportResult(_99=14711568,_82=3); | | 2761973 | user.s4_1(76:lng,"2010-07-03 00:00:00","2010-07-10 00:00:00"); | +---------+----------------------------------------------------------------------------------------+

1 0

[MonetDB-users] performance degrades when date related function is used
by Ramkumarlak 03 Feb '11

03 Feb '11

Hi, I am looking for monerdb to replace mysql. I tried some queries with my test data and all were much more faster than mysql. But our actual table contains a column which stores date in unix time format and we do generate reports based on week, day, quarter, month and year. For that we are using a lot mysql date related functions. I too tried the following approach for one of my requirements. cast(timestamp '1970-01-01 00:00:00' + cast(created_date/1000 as interval second) as date) but it takes longer time. 1) Is there any other way to make this query faster ? 2) what are all the options are available in monetdb to deal these (week,quarter,hour,day and year wise reporting) kind of queries. Any help will be appreciated. Thanks and Regards Ram -- View this message in context: http://old.nabble.com/performance-degrades-when-date-related-function-is-us… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] query slowdown with concurrent write
by Bennett Hiles 03 Feb '11

03 Feb '11

Hi, MonetDB has great response times for the queries I have been running, but once I start inserting data to the table and concurrently running the queries, the queries slow down. To give you concrete numbers, I am initializing the database with 8 million records. I then run random aggregation queries repeatedly. Before I insert additional data the max query time is 0.3 seconds. If I then insert 10 rows the queries slow to a max of 1.5s. The only solution I have to getting the queries to resume their original speed is to drop the data and reload the 8 millions rows into the table. I have also noticed that if I run my inserts while I am not concurrently making queries to the database, I don't run into the slowness, and the queries keep their 0.3s response time. Has anyone else encountered this problem before? Is there a way around it? I have also tried using the copy command instead of inserts but get the same result. Thanks, Bennett

3 2

[MonetDB-users] MAL
by Michael Kusber 02 Feb '11

02 Feb '11

Hello guys, I aplogize for bothering you with what might be a simple issue for you, but I am new to monetDB and I have to solve some complex problems for which I cannot find a solution by myself. I've read about MAL in the M5 reference manual. I recognized, the normal way to execute commands is via MAPI and JDBC and stuff based on MAPI. This will be parsed into MAL, optimized and executed. Is there a way to write MAL code and give it as input to the server directly?? Another question: Is it possible to explicitely invoke execution of a query in multiple threads via MAPI or other means? Kind regards, michael

3 2

[MonetDB-users] After delete records with a field criteria, still shows in group by SQL
by Osmin Castillo 02 Feb '11

02 Feb '11

Hi there. I'm having this extrange problem with MonetDB. After delete some records with a criteria for example: field_id = 201011 The records are deleted, but when I use a select group by, the count(0) shows results for field_id = 201011 Thanks in advance

1 0

[MonetDB-users] collation - sorting order
by Ramkumarlak 28 Jan '11

28 Jan '11

Hi, Can we control the sorting order in MonetDB? Is there any way we can specify collation ...? Regards Ram -- View this message in context: http://old.nabble.com/collation---sorting-order-tp30786899p30786899.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] date functions in monetdb
by Ramkumarlak 27 Jan '11

27 Jan '11

Hi, Let me know whether the following MYSQL functions are available in MonetDB. 1) concat() 2) date() 3) from_unix_time() 4) interval() 5) Weekday() If not, are there any equivalent functions available ? Regards Ram -- View this message in context: http://old.nabble.com/date-functions-in-monetdb-tp30775963p30775963.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

[MonetDB-users] Performance question between schema
by brien colwell 27 Jan '11

27 Jan '11

With Monet DB I've encountered a counter- intuitive performance characteristic. I have a dataset of 33554432 (2^25) triples, (a, b, c). The values a and b are uniformly distributed on [0, 16), and c is unique to each triple, on [0, 2^25). I store them in two ways: Schema 1: One table with three columns (a, b, c) Schema 2: Two tables with two columns each, (a, c) and (b, c) I'm running SQL queries to select the "c" value from the triples based on a=X and b=Y. With schema 1, this is a single select. With schema 2, this is an inner join between two selects. What surprised me is that the query against schema 2 is on average >50 ms faster to return than the query against schema 1. To me this was counter- intuitive since schema 1 stores less data, and the data are pre-joined (in a row-oriented sense) for the type of filter applied. However, I'm guessing this has to do with how MonetDB stores data in columns. Is the query against schema 1 actually doing three joins, "a" -> "b" -> "c", on an artificial join key? For example, -- Schema 1) create table abc (a bigint, b bigint, c bigint); -- Schema 2) create table ac (a bigint, c bigint); create table bc (b bigint, c bigint); copy 33554432 records into abc from '/home/hadoop/abc.monetdb'; copy 33554432 records into ac from '/home/hadoop/ac.monetdb'; copy 33554432 records into bc from '/home/hadoop/bc.monetdb'; -- Query Schema 1) select abc.c from abc where abc.a=3 and abc.b=7 order by abc.c asc -- Times to return (JDBC) -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.287, SQL query: 0.287, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.288, SQL query: 0.288, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.35, SQL query: 0.35, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.372, SQL query: 0.372, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.405, SQL query: 0.405, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.285, SQL query: 0.285, Building output: 0 -- Query Schema 2) select ac.c from ac inner join bc on bc.b=7 and bc.c = ac.c where ac.a = 3 order by ac.c asc -- Times to return (JDBC) -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.218, SQL query: 0.218, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.216, SQL query: 0.216, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.318, SQL query: 0.318, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.322, SQL query: 0.322, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.218, SQL query: 0.218, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.219, SQL query: 0.219, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.222, SQL query: 0.222, Building output: 0

2 1

[MonetDB-users] "similar to" text matching syntax
by brien colwell 25 Jan '11

25 Jan '11

I came across a commit for a "similar to" syntax for text columns -- http://www.mail-archive.com/checkin-list@monetdb.org/msg01289.html Very interested. Are there any updates on this commit -- stable, not stable, not tested? Does MonetDB publish nightly builds?

2 2

[MonetDB-users] quer cache
by Ramkumarlak 24 Jan '11

24 Jan '11

Hi, Does monetdb have query cache ? If so, how do we disable it ? I searched in monetdb5.cong but i did not find any option. Regards Ram -- View this message in context: http://old.nabble.com/quer-cache-tp30747505p30747505.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 3

[MonetDB-users] copy select
by Nafees Ur Rehman 19 Jan '11

19 Jan '11

Hi, With executing the statement below: COPY SELECT * FROM adv LIMIT 10 INTO 'D:/exp.txt' I get the following error: !TypeException: user.s0_1[4703]:'aggr.count' undefined in : tuples:wrd :=aggr.count(_4695:str) !SQLException:SQLengine:Program contains errors What am I doing wrong? Thanks Nafees

1 0

[MonetDB-users] ODBC error 'Function Sequence Error' occurredwhen loaded TPC-H data of MonetDB
by JinBinge 16 Jan '11

16 Jan '11

Hi, Benchmark Factory for Database was used to test TPC-H of MonetDB. Errors occurred when creating objects for TPC-H option was selected,such as ODBC error 'Function Sequence Error'. And tables h_nation,h_region,h_supplier' data were loaded successful,the other five tables failed to load data. follows are the test configuration: OS: Linux Hardware: amd64 MonetDB version: Oct2010 Benchmark Factory for Database version: v5.7.1g.504 load data'SF=1G Are there errors in ODBC of MonetDB? Yours sincerely Binge Jin

2 1

[MonetDB-users] Query Error Question
by Brett Spurrier 14 Jan '11

14 Jan '11

Hi all, I have yet another query that I am having trouble running with Monet, and I am hoping a seasoned expert can help me figure out what is going on here. My query is: SELECT fid, title, D0 AS ed FROM signatures t WHERE fid = ( SELECT fid FROM signatures ti WHERE ti.guid = t.guid ORDER BY D0 ASC LIMIT 1 OFFSET 1) ORDER BY D0 ASC, fid DESC; but I am getting the error: "!syntax error, unexpected ORDER, expecting INTERSECT or EXCEPT or UNION or ')' in: "select fid, title, d0 as ed from signatures t where fid = ( select fid from signatures ti where ti.guid = t.guid order" So it appears to not like the ORDER BY term, and I can't figure out why. I am trying to grab the second element (after ordering by D0) based on distinct GUIDs. The query iteself is based on the answer for a question from stackoverflow that I posted: http://stackoverflow.com/questions/4634068/sql-query-with-order-by-part-2 Can anyone help me figure this out? The syntax looks fine to me. By the way, I have "D0 AS ed" because D0 will eventually be replaced with a simple math operator (i.e., D0/D1). Many thanks, Brett

4 5

[MonetDB-users] An error about MonetDB' TPCH power test
by 金炳娥 14 Jan '11

14 Jan '11

Hi all, Recently I used the Benchmark Factory tool to test TPCH of MonetDB(SF=100). When running power test,there is an error,such as follows: Started test TPC-H Power Test Agent(bejin) Error: ODBC Error (HY000,0) - [MonetDB][ODBC Driver 1.0]SQLException:convert:too many digits (21 > 19) Do you know the reason and how to reslove the problem? Looking forwards to yours answer. Yours sincerely Binge Jin

1 0

[MonetDB-users] Add primary key performance
by paulr135 13 Jan '11

13 Jan '11

I've loaded a 27GB (~165M rows) file into a table in MonetDB and now I am trying to add a primary key which consists of 4 fields - 2 varchars and 2 ints. It has been running for a few days (since Friday of last week). I've followed the instructions laid out on this post: http://old.nabble.com/MonetDB-SQL:-bulk-loading-large-data-sets---adding-ke… which involves setting the optimizer variable to "nov2009_pipe". I am unable to log in via mclient - I am able to put in my credentials but it just hangs at the "Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2010-SP2)" line. So I have a few questions: 1. Is there any way I can see the status of the alter table ... add primary key command? 2. Is it possible to stop the alter table gracefully? Or will I have to shut the db down? 3. Is there anything I should do in future to speed up adding of primary keys? Thanks. -- View this message in context: http://old.nabble.com/Add-primary-key-performance-tp30397515p30397515.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] Consultation about TPCH throughput test errors with MonetDB
by 金炳娥 12 Jan '11

12 Jan '11

Hi, Recently I has a problem when running TPCH with MonetDB. Follows are the configuration. Hardware:amd64 OS:Linux MonetDB Version:Oct2010 I loaded 100G' data to the MonetDB.Using TPCH Tester tools,I runned power test.After power test successed,running throughput test. There is an error. ERR TPCH[17170]: *** glibc detected *** /mnt/sdb1/tpch/MonetDB/install/bin/mserver5: free(): invalid pointer: 0x00002ac5381b5008 *** 2011-01-04 14:00:41 ERR TPCH[17170]: ======= Backtrace: ========= 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libc.so.6[0x3aa5c71ce2] 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libc.so.6(cfree+0x8c)[0x3aa5c7590c] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1[0x2b67e8c062f6] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1(HEAPfree+0xb9)[0x2b67e8b36511] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1(BATdelete+0x196)[0x2b67e8e1769a] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1[0x2b67e8b33697] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1[0x2b67e8b32fa8] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libmonetdb5.so.5[0x2b67e7f1da68] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libmonetdb5.so.5[0x2b67e7f1f6f1] 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libpthread.so.0[0x3aa6806367] 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libc.so.6(clone+0x6d)[0x3aa5cd30ad] 2011-01-04 14:00:41 ERR TPCH[17170]: ======= Memory map: ======== 2011-01-04 14:00:41 ERR TPCH[17170]: 00400000-00404000 r-xp 00000000 08:11 179159655 /mnt/sdb1/tpch/MonetDB/install/bin/mserver5 2011-01-04 14:00:41 ERR TPCH[17170]: 00604000-00605000 rw-p 00004000 08:11 179159655 /mnt/sdb1/tpch/MonetDB/install/bin/mserver5 2011-01-04 14:00:41 ERR TPCH[17170]: 00605000-00612000 rw-p 00605000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 0a03f000-0e488000 rw-p 0a03f000 00:00 0 [heap] 2011-01-04 14:00:41 ERR TPCH[17170]: 4001c000-4001d000 ---p 4001c000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 4001d000-4021d000 rw-p 4001d000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 40287000-40288000 ---p 40287000 00:00 0 … 2011-01-04 14:00:41 ERR TPCH[17170]: 4bc66000-4be66000 rw-p 4bc66000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5800000-3aa581c000 r-xp 00000000 fd:00 95158276 /lib64/ld-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5a1b000-3aa5a1c000 r--p 0001b000 fd:00 95158276 /lib64/ld-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5a1c000-3aa5a1d000 rw-p 0001c000 fd:00 95158276 /lib64/ld-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5c00000-3aa5d4c000 r-xp 00000000 fd:00 95158283 /lib64/libc-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5d4c000-3aa5f4c000 ---p 0014c000 fd:00 95158283 /lib64/libc-2.5.so The test data are correct. Do you know the reason above errors? I hope to have a letter from you before long.With every good wish. Yours sincerely， Binge Jin

2 1

[MonetDB-users] Concurrent aggregation crashes MonetDB
by Dávid Karnok 06 Jan '11

06 Jan '11

Hi! I'm running an evaluation test on the latest MonetDB/SQL stable version and always crashes possibly due an out-of-memory problem. My system: Windows 7 x86, 4GB memory, Core i7 920, data files are on an SSD disk MonetDB (default install): # MonetDB server v5.22.1, based on kernel v1.40.1 # Release Oct2010 # Serving database 'demo', using 8 threads # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Found 3.186 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\Program Files\MonetDB\MonetDB5\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.40.1 loaded Using monetdb-1.17-jdbc driver. The test involves a simple, two column table: CREATE TABLE a (text VARCHAR(50), value INT) Which is then filled with 10.000.000 records of random values. The data files take up to 150MB disk space. The test routine creates multiple threads which connect to the local demo database, and simultaneously execute the same aggregation query. The test simulates a concurrent analysis-data mining scenario running on the same data. void parallelTest(int threadCount) { ExecutorService exec = Executors.newFixedThreadPool(threadCount); final CountDownLatch cnt = new CountDownLatch(threadCount); for (int i = 0; i < threadCount; i++) { final int j = i; exec.submit(new Runnable() { @Override public void run() { try { Class.forName("nl.cwi.monetdb.jdbc.MonetDriver"); Connection conn = DriverManager.getConnection("jdbc:monetdb://localhost/demo", "monetdb", "monetdb"); try { conn.setAutoCommit(false); long t1 = System.currentTimeMillis(); Statement stmt = conn.createStatement(); try { cnt.countDown(); cnt.await(); ResultSet rs = stmt.executeQuery("SELECT text, AVG(value) FROM a GROUP BY text "); while (rs.next()) { rs.getString(1); rs.getDouble(2); } System.out.printf("[%02d] Select time: %.2fs%n", j, (System.currentTimeMillis() - t1) / 1000.0); } finally { stmt.close(); } } finally { conn.close(); } } catch (Exception ex) { ex.printStackTrace(); } } }); } exec.shutdown(); try { exec.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS); } catch (InterruptedException ex) { } } I noticed that if the threadCount >= 4 the MonetDB console starts to print VirtualAlloc failed messages and sooner or later terminates. The memory usage seems to jump by 600MB, e.g. with 3 threads it consumes 1.7 GB memory, but with 4 threads, it can't consume 2.4GB. Replacing the query with a single result "SELECT MAX(VALUE) FROM a" causes the same crash after 5 threads. The same test running with the latest MySQL works perfectly, although slowly. No crashes or memory issues. Is this a memory leak in MonetDB or the result of its architecture, meaning I should use a 64bit system with lots of RAM? Regards, David Karnok

1 0

[MonetDB-users] MonetDB LIMIT usage
by Brett Spurrier 05 Jan '11

05 Jan '11

Hi all, I am trying to use the LIMIT clause in my query, and while this should be fairly straight froward, I am having trouble learning the differences between Monet's LIMIT and other T-sql LIMITS. I have the simple query: SELECT * FROM signatures ORDER BY scale ASC LIMIT 1; which works great and as expected. However, I really want the second record, so in other databases I can use: SELECT * FROM signatures ORDER BY scale ASC LIMIT 1, 1; I have also tried: SELECT * FROM signatures ORDER BY scale ASC LIMIT 1 1; However, these both fail in MonetDB. According to: http://monetdb.cwi.nl/SQL/Documentation/Queries.html#Queries it looks like it should be supported. Does anyone have any insight as to how to grab the second record in MonetDB (I.e., limiting the record results to one with an offset of one)??? I am at a loss here. Cheers, Brett

3 4

[MonetDB-users] Issues with commandline copy into and opening files
by Lex Slaghuis 03 Jan '11

03 Jan '11

Hi All Eversince I updated my system; The following blockbased import script doesn't work anymore: #!/bin/bash BLOCKSIZE=250000 #RECORDCOUNT L=$(wc -l $1 | awk '{ print $1}'); #MERGE BLOCKS INTO TABLE for I in $(seq 0 $BLOCKSIZE $L); do echo "$BLOCKSIZE $I $1/$L loading... "; mclient --time -d test --statement="COPY $BLOCKSIZE OFFSET $I records into \ wptool.wpstats from '$1' using delimiters ' ','\n','' null as '';"; echo "$BLOCKSIZE $I $1/$L loading...finished "; done Returns the following error (it used to work): IOException:streams.open:Could not open file ERROR: mnstr_open: could not open file '/var/www/loadstats/pagecounts-20110102-160000' The script is started through: sh importintodb.sh /var/www/loadstats/pagecounts-20110102-160000 Does anyone has some thoughts on this, or should I report a bug? I would really like to use commandline, as the hourly imports are 6M records big... The .monetdb file is placed locally and automatically authenticates with monetdb (admin). Lang=SQL, history=yes Regards Lex Slaghuis

2 1

[MonetDB-users] Monnetdb/Xquery
by Ramachandran, Prakash 30 Dec '10

30 Dec '10

Hi, I am currently evaluating software's for metadata storage that can be used in our product. I came across your product and found it very interesting and we might consider using it for our metadata storage. I just wanted to know if Monnetdb provides commercial support, if so can you please provide the detail on sales contacts. Thank in advance. Regards, Prakash R

1 0

[MonetDB-users] MAPI interface error
by Brett Spurrier 24 Dec '10

24 Dec '10

Hi all, I am having a bit of trouble and I am hoping a seasoned MonetDB expert can lend some advice. I am using the MAPI interface in my C/C++ application to talk to my MonetDB instance. I have a recursive INSERT INTO statement which populates the database, and it initially works pretty well. However, as I populate more (past 200 inserts in my case), I get a MAPI error (exampled by: mapi_explain_result()): MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !Connection terminated No real reason as to why. Has anyone seen this before? My code: MapiHdl query(Mapi dbh, char *q) { MapiHdl ret = NULL; if ((ret = mapi_query(dbh, q)) == NULL || mapi_error(dbh) != MOK) die(dbh, ret); return(ret); } void update(Mapi dbh, char *q) { MapiHdl ret = query(dbh, q); if (mapi_close_handle(ret) != MOK) die(dbh, ret); } void main() { dbh = mapi_connect("localhost", 50000, "monetdb", "monetdb", "sql", "dbname"); if (mapi_error(dbh)) { die(dbh, hdl); } for (int i=0; i< 2500; i++) { char *p = INSERT_INTO_STATEMENT; update(dbh, p); } } Does anyone see any reason that the MAPI would choke mid way through the For loop? Any help would be majorly appreciated. I am at a loss. Cheers, Brett

3 14

[MonetDB-users] Announcement: New Jun2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 21 Dec '10

21 Dec '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2010-SP1 bug fix release of the MonetDB suite of programs. The most important and visible change is a security fix in the SQL COPY INTO command. This is detailed in the release notes. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Oct2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] How to limit monetdb memory consumption on Windows
by Bianco Prieto-Ilarion 18 Dec '10

18 Dec '10

Hi all! I have a problem. I'm using Oct2010 64-bit MonetDB build. I have a table with 8M records and when I'm trying to execute LEFT JOIN (from JDBC) MonetDB is crashed. I can see that it crashed due to memory (my laptop has 8Gb) consumption is unlimited. What can you suggest (just use Linux is a final step:)) Thanks, Bina Prieto

1 0

[MonetDB-users] MonetDB Installation
by Rob Fynes 16 Dec '10

16 Dec '10

Hello, Regarding the pre-compiled binaries directories for deb distributions "karmic" and "lucid" in the downloads section, there doesn't appear to be anything for x-query. Is this intentional? Regards, -Rob

2 2

[MonetDB-users] mserver5 ignores ulimit?
by Siva Manoharan 14 Dec '10

14 Dec '10

Hi. I am currently running monetdb (release Oct2010) on Debian etch, with 8GB physical RAM and 10GB virutal memory. During unrestricted operation (using the merovingian daemon) monetdb takes up a significant majority of the resources, which is not desirable as there are other processes running on the same server. To limit the memory, I tried using ulimit (setting -d, -v and -m all to 2GB). According to the logs, merovingian notices the limit (a line from log states "Found 1.907 GiB available main-memory"). However, memory usage of mserver still exceeds that limit and is killed off by the OS, abandoning an ongoing query. (merovingian is not killed and mserver restarts without any problems). Is this expected behaviour or is it a bug? Is there another way to limit monetdb's memory usage? As a note, none of the tables (and therefore none of the columns) are anywhere near 2GB in size. The biggest table has ~2.5M rows, where each row has 2 columns ( 8byte bigint and 4 byte int). However, there are thousands of tables, and mserver exceeds the limit most often when a large number of tables are being updated within a short time (in a single php process, however). Is it possible that mserver doesn't release memory fast enough to allow multiple tables to be accessed? Thanks.

1 0

[MonetDB-users] crashes with copy into (270MB file)
by Lex Slaghuis 13 Dec '10

13 Dec '10

Hello I Experience crashes with the copy into statement, with large files (270MB utf8) ... sql>COPY 6000000 records into wptool.wpstats from '/var/www/pagecounts' using delimiters ' ','\n','' null as ''; MAPI = testuser@localhost:50000 ACTION= read_line QUERY = COPY 6000000 records into wptool.wpstats from '/var/www/pagecounts' using delimiters ' ','\n','' null as ''; ERROR = !Connection terminated Timer 5846.447 msec...... Number of records: wc -l /var/www/pagecounts 5983201 /var/www/pagecounts My memory: free -m(egabytes) total used free shared buffers cached Mem: 2007 431 1576 0 20 304 -/+ buffers/cache: 106 1900 Swap: 16769 0 16769 Error-logs: 2010-12-13 14:17:41 MSG merovingian[902]: proxying client localhost:52737 for database 'test' to mapi:monetdb:///var/MonetDB5/dbf arm/test/.mapi.sock?database=test 2010-12-13 14:17:41 MSG merovingian[902]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2010-12-13 14:18:30 MSG merovingian[902]: database 'test' (1142) was killed by signal SIGSEGV 2010-12-13 14:18:38 ERR control[902]: (local): received stop signal for non running database: test 2010-12-13 14:18:46 MSG merovingian[902]: database 'test' has crashed after start on 2010-12-13 14:17:33, attempting restart, up min/avg/max: 14s/22h/2d, crash average: 1.00 0.70 0.23 (10-3=7) Some specs: uname -a Linux ubuntu 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux 2010-12-13 14:17:33 MSG test[1142]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2010-12-13 14:17:33 MSG test[1142]: # Found 1.961 GiB available main-memory. If anyone has some pointers on this , it would really help out. I think it would be really to chop up the files into smaller pieces, just for import. Regards Lex Slaghuis -----Original Message----- From: monetdb-users-request(a)lists.sourceforge.net [mailto:monetdb-users-request@lists.sourceforge.net] Sent: vrijdag 10 december 2010 11:12 To: monetdb-users(a)lists.sourceforge.net Subject: MonetDB-users Digest, Vol 55, Issue 2 Send MonetDB-users mailing list submissions to monetdb-users(a)lists.sourceforge.net To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/monetdb-users or, via email, send a message with subject or body 'help' to monetdb-users-request(a)lists.sourceforge.net You can reach the person managing the list at monetdb-users-owner(a)lists.sourceforge.net When replying, please edit your Subject line so it is more specific than "Re: Contents of MonetDB-users digest..." Today's Topics: 1. COPY INTO: wrong length with backslashed chars? (Daniel Boesswetter) 2. Re: COPY INTO: wrong length with backslashed chars? (Stefan de Konink) 3. Some questions regarding parallelism and other stuff... (Mike De-La-Columnar) 4. Re: Some questions regarding parallelism and other stuff... (Martin Kersten) 5. Add primary key performance (paulr135) 6. Re: Add primary key performance (Stefan Manegold) 7. Re: COPY INTO: wrong length with backslashed chars? (Daniel Boesswetter) ---------------------------------------------------------------------- Message: 1 Date: Sun, 05 Dec 2010 19:08:01 +0100 From: Daniel Boesswetter <daniel.boesswetter(a)fu-berlin.de> Subject: [MonetDB-users] COPY INTO: wrong length with backslashed chars? To: monetdb-users(a)lists.sourceforge.net Message-ID: <4CFBD501.9030709(a)fu-berlin.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi all, I'm using MonetDB for the analysis for ~90GB of data which come from a MySQL-based web-application. The data is dumped into CSV-files on a regular basis and then imported into MonetDB with the COPY command. I have experienced some strange behaviour: obviously, MonetDB counts the number of chars for a VARCHAR-field before removing backslashes which are used e.g. for escaping newlines or other backslashes and rejects strings that should actually fit. A simple reproduction is this: 1. In mclient: CREATE TABLE t (x VARCHAR(1)); 2. Create a textfile foo.txt containing two backslashes: \\ 3. In mclient: sql>copy into t from '/tmp/foo.txt'; SQLException:importTable:value '\\' from line 1 field 1 not inserted, expecting type varchar(1) failed to import table Performing exactly the same steps with a VARCHAR(2) column works without errors and results in t.x containg the de-escaped single backslash. I'm using the Oct-2010 release. My current workaround is to enlarge VARCHARs which caused problems in the past, but this is of course no guarantee that it won't happen again. Although I found a few monetdb-bugs entries concerning the COPY command (e.g. about the misleading error message seen above), none seems to deal with my problem. Is this a bug? Best regards, Daniel ------------------------------ Message: 2 Date: Sun, 05 Dec 2010 19:15:36 +0100 From: Stefan de Konink <stefan(a)konink.de> Subject: Re: [MonetDB-users] COPY INTO: wrong length with backslashed chars? To: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Message-ID: <4CFBD6C8.30208(a)konink.de> Content-Type: text/plain; charset=ISO-8859-1 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 05-12-10 19:08, Daniel Boesswetter schreef: > Hi all, > > I'm using MonetDB for the analysis for ~90GB of data which come from a > MySQL-based web-application. The data is dumped into CSV-files on a > regular basis and then imported into MonetDB with the COPY command. I > have experienced some strange behaviour: obviously, MonetDB counts the > number of chars for a VARCHAR-field before removing backslashes which > are used e.g. for escaping newlines or other backslashes and rejects > strings that should actually fit. > > A simple reproduction is this: > > 1. In mclient: CREATE TABLE t (x VARCHAR(1)); 2. Create a textfile > foo.txt containing two backslashes: \\ 3. In mclient: > sql>copy into t from '/tmp/foo.txt'; > SQLException:importTable:value '\\' from line 1 field 1 not inserted, > expecting type varchar(1) failed to import table > > Performing exactly the same steps with a VARCHAR(2) column works > without errors and results in t.x containg the de-escaped single backslash. > > I'm using the Oct-2010 release. My current workaround is to enlarge > VARCHARs which caused problems in the past, but this is of course no > guarantee that it won't happen again. > > Although I found a few monetdb-bugs entries concerning the COPY > command (e.g. about the misleading error message seen above), none > seems to deal with my problem. > > Is this a bug? What are you escaping? Paste the line that fails ;) Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkz71sgACgkQYH1+F2Rqwn3AmgCfayhLg6zjXZgvqhV9OHJbj5Kh z64An2xZJy7H1wv3M6hr/SHQaK2C8IM3 =//V/ -----END PGP SIGNATURE----- ------------------------------ Message: 3 Date: Mon, 6 Dec 2010 05:07:46 -0800 (PST) From: Mike De-La-Columnar <mikevermondusa(a)gmail.com> Subject: [MonetDB-users] Some questions regarding parallelism and other stuff... To: monetdb-users(a)lists.sourceforge.net Message-ID: <30366715.post(a)talk.nabble.com> Content-Type: text/plain; charset=us-ascii Hi fellas, If its not too much of a hassle, i have a few questions regarding parallelism & the internal algorithms 1. What is the difference between Data-Flow, and Mitosis? 2. Is there a special reason why the distinct count aggregation is not parallelized just like the other aggregation functions? Is it just a matter of not making it into the release, or is there a fundamental problem here. Note: even though the parallel distinct can be slower in certain distributions, it could still be useful... 3. Are there any plans on supporting parallel frameworks like OpenMP/Intel TBB directly within the low-level MAL functions? Providing data-parallelism on the loop-code level could even increase performance. Or did you already play with it and decided that the performance gain is just not worth it? 4. Is there a way for me to prevent MonetDB server from cleaning up all intermediate BATs between server restarts. I've read here that some intermediates (like join indices etc.) are lost between restarts... is it still correct? Thank you very much for your time, Keep up the amazing job of building the most powerful columnar database out there! -- View this message in context: http://old.nabble.com/Some-questions-regarding-parallelism-and-other-stuff.… Sent from the monetdb-users mailing list archive at Nabble.com. ------------------------------ Message: 4 Date: Mon, 06 Dec 2010 20:13:55 +0100 From: Martin Kersten <Martin.Kersten(a)cwi.nl> Subject: Re: [MonetDB-users] Some questions regarding parallelism and other stuff... To: monetdb-users(a)lists.sourceforge.net Message-ID: <4CFD35F3.1060001(a)cwi.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed On 12/6/10 2:07 PM, Mike De-La-Columnar wrote: > > Hi fellas, Hi Mike > > If its not too much of a hassle, i have a few questions regarding > parallelism& the internal algorithms > > 1. > What is the difference between Data-Flow, and Mitosis? See optimizer descriptions: http://monetdb.cwi.nl/MonetDB/Documentation/Optimizer-Toolkit.html Orthogonal issues. > > 2. > Is there a special reason why the distinct count aggregation is not > parallelized just like the other aggregation functions? > Is it just a matter of not making it into the release, or is there a > fundamental problem here. Distinct is a global property, not a local one. A distributed version may easily lead to throwing the complete underlying table around. > > Note: even though the parallel distinct can be slower in certain > distributions, it could still be useful... > > 3. > Are there any plans on supporting parallel frameworks like OpenMP/Intel TBB > directly within the low-level MAL functions? > Providing data-parallelism on the loop-code level could even increase > performance. It could improve, but there should be a clear driving application. > Or did you already play with it and decided that the performance gain is > just not worth it? Depends on the amount of work per byte unit. > > 4. > Is there a way for me to prevent MonetDB server from cleaning up all > intermediate BATs between server restarts. > I've read here that some intermediates (like join indices etc.) are lost > between restarts... is it still correct? All temporaries are indeed lost. Join indices for foreign key checking are retained on disk. Rebuilding hashes is cheaper (mostly) then write/read them from disk > > > Thank you very much for your time, > Keep up the amazing job of building the most powerful columnar database out > there! Thanks for the support !. Martin > > > ------------------------------ Message: 5 Date: Tue, 7 Dec 2010 07:44:14 -0800 (PST) From: paulr135 <romesh135(a)gmail.com> Subject: [MonetDB-users] Add primary key performance To: monetdb-users(a)lists.sourceforge.net Message-ID: <30397515.post(a)talk.nabble.com> Content-Type: text/plain; charset=us-ascii I've loaded a 27GB (~165M rows) file into a table in MonetDB and now I am trying to add a primary key which consists of 4 fields - 2 varchars and 2 ints. It has been running for a few days (since Friday of last week). I've followed the instructions laid out on this post: http://old.nabble.com/MonetDB-SQL:-bulk-loading-large-data-sets---adding-ke… which involves setting the optimizer variable to "nov2009_pipe". I am unable to log in via mclient - I am able to put in my credentials but it just hangs at the "Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2010-SP2)" line. So I have a few questions: 1. Is there any way I can see the status of the alter table ... add primary key command? 2. Is it possible to stop the alter table gracefully? Or will I have to shut the db down? 3. Is there anything I should do in future to speed up adding of primary keys? Thanks. -- View this message in context: http://old.nabble.com/Add-primary-key-performance-tp30397515p30397515.html Sent from the monetdb-users mailing list archive at Nabble.com. ------------------------------ Message: 6 Date: Wed, 8 Dec 2010 09:38:23 +0100 From: Stefan Manegold <Stefan.Manegold(a)cwi.nl> Subject: Re: [MonetDB-users] Add primary key performance To: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Message-ID: <20101208083823.GA24931(a)cwi.nl> Content-Type: text/plain; charset=us-ascii On Tue, Dec 07, 2010 at 07:44:14AM -0800, paulr135 wrote: > > I've loaded a 27GB (~165M rows) file into a table in MonetDB and now I am > trying to add a primary key which consists of 4 fields - 2 varchars and 2 > ints. > It has been running for a few days (since Friday of last week). > > I've followed the instructions laid out on this post: > http://old.nabble.com/MonetDB-SQL:-bulk-loading-large-data-sets---adding-ke… > which involves setting the optimizer variable to "nov2009_pipe". Using the nov2009_pipe optimizer pipeline for adding constraint only holds for the Feb2010 release of MonetDB. It should not be required anymore for later releases as of Jun2010-SP1, incl. the latest Oct2010 release. Which version of MonetDB are you using? Which OS are you running on? What kind of manchine/hardware are you running on, in particular how much memory does it have? Can you check what the machine is doing? Is it running out of physical memory and swapping, i.e., is the virtual size of the mserver5 (much) larger than the machine's physical memory size? > I am unable to log in via mclient - I am able to put in my credentials but > it just hangs at the "Welcome to mclient, the MonetDB/SQL interactive > terminal (Jun2010-SP2)" line. > > So I have a few questions: > 1. Is there any way I can see the status of the alter table ... add primary > key command? You can use MonetDB's stethoscope tool to "listen" what the server is doing; however, while it is busy with a single (MAL-)operation, there will not be any output; hence, you might have to wait for some time to see some output. Unless very familiar with MonetDB internals, the output of stethoscope might not be too informative for you. But you're welcome to send us the output to have a look at it. In addition, you can check what the mserver5 is doing as suggested above. > 2. Is it possible to stop the alter table gracefully? Or will I have to > shut the db down? No, unfortunately there is no way to gracefully stop a running query. You'd have to way until it finishes (successfully or with an error), or shut down the DB. > 3. Is there anything I should do in future to speed up adding of primary > keys? In priciple not. We are aware that with data sizes (i.e., total size of all columns that the primary keys consists of) that significantly exceed main memory, adding the key (i.e., checking whether the data fulfills the key condition) does not work as efficiently as it could/should, in particular with multi-column keys --- we're working on that ... You could, though, run the ALTER TABLE statement in trace mode, i.e., `TRACE ALTER TABLE ...`, collect the output and send it to us for analysis; that could help us to improve the creation of large multi-column primary keys. Regards, Stefan > Thanks. > -- > View this message in context: http://old.nabble.com/Add-primary-key-performance-tp30397515p30397515.html > Sent from the monetdb-users mailing list archive at Nabble.com. > > > ------------------------------------------------------------------------------ > What happens now with your Lotus Notes apps - do you make another costly > upgrade, or settle for being marooned without product support? Time to move > off Lotus Notes and onto the cloud with Force.com, apps are easier to build, > use, and manage than apps on traditional platforms. Sign up for the Lotus > Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > > > > > > > > > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ------------------------------ Message: 7 Date: Fri, 10 Dec 2010 11:11:34 +0100 From: Daniel Boesswetter <daniel.boesswetter(a)fu-berlin.de> Subject: Re: [MonetDB-users] COPY INTO: wrong length with backslashed chars? To: monetdb-users(a)lists.sourceforge.net Message-ID: <4D01FCD6.9040201(a)fu-berlin.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi Stefan, > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Op 05-12-10 19:08, Daniel Boesswetter schreef: > > Hi all, > > > > I'm using MonetDB for the analysis for ~90GB of data which come from a > > MySQL-based web-application. The data is dumped into CSV-files on a > > regular basis and then imported into MonetDB with the COPY command. I > > have experienced some strange behaviour: obviously, MonetDB counts the > > number of chars for a VARCHAR-field before removing backslashes which > > are used e.g. for escaping newlines or other backslashes and rejects > > strings that should actually fit. > > > > A simple reproduction is this: > > > > 1. In mclient: CREATE TABLE t (x VARCHAR(1)); > > 2. Create a textfile foo.txt containing two backslashes: \\ > > 3. In mclient: > > sql>copy into t from '/tmp/foo.txt'; > > SQLException:importTable:value '\\' from line 1 field 1 not inserted, > > expecting type varchar(1) > > failed to import table > > > > Performing exactly the same steps with a VARCHAR(2) column works without > > errors and results in t.x containg the de-escaped single backslash. > > > > I'm using the Oct-2010 release. My current workaround is to enlarge > > VARCHARs which caused problems in the past, but this is of course no > > guarantee that it won't happen again. > > > > Although I found a few monetdb-bugs entries concerning the COPY command > > (e.g. about the misleading error message seen above), none seems to deal > > with my problem. > > > > Is this a bug? > > What are you escaping? Paste the line that fails ;) > > As I said: try to import a single "\" into a VARCHAR(1) with COPY. Writing "\" does not work because it escapes the newline. Writing "\\" does not work because it is longer than 1 char. Regards, Daniel > Stefan > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.16 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEAREKAAYFAkz71sgACgkQYH1+F2Rqwn3AmgCfayhLg6zjXZgvqhV9OHJbj5Kh > z64An2xZJy7H1wv3M6hr/SHQaK2C8IM3 > =//V/ > -----END PGP SIGNATURE----- > ------------------------------ ------------------------------------------------------------------------------ ------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users End of MonetDB-users Digest, Vol 55, Issue 2 ********************************************

1 0

[MonetDB-users] COPY INTO: wrong length with backslashed chars?
by Daniel Boesswetter 10 Dec '10

10 Dec '10

Hi all, I'm using MonetDB for the analysis for ~90GB of data which come from a MySQL-based web-application. The data is dumped into CSV-files on a regular basis and then imported into MonetDB with the COPY command. I have experienced some strange behaviour: obviously, MonetDB counts the number of chars for a VARCHAR-field before removing backslashes which are used e.g. for escaping newlines or other backslashes and rejects strings that should actually fit. A simple reproduction is this: 1. In mclient: CREATE TABLE t (x VARCHAR(1)); 2. Create a textfile foo.txt containing two backslashes: \\ 3. In mclient: sql>copy into t from '/tmp/foo.txt'; SQLException:importTable:value '\\' from line 1 field 1 not inserted, expecting type varchar(1) failed to import table Performing exactly the same steps with a VARCHAR(2) column works without errors and results in t.x containg the de-escaped single backslash. I'm using the Oct-2010 release. My current workaround is to enlarge VARCHARs which caused problems in the past, but this is of course no guarantee that it won't happen again. Although I found a few monetdb-bugs entries concerning the COPY command (e.g. about the misleading error message seen above), none seems to deal with my problem. Is this a bug? Best regards, Daniel

3 4

[MonetDB-users] Some questions regarding parallelism and other stuff...
by Mike De-La-Columnar 06 Dec '10

06 Dec '10

Hi fellas, If its not too much of a hassle, i have a few questions regarding parallelism & the internal algorithms 1. What is the difference between Data-Flow, and Mitosis? 2. Is there a special reason why the distinct count aggregation is not parallelized just like the other aggregation functions? Is it just a matter of not making it into the release, or is there a fundamental problem here. Note: even though the parallel distinct can be slower in certain distributions, it could still be useful... 3. Are there any plans on supporting parallel frameworks like OpenMP/Intel TBB directly within the low-level MAL functions? Providing data-parallelism on the loop-code level could even increase performance. Or did you already play with it and decided that the performance gain is just not worth it? 4. Is there a way for me to prevent MonetDB server from cleaning up all intermediate BATs between server restarts. I've read here that some intermediates (like join indices etc.) are lost between restarts... is it still correct? Thank you very much for your time, Keep up the amazing job of building the most powerful columnar database out there! -- View this message in context: http://old.nabble.com/Some-questions-regarding-parallelism-and-other-stuff.… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] MonetDB/XQuery Installation
by Rob Fynes 01 Dec '10

01 Dec '10

Hello, Apologies if this is the improper place to be asking this. I'm a new user to MonetDB, and I'm having trouble installing MonetDB/XQuery on Ubuntu 10.04 32-bit. Actually, the installation did seem to succeed (running the auto install script from the super source tarball) and I can start MServer, but with the error: # MonetDB Server v4.40.2 # based on GDK v1.40.2 # not released # Copyright (c) 1993-July 2008, CWI. All rights reserved. # Copyright (c) August 2008-2010, MonetDB B.V.. All rights reserved. # Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs; dynamically linked. # Detected 3.919 GiB (4208013312 bytes) main-memory. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: moduleClient: module(pathfinder) load error. The problem, its seems anyway, is during the configuration of pathfinder: ... ... checking for signal.h... yes checking return type of signal handlers... (cached) void checking for strings.h... (cached) yes checking for strcasecmp... yes checking for xml2-config.32... no checking for xml2-32-config... no checking for xml2-config... no checking for libxml/parser.h... no configure: error: in `/home/rob/MonetDB': configure: error: The libxml2 XML parsing library and/or include files were not found or you have requested to not use it. This is required dependence, core components like the XML Schema import functionality depends on it. See `config.log' for more details. I do have libxml2 installed (by default with Lucid). I've run the pathfinder configure script a few times, with the option --with-libxml2=*some variations*, but I'm not really confident about what I'm doing. I was wondering if anyone could point me in the right direction. Thanks. -Rob

3 5

[MonetDB-users] Granting TMP Table Creation Privileges to User
by Reeves, Matthew J. 30 Nov '10

30 Nov '10

I'm not sure what I'm missing but how does one grant permissions to a user to create temporary tables? I'm totally stumped. Thanks, Matt

1 0

[MonetDB-users] issue with concurrent access to MonetDB/SQL via JDBC
by Wouter Alink 26 Nov '10

26 Nov '10

Dear users, I'm currently struggling with concurrent access to MonetDB/SQL via JDBC. Version: MonetDB server v5.20.4 (64-bit), based on kernel v1.38.4 (64-bit oids). I might still have some concurrency issues with my own code, but somehow I frequently end up with the following error message: (while stress-testing by querying with 10 threads at the same time) class java.sql.SQLException: TypeException:user.s3_1[448]:'user.contains' undefined in: _1933:any := user.contains(_1929:str, _1931:str, _474:bit) TypeException:user.s3_1[449]:'bat.insert' undefined in: _1934:any := bat.insert(_1923:bat[:oid,:bit], _1928:oid, _1933:any) TypeException:user.s3_1[645]:'user.contains' undefined in: _1946:any := user.contains(_1942:str, _1944:str, _474:bit) TypeException:user.s3_1[646]:'bat.insert' undefined in: _1947:any := bat.insert(_1938:bat[:oid,:bit], _1941:oid, _1946:any) TypeException:user.s3_1[1042]:'user.contains' undefined in: _1959:any := user.contains(_1955:str, _1957:str, _474:bit) TypeException:user.s3_1[1043]:'bat.insert' undefined in: _1960:any := bat.insert(_1951:bat[:oid,:bit], _1954:oid, _1959:any) TypeException:user.s3_1[1226]:'user.contains' undefined in: _1974:any := user.contains(_1970:str, _1972:str, _474:bit) TypeException:user.s3_1[1227]:'bat.insert' undefined in: _1975:any := bat.insert(_1964:bat[:oid,:bit], _1969:oid, _1974:any) SQLException:SQLengine:Program contains errors nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2209) nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:1968) nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:506) nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:347) nl.cwi.monetdb.jdbc.MonetStatement.executeUpdate(MonetStatement.java:542) Does anyone else have experienced such error messages before, or does anyone have a clue to where to look? (like too many open connections, concurrent access of the same connection, querying a closed connection, or perhaps a bug inside monetdb)? Thanks, Wouter

2 1

[MonetDB-users] MonetDB Debain and RDF
by Andreas Harth 25 Nov '10

25 Nov '10

Hi, I'd like to use Lefteris' RDF package on Debian. What do I have to do to create and connect to a database as user on Debian? I've installed the following packages: # dpkg -l | grep -e monetdb ii libmonetdb-client1 1.40.1-20101110 MonetDB database client libraries ii libmonetdb1 1.40.1-20101110 database system for high-performance applications, libraries ii libmonetdb5-server5 5.22.1-20101110 MonetDB database server version 5, libraries ii libmonetdb5-sql2 2.40.1-20101110 MonetDB SQL libraries for monetdb5-server ii monetdb-client 1.40.1-20101110 MonetDB database client ii monetdb5-server 5.22.1-20101110 MonetDB database server version 5 ii monetdb5-sql 2.40.1-20101110 MonetDB SQL support for monetdb5 I can start MonetDB: # /etc/init.d/monetdb5-sql start Starting MonetDB SQL server: merovingian. When I try do create the tables (as a user): $ mclient -lsql create_tables.sql user(aharth): password: merovingian: please specify a database $ $ mclient -lsql create_tables.sql -d demo user(aharth): password: merovingian: internal error while starting mserver, please refer to the logs $ The log says: 2010-11-25 16:20:19 MSG merovingian[3685]: starting database 'demo', up min/avg/max: 3s/4s/5s, crash average: 0.00 0.00 0.00 (2-2=0) 2010-11-25 16:20:19 ERR merovingian[3685]: client error: cannot start database 'demo': no .vaultkey found (did you create the database with `monetdb create demo`?) I'm stuck here. How can I create a database as a user? $ monetdb create demo2 permission denied $ As root, I only can create a database in maintainance mode: # monetdb create demo2 created database in maintenance mode: demo2 # Any pointers to howto's that'll help? Cheers, Andreas.

3 5

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 23 Nov '10

23 Nov '10

I see --- if you consider it a bug, please file a detailed bug report including information & instructions to reporduce the problem as well as infornmation about the exact MonetDB version and OS you're using via http://bugs.MonetDB.org/ Stefan ----- Oorspronkelijk bericht ----- Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> Verzonden: maandag 25 oktober 2010 16:32 Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB Hi Stefan, Nope, the name of the column is "deliveryDays" :) Freddy 2010/10/25 Stefan Manegold <Stefan.Manegold(a)cwi.nl> > Hi Freddy, > > I don't think that the error you experience is triggered by the column > type, but rather by the fact that your date column ha? a name - say "date" - > that is a reserved key word in SQL an hence needs to be double-quoted > according to the SQL standard (this is not a MonetDB specific choice, but a > "feature" of the SQL standard) --- not knowing your table & column names, I > can only guess, though ... > > Stefan > > > ----- Oorspronkelijk bericht ----- > Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > Verzonden: maandag 25 oktober 2010 16:03 > Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> > Onderwerp: [MonetDB-users] Querying date type column in MonetDB > > Dear All, > > I am new to MonetDB, so most likely my question will look obvious. > > I would like to ask, how can columns with date type is queried in monetdb. > Based on my experience, it seems that date type columns need special > treatment. It seems that the only way to query them is with the syntax > "column_name" while other columns seems to work with the format > table_name.column_name without the need of double quote. When I tried to > query date column with the syntax table_name.column_name, it gave me an > error "no such column" > > So my question is, is it true that the only way to query date columns is > using "column_name" format? Another question will be, what happen if I need > to query same name columns coming from different table? > > Thanks in advance, > Freddy > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users >

4 6

[MonetDB-users] Copy into issues with mixed ascii / hex data
by paulr135 23 Nov '10

23 Nov '10

I have a tab-delimited data file that is mostly ascii, but has a few hex characters in it. For example: ... 3263640152 ABC<99> ... where <99> is a hex value for (tm) - trademark. When I try and load this data file into monetdb, I get the following error: [root@bidbtest ~]# /usr/local/MonetDB/bin/mclient -lsql --database=myschema -t -s "COPY 20603 records into test_table from '/tmp/testdata.txt' using delimiters '\t','\n','\"' null as '';" user(root):monetdb password: MAPI = (monetdb) /usr/local/MonetDB/var/MonetDB5/dbfarm/mapi_socket QUERY = COPY 20603 records into test_table from '/tmp/testdata.txt' using delimiters '\t','\n','"' null as ''; ERROR = !SQLException:importTable:value 'failed to import table Timer 790.415 msec The table def is: CREATE TABLE creative_dim ( id bigint NOT NULL, headline varchar(100) DEFAULT NULL ) How can I go about loading this file? Ideally, I would like to get these characters loaded as is. I saw that mclient has a -E flag that accepts an encoding type. However, I'm not sure what to set it to for a file with ascii / hex data. A complicating factor is that I don't know all the hex characters that are in my file. I luckily spotted the <99>/tm one above as it was one of the first lines in the file. But after replacing all instances of that character with the empty string, I still get a similar error to what I pasted above. I manually paged through the file and found different hex characters. It isn't practical for me to page through an entire file and monetdb doesn't tell me which line number the problem is on. By the way, I am using MonetDB v5.20.5. Thanks for the help! -- View this message in context: http://old.nabble.com/Copy-into-issues-with-mixed-ascii---hex-data-tp302389… Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] compiling from hg
by Dean Serenevy 18 Nov '10

18 Nov '10

Hello, I am attempting to compile from mercurial sources. I have tried from the Oct2010_1 tag as well as tip and both error out at the same point (I ran de-bootstrap between attempts). As far as I can tell i have all prerequisites installed. Full command output is included at the end of the message. Can anyone tell me what I am doing wrong? Thaks, Dean [MonetDB.hg/buildtools] (hg:default)$ hg update tip 0 files updated, 0 files merged, 0 files removed, 0 files unresolved [MonetDB.hg/buildtools] (hg:default)$ ./bootstrap [MonetDB.build/buildtools]$ ../MonetDB.hg/buildtools/configure --prefix=/opt/MonetDB --enable-burg --enable-mel --enable-debug=no --enable-assert=no --enable-optimize=yes [MonetDB.build/buildtools]$ make -j4 ... g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.yy.o -MD -MP -MF .deps/mel.yy.Tpo -c -o mel.yy.o mel.yy.cc y.tab.c:178:1: warning: "MEL_MODULE" redefined In file included from /home/duelafn/MonetDB.hg/buildtools/mel/command.h:26, from /home/duelafn/MonetDB.hg/buildtools/mel/mel.yy:47: /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:1:1: warning: this is the location of the previous definition y.tab.c:179:1: warning: "MEL_PRELUDE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:2:1: warning: this is the location of the previous definition y.tab.c:180:1: warning: "MEL_EPILOGUE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:3:1: warning: this is the location of the previous definition y.tab.c:181:1: warning: "MEL_END" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:4:1: warning: this is the location of the previous definition y.tab.c:182:1: warning: "MEL_COMMAND" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:5:1: warning: this is the location of the previous definition y.tab.c:183:1: warning: "MEL_OPERATOR0" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:6:1: warning: this is the location of the previous definition y.tab.c:184:1: warning: "MEL_OPERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:7:1: warning: this is the location of the previous definition y.tab.c:185:1: warning: "MEL_OPERATOR1" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:8:1: warning: this is the location of the previous definition y.tab.c:186:1: warning: "MEL_FIXEDATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:9:1: warning: this is the location of the previous definition y.tab.c:187:1: warning: "MEL_ATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:10:1: warning: this is the location of the previous definition y.tab.c:188:1: warning: "MEL_USE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:11:1: warning: this is the location of the previous definition y.tab.c:189:1: warning: "MEL_ATOMOP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:12:1: warning: this is the location of the previous definition y.tab.c:190:1: warning: "MEL_STRING" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:13:1: warning: this is the location of the previous definition y.tab.c:191:1: warning: "MEL_IDENTIFIER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:14:1: warning: this is the location of the previous definition y.tab.c:192:1: warning: "MEL_USED" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:15:1: warning: this is the location of the previous definition y.tab.c:193:1: warning: "MEL_NUMBER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:16:1: warning: this is the location of the previous definition y.tab.c:194:1: warning: "MEL_ANY" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:17:1: warning: this is the location of the previous definition y.tab.c:195:1: warning: "MEL_BAT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:18:1: warning: this is the location of the previous definition y.tab.c:196:1: warning: "MEL_TYPE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:19:1: warning: this is the location of the previous definition y.tab.c:197:1: warning: "MEL_VOID" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:20:1: warning: this is the location of the previous definition y.tab.c:198:1: warning: "MEL_BUILTIN" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:21:1: warning: this is the location of the previous definition y.tab.c:199:1: warning: "MEL_OBJECT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:22:1: warning: this is the location of the previous definition y.tab.c:200:1: warning: "MEL_VARARGS" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:23:1: warning: this is the location of the previous definition y.tab.c:201:1: warning: "MEL_SEP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:24:1: warning: this is the location of the previous definition y.tab.c:202:1: warning: "MEL_ITERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:25:1: warning: this is the location of the previous definition mv -f .deps/glue.Tpo .deps/glue.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mil.o -MD -MP -MF .deps/mil.Tpo -c -o mil.o mil.cc y.tab.c:150: error: expected identifier before numeric constant y.tab.c:150: error: expected ‘}’ before numeric constant y.tab.c:150: error: expected unqualified-id before numeric constant y.tab.c:175: error: expected declaration before ‘}’ token make[3]: *** [mel.tab.o] Error 1 make[3]: *** Waiting for unfinished jobs.... mv -f .deps/mel.Tpo .deps/mel.Po mv -f .deps/mil.Tpo .deps/mil.Po mv -f .deps/mel.yy.Tpo .deps/mel.yy.Po make[3]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/duelafn/MonetDB.build' make: *** [all] Error 2 FULL COMMAND OUTPUT [MonetDB.hg/buildtools] (hg:default)$ ./bootstrap configure.ac:24: installing `./install-sh' configure.ac:24: installing `./missing' Makefile.am: installing `./depcomp' configure.ac:24: installing `./install-sh' configure.ac:24: installing `./missing' Makefile.am: installing `./depcomp' configure.ac:24: installing `./install-sh' configure.ac:24: installing `./missing' Makefile.am: installing `./depcomp' configure.ac:25: installing `./install-sh' configure.ac:25: installing `./missing' [MonetDB.build/buildtools]$ ../MonetDB.hg/buildtools/configure --prefix=/opt/MonetDB --enable-burg --enable-mel --enable-debug=no --enable-assert=no --enable-optimize=yes checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for python... /usr/bin/python configure: WARNING: unrecognized options: --enable-burg, --enable-mel, --enable-debug, --enable-assert, --enable-optimize checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make sets $(MAKE)... (cached) yes checking for g++... g++ checking whether the C++ compiler works... yes checking for C++ compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking for style of include used by make... GNU checking dependency style of g++... gcc3 checking for gcc... gcc checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking dependency style of gcc... gcc3 checking for lockfile... echo checking for flex... flex checking lex output file root... lex.yy checking lex library... -lfl checking whether yytext is a pointer... yes checking for bison... bison -y checking whether ln -s works... yes checking for rm... rm -f checking for mv... mv -f checking for flex version... flex 2.5.35 checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking whether time.h and sys/time.h may both be included... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking malloc.h usability... yes checking malloc.h presence... yes checking for malloc.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking for unistd.h... (cached) yes checking for iostream... yes checking for cstdio... yes checking for an ANSI C-conforming const... yes checking for inline... inline checking for size_t... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for memset... yes checking for strchr... yes checking for strdup... yes checking for strrchr... yes configure: creating ./config.status config.status: creating Makefile config.status: creating mel_config.h config.status: executing depfiles commands configure: WARNING: unrecognized options: --enable-burg, --enable-mel, --enable-debug, --enable-assert, --enable-optimize configure: creating ./config.status config.status: creating Makefile config.status: creating autogen/Makefile config.status: creating conf/Makefile config.status: creating conf/Mbootstrap config.status: creating conf/buildtools-config config.status: creating dummy.h === configuring in Mx (/home/duelafn/MonetDB.build/Mx) configure: running /bin/bash ../../MonetDB.hg/buildtools/Mx/configure --disable-option-checking '--prefix=/opt/MonetDB' '--enable-burg' '--enable-mel' '--enable-debug=no' '--enable-assert=no' '--enable-optimize=yes' --cache-file=/dev/null --srcdir=../../MonetDB.hg/buildtools/Mx checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for bash... /bin/bash checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking getopt.h usability... yes checking getopt.h presence... yes checking for getopt.h... yes checking limits.h usability... yes checking limits.h presence... yes checking for limits.h... yes checking stddef.h usability... yes checking stddef.h presence... yes checking for stddef.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking for unistd.h... (cached) yes checking utime.h usability... yes checking utime.h presence... yes checking for utime.h... yes checking whether stat file-mode macros are broken... no checking for an ANSI C-conforming const... yes checking for size_t... yes checking for ptrdiff_t... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible realloc... yes checking whether lstat correctly handles trailing slash... yes checking whether stat accepts an empty string... no checking for utime.h... (cached) yes checking whether utime accepts a null argument... yes checking for vprintf... yes checking for _doprnt... no checking for memset... yes checking for mkdir... yes checking for strchr... yes checking for strcspn... yes checking for strrchr... yes checking for strstr... yes checking for utime... yes checking whether getopt is declared... yes configure: creating ./config.status config.status: creating Makefile config.status: creating mx_config.h config.status: executing depfiles commands === configuring in burg (/home/duelafn/MonetDB.build/burg) configure: running /bin/bash ../../MonetDB.hg/buildtools/burg/configure --disable-option-checking '--prefix=/opt/MonetDB' '--enable-burg' '--enable-mel' '--enable-debug=no' '--enable-assert=no' '--enable-optimize=yes' --cache-file=/dev/null --srcdir=../../MonetDB.hg/buildtools/burg checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for flex... flex checking lex output file root... lex.yy checking lex library... -lfl checking whether yytext is a pointer... yes checking for bison... bison -y checking for lockfile... echo checking for rm... rm -f checking for mv... mv -f checking for flex version... flex 2.5.35 checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking stdbool.h usability... yes checking stdbool.h presence... yes checking for stdbool.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking for stdbool.h that conforms to C99... (cached) yes checking for _Bool... yes checking for an ANSI C-conforming const... yes checking for size_t... yes checking for long long... yes checking for __int64... no checking size of void *... 8 checking size of int... 4 checking size of long... 8 checking size of long long... 8 checking size of __int64... 0 checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for memset... yes checking for strdup... yes configure: creating ./config.status config.status: creating Makefile config.status: creating burg_config.h config.status: executing depfiles commands [MonetDB.build/buildtools]$ make -j4 (CDPATH="${ZSH_VERSION+.}:" && cd ../MonetDB.hg/buildtools && /bin/bash /home/duelafn/MonetDB.hg/buildtools/missing --run autoheader) rm -f stamp-h1 touch ../MonetDB.hg/buildtools/dummy.h.in cd . && /bin/bash ./config.status dummy.h config.status: creating dummy.h config.status: dummy.h is unchanged /usr/bin/make all-recursive make[1]: Entering directory `/home/duelafn/MonetDB.build' Making all in autogen make[2]: Entering directory `/home/duelafn/MonetDB.build/autogen' mkdir -p /home/duelafn/MonetDB.build/autogen/autogen cp ../../MonetDB.hg/buildtools/autogen/setup.py /home/duelafn/MonetDB.build/autogen/setup.py cp ../../MonetDB.hg/buildtools/autogen/autogen.py /home/duelafn/MonetDB.build/autogen/autogen.py cp ../../MonetDB.hg/buildtools/autogen/autogen/am.py /home/duelafn/MonetDB.build/autogen/autogen/am.py cp ../../MonetDB.hg/buildtools/autogen/autogen/msc.py /home/duelafn/MonetDB.build/autogen/autogen/msc.py cp ../../MonetDB.hg/buildtools/autogen/autogen/var.py /home/duelafn/MonetDB.build/autogen/autogen/var.py cp ../../MonetDB.hg/buildtools/autogen/autogen/codegen.py /home/duelafn/MonetDB.build/autogen/autogen/codegen.py cp ../../MonetDB.hg/buildtools/autogen/autogen/__init__.py /home/duelafn/MonetDB.build/autogen/autogen/__init__.py /usr/bin/python setup.py build running build running build_py creating build creating build/lib.linux-x86_64-2.6 creating build/lib.linux-x86_64-2.6/autogen copying autogen/var.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/codegen.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/__init__.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/am.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/msc.py -> build/lib.linux-x86_64-2.6/autogen running build_scripts creating build/scripts-2.6 copying and adjusting autogen.py -> build/scripts-2.6 changing mode of build/scripts-2.6/autogen.py from 644 to 755 make[2]: Leaving directory `/home/duelafn/MonetDB.build/autogen' Making all in Mx make[2]: Entering directory `/home/duelafn/MonetDB.build/Mx' (CDPATH="${ZSH_VERSION+.}:" && cd ../../MonetDB.hg/buildtools/Mx && /bin/bash /home/duelafn/MonetDB.hg/buildtools/Mx/missing --run autoheader) rm -f stamp-h1 touch ../../MonetDB.hg/buildtools/Mx/mx_config.h.in cd . && /bin/bash ./config.status mx_config.h config.status: creating mx_config.h config.status: mx_config.h is unchanged /usr/bin/make all-am make[3]: Entering directory `/home/duelafn/MonetDB.build/Mx' gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Mx.o -MD -MP -MF .deps/Mx.Tpo -c -o Mx.o ../../MonetDB.hg/buildtools/Mx/Mx.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Def.o -MD -MP -MF .deps/Def.Tpo -c -o Def.o ../../MonetDB.hg/buildtools/Mx/Def.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Form.o -MD -MP -MF .deps/Form.Tpo -c -o Form.o ../../MonetDB.hg/buildtools/Mx/Form.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Tok.o -MD -MP -MF .deps/Tok.Tpo -c -o Tok.o ../../MonetDB.hg/buildtools/Mx/Tok.c mv -f .deps/Mx.Tpo .deps/Mx.Po mv -f .deps/Tok.Tpo .deps/Tok.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Code.o -MD -MP -MF .deps/Code.Tpo -c -o Code.o ../../MonetDB.hg/buildtools/Mx/Code.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Display.o -MD -MP -MF .deps/Display.Tpo -c -o Display.o ../../MonetDB.hg/buildtools/Mx/Display.c mv -f .deps/Form.Tpo .deps/Form.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Print.o -MD -MP -MF .deps/Print.Tpo -c -o Print.o ../../MonetDB.hg/buildtools/Mx/Print.c mv -f .deps/Display.Tpo .deps/Display.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Io.o -MD -MP -MF .deps/Io.Tpo -c -o Io.o ../../MonetDB.hg/buildtools/Mx/Io.c mv -f .deps/Def.Tpo .deps/Def.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Sys.o -MD -MP -MF .deps/Sys.Tpo -c -o Sys.o ../../MonetDB.hg/buildtools/Mx/Sys.c mv -f .deps/Print.Tpo .deps/Print.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT disclaimer.o -MD -MP -MF .deps/disclaimer.Tpo -c -o disclaimer.o ../../MonetDB.hg/buildtools/Mx/disclaimer.c mv -f .deps/Code.Tpo .deps/Code.Po mv -f .deps/Sys.Tpo .deps/Sys.Po mv -f .deps/disclaimer.Tpo .deps/disclaimer.Po mv -f .deps/Io.Tpo .deps/Io.Po gcc -g -O2 -o Mx Mx.o Def.o Form.o Tok.o Code.o Display.o Print.o Io.o Sys.o disclaimer.o make[3]: Leaving directory `/home/duelafn/MonetDB.build/Mx' make[2]: Leaving directory `/home/duelafn/MonetDB.build/Mx' Making all in mel make[2]: Entering directory `/home/duelafn/MonetDB.build/mel' (CDPATH="${ZSH_VERSION+.}:" && cd /home/duelafn/MonetDB.hg/buildtools/mel && /bin/bash /home/duelafn/MonetDB.hg/buildtools/mel/missing --run autoheader) rm -f stamp-h1 touch /home/duelafn/MonetDB.hg/buildtools/mel/mel_config.h.in cd . && /bin/bash ./config.status mel_config.h config.status: creating mel_config.h config.status: mel_config.h is unchanged /usr/bin/make all-am make[3]: Entering directory `/home/duelafn/MonetDB.build/mel' ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/any_arg.mx /home/duelafn/MonetDB.hg/buildtools/mel/any_arg.mx: ./any_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/atom_arg.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/atom.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/atomops.mx /home/duelafn/MonetDB.hg/buildtools/mel/atom_arg.mx: ./atom_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/bat_arg.mx /home/duelafn/MonetDB.hg/buildtools/mel/atom.mx: ./atom.cc - created /home/duelafn/MonetDB.hg/buildtools/mel/atomops.mx: ./atomops.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/builtin.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/command.mx /home/duelafn/MonetDB.hg/buildtools/mel/bat_arg.mx: ./bat_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/debug.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/depend.mx /home/duelafn/MonetDB.hg/buildtools/mel/builtin.mx: ./builtin.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/dependency.mx /home/duelafn/MonetDB.hg/buildtools/mel/command.mx: ./command.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/epilogue.mx /home/duelafn/MonetDB.hg/buildtools/mel/debug.mx: ./debug.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/FileInfo.mx /home/duelafn/MonetDB.hg/buildtools/mel/depend.mx: ./depend.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/glue.mx /home/duelafn/MonetDB.hg/buildtools/mel/dependency.mx: ./dependency.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/html.mx /home/duelafn/MonetDB.hg/buildtools/mel/epilogue.mx: ./epilogue.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/iterator.mx /home/duelafn/MonetDB.hg/buildtools/mel/FileInfo.mx: ./FileInfo.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/link.mx /home/duelafn/MonetDB.hg/buildtools/mel/html.mx: ./html.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/ListIterator.mx /home/duelafn/MonetDB.hg/buildtools/mel/iterator.mx: ./iterator.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/mel.mx /home/duelafn/MonetDB.hg/buildtools/mel/ListIterator.mx: ./ListIterator.cc - created echo waiting /home/duelafn/MonetDB.hg/buildtools/mel/link.mx: ./link.cc - created flex /home/duelafn/MonetDB.hg/buildtools/mel/mel.ll waiting bison -y -d /home/duelafn/MonetDB.hg/buildtools/mel/mel.yy || { rm -f waiting ; exit 1 ; } /home/duelafn/MonetDB.hg/buildtools/mel/mel.mx: ./mel.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/mil.mx /home/duelafn/MonetDB.hg/buildtools/mel/glue.mx: ./glue.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/module.mx /home/duelafn/MonetDB.hg/buildtools/mel/mil.mx: ./mil.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/object.mx /home/duelafn/MonetDB.hg/buildtools/mel/module.mx: ./module.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/operator.mx conflicts: 32 reduce/reduce /home/duelafn/MonetDB.hg/buildtools/mel/object.mx: ./object.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/ops.mx /home/duelafn/MonetDB.hg/buildtools/mel/operator.mx: ./operator.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/prelude.mx /home/duelafn/MonetDB.hg/buildtools/mel/prelude.mx: ./prelude.cc - created /home/duelafn/MonetDB.hg/buildtools/mel/ops.mx: ./ops.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/proto.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/symbol.mx if [ -f lex.yy.c ]; then mv lex.yy.c mel.yy.cc ; fi /home/duelafn/MonetDB.hg/buildtools/mel/proto.mx: ./proto.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/symtable.mx /home/duelafn/MonetDB.hg/buildtools/mel/symbol.mx: ./symbol.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/type_arg.mx mv mel.yy.cc mel.yy.cc.tmp /home/duelafn/MonetDB.hg/buildtools/mel/symtable.mx: ./symtable.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/use.mx /home/duelafn/MonetDB.hg/buildtools/mel/type_arg.mx: ./type_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/var_arg.mx /home/duelafn/MonetDB.hg/buildtools/mel/use.mx: ./use.cc - created echo '#include <'"mel_config.h"'>' > mel.yy.cc grep -v '^#include.*[<"]'"mel_config.h"'[">]' mel.yy.cc.tmp >> mel.yy.cc g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT any_arg.o -MD -MP -MF .deps/any_arg.Tpo -c -o any_arg.o any_arg.cc /home/duelafn/MonetDB.hg/buildtools/mel/var_arg.mx: ./var_arg.cc - created g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT atom_arg.o -MD -MP -MF .deps/atom_arg.Tpo -c -o atom_arg.o atom_arg.cc rm -f mel.yy.cc.tmp g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT atom.o -MD -MP -MF .deps/atom.Tpo -c -o atom.o atom.cc if [ -f y.tab.c ]; then mv y.tab.c mel.tab.cc ; fi rm -f waiting g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT atomops.o -MD -MP -MF .deps/atomops.Tpo -c -o atomops.o atomops.cc mv -f .deps/atom.Tpo .deps/atom.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT bat_arg.o -MD -MP -MF .deps/bat_arg.Tpo -c -o bat_arg.o bat_arg.cc mv -f .deps/any_arg.Tpo .deps/any_arg.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT builtin.o -MD -MP -MF .deps/builtin.Tpo -c -o builtin.o builtin.cc mv -f .deps/atom_arg.Tpo .deps/atom_arg.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT command.o -MD -MP -MF .deps/command.Tpo -c -o command.o command.cc mv -f .deps/atomops.Tpo .deps/atomops.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT debug.o -MD -MP -MF .deps/debug.Tpo -c -o debug.o debug.cc mv -f .deps/bat_arg.Tpo .deps/bat_arg.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT depend.o -MD -MP -MF .deps/depend.Tpo -c -o depend.o depend.cc mv -f .deps/builtin.Tpo .deps/builtin.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT dependency.o -MD -MP -MF .deps/dependency.Tpo -c -o dependency.o dependency.cc mv -f .deps/command.Tpo .deps/command.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT epilogue.o -MD -MP -MF .deps/epilogue.Tpo -c -o epilogue.o epilogue.cc mv -f .deps/debug.Tpo .deps/debug.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT FileInfo.o -MD -MP -MF .deps/FileInfo.Tpo -c -o FileInfo.o FileInfo.cc mv -f .deps/depend.Tpo .deps/depend.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT glue.o -MD -MP -MF .deps/glue.Tpo -c -o glue.o glue.cc mv -f .deps/dependency.Tpo .deps/dependency.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT html.o -MD -MP -MF .deps/html.Tpo -c -o html.o html.cc mv -f .deps/epilogue.Tpo .deps/epilogue.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT iterator.o -MD -MP -MF .deps/iterator.Tpo -c -o iterator.o iterator.cc mv -f .deps/FileInfo.Tpo .deps/FileInfo.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT link.o -MD -MP -MF .deps/link.Tpo -c -o link.o link.cc mv -f .deps/iterator.Tpo .deps/iterator.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT ListIterator.o -MD -MP -MF .deps/ListIterator.Tpo -c -o ListIterator.o ListIterator.cc mv -f .deps/ListIterator.Tpo .deps/ListIterator.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.o -MD -MP -MF .deps/mel.Tpo -c -o mel.o mel.cc mv -f .deps/html.Tpo .deps/html.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.tab.o -MD -MP -MF .deps/mel.tab.Tpo -c -o mel.tab.o mel.tab.cc mv -f .deps/link.Tpo .deps/link.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.yy.o -MD -MP -MF .deps/mel.yy.Tpo -c -o mel.yy.o mel.yy.cc y.tab.c:178:1: warning: "MEL_MODULE" redefined In file included from /home/duelafn/MonetDB.hg/buildtools/mel/command.h:26, from /home/duelafn/MonetDB.hg/buildtools/mel/mel.yy:47: /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:1:1: warning: this is the location of the previous definition y.tab.c:179:1: warning: "MEL_PRELUDE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:2:1: warning: this is the location of the previous definition y.tab.c:180:1: warning: "MEL_EPILOGUE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:3:1: warning: this is the location of the previous definition y.tab.c:181:1: warning: "MEL_END" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:4:1: warning: this is the location of the previous definition y.tab.c:182:1: warning: "MEL_COMMAND" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:5:1: warning: this is the location of the previous definition y.tab.c:183:1: warning: "MEL_OPERATOR0" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:6:1: warning: this is the location of the previous definition y.tab.c:184:1: warning: "MEL_OPERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:7:1: warning: this is the location of the previous definition y.tab.c:185:1: warning: "MEL_OPERATOR1" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:8:1: warning: this is the location of the previous definition y.tab.c:186:1: warning: "MEL_FIXEDATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:9:1: warning: this is the location of the previous definition y.tab.c:187:1: warning: "MEL_ATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:10:1: warning: this is the location of the previous definition y.tab.c:188:1: warning: "MEL_USE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:11:1: warning: this is the location of the previous definition y.tab.c:189:1: warning: "MEL_ATOMOP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:12:1: warning: this is the location of the previous definition y.tab.c:190:1: warning: "MEL_STRING" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:13:1: warning: this is the location of the previous definition y.tab.c:191:1: warning: "MEL_IDENTIFIER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:14:1: warning: this is the location of the previous definition y.tab.c:192:1: warning: "MEL_USED" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:15:1: warning: this is the location of the previous definition y.tab.c:193:1: warning: "MEL_NUMBER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:16:1: warning: this is the location of the previous definition y.tab.c:194:1: warning: "MEL_ANY" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:17:1: warning: this is the location of the previous definition y.tab.c:195:1: warning: "MEL_BAT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:18:1: warning: this is the location of the previous definition y.tab.c:196:1: warning: "MEL_TYPE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:19:1: warning: this is the location of the previous definition y.tab.c:197:1: warning: "MEL_VOID" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:20:1: warning: this is the location of the previous definition y.tab.c:198:1: warning: "MEL_BUILTIN" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:21:1: warning: this is the location of the previous definition y.tab.c:199:1: warning: "MEL_OBJECT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:22:1: warning: this is the location of the previous definition y.tab.c:200:1: warning: "MEL_VARARGS" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:23:1: warning: this is the location of the previous definition y.tab.c:201:1: warning: "MEL_SEP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:24:1: warning: this is the location of the previous definition y.tab.c:202:1: warning: "MEL_ITERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:25:1: warning: this is the location of the previous definition mv -f .deps/glue.Tpo .deps/glue.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mil.o -MD -MP -MF .deps/mil.Tpo -c -o mil.o mil.cc y.tab.c:150: error: expected identifier before numeric constant y.tab.c:150: error: expected ‘}’ before numeric constant y.tab.c:150: error: expected unqualified-id before numeric constant y.tab.c:175: error: expected declaration before ‘}’ token make[3]: *** [mel.tab.o] Error 1 make[3]: *** Waiting for unfinished jobs.... mv -f .deps/mel.Tpo .deps/mel.Po mv -f .deps/mil.Tpo .deps/mil.Po mv -f .deps/mel.yy.Tpo .deps/mel.yy.Po make[3]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/duelafn/MonetDB.build' make: *** [all] Error 2

3 7

[MonetDB-users] Large Database / Is there hope?
by Dean Serenevy 18 Nov '10

18 Nov '10

Hello, I am still having problems compiling from sources, however, I would like to determine whether these is hope in continuing. We have a corpus of about 18,000 documents averaging about 5000 words each. We additionally have stand-off annotations that we would like to add and query against. This totals about 300 million XML elements. We are running 64-bit Linux with 6GiB RAM. Database A: Loading the documents (in batches of 100) as separate collections takes about 30 min and consumes 30GiB disk space in var/MonetDB4/dbfarm. Simple queries against database A take a long time, consume only 5% CPU and heavily work kswapd count( for $d in pf:documents-unsafe() return doc($d)/tCorpus ) # killed after1 hour Database B: Loading the documents (in batches of 100) into a single huge collection takes about 6 hours and consumes 120GiB disk space in var/MonetDB4/dbfarm. Simple queries against Database B take a long time, consume only 5% CPU and heavily work pdflush count( pf:collection("tcorpus")//tCorpus ) # killed after 3 hours My questions: 1. Is there any hope of successfully performing non-trivial queries on either of these databases using MonetDB? 2. If so, Is loading into a single collection or separate collections likely to be preferable? 3. The above work was done using the Jun2010-SP2 SuperBall, which is the only version I have been able to compile. Has any relevant work been done since then on MonetDB4 or any of the XQuery code that might improve performance? Thank you, Dean

1 0

[MonetDB-users] MonetDB-client 1.40.1 without Python support?
by W-Mark Kubacki 16 Nov '10

16 Nov '10

Hi. I've noticed that MonetDB-client 1.40.1 doesn't ship with Python libraries anymore. Yet there is a --with-python configuration option and Python examples. The corresponding changelog page [1] says nothing about the matter. What happened? -- Mark [1] http://monetdb.cwi.nl/Development/Releases/Oct2010/ [2] https://github.com/wmark/ossdl-overlay

3 2

[MonetDB-users] Announcement: New Oct2010 feature release of MonetDB suite
by Sjoerd Mullender 15 Nov '10

15 Nov '10

It is not October anymore, but nevertheless, the MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Oct2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] !MALException:setScenario:Scenario not initialized 'sql'
by Osmin Castillo 13 Nov '10

13 Nov '10

Hi guys, after install *MonetDB RPMs for Centos 5<http://code.google.com/p/monetdb-rhel-5/> * and mserver5 starts , I receive this error when execute the client: mclient -lsql !MALException:setScenario:Scenario not initialized 'sql' There's something missing?.. please help.

4 8

[MonetDB-users] MonetDB memory issue
by Andrey Timofeyev 12 Nov '10

12 Nov '10

Hello, All, We began to use MonetDB several weeks ago. As I can see people use monetdb several years and operate with millions of records. But we could not insert even several thousands records. After several minutes monetdb eat all 7 Gb memory and crush without any message. It seems that we use monetdb incorrectly. Do there exist any special options for start monetdb? MonetDB version: MonetDB server v5.20.5 (64-bit), based on kernel v1.38.5 (32-bit oids) Release Jun2010-SP2 Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 7.9GiB available memory Configured for prefix: C:\Users\sjoerd\Documents\src\stable\vs64\NTv64 Libraries: libpcre: 7.9 2009-04-11 openssl: OpenSSL 1.0.0a 1 Jun 2010 libxml2: 2.7.7 Compiled by: sjoerd@ALGOL (i686-pc-win32) Compilation: cl -GF -W3 -WX -wd4273 -wd4102 -MD -nologo -Ox Linking : cl -GF -W3 -WX -wd4273 -wd4102 -MD -nologo -Ox OS: Windows 7 x64 (Версия 6.1.7600 Сборка 7600) (We also try it on the ubuntu 10.4 - same effect) -- Best regards, Andrey.

3 11

[MonetDB-users] Sept-2010 Release
by Henry Addington 11 Nov '10

11 Nov '10

Hello: Do you have an estimated release date for the Oct-2010 release? I thought the major releases came out every 3 to 4 months..... Thanks for the update. Best regards, Henry

2 3

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 25 Oct '10

25 Oct '10

"T.c" != "T"."c" ;-) -- | Stefan.Manegold @ CWI .nl | DB Architectures (INS1) | | http://www.cwi.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Oorspronkelijk bericht ----- Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> Verzonden: maandag 25 oktober 2010 17:11 Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB Dear Arjen, Thanks. I did check the uppercase letter. With a date-type column "deliveryDays" in a table "bsbm.offer", here is the result - SELECT "deliveryDays" FROM bsbm.offer --> WORKS - SELECT "deliverydays" FROM bsbm.offer --> FAILS - SELECT "offer.deliveryDays" FROM bsbm.offer --> FAILS - SELECT deliveryDays FROM bsbm.offer --> FAILS - SELECT offer.deliveryDays FROM bsbm.offer --> FAILS - SELECT offer."deliveryDays" FROM bsbm.offer ---> WORKS The last query is the one I need :) Again, thanks for the help. Regards Freddy 2010/10/25 Arjen de Rijke <Arjen.de.Rijke(a)cwi.nl> > Stefan Manegold <Stefan.Manegold(a)cwi.nl> writes: > > > I see --- if you consider it a bug, please file a detailed bug report > including information & instructions to reporduce the problem as well as > infornmation about the exact MonetDB version and OS you're using via > http://bugs.MonetDB.org/ > > I would not consider it a bug. The catch is the uppercase letter in > deliveryDays. The default handling of upper/lowercase characters is > different than in other RDMBS. So in some cases the quotes are necessary. > > Arjen de Rijke > > > > Stefan > > > > > > ----- Oorspronkelijk bericht ----- > > Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > > Verzonden: maandag 25 oktober 2010 16:32 > > Aan: Communication channel for MonetDB users < > monetdb-users(a)lists.sourceforge.net> > > Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB > > > > Hi Stefan, > > > > Nope, the name of the column is "deliveryDays" :) > > > > Freddy > > > > 2010/10/25 Stefan Manegold <Stefan.Manegold(a)cwi.nl> > > > >> Hi Freddy, > >> > >> I don't think that the error you experience is triggered by the column > >> type, but rather by the fact that your date column ha? a name - say > "date" - > >> that is a reserved key word in SQL an hence needs to be double-quoted > >> according to the SQL standard (this is not a MonetDB specific choice, > but a > >> "feature" of the SQL standard) --- not knowing your table & column > names, I > >> can only guess, though ... > >> > >> Stefan > >> > >> > >> ----- Oorspronkelijk bericht ----- > >> Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > >> Verzonden: maandag 25 oktober 2010 16:03 > >> Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> > >> Onderwerp: [MonetDB-users] Querying date type column in MonetDB > >> > >> Dear All, > >> > >> I am new to MonetDB, so most likely my question will look obvious. > >> > >> I would like to ask, how can columns with date type is queried in > monetdb. > >> Based on my experience, it seems that date type columns need special > >> treatment. It seems that the only way to query them is with the syntax > >> "column_name" while other columns seems to work with the format > >> table_name.column_name without the need of double quote. When I tried to > >> query date column with the syntax table_name.column_name, it gave me an > >> error "no such column" > >> > >> So my question is, is it true that the only way to query date columns is > >> using "column_name" format? Another question will be, what happen if I > need > >> to query same name columns coming from different table? > >> > >> Thanks in advance, > >> Freddy > >> > >> > >> > ------------------------------------------------------------------------------ > >> Nokia and AT&T present the 2010 Calling All Innovators-North America > >> contest > >> Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > >> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > >> marketing > >> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > >> http://p.sf.net/sfu/nokia-dev2dev > >> _______________________________________________ > >> MonetDB-users mailing list > >> MonetDB-users(a)lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/monetdb-users > >> > > > > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > > Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > > MonetDB-users mailing list > > MonetDB-users(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > -- > ==================================================================== > CWI, Kamer L330 Centrum voor Wiskunde en Informatica > Science Park 123 Email: arjen.de.rijke(a)cwi.nl > 1098 XG Amsterdam tel: +31-(0)20-5924305 > Nederland +31-(0)6-51899284 > fax: +31-(0)20-5924312 > ===================== http://www.cwi.nl/~rijke/ ==================== > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users >

1 0

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 25 Oct '10

25 Oct '10

Good point, Arjen! Indeed CaMeL case identifiers need to be double-quoted in MonetDB/SQL ... Stefan -- | Stefan.Manegold @ CWI .nl | DB Architectures (INS1) | | http://www.cwi.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Oorspronkelijk bericht ----- Van: Arjen de Rijke <Arjen.de.Rijke(a)cwi.nl> Verzonden: maandag 25 oktober 2010 17:00 Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB Stefan Manegold <Stefan.Manegold(a)cwi.nl> writes: > I see --- if you consider it a bug, please file a detailed bug report including information & instructions to reporduce the problem as well as infornmation about the exact MonetDB version and OS you're using via http://bugs.MonetDB.org/ I would not consider it a bug. The catch is the uppercase letter in deliveryDays. The default handling of upper/lowercase characters is different than in other RDMBS. So in some cases the quotes are necessary. Arjen de Rijke > > Stefan > > > ----- Oorspronkelijk bericht ----- > Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > Verzonden: maandag 25 oktober 2010 16:32 > Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> > Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB > > Hi Stefan, > > Nope, the name of the column is "deliveryDays" :) > > Freddy > > 2010/10/25 Stefan Manegold <Stefan.Manegold(a)cwi.nl> > >> Hi Freddy, >> >> I don't think that the error you experience is triggered by the column >> type, but rather by the fact that your date column ha? a name - say "date" - >> that is a reserved key word in SQL an hence needs to be double-quoted >> according to the SQL standard (this is not a MonetDB specific choice, but a >> "feature" of the SQL standard) --- not knowing your table & column names, I >> can only guess, though ... >> >> Stefan >> >> >> ----- Oorspronkelijk bericht ----- >> Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> >> Verzonden: maandag 25 oktober 2010 16:03 >> Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> >> Onderwerp: [MonetDB-users] Querying date type column in MonetDB >> >> Dear All, >> >> I am new to MonetDB, so most likely my question will look obvious. >> >> I would like to ask, how can columns with date type is queried in monetdb. >> Based on my experience, it seems that date type columns need special >> treatment. It seems that the only way to query them is with the syntax >> "column_name" while other columns seems to work with the format >> table_name.column_name without the need of double quote. When I tried to >> query date column with the syntax table_name.column_name, it gave me an >> error "no such column" >> >> So my question is, is it true that the only way to query date columns is >> using "column_name" format? Another question will be, what happen if I need >> to query same name columns coming from different table? >> >> Thanks in advance, >> Freddy >> >> >> ------------------------------------------------------------------------------ >> Nokia and AT&T present the 2010 Calling All Innovators-North America >> contest >> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada >> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in >> marketing >> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >> http://p.sf.net/sfu/nokia-dev2dev >> _______________________________________________ >> MonetDB-users mailing list >> MonetDB-users(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-users >> > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users -- ==================================================================== CWI, Kamer L330 Centrum voor Wiskunde en Informatica Science Park 123 Email: arjen.de.rijke(a)cwi.nl 1098 XG Amsterdam tel: +31-(0)20-5924305 Nederland +31-(0)6-51899284 fax: +31-(0)20-5924312 ===================== http://www.cwi.nl/~rijke/ ==================== ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

1 0

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 25 Oct '10

25 Oct '10

Hi Freddy, I don't think that the error you experience is triggered by the column type, but rather by the fact that your date column ha? a name - say "date" - that is a reserved key word in SQL an hence needs to be double-quoted according to the SQL standard (this is not a MonetDB specific choice, but a "feature" of the SQL standard) --- not knowing your table & column names, I can only guess, though ... Stefan ----- Oorspronkelijk bericht ----- Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> Verzonden: maandag 25 oktober 2010 16:03 Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> Onderwerp: [MonetDB-users] Querying date type column in MonetDB Dear All, I am new to MonetDB, so most likely my question will look obvious. I would like to ask, how can columns with date type is queried in monetdb. Based on my experience, it seems that date type columns need special treatment. It seems that the only way to query them is with the syntax "column_name" while other columns seems to work with the format table_name.column_name without the need of double quote. When I tried to query date column with the syntax table_name.column_name, it gave me an error "no such column" So my question is, is it true that the only way to query date columns is using "column_name" format? Another question will be, what happen if I need to query same name columns coming from different table? Thanks in advance, Freddy

2 1

[MonetDB-users] Querying date type column in MonetDB
by Freddy Priyatna 25 Oct '10

25 Oct '10

Dear All, I am new to MonetDB, so most likely my question will look obvious. I would like to ask, how can columns with date type is queried in monetdb. Based on my experience, it seems that date type columns need special treatment. It seems that the only way to query them is with the syntax "column_name" while other columns seems to work with the format table_name.column_name without the need of double quote. When I tried to query date column with the syntax table_name.column_name, it gave me an error "no such column" So my question is, is it true that the only way to query date columns is using "column_name" format? Another question will be, what happen if I need to query same name columns coming from different table? Thanks in advance, Freddy

1 0

[MonetDB-users] calc.date in SQL to MAL.
by Guillaume Yziquel 22 Oct '10

22 Oct '10

Hi. I have resumed work on the OCaml binding for MonetDB. I have been playing a bit with it and with MonetDB, and noticed something rather weird when trying to explain a SQL statement to MAL. Compilation, and launching a small test in an OCaml toplevel: yziquel@seldon:~/data$ make ocamlbuild -use-ocamlfind import.cma Finished, 3 targets (0 cached) in 00:00:00. yziquel@seldon:~/data$ make test cp ocamlinit _build/ cd _build; ledit ocaml -init ocamlinit Objective Caml version 3.12.0 Findlib has been successfully loaded. Additional directives: #require "package";; to load a package #list;; to list the available packages #camlp4o;; to load camlp4 (standard syntax) #camlp4r;; to load camlp4 (revised syntax) #predicates "p,q,...";; to set these predicates Topfind.reset();; to force that packages will be reloaded #thread;; to enable threads /usr/lib/ocaml/csv: added to search path /usr/lib/ocaml/csv/csv.cma: loaded /usr/lib/ocaml/unix.cma: loaded /usr/lib/ocaml/monetdb5: added to search path /usr/lib/ocaml/monetdb5/monetDB5.cma: loaded /usr/lib/ocaml/dynlink.cma: loaded /usr/lib/ocaml/camlp4: added to search path /usr/lib/ocaml/pcre: added to search path /usr/lib/ocaml/pcre/pcre.cma: loaded /usr/local/lib/ocaml/3.12.0/tophide: added to search path /usr/local/lib/ocaml/3.12.0/tophide/tophide.cmo: loaded /usr/local/lib/ocaml/3.12.0/mikmatch_pcre: added to search path /usr/local/lib/ocaml/3.12.0/mikmatch_pcre/run_mikmatch_pcre.cma: loaded So much for the prologue, and dynamic loading. So the test starts here. Everytime you see "Asking", it is the statement that is fed to the connection handler in order to retrieve the query handler. (With mapilib). Asking: select name from tables; Asking: create table indu (Date date, Open decimal(15,2), High decimal(15,2), Low decimal(15,2), Close decimal(15,2), Volume decimal(15,2), OI decimal(15,2)); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-02-01', 108.80, 108.80, 108.80, 108.80, 1156, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-05-01', 108.90, 108.90, 108.90, 108.90, 1305, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-06-01', 107.40, 107.40, 107.40, 107.40, 1251, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-07-01', 107.60, 107.60, 107.60, 107.60, 771, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-08-01', 107.20, 107.20, 107.20, 107.20, 843, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-09-01', 106.60, 106.60, 106.60, 106.60, 736, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-12-01', 104.20, 104.20, 104.20, 104.20, 1049, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-13-01', 104.50, 104.50, 104.50, 104.50, 908, 0); And on this last statement, the server doesn't seem happy at all: Exception: MonetDB5.Mapi.Error (Some (MonetDB5.Mapi.MSERVER, None)). So I've been trying to 'explain' the SQL to MAL. The first two work fine and output a very similar MAL statement. The third SQL statements, however, is getting translated to something somewhat different: You see a call to 'calc.date' on the 4th MAL line: # ask "explain insert into indu (Date, Open, High, Low, Close, Volume, # OI) values ('1920-07-01', 107.60, 107.60, 107.60, 107.60, 771, 0);";; Asking: explain insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-07-01', 107.60, 107.60, 107.60, 107.60, 771, 0); - : string option list list = [[Some "function user.s0_1{autoCommit=true}():void;"]; [Some " _2 := sql.mvc();"]; [Some "barrier _32 := language.dataflow();"]; [Some " _3 := sql.append(_2,\"sys\",\"indu\",\"date\",1920-07-01);"]; [Some " _7 := sql.append(_3,\"sys\",\"indu\",\"open\",10760:lng);"]; [Some " _9 := sql.append(_7,\"sys\",\"indu\",\"high\",10760:lng);"]; [Some " _11 := sql.append(_9,\"sys\",\"indu\",\"low\",10760:lng);"]; [Some " _13 := sql.append(_11,\"sys\",\"indu\",\"close\",10760:lng);"]; [Some " _15 := sql.append(_13,\"sys\",\"indu\",\"volume\",77100:lng);"]; [Some " sql.append(_15,\"sys\",\"indu\",\"oi\",0:lng);"]; [Some "exit _32;"]; [Some " sql.affectedRows(1:wrd,\"\");"]; [Some "end s0_1;"]] # ask "explain insert into indu (Date, Open, High, Low, Close, Volume, # OI) values ('1920-12-01', 104.20, 104.20, 104.20, 104.20, 1049, 0);";; Asking: explain insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-12-01', 104.20, 104.20, 104.20, 104.20, 1049, 0); - : string option list list = [[Some "function user.s1_1{autoCommit=true}():void;"]; [Some " _2 := sql.mvc();"]; [Some "barrier _32 := language.dataflow();"]; [Some " _3 := sql.append(_2,\"sys\",\"indu\",\"date\",1920-12-01);"]; [Some " _7 := sql.append(_3,\"sys\",\"indu\",\"open\",10420:lng);"]; [Some " _9 := sql.append(_7,\"sys\",\"indu\",\"high\",10420:lng);"]; [Some " _11 := sql.append(_9,\"sys\",\"indu\",\"low\",10420:lng);"]; [Some " _13 := sql.append(_11,\"sys\",\"indu\",\"close\",10420:lng);"]; [Some " _15 := sql.append(_13,\"sys\",\"indu\",\"volume\",104900:lng);"]; [Some " sql.append(_15,\"sys\",\"indu\",\"oi\",0:lng);"]; [Some "exit _32;"]; [Some " sql.affectedRows(1:wrd,\"\");"]; [Some "end s1_1;"]] # ask "explain insert into indu (Date, Open, High, Low, Close, Volume, # OI) values ('1920-13-01', 104.50, 104.50, 104.50, 104.50, 908, 0);";; Asking: explain insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-13-01', 104.50, 104.50, 104.50, 104.50, 908, 0); - : string option list list = [[Some "function user.s2_1{autoCommit=true}():void;"]; [Some " _2 := sql.mvc();"]; [Some "barrier _33 := language.dataflow();"]; [Some " _3 := calc.date(\"1920-13-01\");"]; [Some " _4 := sql.append(_2,\"sys\",\"indu\",\"date\",_3);"]; [Some " _8 := sql.append(_4,\"sys\",\"indu\",\"open\",10450:lng);"]; [Some " _10 := sql.append(_8,\"sys\",\"indu\",\"high\",10450:lng);"]; [Some " _12 := sql.append(_10,\"sys\",\"indu\",\"low\",10450:lng);"]; [Some " _14 := sql.append(_12,\"sys\",\"indu\",\"close\",10450:lng);"]; [Some " _16 := sql.append(_14,\"sys\",\"indu\",\"volume\",90800:lng);"]; [Some " sql.append(_16,\"sys\",\"indu\",\"oi\",0:lng);"]; [Some "exit _33;"]; [Some " sql.affectedRows(1:wrd,\"\");"]; [Some "end s2_1;"]] # Why is that happening? The SQL statements seem so similar that I do not see why it gets compiled differently to MAL. Best regards, -- Guillaume Yziquel http://yziquel.homelinux.org

2 4

[MonetDB-users] installation corrupted
by Nafees Ur Rehman 20 Oct '10

20 Oct '10

Hi, I am using monetDB/SQL for windows on an xp machine. Two days ago, the server stopped responding. Tried to repair and change it...but didn't work. Also got it formally un-installed and then tried re-installation...but no luck. Any hints. The server does not mount at all. Regards Nafees Ur Rehman Konstanz, Germany

1 0

[MonetDB-users] MonetDB performance question
by Tudor-Ioan Salomie 18 Oct '10

18 Oct '10

Hi, I have recently tried doing some experiments with MonetDB, measuring its performance for the following dataset+workload: * 1 table (50 attributes), 1GB, comprising 3.000.000 tuples * the attributes are fixed length strings and integer types * the workload is query (SELECT) only, where each query has the form: SELECT <list_of_projections> FROM MyTable WHERE <list_of_predicates>; * the list of predicates is a conjunction of equality operators * more than 99% of the queries predicate on the same attribute (let's call it A1) that has a *very* good selectivity * the average number of predicated attributes per query is 8 * for this workload, averaging over 4000 queries, there is 1 result tuple per query (actual result tuples per query are in the range 0-10) I have installed MonetDB on a 64-bit Debian machines, using the .deb files provided on the internet: * libmonetdb1_1.38.5-20100824_amd64.deb * libmonetdb5-server5_5.20.5-20100824_amd64.deb * libmonetdb5-sql2_2.38.5-20100824_amd64.deb * libmonetdb-client1_1.38.5-20100824_amd64.deb * monetdb5-server_5.20.5-20100824_amd64.deb * monetdb5-sql_2.38.5-20100824_amd64.deb * monetdb-client_1.38.5-20100824_amd64.deb For running the queries I use the SQL front end. I have run the queries both through "mclient" (directly on the server) and through the JDBC driver (from another client machine). The MonetDB server is bound to run on only one core on the machine (though I have not seen any difference if running it on multiple cores). To my surprise, the throughput for MonetDB for this workload is very low. I get about 3 queries per second, which seems to indicate that MonetDB does full table scanning only. Am I correct in this assumption? Is there anything that can be done to speed it up? Is this not a workload suitable for MonetDB? Looking forward to your input! Regards, Tudor.

2 1

[MonetDB-users] FW: Comment via CWI webform MonetDB
by Peter Boncz 15 Oct '10

15 Oct '10

Narayan Sujay, Thanks for your interest in MonetDB. Rather than directing technical questions to the CWI organization webmaster, you are supposed to post on one of the relevant mailing lists such as this one (monetdb-user(a)lists.sourceforge.net) Peter -----Original Message----- From: Minnie Middelberg [mailto:Minnie.Middelberg@cwi.nl] Sent: Friday, October 15, 2010 9:50 AM To: Peter Boncz Subject: Fwd: Comment via CWI webform MonetDB Hallo Peter, Onderstaand is binnengekomen via het CWI Comment formulier en het betreft MonetDB. Wetende dat jij en Marting Kersten achter MonetDB zitten forward ik het maar naar jou. Mocht er iemand zijn die dit soort vragen belast is dan hoor ik het graag van je. Mvg Minnie ----- Forwarded Message ----- From: "Narayan Sujay" <316sujay(a)gmail.com> To: voorlichting(a)cwi.nl, webmaster(a)cwi.nl Sent: Thursday, 14 October, 2010 6:15:15 PM Subject: Comment via CWI webform Submitted on 14.10.2010 - 18:15 Submitted by anonymous user: [198.134.51.12] Submitted values are: Name sender: Narayan Sujay Email address: 316sujay(a)gmail.com Telephone number: 4193780532 Your comment: Hello, I have downloaded MonetDB and the native client and server work well. I can write simple queries in the client cmd window. I am trying to connect MonetDB with one of the SQL frontends like DB visualizer etc and I am not able to do this. Please help me out with instructions. I followed the instructions on the webpage but still getting errors connecting through db visualizer. I think it is the problem with the JDBC drivers. Please let me know excatly where to find the driver on the local computer after I have installed Monet DB. My Main goal here is to load large Data sets in MonetDB. Please tell me a way to do this for Monet DB. The error I get in DB Visualizer is: Long Message: Cannot connect to ORB Details: Type: javax.naming.CommunicationException Stack Trace: javax.naming.CommunicationException: Cannot connect to ORB [Root exception is org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No] at com.sun.jndi.cosnaming.CNCtx.setOrbAndRootContext(Unknown Source) at com.sun.jndi.cosnaming.CNCtx.initOrbAndRootContext(Unknown Source) at com.sun.jndi.cosnaming.CNCtx.<init>(Unknown Source) at com.sun.jndi.cosnaming.CNCtxFactory.getInitialContext(Unknown Source) at javax.naming.spi.NamingManager.getInitialContext(Unknown Source) at javax.naming.InitialContext.getDefaultInitCtx(Unknown Source) at javax.naming.InitialContext.init(Unknown Source) at javax.naming.InitialContext.<init>(Unknown Source) at com.onseven.dbvis.db.A.G.Ą(Z:3155) at com.onseven.dbvis.db.A.G.Ĭ(Z:2675) at com.onseven.dbvis.db.A.E.Ĭ(Z:98) at com.onseven.dbvis.db.A.D.ā(Z:2056) at se.pureit.swing.A.O$2.run(Z:1192) at java.lang.Thread.run(Unknown Source) Caused by: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(Unknown Source) at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(Unknown Source) at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(Unknown Source) at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(Unknown Source) at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(Unknown Source) at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(Unknown Source) at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(Unknown Source) at com.sun.corba.se.impl.resolver.BootstrapResolverImpl.invoke(Unknown Source) at com.sun.corba.se.impl.resolver.BootstrapResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.resolver.CompositeResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.resolver.CompositeResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.resolver.CompositeResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.orb.ORBImpl.resolve_initial_references(Unknown Source) ... 14 more Caused by: java.net.ConnectException: Connection refused: connect at sun.nio.ch.Net.connect(Native Method) at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) at java.nio.channels.SocketChannel.open(Unknown Source) at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(Unknown Source) ... 25 more The results of this submission may be viewed at: http://www.cwi.nl/en/node/757/submission/3152

1 0

[MonetDB-users] upgrade notes: Feb2010 --> Jun2010
by Mark Bucciarelli 07 Oct '10

07 Oct '10

Hi, I'd like to upgrade to the June release. I'm running Feb 2010. Can I just build the June release and expect it will just work with my database? Or do I need to upgrade the data somehow? I poked around dev.monetdb.org/hg, looking for release notes, changelist, etc. but didn't find anything. Thanks, m

2 3

[MonetDB-users] bug #2634 in Jun2010? [was: upgrade notes: Feb2010 --> Jun2010]
by Mark Bucciarelli 07 Oct '10

07 Oct '10

On Thu, Oct 7, 2010 at 9:15 AM, Sjoerd Mullender <sjoerd(a)acm.org> wrote: > On 2010-10-07 14:41, Mark Bucciarelli wrote: > >> I poked around dev.monetdb.org/hg, looking for release >> notes, changelist, etc. but didn't find anything. > > http://monetdb.cwi.nl/Development/Releases/Jun2010/ > Thanks. Was the commit in bug #2634 in the release? I don't see it in the SQL notes. He closed as NEEDS MORE INFO (I think---I can't tell from ticket history) because I did not include a test case. But he did commit some change that fixed the issue (I was able to upload a test case this week, and verified his fix solved my case). Thanks, m http://bugs.monetdb.org/show_bug.cgi?id=2634

2 1

[MonetDB-users] Performance improvements over equal queries limits to operators?
by Stefan de Konink 06 Oct '10

06 Oct '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, I was quite surprised about the performance of some queries, that I expected to be heavy weighted, but actually performed pretty well. To show some things: sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (259.000ms) sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (259.000ms) sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (258.000ms) sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (257.000ms) sql>select * from kvk where postcode = '2265CA' and adres like '%7%'; 3 tuples (11.000ms) sql>select * from kvk where postcode = '2265CA' and adres like '%7%'; 3 tuples (6.000ms) sql>select * from kvk where postcode = '2265CA' and adres like '%7%'; 3 tuples (6.000ms) sql>select * from kvk where kvk = 273121520000; 1 tuple (473.000ms) sql>select * from kvk where kvk = 273121520000; 1 tuple (5.000ms) I wonder: what makes the 11ms to 6ms possible in the like vs = scenario? While a repeat of the like operator itself doesn't give a true improvement? As you can observe the like argument doesn't have any modifiers, so it should actually become a normal equal. (I noticed this later, goes beyond my original point, will file a bug for enhancement.) Also I'm quite surprised that the primary key lookup still takes a fair amount of time, could be i/o related obviously, but the performance on the second run is again huge, even for other arguments. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkysWw8ACgkQYH1+F2Rqwn19AgCgk9lkRBLcWdylxxz9M4B8a8f5 5JUAn1TOU1VxpvVfVWJMLwlwp9Pz4a0x =IqCY -----END PGP SIGNATURE-----

2 1

[MonetDB-users] Dump schema w/ mclient?
by Mark Bucciarelli 05 Oct '10

05 Oct '10

Is there a way to user mclient to dump the db schema without dumping the data? I tried creating a one line SQL file with \d <tablename> and piping to mclient stdin, but it didn't work. Thanks, m

3 2

[MonetDB-users] add a unique constraint, need to create index?
by Mark Bucciarelli 04 Oct '10

04 Oct '10

Hi, I added a unique constraint on two fields, and noticed there was not an index created on those two fields. Do I need to do a CREATE INDEX or is it implicit? I will be using those two fields a lot in WHERE clauses. Thanks, m

1 1

[MonetDB-users] Large XML instances with MonetDB/XQuery 4.38.5
by Christian Grün 02 Oct '10

02 Oct '10

Hi everyone, is it still possible to shred large XML instances with MonetDB/XQuery, or do I need more memory for shredding? I was just trying to the 1GB instance of XMark with WinXP 32Bit and 2GB RAM, and this is what I got: MonetDB>shred_doc("1gb.xml", "xmark"); #GDKmmap(536870912) fails, try to free up space [memory in use=36143780,virtual memory in use=1015480320] #GDKmmap(536870912) result [mem=26016940,vm=1005387776] !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 536870912 for 06\632theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( follow murders prov shoes p resent eager delivered defy bend viol here marquis powers complexion pillar cran ts agate ), BUNappend fails !ERROR: CMDshred_url: operation failed. pf:add-doc(...) produces the same output; the XML document itself is well-formed. Thanks, Christian

2 6

[MonetDB-users] Embedded SQL Example Hangs?
by Hering Cheng 23 Sep '10

23 Sep '10

Hi, I am trying to run the example embedded MonetDB server using the June 2010 SP2 release: $ ./Mbeddedsql5 --config=monetdb5.conf --time --dbname=test_db After entering a simple SQL statement on the standard input and pressing <ENTER>, the program hangs at mapi_query. Here is the stack trace: [1] _read(0x5, 0xffffffff7fffdf16, 0x2, 0x0, 0x0, 0x0), at 0xffffffff7a6d3d90 [2] read(0x5, 0xffffffff7fffdf16, 0x2, 0x0, 0x0, 0x0), at 0xffffffff7a6c48b8 [3] pipe_read(s = 0x10015fe50, buf = 0xffffffff7fffdf16, elmsize = 2U, cnt = 1U), line 3194 in "stream.c" [4] stream_readSht(s = 0x10015fe50, val = 0xffffffff7fffdf16), line 2679 in "stream.c" [5] bs_read(ss = 0x10015fcd0, buf = 0x100f7a012, elmsize = 1U, cnt = 8190U), line 2368 in "stream.c" [6] stream_read(s = 0x10015fcd0, buf = 0x100f7a012, elmsize = 1U, cnt = 8190U), line 181 in "stream.c" [7] read_line(mid = 0x100133950), line 2515 in "Mapi.c" [8] read_into_cache(hdl = 0x100f73f30, lookahead = 1), line 2993 in "Mapi.c" [9] mapi_execute_internal(hdl = 0x100f73f30), line 3240 in "Mapi.c" [10] mapi_query(mid = 0x100133950, cmd = 0x100147010 "select count(1) from taq.trade\n"), line 3277 in "Mapi.c" [11] main(argc = 4, av = 0xffffffff7fffe628), line 212 in "prog.c" Here is my MonetDB configuration: $ ./mserver5 --version MonetDB server v5.20.5 (64-bit), based on kernel v1.38.5 (64-bit oids) Release Jun2010-SP2 Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 64.0GiB available memory, 16 available cpu cores Configured for prefix: /GAAL/chenher/share/monetdb/distro-sparc-jun2010-sp2-64bit Libraries: libpcre: 8.10 2010-06-25 (compiled with 8.10) openssl: OpenSSL 1.0.0a 1 Jun 2010 (compiled with ) libxml2: 2.6.23 (compiled with 2.6.23) Compiled by: chenher@sfouxsrv073 (sparc-sun-solaris2.10) Compilation: cc -m64 -xcode=pic32 -I/GAAL/chenher/share/hans_boehm_gc/distro-sparc-7.1-64bit//include/ -g Linking : /usr/ccs/bin/ld -m64 -L/GAAL/chenher/share/openssl/distro-sparc-1.0.0a-64bit//lib/64/ -L/GAAL/chenher/share/pcre/distro-sparc-8.10-64bit//lib -L/GAAL/chenher/share/hans_boehm_gc/distro-sparc-7.1-64bit//lib/ The database test_db was created by a non-embedded mserver5 using the same configuration file. The non-embedded mserver5 was gracefully shut down just before running the embedded version. Thanks. Hering

1 0

Re: [MonetDB-users] newbies question
by Peter Boncz 21 Sep '10

21 Sep '10

Hello Olivier, MonetDB/XQuery and MonetDB/SQL are currently different servers, based on different MonetDB kernel versions (4 and 5). The new python library you are probably referring to, however, only works for MonetDB5 (thus SQL). MonetDB/XQuery can be accessed using JDBC, XRPC and C (mapi), as per the MonetDB/XQuery API documentation. Peter > -----Original Message----- > From: o.fournier(a)iktios.fr [mailto:o.fournier@iktios.fr] > Sent: Tuesday, September 21, 2010 5:57 PM > To: monetdb-users(a)lists.sourceforge.net > Subject: [MonetDB-users] newbies question > > Hi, > > I find the concept of MonnetDb interresting... > > So, I would like use MonnetDB in SQL and XML mode with python > interface. > Could I use the python MAPI module to request XML mode? > If not, have you plan to do it? > Or what method could I use to access XML Monnetdb from python? > > Thank you > > Olivier Founier > > > > ---------------------------------------------------------------------- > -------- > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing. > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users

1 0

[MonetDB-users] newbies question
by o.fournier＠iktios.fr 21 Sep '10

21 Sep '10

Hi, I find the concept of MonnetDb interresting... So, I would like use MonnetDB in SQL and XML mode with python interface. Could I use the python MAPI module to request XML mode? If not, have you plan to do it? Or what method could I use to access XML Monnetdb from python? Thank you Olivier Founier

1 0

[MonetDB-users] about create index command
by kun ren 17 Sep '10

17 Sep '10

Dear all, I have a problem on create index command, my table is t1,and have two column, c1 and c2. when I execute the sql command "create index index_c1 on t1(c1);" but the MonetDB said "Segmentation fault", what the question? Best regards!

2 2

Re: [MonetDB-users] MonetDB Copy File INTO
by Fabian Groffen 14 Sep '10

14 Sep '10

On 14-09-2010 01:04:41 +0000, mikeschill(a)users.sourceforge.net wrote: > Username: MikeSchill > > I really want to use the MonetDB for our processing but getting > problems trying to do a COPY INTO: Always fails on last line stating > missing '(field delimeter) line (last line) field (last field) > failed to import table > > COPY into STATEDATA from > C:\\Books\\RED.txt' using > delimiters '\t','\n'; > > No idea what I am doing wrong....fails on record 101 if I say COPY 100 > Records... Can you give us any details on what version of MonetDB you are using (such as the release, e.g. Jun2010-SP2)? Also is your data public, or can we get a copy of it to see if we can reproduce the problem? Regards, Fabian Groffen

1 0

[MonetDB-users] 64 bit build with 64 bit OIDs - Invalid data for large string table
by Alex Bo. 07 Sep '10

07 Sep '10

Hi, I built June-SP1 on Windows 64 bit 64 bit OIDs. I loaded a table with one string field with approximately 500M strings. I am running 'Select * from table limit 10', and what I see is this: #1: myVal1 #2: null #3: myVal2 #4: null #5: myVal3 #6: null ... It seems that nulls are inserted between every two consecutive values in the source. The rowcount of the table is correct. I looked inside the dbfarm folders and saw that the theap file is ~26GB, and the tail ~4.6GB. Inserting only 1M of the same data worked fine. Please advise, Thanks. -- View this message in context: http://old.nabble.com/64-bit-build-with-64-bit-OIDs---Invalid-data-for-larg… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Building June10-SP1 64 with 64 OID on Windows
by Alex Bo. 06 Sep '10

06 Sep '10

When using MONET_OID64=1 to build June10-SP1 on Windows 64, I get this in the MonetDB5 module: .\..\..\..\..\src\modules\mal\tokenizer.mx(562) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data How can I overcome this? (This did not happen with 32 bit OIDs) Thanks. -- View this message in context: http://old.nabble.com/Building-June10-SP1-64-with-64-OID-on-Windows-tp29632… Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] strPut error with large string tables
by Alex Bo. 02 Sep '10

02 Sep '10

Hi, (June2010 SP1 built on Windows) I loaded a table with a single string column successfully with approximately 500M string values. Running 'Select * from table limit 10' gives me this error: 'strPut: string heaps gets larger than 15GB.' What does this mean? -- View this message in context: http://old.nabble.com/strPut-error-with-large-string-tables-tp29593089p2959… Sent from the monetdb-users mailing list archive at Nabble.com.

3 3

[MonetDB-users] June2010 SP2 failed to build on Windows 64
by Alex Bo. 02 Sep '10

02 Sep '10

Hi, I have successfully built June2010 SP2 on Windows 32 bit. However, the same on 64 bit fails in the Monet module while trying to build 'monet_options.c'. I am getting a bunch of errors related to ws2tcpip.h: C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\include\ws2tcpip.h(29) : error C2079: 'imr_multiaddr' uses undefined struct 'in_addr' C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\include\ws2tcpip.h(30) : error C2079: 'imr_interface' uses undefined struct 'in_addr' C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\include\ws2tcpip.h(38) : error C2079: 'imr_multiaddr' uses undefined struct 'in_addr' C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\include\ws2tcpip.h(39) : error C2079: 'imr_sourceaddr' uses undefined struct 'in_addr' C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\include\ws2tcpip.h(40) : error C2079: 'imr_interface' uses undefined struct 'in_addr' C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\include\ws2tcpip.h(46) : error C2079: 'imsf_multiaddr' uses undefined struct 'in_addr' Please advise. Thanks. -- View this message in context: http://old.nabble.com/June2010-SP2-failed-to-build-on-Windows-64-tp29593272… Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] Where is the right place to fill bug report ?
by Krzysztof Kowalczyk 02 Sep '10

02 Sep '10

Where is the right place to fill a bug report ? I am using sophisticated xquery on MonetDB on Windows x64 but my work has been stopped by a critical bug - probably in Pathfinder. The problem is that Monet fails to compile a query, instead it just take whole available memory and system needs hard restart. Sometimes I can rewrite query to different form with same functionality and this help, but it take a lot of time to find which code part is cause the error. Regards, Krzysztof Kowalczyk

3 4

[MonetDB-users] Joining Across Multiple Databases
by Hering Cheng 31 Aug '10

31 Aug '10

Hi, Is it possible to join two tables residing in different databases in a query? Here, by "database" I mean the result of running "monetdb create <database>". If so, what syntax should we use for the table names? Thanks. Hering Cheng

2 1

[MonetDB-users] Bulk Data Extraction & Embedded mserver5
by Hering Cheng 30 Aug '10

30 Aug '10

Hi, I was wondering if you can give me some ideas on how to retrieve data from MonetDB at very high throughput. The query will be very simple, issued against a single table. This table contains billions of records and I'd like to retrieve the data at a rate of at least one million records per second. From my naive tests with mclient running on the same 16-core machine as mserver5, I am only able to extract the data at about 20,000 records per second with the Feb2010 release. As a baseline case, with the same data in compressed (gzip'd) ASCII form stored in a regular file on the same (SAN) file system as MonetDB, I am able to read at the desired speed of one million records per second. I understand that there is communication overhead between mserver5 and mclient (or whatever client I use). Therefore, one possibility is to embed my application within mserver5. The embedded application basically just needs to be able to issue a SQL (or even MAL) query against the enclosing mserver5 and process the result set. If this is a viable approach, I'd like some guidance on where the hooks are. Thanks. Hering Cheng

5 8

[MonetDB-users] Installation error
by Alexander Rutkowski 30 Aug '10

30 Aug '10

Hi, I tried to install MonetDB on a Debian Linux system with ./monetdb-install.sh --prefix=/export/local-1/public/node/MonetDB --enable-sql --enable-xquery --enable-optimize but I got the error message ... test -z "/export/local-1/public/node/MonetDB/lib" || /bin/mkdir -p "/export/local-1/public/node/MonetDB/lib" /bin/sh ../../libtool --mode=install /usr/bin/install -c libstream.la '/export/local-1/public/node/MonetDB/lib' libtool: install: error: cannot install `libstream.la' to a directory not ending in /usr/local/lib make[6]: *** [install-streamLTLIBRARIES] Error 1 ... installing MonetDB failed I have no root rights so I need the --prefix argument. Thank you very much Alexander

3 3

[MonetDB-users] Announcement: New Jun2010-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 30 Aug '10

30 Aug '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010-SP2 bug fix release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] Response time
by Mitja Kleider 14 Aug '10

14 Aug '10

Hi, I am experiencing a huge difference between query response times of "mserver5 --readonly" and plain "mserver5". The difference is in the order of 25 sec vs 25 min. The disks are rather slow and there is a lot of disk write going on. (Not in readonly, obviously.) How can I investigate this further? Regards, Mitja

2 1

[MonetDB-users] Problems with huge SQL queries.
by Manuel Mayr 10 Aug '10

10 Aug '10

Hello everyone, I'm currently using Monet5 for some experiments with ActiveRecord but some SQL queries seem to be too big for the default configuration of monet. The prominent characteristic of the query is a huge IN part in the WHERE clause (nearly 13Kb). $ mclient -d tpch01 -h localhost < q1.sql MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = SELECT states.* FROM states WHERE (states.country_id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141 ERROR = !Connection terminated Is there a parameter that allows me to increase the SQL query buffer? Beste regards, Manuel PS.: I'm not sure if it is really the size of the SQL buffer since the query works when I modify it to run it against sys.tables, but it doesn't seem to return any result. Even prefix the query with "plan" doesn't come back within minutes. Maybe a performance bottleneck in the compiler or planner? -- Manuel Mayr (http://www-db.informatik.uni-tuebingen.de/team/mayr) Lehrstuhl Datenbanksysteme Wilhelm-Schickard Institut für Informatik Eberhard-Karls-Universität Tübingen

7 18

[MonetDB-users] geom performance
by Mitja Kleider 06 Aug '10

06 Aug '10

Hi, I am having trouble with the geom module (Jun2010-SP1). A query constructing a Point(x,y) from longitude (x) and latitude (y) takes much longer than expected, see below: # MonetDB server v5.20.3, based on kernel v1.38.3 # Serving database 'germany3', using 2 threads # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked # MonetDB/SQL module v2.38.3 loaded # MonetDB/GIS module v0.18.3 loaded sql>\d nodes CREATE TABLE "sys"."nodes" ( "id" int NOT NULL DEFAULT next value for "sys"."s_nodes", "long" decimal(10,7), "lat" decimal(10,7), CONSTRAINT "pk_nodes_id" PRIMARY KEY ("id") ); sql>select long,lat from nodes limit 1; +--------------+--------------+ | long | lat | +==============+==============+ | 8.9766191 | 47.7913748 | +--------------+--------------+ 1 tuple (1.0s) sql>select point(8.9766191, 47.7913748); +------------------------------------------------+ | point_single_value | +================================================+ | POINT (8.9766191000000006 47.7913747999999998) | +------------------------------------------------+ 1 tuple (0.897ms) sql>select point(long,lat) from nodes limit 1; +------------------------------------------------+ | point_long | +================================================+ | POINT (8.9766191000000006 47.7913747999999998) | +------------------------------------------------+ 1 tuple (1m 23s) And executing something similar for the whole table caused a segfault after eating a lot of virtual memory: sql>ALTER TABLE nodes ADD geom POINT DEFAULT NULL; sql>UPDATE nodes SET geom = Point(long,lat) WHERE geom IS NULL; In case you would like to try yourself, here is the import script and the parser to get such a database: http://gitorious.org/osm-poi-tools/monetdb/blobs/master/scripts/import.sh http://gitorious.org/osm-poi-tools/monetdb/blobs/master/osmparser/src/osmpa… Thanks for your help, Mitja

2 1

[MonetDB-users] Example of XRPC/Java application
by Ryad Ben-El-Kezadri 05 Aug '10

05 Aug '10

Hello, For whom interested by XRPC and Java GUI, I have posted a video http://www.youtube.com/watch?v=3WzM0ZOJw9k showing the great things that can be done with MonetDB/XQuery Regards Ryad PS: this is a networking application

3 2

[MonetDB-users] How to log every sql query?
by Nicolas Lemay 02 Aug '10

02 Aug '10

Hello, I've been using MonetDB for quite some weeks, but I never find the way to log every sql query my database receive. I've tried googling this, but did not find a lot of docs about that. Is it possible to do this? If yes, how can I accomplish this? Thank you!

1 0

[MonetDB-users] Loading directly into memory.
by Nozhup 31 Jul '10

31 Jul '10

Hi, The first query after a load action takes longer because the new data is still 'cold'. Is there a way in MonetDB to load data directly with a 'hot' status? Thank in advance. -- View this message in context: http://old.nabble.com/Loading-directly-into-memory.-tp29105569p29105569.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

Re: [MonetDB-users] MonetDB-users Digest, Vol 50, Issue 11
by Shailendra Mishra 29 Jul '10

29 Jul '10

Hi Niels: I don't care if the log gets written on table creation. I tried the flag but it didn't do anything. The default value is 0. In the FebPS2 version it's a debug flag - actually setting it to 128 doesn't help. On closer examination it appears that it checks for if (debug & 1) then it would write a log. Setting debug to 128, would imply that debug & 1 = 0 and so no debug output is written. Can you tell me where the log writer code is ? Is it gdk_logger.c ? if yes, which is the function that writes the log_write_format, log_write_string, if not which ones. -- Shailendra ------------------------------ > > Message: 5 > Date: Wed, 28 Jul 2010 08:40:06 +0200 > From: Niels Nes <Niels.Nes(a)cwi.nl> > Subject: Re: [MonetDB-users] Creating table with nologging. > To: Communication channel for MonetDB users > <monetdb-users(a)lists.sourceforge.net> > Message-ID: <20100728064006.GA6626(a)cwi.nl> > Content-Type: text/plain; charset=utf-8 > > On Tue, Jul 27, 2010 at 04:28:56PM -0700, Shailendra Mishra wrote: > > I have the need for creating table with nologging enabled. I do > > understand that MonetDB is not meant to be used this way. However, I am > > trying to use it to ingest data off data-feeds. My db lives on a SSD, > > and the only DML that I execute is INSERT. I would appreciate if folks > > familiar with the server code could point me source files ?where I > > could make the change to achieve this. Thanks in advance for your help. > > > > -- Shailendra > > Shailendra > > I haven't tried this in a while but it used to be possible to switch > logging of inserts of using a debug flag (passed to the logger code). > > See the file src/storage/bat/bat_logger.mx > function bl_create. The first argument to logger_create should > be 128 to switch off logging. > > This will only disable that every insert will also write the inserted > values to the log. It still creates a log for your initial create table > statement. > > Indeed this is un documented and far from normal supported usage of > monetdb. > > Niels > >

2 2

[MonetDB-users] MonetDB XQuery newbie
by Stephen Thomas 28 Jul '10

28 Jul '10

I'm a newbie MonetDB user and have two issues out of the box. (I have downloaded and installed the latest stable version, on Mac OS X 10.6.3.) 1. I cannot get the command line history function to work with XQuery. I invoke the server with: Mserver --dbinit="module(pathfinder);" and invoke a client with: mclient -lx --history However, the command line history does not work; hitting the up arrow after a command results in "^[[A" being displayed: shell> mclient -lx --history Welcome to mclient, the MonetDB/XQuery interactive terminal Type \q to quit, \? for a list of available commands xquery>\<test1.xq <hello>World</hello> xquery>^[[A 2. When loading namespaces and modules, and making an error, the error can only be corrected *after* I restart the client and server processes. For example, when I try to import a module with namespace "http://www.foo.edu" but accidentally type "http://www.food.edu", I receive an error message. This is expected. However, if I try again to import the module again, this time with the correct namespace, I get the same exact error message (complete with "http://www.food.edu"). This continues unless I restart both the client and server. Is this a "feature", or am I doing something incorrectly? Thanks for your help, Steve

2 1

[MonetDB-users] Creating table with nologging.
by Shailendra Mishra 28 Jul '10

28 Jul '10

I have the need for creating table with nologging enabled. I do understand that MonetDB is not meant to be used this way. However, I am trying to use it to ingest data off data-feeds. My db lives on a SSD, and the only DML that I execute is INSERT. I would appreciate if folks familiar with the server code could point me source files where I could make the change to achieve this. Thanks in advance for your help. -- Shailendra

2 1

[MonetDB-users] XRPC/execute at not working from mclient
by Ryad Ben-El-Kezadri 27 Jul '10

27 Jul '10

Hello, I found some problems to run a remote Xquery from mclient. The query is the following (it works fine with the Java XRPC API but not with mclient). mclient -lx -s "execute at {'192.168.0.12'} {pf:collections()}" I get the following error. user(win32): password: MAPI = win32@localhost:50000 QUERY = execute at {'192.168.0.12'} {pf:collections()} ERROR = !ERROR: interpret: no matching MIL operator to 'reverse(void)'. !MAYBE YOU MEAN: ! reverse(BAT[any::1,any::2]) : BAT[any::2,any::1] !ERROR: interpret_params: sort(param 1): evaluation error. !ERROR: interpret_params: reverse(param 1): evaluation error. Did anyone had already successfully run such kind of query? PS1: there are 2 Mserver running: one on the local host/one on the remote host. The Java XRPC API does not need to run a Mserver on the local host, but mclient do (else I get a "initiating connection socket failed" error ) PS2:I have Mserver 4.39.0 on the remote linux server and Mserver 4.36.5 on the windows client (The query also fails with client/linux/4.39.0 & server/linux/4.39.0) PS3: the default example provided in the XRPC doc fails (no MIL operator) the same way: import module namespace test = "xrpc-test-function" at " http://192.168.0.12:50001/export/xrpc-mod.xq"; execute at {"192.168.0.12"} {test:add(100, 200)} Regards Ryad

2 1

Re: [MonetDB-users] about the loadfile size
by Stefan Manegold 20 Jul '10

20 Jul '10

Dear Kun Ren, first of all, I would highly appreciate if you could stick to using the MonetDB mailing lists for such questions about MonetDB instead of sending me private email. On the one hand, this increases the chances for you to get quick and suitable answers, as more people read the mailing lists. On the other hand, other users might have similar questions and benefit from the answers sent to the mailing lists. Having said that, given that you mention neither the version of MonetDB you're using or any specification of your Linux & Windows machine(s) (in particular whether MonetDB, the OS, and the hardware architecture are 32- or 64-bit), nor whether you get any error or not (What exactly does "auto stop mean"? What exactly happens?) I can only guess that your Linux system is 64-bit while your Windows system (or at least the MonetDB version you installed on it) is 32-bit. For a detailed discussion of the impact of 32-bit vs. 64-bit on the scalability of MonetDB, please see http://sourceforge.net/mailarchive/message.php?msg_name=20100319145945.GA91… Hope this helps. Stefan On Tue, Jul 20, 2010 at 05:19:40PM +0800, kun ren wrote: > Dear professer Stefan, > Recently,I use TPC-H test the Monetdb, On windows ,so I use "copy > ,, records,,,into,,from" to load data to Monetdb,when the size of the file > is about 700M,It is OK.But when the size is 1.4G,the Monetdb server is auto > stop when loading. But when I am in Linux, it is no problem. So I want to > ask why it is wrong on windows when the file is so large? > Best regards, -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 1

[MonetDB-users] Announcement: New Jun2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 16 Jul '10

16 Jul '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010-SP1 bug fix release of the MonetDB suite of programs. Lots of problems have been fixed, the most important one being the fix in the handling of database upgrades of databases created with the Feb2010 release to the current version. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location has changed to <http://dev.monetdb.org/downloads/>. Please fix any bookmarks you may have. -- Sjoerd Mullender

1 0

[MonetDB-users] php mapi
by dariuszs 15 Jul '10

15 Jul '10

Hi, PHP does not return rows from the query when the result set reaches the size about 30KB, returns fine when it's less. Thanks.

4 8

[MonetDB-users] SQL text normalisation principles in relation to MonetDB
by Stefan de Konink 15 Jul '10

15 Jul '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, I rather have a general question and I would like to know something about normalisation in MonetDB in relation to the underlying storage model. Since my first courses in SQL, that was PostgreSQL at the time. It was a strong advise always try to normalise duplications of data, because it would always increase your search preformance to look up numerical keys over string comparisons. Given that in MonetDB, text-strings are deduplicated in a best effort way, factually become numbers, how does this compare to the additional costs of: - enums - foreign key relations For example, some database schema's use enums (or worse: varchars) for two values. For example 'accessible', 'inaccessible'. It is clear that the storage size for this field BOOLEAN NOT NULL, would be sufficient and of the length of one bit (best case). Now within MonetDB the string data is currently optimised, would it therefore cost additional time to create a secondary table to join to vs using a string field. Thus, does normalisation for 'manual' deduplication, hurt or not? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkw9uQAACgkQYH1+F2Rqwn2sjACeNnvDQS+cXjzs1USpomkL6rz8 7wkAn2rF2ZRvEJbQ1cX+oxJsWPcMoTk6 =RzDh -----END PGP SIGNATURE-----

3 2

[MonetDB-users] how to write function like oracle nvl()?
by Mike Jason 13 Jul '10

13 Jul '10

Hi, I need some help on how to write some special functions that monetdb currently doesn't have. E.g., I want to use the oralce funtion nvl(), which is described as follows: http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/functions105… Then, how can I write a function that has the some ability as the oralce's? By the way, now I use MonetDB-Feb2010.

2 5

[MonetDB-users] MonetDB - Function Sequence Error
by r8rdarklord 13 Jul '10

13 Jul '10

I have written a multithreaded ODBC program which tries to insert data to 11 tables in MonetDB. Each thread has a connection handle, a statment handle (comprising of insert into blah!....). Each thread on instantiation creates the handles (connection, statement etc.), connects to the db, binds meta-data, prepares query and executes the insert in a loop after reading the bind values from a file. The program runs fine to a point following which: SQLExecute: Error: SQLstate HY010, Errnr 0, Message [MonetDB][ODBC Driver 1.0]Function sequence error On closer examination it's the statement handle that is not in Prepared state. I thought that this maybe because the asynchronous execution of the previous execution of the commit isn't complete yet. I recoded my commit as follows: ret_sts = SQLEndTran(SQL_HANDLE_DBC, dbch, SQL_COMMIT); while (ret_sts == SQL_STILL_EXECUTING) ret_sts = SQLEndTran(SQL_HANDLE_DBC, dbch, SQL_COMMIT); check(ret_sts, SQL_HANDLE_DBC, dbch, "SQLEndTran"); However, the ret_sts returns -1 which means a SQL_ERROR This has me perplexed. Clearly the SQL is good else it wouldn't have executed a no. of times in the first place. The stmth also look good, so there is no stepping by threads on each others memory. I can post the whole program if needed. -- View this message in context: http://old.nabble.com/MonetDB---Function-Sequence-Error-tp29148194p29148194… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

Re: [MonetDB-users] [Monetdb-developers] about import *.csv
by Stefan de Konink 12 Jul '10

12 Jul '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 You must reply to the mailinglist, and not to me directly. In order to help you with your segmentation fault you also must provide what version you are using. Op 12-07-10 06:57, kun ren schreef: > Thank you, when I am trying,something is wrong, > when i use command," mclient -l sql -d demo" > and then use command "create table t1(name char(100),number int);" to > create a table; > and then I create a csv file, the content of the file is: > renkun,1 > shaolong,2 > laoluo,3 > and then i use command as you said,"copy 3 records into t1 from 1.csv > delimiters ',','\n';" but the server said:"Segmentation fault > ",what wrong with me?thanks > Best regards; > > > > > > 2010/7/12 Stefan de Konink <stefan(a)konink.de <mailto:stefan@konink.de>> > > On Mon, 12 Jul 2010, kun ren wrote: > > Dear all, > I want to import *.csv into MonetDB,what interface do i will > use？ > > > Just mclient, create your table. And use copy into so for example > > copy i<<numofrows>> records into <<justcreatedtable>> from > 'filename' delimiters ',', '\n', null as ''; > > > Stefan > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkw6oSYACgkQYH1+F2Rqwn1OzwCglSIGOl+tKxMk+bbDPPaLUX+i plwAn0aeeJKs3TLiNk4cHPURovJqh8T7 =+Tr/ -----END PGP SIGNATURE-----

3 4

[MonetDB-users] hash accelerator
by Miguel Ángel Águila Lorente 09 Jul '10

09 Jul '10

Hello, I'm working with mal and I've seen in the web documentation, in the part of the inner core that there are built-in accelerator functions. As I can see, there is one search accelerator: hashing. ¿Could you tell me how to use this accelerator using mal language? Looking forward to hearing from you. Thank you. Mike

2 2

[MonetDB-users] MonetDB scalability
by Chris 09 Jul '10

09 Jul '10

Hi, I'm currently evaluating MonetDB for business reporting, and looking for information on scalability, and I can't seem to find any information on clustering options. Is there any support in MonetDB for clustering across multiple physical servers to improve performance? Thanks, Chris

6 10

[MonetDB-users] sql_debug magic number?
by Mark Bucciarelli 06 Jul '10

06 Jul '10

What's a good value for sql_debug in monetdb5.conf? I tried 1 and 17, but got nothing in merovingian.log. Looks like perhaps something was written to dbfarm/<db>/sql_logs/sql/log, but I couldn't make any sense of it. Thanks, m

1 0

Re: [MonetDB-users] SQL: count() counts distinct?
by Ying Zhang 04 Jul '10

04 Jul '10

Hi Niels, I have just updated my stable branch with your backport. It's fixed. Thanks! Jennie On Jul 04, 2010, at 18:42 , Niels Nes wrote: > On Sun, Jul 04, 2010 at 02:49:11PM +0200, Ying Zhang wrote: >> Dear all, >> >> I have a table "a" as the following: >> >> sql>select * from a; >> +------+------+ >> | x | y | >> +======+======+ >> | 1 | 1 | >> | 1 | 2 | >> | 1 | 3 | >> | 2 | 1 | >> | 2 | 2 | >> | 2 | 3 | >> | 2 | 1 | >> | 1 | 2 | >> | 2 | 3 | >> +------+------+ >> 9 tuples >> >> I want to count the number of distinct tuples in the table, so I did the following: >> >> sql>select count(*) from (select distinct x,y from a) as b; >> +------+ >> | L17 | >> +======+ >> | 2 | >> +------+ >> 1 tuple >> sql>select count(x) from (select distinct x,y from a) as b; >> +------+ >> | L20 | >> +======+ >> | 2 | >> +------+ >> 1 tuple >> sql>select count(y) from (select distinct x,y from a) as b; >> +------+ >> | L21 | >> +======+ >> | 3 | >> +------+ >> 1 tuple >> >> It looks like that count() automatically counts the distinct values of x and y, while "select distinct x,y from b" does give me what I want: >> >> sql>select distinct x, y from b; >> +------+------+ >> | x | y | >> +======+======+ >> | 1 | 1 | >> | 1 | 2 | >> | 1 | 3 | >> | 2 | 1 | >> | 2 | 2 | >> | 2 | 3 | >> +------+------+ >> 6 tuples >> >> Is this an expected behaviour of count()? If yes, how can I get the #tuples of "select distinct x,y from b"? >> >> Thanks in advance! >> >> Jennie >> >> PS> info about mserver5: > Jenny it seems one of the m5 optimizers breaks the count (select distinct) case. > In current this seems fixed again. > > Niels >> >> $ mserver5 >> # MonetDB server v5.18.6, based on kernel v1.36.6 >> # Serving database 'demo', using 4 threads >> # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked >> # Found 7.751 GiB available main-memory. >> # Copyright (c) 1993-July 2008 CWI. >> # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved >> # Visit http://monetdb.cwi.nl/ for further information >> #warning: please don't forget to set your vault key! >> #(see [prefix]/stable/debug/etc/monetdb5.conf) >> # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ >>> include sql; >> # MonetDB/SQL module v2.36.6 loaded >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Sprint >> What will you do first with EVO, the first 4G phone? >> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first >> _______________________________________________ >> MonetDB-users mailing list >> MonetDB-users(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-users > > -- > Niels Nes, Centrum Wiskunde & Informatica (CWI) > Science Park 123, 1098 XG Amsterdam, The Netherlands > room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl > url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

[MonetDB-users] SQL: count() counts distinct?
by Ying Zhang 04 Jul '10

04 Jul '10

Dear all, I have a table "a" as the following: sql>select * from a; +------+------+ | x | y | +======+======+ | 1 | 1 | | 1 | 2 | | 1 | 3 | | 2 | 1 | | 2 | 2 | | 2 | 3 | | 2 | 1 | | 1 | 2 | | 2 | 3 | +------+------+ 9 tuples I want to count the number of distinct tuples in the table, so I did the following: sql>select count(*) from (select distinct x,y from a) as b; +------+ | L17 | +======+ | 2 | +------+ 1 tuple sql>select count(x) from (select distinct x,y from a) as b; +------+ | L20 | +======+ | 2 | +------+ 1 tuple sql>select count(y) from (select distinct x,y from a) as b; +------+ | L21 | +======+ | 3 | +------+ 1 tuple It looks like that count() automatically counts the distinct values of x and y, while "select distinct x,y from b" does give me what I want: sql>select distinct x, y from b; +------+------+ | x | y | +======+======+ | 1 | 1 | | 1 | 2 | | 1 | 3 | | 2 | 1 | | 2 | 2 | | 2 | 3 | +------+------+ 6 tuples Is this an expected behaviour of count()? If yes, how can I get the #tuples of "select distinct x,y from b"? Thanks in advance! Jennie PS> info about mserver5: $ mserver5 # MonetDB server v5.18.6, based on kernel v1.36.6 # Serving database 'demo', using 4 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 7.751 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see [prefix]/stable/debug/etc/monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ >include sql; # MonetDB/SQL module v2.36.6 loaded

1 0

[MonetDB-users] Announcement: New Jun2010 feature release of MonetDB suite
by Sjoerd Mullender 02 Jul '10

02 Jul '10

It is not June anymore, but nevertheless, the MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location has changed to <http://dev.monetdb.org/downloads/>. Please fix any bookmarks you may have. -- Sjoerd Mullender

1 2

[MonetDB-users] mclient --password
by Matthieu Guamis 01 Jul '10

01 Jul '10

Hello, Wow! the "--password" option is no more accepted by mclient utility :) Is there any alternative for automatic login? Thanks a lot -- *Matthieu Guamis* *Logo Axège <http://www.axege.com/>* /Axège// 23,rue Saint-Simon 63000 Clermont-Ferrand/ Tél: +33 (0)4 63 05 95 40 Fax: +33 (0)4.73.70.65.29 Email: matthieu.guamis(a)axege.com <mailto:matthieu.guamis@axege.com>

3 2

[MonetDB-users] Server requires 'SHA512' hash, but client support was not compiled in
by Matthieu Guamis 01 Jul '10

01 Jul '10

Hello, I get this message "Server requires 'SHA512' hash, but client support was not compiled in" when trying to connect to database. I installed MonetDB on RHEL5 system (more precisely Oracle Entreprise Linux with SHA256 not available in Openssl) from the script "monetdb-install.sh". In a previous post <http://sourceforge.net/mailarchive/forum.php?thread_name=20090731161927.GC2…> Fabian Groffen wrote: > Adding --with-password-backend=XXX to the configure call should be > enough. > But I'm not sure what does it mean (sorry) so my question is, how do you do that? Thanks a lot for your help PS: mserver5 --version MonetDB server v5.18.6 (64-bit), based on kernel v1.36.6 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 2.9GiB available memory, 8 available cpu cores Configured for prefix: /home/admaxg/MonetDB Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8e-rhel5 01 Jul 2008 (compiled with OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008) Compiled by: admaxg(a)localhost.localdomain (x86_64-unknown-linux-gnu) Compilation: gcc -O2 Linking : ld -IPA -m elf_x86_64 -- *Matthieu Guamis* *Logo Axège <http://www.axege.com/>* /Axège// 23,rue Saint-Simon 63000 Clermont-Ferrand/ Tél: +33 (0)4 63 05 95 40 Fax: +33 (0)4.73.70.65.29 Email: matthieu.guamis(a)axege.com <mailto:matthieu.guamis@axege.com>

1 0

[MonetDB-users] sql: getting (or specifying) sequence name
by Mark Bucciarelli 01 Jul '10

01 Jul '10

Hi, Given a table with a sequence that controls the primary id, is there a SQL command I can run to get the name of that sequence? Alternatively, is there a way to specify the sequence name when creating the table? Thanks, m

3 5

[MonetDB-users] full text index?
by Patrick Schäfer 30 Jun '10

30 Jun '10

Hi, is there any support for a full text index? In the documentation it says: "port PF/Tijah to support the XQuery Full-Text specification." Thank you Patrick

1 0

[MonetDB-users] How to recover data from a crash?
by 李坤 30 Jun '10

30 Jun '10

Hi all. I plan to deploy monetdb into production environment, but i don't know how to recover from a crash. I find some sql log files in var\sql_logs\$dbname\sql\, can i use these log files to recover, and how? Or are there any other methods to recover from crashes?

3 3

[MonetDB-users] problem with monetdb
by Andrea Panetta 28 Jun '10

28 Jun '10

Hi at All. I've a problem with monetdb... I need to use it with jdbcDriver eith iSQL viewer but, when I user mi driver, and i load it with my credential, I've the following error Error: ResultSet is closed. It's strange because, when I try the driver with eclipse, my programs work well but, when I use this driver with an external software, like isql viewer, it don't work. How I can do to fix this error? Thank you so much. (PS: I've the monetdb jdbc driver version 1.15, and it should be the laste updated driver) -- Panetta Andrea Via Pertini 22 87055 - San Giovanni in Fiore (CS) tf - 329 / 3554684

3 3

[MonetDB-users] MonetDB aborts when importing over 10 million rows
by Luis Fernando Kauer 26 Jun '10

26 Jun '10

Hi, I'm still trying to evaluate MonetDB but I keep getting errors when trying to import over 10 million rows to a table. The server simply aborts and closes. I'm using MonetDB on Windows XP. Whenener I restart after the server had aborted it shows messages like this: #GDKmmap(71827456) fails, try to free up space [memory in use=18552560,virtual m emory in use=1837957120] #GDKmmap(71827456) result [mem=18552560,vm=1837957120] #GDKmmap(36175872) fails, try to free up space [memory in use=16499872,virtual m emory in use=1844903936] #GDKmmap(36175872) result [mem=16499872,vm=1844903936] #GDKmmap(143130624) fails, try to free up space [memory in use=15697728,virtual memory in use=1744764928] #GDKmmap(143130624) result [mem=15697728,vm=1744764928] #GDKmmap(143130624) fails, try to free up space [memory in use=13554656,virtual memory in use=1654456320] #GDKmmap(143130624) result [mem=13554656,vm=1654456320] #GDKmmap(143130624) fails, try to free up space [memory in use=8747960,virtual m emory in use=1650917376] #GDKmmap(143130624) result [mem=8747960,vm=1650917376] #GDKmmap(143130624) fails, try to free up space [memory in use=7946840,virtual m emory in use=1531052032] #GDKmmap(143130624) result [mem=7946840,vm=1531052032] !mvc_init: unable to create system tables !SQLException:SQLinit:Catalogue initialization failed !ERROR: HEAPextend: failed to extend to 71722596 for 14\1475tail !ERROR: HEAPextend: failed to extend to 35861298 for 15\1537tail !ERROR: HEAPextend: failed to extend to 142816048 for 15\1543tail !ERROR: HEAPextend: failed to extend to 142816048 for 15\1550tail !ERROR: HEAPextend: failed to extend to 142816048 for 15\1556tail !ERROR: HEAPextend: failed to extend to 142816048 for 15\1557tail #function user.main():void; # sql.prelude(); #end main; First I was trying to import files with about 3 million rows each but it started giving errors after the 3rd file. Then I tried to import through files with about 1 million rows each and I was able to import about 14 million rows, but after about 10 million rows I was only able to import one or two files in a row without MonetDB resetting. Is this a bug or a size limit for the available memory. Anything I could try? With regards, Luis Fernando

2 5

[MonetDB-users] "Parent Directory" links in download area give 404.
by Mark Bucciarelli 25 Jun '10

25 Jun '10

(1) open http://monetdb.cwi.nl/downloads/Ubuntu/ (2) click "Parent Directory" link (3) 404 I saw same behavior after opening http://monetdb.cwi.nl/downloads/sources/Feb2010-SP2 so it may be a global issue. Thanks, m

2 1

[MonetDB-users] Copy Into and .new files
by Alex Bo. 23 Jun '10

23 Jun '10

Hi, I am doing a lot of COPY INTOs, and I have noticed that sometimes while doing this MonetDB seems to hang while its process memory is slowly rising. I then noticed that while this is happening, files with a '.new' extension are added to the dbfarm folders. Now suddenly I reached a state in which no '.new' files are created at all, and the COPY INTOs work much faster because no 'hanging' occurs in which something is loaded into memory. What's the explanation for this? What can I do in order to always have COPY INTOs write directly to the final files without the '.new' intermediates? Thanks. -- View this message in context: http://old.nabble.com/Copy-Into-and-.new-files-tp28973215p28973215.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

Re: [MonetDB-users] Embedded server in Java with JDBC
by Luis Fernando Kauer 23 Jun '10

23 Jun '10

Hi, Thanks for the quick reply. When I say embedded I mean like in HSQLDB, which allows to embed the db server in the application instead of starting the db server in a separete process accepting connections from all clients. My requirements are that the end user should not have to deal with the database directly, like start and stop it, not even need to install/configure it separetely. But I would like to access the data through JDBC driver because the libraries I'm working with require it. I also would like a small footprint. If that's not possible, then I would like to be able to start and stop the server through the application, even if it runs in a separete process. The application allows the user to analyse and audit a large volume of data on different tables and MonetDB seems to be really fast for this. Each user project will have its own set of tables so it would be nice to be able to set the database files where the user chooses for each project. If that's not possible I'll have to stick with different schemas for each project. I'm still having some problems importing large tables (> 10 million rows divided in 10 files with 1 million rows each) when MonetDB is reporting the following error SQLException:sql:Failed to extend the BAT, perhaps disk full SQLException:importTable:failed to import table and on the console appears many times the follwing: #GDKmmap(111673344) fails, try to free up space [memory in use=56663400,virtual memory in use=1521352704] #GDKmmap(111673344) result [mem=56663160,vm=1127612416] #GDKmmap: recovery ok. Continuing.. Is there any configuration or a way to make MonetDB work with large tables without having more memory available? Thanks for your help! Regards, Luis Fernando

2 1

[MonetDB-users] Embedded server in Java with JDBC
by Luis Fernando Kauer 22 Jun '10

22 Jun '10

Hi, I would like to use the embedded server within a Java application connecting through JDBC. Is it possible? I would probably need to start the embedded server before trying to connect through JDBC, right? Or is there a connection string that makes the jdbc driver start and use an embedded server? That would be great! If not, what's the best way to start the embedded server in Java? Thanks for this great and fast dabatase. Regards, Luis Fernando

2 1

[MonetDB-users] Mapiclient password prompt
by jz＠uva 21 Jun '10

21 Jun '10

I use the Mclient in conjunction with XQuery. When I do a CVS checkout, I get the version of Mapiclient where the -P option has disappeared, yet it still asks me for a password. My question is: how do I disable the password prompt? It is a bit too much to enter the password for thousands of queries, although I can type quite quickly. ;-) junte

3 4

[MonetDB-users] Get average value for intervals
by Gustav Koller 18 Jun '10

18 Jun '10

Hello, I've some time series data (time + a few value columns), now I want to generate average values for each hour. Currently I'm using the following query: SELECT (time - (time % 3600)) AS timegroup, AVG(value0) AS avgvalue0 FROM dbbench WHERE time > x AND time <= y GROUP BY timegroup; I've a few GB of data and the query is running for several seconds. Does anyone have any recommendations on how to improve the query for faster execution time? Greetings Gustav

2 2

[MonetDB-users] Limit memory
by Miguel Ángel Águila Lorente 17 Jun '10

17 Jun '10

Hello, I'm trying to limit the memory that monetdb can use during its executions. I'm using mal language and I've found this operation in order to do it. command mem_maxsize(v:lng) :void address set_mem_maxsize comment "Set the maximum usable amount of physical swapspace in KB."; When I use this operation the connection is terminated. I let here some examples: (I'm working with a machine with 32 GB) mal>status.mem_maxsize(4294967296:lng); MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = status.mem_maxsize(4294967296:lng); ERROR = !Connection terminated mal>status.mem_maxsize(1073741824:lng); MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = status.mem_maxsize(1073741824:lng); ERROR = !Connection terminated mal>status.mem_maxsize(1048576:lng); MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = status.mem_maxsize(1048576:lng); ERROR = !Connection terminated Looking forward to hearing from you. Thank you. Mike

4 6

Re: [MonetDB-users] Meaning of: Scenario not initialized 'sql'
by Yel 17 Jun '10

17 Jun '10

Hi, I know this topic has been discussed last year, but with the MonetDB-Feb2010-SuperBall-SP2 installed, I repeatedly got the error like MALException:setScenario:Scenario not initialized 'sql' almost every time when I try to start mclient with "mclient -l sql -d dbname", even when i try it later on. Moreover, only "mclient -l mal -d dbname" is OK, sql, xquery, mil all got the error above. Any idea ? BTW: 1. MonetDB is installed by ./monetdb-install.sh --enable-sql --enable-debug 2. mapi_port is set to 50003 in etc/monetdb5.conf; Thanks. ---- Best Regards Ye Liang Sjoerd Mullender-2 wrote: > Alex Bo. wrote: > > Hi, > > > > Sometimes I get !MALException:setScenario:Scenario not initialized 'sql' > > from the server when I try to start it in scenarios that usually work > > without any problems. I can't reproduce it consistently, but sometimes I get > > it. What does this exception mean? > > > > Thanks. > > This exception means that the server is not yet ready for accepting SQL > connections. The solution is to try again later. > > When the server starts up, it pretty quickly starts accepting > connections, but initially only for MAL. When the SQL engine is > initialized (which may take a while if there was any recovery to be > done), it starts accepting SQL connections as well. You get this > exception when you connect before the SQL engine is ready. > > -- > Sjoerd Mullender

4 4

Re: [MonetDB-users] MonetDB XML - slow execution of query, with test on attribute
by Patrick Schäfer 17 Jun '10

17 Jun '10

Hi Jan, thank you for your response. I thought that all documents within a collection are stored within one (or several) relational tables? I am not sure, if merging all documents into one big document is an option. The documents are supposed to be updateable ... does that give a good performance for one large document? Patrick > Message: 7 > Date: Tue, 15 Jun 2010 21:58:44 +0200 > From: Jan Rittinger <jan.rittinger(a)uni-tuebingen.de> > Subject: Re: [MonetDB-users] MonetDB XML - slow execution of query > with test on attribute > To: Communication channel for MonetDB users > <monetdb-users(a)lists.sourceforge.net> > Message-ID: <702CE627-774E-47C5-B3B7-34A5431F20CF(a)uni-tuebingen.de> > Content-Type: text/plain; charset="windows-1252" > > Hi Patrick, > > the evaluation plan of your query looks quite bad -- with respect to what could be possible... > > If you could use 'doc("tu")' (with a single big document containing all small documents), you would get the result in *no* time, as then the attribute index is used. Unfortunately I'm not aware of the difference in the attribute index with respect to collections... > > Regards, > Jan >

2 1

Re: [MonetDB-users] MonetDB XML - slow execution of query with test on attribute
by Patrick Schäfer 17 Jun '10

17 Jun '10

Hi Peter, thank you for your answer. using pf:collection gives me a "Speicherzugriffsfehler" (/Segmentation fault)/ when executing the query you proposed. Is that a bug in MonetDB XML? replacing "pf:collection" by "fn:collection" leads to Trans 55.436 msec Shred 19.083 msec Query 29241.029 msec Print 0.207 msec Timer 29582.455 msec So it is basically the same query time :( Patrick > Hi, > > How about: > > smiles> { > for $i in pf:collection("tu") > where $i//xq:molecule/@id = "1" > return > <smile>{$i}</smile> > } </smiles> > > is that any faster? > > Peter > > >

1 0

Re: [MonetDB-users] Meaning of: Scenario not initialized 'sql'
by yel 17 Jun '10

17 Jun '10

Hi, I know this topic has been discussed last year, but with the MonetDB-Feb2010-SuperBall-SP2 installed, I repeatedly got the error like MALException:setScenario:Scenario not initialized 'sql' almost every time when I try to start mclient with "mclient -l sql -d dbname", even when i try it later on. Moreover, only "mclient -l mal -d dbname" is OK, sql, xquery, mil all got the error above. Any idea ? BTW: 1. MonetDB is installed by ./monetdb-install.sh --enable-sql --enable-debug 2. mapi_port is set to 50003 in etc/monetdb5.conf; Thanks. ---- Best Regards Ye Liang > > Sjoerd Mullender-2 wrote: > Alex Bo. wrote: > > Hi, > > > > Sometimes I get !MALException:setScenario:Scenario not initialized 'sql' > > from the server when I try to start it in scenarios that usually work > > without any problems. I can't reproduce it consistently, but sometimes I get > > it. What does this exception mean? > > > > Thanks. > > This exception means that the server is not yet ready for accepting SQL > connections. The solution is to try again later. > > When the server starts up, it pretty quickly starts accepting > connections, but initially only for MAL. When the SQL engine is > initialized (which may take a while if there was any recovery to be > done), it starts accepting SQL connections as well. You get this > exception when you connect before the SQL engine is ready. > > -- > Sjoerd Mullender >

1 0

[MonetDB-users] Meaning of: Scenario not initialized 'sql'
by Alex Bo. 17 Jun '10

17 Jun '10

Hi, Sometimes I get !MALException:setScenario:Scenario not initialized 'sql' from the server when I try to start it in scenarios that usually work without any problems. I can't reproduce it consistently, but sometimes I get it. What does this exception mean? Thanks. -- View this message in context: http://www.nabble.com/Meaning-of%3A-Scenario-not-initialized-%27sql%27-tp26… Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

[MonetDB-users] MonetDB XML - slow execution of query with test on attribute
by Patrick Schäfer 15 Jun '10

15 Jun '10

Hi, I am trying to write a xquery-expression that does a test on an attribute (id="1"). Unfortunately the query execution time for this query is really bad (>30 seconds). Is there any way to find out why this query takes so long? 30 seconds for a test on one single attribute is just way to high for our use-case. I read that MonetDB automatically creates indices on all attribute - is there a way to test, if there was an index created for that attribute and if that index was used for query execution? The query (in File q3.xp) is supposed to return the molecule with id "1" declare namespace xq="http://www.xml-cml.org/schema"; <smiles> { for $i in collection("tu") where $i/xq:molecule/@id = "1" return <smile>{$i}</smile> } </smiles> The query is executed like this: # mclient -t -lx -G q3.xp ... Trans 55.684 msec Shred 18.818 msec Query 29335.766 msec Print 0.258 msec The database contains 1.000.000 XML-documents in a single collection, is updatable and is about 2601 MiB. A document looks similar to this one: <?xml version="1.0" encoding="UTF-8"?> <molecule id="1" xmlns="http://www.xml-cml.org/schema"> <bondArray atomRef1="..." order="..."/> <atomArray atomID="..." elementType="..." x3="..."/> <identifier convention="iupac:smile" value="..."/> <identifier convention="iupac:inchi" value="..."/> </molecule> We are using: "MonetDB/XQuery 0.36.5, based on the pathfinder <http://www.pathfinder-xquery.org> compiler and MonetDB 4.36.1 <http://monetdb.cwi.nl>" Thank you very much for any help! Patrick

3 2

[MonetDB-users] Decimal(p,s)
by Nozhup 15 Jun '10

15 Jun '10

What is the reason that MonetDB only supports a maximum p value of 18 while for example SQL Server (see link) supports up to 38? http://www.w3schools.com/SQL/sql_datatypes.asp -- View this message in context: http://old.nabble.com/Decimal%28p%2Cs%29-tp28892721p28892721.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] MIL and GROUP BY
by Hannes Rauhe 14 Jun '10

14 Jun '10

Hello together, I installed MonetDB with the help of the latest monetdb-install.sh and played around with MIL. Now I'm stuck with translating a simple SQL-Query containing a GROUPBY. After starting the server with # Mserver --dbname test --dbinit 'module(mapi); mil_start();' and the client with # ./mclient -lmil -utest -dname=test I tried the following: var a:= bat(oid,int); a.insert(1@0, 1995); a.insert(2@0, 1992); a.insert(3@0, 1993); a.insert(4@0, 1994); a.insert(5@0, 1995); a.insert(6@0, 1996); CTgroup(a).print(); the answer is: ERROR: const operation on a BAT ERROR: CMDconstCopy: operation failed. ERROR: interpret_params: print(param 1): evaluation error. Another try after loading the xtables-Module: mil>module("xtables"); mil>CTgroup(a).print(); +-------------------------------------------------------+ | � | +-------------------------------------------------------+ 1 tuple I used a few papers and http://monetdb.cwi.nl/MonetDB/Version4/Documentation/mil/index.html (Chapter 3.7 is what I'm looking for) as tutorial but didn't find anything helpful. Where's my mistake? Best Regards, Hannes

2 1

[MonetDB-users] Performance degradation
by Michael W Schatzow 11 Jun '10

11 Jun '10

hello, I am having performance degradation issues with repeated queries of monetdb. I am attempting to use monetdb for financial tick data and make sql queries requesting all quotes for a symbol on date x where time > y and time <z. Previously monetdb had worked great for this task and was extremely quick. However, I had a crash last week of the server it was built on. After this monetdb ran incredibly slow on the order of 25 times slower than it was running before. I rebuilt monetdb and it ran quickly for a day or two and started degrading in speed. Could someone give a hint or two on how I should go about debugging this? Is there a optimization task that I need to run? I am really at a loss of how I should handle this. I have been running queries via mclient -s from my program written in python. I previously found the python bindings too slow. Stats: cpu: 2 opteron 2222 os:cent os 5.4 memory 8gb database Size: only testing with csv data of 1.3 GB indexed by date and time built with options:--enable-optimise --enable-sql Thanks for any help, Michael

1 0

[MonetDB-users] libbat.dll crash
by Alex Bo. 09 Jun '10

09 Jun '10

Hi, I am experiencing a consistent Monet crash on an XP 32 bit machine, because of libbat.dll. The exact problem is mentioned here: http://old.nabble.com/--monetdb-Bugs-2982639---Crash-during-server-startup-… However, the link over the there to sourceforge doesn't work, so I can't comment or see what's going on with that bug. I tried it on Aug09-SP1 and Feb2010-SP2 (built from CSV), and both crashed. The link suggest that maybe older Monet dlls are laying around in the system32 folder, but I do not have any of the suggested dlls in there. Any suggestions on this issue? -- View this message in context: http://old.nabble.com/libbat.dll-crash-tp28826934p28826934.html Sent from the monetdb-users mailing list archive at Nabble.com.

4 3

[MonetDB-users] PF/Tijah Question: BM25F?
by jz＠uva 07 Jun '10

07 Jun '10

I was wondering whether it would be possible to use BM25F? http://trec.nist.gov/pubs/trec13/papers/microsoft-cambridge.web.hard.pdf Anyone else tried this before with an XQuery? junte

4 7

Re: [MonetDB-users] Server Crash Upon Connect
by Hering Cheng 28 May '10

28 May '10

Resent, with some of thread details removed due to limits on message length. On Thu, May 27, 2010 at 2:49 PM, Hering Cheng <hering.cheng(a)computer.org>wrote: > Hi, > > MonetDB core dumped after I had 10 Java threads retrieving a total of 23 > million records from a single table via JDBC. The threads actually all > completed successfully. The crash seems to occur when another process tried > to open a connection to the same database. I can reproduce the crash > consistently. > > After the core file is produced, processes can connect to MonetDB > successfully, with Merovingian starting up the dead mserver5 automatically. > > $ tail > ~/gaal/rdcuxsrv220-local-disk/chenher/monetdb/feb2010/var/log/MonetDB/merovingian.log > 2010-05-27 14:27:39 MSG merovingian[19478]: client rdcuxsrv220:61292 has > disconnected from proxy > 2010-05-27 14:27:42 MSG merovingian[19478]: client rdcuxsrv220:61296 has > disconnected from proxy > 2010-05-27 14:27:42 MSG merovingian[19478]: client rdcuxsrv220:61294 has > disconnected from proxy > 2010-05-27 14:27:42 MSG merovingian[19478]: client rdcuxsrv220:61298 has > disconnected from proxy > 2010-05-27 14:27:42 MSG merovingian[19478]: client rdcuxsrv220:61288 has > disconnected from proxy > 2010-05-27 14:27:42 MSG merovingian[19478]: client rdcuxsrv220:61304 has > disconnected from proxy > 2010-05-27 14:27:42 MSG merovingian[19478]: client rdcuxsrv220:61302 has > disconnected from proxy > 2010-05-27 14:32:33 MSG merovingian[19478]: proxying client > rdcuxsrv220:61784 for database 'taq' to mapi:monetdb://127.0.0.1:50001/taq > 2010-05-27 14:33:01 MSG merovingian[19478]: client rdcuxsrv220:61784 has > disconnected from proxy > 2010-05-27 14:34:06 MSG merovingian[19478]: database 'taq' (17289) has > crashed (dumped core) > > This is a Solaris SPARC server and MonetDB was built from the Feb2010 > source code using Sun Studio 12.1: > > $ ( ulimit -d $[32*1024*1024] && ulimit -n $[10*1024]; export > LD_PRELOAD_64=/usr/lib/64/libumem.so:${LD_PRELOAD_64}; export > LD_PRELOAD=/usr/lib/libumem.so:${LD_PRELOAD}; export > MONETDB5CONF=/GAAL/chenher/rdcuxsrv220-local-disk/chenher/monetdb/feb2010/etc/monetdb5.conf; > /GAAL/chenher/share/monetdb/distro-sparc-feb2010-64bit-debug/bin/mserver5 > --version; ) > MonetDB server v5.18.1 (64-bit), based on kernel v1.36.1 (64-bit oids) > Copyright (c) 1993-July 2008 CWI > Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved > Visit http://monetdb.cwi.nl/ for further information > Found 32.0GiB available memory, 16 available cpu cores > Configured for prefix: > /GAAL/chenher/share/monetdb/distro-sparc-feb2010-64bit-debug > Libraries: > libpcre: 8.01 2010-01-19 (compiled with 8.01) > openssl: OpenSSL 0.9.8k 25 Mar 2009 (compiled with ) > libxml2: 2.6.23 (compiled with 2.6.23) > Compiled by: chenher@rdcuxsrv220 (sparc-sun-solaris2.10) > Compilation: cc -m64 -xcode=pic32 > -I/GAAL/chenher/share/hans_boehm_gc/distro-sparc-6.8-64bit/include/ -g > Linking : /usr/ccs/bin/ld -m64 -L/GAAL/chenher/share/openssl-64bit/lib > -L/GAAL/chenher/share/pcre-64bit/lib > -L/GAAL/chenher/share/hans_boehm_gc/distro-sparc-6.8-64bit/lib/ > > $ dbx ~/gaal/share/monetdb/distro-sparc-feb2010-64bit-debug/bin/mserver5 > ~/gaal/rdcuxsrv220-local-disk/chenher/monetdb/feb2010/var/MonetDB5/dbfarm/taq/core > For information about new features see `help changes' > To remove this message, put `dbxenv suppress_startup_message 7.7' in your > .dbxrc > Reading mserver5 > ... > Reading lib_logger.so.5.18.1 > Reading libuuid.so.1 > t@4 (l@4) terminated by signal SEGV (no mapping at the fault address) > Current function is putName > 174 for(l= nme[0]; l && namespace.nme[l]; l= > namespace.link[l]){ > (dbx) where > current thread: t@4 > =>[1] putName(nme = 0xffffffff6aec3928 "exportValue", len = 11U), line 174 > in "mal_namespace.c" > [2] initSQLreferences(), line 49 in "sql_gencode.c" > [3] SQLinitClient(c = 0xffffffff7f352628), line 379 in "sql_scenario.c" > [4] runPhase(c = 0xffffffff7f352628, phase = 5), line 363 in > "mal_scenario.c" > [5] runScenarioBody(c = 0xffffffff7f352628), line 392 in "mal_scenario.c" > [6] runScenario(c = 0xffffffff7f352628), line 438 in "mal_scenario.c" > [7] MSserveClient(dummy = 0xffffffff7f352628), line 368 in > "mal_session.c" > (dbx) threads > t@1 a l@1 ?() LWP suspended in __pollsys() > t@2 a l@2 SERVERlistenThread() LWP suspended in __pollsys() > t@3 a l@3 mvc_logmanager() LWP suspended in __pollsys() > o> t@4 a l@4 ?() signal SIGSEGV in putName() > t@5 a l@5 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@6 a l@6 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@7 a l@7 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@8 a l@8 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@9 a l@9 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@10 a l@10 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@11 a l@11 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@12 a l@12 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@13 a l@13 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@14 a l@14 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@15 a l@15 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@16 a l@16 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@17 a l@17 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@18 a l@18 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@19 a l@19 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@20 a l@20 runDFLOWworker() sleep on 0x100c14b08 in > __lwp_park() > t@21 a l@21 ?() sleep on 0xffffffff7f352a98 in __lwp_park() > t@22 a l@22 ?() sleep on 0xffffffff7f352d58 in __lwp_park() > t@23 a l@23 ?() sleep on 0xffffffff7f353018 in __lwp_park() > t@24 a l@24 ?() sleep on 0xffffffff7f3532d8 in __lwp_park() > t@25 a l@25 ?() sleep on 0xffffffff7f353598 in __lwp_park() > t@26 a l@26 ?() sleep on 0xffffffff7f353858 in __lwp_park() > t@27 a l@27 ?() sleep on 0xffffffff7f353b18 in __lwp_park() > t@28 a l@28 ?() sleep on 0xffffffff7f353dd8 in __lwp_park() > t@29 a l@29 ?() sleep on 0xffffffff7f354098 in __lwp_park() > t@30 a l@30 ?() sleep on 0xffffffff7f354358 in __lwp_park() > t@31 a l@31 runDFLOWworker() sleep on 0x1012f8988 in > __lwp_park() > t@32 a l@32 runDFLOWworker() sleep on 0x1012f8988 in > __lwp_park() > t@33 a l@33 runDFLOWworker() sleep on 0x1012f8988 in > __lwp_park() > ... > t@174 a l@174 runDFLOWworker() sleep on 0x102c9f908 in > __lwp_park() > t@175 a l@175 runDFLOWworker() sleep on 0x102c9f908 in > __lwp_park() > > Thanks. > Hering >

1 0

[MonetDB-users] Loading problems since FEB2010+SP1 release
by Nozhup 26 May '10

26 May '10

Since the FEB2010+SP1 release we're encountering loading issues. With the FEB2010 release we were able to load records until the HDD was full (500+ million records) on PCs with 2,3 or 24 GB ram memory without going to the SWAP. This was done by loading CSV-files with 1 million records each through the COPY INTO command (in the mclient and through a c-script). Since the FEB2010+SP1 release the system starts to SWAP after loading the first couple of million records. This results in increased loadtimes per loaded CSV-file: 1th million: 14 sec 2nd million: 26 sec 3rd million: 44 sec 4th million: 50 sec 5th million: 69 sec etc. We have tested this on several PCs and all PCs encounter the same problem. Test files used are the same for both releases as is the OS (Ubuntu 9.10). The only variable that has changed is MonetDB. -- View this message in context: http://old.nabble.com/Loading-problems-since-FEB2010%2BSP1-release-tp282876… Sent from the monetdb-users mailing list archive at Nabble.com.

5 7

[MonetDB-users] June Candidate release
by Nozhup 26 May '10

26 May '10

Hi, I'm having trouble making the June Candidate release. OS Used Ubuntu 10.04 Versions Tried 20100516/ 20100525/ 20100526/ Command Used ./monetdb-install.sh --enable-sql Part Of Error Log SQLExtendedFetch.c:43: error: conflicting types for 'SQLExtendedFetch' /usr/include/sqlext.h:1853: note: previous declaration of 'SQLExtendedFetch' was here make[7]: *** [libMonetODBC_la-SQLExtendedFetch.lo] Error 1 make[7]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients/src/odbc/driver' make[6]: *** [all] Error 2 make[6]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients/src/odbc/driver' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients/src/odbc' make[4]: *** [all] Error 2 make[4]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients/src/odbc' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/var/tmp/MonetDB-1274859923/MonetDB-client/clients' -- View this message in context: http://old.nabble.com/June-Candidate-release-tp28677743p28677743.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 5

[MonetDB-users] Mapi - mal
by Petros Tsialiamanis 20 May '10

20 May '10

Hello, I would like to get the oid of a stored uri using the Mapi library. The mal commands that need to be executed are: tokenizer.open("rdf"); id := tokenizer.locate("http://uri"); io.print(); tokenizer.close(); I tried to run the following program ...tokenizer.open("rdf");... ... if ((hdl = mapi_query(dbh, "tokenizer.locate(\"http://uri\")")) == NULL || mapi_error(dbh) != MOK) die(dbh, hdl); if (mapi_close_handle(hdl) != MOK) die(dbh, hdl); while (mapi_fetch_row(hdl)) result=mapi_fetch_field(hdl, 0); ... ...tokenizer.close();... .. but I get the following error Mapi.mx:5141: mapi_fetch_row: Assertion `(hdl)->mid' failed. Could you tell me how can I solve this problem? Kind regards, Petros

2 2

[MonetDB-users] Loading and querying on the same database.
by Nozhup 19 May '10

19 May '10

Loading and querying simultaneously seems to slowdown the database significantly. I can understand if the queries became slower during loading, but it seems that only the first query before loading is finished is slow and all queries coming after. I can only fix the slowdown by using "killall mserver5" in a terminal, which is not an acceptable solution. I start 1 "copy into" action with one connection and at the same time a series of simple queries on another connection. The Simple query looks something like this: SELECT x,y FROM table WHERE x = '519'; Used OS: Ubuntu 9.10 and 10.04 MonetDB: Feb2010 release (SP! and SP2 are no option due to them giving loading problems) Results Query: 1 --> 0.000191 SECONDS : : : Query: 10501 --> 0.000170 SECONDS Query: 10502 --> 0.000169 SECONDS Query: 10503 --> 0.000169 SECONDS Query: 10504 --> 0.000172 SECONDS Query: 10505 --> 0.000172 SECONDS Query: 10506 --> 0.000174 SECONDS Query: 10507 --> 0.000178 SECONDS Query: 10508 --> 1.199149 SECONDS Load : 1 --> 3.238532 SECONDS Query: 1-10509 --> 1.238804 SECONDS Query: 1-10510 --> 1.175408 SECONDS Query: 1-10511 --> 1.141172 SECONDS Query: 1-10512 --> 1.132525 SECONDS Query: 1-10513 --> 1.159554 SECONDS Query: 1-10514 --> 1.158303 SECONDS Query: 1-10515 --> 1.160997 SECONDS How do I solve this or is this an error in MonetDB? Thanks in advance. -- View this message in context: http://old.nabble.com/Loading-and-querying-on-the-same-database.-tp28605728… Sent from the monetdb-users mailing list archive at Nabble.com.

2 5

[MonetDB-users] updates in a transaction
by Min Lu 19 May '10

19 May '10

Hi, I tried to update a column of a record more than once within a transaction. But it seemed that only the first update took effect. For example, for a table "CREATE TABLE people (id TINYINT PRIMARY KEY, name VARCHAR(128) NOT NULL)", sql>select * from people; +------+-----------------------------+ | id | name | +======+=============+ | 0 | Phil Ivey | | 1 | Michael Jordan | | 2 | Lionel Messi | +------+-----------------------------+ sql>start transaction; auto commit mode: off sql>update people set id = -1 where name='Phil Ivey'; 1 affected row sql>select * from people; +------+-----------------------------+ | id | name | +======+=============+ | 1 | Michael Jordan | | 2 | Lionel Messi | | -1 | Phil Ivey | +------+-----------------------------+ sql>update people set id = -2 where name='Phil Ivey'; 1 affected row sql>select * from people; +------+-----------------------------+ | id | name | +======+=============+ | 1 | Michael Jordan | | 2 | Lionel Messi | | -1 | Phil Ivey | +------+-----------------------------+ sql>commit; auto commit mode: on sql>select * from people; +------+-----------------------------+ | id | name | +======+=============+ | -1 | Phil Ivey | | 1 | Michael Jordan | | 2 | Lionel Messi | +------+-----------------------------+ It is not what we would expect, isn't it? My MonetDB version is v5.18.3 Min

3 5

[MonetDB-users] crash when doing count(distinct col)
by Sam Mason 18 May '10

18 May '10

Hi, I've just pulled a largish dataset into monetdb (419851473 rows, 8 columns) and it's crashing when I try to do the following: select count(distinct a) from calldata; "a" is an integer column and monetdb gets the right answer of 886693 when I run either: select count(a) from (select a from calldata group by a) x; or select count(a) from (select distinct a from calldata) x; I'm running the latest 64bit .deb packages under ubuntu 10.4. The following is what I get back from mclient: sql>select count(distinct a) from calldata; MALException:!ERROR: HEAPalloc: Insufficient space for HEAP of 3358812160 bytes. ERROR: HEAPalloc: Insufficient space for HEAP of 3358812160 bytes. ERROR: HEAPalloc: Insufficient space for HEAP of 3358812160 bytes. Timer 93948.763 msec 1 rows I've not got much swap space, does it just happen to need it for the first version but not the other versions? -- Sam http://samason.me.uk/

2 2

[MonetDB-users] MonetDB performance
by dariuszs 18 May '10

18 May '10

Hi, So what happens (performance wise) when you have 50GB of memory, 300GB of data and on average 100 concurrent users using all the tables and columns in your database? Thanks. Dariusz.

7 13

[MonetDB-users] MonetDB/SQL: bulk-loading large data sets & adding key constraints
by Stefan Manegold 15 May '10

15 May '10

Dear all, derived from recent experiments and intensive testing, here are some recommendations for bulk-loading large data sets (in particular those larger than your available main memory) in MonetDB/SQL and adding constraints, i.e., primary and foreign keys. First of all, bulk loading should be done from (possibly gzip or bzip2 compressed) CSV files using the SQL `COPY INTO` command; cf., http://monetdb.cwi.nl/SQL/Documentation/Data-Manipulation.html In case your tables come with primary or foreign key constraints, it is recommended to omit these constrains when initially creating the tables. Then bulk-load (`COPY INTO`) your data. Once the data is loaded add the key constraints one by one using the SQL `ALTER TABLE ... ADD CONSTRAINT ...` command. Note, though, that the current Feb2010 release of MonetDB/SQL might experience some performance problems when adding constraints on data sets larger than main memory on multi-core machines. The reason it a non-optimal query translation and plan parallelization. For the time being, i.e., until we manage to fix this problem properly, it is recommended to resort to the optimizer pipeline of the Nov2009 release (for insiders, this effectively means disabling the "mitosis" optimizer), (only) while adding the key constraints. To do so, simply activate the Nov2009 optimizer pipeline via SET optimizer='nov2009_pipe'; just before adding the key constraints via `ALTER TABLE ... ADD CONSTRAINTS` and then restore the default optimizer pipeline via SET optimizer='default_pipe'; once you're done with adding constraints. For TPCH scale factor 100 (100 GB) on a 8 GB 4-core machine, this reduces the total time to add all key constraints from ~1.5h to ~1h. For SSB scale factor 100 (100 GB) on a 8 GB 4-core machine, this reduces the total time to add all key constraints from ~4h to ~1h. Hope this helps. Please don't hesitate to comment or ask questions. We'll let you know, one we managed to fix this problem properly. Kind regards, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[MonetDB-users] Help,how to create hash index on table
by 冯亮 15 May '10

15 May '10

Hello, Current I am using MonetDB server v5.18.3 , and as i see in Wiki , MonetDB support hash index ,I intended to create a hash index on a table. But how to do it , i have referred to sqlmanual downloaded from monetdb.cwi.nl, but it simply wrote following format: CREATE [UNIQUE] INDEX <index_name> ON <table_name> ( <column_name>, ...) ; Is that Btree index or hash index or others? Does MonetDB support multiple column hash index? I am looking forward to your reply. Thanks a lot. Andrew. Feng

4 3

[MonetDB-users] Help,how to create hash index on table
by fll 09 May '10

09 May '10

Hello, Current I am using MonetDB server v5.18.3 , and as i see in Wiki , MonetDB support hash index ,I intended to create a hash index on a table. But how to do it , i have referred to sqlmanual downloaded from monetdb.cwi.nl, but it simply wrote following format: CREATE [UNIQUE] INDEX <index_name> ON <table_name> ( <column_name>, ...) ; Is that Btree index or hash index or others? Does MonetDB support multiple column hash index? I am looking forward to your reply. Thanks a lot. Andrew. Feng

2 1

[MonetDB-users] Move of SourceForge bugtracker to MonetDB Bugzilla
by Fabian Groffen 07 May '10

07 May '10

This message is important for all that used to report bugs on our SourceForge tracker, or are planning to report bugs in the future. With the move of our CVS repository to a new Mercurial repository, we have also migrated the bugtracker from SourceForge to a Bugzilla instance. The old SourceForge trackers are no longer in use. Please DO NOT file any bugs there, as they will get lost. We have removed their presence as far as possible to avoid any confusion here. The new location to search, read and file new bugs is here: http://dev.monetdb.org/bugzilla/ (Please bear with us while we try to get the interface/skin well suited for our project.) All bugs that were previously on the SourceForge trackers have been migrated to the new Bugzilla instance. The bug numbers could not be retained, hence all have been renumbered. See [1] for dealing with this in Bugzilla. All users that have ever reported a bug, commented on a bug, or been an assignee for a bug on SourceForge have been imported in Bugzilla. This means that for all those users a <sf-nickname>@users.sourceforge.net account has been created. Exception to this are users who work at CWI, whose addresses have been mapped to their CWI address. If you want to use your migrated account, simply click the "Forgot Password" link on the Bugzilla main page. It will send an email to your (SourceForge) email address with a token to allow you to set a password. If you are a new user, you can simply sign up on our Bugzilla using the "Open a New Account" button. In both cases, make sure you look through the preferences, in particular the "Email Preferences". For migrated accounts these preferences are set to exclude sending any bug email, which may or may not be what you prefer. Feel free to ask for any help if problems arise. [1] Searching an old bug using its bugnumber: If you have an old SourceForge bug number and need to find tha associated bug in Bugzilla, open up an advanced search, and enter the old bug number in the URL field. Make sure that you select the appropriate bug resolutions (e.g. include RESOLVED) and search. Each migrated bug has an URL set to the original bug on SourceForge, hence is easily found this way.

3 2

[MonetDB-users] Parallel Processing
by Nafees Ur Rehman 04 May '10

04 May '10

Hi, Does monetdb has support for parallel processing? If some can guide me on that. Regards Nafees Ur Rehman

4 8

[MonetDB-users] trigger ambiguous issue
by Huabing 03 May '10

03 May '10

I have following trigger statement: CREATE TRIGGER delete_pos_cp AFTER DELETE ON cp referencing old row as o FOR EACH ROW WHEN o.new_id_ IS NOT NULL UPDATE cp SET old_id_ = o.old_id_ WHERE rec_id_ = o.rec_id_; Following error is issued: SELECT: identifier 'rec_id_' ambiguous The last "o.rec_id_" gives rise to this ambigious. Any suggestion on this.

2 1

Re: [MonetDB-users] sql query size
by dariuszs 27 Apr '10

27 Apr '10

Hi, This still does not work in the latest version - Feb2010+SP2 - on Windows XP 32bit, Windows Server 2003 64bit,Ubuntu 9.10 64 bit,Fedora 12 64bit. Dariusz. Martin Kersten wrote: > Thanks for the example, we will look into it. > > dariuszs wrote: >> Martin, >> 1) Table schema >> sql>\d zip_codes >> CREATE TABLE "sys"."zip_codes" ( >> "zip_code" varchar(5) >> ); >> sql> >> >> 2) From mclient >> sql>\< /tmp/test.sql >> MAPI = monetdb@localhost:50000 >> ACTION= read_line >> QUERY = select count(*) from zip_codes WHERE zip_code IN >> ('00501','00544','01001','01002','01003','01004','01005','01007','01008','01009','01010','01011','01012','01013','01014','01020','01021','01022','01026','01027','01028','01029','01030','01031','01032','01033','01034','01035','01036','01037','01038','01039','01040','01041','01050','01053','01054','01056','01057','01059','01060','01061','01062','01063','01066','01068','01069','01070','01071','01072','01073','01074','01075','01077','01079','01080','01081','01082 >> >> ERROR = Connection terminated >> sql> >> 3) From shell >> root@monetdb:/tmp# mclient -lsql < test.sql >> MAPI = monetdb@localhost:50000 >> ACTION= read_line >> QUERY = select count(*) from zip_codes WHERE zip_code IN >> ('00501','00544','01001','01002','01003','01004','01005','01007','01008','01009','01010','01011','01012','01013','01014','01020','01021','01022','01026','01027','01028','01029','01030','01031','01032','01033','01034','01035','01036','01037','01038','01039','01040','01041','01050','01053','01054','01056','01057','01059','01060','01061','01062','01063','01066','01068','01069','01070','01071','01072','01073','01074','01075','01077','01079','01080','01081','01082 >> >> ERROR = Connection terminated >> root@monetdb:/tmp# >> 4) SQL >> >> >> Dariusz >> >> Martin Kersten wrote: >>> Hi Dariusz, >>> >>> A bug or a system limitation. It should at least never crash. >>> >>> How did you submit the query? To mclient? or mserver? or through >>> a mapi libary call? >>> >>> I would have to look into the SQL parser. The underlying stream would >>> expand upon need. It is, however, an interesting query. What kind of >>> application generates it? >>> >>> regards, Martin >>> >>> ps. we should keep a profile of users to avoid asking platform >>> details. >>> >>> dariuszs wrote: >>>> Hi Martin, >>>> Then I guess it's a bug. Please try running a query over 44k, >>>> you'll get >>>> MAPI = monetdb@localhost:50000 >>>> ACTION= read_line. >>>> QUERY = select count(*) from plk100 WHERE c15 IN >>>> ('00501','00544','01001','01002','01003'.......... >>>> ERROR = Connection terminated >>>> Any query under 44k runs fine, so I'm assuming the limit is 64k >>>> (query + other packet data) >>>> >>>> Thanks. Dariusz. >>>> >>>> >>>> >>>> Martin Kersten wrote: >>>>> dariuszs wrote: >>>>> >>>>>> Hi, >>>>>> Is there any limit on how big the sql query can be (in bytes)? >>>>>> Thanks. Dariusz. >>>>>> >>>>>> >>>>> As far as i am aware no. >>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> >>>>>> Download Intel® Parallel Studio Eval >>>>>> Try the new software tools for yourself. Speed compiling, find bugs >>>>>> proactively, and fine-tune applications for parallel performance. >>>>>> See why Intel Parallel Studio got high marks during beta. >>>>>> http://p.sf.net/sfu/intel-sw-dev >>>>>> _______________________________________________ >>>>>> MonetDB-users mailing list >>>>>> MonetDB-users(a)lists.sourceforge.net >>>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> Download Intel® Parallel Studio Eval >>>>> Try the new software tools for yourself. Speed compiling, find bugs >>>>> proactively, and fine-tune applications for parallel performance. >>>>> See why Intel Parallel Studio got high marks during beta. >>>>> http://p.sf.net/sfu/intel-sw-dev >>>>> _______________________________________________ >>>>> MonetDB-users mailing list >>>>> MonetDB-users(a)lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >>> >>> >>> >> > > > >

1 0

[MonetDB-users] ODBC connection problem.
by Nozhup 27 Apr '10

27 Apr '10

I get the following error when trying to connect a MonetDB database with 'OpenOffice.org Database' through the ODBC driver. [unixODBC][MonetDB][ODBC Driver 1.0]Client unable to establish connection .ini-files used: odbc.ini: [MonetDB_User] Description = Default MonetDB Data Source Driver = MonetDB Host = localhost Port = 50000 Database = TestDB User = monetdb Password = monetdb Debug = odbcinst.ini [MonetDB] Description = MonetDB Driver Driver = Driver64 = /home/breplu/MonetDB/lib/libMonetODBC.so Setup = Setup64 = /home/breplu/MonetDB/lib/libMonetODBCs.so UsageCount = 1 CPTimeout = CPReuse = MonetDB version: Feb2010 release A ODBC connection between MySql and OpenOffice works correctly, so I don't think the problem lies with OpenOffice. Am I doing something wrong or is this a know issue with MonetDB? Thanks in advance. -- View this message in context: http://old.nabble.com/ODBC-connection-problem.-tp28364397p28364397.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 1

[MonetDB-users] SMELLIEST data types???
by Steve Schmechel 27 Apr '10

27 Apr '10

I'm guessing that this was meant to be "SMALLINT". From: http://monetdb.cwi.nl/SQL/Documentation/SQL-Features.html Feature ID Feature name S/N B011-B017 Embedded Language support. Core SQL:1999 says that at least one of Embedded Ada, Embedded C, Embedded Cobol, Embedded Fortran, Embedded MUMPS, Embedded Pascal or Embedded PL/I 1 should be supported. N E011 Numeric data types (FULL support) S E011-01 INTEGER and SMELLIEST data types ^^^^^^^^^ I know I read that page before, funny I never noticed that. (I have to admit, I did check the standard just to make sure it wasn't in there and I had just missed it all these years.) I have seen a few database tables that had some really stinky data types, but I'm glad I never came across one that contained the smelliest. :-) Steve

2 1

Re: [MonetDB-users] SQL Case Statement
by Steve Schmechel 27 Apr '10

27 Apr '10

--- On Mon, 4/26/10, Mark Bucciarelli <mkbucc(a)gmail.com> wrote: > From: Mark Bucciarelli <mkbucc(a)gmail.com> > $ cd src/monet_2009Nov/ > $ find . -name "*sql*.y" > ./sql/src/server/sql_parser.y > $ > > Well commented YACC source. > > I use that when I have a SQL question. This file is not in any of the tarballs, including Nov2009. Do you have to generate it? If so, that would be a horrible way to document an external API: 1) Download source code. 2) Run lexical analyzer on source code. 3) Try to understand code comments. 4) Retry your query in the client application. Is there another document somewhere? If the comments are very good, maybe that could be quickly refined to become the official documentation. Thanks, Steve

3 2

[MonetDB-users] SQL Case Statement
by Hering Cheng 26 Apr '10

26 Apr '10

Hi, Is there any plan to add support for CASE in SELECT clauses to MonetDB? Thanks. Hering

4 6

[MonetDB-users] how to grant privileges to creating tables?
by Min Lu 25 Apr '10

25 Apr '10

Dear MonetDB users, Is it all possible to grant the privilege of "creating tables" to some users in MonetDB? For example, something like "grant create table to user_name" or similar? Thanks, Min

2 2

[MonetDB-users] how to grant privileges to creating tables? (more specific version)
by Min Lu 25 Apr '10

25 Apr '10

Dear all, I just want to address my question in last message in a more specific way. Actually I am wondering if I can grant privileges to creating local/global temporary tables to a normal user other than monetdb. And what is the complete list of priveleges known to MonetDB? Thanks, Min

2 2

[MonetDB-users] Fedora 12 cannot install public key
by dariuszs 23 Apr '10

23 Apr '10

Hi, Just trying to install public key on another server [root@monetdb ~]# rpm --import http://monetdb.cwi.nl/downloads/MonetDB-GPG-KEY error: http://monetdb.cwi.nl/downloads/MonetDB-GPG-KEY: import failed. [root@monetdb ~]#

1 1

[MonetDB-users] mal index
by Miguel Ángel Águila Lorente 23 Apr '10

23 Apr '10

Hello, I'm working in a research problem and we need to create a graph database. We think that MonetDB can be useful to represent this graph since the use of bats may make the graph traversal efficient. Nevertheless, SQL is limited for the type of operations that we need to do, and that is why we have decided to resort to MAL. Our database is like this: - Nodes: a) node ident b) name - Edges: a) edge ident b) head node ident c) tail node ident One of the operations is to find the node with maximum out degree and, after that, to do a BFS using this node as the source. I have stored all the information in bats: batNode1=[oid,node ident] batNode2=[oid,name] batEdge=[oid, edge ident] batEdgeHead=[oid, head node ident] batEdgeTail=[oid,tail node ident] For getting the node with the maximum out degree, I access batEdgeHead and store the number of occurrences of every node id in a batAux. batAux=[node ident, times] After that I only select the node id with the largest number of occurrences. Now, we are ready to do the BFS. In every iteration, I have to save all the tail nodes for exploring them in the next generation. This is the pseudocode explanation: Put the node id to NodesToVisit While there are NodesToVisit currentNode=NodesToVisit; edgeToVisit=get edges idents from batEdgeHead from currentNode ident While there are edgesToVisit currentEdge=edgesToVisit nodeFinal=get tail node ident from batEdgeTail from currentEdge ident If notExist nodeFinal in NodesToVisit put nodeFinal in NodesToVisit endIf endWhile endWhile The main problem happens when I perform a select from batEdgeHead from the ident node. batEdgeHead might become very large and accessing it might be very expensive in terms of time. Is there any kind of indexing? I tried with crackers and AVL index but, if I am not wrong, this is only useful if I had to do the same selection more than once. The number of the nodes is 325.000 and the number of edges 925.000. At the end of this e-mail you will find the code. May any one help in improving the performance of this code? Best regards, Miquel Àngel Àguila Bat equivalence: batEdge, batNode1 --> b1:= bat.new(:oid,:lng); batNode2 --> b2:= bat.new(:oid,:str); batEdgeHead --> rh:= bat.new(:oid,:lng); batEdgeTail --> rt:= bat.new(:oid,:lng); batType --> b4:= bat.new(:oid, int) (Type: 1. nodeType1, 2. nodeType2 , 3. edgeType1, 4. edgeType2, 5. edgeType3) Loaded the bats, here is the code: Maxium out degree:----------------------------------------------------- edart:=algebra.select(b4,3); nodOut:=algebra.semijoin(rh,edart); res:=bat.new(:lng,:int); valor:=0; barrier (mloop16:lng ,h16:oid ,tl16:lng ) := bat.newIterator(nodOut); bite1:=algebra.exist(res,tl16); barrier ifpart:= bite1; valor:=algebra.find(res, tl16); valor:=calc.+(valor,1); res:=bat.replace(res,tl16,valor); exit ifpart; barrier elsepart:= calc.!=(bite1,true); bat.insert(res,tl16,1); exit elsepart; redo (mloop16:lng ,h16:oid ,tl16:lng ) := bat.hasMoreElements(nodOut); exit mloop16:lng ; maxOutDegree:=aggr.max(res); node:=algebra.select(res,maxOutDegree); io.print(node); nodart:=algebra.select(b4,1); nodart2:=algebra.semijoin(b1,nodart); ident2:=algebra.join(nodart2,node); name2:=algebra.semijoin(b3,ident2); secondtime:=alarm.time(); #Traversals------------------------------------------------------------ node2:=algebra.semijoin(b1,name2); aux:=0:lng; idoid:=0@0; barrier (mloop17:lng ,h17:oid ,tl17:lng) := bat.newIterator(node2); aux:=tl17; idoid:=h17; exit mloop17:lng ; nod:=bat.new(:lng,:lng); dif:=bat.new(:lng,:lng); vis:=bat.new(:lng,:lng); ar:=bat.new(:oid,:oid); tr:=bat.new(:oid,:oid); bat.insert(nod,aux,aux); dif:=algebra.difference(nod,vis); outer6:= aggr.count(dif); rhtaux:=bat.new(:oid,:int); rhaux:=bat.new(:oid,:lng); rtaux:=bat.new(:oid,:lng); rhtaux:=algebra.select(b4,3); rtaux:=algebra.semijoin(rt,rhtaux); rhaux:=algebra.semijoin(rh,rhtaux); barrier outer:=true; barrier (mloop18:lng ,h18:lng ,tl18:lng ) := bat.newIterator(dif); # This is the problem, I need to do a lot of selects rhaux2:=algebra.select(rhaux,tl18); rtaux2:=algebra.semijoin(rtaux,rhaux2); barrier (mloop19:lng ,h19:oid,tl19:lng ) := bat.newIterator(rtaux2); bite1:=algebra.exist(nod,tl19); barrier ifpart:= bite1; bat.insert(tr,h19,h19); exit ifpart; barrier elsepart:= calc.!=(bite1,true); bat.insert(ar,h19,h19); bat.insert(nod,tl19,tl19); exit elsepart; redo (mloop19:lng ,h19:oid ,tl19:lng ) := bat.hasMoreElements(rtaux2); exit mloop19:lng ; bat.insert(vis,h18,tl18); redo (mloop18:lng ,h18:lng ,tl18:lng ) := bat.hasMoreElements(dif); exit mloop18:lng ; dif:=algebra.difference(nod,vis); outer6:= aggr.count(dif); outer:=calc.!=(outer6,0); redo outer:=calc.!=(outer6,0); exit outer;

4 4

Re: [MonetDB-users] Core Dump upon COPY on SPARC
by Hering Cheng 23 Apr '10

23 Apr '10

Using Feb2010 release resulted in a different error. This time, mserver5 did not crash: MAPI = chenher@rdcuxsrv220:50000 QUERY = COPY 34274958 RECORDS INTO app.t_compute_all_closeprice FROM '/home/chenher/gaal/tmp/tar/t_compute_all_closeprice.txt' DELIMITERS '|', '\n'; ERROR = !MALException:!ERROR: HEAPextend: failed to extend to 274199664 for 17/1745tail Here is the configuration from log file: 2010-04-21 13:11:51 MSG tar[537]: # MonetDB server v5.18.1, based on kernel v1.36.1 2010-04-21 13:11:51 MSG tar[537]: # Serving database 'tar', using 16 threads 2010-04-21 13:11:51 MSG tar[537]: # Compiled for sparc-sun-solaris2.10/64bit with 64bit OIDs dynamically linked 2010-04-21 13:11:51 MSG tar[537]: # Found 32.000 GiB available main-memory. 2010-04-21 13:11:51 MSG tar[537]: # Copyright (c) 1993-July 2008 CWI. 2010-04-21 13:11:51 MSG tar[537]: # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved 2010-04-21 13:11:51 MSG tar[537]: # Visit http://monetdb.cwi.nl/ for further information 2010-04-21 13:11:51 MSG tar[537]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50001/ 2010-04-21 13:11:52 MSG tar[537]: # MonetDB/SQL module v2.36.1 loaded Any suggestion? On Wed, Apr 21, 2010 at 1:16 PM, Hering Cheng <hering.cheng(a)computer.org>wrote: > The statement I gave is the only statement that I passed to mclient, using > --statement argument. > > > On Wed, Apr 21, 2010 at 11:32 AM, Niels Nes <Niels.Nes(a)cwi.nl> wrote: > >> On Wed, Apr 21, 2010 at 11:08:46AM -0700, Hering Cheng wrote: >> > Here is the single COPY statement I was using: >> > >> > COPY 34274958 RECORDS INTO app.t_compute_all_closeprice FROM '/home/ >> > chenher/gaal/tmp/tar/t_compute_all_closeprice.txt' DELIMITERS '|', '\ >> > n'; >> >> I was wondering because the crash you have seems to indicate its >> in the commit, ie your load finished but then the commit (auto or not) >> crashes. Probably because of some out of memory problem. >> >> niels >> > >> > On Wed, Apr 21, 2010 at 11:02 AM, Niels Nes <Niels.Nes(a)cwi.nl> wrote: >> > >> > On Wed, Apr 21, 2010 at 10:48:48AM -0700, Hering Cheng wrote: >> > > Hi, >> > > >> > > I am trying to load 77 GB of data (or 34,274,958 records) into >> > MonetDB, >> > > using the Feb2010 SP1 source code compiled on Solaris SPARC with >> > Sun >> > > Studio 12.1 compiler with debugging enabled, and the database >> > crashes 5 >> > > hours into the loading. Here is some information about my >> > > configuration: >> > > >> > > I start up with Merovingian like this: >> > > >> > > $ ( ulimit -d $[32*1024*1024]; export LD_PRELOAD_64=/usr/lib/64/ >> > > libumem.so:${LD_PRELOAD_64}; export LD_PRELOAD=/usr/lib/ >> > libumem.so:$ >> > > {LD_PRELOAD}; export MONETDB5CONF=/GAAL/chenher/ >> > rdcuxsrv220-local-disk/ >> > > chenher/monetdb/feb2010sp1/etc/monetdb5.conf; /GAAL/chenher/share >> > / >> > > monetdb/distro-sparc-feb2010-sp1-64bit/bin/merovingian; ) >> > > >> > > Here are the relevent log sections: >> > > >> > > 2010-04-16 13:25:28 MSG merovingian[4302]: starting database >> > 'tar', up >> > > min/avg/max: 4m/4m/4m, crash average: 0.00 0.00 0.00 (1-1=0) >> > > 2010-04-16 13:25:29 MSG tar[15594]: arguments: /GAAL/chenher/ >> > share/ >> > > monetdb/distro-sparc-feb2010-sp1-64bit/bin/mserver5 --config=/ >> > GAAL/ >> > > chenher/rdcuxsrv220-local-disk/chenher/monetdb/feb2010sp1/etc/ >> > > monetdb5.conf --dbname=tar --dbinit=include sql; --set >> > merovingian_uri= >> > > mapi:monetdb://rdcuxsrv220:50000/tar --set monet_daemon=yes --set >> > > mapi_open=false --set mapi_autosense=true --set mapi_port=50001 >> > --set >> > > monet_vault_key=/GAAL/chenher/rdcuxsrv220-local-disk/chenher/ >> > monetdb/ >> > > feb2010sp1/var/MonetDB5/dbfarm/tar/.vaultkey --set sql_optimizer= >> > > default_pipe >> > > 2010-04-16 13:25:34 MSG tar[15594]: # MonetDB server v5.18.3, >> > based on >> > > kernel v1.36.3 >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Serving database 'tar', >> > using 16 >> > > threads >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Compiled for >> > > sparc-sun-solaris2.10/64bit with 64bit OIDs dynamically linked >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Found 32.000 GiB available >> > > main-memory. >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Copyright (c) 1993-July >> > 2008 CWI. >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Copyright (c) August >> > 2008-2010 >> > > MonetDB B.V., all rights reserved >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Visit http://monetdb.cwi.nl >> > / for >> > > further information >> > > 2010-04-16 13:25:34 MSG tar[15594]: # Listening for connection >> > requests >> > > on mapi:monetdb://127.0.0.1:50001/ >> > > 2010-04-16 13:25:35 MSG tar[15594]: # MonetDB/SQL module v2.36.3 >> > loaded >> > > 2010-04-16 13:25:35 MSG control[4302]: rdcuxsrv220:58146: started >> > > database 'tar' >> > > ... >> > > 2010-04-21 10:23:37 MSG merovingian[4302]: database 'tar' (15594) >> > has >> > > crashed (dumped core) >> > > >> > > Using Sun's DBX tool to examine the crash: >> > > >> > > ... >> > > t@212 (l@212) terminated by signal SEGV (no mapping at the fault >> > > address) >> > > Current function is temp_create >> > > 87 temp_dup(b->batCacheid); >> > > (dbx) where >> > > current thread: t@212 >> > > =>[1] temp_create(b = (nil)), line 87 in "bat_utils.c" >> > > [2] ebat2real(b = 5714, ibase = 0), line 162 in "bat_utils.c" >> > > [3] delta_append_bat(bat = 0x104923c98, i = 0x10143ec98), line >> > 234 in >> > > "bat_storage.c" >> > > [4] append_col(tr = 0x104fe1e98, c = 0x1045c7af8, i = >> > 0x10143ec98, >> > > tpe = 5), line 267 in "bat_storage.c" >> > > [5] mvc_append_wrap(cntxt = 0xffffffff7f352c00, mb = >> > 0x1050d5d98, stk >> > > = 0x101816018, pci = 0x1056a7c18), line 1120 in "sql.c" >> > > [6] runMALsequence(cntxt = 0xffffffff7f352c00, mb = >> > 0x1050d5d98, >> > > startpc = 1, stoppc = 0, stk = 0x101816018, env = (nil), >> > pcicaller = >> > > (nil)), line 2908 in "mal_interpreter.c" >> > > [7] callMAL(cntxt = 0xffffffff7f352c00, mb = 0x1050d5d98, env = >> > > 0xfffffffddabff908, argv = 0xfffffffddabff968, debug = '\0'), >> > line 402 >> > > in "mal_interpreter.c" >> > > [8] SQLexecutePrepared(c = 0xffffffff7f352c00, be = >> > 0x1050f0ec8, q = >> > > 0x104fe1b18), line 1196 in "sql_scenario.c" >> > > [9] SQLengineIntern(c = 0xffffffff7f352c00, be = 0x1050f0ec8), >> > line >> > > 1249 in "sql_scenario.c" >> > > [10] SQLengine(c = 0xffffffff7f352c00), line 1349 in >> > "sql_scenario.c" >> > > [11] runPhase(c = 0xffffffff7f352c00, phase = 4), line 363 in >> > > "mal_scenario.c" >> > > [12] runScenarioBody(c = 0xffffffff7f352c00), line 412 in >> > > "mal_scenario.c" >> > > [13] runScenario(c = 0xffffffff7f352c00), line 438 in >> > > "mal_scenario.c" >> > > [14] MSserveClient(dummy = 0xffffffff7f352c00), line 368 in >> > > "mal_session.c" >> > > (dbx) threads >> > > t@1 a l@1 ?() LWP suspended in __pollsys() >> > > t@2 a l@2 SERVERlistenThread() LWP suspended in >> > __pollsys >> > > () >> > > t@3 a l@3 mvc_logmanager() LWP suspended in >> > __pollsys() >> > > t@4 b l@4 umem_update_thread() sleep on (unknown) in >> > > __lwp_park() >> > > t@13 a l@13 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@14 a l@14 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@15 a l@15 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@16 a l@16 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@17 a l@17 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@18 a l@18 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@19 a l@19 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@20 a l@20 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@21 a l@21 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@22 a l@22 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@23 a l@23 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@24 a l@24 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@25 a l@25 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@26 a l@26 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@27 a l@27 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > t@28 a l@28 runDFLOWworker() sleep on 0x101ea9a48 in >> > > __lwp_park() >> > > o> t@212 a l@212 ?() signal SIGSEGV in temp_create() >> > > >> > > With Feb2010 release, I managed to load a much larger database >> > without >> > > a crash. I will try loading this smaller data set again with the >> > > Feb2010 release and report what happens. In the meantime, can >> > you give >> > > me some suggestions? >> > > >> > > Thanks. >> > > Hering >> > > >> > >> > Are you loading with a single COPY into or with several insert >> > into? >> > >> > Niels >> > >> > > >> > >> ------------------------------------------------------------------------------ >> > >> > > _______________________________________________ >> > > MonetDB-users mailing list >> > > MonetDB-users(a)lists.sourceforge.net >> > > https://lists.sourceforge.net/lists/listinfo/monetdb-users >> > >> > >> > -- >> > Niels Nes, Centrum Wiskunde & Informatica (CWI) >> > Science Park 123, 1098 XG Amsterdam, The Netherlands >> > room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl<sip%3A4098(a)sip.cwi.nl> >> > url: http://www.cwi.nl/~niels <http://www.cwi.nl/%7Eniels> >> e-mail: Niels.Nes(a)cwi.nl >> > >> > >> >> -- >> Niels Nes, Centrum Wiskunde & Informatica (CWI) >> Science Park 123, 1098 XG Amsterdam, The Netherlands >> room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl<sip%3A4098(a)sip.cwi.nl> >> url: http://www.cwi.nl/~niels <http://www.cwi.nl/%7Eniels> e-mail: >> Niels.Nes(a)cwi.nl >> > >

2 3

[MonetDB-users] copy into file
by dariuszs 22 Apr '10

22 Apr '10

Hi, It looks like the same problem exists no matter what the OS (Fedora 12 or Ubuntu 9.10 or Windows) and this worked in Nov2009-SP1 release. When you try: sql>copy select * from table1 limit 1000 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from table1 limit 1000 into '/tmp/output.txt'; ERROR = !Connection terminated Timer 1105.876 msec You have mail in /var/spool/mail/root [root@monetdb ~]# [root@monetdb /]# mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0-beta4 10 Nov 2009 (compiled with OpenSSL 1.0.0-fips-beta4 10 Nov 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 Please help. Thanks. Dariusz.

4 27

[MonetDB-users] SQL constraints and performance
by Isidor Zeuner 22 Apr '10

22 Apr '10

Dear MonetDB developers, from my own experiences regarding database engine design, I know that SQL constraints can be used to increase performance because the query planner can use them to reason about certain regularities in the data. On the other hand, they come with a cost as the engine is supposed to enforce them. Are there any common rules on how constraints and performance correlate in MonetDB, which could be used to adapt a schema to work especially fast with MonetDB? Best regards, Isidor

1 0

[MonetDB-users] Core Dump upon COPY on SPARC
by Hering Cheng 21 Apr '10

21 Apr '10

Hi, I am trying to load 77 GB of data (or 34,274,958 records) into MonetDB, using the Feb2010 SP1 source code compiled on Solaris SPARC with Sun Studio 12.1 compiler with debugging enabled, and the database crashes 5 hours into the loading. Here is some information about my configuration: I start up with Merovingian like this: $ ( ulimit -d $[32*1024*1024]; export LD_PRELOAD_64=/usr/lib/64/libumem.so:${LD_PRELOAD_64}; export LD_PRELOAD=/usr/lib/libumem.so:${LD_PRELOAD}; export MONETDB5CONF=/GAAL/chenher/rdcuxsrv220-local-disk/chenher/monetdb/feb2010sp1/etc/monetdb5.conf; /GAAL/chenher/share/monetdb/distro-sparc-feb2010-sp1-64bit/bin/merovingian; ) Here are the relevent log sections: 2010-04-16 13:25:28 MSG merovingian[4302]: starting database 'tar', up min/avg/max: 4m/4m/4m, crash average: 0.00 0.00 0.00 (1-1=0) 2010-04-16 13:25:29 MSG tar[15594]: arguments: /GAAL/chenher/share/monetdb/distro-sparc-feb2010-sp1-64bit/bin/mserver5 --config=/GAAL/chenher/rdcuxsrv220-local-disk/chenher/monetdb/feb2010sp1/etc/monetdb5.conf --dbname=tar --dbinit=include sql; --set merovingian_uri=mapi:monetdb://rdcuxsrv220:50000/tar --set monet_daemon=yes --set mapi_open=false --set mapi_autosense=true --set mapi_port=50001 --set monet_vault_key=/GAAL/chenher/rdcuxsrv220-local-disk/chenher/monetdb/feb2010sp1/var/MonetDB5/dbfarm/tar/.vaultkey --set sql_optimizer=default_pipe 2010-04-16 13:25:34 MSG tar[15594]: # MonetDB server v5.18.3, based on kernel v1.36.3 2010-04-16 13:25:34 MSG tar[15594]: # Serving database 'tar', using 16 threads 2010-04-16 13:25:34 MSG tar[15594]: # Compiled for sparc-sun-solaris2.10/64bit with 64bit OIDs dynamically linked 2010-04-16 13:25:34 MSG tar[15594]: # Found 32.000 GiB available main-memory. 2010-04-16 13:25:34 MSG tar[15594]: # Copyright (c) 1993-July 2008 CWI. 2010-04-16 13:25:34 MSG tar[15594]: # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved 2010-04-16 13:25:34 MSG tar[15594]: # Visit http://monetdb.cwi.nl/ for further information 2010-04-16 13:25:34 MSG tar[15594]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50001/ 2010-04-16 13:25:35 MSG tar[15594]: # MonetDB/SQL module v2.36.3 loaded 2010-04-16 13:25:35 MSG control[4302]: rdcuxsrv220:58146: started database 'tar' ... 2010-04-21 10:23:37 MSG merovingian[4302]: database 'tar' (15594) has crashed (dumped core) Using Sun's DBX tool to examine the crash: ... t@212 (l@212) terminated by signal SEGV (no mapping at the fault address) Current function is temp_create 87 temp_dup(b->batCacheid); (dbx) where current thread: t@212 =>[1] temp_create(b = (nil)), line 87 in "bat_utils.c" [2] ebat2real(b = 5714, ibase = 0), line 162 in "bat_utils.c" [3] delta_append_bat(bat = 0x104923c98, i = 0x10143ec98), line 234 in "bat_storage.c" [4] append_col(tr = 0x104fe1e98, c = 0x1045c7af8, i = 0x10143ec98, tpe = 5), line 267 in "bat_storage.c" [5] mvc_append_wrap(cntxt = 0xffffffff7f352c00, mb = 0x1050d5d98, stk = 0x101816018, pci = 0x1056a7c18), line 1120 in "sql.c" [6] runMALsequence(cntxt = 0xffffffff7f352c00, mb = 0x1050d5d98, startpc = 1, stoppc = 0, stk = 0x101816018, env = (nil), pcicaller = (nil)), line 2908 in "mal_interpreter.c" [7] callMAL(cntxt = 0xffffffff7f352c00, mb = 0x1050d5d98, env = 0xfffffffddabff908, argv = 0xfffffffddabff968, debug = '\0'), line 402 in "mal_interpreter.c" [8] SQLexecutePrepared(c = 0xffffffff7f352c00, be = 0x1050f0ec8, q = 0x104fe1b18), line 1196 in "sql_scenario.c" [9] SQLengineIntern(c = 0xffffffff7f352c00, be = 0x1050f0ec8), line 1249 in "sql_scenario.c" [10] SQLengine(c = 0xffffffff7f352c00), line 1349 in "sql_scenario.c" [11] runPhase(c = 0xffffffff7f352c00, phase = 4), line 363 in "mal_scenario.c" [12] runScenarioBody(c = 0xffffffff7f352c00), line 412 in "mal_scenario.c" [13] runScenario(c = 0xffffffff7f352c00), line 438 in "mal_scenario.c" [14] MSserveClient(dummy = 0xffffffff7f352c00), line 368 in "mal_session.c" (dbx) threads t@1 a l@1 ?() LWP suspended in __pollsys() t@2 a l@2 SERVERlistenThread() LWP suspended in __pollsys() t@3 a l@3 mvc_logmanager() LWP suspended in __pollsys() t@4 b l@4 umem_update_thread() sleep on (unknown) in __lwp_park() t@13 a l@13 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@14 a l@14 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@15 a l@15 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@16 a l@16 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@17 a l@17 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@18 a l@18 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@19 a l@19 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@20 a l@20 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@21 a l@21 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@22 a l@22 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@23 a l@23 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@24 a l@24 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@25 a l@25 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@26 a l@26 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@27 a l@27 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() t@28 a l@28 runDFLOWworker() sleep on 0x101ea9a48 in __lwp_park() o> t@212 a l@212 ?() signal SIGSEGV in temp_create() With Feb2010 release, I managed to load a much larger database without a crash. I will try loading this smaller data set again with the Feb2010 release and report what happens. In the meantime, can you give me some suggestions? Thanks. Hering

2 1

[MonetDB-users] Feb2010-SP1/Fedora 12/Ubuntu 9.10/Windows - creating output problem
by dariuszs 21 Apr '10

21 Apr '10

Hi, It looks like the same problem exists no matter what the OS (Fedora 12 or Ubuntu 9.10 or Windows) and this worked in Nov2009-SP1 release. When you try: 'copy select * from table1 limit 1000 into '/tmp/output.txt' You get: sql>copy select * from plk100 limit 1000 into '/tmp/output.txt'; MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = copy select * from plk100 limit 1000 into '/tmp/output.txt'; ERROR = !Connection terminated Timer 1105.876 msec You have mail in /var/spool/mail/root [root@monetdb ~]# [root@monetdb /]# mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0-beta4 10 Nov 2009 (compiled with OpenSSL 1.0.0-fips-beta4 10 Nov 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -O2 -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 Please help. Thanks. Dariusz.

1 0

[MonetDB-users] monetdb/merovingian
by dariuszs 20 Apr '10

20 Apr '10

Hi, I've got this messages after rebooting the server and trying to start merovingian again: unable to open '/var/run/MonetDB/merovingian.pid' for writing: No such file or directory Software: Ubuntu 9.10 MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Please help. Dariusz.

5 12

[MonetDB-users] Python Error
by Matthew Jones 20 Apr '10

20 Apr '10

Hey folks, I'm seeing an error in the Python API. When connecting to a remote Monet instance with the python client and setting the cursor's arraysize to something significant (100, as an example) then I get the following error: Traceback (most recent call last): File "afile.py", line 9, in <module> cur.execute("SELECT max(val), otherval" File "build/bdist.linux-x86_64/egg/monetdb/sql/cursors.py", line 203, in execute File "build/bdist.linux-x86_64/egg/monetdb/sql/cursors.py", line 445, in __store_result File "build/bdist.linux-x86_64/egg/monetdb/sql/cursors.py", line 529, in __parse_tuple File "build/bdist.linux-x86_64/egg/monetdb/sql/cursors.py", line 569, in __exception_handler monetdb.monetdb_exceptions.InterfaceError: length of row doesn't match header This doesn't happen when the client is used locally on the machine running the database. I will open a bug about this, also. It prevents the .arraysize cursor property from being used remotely. It's default value is 1, which causes the server to take a ridiculously long time to send back large data sets. 73, Matthew W. Jones (KI4ZIB) http://matburt.net

2 1

[MonetDB-users] Foreign key validation issue with "COPY INTO..." ?
by Matthieu Guamis 19 Apr '10

19 Apr '10

Hello, "COPY INTO" command fails with "FOREIGN KEY constraint 'xxx' violated" whereas "INSERT INTO" succeeded. Simple 3 tables example (it works with 2 tables, one referencing the other): CREATE TABLE a ( a1 VARCHAR(3) NOT NULL, CONSTRAINT pka PRIMARY KEY (a1) ); CREATE TABLE b ( b1 VARCHAR(3) NOT NULL, b2 VARCHAR(3) NOT NULL, CONSTRAINT pkb PRIMARY KEY (b1,b2), CONSTRAINT fk1 FOREIGN KEY (b2) REFERENCES a (a1) ); CREATE TABLE c ( c1 varchar(3) NOT NULL, c2 varchar(3) NOT NULL, c3 varchar(3), CONSTRAINT pkc PRIMARY KEY (c1,c2), CONSTRAINT fk2 FOREIGN KEY (c2) REFERENCES a (a1), CONSTRAINT fk3 FOREIGN KEY (c2,c3) REFERENCES b (b1,b2) ); Using INSERT INTO: sql>insert into a values ('aaa'); 1 affected row sql>insert into c values ('cc1','aaa',null); 1 affected row Using COPY INTO with one line input file "test.txt" containing : cc2|aaa| sql>COPY INTO c FROM '/home/mat/test.txt' USING DELIMITERS '|',' more>' NULL as ''; SQLException:assert:INSERT INTO: FOREIGN KEY constraint 'c.fk3' violated Bug or misunderstanding? Thanks a lot, Matthieu PS: mserver5 --version MonetDB server v5.18.3 (64-bit), based on kernel v1.36.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 2.0GiB available memory, 2 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

4 4

[MonetDB-users] XRPC queries do not behave like Mclient queries
by Ryad Ben-El-Kezadri 16 Apr '10

16 Apr '10

Hello, 1°)I am querying Monetdb through the XRPC java API. I have implemented a xquery in a function in a XRPC module and I call it from java. I noticed that my XRPC module is not loaded when the xquery compares an attribute to an integer value (for instance $n[@attr=1]) In this case, I get the following error message embedded in the XRPC reply: Error occurred during execution fatal error: function #pf:number is not supported. xquery_module_load: could not load module. Comparing an attribute to a string works fine however (ex $n[@attr="1"]) 2°)What is strange is that the same function (fct) works perfect when I call it from the Mclient (instead of java). Example import module namespace f="xrpc-extract-function" at " http://127.0.0.1:50001/export/extract.xq"; f:fct("file.xml", 0, 1000000) 3°)I don't know if this problem is general or specifically related to my configuration. Indeed I had to load the server modules myself (else the server crashes) with the following command in the bat server script: "%MONETDB%\bin\Mserver.exe" --dbinit="module(pf_support);module(logger);module(xrpc_client);module(xrpc_server);module(pf_standoff);module(mkey);module(pathfinder);module(mapi);" --set "prefix=%MONETDB%" --set "exec_prefix=%MONETDB%" %MONETDBFARM% %* Your help will be really helpful, because right now I can only do string comparisons on attributes (which slows the program a lot,....). My guess is that, when invoked from java, Monetdb looks (at load time) for pf:number instead of fn:number to convert the string value of the attribute, but that this function does not exist Warm regards Ryad ////////////////////////////////////////// Here is my java call, I guess it is exaclty the same signature that with the Mclient (so there is no problem with it) String call = XRPCMessageMine.XRPC_SEQ(XRPCMessageMine.XRPC_ATOM("string", file)) + XRPCMessageMine.XRPC_SEQ(XRPCMessageMine.XRPC_ATOM("double", Double.toString(timeInterval[0]))) + XRPCMessageMine.XRPC_SEQ(XRPCMessageMine.XRPC_ATOM("double", Double.toString(timeInterval[1])));

1 0

[MonetDB-users] monetdb sql - same query different results
by dariuszs 16 Apr '10

16 Apr '10

Hi, Same query produces different results: sql>select count(*) from plk100 WHERE 0=0 and c15 in (select zip from zip_list) and ( c102 in ('1','0')) and ( c28 in ('1','0')) and ( c66 in ('01')); +-------+ | L73 | +=======+ | 14945 | +-------+ 1 tuple Timer 69.446 msec 1 rows sql>select count(*) from plk100 WHERE 0=0 and c15 in (select zip from zip_list) and ( c102 in ('1','0')) and ( c28 in ('1','0')) and ( c66 in ('01')); +-------+ | L73 | +=======+ | 57257 | +-------+ 1 tuple Timer 146.666 msec 1 rows sql> Please help.

2 8

[MonetDB-users] monetdb python API, howto install
by Martin Körner 16 Apr '10

16 Apr '10

Hello Guys, i have installed the MonetDB 5 SQL client-server package from http://monetdb.cwi.nl/downloads/Windows/Feb2010-SP1/ The downloaded file was MonetDB5-SQL-Installer-i386-20100322.msi. So i use a Windows XP 32bit system. My installed Python version is now 2.6. Now i want to access MonetDB with Python. But i found no way to install the MonetDB Python API. I tried it like it is shown on http://monetdb.cwi.nl/SQL/Documentation/The-Python-Library.html But there is no file 'setup.py' to install. Is there any other way to install the API? Where i can download an install package for the API? With best greetings from Germany, Martin -- heidemann added insight GmbH In der Weidbach 2 63477 Maintal Telefon: +49 6181 9080739 Internet: www.added-insight.com USt-Id DE 2130 86 540 Registernummer: HRB 91533 (Amtsgericht Hanau) Geschäftsführer: Dr. Jeffrey Heidemann ********************************************************************* Diese E-Mail kann vertrauliche und/oder rechtlich geschuetzte Informationen enthalten. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. **********************************************************************

1 1

[MonetDB-users] MonetDB/XQuey - how to ignore DTDs?
by Roberto Cornacchia 15 Apr '10

15 Apr '10

Hi all, I wonder is there is an option to make MonetDB/XQuery shredder ignore DTD files. What triggered this question is the need to shred about 20M files, each of them including *the same* online DTD. What happens is that, for each xml file, the DTD is downloaded and shredded/used(?). It is not even cached (I tried using xquery_cacherules, but apparently it has no effect on DTDs). This makes shredding speed drop down to more than 3 seconds per document, which is not an option of course, as it would take me roughly 700 days to shred the entire collection :) Removing DTD link from the original xml files is not an option either, the collection is simply too big to be replicated. How can the DTD be ignored? Roberto -- | M.Sc. Roberto Cornacchia | CWI (Centrum voor Wiskunde en Informatica) | Science Park 123, 1098XG Amsterdam, The Netherlands | tel: +31 20 592 4322 , http://www.cwi.nl/~roberto

4 4

[MonetDB-users] grouping sets
by Nafees Ur Rehman 15 Apr '10

15 Apr '10

Hi, Whats is the syntax of grouping sets in monetdb? I am trying Select has_nurs,form,health,class,count(*) from nursery group by GROUPING SETS ((has_nurs,class),(form,class),(health,class)) but it does not work Regards Nafees Ur Rehman

3 3

Re: [MonetDB-users] [Monetdb-developers] MonetDB memory management
by Stefan de Konink 15 Apr '10

15 Apr '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 15-04-10 11:00, Xiao Lei schreef: > Then how can i release the memory? By not posting to me directly but reply to the mailinglist ;) It will not directly solve your problem, but keep other people in the loop. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkvG238ACgkQYH1+F2Rqwn1xdwCeOxnqWgALhsk4ue2vEv/D/ZsM IC0AnRyRAUabgy7Z2u2UZHGOXl4EJLYS =fsie -----END PGP SIGNATURE-----

2 1