- users-list - monetdb.org

[MonetDB-users] Restore db on different platform / OID size
by Sander Kruseman 15 Apr '11

15 Apr '11

When attempting to restore a db dump, made on Windows 7 with 32 bits sized object ID's, on OSX with 64 bits sized object ID's, I run into errors. Like: CREATE FUNCTION: name 'env' already in use If I tweak the dump script by first dropping the functions that appear to exist, I get stuff like: ' from line 1 field 5 not inserted, expecting type real failed to import table Should what I try to do be possible anyway? Am I doing anything wrong? Thanks for your feedback. Sander Kruseman

2 1

[MonetDB-users] bulk upload data
by Nic Chidu 14 Apr '11

14 Apr '11

What way do you suggest for bulk uploading data? Thanks. Nic

3 2

[MonetDB-users] documentation on metadata statement
by Nic Chidu 13 Apr '11

13 Apr '11

Where can I find doc about metadata statements. Stuff like showing database, table, running environment. Thanks, Nic

2 1

[MonetDB-users] Series UPDATE statements will significantly decrease server performance
by Vladimir Vlach 13 Apr '11

13 Apr '11

Hi, We are experiencing performance issue with MonetDB when running UPDATE. We have a table with 1M rows and 3.000 columns. All columns except primary_id are FLOAT type. When I execute series of these statements the performance on first 30-35 statements is quite acceptable (less then 1s) but then suddenly it gets really slow. One UPDATE takes 30 seconds or more. After next 5-10 statements the performance decreases and one UPDATE takes 1 minute or longer. We run this with AutoCommit. The behavior is very persistent. UPDATE table SET column=column+1 WHERE primary_id='e1a7d7c7204c292e56e942dbdad118d3' ; Would you say MonetDB is not suitable for bulk UPDATE operations? Thanks, Vlad

1 0

[MonetDB-users] UPDATE INNER JOIN syntax
by Osmin Castillo 12 Apr '11

12 Apr '11

Hi guys: How can I to update a table field based on a inner join update? What's the syntax for MonetDB? thanks in advance Osmin Castillo

2 1

[MonetDB-users] How to recover Database after crash
by Vladimir Vlach 11 Apr '11

11 Apr '11

Hi, is there guide on how to recover DB after server crash? I get following error: root@monetdb:/var/monetdb5/dbfarm# monetdb release vlad_a-f taken database out of maintenance mode: vlad_a-f root@monetdb:/var/monetdb5/dbfarm# monetdb start vlad_a-f starting database 'vlad_a-f'... FAILED start: starting 'vlad_a-f' failed: database 'vlad_a-f' appears to shut itself down after starting, check merovingian's logfile for possible hints root@monetdb:/var/monetdb5/dbfarm# tail /var/log/monetdb/merovingian.log 2011-04-11 11:40:53 MSG merovingian[2477]: starting database 'vlad_a-f', up min/avg/max: 2s/1h/15h, crash average: 0.00 0.30 0.20 (18-12=6) 2011-04-11 11:40:53 MSG vlad_a-f[2783]: arguments: /usr/bin/mserver5 --config=/etc/monetdb5.conf --dbname=vlad_a-f --set merovingian_uri=mapi:monetdb://monetdb:50000/vlad_a-f --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/monetdb5/dbfarm/vlad_a-f/.mapi.sock --set monet_vault_key=/var/monetdb5/dbfarm/vlad_a-f/.vaultkey --set monet_daemon=yes 2011-04-11 11:40:53 ERR vlad_a-f[2783]: !FATAL: BBPinit: could not write bat/BBP.dir 2011-04-11 11:40:53 MSG merovingian[2477]: database 'vlad_a-f' (2783) has exited with exit status 1 2011-04-11 11:41:02 ERR control[2477]: (local): failed to fork mserver: database 'vlad_a-f' appears to shut itself down after starting, check merovingian's logfile for possible hints Is there a procedure how to recover this? Thanks, Vlad

3 3

[MonetDB-users] Temporary tables
by Charalampos Nikolaou 10 Apr '11

10 Apr '11

Hi, this page (http://monetdb.cwi.nl/SQL/Documentation/Table-Definition.html) states that only the LOCAL TEMPORARY scope of SQL is implemented. Considering the following interaction with in mclient, it does not seem that is implemented at all. Am I missing something here or it is a bug? sql>CREATE LOCAL TEMPORARY TABLE transaction_statements ( ctx INTEGER NOT NULL, subj INTEGER NOT NULL, pred INTEGER NOT NULL, obj INTEGER NOT NULL, expl BOOL NOT NULL ); operation successful sql>INSERT INTO transaction_statements (ctx, subj, pred, obj, expl) VALUES (0, 1, 2, 805306369, 1); 1 affected row (10.450ms) sql>SELECT * FROM transaction_statements; +-----+------+------+-----+------+ | ctx | subj | pred | obj | expl | +=====+======+======+ +-----+------+------+-----+------+ 0 tuples (1.490ms) sql>SELECT * FROM tmp.transaction_statements; +-----+------+------+-----+------+ | ctx | subj | pred | obj | expl | +=====+======+======+ +-----+------+------+-----+------+ 0 tuples (1.443ms) sql>SELECT * FROM tmp._tables; +------+------------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-------+------+-------+ | id | name | sche | query | type | syste | comm | reado | : : : ma_i : : : m : it_a : nly : : : : d : : : : ctio : : : : : : : : : n : : +======+========================+======================================================================+======+=======+======+=======+ | 5678 | transaction_statements | 2103 | null | 0 | false | 1 | false | +------+------------------------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-------+------+-------+ 1 tuple (2.013ms) Thanks, Babis

2 1

[MonetDB-users] build failed on undefined crypt references
by Sander Kruseman 10 Apr '11

10 Apr '11

Hi, Building from recent tar.bz2 (11.1.1) on Linux/x86_64 I get: libtool: link: gcc -g -O2 -Wno-declaration-after-statement -D_REENTRANT -o .libs/mclient mclient.o ReadlineTools.o -pthread -Wl,--export-dynamic ./.libs/libmcutil.a ../mapilib/.libs/libmapi.so /var/scratch/sander/MonetDB-11.1.1/common/stream/.libs/libstream.so ../../common/stream/.libs/libstream.so -lbz2 -lreadline -ltermcap -lcrypt -pthread -Wl,-rpath -Wl,/usr/local/lib ../mapilib/.libs/libmapi.so: undefined reference to `MD5' ../mapilib/.libs/libmapi.so: undefined reference to `SHA224' ../mapilib/.libs/libmapi.so: undefined reference to `RIPEMD160' ../mapilib/.libs/libmapi.so: undefined reference to `SHA512' ../mapilib/.libs/libmapi.so: undefined reference to `SHA384' ../mapilib/.libs/libmapi.so: undefined reference to `SHA256' ../mapilib/.libs/libmapi.so: undefined reference to `SHA1' What could be causing this? Any help is much appreciated. Sander Kruseman

2 1

[MonetDB-users] How to access MonetDB in Javascript without XRPC/XQuery?
by dsuh 09 Apr '11

09 Apr '11

Now that XRPC/XQuery is being removed from the MonetDB source, are there any options for client-side browser Javascript access to SQL queries? Would it end up looking like using the PHP interface in the web server side and passing the results to the client-side Javascript? Exactly how would this look like? I am interested in using HTML5/Javascript on the client-side to create a mashup of different data including GIS information from MonetDB. Can anyone suggest possible solutions? David Suh

2 1

[MonetDB-users] history support in mclient across sessions
by Charalampos Nikolaou 07 Apr '11

07 Apr '11

Hi, does mclient support saving of commands in a history file and accessing them again in a future mclient session? Currently, there is a -H option, but it doesn't seem to work (Mar2011 release). Thanks, Babis

2 1

[MonetDB-users] Killing long running queries
by Ramkumarlak 06 Apr '11

06 Apr '11

Is there any way to know the current running queries (like mysql's "show full processlist"). Also is there any way of killing long running queries. -- View this message in context: http://old.nabble.com/Killing-long-running-queries-tp31276295p31276295.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 5

Re: [MonetDB-users] [expath] Kudos on CXAN
by Christian Grün 05 Apr '11

05 Apr '11

…time for MonetDB to join up ;) ___________________________ On Tue, Apr 5, 2011 at 12:04 AM, Florent Georges <fgeorges(a)gmail.com> wrote: > On 3 April 2011 14:31, Chris Maloney wrote: > >> Florent, I suddenly realized this morning why you have the >> image that you do on the CXAN site -- it's a pronunciation >> hint! > > Good catch! > > For the others, the picture on the home page of CXAN > <http://cxan.org/beta/> is a painting of Paul Cezanne > <http://en.wikipedia.org/wiki/Paul_C%C3%A9zanne>, a French > painter. > > I hesitated with a self-portrait (this one is a portrait of > Victor Chocquet, dont ask me who he is), but I liked this one. > The hint is that the X is actually pronounced like a Z, to make > the name fluent (either a la French, with a C like in sailer, or > English style, with a C like in sea). > > Very good catch :-) > > -- > Florent Georges > http://fgeorges.org/ > > -- > You received this message because you are subscribed to the Google Groups "EXPath" group. > To post to this group, send email to expath(a)googlegroups.com. > To unsubscribe from this group, send email to expath+unsubscribe(a)googlegroups.com. > For more options, visit this group at http://groups.google.com/group/expath?hl=en.

1 0

[MonetDB-users] query dit not produce result set
by Bas Kaptijn 04 Apr '11

04 Apr '11

Hi, with the MAR2010 release (x64 windows through installer) (and also the release Feb2010-SP2) I get an exception running a particular query, whilst a slightly modified one with the expected result (except for the naming of a result column) does not. I narrowed the problem down to this: sql>CREATE TABLE dbg (a INT, b INT); operation successful sql>INSERT INTO dbg (a,b) VALUES (10,10); 1 affected row (32.410ms) sql>SELECT a as d, SUM(b), (2 * (SUM(b) / (SELECT 2))) as f FROM dbg GROUP BY d; +------+------+------+ | d | L451 | f | +======+======+======+ | 10 | 10 | 10 | +------+------+------+ 1 tuple (8.841ms) sql>SELECT a as d, SUM(b) as e, (2 * (SUM(b) / (SELECT 2))) as f FROM dbg GROUP BY d; sql> The second select query does not produce a result set (as stated in the SQLException I get when doing this through JDBC). The monetdb client just shows a new prompt without stating any warning/error. The query works when I remove the "as e" ... or the "select 2" but in the real case I get this problem this select does much more than just selecting the number 2. I think this is a bug. Or am I doing something that is not (yet) supported in MonetDB? Thanks Bas Kaptijn

2 1

[MonetDB-users] for help(software information is added)
by Xianmin Liu 31 Mar '11

31 Mar '11

Hi, Sorry for missing the software information in last letter. I used MonetDB4-XQuery-Installer-x86_64-20110324 version, it seems to be the latest. If you have read my last letter, pls ignore followings. I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following. ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB. !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy hand helms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec Can you help me? How can I fix it? Thanks a lot. -- Xianmin Liu

2 2

[MonetDB-users] DROP TABLE IF EXISTS <table>;
by Michael Sioutis 31 Mar '11

31 Mar '11

Hello, I want to prevent an SQLException from occurring for tables that do not exist in my database. For example: sql>DROP TABLE "locked"; SQLException:sql.catalog:DROP TABLE: no such table 'locked' It seems (and is documented) that there is no such thing like "IF EXISTS" in MonetDB. Is there a way to circumvent the SQLException? Mike

5 8

Re: [MonetDB-users] for help(software information is added)
by Peter Boncz 31 Mar '11

31 Mar '11

Xianmin, On Windows we only build with 32bits oids (even with 64bits pointers). When working with 32-bits offsets string heaps are restricted to 16GB. For large data, we recommend 64-bits linux. Then grab a oid64 rpm. Peter From: Xianmin Liu [mailto:xianmliu@gmail.com] Sent: donderdag 31 maart 2011 11:09 To: monetdb-users(a)lists.sourceforge.net; Peter Boncz Cc: Wenfei Fan; Gao Cong; Nan Tang; Anastasios (Tasos) Kementsietsidis Subject: for help(software information is added) Hi, Sorry for missing the software information in last letter. I used MonetDB4-XQuery-Installer-x86_64-20110324 version, it seems to be the latest. If you have read my last letter, pls ignore followings. I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following. ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB. !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy hand helms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec Can you help me? How can I fix it? Thanks a lot. -- Xianmin Liu

1 0

[MonetDB-users] for help
by Xianmin Liu 31 Mar '11

31 Mar '11

Hi, I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following. ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB. !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy hand helms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec Can you help me? How can I fix it? Thanks a lot. -- Xianmin Liu

2 1

[MonetDB-users] DROP TABLE IF EXISTS <table>;
by Michael Sioutis 30 Mar '11

30 Mar '11

Hello, I want to prevent an SQLException from occurring for tables that do not exist in my database. For example: sql>DROP TABLE "locked"; SQLException:sql.catalog:DROP TABLE: no such table 'locked' It seems (and is documented) that there is no such thing like "IF EXISTS" in MonetDB. Is there a way to circumvent the SQLException? Mike

1 0

[MonetDB-users] How to determine size of a table and its indexes
by Nekas 30 Mar '11

30 Mar '11

In Oracle and MySQL for example, we can obtain the space occupied in the DB by tables and indexes in the user space. Is this possible in MonetDB? If so, how can I do it? I often need it, so it would be quite helpful to me. Thanks you in advance, Ana T.

1 2

[MonetDB-users] Announcement: New Mar2011 feature release of MonetDB suite
by Sjoerd Mullender 28 Mar '11

28 Mar '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Mar2011 feature release of the MonetDB suite of programs. This release marks the last planned release of MonetDB4 and Pathfinder, and hence the last planned release of MonetDB/XQuery. Development of the Pathfinder compiler continues at the University of Tübingen. The next release, currently planned for April 2011, should be a smooth upgrade from this release, even if MonetDB/XQuery is installed (which will of course then not be upgraded). This release also marks the first release of the new way of building the MonetDB suite. For details on both these issues, and on the other features and bug fixes of this release, please see the release notes at <http://monetdb.cwi.nl/Development/Releases/Mar2011/>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] Recycler Optimizer
by Henry Addington 24 Mar '11

24 Mar '11

Hello: I understand some of the optimizers/pipelines are experimental as indicated in the monetdb5.conf file. Has anyone used the Recycle pipeline for Skyserver? Would you mind sharing your experience, in terms of performance improvement and whether it can bet used in a production environment? We are using the Oct2010-SP1 release for a db that is read-only. I would guess if there are issues, I would think the problems would likely show up in read/write systems. Any feedback would be appreciated.... It looks like the Recycle pipeline for Skyserver pipeline is based on the Nov-2009 pipeline. Has anyone tried the recycler optimizer with the Feb2010 default pipeline? I was going to add "recycle" between "deadcode" and "reduce" in the default pipeline and see if it would work. Thanks. Henry

3 4

[MonetDB-users] ask for help
by Xianmin Liu 23 Mar '11

23 Mar '11

Hi, It's me again. Recently, I was doing some experiments using MonetDB/XQuery. And I asked one question about errors when loading XML data some days ago, Peter solved my problem very kindly. Now, there is an error again, pls help me. Following is my setting, CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB System: Ubuntu 10.04 64bits MonetDB: setup by deb, serverV4.40.3 GDKv1.40.3, MonetDB/XQuery module 0.40.3 loaded. I used pf:add-doc to add data, the data size is 3GB(generated by xmark generator with factor 30), error returned: ERROR: !xquery> -- Xianmin Liu

2 1

[MonetDB-users] User-defined MAL functions
by Petros Tsialiamanis 22 Mar '11

22 Mar '11

Hi, I have some questions about using user-defined MAL functions I would like to create a function in MAL. 1) Is it possible that the function can take variable number of BATs as argument like in the case of algebra.joinPath(l:bat[:any,:any]... ) ? 2) If yes, how I can get the number of BATs passed as argument and 3) how I can get the n-th element of the argument? Thank you in advance Petros

2 1

[MonetDB-users] Some questions
by Henning Grosch 22 Mar '11

22 Mar '11

Hi everyone, I have a some questions about using monetDB. First thing I noticed when using extract on a timestamp field I can only cast the timestamp as year, month, day and so on. Is it somehow possible to extract the week of the year? Week starts with monday. Another thing is, can i use if statements in a query like i could in mysql? And the last one, i already searched in the archiv and found a post about win 32 odbc driver and column names which appear as questionmarks or other symbols like when using a wrong encoding, does anybody has the same problem or knows a solution? Regards Henning

1 0

Re: [MonetDB-users] ask for help
by Peter Boncz 21 Mar '11

21 Mar '11

Hi, You cannot load the 1G of data in the 32-bits windows version on MXQ. Simply won’t fit the virtual address space. Use the 64-bits version (best: use the 64-bits Linux version). Peter From: Xianmin Liu [mailto:xianmliu@gmail.com] Sent: Monday, March 21, 2011 2:11 PM To: monetdb-users(a)lists.sourceforge.net Cc: Nan Tang; Wenfei Fan Subject: [MonetDB-users] ask for help Hi, I got some errors when using MonetDB/XQuery, could you pls give me some help? I was running experiments over large XML data using MonetDB/XQuery. My hardware: CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB MonetDB: Server v4.32.0 based on GDK v1.32.0, Compiled for i686-pc-win32/32bit, MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra') When I try to load xmark1G.xml (generated by xmark generator with factor 10) by command 'pf:add-doc("H:\lxm\1G\xmark1000M.xml","xmark1G.xml")<>' Return errors: ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 278712278 for 03\373theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: CMDshred_url: operation failed. -- Xianmin Liu

1 0

[MonetDB-users] ask for help
by Xianmin Liu 21 Mar '11

21 Mar '11

Hi, I got some errors when using MonetDB/XQuery, could you pls give me some help? I was running experiments over large XML data using MonetDB/XQuery. My hardware: CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB MonetDB: Server v4.32.0 based on GDK v1.32.0, Compiled for i686-pc-win32/32bit, MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra') When I try to load xmark1G.xml (generated by xmark generator with factor 10) by command 'pf:add-doc("H:\lxm\1G\xmark1000M.xml","xmark1G.xml")<>' Return errors: ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 278712278 for 03\373theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: CMDshred_url: operation failed. -- Xianmin Liu

1 0

[MonetDB-users] EXTRACT(SECOND FROM timestamp) gives 0 seconds
by spamme.gs＠googlemail.com 18 Mar '11

18 Mar '11

Hello, I can't understand the following behaviour : Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2010-SP1) Database: MonetDB v5.22.3, 'tradedb' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select extract(second from timestamp '2011-3-10 9:30:42.246'); +---------------------+ | second_single_value | +=====================+ | 0.246 | +---------------------+ Whoops. Well maybe monetdb can EXPLAIN what's goin on sql>explain select extract(second from timestamp '2011-3-10 9:30:42.246'); | function user.s2_1{autoCommit=true}():void; | | sql.mvc(); | | _4 := mtime.milliseconds(2011-03-10 09:30:42.246:timestamp); | | sql.exportValue(1,".","second_single_value","decimal",9,3,8,_4,""); | | end s2_1; | Ok ... Looks like this function is declared here: (modules/atoms/mtime.mal) 413 command milliseconds(d:daytime) :int 414 address MTIMEdaytime_extract_milliseconds And digging into the source of _that_ function shows us that it's just returning the millisecond part. So, any chance of a quick fix here? It looks like I'm not the first to ask this question: http://sourceforge.net/mailarchive/message.php?msg_id=23238837 but that poor guy was completely ignored.

2 1

[MonetDB-users] LC_ALL=C in monetdb-install.sh breaks RDoc
by Lazaros Koromilas 18 Mar '11

18 Mar '11

Hello, all! I did the following on an Arch Linux yystem: MonetDB-Oct2010-SuperBall-SP1> ./monetdb-install.sh --enable-xquery ... make[8]: Entering directory `/var/tmp/MonetDB-1299754542/MonetDB-client/clients/ src/ruby/adapter' mkdir -p /home/myuser/MonetDB/lib/ruby/gems/1.9.1 gem install --local --install-dir /home/myuser/MonetDB/lib/ruby/gems/1.9.1 --forc e --rdoc activerecord-monetdb-adapter-0.1.gem ERROR: While executing gem ... (Encoding::UndefinedConversionError) U+2019 from UTF-8 to US-ASCII Successfully installed activerecord-monetdb-adapter-0.1 1 gem installed Installing ri documentation for activerecord-monetdb-adapter-0.1... Installing RDoc documentation for activerecord-monetdb-adapter-0.1... make[8]: *** [install-exec-local-activerecord-monetdb-adapter-0_1_gemspec] Error 1 ... Setting LC_ALL to UTF-8 fixes my problem. --- monetdb-install.sh.orig 2011-03-10 13:34:58.000000000 +0200 +++ monetdb-install.sh 2011-03-10 13:35:05.000000000 +0200 @@ -108,7 +108,7 @@ : ${FETCHCOMMAND:="wget --progress=dot:mega -t 5 --passive-ftp -O \"\${FILE}\" \"\${URI}\""} # make sure we get English error messages and so on -export LC_ALL=C +export LC_ALL="en_US.UTF-8" # parse command line options ORGARG=$*

2 3

[MonetDB-users] MonetDB/XQuery order by problem
by Rob Fynes 13 Mar '11

13 Mar '11

Hello, I'm trying to produce a sorted list of results with MonetDB/XQuery using 'order by', with a query that goes something like this: for $cname in pf:collections()/text() return if (local:is-date-between($cname, **start_date_here**, **end_date_here**)) then for $s in pf:collection($cname)//Dublin/stations[time/hour=13 and time/minute>=22 and time/minute<=25] order by $s/timeOfDay return $s else () ...the 'is-date-between' is just some custom function to check if the $cname (collection names are dates) is between the start and end dates supplied to the function. However, the problem is with the nested for loop. I want each $s that satisfies the XPath expression to be returned, but ordered by $s/timeOfDay. When I submit this query, I get the following error: ERROR = !type error: [err:XPTY0004] Orderspec requires at most one atomic item (given: string*). I've tried a number of variations on this, but the error is always the same. If I remove the order by clause, the results will return correctly, but in the unsorted order that they appear in the database. I'm still a beginner with XQuery, so I'm unsure where I'm going wrong, and documentation and tutorials have been unhelpful so far. Any help is appreciated! Regards, -Rob

2 3

[MonetDB-users] Copy records into table failed
by Tibor Barna 11 Mar '11

11 Mar '11

Hello, I am trying to import data from MySQL into MonetDB usig the bulk loader, the csv has ~11GB, 163Mil Rows. When running: sql>copy 163715240 records into sys.clicks from '/data/csv_monet_dump/clicks.tbl' using delimiters '|',' more>', '"' null as ''; I always get this error message: SQLException:importTable:failed to import table ERROR: HEAPextend: failed to extend to 654860960 for 32/3231tail ERROR: TABLETcreate_bats: Failed to create bat of size 163715240 merovingian.log: 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) fails, try to free up space [memory in use=41411528,virtual memory in use=12310675456] 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) result [mem=32087208,vm=12310675456] I am using 64bit openSUSE 11.3, 12GB RAM, 4GB Swap. Does anybody know how can i overcome this error? Any help would be most welcome :) Cherers, Tibor. -- View this message in context: http://old.nabble.com/Copy-records-into-table-failed-tp31115783p31115783.ht… Sent from the monetdb-users mailing list archive at Nabble.com.

3 7

[MonetDB-users] Get MAL from SQL
by Kusber, Michael 10 Mar '11

10 Mar '11

Hi, is there a possibility to see the MAL code generated from a specific SQL statement? Kind regards, Michael

3 2

[MonetDB-users] Are these bugs？
by tangray 09 Mar '11

09 Mar '11

hi all when i use c++ pragrm to insert some data into a table,the program crashes,the message is below: so i use gdb to debug the bug, i find that these places may be bugs： 1. out of bounds access file mapi.c function: mapi_param_store 2356 strncpy(hdl->query + k, p, q - p); 2357 k += q - p; 2358 hdl->query[k] = 0; q-p is equal or bigger than lim - k ,the strncpy will hava a out of bounds access. this may be the reason to crash. 2. out of bounds access file:mapi.c function: mapi_param_store 2411 case MAPI_DOUBLE: 2412 checkSpace(20); 2413 sprintf(hdl->query + k, "%.17g", *(double *) src); 2414 break; sprintf(hdl->query + k, "%.17g", *(double *) src) , sprintf function with "%.17g" format may produces a string longer than 20,eg： double d = 0.000200000000000001; double d = 0.000200000000000001; printf("%.17g",d); the result is : 0.00020000000000000101 so i think, in some paticular condition ,this will hava a out of bounds access too. I'm working with MonetDB Server v5.22.3, based on kernel 1.40.3 over fedora 12.

2 1

[MonetDB-users] lose some datas of a table's one field, how can this happen?
by tangray 08 Mar '11

08 Mar '11

Hi when execute sql : select * from td_ind_security_held where id = 1560000 in DbVisualizer 7.1.1,it appears "protocol violation: unexpected line in data block" td_ind_security_held has 30 columns(1 int,6 varchar, others decimal) and 7219618 rows. so i count field's records one by one , all fields's records are equal to 7219618 except the allgain's records, it is only 658337. it obviously lost some data. when program inserted the data into the table , the monet server carshed out some times, can this be the reson for the lose? I'm working with MonetDB Server v5.22.3, based on kernel 1.40.3 (downloaded msi from sourceforge) over Windows XP SP3.

2 1

Re: [MonetDB-users] MonetDB/XQuery JDBC result issue
by Wouter Alink 04 Mar '11

04 Mar '11

Hello Rob, Yes, the good thing about the JDBC interface is that the result is streamed to the client, so that there is no large memory-consumption on the client-side. However, be aware that, due to the way MonetDB works internally, the whole result is first computed and kept in memory at the server, before it is shipped to the client. This will for most use-cases not be a problem, but if you are calculating huge result sets it might. If you only fetch the first 10 rows of a result at the client, you may want to wrap a "subsequence(<your-query>, 1, 10)" around your query. Kind regards, Wouter On Fri, Mar 4, 2011 at 3:42 PM, Rob Fynes <monetdbmail(a)gmail.com> wrote: > Hi Wouter, > > Thanks for the response. You were right about that, it's working fine now. > I was thrown slightly when I saw the MonetDB documentation that said "The > result comes back as a single-row single-column multi-line string result". I > figured that the 'entire' set of results would be stuffed into one row and > one column in the result set object. In retrospect, this wouldn't make much > sense, since you could end up with a *really* big string! > > Thanks, > -Rob > > > On Wed, Mar 2, 2011 at 1:46 PM, Wouter Alink <wouter.alink(a)gmail.com>wrote: > >> Dear Rob Fynes, >> >> You have already provided the answer in your email. XQuery specifies that >> a result is actually a sequence of items. The "XQueryResult"-node has only >> been added to convert the actual result sequence into a valid XML fragment. >> Using XQuery over JDBC works like SQL; you can iterate over the results. >> There is no artificial XQueryResult-node added. >> >> For example: >> >> ResultSet rs = conn.executeQuery("...."); >> while(rs.next()) { >> .. do something with the result .. >> } >> >> With kind regards, >> Wouter Alink >> >> >> >> On Tue, Mar 1, 2011 at 10:13 PM, Rob Fynes <monetdbmail(a)gmail.com>wrote: >> >>> Hello, >>> >>> I've been having an issue with MonetDB/XQuery using the JDBC driver. >>> Essentially, when I supply an *XPath* query such as the following: >>> >>> pf:collection("collection_name")//stations[date/year>=2010] >>> >>> ... using the mclient utility, there's no problem. The result is an >>> <XQueryResult> root node containing all the relevant 'stations' nodes as >>> requested by the query. >>> However, if I issue the same query to the server from a Java application >>> using the JDBC driver for MonetDB, the string result that is returned merely >>> contains one 'stations' node (the first one that gets processed, I think). >>> >>> My guess is that the resultset (of 'stations' nodes) is not being wrapped >>> in a root node, as is the case when querying using the mclient utility. >>> But I'm unsure if this due to a bug, or if there's something that I'm not >>> doing correctly. >>> >>> I'm also aware that this issue can be resolved by modifying the query >>> (when using JDBC) to something such as: >>> >>> <result> { >>> pf:collection("collection_name")//stations[date/year>=2010] >>> } </result> >>> >>> ... but I have reasons for not wanting to do this. >>> Any help is much appreciated. If anyone needs more information, just let >>> me know (and if it makes any difference, I'm running a local server >>> instance). >>> >>> Cheers, >>> -Rob >>> >>> >>> ------------------------------------------------------------------------------ >>> Free Software Download: Index, Search & Analyze Logs and other IT data in >>> Real-Time with Splunk. Collect, index and harness all the fast moving IT >>> data >>> generated by your applications, servers and devices whether physical, >>> virtual >>> or in the cloud. Deliver compliance at lower cost and gain new business >>> insights. http://p.sf.net/sfu/splunk-dev2dev >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >>> >> >

1 0

[MonetDB-users] MonetDB/XQuery JDBC result issue
by Rob Fynes 02 Mar '11

02 Mar '11

Hello, I've been having an issue with MonetDB/XQuery using the JDBC driver. Essentially, when I supply an *XPath* query such as the following: pf:collection("collection_name")//stations[date/year>=2010] ... using the mclient utility, there's no problem. The result is an <XQueryResult> root node containing all the relevant 'stations' nodes as requested by the query. However, if I issue the same query to the server from a Java application using the JDBC driver for MonetDB, the string result that is returned merely contains one 'stations' node (the first one that gets processed, I think). My guess is that the resultset (of 'stations' nodes) is not being wrapped in a root node, as is the case when querying using the mclient utility. But I'm unsure if this due to a bug, or if there's something that I'm not doing correctly. I'm also aware that this issue can be resolved by modifying the query (when using JDBC) to something such as: <result> { pf:collection("collection_name")//stations[date/year>=2010] } </result> ... but I have reasons for not wanting to do this. Any help is much appreciated. If anyone needs more information, just let me know (and if it makes any difference, I'm running a local server instance). Cheers, -Rob

2 1

[MonetDB-users] draft message
by Martin Kersten 01 Mar '11

01 Mar '11

Dear MonetDB users and developers, The preparations for the March 2011 release are on target. This release will feature a much simplified installation and compilation process using a unified bootstrap/configure/make/install. Ofcourse, installers and RPMs are also provided. Furthermore, the March 2011 release will be the freeze of the MonetDB XQuery source tree. It will remain downloadable from the website, but no active development or maintenance is foreseen. In April 2011 a MonetDB/SQL feature release will mark the split of the two code bases. From that point onward, we will primarily be concerned with SQL and its enhancements. Saying goodbye to the very old mserver4 code base, also triggered a major code renovation activity. The MonetDB/SQL code base will be cleaned from legacy code, 1-time experimental code, and all that is not strictly needed for the intended functionality. We expect that this cleanup will simplify maintenance in the future. The results of the renovation step will become visible in the fall of 2011. We plan for a smooth transition step for existing SQL environments. However, under the hood, we will change the MAL and kernel library interfaces significantly. To assess the consequences in the community we would like to learn about projects that directly address the MAL or GDK layer. Please send us an email at info(a)monetdb.org with the project highlights. regards, Martin Kersten

1 0

[MonetDB-users] protocol violation: unexpected line in data block
by tangray 28 Feb '11

28 Feb '11

Hi when execute sql : select * from td_ind_security_held where id = 1560000 in DbVisualizer 7.1.1,it appears "protocol violation: unexpected line in data block" td_ind_security_held has 30 columns(1 int,6 varchar, others decimal) and about 7 million rows. I'm working with MonetDB Server v5.22.3, based on kernel 1.40.3 (downloaded msi from sourceforge) over Windows XP SP3.

2 3

Re: [MonetDB-users] Query time takes equally long on whole
by dsuh 24 Feb '11

24 Feb '11

We have experienced big improvements from Nov-2009SP2 to Oct-2010SP1 as well except for views. It appears that the mitosis optimizer has problems with views and queries on them run slower than without it. (~9 sec slower on 1 min queries on views and ~20 sec slower on 2 min queries on views using default_pipe rather than no_mitosis_pipe or nov2009_pipe) The mitosis optimizer was the big change in the optimizer pipeline between these versions. Is it possible that this is still from the bug?: http://bugs.monetdb.org/show_bug.cgi?id=2472 Can anyone tell me if there are any plans for optimization improvements (either mitosis, commonTerms, or otherwise) with queries on views? David Suh > > Hello Henry, > Thank you for your reply! See inline. > > On Thu, Feb 24, 2011 at 7:54 AM, Henry Addington <hs.addington(a)gmail.com > >wrote: > > > I am no technical expert on MonetDB. But, we have been using MonetDB for > > more than a year now. So, I can give you couple of suggestions > > (or workarounds) that might help based on our experience. We are using a > > third-party application that generates queries and sends them to MonetDB > > which has a transaction table with more than 1.6billion records and > growing > > (timeseries data like yours..) linked to dimensional tables. The kinds of > > queries that are being generated are almost exact type that you are > showing, > > some sort of an aggregation query using group by and filtered with where > > clause. > > > > One thing that we noticed with queries with joins is that inner join > seems > > to be significantly faster than left/right joins even if the results are > no > > different between the 3 types of joins. I know that might not be an > option > > for you. But, it works for our use case. > > > > I've changed my queries into two separate ones: one for the total > timescope, and one for (occasionally) getting the results spread over the > interval for one item. The new total timescope query is faster on less time > as scope, and as fast as before on the total 24 hour scope, so definitely > an > improvement for me. > > > > > > Another thing that we noticed is if you add constraints, ie primary and > > foreign keys, you will get better performance. I think you might already > > have them in your tables though.... > > > > I do have them, but i didn't add them in my create statement. I''m also > going to try if this adds extra speed. > > > > > > We are in the process of testing Oct-2010SP1 release so we can upgrade > > our Nov-2009SP2 environment. We are seeing significant performance > > improvement in most of our typical queries and much smaller memory > footprint > > with the Oct-2010SP1 release. But, we see some performance degradation > with > > queries based on views instead of tables. You might want to try the > Nov-2009 > > optimizer and see if that helps. You can do this by specifyng the > Nov-2009 > > optimizer in the monetdb5.conf file. > > > > This didn't show any difference on the original query, but thanks for > mentioning. I might need it in the future. > > Kind regards, > Rob Berentsen > > > > > On Wed, Feb 23, 2011 at 4:24 AM, Rob Berentsen <rhberentsen(a)gmail.com > >wrote: > > > >> Hello, > >> > >> I'm new to monetdb and installed it two weeks ago to do some tests. I > was > >> very much impressed with the speed on large tables, but in some of my > latest > >> tests regarding subparts of these tables based on a unix timestamp the > time > >> was the same as querying the whole table. Does anyone know what I can do > to > >> get higher speeds on queries in monetdb that only use part of the table, > >> like partition pruning, explicit foreign keys or unique statements, > >> indexes or some other way? > >> > >> I have the following two tables and number of records: > >> > >> CREATE TABLE table1 (T1vid INT NOT NULL, T1Field1 VARCHAR(20) NOT > >> NULL, T1Field2 TINYINT NOT NULL); > >> This table has about 400.000 records. The records are non time bounded, > >> it's just a list. The T1vid is an increased ID, but not generated in > >> momentdb at the moment, but in mysql > >> > >> CREATE TABLE table2 (T2mid INT NOT NULL, T2vid INT NOT NULL, T2timestamp > >> INT NOT NULL, T2Field1 INT NOT NULL, T2Field2 INT NOT NULL); > >> This table has about 1.000.000 records and a unix_timestamp field. It > get > >> a few new records every second and records older then 24 hours are > deleted. > >> The T2mid-field refers to some other table that's not in momentdb's > >> database, so just an integer. The T2vid-field however refers to the > >> T1vid-field in table1 (a foreign key, but i didn't define it that way). > >> > >> The following query takes about 3,5 seconds, it doesnt't have the > >> timestamp included in the WHERE-clause: > >> SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / > 3600) > >> as interval > >> FROM table2 AS t2 > >> LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid > >> WHERE t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique > >> v.T1vid ID's*) > >> GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY > interval > >> DESC; > >> Resulting is a list of the 15 v.T1vid ID's times the number of > interval's. > >> t2.T2timestamp is now devided by 3600 and grouped on this result, so > each > >> set of 15 v.T1vid ID's would reflect 1 hour, returning max 24x15 rows. > >> ** > >> The following query also takes about 3,5 seconds, but it does have the > >> timestamp included in the WHERE-clause. It only needs to access 1 hour > of > >> data instead of the whole 24 hours, and groups it into intervals of 5 > >> minutes (300 sec): > >> SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / > 300) > >> as interval > >> FROM table2 AS t2 > >> LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid > >> WHERE* t2.T2timestamp BETWEEN 1298360000 AND 1298363600* > >> AND t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique > v.T1vid > >> ID's*) > >> GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY > interval > >> DESC; > >> > >> t2.T2timestamp is now devided by 300, so each set of 15 v.T1vid ID's > would > >> reflect 5 minuntes, returning max 12x15 rows. > >> > >> I would very much appreciate any help or hints; please let me know any > >> question you might have. > >> > >> > >> Kind regards, > >> Rob Berentsen > >> > >> > >> > ------------------------------------------------------------------------------ > >> Free Software Download: Index, Search & Analyze Logs and other IT data > in > >> Real-Time with Splunk. Collect, index and harness all the fast moving IT > >> data > >> generated by your applications, servers and devices whether physical, > >> virtual > >> or in the cloud. Deliver compliance at lower cost and gain new business > >> insights. http://p.sf.net/sfu/splunk-dev2dev > >> _______________________________________________ > >> MonetDB-users mailing list > >> MonetDB-users(a)lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/monetdb-users > >> > >> > > > > > > > ------------------------------------------------------------------------------ > > Free Software Download: Index, Search & Analyze Logs and other IT data in > > Real-Time with Splunk. Collect, index and harness all the fast moving IT > > data > > generated by your applications, servers and devices whether physical, > > virtual > > or in the cloud. Deliver compliance at lower cost and gain new business > > insights. http://p.sf.net/sfu/splunk-dev2dev > > _______________________________________________ > > MonetDB-users mailing list > > MonetDB-users(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > > > >

1 0

[MonetDB-users] monetdb on ubuntu 10.10 - xquery results displaying only in single line!
by sagarjauhari 24 Feb '11

24 Feb '11

Hello, my xquery results are displaying only in a single line. The hello world example on http://monetdb.cwi.nl/XQuery/QuickTour/DOCMGT/ displays result correctly as: > doc("HelloWorld.xml") <?xml version="1.0" encoding="utf-8"?> <doc> <greet kind="informal">Hi </greet> <greet kind="casual">Hello </greet> <location kind="global">World</location> <location kind="local">Amsterdam</location> </doc> But this: xquery>for $a in (1 to 3) return <number>{$a}</number> outputs this: <XQueryResult><number>1</number><number>2</number><number>3</number></XQueryResult> and for the file :test.xq: (1, 42.0, "Hello World", <node attr="value">test</node>) the xquery mclient -lx -f xml test.xq outputs <?xml version="1.0" encoding="utf-8"?> <XQueryResult>1 42.000000 Hello World<node attr="value">test</node></XQueryResult> Any suggestions? Thanks in advance -- View this message in context: http://old.nabble.com/monetdb-on-ubuntu-10.10---xquery-results-displaying-o… Sent from the monetdb-users mailing list archive at Nabble.com.

2 4

[MonetDB-users] Stored Procedures in MonetDB
by Irini Fundulaki 24 Feb '11

24 Feb '11

Hi all, would it be possible to let us know whether MonetDB/SQL supports stored procedures and provide us with pointers to the features supported (e.g., recursion). Many Thanks Irini ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

[MonetDB-users] Query time takes equally long on whole and just small part of table
by Rob Berentsen 24 Feb '11

24 Feb '11

Hello, I'm new to monetdb and installed it two weeks ago to do some tests. I was very much impressed with the speed on large tables, but in some of my latest tests regarding subparts of these tables based on a unix timestamp the time was the same as querying the whole table. Does anyone know what I can do to get higher speeds on queries in monetdb that only use part of the table, like partition pruning, explicit foreign keys or unique statements, indexes or some other way? I have the following two tables and number of records: CREATE TABLE table1 (T1vid INT NOT NULL, T1Field1 VARCHAR(20) NOT NULL, T1Field2 TINYINT NOT NULL); This table has about 400.000 records. The records are non time bounded, it's just a list. The T1vid is an increased ID, but not generated in momentdb at the moment, but in mysql CREATE TABLE table2 (T2mid INT NOT NULL, T2vid INT NOT NULL, T2timestamp INT NOT NULL, T2Field1 INT NOT NULL, T2Field2 INT NOT NULL); This table has about 1.000.000 records and a unix_timestamp field. It get a few new records every second and records older then 24 hours are deleted. The T2mid-field refers to some other table that's not in momentdb's database, so just an integer. The T2vid-field however refers to the T1vid-field in table1 (a foreign key, but i didn't define it that way). The following query takes about 3,5 seconds, it doesnt't have the timestamp included in the WHERE-clause: SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / 3600) as interval FROM table2 AS t2 LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid WHERE t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique v.T1vid ID's*) GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY interval DESC; Resulting is a list of the 15 v.T1vid ID's times the number of interval's. t2.T2timestamp is now devided by 3600 and grouped on this result, so each set of 15 v.T1vid ID's would reflect 1 hour, returning max 24x15 rows. ** The following query also takes about 3,5 seconds, but it does have the timestamp included in the WHERE-clause. It only needs to access 1 hour of data instead of the whole 24 hours, and groups it into intervals of 5 minutes (300 sec): SELECT SUM(t2.T2Field2), t1.T1Field1, t1.T1Field2, (t2.T2timestamp / 300) as interval FROM table2 AS t2 LEFT JOIN table 1 AS t1 ON t1.T1vid=t2.T2vid WHERE* t2.T2timestamp BETWEEN 1298360000 AND 1298363600* AND t2.T2Field1=8 AND t2.T2vid IN (*list of 15 unique v.T1vid ID's*) GROUP BY interval, t1.T1Field1, t1.T1Field2 ORDER BY interval DESC; t2.T2timestamp is now devided by 300, so each set of 15 v.T1vid ID's would reflect 5 minuntes, returning max 12x15 rows. I would very much appreciate any help or hints; please let me know any question you might have. Kind regards, Rob Berentsen

2 2

[MonetDB-users] MAL changes in March2011
by Mike De-La-Columnar 22 Feb '11

22 Feb '11

Hi, It seems like the whole MAL language is being upgraded. (the use of columns instead of BATs) are those changes just cosmetic, or is there something else that will benefit from it? also, will the old MAL work in the new version? -- View this message in context: http://old.nabble.com/MAL-changes-in-March2011-tp30985134p30985134.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] rdf.sql missing from Mar2011 branch
by Babis Nikolaou 18 Feb '11

18 Feb '11

Hello, the rdf.sql file for creating the schema of an RDF database is missing from the branch of Mar2011. Could you add it please? Thanks, Babis

4 12

[MonetDB-users] Kernel / FS tuning
by Franck Routier 18 Feb '11

18 Feb '11

Hi, I was wondering if anyone has suggestions about what tweaks might best suit MonetDB's needs regarding : - (linux) kernel parameters : memory management, ... - FS (type and parameters) : block size, write barriers, ... Is there any kind of best practice or even some thoughts on this ? Thanks in advance for your inputs, Franck

1 0

[MonetDB-users] Problem starting monetdbd after installing MonetDB
by Babis Nikolaou 10 Feb '11

10 Feb '11

Hi all, i get the following error after installing MonetDB from CVS (tip) and running monetdb/d. $ monetdbd cannot open config file (null) unable to retrieve startup status $ monetdb cannot open config file (null) As opposed to the previous version i had (a version of CVS during November), now there is no /etc subdirectory inside the MonetDB installation directory, where the configuration files lie. I also searched for any configuration files inside the build directory, in case sth had gone wrong during installation, but nothing has been generated. I also noticed that the installation procedure when compiling from CVS has been changed. Do i miss something? I just run bootstrap, configure, make, and last make install. One last remark: in the manpage of monetdbd (and monetdb), the configuration file in question is referenced as @MONETDB5_CONFFILE@. Shouldn't had been mapped to a filename? Thanks, Babis

3 3

[MonetDB-users] Bug in EXTRACT?
by Alex Bo. 06 Feb '11

06 Feb '11

Hi, There seems to be a bug in the October SP. If you run this query: SELECT EXTRACT(YEAR FROM f1 .a1) as a2, EXTRACT(YEAR FROM f1 .a1) as a3 FROM (select '1996-02-01' as a2, '1996-07-01' as a1) f1 it returns: 1996, 1996-02-01 Instead of: 1996, 1996 The second column is returned as timestamp instead of int. Also, this does not happen if the first aliases (a2, a3) are changed to different names, nor does it happen if the two EXTRACTS are not identical. If one is YEAR and the second is MONTH it works, but if both are MONTH the same bug happens. I tried it also on the August build and it works fine there. Any ideas? -- View this message in context: http://old.nabble.com/Bug-in-EXTRACT--tp30855882p30855882.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

Re: [MonetDB-users] query slowdown with concurrent write
by Bennett Hiles 04 Feb '11

04 Feb '11

Hi Martin and Stefan, Thanks for the quick response, and sorry if this created another thread, I'm not familiar with how to use this mailing list. I am using: MonetDB v5.22.3 on ubuntu 9.04 64bit Queries and data loading use the php library. To further explain my use case, I am planning on having monetdb be updated about once a minute with 1000-10000 new rows of data. At the same time I will be having as many clients as possible making queries on the database. I have so far seen that 5 clients can be handled with satisfiable response times. Is this a suitable application for monetdb? The query times I have seen so far are great. My problems are handling the queries with concurrent writes and if possible allowing for more clients to make simultaneous queries. Re: Martin's questions: I tried using the recycle optimizer in the default pipe and my queries didn't slow down, which is awesome. But, my problem now is that the values aren't actually added to the dataset until an insert transaction occurs when I am not querying the database. Therefore, if I continuously query the database then the inserts are never seen, but if I stop querying for a second and another insert happens the database is updated. I am fine with some delay in the insert time, but is there anyway I could have it executed slowly alongside the queries, or am I too greedy :) The pipe I created is the same as the default pipe with the recycler optimizer added: bennett_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,recycle,reduce,dataflow,history,multiplex,garbageCollector My schema and and an example query are: CREATE TABLE my_test_table ( account_id bigint, dim1 bigint, dim2 bigint, dim3 bigint, dim5 varchar(50), dim6 varchar(50), dim7 varchar(256), dim8 bigint, dim9 varchar(256), dim10 varchar(256), unique_id bigint, my_time timestamp, summable int ); QUERY: select dim5, sum(summable), count(distinct unique_id) from my_test_table where account_id = 76 and my_time >= '2010-07-03 00:00:00' and my_time < '2010-07-10 00:00:00' group by dim5; Re Stefan's questions: The results from the trace are below. I apologize if any of the data looks incorrect, I scrubbed my table's field names to be more generic. I let the database cool down for 1 minute and 10 minutes, but the queries remained slow. If I restarted the database, the queries resumed their quick response times. FAST QUERY TIME: sql>trace select dim5, sum(summable) as summables, count(distinct unique_id) as unqiue_ids from my_test_table where account_id = 76 and my_time >= '2010-07-03 00:00:00' and my_time < '2010-07-10 00:00:00' group by dim5; +-----------------------+--------+--------+ | dim1 | summables | unique_ids | +=======================+========+========+ | test1 | 251 | 18 | | test2 | 48 | 11 | | test3 | 1 | 1 | | test4 | 2 | 1 | | test4 | 106 | 2 | +-----------------------+--------+--------+ 5 tuples (125.534ms) +-------+------------------------------------------------------------------------------------------+ | ticks | stmt | +=======+==========================================================================================+ | 3 | _5 := sql.mvc(); | | 18 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","account_id | : : ",2); : | 7 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","accou | : : nt_id",0); : | 4 | _15:bat[:oid,:lng] <tmp_1102>[0] := sql.bind(_5=21685368,"sys","my_test_table","account_id | : : ",1); : | 6 | _18:bat[:oid,:oid] <tmp_1357>[0] := sql.bind_dbat(_5=21685368,"sys","my_test_table",1); | | 21 | _13<tmp_1527>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat[: | : : oid,:lng][0]); : | 4 | _19<tmpr_1357>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 10 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=21685368,"sys","my_test_table" | : : ,"my_time",0); : | 27 | _14<tmp_1532>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 11 | _23 := calc.timestamp(A1="2010-07-03 00:00:00"); | | 12 | _17<tmp_1527>[8000000] := algebra.kunion(_14=nil,_15=nil:bat[:oid,:lng]); | | 4 | _24 := calc.timestamp(A2="2010-07-10 00:00:00"); | | 7 | _20<tmp_1532>[8000000] := algebra.kdifference(_17=nil,_19=<tmpr_1357>[0]); | | 4 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=21685368,"sys","my_test_table","my_t | : : ime",2); : | 29575 | _25<tmp_1531>[359554] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-07-03 00: | : : 00:00.000,_24=2010-07-10 00:00:00.000,true,false); : | 12 | _31:bat[:oid,:timestamp] <tmp_1132>[0] := sql.bind(_5=21685368,"sys","my_test_table","my_t | : : ime",1); : | 24 | _28<tmp_1527>[359554] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestamp] | : : [0]); : | 23 | _32<tmp_1536>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00:00 | : : .000,_24=2010-07-10 00:00:00.000,true,false); : | 35 | _29<tmp_1501>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00:00 | : : .000,_24=2010-07-10 00:00:00.000,true,false); : | 8 | _39:bat[:oid,:str] <tmp_1364>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","dim5",0); : | 12 | _30<tmp_1534>[359554] := algebra.kunion(_28=nil,_29=nil); | | 5 | _44:bat[:oid,:str] <tmp_1103>[0] := sql.bind(_5=21685368,"sys","my_test_table","dim5 | : : e",2); : | 23 | _33<tmp_1501>[359554] := algebra.kunion(_30=nil,_32=nil); | | 6 | _46<tmp_1536>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:str],_44=<tmp_1103>:bat[ | : : :oid,:str][0]); : | 8 | _34<tmp_1534>[359554] := algebra.kdifference(_33=nil,_19=nil); | | 9 | _47<tmp_1501>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:str]); | | 4 | _48:bat[:oid,:str] <tmp_1104>[0] := sql.bind(_5=21685368,"sys","my_test_table","dim5",1); : | 10 | _50<tmp_1527>[8000000] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:str]); | | 6 | _66:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",0); : | 3 | _68:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",2); : | 7 | _69<tmp_1501>[8000000] := algebra.kdifference(_66=nil:bat[:oid,:lng],_68=<tmp_1101>:bat[ | : : :oid,:lng][0]); : | 9 | _70<tmp_1541>[8000000] := algebra.kunion(_69=nil,_68=nil:bat[:oid,:lng]); | | 4 | _71:bat[:oid,:lng] <tmp_1102>[0] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",1); : | 8 | _74<tmp_1501>[8000000] := algebra.kunion(_70=nil,_71=nil:bat[:oid,:lng]); | | 4 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=21685368,"sys","my_test_table","summable",2) | : : ; : | 7 | _59<tmp_1541>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:bat[ | : : :oid,:int][0]); : | 8 | _60<tmp_1524>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 3 | _61:bat[:oid,:int] <tmp_1066>[0] := sql.bind(_5=21685368,"sys","my_test_table","summable",1) | : : ; : | 8 | _62<tmp_1541>[8000000] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 15525 | _35<tmp_1536>[359554] := algebra.semijoin(_20=nil,_34=nil); | | 2446 | _36<tmp_1534>[139] := algebra.uselect(_35=nil,A0=76:lng); | | 10 | _37<tmp_1536>[139] := algebra.markT(_36=nil,0@0:oid); | | 4 | _38<tmpr_1536>[139] := bat.reverse(_37=nil); | | 33 | _75<tmp_1477>[139] := algebra.leftjoin(_38=<tmpr_1536>[139],_74=nil); | | 47 | _51<tmp_1403>[139] := algebra.leftjoin(_38=<tmpr_1536>[139],_50=nil); | | 4 | _76<tmp_1477>[139] := algebra.selectNotNil(_75=nil); | | 47 | (ext61<tmp_1502>[5],grp59<tmp_1476>[139]) := group.done(_51=<tmp_1403>[139]); | | 19 | _63<tmp_1363>[139] := algebra.leftjoin(_38=nil,_62=nil); | | 8 | _54<tmp_1541>[5] := bat.mirror(ext61=<tmp_1502>[5]); | | 3 | _64<tmp_1363>[139] := algebra.selectNotNil(_63=nil); | | 47 | _55<tmp_1471>[5] := algebra.join(_54=<tmp_1541>[5],_51=nil); | | 43 | (_77<tmp_1344>[33],grp85) := group.derive(ext61=nil,grp59=<tmp_1476>[139],_76=<tmp_1477> | : : [139]); : | 6 | _79<tmp_1527>[33] := bat.mirror(_77=nil); | | 22 | _80<tmp_1506>[33] := algebra.join(_79=nil,_76=nil); | | 20 | _81:bat[:oid,:wrd] <tmp_1527>[5] := aggr.count(_80=nil,grp59=<tmp_1476>[139],_54=<tmp_15 | : : 41>[5]); : | 21 | _65:bat[:oid,:lng] <tmp_1536>[5] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 63484 | barrier _137 := language.dataflow(); | | 9 | _82 := sql.resultSet(3,1,_55=<tmp_1471>[5]); | | 7 | sql.rsColumn(_82=1,"sys.my_test_table","dim5","varchar",50,0,_55=nil); | | 5 | sql.rsColumn(_82=1,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 5 | sql.rsColumn(_82=1,"sys.my_test_table","unique_ids","wrd",64,0,_81=nil:bat[:oid,:wrd]); | | 3 | _99 := io.stdout(); | | 41 | sql.exportResult(_99=14711568,_82=1); | | 83716 | user.s4_1(76:lng,"2010-07-03 00:00:00","2010-07-10 00:00:00"); | +-------+------------------------------------------------------------------------------------------+ SLOW QUERY TIME: sql>trace select dim5, sum(summable) as summables, count(distinct unique_id) as unqiue_ids from my_test_table where account_id = 76 and my_time >= '2010-07-03 00:00:00' and my_time < '2010-07-10 00:00:00' group by dim5; +-----------------------+--------+--------+ | dim1 | summables | unique_ids | +=======================+========+========+ | test1 | 251 | 18 | | test2 | 48 | 11 | | test3 | 1 | 1 | | test4 | 2 | 1 | | test4 | 106 | 2 | +-----------------------+--------+--------+ 5 tuples (2.7s) +---------+----------------------------------------------------------------------------------------+ | ticks | stmt | +=========+========================================================================================+ | 3 | _5 := sql.mvc(); | | 20 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","account_ | : : id",2); : | 5 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","acc | : : ount_id",0); : | 25 | _13<tmp_1705>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat | : : [:oid,:lng][0]); : | 4 | _15:bat[:oid,:lng] <tmp_1664>[10] := sql.bind(_5=21685368,"sys","my_test_table","account | : : _id",1); : | 20 | _14<tmp_1706>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 4 | _18:bat[:oid,:oid] <tmp_1070>[0] := sql.bind_dbat(_5=21685368,"sys","my_test_table",1); | | 8 | _19<tmpr_1070>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 31627 | _51<tmp_1670>[0] := algebra.leftjoin(_38=<tmpr_1665>[0],_50=nil); | | 20 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","my_time",0); : | 10 | _23 := calc.timestamp(A1="2010-07-03 00:00:00"); | | 3 | _24 := calc.timestamp(A2="2010-07-10 00:00:00"); | | 7243 | (ext61<tmp_1710>[0],grp59<tmp_1661>[0]) := group.done(_51=<tmp_1670>[0]); | | 16 | _54<tmp_1665>[0] := bat.mirror(ext61=<tmp_1710>[0]); | | 47358 | _63<tmp_1356>[0] := algebra.leftjoin(_38=nil,_62=nil); | | 6 | _64<tmp_1356>[0] := algebra.selectNotNil(_63=nil); | | 13 | (_77<tmp_1710>[0],grp85) := group.derive(ext61=nil,grp59=<tmp_1661>[0],_76=<tmp_1651>[ | : : 0]); : | 16 | _79<tmp_1674>[0] := bat.mirror(_77=nil); | | 11 | _80<tmp_1710>[0] := algebra.join(_79=nil,_76=nil); | | 10 | _81:bat[:oid,:wrd] <tmp_1651>[0] := aggr.count(_80=nil,grp59=<tmp_1661>[0],_54=<tmp_16 | : : 65>[0]); : | 20 | _55<tmp_1711>[0] := algebra.join(_54=<tmp_1665>[0],_51=nil); | | 11 | _65:bat[:oid,:lng] <tmp_1674>[0] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 11 | _82 := sql.resultSet(3,1,_55=<tmp_1711>[0]); | | 9 | sql.rsColumn(_82=235,"sys.my_test_table","dim5","varchar",50,0,_55=nil); | | 5 | sql.rsColumn(_82=235,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 4 | sql.rsColumn(_82=235,"sys.my_test_table","unique_ids","wrd",64,0,_81=nil:bat[:oid,:wrd]); | | 2 | _99 := io.stdout(); | | 30 | sql.exportResult(_99=24880032,_82=235); | | 420626 | _17<tmp_1705>[8000010] := algebra.kunion(_14=nil,_15=nil:bat[:oid,:lng]); | | 17 | _20<tmp_1665>[8000010] := algebra.kdifference(_17=nil,_19=<tmpr_1070>[0]); | | 13 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=21685368,"sys","my_test_table","my | : : _time",2); : | 5 | _31:bat[:oid,:timestamp] <tmp_1704>[10] := sql.bind(_5=21685368,"sys","my_test_table","m | : : y_time",1); : | 13 | _32<tmp_1673>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00: | : : 00.000,_24=2010-07-10 00:00:00.000,true,false); : | 8 | _39:bat[:oid,:str] <tmp_1364>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","dim5",0); : | 4 | _44:bat[:oid,:str] <tmp_1103>[0] := sql.bind(_5=21685368,"sys","my_test_table","dim5",2); : | 7 | _46<tmp_1661>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:str],_44=<tmp_1103>:ba | : : t[:oid,:str][0]); : | 16 | _47<tmp_1706>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:str]); | | 3 | _5 := sql.mvc(); | | 4 | _48:bat[:oid,:str] <tmp_1631>[10] := sql.bind(_5=21685368,"sys","my_test_table","dim5",1); : | 21 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","account_ | : : id",2); : | 5 | _15:bat[:oid,:lng] <tmp_1637>[10] := sql.bind(_5=25090760,"sys","my_test_table","account | : : _id",1); : | 4 | _18:bat[:oid,:oid] <tmp_1070>[0] := sql.bind_dbat(_5=25090760,"sys","my_test_table",1); | | 3 | _19<tmpr_1070>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 5 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","my_time",0); : | 8 | _23 := calc.timestamp(A1="2010-07-27 00:00:00"); | | 3 | _24 := calc.timestamp(A2="2010-08-03 00:00:00"); | | 31883 | _25<tmp_1712>[391659] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-07-27 0 | : : 0:00:00.000,_24=2010-08-03 00:00:00.000,true,false); : | 11 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=25090760,"sys","my_test_table","my | : : _time",2); : | 21 | _28<tmp_1674>[391659] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestam | : : p][0]); : | 33 | _29<tmp_1714>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-07-27 00:00: | : : 00.000,_24=2010-08-03 00:00:00.000,true,false); : | 11 | _30<tmp_1713>[391659] := algebra.kunion(_28=nil,_29=nil); | | 4 | _31:bat[:oid,:timestamp] <tmp_1660>[10] := sql.bind(_5=25090760,"sys","my_test_table","m | : : y_time",1); : | 12 | _32<tmp_1714>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-07-27 00:00: | : : 00.000,_24=2010-08-03 00:00:00.000,true,false); : | 10 | _33<tmp_1674>[391659] := algebra.kunion(_30=nil,_32=nil); | | 7 | _39:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",0); : | 4 | _44:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","unqiue_id",2); : | 6 | _46<tmp_1713>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:lng],_44=<tmp_1101>:ba | : : t[:oid,:lng][0]); : | 12 | _47<tmp_1715>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:lng]); | | 4 | _48:bat[:oid,:lng] <tmp_1657>[10] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",1); : | 366305 | _50<tmp_1713>[8000010] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:lng]); | | 14 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=25090760,"sys","my_test_table","summable", | : : 2); : | 17 | _59<tmp_1715>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:ba | : : t[:oid,:int][0]); : | 16 | _60<tmp_1716>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 4 | _61:bat[:oid,:int] <tmp_1362>[10] := sql.bind(_5=25090760,"sys","my_test_table","summable" | : : ,1); : | 5 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","acc | : : ount_id",0); : | 10 | _13<tmp_1717>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat | : : [:oid,:lng][0]); : | 298001 | _62<tmp_1715>[8000010] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 21 | _14<tmp_1716>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 374628 | _17<tmp_1717>[8000010] := algebra.kunion(_14=nil,_15=nil:bat[:oid,:lng]); | | 20 | _20<tmp_1716>[8000010] := algebra.kdifference(_17=nil,_19=<tmpr_1070>[0]); | | 15 | _34<tmp_1720>[391659] := algebra.kdifference(_33=nil,_19=nil); | | 65082 | _35<tmp_1674>[391659] := algebra.semijoin(_20=nil,_34=nil); | | 2960 | _36<tmp_1714>[0] := algebra.uselect(_35=nil,A0=239:lng); | | 12 | _37<tmp_1674>[0] := algebra.markT(_36=nil,0@0:oid); | | 5 | _38<tmpr_1674>[0] := bat.reverse(_37=nil); | | 58011 | _51<tmp_1651>[0] := algebra.leftjoin(_38=<tmpr_1674>[0],_50=nil); | | 26 | (ext61<tmp_1710>[0],grp59<tmp_1713>[0]) := group.done(_51=<tmp_1651>[0]); | | 5 | _54<tmp_1674>[0] := bat.mirror(ext61=<tmp_1710>[0]); | | 27086 | _63<tmp_1356>[0] := algebra.leftjoin(_38=nil,_62=nil); | | 24 | _55<tmp_1715>[0] := algebra.join(_54=<tmp_1674>[0],_51=<tmp_1651>[0]); | | 3 | _66<tmp_1651>[0] := algebra.selectNotNil(_51=nil); | | 11 | (_67<tmp_1710>[0],grp75) := group.derive(ext61=nil,grp59=<tmp_1713>[0],_66=<tmp_1651>[ | : : 0]); : | 10 | _69<tmp_1712>[0] := bat.mirror(_67=nil); | | 5 | _70<tmp_1710>[0] := algebra.join(_69=nil,_66=nil); | | 9 | _71:bat[:oid,:wrd] <tmp_1651>[0] := aggr.count(_70=nil,grp59=<tmp_1713>[0],_54=<tmp_16 | : : 74>[0]); : | 3 | _64<tmp_1356>[0] := algebra.selectNotNil(_63=nil); | | 11 | _65:bat[:oid,:lng] <tmp_1712>[0] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 1226556 | barrier _119 := language.dataflow(); | | 9 | _72 := sql.resultSet(3,1,_55=<tmp_1715>[0]); | | 8 | sql.rsColumn(_72=236,"sys.my_test_table","unique_id","bigint",64,0,_55=nil); | | 4 | sql.rsColumn(_72=236,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 3 | sql.rsColumn(_72=236,"sys.my_test_table","unique_ids","wrd",64,0,_71=nil:bat[:oid,:wrd]); | | 3 | _85 := io.stdout(); | | 37 | sql.exportResult(_85=24880032,_72=236); | | 2 | _5 := sql.mvc(); | | 18 | _11:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","account_ | : : id",2); : | 4 | _15:bat[:oid,:lng] <tmp_1637>[10] := sql.bind(_5=25090760,"sys","my_test_table","account | : : _id",1); : | 3 | _18:bat[:oid,:oid] <tmp_1070>[0] := sql.bind_dbat(_5=25090760,"sys","my_test_table",1); | | 3 | _19<tmpr_1070>[0] := bat.reverse(_18=nil:bat[:oid,:oid]); | | 6 | _21:bat[:oid,:timestamp] <tmp_1412>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","my_time",0); : | 8 | _23 := calc.timestamp(A1="2010-06-24 00:00:00"); | | 3 | _24 := calc.timestamp(A2="2010-07-01 00:00:00"); | | 18 | _6:bat[:oid,:lng] <tmp_1410>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","acc | : : ount_id",0); : | 678 | _13<tmp_1713>[8000000] := algebra.kdifference(_6=nil:bat[:oid,:lng],_11=<tmp_1101>:bat | : : [:oid,:lng][0]); : | 22 | _14<tmp_1674>[8000000] := algebra.kunion(_13=nil,_11=nil:bat[:oid,:lng]); | | 49964 | _25<tmp_1714>[372221] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-06-24 0 | : : 0:00:00.000,_24=2010-07-01 00:00:00.000,true,false); : | 14 | _27:bat[:oid,:timestamp] <tmp_1131>[0] := sql.bind(_5=25090760,"sys","my_test_table","my | : : _time",2); : | 21 | _28<tmp_1713>[372221] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestam | : : p][0]); : | 24 | _29<tmp_1720>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-06-24 00:00: | : : 00.000,_24=2010-07-01 00:00:00.000,true,false); : | 11 | _30<tmp_1712>[372221] := algebra.kunion(_28=nil,_29=nil); | | 4 | _31:bat[:oid,:timestamp] <tmp_1660>[10] := sql.bind(_5=25090760,"sys","my_test_table","m | : : y_time",1); : | 13 | _32<tmp_1720>[0] := algebra.uselect(_31=nil:bat[:oid,:timestamp],_23=2010-06-24 00:00: | : : 00.000,_24=2010-07-01 00:00:00.000,true,false); : | 10 | _33<tmp_1713>[372221] := algebra.kunion(_30=nil,_32=nil); | | 5 | _39:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",0); : | 5 | _44:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",2); : | 6 | _46<tmp_1712>[8000000] := algebra.kdifference(_39=nil:bat[:oid,:lng],_44=<tmp_1101>:ba | : : t[:oid,:lng][0]); : | 8 | _47<tmp_1717>[8000000] := algebra.kunion(_46=nil,_44=nil:bat[:oid,:lng]); | | 3 | _48:bat[:oid,:lng] <tmp_1657>[10] := sql.bind(_5=25090760,"sys","my_test_table","unique_id",1); : | 1381664 | _50<tmp_1661>[8000010] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:str]); | | 17 | _66:bat[:oid,:lng] <tmp_1413>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",0); : | 4 | _68:bat[:oid,:lng] <tmp_1101>[0] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",2); : | 19 | _69<tmp_1706>[8000000] := algebra.kdifference(_66=nil:bat[:oid,:lng],_68=<tmp_1101>:ba | : : t[:oid,:lng][0]); : | 17 | _70<tmp_1721>[8000000] := algebra.kunion(_69=nil,_68=nil:bat[:oid,:lng]); | | 4 | _71:bat[:oid,:lng] <tmp_1703>[10] := sql.bind(_5=21685368,"sys","my_test_table","unique_id",1); : | 405606 | _50<tmp_1712>[8000010] := algebra.kunion(_47=nil,_48=nil:bat[:oid,:lng]); | | 15 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=25090760,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=25090760,"sys","my_test_table","summable", | : : 2); : | 20 | _59<tmp_1717>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:ba | : : t[:oid,:int][0]); : | 17 | _60<tmp_1722>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 4 | _61:bat[:oid,:int] <tmp_1362>[10] := sql.bind(_5=25090760,"sys","my_test_table","summable" | : : ,1); : | 374062 | _74<tmp_1706>[8000010] := algebra.kunion(_70=nil,_71=nil:bat[:oid,:lng]); | | 16 | _56:bat[:oid,:int] <tmp_1366>[8000000] := sql.bind(_5=21685368,"sys","my_test_table","summable",0); : | 4 | _58:bat[:oid,:int] <tmp_1065>[0] := sql.bind(_5=21685368,"sys","my_test_table","summable", | : : 2); : | 18 | _59<tmp_1721>[8000000] := algebra.kdifference(_56=nil:bat[:oid,:int],_58=<tmp_1065>:ba | : : t[:oid,:int][0]); : | 16 | _60<tmp_1723>[8000000] := algebra.kunion(_59=nil,_58=nil:bat[:oid,:int]); | | 4 | _61:bat[:oid,:int] <tmp_1361>[10] := sql.bind(_5=21685368,"sys","my_test_table","summable" | : : ,1); : | 1884678 | _25<tmp_1707>[359554] := algebra.uselect(_21=nil:bat[:oid,:timestamp],_23=2010-07-03 0 | : : 0:00:00.000,_24=2010-07-10 00:00:00.000,true,false); : | 28 | _28<tmp_1724>[359554] := algebra.kdifference(_25=nil,_27=<tmp_1131>:bat[:oid,:timestam | : : p][0]); : | 30 | _29<tmp_1726>[0] := algebra.uselect(_27=nil:bat[:oid,:timestamp],_23=2010-07-03 00:00: | : : 00.000,_24=2010-07-10 00:00:00.000,true,false); : | 11 | _30<tmp_1725>[359554] := algebra.kunion(_28=nil,_29=nil); | | 10 | _33<tmp_1726>[359554] := algebra.kunion(_30=nil,_32=nil); | | 7 | _34<tmp_1725>[359554] := algebra.kdifference(_33=nil,_19=nil); | | 63572 | _35<tmp_1726>[359554] := algebra.semijoin(_20=nil,_34=nil); | | 2812 | _36<tmp_1673>[139] := algebra.uselect(_35=nil,A0=76:lng); | | 10 | _37<tmp_1726>[139] := algebra.markT(_36=nil,0@0:oid); | | 5 | _38<tmpr_1726>[139] := bat.reverse(_37=nil); | | 58840 | _75<tmp_1715>[139] := algebra.leftjoin(_38=<tmpr_1726>[139],_74=nil); | | 6 | _76<tmp_1715>[139] := algebra.selectNotNil(_75=nil); | | 21903 | _51<tmp_1711>[139] := algebra.leftjoin(_38=<tmpr_1726>[139],_50=nil); | | 62 | (ext61<tmp_1344>[5],grp59<tmp_1651>[139]) := group.done(_51=<tmp_1711>[139]); | | 11 | _54<tmp_1661>[5] := bat.mirror(ext61=<tmp_1344>[5]); | | 46 | _55<tmp_1570>[5] := algebra.join(_54=<tmp_1661>[5],_51=nil); | | 34 | (_77<tmp_1500>[33],grp85) := group.derive(ext61=nil,grp59=<tmp_1651>[139],_76=<tmp_171 | : : 5>[139]); : | 6 | _79<tmp_1706>[33] := bat.mirror(_77=nil); | | 24 | _80<tmp_1502>[33] := algebra.join(_79=nil,_76=nil); | | 22 | _81:bat[:oid,:wrd] <tmp_1706>[5] := aggr.count(_80=nil,grp59=<tmp_1651>[139],_54=<tmp_ | : : 1661>[5]); : | 554759 | _62<tmp_1717>[8000010] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 541977 | _62<tmp_1721>[8000010] := algebra.kunion(_60=nil,_61=nil:bat[:oid,:int]); | | 41224 | _63<tmp_1356>[139] := algebra.leftjoin(_38=nil,_62=nil); | | 10 | _64<tmp_1356>[139] := algebra.selectNotNil(_63=nil); | | 48 | _65:bat[:oid,:lng] <tmp_1721>[5] := aggr.sum(_64=nil,grp59=nil,_54=nil); | | 2761640 | barrier _137 := language.dataflow(); | | 12 | _82 := sql.resultSet(3,1,_55=<tmp_1570>[5]); | | 8 | sql.rsColumn(_82=3,"sys.my_test_table","dim5","varchar",50,0,_55=nil); | | 5 | sql.rsColumn(_82=3,"sys.my_test_table","summables","bigint",32,0,_65=nil:bat[:oid,:lng]); | | 6 | sql.rsColumn(_82=3,"sys.my_test_table","unique_ids","wrd",64,0,_81=nil:bat[:oid,:wrd]); | | 3 | _99 := io.stdout(); | | 45 | sql.exportResult(_99=14711568,_82=3); | | 2761973 | user.s4_1(76:lng,"2010-07-03 00:00:00","2010-07-10 00:00:00"); | +---------+----------------------------------------------------------------------------------------+

1 0

[MonetDB-users] performance degrades when date related function is used
by Ramkumarlak 03 Feb '11

03 Feb '11

Hi, I am looking for monerdb to replace mysql. I tried some queries with my test data and all were much more faster than mysql. But our actual table contains a column which stores date in unix time format and we do generate reports based on week, day, quarter, month and year. For that we are using a lot mysql date related functions. I too tried the following approach for one of my requirements. cast(timestamp '1970-01-01 00:00:00' + cast(created_date/1000 as interval second) as date) but it takes longer time. 1) Is there any other way to make this query faster ? 2) what are all the options are available in monetdb to deal these (week,quarter,hour,day and year wise reporting) kind of queries. Any help will be appreciated. Thanks and Regards Ram -- View this message in context: http://old.nabble.com/performance-degrades-when-date-related-function-is-us… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] query slowdown with concurrent write
by Bennett Hiles 03 Feb '11

03 Feb '11

Hi, MonetDB has great response times for the queries I have been running, but once I start inserting data to the table and concurrently running the queries, the queries slow down. To give you concrete numbers, I am initializing the database with 8 million records. I then run random aggregation queries repeatedly. Before I insert additional data the max query time is 0.3 seconds. If I then insert 10 rows the queries slow to a max of 1.5s. The only solution I have to getting the queries to resume their original speed is to drop the data and reload the 8 millions rows into the table. I have also noticed that if I run my inserts while I am not concurrently making queries to the database, I don't run into the slowness, and the queries keep their 0.3s response time. Has anyone else encountered this problem before? Is there a way around it? I have also tried using the copy command instead of inserts but get the same result. Thanks, Bennett

3 2

[MonetDB-users] MAL
by Michael Kusber 02 Feb '11

02 Feb '11

Hello guys, I aplogize for bothering you with what might be a simple issue for you, but I am new to monetDB and I have to solve some complex problems for which I cannot find a solution by myself. I've read about MAL in the M5 reference manual. I recognized, the normal way to execute commands is via MAPI and JDBC and stuff based on MAPI. This will be parsed into MAL, optimized and executed. Is there a way to write MAL code and give it as input to the server directly?? Another question: Is it possible to explicitely invoke execution of a query in multiple threads via MAPI or other means? Kind regards, michael

3 2

[MonetDB-users] After delete records with a field criteria, still shows in group by SQL
by Osmin Castillo 02 Feb '11

02 Feb '11

Hi there. I'm having this extrange problem with MonetDB. After delete some records with a criteria for example: field_id = 201011 The records are deleted, but when I use a select group by, the count(0) shows results for field_id = 201011 Thanks in advance

1 0

[MonetDB-users] collation - sorting order
by Ramkumarlak 28 Jan '11

28 Jan '11

Hi, Can we control the sorting order in MonetDB? Is there any way we can specify collation ...? Regards Ram -- View this message in context: http://old.nabble.com/collation---sorting-order-tp30786899p30786899.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] date functions in monetdb
by Ramkumarlak 27 Jan '11

27 Jan '11

Hi, Let me know whether the following MYSQL functions are available in MonetDB. 1) concat() 2) date() 3) from_unix_time() 4) interval() 5) Weekday() If not, are there any equivalent functions available ? Regards Ram -- View this message in context: http://old.nabble.com/date-functions-in-monetdb-tp30775963p30775963.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

[MonetDB-users] Performance question between schema
by brien colwell 27 Jan '11

27 Jan '11

With Monet DB I've encountered a counter- intuitive performance characteristic. I have a dataset of 33554432 (2^25) triples, (a, b, c). The values a and b are uniformly distributed on [0, 16), and c is unique to each triple, on [0, 2^25). I store them in two ways: Schema 1: One table with three columns (a, b, c) Schema 2: Two tables with two columns each, (a, c) and (b, c) I'm running SQL queries to select the "c" value from the triples based on a=X and b=Y. With schema 1, this is a single select. With schema 2, this is an inner join between two selects. What surprised me is that the query against schema 2 is on average >50 ms faster to return than the query against schema 1. To me this was counter- intuitive since schema 1 stores less data, and the data are pre-joined (in a row-oriented sense) for the type of filter applied. However, I'm guessing this has to do with how MonetDB stores data in columns. Is the query against schema 1 actually doing three joins, "a" -> "b" -> "c", on an artificial join key? For example, -- Schema 1) create table abc (a bigint, b bigint, c bigint); -- Schema 2) create table ac (a bigint, c bigint); create table bc (b bigint, c bigint); copy 33554432 records into abc from '/home/hadoop/abc.monetdb'; copy 33554432 records into ac from '/home/hadoop/ac.monetdb'; copy 33554432 records into bc from '/home/hadoop/bc.monetdb'; -- Query Schema 1) select abc.c from abc where abc.a=3 and abc.b=7 order by abc.c asc -- Times to return (JDBC) -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.287, SQL query: 0.287, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.288, SQL query: 0.288, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.35, SQL query: 0.35, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.372, SQL query: 0.372, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.405, SQL query: 0.405, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.285, SQL query: 0.285, Building output: 0 -- Query Schema 2) select ac.c from ac inner join bc on bc.b=7 and bc.c = ac.c where ac.a = 3 order by ac.c asc -- Times to return (JDBC) -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.218, SQL query: 0.218, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.216, SQL query: 0.216, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.318, SQL query: 0.318, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.322, SQL query: 0.322, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.218, SQL query: 0.218, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.219, SQL query: 0.219, Building output: 0 -- Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.222, SQL query: 0.222, Building output: 0

2 1

[MonetDB-users] "similar to" text matching syntax
by brien colwell 25 Jan '11

25 Jan '11

I came across a commit for a "similar to" syntax for text columns -- http://www.mail-archive.com/checkin-list@monetdb.org/msg01289.html Very interested. Are there any updates on this commit -- stable, not stable, not tested? Does MonetDB publish nightly builds?

2 2

[MonetDB-users] quer cache
by Ramkumarlak 24 Jan '11

24 Jan '11

Hi, Does monetdb have query cache ? If so, how do we disable it ? I searched in monetdb5.cong but i did not find any option. Regards Ram -- View this message in context: http://old.nabble.com/quer-cache-tp30747505p30747505.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 3

[MonetDB-users] copy select
by Nafees Ur Rehman 19 Jan '11

19 Jan '11

Hi, With executing the statement below: COPY SELECT * FROM adv LIMIT 10 INTO 'D:/exp.txt' I get the following error: !TypeException: user.s0_1[4703]:'aggr.count' undefined in : tuples:wrd :=aggr.count(_4695:str) !SQLException:SQLengine:Program contains errors What am I doing wrong? Thanks Nafees

1 0

[MonetDB-users] ODBC error 'Function Sequence Error' occurredwhen loaded TPC-H data of MonetDB
by JinBinge 16 Jan '11

16 Jan '11

Hi, Benchmark Factory for Database was used to test TPC-H of MonetDB. Errors occurred when creating objects for TPC-H option was selected,such as ODBC error 'Function Sequence Error'. And tables h_nation,h_region,h_supplier' data were loaded successful,the other five tables failed to load data. follows are the test configuration: OS: Linux Hardware: amd64 MonetDB version: Oct2010 Benchmark Factory for Database version: v5.7.1g.504 load data'SF=1G Are there errors in ODBC of MonetDB? Yours sincerely Binge Jin

2 1

[MonetDB-users] Query Error Question
by Brett Spurrier 14 Jan '11

14 Jan '11

Hi all, I have yet another query that I am having trouble running with Monet, and I am hoping a seasoned expert can help me figure out what is going on here. My query is: SELECT fid, title, D0 AS ed FROM signatures t WHERE fid = ( SELECT fid FROM signatures ti WHERE ti.guid = t.guid ORDER BY D0 ASC LIMIT 1 OFFSET 1) ORDER BY D0 ASC, fid DESC; but I am getting the error: "!syntax error, unexpected ORDER, expecting INTERSECT or EXCEPT or UNION or ')' in: "select fid, title, d0 as ed from signatures t where fid = ( select fid from signatures ti where ti.guid = t.guid order" So it appears to not like the ORDER BY term, and I can't figure out why. I am trying to grab the second element (after ordering by D0) based on distinct GUIDs. The query iteself is based on the answer for a question from stackoverflow that I posted: http://stackoverflow.com/questions/4634068/sql-query-with-order-by-part-2 Can anyone help me figure this out? The syntax looks fine to me. By the way, I have "D0 AS ed" because D0 will eventually be replaced with a simple math operator (i.e., D0/D1). Many thanks, Brett

4 5

[MonetDB-users] An error about MonetDB' TPCH power test
by 金炳娥 14 Jan '11

14 Jan '11

Hi all, Recently I used the Benchmark Factory tool to test TPCH of MonetDB(SF=100). When running power test,there is an error,such as follows: Started test TPC-H Power Test Agent(bejin) Error: ODBC Error (HY000,0) - [MonetDB][ODBC Driver 1.0]SQLException:convert:too many digits (21 > 19) Do you know the reason and how to reslove the problem? Looking forwards to yours answer. Yours sincerely Binge Jin

1 0

[MonetDB-users] Add primary key performance
by paulr135 13 Jan '11

13 Jan '11

I've loaded a 27GB (~165M rows) file into a table in MonetDB and now I am trying to add a primary key which consists of 4 fields - 2 varchars and 2 ints. It has been running for a few days (since Friday of last week). I've followed the instructions laid out on this post: http://old.nabble.com/MonetDB-SQL:-bulk-loading-large-data-sets---adding-ke… which involves setting the optimizer variable to "nov2009_pipe". I am unable to log in via mclient - I am able to put in my credentials but it just hangs at the "Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2010-SP2)" line. So I have a few questions: 1. Is there any way I can see the status of the alter table ... add primary key command? 2. Is it possible to stop the alter table gracefully? Or will I have to shut the db down? 3. Is there anything I should do in future to speed up adding of primary keys? Thanks. -- View this message in context: http://old.nabble.com/Add-primary-key-performance-tp30397515p30397515.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] Consultation about TPCH throughput test errors with MonetDB
by 金炳娥 12 Jan '11

12 Jan '11

Hi, Recently I has a problem when running TPCH with MonetDB. Follows are the configuration. Hardware:amd64 OS:Linux MonetDB Version:Oct2010 I loaded 100G' data to the MonetDB.Using TPCH Tester tools,I runned power test.After power test successed,running throughput test. There is an error. ERR TPCH[17170]: *** glibc detected *** /mnt/sdb1/tpch/MonetDB/install/bin/mserver5: free(): invalid pointer: 0x00002ac5381b5008 *** 2011-01-04 14:00:41 ERR TPCH[17170]: ======= Backtrace: ========= 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libc.so.6[0x3aa5c71ce2] 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libc.so.6(cfree+0x8c)[0x3aa5c7590c] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1[0x2b67e8c062f6] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1(HEAPfree+0xb9)[0x2b67e8b36511] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1(BATdelete+0x196)[0x2b67e8e1769a] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1[0x2b67e8b33697] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libbat.so.1[0x2b67e8b32fa8] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libmonetdb5.so.5[0x2b67e7f1da68] 2011-01-04 14:00:41 ERR TPCH[17170]: /mnt/sdb1/tpch/MonetDB/install/lib/libmonetdb5.so.5[0x2b67e7f1f6f1] 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libpthread.so.0[0x3aa6806367] 2011-01-04 14:00:41 ERR TPCH[17170]: /lib64/libc.so.6(clone+0x6d)[0x3aa5cd30ad] 2011-01-04 14:00:41 ERR TPCH[17170]: ======= Memory map: ======== 2011-01-04 14:00:41 ERR TPCH[17170]: 00400000-00404000 r-xp 00000000 08:11 179159655 /mnt/sdb1/tpch/MonetDB/install/bin/mserver5 2011-01-04 14:00:41 ERR TPCH[17170]: 00604000-00605000 rw-p 00004000 08:11 179159655 /mnt/sdb1/tpch/MonetDB/install/bin/mserver5 2011-01-04 14:00:41 ERR TPCH[17170]: 00605000-00612000 rw-p 00605000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 0a03f000-0e488000 rw-p 0a03f000 00:00 0 [heap] 2011-01-04 14:00:41 ERR TPCH[17170]: 4001c000-4001d000 ---p 4001c000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 4001d000-4021d000 rw-p 4001d000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 40287000-40288000 ---p 40287000 00:00 0 … 2011-01-04 14:00:41 ERR TPCH[17170]: 4bc66000-4be66000 rw-p 4bc66000 00:00 0 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5800000-3aa581c000 r-xp 00000000 fd:00 95158276 /lib64/ld-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5a1b000-3aa5a1c000 r--p 0001b000 fd:00 95158276 /lib64/ld-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5a1c000-3aa5a1d000 rw-p 0001c000 fd:00 95158276 /lib64/ld-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5c00000-3aa5d4c000 r-xp 00000000 fd:00 95158283 /lib64/libc-2.5.so 2011-01-04 14:00:41 ERR TPCH[17170]: 3aa5d4c000-3aa5f4c000 ---p 0014c000 fd:00 95158283 /lib64/libc-2.5.so The test data are correct. Do you know the reason above errors? I hope to have a letter from you before long.With every good wish. Yours sincerely， Binge Jin

2 1

[MonetDB-users] Concurrent aggregation crashes MonetDB
by Dávid Karnok 06 Jan '11

06 Jan '11

Hi! I'm running an evaluation test on the latest MonetDB/SQL stable version and always crashes possibly due an out-of-memory problem. My system: Windows 7 x86, 4GB memory, Core i7 920, data files are on an SSD disk MonetDB (default install): # MonetDB server v5.22.1, based on kernel v1.40.1 # Release Oct2010 # Serving database 'demo', using 8 threads # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Found 3.186 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\Program Files\MonetDB\MonetDB5\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.40.1 loaded Using monetdb-1.17-jdbc driver. The test involves a simple, two column table: CREATE TABLE a (text VARCHAR(50), value INT) Which is then filled with 10.000.000 records of random values. The data files take up to 150MB disk space. The test routine creates multiple threads which connect to the local demo database, and simultaneously execute the same aggregation query. The test simulates a concurrent analysis-data mining scenario running on the same data. void parallelTest(int threadCount) { ExecutorService exec = Executors.newFixedThreadPool(threadCount); final CountDownLatch cnt = new CountDownLatch(threadCount); for (int i = 0; i < threadCount; i++) { final int j = i; exec.submit(new Runnable() { @Override public void run() { try { Class.forName("nl.cwi.monetdb.jdbc.MonetDriver"); Connection conn = DriverManager.getConnection("jdbc:monetdb://localhost/demo", "monetdb", "monetdb"); try { conn.setAutoCommit(false); long t1 = System.currentTimeMillis(); Statement stmt = conn.createStatement(); try { cnt.countDown(); cnt.await(); ResultSet rs = stmt.executeQuery("SELECT text, AVG(value) FROM a GROUP BY text "); while (rs.next()) { rs.getString(1); rs.getDouble(2); } System.out.printf("[%02d] Select time: %.2fs%n", j, (System.currentTimeMillis() - t1) / 1000.0); } finally { stmt.close(); } } finally { conn.close(); } } catch (Exception ex) { ex.printStackTrace(); } } }); } exec.shutdown(); try { exec.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS); } catch (InterruptedException ex) { } } I noticed that if the threadCount >= 4 the MonetDB console starts to print VirtualAlloc failed messages and sooner or later terminates. The memory usage seems to jump by 600MB, e.g. with 3 threads it consumes 1.7 GB memory, but with 4 threads, it can't consume 2.4GB. Replacing the query with a single result "SELECT MAX(VALUE) FROM a" causes the same crash after 5 threads. The same test running with the latest MySQL works perfectly, although slowly. No crashes or memory issues. Is this a memory leak in MonetDB or the result of its architecture, meaning I should use a 64bit system with lots of RAM? Regards, David Karnok

1 0

[MonetDB-users] MonetDB LIMIT usage
by Brett Spurrier 05 Jan '11

05 Jan '11

Hi all, I am trying to use the LIMIT clause in my query, and while this should be fairly straight froward, I am having trouble learning the differences between Monet's LIMIT and other T-sql LIMITS. I have the simple query: SELECT * FROM signatures ORDER BY scale ASC LIMIT 1; which works great and as expected. However, I really want the second record, so in other databases I can use: SELECT * FROM signatures ORDER BY scale ASC LIMIT 1, 1; I have also tried: SELECT * FROM signatures ORDER BY scale ASC LIMIT 1 1; However, these both fail in MonetDB. According to: http://monetdb.cwi.nl/SQL/Documentation/Queries.html#Queries it looks like it should be supported. Does anyone have any insight as to how to grab the second record in MonetDB (I.e., limiting the record results to one with an offset of one)??? I am at a loss here. Cheers, Brett

3 4

[MonetDB-users] Issues with commandline copy into and opening files
by Lex Slaghuis 03 Jan '11

03 Jan '11

Hi All Eversince I updated my system; The following blockbased import script doesn't work anymore: #!/bin/bash BLOCKSIZE=250000 #RECORDCOUNT L=$(wc -l $1 | awk '{ print $1}'); #MERGE BLOCKS INTO TABLE for I in $(seq 0 $BLOCKSIZE $L); do echo "$BLOCKSIZE $I $1/$L loading... "; mclient --time -d test --statement="COPY $BLOCKSIZE OFFSET $I records into \ wptool.wpstats from '$1' using delimiters ' ','\n','' null as '';"; echo "$BLOCKSIZE $I $1/$L loading...finished "; done Returns the following error (it used to work): IOException:streams.open:Could not open file ERROR: mnstr_open: could not open file '/var/www/loadstats/pagecounts-20110102-160000' The script is started through: sh importintodb.sh /var/www/loadstats/pagecounts-20110102-160000 Does anyone has some thoughts on this, or should I report a bug? I would really like to use commandline, as the hourly imports are 6M records big... The .monetdb file is placed locally and automatically authenticates with monetdb (admin). Lang=SQL, history=yes Regards Lex Slaghuis

2 1

[MonetDB-users] Monnetdb/Xquery
by Ramachandran, Prakash 30 Dec '10

30 Dec '10

Hi, I am currently evaluating software's for metadata storage that can be used in our product. I came across your product and found it very interesting and we might consider using it for our metadata storage. I just wanted to know if Monnetdb provides commercial support, if so can you please provide the detail on sales contacts. Thank in advance. Regards, Prakash R

1 0

[MonetDB-users] MAPI interface error
by Brett Spurrier 24 Dec '10

24 Dec '10

Hi all, I am having a bit of trouble and I am hoping a seasoned MonetDB expert can lend some advice. I am using the MAPI interface in my C/C++ application to talk to my MonetDB instance. I have a recursive INSERT INTO statement which populates the database, and it initially works pretty well. However, as I populate more (past 200 inserts in my case), I get a MAPI error (exampled by: mapi_explain_result()): MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !Connection terminated No real reason as to why. Has anyone seen this before? My code: MapiHdl query(Mapi dbh, char *q) { MapiHdl ret = NULL; if ((ret = mapi_query(dbh, q)) == NULL || mapi_error(dbh) != MOK) die(dbh, ret); return(ret); } void update(Mapi dbh, char *q) { MapiHdl ret = query(dbh, q); if (mapi_close_handle(ret) != MOK) die(dbh, ret); } void main() { dbh = mapi_connect("localhost", 50000, "monetdb", "monetdb", "sql", "dbname"); if (mapi_error(dbh)) { die(dbh, hdl); } for (int i=0; i< 2500; i++) { char *p = INSERT_INTO_STATEMENT; update(dbh, p); } } Does anyone see any reason that the MAPI would choke mid way through the For loop? Any help would be majorly appreciated. I am at a loss. Cheers, Brett

3 14

[MonetDB-users] Announcement: New Jun2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 21 Dec '10

21 Dec '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2010-SP1 bug fix release of the MonetDB suite of programs. The most important and visible change is a security fix in the SQL COPY INTO command. This is detailed in the release notes. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Oct2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] How to limit monetdb memory consumption on Windows
by Bianco Prieto-Ilarion 18 Dec '10

18 Dec '10

Hi all! I have a problem. I'm using Oct2010 64-bit MonetDB build. I have a table with 8M records and when I'm trying to execute LEFT JOIN (from JDBC) MonetDB is crashed. I can see that it crashed due to memory (my laptop has 8Gb) consumption is unlimited. What can you suggest (just use Linux is a final step:)) Thanks, Bina Prieto

1 0

[MonetDB-users] MonetDB Installation
by Rob Fynes 16 Dec '10

16 Dec '10

Hello, Regarding the pre-compiled binaries directories for deb distributions "karmic" and "lucid" in the downloads section, there doesn't appear to be anything for x-query. Is this intentional? Regards, -Rob

2 2

[MonetDB-users] mserver5 ignores ulimit?
by Siva Manoharan 14 Dec '10

14 Dec '10

Hi. I am currently running monetdb (release Oct2010) on Debian etch, with 8GB physical RAM and 10GB virutal memory. During unrestricted operation (using the merovingian daemon) monetdb takes up a significant majority of the resources, which is not desirable as there are other processes running on the same server. To limit the memory, I tried using ulimit (setting -d, -v and -m all to 2GB). According to the logs, merovingian notices the limit (a line from log states "Found 1.907 GiB available main-memory"). However, memory usage of mserver still exceeds that limit and is killed off by the OS, abandoning an ongoing query. (merovingian is not killed and mserver restarts without any problems). Is this expected behaviour or is it a bug? Is there another way to limit monetdb's memory usage? As a note, none of the tables (and therefore none of the columns) are anywhere near 2GB in size. The biggest table has ~2.5M rows, where each row has 2 columns ( 8byte bigint and 4 byte int). However, there are thousands of tables, and mserver exceeds the limit most often when a large number of tables are being updated within a short time (in a single php process, however). Is it possible that mserver doesn't release memory fast enough to allow multiple tables to be accessed? Thanks.

1 0

[MonetDB-users] crashes with copy into (270MB file)
by Lex Slaghuis 13 Dec '10

13 Dec '10

Hello I Experience crashes with the copy into statement, with large files (270MB utf8) ... sql>COPY 6000000 records into wptool.wpstats from '/var/www/pagecounts' using delimiters ' ','\n','' null as ''; MAPI = testuser@localhost:50000 ACTION= read_line QUERY = COPY 6000000 records into wptool.wpstats from '/var/www/pagecounts' using delimiters ' ','\n','' null as ''; ERROR = !Connection terminated Timer 5846.447 msec...... Number of records: wc -l /var/www/pagecounts 5983201 /var/www/pagecounts My memory: free -m(egabytes) total used free shared buffers cached Mem: 2007 431 1576 0 20 304 -/+ buffers/cache: 106 1900 Swap: 16769 0 16769 Error-logs: 2010-12-13 14:17:41 MSG merovingian[902]: proxying client localhost:52737 for database 'test' to mapi:monetdb:///var/MonetDB5/dbf arm/test/.mapi.sock?database=test 2010-12-13 14:17:41 MSG merovingian[902]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2010-12-13 14:18:30 MSG merovingian[902]: database 'test' (1142) was killed by signal SIGSEGV 2010-12-13 14:18:38 ERR control[902]: (local): received stop signal for non running database: test 2010-12-13 14:18:46 MSG merovingian[902]: database 'test' has crashed after start on 2010-12-13 14:17:33, attempting restart, up min/avg/max: 14s/22h/2d, crash average: 1.00 0.70 0.23 (10-3=7) Some specs: uname -a Linux ubuntu 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux 2010-12-13 14:17:33 MSG test[1142]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2010-12-13 14:17:33 MSG test[1142]: # Found 1.961 GiB available main-memory. If anyone has some pointers on this , it would really help out. I think it would be really to chop up the files into smaller pieces, just for import. Regards Lex Slaghuis -----Original Message----- From: monetdb-users-request(a)lists.sourceforge.net [mailto:monetdb-users-request@lists.sourceforge.net] Sent: vrijdag 10 december 2010 11:12 To: monetdb-users(a)lists.sourceforge.net Subject: MonetDB-users Digest, Vol 55, Issue 2 Send MonetDB-users mailing list submissions to monetdb-users(a)lists.sourceforge.net To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/monetdb-users or, via email, send a message with subject or body 'help' to monetdb-users-request(a)lists.sourceforge.net You can reach the person managing the list at monetdb-users-owner(a)lists.sourceforge.net When replying, please edit your Subject line so it is more specific than "Re: Contents of MonetDB-users digest..." Today's Topics: 1. COPY INTO: wrong length with backslashed chars? (Daniel Boesswetter) 2. Re: COPY INTO: wrong length with backslashed chars? (Stefan de Konink) 3. Some questions regarding parallelism and other stuff... (Mike De-La-Columnar) 4. Re: Some questions regarding parallelism and other stuff... (Martin Kersten) 5. Add primary key performance (paulr135) 6. Re: Add primary key performance (Stefan Manegold) 7. Re: COPY INTO: wrong length with backslashed chars? (Daniel Boesswetter) ---------------------------------------------------------------------- Message: 1 Date: Sun, 05 Dec 2010 19:08:01 +0100 From: Daniel Boesswetter <daniel.boesswetter(a)fu-berlin.de> Subject: [MonetDB-users] COPY INTO: wrong length with backslashed chars? To: monetdb-users(a)lists.sourceforge.net Message-ID: <4CFBD501.9030709(a)fu-berlin.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi all, I'm using MonetDB for the analysis for ~90GB of data which come from a MySQL-based web-application. The data is dumped into CSV-files on a regular basis and then imported into MonetDB with the COPY command. I have experienced some strange behaviour: obviously, MonetDB counts the number of chars for a VARCHAR-field before removing backslashes which are used e.g. for escaping newlines or other backslashes and rejects strings that should actually fit. A simple reproduction is this: 1. In mclient: CREATE TABLE t (x VARCHAR(1)); 2. Create a textfile foo.txt containing two backslashes: \\ 3. In mclient: sql>copy into t from '/tmp/foo.txt'; SQLException:importTable:value '\\' from line 1 field 1 not inserted, expecting type varchar(1) failed to import table Performing exactly the same steps with a VARCHAR(2) column works without errors and results in t.x containg the de-escaped single backslash. I'm using the Oct-2010 release. My current workaround is to enlarge VARCHARs which caused problems in the past, but this is of course no guarantee that it won't happen again. Although I found a few monetdb-bugs entries concerning the COPY command (e.g. about the misleading error message seen above), none seems to deal with my problem. Is this a bug? Best regards, Daniel ------------------------------ Message: 2 Date: Sun, 05 Dec 2010 19:15:36 +0100 From: Stefan de Konink <stefan(a)konink.de> Subject: Re: [MonetDB-users] COPY INTO: wrong length with backslashed chars? To: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Message-ID: <4CFBD6C8.30208(a)konink.de> Content-Type: text/plain; charset=ISO-8859-1 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 05-12-10 19:08, Daniel Boesswetter schreef: > Hi all, > > I'm using MonetDB for the analysis for ~90GB of data which come from a > MySQL-based web-application. The data is dumped into CSV-files on a > regular basis and then imported into MonetDB with the COPY command. I > have experienced some strange behaviour: obviously, MonetDB counts the > number of chars for a VARCHAR-field before removing backslashes which > are used e.g. for escaping newlines or other backslashes and rejects > strings that should actually fit. > > A simple reproduction is this: > > 1. In mclient: CREATE TABLE t (x VARCHAR(1)); 2. Create a textfile > foo.txt containing two backslashes: \\ 3. In mclient: > sql>copy into t from '/tmp/foo.txt'; > SQLException:importTable:value '\\' from line 1 field 1 not inserted, > expecting type varchar(1) failed to import table > > Performing exactly the same steps with a VARCHAR(2) column works > without errors and results in t.x containg the de-escaped single backslash. > > I'm using the Oct-2010 release. My current workaround is to enlarge > VARCHARs which caused problems in the past, but this is of course no > guarantee that it won't happen again. > > Although I found a few monetdb-bugs entries concerning the COPY > command (e.g. about the misleading error message seen above), none > seems to deal with my problem. > > Is this a bug? What are you escaping? Paste the line that fails ;) Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkz71sgACgkQYH1+F2Rqwn3AmgCfayhLg6zjXZgvqhV9OHJbj5Kh z64An2xZJy7H1wv3M6hr/SHQaK2C8IM3 =//V/ -----END PGP SIGNATURE----- ------------------------------ Message: 3 Date: Mon, 6 Dec 2010 05:07:46 -0800 (PST) From: Mike De-La-Columnar <mikevermondusa(a)gmail.com> Subject: [MonetDB-users] Some questions regarding parallelism and other stuff... To: monetdb-users(a)lists.sourceforge.net Message-ID: <30366715.post(a)talk.nabble.com> Content-Type: text/plain; charset=us-ascii Hi fellas, If its not too much of a hassle, i have a few questions regarding parallelism & the internal algorithms 1. What is the difference between Data-Flow, and Mitosis? 2. Is there a special reason why the distinct count aggregation is not parallelized just like the other aggregation functions? Is it just a matter of not making it into the release, or is there a fundamental problem here. Note: even though the parallel distinct can be slower in certain distributions, it could still be useful... 3. Are there any plans on supporting parallel frameworks like OpenMP/Intel TBB directly within the low-level MAL functions? Providing data-parallelism on the loop-code level could even increase performance. Or did you already play with it and decided that the performance gain is just not worth it? 4. Is there a way for me to prevent MonetDB server from cleaning up all intermediate BATs between server restarts. I've read here that some intermediates (like join indices etc.) are lost between restarts... is it still correct? Thank you very much for your time, Keep up the amazing job of building the most powerful columnar database out there! -- View this message in context: http://old.nabble.com/Some-questions-regarding-parallelism-and-other-stuff.… Sent from the monetdb-users mailing list archive at Nabble.com. ------------------------------ Message: 4 Date: Mon, 06 Dec 2010 20:13:55 +0100 From: Martin Kersten <Martin.Kersten(a)cwi.nl> Subject: Re: [MonetDB-users] Some questions regarding parallelism and other stuff... To: monetdb-users(a)lists.sourceforge.net Message-ID: <4CFD35F3.1060001(a)cwi.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed On 12/6/10 2:07 PM, Mike De-La-Columnar wrote: > > Hi fellas, Hi Mike > > If its not too much of a hassle, i have a few questions regarding > parallelism& the internal algorithms > > 1. > What is the difference between Data-Flow, and Mitosis? See optimizer descriptions: http://monetdb.cwi.nl/MonetDB/Documentation/Optimizer-Toolkit.html Orthogonal issues. > > 2. > Is there a special reason why the distinct count aggregation is not > parallelized just like the other aggregation functions? > Is it just a matter of not making it into the release, or is there a > fundamental problem here. Distinct is a global property, not a local one. A distributed version may easily lead to throwing the complete underlying table around. > > Note: even though the parallel distinct can be slower in certain > distributions, it could still be useful... > > 3. > Are there any plans on supporting parallel frameworks like OpenMP/Intel TBB > directly within the low-level MAL functions? > Providing data-parallelism on the loop-code level could even increase > performance. It could improve, but there should be a clear driving application. > Or did you already play with it and decided that the performance gain is > just not worth it? Depends on the amount of work per byte unit. > > 4. > Is there a way for me to prevent MonetDB server from cleaning up all > intermediate BATs between server restarts. > I've read here that some intermediates (like join indices etc.) are lost > between restarts... is it still correct? All temporaries are indeed lost. Join indices for foreign key checking are retained on disk. Rebuilding hashes is cheaper (mostly) then write/read them from disk > > > Thank you very much for your time, > Keep up the amazing job of building the most powerful columnar database out > there! Thanks for the support !. Martin > > > ------------------------------ Message: 5 Date: Tue, 7 Dec 2010 07:44:14 -0800 (PST) From: paulr135 <romesh135(a)gmail.com> Subject: [MonetDB-users] Add primary key performance To: monetdb-users(a)lists.sourceforge.net Message-ID: <30397515.post(a)talk.nabble.com> Content-Type: text/plain; charset=us-ascii I've loaded a 27GB (~165M rows) file into a table in MonetDB and now I am trying to add a primary key which consists of 4 fields - 2 varchars and 2 ints. It has been running for a few days (since Friday of last week). I've followed the instructions laid out on this post: http://old.nabble.com/MonetDB-SQL:-bulk-loading-large-data-sets---adding-ke… which involves setting the optimizer variable to "nov2009_pipe". I am unable to log in via mclient - I am able to put in my credentials but it just hangs at the "Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2010-SP2)" line. So I have a few questions: 1. Is there any way I can see the status of the alter table ... add primary key command? 2. Is it possible to stop the alter table gracefully? Or will I have to shut the db down? 3. Is there anything I should do in future to speed up adding of primary keys? Thanks. -- View this message in context: http://old.nabble.com/Add-primary-key-performance-tp30397515p30397515.html Sent from the monetdb-users mailing list archive at Nabble.com. ------------------------------ Message: 6 Date: Wed, 8 Dec 2010 09:38:23 +0100 From: Stefan Manegold <Stefan.Manegold(a)cwi.nl> Subject: Re: [MonetDB-users] Add primary key performance To: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Message-ID: <20101208083823.GA24931(a)cwi.nl> Content-Type: text/plain; charset=us-ascii On Tue, Dec 07, 2010 at 07:44:14AM -0800, paulr135 wrote: > > I've loaded a 27GB (~165M rows) file into a table in MonetDB and now I am > trying to add a primary key which consists of 4 fields - 2 varchars and 2 > ints. > It has been running for a few days (since Friday of last week). > > I've followed the instructions laid out on this post: > http://old.nabble.com/MonetDB-SQL:-bulk-loading-large-data-sets---adding-ke… > which involves setting the optimizer variable to "nov2009_pipe". Using the nov2009_pipe optimizer pipeline for adding constraint only holds for the Feb2010 release of MonetDB. It should not be required anymore for later releases as of Jun2010-SP1, incl. the latest Oct2010 release. Which version of MonetDB are you using? Which OS are you running on? What kind of manchine/hardware are you running on, in particular how much memory does it have? Can you check what the machine is doing? Is it running out of physical memory and swapping, i.e., is the virtual size of the mserver5 (much) larger than the machine's physical memory size? > I am unable to log in via mclient - I am able to put in my credentials but > it just hangs at the "Welcome to mclient, the MonetDB/SQL interactive > terminal (Jun2010-SP2)" line. > > So I have a few questions: > 1. Is there any way I can see the status of the alter table ... add primary > key command? You can use MonetDB's stethoscope tool to "listen" what the server is doing; however, while it is busy with a single (MAL-)operation, there will not be any output; hence, you might have to wait for some time to see some output. Unless very familiar with MonetDB internals, the output of stethoscope might not be too informative for you. But you're welcome to send us the output to have a look at it. In addition, you can check what the mserver5 is doing as suggested above. > 2. Is it possible to stop the alter table gracefully? Or will I have to > shut the db down? No, unfortunately there is no way to gracefully stop a running query. You'd have to way until it finishes (successfully or with an error), or shut down the DB. > 3. Is there anything I should do in future to speed up adding of primary > keys? In priciple not. We are aware that with data sizes (i.e., total size of all columns that the primary keys consists of) that significantly exceed main memory, adding the key (i.e., checking whether the data fulfills the key condition) does not work as efficiently as it could/should, in particular with multi-column keys --- we're working on that ... You could, though, run the ALTER TABLE statement in trace mode, i.e., `TRACE ALTER TABLE ...`, collect the output and send it to us for analysis; that could help us to improve the creation of large multi-column primary keys. Regards, Stefan > Thanks. > -- > View this message in context: http://old.nabble.com/Add-primary-key-performance-tp30397515p30397515.html > Sent from the monetdb-users mailing list archive at Nabble.com. > > > ------------------------------------------------------------------------------ > What happens now with your Lotus Notes apps - do you make another costly > upgrade, or settle for being marooned without product support? Time to move > off Lotus Notes and onto the cloud with Force.com, apps are easier to build, > use, and manage than apps on traditional platforms. Sign up for the Lotus > Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > > > > > > > > > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ------------------------------ Message: 7 Date: Fri, 10 Dec 2010 11:11:34 +0100 From: Daniel Boesswetter <daniel.boesswetter(a)fu-berlin.de> Subject: Re: [MonetDB-users] COPY INTO: wrong length with backslashed chars? To: monetdb-users(a)lists.sourceforge.net Message-ID: <4D01FCD6.9040201(a)fu-berlin.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi Stefan, > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Op 05-12-10 19:08, Daniel Boesswetter schreef: > > Hi all, > > > > I'm using MonetDB for the analysis for ~90GB of data which come from a > > MySQL-based web-application. The data is dumped into CSV-files on a > > regular basis and then imported into MonetDB with the COPY command. I > > have experienced some strange behaviour: obviously, MonetDB counts the > > number of chars for a VARCHAR-field before removing backslashes which > > are used e.g. for escaping newlines or other backslashes and rejects > > strings that should actually fit. > > > > A simple reproduction is this: > > > > 1. In mclient: CREATE TABLE t (x VARCHAR(1)); > > 2. Create a textfile foo.txt containing two backslashes: \\ > > 3. In mclient: > > sql>copy into t from '/tmp/foo.txt'; > > SQLException:importTable:value '\\' from line 1 field 1 not inserted, > > expecting type varchar(1) > > failed to import table > > > > Performing exactly the same steps with a VARCHAR(2) column works without > > errors and results in t.x containg the de-escaped single backslash. > > > > I'm using the Oct-2010 release. My current workaround is to enlarge > > VARCHARs which caused problems in the past, but this is of course no > > guarantee that it won't happen again. > > > > Although I found a few monetdb-bugs entries concerning the COPY command > > (e.g. about the misleading error message seen above), none seems to deal > > with my problem. > > > > Is this a bug? > > What are you escaping? Paste the line that fails ;) > > As I said: try to import a single "\" into a VARCHAR(1) with COPY. Writing "\" does not work because it escapes the newline. Writing "\\" does not work because it is longer than 1 char. Regards, Daniel > Stefan > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.16 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEAREKAAYFAkz71sgACgkQYH1+F2Rqwn3AmgCfayhLg6zjXZgvqhV9OHJbj5Kh > z64An2xZJy7H1wv3M6hr/SHQaK2C8IM3 > =//V/ > -----END PGP SIGNATURE----- > ------------------------------ ------------------------------------------------------------------------------ ------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users End of MonetDB-users Digest, Vol 55, Issue 2 ********************************************

1 0

[MonetDB-users] COPY INTO: wrong length with backslashed chars?
by Daniel Boesswetter 10 Dec '10

10 Dec '10

Hi all, I'm using MonetDB for the analysis for ~90GB of data which come from a MySQL-based web-application. The data is dumped into CSV-files on a regular basis and then imported into MonetDB with the COPY command. I have experienced some strange behaviour: obviously, MonetDB counts the number of chars for a VARCHAR-field before removing backslashes which are used e.g. for escaping newlines or other backslashes and rejects strings that should actually fit. A simple reproduction is this: 1. In mclient: CREATE TABLE t (x VARCHAR(1)); 2. Create a textfile foo.txt containing two backslashes: \\ 3. In mclient: sql>copy into t from '/tmp/foo.txt'; SQLException:importTable:value '\\' from line 1 field 1 not inserted, expecting type varchar(1) failed to import table Performing exactly the same steps with a VARCHAR(2) column works without errors and results in t.x containg the de-escaped single backslash. I'm using the Oct-2010 release. My current workaround is to enlarge VARCHARs which caused problems in the past, but this is of course no guarantee that it won't happen again. Although I found a few monetdb-bugs entries concerning the COPY command (e.g. about the misleading error message seen above), none seems to deal with my problem. Is this a bug? Best regards, Daniel

3 4

[MonetDB-users] Some questions regarding parallelism and other stuff...
by Mike De-La-Columnar 06 Dec '10

06 Dec '10

Hi fellas, If its not too much of a hassle, i have a few questions regarding parallelism & the internal algorithms 1. What is the difference between Data-Flow, and Mitosis? 2. Is there a special reason why the distinct count aggregation is not parallelized just like the other aggregation functions? Is it just a matter of not making it into the release, or is there a fundamental problem here. Note: even though the parallel distinct can be slower in certain distributions, it could still be useful... 3. Are there any plans on supporting parallel frameworks like OpenMP/Intel TBB directly within the low-level MAL functions? Providing data-parallelism on the loop-code level could even increase performance. Or did you already play with it and decided that the performance gain is just not worth it? 4. Is there a way for me to prevent MonetDB server from cleaning up all intermediate BATs between server restarts. I've read here that some intermediates (like join indices etc.) are lost between restarts... is it still correct? Thank you very much for your time, Keep up the amazing job of building the most powerful columnar database out there! -- View this message in context: http://old.nabble.com/Some-questions-regarding-parallelism-and-other-stuff.… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] MonetDB/XQuery Installation
by Rob Fynes 01 Dec '10

01 Dec '10

Hello, Apologies if this is the improper place to be asking this. I'm a new user to MonetDB, and I'm having trouble installing MonetDB/XQuery on Ubuntu 10.04 32-bit. Actually, the installation did seem to succeed (running the auto install script from the super source tarball) and I can start MServer, but with the error: # MonetDB Server v4.40.2 # based on GDK v1.40.2 # not released # Copyright (c) 1993-July 2008, CWI. All rights reserved. # Copyright (c) August 2008-2010, MonetDB B.V.. All rights reserved. # Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs; dynamically linked. # Detected 3.919 GiB (4208013312 bytes) main-memory. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: moduleClient: module(pathfinder) load error. The problem, its seems anyway, is during the configuration of pathfinder: ... ... checking for signal.h... yes checking return type of signal handlers... (cached) void checking for strings.h... (cached) yes checking for strcasecmp... yes checking for xml2-config.32... no checking for xml2-32-config... no checking for xml2-config... no checking for libxml/parser.h... no configure: error: in `/home/rob/MonetDB': configure: error: The libxml2 XML parsing library and/or include files were not found or you have requested to not use it. This is required dependence, core components like the XML Schema import functionality depends on it. See `config.log' for more details. I do have libxml2 installed (by default with Lucid). I've run the pathfinder configure script a few times, with the option --with-libxml2=*some variations*, but I'm not really confident about what I'm doing. I was wondering if anyone could point me in the right direction. Thanks. -Rob

3 5

[MonetDB-users] Granting TMP Table Creation Privileges to User
by Reeves, Matthew J. 30 Nov '10

30 Nov '10

I'm not sure what I'm missing but how does one grant permissions to a user to create temporary tables? I'm totally stumped. Thanks, Matt

1 0

[MonetDB-users] issue with concurrent access to MonetDB/SQL via JDBC
by Wouter Alink 26 Nov '10

26 Nov '10

Dear users, I'm currently struggling with concurrent access to MonetDB/SQL via JDBC. Version: MonetDB server v5.20.4 (64-bit), based on kernel v1.38.4 (64-bit oids). I might still have some concurrency issues with my own code, but somehow I frequently end up with the following error message: (while stress-testing by querying with 10 threads at the same time) class java.sql.SQLException: TypeException:user.s3_1[448]:'user.contains' undefined in: _1933:any := user.contains(_1929:str, _1931:str, _474:bit) TypeException:user.s3_1[449]:'bat.insert' undefined in: _1934:any := bat.insert(_1923:bat[:oid,:bit], _1928:oid, _1933:any) TypeException:user.s3_1[645]:'user.contains' undefined in: _1946:any := user.contains(_1942:str, _1944:str, _474:bit) TypeException:user.s3_1[646]:'bat.insert' undefined in: _1947:any := bat.insert(_1938:bat[:oid,:bit], _1941:oid, _1946:any) TypeException:user.s3_1[1042]:'user.contains' undefined in: _1959:any := user.contains(_1955:str, _1957:str, _474:bit) TypeException:user.s3_1[1043]:'bat.insert' undefined in: _1960:any := bat.insert(_1951:bat[:oid,:bit], _1954:oid, _1959:any) TypeException:user.s3_1[1226]:'user.contains' undefined in: _1974:any := user.contains(_1970:str, _1972:str, _474:bit) TypeException:user.s3_1[1227]:'bat.insert' undefined in: _1975:any := bat.insert(_1964:bat[:oid,:bit], _1969:oid, _1974:any) SQLException:SQLengine:Program contains errors nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2209) nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:1968) nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:506) nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:347) nl.cwi.monetdb.jdbc.MonetStatement.executeUpdate(MonetStatement.java:542) Does anyone else have experienced such error messages before, or does anyone have a clue to where to look? (like too many open connections, concurrent access of the same connection, querying a closed connection, or perhaps a bug inside monetdb)? Thanks, Wouter

2 1

[MonetDB-users] MonetDB Debain and RDF
by Andreas Harth 25 Nov '10

25 Nov '10

Hi, I'd like to use Lefteris' RDF package on Debian. What do I have to do to create and connect to a database as user on Debian? I've installed the following packages: # dpkg -l | grep -e monetdb ii libmonetdb-client1 1.40.1-20101110 MonetDB database client libraries ii libmonetdb1 1.40.1-20101110 database system for high-performance applications, libraries ii libmonetdb5-server5 5.22.1-20101110 MonetDB database server version 5, libraries ii libmonetdb5-sql2 2.40.1-20101110 MonetDB SQL libraries for monetdb5-server ii monetdb-client 1.40.1-20101110 MonetDB database client ii monetdb5-server 5.22.1-20101110 MonetDB database server version 5 ii monetdb5-sql 2.40.1-20101110 MonetDB SQL support for monetdb5 I can start MonetDB: # /etc/init.d/monetdb5-sql start Starting MonetDB SQL server: merovingian. When I try do create the tables (as a user): $ mclient -lsql create_tables.sql user(aharth): password: merovingian: please specify a database $ $ mclient -lsql create_tables.sql -d demo user(aharth): password: merovingian: internal error while starting mserver, please refer to the logs $ The log says: 2010-11-25 16:20:19 MSG merovingian[3685]: starting database 'demo', up min/avg/max: 3s/4s/5s, crash average: 0.00 0.00 0.00 (2-2=0) 2010-11-25 16:20:19 ERR merovingian[3685]: client error: cannot start database 'demo': no .vaultkey found (did you create the database with `monetdb create demo`?) I'm stuck here. How can I create a database as a user? $ monetdb create demo2 permission denied $ As root, I only can create a database in maintainance mode: # monetdb create demo2 created database in maintenance mode: demo2 # Any pointers to howto's that'll help? Cheers, Andreas.

3 5

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 23 Nov '10

23 Nov '10

I see --- if you consider it a bug, please file a detailed bug report including information & instructions to reporduce the problem as well as infornmation about the exact MonetDB version and OS you're using via http://bugs.MonetDB.org/ Stefan ----- Oorspronkelijk bericht ----- Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> Verzonden: maandag 25 oktober 2010 16:32 Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB Hi Stefan, Nope, the name of the column is "deliveryDays" :) Freddy 2010/10/25 Stefan Manegold <Stefan.Manegold(a)cwi.nl> > Hi Freddy, > > I don't think that the error you experience is triggered by the column > type, but rather by the fact that your date column ha? a name - say "date" - > that is a reserved key word in SQL an hence needs to be double-quoted > according to the SQL standard (this is not a MonetDB specific choice, but a > "feature" of the SQL standard) --- not knowing your table & column names, I > can only guess, though ... > > Stefan > > > ----- Oorspronkelijk bericht ----- > Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > Verzonden: maandag 25 oktober 2010 16:03 > Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> > Onderwerp: [MonetDB-users] Querying date type column in MonetDB > > Dear All, > > I am new to MonetDB, so most likely my question will look obvious. > > I would like to ask, how can columns with date type is queried in monetdb. > Based on my experience, it seems that date type columns need special > treatment. It seems that the only way to query them is with the syntax > "column_name" while other columns seems to work with the format > table_name.column_name without the need of double quote. When I tried to > query date column with the syntax table_name.column_name, it gave me an > error "no such column" > > So my question is, is it true that the only way to query date columns is > using "column_name" format? Another question will be, what happen if I need > to query same name columns coming from different table? > > Thanks in advance, > Freddy > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users >

4 6

[MonetDB-users] Copy into issues with mixed ascii / hex data
by paulr135 23 Nov '10

23 Nov '10

I have a tab-delimited data file that is mostly ascii, but has a few hex characters in it. For example: ... 3263640152 ABC<99> ... where <99> is a hex value for (tm) - trademark. When I try and load this data file into monetdb, I get the following error: [root@bidbtest ~]# /usr/local/MonetDB/bin/mclient -lsql --database=myschema -t -s "COPY 20603 records into test_table from '/tmp/testdata.txt' using delimiters '\t','\n','\"' null as '';" user(root):monetdb password: MAPI = (monetdb) /usr/local/MonetDB/var/MonetDB5/dbfarm/mapi_socket QUERY = COPY 20603 records into test_table from '/tmp/testdata.txt' using delimiters '\t','\n','"' null as ''; ERROR = !SQLException:importTable:value 'failed to import table Timer 790.415 msec The table def is: CREATE TABLE creative_dim ( id bigint NOT NULL, headline varchar(100) DEFAULT NULL ) How can I go about loading this file? Ideally, I would like to get these characters loaded as is. I saw that mclient has a -E flag that accepts an encoding type. However, I'm not sure what to set it to for a file with ascii / hex data. A complicating factor is that I don't know all the hex characters that are in my file. I luckily spotted the <99>/tm one above as it was one of the first lines in the file. But after replacing all instances of that character with the empty string, I still get a similar error to what I pasted above. I manually paged through the file and found different hex characters. It isn't practical for me to page through an entire file and monetdb doesn't tell me which line number the problem is on. By the way, I am using MonetDB v5.20.5. Thanks for the help! -- View this message in context: http://old.nabble.com/Copy-into-issues-with-mixed-ascii---hex-data-tp302389… Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] compiling from hg
by Dean Serenevy 18 Nov '10

18 Nov '10

Hello, I am attempting to compile from mercurial sources. I have tried from the Oct2010_1 tag as well as tip and both error out at the same point (I ran de-bootstrap between attempts). As far as I can tell i have all prerequisites installed. Full command output is included at the end of the message. Can anyone tell me what I am doing wrong? Thaks, Dean [MonetDB.hg/buildtools] (hg:default)$ hg update tip 0 files updated, 0 files merged, 0 files removed, 0 files unresolved [MonetDB.hg/buildtools] (hg:default)$ ./bootstrap [MonetDB.build/buildtools]$ ../MonetDB.hg/buildtools/configure --prefix=/opt/MonetDB --enable-burg --enable-mel --enable-debug=no --enable-assert=no --enable-optimize=yes [MonetDB.build/buildtools]$ make -j4 ... g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.yy.o -MD -MP -MF .deps/mel.yy.Tpo -c -o mel.yy.o mel.yy.cc y.tab.c:178:1: warning: "MEL_MODULE" redefined In file included from /home/duelafn/MonetDB.hg/buildtools/mel/command.h:26, from /home/duelafn/MonetDB.hg/buildtools/mel/mel.yy:47: /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:1:1: warning: this is the location of the previous definition y.tab.c:179:1: warning: "MEL_PRELUDE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:2:1: warning: this is the location of the previous definition y.tab.c:180:1: warning: "MEL_EPILOGUE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:3:1: warning: this is the location of the previous definition y.tab.c:181:1: warning: "MEL_END" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:4:1: warning: this is the location of the previous definition y.tab.c:182:1: warning: "MEL_COMMAND" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:5:1: warning: this is the location of the previous definition y.tab.c:183:1: warning: "MEL_OPERATOR0" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:6:1: warning: this is the location of the previous definition y.tab.c:184:1: warning: "MEL_OPERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:7:1: warning: this is the location of the previous definition y.tab.c:185:1: warning: "MEL_OPERATOR1" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:8:1: warning: this is the location of the previous definition y.tab.c:186:1: warning: "MEL_FIXEDATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:9:1: warning: this is the location of the previous definition y.tab.c:187:1: warning: "MEL_ATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:10:1: warning: this is the location of the previous definition y.tab.c:188:1: warning: "MEL_USE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:11:1: warning: this is the location of the previous definition y.tab.c:189:1: warning: "MEL_ATOMOP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:12:1: warning: this is the location of the previous definition y.tab.c:190:1: warning: "MEL_STRING" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:13:1: warning: this is the location of the previous definition y.tab.c:191:1: warning: "MEL_IDENTIFIER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:14:1: warning: this is the location of the previous definition y.tab.c:192:1: warning: "MEL_USED" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:15:1: warning: this is the location of the previous definition y.tab.c:193:1: warning: "MEL_NUMBER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:16:1: warning: this is the location of the previous definition y.tab.c:194:1: warning: "MEL_ANY" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:17:1: warning: this is the location of the previous definition y.tab.c:195:1: warning: "MEL_BAT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:18:1: warning: this is the location of the previous definition y.tab.c:196:1: warning: "MEL_TYPE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:19:1: warning: this is the location of the previous definition y.tab.c:197:1: warning: "MEL_VOID" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:20:1: warning: this is the location of the previous definition y.tab.c:198:1: warning: "MEL_BUILTIN" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:21:1: warning: this is the location of the previous definition y.tab.c:199:1: warning: "MEL_OBJECT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:22:1: warning: this is the location of the previous definition y.tab.c:200:1: warning: "MEL_VARARGS" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:23:1: warning: this is the location of the previous definition y.tab.c:201:1: warning: "MEL_SEP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:24:1: warning: this is the location of the previous definition y.tab.c:202:1: warning: "MEL_ITERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:25:1: warning: this is the location of the previous definition mv -f .deps/glue.Tpo .deps/glue.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mil.o -MD -MP -MF .deps/mil.Tpo -c -o mil.o mil.cc y.tab.c:150: error: expected identifier before numeric constant y.tab.c:150: error: expected ‘}’ before numeric constant y.tab.c:150: error: expected unqualified-id before numeric constant y.tab.c:175: error: expected declaration before ‘}’ token make[3]: *** [mel.tab.o] Error 1 make[3]: *** Waiting for unfinished jobs.... mv -f .deps/mel.Tpo .deps/mel.Po mv -f .deps/mil.Tpo .deps/mil.Po mv -f .deps/mel.yy.Tpo .deps/mel.yy.Po make[3]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/duelafn/MonetDB.build' make: *** [all] Error 2 FULL COMMAND OUTPUT [MonetDB.hg/buildtools] (hg:default)$ ./bootstrap configure.ac:24: installing `./install-sh' configure.ac:24: installing `./missing' Makefile.am: installing `./depcomp' configure.ac:24: installing `./install-sh' configure.ac:24: installing `./missing' Makefile.am: installing `./depcomp' configure.ac:24: installing `./install-sh' configure.ac:24: installing `./missing' Makefile.am: installing `./depcomp' configure.ac:25: installing `./install-sh' configure.ac:25: installing `./missing' [MonetDB.build/buildtools]$ ../MonetDB.hg/buildtools/configure --prefix=/opt/MonetDB --enable-burg --enable-mel --enable-debug=no --enable-assert=no --enable-optimize=yes checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for python... /usr/bin/python configure: WARNING: unrecognized options: --enable-burg, --enable-mel, --enable-debug, --enable-assert, --enable-optimize checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make sets $(MAKE)... (cached) yes checking for g++... g++ checking whether the C++ compiler works... yes checking for C++ compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking for style of include used by make... GNU checking dependency style of g++... gcc3 checking for gcc... gcc checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking dependency style of gcc... gcc3 checking for lockfile... echo checking for flex... flex checking lex output file root... lex.yy checking lex library... -lfl checking whether yytext is a pointer... yes checking for bison... bison -y checking whether ln -s works... yes checking for rm... rm -f checking for mv... mv -f checking for flex version... flex 2.5.35 checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking whether time.h and sys/time.h may both be included... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking malloc.h usability... yes checking malloc.h presence... yes checking for malloc.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking for unistd.h... (cached) yes checking for iostream... yes checking for cstdio... yes checking for an ANSI C-conforming const... yes checking for inline... inline checking for size_t... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for memset... yes checking for strchr... yes checking for strdup... yes checking for strrchr... yes configure: creating ./config.status config.status: creating Makefile config.status: creating mel_config.h config.status: executing depfiles commands configure: WARNING: unrecognized options: --enable-burg, --enable-mel, --enable-debug, --enable-assert, --enable-optimize configure: creating ./config.status config.status: creating Makefile config.status: creating autogen/Makefile config.status: creating conf/Makefile config.status: creating conf/Mbootstrap config.status: creating conf/buildtools-config config.status: creating dummy.h === configuring in Mx (/home/duelafn/MonetDB.build/Mx) configure: running /bin/bash ../../MonetDB.hg/buildtools/Mx/configure --disable-option-checking '--prefix=/opt/MonetDB' '--enable-burg' '--enable-mel' '--enable-debug=no' '--enable-assert=no' '--enable-optimize=yes' --cache-file=/dev/null --srcdir=../../MonetDB.hg/buildtools/Mx checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for bash... /bin/bash checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking getopt.h usability... yes checking getopt.h presence... yes checking for getopt.h... yes checking limits.h usability... yes checking limits.h presence... yes checking for limits.h... yes checking stddef.h usability... yes checking stddef.h presence... yes checking for stddef.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking for unistd.h... (cached) yes checking utime.h usability... yes checking utime.h presence... yes checking for utime.h... yes checking whether stat file-mode macros are broken... no checking for an ANSI C-conforming const... yes checking for size_t... yes checking for ptrdiff_t... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible realloc... yes checking whether lstat correctly handles trailing slash... yes checking whether stat accepts an empty string... no checking for utime.h... (cached) yes checking whether utime accepts a null argument... yes checking for vprintf... yes checking for _doprnt... no checking for memset... yes checking for mkdir... yes checking for strchr... yes checking for strcspn... yes checking for strrchr... yes checking for strstr... yes checking for utime... yes checking whether getopt is declared... yes configure: creating ./config.status config.status: creating Makefile config.status: creating mx_config.h config.status: executing depfiles commands === configuring in burg (/home/duelafn/MonetDB.build/burg) configure: running /bin/bash ../../MonetDB.hg/buildtools/burg/configure --disable-option-checking '--prefix=/opt/MonetDB' '--enable-burg' '--enable-mel' '--enable-debug=no' '--enable-assert=no' '--enable-optimize=yes' --cache-file=/dev/null --srcdir=../../MonetDB.hg/buildtools/burg checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for flex... flex checking lex output file root... lex.yy checking lex library... -lfl checking whether yytext is a pointer... yes checking for bison... bison -y checking for lockfile... echo checking for rm... rm -f checking for mv... mv -f checking for flex version... flex 2.5.35 checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking stdbool.h usability... yes checking stdbool.h presence... yes checking for stdbool.h... yes checking for stdlib.h... (cached) yes checking for string.h... (cached) yes checking for stdbool.h that conforms to C99... (cached) yes checking for _Bool... yes checking for an ANSI C-conforming const... yes checking for size_t... yes checking for long long... yes checking for __int64... no checking size of void *... 8 checking size of int... 4 checking size of long... 8 checking size of long long... 8 checking size of __int64... 0 checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for memset... yes checking for strdup... yes configure: creating ./config.status config.status: creating Makefile config.status: creating burg_config.h config.status: executing depfiles commands [MonetDB.build/buildtools]$ make -j4 (CDPATH="${ZSH_VERSION+.}:" && cd ../MonetDB.hg/buildtools && /bin/bash /home/duelafn/MonetDB.hg/buildtools/missing --run autoheader) rm -f stamp-h1 touch ../MonetDB.hg/buildtools/dummy.h.in cd . && /bin/bash ./config.status dummy.h config.status: creating dummy.h config.status: dummy.h is unchanged /usr/bin/make all-recursive make[1]: Entering directory `/home/duelafn/MonetDB.build' Making all in autogen make[2]: Entering directory `/home/duelafn/MonetDB.build/autogen' mkdir -p /home/duelafn/MonetDB.build/autogen/autogen cp ../../MonetDB.hg/buildtools/autogen/setup.py /home/duelafn/MonetDB.build/autogen/setup.py cp ../../MonetDB.hg/buildtools/autogen/autogen.py /home/duelafn/MonetDB.build/autogen/autogen.py cp ../../MonetDB.hg/buildtools/autogen/autogen/am.py /home/duelafn/MonetDB.build/autogen/autogen/am.py cp ../../MonetDB.hg/buildtools/autogen/autogen/msc.py /home/duelafn/MonetDB.build/autogen/autogen/msc.py cp ../../MonetDB.hg/buildtools/autogen/autogen/var.py /home/duelafn/MonetDB.build/autogen/autogen/var.py cp ../../MonetDB.hg/buildtools/autogen/autogen/codegen.py /home/duelafn/MonetDB.build/autogen/autogen/codegen.py cp ../../MonetDB.hg/buildtools/autogen/autogen/__init__.py /home/duelafn/MonetDB.build/autogen/autogen/__init__.py /usr/bin/python setup.py build running build running build_py creating build creating build/lib.linux-x86_64-2.6 creating build/lib.linux-x86_64-2.6/autogen copying autogen/var.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/codegen.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/__init__.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/am.py -> build/lib.linux-x86_64-2.6/autogen copying autogen/msc.py -> build/lib.linux-x86_64-2.6/autogen running build_scripts creating build/scripts-2.6 copying and adjusting autogen.py -> build/scripts-2.6 changing mode of build/scripts-2.6/autogen.py from 644 to 755 make[2]: Leaving directory `/home/duelafn/MonetDB.build/autogen' Making all in Mx make[2]: Entering directory `/home/duelafn/MonetDB.build/Mx' (CDPATH="${ZSH_VERSION+.}:" && cd ../../MonetDB.hg/buildtools/Mx && /bin/bash /home/duelafn/MonetDB.hg/buildtools/Mx/missing --run autoheader) rm -f stamp-h1 touch ../../MonetDB.hg/buildtools/Mx/mx_config.h.in cd . && /bin/bash ./config.status mx_config.h config.status: creating mx_config.h config.status: mx_config.h is unchanged /usr/bin/make all-am make[3]: Entering directory `/home/duelafn/MonetDB.build/Mx' gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Mx.o -MD -MP -MF .deps/Mx.Tpo -c -o Mx.o ../../MonetDB.hg/buildtools/Mx/Mx.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Def.o -MD -MP -MF .deps/Def.Tpo -c -o Def.o ../../MonetDB.hg/buildtools/Mx/Def.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Form.o -MD -MP -MF .deps/Form.Tpo -c -o Form.o ../../MonetDB.hg/buildtools/Mx/Form.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Tok.o -MD -MP -MF .deps/Tok.Tpo -c -o Tok.o ../../MonetDB.hg/buildtools/Mx/Tok.c mv -f .deps/Mx.Tpo .deps/Mx.Po mv -f .deps/Tok.Tpo .deps/Tok.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Code.o -MD -MP -MF .deps/Code.Tpo -c -o Code.o ../../MonetDB.hg/buildtools/Mx/Code.c gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Display.o -MD -MP -MF .deps/Display.Tpo -c -o Display.o ../../MonetDB.hg/buildtools/Mx/Display.c mv -f .deps/Form.Tpo .deps/Form.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Print.o -MD -MP -MF .deps/Print.Tpo -c -o Print.o ../../MonetDB.hg/buildtools/Mx/Print.c mv -f .deps/Display.Tpo .deps/Display.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Io.o -MD -MP -MF .deps/Io.Tpo -c -o Io.o ../../MonetDB.hg/buildtools/Mx/Io.c mv -f .deps/Def.Tpo .deps/Def.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT Sys.o -MD -MP -MF .deps/Sys.Tpo -c -o Sys.o ../../MonetDB.hg/buildtools/Mx/Sys.c mv -f .deps/Print.Tpo .deps/Print.Po gcc -DHAVE_CONFIG_H -I. -I../../MonetDB.hg/buildtools/Mx -g -O2 -MT disclaimer.o -MD -MP -MF .deps/disclaimer.Tpo -c -o disclaimer.o ../../MonetDB.hg/buildtools/Mx/disclaimer.c mv -f .deps/Code.Tpo .deps/Code.Po mv -f .deps/Sys.Tpo .deps/Sys.Po mv -f .deps/disclaimer.Tpo .deps/disclaimer.Po mv -f .deps/Io.Tpo .deps/Io.Po gcc -g -O2 -o Mx Mx.o Def.o Form.o Tok.o Code.o Display.o Print.o Io.o Sys.o disclaimer.o make[3]: Leaving directory `/home/duelafn/MonetDB.build/Mx' make[2]: Leaving directory `/home/duelafn/MonetDB.build/Mx' Making all in mel make[2]: Entering directory `/home/duelafn/MonetDB.build/mel' (CDPATH="${ZSH_VERSION+.}:" && cd /home/duelafn/MonetDB.hg/buildtools/mel && /bin/bash /home/duelafn/MonetDB.hg/buildtools/mel/missing --run autoheader) rm -f stamp-h1 touch /home/duelafn/MonetDB.hg/buildtools/mel/mel_config.h.in cd . && /bin/bash ./config.status mel_config.h config.status: creating mel_config.h config.status: mel_config.h is unchanged /usr/bin/make all-am make[3]: Entering directory `/home/duelafn/MonetDB.build/mel' ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/any_arg.mx /home/duelafn/MonetDB.hg/buildtools/mel/any_arg.mx: ./any_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/atom_arg.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/atom.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/atomops.mx /home/duelafn/MonetDB.hg/buildtools/mel/atom_arg.mx: ./atom_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/bat_arg.mx /home/duelafn/MonetDB.hg/buildtools/mel/atom.mx: ./atom.cc - created /home/duelafn/MonetDB.hg/buildtools/mel/atomops.mx: ./atomops.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/builtin.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/command.mx /home/duelafn/MonetDB.hg/buildtools/mel/bat_arg.mx: ./bat_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/debug.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/depend.mx /home/duelafn/MonetDB.hg/buildtools/mel/builtin.mx: ./builtin.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/dependency.mx /home/duelafn/MonetDB.hg/buildtools/mel/command.mx: ./command.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/epilogue.mx /home/duelafn/MonetDB.hg/buildtools/mel/debug.mx: ./debug.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/FileInfo.mx /home/duelafn/MonetDB.hg/buildtools/mel/depend.mx: ./depend.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/glue.mx /home/duelafn/MonetDB.hg/buildtools/mel/dependency.mx: ./dependency.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/html.mx /home/duelafn/MonetDB.hg/buildtools/mel/epilogue.mx: ./epilogue.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/iterator.mx /home/duelafn/MonetDB.hg/buildtools/mel/FileInfo.mx: ./FileInfo.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/link.mx /home/duelafn/MonetDB.hg/buildtools/mel/html.mx: ./html.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/ListIterator.mx /home/duelafn/MonetDB.hg/buildtools/mel/iterator.mx: ./iterator.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/mel.mx /home/duelafn/MonetDB.hg/buildtools/mel/ListIterator.mx: ./ListIterator.cc - created echo waiting /home/duelafn/MonetDB.hg/buildtools/mel/link.mx: ./link.cc - created flex /home/duelafn/MonetDB.hg/buildtools/mel/mel.ll waiting bison -y -d /home/duelafn/MonetDB.hg/buildtools/mel/mel.yy || { rm -f waiting ; exit 1 ; } /home/duelafn/MonetDB.hg/buildtools/mel/mel.mx: ./mel.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/mil.mx /home/duelafn/MonetDB.hg/buildtools/mel/glue.mx: ./glue.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/module.mx /home/duelafn/MonetDB.hg/buildtools/mel/mil.mx: ./mil.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/object.mx /home/duelafn/MonetDB.hg/buildtools/mel/module.mx: ./module.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/operator.mx conflicts: 32 reduce/reduce /home/duelafn/MonetDB.hg/buildtools/mel/object.mx: ./object.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/ops.mx /home/duelafn/MonetDB.hg/buildtools/mel/operator.mx: ./operator.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/prelude.mx /home/duelafn/MonetDB.hg/buildtools/mel/prelude.mx: ./prelude.cc - created /home/duelafn/MonetDB.hg/buildtools/mel/ops.mx: ./ops.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/proto.mx ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/symbol.mx if [ -f lex.yy.c ]; then mv lex.yy.c mel.yy.cc ; fi /home/duelafn/MonetDB.hg/buildtools/mel/proto.mx: ./proto.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/symtable.mx /home/duelafn/MonetDB.hg/buildtools/mel/symbol.mx: ./symbol.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/type_arg.mx mv mel.yy.cc mel.yy.cc.tmp /home/duelafn/MonetDB.hg/buildtools/mel/symtable.mx: ./symtable.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/use.mx /home/duelafn/MonetDB.hg/buildtools/mel/type_arg.mx: ./type_arg.cc - created ../Mx/Mx -x C /home/duelafn/MonetDB.hg/buildtools/mel/var_arg.mx /home/duelafn/MonetDB.hg/buildtools/mel/use.mx: ./use.cc - created echo '#include <'"mel_config.h"'>' > mel.yy.cc grep -v '^#include.*[<"]'"mel_config.h"'[">]' mel.yy.cc.tmp >> mel.yy.cc g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT any_arg.o -MD -MP -MF .deps/any_arg.Tpo -c -o any_arg.o any_arg.cc /home/duelafn/MonetDB.hg/buildtools/mel/var_arg.mx: ./var_arg.cc - created g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT atom_arg.o -MD -MP -MF .deps/atom_arg.Tpo -c -o atom_arg.o atom_arg.cc rm -f mel.yy.cc.tmp g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT atom.o -MD -MP -MF .deps/atom.Tpo -c -o atom.o atom.cc if [ -f y.tab.c ]; then mv y.tab.c mel.tab.cc ; fi rm -f waiting g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT atomops.o -MD -MP -MF .deps/atomops.Tpo -c -o atomops.o atomops.cc mv -f .deps/atom.Tpo .deps/atom.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT bat_arg.o -MD -MP -MF .deps/bat_arg.Tpo -c -o bat_arg.o bat_arg.cc mv -f .deps/any_arg.Tpo .deps/any_arg.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT builtin.o -MD -MP -MF .deps/builtin.Tpo -c -o builtin.o builtin.cc mv -f .deps/atom_arg.Tpo .deps/atom_arg.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT command.o -MD -MP -MF .deps/command.Tpo -c -o command.o command.cc mv -f .deps/atomops.Tpo .deps/atomops.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT debug.o -MD -MP -MF .deps/debug.Tpo -c -o debug.o debug.cc mv -f .deps/bat_arg.Tpo .deps/bat_arg.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT depend.o -MD -MP -MF .deps/depend.Tpo -c -o depend.o depend.cc mv -f .deps/builtin.Tpo .deps/builtin.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT dependency.o -MD -MP -MF .deps/dependency.Tpo -c -o dependency.o dependency.cc mv -f .deps/command.Tpo .deps/command.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT epilogue.o -MD -MP -MF .deps/epilogue.Tpo -c -o epilogue.o epilogue.cc mv -f .deps/debug.Tpo .deps/debug.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT FileInfo.o -MD -MP -MF .deps/FileInfo.Tpo -c -o FileInfo.o FileInfo.cc mv -f .deps/depend.Tpo .deps/depend.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT glue.o -MD -MP -MF .deps/glue.Tpo -c -o glue.o glue.cc mv -f .deps/dependency.Tpo .deps/dependency.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT html.o -MD -MP -MF .deps/html.Tpo -c -o html.o html.cc mv -f .deps/epilogue.Tpo .deps/epilogue.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT iterator.o -MD -MP -MF .deps/iterator.Tpo -c -o iterator.o iterator.cc mv -f .deps/FileInfo.Tpo .deps/FileInfo.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT link.o -MD -MP -MF .deps/link.Tpo -c -o link.o link.cc mv -f .deps/iterator.Tpo .deps/iterator.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT ListIterator.o -MD -MP -MF .deps/ListIterator.Tpo -c -o ListIterator.o ListIterator.cc mv -f .deps/ListIterator.Tpo .deps/ListIterator.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.o -MD -MP -MF .deps/mel.Tpo -c -o mel.o mel.cc mv -f .deps/html.Tpo .deps/html.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.tab.o -MD -MP -MF .deps/mel.tab.Tpo -c -o mel.tab.o mel.tab.cc mv -f .deps/link.Tpo .deps/link.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mel.yy.o -MD -MP -MF .deps/mel.yy.Tpo -c -o mel.yy.o mel.yy.cc y.tab.c:178:1: warning: "MEL_MODULE" redefined In file included from /home/duelafn/MonetDB.hg/buildtools/mel/command.h:26, from /home/duelafn/MonetDB.hg/buildtools/mel/mel.yy:47: /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:1:1: warning: this is the location of the previous definition y.tab.c:179:1: warning: "MEL_PRELUDE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:2:1: warning: this is the location of the previous definition y.tab.c:180:1: warning: "MEL_EPILOGUE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:3:1: warning: this is the location of the previous definition y.tab.c:181:1: warning: "MEL_END" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:4:1: warning: this is the location of the previous definition y.tab.c:182:1: warning: "MEL_COMMAND" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:5:1: warning: this is the location of the previous definition y.tab.c:183:1: warning: "MEL_OPERATOR0" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:6:1: warning: this is the location of the previous definition y.tab.c:184:1: warning: "MEL_OPERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:7:1: warning: this is the location of the previous definition y.tab.c:185:1: warning: "MEL_OPERATOR1" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:8:1: warning: this is the location of the previous definition y.tab.c:186:1: warning: "MEL_FIXEDATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:9:1: warning: this is the location of the previous definition y.tab.c:187:1: warning: "MEL_ATOM" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:10:1: warning: this is the location of the previous definition y.tab.c:188:1: warning: "MEL_USE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:11:1: warning: this is the location of the previous definition y.tab.c:189:1: warning: "MEL_ATOMOP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:12:1: warning: this is the location of the previous definition y.tab.c:190:1: warning: "MEL_STRING" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:13:1: warning: this is the location of the previous definition y.tab.c:191:1: warning: "MEL_IDENTIFIER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:14:1: warning: this is the location of the previous definition y.tab.c:192:1: warning: "MEL_USED" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:15:1: warning: this is the location of the previous definition y.tab.c:193:1: warning: "MEL_NUMBER" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:16:1: warning: this is the location of the previous definition y.tab.c:194:1: warning: "MEL_ANY" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:17:1: warning: this is the location of the previous definition y.tab.c:195:1: warning: "MEL_BAT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:18:1: warning: this is the location of the previous definition y.tab.c:196:1: warning: "MEL_TYPE" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:19:1: warning: this is the location of the previous definition y.tab.c:197:1: warning: "MEL_VOID" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:20:1: warning: this is the location of the previous definition y.tab.c:198:1: warning: "MEL_BUILTIN" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:21:1: warning: this is the location of the previous definition y.tab.c:199:1: warning: "MEL_OBJECT" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:22:1: warning: this is the location of the previous definition y.tab.c:200:1: warning: "MEL_VARARGS" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:23:1: warning: this is the location of the previous definition y.tab.c:201:1: warning: "MEL_SEP" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:24:1: warning: this is the location of the previous definition y.tab.c:202:1: warning: "MEL_ITERATOR" redefined /home/duelafn/MonetDB.hg/buildtools/mel/mel.tab.h:25:1: warning: this is the location of the previous definition mv -f .deps/glue.Tpo .deps/glue.Po g++ -DHAVE_CONFIG_H -I. -I/home/duelafn/MonetDB.hg/buildtools/mel -g -MT mil.o -MD -MP -MF .deps/mil.Tpo -c -o mil.o mil.cc y.tab.c:150: error: expected identifier before numeric constant y.tab.c:150: error: expected ‘}’ before numeric constant y.tab.c:150: error: expected unqualified-id before numeric constant y.tab.c:175: error: expected declaration before ‘}’ token make[3]: *** [mel.tab.o] Error 1 make[3]: *** Waiting for unfinished jobs.... mv -f .deps/mel.Tpo .deps/mel.Po mv -f .deps/mil.Tpo .deps/mil.Po mv -f .deps/mel.yy.Tpo .deps/mel.yy.Po make[3]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/duelafn/MonetDB.build/mel' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/duelafn/MonetDB.build' make: *** [all] Error 2

3 7

[MonetDB-users] Large Database / Is there hope?
by Dean Serenevy 18 Nov '10

18 Nov '10

Hello, I am still having problems compiling from sources, however, I would like to determine whether these is hope in continuing. We have a corpus of about 18,000 documents averaging about 5000 words each. We additionally have stand-off annotations that we would like to add and query against. This totals about 300 million XML elements. We are running 64-bit Linux with 6GiB RAM. Database A: Loading the documents (in batches of 100) as separate collections takes about 30 min and consumes 30GiB disk space in var/MonetDB4/dbfarm. Simple queries against database A take a long time, consume only 5% CPU and heavily work kswapd count( for $d in pf:documents-unsafe() return doc($d)/tCorpus ) # killed after1 hour Database B: Loading the documents (in batches of 100) into a single huge collection takes about 6 hours and consumes 120GiB disk space in var/MonetDB4/dbfarm. Simple queries against Database B take a long time, consume only 5% CPU and heavily work pdflush count( pf:collection("tcorpus")//tCorpus ) # killed after 3 hours My questions: 1. Is there any hope of successfully performing non-trivial queries on either of these databases using MonetDB? 2. If so, Is loading into a single collection or separate collections likely to be preferable? 3. The above work was done using the Jun2010-SP2 SuperBall, which is the only version I have been able to compile. Has any relevant work been done since then on MonetDB4 or any of the XQuery code that might improve performance? Thank you, Dean

1 0

[MonetDB-users] MonetDB-client 1.40.1 without Python support?
by W-Mark Kubacki 16 Nov '10

16 Nov '10

Hi. I've noticed that MonetDB-client 1.40.1 doesn't ship with Python libraries anymore. Yet there is a --with-python configuration option and Python examples. The corresponding changelog page [1] says nothing about the matter. What happened? -- Mark [1] http://monetdb.cwi.nl/Development/Releases/Oct2010/ [2] https://github.com/wmark/ossdl-overlay

3 2

[MonetDB-users] Announcement: New Oct2010 feature release of MonetDB suite
by Sjoerd Mullender 15 Nov '10

15 Nov '10

It is not October anymore, but nevertheless, the MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Oct2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] !MALException:setScenario:Scenario not initialized 'sql'
by Osmin Castillo 13 Nov '10

13 Nov '10

Hi guys, after install *MonetDB RPMs for Centos 5<http://code.google.com/p/monetdb-rhel-5/> * and mserver5 starts , I receive this error when execute the client: mclient -lsql !MALException:setScenario:Scenario not initialized 'sql' There's something missing?.. please help.

4 8

[MonetDB-users] MonetDB memory issue
by Andrey Timofeyev 12 Nov '10

12 Nov '10

Hello, All, We began to use MonetDB several weeks ago. As I can see people use monetdb several years and operate with millions of records. But we could not insert even several thousands records. After several minutes monetdb eat all 7 Gb memory and crush without any message. It seems that we use monetdb incorrectly. Do there exist any special options for start monetdb? MonetDB version: MonetDB server v5.20.5 (64-bit), based on kernel v1.38.5 (32-bit oids) Release Jun2010-SP2 Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 7.9GiB available memory Configured for prefix: C:\Users\sjoerd\Documents\src\stable\vs64\NTv64 Libraries: libpcre: 7.9 2009-04-11 openssl: OpenSSL 1.0.0a 1 Jun 2010 libxml2: 2.7.7 Compiled by: sjoerd@ALGOL (i686-pc-win32) Compilation: cl -GF -W3 -WX -wd4273 -wd4102 -MD -nologo -Ox Linking : cl -GF -W3 -WX -wd4273 -wd4102 -MD -nologo -Ox OS: Windows 7 x64 (Версия 6.1.7600 Сборка 7600) (We also try it on the ubuntu 10.4 - same effect) -- Best regards, Andrey.

3 11

[MonetDB-users] Sept-2010 Release
by Henry Addington 11 Nov '10

11 Nov '10

Hello: Do you have an estimated release date for the Oct-2010 release? I thought the major releases came out every 3 to 4 months..... Thanks for the update. Best regards, Henry

2 3

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 25 Oct '10

25 Oct '10

"T.c" != "T"."c" ;-) -- | Stefan.Manegold @ CWI .nl | DB Architectures (INS1) | | http://www.cwi.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Oorspronkelijk bericht ----- Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> Verzonden: maandag 25 oktober 2010 17:11 Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB Dear Arjen, Thanks. I did check the uppercase letter. With a date-type column "deliveryDays" in a table "bsbm.offer", here is the result - SELECT "deliveryDays" FROM bsbm.offer --> WORKS - SELECT "deliverydays" FROM bsbm.offer --> FAILS - SELECT "offer.deliveryDays" FROM bsbm.offer --> FAILS - SELECT deliveryDays FROM bsbm.offer --> FAILS - SELECT offer.deliveryDays FROM bsbm.offer --> FAILS - SELECT offer."deliveryDays" FROM bsbm.offer ---> WORKS The last query is the one I need :) Again, thanks for the help. Regards Freddy 2010/10/25 Arjen de Rijke <Arjen.de.Rijke(a)cwi.nl> > Stefan Manegold <Stefan.Manegold(a)cwi.nl> writes: > > > I see --- if you consider it a bug, please file a detailed bug report > including information & instructions to reporduce the problem as well as > infornmation about the exact MonetDB version and OS you're using via > http://bugs.MonetDB.org/ > > I would not consider it a bug. The catch is the uppercase letter in > deliveryDays. The default handling of upper/lowercase characters is > different than in other RDMBS. So in some cases the quotes are necessary. > > Arjen de Rijke > > > > Stefan > > > > > > ----- Oorspronkelijk bericht ----- > > Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > > Verzonden: maandag 25 oktober 2010 16:32 > > Aan: Communication channel for MonetDB users < > monetdb-users(a)lists.sourceforge.net> > > Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB > > > > Hi Stefan, > > > > Nope, the name of the column is "deliveryDays" :) > > > > Freddy > > > > 2010/10/25 Stefan Manegold <Stefan.Manegold(a)cwi.nl> > > > >> Hi Freddy, > >> > >> I don't think that the error you experience is triggered by the column > >> type, but rather by the fact that your date column ha? a name - say > "date" - > >> that is a reserved key word in SQL an hence needs to be double-quoted > >> according to the SQL standard (this is not a MonetDB specific choice, > but a > >> "feature" of the SQL standard) --- not knowing your table & column > names, I > >> can only guess, though ... > >> > >> Stefan > >> > >> > >> ----- Oorspronkelijk bericht ----- > >> Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > >> Verzonden: maandag 25 oktober 2010 16:03 > >> Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> > >> Onderwerp: [MonetDB-users] Querying date type column in MonetDB > >> > >> Dear All, > >> > >> I am new to MonetDB, so most likely my question will look obvious. > >> > >> I would like to ask, how can columns with date type is queried in > monetdb. > >> Based on my experience, it seems that date type columns need special > >> treatment. It seems that the only way to query them is with the syntax > >> "column_name" while other columns seems to work with the format > >> table_name.column_name without the need of double quote. When I tried to > >> query date column with the syntax table_name.column_name, it gave me an > >> error "no such column" > >> > >> So my question is, is it true that the only way to query date columns is > >> using "column_name" format? Another question will be, what happen if I > need > >> to query same name columns coming from different table? > >> > >> Thanks in advance, > >> Freddy > >> > >> > >> > ------------------------------------------------------------------------------ > >> Nokia and AT&T present the 2010 Calling All Innovators-North America > >> contest > >> Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > >> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > >> marketing > >> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > >> http://p.sf.net/sfu/nokia-dev2dev > >> _______________________________________________ > >> MonetDB-users mailing list > >> MonetDB-users(a)lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/monetdb-users > >> > > > > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > > Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > > MonetDB-users mailing list > > MonetDB-users(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > -- > ==================================================================== > CWI, Kamer L330 Centrum voor Wiskunde en Informatica > Science Park 123 Email: arjen.de.rijke(a)cwi.nl > 1098 XG Amsterdam tel: +31-(0)20-5924305 > Nederland +31-(0)6-51899284 > fax: +31-(0)20-5924312 > ===================== http://www.cwi.nl/~rijke/ ==================== > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users >

1 0

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 25 Oct '10

25 Oct '10

Good point, Arjen! Indeed CaMeL case identifiers need to be double-quoted in MonetDB/SQL ... Stefan -- | Stefan.Manegold @ CWI .nl | DB Architectures (INS1) | | http://www.cwi.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Oorspronkelijk bericht ----- Van: Arjen de Rijke <Arjen.de.Rijke(a)cwi.nl> Verzonden: maandag 25 oktober 2010 17:00 Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB Stefan Manegold <Stefan.Manegold(a)cwi.nl> writes: > I see --- if you consider it a bug, please file a detailed bug report including information & instructions to reporduce the problem as well as infornmation about the exact MonetDB version and OS you're using via http://bugs.MonetDB.org/ I would not consider it a bug. The catch is the uppercase letter in deliveryDays. The default handling of upper/lowercase characters is different than in other RDMBS. So in some cases the quotes are necessary. Arjen de Rijke > > Stefan > > > ----- Oorspronkelijk bericht ----- > Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> > Verzonden: maandag 25 oktober 2010 16:32 > Aan: Communication channel for MonetDB users <monetdb-users(a)lists.sourceforge.net> > Onderwerp: Re: [MonetDB-users] Querying date type column in MonetDB > > Hi Stefan, > > Nope, the name of the column is "deliveryDays" :) > > Freddy > > 2010/10/25 Stefan Manegold <Stefan.Manegold(a)cwi.nl> > >> Hi Freddy, >> >> I don't think that the error you experience is triggered by the column >> type, but rather by the fact that your date column ha? a name - say "date" - >> that is a reserved key word in SQL an hence needs to be double-quoted >> according to the SQL standard (this is not a MonetDB specific choice, but a >> "feature" of the SQL standard) --- not knowing your table & column names, I >> can only guess, though ... >> >> Stefan >> >> >> ----- Oorspronkelijk bericht ----- >> Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> >> Verzonden: maandag 25 oktober 2010 16:03 >> Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> >> Onderwerp: [MonetDB-users] Querying date type column in MonetDB >> >> Dear All, >> >> I am new to MonetDB, so most likely my question will look obvious. >> >> I would like to ask, how can columns with date type is queried in monetdb. >> Based on my experience, it seems that date type columns need special >> treatment. It seems that the only way to query them is with the syntax >> "column_name" while other columns seems to work with the format >> table_name.column_name without the need of double quote. When I tried to >> query date column with the syntax table_name.column_name, it gave me an >> error "no such column" >> >> So my question is, is it true that the only way to query date columns is >> using "column_name" format? Another question will be, what happen if I need >> to query same name columns coming from different table? >> >> Thanks in advance, >> Freddy >> >> >> ------------------------------------------------------------------------------ >> Nokia and AT&T present the 2010 Calling All Innovators-North America >> contest >> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada >> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in >> marketing >> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >> http://p.sf.net/sfu/nokia-dev2dev >> _______________________________________________ >> MonetDB-users mailing list >> MonetDB-users(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-users >> > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users -- ==================================================================== CWI, Kamer L330 Centrum voor Wiskunde en Informatica Science Park 123 Email: arjen.de.rijke(a)cwi.nl 1098 XG Amsterdam tel: +31-(0)20-5924305 Nederland +31-(0)6-51899284 fax: +31-(0)20-5924312 ===================== http://www.cwi.nl/~rijke/ ==================== ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

1 0

Re: [MonetDB-users] Querying date type column in MonetDB
by Stefan Manegold 25 Oct '10

25 Oct '10

Hi Freddy, I don't think that the error you experience is triggered by the column type, but rather by the fact that your date column ha? a name - say "date" - that is a reserved key word in SQL an hence needs to be double-quoted according to the SQL standard (this is not a MonetDB specific choice, but a "feature" of the SQL standard) --- not knowing your table & column names, I can only guess, though ... Stefan ----- Oorspronkelijk bericht ----- Van: Freddy Priyatna <freddy.priyatna(a)gmail.com> Verzonden: maandag 25 oktober 2010 16:03 Aan: MonetDB-users <monetdb-users(a)lists.sourceforge.net> Onderwerp: [MonetDB-users] Querying date type column in MonetDB Dear All, I am new to MonetDB, so most likely my question will look obvious. I would like to ask, how can columns with date type is queried in monetdb. Based on my experience, it seems that date type columns need special treatment. It seems that the only way to query them is with the syntax "column_name" while other columns seems to work with the format table_name.column_name without the need of double quote. When I tried to query date column with the syntax table_name.column_name, it gave me an error "no such column" So my question is, is it true that the only way to query date columns is using "column_name" format? Another question will be, what happen if I need to query same name columns coming from different table? Thanks in advance, Freddy

2 1

[MonetDB-users] Querying date type column in MonetDB
by Freddy Priyatna 25 Oct '10

25 Oct '10

Dear All, I am new to MonetDB, so most likely my question will look obvious. I would like to ask, how can columns with date type is queried in monetdb. Based on my experience, it seems that date type columns need special treatment. It seems that the only way to query them is with the syntax "column_name" while other columns seems to work with the format table_name.column_name without the need of double quote. When I tried to query date column with the syntax table_name.column_name, it gave me an error "no such column" So my question is, is it true that the only way to query date columns is using "column_name" format? Another question will be, what happen if I need to query same name columns coming from different table? Thanks in advance, Freddy

1 0

[MonetDB-users] calc.date in SQL to MAL.
by Guillaume Yziquel 22 Oct '10

22 Oct '10

Hi. I have resumed work on the OCaml binding for MonetDB. I have been playing a bit with it and with MonetDB, and noticed something rather weird when trying to explain a SQL statement to MAL. Compilation, and launching a small test in an OCaml toplevel: yziquel@seldon:~/data$ make ocamlbuild -use-ocamlfind import.cma Finished, 3 targets (0 cached) in 00:00:00. yziquel@seldon:~/data$ make test cp ocamlinit _build/ cd _build; ledit ocaml -init ocamlinit Objective Caml version 3.12.0 Findlib has been successfully loaded. Additional directives: #require "package";; to load a package #list;; to list the available packages #camlp4o;; to load camlp4 (standard syntax) #camlp4r;; to load camlp4 (revised syntax) #predicates "p,q,...";; to set these predicates Topfind.reset();; to force that packages will be reloaded #thread;; to enable threads /usr/lib/ocaml/csv: added to search path /usr/lib/ocaml/csv/csv.cma: loaded /usr/lib/ocaml/unix.cma: loaded /usr/lib/ocaml/monetdb5: added to search path /usr/lib/ocaml/monetdb5/monetDB5.cma: loaded /usr/lib/ocaml/dynlink.cma: loaded /usr/lib/ocaml/camlp4: added to search path /usr/lib/ocaml/pcre: added to search path /usr/lib/ocaml/pcre/pcre.cma: loaded /usr/local/lib/ocaml/3.12.0/tophide: added to search path /usr/local/lib/ocaml/3.12.0/tophide/tophide.cmo: loaded /usr/local/lib/ocaml/3.12.0/mikmatch_pcre: added to search path /usr/local/lib/ocaml/3.12.0/mikmatch_pcre/run_mikmatch_pcre.cma: loaded So much for the prologue, and dynamic loading. So the test starts here. Everytime you see "Asking", it is the statement that is fed to the connection handler in order to retrieve the query handler. (With mapilib). Asking: select name from tables; Asking: create table indu (Date date, Open decimal(15,2), High decimal(15,2), Low decimal(15,2), Close decimal(15,2), Volume decimal(15,2), OI decimal(15,2)); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-02-01', 108.80, 108.80, 108.80, 108.80, 1156, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-05-01', 108.90, 108.90, 108.90, 108.90, 1305, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-06-01', 107.40, 107.40, 107.40, 107.40, 1251, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-07-01', 107.60, 107.60, 107.60, 107.60, 771, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-08-01', 107.20, 107.20, 107.20, 107.20, 843, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-09-01', 106.60, 106.60, 106.60, 106.60, 736, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-12-01', 104.20, 104.20, 104.20, 104.20, 1049, 0); Asking: insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-13-01', 104.50, 104.50, 104.50, 104.50, 908, 0); And on this last statement, the server doesn't seem happy at all: Exception: MonetDB5.Mapi.Error (Some (MonetDB5.Mapi.MSERVER, None)). So I've been trying to 'explain' the SQL to MAL. The first two work fine and output a very similar MAL statement. The third SQL statements, however, is getting translated to something somewhat different: You see a call to 'calc.date' on the 4th MAL line: # ask "explain insert into indu (Date, Open, High, Low, Close, Volume, # OI) values ('1920-07-01', 107.60, 107.60, 107.60, 107.60, 771, 0);";; Asking: explain insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-07-01', 107.60, 107.60, 107.60, 107.60, 771, 0); - : string option list list = [[Some "function user.s0_1{autoCommit=true}():void;"]; [Some " _2 := sql.mvc();"]; [Some "barrier _32 := language.dataflow();"]; [Some " _3 := sql.append(_2,\"sys\",\"indu\",\"date\",1920-07-01);"]; [Some " _7 := sql.append(_3,\"sys\",\"indu\",\"open\",10760:lng);"]; [Some " _9 := sql.append(_7,\"sys\",\"indu\",\"high\",10760:lng);"]; [Some " _11 := sql.append(_9,\"sys\",\"indu\",\"low\",10760:lng);"]; [Some " _13 := sql.append(_11,\"sys\",\"indu\",\"close\",10760:lng);"]; [Some " _15 := sql.append(_13,\"sys\",\"indu\",\"volume\",77100:lng);"]; [Some " sql.append(_15,\"sys\",\"indu\",\"oi\",0:lng);"]; [Some "exit _32;"]; [Some " sql.affectedRows(1:wrd,\"\");"]; [Some "end s0_1;"]] # ask "explain insert into indu (Date, Open, High, Low, Close, Volume, # OI) values ('1920-12-01', 104.20, 104.20, 104.20, 104.20, 1049, 0);";; Asking: explain insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-12-01', 104.20, 104.20, 104.20, 104.20, 1049, 0); - : string option list list = [[Some "function user.s1_1{autoCommit=true}():void;"]; [Some " _2 := sql.mvc();"]; [Some "barrier _32 := language.dataflow();"]; [Some " _3 := sql.append(_2,\"sys\",\"indu\",\"date\",1920-12-01);"]; [Some " _7 := sql.append(_3,\"sys\",\"indu\",\"open\",10420:lng);"]; [Some " _9 := sql.append(_7,\"sys\",\"indu\",\"high\",10420:lng);"]; [Some " _11 := sql.append(_9,\"sys\",\"indu\",\"low\",10420:lng);"]; [Some " _13 := sql.append(_11,\"sys\",\"indu\",\"close\",10420:lng);"]; [Some " _15 := sql.append(_13,\"sys\",\"indu\",\"volume\",104900:lng);"]; [Some " sql.append(_15,\"sys\",\"indu\",\"oi\",0:lng);"]; [Some "exit _32;"]; [Some " sql.affectedRows(1:wrd,\"\");"]; [Some "end s1_1;"]] # ask "explain insert into indu (Date, Open, High, Low, Close, Volume, # OI) values ('1920-13-01', 104.50, 104.50, 104.50, 104.50, 908, 0);";; Asking: explain insert into indu (Date, Open, High, Low, Close, Volume, OI) values ('1920-13-01', 104.50, 104.50, 104.50, 104.50, 908, 0); - : string option list list = [[Some "function user.s2_1{autoCommit=true}():void;"]; [Some " _2 := sql.mvc();"]; [Some "barrier _33 := language.dataflow();"]; [Some " _3 := calc.date(\"1920-13-01\");"]; [Some " _4 := sql.append(_2,\"sys\",\"indu\",\"date\",_3);"]; [Some " _8 := sql.append(_4,\"sys\",\"indu\",\"open\",10450:lng);"]; [Some " _10 := sql.append(_8,\"sys\",\"indu\",\"high\",10450:lng);"]; [Some " _12 := sql.append(_10,\"sys\",\"indu\",\"low\",10450:lng);"]; [Some " _14 := sql.append(_12,\"sys\",\"indu\",\"close\",10450:lng);"]; [Some " _16 := sql.append(_14,\"sys\",\"indu\",\"volume\",90800:lng);"]; [Some " sql.append(_16,\"sys\",\"indu\",\"oi\",0:lng);"]; [Some "exit _33;"]; [Some " sql.affectedRows(1:wrd,\"\");"]; [Some "end s2_1;"]] # Why is that happening? The SQL statements seem so similar that I do not see why it gets compiled differently to MAL. Best regards, -- Guillaume Yziquel http://yziquel.homelinux.org

2 4

[MonetDB-users] installation corrupted
by Nafees Ur Rehman 20 Oct '10

20 Oct '10

Hi, I am using monetDB/SQL for windows on an xp machine. Two days ago, the server stopped responding. Tried to repair and change it...but didn't work. Also got it formally un-installed and then tried re-installation...but no luck. Any hints. The server does not mount at all. Regards Nafees Ur Rehman Konstanz, Germany

1 0

[MonetDB-users] MonetDB performance question
by Tudor-Ioan Salomie 18 Oct '10

18 Oct '10

Hi, I have recently tried doing some experiments with MonetDB, measuring its performance for the following dataset+workload: * 1 table (50 attributes), 1GB, comprising 3.000.000 tuples * the attributes are fixed length strings and integer types * the workload is query (SELECT) only, where each query has the form: SELECT <list_of_projections> FROM MyTable WHERE <list_of_predicates>; * the list of predicates is a conjunction of equality operators * more than 99% of the queries predicate on the same attribute (let's call it A1) that has a *very* good selectivity * the average number of predicated attributes per query is 8 * for this workload, averaging over 4000 queries, there is 1 result tuple per query (actual result tuples per query are in the range 0-10) I have installed MonetDB on a 64-bit Debian machines, using the .deb files provided on the internet: * libmonetdb1_1.38.5-20100824_amd64.deb * libmonetdb5-server5_5.20.5-20100824_amd64.deb * libmonetdb5-sql2_2.38.5-20100824_amd64.deb * libmonetdb-client1_1.38.5-20100824_amd64.deb * monetdb5-server_5.20.5-20100824_amd64.deb * monetdb5-sql_2.38.5-20100824_amd64.deb * monetdb-client_1.38.5-20100824_amd64.deb For running the queries I use the SQL front end. I have run the queries both through "mclient" (directly on the server) and through the JDBC driver (from another client machine). The MonetDB server is bound to run on only one core on the machine (though I have not seen any difference if running it on multiple cores). To my surprise, the throughput for MonetDB for this workload is very low. I get about 3 queries per second, which seems to indicate that MonetDB does full table scanning only. Am I correct in this assumption? Is there anything that can be done to speed it up? Is this not a workload suitable for MonetDB? Looking forward to your input! Regards, Tudor.

2 1

[MonetDB-users] FW: Comment via CWI webform MonetDB
by Peter Boncz 15 Oct '10

15 Oct '10

Narayan Sujay, Thanks for your interest in MonetDB. Rather than directing technical questions to the CWI organization webmaster, you are supposed to post on one of the relevant mailing lists such as this one (monetdb-user(a)lists.sourceforge.net) Peter -----Original Message----- From: Minnie Middelberg [mailto:Minnie.Middelberg@cwi.nl] Sent: Friday, October 15, 2010 9:50 AM To: Peter Boncz Subject: Fwd: Comment via CWI webform MonetDB Hallo Peter, Onderstaand is binnengekomen via het CWI Comment formulier en het betreft MonetDB. Wetende dat jij en Marting Kersten achter MonetDB zitten forward ik het maar naar jou. Mocht er iemand zijn die dit soort vragen belast is dan hoor ik het graag van je. Mvg Minnie ----- Forwarded Message ----- From: "Narayan Sujay" <316sujay(a)gmail.com> To: voorlichting(a)cwi.nl, webmaster(a)cwi.nl Sent: Thursday, 14 October, 2010 6:15:15 PM Subject: Comment via CWI webform Submitted on 14.10.2010 - 18:15 Submitted by anonymous user: [198.134.51.12] Submitted values are: Name sender: Narayan Sujay Email address: 316sujay(a)gmail.com Telephone number: 4193780532 Your comment: Hello, I have downloaded MonetDB and the native client and server work well. I can write simple queries in the client cmd window. I am trying to connect MonetDB with one of the SQL frontends like DB visualizer etc and I am not able to do this. Please help me out with instructions. I followed the instructions on the webpage but still getting errors connecting through db visualizer. I think it is the problem with the JDBC drivers. Please let me know excatly where to find the driver on the local computer after I have installed Monet DB. My Main goal here is to load large Data sets in MonetDB. Please tell me a way to do this for Monet DB. The error I get in DB Visualizer is: Long Message: Cannot connect to ORB Details: Type: javax.naming.CommunicationException Stack Trace: javax.naming.CommunicationException: Cannot connect to ORB [Root exception is org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No] at com.sun.jndi.cosnaming.CNCtx.setOrbAndRootContext(Unknown Source) at com.sun.jndi.cosnaming.CNCtx.initOrbAndRootContext(Unknown Source) at com.sun.jndi.cosnaming.CNCtx.<init>(Unknown Source) at com.sun.jndi.cosnaming.CNCtxFactory.getInitialContext(Unknown Source) at javax.naming.spi.NamingManager.getInitialContext(Unknown Source) at javax.naming.InitialContext.getDefaultInitCtx(Unknown Source) at javax.naming.InitialContext.init(Unknown Source) at javax.naming.InitialContext.<init>(Unknown Source) at com.onseven.dbvis.db.A.G.Ą(Z:3155) at com.onseven.dbvis.db.A.G.Ĭ(Z:2675) at com.onseven.dbvis.db.A.E.Ĭ(Z:98) at com.onseven.dbvis.db.A.D.ā(Z:2056) at se.pureit.swing.A.O$2.run(Z:1192) at java.lang.Thread.run(Unknown Source) Caused by: org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 201 completed: No at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(Unknown Source) at com.sun.corba.se.impl.logging.ORBUtilSystemException.connectFailure(Unknown Source) at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(Unknown Source) at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.<init>(Unknown Source) at com.sun.corba.se.impl.transport.SocketOrChannelContactInfoImpl.createConnection(Unknown Source) at com.sun.corba.se.impl.protocol.CorbaClientRequestDispatcherImpl.beginRequest(Unknown Source) at com.sun.corba.se.impl.protocol.CorbaClientDelegateImpl.request(Unknown Source) at com.sun.corba.se.impl.resolver.BootstrapResolverImpl.invoke(Unknown Source) at com.sun.corba.se.impl.resolver.BootstrapResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.resolver.CompositeResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.resolver.CompositeResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.resolver.CompositeResolverImpl.resolve(Unknown Source) at com.sun.corba.se.impl.orb.ORBImpl.resolve_initial_references(Unknown Source) ... 14 more Caused by: java.net.ConnectException: Connection refused: connect at sun.nio.ch.Net.connect(Native Method) at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) at java.nio.channels.SocketChannel.open(Unknown Source) at com.sun.corba.se.impl.transport.DefaultSocketFactoryImpl.createSocket(Unknown Source) ... 25 more The results of this submission may be viewed at: http://www.cwi.nl/en/node/757/submission/3152

1 0

[MonetDB-users] upgrade notes: Feb2010 --> Jun2010
by Mark Bucciarelli 07 Oct '10

07 Oct '10

Hi, I'd like to upgrade to the June release. I'm running Feb 2010. Can I just build the June release and expect it will just work with my database? Or do I need to upgrade the data somehow? I poked around dev.monetdb.org/hg, looking for release notes, changelist, etc. but didn't find anything. Thanks, m

2 3

[MonetDB-users] bug #2634 in Jun2010? [was: upgrade notes: Feb2010 --> Jun2010]
by Mark Bucciarelli 07 Oct '10

07 Oct '10

On Thu, Oct 7, 2010 at 9:15 AM, Sjoerd Mullender <sjoerd(a)acm.org> wrote: > On 2010-10-07 14:41, Mark Bucciarelli wrote: > >> I poked around dev.monetdb.org/hg, looking for release >> notes, changelist, etc. but didn't find anything. > > http://monetdb.cwi.nl/Development/Releases/Jun2010/ > Thanks. Was the commit in bug #2634 in the release? I don't see it in the SQL notes. He closed as NEEDS MORE INFO (I think---I can't tell from ticket history) because I did not include a test case. But he did commit some change that fixed the issue (I was able to upload a test case this week, and verified his fix solved my case). Thanks, m http://bugs.monetdb.org/show_bug.cgi?id=2634

2 1

[MonetDB-users] Performance improvements over equal queries limits to operators?
by Stefan de Konink 06 Oct '10

06 Oct '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, I was quite surprised about the performance of some queries, that I expected to be heavy weighted, but actually performed pretty well. To show some things: sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (259.000ms) sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (259.000ms) sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (258.000ms) sql>select * from kvk where postcode like '2265CA' and adres like '%7%'; 3 tuples (257.000ms) sql>select * from kvk where postcode = '2265CA' and adres like '%7%'; 3 tuples (11.000ms) sql>select * from kvk where postcode = '2265CA' and adres like '%7%'; 3 tuples (6.000ms) sql>select * from kvk where postcode = '2265CA' and adres like '%7%'; 3 tuples (6.000ms) sql>select * from kvk where kvk = 273121520000; 1 tuple (473.000ms) sql>select * from kvk where kvk = 273121520000; 1 tuple (5.000ms) I wonder: what makes the 11ms to 6ms possible in the like vs = scenario? While a repeat of the like operator itself doesn't give a true improvement? As you can observe the like argument doesn't have any modifiers, so it should actually become a normal equal. (I noticed this later, goes beyond my original point, will file a bug for enhancement.) Also I'm quite surprised that the primary key lookup still takes a fair amount of time, could be i/o related obviously, but the performance on the second run is again huge, even for other arguments. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkysWw8ACgkQYH1+F2Rqwn19AgCgk9lkRBLcWdylxxz9M4B8a8f5 5JUAn1TOU1VxpvVfVWJMLwlwp9Pz4a0x =IqCY -----END PGP SIGNATURE-----

2 1

[MonetDB-users] Dump schema w/ mclient?
by Mark Bucciarelli 05 Oct '10

05 Oct '10

Is there a way to user mclient to dump the db schema without dumping the data? I tried creating a one line SQL file with \d <tablename> and piping to mclient stdin, but it didn't work. Thanks, m

3 2

[MonetDB-users] add a unique constraint, need to create index?
by Mark Bucciarelli 04 Oct '10

04 Oct '10

Hi, I added a unique constraint on two fields, and noticed there was not an index created on those two fields. Do I need to do a CREATE INDEX or is it implicit? I will be using those two fields a lot in WHERE clauses. Thanks, m

1 1