- users-list - monetdb.org

Announcement: New Feb2013-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 29 Apr '13

29 Apr '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>.

1 0

All tested on the same column, some functions work, some don't?
by Anthony Damico 24 Apr '13

24 Apr '13

Hi, I'm using MonetDB on Windows with the MonetDB.R connector. I'm getting some very strange errors on a particular column (called `rx_cso_num`) where *some* of the math functions work and others do not? I tested the median() function using mclient - I get the same error - so this has nothing to do with the fact that I'm connecting through R. I'm running these queries on a confidential data set, so unfortunately I cannot provide a reproducible example with this.. I'm sorry for the hassle, and hoping the error messages below might provide enough of a clue to resolve the issue? If this really isn't enough, I can send some snipped mclient -X output and/or try my best to create some fake data that also creates some of these problems? Thanks!! :) Here's a list of functions that *work* on the column without a problem: "count","distinct","min", "max", "sum","avg","abs","sqrt","floor","ceiling","exp","log","cos","sin","tan","acos","asin","atan","cosh","sinh","tanh" Here's the errors I get for each of the functions that do not: [1] "median" Error in dbGetQuery(db, paste0("select ", i, "(rx_cso_num) from x07")) : Unable to execute statement 'select median(rx_cso_num) from x07'. Server says '!22003!overflow in conversion of 133 to bte.'. [1] "sign" Error in dbGetQuery(db, paste0("select ", i, "(rx_cso_num) from x07")) : Unable to execute statement 'select sign(rx_cso_num) from x07'. Server says '!TypeException:user.s5_1[25]:'bat.insert' undefined in: _93:any := bat.insert(_83:bat[:oid,:int], _91:oid, _90:bte)'. [1] "stddev_pop" Error in dbGetQuery(db, paste0("select ", i, "(rx_cso_num) from x07")) : Unable to execute statement 'select stddev_pop(rx_cso_num) from x07'. Server says '!SELECT: no such unary operator 'stddev_pop(wrd)''. [1] "stddev" Error in dbGetQuery(db, paste0("select ", i, "(rx_cso_num) from x07")) : Unable to execute statement 'select stddev(rx_cso_num) from x07'. Server says '!SELECT: no such unary operator 'stddev(wrd)''. [1] "prod" Error in dbGetQuery(db, paste0("select ", i, "(rx_cso_num) from x07")) : Unable to execute statement 'select prod(rx_cso_num) from x07'. Server says '!22003!overflow in conversion of 133 to bte.'. Here's my mserver window: # MonetDB 5 server v11.15.3 "Feb2013-SP1" # Serving database 'medicare_sample', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 7.860 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:49800/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded > Thanks!!!

2 9

problem with foreign key constrain violation
by Gijs Molenaar 23 Apr '13

23 Apr '13

Hi MonetDB, I think I'm really confused or I found a bug. Please have a look below. I'm expecting a 'aggregation_book_contact_id_fkey' violation, since that row is clearly missing. But I'm getting a 'aggregation_book.aggregation_book_publisher_id_fkey' violation. Is it me or is this error message wrong? sql>\d sys.aggregation_book CREATE TABLE "sys"."aggregation_book" ( "id" INTEGER NOT NULL DEFAULT next value for "sys"."seq_6205", "isbn" VARCHAR(9) NOT NULL, "name" VARCHAR(255) NOT NULL, "pages" INTEGER NOT NULL, "rating" DOUBLE NOT NULL, "price" DECIMAL(6,2) NOT NULL, "contact_id" INTEGER NOT NULL, "publisher_id" INTEGER NOT NULL, "pubdate" DATE NOT NULL, CONSTRAINT "aggregation_book_id_pkey" PRIMARY KEY ("id"), CONSTRAINT "aggregation_book_contact_id_fkey" FOREIGN KEY ("contact_id") REFERENCES "sys"."aggregation_author" ("id"), CONSTRAINT "aggregation_book_publisher_id_fkey" FOREIGN KEY ("publisher_id") REFERENCES "sys"."aggregation_publisher" ("id") ); sql>select * from aggregation_publisher; +------+--------+------------+ | id | name | num_awards | +======+========+============+ | 1 | Apress | 3 | +------+--------+------------+ 1 tuple (0.588ms) sql>select * from aggregation_author; +----+------+-----+ | id | name | age | +====+======+=====+ +----+------+-----+ 0 tuples (0.570ms) sql>INSERT INTO "aggregation_book" ("id", "isbn", "name", "pages", more>"rating", "price", "contact_id", "publisher_id", "pubdate") VALUES (1, more>'159059725', 'The Definitive Guide to Django: Web Development Done more>Right', 447, 4.5, '30.00', 1, 1, '2007-12-06'); INSERT INTO: FOREIGN KEY constraint 'aggregation_book.aggregation_book_publisher_id_fkey' violated -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

4 6

Re: Bug grouping on string column, only when results filtered
by Stefan Manegold 17 Apr '13

17 Apr '13

as string-related bug has recently been fixed (see MonetDB bug tracker; I don't recall the bug ID right now) and will be shipped with the upcoming Feb2013-SP2 release, a preview of which is avilable from the testing repository. Stefan Pete Hollobon <hollobon(a)gmail.com> wrote: Hi Hannes, Thanks for looking into this. I'm using a 64-bit build. Here's my uname and the versions from the MonetDB binaries. ~% uname -a Linux lx1 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux ~% /usr/bin/monetdbd version MonetDB Database Server v1.7 (Feb2013-SP1) ~% /usr/bin/monetdb version MonetDB Database Server Toolkit v1.1 (Feb2013-SP1) ~% /usr/bin/mclient --version mclient, the MonetDB interactive terminal (Feb2013-SP1) support for command-line editing compiled-in character encoding: UTF-8 However, I realised that my local Feb2013 build was using the shared libraries from the Ubuntu debs. I fixed this, and that build now seems to be working properly. I can still reproduce it with the distributed binaries though. Whatever bug it was must have been fixed already - apologies for wasting your time. Regards, Pete. On 17 April 2013 12:56, Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl> wrote: > Hello Pete, > > thank you for your bug report and the effort you have put into making this > reproducible for us. However, having followed your instructions with both > the Feb2013 version as well as the current development branch, I was not > able to also provoke this issue. > > I got the following results (Both Feb2013 and default branch): > > ➜ ~ mclient bug -s "select a, count(*) from test where b = 20227 group by > a" > +------+--------+ > | a | L1 | > +======+========+ > | 2T | 193058 | > | 1T | 202267 | > | 5T | 87038 | > | 4T | 100302 | > | 3T | 142326 | > | 15T | 95 | > | 20T | 95 | > | 30T | 14277 | > | 7T | 767 | > | 6T | 22863 | > > | 10T | 127 | > | 6V | 216594 | > | 9T | 104 | > | 8T | 88 | > +------+--------+ > 14 tuples > > ➜ ~ mclient bug -s "select a, count(*) from test group by a" > > +------+---------+ > | a | L1 | > +======+=========+ > | 2T | 5799418 | > | 1T | 6072825 | > | 5T | 2605374 | > | 4T | 2998157 | > | 3T | 4262607 | > | 15T | 2703 | > | 20T | 2818 | > | 30T | 497657 | > | 7T | 22460 | > | 6T | 686789 | > | 10T | 3496 | > | 6V | 6550540 | > | 9T | 1853 | > | 8T | 1639 | > +------+---------+ > 14 tuples > > > Can you perhaps give more information abuont your environment? For example, > this a 64 or 32 bit build? You could also try the development default branch > and see if the problem also occurs there. > > Best, > > Hannes > > > > > > On 17/04/13 10:33, Pete Hollobon wrote: >> >> Hi, >> >> I think I've found a bug in MonetDB. It only occurs when grouping by a >> text column and filtering the results; with no "where" clause, the >> results are fine. The table is fairly large (~30 million rows). >> >> ~% ./generate_test_csv.py test.csv >> ~% mclient bug -s "create table test (b integer, a varchar(5))" >> operation successful >> ~% mclient bug -s "copy into test FROM '/home/pete/test.csv' DELIMITERS >> ','" >> 29508336 affected rows >> >> There are no nulls or empty strings in column "a", however: >> >> ~% mclient bug -s "select a, count(*) from test where b = 20227 group by >> a" >> +------+--------+ >> | a | L1 | >> +======+========+ >> | | 763088 | >> | 10T | 127 | >> | 6V | 216594 | >> | 9T | 104 | >> | 8T | 88 | >> +------+--------+ >> >> When the "where" clause is removed, no nulls are returned: >> >> ~% mclient bug -s "select a, count(*) from test group by a" >> +------+---------+ >> | a | L1 | >> +======+=========+ >> | 2T | 5799418 | >> | 1T | 6072825 | >> | 5T | 2605374 | >> | 4T | 2998157 | >> | 3T | 4262607 | >> | 15T | 2703 | >> | 20T | 2818 | >> | 30T | 497657 | >> | 7T | 22460 | >> | 6T | 686789 | >> | 10T | 3496 | >> | 6V | 6550540 | >> | 9T | 1853 | >> | 8T | 1639 | >> +------+---------+ >> 14 tuples >> >> I have tested with the latest MonetDB Ubuntu debs and also with a >> local build of the Hg Feb2013 branch tip. Both exhibit the same >> problem. I'm running Linux (Ubuntu 12.10) on a 6 core Xeon with 16Gb >> RAM. I've tried with minimal_pipe and the results are the same. >> >> Please let me know if any other information would be useful. >> >> Regards, >> Pete. >> >> >> >> _______________________________________________ >> users-list mailing list >> users-list(a)monetdb.org >> http://mail.monetdb.org/mailman/listinfo/users-list >> > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

Bug grouping on string column, only when results filtered
by Pete Hollobon 17 Apr '13

17 Apr '13

Hi, I think I've found a bug in MonetDB. It only occurs when grouping by a text column and filtering the results; with no "where" clause, the results are fine. The table is fairly large (~30 million rows). ~% ./generate_test_csv.py test.csv ~% mclient bug -s "create table test (b integer, a varchar(5))" operation successful ~% mclient bug -s "copy into test FROM '/home/pete/test.csv' DELIMITERS ','" 29508336 affected rows There are no nulls or empty strings in column "a", however: ~% mclient bug -s "select a, count(*) from test where b = 20227 group by a" +------+--------+ | a | L1 | +======+========+ | | 763088 | | 10T | 127 | | 6V | 216594 | | 9T | 104 | | 8T | 88 | +------+--------+ When the "where" clause is removed, no nulls are returned: ~% mclient bug -s "select a, count(*) from test group by a" +------+---------+ | a | L1 | +======+=========+ | 2T | 5799418 | | 1T | 6072825 | | 5T | 2605374 | | 4T | 2998157 | | 3T | 4262607 | | 15T | 2703 | | 20T | 2818 | | 30T | 497657 | | 7T | 22460 | | 6T | 686789 | | 10T | 3496 | | 6V | 6550540 | | 9T | 1853 | | 8T | 1639 | +------+---------+ 14 tuples I have tested with the latest MonetDB Ubuntu debs and also with a local build of the Hg Feb2013 branch tip. Both exhibit the same problem. I'm running Linux (Ubuntu 12.10) on a 6 core Xeon with 16Gb RAM. I've tried with minimal_pipe and the results are the same. Please let me know if any other information would be useful. Regards, Pete.

2 2

beginning with monetdb on windows
by Satoshi Nagayasu 09 Apr '13

09 Apr '13

Hi, I'm new to the MonetDB, and trying MonetDB on Windows. At first, I have installed MonetDB on Windows with using MonetDB5-SQL-Installer-x86_64-20130227.msi. http://dev.monetdb.org/downloads/Windows/Feb2013-SP1/ Then, I attempted to follow the Getting Started guide. http://www.monetdb.org/Documentation/Tutorial However, I couldn't find monetdb.exe(bat?) and monetdbd.exe(bat?) in my monetdb installed directory. What can I do for the first step? Any comments? Regards, -- Satoshi Nagayasu <snaga(a)uptime.jp> Uptime Technologies, LLC. http://www.uptime.jp

4 6

MonetDB server crashing on Windows for seemingly simple SELECT statements
by Anthony Damico 01 Apr '13

01 Apr '13

Hi, I'm hitting some very strange errors with a few of my data tables. The data users agreement for these datasets does not allow me to share the actual output and I'm having trouble getting a similar error to occur on public data, so hopefully my description of what's going on will help the troubleshooting. I can send more information about the structure of these tables if that would be helpful? This command causes mserver.exe to crash every time: select state_cd , sum(sex) from x01 where (bene_age_at_end_ref_yr = 64) group by state_cd order by state_cd; This command causes mserver.exe to crash after running it three consecutive times. The first two times, it works. select state_cd , sum(sex) from x01 where (bene_age_at_end_ref_yr < 65) group by state_cd order by state_cd; ALL of these commands work without any trouble on the same data table: select state_cd from x01 where (bene_age_at_end_ref_yr > 64) group by state_cd order by state_cd; select state_cd , sum( metro ) as metro from x01 group by state_cd order by state_cd; select state_cd , sum( metro ) as metro from x01 where (bene_age_at_end_ref_yr > 64) group by state_cd order by state_cd; select distinct bene_age_at_end_ref_yr from x01 order by bene_age_at_end_ref_yr' ) select state_cd , count(sex) from x01 where (bene_age_at_end_ref_yr = 64) group by state_cd order by state_cd; select state_cd , sum(sex) from x01 group by state_cd order by state_cd; select state_cd , sum(sex) from x01 where (bene_age_at_end_ref_yr > 64) group by state_cd order by state_cd; select state_cd , sum(sex) from x01 where (bene_age_at_end_ref_yr >= 64) group by state_cd order by state_cd; I have no idea where to start troubleshooting what's going on here. Can anyone recommend additional tests to diagnose the root of the problem? I really don't understand how flipping the greater than sign on the otherwise same query could cause it to break.. I've pasted a bunch of diagnostics and the mclient in -X mode below, but it really doesn't seem helpful. Thanks!!!!! Here's my mserver.exe version: # MonetDB 5 server v11.15.1 "Feb2013" # Serving database 'medicare_sample', using 4 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 20.000 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:49800/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded > When running the first command that breaks inside mclient -X, here's what happens: user(win32):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Feb2013) mapi_query:128:SELECT "name", "value" FROM sys.env() AS env WHERE "name" IN ('gd k_dbname', 'monet_version', 'monet_release', 'merovingian_uri') fetch next block: start at:4 got next block: length:200 text:&1 0 3 2 3 % .env, .env # table_name % name, value # name % varchar, varchar # type % 13, 15 # length [ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] got complete block: text:&1 0 3 2 3 % .env, .env # table_name % name, value # name % varchar, varchar # type % 13, 15 # length [ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:&1 0 3 2 3 allocating new result set got complete block: text:% .env, .env # table_name % name, value # name % varchar, varchar # type % 13, 15 # length [ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:% .env, .env # table_name got complete block: text:% name, value # name % varchar, varchar # type % 13, 15 # length [ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:% name, value # name got complete block: text:% varchar, varchar # type % 13, 15 # length [ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:% varchar, varchar # type got complete block: text:% 13, 15 # length [ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:% 13, 15 # length got complete block: text:[ "gdk_dbname", "medicare_sample" ] [ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:[ "gdk_dbname", "medicare_sample" ] got complete block: text:[ "monet_version", "11.15.1" ] [ "monet_release", "Feb2013" ] read_line:[ "monet_version", "11.15.1" ] got complete block: text:[ "monet_release", "Feb2013" ] read_line:[ "monet_release", "Feb2013" ] fetch next block: start at:204 got next block: length:0 text: got complete block: text:? read_line:? Database: MonetDB v11.15.1 (Feb2013), 'medicare_sample' closing result set Type \q to quit, \? for a list of available commands auto commit mode: on mapi_query:46:SET TIME ZONE INTERVAL '-05:00' HOUR TO MINUTE fetch next block: start at:206 got next block: length:3 text:&3 got complete block: text:&3 read_line:&3 allocating new result set fetch next block: start at:209 got next block: length:0 text: got complete block: text:? read_line:? closing result set sql>select state_cd , sum(sex) from x01 where (bene_age_at_end_ref_yr = 64) grou p by state_cd order by state_cd; mapi_query_part:109:select state_cd , sum(sex) from x01 where (bene_age_at_end_r ef_yr = 64) group by state_cd order by state_cd; fetch next block: start at:211 Connection terminated Press any key to continue . . . When running the second command that breaks inside mclient -X, here's what happens: The first two times, the command runs fine.. Then the third time, it gives: read_line:? +----------+-------+ 67 tuples (241.424ms) closing result set sql>select state_cd , sum(sex) from x01 where (bene_age_at_end_ref_yr < 65) grou p by state_cd order by state_cd; mapi_query_part:109:select state_cd , sum(sex) from x01 where (bene_age_at_end_r ef_yr < 65) group by state_cd order by state_cd; fetch next block: start at:2403 Connection terminated Press any key to continue . . .

3 2

select + group by expression alias from a view breaks connection
by Budulinku Dejmihrasku 29 Mar '13

29 Mar '13

Hi, i hit a strange problem when using alias for an expression in group by i run into this with a complex view containing multiple UNION ALL queries but after a while everything boiled down to a fairly simple test case that will always produce the error Welcome to mclient, the MonetDB/SQL interactive terminal (Feb2013-SP1) Database: MonetDB v11.15.3 (Feb2013-SP1), 'mapi:monetdb://lux-analytics:50000/zkdev1' Type \q to quit, \? for a list of available commands auto commit mode: on sql>CREATE VIEW mmtest_v AS SELECT date'2012-01-01' as period, 1 as quantity; operation successful (213.074ms) sql>SELECT extract(month from period) as m, count(1) as cnt FROM mmtest_v GROUP BY m; *Connection terminated* I am on Ubuntu 12.04.2 LTS As the alias in GROUP BY works in other cases i expect this is a bug - pls confirm and i will be happy to file it. BTW it looks like MonetDB has generally issues with selects from more complex views - i.e. views based on another view(s). In such cases the results do not match expectations or there is even an error. I can reproduce the problem but i only on quite large set of data (typically 20+ mil. records) and after some data manipulation got involved, so i cannot give a simple test case as above. Any idea how to pass such problem (when large data-set is involved) to the development team? Thanks in advance milan

3 3

RAM consumption of MonetDB
by Hassan, Ahmad 27 Mar '13

27 Mar '13

Hi Team, Please may I ask what main memory management strategy has been adopted for MonetDB in terms of memory utilization. For example I loaded the TPCH benchmark dataset in MonetDB (v11.13.5) with scale factor 40. The data on disk is 43Gb in size. But when I run the TPCH queries then the main memory used by mserver stays in the range of 0.5GB to 3GB. I was thinking that the monetDB would load the tables into memory because TPCH queries are doing joins over various tables but it seems that monetdb loads only small subset of the complete dataset. Please can someone explain what is going in the background and why I always see 0.5GB to 3GB memory resident size of mserver process even for dataset of size 43G and if this is the normal expected behaviour of monetdb? The farm size on disk is: farms/f1/db1/bat# du . -hs 48G . Thanks. Kind Regards, Ahmad

3 5

Re: Quickstart Guide
by Stefan Manegold 20 Mar '13

20 Mar '13

Sterling, Thanks for notifing! Could you please file a bug report via http://bugs.monetdb.org/ ? Thanks! Stefan Sterling Paramore <gnilrets(a)gmail.com> wrote: The link at http://www.monetdb.org/Downloads/Tutorial is broken and I don't see any way to contact the creators of the web page. How can I get information about how to install MonetDB? Thanks, Sterling

4 4