- users-list - monetdb.org

[MonetDB-users] Announcement: New Aug2011-SP1 bugfix release of MonetDB suite
by Fabian Groffen 30 Sep '11

30 Sep '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Aug 2011-SP1 bugfix release SQL * Fixed a bug when using default values for interval columns. Bug 2877. * A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882. * A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883. Client Package * Fixed Perl DBD rowcount for larger results, bug #2889 * Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema). * Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois. * Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois. * mclient: fix display of varchar columns with only NULL values. * Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables. MonetDB5 Server * Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888. Bug Fixes * 2865: mserver5 --set gdk_nr_threads=2 --forcemito: deadlock during first SQL client connect on virgin (empty) DB * 2877: assertion failed when creating table with default interval * 2879: A SELECT query fails to produce some results * 2882: Delete of some records don't work on Monetdb * 2883: insert into t (select * from t) bogus in multithreaded execution * 2884: Mapi.pm and GetInfo.pm syntax fixes for array usage * 2885: DBD::monetdb table_info error * 2887: Division by zero error, when should not divide at 0 * 2888: test/null/null_in_case.sql hangs server * 2889: Mapi::getBlock is erasing query count on second call

1 0

[MonetDB-users] copy ... from stdin in perl script
by Eugene Prokopiev 28 Sep '11

28 Sep '11

Hi, I have table t1(f1 integer) I can copy to it by executing this file via mclient: $ cat /tmp/f1.sql copy into t1 from stdin; 1 2 3 But I can't copy to it by this perl script: $ cat /tmp/f1.pl #!/usr/bin/perl -w use DBI; my $dbh = DBI->connect('dbi:monetdb:database=db', 'dbadmin', "dbpwd"); $dbh->do("copy into t1 from stdin;\n1\n2\n3\n"); I got: Missing argument in sprintf at /usr/lib64/perl5/DBD/monetdb/GetInfo.pm line 25. No query result at /usr/lib64/perl5/MonetDB/CLI/MapiLib.pm line 80. Use of uninitialized value in die at /usr/lib64/perl5/MonetDB/CLI/MapiLib.pm line 180. (in cleanup) Died at /usr/lib64/perl5/MonetDB/CLI/MapiLib.pm line 180. Why can it be? -- Thanks, Eugene Prokopiev

2 9

[MonetDB-users] CSV Data Fault still used ?
by Stéphane NOUHAUD 26 Sep '11

26 Sep '11

Hi, I try to use CSV Data Fault (Documented here http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/CSVvault) without great succes... Just quick question is CSV Data Fault function still included in last version of MonetDB ? Except that MonetDB is really fantastic software. Thanks for sharing it ! Regards, -- *Stéphane NOUHAUD* * *

1 0

[MonetDB-users] Parallel COPY INTO
by zack_pcd 22 Sep '11

22 Sep '11

I have a large table with about 50 billion records. The files to be loaded are not in CSV format. The files are processed in java and the data uploaded to the database using COPY INTO through the MapiSocket interface. I do not have the record count readily available. My question is this: If I open multiple MapiSockets and upload multiple streams of data at the same time to the same table, is this supported and stable? I am asking because the database appears to become unresponsive after the bulk load. (mclient seems to hang on connect) If I stop and the start the database after the bulk load, I get: "failed to fork mserver: database 'test_db' has inconsistent state (sabaoth administration reports running, but process seems gone)" -- View this message in context: http://old.nabble.com/Parallel-COPY-INTO-tp32503834p32503834.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Partitioning on Monetdb
by Benji07 21 Sep '11

21 Sep '11

Does Monetdb supports basic table partitioning using MAPI? I need to split one large table into smaller physical pieces and improve query performance. I didn't find help in documentation. Thanks for help -- View this message in context: http://old.nabble.com/Partitioning-on-Monetdb-tp32493404p32493404.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] EPS / SVG of new MonetDB logo
by Stefan de Konink 20 Sep '11

20 Sep '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, The new logo is online on the new website for a while now, is there any EPS/SVG or other vector format available? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk547rYACgkQYH1+F2Rqwn14VwCgioRrllvOlEUwHJke6eLxdXca rm0An2CwkBr1KoUWmgYewB64U5B49GtS =ef+A -----END PGP SIGNATURE-----

2 3

[MonetDB-users] GUID
by Michal Dvorak 20 Sep '11

20 Sep '11

Hi MonetDB-Team, are you going to implement 128b GUID? If not, any idea how to bypass it? 2x Int64 or BLOB(128)? Thank you very much for your answer J Best regards, Michal

3 2

[MonetDB-users] Maximum Parallelism and Recommended Hardware for MonetDB
by brien colwell 20 Sep '11

20 Sep '11

I set up a MonetDB server for a data mining team on a single node. The node's specs are 32 cores (4x8 AMD 2.3GHz) 128GiB RAM, with the dbfarm on a RAID0 2xSSD (I think they are Intel SSDs). MonetDB on this hardware works well for aggregates and joins on integers, but chugs considerably on inline views with strings and joins with strings. The team usually has 2-3 concurrent jobs at any given time, some materializing new tables, with lots of bulk loading ("copy into ... from file ..."). A typical table size is 2GiB, and queries may join 5-10 tables. When I look at process activity on the node, I usually see each DB's mserver5 process pegged on CPU, using just a single core. Since I rarely see more than one core used per mserver5, I'm starting to think either I haven't configured MonetDB to favor parallel plans, or I chose the wrong hardware for MonetDB. So, as questions to everyone on the list, I'm curious 1) Are there knobs in MonetDB to favor parallel execution plans? 2) What's the ideal hardware for a single-node MonetDB server to support a workload of 2-5 concurrent queries? 3) What's the ideal disk configuration for the above? Brien

2 1

[MonetDB-users] How to compress a table
by Alexis Berlemont 20 Sep '11

20 Sep '11

Hi, I am currently trying to evaluate MonetDB as a solution to hold significant amounts of data. Because of limited storage capacities, I am trying to enable compression in MonetDB. I did not manage to find any tutorial on that subject, so I did not go very far. I cannot even tell that I wholly understand the problem. According to the page "Documentation->User Guide->compression", I understood that I had to inject a compression optimizer inside a pipeline. This "thing" is supposed to be possible via the setting of a sql environment variable named "optimizer". In the (generated) source file monetdb5/optimizer/opt_pipes.c, I found a default + compression pipeline. Then, in a mclient session, I typed: sql>set optimizer='inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector'; After that, I tried to use a sql function which was supposed to compress (thanks to the zlib) a table: sql.gzcompress sql>call gzcompress(myschema, mytable); Unfortunately, I am stuck here with the error message 'SELECT: identifier 'myschema' unknown'. I think I located the error message in "sql/server/rel_select.c" but I did not manage to deduce anything interesting. Do you have any idea ? Thanks in advance. Alexis.

2 1

[MonetDB-users] How to change in/out date format
by Eugene Prokopiev 20 Sep '11

20 Sep '11

Hi, is it possible to use "yy-MM-dd HH:mm:ss.SSS" date format via something like SET DATESTYLE in COPY INTO ... FROM ... with mclient to reuse existing files? -- Thanks, Eugene Prokopiev

2 1