- users-list - monetdb.org

[MonetDB-users] Database crash
by Stathis Gkotsis 16 Mar '12

16 Mar '12

Hi all, yesterday my server lost power and when I rebooted my monetdb database would not start. This is what I see in the logs: gdk_dbfarm=/var/monetdb5/dbfarm --dbname=mydatabase --set merovingian_uri=mapi:monetdb://mydatabase:50000/mydatabase --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/monetdb5/dbfarm/mydatabase/.mapi.sock --set monet_vault_key=/var/monetdb5/dbfarm/mydatabase/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2012-03-16 01:15:34 MSG merovingian[897]: database 'mydatabase' (1793) has exited with exit status 1 2012-03-16 01:15:34 ERR merovingian[897]: client error: database 'mydatabase' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2012-03-16 01:15:36 MSG merovingian[897]: database 'mydatabase' has crashed after start on 2012-03-13 17:57:44, attempting restart, up min/avg/max: 32s/32s/32s, crash average: 1.00 0.10 0.03 (2-1=1) 20 Any ideas on how to fix this? Thank you, Stathis

1 0

[MonetDB-users] Announcement: New Dec2011-SP2 Bugfix release of MonetDB suite
by Fabian Groffen 16 Mar '12

16 Mar '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2011-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2011-SP2 bugfix release Java Module * Implemented missing Number types support in PreparedStatement.setObject() Bug Fixes * 2915: Query (mclient or jdbc) with 2 outer joins appears to crash connectionor server * 2947: Queries where a "is null" clause is combined with two equals clauses fail to produce the expected result. * 2968: Memory Leaks * 2974: thetajoin produces invalid/inconsistent bat * 2988: Subquery in select part goes wrong * 2998: Equivalent multi-way join queries give different results * 3002: String plus interval result in program errors upon comparison * 3011: count(*) in subquery results in 1 for query without results * 3012: bitwise comparison fails after a cast * 3029: test sql/test/Dependencies/Tests/Dependencies.sql fails after recent changes * 3031: Virtually all SQL tests fail on Windows * 3036: optimizers are not getting set correctly through set optimizer * 3038: ~190 SQL tests fail on Darwin, FreeBSD & Solaris (only) since introduction of "(extended) relational algebra" * 3043: WITH clause is not allowed in derived table * 3045: algebra.project(0@0,bat) fails at propcheck * 3052: Wrong Error Code on Rollback Exception

1 0

[MonetDB-users] rename table?
by Christian Braun 16 Mar '12

16 Mar '12

Hello! Is it possible to rename a table? Regards, Christian.

1 0

[MonetDB-users] Query ignored
by Mints, Alexey 15 Mar '12

15 Mar '12

Hello! I have an issue with MonetDB query. This one select count(*) from catalogedsources c1 left outer join catalogedsources c2 on ( c2.cat_id = c1.cat_id + 2 AND ((c1.decl - c2.decl) * (c1.decl - c2.decl))< 1e-06) where c1.cat_id = 4 AND c1.decl BETWEEN 77.0 AND 83.0 AND c1.ra BETWEEN 42.0 AND 48.0 ; works fine. But if I replace "c2.cat_id = c1.cat_id + 2" with "c2.cat_id = 6" - i receive nothing. Not zero, not empty dataset, but just nothing at all. "EXPLAIN" does not work as well. Also, with more complex join conditions I have no result as well. I am using: Welcome to mclient, the MonetDB/SQL interactive terminal (Dec2011-SP1) Database: MonetDB v11.7.7 (Dec2011-SP1), 'mapi:monetdb://linux-6d2s.site:50000/*****' What can this be? Regards, Alexey

1 0

[MonetDB-users] MonetDB out of memory errors
by Stathis Gkotsis 14 Mar '12

14 Mar '12

Hi all, I am using MonetDB v11.7.7 (Dec2011-SP1) in an application which constantly feeds data in tables (daily tables). The communication with the server is done through the mapi API. This is accomplished through the "COPY INTO" command. From time to time (every 1-2 days) I see errors in the application logs such as: exception at 00000000004C0109: Connect error (Error code -4): monetdbd: an internal error has occurred, refer to the logs for details, please try again later . exception at 00000000004C0109: Connect error (Error code -4): monetdbd: there are no available connections for 'mydatabase' . exception at 00000000004C0109: Connect error (Error code -4): monetdbd: there are no available connections for 'mydatabase' Then looking at dmesg log: [964932.242080] 251025 pages non-shared [964932.242083] Out of memory: kill process 11585 (monetdbd) score 787911 or a child [964932.302785] Killed process 11781 (mserver5) Sometimes, monetdb can recover from this error and then the loading continues normally, some others I have to manually restart monetdb. Is there anything I can do to troubleshoot this issue? Is there a possibility for me to fall back to an earlier version of MonetDB with potentially less features but more stability? Thanks for all the help, Stathis

2 2

[MonetDB-users] rdf support
by Jürgen Jakobitsch 14 Mar '12

14 Mar '12

hi, first of all congrats to this great project! i came to monetDB via a search for "column stores", since i know that one of the big players in the field of linked data and rdf storage switched to usage of a column store in one of their latest releases. i just built from source without any hassle on openSuse 11.4 with support for rdf since this is my primary interest at the moment. i didn't find much of an info regarding monetDB and rdf besides an old nabble-mail and a small-meeting wiki. is there some more info available? any pointer really appreciated. wkr jürgen -- | Jürgen Jakobitsch, | Software Developer | Semantic Web Company GmbH | Mariahilfer Straße 70 / Neubaugasse 1, Top 8 | A - 1070 Wien, Austria | Mob +43 676 62 12 710 | Fax +43.1.402 12 35 - 22 COMPANY INFORMATION | http://www.semantic-web.at/ PERSONAL INFORMATION | web : http://www.turnguard.com | foaf : http://www.turnguard.com/turnguard | skype : jakobitsch-punkt | xmlns:tg = "http://www.turnguard.com/turnguard#"

1 0

[MonetDB-users] Replication/Redundancy
by Joseph Brower 13 Mar '12

13 Mar '12

We're using MonetDB quite heavily and are really enjoying it. We're now wanting to add replication/redundancy to ensure we have as little downtime as possible. We're also thinking of balancing reads if that is possible. I've heard this is possible, but I'm unsure on the best way to approach it. I've not been able to find many docs that talk about this either. Could someone point me in the right direction? Thanks, Joseph Brower

4 10

[MonetDB-users] monetdbd: binding to datagram socket port 50000 failed: no available address
by Charalampos Nikolaou 09 Mar '12

09 Mar '12

When I start monetdbd I get the following error "monetdbd: binding to datagram socket port 50000 failed: no available address" Prior to this error I had properly terminated the server, so there is no server running. How can I overcome that problem? Also, netstat gives the following: $ netstat -p udp | grep 50000 udp4 1562 0 *.50000 *.* Thanks, Babis P.S.: I am on a Mac if that plays any role.

2 2

[MonetDB-users] some questions regarding queries in MonetDB
by Freddy Priyatna 06 Mar '12

06 Mar '12

Dear all, I've been using MonetDB version 5 on Ubuntu machine for evaluating BSBM queries. The dataset and queries can be found here[1]. It seems that there are some queries that are not supported on MonetDB : 1) SELECT p.nr, p.label FROM product p, producttypeproduct ptp WHERE p.nr=ptp.product AND "productType"=35 AND "propertyNum1">500 AND "propertyNum3"<2000 AND 168 IN (SELECT "productFeature" FROM productfeatureproduct WHERE productfeatureproduct.product=p.nr) AND 147 NOT IN (SELECT "productFeature" FROM productfeatureproduct WHERE product=p.nr) >From the error, the subquery seems can't refer to the table alias "p". 2) SELECT distinct p.nr, p.label FROM product p, product po, (Select distinct pfp1.product FROM productfeatureproduct pfp1, (SELECT "productFeature" FROM productfeatureproduct WHERE product=2) pfp2 WHERE pfp2."productFeature"=pfp1."productFeature") pfp WHERE p.nr=pfp.product AND po.nr=2 AND p.nr <> po.nr AND p."propertyNum1" < (po."propertyNum1"+120) AND p."propertyNum1" > (po."propertyNum1"-120) AND p."propertyNum2" < (po."propertyNum2"+170) AND p."propertyNum2" > (po."propertyNum2"-170) This is strange, because if I just comment one of the last 2 lines, the query works. But the last two lines can't be used together. 3) SELECT nr, label FROM product WHERE label like "%countries%"; It seems that MonetDB doesn't support regex. 4) SELECT * FROM (select label from product where nr=2) p left join ((select o.nr as onr, o.price, v.nr as vnr, v.label from offer o, vendor v where 2=o.product AND o.vendor=v.nr AND v.country='GB' AND o."validTo">'2008-07-01') ov right join (select r.nr as rnr, r.title, pn.nr as pnnr, pn.name, r.rating1, r.rating2 from review r, person pn where r.product=2 AND r.person=pn.nr) rpn on (1=1)) on (1=1); It seems that MonetDB can't do join with true condition because when I tried to run a simple query : SELECT * FROM product p RIGHT JOIN offer o ON (1=1) Then MonetDB gives an error. Could anyone please help me how to run the queries above? Many thanks in advance. Freddy [1] http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/ExploreUse…

5 11

[MonetDB-users] duplicate tables in schema
by Bas Kaptijn 05 Mar '12

05 Mar '12

Hi, repeatedly we get in a situation where tables end up duplicate in the sys.tables table: SELECT count(*) FROM tables WHERE name='mytable'; returns: 2 We think this should never be the case. It seems this gives problems with dumping the database and in other situations. This is an older version of MonetDB (Apr2011 SP1). We want to know whether someone knows this was/is a symptom of any known issue Sincerely, Bas

3 2

[MonetDB-users] Slow range join performance / regression with Dec2011-SP1
by Viktor Rosenfeld 02 Mar '12

02 Mar '12

Hi, I have a few queries that join a table with itself using a ranged predicate on the primary key. The simplest query is: SELECT count(*) AS count FROM rank_copy rank1, rank_copy rank2 WHERE rank1.cat_s = true AND rank2.pos_vvfin = true AND rank1.pre < rank2.pre AND rank2.pre < rank1.post As you can see, the range lookup on rank2.pre is restricted on both sides by rank1.pre and rank1.post. The selectivity of the attributes and the complete query is: - row count : 2431307 - query : 95977 (0.039) - cat_s : 148799 (0.061) - pos_vvfin : 71965 (0.030) PostgreSQL is able to evaluate the query in 2 seconds using an index on pre. (This index is automatically generated because pre is the primary key.) MonetDB Dec2011 requires about 35 seconds. The MAL plan is posted below. I've also tried sorting the table on pre, and created an (advisory) index on pre but this makes no difference. +---------------------------------------------------------------------------+ | mal | +===========================================================================+ | function user.s1_1{autoCommit=true}():void; | | X_2 := sql.mvc(); | | X_9:bat[:oid,:bit] := sql.bind(X_2,"sys","rank_copy","pos_vvfin",0); | | X_10 := algebra.uselect(X_9,true:bit); | | X_11 := algebra.markT(X_10,0@0:oid); | | X_12 := bat.reverse(X_11); | | X_4:bat[:oid,:int] := sql.bind(X_2,"sys","rank_copy","pre",0); | | X_13 := algebra.leftjoin(X_12,X_4); | | X_17:bat[:oid,:bit] := sql.bind(X_2,"sys","rank_copy","cat_s",0); | | X_18 := algebra.uselect(X_17,true:bit); | | X_19 := algebra.markT(X_18,0@0:oid); | | X_20 := bat.reverse(X_19); | | X_23 := algebra.leftjoin(X_20,X_4); | | X_15:bat[:oid,:int] := sql.bind(X_2,"sys","rank_copy","post",0); | | X_21 := algebra.leftjoin(X_20,X_15); | | X_24 := algebra.join(X_13,X_23,X_21,false,false); | | X_26 := algebra.markT(X_24,0@0:oid); | | X_27 := bat.reverse(X_26); | | X_28 := algebra.leftjoin(X_27,X_13); | | X_29 := aggr.count(X_28); | | sql.exportValue(1,"sys.rank2","count":str,"wrd",64,0,6,X_29,""); | | end s1_1; | +---------------------------------------------------------------------------+ 22 tuples If I use a slightly more complicated query that has the boolean attributes pos_vvfin and cat_s on another table that is joined by rank, the situation is even worse. SELECT count(*) AS count FROM node_copy node1 JOIN rank_copy rank1 ON (node1.id = rank1.node_ref), node_copy node2 JOIN rank_copy rank2 ON (node2.id = rank2.node_ref) WHERE node1.cat_s = true AND node2.pos_vvfin = true AND rank1.pre < rank2.pre AND rank2.pre < rank1.post PostgreSQL evaluates the query in 9 seconds, again using the primary key indexes on node.id and rank.pre. MonetDB Dec2011 requires 14 minutes. The MAL plan follows: +-----------------------------------------------------------------------------------------------+ | mal | +===============================================================================================+ | function user.s1_1{autoCommit=true}():void; | | X_2 := sql.mvc(); | | X_19:bat[:oid,:oid] := sql.bind_idxbat(X_2,"sys","rank_copy","FK_rank_copy_node_ref",0); | | X_13:bat[:oid,:bit] := sql.bind(X_2,"sys","node_copy","pos_vvfin",0); | | X_14 := algebra.uselect(X_13,true:bit); | | X_15 := algebra.markT(X_14,0@0:oid); | | X_16 := bat.reverse(X_15); | | X_9:bat[:oid,:int] := sql.bind(X_2,"sys","node_copy","id",0); | | X_11 := bat.mirror(X_9); | | X_17 := algebra.leftjoin(X_16,X_11); | | X_18 := bat.reverse(X_17); | | X_37:bat[:oid,:bit] := sql.bind(X_2,"sys","node_copy","cat_s",0); | | X_39 := algebra.uselect(X_37,true:bit); | | X_40 := algebra.markT(X_39,0@0:oid); | | X_41 := bat.reverse(X_40); | | X_42 := algebra.leftjoin(X_41,X_11); | | X_43 := bat.reverse(X_42); | | X_24 := algebra.join(X_19,X_18); | | X_25 := algebra.markT(X_24,0@0:oid); | | X_26 := bat.reverse(X_25); | | X_4:bat[:oid,:int] := sql.bind(X_2,"sys","rank_copy","pre",0); | | X_27 := algebra.leftjoin(X_26,X_4); | | X_29:bat[:oid,:int] := sql.bind(X_2,"sys","rank_copy","post",0); | | X_30 := algebra.join(X_27,X_4,X_29,false,false); | | X_32 := algebra.markT(X_30,0@0:oid); | | X_33 := bat.reverse(X_32); | | X_44 := bat.reverse(X_30); | | X_45 := algebra.markT(X_44,0@0:oid); | | X_46 := bat.reverse(X_45); | | X_47 := algebra.leftjoin(X_46,X_19); | | X_48 := algebra.join(X_47,X_43); | | X_49 := algebra.markT(X_48,0@0:oid); | | X_50 := bat.reverse(X_49); | | X_51:bat[:oid,:int] := algebra.leftjoinPath(X_50,X_33,X_27); | | X_52 := aggr.count(X_51); | | sql.exportValue(1,"sys.rank2","count","wrd",64,0,6,X_52,""); | | end s1_1; | +-----------------------------------------------------------------------------------------------+ 37 tuples The Dec2011-SP1 release takes even longer, about 18 minutes. The MAL plan is unchanged. The culprit appears to be the ranged lookup because if I replace it with an equi-join on another attribute MonetDB evaluates it in about 100 ms. SELECT count(*) AS count FROM node_copy node1 JOIN rank_copy rank1 ON (node1.id = rank1.node_ref), node_copy node2 JOIN rank_copy rank2 ON (node2.id = rank2.node_ref) WHERE node1.cat_s = true AND node2.pos_vvfin = true AND rank1.pre = rank2.parent I've tried rewriting the query using WITH and had some success. The first query again takes 35 seconds: WITH rank1 AS (SELECT * FROM rank_copy WHERE cat_s = true), rank2 AS (SELECT * FROM rank_copy WHERE pos_vvfin = true) SELECT count(*) FROM rank1, rank2 WHERE rank1.pre < rank2.pre AND rank2.pre < rank1.post The second query also takes 35 seconds when rewritten: WITH rank1 AS (SELECT * FROM rank_copy JOIN node_copy ON (rank_copy.node_ref = node_copy.id) WHERE node_copy.cat_s = true), rank2 AS (SELECT * FROM rank_copy JOIN node_copy ON (rank_copy.node_ref = node_copy.id) WHERE node_copy.pos_vvfin = true) SELECT count(*) FROM rank1, rank2 WHERE rank1.pre < rank2.pre AND rank2.pre < rank1.post MAL plan: +-----------------------------------------------------------------------------------------------+ | mal | +===============================================================================================+ | function user.s1_2{autoCommit=true}():void; | | X_2 := sql.mvc(); | | X_20:bat[:oid,:oid] := sql.bind_idxbat(X_2,"sys","rank_copy","FK_rank_copy_node_ref",0); | | X_13:bat[:oid,:bit] := sql.bind(X_2,"sys","node_copy","pos_vvfin",0); | | X_14 := algebra.uselect(X_13,true:bit); | | X_15 := algebra.markT(X_14,0@0:oid); | | X_16 := bat.reverse(X_15); | | X_9:bat[:oid,:int] := sql.bind(X_2,"sys","node_copy","id",0); | | X_11 := bat.mirror(X_9); | | X_17 := algebra.leftjoin(X_16,X_11); | | X_18 := bat.reverse(X_17); | | X_32:bat[:oid,:bit] := sql.bind(X_2,"sys","node_copy","cat_s",0); | | X_33 := algebra.uselect(X_32,true:bit); | | X_34 := algebra.markT(X_33,0@0:oid); | | X_35 := bat.reverse(X_34); | | X_36 := algebra.leftjoin(X_35,X_11); | | X_37 := bat.reverse(X_36); | | X_22 := algebra.join(X_20,X_18); | | X_23 := algebra.markT(X_22,0@0:oid); | | X_24 := bat.reverse(X_23); | | X_5:bat[:oid,:int] := sql.bind(X_2,"sys","rank_copy","pre",0); | | X_25 := algebra.leftjoin(X_24,X_5); | | X_38 := algebra.join(X_20,X_37); | | X_39 := algebra.markT(X_38,0@0:oid); | | X_40 := bat.reverse(X_39); | | X_43 := algebra.leftjoin(X_40,X_5); | | X_27:bat[:oid,:int] := sql.bind(X_2,"sys","rank_copy","post",0); | | X_41 := algebra.leftjoin(X_40,X_27); | | X_44 := algebra.join(X_25,X_43,X_41,false,false); | | X_46 := algebra.markT(X_44,0@0:oid); | | X_47 := bat.reverse(X_46); | | X_48 := algebra.leftjoin(X_47,X_25); | | X_49 := aggr.count(X_48); | | sql.exportValue(1,"sys.rank2","L1","wrd",64,0,6,X_49,""); | | end s1_2; | +-----------------------------------------------------------------------------------------------+ 35 tuples Is there anything I can do to further speed up the execution of these queries? Thanks, Viktor

1 0

[MonetDB-users] "ERROR = !Connection terminated" for some SQL queries
by Viktor Rosenfeld 01 Mar '12

01 Mar '12

Hi, the following query results in a "Connection terminated error": SELECT count(*) FROM ( SELECT DISTINCT node1.id AS id1, node2.id AS id2, node1.toplevel_corpus FROM node AS node1, rank AS rank1, component AS component1, node AS node2, rank AS rank2, component AS component2 WHERE component1.id = component2.id AND component1.name IS NULL AND component1.type = 'd' AND component2.name IS NULL AND component2.type = 'd' AND rank1.component_ref = component1.id AND rank1.node_ref = node1.id AND rank1.pre = rank2.parent AND rank2.component_ref = component2.id AND rank2.node_ref = node2.id ) AS solutions; The full error message is: MAPI = annis@moos.local:50000 ACTION= read_line QUERY = SELECT count(*) FROM ( SELECT DISTINCT node1.id AS id1, node2.id AS id2, node1.toplevel_corpus FROM node AS node1, rank AS rank1, component AS component1, node AS node2, rank AS rank2, component AS component2 WHERE component1.id = component2.id AND component1.name IS NULL AND component1.type = 'd' AND component2.name IS NULL AND component2.type = 'd' AND rank1.component_ref = component1.id AND rank1.node_ref = node1.id AND rank1.pre = rank2.parent AND rank2.component_ref = component2.id AND rank2.node_ref = node2.id ) AS solutions; ERROR = !Connection terminated I can get it to run if I remove the following line: component1.id = component2.id AND or if I remove the following 4 lines: component1.name IS NULL AND component1.type = 'd' AND component2.name IS NULL AND component2.type = 'd' AND However, if I include all constraints on component, the error appears. Let me know, if you need more information. Cheers, Viktor

2 3

[MonetDB-users] Database crash
by Stathis Gkotsis 29 Feb '12

29 Feb '12

Hi all, I am using MonetDB Dec 2011 version. Seeing that I could not connect to 'mydatabase' from mclient, I tailed the log: 2012-02-29 11:39:32 MSG mydatabase[7045]: # Serving database 'mydatabase', using 2 threads 2012-02-29 11:39:32 MSG mydatabase[7045]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2012-02-29 11:39:32 MSG mydatabase[7045]: # Found 999.238 MiB available main-memory. 2012-02-29 11:39:32 MSG mydatabase[7045]: # Copyright (c) 1993-July 2008 CWI. 2012-02-29 11:39:32 MSG mydatabase[7045]: # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved 2012-02-29 11:39:32 MSG mydatabase[7045]: # Visit http://www.monetdb.org/ for further information 2012-02-29 11:39:32 MSG mydatabase[7045]: # Listening for UNIX domain connection requests on mapi:monetdb:///var/monetdb5/dbfarm/mydatabase/.mapi.sock 2012-02-29 11:39:32 MSG mydatabase[7045]: # MonetDB/SQL module loaded 2012-02-29 11:39:32 MSG merovingian[6978]: database 'mydatabase' (7045) was killed by signal SIGSEGV 2012-02-29 11:39:32 ERR merovingian[6978]: client error: database 'mydatabase' has crashed after starting, manual intervention needed, check monetdbd's logfile for details Any ideas? Thank you.

1 0

[MonetDB-users] Very Slow Selects
by Joseph Brower 29 Feb '12

29 Feb '12

We're using MonetDB and hitting it fairly heavily with read queries. We only use importing of text files to insert information, and that happens every 30 seconds or so. When we're only inserting information, we consume less than 1% of the cpu and things run just fine. We are also able to occasionally query the DB for various peices of information (counts, averages, etc) and it returns quite quickly. The problem is that after we've been running for 15-20 minutes of importing and querying heavily, performance drops to where queries take multiple minutes to finish. The queries generally finish in less than half a second. Does anyone have any ideas on how we can best track down this issue? Is there any more information that I need to provide? Thanks, Joseph Brower

6 7

[MonetDB-users] Announcement: New Dec2011-SP1 Bugfix release of MonetDB suite
by Fabian Groffen 29 Feb '12

29 Feb '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2011-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2011-SP1 bugfix release Build Environment * Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case. * On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997. SQL * Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient. * Stripped off implementation-specific parts from error messages before they get presented to the user. Java Module * Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument. * Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages MonetDB5 Server * Fixed potential crash by dealing better with non-standard types. Merovingian * Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries. * Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982. * Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983. * Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984. * Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984. Bug Fixes * 2862: foreign key self references cause violation upon alter table add constraint... * 2933: "TypeException: algebra.join undefined" when evaluating EXISTS (SELECT ...) predicate * 2946: Invalid BAT if left join condition has only columns from right table * 2964: LEFT OUTER JOIN with constants returns errors * 2969: Result precision issue with Decimal data type * 2973: Date coercion functionality tests * 2979: misc merovingian control problems * 2982: monetdbd status and mserver properties have random values * 2983: monetdbd attempts to stop non-existing database 'control' * 2984: monetdb create/release/start/stop returns error "OK" for remote connections * 2995: Test geom/sql/Tests/example.sql crashes since changeset 6399077ff8a4 * 2997: monetdb-python packages installed in wrong dir on Ubuntu 10.04 LTS (lucid) * 3004: Incorrect evaluation of conditions in WHERE part of SQL statement * 3009: Segmentation fault on simple query * 3013: calc.and is not yet defined for dbl * 3014: dayofweek comment is wrong in mtime.mx:700 * 3017: rel_bin.c:611: exp_bin: Assertion `0' failed. with between on multicolumns * 3020: Assertion error in debugging mode * 3034: INNER JOIN with Subquery produces mal errors * 3041: WHERE condition ignored after 2 LEFT JOINs with subqueries

1 0

[MonetDB-users] max concurrent limit?
by Allen Zhang 28 Feb '12

28 Feb '12

Hi, When connecting to monetdb Aug 2011 SP2 version, we encountered error states : Maximum concurrent client connection (64) reached, any solutions? Best Regards Allen

4 4

[MonetDB-users] MonetDB JDBC 1.9 question
by Charalampos Nikolaou 23 Feb '12

23 Feb '12

Hi, I am getting some strange exceptions using MonetDB JDBC 1.8/1.9, and I decided to look into the code. While inspecting the code I came across these strange lines in MonetConnection.executeQuery(String[] templ, String query) method (http://dev.monetdb.org/hg/MonetDB/file/718da8ca0a1a/java/src/nl/cwi/monetdb…) In my case, these lines are executed for sure when doing a batch insert. So, suppose that the batch contains the following commands (as constructed by the MonetStatement.executeBatch() method): exec 1(1, 957339737330229055); exec 1(2, 278262503670654331); exec 1(805306369, 3763943296910752235) The lines in question take the above commands as a string, prepend a 's' character and append a ';' at the end. The resulting commands, which are written in the server's socket, are the following: sexec 1(1, 957339737330229055); exec 1(2, 278262503670654331); exec 1(805306369, 3763943296910752235); First of all, I am not familiar with the internals of JDBC drivers. Taking this into account, is this what it should be? From a symmetric point of view, I would assume that the correct would be the following: sexec 1(1, 957339737330229055); sexec 1(2, 278262503670654331); sexec 1(805306369, 3763943296910752235); That is, it should prepend a 's' character before an exec command. Last, are these exec commands (with or without a prepending 's') specific to MonetDB? In either case, is there any documentation to get familiar with their meaning? Thanks a lot, Babis

6 20

[MonetDB-users] Prepared statement not found (error: 07003)
by George Garbis 23 Feb '12

23 Feb '12

Hello all, I have a java project that creates a database using more than one connections and I get the following error: > 22000!07003:EXEC: no prepared statement with id: 1 > After digging in log files I created a simplified version of the problematic commands: Class.forName("nl.cwi.monetdb.jdbc.MonetDriver"); > Connection con1 = > DriverManager.getConnection("jdbc:monetdb://localhost/db1", "monetdb", > "monetdb"); > con1.setAutoCommit(true); > Connection con2 = > DriverManager.getConnection("jdbc:monetdb://localhost/db1", "monetdb", > "monetdb"); > con2.setAutoCommit(true); > > PreparedStatement ps2 = con2.prepareStatement("INSERT INTO table1 (a) > VALUES (CAST(? AS INTEGER))"); > ps2.setInt(1,1); ps2.addBatch(); > > Statement s1 = con1.createStatement(); > s1.execute("CREATE TABLE table2 ( a INTEGER )"); > s1.close(); > > ps2.executeBatch(); > During execution of ps2.executeBatch() the aforementioned error is occurred. The problem occurs because between creating the first prepared statement and executing it another statement is created and executed (this odd sequence of commands happens because the project is multi-threaded). The two queries change different tables so i wonder if there is something wrong in the commands above. It seems that MonetDB handles in a special way prepared statements and I have to execute each one before I define a new one. If yes can you give me some hints why this is happening or how i can work around this problem? Thank you in advance, George Garbis

3 3

[MonetDB-users] SQL:2003 support
by Gatis Ozoliņš 22 Feb '12

22 Feb '12

Are there any plans for SQL:2003 support? I am particularly interested in window functions (LAG, LEAD, DENSE_RANK ...) and MERGE statement.

1 0

[MonetDB-users] Transaction management in stored procedures
by Vassilis Papakonstantinou 20 Feb '12

20 Feb '12

Hi all, Does anyone knows how can I handle transactions in a stored procedure? In procedure definition on SQL Reference says that a procedure_statement may consists of transaction_statement etc. How transaction_statement implemented in a stored procedure? I tried (inside the BEGIN ... END block of stored procedure) the: START TRANSACTION; sql statements COMMIT; and: sql statements COMMIT; but both they do not work. Thanks, Vassilis

2 4

[MonetDB-users] Subquery with INNER JOIN
by George Garbis 20 Feb '12

20 Feb '12

Hello to all, I have some problems trying to use subqueries with INNER JOIN. Assume the following schema: CREATE TABLE table1 (a INTEGER); > CREATE TABLE table2 (a INTEGER, b INTEGER); > > INSERT INTO table1 VALUES(1); > INSERT INTO table1 VALUES(2); > INSERT INTO table2 VALUES(2, 2); > I pose this query > SELECT * FROM (SELECT a, 2 AS b FROM table1) t1 INNER JOIN table2 t2 ON > (t1.b = t2.b); > and I get the following mal errors: > TypeException:user.s5_1[44]:'bat.mirror' undefined in: _62:any := > bat.mirror(_61:int) > TypeException:user.s5_1[46]:'algebra.join' undefined in: _64:any := > algebra.join(_62:any, _63:bat[:oid,:oid]) > TypeException:user.s5_1[47]:'algebra.markH' undefined in: _65:any := > algebra.markH(_64:any, _66:oid) > TypeException:user.s5_1[48]:'algebra.markT' undefined in: _67:any := > algebra.markT(_64:any, _66:oid) > TypeException:user.s5_1[49]:'bat.reverse' undefined in: _68:any := > bat.reverse(_67:any) > TypeException:user.s5_1[50]:'algebra.join' undefined in: _69:any := > algebra.join(_68:any, _61:int) > TypeException:user.s5_1[51]:'algebra.join' undefined in: _70:any := > algebra.join(_65:any, _59:bat[:oid,:int]) > TypeException:user.s5_1[55]:'algebra.join' undefined in: _77:any := > algebra.join(_76:bat[:void,:oid], _65:any) > TypeException:user.s5_1[56]:'bat.reverse' undefined in: _78:any := > bat.reverse(_77:any) > TypeException:user.s5_1[57]:'algebra.markT' undefined in: _79:any := > algebra.markT(_78:any, _43:oid) > TypeException:user.s5_1[58]:'bat.reverse' undefined in: _80:any := > bat.reverse(_79:any) > TypeException:user.s5_1[59]:'algebra.leftjoin' undefined in: _81:any := > algebra.leftjoin(_80:any, _42:bat[:oid,:int]) > TypeException:user.s5_1[64]:'algebra.project' undefined in: _86:any := > algebra.project(_81:any, _83:int) > TypeException:user.s5_1[65]:'algebra.join' undefined in: _87:any := > algebra.join(_86:any, _85:bat[:int,:oid]) > TypeException:user.s5_1[66]:'bat.reverse' undefined in: _88:any := > bat.reverse(_87:any) > TypeException:user.s5_1[67]:'algebra.markT' undefined in: _89:any := > algebra.markT(_88:any, _21:oid) > TypeException:user.s5_1[68]:'bat.reverse' undefined in: _90:any := > bat.reverse(_89:any) > TypeException:user.s5_1[69]:'algebra.leftjoin' undefined in: _91:any := > algebra.leftjoin(_90:any, _20:bat[:oid,:int]) > TypeException:user.s5_1[76]:'algebra.markT' undefined in: _103:any := > algebra.markT(_87:any, _101:oid) > TypeException:user.s5_1[77]:'bat.reverse' undefined in: _104:any := > bat.reverse(_103:any) > TypeException:user.s5_1[78]:'algebra.leftjoin' undefined in: _105:any := > algebra.leftjoin(_104:any, _81:any) > TypeException:user.s5_1[79]:'algebra.project' undefined in: _106:any := > algebra.project(_105:any, _96:bte) > TypeException:user.s5_1[86]:'algebra.leftjoin' undefined in: _113:any := > algebra.leftjoin(_80:any, _59:bat[:oid,:int]) > TypeException:user.s5_1[87]:'algebra.leftjoin' undefined in: _114:any := > algebra.leftjoin(_104:any, _113:any) > 39000!program contains errors > If i don't use a subquery and i first create the table i need, the query executes flawlessly and it returns the expected results. CREATE TABLE table3 (a INTEGER, b INTEGER); > INSERT INTO table3(a,b) (SELECT a, 2 AS b FROM table1); > SELECT * FROM (SELECT a, b FROM table3) t1 INNER JOIN table2 t2 ON (t1.b = > t2.b); > Results: > +------+------+------+----+ > | a | b | a | b | > +==+===+===+===+ > | 2 | 2 | 2 | 2 | > | 1 | 2 | 2 | 2 | > +------+------+------+----+ > Can anyone help? Is there something wrong in my first query or it is a bug? Thank you in advance, George Garbis

2 1

[MonetDB-users] mapi get field types
by Stathis Gkotsis 18 Feb '12

18 Feb '12

Hi, I am issuing a query using the mapi using mapi_query. Is there a way to get the returned field types through the api? Thank you, Stathis

2 1

[MonetDB-users] Ending up with a sql/log.XX of 4GB
by Stefan de Konink 17 Feb '12

17 Feb '12

Over the night my database was running. Doing both insert (constant) and select jobs (incidental). It seems that the combination is fatal. Doing each thing on independant databases I don't have problems at all, but doing them within the same database all kind of badness arrises. The next badness is the internal sql log of 4GB, MonetDB reports disk full. Which is actually the case. The actual database contents is only 49MB upon start (after inserting static data). I expect 200.000 extra rows of dynamic data, with a cyclic pattern, to be added over a week time, overwritting eachother. I wonder if it could be related to 'no-autocommit' behavior of select queries outside of transactions. I would assume it does not make sense at all, but that seems the most 'logical' reason... I could recover from disk full to free up space, wait a long time with a high load. SQL log was empty. Please advise, Stefan

1 0

[MonetDB-users] Table column order and type
by Pete Black 16 Feb '12

16 Feb '12

Hi, From what I understand indexes are indicative[1], sortedness is determined at runtime[2] and the storage model should not effect the general schema design[3]. Has anyone had any experience with large time series where the column order and types have effected performance? For example having data with less variance and "uniqueness" earlier in a table and splitting up timestamps and storing their date and daytime components separately. Thanks, Pete [1]http://www.monetdb.org/Documentation/Manuals/SQLreference/Indices [2]http://old.nabble.com/column-sortedness-to33053179.html#a33053179 [3]http://old.nabble.com/General-Questions-on-Columnar-DB-Design-and-Queryin…

2 1

[MonetDB-users] Microsecond timestamp precision?
by Pete Black 16 Feb '12

16 Feb '12

Hi, The documentation doesn't mention the resolution of the timestamp[1] directly, but given the examples for the daytime one can assume that the it holds millisecond resolution. Why then are the trailing zeros below printed out? And is there a way to get microsecond resolution for example using mal instead? sql>DECLARE t timestamp(9); sql>SET t='2012-02-16 01:15:25.123456'; sql>SELECT t; +----------------------------+ | single_value | +============================+ | 2012-02-16 01:15:25.123000 | +----------------------------+ 1 tuple (1.293ms) Thanks, Pete [1]http://www.monetdb.org/Documentation/SQLreference/Temporal

1 0

[MonetDB-users] Does Monetdb supports Date functions?
by Dongli 16 Feb '12

16 Feb '12

Hi guys, Does Monetdb supports "datediff" function? I didn't found this function while I run "select name from functions". Besides, Does Monetdb supports other Date relevant functions? Any advices will be highly appreciated. Thanks in advance, Tony

4 3

[MonetDB-users] Does Monetdb supports UTF-8 characters when I use 'as' clause?
by Dongli 15 Feb '12

15 Feb '12

Hi folks, Please take a look at following issue that I got today: The following queries I executed aim to test if Monetdb supports GBK characters(Chinese), since I have to use Chinese characters in my system to display the reports. I’m not sure if I missed some setting such as ‘set characters=’utf-8’? The following alias after the ‘as’ is a Chinese character ‘I’. sql>select source as '我' from session; 42000!syntax error, unexpected STRING in: "select source as '我'" sql>select source as 我 from session; unexpected character (U+6211) sql>select source as "我" from session; Invalid identifier '我' Thanks in advance, Tony

3 3

[MonetDB-users] database 'dw' has crashed after starting, manual intervention needed, check monetdbd's logfile for details
by Dongli 15 Feb '12

15 Feb '12

Hi folks, How can I recover from a crashed database? I got this error when I restart a running database 'dw', Do you have any ideas on this? 2012-02-15 13:59:28 MSG control[27007]: (local): served status list 2012-02-15 13:59:28 MSG merovingian[27007]: startup of database under maintenance 'dw' forced 2012-02-15 13:59:28 MSG merovingian[27007]: database 'dw' has crashed after start on 2012-02-15 13:58:47, attempting restart, up min/avg/max: 1d/1d/1d, crash average: 1.00 1.00 0.97 (45-1=44) 2012-02-15 13:59:28 MSG dw[27831]: arguments: /usr/lib/monetdb/bin/mserver5 --set gdk_dbfarm=/data/warehouse --dbname=dw --set merovingian_uri=mapi:monetdb://monet00:50000/dw --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data/warehouse/dw/.mapi.sock --set monet_vault_key=/data/warehouse/dw/.vaultkey --set gdk_nr_threads=24 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2012-02-15 13:59:28 MSG dw[27831]: # MonetDB 5 server v11.7.5 "Dec2011" 2012-02-15 13:59:28 MSG dw[27831]: # Serving database 'dw', using 24 threads 2012-02-15 13:59:28 MSG dw[27831]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked 2012-02-15 13:59:28 MSG dw[27831]: # Found 23.533 GiB available main-memory. 2012-02-15 13:59:28 MSG dw[27831]: # Copyright (c) 1993-July 2008 CWI. 2012-02-15 13:59:28 MSG dw[27831]: # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved 2012-02-15 13:59:28 MSG dw[27831]: # Visit http://www.monetdb.org/ for further information 2012-02-15 13:59:28 MSG dw[27831]: # Listening for UNIX domain connection requests on mapi:monetdb:///data/warehouse/dw/.mapi.sock 2012-02-15 13:59:28 MSG dw[27831]: # MonetDB/SQL module loaded 2012-02-15 13:59:28 MSG merovingian[27007]: database 'dw' (27831) was killed by signal SIGSEGV 2012-02-15 13:59:28 ERR control[27007]: (local): failed to fork mserver: database 'dw' has crashed after starting, manual intervention needed, check monetdbd's logfile for details >>The following is my platform information: MonetDB 5 server v11.7.5 "Dec2011" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 23.5GiB available memory, 24 available cpu cores Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8e-rhel5 01 Jul 2008 (compiled with OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008) libxml2: 2.6.26 (compiled with 2.6.26) Compiled by: root@monet00 (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 Thanks in advance, Tony

1 0

[MonetDB-users] failed to fork mserver: database 'dw' has crashed after starting, manual intervention needed, check monetdbd's logfile for details
by Tony 15 Feb '12

15 Feb '12

Hi folks, How can I recover from a crashed database? I got this error when I restart a running database 'dw', Do you have any ideas on this? 2012-02-15 13:59:28 MSG control[27007]: (local): served status list 2012-02-15 13:59:28 MSG merovingian[27007]: startup of database under maintenance 'dw' forced 2012-02-15 13:59:28 MSG merovingian[27007]: database 'dw' has crashed after start on 2012-02-15 13:58:47, attempting restart, up min/avg/max: 1d/1d/1d, crash average: 1.00 1.00 0.97 (45-1=44) 2012-02-15 13:59:28 MSG dw[27831]: arguments: /usr/lib/monetdb/bin/mserver5 --set gdk_dbfarm=/data/warehouse --dbname=dw --set merovingian_uri=mapi:monetdb://monet00:50000/dw --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data/warehouse/dw/.mapi.sock --set monet_vault_key=/data/warehouse/dw/.vaultkey --set gdk_nr_threads=24 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2012-02-15 13:59:28 MSG dw[27831]: # MonetDB 5 server v11.7.5 "Dec2011" 2012-02-15 13:59:28 MSG dw[27831]: # Serving database 'dw', using 24 threads 2012-02-15 13:59:28 MSG dw[27831]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked 2012-02-15 13:59:28 MSG dw[27831]: # Found 23.533 GiB available main-memory. 2012-02-15 13:59:28 MSG dw[27831]: # Copyright (c) 1993-July 2008 CWI. 2012-02-15 13:59:28 MSG dw[27831]: # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved 2012-02-15 13:59:28 MSG dw[27831]: # Visit http://www.monetdb.org/ for further information 2012-02-15 13:59:28 MSG dw[27831]: # Listening for UNIX domain connection requests on mapi:monetdb:///data/warehouse/dw/.mapi.sock 2012-02-15 13:59:28 MSG dw[27831]: # MonetDB/SQL module loaded 2012-02-15 13:59:28 MSG merovingian[27007]: database 'dw' (27831) was killed by signal SIGSEGV 2012-02-15 13:59:28 ERR control[27007]: (local): failed to fork mserver: database 'dw' has crashed after starting, manual intervention needed, check monetdbd's logfile for details >>The following is my platform information: MonetDB 5 server v11.7.5 "Dec2011" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 23.5GiB available memory, 24 available cpu cores Libraries: libpcre: 6.6 06-Feb-2006 (compiled with 6.6) openssl: OpenSSL 0.9.8e-rhel5 01 Jul 2008 (compiled with OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008) libxml2: 2.6.26 (compiled with 2.6.26) Compiled by: root@monet00 (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 Thanks in advance, Tony

2 1

[MonetDB-users] Odd Single Thread Only Processing in Dec 2011 Release
by Henry Addington 15 Feb '12

15 Feb '12

Hi: We are testing Dec 2011 Release to upgrade from Oct 2010 SP1 release on CentOS 5.2 64-bit with 100+ GB memory. We have a BI tool that users use to generate various reports. We have access to the SQL statements that get generated by the BI tool and sent to the MonetDB. So, we know exactly the SQL statements that are being sent to MonetDB. One very odd thing that we are seeing the December release is that the reports from the BI tool are very, very slow (x10 - x15 from Oct releasee). It is obvious why looking at the server resource use. It looks like only 1 cpu is being used for all the queries! However, if I take the SQL statements generated by the BI tool and run them in DBVisualizer, all 20 cpu cores are used and the results return quickly. I am at a loss as to why we are seeing such different performance from identical SQL statements... Is there a configuration setting or something in the December release that we overlooked? Can you please help us troubleshoot? Thank you. HS

2 3

[MonetDB-users] select distinct ... where algorithm
by zack_pcd 13 Feb '12

13 Feb '12

I am seeing the following: select count(*) from f_sorted; +-----------+ | L7 | +===========+ | 102612097 | +-----------+ 1 tuple (0.988ms) select count(*) from f_sorted where yyyymm='201110'; +----------+ | L10 | +==========+ | 23505343 | +----------+ 1 tuple (1.0s) select count(distinct uuid) from f_sorted where yyyymm='201110'; +---------+ | L6 | +=========+ | 5941581 | +---------+ 1 tuple (23.8s) Very fast thus far! However: copy (select distinct(uuid) from f_sorted where yyyymm='201110') into '/home/user/filename' USING DELIMITERS ',','\n','"' NULL as 'null'; This has been running for hours and has yet to start writing to the file. I get this when I explain the query: +----------------------------------------------------------------------------------------+ | mal | +========================================================================================+ | function user.s5_3{autoCommit=true}():void; | | X_2 := sql.mvc(); | | X_22:bat[:oid,:str] := sql.bind(X_2,"sys","f_sorted","yyyymm",0); | | X_23 := algebra.uselect(X_22,"201110":str); | | X_20:bat[:oid,:str] := sql.bind(X_2,"sys","f_sorted","yyyymm",2); | | X_24 := algebra.kdifference(X_23,X_20); | | X_21 := algebra.uselect(X_20,"201110":str); | | X_25 := algebra.kunion(X_24,X_21); | | X_18:bat[:oid,:str] := sql.bind(X_2,"sys","f_sorted","yyyymm",1); | | X_19 := algebra.uselect(X_18,"201110":str); | | X_26 := algebra.kunion(X_25,X_19); | | X_16:bat[:oid,:oid] := sql.bind_dbat(X_2,"sys","f_sorted",1); | | X_17 := bat.reverse(X_16); | | X_27 := algebra.kdifference(X_26,X_17); | | X_28 := algebra.markT(X_27,0@0:oid); | | X_29 := bat.reverse(X_28); | | X_10:bat[:oid,:str] := sql.bind(X_2,"sys","f_sorted","uuid",0); | | X_8:bat[:oid,:str] := sql.bind(X_2,"sys","f_sorted","uuid",2); | | X_12 := algebra.kdifference(X_10,X_8); | | X_13 := algebra.kunion(X_12,X_8); | | X_5:bat[:oid,:str] := sql.bind(X_2,"sys","f_sorted","uuid",1); | | X_14 := algebra.kunion(X_13,X_5); | | X_30 := algebra.leftjoin(X_29,X_14); | | (ext44,grp42) := group.done(X_30); | | X_33 := bat.mirror(ext44); | | X_34 := algebra.leftjoin(X_33,X_30); | | X_35 := sql.resultSet(1,1,X_34); | | sql.rsColumn(X_35,"sys.f_sorted","uuid":str,"varchar",2048,0,X_34); | | X_43 := io.stdout(); | | sql.exportResult(X_43,X_35); | | end s5_3; | +----------------------------------------------------------------------------------------+ 31 tuples (1.350ms) Is it doing something wrong? I don't expect the query to take so long (hours) to produce the output, especially since the distinct count is also very fast. The copy-into *seems* to be irrelevant to the problem, but I decided to include it just in case, as that's how I am using the query. -- View this message in context: http://old.nabble.com/select-distinct-...-where-algorithm-tp33297578p332975… Sent from the monetdb-users mailing list archive at Nabble.com.

4 6

[MonetDB-users] False dependency
by Stathis Gkotsis 10 Feb '12

10 Feb '12

Hi all, trying to install: libmonetdb-client-dev_11.7.5-20120120_amd64.deb on Ubuntu 10.04, I see that there is a wrong dependency on libmonetdb-client1. Is there a way around this? Thank you.

1 0

[MonetDB-users] re cycler_pipe and cracker_pipe
by zack_pcd 08 Feb '12

08 Feb '12

Is it possible to have a mix of these pipe's optimizers so you get the benefit of both? Or are these two pipes incompatible? (recycler_pipe and cracker_pipe) -- View this message in context: http://old.nabble.com/recycler_pipe-and-cracker_pipe-tp33282420p33282420.ht… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] LIMIT clause not recognized in nested query in FROM clause
by Viktor Rosenfeld 07 Feb '12

07 Feb '12

Hi, it appears that clauses like LIMIT or OFFSET are not allowed if the query is used inside a FROM clause. The following query throws an error: SELECT * FROM ( SELECT id FROM corpus LIMIT 10 ) AS inner_table; The error message is: syntax error, unexpected LIMIT, expecting INTERSECT or EXCEPT or UNION or ')' in: "select * Cheers, Viktor

1 0

[MonetDB-users] How can I execute SQL via mclient directly?
by bai dongli 02 Feb '12

02 Feb '12

Hi guys, I'm pretty new for Monetdb, As per the introduce of the online document, I have to type the password on the command line window when I want to execute SQL via the mclient program. It there a way like this which can be used in Mysql: "mysql --user --password databasename -e SQL" in monetdb? I didn't found the argument that can used for the "password" in mclient. Thanks, Tony

2 4

[MonetDB-users] Can't drop "global temporary table" in SCHEMA tmp
by bai dongli 02 Feb '12

02 Feb '12

Hi guys, I got a problem when I trying to drop a global temporary table in my database, I got following errors: sql>drop table tmp.test; 42S02!DROP TABLE: no such table 'test' I got the same error as above when I trying to execute a "select" query on that table, It's so weird, cause the table "test" was been there: sql>\d TABLE tmp.test The even more weird things happened when I execute following command: sql>create global temporary table tmp.test(id int); operation successful (1.888ms) sql>\d TABLE tmp.test sql>\d test table tmp.test is not unique, corrupt catalog? sql>\d TABLE tmp.test sql>drop table tmp.test more>; operation successful (1.470ms) sql>\d TABLE tmp.test Please pay attention on this line "table tmp.test is not unique, corrupt catalog?". Do anybody can help me out of here? Thanks, Tony

1 0

[MonetDB-users] Batch sql inserts in monetdb
by Federico Dayan 01 Feb '12

01 Feb '12

Hello all, I'm working in a project and we are planning to use monetdb. Our Data: We have around 15 millon users and 6,000 columns. Our Problem We need to be able to query specific log data. We use Hadoop to massage the data and create a sql file with sql inserts (around 15 millon insert with 3000 columns) Out Question Is it efficent to run a command like. By efficent I refe:. betweent 1 to 6 hours efficent, after that, a big problem. monetdb -u ....... < my_sql_insert.sql with 15 millon inserts and between 1000 to 3000 columns? (I'm not always going to create insert for the 6000 columns) could monetdb handle multiple of those command if a split the file. monetdb -u ....... < my_sql_insert_1.sql monetdb -u ....... < my_sql_insert_2.sql monetdb -u ....... < my_sql_insert_3.sql ... Is SQL COPY more efficient than a sql scrip file with inserts? BAT files is it significant faster ? I'm trying to avoid this approach since it seems a little bit obscue. any other suggestion? I'm not able to run all this test and get the info myself becuase I will not have a monetdb ready soon. Thank you very much and I would be glad to send back an email with the results I got and contribute in that way. Federico

4 3

[MonetDB-users] size of table
by Pavel Stehule 30 Jan '12

30 Jan '12

Hello I can't to search in doc a way to get size of tables. Is there some function or statement how to do it? Thank you Pavel Stehule

3 3

[MonetDB-users] Best backup strategy
by Sékine Coulibaly 27 Jan '12

27 Jan '12

Hi there, Thanks again for your last replies regarding the insertion into base. Using COPY INTO worked like a charm. A production deployment means : - Backup - High availability So my questions here, not addressed in the online docs are : - Regarding backup, is there a way to make a consistent backup (I mean putting the DB in a "backuping state", and save/archive the files) - To allow for high availability, can I use clusterization ? Can horizontal scalability be achieved through clusterization ? Thank you !

1 0

[MonetDB-users] Announcement: New Dec2011 Feature release of MonetDB suite
by Sjoerd Mullender 26 Jan '12

26 Jan '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2011 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2011 feature release SQL * mclient: The csv output format can now also be of the form csv=c and csv+c where c is the column separator. The form with csv+ produces a single header line with column names. * Changed a bug where the sign() function returned the same type as its argument instead of always an INTEGER. * Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements. * Fixed a bug when using default values for interval columns. Bug 2877. * A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882. * A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883. Java Module * Implemented getMetaData() method of PreparedStatement. * Fixed an AssertionError for special results from e.g. EXPLAIN queries. * Fixed overflow error when batching large statements, bug #2952 * Resolved a concurrency problem where ResultSet's date-related getters could cause odd stack traces when used by multiple threads at the same time, bug #2950. * JDBC now implements JDBCv4.1 which makes it possible to be built with Java 7 and up. JDBCv4.1 is a maintenance release of JDBCv4, and hence can be compiled with Java 6, although the added methods obviously are not part of the java.sql interfaces. * SQLExceptions thrown now carry a SQLSTATE. Until the server starts sending correct SQLSTATEs for all errors, server originated errors without SQLSTATE are considered generic data exceptions (22000). * Fixed a bug where closing ResultSets and PreparedStatements could lead to errors on concurrent running queries using the same Connection due to a race condition. * Changed version scheme of JDBC driver and MCL jar to be more standard, from monetdb-X.Y-<thing>.jar to monetdb-<thing>-X.Y.jar, bug #2943 * The embedded monet instance capability of MonetConnection was removed. * Bump JDBC version to 2.0 (Liberica). JDBC now implements JDBCv4 which makes it possible to be built with Java 6 and up. Java 5 and before are no longer supported, and can use the 1.x releases of the driver. * INTERVAL columns are now treated as decimals, since they can have sub-second precision. Client Package * Fix making connections with stethoscope to hosts without monetdbd. Bug #2944. * Fixed some bugs in the PHP interface affecting the mapi_ping(), monetdb_insert_id() and auto_commit() functions. Bugs #2936, #2937, #2938. * ODBC: Fixed SQLNumResultCols and SQLDescribeCol to return useful information after a call to SQLPrepare and before any SQLExecute. * mclient: The exact interpretation of the -i (--interactive) option and the - filename argument have changed. The - filename argument means read from standard input and no longer implies that no \ interpretation is done on the input. Instead, \ interpretation is done if either standard input is a terminal, or if the -i option is given. The -i option no longer causes mclient to read from standard input. It only means to do \ interpretation when reading from standard input. Use the - filename argument to read from standard input. Note that if no -s option is specified and no filename arguments are present, mclient still reads from standard input. * ODBC: Improved internal query for SQLSpecialColumns. Before, the query returned all columns taking part in a PRIMARY KEY *and* all columns taking part in a UNIQUE constraint. Now it returns only one or the other set. * ODBC: The database name is now used as the catalog name throughout. Functions that return a catalog name return the database name, and functions that match on catalog name match it with the database name. * ODBC: Implemented an easier way to create a log file of interactions with the ODBC driver. You can now add a connection attribute "LOGFILE=filename" to the connection string parameter of SQLBrowseConnect and SQLDriverConnect, and to the relevant part of the Windows registry or odbc.ini file. This value is only used if there is no environment variable ODBCDEBUG. * Quoting of object names for mclient's \d command is now more flexible and consistent with standard SQL quoting rules, bug #2846. * ODBC: Implemented SQLColumnPrivileges function. * Install new program, mnc, which provides netcat functionality, but based on MonetDB's communication libraries * Report full URI to database instead of just database when available in mclient. * The time format of the timer output can now be controlled with an optional argument to the -i option. ms, s and m force the time to be formatted as milliseconds, seconds or minutes + seconds respectively. * ODBC: Implemented a workaround in SQLTables for bug 2908. * Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability. * Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901 * mclient now checks the result of encoding conversions using the iconv library. * Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv). * Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois. * Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output * Fixed Perl DBD rowcount for larger results, bug #2889 * Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema). * Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois. * Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois. * mclient: fix display of varchar columns with only NULL values. * Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables. MonetDB5 Server * A couple of memory leaks have been plugged. * Introduced new variable max_clients that allows to define how many concurrent connections are allowed to be made against the database. * mal_interpreter.mx Protect against concurrent exceptions If multiple parallel blocks access the exception variables and perform GDKfree on old messages, then we may enter a case that one thread attempts a second free. Simple lock-based protection is the first line of defense. * bpm and partitions have been moved to the attic. It is replaced by the partition optimizer, still under construction. * The MAL debugger next/step operations semantics have been fixed. The profiler also now clearly shows entrance/exit of a MAL function. * The dataflow optimizer uses a less strict side-effect test for BAT new operations and better recognizes trivial plans. * Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888. Merovingian * Fixed crash in monetdb when an invalid property was retrieved using the get command, bug #2953. * monetdb get output is now grouped by database instead of by property * Unlike in previous releases, 'monetdbd get mserver' now returns the path to the mserver5 binary only for dbfarms that have a running monetdbd, instead of only for those that are not served by a monetdbd. * Add nclients property that controls the maximum number of concurrent clients allowed to access the database * Starting monetdbd without any arguments or without dbfarm is no longer supported. A dbfarm now must be provided for each command. * The control passphrase has been turned into a hash of the password, for a more stronger authorisation model as used by mclients. On upgrade, any existing passphrase is converted to the hashed version of the passphrase. * The monetdbd controlport option has been removed in favour of a boolean control option. On upgrade, when controlport was set to a non-zero value and a passphrase was set, control is set to true, or false otherwise. * The monetdbd discoveryport option has been removed in favour of a boolean discovery option. On upgrade, when discoveryport was set to a non-zero value, discovery is set to true, or false otherwise. * Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port. * Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped. * monetdbd get status now also reports the version of the running monetdbd * Improved response time of 'monetdb start' when the database fails to start. * Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database. * To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client. MonetDB Common * Fixed memory detection on 32-bits Solaris systems with more memory than can be addressed in 32-bits mode, bug #2935 Bug Fixes

1 0

[MonetDB-users] measure performance time
by jfergar 26 Jan '12

26 Jan '12

Hi, I need to measure the sql query execution time of several queries. The mclient "-t" option does not exist in the newer version, and the "\< file_with_queries" SQL command does not seem to show the time when importing from a file. Any ideas?? -- View this message in context: http://old.nabble.com/measure-performance-time-tp33201681p33201681.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] Strange disk I/O behavior
by Viktor Rosenfeld 26 Jan '12

26 Jan '12

Hi, I'm observing disk I/O when evaluating some queries which I am unable to explain. I have two datasets, one relatively small with the BATs taking up 1.6 MB on disk and one fairly large with 283 MB on disk. According to iostat, when I evaluate the attached query on the small dataset, 100 MB are written to disk, but none are read. This is on a Linux machine with 4 GB RAM running the Aug2011-SP3 release. I don't understand why data is written on disk, that (a) is never read again and (b) would easily fit into main memory. I am unable to evaluate the queries on the large dataset because MonetDB writes so much data that the disk fills up after a few minutes (more than 22GB). Please let me know if I can provide more information. Cheers, Viktor

3 12

[MonetDB-users] MonetDB to X100 relations
by Pavel Stehule 24 Jan '12

24 Jan '12

Hello I am interested about current state of MonetDB/X100 project? It is freely available? It is part of current MonetDB 5 project? Can we expect some progress and possible availability under some open source licence? Thank you for information Best regards Pavel Stehule

2 2

[MonetDB-users] ODBC Error
by Luiz Gustavo de Souza 18 Jan '12

18 Jan '12

Greetings MonetDB developers! I'm preparing MonetDB for an application, and I got the current error from the ODBC: [MonetDB][ODBC Driver 11.5.9]Cannot return the number of output columns. Query must be executed first. I'm using Windows 7 Ultimate 64bits with both ODBC x86 and x64, but the system and also the Microsoft Excel only find the x86 driver. How can I overcome this? Thank you very much.

2 1

[MonetDB-users] Slow import from a SQL (20 tps)
by Sékine Coulibaly 18 Jan '12

18 Jan '12

I'm evaluating MonetDB and importing an SQL file containing INSERT statements, so that I can populate the following table : CREATE TABLE session ( cmd character varying(25), date bigint, session_id character varying(64), score integer, src_ip character varying(25), dst_ip character varying(25), tx_bytes integer, rx_bytes integer, retx_bytes integer, scrambling character varying(16), url character varying(64), country character varying(32), orig character varying(3), status integer ) No index was created. >From what I can see, I'm somewhere between 20 and 40 insertions per second (Core i5, 4GB RAM, Windows7, MonetDb MonetDB5-SQL-Installer-i386-20111021.msi). The Windows task manager shows a disk write throughput of 300 kB/s which looks like very small, processor usage around 5%. I imported the file with mclient -u monetdb < myfile.sql I expected to import at 200 rows/second minimum. Am I doing something wrong ? What can be done to improve the loading speed ? Thank you SC

2 2

[MonetDB-users] How to find implementation function of a MAL function
by Charalampos Nikolaou 18 Jan '12

18 Jan '12

Hello, I would like to know whether there is any way to find out the C function implementing a MAL function through mclient (maybe with the ?pattern command). Let's say I search for batcalc.and(:bat, :bat) function. If there isn't any, then what is the C function implementing the MAL function I mentioned previously? Thanks, Babis

2 1

[MonetDB-users] C++ Driver (postgresql proxy)
by Sékine Coulibaly 18 Jan '12

18 Jan '12

Hi, I plan to use MonetDb in a C++ project and have the following questions/concerns : - Using an third party ODBC library (such as SOCI) to add ODBC support to my project looks like risky. Is MonetDB fully compliant with ODBC ? Does anyone know if it works well with SOCI (or any other open source ODBC library) ? - I really would prefer using an integrated library (Postgresql for example ships with a client library for embedding SQL in C/C++ code, and a libpq library). I guess this is not available for MonetDb. Confirmed ? - Previously, I came through a driver that allowed to proxy MonetDb to a virtual Postgresql. I mean that the proxy attacks MonetDb in ODBC (and hopefully it does it well), and then exposes the MonetDb database through the standard Postgesql interface. Hence you can use PgAdminIII, pgadmin... on port 5432 which is the Postgesql port. I can't find this proxy driver anymore. Does anyone ? Thank you !

1 0

[MonetDB-users] Stored Procedure code example.
by Miguel Chang 18 Jan '12

18 Jan '12

Hello there, before I start with my question I would like to thank all those developers who put so much effort on this great Open Source Database. My name is Miguel, I´m currently using the MonetDB database for a university project and so far it´s been great. By following the documentation on the webpage I´ve build a fully functional system but I´ve encoutered one problem: So far I´ve been unable to create an stored procedure, even after following all the instructions on the documentation page: http://www.monetdb.org/Documentation/Manuals/SQLreference/Procedures Here´s an example I´ve tried but that does not works: CREATE PROCEDURE SP_GET_ALL_EVENTS () BEGIN SELECT * FROM DEMO.EVENTS; END; >From the many variation I tried with the last query I always get these errors: "Unexpected From, Expected In" or "unexpected END" I´ve been trying many variations (INSERT, UPDATES, DELETES) and I get the same responses... So can someone please point me what´s the error in my code? and also show me some examples codes for stored procedure creation? Much Appreciated.

2 2

[MonetDB-users] MonetDB unexpected count/group by slowdown
by zack_pcd 17 Jan '12

17 Jan '12

I have a table f_trigger with 171,452,378 records. It has fields script_id and machine_uuid (and other fields not used here). I want to run select script_id,count(distinct machine_uuid) from f_trigger group by script_id; First, though, let's get the number of expected records only: select count(*) from (select script_id,count(distinct machine_uuid) from f_trigger group by script_id) as subq; +------+ | L7 | +======+ | 208 | +------+ 1 tuple (359.124ms) Very nice. Let's explain the query before I run it: explain select script_id,count(distinct machine_uuid) from f_trigger group by script_id; +-------------------------------------------------------------------------------+ | mal | +===============================================================================+ | function user.s0_1{autoCommit=true}():void; | | X_21 := nil:bat[:oid,:str]; | | X_39:bat[:oid,:wrd] := nil:bat[:oid,:wrd]; | | barrier X_596 := language.dataflow(); | | X_2 := sql.mvc(); | | X_12:bat[:oid,:str] := sql.bind(X_2,"sys","f_trigger","script_id",0); | | X_10:bat[:oid,:str] := sql.bind(X_2,"sys","f_trigger","script_id",2); | | X_14 := algebra.kdifference(X_12,X_10); | | X_15 := algebra.kunion(X_14,X_10); | | X_9:bat[:oid,:str] := sql.bind(X_2,"sys","f_trigger","script_id",1); | | X_16 := algebra.kunion(X_15,X_9); | | X_5:bat[:oid,:oid] := sql.bind_dbat(X_2,"sys","f_trigger",1); | | X_8 := bat.reverse(X_5); | | X_17 := algebra.kdifference(X_16,X_8); | | (ext28,grp26) := group.done(X_17); | | X_20 := bat.mirror(ext28); | | X_21 := algebra.join(X_20,X_17); | | X_29:bat[:oid,:str] := sql.bind(X_2,"sys","f_trigger","machine_uuid",0); | | X_30 := algebra.selectNotNil(X_29); | | X_26:bat[:oid,:str] := sql.bind(X_2,"sys","f_trigger","machine_uuid",2); | | X_31 := algebra.kdifference(X_30,X_26); | | X_28 := algebra.selectNotNil(X_26); | | X_32 := algebra.kunion(X_31,X_28); | | X_24:bat[:oid,:str] := sql.bind(X_2,"sys","f_trigger","machine_uuid",1); | | X_25 := algebra.selectNotNil(X_24); | | X_33 := algebra.kunion(X_32,X_25); | | X_34 := algebra.kdifference(X_33,X_8); | | (X_35,grp55) := group.derive(ext28,grp26,X_34); | | X_37 := bat.mirror(X_35); | | X_38 := algebra.join(X_37,X_34); | | X_39:bat[:oid,:wrd] := aggr.count(X_38,grp26,X_20); | | exit X_596; | | X_40 := sql.resultSet(2,1,X_21); | | sql.rsColumn(X_40,"sys.f_trigger","script_id","varchar",32768,0,X_21); | | sql.rsColumn(X_40,"sys.f_trigger","L10","wrd",64,0,X_39); | | X_53 := io.stdout(); | | sql.exportResult(X_53,X_40); | | end s0_1; | +-------------------------------------------------------------------------------+ 38 tuples (3.136ms) Running the query takes a very long time. While the query is running, I notice VERY heavy disk writes, and almost no reads and almost no CPU activity. Based on other information on the mailing list, I suspect the query generates intermediate data that gets writted to disk (instead of kept in memory). We are running monetdb on a 32 core linux machine with 256 GB of RAM and a fast raid array. Memory usage is under 20%, so I would not expect monetdb to be writing to disk so much (50-500MB/s writes)... Database settings: shared default yes nthreads local 32 optpipe local recycler_pipe master default no slave default <unknown> readonly local yes Stethoscope output: [1] # event, time, thread, usec, stmt, # name [1] [ 256, "14:02:38.525477", 2, 0, "calc.str(\"script_id\");" ] [1] [ 257, "14:02:38.525477", 2, 134, "calc.str(\"script_id\");" ] [1] [ 258, "14:02:38.525634", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 259, "14:02:38.525634", 2, 18, "calc.str(\"f_trigger\");" ] [1] [ 260, "14:02:38.525670", 2, 0, "calc.str(\"script_id\");" ] [1] [ 261, "14:02:38.525670", 2, 16, "calc.str(\"script_id\");" ] [1] [ 262, "14:02:38.525704", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 263, "14:02:38.525704", 2, 17, "calc.str(\"f_trigger\");" ] [1] [ 264, "14:02:38.525740", 2, 0, "calc.str(\"script_id\");" ] [1] [ 265, "14:02:38.525740", 2, 17, "calc.str(\"script_id\");" ] [1] [ 266, "14:02:38.525777", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 267, "14:02:38.525777", 2, 19, "calc.str(\"f_trigger\");" ] [1] [ 268, "14:02:38.525812", 2, 0, "calc.str(\"script_id\");" ] [1] [ 269, "14:02:38.525812", 2, 19, "calc.str(\"script_id\");" ] [1] [ 270, "14:02:38.525848", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 271, "14:02:38.525848", 2, 19, "calc.str(\"f_trigger\");" ] [1] [ 272, "14:02:38.525884", 2, 0, "calc.str(\"L11\");" ] [1] [ 273, "14:02:38.525884", 2, 19, "calc.str(\"L11\");" ] [1] [ 274, "14:02:38.525925", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 275, "14:02:38.525925", 2, 20, "calc.str(\"f_trigger\");" ] [1] [ 276, "14:02:38.525962", 2, 0, "calc.str(\"L11\");" ] [1] [ 277, "14:02:38.525962", 2, 18, "calc.str(\"L11\");" ] [1] [ 278, "14:02:38.526000", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 279, "14:02:38.526000", 2, 18, "calc.str(\"f_trigger\");" ] [1] [ 280, "14:02:38.526039", 2, 0, "calc.str(\"machine_uuid\");" ] [1] [ 281, "14:02:38.526039", 2, 19, "calc.str(\"machine_uuid\");" ] [1] [ 282, "14:02:38.526077", 2, 0, "calc.str(\"f_trigger\");" ] [1] [ 283, "14:02:38.526077", 2, 20, "calc.str(\"f_trigger\");" ] [1] [ 284, "14:02:38.526116", 2, 0, "calc.str(nil:str);" ] [1] [ 285, "14:02:38.526116", 2, 22, "calc.str(nil:str);" ] [1] [ 286, "14:02:38.528707", 2, 0, "X_21 := nil:bat[:oid,:str];" ] [1] [ 287, "14:02:38.528707", 2, 64, "X_21 := nil:bat[:oid,:str];" ] [1] [ 288, "14:02:38.528790", 2, 0, "X_39:bat[:oid,:wrd] := nil:bat[:oid,:wrd];" ] [1] [ 289, "14:02:38.528790", 2, 27, "X_39:bat[:oid,:wrd] := nil:bat[:oid,:wrd];" ] [1] [ 290, "14:02:38.528835", 2, 0, "barrier X_596 := language.dataflow();" ] [1] [ 291, "14:02:38.528928", 40, 0, "X_2 := sql.mvc();" ] [1] [ 292, "14:02:38.528928", 40, 141, "X_2 := sql.mvc();" ] [1] [ 293, "14:02:38.529100", 40, 0, "X_12:bat[:oid,:str] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"script_id\",0);" ] [1] [ 294, "14:02:38.529100", 40, 78, "X_12:bat[:oid,:str] =<tmp_7153>[171452378] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"script_id\",0);" ] [1] [ 295, "14:02:38.529255", 52, 0, "X_10:bat[:oid,:str] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"script_id\",2);" ] [1] [ 296, "14:02:38.529255", 52, 229, "X_10:bat[:oid,:str] =<tmp_11631>[0] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"script_id\",2);" ] [1] [ 297, "14:02:38.529491", 45, 0, "X_5:bat[:oid,:oid] := sql.bind_dbat(X_2=0,\"sys\",\"f_trigger\",1);" ] [1] [ 298, "14:02:38.529491", 45, 201, "X_5:bat[:oid,:oid] =<tmp_11616>[0] := sql.bind_dbat(X_2=0,\"sys\",\"f_trigger\",1);" ] [1] [ 299, "14:02:38.529699", 50, 0, "X_9:bat[:oid,:str] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"script_id\",1);" ] [1] [ 300, "14:02:38.529699", 50, 227, "X_9:bat[:oid,:str] =<tmp_11632>[0] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"script_id\",1);" ] [1] [ 301, "14:02:38.529951", 56, 0, "X_24:bat[:oid,:str] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"machine_uuid\",1);" ] [1] [ 302, "14:02:38.530203", 52, 0, "X_14 := algebra.kdifference(X_12=<tmp_7153>:bat[:oid,:str][171452378],X_10=<tmp_11631>:bat[:oid,:str][0]);" ] [1] [ 303, "14:02:38.529951", 56, 259, "X_24:bat[:oid,:str] =<tmp_11632>[0] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"machine_uuid\",1);" ] [1] [ 304, "14:02:38.530203", 52, 112, "X_14=<tmp_12321>[171452378] := algebra.kdifference(X_12=nil:bat[:oid,:str],X_10=<tmp_11631>:bat[:oid,:str][0]);" ] [1] [ 305, "14:02:38.530308", 45, 0, "X_8 := bat.reverse(X_5=<tmp_11616>:bat[:oid,:oid][0]);" ] [1] [ 306, "14:02:38.530308", 45, 143, "X_8=<tmpr_11616>[0] := bat.reverse(X_5=nil:bat[:oid,:oid]);" ] [1] [ 307, "14:02:38.530465", 57, 0, "X_29:bat[:oid,:str] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"machine_uuid\",0);" ] [1] [ 308, "14:02:38.530668", 56, 0, "X_25 := algebra.selectNotNil(X_24=<tmp_11632>:bat[:oid,:str][0]);" ] [1] [ 309, "14:02:38.530465", 57, 210, "X_29:bat[:oid,:str] =<tmp_10750>[171452378] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"machine_uuid\",0);" ] [1] [ 310, "14:02:38.530668", 38, 0, "X_26:bat[:oid,:str] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"machine_uuid\",2);" ] [1] [ 311, "14:02:38.530668", 56, 44, "X_25=<tmp_11632>[0] := algebra.selectNotNil(X_24=nil:bat[:oid,:str]);" ] [1] [ 312, "14:02:38.530668", 38, 1743, "X_26:bat[:oid,:str] =<tmp_11631>[0] := sql.bind(X_2=0,\"sys\",\"f_trigger\",\"machine_uuid\",2);" ] [1] [ 313, "14:02:38.530946", 61, 0, "X_15 := algebra.kunion(X_14=<tmp_12321>[171452378],X_10=<tmp_11631>:bat[:oid,:str][0]);" ] [1] [ 314, "14:02:38.531115", 57, 0, "X_30 := algebra.selectNotNil(X_29=<tmp_10750>:bat[:oid,:str][171452378]);" ] [1] [ 315, "14:02:38.531115", 57, 39, "X_30=<tmp_10750>[171452378] := algebra.selectNotNil(X_29=nil:bat[:oid,:str]);" ] [1] [ 316, "14:02:38.530946", 61, 210, "X_15=<tmp_12211>[171452378] := algebra.kunion(X_14=nil,X_10=nil:bat[:oid,:str]);" ] [1] [ 317, "14:02:38.531190", 57, 0, "X_31 := algebra.kdifference(X_30=<tmp_10750>[171452378],X_26=<tmp_11631>:bat[:oid,:str][0]);" ] [1] [ 318, "14:02:38.531271", 61, 0, "X_16 := algebra.kunion(X_15=<tmp_12211>[171452378],X_9=<tmp_11632>:bat[:oid,:str][0]);" ] [1] [ 319, "14:02:38.531190", 57, 99, "X_31=<tmp_12321>[171452378] := algebra.kdifference(X_30=nil,X_26=<tmp_11631>:bat[:oid,:str][0]);" ] [1] [ 320, "14:02:38.531271", 61, 42, "X_16=<tmp_12216>[171452378] := algebra.kunion(X_15=nil,X_9=nil:bat[:oid,:str]);" ] [1] [ 321, "14:02:38.531346", 57, 0, "X_28 := algebra.selectNotNil(X_26=<tmp_11631>:bat[:oid,:str][0]);" ] [1] [ 322, "14:02:38.531346", 57, 55, "X_28=<tmp_11631>[0] := algebra.selectNotNil(X_26=nil:bat[:oid,:str]);" ] [1] [ 323, "14:02:38.531408", 61, 0, "X_17 := algebra.kdifference(X_16=<tmp_12216>[171452378],X_8=<tmpr_11616>[0]);" ] [1] [ 324, "14:02:38.531466", 57, 0, "X_32 := algebra.kunion(X_31=<tmp_12321>[171452378],X_28=<tmp_11631>[0]);" ] [1] [ 325, "14:02:38.531408", 61, 59, "X_17=<tmp_12211>[171452378] := algebra.kdifference(X_16=nil,X_8=<tmpr_11616>[0]);" ] [1] [ 326, "14:02:38.531466", 57, 49, "X_32=<tmp_12216>[171452378] := algebra.kunion(X_31=nil,X_28=nil);" ] [1] [ 327, "14:02:38.531537", 61, 0, "(ext28,grp26) := group.done(X_17=<tmp_12211>[171452378]);" ] [1] [ 328, "14:02:38.531596", 57, 0, "X_33 := algebra.kunion(X_32=<tmp_12216>[171452378],X_25=<tmp_11632>[0]);" ] [1] [ 329, "14:02:38.531596", 57, 34, "X_33=<tmp_12261>[171452378] := algebra.kunion(X_32=nil,X_25=nil);" ] [1] [ 330, "14:02:38.531660", 57, 0, "X_34 := algebra.kdifference(X_33=<tmp_12261>[171452378],X_8=<tmpr_11616>[0]);" ] [1] [ 331, "14:02:38.531660", 57, 28, "X_34=<tmp_12216>[171452378] := algebra.kdifference(X_33=nil,X_8=nil);" ] [1] [ 332, "14:02:38.531537", 61, 4488161, "(ext28=<tmp_12207>[208],grp26=<tmp_12321>[171452378]) := group.done(X_17=<tmp_12211>[171452378]);" ] [1] [ 333, "14:02:43.019887", 61, 0, "X_20 := bat.mirror(ext28=<tmp_12207>[208]);" ] [1] [ 334, "14:02:43.019887", 61, 39, "X_20=<tmp_12261>[208] := bat.mirror(ext28=<tmp_12207>[208]);" ] [1] [ 335, "14:02:43.019962", 61, 0, "(X_35,grp55) := group.derive(ext28=<tmp_12207>[208],grp26=<tmp_12321>[171452378],X_34=<tmp_12216>[171452378]);" ] [1] [ 336, "14:02:43.020045", 60, 0, "X_21 := algebra.join(X_20=<tmp_12261>[208],X_17=<tmp_12211>[171452378]);" ] [1] [ 337, "14:02:43.020045", 60, 505, "X_21=<tmp_12332>[208] := algebra.join(X_20=<tmp_12261>[208],X_17=nil);" ] [1] [ 338, "13:50:05.243130", 21, 2365556004, "(X_41=<tmp_12243>[90737912],grp74) := group.derive(ext30=nil,grp28=<tmp_12111>[171452378],X_40=<tmp_12110>[171452378]);" ] [1] [ 339, "14:29:30.799335", 21, 0, "X_43 := bat.mirror(X_41=<tmp_12243>[90737912]);" ] [1] [ 340, "14:29:30.799335", 21, 44, "X_43=<tmp_12113>[90737912] := bat.mirror(X_41=nil);" ] [1] [ 341, "14:29:30.799408", 21, 0, "X_44 := algebra.join(X_43=<tmp_12113>[90737912],X_40=<tmp_12110>[171452378]);" ] [1] [ 342, "14:29:30.799408", 21, 1931164, "X_44=<tmp_12117>[90737912] := algebra.join(X_43=nil,X_40=nil);" ] [1] [ 343, "14:29:32.730825", 21, 0, "X_45:bat[:oid,:wrd] := aggr.count(X_44=<tmp_12117>[90737912],grp28=<tmp_12111>[171452378],X_20=<tmp_12107>[208]);" ] [1] [ 344, "14:29:32.730825", 21, 2686923, "X_45:bat[:oid,:wrd] =<tmp_12110>[208] := aggr.count(X_44=nil,grp28=nil,X_20=nil);" ] [1] [ 345, "13:49:55.404991", 35, 2380012994, "barrier X_602 := language.dataflow();" ] [1] [ 346, "14:29:35.418135", 35, 0, "X_47 := sql.resultSet(2,1,X_25=<tmp_12112>[10]);" ] [1] [ 347, "14:29:35.418135", 35, 53, "X_47 := sql.resultSet(2,1,X_25=<tmp_12112>[10]);" ] [1] [ 348, "14:29:35.418208", 35, 0, "X_46 := algebra.leftjoin(X_24=<tmp_12115>[10],X_45=<tmp_12110>:bat[:oid,:wrd][208]);" ] [1] [ 349, "14:29:35.418208", 35, 130, "X_46=<tmp_12172>[10] := algebra.leftjoin(X_24=nil,X_45=nil:bat[:oid,:wrd]);" ] [1] [ 350, "14:29:35.418370", 35, 0, "sql.rsColumn(X_47=1,\"sys.f_trigger\",\"script_id\",\"varchar\",32768,0,X_25=<tmp_12112>[10]);" ] [1] [ 351, "14:29:35.418370", 35, 36, "sql.rsColumn(X_47=1,\"sys.f_trigger\",\"script_id\",\"varchar\",32768,0,X_25=nil);" ] [1] [ 352, "14:29:35.418429", 35, 0, "sql.rsColumn(X_47=1,\"sys.f_trigger\",\"L6\",\"wrd\",64,0,X_46=<tmp_12172>[10]);" ] [1] [ 353, "14:29:35.418429", 35, 34, "sql.rsColumn(X_47=1,\"sys.f_trigger\",\"L6\",\"wrd\",64,0,X_46=nil);" ] [1] [ 354, "14:29:35.418483", 35, 0, "X_57 := io.stdout();" ] [1] [ 355, "14:29:35.418483", 35, 18, "X_57 := io.stdout();" ] [1] [ 356, "14:29:35.418515", 35, 0, "sql.exportResult(X_57==\"42083632\":streams,X_47=1);" ] [1] [ 357, "14:29:35.418515", 35, 73, "sql.exportResult(X_57==\"42083632\":streams,X_47=1);" ] [1] [ 358, "14:29:35.418611", 35, 0, "end s3_2;" ] [1] [ 359, "14:02:43.019962", 61, 2633497166, "(X_35=<tmp_12112>[90737912],grp55) := group.derive(ext28=nil,grp26=<tmp_12321>[171452378],X_34=<tmp_12216>[171452378]);" ] [1] [ 360, "14:46:36.517329", 61, 0, "X_37 := bat.mirror(X_35=<tmp_12112>[90737912]);" ] [1] [ 361, "14:46:36.517329", 61, 43, "X_37=<tmp_12206>[90737912] := bat.mirror(X_35=nil);" ] [1] [ 362, "14:46:36.517404", 61, 0, "X_38 := algebra.join(X_37=<tmp_12206>[90737912],X_34=<tmp_12216>[171452378]);" ] [1] [ 363, "14:46:36.517404", 61, 2374517, "X_38=<tmp_12211>[90737912] := algebra.join(X_37=nil,X_34=nil);" ] [1] [ 364, "14:46:38.892126", 61, 0, "X_39:bat[:oid,:wrd] := aggr.count(X_38=<tmp_12211>[90737912],grp26=<tmp_12321>[171452378],X_20=<tmp_12261>[208]);" ] [1] [ 365, "14:46:38.892126", 61, 2690276, "X_39:bat[:oid,:wrd] =<tmp_12216>[208] := aggr.count(X_38=nil,grp26=nil,X_20=nil);" ] [1] [ 366, "14:02:38.528835", 2, 2643053835, "barrier X_596 := language.dataflow();" ] [1] [ 367, "14:46:41.583009", 2, 0, "X_40 := sql.resultSet(2,1,X_21=<tmp_12332>[208]);" ] [1] [ 368, "14:46:41.583009", 2, 48, "X_40 := sql.resultSet(2,1,X_21=<tmp_12332>[208]);" ] [1] [ 369, "14:46:41.583076", 2, 0, "sql.rsColumn(X_40=1,\"sys.f_trigger\",\"script_id\",\"varchar\",32768,0,X_21=<tmp_12332>[208]);" ] [1] [ 370, "14:46:41.583076", 2, 40, "sql.rsColumn(X_40=1,\"sys.f_trigger\",\"script_id\",\"varchar\",32768,0,X_21=nil);" ] [1] [ 371, "14:46:41.583140", 2, 0, "sql.rsColumn(X_40=1,\"sys.f_trigger\",\"L11\",\"wrd\",64,0,X_39=<tmp_12216>:bat[:oid,:wrd][208]);" ] [1] [ 372, "14:46:41.583140", 2, 35, "sql.rsColumn(X_40=1,\"sys.f_trigger\",\"L11\",\"wrd\",64,0,X_39=nil:bat[:oid,:wrd]);" ] [1] [ 373, "14:46:41.583198", 2, 0, "X_53 := io.stdout();" ] [1] [ 374, "14:46:41.583198", 2, 18, "X_53 := io.stdout();" ] [1] [ 375, "14:46:41.583230", 2, 0, "sql.exportResult(X_53==\"35761888\":streams,X_40=1);" ] [1] [ 376, "14:46:41.583230", 2, 403, "sql.exportResult(X_53==\"35761888\":streams,X_40=1);" ] [1] [ 377, "14:46:41.583661", 2, 0, "end s1_1;" ] The query took 44 minutes to complete. Is this a bug (as I suspect)? Or is the group by/count distinct operation just slow? If it is a bug, is this fixed in dec-2011? If that is the case, I would like to verify this was fixed for us in dec-2011 - however the last "latest" build I tried was crashing for me. I am not sure which version of dec-2011 I should try for testing this. Thanks -- View this message in context: http://old.nabble.com/MonetDB-unexpected-count-group-by-slowdown-tp33137428… Sent from the monetdb-users mailing list archive at Nabble.com.

3 4

[MonetDB-users] column sortedness
by David Pennell 13 Jan '12

13 Jan '12

I'm running some experiments with a 1B row fact table (10 columns). - How can I tell if a column is sorted? - If I generate files in sorted order and use binary bulk loading, will monetdb consider the column sorted? If not, can I force it? I've been reading code, but understanding all the assignments to bn->tsorted is going to take a while. I get the general theme of setting tsorted during bat operations, but I've been unsuccessful in finding where it is initialized when a column is first accessed. TIA, -david

2 2

[MonetDB-users] Prepare statements as performance upgrade?
by Stefan de Konink 12 Jan '12

12 Jan '12

Hi, When I look into the executemany implementation of the Python DBAPI, I see that executemany is implemented as loop over the parameters given the same statement. In the mapi documentation there is written about using mapi_prepare_array() and mapi_execute_array(). Would such thing be beneficial in the general case? In my own program I have the following: while True: received = data_from_network() update_monetdb(received) It seems that the standard DBAPI doesn't expose prepared statements as some sort of transparent usage, so using SQL PREPARE sounds as the most logical way. What is the SQL statement to 'forget' the stored prepared statement? Stefan

1 0

[MonetDB-users] Disk space of a table
by Vassilis Papakonstantinou 11 Jan '12

11 Jan '12

Hello, Is there a way to get the disk space of a table? Vassilis

3 4

[MonetDB-users] how to load NaN as REAL
by bigu 10 Jan '12

10 Jan '12

I have a csv file and some columns are defined as REAL. The values some time have 'NaN' so when trying to load in the db it says cannot load NaN into REAL. COPY 1000000 OFFSET 2 RECORDS into mytable from 'C:\temp.csv' delimiters ',', '\n'; SQLException:importTable:value 'NaN' from line 11 field 10 not inserted, expecting type real failed to import table -- View this message in context: http://old.nabble.com/how-to-load-NaN-as-REAL-tp33107777p33107777.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 2

[MonetDB-users] load CSV without header row
by bigu 09 Jan '12

09 Jan '12

Today installed monetdb for first time and wanted to play with some of the features but stuck with loading the file. Excuse me if this looks too basic. There is a 3GB csv file (1M rows and 400 columns). I am trying load in monetdb on my windows 7 machine. I first created a table mostly data is numeric exect for few that are metadata strings. How do i load this file without removing the header. The error says column_name not of type integer since the column_name is the actual header string but the values are integers. I can remove the header manually before loading but if i plan to scale then it is not a good option. Is there a way to say load "n" records from file starting at 'x' to 'y'? or ignore first line. COPY 1000000 RECORDS into mytable from 'C:\data.csv' delimiters ',', '\n'; Thanks. -- View this message in context: http://old.nabble.com/load-CSV-without-header-row-tp33096188p33096188.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] [PATCH]: python-monetdb does not support RIPEMD160
by W-Mark Kubacki 09 Jan '12

09 Jan '12

Hello! Although MonetDB supports RIPEMD160, python-monetdb (version 11.5.9) does not. Patch [1] fixes that. Unfortunately Python's hashlib.algorithms and dir(hashlib) is "incomplete" in this regards and lists only algorithms which are guaranteed to be supported. Yet more are actually implemented. The patch moves the responsibility for checking whether RIPEMD160 (and others) is supported to hashlib, retaining the NotSupportedError. -- Mark Kubacki [1] https://raw.github.com/wmark/ossdl-overlay/master/dev-python/python-monetdb…

2 1

[MonetDB-users] Max Connection Limit
by Joseph Brower 06 Jan '12

06 Jan '12

Right now we are hitting the limit of 64 connections to MonetDB. We're working on some middleware that will consolidate those into only 2 or 3 connections, but until we finish it, we need to increase the limit. How do we go about doing that? Thanks, Joseph Brower

3 7

[MonetDB-users] Error in SELECT from a function via Mapi
by Iain Buchanan 02 Jan '12

02 Jan '12

Hi, I'm having problems calling a function via the Mapi interface (using Visual C++ under Windows and the 15th November download). It works fine when I call it directly, but fails when I call via Mapi. I create the function: create function tick.dummy(a integer) returns table (entry integer) begin return (select 1); end; I can then call this directly from mclient without any problems: sql>select entry from tick.dummy(1); +-------+ | entry | +=======+ | 1 | +-------+ 1 tuple (1.074ms) sql> However, if I call via Mapi I receive an error: SELECT: no such unary operator 'dummy(tinyint)' I'm using the following sequence of API calls: MapiHdl handle = mapi_prepare(connection, "SELECT entry FROM tick.dummy(?)"); int param(0); mapi_param_type(handle, 0, MAPI_INT, MAPI_AUTO, &param); if (mapi_execute(handle) != MOK) { // Error here } The above seems to work fine for a general SELECT statement, but fails for one with a function. Is there a different syntax for calling via Mapi? Iain

1 0

[MonetDB-users] ENUM style table with JOIN versus automatically deduplicated VARCHAR
by Stefan de Konink 02 Jan '12

02 Jan '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, MonetDB 'deduplicates' strings, is there any benchmark that outlines the performance difference of an ENUM with a low number of (string)items versus the use of a VARCHAR field? The technical question would be; Is it more efficient to do: select * from my_table, my_enum where mytable.my_enum_id = my_enum.id and my_enum.name = 'Something'; or select * from my_table where my_enum_varchar = 'something'; I can imagine that the following would out perform a string comparison, but implies hardcoded values. select * from my_table, my_enum where mytable.my_enum_id = hardcoded; Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk78fnkACgkQYH1+F2Rqwn3BUQCeOUtPIGzcHT1xpnsF4n78BQ/1 JWAAnjnzPeT8D7CtsjYgAiGg/WRIgNbQ =989f -----END PGP SIGNATURE-----

2 1

[MonetDB-users] MonetDB internals, does UPDATE update in place?
by Stefan de Konink 29 Dec '11

29 Dec '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Just to be very sure; I know that the DELETE, INSERT strategy to update the database will guarantee a database that keeps on growing, because of the lack of VACUUM. Is the normal UPDATE statement for fixed width datatypes in place, or is it also a form of copy-on-write? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk78fPwACgkQYH1+F2Rqwn2fxACfWdXvwr4pWODNo4ZH9uq5oVS9 BmwAn16XGh8h/vYsGkvthKej3MA8Nyvi =KK5T -----END PGP SIGNATURE-----

1 0

[MonetDB-users] Maximum size for SQL command in mclient
by Charalampos Nikolaou 28 Dec '11

28 Dec '11

Hi, is there a maximum size for an SQL command (e.g., INSERT INTO statement) that is imposed in mclient? The connection is terminated when I execute several commands from a sql file one of which is quite long. The point where I get the error is at the 8192th byte (which is the last of 8KBs). Babis

2 2

[MonetDB-users] protocol violation: unexpected line in data block
by tangray 27 Dec '11

27 Dec '11

the whole log file : RD 1298855541531: read final block: 57 bytes RX 1298855541531: onxRWd3Yv:mserver:9:RIPEMD160,SHA256,SHA1,MD5:LIT:SHA512: RD 1298855541531: inserting prompt TD 1298855541531: write final block: 92 bytes TX 1298855541531: BIG:risk:{SHA256}01fa94fefcadb5918473e49aaa8e425538f8d40960b2d649348cb17c599031ae:sql:krisk: RD 1298855541546: read final block: 0 bytes RX 1298855541546: RD 1298855541546: inserting prompt TD 1298855541546: write final block: 48 bytes TX 1298855541546: sSET TIME ZONE INTERVAL '+08:00' HOUR TO MINUTE; RD 1298855541546: read final block: 0 bytes RX 1298855541546: RD 1298855541546: inserting prompt TD 1298855541546: write final block: 15 bytes TX 1298855541546: Xreply_size 250 RD 1298855541546: read final block: 0 bytes RX 1298855541546: RD 1298855541546: inserting prompt TD 1298855541546: write final block: 46 bytes TX 1298855541546: sSELECT "name", "value" FROM sys.env() as env; RD 1298855541546: read final block: 4246 bytes RX 1298855541546: &1 0 46 2 46 % .env, .env # table_name % name, value # name % varchar, varchar # type % 18, 203 # length [ "gdk_arch", "32biti686-pc-win32" ] [ "gdk_version", "1.40.3" ] [ "prefix", "D:\\Program Files\\MonetDB\\MonetDB5" ] [ "exec_prefix", "D:\\Program Files\\MonetDB\\MonetDB5" ] [ "gdk_dbname", "krisk" ] [ "gdk_dbfarm", "C:\\Documents and Settings\\swordtan\\Application Data\\MonetDB5\\dbfarm" ] [ "gdk_debug", "0" ] [ "gdk_alloc_map", "no" ] [ "gdk_vmtrim", "yes" ] [ "monet_admin", "adm" ] [ "monet_prompt", ">" ] [ "monet_welcome", "yes" ] [ "monet_mod_path", "D:\\Program Files\\MonetDB\\MonetDB5\\lib\\MonetDB5;D:\\Program Files\\MonetDB\\MonetDB5\\lib\\MonetDB5\\lib;D:\\Program Files\\MonetDB\\MonetDB5\\lib\\MonetDB5\\bin" ] [ "monet_daemon", "no" ] [ "host", "localhost" ] [ "mapi_port", "50000" ] [ "mapi_clients", "2" ] [ "mapi_open", "true" ] [ "mapi_autosense", "false" ] [ "default_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "minimal_pipe", "inline,remap,deadcode,multiplex,garbageCollector" ] [ "sql_optimizer", "default_pipe" ] [ "sql_debug", "0" ] [ "standoff_ns", "" ] [ "standoff_start", "start" ] [ "standoff_end", "end" ] [ "config", "D:\\Program Files\\MonetDB\\MonetDB5\\etc\\monetdb5.conf" ] [ "mero_pidfile", "D:\\Program Files\\MonetDB\\MonetDB5\\var\\run\\MonetDB\\merovingian.pid" ] [ "mero_controlport", "50001" ] [ "nov2009_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "replication_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,replication,multiplex,garbageCollector" ] [ "accumulator_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,accumulators,dataflow,history,multiplex,garbageCollector" ] [ "recycler_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,deadcode,recycle,reduce,dataflow,history,multiplex,garbageCollector" ] [ "cracker_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,selcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "sidcrack_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,sidcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "datacell_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,datacell,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "octopus_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,constants,commonTerms,joinPath,octopus,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "mapreduce_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mapreduce,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "datacyclotron_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,datacyclotron,mergetable,deadcode,constants,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,replication,multiplex,garbageCollector" ] [ "derive_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,derivePath,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "dictionary_pipe", "inline,remap,dictionary,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector" ] [ "compression_pipe", "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector" ] [ "gdk_mem_pagebits", "16" ] [ "gdk_mem_bigsize", "1048576" ] [ "monet_pid", "984" ] [ "monet_version", "5.22.3" ] RD 1298855541546: inserting prompt TD 1298855541562: write final block: 60 bytes TX 1298855541562: sselect * from risk.td_ind_security_held where id = 1560000; RD 1298855541578: read final block: 1600 bytes RX 1298855541578: &1 1 1 30 1 % risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held # table_name % id, bizdate, painnercode, currency, exchange, seccode, basickind, unrealizedgainchange, unrealizedgain, margingain, interestgain, othergain, yieldrate, accrualinterest, allgain, fullmv, marketvalue, lastfullmv, lastmarketvalue, lastcost, cost, buysum, sellsum, realizedgain, holdingqty, lastholdingqty, mvweight, lastmvweight, fullmvweight, lastfullmvweight # name % int, varchar, varchar, varchar, varchar, varchar, varchar, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal # type % 7, 8, 8, 3, 5, 6, 3, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 14, 14, 14, 14 # length RD 1298855541578: inserting prompt TD 1298855762109: write final block: 60 bytes TX 1298855762109: sselect * from risk.td_ind_security_held where id = 1560000; RD 1298855762125: read final block: 1600 bytes RX 1298855762125: &1 2 1 30 1 % risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held # table_name % id, bizdate, painnercode, currency, exchange, seccode, basickind, unrealizedgainchange, unrealizedgain, margingain, interestgain, othergain, yieldrate, accrualinterest, allgain, fullmv, marketvalue, lastfullmv, lastmarketvalue, lastcost, cost, buysum, sellsum, realizedgain, holdingqty, lastholdingqty, mvweight, lastmvweight, fullmvweight, lastfullmvweight # name % int, varchar, varchar, varchar, varchar, varchar, varchar, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal, decimal # type % 7, 8, 8, 3, 5, 6, 3, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 14, 14, 14, 14 # length RD 1298855762125: inserting prompt TD 1298856279453: write final block: 69 bytes TX 1298856279453: sselect id,bizdate from risk.td_ind_security_held where id = 1560000; RD 1298856279453: read final block: 162 bytes RX 1298856279453: &1 3 1 2 1 % risk.td_ind_security_held, risk.td_ind_security_held # table_name % id, bizdate # name % int, varchar # type % 7, 8 # length [ 1560000, "20090702" ] RD 1298856279453: inserting prompt TD 1298856279468: write final block: 1797 bytes TX 1298856279468: sSELECT 'krisk' AS "TABLE_CAT", "schemas"."name" AS "TABLE_SCHEM", "tables"."name" AS "TABLE_NAME", "columns"."name" AS "COLUMN_NAME", cast(CASE "columns"."type" WHEN 'wrd' THEN -5 WHEN 'sec_interval' THEN -5 WHEN 'bigint' THEN -5 WHEN 'timetz' THEN 92 WHEN 'char' THEN 1 WHEN 'int' THEN 4 WHEN 'clob' THEN 2005 WHEN 'month_interval' THEN 4 WHEN 'tinyint' THEN -6 WHEN 'table' THEN 2003 WHEN 'date' THEN 91 WHEN 'double' THEN 8 WHEN 'decimal' THEN 3 WHEN 'timestamptz' THEN 93 WHEN 'timestamp' THEN 93 WHEN 'blob' THEN 2004 WHEN 'real' THEN 7 WHEN 'time' THEN 92 WHEN 'oid' THEN 1111 WHEN 'boolean' THEN 16 WHEN 'smallint' THEN 5 WHEN 'varchar' THEN 12 ELSE 1111 END AS smallint) AS "DATA_TYPE", "columns"."type" AS "TYPE_NAME", "columns"."type_digits" AS "COLUMN_SIZE", "columns"."type_scale" AS "DECIMAL_DIGITS", 0 AS "BUFFER_LENGTH", 10 AS "NUM_PREC_RADIX", cast(CASE "null" WHEN true THEN 1 WHEN false THEN 0 END AS int) AS "NULLABLE", cast(null AS varchar(1)) AS "REMARKS", "columns"."default" AS "COLUMN_DEF", 0 AS "SQL_DATA_TYPE", 0 AS "SQL_DATETIME_SUB", 0 AS "CHAR_OCTET_LENGTH", "columns"."number" + 1 AS "ORDINAL_POSITION", cast(null AS varchar(1)) AS "SCOPE_CATALOG", cast(null AS varchar(1)) AS "SCOPE_SCHEMA", cast(null AS varchar(1)) AS "SCOPE_TABLE", cast(1111 AS smallint) AS "SOURCE_DATA_TYPE", CASE "null" WHEN true THEN CAST ('YES' AS varchar(3)) WHEN false THEN CAST ('NO' AS varchar(3)) END AS "IS_NULLABLE" FROM "sys"."columns" AS "columns", "sys"."tables" AS "tables", "sys"."schemas" AS "schemas" WHERE "columns"."table_id" = "tables"."id" AND "tables"."schema_id" = "schemas"."id" AND LOWER("schemas"."name") LIKE 'risk' AND LOWER("tables"."name") LIKE 'td_ind_security_held' AND LOWER("columns"."name") LIKE 'id' ORDER BY "TABLE_SCHEM", "TABLE_NAME", "ORDINAL_POSITION"; RD 1298856279484: read final block: 853 bytes RX 1298856279484: &1 4 1 22 1 % ., sys.schemas, .tables, .columns, ., .columns, .columns, .columns, ., ., ., ., .columns, ., ., ., ., ., ., ., ., . # table_name % TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, DECIMAL_DIGITS, BUFFER_LENGTH, NUM_PREC_RADIX, NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB, CHAR_OCTET_LENGTH, ORDINAL_POSITION, SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE, SOURCE_DATA_TYPE, IS_NULLABLE # name % char, varchar, varchar, varchar, smallint, varchar, int, int, tinyint, tinyint, int, varchar, varchar, tinyint, tinyint, tinyint, int, varchar, varchar, varchar, smallint, varchar # type % 5, 4, 20, 2, 1, 3, 2, 1, 1, 2, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 4, 2 # length [ "krisk", "risk", "td_ind_security_held", "id", 4, "int", 32, 0, 0, 10, 0, NULL, NULL, 0, 0, 0, 1, NULL, NULL, NULL, 1111, "NO" ] RD 1298856279484: inserting prompt TD 1298856279500: write final block: 1802 bytes TX 1298856279500: sSELECT 'krisk' AS "TABLE_CAT", "schemas"."name" AS "TABLE_SCHEM", "tables"."name" AS "TABLE_NAME", "columns"."name" AS "COLUMN_NAME", cast(CASE "columns"."type" WHEN 'wrd' THEN -5 WHEN 'sec_interval' THEN -5 WHEN 'bigint' THEN -5 WHEN 'timetz' THEN 92 WHEN 'char' THEN 1 WHEN 'int' THEN 4 WHEN 'clob' THEN 2005 WHEN 'month_interval' THEN 4 WHEN 'tinyint' THEN -6 WHEN 'table' THEN 2003 WHEN 'date' THEN 91 WHEN 'double' THEN 8 WHEN 'decimal' THEN 3 WHEN 'timestamptz' THEN 93 WHEN 'timestamp' THEN 93 WHEN 'blob' THEN 2004 WHEN 'real' THEN 7 WHEN 'time' THEN 92 WHEN 'oid' THEN 1111 WHEN 'boolean' THEN 16 WHEN 'smallint' THEN 5 WHEN 'varchar' THEN 12 ELSE 1111 END AS smallint) AS "DATA_TYPE", "columns"."type" AS "TYPE_NAME", "columns"."type_digits" AS "COLUMN_SIZE", "columns"."type_scale" AS "DECIMAL_DIGITS", 0 AS "BUFFER_LENGTH", 10 AS "NUM_PREC_RADIX", cast(CASE "null" WHEN true THEN 1 WHEN false THEN 0 END AS int) AS "NULLABLE", cast(null AS varchar(1)) AS "REMARKS", "columns"."default" AS "COLUMN_DEF", 0 AS "SQL_DATA_TYPE", 0 AS "SQL_DATETIME_SUB", 0 AS "CHAR_OCTET_LENGTH", "columns"."number" + 1 AS "ORDINAL_POSITION", cast(null AS varchar(1)) AS "SCOPE_CATALOG", cast(null AS varchar(1)) AS "SCOPE_SCHEMA", cast(null AS varchar(1)) AS "SCOPE_TABLE", cast(1111 AS smallint) AS "SOURCE_DATA_TYPE", CASE "null" WHEN true THEN CAST ('YES' AS varchar(3)) WHEN false THEN CAST ('NO' AS varchar(3)) END AS "IS_NULLABLE" FROM "sys"."columns" AS "columns", "sys"."tables" AS "tables", "sys"."schemas" AS "schemas" WHERE "columns"."table_id" = "tables"."id" AND "tables"."schema_id" = "schemas"."id" AND LOWER("schemas"."name") LIKE 'risk' AND LOWER("tables"."name") LIKE 'td_ind_security_held' AND LOWER("columns"."name") LIKE 'bizdate' ORDER BY "TABLE_SCHEM", "TABLE_NAME", "ORDINAL_POSITION"; RD 1298856279515: read final block: 862 bytes RX 1298856279515: &1 5 1 22 1 % ., sys.schemas, .tables, .columns, ., .columns, .columns, .columns, ., ., ., ., .columns, ., ., ., ., ., ., ., ., . # table_name % TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, DECIMAL_DIGITS, BUFFER_LENGTH, NUM_PREC_RADIX, NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB, CHAR_OCTET_LENGTH, ORDINAL_POSITION, SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE, SOURCE_DATA_TYPE, IS_NULLABLE # name % char, varchar, varchar, varchar, smallint, varchar, int, int, tinyint, tinyint, int, varchar, varchar, tinyint, tinyint, tinyint, int, varchar, varchar, varchar, smallint, varchar # type % 5, 4, 20, 7, 2, 7, 1, 1, 1, 2, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 4, 2 # length [ "krisk", "risk", "td_ind_security_held", "bizdate", 12, "varchar", 8, 0, 0, 10, 0, NULL, NULL, 0, 0, 0, 2, NULL, NULL, NULL, 1111, "NO" ] RD 1298856279515: inserting prompt TD 1298856508906: write final block: 77 bytes TX 1298856508906: sselect id,bizdate,allgain from risk.td_ind_security_held where id = 1560000; RD 1298856508906: read final block: 187 bytes RX 1298856508906: &1 6 1 3 1 % risk.td_ind_security_held, risk.td_ind_security_held, risk.td_ind_security_held # table_name % id, bizdate, allgain # name % int, varchar, decimal # type % 7, 8, 20 # length RD 1298856508906: inserting prompt

2 1

[MonetDB-users] java.lang.AssertionError: block 0 should have been fetched by now
by Viktor Rosenfeld 27 Dec '11

27 Dec '11

Hi, I get an error message when I retrieve a MAL plan over JDBC after the 250th call to ResultSet.next(). The relevant part of the stack trace is: java.lang.AssertionError: block 0 should have been fetched by now :( at nl.cwi.monetdb.jdbc.MonetConnection$ResultSetResponse.getLine(MonetConnection.java:1585) at nl.cwi.monetdb.jdbc.MonetResultSet.absolute(MonetResultSet.java:194) at nl.cwi.monetdb.jdbc.MonetResultSet.relative(MonetResultSet.java:2149) at nl.cwi.monetdb.jdbc.MonetResultSet.next(MonetResultSet.java:2116) The call to EXPLAIN works in mclient and returns a MAL plan that is 402 lines long. I can reproduce the error when I try to create the plan via jdbcclient, except that it says that block 1 should have been fetched and the exporter stops after line 160 or so. However, other SQL queries that also produce more than 250 results work over JDBC. Not sure what the difference is, but these queries only return column of integers. I'm using the Aug2011-SP3 release on OS X 10.6.8. Please let me know if I should provide other information like the SQL query or the MAL plan. Cheers, Viktor

2 3

[MonetDB-users] time series data scaling
by David Pennell 20 Dec '11

20 Dec '11

In a large scale time series row-store DB (Oracle), I'm accustomed to partitioning fact tables on time. This has has a couple of benefits: - since queries always include predicates on the time field, partition elimination can help a lot - when the time based partitions map one-to-one with tablespaces, removing old data is extremely fast. (when you've reached the maximum data retention period, you need to remove data as fast as you can load it). Are there any features in monetdb that would facilitate these needs (or ameliorate them)? TIA, -david

2 2

[MonetDB-users] Benefits of sorted columns
by Viktor Rosenfeld 19 Dec '11

19 Dec '11

Hi, I know that when a table is sorted on a column, value look-ups on that column will use a binary search instead of a linear search. Is there a similar benefit for joins? E.g. if I have 2 tables that contain a sorted column C and I do an equality join on this column (or a self-join using 2 aliases of the same table), I would expect a traditional DBMS to do a merge-join. Cheers, Viktor

2 2

[MonetDB-users] SQLException:SQLengine:Program contains errors
by Charalampos Nikolaou 17 Dec '11

17 Dec '11

Hi, I am getting the error "SQLException:SQLengine:Program contains errors" when I define a simple function that returns a table. The SQL reference on the site of MonetDB (http://www.monetdb.org/Documentation/SQLreference/Flowofcontrol) states that the accepted expression to a "TABLE" expression should be "select_no_parens_orderby" which is not defined in any place (or haven't found it) other than sql_parser.y. The function I define is the following and apart from the fact that is inserted successfully into the "functions" table, I checked that it can be produced by the grammar in sql_parser.y: CREATE FUNCTION refine () RETURNS TABLE (u varchar(32), v varchar(32), r varchar(5)) BEGIN RETURN TABLE ( ( SELECT R.u, R.v, R.r FROM R INTERSECT SELECT R.u, R.v, R.r FROM R ) ); END; Further, what is the proper way to deleting a function, in order to re-define it in the same session? I observed that is not sufficient to just delete it from sys.functions table. An entry #function_name is also present in sys._tables. Thanks, Babis

3 5

Re: [MonetDB-users] Information request
by Ying Zhang 16 Dec '11

16 Dec '11

On Fri, Dec 16, 2011 at 10:51:18PM +0800, dj zhang wrote: > Hello Jennie, > > Glad to hear from you! Thanks for your reply! I'm sorry that I didn't > realize that you guys are in a new year vacation. Hello Dongjie, I'd like to recommend you to subscribe to the MonetDB users mailling list: http://www.monetdb.org/Developers/Mailinglists , and sent your feature questions to the mailinglist. This way, your questions will reach the whole developer team at one, and you'll most probably get answers faster. I felt free to forward this e-mail to the mailing list. > Maybe I didn't express myself very clear. I've generated a table file in > which each line represents a record, and the fields of a record are > separated by "|", e.g. "abc|1.2.3.3|www.monetdb.org|3". I can bulk load > this file to MonetDB by SQL COPY INTO command. Unfortunately, you can't load this table using the binary bulk load feature of MonetDB, since this feature only handles numerical data, and doesn't not work with strings. How large (in GBs) is your total dataset? Maybe you don't have to use the binary bulk load feature... > Now I want to change the table file to column files, each file just > contains one column's values which also stored line by line. I want to know > if I can load this table by loading these column files. > > From your information, Can I say that I have to change these column files > to binary format that MonetDB recognise(BAT storage), then use binary bulk > load command to load the table to MonetDB. And if I change the files to > other binary format it will not work. Is that right? That's right. MonetDB just works with one binary format. Regards, Jennie > > Thank you so much! Once again happy new year! > > Best wishes, > > Dongjie > ------------------------------------- > > On Fri, Dec 16, 2011 at 8:55 PM, Ying Zhang <Y.Zhang(a)cwi.nl> wrote: > > > Hello Zhang Dongjie, > > > > First of all, thank you very much for your interest in MonetDB. > > > > > I have learn from the website document that I can load data use column > > > files when migrate between MonetDB instances, but the file has to be in > > the > > > format of binary version of the BAT storage if I didn't misunderstand. > > > > If each column of a table is already stored in binary format in a file, > > the table can be loaded using the "binary bulk load" feature of MonetDB, > > as described here: > > http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/BinaryBulkLoad > > > > If you are migrating your existing database to an exactly the same > > machine, you can just copy dbfarm sub-directory for your database. If > > you are migrating to a machine with different hardware, you need to dump > > the database and reload it on the destination machine, as described here: > > http://www.monetdb.org/Documentation/UserGuide/DumpRestore > > However, in this case, the dumped data can not be loaded using the > > binary bulk load feature, because the file containing dumpted data is an > > ASCII file. > > > > > So my question is, is there any way to load data to monetdb by column as > > > long as the record is store by column. Or do I have to first change the > > > column files to the binary version of BAT format then use the Binary Bulk > > > Load commands. > > > > You can only use the binary bulk load feature, iff the data of each > > column is stored in a separate file in binary format. Hence, yes, > > you'll have to generate those binary files, if you don't have them yet. > > However, if your data is already in, e.g., CSV format, you don't have to > > create binary files for the columns, since you can just load the CSV > > file. If your (CSV) data is not too large, the standard SQL COPY INTO > > feature should just work. > > > > > And if I have to change the format, is there any way to use google > > protobuf > > > to serialize the column file to the BAT format, or is there any solutions > > > like handlersocket for mysql? > > > > I'm unfortunately not familiar with protobuf and handlersocket. Maybe > > someone else in our group can answer these questions. > > > > Hope this helps a bit. > > > > > > With kind regads, > > > > Jennie Zhang > > > > > > > > > > By the way I think MonetDB is a great project and I wish it can find it's > > > way to commercialize very soon! > > > > > > Thank you very much, and Merry Christmas! > > > > > > Best regards, > > > > > > Dongjie > > > ------------------ > > > Dongjie Zhang > > > > > > Services Computing Technology and System Lab, > > > Cluster and Grid Computing Lab, > > > School of Computer Science and Technology, > > > Huazhong University of Science and Technology, > > > Wuhan, 430074, China > > > > > > 华中科技大学计算机科学与技术学院 > > > 服务计算技术与系统教育部重点实验室集群与网格计算湖北省重点实验室 > > > 湖北武汉 430074 > > > > > _______________________________________________ > > > Info mailing list > > > Info(a)monetdb.org > > > http://mail.monetdb.org/mailman/listinfo/info > > > > > > > -- > ------------------ > Dongjie Zhang > > Services Computing Technology and System Lab, > Cluster and Grid Computing Lab, > School of Computer Science and Technology, > Huazhong University of Science and Technology, > Wuhan, 430074, China > > 华中科技大学计算机科学与技术学院 > 服务计算技术与系统教育部重点实验室集群与网格计算湖北省重点实验室 > 湖北武汉 430074

1 0

[MonetDB-users] Invalid BAT if left join condition has only columns from right table
by George Garbis 15 Dec '11

15 Dec '11

Hello to all, a left join which have only right table columns in join condition seems to produce invalid bat. Assume we have the tables below: CREATE TABLE tableA(x integer, y integer); INSERT INTO tableA values(1,10); INSERT INTO tableA values(2,20); CREATE TABLE tableB(x integer, y integer); INSERT INTO tableB values(1,-10); INSERT INTO tableB values(2,-20); and pose the query: SELECT A.x, A.y, B.x, B.y FROM tableA A LEFT JOIN tableB B ON ( B.y < -10 ) ; the expected results are: +------+----- +------+------+ | x | y | x | y | +===+===+===+===+ | 1 | 10 | 2 | -20 | | 2 | 20 | 2 | -20 | +------+------+------+------+ but mclient prints this message: TypeException:user.s1_1[36]:'algebra.kdifference' undefined in: _55:any := algebra.kdifference(_54:bat[:oid,:int], _44:bat[:void,:oid]) TypeException:user.s1_1[37]:'algebra.markT' undefined in: _56:any := algebra.markT(_55:any, _21:oid) TypeException:user.s1_1[38]:'bat.reverse' undefined in: _57:any := bat.reverse(_56:any) TypeException:user.s1_1[39]:'algebra.join' undefined in: _58:any := algebra.join(_57:any, _20:bat[:oid,:int]) TypeException:user.s1_1[41]:'algebra.leftjoin' undefined in: _60:any := algebra.leftjoin(_59:bat[:oid,:void], _54:bat[:oid,:int]) TypeException:user.s1_1[43]:'bat.append' undefined in: _64:any := bat.append(_61:bat[:oid,:int], _60:any, _65:bit) TypeException:user.s1_1[44]:'bat.append' undefined in: _66:any := bat.append(_64:any, _58:any, _65:bit) TypeException:user.s1_1[55]:'algebra.join' undefined in: _80:any := algebra.join(_57:any, _79:bat[:oid,:int]) TypeException:user.s1_1[61]:'algebra.leftjoin' undefined in: _90:any := algebra.leftjoin(_59:bat[:oid,:void], _89:bat[:oid,:int]) TypeException:user.s1_1[63]:'bat.append' undefined in: _92:any := bat.append(_91:bat[:oid,:int], _90:any, _65:bit) TypeException:user.s1_1[64]:'bat.append' undefined in: _93:any := bat.append(_92:any, _80:any, _65:bit) TypeException:user.s1_1[70]:'algebra.project' undefined in: _102:any := algebra.project(_57:any, _100:int) TypeException:user.s1_1[83]:'bat.append' undefined in: _119:any := bat.append(_117:bat[:oid,:int], _102:any, _118:bit) TypeException:user.s1_1[89]:'algebra.project' undefined in: _126:any := algebra.project(_57:any, _125:int) TypeException:user.s1_1[96]:'bat.append' undefined in: _133:any := bat.append(_132:bat[:oid,:int], _126:any, _118:bit) SQLException:SQLengine:Program contains errors For ease I attach sql scripts for creating tables and posing the query Best regards, George Garbis

2 1

[MonetDB-users] ODBC Stops inserting
by Joseph Brower 13 Dec '11

13 Dec '11

Hello everyone! We're using the odbc driver to insert records every half second or so. These records are for analytics information about visitors to various websites that we host. (We found MonetDB to be very fast for fetching information like average request time, number of visitors, general analytics information like that.) We've run into a few issues though. The biggest concern is that we stop seeing new records entered after a while. The log shows 2011-12-13 17:00:00 MSG merovingian[86]: proxying client 10.60.5.13:37359 for database 'web_requests' to mapi:monetdb:///var/lib/monetdb/web_requests/web_requests/.mapi.sock?database=web_requests 2011-12-13 17:00:00 MSG merovingian[86]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying over and over for each machine that would be inserting these records. But no new records are ever added. Another concern is that in our installation, we don't see a MonetDB.conf file. I was hoping that we could specify how much RAM we'd like to allow MonetDB to consume. This is to ensure that our memory manager doesn't kill MonetDB, since that's the primary role of this server. Is this a file that we should create? I'm still fairly new to MonetDB and as a result, am still trying to find my way with it. The last concern I have is the log file size. Is there a way (once we've ironed out the before mentioned issues) to instruct MonetDB to only log things of a certain importance or higher? For example, to only log ERR rather than ERR and MSG messages? Our log files will become quite large and unmanageable if we don't disable or implement some sort of log rotation system. The version of MonetDB that we are running is MonetDB 5 server v11.5.7 "Aug2011-SP2". We are running under Ubuntu 10.04 64Bit. Thanks, Joseph Brower

2 3

[MonetDB-users] SpagoBI
by Jeremy OSullivan 09 Dec '11

09 Dec '11

Hello,We are trying to get SpagoBI to work with MonetDB. SpagoBI uses a JDBC interface. The problem is as follows: The import of the schema works fine. However, when you save the model in SpagoBI, the schema in MonetDB can no longer be viewed. We have tried this with MySQL and it works fine.Regards,Jeremy

1 0

[MonetDB-users] Can't create a csv vault?
by Mark Aufflick 08 Dec '11

08 Dec '11

Hi all, I'm very interested in the vault feature of monetdb, both for csv and for creating my own formats. Trouble is I can't make a simple example work. I've flattended the voc sample data into a csv file, and then try to load it so: create table test (type varchar(255), number integer, number_sup varchar(255), trip integer, trip_sup varchar(255), onboard_at_departure integer, death_at_cap integer, left_at_cape integer, onboard_at_cape integer, death_during_voyage integer, onboard_at_arrival integer) external csv '/tmp/all.csv'; But I get an error: syntax error, unexpected EXTERNAL, expecting SCOLON in: ... Do I need to enable vaults at configure time or create time or something? I didn't see an option for it. I built from the "Latest" tarball on OSX Lion. Cheers, Mark.

2 3

[MonetDB-users] stethoscope on OS X can't connect to database
by Viktor Rosenfeld 07 Dec '11

07 Dec '11

Hi, when I try to connect to a database with stethoscope I get an error: $ stethoscope -d annis_test -u annis_test -P ... +tis stethoscope: no databases found for 'annis_test' I'm using the Aug2011-SP3 release on OS X 10.6.8. The same command works under Linux. The instances are set up identically. Cheers, Viktor

2 2

[MonetDB-users] JDBC documentation inconsistencies
by Viktor Rosenfeld 29 Nov '11

29 Nov '11

Hi, the example at the top of the documentation of the JDBC client at http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/Clients/JDBC uses the -d switch to specify the table and the -b switch to specify the database. However the current version does not recognize the -b switch and the -d switch can be used to select the database as is documented further on the page. Also, the page says that TRACE is not possible via JDBC. However, as the jdbcclient.jar demonstrates it is indeed possible to use TRACE on a query by checking additional result sets. Cheers, Viktor

2 4

[MonetDB-users] Announcement: New Aug2011-SP3 Bugfix release of MonetDB suite
by Fabian Groffen 22 Nov '11

22 Nov '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Note that this is the last release of the Aug2011 branch. From now on, no further development will take place on this branch. Aug 2011-SP3 bugfix release SQL * Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements. Merovingian * Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port. Bug Fixes * 2875: incomplete database initialization * 2912: Function body disappears after loading data - error message: SQLException:sql.resultSet:Cannot access descriptor * 2919: Add column and select in same transaction * 2920: Updated database from Aug2011 to Dec2011 make queries fail with "Subquery result missing". * 2921: complex where clause broken * 2923: Multiplication and cast from decimal to integer leads to 0 value * 2924: use LIKE with ESCAPE ignores OR operator * 2928: Cannot update a field with a count query based on an other table

1 0

[MonetDB-users] Workflow for independent, read only, discretely updated tables.
by Hedge Hog 20 Nov '11

20 Nov '11

Hi, I'm evaluating MonetDB, and the following is based on reading the docs and some of the monetdb-users email list archive. I wonder if the following setup+workflow is correct, and if so, is it monetdb best practice, given this use case: We will have read only tables, each distributed across several machines/servers, they will be updated daily. There will be no cross table queries, i.e. only one table touched by each quert. We would like to be able to update each table while not affecting the availability of any other table. The table_db's are not horizontally sharded, i.e. all the data for a query will always come from that one table. We cannot use udp multicast/broadcast so a monetdb cluster is not possible (unless a localhost cluster is possible/sensible?). The setup+workflow: - one table per database (this allows for independent table updates), let these be <table_db>. - update a master (writable) instance of the table on a 'special' monetdbd/machine. - copy the updated table to each machine. - To update the table on a machine: $> monetbd lock <mytable_db> $> monetbd stop <mytable_db> $> mclient -u monetdb <mytable_db> updatefile $> cat updatefile copy into MyTable from ('path_to_mytable_col_file_i', 'path_to_mytable_col_file_f', 'path_to_mytable_col_file_s'); $> monetbd release <mytable_db> $> monetbd start <mytable_db> Is the above the best pattern/architecture of monetdb for such a use case? Appreciate any insights people can offer TIA Hedge -- πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα [The fox knows many things, but the hedgehog knows one big thing.] Archilochus, Greek poet (c. 680 BC – c. 645 BC) http://hedgehogshiatus.com

3 4

[MonetDB-users] Unable to use Tableau
by Sékine Coulibaly 18 Nov '11

18 Nov '11

Hi there, I'm a new user to both Tableau and MonetDb. I run under Windows, and my MonetDb base is up and running, a table "matable" was created and populated. Although I can connect the MonetDb base to Tableau (although I get some warnings), as soon as Tableau sends a request to MonetDb, the ODBC driver returns the following : « Database error (0x80004005) : [MonetDB][ODBC Driver 11.5.7]Invalid identifier 'none:name:nk' » As can be seen here : http://puu.sh/8STw Has any of you already used Tableau and solved this issue ? Shall I consider using Pentaho ? Thank you ! Regards

2 1

[MonetDB-users] pcre not found when configure monetdb5
by xldelg＠sohu.com 18 Nov '11

18 Nov '11

hi, I tried to compile the latest monetdb(11.5.3) on centos6 x64, but the pcre library is not found when configure. I'm sure that the pcre library has been installed via compiling the source, the libpcre.la, libpcre.a, libpcre.so has been installed in /usr/local/lib. I also configure monetdb like: pcre_LIBS=/usr/local/lib pcre_CFLAGS=/usr/local/include ./configure, but it doesn't work. How to fix this problem? Any reply will be apprecitated.    

2 2

[MonetDB-users] "TypeException: algebra.join undefined" when evaluating EXISTS (SELECT ...) predicate
by Viktor Rosenfeld 16 Nov '11

16 Nov '11

Hi, I get an error message when executing the following query: SELECT count(*) FROM _rank AS _rank1, _rank AS _rank2 WHERE _rank1.pre = 23 AND _rank2.pre = 33 AND EXISTS (SELECT 1 FROM _rank AS ancestor WHERE ancestor.pre < _rank1.pre AND _rank1.pre < ancestor.post AND ancestor.pre < _rank2.pre AND _rank2.pre < ancestor.post); The error message is: TypeException:user.s0_1[99]:'algebra.join' undefined in: _129:any := algebra.join(_128:bat[:oid,:oid], _108:bat[:int,:oid]) SQLException:SQLengine:Program contains errors A minimal example of test data to run this query follows: CREATE TABLE _rank (pre INTEGER, post INTEGER); INSERT INTO _rank VALUES (22, 37); INSERT INTO _rank VALUES (23, 24); INSERT INTO _rank VALUES (33, 34); Cheers, Viktor

2 2

[MonetDB-users] General Questions on Columnar DB Design and Querying
by Sean McNamara 13 Nov '11

13 Nov '11

Hi all, I've managed to get MonetDB running and loaded with a chunk of data (thanks to some great help from the list) and am now trying to figure out the "right" way to make use of it. I don't have a whole lot of exposure to columnar stores, and am therefore not sure if I'm doing things in an idiomatically correct way. I have a couple general questions about columnar DBs, and hope its appropriate to ask them here even thought they aren't specific to MonetDB. If these questions aren't considered appropriate for this list I apologize in advance and will happily move them to a different forum (but would appreciate pointers to where such a forum might be.) Two questions are general, and one is fairly specific. First the general: 1. Does working with a columnar store fundamentally change the way one would structure the tables? I'm somewhat familiar with star-schema designs where a central fact table holds the entire attribute set in as rows of indexes with supporting lookup tables for each column type, but it seems like columnar stores achieve the same thing without actually requiring the developer to specifically build those structures. Tables are constructed in a de-normalized RDBMS manner and the underlying DB takes care of the vectorization. Is this a valid description, or am I fundamentally misunderstanding the basics? 2. If there are significant differences in the way columnar stores should be designed, can folks point me to any resources (web, book, article, etc.) to help educate me on the topic? Now the specific: I have several sets of time-series data modeled as follows: create table sensordata ( sensor varchar(32), epoch_ts bigint, reading double, CONSTRAINT SENSORDATA_PKEY PRIMARY KEY (sensor, epoch_ts) ); I've populated that data, and now want to query data for a subset of sensors and generate a table that shows the value of the sensors at each point in time. I'm not sure what the right way to do this is. My desired out:ut would be: epoch_ts sensor1 sensor2 delta -------------- ------------ ------------ ------ <time1> 23.5 24.1 0.6 <time2> 19.3 25.2 5.9 ... <timeN> 25.1 19.3 -5.8 I tried to build that query as follows but end up with NULL results (note: I'm in no way a SQL pro so it might be really ugly.) select a.epoch_ts as epoch, a.reading as sensor1_reading, b.reading as sensor2_reading, b.reading - a.reading as delta from (select epoch_ts, reading from sensordata where sensor='sensor1') a, (select epoch_ts, reading from sensordata where sensor='sensor2') b where b.epoch_ts = a.epoch_ts order by epoch asc; Is this a sane way to approach the problem, or am I completely off track here. Each subselect returns results, but they obviously aren't joining correctly and it feels like it might simply be the wrong way to try to build the result set. I appreciate any direction you might provide. Thanks!

2 1

[MonetDB-users] MonetDB Permission/COPY INTO Questions
by Sean McNamara 13 Nov '11

13 Nov '11

Hi All, I'm just staring to play with MonetDB, but am running into some trouble with a few basic tasks. I've managed to create a new DB and have added a table. I'd now like to bulk-load the table with a CSV file full of data. When I try to do so, I get the following: WS3:~ tank$ mclient -u testuser testdb Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011) Database: MonetDB v11.5.1, 'testdb' Type \q to quit, \? for a list of available commands auto commit mode: on sql>COPY 1800 records into testdata from 'testdata.csv'; COPY INTO: insufficient privileges: COPY INTO from file(s) requires administrator rights, use 'COPY INTO "testdata" FROM STDIN' instead I then try to copy into via STDIN: WS3:~ tank$ mclient -u testuser -s "COPY 1800 records into testdata from STDIN using delimiters ',';" testdb < testdata.csv WS3:~ tank$ mclient -u testuser testdb Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011) Database: MonetDB v11.5.1, 'testdb' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select * testdb; +-----+-----+------+ | foo | bar | baz | +===+==+===+ +-----+-----+------+ 0 tuples (2.307ms) sql> It seems clear I have some sort of permission/privilege issue, but I'm not clear how to resolve it. I've looked at all the tables and see the auths, users, user_role, and other tables, but it's not clear what actually needs to be tweaked there or if I should be using other DB commands to set things up. I've tried signing on as the monetdb user to do the import, but the monetdb user doesn't appear to be able to see the table: WS3:~ tank$ mclient -u monetdb testdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011) Database: MonetDB v11.5.1, 'testdb' Type \q to quit, \? for a list of available commands auto commit mode: on sql>COPY 1800 records into testdata from 'testdata.csv'; COPY INTO: no such table 'testdata' I realize this should all be quite simple, but am not finding my way forward. I've been searching the archives and hunting on google but am coming up empty. Can anyone point the way? Thanks!

6 11

[MonetDB-users] General questions on columnar DB design and querying
by Sean McNamara 13 Nov '11

13 Nov '11

Hi all, I've managed to get MonetDB running and loaded with a chunk of data (thanks to some great help from the list) and am now trying to figure out the "right" way to make use of it. I don't have a whole lot of exposure to columnar stores, and am therefore not sure if I'm doing things in an idiomatically correct way. I have a couple general questions about columnar DBs, and hope its appropriate to ask them here even thought they aren't specific to MonetDB. If these questions aren't considered appropriate for this list I apologize in advance and will happily move them to a different forum (but would appreciate pointers to where such a forum might be.) Two questions are general, and one is fairly specific. First the general: 1. Does working with a columnar store fundamentally change the way one would structure the tables? I'm somewhat familiar with star-schema designs where a central fact table holds the entire attribute set in as rows of indexes with supporting lookup tables for each column type, but it seems like columnar stores achieve the same thing without actually requiring the developer to specifically build those structures. Tables are constructed in a de-normalized RDBMS manner and the underlying DB takes care of the vectorization. Is this a valid description, or am I fundamentally misunderstanding the basics? 2. If there are significant differences in the way columnar stores should be designed, can folks point me to any resources (web, book, article, etc.) to help educate me on the topic? Now the specific: I have several sets of time-series data modeled as follows: create table sensordata ( sensor varchar(32), epoch_ts bigint, reading double, CONSTRAINT SENSORDATA_PKEY PRIMARY KEY (sensor, epoch_ts) ); I've populated that data, and now want to query data for a subset of sensors and generate a table that shows the value of the sensors at each point in time. I'm not sure what the right way to do this is. My desired out:ut would be: epoch_ts sensor1 sensor2 delta -------------- ------------ ------------ ------ <time1> 23.5 24.1 0.6 <time2> 19.3 25.2 5.9 ... <timeN> 25.1 19.3 -5.8 I tried to build that query as follows but end up with NULL results (note: I'm in no way a SQL pro so it might be really ugly.) select a.epoch_ts as epoch, a.reading as sensor1_reading, b.reading as sensor2_reading, b.reading - a.reading as delta from (select epoch_ts, reading from sensordata where sensor='sensor1') a, (select epoch_ts, reading from sensordata where sensor='sensor2') b where b.epoch_ts = a.epoch_ts order by epoch asc; Is this a sane way to approach the problem, or am I completely off track here. Each subselect returns results, but they obviously aren't joining correctly and it feels like it might simply be the wrong way to try to build the result set. I appreciate any direction you might provide. Thanks!

1 0

Re: [MonetDB-users] MonetDB Permission/COPY INTO Questions
by Allen Zhang 12 Nov '11

12 Nov '11

Sean, Maybe here is why you did not spot the duplicate: cat /tmp/testdata2.csv | cut -d, -f1,5 | sort | uniq -d Regards Allen On 11/12/11 12:39 PM, "Sean McNamara" <tank(a)thursday.org> wrote: >Looks like I had too much faith in the "uniq" command (or >misunderstand its constraints.) There are definitely duplicates in my >import set. :-( > >Back to data scrubbing... > >On Sat, Nov 12, 2011 at 12:08 PM, Allen Zhang <azhang(a)pmci-usa.com> wrote: >> Sean, >> >> Use sql select to find out duplicates. >> >> Regards >> Allen >> >> On 11/12/11 12:01 PM, "Sean McNamara" <tank(a)thursday.org> wrote: >> >>>I tried that (as both the normal dbuser and as admin). Both fail the >>>same way but don't give an indicator of where it feels the constraint >>>is violated: >>> >>>$ mclient -u monetdb -s "alter table mduser.testdata add constraint >>>testdata_pkey primary key (foo,epoch);" testdb >>>password: >>>SQLException:assert:UPDATE: PRIMARY KEY constraint >>>'testdata.testdata_pkey' violated >>> >>> >>>On Sat, Nov 12, 2011 at 11:42 AM, Allen Zhang <azhang(a)pmci-usa.com> >>>wrote: >>>> How about add constraint after you import? I think it might tell you >>>>which >>>> one had problem. >>>> >>>> Regards >>>> Allen >>>> >>>> On 11/12/11 11:19 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >>>> >>>>>Brilliant suggestion, and I think it exposed the problem. >>>>> >>>>>$ mclient -u monetdb -s "copy into mduser.testdata from >>>>>'/tmp/testdata2.csv' using delimiters ',','\n','\'';" testdb >>>>>password: >>>>>SQLException:importTable:value 'ONE_SECOND' from line 12734 field 3 >>>>>not inserted, expecting type varchar(8) >>>>>failed to import table >>>>> >>>>>It looks like the field I created is too small to handle some of the >>>>>values in the file. After expanding all the varchar fields to 32 and >>>>>re-running, I get a different error: >>>>> >>>>>$ mclient -u monetdb -s "copy into mduser.testdata from >>>>>'/tmp/testdata2.csv' using delimiters ',','\n','\'';" testdb >>>>>password: >>>>>SQLException:assert:INSERT INTO: PRIMARY KEY constraint >>>>>'testdata.testdata_pkey' violated >>>>> >>>>>That makes me think I have duplicate data. The PKEY is comprised of >>>>>fields #1 and #5, so I check for that but it looks fine: >>>>> >>>>>$ cat /tmp/testdata2.csv | cut -d, -f1,5 |uniq -d >>>>>$ >>>>> >>>>>However if I drop and re-add the table without the constraint, it >>>>>works: >>>>> >>>>>sql>create table testdata ( >>>>>more> foo varchar(32), >>>>>more> bar varchar(32), >>>>>more> baz varchar(32), >>>>>more> woz varchar(32), >>>>>more> epoch bigint >>>>>more>); >>>>>operation successful (1.477ms) >>>>>sql>\q >>>>>$ mclient -u monetdb -s "copy into mduser.testdata from >>>>>'/tmp/testdata2.csv' using delimiters ',','\n','\'';" testdb >>>>>password: >>>>>2519931 affected rows >>>>> >>>>> >>>>>Thanks Allen! This gets me able to import. Now I'm wondering what >>>>>might be wrong with the constraint. Any insight there? >>>>> >>>>> >>>>> >>>>>On Sat, Nov 12, 2011 at 10:58 AM, Allen Zhang <azhang(a)pmci-usa.com> >>>>>wrote: >>>>>> What about just issue the following: >>>>>> >>>>>> mclient -s "copy into testdata from '/tmp/testdata2.csv' using >>>>>>delimiters >>>>>> ',','\n','\'';" >>>>>> >>>>>> >>>>>> Regards >>>>>> Allen >>>>>> >>>>>> >>>>>> On 11/12/11 10:48 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >>>>>> >>>>>>>Thanks Allen. Red-herring there. My first file had "SPY" in the >>>>>>>first column, and the second had "testdata1", so the difference >>>>>>>there >>>>>>>is just because of the contents. If I convert the original SPY file >>>>>>>and re-run, I get the same behavior. >>>>>>> >>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>',','\n','\'';" testdb - < /tmp/testdata2.csv >>>>>>>syntax error, unexpected STRING in: "'SPY'" >>>>>>> >>>>>>> >>>>>>>On Sat, Nov 12, 2011 at 10:43 AM, Allen Zhang <azhang(a)pmci-usa.com> >>>>>>>wrote: >>>>>>>> Hi Sean, >>>>>>>> >>>>>>>> You should inspect your data quality, it seems in the first 3 >>>>>>>>imports, >>>>>>>>it >>>>>>>> stopped at "spy", I suspect it might be some \t in that supposed >>>>>>>>to >>>>>>>>be >>>>>>>> string, and last import you put single-quote around string, then >>>>>>>>it >>>>>>>> stopped at "testvalue1", search your import file, look for that >>>>>>>> "testvalue1", the problem should around there, maybe an extra >>>>>>>>field >>>>>>>> because wrong replace by sed. >>>>>>>> >>>>>>>> Best Regards >>>>>>>> Allen >>>>>>>> >>>>>>>> On 11/12/11 10:26 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >>>>>>>> >>>>>>>>>Thanks Viktor, but no dice. I also tried converting from CSV to >>>>>>>>>TAB-delim to match what you were doing, but it's still not >>>>>>>>>working... >>>>>>>>> >>>>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>>>',','\n','';" testdb - < /tmp/testdata.csv >>>>>>>>>syntax error, unexpected IDENT in: "spy" >>>>>>>>> >>>>>>>>>$ sed 's/,/\'$'\t''/g' /tmp/testdata.csv > /tmp/testdata.dat >>>>>>>>> >>>>>>>>>$ mclient -s "copy into testdata from STDIN;" testdb - < >>>>>>>>>/tmp/testdata.dat >>>>>>>>>syntax error, unexpected IDENT in: "spy" >>>>>>>>> >>>>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>>>'\t','\n','';" testdb - < /tmp/testdata.dat >>>>>>>>>syntax error, unexpected IDENT in: "spy" >>>>>>>>> >>>>>>>>>I'm understanding the format of the documentation quite a bit >>>>>>>>>better >>>>>>>>>now after spending a few days with it, but I'm still not clear on >>>>>>>>>why >>>>>>>>>this insert is failing. >>>>>>>>>I went one step further and converted the input file to >>>>>>>>>single-quote >>>>>>>>>the string fields, but this fails as well: >>>>>>>>> >>>>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>>>',','\n','\'';" testdb - < /tmp/testdata2.csv >>>>>>>>>syntax error, unexpected STRING in: "'testvalue1'" >>>>>>>>> >>>>>>>>> >>>>>>>>>The problem has to be something trivial, but I have no idea what >>>>>>>>>it >>>>>>>>>is... >>>>>>>>> >>>>>>>>>On Sat, Nov 12, 2011 at 5:30 AM, Viktor Rosenfeld >>>>>>>>><listuser36(a)googlemail.com> wrote: >>>>>>>>>> Hi Sean, >>>>>>>>>> >>>>>>>>>> Sean McNamara wrote: >>>>>>>>>> >>>>>>>>>>> $ mclient -s "copy 2519931 records into testdata from STDIN >>>>>>>>>>>using >>>>>>>>>>> delimiters ',';" testdb - < /tmp/testdata.csv >>>>>>>>>>> syntax error, unexpected IDENT in: "testvalue1" >>>>>>>>>>> >>>>>>>>>>> where testvalue1 is one of the actual values in the FOO column >>>>>>>>>>> (actually the only value in this particular file.) >>>>>>>>>>> >>>>>>>>>>> I'm wondering if this is telling me it's expecting the field to >>>>>>>>>>>be >>>>>>>>>>> quoted, but that seems unlikely as it would make working with >>>>>>>>>>>CSVs >>>>>>>>>>> quite a bit painful. >>>>>>>>>>> >>>>>>>>>>> Can you guys kindly help me take another step forward. I >>>>>>>>>>>really >>>>>>>>>>> appreciate the guidance. >>>>>>>>>> >>>>>>>>>> I'm using >>>>>>>>>> >>>>>>>>>> COPY ... USING DELIMITERS '\t','\n',''; >>>>>>>>>> >>>>>>>>>> to bulk-load tab-delimited unquoted data. The '\t' is the >>>>>>>>>>default >>>>>>>>>>field >>>>>>>>>> separator, the '\n' is the default record separator. The final >>>>>>>>>>'' >>>>>>>>>>is >>>>>>>>>>the >>>>>>>>>> quote string -- empty in my case. >>>>>>>>>> >>>>>>>>>> See: >>>>>>>>>>http://www.monetdb.org/Documentation/Manuals/SQLreference/CopyInt >>>>>>>>>>o >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Viktor >>>>>>>>>> >>>>>>>>> >>>>>>>>>------------------------------------------------------------------ >>>>>>>>>-- >>>>>>>>>-- >>>>>>>>>-- >>>>>>>>>-- >>>>>>>>>---- >>>>>>>>>RSA(R) Conference 2012 >>>>>>>>>Save $700 by Nov 18 >>>>>>>>>Register now >>>>>>>>>http://p.sf.net/sfu/rsa-sfdev2dev1 >>>>>>>>>_______________________________________________ >>>>>>>>>MonetDB-users mailing list >>>>>>>>>MonetDB-users(a)lists.sourceforge.net >>>>>>>>>https://lists.sourceforge.net/lists/listinfo/monetdb-users >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >> >> >>

1 0

Re: [MonetDB-users] MonetDB Permission/COPY INTO Questions
by Allen Zhang 12 Nov '11

12 Nov '11

Sean, Use sql select to find out duplicates. Regards Allen On 11/12/11 12:01 PM, "Sean McNamara" <tank(a)thursday.org> wrote: >I tried that (as both the normal dbuser and as admin). Both fail the >same way but don't give an indicator of where it feels the constraint >is violated: > >$ mclient -u monetdb -s "alter table mduser.testdata add constraint >testdata_pkey primary key (foo,epoch);" testdb >password: >SQLException:assert:UPDATE: PRIMARY KEY constraint >'testdata.testdata_pkey' violated > > >On Sat, Nov 12, 2011 at 11:42 AM, Allen Zhang <azhang(a)pmci-usa.com> wrote: >> How about add constraint after you import? I think it might tell you >>which >> one had problem. >> >> Regards >> Allen >> >> On 11/12/11 11:19 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >> >>>Brilliant suggestion, and I think it exposed the problem. >>> >>>$ mclient -u monetdb -s "copy into mduser.testdata from >>>'/tmp/testdata2.csv' using delimiters ',','\n','\'';" testdb >>>password: >>>SQLException:importTable:value 'ONE_SECOND' from line 12734 field 3 >>>not inserted, expecting type varchar(8) >>>failed to import table >>> >>>It looks like the field I created is too small to handle some of the >>>values in the file. After expanding all the varchar fields to 32 and >>>re-running, I get a different error: >>> >>>$ mclient -u monetdb -s "copy into mduser.testdata from >>>'/tmp/testdata2.csv' using delimiters ',','\n','\'';" testdb >>>password: >>>SQLException:assert:INSERT INTO: PRIMARY KEY constraint >>>'testdata.testdata_pkey' violated >>> >>>That makes me think I have duplicate data. The PKEY is comprised of >>>fields #1 and #5, so I check for that but it looks fine: >>> >>>$ cat /tmp/testdata2.csv | cut -d, -f1,5 |uniq -d >>>$ >>> >>>However if I drop and re-add the table without the constraint, it works: >>> >>>sql>create table testdata ( >>>more> foo varchar(32), >>>more> bar varchar(32), >>>more> baz varchar(32), >>>more> woz varchar(32), >>>more> epoch bigint >>>more>); >>>operation successful (1.477ms) >>>sql>\q >>>$ mclient -u monetdb -s "copy into mduser.testdata from >>>'/tmp/testdata2.csv' using delimiters ',','\n','\'';" testdb >>>password: >>>2519931 affected rows >>> >>> >>>Thanks Allen! This gets me able to import. Now I'm wondering what >>>might be wrong with the constraint. Any insight there? >>> >>> >>> >>>On Sat, Nov 12, 2011 at 10:58 AM, Allen Zhang <azhang(a)pmci-usa.com> >>>wrote: >>>> What about just issue the following: >>>> >>>> mclient -s "copy into testdata from '/tmp/testdata2.csv' using >>>>delimiters >>>> ',','\n','\'';" >>>> >>>> >>>> Regards >>>> Allen >>>> >>>> >>>> On 11/12/11 10:48 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >>>> >>>>>Thanks Allen. Red-herring there. My first file had "SPY" in the >>>>>first column, and the second had "testdata1", so the difference there >>>>>is just because of the contents. If I convert the original SPY file >>>>>and re-run, I get the same behavior. >>>>> >>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>',','\n','\'';" testdb - < /tmp/testdata2.csv >>>>>syntax error, unexpected STRING in: "'SPY'" >>>>> >>>>> >>>>>On Sat, Nov 12, 2011 at 10:43 AM, Allen Zhang <azhang(a)pmci-usa.com> >>>>>wrote: >>>>>> Hi Sean, >>>>>> >>>>>> You should inspect your data quality, it seems in the first 3 >>>>>>imports, >>>>>>it >>>>>> stopped at "spy", I suspect it might be some \t in that supposed to >>>>>>be >>>>>> string, and last import you put single-quote around string, then it >>>>>> stopped at "testvalue1", search your import file, look for that >>>>>> "testvalue1", the problem should around there, maybe an extra field >>>>>> because wrong replace by sed. >>>>>> >>>>>> Best Regards >>>>>> Allen >>>>>> >>>>>> On 11/12/11 10:26 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >>>>>> >>>>>>>Thanks Viktor, but no dice. I also tried converting from CSV to >>>>>>>TAB-delim to match what you were doing, but it's still not >>>>>>>working... >>>>>>> >>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>',','\n','';" testdb - < /tmp/testdata.csv >>>>>>>syntax error, unexpected IDENT in: "spy" >>>>>>> >>>>>>>$ sed 's/,/\'$'\t''/g' /tmp/testdata.csv > /tmp/testdata.dat >>>>>>> >>>>>>>$ mclient -s "copy into testdata from STDIN;" testdb - < >>>>>>>/tmp/testdata.dat >>>>>>>syntax error, unexpected IDENT in: "spy" >>>>>>> >>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>'\t','\n','';" testdb - < /tmp/testdata.dat >>>>>>>syntax error, unexpected IDENT in: "spy" >>>>>>> >>>>>>>I'm understanding the format of the documentation quite a bit better >>>>>>>now after spending a few days with it, but I'm still not clear on >>>>>>>why >>>>>>>this insert is failing. >>>>>>>I went one step further and converted the input file to single-quote >>>>>>>the string fields, but this fails as well: >>>>>>> >>>>>>>$ mclient -s "copy into testdata from STDIN using delimiters >>>>>>>',','\n','\'';" testdb - < /tmp/testdata2.csv >>>>>>>syntax error, unexpected STRING in: "'testvalue1'" >>>>>>> >>>>>>> >>>>>>>The problem has to be something trivial, but I have no idea what it >>>>>>>is... >>>>>>> >>>>>>>On Sat, Nov 12, 2011 at 5:30 AM, Viktor Rosenfeld >>>>>>><listuser36(a)googlemail.com> wrote: >>>>>>>> Hi Sean, >>>>>>>> >>>>>>>> Sean McNamara wrote: >>>>>>>> >>>>>>>>> $ mclient -s "copy 2519931 records into testdata from STDIN using >>>>>>>>> delimiters ',';" testdb - < /tmp/testdata.csv >>>>>>>>> syntax error, unexpected IDENT in: "testvalue1" >>>>>>>>> >>>>>>>>> where testvalue1 is one of the actual values in the FOO column >>>>>>>>> (actually the only value in this particular file.) >>>>>>>>> >>>>>>>>> I'm wondering if this is telling me it's expecting the field to >>>>>>>>>be >>>>>>>>> quoted, but that seems unlikely as it would make working with >>>>>>>>>CSVs >>>>>>>>> quite a bit painful. >>>>>>>>> >>>>>>>>> Can you guys kindly help me take another step forward. I really >>>>>>>>> appreciate the guidance. >>>>>>>> >>>>>>>> I'm using >>>>>>>> >>>>>>>> COPY ... USING DELIMITERS '\t','\n',''; >>>>>>>> >>>>>>>> to bulk-load tab-delimited unquoted data. The '\t' is the default >>>>>>>>field >>>>>>>> separator, the '\n' is the default record separator. The final '' >>>>>>>>is >>>>>>>>the >>>>>>>> quote string -- empty in my case. >>>>>>>> >>>>>>>> See: >>>>>>>>http://www.monetdb.org/Documentation/Manuals/SQLreference/CopyInto >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Viktor >>>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>>-- >>>>>>>-- >>>>>>>---- >>>>>>>RSA(R) Conference 2012 >>>>>>>Save $700 by Nov 18 >>>>>>>Register now >>>>>>>http://p.sf.net/sfu/rsa-sfdev2dev1 >>>>>>>_______________________________________________ >>>>>>>MonetDB-users mailing list >>>>>>>MonetDB-users(a)lists.sourceforge.net >>>>>>>https://lists.sourceforge.net/lists/listinfo/monetdb-users >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >> >> >>

1 0

Re: [MonetDB-users] MonetDB Permission/COPY INTO Questions
by Allen Zhang 12 Nov '11

12 Nov '11

What about just issue the following: mclient -s "copy into testdata from '/tmp/testdata2.csv' using delimiters ',','\n','\'';" Regards Allen On 11/12/11 10:48 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >Thanks Allen. Red-herring there. My first file had "SPY" in the >first column, and the second had "testdata1", so the difference there >is just because of the contents. If I convert the original SPY file >and re-run, I get the same behavior. > >$ mclient -s "copy into testdata from STDIN using delimiters >',','\n','\'';" testdb - < /tmp/testdata2.csv >syntax error, unexpected STRING in: "'SPY'" > > >On Sat, Nov 12, 2011 at 10:43 AM, Allen Zhang <azhang(a)pmci-usa.com> wrote: >> Hi Sean, >> >> You should inspect your data quality, it seems in the first 3 imports, >>it >> stopped at "spy", I suspect it might be some \t in that supposed to be >> string, and last import you put single-quote around string, then it >> stopped at "testvalue1", search your import file, look for that >> "testvalue1", the problem should around there, maybe an extra field >> because wrong replace by sed. >> >> Best Regards >> Allen >> >> On 11/12/11 10:26 AM, "Sean McNamara" <tank(a)thursday.org> wrote: >> >>>Thanks Viktor, but no dice. I also tried converting from CSV to >>>TAB-delim to match what you were doing, but it's still not working... >>> >>>$ mclient -s "copy into testdata from STDIN using delimiters >>>',','\n','';" testdb - < /tmp/testdata.csv >>>syntax error, unexpected IDENT in: "spy" >>> >>>$ sed 's/,/\'$'\t''/g' /tmp/testdata.csv > /tmp/testdata.dat >>> >>>$ mclient -s "copy into testdata from STDIN;" testdb - < >>>/tmp/testdata.dat >>>syntax error, unexpected IDENT in: "spy" >>> >>>$ mclient -s "copy into testdata from STDIN using delimiters >>>'\t','\n','';" testdb - < /tmp/testdata.dat >>>syntax error, unexpected IDENT in: "spy" >>> >>>I'm understanding the format of the documentation quite a bit better >>>now after spending a few days with it, but I'm still not clear on why >>>this insert is failing. >>>I went one step further and converted the input file to single-quote >>>the string fields, but this fails as well: >>> >>>$ mclient -s "copy into testdata from STDIN using delimiters >>>',','\n','\'';" testdb - < /tmp/testdata2.csv >>>syntax error, unexpected STRING in: "'testvalue1'" >>> >>> >>>The problem has to be something trivial, but I have no idea what it >>>is... >>> >>>On Sat, Nov 12, 2011 at 5:30 AM, Viktor Rosenfeld >>><listuser36(a)googlemail.com> wrote: >>>> Hi Sean, >>>> >>>> Sean McNamara wrote: >>>> >>>>> $ mclient -s "copy 2519931 records into testdata from STDIN using >>>>> delimiters ',';" testdb - < /tmp/testdata.csv >>>>> syntax error, unexpected IDENT in: "testvalue1" >>>>> >>>>> where testvalue1 is one of the actual values in the FOO column >>>>> (actually the only value in this particular file.) >>>>> >>>>> I'm wondering if this is telling me it's expecting the field to be >>>>> quoted, but that seems unlikely as it would make working with CSVs >>>>> quite a bit painful. >>>>> >>>>> Can you guys kindly help me take another step forward. I really >>>>> appreciate the guidance. >>>> >>>> I'm using >>>> >>>> COPY ... USING DELIMITERS '\t','\n',''; >>>> >>>> to bulk-load tab-delimited unquoted data. The '\t' is the default >>>>field >>>> separator, the '\n' is the default record separator. The final '' is >>>>the >>>> quote string -- empty in my case. >>>> >>>> See: >>>>http://www.monetdb.org/Documentation/Manuals/SQLreference/CopyInto >>>> >>>> Cheers, >>>> Viktor >>>> >>> >>>------------------------------------------------------------------------ >>>-- >>>---- >>>RSA(R) Conference 2012 >>>Save $700 by Nov 18 >>>Register now >>>http://p.sf.net/sfu/rsa-sfdev2dev1 >>>_______________________________________________ >>>MonetDB-users mailing list >>>MonetDB-users(a)lists.sourceforge.net >>>https://lists.sourceforge.net/lists/listinfo/monetdb-users >> >> >>

1 0

Re: [MonetDB-users] Extended Fetch ODBC Error
by Jeremy O'Sullivan 10 Nov '11

10 Nov '11

Hi Sjoerd, Would it be possible to test your thesis ? The OBI guys reckon that for odbc 2 connections, they use the function SQLExtendedFetch. This has bee replaced by SQLFetchScroll in odbc 3.5. Not sure if that helps, I know little or nothing about odbc, other than many BI tools only use this interface and not jdbc. Regards, Jeremy It is unclear from this message what exactly fails. My guess (since I saw something like that recently) is that Oracle's OBI EE tries to set the statement option (or attribute) SQL_ATTR_CONCURRENCY to a value that is not supported. We only support setting the value to SQL_CONCUR_READ_ONLY. Attempting to set the attribute to any other value will return the above error (not implemented). Perhaps what we should do instead here is accept the value and return a message (in the ODBC way) that instead of the value that the application asked for we set it to SQL_CONCUR_READ_ONLY. Other possible statement attributes that could return the feature not implemented error are SQL_ATTR_ASYNC_ENABLE, SQL_ATTR_CURSOR_SENSITIVITY, SQL_ATTR_ENABLE_AUTO_IPD, SQL_ATTR_FETCH_BOOKMARK_PTR, SQL_ATTR_KEYSET_SIZE, SQL_ATTR_MAX_LENGTH, SQL_ATTR_MAX_ROWS, SQL_ATTR_QUERY_TIMEOUT, SQL_ATTR_ROW_NUMBER, SQL_ATTR_SIMULATE_CURSOR, and SQL_ATTR_USE_BOOKMARKS. On 9 Nov 2011, at 12:26, Jeremy O'Sullivan wrote: > Hello, > > When trying to connect to MonetDB using Oracle's OBI EE, I get the error below. Has anyone seen this before ? > Regards, > Jeremy > > [MonetDB][ODBC Driver 11.5.7]Optional feature not implemented. > > [nQSError: 16006] Setting the statement option for extended fetch failed. > >

2 1

[MonetDB-users] Extended Fetch ODBC Error
by Jeremy O'Sullivan 09 Nov '11

09 Nov '11

Hello, When trying to connect to MonetDB using Oracle's OBI EE, I get the error below. Has anyone seen this before ? Regards, Jeremy [MonetDB][ODBC Driver 11.5.7]Optional feature not implemented. [nQSError: 16006] Setting the statement option for extended fetch failed.

2 1

[MonetDB-users] Tableau Alias error
by Jeremy O'Sullivan 09 Nov '11

09 Nov '11

Hello, When connecting from Tableau, the following syntax generates and error on the command line, is this correct ? (The :w triggers the error.) Regards, Jeremy select count(context_key) as "context_key:w" from context_dim;

2 1

[MonetDB-users] How to check wether a record exists or not in stored procedure
by dengxk 08 Nov '11

08 Nov '11

I am writing a stored procedure to insert an record into my table. If the record exists, just update it. it is easier to acheive in microsoft sql server with the sql: if exists( select column from table where condition ...). As monetdb doesn't support select statment and cursor in stored procedure, is there other walkround?

1 0

Re: [MonetDB-users] confirm e5d538cae9aefced13c1ef02f7f06476bbdd59e0
by charles 07 Nov '11

07 Nov '11

Hello Everyone, I have been starting to work with MonetDB for use with our large production database. It is a 'data warehouse' of sorts. Few comments: a) the php client was not included with built/binary distro b) user and password was not specified in http://www.monetdb.org/Documentation/monetdbd at "Next thing, making a connection to the database using mclient requires the -d argument to specify which database to connect to:" c) 'last crash' field is scary to sys admins/users :) I am going to investigate more, just wanted to pop my head in and say hello. As you can guess, our main dev platform is php, so I will need to be using the php extensions. Cheers, Charles

3 7

[MonetDB-users] alias error
by Jeremy O'Sullivan 07 Nov '11

07 Nov '11

Hello, The following syntax generates and error on the command line, is this correct ? The :w triggers the error. Regards, Jeremy select count(context_key) as "context_key:w" from context_dim;

1 0

[MonetDB-users] ODBC-Extended Fetch
by Jeremy O'Sullivan 07 Nov '11

07 Nov '11

Hello, When trying to connect to MonetDB using Oracle's OBI EE, I get the error below. Has anyone seen this before ? Regards, Jeremy [nQSError: 16001] ODBC error state: HYC00 code: 0 message: [MonetDB][ODBC Driver 11.5.7]Optional feature not implemented. [nQSError: 16006] Setting the statement option for extended fetch failed.

1 0

[MonetDB-users] Errors after executing "set history=true" for saving the query history
by Roberto Fernandez 05 Nov '11

05 Nov '11

Hi, I was trying to save my history based on this documentation http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History And this is what is happening: $ mclient -dtest-db -umonetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011-SP2) Database: MonetDB v11.5.7 (Aug2011-SP2), 'test-db' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.623ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; +------+ | L5 | +======+ | 2785 | +------+ 1 tuple (1.6s) sql>set history=true; sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.232ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _23:any := user.keepquery(_24:wrd, _25:str, _26:lng, _27:lng) TypeException:user.s0_1[32]:'user.keepcall' undefined in: _73:any := user.keepcall(_24:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select * from queryHistory; TypeException:user.s5_1[1]:'user.keepquery' undefined in: _43:any := user.keepquery(_44:wrd, _45:str, _46:lng, _47:lng) TypeException:user.s5_1[48]:'user.keepcall' undefined in: _111:any := user.keepcall(_44:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select 1; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _12:any := user.keepquery(_13:wrd, _14:str, _15:lng, _16:lng) TypeException:user.s0_1[24]:'user.keepcall' undefined in: _55:any := user.keepcall(_13:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>set history=false; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _2:any := user.keepquery(_3:wrd, _4:str, _5:lng, _6:lng) TypeException:user.s0_1[19]:'user.keepcall' undefined in: _41:any := user.keepcall(_3:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql> SERVER INFORMATION: It is a Amazon Web Services Linux distribution (based on CentOS) $ mserver5 --version MonetDB 5 server v11.5.7 "Aug2011-SP2" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 7.3GiB available memory, 2 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0e 6 Sep 2011 (compiled with OpenSSL 1.0.0e-fips 6 Sep 2011) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: ec2-user@domU-12-31-39-0E-8D-F2 (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 I will really appreciate any help provided. Thanks, Roberto F -- View this message in context: http://old.nabble.com/Errors-after-executing-%22set-history%3Dtrue%22-for-s… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Errors saving query history
by Roberto Fernandez 05 Nov '11

05 Nov '11

Hi, I was trying to save my history based on this documentation http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History And this is what is happening: $ mclient -dtest-db -umonetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011-SP2) Database: MonetDB v11.5.7 (Aug2011-SP2), 'test-db' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.623ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; +------+ | L5 | +======+ | 2785 | +------+ 1 tuple (1.6s) sql>set history=true; sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.232ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _23:any := user.keepquery(_24:wrd, _25:str, _26:lng, _27:lng) TypeException:user.s0_1[32]:'user.keepcall' undefined in: _73:any := user.keepcall(_24:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select * from queryHistory; TypeException:user.s5_1[1]:'user.keepquery' undefined in: _43:any := user.keepquery(_44:wrd, _45:str, _46:lng, _47:lng) TypeException:user.s5_1[48]:'user.keepcall' undefined in: _111:any := user.keepcall(_44:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select 1; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _12:any := user.keepquery(_13:wrd, _14:str, _15:lng, _16:lng) TypeException:user.s0_1[24]:'user.keepcall' undefined in: _55:any := user.keepcall(_13:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>set history=false; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _2:any := user.keepquery(_3:wrd, _4:str, _5:lng, _6:lng) TypeException:user.s0_1[19]:'user.keepcall' undefined in: _41:any := user.keepcall(_3:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql> SERVER INFORMATION: It is a Amazon Web Services Linux distribution (based on CentOS) $ mserver5 --version MonetDB 5 server v11.5.7 "Aug2011-SP2" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 7.3GiB available memory, 2 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0e 6 Sep 2011 (compiled with OpenSSL 1.0.0e-fips 6 Sep 2011) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: ec2-user@domU-12-31-39-0E-8D-F2 (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 I will really appreciate any help provided. Thanks, Roberto F -- View this message in context: http://old.nabble.com/Errors-saving-query-history-tp32786364p32786364.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Errors saving query history
by Roberto Fernandez 05 Nov '11

05 Nov '11

Hi, I was trying to save my history based on this documentation http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History And this is what is happening: $ mclient -dtest-db -umonetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011-SP2) Database: MonetDB v11.5.7 (Aug2011-SP2), 'test-db' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.623ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; +------+ | L5 | +======+ | 2785 | +------+ 1 tuple (1.6s) sql>set history=true; sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.232ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _23:any := user.keepquery(_24:wrd, _25:str, _26:lng, _27:lng) TypeException:user.s0_1[32]:'user.keepcall' undefined in: _73:any := user.keepcall(_24:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select * from queryHistory; TypeException:user.s5_1[1]:'user.keepquery' undefined in: _43:any := user.keepquery(_44:wrd, _45:str, _46:lng, _47:lng) TypeException:user.s5_1[48]:'user.keepcall' undefined in: _111:any := user.keepcall(_44:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select 1; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _12:any := user.keepquery(_13:wrd, _14:str, _15:lng, _16:lng) TypeException:user.s0_1[24]:'user.keepcall' undefined in: _55:any := user.keepcall(_13:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>set history=false; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _2:any := user.keepquery(_3:wrd, _4:str, _5:lng, _6:lng) TypeException:user.s0_1[19]:'user.keepcall' undefined in: _41:any := user.keepcall(_3:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql> SERVER INFORMATION: It is a Amazon Web Services Linux distribution (based on CentOS) $ mserver5 --version MonetDB 5 server v11.5.7 "Aug2011-SP2" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 7.3GiB available memory, 2 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0e 6 Sep 2011 (compiled with OpenSSL 1.0.0e-fips 6 Sep 2011) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: ec2-user@domU-12-31-39-0E-8D-F2 (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 I will really appreciate any help provided. Thanks, Roberto F -- View this message in context: http://old.nabble.com/Errors-saving-query-history-tp32786363p32786363.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Errors saving query history
by Roberto Fernandez 05 Nov '11

05 Nov '11

Hi, I was trying to save my history based on this documentation http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History And this is what is happening: $ mclient -dtest-db -umonetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011-SP2) Database: MonetDB v11.5.7 (Aug2011-SP2), 'test-db' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.623ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; +------+ | L5 | +======+ | 2785 | +------+ 1 tuple (1.6s) sql>set history=true; sql>select 1; +--------------+ | single_value | +==============+ | 1 | +--------------+ 1 tuple (0.232ms) sql>select count(*) from narvq.narvq where zip5 = '81657'; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _23:any := user.keepquery(_24:wrd, _25:str, _26:lng, _27:lng) TypeException:user.s0_1[32]:'user.keepcall' undefined in: _73:any := user.keepcall(_24:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select * from queryHistory; TypeException:user.s5_1[1]:'user.keepquery' undefined in: _43:any := user.keepquery(_44:wrd, _45:str, _46:lng, _47:lng) TypeException:user.s5_1[48]:'user.keepcall' undefined in: _111:any := user.keepcall(_44:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>select 1; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _12:any := user.keepquery(_13:wrd, _14:str, _15:lng, _16:lng) TypeException:user.s0_1[24]:'user.keepcall' undefined in: _55:any := user.keepcall(_13:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql>set history=false; TypeException:user.s0_1[1]:'user.keepquery' undefined in: _2:any := user.keepquery(_3:wrd, _4:str, _5:lng, _6:lng) TypeException:user.s0_1[19]:'user.keepcall' undefined in: _41:any := user.keepcall(_3:wrd, ctime:timestamp, args:str, xtime:lng, rtime:lng, foot:lng, memory:lng, tuples:wrd, inblock:lng, oublock:lng) SQLException:SQLengine:Program contains errors sql> SERVER INFORMATION: It is a Amazon Web Services Linux distribution (based on CentOS) $ mserver5 --version MonetDB 5 server v11.5.7 "Aug2011-SP2" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 7.3GiB available memory, 2 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0e 6 Sep 2011 (compiled with OpenSSL 1.0.0e-fips 6 Sep 2011) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: ec2-user@domU-12-31-39-0E-8D-F2 (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 I will really appreciate any help provided. Thanks, Roberto F -- View this message in context: http://old.nabble.com/Errors-saving-query-history-tp32786355p32786355.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] compile error when build MonetDB 11.5.7 on CentOS 5.5
by Allen Zhang 04 Nov '11

04 Nov '11

Hi, When compile on CentOS 5.5, following error pop up, can any one tell me which version of libcom_err.so is needed for this MonetDB version? Thanks Allen make[5]: Entering directory `/usr/src/MonetDB-11.5.7/common/stream' /bin/sh ../../libtool --tag=CC --mode=link gcc -DLIBSTREAM -g -O2 -D_REENTRANT -version-info 2:2:0 -o libstream.la -rpath /usr/local/MonetDBAug2011-SP2/lib libstream_la-stream.lo -lbz2 -L/usr/kerberos/lib64 -lssl -lcrypto -ldl -lz -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz -pthread libtool: link: gcc -shared -fPIC -DPIC .libs/libstream_la-stream.o -lbz2 -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz -O2 -pthread -pthread -Wl,-soname -Wl,libstream.so.2 -o .libs/libstream.so.2.0.2 /usr/bin/ld: skipping incompatible /usr/lib/libcom_err.so when searching for -lcom_err /usr/bin/ld: skipping incompatible /usr/lib/libcom_err.a when searching for -lcom_err /usr/bin/ld: cannot find -lcom_err collect2: ld returned 1 exit status

2 2

Re: [MonetDB-users] compile error when build MonetDB 11.5.7 on CentOS 5.5
by Stefan Manegold 04 Nov '11

04 Nov '11

Hi Allen, from your log, I guess your CentS is 64 bit, right? Is the installed libcom_err also 64 bit or 32 bit? The linker says the sitcom_err it finds is "incompatible" ... Stefan ----- Reply message ----- From: "Allen Zhang" <azhang(a)pmci-usa.com> To: <monetdb-users(a)lists.sourceforge.net> Subject: [MonetDB-users] compile error when build MonetDB 11.5.7 on CentOS 5.5 Date: Fri, Nov 4, 2011 01:07 Hi, When compile on CentOS 5.5, following error pop up, can any one tell me which version of libcom_err.so is needed for this MonetDB version? Thanks Allen make[5]: Entering directory `/usr/src/MonetDB-11.5.7/common/stream' /bin/sh ../../libtool --tag=CC --mode=link gcc -DLIBSTREAM -g -O2 -D_REENTRANT -version-info 2:2:0 -o libstream.la -rpath /usr/local/MonetDBAug2011-SP2/lib libstream_la-stream.lo -lbz2 -L/usr/kerberos/lib64 -lssl -lcrypto -ldl -lz -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz -pthread libtool: link: gcc -shared -fPIC -DPIC .libs/libstream_la-stream.o -lbz2 -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz -O2 -pthread -pthread -Wl,-soname -Wl,libstream.so.2 -o .libs/libstream.so.2.0.2 /usr/bin/ld: skipping incompatible /usr/lib/libcom_err.so when searching for -lcom_err /usr/bin/ld: skipping incompatible /usr/lib/libcom_err.a when searching for -lcom_err /usr/bin/ld: cannot find -lcom_err collect2: ld returned 1 exit status

1 0

[MonetDB-users] I want to join this group:)
by Tony Bai 02 Nov '11

02 Nov '11

Could you add me as a member of Monetdb please? Thanks, Tony

1 0

[MonetDB-users] Temporal coalescing query error
by Alexander S 29 Oct '11

29 Oct '11

Hello, I'm attempting to use monetdb for some temporal queries. Some queries require the output to be coalesced. For example, with schema emp(empno int, ts date, te date) the tuples (00001, '1/1/2001', '2/1/2001'), (00001, '2/1/2001', '3/1/2001') are output as (00001, '1/1/2001', '3/1/2001'). My coalescing query works in both MS SQL Server and DB2 but MonetDB gives me the error "SELECT: no such column 'm.ts'". 1) A leaner alternative query form using window clauses (ROWS UNBOUNDED PRECEDING) doesn't appear to be supported in MonetDB. So I'm stuck. Any help would be appreciated. Query: WITH Emps(empno, ts, te) AS (SELECT empno, ts, te FROM emp) SELECT DISTINCT F.empno, F.ts, L.te FROM Emps AS F, Emps AS L WHERE F.ts < L.te AND F.empno = L.empno AND NOT EXISTS (SELECT * FROM Emps AS M WHERE M.empno = F.empno AND F.ts < M.ts AND M.ts < L.te AND NOT EXISTS (SELECT * FROM Emps AS T1 WHERE T1.empno = F.empno AND T1.ts < M.ts AND M.ts <= T1.te) ) AND NOT EXISTS (SELECT * FROM Emps AS T2 WHERE T2.empno = F.empno AND ((T2.ts < F.ts AND F.ts <= T2.te) OR (T2.ts <= L.te AND L.te < T2.te)) ); Thanks, Alex -- View this message in context: http://old.nabble.com/Temporal-coalescing-query-error-tp32742108p32742108.h… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] eEnd of Stream
by nat lu 28 Oct '11

28 Oct '11

I'm using Monet 11.5 on Ubuntu 11.04. Ive created a database and am attempting to run a series of queries against it with a test script. This is in attempt to use Monet with Apache Jena. I had earlier mailed about a problem with temp tables, but have dodged that by using the monetdb account for now. The first few succeed, but then I get the following error: java.sql.SQLException: Read from localhost:50000: End of stream reached (mserver still alive?) Why does this happen, under what circumstances does it arise ? Is there anything I can do to avoid it ? This is the query that it fails on: SELECT R_1.lex AS V_1_lex, R_1.datatype AS V_1_datatype, R_1.lang AS V_1_lang, R_1.type AS V_1_type, R_2.lex AS V_2_lex, R_2.datatype AS V_2_datatype, R_2.lang AS V_2_lang, R_2.type AS V_2_type FROM ( SELECT T_1.s AS X_1 FROM Triples AS T_1 WHERE ( T_1.p = 2004134117598721274 AND T_1.o = 435905340492217258 ) ) AS T_1 LEFT OUTER JOIN Triples AS T_2 ON ( T_2.p = 3816485599920428794 AND T_1.X_1 = T_2.s ) LEFT OUTER JOIN Nodes AS R_1 ON ( T_1.X_1 = R_1.hash ) LEFT OUTER JOIN Nodes AS R_2 ON ( T_2.o = R_2.hash )

3 7

[MonetDB-users] IsNumeric() clause in MonetDB
by Osmin Castillo 26 Oct '11

26 Oct '11

Hi everyone: How can I test if a VARCHAR field has numeric data?, at SQLServer I use IsNumeric() clause, what can i use in MonetDB? thanks in advance Osmin

1 0

[MonetDB-users] Announcement: New Aug2011-SP2 Bugfix release of MonetDB suite
by MonetDB Release Engineer 26 Oct '11

26 Oct '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Aug 2011-SP2 bugfix release Client Package * ODBC: Implemented a workaround in SQLTables for bug 2908. * Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability. * Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901 * mclient now checks the result of encoding conversions using the iconv library. * Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv). * Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois. * Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output Merovingian * Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped. * monetdbd get status now also reports the version of the running monetdbd * Improved response time of 'monetdb start' when the database fails to start. * Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database. * To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client. Bug Fixes * 2346: mkey.bulk_rotate_xor_hash MAL exception * 2712: un-rolled-back savepoints cause crash upon scenario exit * 2774: COPY INTO of double (~1.0e-310) cannot be imported. * 2890: Slow execution of COUNT in combination with DISTINCT * 2894: No consistent behavior of implicit cast of integer to double * 2897: Mapi.pm: short reads not handled when reading blocks from server * 2898: order by for multiple (two) columns fails for large tables when second ordering is 'desc': "mat.refine_reverse' undefined" * 2900: rand(int) should not return NULL * 2901: Make keyword arguments in Python DB API connect() function more standard * 2904: TypeException after calling UDF in other than sys schema (when it is initialised) * 2906: "ALTER TABLE mytable SET READ ONLY" results in server segfault * 2908: SQL query produces error with MAL statements * 2909: query result is not displayed when history = true

1 0

[MonetDB-users] Newbie just joined this group :)
by Tony Bai 26 Oct '11

26 Oct '11

Hi guys, I just joined this amazing group, May I asking a stupid question about Monetdb if I haven't run out my 'ask stupid question' budget? My question are as following: * Regarding to the work cluster, Does Monetdb support smoothly expand while I have to add new server to the work cluster in necessary? How to add a new server to a work cluster? * Does Monetdb supply any analytical functions? Do we have any documents or materials related to what kind of analytical functions can be used? BTW: Any advices or suggestions regarding building data mart with Monetdb would be highly appreciated! Thanks in advance, Tony

1 0

[MonetDB-users] Examples of usage
by Jeremy O'Sullivan 25 Oct '11

25 Oct '11

Hello, I am presently evaluating MonetDB's performance for a client using big data sets. We are very happy with the impressive results. Something that is harder to evaluate is how wide spread is MonetDB's usage. There are some references on the home page, but as it says in the introductory passage, most usage applications are anonymous. Would anyone like to share examples of applications that are using MonetDB ? Regards, Jeremy

2 1

[MonetDB-users] Network unreachable errors in log
by Jeremy O'Sullivan 21 Oct '11

21 Oct '11

Hello, The log file, merovingian.log, contains many occurrences of the below: 2011-10-21 17:20:47 ERR discovery[4680]: error while sending broadcast message: Network is unreachable The database seems to be performing ok. What causes this issue, would this affect the performance of the database ? Regards, Jeremy

2 1

Re: [MonetDB-users] ODBC connection issues
by Jeremy O'Sullivan 20 Oct '11

20 Oct '11

Hi, I have downloaded the patch for the ODBC driver on windows 2003 32 bit (after uninstalling the existing odbc driver) and tried to view the schema from excel, through the excel odbc connection. I get the exact same error as before. Regards, Jeremy

3 2

Re: [MonetDB-users] ODBC connection issues
by Jeremy OSullivan 20 Oct '11

20 Oct '11

Hi, The dialogue comes back with the same error message. I tried it in access as well. I am not able to see the schema, not to mind query it. The log may be different, I have attached the output. I asked my colleague to download and try on his machine, he got the same issue. Regards, Jeremy MSACCESS f68-f54 ENTER SQLAllocConnect HENV 09E41788 HDBC * 00139820 MSACCESS f68-f54 EXIT SQLAllocConnect with return code 0 (SQL_SUCCESS) HENV 09E41788 HDBC * 0x00139820 ( 0x09e426b8) MSACCESS f68-f54 ENTER SQLSetConnectOption HDBC 09E426B8 SQLINTEGER 103 <SQL_LOGIN_TIMEOUT> SQLPOINTER 0x00000014 MSACCESS f68-f54 EXIT SQLSetConnectOption with return code 0 (SQL_SUCCESS) HDBC 09E426B8 SQLINTEGER 103 <SQL_LOGIN_TIMEOUT> SQLPOINTER 0x00000014 (BADMEM) MSACCESS f68-f54 ENTER SQLSetConnectAttrW SQLHDBC 09E426B8 SQLINTEGER 30002 <unknown> SQLPOINTER [Unknown attribute 30002] SQLINTEGER -3 MSACCESS f68-f54 EXIT SQLSetConnectAttrW with return code 0 (SQL_SUCCESS) SQLHDBC 09E426B8 SQLINTEGER 30002 <unknown> SQLPOINTER [Unknown attribute 30002] SQLINTEGER -3 MSACCESS f68-f54 ENTER SQLDriverConnectW HDBC 09E426B8 HWND 0018019E WCHAR * 0x4BF77CD4 [ -3] "******\ 0" SWORD -3 WCHAR * 0x4BF77CD4 SWORD 2 SWORD * 0x00000000 UWORD 1 <SQL_DRIVER_COMPLETE> MSACCESS f68-f54 EXIT SQLDriverConnectW with return code 1 (SQL_SUCCESS_WITH_INFO) HDBC 09E426B8 HWND 0018019E WCHAR * 0x4BF77CD4 [ -3] "******\ 0" SWORD -3 WCHAR * 0x4BF77CD4 SWORD 2 SWORD * 0x00000000 UWORD 1 <SQL_DRIVER_COMPLETE> DIAG [IM006] [Microsoft][ODBC Driver Manager] Driver's SQLSetConnectAttr failed (0) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 9 <SQL_ODBC_API_CONFORMANCE> PTR 0x0013981C SWORD 2 SWORD * 0x00139824 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 9 <SQL_ODBC_API_CONFORMANCE> PTR 0x0013981C (1) SWORD 2 SWORD * 0x00139824 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 6 <SQL_DRIVER_NAME> PTR 0x00139828 SWORD 200 SWORD * 0x00139824 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 6 <SQL_DRIVER_NAME> PTR 0x00139828 [ 28] "MonetDBODBClib" SWORD 200 SWORD * 0x00139824 (28) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 46 <SQL_TXN_CAPABLE> PTR 0x001393DC SWORD 2 SWORD * 0x001393A0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 46 <SQL_TXN_CAPABLE> PTR 0x001393DC (2) SWORD 2 SWORD * 0x001393A0 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 23 <SQL_CURSOR_COMMIT_BEHAVIOR> PTR 0x039C6138 SWORD 2 SWORD * 0x001393A0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 23 <SQL_CURSOR_COMMIT_BEHAVIOR> PTR 0x039C6138 (0) SWORD 2 SWORD * 0x001393A0 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 24 <SQL_CURSOR_ROLLBACK_BEHAVIOR> PTR 0x039C613A SWORD 2 SWORD * 0x001393A0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 24 <SQL_CURSOR_ROLLBACK_BEHAVIOR> PTR 0x039C613A (0) SWORD 2 SWORD * 0x001393A0 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 1 <SQL_ACTIVE_STATEMENTS> PTR 0x001393E4 SWORD 2 SWORD * 0x001393B8 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 1 <SQL_ACTIVE_STATEMENTS> PTR 0x001393E4 (0) SWORD 2 SWORD * 0x001393B8 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 17 <SQL_DBMS_NAME> PTR 0x001397E4 SWORD 200 SWORD * 0x001397E0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 17 <SQL_DBMS_NAME> PTR 0x001397E4 [ 14] "MonetDB" SWORD 200 SWORD * 0x001397E0 (14) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 0 <SQL_ACTIVE_CONNECTIONS> PTR 0x001398E0 SWORD 2 SWORD * 0x001398E8 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 0 <SQL_ACTIVE_CONNECTIONS> PTR 0x001398E0 (0) SWORD 2 SWORD * 0x001398E8 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 25 <SQL_DATA_SOURCE_READ_ONLY> PTR 0x001398EC SWORD 20 SWORD * 0x001398E8 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 25 <SQL_DATA_SOURCE_READ_ONLY> PTR 0x001398EC [ 2] "N" SWORD 20 SWORD * 0x001398E8 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 46 <SQL_TXN_CAPABLE> PTR 0x001398E4 SWORD 2 SWORD * 0x001398C0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 46 <SQL_TXN_CAPABLE> PTR 0x001398E4 (2) SWORD 2 SWORD * 0x001398C0 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 23 <SQL_CURSOR_COMMIT_BEHAVIOR> PTR 0x001398DC SWORD 2 SWORD * 0x001398C0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 23 <SQL_CURSOR_COMMIT_BEHAVIOR> PTR 0x001398DC (0) SWORD 2 SWORD * 0x001398C0 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 24 <SQL_CURSOR_ROLLBACK_BEHAVIOR> PTR 0x001398D8 SWORD 2 SWORD * 0x001398C0 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 24 <SQL_CURSOR_ROLLBACK_BEHAVIOR> PTR 0x001398D8 (0) SWORD 2 SWORD * 0x001398C0 (2) MSACCESS f68-f54 ENTER SQLGetInfoW HDBC 09E426B8 UWORD 29 <SQL_IDENTIFIER_QUOTE_CHAR> PTR 0x001398EC SWORD 20 SWORD * 0x001398E8 MSACCESS f68-f54 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 09E426B8 UWORD 29 <SQL_IDENTIFIER_QUOTE_CHAR> PTR 0x001398EC [ 2] """ SWORD 20 SWORD * 0x001398E8 (2) MSACCESS f68-f54 ENTER SQLAllocStmt HDBC 09E426B8 HSTMT * 00137C38 MSACCESS f68-f54 EXIT SQLAllocStmt with return code 0 (SQL_SUCCESS) HDBC 09E426B8 HSTMT * 0x00137C38 ( 0x09e41a28) MSACCESS f68-f54 ENTER SQLGetStmtOption HSTMT 09E41A28 UWORD 0 PTR 0x00137B40 MSACCESS f68-f54 EXIT SQLGetStmtOption with return code -1 (SQL_ERROR) HSTMT 09E41A28 UWORD 0 PTR 0x00137B40 DIAG [HYC00] [MonetDB][ODBC Driver 11.5.6]Optional feature not implemented (0) MSACCESS f68-f54 ENTER SQLErrorW HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137AEC (NYI) SDWORD * 0x00137AB8 WCHAR * 0x0374A000 SWORD 4095 SWORD * 0x00137AAC MSACCESS f68-f54 EXIT SQLErrorW with return code 0 (SQL_SUCCESS) HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137AEC (NYI) SDWORD * 0x00137AB8 (0) WCHAR * 0x0374A000 [ 61] "[MonetDB][ODBC Driver 11.5.6]Optional feature not implemented" SWORD 4095 SWORD * 0x00137AAC (61) MSACCESS f68-f54 ENTER SQLErrorW HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137AEC (NYI) SDWORD * 0x00137AB8 WCHAR * 0x0374A08C SWORD 4025 SWORD * 0x00137AAC MSACCESS f68-f54 EXIT SQLErrorW with return code 100 (SQL_NO_DATA_FOUND) HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137AEC (NYI) SDWORD * 0x00137AB8 WCHAR * 0x0374A08C SWORD 4025 SWORD * 0x00137AAC MSACCESS f68-f54 ENTER SQLTablesW HSTMT 09E41A28 WCHAR * 0x00000000 SWORD 0 WCHAR * 0x00000000 SWORD 0 WCHAR * 0x00000000 SWORD 0 WCHAR * 0x09E42640 [ -3] "'TABLE','VIEW','SYSTEM TABLE','ALIAS','SYNONYM'\ 0" SWORD -3 MSACCESS f68-f54 EXIT SQLTablesW with return code -1 (SQL_ERROR) HSTMT 09E41A28 WCHAR * 0x00000000 SWORD 0 WCHAR * 0x00000000 SWORD 0 WCHAR * 0x00000000 SWORD 0 WCHAR * 0x09E42640 [ -3] "'TABLE','VIEW','SYSTEM TABLE','ALIAS','SYNONYM'\ 0" SWORD -3 DIAG [HY000] [MonetDB][ODBC Driver 11.5.6]TypeException:user.s2_2[34]:'algebra.markT' undefined in: _74:any := algebra.markT(_73:bit, _69:oid) TypeException:user.s2_2[56]:'algebra.kunion' undefined in: _102:any := algebra.kunion(_101:bat[:oid,:oid], _74:any) TypeException:user.s2_2[57]:'algebra.markT' undefined in: _103:any := algebra.markT(_102:any, _68:oid) TypeException:user.s2_2[62]:'algebra.markT' undefined in: _110:any := algebra.markT(_109:bit, _104:oid) TypeException:user.s2_2[63]:'algebra.kunion' undefined in: _111:any := algebra.kunion(_110:any, _103:any) TypeException:user.s2_2[80]:'algebra.semijoin' undefined in: _133:any := algebra.semijoin(_132:bat[:oid,:void], _111:any) TypeException:user.s2_2[81]:'algebra.markT' undefined in: _134:any := algebra.markT(_133:any, _67:oid) TypeException:user.s2_2[82]:'bat.reverse' undefined in: _135:any := bat.reverse(_134:any) TypeException:user.s2_2[83]:'algebra.leftjoin' undefined in: _136:any := algebra.leftjoin(_135:any, _64:bat[:oid,:int]) TypeException:user.s2_2[84]:'bat.reverse' undefined in: _137:any := bat.reverse(_136:any) TypeException:user.s2_2[96]:'algebra.join' undefined in: _156:any := algebra.join(_155:bat[:oid,:int], _137:any) TypeException:user.s2_2[97]:'algebra.markT' undefined in: _157:any := algebra.markT(_156:any, _58:oid) TypeException:user.s2_2[98]:'bat.reverse' undefined in: _158:any := bat.reverse(_157:any) TypeException:user.s2_2[99]:'algebra.leftjoin' undefined in: _159:any := algebra.leftjoin(_158:any, _57:bat[:oid,:str]) TypeException:user.s2_2[119]:'algebra.markT' undefined in: _185:any := algebra.markT(_184:bit, _180:oid) TypeException:user.s2_2[195]:'algebra.kunion' undefined in: _273:any := algebra.kunion(_272:bat[:oid,:oid], _185:any) TypeException:user.s2_2[196]:'algebra.markT' undefined in: _274:any := algebra.markT(_273:any, _179:oid) TypeException:user.s2_2[201]:'algebra.markT' undefined in: _281:any := algebra.markT(_280:bit, _275:oid) TypeException:user.s2_2[202]:'algebra.kunion' undefined (0) MSACCESS f68-f54 ENTER SQLErrorW HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137B00 (NYI) SDWORD * 0x00137ACC WCHAR * 0x0374A000 SWORD 4095 SWORD * 0x00137AC0 MSACCESS f68-f54 EXIT SQLErrorW with return code 0 (SQL_SUCCESS) HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137B00 (NYI) SDWORD * 0x00137ACC (0) WCHAR * 0x0374A000 [ 4095] "[MonetDB][ODBC Driver 11.5.6]TypeException:user.s2_2[34]:'algebra.markT' undefined in: _74:any := algebra.markT(_73:bit, _69:oid)\ aTypeException:user.s2_2[56]:'algebra.kunion' undefined in: _102:any := algebra.kunion(_101:bat[:oid,:oid], _74:any)\ aTypeException:user.s2_2[57]:'algebra.markT' undefined in: _103:any := algebra.markT(_102:any, _68:oid)\ aTypeException:user.s2_2[62]:'algebra.markT' undefined in: _110:any := algebra.markT(_109:bit, _104:oid)\ aTypeException:user.s2_2[63]:'algebra.kunion' undefined in:\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0..." SWORD 4095 SWORD * 0x00137AC0 (8055) MSACCESS f68-f54 ENTER SQLErrorW HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137B00 (NYI) SDWORD * 0x00137ACC WCHAR * 0x0374A410 SWORD 3575 SWORD * 0x00137AC0 MSACCESS f68-f54 EXIT SQLErrorW with return code 100 (SQL_NO_DATA_FOUND) HENV 09E41788 HDBC 09E426B8 HSTMT 09E41A28 WCHAR * 0x00137B00 (NYI) SDWORD * 0x00137ACC WCHAR * 0x0374A410 SWORD 3575 SWORD * 0x00137AC0 MSACCESS f68-f54 ENTER SQLFreeStmt HSTMT 09E41A28 UWORD 1 <SQL_DROP> MSACCESS f68-f54 EXIT SQLFreeStmt with return code 0 (SQL_SUCCESS) HSTMT 09E41A28 UWORD 1 <SQL_DROP> MSACCESS f68-f54 ENTER SQLDisconnect HDBC 09E426B8 MSACCESS f68-f54 EXIT SQLDisconnect with return code 0 (SQL_SUCCESS) HDBC 09E426B8 MSACCESS f68-f54 ENTER SQLFreeConnect HDBC 09E426B8 MSACCESS f68-f54 EXIT SQLFreeConnect with return code 0 (SQL_SUCCESS) HDBC 09E426B8

1 0

[MonetDB-users] Problems with the query log
by Viktor Rosenfeld 20 Oct '11

20 Oct '11

Hi, I get errors, when I try to use the query history feature as described here: http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/History After history is set to true, I get the following error message every time I execute a query: invalid/unknown response from server, ignoring output ? = 0:0 profiler.setFootprintFlag(); ? = 0:0 profiler.setMemoryFlag(); ? = 0:0 inblock := profiler.getDiskReads(); ? = 0:0 oublock := profiler.getDiskWrites(); ? = 0:0 xtime := alarm.usec(); (remaining output omitted, use \fraw to examine in detail) This only stops if I set history back to false. I'm using the latest MonetDB 11.5.3 on OS X 10.6. Below is an example session. Cheers, Viktor -----8< sample session >8----- $ mclient -d demo -u monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Aug2011-SP1) Database: MonetDB v11.5.3, 'demo' Type \q to quit, \? for a list of available commands auto commit mode: on sql>set history=true; sql>select 1; invalid/unknown response from server, ignoring output ? = 0:0 profiler.setFootprintFlag(); ? = 0:0 profiler.setMemoryFlag(); ? = 0:0 inblock := profiler.getDiskReads(); ? = 0:0 oublock := profiler.getDiskWrites(); ? = 0:0 xtime := alarm.usec(); (remaining output omitted, use \fraw to examine in detail) sql>select * from queryHistory; invalid/unknown response from server, ignoring output ? = 0:0 profiler.setFootprintFlag(); ? = 0:0 profiler.setMemoryFlag(); ? = 0:0 inblock := profiler.getDiskReads(); ? = 0:0 oublock := profiler.getDiskWrites(); ? = 0:0 xtime := alarm.usec(); (remaining output omitted, use \fraw to examine in detail) sql>set history=false; invalid/unknown response from server, ignoring output ? = 0:0 profiler.setFootprintFlag(); ? = 0:0 profiler.setMemoryFlag(); ? = 0:0 inblock := profiler.getDiskReads(); ? = 0:0 oublock := profiler.getDiskWrites(); ? = 0:0 xtime := alarm.usec(); (remaining output omitted, use \fraw to examine in detail) sql>select * from queryHistory; +---------+----------------------------+---------+-----------------------------+-------+----------+ | id | defined | name | query | parse | optimize | +=========+============================+=========+=============================+=======+==========+ | 1035769 | 2011-10-18 15:51:36.000000 | monetdb | select 1; | 236 | 208 | | 1035885 | 2011-10-18 15:51:47.000000 | monetdb | select * from queryhistory; | 300 | 860 | | 1036047 | 2011-10-18 15:51:50.000000 | monetdb | set history=false; | 130 | 466 | +---------+----------------------------+---------+-----------------------------+-------+----------+ 3 tuples (1.613ms) sql>

2 2

Re: [MonetDB-users] ODBC connection issues
by Jeremy O'Sullivan 20 Oct '11

20 Oct '11

Hi, How do I get the patch, there is no attachment on the bug. Is there a new .msi ? Regards, Jeremy

2 1

[MonetDB-users] Problem with YEAR, MONTH, DAY functions
by Roula Avg 20 Oct '11

20 Oct '11

Hello, sorry to ask again but i have problem with YEAR, MONTH and DAY functions. According to the references YEAR for example can take as argument a date or a timestamp or a month_interval value. So if i run the query: sql> select year(current_timestamp); or sql> select year( '2002-03-21') it should return 2011 and 2002. Instead of this, i take: *syntax error,unexpected YEAR in: "select year". *What am i doing wrong? Thank you very much in advance

3 2

[MonetDB-users] MonetDB Temporary tables
by ultanocarroll 19 Oct '11

19 Oct '11

Hi, I'm trying to create an adapter for Apache Jena[1] to MonetDB[2]. Part of this process involves creating temporary tables in Monet. During the Jena db setup process these tables are created with SQL statements. For most RDBMS these take the form of CREATE GLOBAL TEMPORARY... However, with MonetDB (Aug 11 release on Ubuntu 11.04) I have been unable to use this statement form, or CREATE LOCAL, or CREATE TEMPORARY TABLE. Monet always returns "insufficient privileges", a la : sql>create global temporary table t (col1 int); CREATE TABLE: insufficient privileges for user 'jena' in schema 'tmp' sql>create local temporary table t (col1 int); CREATE TABLE: insufficient privileges for user 'jena' in schema 'tmp' sql>create temporary table t (col1 int); CREATE TABLE: insufficient privileges for user 'jena' in schema 'tmp' (Jena's table is a little more sophisticated but raises the same error ! :-) I have created the user for this operation, an accompanying schema with authorization and reset the users schema to this, as per Getting Started [3]. The only operation I can seem to get to work in place of temporary tables is to use CREATE TABLE. However, when I do this, Jena unit tests fail when it tries to re-create, for a second group of tests, these temp tables, with the exception java.sql.SQLException: CREATE TABLE: name 'nnodetriples' already in use Jena then tries to drop the tables (using execSilent) and create anew, but at this point gets another exception. Caused by: java.sql.SQLException: current transaction is aborted (please ROLLBACK) Any suggestions on this ? 1. It seems to me the root of the problem is that the global temp tables cannot be created, or that the 'already in use' exception is escalating to a transaction abort, incorrectly. 2. Do I need (can I) grant the user privileges to the tmp schema ? [1] http://incubator.apache.org/jena/ [2] https://issues.apache.org/jira/browse/JENA-134 [3] http://www.monetdb.org/Downloads/Tutorial

2 1

Re: [MonetDB-users] ODBC connection issues
by Jeremy O'Sullivan 19 Oct '11

19 Oct '11

Hello, Regarding the ODBC error, I tried connecting to MonetDB using excel and got the same error as I got from OBI EE, see log below: Regards, Jeremy MSQRY32 a18-890 ENTER SQLAllocEnv HENV * 3009DDD4 MSQRY32 a18-890 EXIT SQLAllocEnv with return code 0 (SQL_SUCCESS) HENV * 0x3009DDD4 ( 0x00aa1788) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 31 <unknown> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1C4 WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 31 <unknown> WCHAR * 0x00AA1830 [ 11] "dBASE" SWORD 34 SWORD * 0x0012F1C4 (11) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 (51) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1C4 WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 11] "Excel" SWORD 34 SWORD * 0x0012F1C4 (11) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 (54) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1C4 WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 18] "MS Access" SWORD 34 SWORD * 0x0012F1C4 (18) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 (40) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1C4 WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 100 (SQL_NO_DATA_FOUND) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1C4 WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1B6 MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 32 <unknown> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 32 <unknown> WCHAR * 0x00AA1830 [ 12] "Analyt" SWORD 34 SWORD * 0x0012F1BC (12) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (16) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 20] "IncomeAndE" SWORD 34 SWORD * 0x0012F1BC (20) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (40) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 11] "Forec" SWORD 34 SWORD * 0x0012F1BC (11) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (40) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 17] "IncomeEx" SWORD 34 SWORD * 0x0012F1BC (17) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (40) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 10] "DFB F" SWORD 34 SWORD * 0x0012F1BC (10) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (54) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 10] "DFB F" SWORD 34 SWORD * 0x0012F1BC (10) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (54) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 16] "DFB_KPI_" SWORD 34 SWORD * 0x0012F1BC (16) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (54) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 5] "Wa" SWORD 34 SWORD * 0x0012F1BC (5) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (54) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 12] "Profit" SWORD 34 SWORD * 0x0012F1BC (12) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (30) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 0 (SQL_SUCCESS) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 [ 7] "Mon" SWORD 34 SWORD * 0x0012F1BC (7) WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE (19) MSQRY32 a18-890 ENTER SQLDataSourcesW HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 EXIT SQLDataSourcesW with return code 100 (SQL_NO_DATA_FOUND) HENV 00AA1788 UWORD 1 <SQL_FETCH_NEXT> WCHAR * 0x00AA1830 SWORD 34 SWORD * 0x0012F1BC WCHAR * 0x00000000 SWORD 0 SWORD * 0x0012F1AE MSQRY32 a18-890 ENTER SQLAllocConnect HENV 00AA1788 HDBC * 0012E698 MSQRY32 a18-890 EXIT SQLAllocConnect with return code 0 (SQL_SUCCESS) HENV 00AA1788 HDBC * 0x0012E698 ( 0x00aa1ba8) MSQRY32 a18-890 ENTER SQLSetConnectOption HDBC 00AA1BA8 SQLINTEGER 103 <SQL_LOGIN_TIMEOUT> SQLPOINTER 0x0000002D MSQRY32 a18-890 EXIT SQLSetConnectOption with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 SQLINTEGER 103 <SQL_LOGIN_TIMEOUT> SQLPOINTER 0x0000002D (BADMEM) MSQRY32 a18-890 ENTER SQLDriverConnectW HDBC 00AA1BA8 HWND 0002019A WCHAR * 0x4BF77CD4 [ -3] "******\ 0" SWORD -3 WCHAR * 0x4BF77CD4 SWORD 2 SWORD * 0x00000000 UWORD 1 <SQL_DRIVER_COMPLETE> MSQRY32 a18-890 EXIT SQLDriverConnectW with return code 1 (SQL_SUCCESS_WITH_INFO) HDBC 00AA1BA8 HWND 0002019A WCHAR * 0x4BF77CD4 [ -3] "******\ 0" SWORD -3 WCHAR * 0x4BF77CD4 SWORD 2 SWORD * 0x00000000 UWORD 1 <SQL_DRIVER_COMPLETE> DIAG [IM006] [Microsoft][ODBC Driver Manager] Driver's SQLSetConnectAttr failed (0) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 2 <SQL_DATA_SOURCE_NAME> PTR 0x00AA1CD8 SWORD 2048 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 2 <SQL_DATA_SOURCE_NAME> PTR 0x00AA1CD8 [ 14] "MonetDB" SWORD 2048 SWORD * 0x0012E694 (14) MSQRY32 a18-890 ENTER SQLAllocStmt HDBC 00AA1BA8 HSTMT * 0012E68C MSQRY32 a18-890 EXIT SQLAllocStmt with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 HSTMT * 0x0012E68C ( 0x00aa25c8) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 1 <SQL_ACTIVE_STATEMENTS> PTR 0x0012E640 SWORD 2 SWORD * 0x0012E596 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 1 <SQL_ACTIVE_STATEMENTS> PTR 0x0012E640 (0) SWORD 2 SWORD * 0x0012E596 (2) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 25 <SQL_DATA_SOURCE_READ_ONLY> PTR 0x00AA3038 SWORD 2048 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 25 <SQL_DATA_SOURCE_READ_ONLY> PTR 0x00AA3038 [ 2] "N" SWORD 2048 SWORD * 0x0012E694 (2) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 6 <SQL_DRIVER_NAME> PTR 0x00AA3038 SWORD 2048 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 6 <SQL_DRIVER_NAME> PTR 0x00AA3038 [ 28] "MonetDBODBClib" SWORD 2048 SWORD * 0x0012E694 (28) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 14 <SQL_SEARCH_PATTERN_ESCAPE> PTR 0x00AA3038 SWORD 2048 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 14 <SQL_SEARCH_PATTERN_ESCAPE> PTR 0x00AA3038 SWORD 2048 SWORD * 0x0012E694 (0) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 74 <SQL_CORRELATION_NAME> PTR 0x0012E680 SWORD 2 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 74 <SQL_CORRELATION_NAME> PTR 0x0012E680 (2) SWORD 2 SWORD * 0x0012E694 (2) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 75 <SQL_NON_NULLABLE_COLUMNS> PTR 0x0012E680 SWORD 2 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 75 <SQL_NON_NULLABLE_COLUMNS> PTR 0x0012E680 (1) SWORD 2 SWORD * 0x0012E694 (2) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 41 <SQL_CATALOG_NAME_SEPARATOR> PTR 0x00AA3038 SWORD 2048 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 41 <SQL_CATALOG_NAME_SEPARATOR> PTR 0x00AA3038 SWORD 2048 SWORD * 0x0012E694 (0) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 84 <SQL_FILE_USAGE> PTR 0x0012E654 SWORD 2 SWORD * 0x0012E694 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 84 <SQL_FILE_USAGE> PTR 0x0012E654 (0) SWORD 2 SWORD * 0x0012E694 (2) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 14 <SQL_SEARCH_PATTERN_ESCAPE> PTR 0x00AA1D08 SWORD 130 SWORD * 0x0012DFE0 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 14 <SQL_SEARCH_PATTERN_ESCAPE> PTR 0x00AA1D08 SWORD 130 SWORD * 0x0012DFE0 (0) MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 42 <SQL_CATALOG_TERM> PTR 0x00AA1D08 SWORD 130 SWORD * 0x0012E000 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 42 <SQL_CATALOG_TERM> PTR 0x00AA1D08 SWORD 130 SWORD * 0x0012E000 (0) MSQRY32 a18-890 ENTER SQLFreeStmt HSTMT 00AA25C8 UWORD 0 <SQL_CLOSE> MSQRY32 a18-890 EXIT SQLFreeStmt with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 UWORD 0 <SQL_CLOSE> MSQRY32 a18-890 ENTER SQLFreeStmt HSTMT 00AA25C8 UWORD 2 <SQL_UNBIND> MSQRY32 a18-890 EXIT SQLFreeStmt with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 UWORD 2 <SQL_UNBIND> MSQRY32 a18-890 ENTER SQLGetInfoW HDBC 00AA1BA8 UWORD 32 <SQL_MAX_SCHEMA_NAME_LEN> PTR 0x0012DFF0 SWORD 2 SWORD * 0x0012DFE0 MSQRY32 a18-890 EXIT SQLGetInfoW with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 UWORD 32 <SQL_MAX_SCHEMA_NAME_LEN> PTR 0x0012DFF0 (255) SWORD 2 SWORD * 0x0012DFE0 (2) MSQRY32 a18-890 ENTER SQLTablesW HSTMT 00AA25C8 WCHAR * 0x00AA2590 SWORD 0 WCHAR * 0x00AA2550 [ -3] "%\ 0" SWORD -3 WCHAR * 0x00AA1DA0 SWORD 0 WCHAR * 0x00AA1DB8 SWORD 0 MSQRY32 a18-890 EXIT SQLTablesW with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 WCHAR * 0x00AA2590 SWORD 0 WCHAR * 0x00AA2550 [ -3] "%\ 0" SWORD -3 WCHAR * 0x00AA1DA0 SWORD 0 WCHAR * 0x00AA1DB8 SWORD 0 MSQRY32 a18-890 ENTER SQLBindCol HSTMT 00AA25C8 UWORD 2 SWORD 1 <SQL_C_CHAR> PTR 0x0012E01C SQLLEN 65 SQLLEN * 0x0012DFFC MSQRY32 a18-890 EXIT SQLBindCol with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 UWORD 2 SWORD 1 <SQL_C_CHAR> PTR 0x0012E01C SQLLEN 65 SQLLEN * 0x0012DFFC (11148488) MSQRY32 a18-890 ENTER SQLFetch HSTMT 00AA25C8 MSQRY32 a18-890 EXIT SQLFetch with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 MSQRY32 a18-890 ENTER SQLFetch HSTMT 00AA25C8 MSQRY32 a18-890 EXIT SQLFetch with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 MSQRY32 a18-890 ENTER SQLFetch HSTMT 00AA25C8 MSQRY32 a18-890 EXIT SQLFetch with return code 100 (SQL_NO_DATA_FOUND) HSTMT 00AA25C8 MSQRY32 a18-890 ENTER SQLFreeStmt HSTMT 00AA25C8 UWORD 0 <SQL_CLOSE> MSQRY32 a18-890 EXIT SQLFreeStmt with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 UWORD 0 <SQL_CLOSE> MSQRY32 a18-890 ENTER SQLFreeStmt HSTMT 00AA25C8 UWORD 2 <SQL_UNBIND> MSQRY32 a18-890 EXIT SQLFreeStmt with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 UWORD 2 <SQL_UNBIND> MSQRY32 a18-890 ENTER SQLTablesW HSTMT 00AA25C8 WCHAR * 0x00000000 [ -3] <empty string> SWORD -3 WCHAR * 0x00000000 [ -3] <empty string> SWORD -3 WCHAR * 0x00000000 [ -3] <empty string> SWORD -3 WCHAR * 0x00AA1DA0 [ 24] "'TABLE','VIEW','SYNONYM'" SWORD 24 MSQRY32 a18-890 EXIT SQLTablesW with return code -1 (SQL_ERROR) HSTMT 00AA25C8 WCHAR * 0x00000000 [ -3] <empty string> SWORD -3 WCHAR * 0x00000000 [ -3] <empty string> SWORD -3 WCHAR * 0x00000000 [ -3] <empty string> SWORD -3 WCHAR * 0x00AA1DA0 [ 24] "'TABLE','VIEW','SYNONYM'" SWORD 24 DIAG [HY000] [MonetDB][ODBC Driver 11.5.3]TypeException:user.s3_2[50]:'algebra.markT' undefined in: _96:any := algebra.markT(_95:bit, _90:oid) TypeException:user.s3_2[51]:'algebra.kunion' undefined in: _97:any := algebra.kunion(_96:any, _89:bat[:oid,:oid]) TypeException:user.s3_2[52]:'algebra.markT' undefined in: _98:any := algebra.markT(_97:any, _82:oid) TypeException:user.s3_2[53]:'algebra.semijoin' undefined in: _99:any := algebra.semijoin(_98:any, _81:bat[:oid,:void]) TypeException:user.s3_2[73]:'algebra.kunion' undefined in: _127:any := algebra.kunion(_126:bat[:oid,:oid], _99:any) TypeException:user.s3_2[74]:'algebra.markT' undefined in: _128:any := algebra.markT(_127:any, _66:oid) TypeException:user.s3_2[75]:'bat.reverse' undefined in: _129:any := bat.reverse(_128:any) TypeException:user.s3_2[76]:'algebra.leftjoin' undefined in: _130:any := algebra.leftjoin(_129:any, _63:bat[:oid,:int]) TypeException:user.s3_2[77]:'bat.reverse' undefined in: _131:any := bat.reverse(_130:any) TypeException:user.s3_2[89]:'algebra.join' undefined in: _147:any := algebra.join(_146:bat[:oid,:int], _131:any) TypeException:user.s3_2[90]:'algebra.markT' undefined in: _148:any := algebra.markT(_147:any, _57:oid) TypeException:user.s3_2[91]:'bat.reverse' undefined in: _149:any := bat.reverse(_148:any) TypeException:user.s3_2[92]:'algebra.leftjoin' undefined in: _150:any := algebra.leftjoin(_149:any, _56:bat[:oid,:str]) TypeException:user.s3_2[140]:'algebra.markT' undefined in: _213:any := algebra.markT(_212:bit, _208:oid) TypeException:user.s3_2[141]:'algebra.kunion' undefined in: _214:any := algebra.kunion(_213:any, _207:bat[:oid,:oid]) TypeException:user.s3_2[142]:'algebra.markT' undefined in: _215:any := algebra.markT(_214:any, _194:oid) TypeException:user.s3_2[143]:'algebra.semijoin' undefined in: _216:any := algebra.semijoin(_215:any, _193:bat[:oid,:void]) TypeException:user.s3_2[173]:'algebra.kunion' undefined in: _257:any := algebra.kunion(_256:bat[:oid,:oid], _216:any) TypeException:user.s3 (0) MSQRY32 a18-890 ENTER SQLErrorW HENV 00AA1788 HDBC 00AA1BA8 HSTMT 00AA25C8 WCHAR * 0x0012E4E0 (NYI) SDWORD * 0x0012E564 WCHAR * 0x0012E0E0 SWORD 512 SWORD * 0x0012E56C MSQRY32 a18-890 EXIT SQLErrorW with return code 0 (SQL_SUCCESS) HENV 00AA1788 HDBC 00AA1BA8 HSTMT 00AA25C8 WCHAR * 0x0012E4E0 (NYI) SDWORD * 0x0012E564 (0) WCHAR * 0x0012E0E0 [ 512] "[MonetDB][ODBC Driver 11.5.3]TypeException:user.s3_2[50]:'algebra.markT' undefined in: _96:any := algebra.markT(_95:bit, _90:oid)\ aTypeException:user.s3_2[51]:'algebra.kunion' undefined in: _97:any := algebra.kunion(_96:any, _89:bat[:oid,:oid])\ aTypeException:user.s3_2[52]:'algebra.markT' undefined in: _98:any := algebra.markT(_97:any, _82:oid)\ aTypeException:user.s3_2[53]:'algebra.semijoin' undefined in: _99:any := algebra.semijoin(_98:any, _81:bat[:oid,:void])\ aTypeException:user.s3_2[73]:'algebra.kunion' un\ 0" SWORD 512 SWORD * 0x0012E56C (7860) MSQRY32 a18-890 ENTER SQLErrorW HENV 00AA1788 HDBC 00AA1BA8 HSTMT 00AA25C8 WCHAR * 0x0012E4E0 (NYI) SDWORD * 0x0012E564 WCHAR * 0x0012E0E0 SWORD 512 SWORD * 0x0012E56C MSQRY32 a18-890 EXIT SQLErrorW with return code 100 (SQL_NO_DATA_FOUND) HENV 00AA1788 HDBC 00AA1BA8 HSTMT 00AA25C8 WCHAR * 0x0012E4E0 (NYI) SDWORD * 0x0012E564 WCHAR * 0x0012E0E0 SWORD 512 SWORD * 0x0012E56C MSQRY32 a18-890 ENTER SQLFreeStmt HSTMT 00AA25C8 UWORD 1 <SQL_DROP> MSQRY32 a18-890 EXIT SQLFreeStmt with return code 0 (SQL_SUCCESS) HSTMT 00AA25C8 UWORD 1 <SQL_DROP> MSQRY32 a18-890 ENTER SQLDisconnect HDBC 00AA1BA8 MSQRY32 a18-890 EXIT SQLDisconnect with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 MSQRY32 a18-890 ENTER SQLFreeConnect HDBC 00AA1BA8 MSQRY32 a18-890 EXIT SQLFreeConnect with return code 0 (SQL_SUCCESS) HDBC 00AA1BA8 MSQRY32 a18-890 ENTER SQLFreeEnv HENV 00AA1788 MSQRY32 a18-890 EXIT SQLFreeEnv with return code 0 (SQL_SUCCESS) HENV 00AA1788

2 2

[MonetDB-users] Bulk-importing boolean columns
by Viktor Rosenfeld 19 Oct '11

19 Oct '11

Hi, I need to bulk-import data that was bulk-exported from PostgreSQL. I have only limited control over the raw data. I'm using the following command: COPY INTO table FROM '/path/to/filename' USING DELIMITERS '\t','\n',''; This generally works. The problem is boolean data which is exported by PostgreSQL using the values "t" and "f" (unquoted). MonetDB expects "true" and "false" here. My current work-around is opening the file in Vim and substituting the values (%s/\tt\t/\ttrue\t/g and %s/\tf\t/\tfalse\t/g). Is there a way to make MonetDB accept the original PostgreSQL data? Cheers, Viktor

4 6

[MonetDB-users] Date and time Functions
by Roula Avg 19 Oct '11

19 Oct '11

Hello, i am looking for a manual about all the date and time functions supported in Monetdb. In the official site there is no such manual, can anyone help me please? Thank you very much

3 3

[MonetDB-users] test - ignore <eom>
by nat lu 19 Oct '11

19 Oct '11

1 0

[MonetDB-users] ODBC connection issues
by Jeremy OSullivan 19 Oct '11

19 Oct '11

Hello, I have issues connecting to monetdb using the supplied odbc driver via Oracle's OBI EE admin tool. I am on Windows 2008, 32 bit. When I connect, I get the attached error. I have also attached the odbc configuration. Any help would be greatly appreciated ! Regards, Jeremy

4 4

[MonetDB-users] Insert query into second database
by Seleen 17 Oct '11

17 Oct '11

Hello, I have two databases "mydb" and "test". I want to get a sub-set of values from 'mydb' into 'test'. My code: "insert into test.result select sum(chargfeeprepaid) from mydb.data where callingpartyno = 200 union select sum(chargefeeprepaid) from mydb.sms where callingpartyno = 300;" This works fine in MySQL...but in Monetdb i get the error: INSERT INTO: no such scheme 'test'. Where did i go wrong and what is the correct syntax to do this in monetdb? Greetings Seleen -- View this message in context: http://old.nabble.com/Insert-query-into-second-database-tp32667556p32667556… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] JDBC: Problem with PreparedStatement parameter in Buffer
by George Garbis 10 Oct '11

10 Oct '11

Hi all, I am trying to use PreparedStatement to get a buffer of a geometry invoking the commands bellow PreparedStatement st = con.prepareStatement("CREATE TABLE geom (g > geometry)"); > st.execute(); > st = con.prepareStatement("INSERT INTO geom VALUES(GeomFromText('POINT(1 > 1)', 2100))"); > st.execute(); > st = con.prepareStatement("SELECT Buffer(t.g, ?) AS buffer, t.g AS point > FROM geom AS t"); // This line throws an SQLException > st.setFloat(1, (float)1.0); > ResultSet results = st.executeQuery(); > However the fifth line throws an SQLException with this message: > types geometry(0,0) and double(53,0) are not equal > Replacing the question mark with a number (e.g. 1.0) the program executes properly. Why is this exception thrown? Is something wrong in my code? Thanks in advance, George Garbis

2 2

[MonetDB-users] Monetdb installation on Debian(lenny) doesn't work
by Benji07 05 Oct '11

05 Oct '11

System information : Debian GNU/Linux 5.0.8 (lenny) Hardware Platform : x86_64 GNU/Linux(amd64) I followed the next steps to install Monetdb : 1 - Add next lines in /etc/apt/sources.list : deb http://dev.monetdb.org/downloads/deb/ lenny monetdb deb-src http://dev.monetdb.org/downloads/deb/ lenny monetdb 2 - Install the MonetDB GPG public key 3 - Install the MonetDB packages sudo apt-get update But I get the next message : Failed to fetch http://dev.monetdb.org/downloads/deb/dists/lenny/Release Unable to find expected entry monetdb/binary-amd64/Packages in Meta-index file (malformed Release file?) The file 'Packages' seem to be missing (among others) in this directory, I think the error is related to that. http://dev.monetdb.org/downloads/deb/dists/lenny/monetdb/binary-amd64/ For install of Debian Squeeze, I didn't have any problem. Thanks for you help. -- View this message in context: http://old.nabble.com/Monetdb-installation-on-Debian%28lenny%29-doesn%27t-w… Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] Trigger definitions - ambiguous identifier error
by Benji07 04 Oct '11

04 Oct '11

I've the following table on my database : \d sys.subscriber_data CREATE TABLE "sys"."subscriber_data" ( "subs_id" BIGINT, "imsi" VARCHAR(16), "mccmnc" INTEGER, "msisdn" VARCHAR(16), "imei" VARCHAR(16), "time_idx_print" BIGINT, "time_idx_lastmodified" BIGINT, "time_idx_purge" BIGINT, "merge_subs_id" BIGINT ); I want to create a specific trigger to delete old data in table before insert new data. create trigger oninsert_subscriber before insert on subscriber_data referencing NEW ROW AS new_ident for each row begin atomic delete from subscriber_data where subscriber_data.subs_id = new_ident.subs_id; end; When I executed the statement, I get this error message : SELECT: identifier 'subs_id' ambiguous I don't understand how to use alias in this case to reference old and new data. I wrote the statement according to the documentation: http://www.monetdb.org/Documentation/SQLreference/Triggers However, I found this line in feature overview (unsupported) page T211 Basic trigger capability http://www.monetdb.org/Documentation/Manuals/SQLreference/Features/unsuppor… Does it mean triggers are not yet entirely supported by Monetdb? Thanks for your help. -- View this message in context: http://old.nabble.com/Trigger-definitions---ambiguous-identifier-error-tp32… Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] SQL-MAL correlation
by Liang Bing 30 Sep '11

30 Sep '11

Hallo MonetDB-Team, I have been trying to understand MonetDB-MAL better. I have prepared a table called TEST1. sql>SELECT * FROM TEST1; +-----+ | een | +=====+ +-----+ 0 tuples (1.452ms) In order to see the internal process I use the command EXPLAIN. sql>EXPLAIN SELECT * FROM TEST1; +---------------------------------------------------------------------------------------------------+ | mal | +===================================================================================================+ | function user.s3_2{autoCommit=true}():void; | | barrier _32 := language.dataflow(); | | _2 := sql.mvc(); | | _12:bat[:oid,:str] := sql.bind(_2,"sys","test1","een",0); | | _10:bat[:oid,:str] := sql.bind(_2,"sys","test1","een",2); | | _14 := algebra.kdifference(_12,_10); | | _15 := algebra.kunion(_14,_10); | | _9:bat[:oid,:str] := sql.bind(_2,"sys","test1","een",1); | | _16 := algebra.kunion(_15,_9); | | _5:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","test1",1); | | _8 := bat.reverse(_5); | | _17 := algebra.kdifference(_16,_8); | | exit _32; | | _18 := sql.resultSet(1,1,_17); | | sql.rsColumn(_18,"sys.test1","een","char",2,0,_17); | | _22 := io.stdout(); | | sql.exportResult(_22,_18); | | end s3_2; | +---------------------------------------------------------------------------------------------------+ 18 tuples (1.240ms) Then I have tried to run the command from the lower level. Basically it is almost a copy of the above result. mal>function user.abc{autoCommit=true}():void; mal>barrier _32 := language.dataflow(); mal>_2 := sql.mvc(); mal>_12:bat[:oid,:str] := sql.bind(_2,"sys","test1","een",0); mal>_10:bat[:oid,:str] := sql.bind(_2,"sys","test1","een",2); mal>_14 := algebra.kdifference(_12,_10); mal>_15 := algebra.kunion(_14,_10); mal>_9:bat[:oid,:str] := sql.bind(_2,"sys","test1","een",1); mal>_16 := algebra.kunion(_15,_9); mal>_5:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","test1",1); mal>_8 := bat.reverse(_5); mal>_17 := algebra.kdifference(_16,_8); mal>exit _32; mal>_18 := sql.resultSet(1,1,_17); mal>sql.rsColumn(_18,"sys.test1","een","char",2,0,_17); mal>_22 := io.stdout(); mal>sql.exportResult(_22,_18); mal>end abc; MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = end abc; ERROR = !TypeException:user.abc[13]:'X17' may not be used before being initialized !TypeException:user.abc[14]:'X17' may not be used before being initialized I expected to get a result like this: +-----+ | een | +=====+ +-----+ But instead, I got an error message. Is there probably anyone who could tell me what the error message means and what I need to change in the MAL-syntax? Another question: Maybe you could give a hint where I can find good references about SQL-MAL correlation. Thank you in advance! Kind regards, Liang Bing -- View this message in context: http://old.nabble.com/SQL-MAL-correlation-tp32557812p32557812.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Announcement: New Aug2011-SP1 bugfix release of MonetDB suite
by Fabian Groffen 30 Sep '11

30 Sep '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Aug 2011-SP1 bugfix release SQL * Fixed a bug when using default values for interval columns. Bug 2877. * A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882. * A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883. Client Package * Fixed Perl DBD rowcount for larger results, bug #2889 * Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema). * Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois. * Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois. * mclient: fix display of varchar columns with only NULL values. * Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables. MonetDB5 Server * Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888. Bug Fixes * 2865: mserver5 --set gdk_nr_threads=2 --forcemito: deadlock during first SQL client connect on virgin (empty) DB * 2877: assertion failed when creating table with default interval * 2879: A SELECT query fails to produce some results * 2882: Delete of some records don't work on Monetdb * 2883: insert into t (select * from t) bogus in multithreaded execution * 2884: Mapi.pm and GetInfo.pm syntax fixes for array usage * 2885: DBD::monetdb table_info error * 2887: Division by zero error, when should not divide at 0 * 2888: test/null/null_in_case.sql hangs server * 2889: Mapi::getBlock is erasing query count on second call

1 0

[MonetDB-users] copy ... from stdin in perl script
by Eugene Prokopiev 28 Sep '11

28 Sep '11

Hi, I have table t1(f1 integer) I can copy to it by executing this file via mclient: $ cat /tmp/f1.sql copy into t1 from stdin; 1 2 3 But I can't copy to it by this perl script: $ cat /tmp/f1.pl #!/usr/bin/perl -w use DBI; my $dbh = DBI->connect('dbi:monetdb:database=db', 'dbadmin', "dbpwd"); $dbh->do("copy into t1 from stdin;\n1\n2\n3\n"); I got: Missing argument in sprintf at /usr/lib64/perl5/DBD/monetdb/GetInfo.pm line 25. No query result at /usr/lib64/perl5/MonetDB/CLI/MapiLib.pm line 80. Use of uninitialized value in die at /usr/lib64/perl5/MonetDB/CLI/MapiLib.pm line 180. (in cleanup) Died at /usr/lib64/perl5/MonetDB/CLI/MapiLib.pm line 180. Why can it be? -- Thanks, Eugene Prokopiev

2 9

[MonetDB-users] CSV Data Fault still used ?
by Stéphane NOUHAUD 26 Sep '11

26 Sep '11

Hi, I try to use CSV Data Fault (Documented here http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/CSVvault) without great succes... Just quick question is CSV Data Fault function still included in last version of MonetDB ? Except that MonetDB is really fantastic software. Thanks for sharing it ! Regards, -- *Stéphane NOUHAUD* * *

1 0

[MonetDB-users] Parallel COPY INTO
by zack_pcd 22 Sep '11

22 Sep '11

I have a large table with about 50 billion records. The files to be loaded are not in CSV format. The files are processed in java and the data uploaded to the database using COPY INTO through the MapiSocket interface. I do not have the record count readily available. My question is this: If I open multiple MapiSockets and upload multiple streams of data at the same time to the same table, is this supported and stable? I am asking because the database appears to become unresponsive after the bulk load. (mclient seems to hang on connect) If I stop and the start the database after the bulk load, I get: "failed to fork mserver: database 'test_db' has inconsistent state (sabaoth administration reports running, but process seems gone)" -- View this message in context: http://old.nabble.com/Parallel-COPY-INTO-tp32503834p32503834.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Partitioning on Monetdb
by Benji07 21 Sep '11

21 Sep '11

Does Monetdb supports basic table partitioning using MAPI? I need to split one large table into smaller physical pieces and improve query performance. I didn't find help in documentation. Thanks for help -- View this message in context: http://old.nabble.com/Partitioning-on-Monetdb-tp32493404p32493404.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] EPS / SVG of new MonetDB logo
by Stefan de Konink 20 Sep '11

20 Sep '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, The new logo is online on the new website for a while now, is there any EPS/SVG or other vector format available? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk547rYACgkQYH1+F2Rqwn14VwCgioRrllvOlEUwHJke6eLxdXca rm0An2CwkBr1KoUWmgYewB64U5B49GtS =ef+A -----END PGP SIGNATURE-----

2 3

[MonetDB-users] GUID
by Michal Dvorak 20 Sep '11

20 Sep '11

Hi MonetDB-Team, are you going to implement 128b GUID? If not, any idea how to bypass it? 2x Int64 or BLOB(128)? Thank you very much for your answer J Best regards, Michal

3 2

[MonetDB-users] Maximum Parallelism and Recommended Hardware for MonetDB
by brien colwell 20 Sep '11

20 Sep '11

I set up a MonetDB server for a data mining team on a single node. The node's specs are 32 cores (4x8 AMD 2.3GHz) 128GiB RAM, with the dbfarm on a RAID0 2xSSD (I think they are Intel SSDs). MonetDB on this hardware works well for aggregates and joins on integers, but chugs considerably on inline views with strings and joins with strings. The team usually has 2-3 concurrent jobs at any given time, some materializing new tables, with lots of bulk loading ("copy into ... from file ..."). A typical table size is 2GiB, and queries may join 5-10 tables. When I look at process activity on the node, I usually see each DB's mserver5 process pegged on CPU, using just a single core. Since I rarely see more than one core used per mserver5, I'm starting to think either I haven't configured MonetDB to favor parallel plans, or I chose the wrong hardware for MonetDB. So, as questions to everyone on the list, I'm curious 1) Are there knobs in MonetDB to favor parallel execution plans? 2) What's the ideal hardware for a single-node MonetDB server to support a workload of 2-5 concurrent queries? 3) What's the ideal disk configuration for the above? Brien

2 1

[MonetDB-users] How to compress a table
by Alexis Berlemont 20 Sep '11

20 Sep '11

Hi, I am currently trying to evaluate MonetDB as a solution to hold significant amounts of data. Because of limited storage capacities, I am trying to enable compression in MonetDB. I did not manage to find any tutorial on that subject, so I did not go very far. I cannot even tell that I wholly understand the problem. According to the page "Documentation->User Guide->compression", I understood that I had to inject a compression optimizer inside a pipeline. This "thing" is supposed to be possible via the setting of a sql environment variable named "optimizer". In the (generated) source file monetdb5/optimizer/opt_pipes.c, I found a default + compression pipeline. Then, in a mclient session, I typed: sql>set optimizer='inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector'; After that, I tried to use a sql function which was supposed to compress (thanks to the zlib) a table: sql.gzcompress sql>call gzcompress(myschema, mytable); Unfortunately, I am stuck here with the error message 'SELECT: identifier 'myschema' unknown'. I think I located the error message in "sql/server/rel_select.c" but I did not manage to deduce anything interesting. Do you have any idea ? Thanks in advance. Alexis.

2 1

[MonetDB-users] How to change in/out date format
by Eugene Prokopiev 20 Sep '11

20 Sep '11

Hi, is it possible to use "yy-MM-dd HH:mm:ss.SSS" date format via something like SET DATESTYLE in COPY INTO ... FROM ... with mclient to reuse existing files? -- Thanks, Eugene Prokopiev

2 1

[MonetDB-users] query time
by anagnostes 20 Sep '11

20 Sep '11

Hi. I would like to know if it's possible to get the query time using the input "\<file" and redirecting the output to "\>file" in Windows. When i use the input file it nevers throws the query time. So i can't know it. Thank you. -- View this message in context: http://old.nabble.com/query-time-tp32232809p32232809.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Memory leaks in the latest trunk?
by Stefan de Konink 20 Sep '11

20 Sep '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, Recently I decided it was again a good idea(tm) to 'test' the latest MonetDB trunk on an updated dataset. Sadly over the last 6 days I observed a growing memory usage, today resulting in a segmentation fault. For now nothing more than mserver5[26142]: segfault at 8 ip 00007f1e1e592a31 sp 00007f1e00ca63b8 error 4 in libc-2.13.so[7f1e1e515000+17d000] is known to me, which is development wise totally useless. But an other question did arise: is testing currently taking memory into a count? Especially longer tests might show greater usage of memory. It is obviously no problem replay all the queries that ran against the database and valgrind it... but there might have been work done from development respective that can shed light over the problems. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk5OiAAACgkQYH1+F2Rqwn3d+ACdH5ySymgx46EON89qdnlYof39 ZHsAn1LjL9eOHncOjM99+wAupwMC6OR3 =BsVB -----END PGP SIGNATURE-----

2 1

[MonetDB-users] count slower than expected?
by Joris Slob 20 Sep '11

20 Sep '11

Hello mailing list! This is my first attempt to ask a question here, so please excuse my ignorance. For large tables (>1M rows), I encounter strange behavior when using the COUNT function in SQL in the mclient. When I do the following query: SELECT DISTINCT(predicate) FROM facts; I get a nice resultset of 31 tuples in 20ms. When I try the following query: SELECT COUNT(DISTINCT(predicate)) FROM facts; I get my answer in 565ms. It seems weird that the database doesn't 'optimize' this count to: SELECT COUNT(*) FROM (SELECT DISTINCT(predicate) FROM facts) AS b; I get my answer in 20ms. The facts table is defined as follows: CREATE TABLE "sys"."facts" ( "subject" CHAR(2048), "predicate" CHAR(2048), "object" CHAR(2048) ); This is my first attempt to make a knowledge store and I am aware there is lots I can do to optimize how I store the data. I was just surprised that the count took significantly longer than getting all the results out and counting them. Here is the version information of MonetDB I am using: MonetDB 5 server v11.3.7 "Apr2011-SP2" (32-bit, 32-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 1.0GiB available memory, 2 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8k 25 Mar 2009 (compiled with OpenSSL 0.9.8k 25 Mar 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: root(a)ottar.ins.cwi.nl (i486-pc-linux-gnu) Compilation: gcc -Wall -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : /usr/bin/ld -Wl,-Bsymbolic-functions Greetings, Joris

2 1

[MonetDB-users] omission in documentation?
by Sander Kruseman 20 Sep '11

20 Sep '11

Hi, I'm unable to find documentation on 'mapi_open' on the new website. I read the 5 available man pages, but found no mention of it. The man page for monetdb does however, mention monetdb.conf(5) but I cannot find that one either. Am I overlooking something? regards, Sander Kruseman

2 1

[MonetDB-users] Announcement: New Apr2011-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 16 Sep '11

16 Sep '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011 feature release of the MonetDB suite of programs. The default width for OID columns in the database is now 64 bits for all 64-bit architecture. In releases up to this release, the default width on 64-bit Windows was still 32 bits. This means that with the current release it is possible to have tables with more than about 2 million rows even on Windows (as long as you use a 64-bit build). More information (including release notes) on this release is available at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] Calling a stored procedure from java
by elarios 13 Sep '11

13 Sep '11

Hi, Please, I would like to know the correct sintaxis for calling a stored procedures in MonetDB from a Java application. I was using some code that works fine in MySQL but this doesn't work in MonetDB. The code inside a method is: ------------------------------------------------------------------- Connection con = null; ResultSet rs = null; CallableStatement cs = null; cs = con.prepareCall("{call hts.spInsertFile(?)}"); File fl = new File("/home//Test/example.pdf"); imgfile = new FileInputStream(fl); cs.setBinaryStream(1,imgfile); rs = cs.executeQuery(); ------------------------------------------------------------------- When I execute the stored procedure from Netbeans IDE the stored procedure works fine. I've made a trace in the application and the statement: con.prepareCall("{call hts.spInsertFile(?)}"); returns null and finally throws an exception: Exception in thread "main" java.lang.NullPointerException at testmonetdb.DBase.set_file(DBase.java:86) at testmonetdb.MonetDBTest.main(MonetDBTest.java:34) The MonetDB version I'm using is: MonetDB 5 server v11.3.7 "Apr2011-SP2" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 3.9GiB available memory, 4 available cpu cores Libraries: libpcre: 8.12 2011-01-15 (compiled with 8.10) openssl: OpenSSL 1.0.0d 8 Feb 2011 (compiled with OpenSSL 1.0.0d-fips 8 Feb 2011) libxml2: 2.7.7 (compiled with 2.7.7) Compiled by: mockbuild@ (x86_64-unknown-linux-gnu) Compilation: gcc -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : /usr/bin/ld -m elf_x86_64 I'm also using Netbeans IDE 7.1 with the jdbcclient.jar (http://dev.monetdb.org/downloads/Java/Oct2010-SP1/) Thanks in advance for your help, Enrique. -- View this message in context: http://old.nabble.com/Calling-a-stored-procedure-from-java-tp32431606p32431… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Does MonetDB support materialized view or not?
by Ken Cheng 03 Sep '11

03 Sep '11

Hi All, Does MonetDB support materialized view or not? Yours sincerely, Ken Cheng

3 3

[MonetDB-users] stack protector error
by Michael Sioutis 27 Aug '11

27 Aug '11

Hello, I get an error when trying to compile latest MonetDB default branch on my Ubuntu 10.04 64bit machine. bootstrap: OK configure (../configure --prefix=/opt/MonetDB/ --enable-monetdb5 --enable-sql --enable-rdf --enable-geom --enable-bits=64 --enable-debug --enable-datacell): OK make -j2: ERROR The error is: ../../gdk/gdk_batop.mx: In function ‘BATsetprop’: ../../gdk/gdk_batop.mx:2890: error: not protecting function: no buffer at least 8 bytes long I think flags -fstack-protector-all and -Wstack-protector are to blame, they default to 8 bytes. I can skip the problem if I set 8 byte long dummy buffers to EVERY function that doesn't have such a buffer, e.g., char mybuf[8]; memset(buf, 0, sizeof(buf)); I also thought of setting the buffer size to 1, by passing --param spp-buffer-size=1 as a compiler option, but then I get various other errors. gcc version: gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3 Copyright (C) 2009 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Mike

2 8

[MonetDB-users] Current Pathfinder list/team
by Anton Kolev 21 Aug '11

21 Aug '11

Gentlemen, Which is the proper current list (or personal contact) for the Pathfinder related questions? Are there recent work after Mar2011 branch [1]? Thank you in advance, Anton [1] http://dev.monetdb.org/hg/MonetDB/file/932a58fecbde/pathfinder

2 1

[MonetDB-users] Does MonetDB support regular expression predicates?
by Viktor Rosenfeld 21 Aug '11

21 Aug '11

Hi, does MonetDB support regular expression matching on strings? It's supposedly part of SQL:99, but I couldn't find anything about this on the homepage or the mailing list. I only found the Data Manipulation reference, which lists LIKE predicates. Cheers, Viktor

2 1

[MonetDB-users] Are there any unit test for each checkin/fix
by Ken Cheng 20 Aug '11

20 Aug '11

Hi All, Are there any unit test for regression test for each checkin/fix? Seems I can not find any test related source check in history. Yours sincerely, Ken Cheng

2 1

[MonetDB-users] Compiling MonetDB with JDBC on OS X 10.6
by Viktor Rosenfeld 18 Aug '11

18 Aug '11

Hi, I've tried building the latest MonetDB sources on OS X 10.6.8 and it fails with the following error message: checking for ant... /usr/bin/ant checking for java... /usr/bin/java checking for javac... /usr/bin/javac checking for jar... /usr/bin/jar checking for javadoc... /usr/bin/javadoc configure: error: MonetDB JDBC requires ant and Java 1.5 I used the following build options: ./configure --prefix=/Users/he-sk/unix --enable-strict --disable-assert --disable-debug --enable-optimize --enable-jdbc Searching on the mailing list, I found a post saying that the MonetDB JDBC driver will not build with Java 1.6 (the default on OS X 10.6), because the JDBC API changed from 1.5 to 1.6. However, the post is from September 2008, almost 3 years ago. Is this still the case? Or am I running into a different problem? List post: http://old.nabble.com/Installation-Problems-to19607306.html#a19645299 If MonetDB JDBC does require 1.5, I'm thinking of following the steps outlined here: http://wiki.oneswarm.org/index.php/OS_X_10.6_Snow_Leopard Does anybody have experience with this procedure or does anybody know an alternative? Also, I'm about to upgrade to OS X Lion (10.7). Is anybody running the MonetDB JDBC driver on Lion yet? Are you running it from sources or did you install the binary MonetDB distribution? Cheers, Viktor

2 2

[MonetDB-users] How to update one table from another
by Eugene Prokopiev 12 Aug '11

12 Aug '11

Hi, I have two tables: table1 ----------- id data ----------- 0 0 1 1 2 2 ----------- table2 ----------- id data ----------- 2 2 3 3 ----------- I need to increment data in big table1 by data from small table2 if it exists or insert absent data from table2 to table1. So result must looks like: table1 ----------- id data ----------- 0 0 1 1 2 4 3 3 ----------- I used loop by table2 rows and inserts or updates table1 in this loop in PostgreSQL. What is the most effective way to do it in MonetDB? -- Thanks, Eugene Prokopiev

2 2

[MonetDB-users] JOIN for functions
by Eugene Prokopiev 09 Aug '11

09 Aug '11

Hi, Is it possible to join functions which returns tables? I've got error: sql>select sumdate from summary_ip_in('wan') limit 1; +------------+ | sumdate | +============+ | 2-08-11 | +------------+ 1 tuple (1m 14s) sql>select sumdate from summary_ip_out('wan') limit 1; +------------+ | sumdate | +============+ | 2-08-11 | +------------+ 1 tuple (10m 59s) sql>select sumdate from summary_ip_in('wan') inner join summary_ip_out('wan') on summary_ip_in.sumdate = summary_ip_out.sumdate limit 1; SELECT: no such column 'summary_ip_in.sumdate' sql>select sumdate from summary_ip_in('wan') inner join summary_ip_out('wan') on sumdate = sumdate limit 1; SELECT: identifier 'sumdate' ambiguous What's wrong? -- Thanks, Eugene Prokopiev

2 2

[MonetDB-users] Check for inet equals
by Eugene Prokopiev 06 Aug '11

06 Aug '11

Hi, How to select from table by value in column with INET type? For example, I have table: $ msqldump -d billing -- msqldump dump database Mon Aug 1 11:03:13 2011 -- MonetDB v11.3.3, 'b0' START TRANSACTION; SET SCHEMA "sys"; CREATE TABLE "sys"."transfer_ip" ( "packetdate" TIMESTAMP, "proto" INTEGER, "src" INET, "sport" INTEGER, "sintf" VARCHAR(15), "dst" INET, "dport" INTEGER, "dintf" VARCHAR(15), "packets" BIGINT, "bytes" BIGINT, "prefix" VARCHAR(5), "processed" BOOLEAN ); COMMIT; COPY 10293965 RECORDS INTO "sys"."transfer_ip" FROM stdin USING DELIMITERS '\t','\n','"'; ^C I tried to select in such ways: # mclient -d billing Welcome to mclient, the MonetDB/SQL interactive terminal (Apr2011-SP1) Database: MonetDB v11.3.3, 'billing' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select * from transfer_ip where src = '10.7.1.10' limit 10; SQLException:str_cast:value too long for type (var)char(9) sql>select * from transfer_ip where src = '10.7.1.10':inet limit 10; unexpected symbol (:) sql>select * from transfer_ip where src << '10.7.1.10' limit 10; types inet(0,0) and tinyint(8,0) are not equal for column 'src' What it the right way? -- Thanks, Eugene Prokopiev

2 5

[MonetDB-users] openssl not found when it is present
by Medha Atre 04 Aug '11

04 Aug '11

I am trying to compile MonetDB-11.3.7 on Ubuntu 10.04 distribution (2.6.32-33 64 bit kernel). It fails with "openssl" not found error: I am running configure script with following options: $ ./configure --prefix=/work2/MonetDB-install-11.3.7 --enable-optimize --enable-sql --enable-monetdb5 <snip> checking for iconv... yes checking for working iconv... yes checking for iconv declaration... extern size_t iconv (iconv_t cd, char * *inbuf, size_t *inbytesleft, char * *outbuf, size_t *outbytesleft); checking for library containing uuid_generate... no checking for openssl... no configure: error: OpenSSL library not found but required for MonetDB5 $ openssl libraries are present on my system. I tried: $ cd /lib $ ll *ssl* -rw-r--r-- 1 root root 333856 2011-02-09 20:50 libssl.so.0.9.8 $ $ $ sudo apt-get install openssl Reading package lists... Done Building dependency tree Reading state information... Done openssl is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. $ Can someone please let me know if I am missing any step or need to do something else for the configure script to find SSL libraries? Thanks in anticipation. Medha

4 5

[MonetDB-users] Long running query with count(distinct ...)
by Eugene Prokopiev 02 Aug '11

02 Aug '11

Hi, I have table with 81 887 010 rows. Select count from this table works fine, but I can't wait for the results of select count(distinct ...) from this table. Why can it be? sql>explain select count(src) from transfer_ip; +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mal | +==================================================================================================================================================================================+ | function user.s1_1{autoCommit=true}():void; | | barrier _55 := language.dataflow(); | | _2 := sql.mvc(); | | _39:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",0,0,2); | | _43 := algebra.selectNotNil(_39); | | _35:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",2,0,2); | | _45 := algebra.kdifference(_43,_35); | | _41 := algebra.selectNotNil(_35); | | _47 := algebra.kunion(_45,_41); | | _6:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","transfer_ip",1); | | _9 := bat.reverse(_6); | | _49 := algebra.kdifference(_47,_9); | | _51 := aggr.count(_49); | | _40:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",0,1,2); | | _44 := algebra.selectNotNil(_40); | | _38:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",2,1,2); | | _46 := algebra.kdifference(_44,_38); | | _42 := algebra.selectNotNil(_38); | | _48 := algebra.kunion(_46,_42); | | _52 := aggr.count(_48); | | _10:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",1); | | _11 := algebra.selectNotNil(_10); | | _53 := aggr.count(_11); | | _50 := mat.pack(_51,_52,_53); | | _54 := algebra.selectNotNil(_50); | | _14 := aggr.sum(_54); | | exit _55; | | sql.exportValue(1,"sys.transfer_ip","L4","wrd",64,0,6,_14,""); | | end s1_1; | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 29 tuples (12.925ms) sql>explain select count(distinct src) from transfer_ip; +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mal | +==================================================================================================================================================================================+ | function user.s2_1{autoCommit=true}():void; | | barrier _56 := language.dataflow(); | | _2 := sql.mvc(); | | _42:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",0,0,2); | | _46 := algebra.selectNotNil(_42); | | _38:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",2,0,2); | | _48 := algebra.kdifference(_46,_38); | | _44 := algebra.selectNotNil(_38); | | _50 := algebra.kunion(_48,_44); | | _6:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","transfer_ip",1); | | _9 := bat.reverse(_6); | | _52 := algebra.kdifference(_50,_9); | | _53 := bat.reverse(_52); | | _43:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",0,1,2); | | _47 := algebra.selectNotNil(_43); | | _41:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",2,1,2); | | _49 := algebra.kdifference(_47,_41); | | _45 := algebra.selectNotNil(_41); | | _51 := algebra.kunion(_49,_45); | | _54 := bat.reverse(_51); | | _10:bat[:oid,:inet] := sql.bind(_2,"sys","transfer_ip","src",1); | | _11 := algebra.selectNotNil(_10); | | _55 := bat.reverse(_11); | | _14 := mat.pack(_53,_54,_55); | | _15 := algebra.kunique(_14); | | _16 := bat.reverse(_15); | | _17 := aggr.count(_16); | | exit _56; | | sql.exportValue(1,"sys.transfer_ip","L5","wrd",64,0,6,_17,""); | | end s2_1; | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 30 tuples (1.238ms) -- Thanks, Eugene Prokopiev

3 4

[MonetDB-users] Vacuum command missing
by Eugene Prokopiev 01 Aug '11

01 Aug '11

Hi, I can't find VACUUM command in http://www.monetdb.org/Documentation, I see only vacuum cleaning algorithm mention in http://www.monetdb.org/Documentation/Manuals/SQLreference/Transactions. So, how can phisicaly delete rows after marking them as deleted? -- Thanks, Eugene Prokopiev

2 3

[MonetDB-users] RDF support in MonetDB-11.3.7?
by Medha Atre 31 Jul '11

31 Jul '11

Hello, While compiling MonetDB-11.3.7 from sources, in the configure options, I saw an option "--enable-rdf". I was wondering how this option can be used to load RDF data directly into MonetDB? MonetDB's documentation on the web could not help much. Any pointers will be appreciated. Thanks in anticipation. Medha

3 4

[MonetDB-users] Announcement: New Apr2011-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 29 Jul '11

29 Jul '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2011-SP2 bug fix release of the MonetDB suite of programs. This is the first release with support for Fedora 15. Fedora 13 is no longer supported. One important bug fix has to do with columns with string (char/varchar) values. There was a bug that could cause exponential growth of the file holding the data for the column. This bug has been fixed. If you have a database that is much larger than you would expect for the amount of data, this might be the cause. The remedy is to dump and restore that database (or just the affected tables). Apart from using up too much space, this bug didn't have any other consequences. More information (including release notes) on this release is available at <http://www.monetdb.org/OldReleaseNotes/Apr2011>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[MonetDB-users] Problem with Create Function sintaxis
by elarios＠liacs.nl 20 Jul '11

20 Jul '11

Hi, I'm using MonetDB over Fedora 15 and the driver monetdb-1.20-jdbc.jar I have problems with the sintaxis of the following function: ----------------------- create function hts.Get_idimage() returns int begin declare idimag int; select max(Imag_id) into idimag from hts.Image; returns idimag+1; end; ----------------------- the error message is: ----------------------- Error: syntax error, unexpected $end, expecting WHILE in: "create function hts.get_idimage() returns int begin declare idimag int;" SQLState: null ErrorCode: 0 Error occured in: create function hts.Get_idimage() returns int begin declare idimag int ----------------------- Can you give me some clues of how to fix this sintaxis error? I've checked the SQL 2003 framework and the script looks fine. Thanks in advance for your help. Enrique.

4 4

[MonetDB-users] Table with 1 000 000 000 rows
by Eugene Prokopiev 20 Jul '11

20 Jul '11

Hi, is it possible to store 1 000 000 000 rows in one table or I need to use some partitioning algorithm? -- Thanks, Eugene Prokopiev

3 4

[MonetDB-users] Normalization
by Eugene Prokopiev 20 Jul '11

20 Jul '11

Hi, need I use normalization for tables in colum-oriented databases or it will only decreses perfomance without reducing space usage and other advantages? -- Thanks, Eugene Prokopiev

2 1

[MonetDB-users] Information about Loader
by Lowterm 19 Jul '11

19 Jul '11

Hi, I need some information about MonetDB Loader. Where can I find it? Is there any documentation about it? Thanks -- View this message in context: http://old.nabble.com/Information-about-Loader-tp32084241p32084241.html Sent from the monetdb-users mailing list archive at Nabble.com.

4 6

[MonetDB-users] Clustering, Replication, MultiThreading, Restore
by joel vennin 10 Jul '11

10 Jul '11

Hi, I've just discover MonetDB. I've found some documentation but i've still some questions. 1) About replication, i'm unable to find a documentation about replication support inside MonetDB, if it is not supported do you plan to support it natively, or is there some special script to launch in order to have database in sync. 2) About clustering, my understanding about clustering is to allow one client to launch a sql query on a database through merovingian (if i understood). The query is analysed by all nodes of the cluster and then it return the answer. However, it seems it is not the case, if I follow the documentation on the website about clustering, it seems that I will be connected through a round robin fashion to one of the node. Once connected my query will be executed only on that node. 3) About multithreading, I mean multi query execution per user, I think MonetDb support the fact that multi user can connect to the database and launch different query on it. But is it possible to configure the number of concurrent connection, to setup the number of thread that handle queries ? 4) If MonetDb crash, is it possible that db files become corrupted ? In this case, is there a script to fix it ? Thank you for your help Joel

3 5

[MonetDB-users] how store tiff image in monet db?
by rose26 08 Jul '11

08 Jul '11

-- View this message in context: http://old.nabble.com/how-store-tiff-image-in-monet-db--tp32014566p32014566… Sent from the monetdb-users mailing list archive at Nabble.com.

4 3

[MonetDB-users] Installation problem in Fedora 15
by elarios＠liacs.nl 06 Jul '11

06 Jul '11

Hi, I'm interested in using MonetDB in my project and I'm a beginner in this topic, so I'm sorry for the silly questions that I can make. The problem is: I'm trying to install MonetDB-SQL Server in Fedora 15, but when I execute the command: yum install MonetDB-SQL-server5 MonetDB-client The output makes reference to a repository that doesn't exist: http://dev.monetdb.org/downloads/Fedora/15/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found : http://dev.monetdb.org/downloads/Fedora/15/x86_64/repodata/repomd.xml Trying other mirror. http://dev.monetdb.org/downloads/Fedora/15/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found : http://dev.monetdb.org/downloads/Fedora/15/x86_64/repodata/repomd.xml Trying other mirror. Error: Cannot retrieve repository metadata (repomd.xml) for repository: monetdb. Please verify its path and try again Should I downgrade my operating system to Fedora 14? Do you have any suggestions? Thanks for your time and help, Best regards, Enrique.

3 5

[MonetDB-users] PhD opening at CWI - Adaptive indexing techniques for modern database systems
by Stratos Idreos 23 Jun '11

23 Jun '11

Subject: Adaptive indexing techniques for modern database systems. The Centrum Wiskunde & Informatica (CWI) has a full time vacancy for a talented PhD candidate in the Database Architectures group (http://www.cwi.nl/en/research-groups/Database-Architectures) The position is in the broad area of adaptive indexing techniques for modern database systems. Adaptive indexing is a major challenge in database research; it tries to completely alleviate the need for indexing and tuning steps leading to database systems that require no physical design set-up and can cope immediately with any workload. This leads to database systems that are instantly usable and thus more applicable to numerous new applications that now consider databases too slow, too complex and too old. This position will focus specifically on adaptive indexing for extremely large databases where traditional indexing represents a major bottleneck. The Database Architectures group is one of the world-wide leading groups in system-oriented database research, e.g., it received the 10-year best paper award at VLDB 2009 for their pioneering research on hardware-aware database technology and the column-store model and the 2011 ACM SIGMOD Jim Gray Dissertation award for the database cracking work. The group has a unique methodology that combines fundamental database research with active participation of all group members in database engineering, centred around the open-source analytical database system MonetDB (http://www.monetdb.org/) An internship at one of the partnering database research labs world-wide is considered part of the PhD track. REQUIREMENTS The PhD candidates must have a master degree in computer science (or a 5 year diploma), (specialisation databases, distributed systems, software engineering) mathematics, or related fields. Preferable qualifications for candidates include proven research talent, system programming skills (C/C++), practical experience with using and implementing database systems, and software development in a team. Candidates are expected to have an excellent command of English, and good academic writing and presentation skills. CENTRUM WISKUNDE & INFORMATICA (CWI) Founded in 1946, the Centrum Wiskunde & Informatica (CWI) is the national research institute for mathematics and computer science in the Netherlands. It is located at the Science Park Amsterdam and is part of the “Netherlands Organisation for Scientific Research” (NWO). The institute is internationally focused and renowned for its high quality research, a substantial fraction of which is carried out in the framework of (inter)national programs. Over 160 researchers conduct pioneering research and share their acquired knowledge with society. More than 30 researchers are employed as professors at universities. The institute has generated twenty-one spin-off companies. CWI maintains excellent relations with industry and the academic world, both in the Netherlands as well as abroad. After their research careers at CWI, an increasing number of young staff members find employment in these sectors or in spin-off companies that are based on research results from CWI. Of course, library and computing facilities are first-rate. CWI's non-scientific services to its personnel include career planning, training and courses, assistance in finding housing, and tailor-made solutions to problems that may occasionally arise. TERMS AND CONDITIONS The terms of employment are in accordance with the Dutch Collective Labour Agreement for Research Institutes ("CAO-onderzoeksinstellingen"). Moreover CWI offers attractive working conditions, including flexible scheduling and help with housing for expat employees. The gross monthly salary, for an employee on a full time basis, is €2,037 during the first year and increases to €2,610 over a four year period. CWI also offers excellent and flexible terms of employment, including an employee pension fund. INFORMATION Additional information can be obtained from: dr. Stratos Idreos, email stratos.idreos(a)cwi.nl, telephone +31(0)20 592 4169. or dr. Stefan Manegold, email stefan.manegold(a)cwi.nl, telephone +31(0)20 592 4212, or prof. dr. Martin Kersten, email martin.kersten(a)cwi.nl, telephone +31(0)20 592 4066. APPLICATION Please send your application before 31 July 2011 to: pd(a)cwi.nl. Applications should include a detailed CV, a motivation letter, a list of your M.SC. courses and grades, a copy of your master thesis, and preferably a list of publications.

1 0

[MonetDB-users] Parallelization
by Michael Kusber 20 Jun '11

20 Jun '11

Hi, I'm playing with parallization of databases and I am a little confused about my result on MonetDB. Scenario: 15 Millions of Entries and i want to find the number of entries where the attributes are in a certain sector (i know, a simple select would do the work but i wanted a simple example) i have java code that get all 15 Mio. Data from the database via JDBC and iterates over them with a loop without multithreading. --> 13 ms then i created a stored function: CREATE FUNCTION isInRangexy(x DECIMAL(10,7), y DECIMAL(10,7)) RETURNS BOOL BEGIN ATOMIC IF x > 50 AND y < 50 THEN RETURN true; ELSE RETURN false; END IF; END Then I call it with: SELECT COUNT(*) FROM location WHERE isInRange(x,y) It works, but it is more than 3 times slower than the first java option. I had thought i would get benefit from the fact that i don't need to transfer all the data to the application and of some parallelization and other optimization from monetdb. Did I made a mistake? Can I use another example to get the anticipated results? thanks, michael

2 1

[MonetDB-users] Stored procedures
by Michael Kusber 19 Jun '11

19 Jun '11

Hi, is it possible to achieve the following using SQL stored procedures à la "CREATE PROCEDURE XYZ"? I want to write a function that iterates over a table and adds 2 on a certain attribute. i am realizing that i could do this by an UPDATE, but I wanted to see, whether it would be executed more parallel inside a stored procedure. Is there a way of formulating this in a procedure? do i need to define a function in MAL? would this result in more parallel execution? thanks, michael

2 1

[MonetDB-users] Data compression in MonetDB
by Ken Cheng 18 Jun '11

18 Jun '11

Hi All, I am going to build a DW based on MonetDB. After check on the internet I found that MonetDB does not do data compression. Same data set in MonetDb is much bigger than MySQL and infoBright. Are there any plan to do data compression in the MonetDB? Yours sincerely, Ken Cheng

2 3

[MonetDB-users] Windowing functions ?
by Fabian Pijcke 16 Jun '11

16 Jun '11

Hello everyone, After visiting the MonetDB website, I didn't saw any information about the windowing functions of SQL (ranking, selecting max according to a defined order, etc.) A (very) short introduction can be found on wikipedia: http://en.wikipedia.org/wiki/Select_%28SQL%29#Window_function I know it is implemented into PostGreSQL, Oracle and Microsoft SQL Server. Is there any chance to see this function in MonetDB in the future ? Thanks in advance, sorry for my poor english ... :( Fabian Pijcke

2 2

[MonetDB-users] Create Database - Load Data
by Lowterm 15 Jun '11

15 Jun '11

Hi, how can I create a new Database under Windows and how can I load data in a table form a external file? Thanks -- View this message in context: http://old.nabble.com/Create-Database---Load-Data-tp31653813p31653813.html Sent from the monetdb-users mailing list archive at Nabble.com.

8 24

[MonetDB-users] PhD position at CWI Database Architecture group
by Stefan Manegold 14 Jun '11

14 Jun '11

CWI is an internationally renowned research institute in mathematics and computer science, located in Amsterdam, The Netherlands. The focus is on fundamental research problems, inspired by societal needs. Research is carried out in 18 research groups and a substantial fraction is carried out in the framework of (inter)national programs. VACANCY CWI has a vacancy for a talented PhD candidate in the DB Architectures group (http://www.cwi.nl/en/research-groups/Database-Architectures) DISTRIBUTED DATABASE ARCHITECTURES FOR SCIENCE APPLICATIONS The Database Architectures group is one of the world-wide leading groups in system-oriented database research, e.g., it received the 10-year best paper award at VLDB 2009 for their pioneering research on hardware-aware database technology and the column-store model. The group has a unique methodology that combines fundamental database research with active participation of all group members in database engineering, centered around the open-source analytical database system MonetDB (http://www.monetdb.org/) You will be expected to write both papers and contribute to this code base. The focus of the group is on hard core data management and for this position specifically on database architectures to support large scale scientific applications in the context of the SciLens project (http://www.scilens.org) The goal is to leverage the potential power of a 330 node dedicated database machine to improve science in, e.g., astronomy and seismology. (http://www.scilens.org/content/platform) CWI maintains excellent relations with industry and the academic world, both in the Netherlands as well as abroad. An internship at one of the partnering database research labs world-wide is considered part of the PhD track. After their research careers at CWI, many staff members find employment in these sectors or in spin-off companies based on research results from CWI. Of course, library and computing facilities are first-class. CWI's non-scientific services to its personnel include career planning, training and courses, assistance in finding housing, and tailor-made solutions to problems that may occasionally arise. REQUIREMENTS The PhD candidates must have a master degree in computer science, (specialization distributed systems, database, software engineering) mathematics, or related fields. Preferable qualifications for candidates include proven research talent, system programming skills (C/C++), practical experience with database systems, and software development in a team. One should bring good academic writing and presentation skills. Candidates are expected to have an excellent command of English. TERMS AND CONDITIONS The terms of employment are in accordance with the Dutch "CAO-onderzoeksinstellingen". Moreover, CWI offers attractive working conditions, including flexibility and help with housing for foreigners. The gross monthly salary (on a full time basis) will be â¬ 2,037 during the first year and will reach â¬ 2,610 during the fourth year. INFORMATION Additional information can be obtained from prof.dr. Martin Kersten, email martin.kersten(a)cwi.nl, telephone +31(0)20 592 4066, or dr. Stefan Manegold, email stefan.manegold(a)cwi.nl, telephone +31(0)20 592 4212. -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[MonetDB-users] Select Query crashes the server with "--single-user" option
by Ramak 13 Jun '11

13 Jun '11

Hi, I am trying to use MonetDB in single user mode. While I could load the data using "COPY INTO" command and run a few select queries successfully, the moment i run a query with group by on more than one column, the server crashes saying " ---- >Segmentation fault". Is there a way to get around it? or, am I using it the wrong way? I could recreate the problem doing this--- 1. Start server with "sudo mserver5 --dbfarm='/home/user/work/tmp/data/test' --single-user" 2. start mclient (and create a simple table and fire a select query with group by on multiple columns) >create table test ( market_size int not null, market int not null, state int not null); >select count(*) from test group by market_size, state; -- server crashes on the above query. I am using MonetDB 5 server v11.3.3 "Apr2011-SP1" (64-bit, 64-bit oids) Thanks, Ram -- View this message in context: http://old.nabble.com/Select-Query-crashes-the-server-with-%22--single-user… Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

[MonetDB-users] Production Database Crash - Analysis / Recovery?
by Philippe Hanrigou 12 Jun '11

12 Jun '11

My database is crashing quite systematically in production during some data import (which is in the form as a serie of DELETE ... WHERE / COPY INTO). The relevant information from the merovigian.log seems to be: 2011-06-02 04:53:27 MSG prod_reporting[25449]: !SQLException:SQLinit:Catalogue initialization failed 2011-06-02 04:53:27 MSG prod_reporting[25449]: !ERROR: HEAPextend: failed to extend to 3316460814336 for 11/40/114026theap A) What does this error mean? ----------------------------------------- A lack of memory? I am quite confused as the entire database is barely 5G on disk and there is 7G of RAM on this machine, which is dedicated solely to MonetDB. Moreover I only DELETE/COPY INTO in a single table at a time and the biggest table is barely 1.6G. Cleary there is some memory dynamic that I am not understanding. B) How do I recover the crashed database? ----------------------------------------------------------- It would not start anymore: 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Listening for UNIX domain connection requests on mapi:monetdb:///var/monetdb5/dbfarm/prod_reporting/.mapi.sock 2011-06-02 04:55:19 MSG prod_reporting[1665]: # MonetDB/SQL module loaded 2011-06-02 04:55:19 MSG merovingian[1644]: database 'prod_reporting' (1665) was killed by signal SIGSEGV 2011-06-02 04:55:29 ERR control[1644]: (local): failed to fork mserver: database 'prod_reporting' has crashed after starting, manual intervention needed, check merovingian's logfile for details Thanks in advance, - Philippe -- more complete extract from merovigian.log --- 2011-06-02 04:40:38 MSG merovingian[639]: proxying client ip-10-204-61-105.ec2.internal:52864 for database 'prod_reporting' to mapi:monetdb:///var/monetdb5/dbfarm/prod_reporting/.mapi.sock?database=prod_reporting 2011-06-02 04:40:38 MSG merovingian[639]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2011-06-02 04:53:09 MSG control[639]: (local): served status list 2011-06-02 04:53:23 MSG merovingian[639]: caught SIGTERM, starting shutdown sequence 2011-06-02 04:53:24 MSG control[639]: control channel closed 2011-06-02 04:53:27 MSG merovingian[639]: sending process 25449 (database 'prod_reporting') the TERM signal 2011-06-02 04:53:27 MSG prod_reporting[25449]: !SQLException:SQLinit:Catalogue initialization failed 2011-06-02 04:53:27 MSG prod_reporting[25449]: !ERROR: HEAPextend: failed to extend to 3316460814336 for 11/40/114026theap 2011-06-02 04:53:27 MSG merovingian[639]: database 'prod_reporting' (25449) has exited with exit status 0 2011-06-02 04:53:27 MSG merovingian[639]: database 'prod_reporting' has shut down 2011-06-02 04:53:27 MSG merovingian[639]: Merovingian 1.4 stopped 2011-06-02 04:55:16 MSG merovingian[1644]: Merovingian 1.4 (Apr2011-SP1) starting 2011-06-02 04:55:16 MSG merovingian[1644]: monitoring dbfarm /var/monetdb5/dbfarm 2011-06-02 04:55:16 MSG merovingian[1644]: accepting connections on TCP socket 0.0.0.0:50000 2011-06-02 04:55:16 MSG merovingian[1644]: accepting connections on UNIX domain socket /tmp/.s.monetdb.50000 2011-06-02 04:55:16 MSG discovery[1644]: listening for UDP messages on 0.0.0.0:50000 2011-06-02 04:55:16 MSG control[1644]: accepting connections on UNIX domain socket /tmp/.s.merovingian.50001 2011-06-02 04:55:16 MSG control[1644]: accepting connections on TCP socket 0.0.0.0:50001 2011-06-02 04:55:16 MSG discovery[1644]: new neighbour ip-10-32-111-2 (ip-10-32-111-2.ec2.internal) 2011-06-02 04:55:16 MSG discovery[1644]: new database mapi:monetdb://ip-10-32-111-2:50000/prod_reporting (ttl=660s) 2011-06-02 04:55:16 MSG discovery[1644]: registered neighbour ip-10-32-111-2:50001 2011-06-02 04:55:19 MSG control[1644]: (local): served status list 2011-06-02 04:55:19 MSG merovingian[1644]: starting database 'prod_reporting', up min/avg/max: 1m/31m/1h, crash average: 0.00 0.10 0.03 (6-5=1) 2011-06-02 04:55:19 MSG prod_reporting[1665]: arguments: /usr/bin/mserver5 --set gdk_dbfarm=/var/monetdb5/dbfarm --dbname=prod_reporting --set merovingian_uri=mapi:monetdb://ip-10-32-111-2:50000/prod_reporting --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/monetdb5/dbfarm/prod_reporting/.mapi.sock --set monet_vault_key=/var/monetdb5/dbfarm/prod_reporting/.vaultkey --set monet_daemon=yes 2011-06-02 04:55:19 MSG prod_reporting[1665]: # MonetDB 5 server v11.3.3 "Apr2011-SP1" 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Serving database 'prod_reporting', using 2 threads 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Found 7.294 GiB available main-memory. 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Copyright (c) 1993-July 2008 CWI. 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved 2011-06-02 04:55:19 MSG prod_reporting[1665]: # Listening for UNIX domain connection requests on mapi:monetdb:///var/monetdb5/dbfarm/prod_reporting/.mapi.sock 2011-06-02 04:55:19 MSG prod_reporting[1665]: # MonetDB/SQL module loaded 2011-06-02 04:55:19 MSG merovingian[1644]: database 'prod_reporting' (1665) was killed by signal SIGSEGV 2011-06-02 04:55:29 ERR control[1644]: (local): failed to fork mserver: database 'prod_reporting' has crashed after starting, manual intervention needed, check merovingian's logfile for details 2011-06-02 05:03:18 MSG merovingian[1644]: database 'prod_reporting' has crashed after start on 2011-06-02 04:55:19, attempting restart, up min/avg/max: 1m/31m/1h, crash average: 1.00 0.20 0.07 (7-5=2)

4 8

[MonetDB-users] monetdb checkpoint
by Philippe Hanrigou 11 Jun '11

11 Jun '11

Hi, Online documentation at http://www.monetdb.org/Documentation/Cookbooks/ServerMgmt/ResourceLimits mentions: > The database checkpoint can be made using 'monetdb checkpoint' Yet checkpoint does not seem to be a valid monetdb command. Is it a typo, an historical vestige? Cheers, - Philippe

2 2

[MonetDB-users] Octopus usage in MonetDB
by Andrei Cojocaru 07 Jun '11

07 Jun '11

Hi all, I am trying to setup (and play around with) a MonetDB-Apr11 cluster since the dataset I am expecting to have is beyond what can fit in the memory one node (starting with at least a few hundred gigabytes). I have been following the instructions athttp://www.monetdb.org/Documentation/Cookbooks/Recipes/Cluster and have a few nodes setup. I ran all the commands given there on all the nodes except for the creation of the master database which I ran on only the master. Also, I set the pipe to octopus_pipe on the master database. In mclient, I ran the following commands (from the sample squid test for octopus in the MonetDB source code): create table squidA( Bid int, Cval int); create table squidD( Eid int, Fval int); -- let's fill the tables create procedure fill(lim int) begin declare i int; set i = 0; while i < lim do insert into squidA values(i,rand()); insert into squidD values(i,rand()); set i = i+1; end while; end; declare size int; set size=100; call fill(size); It all runs with no errors. However, when I run: select Bid from squidA, I get a screenful of errors. They are all a variation of: TypeException:octopus.tentacle_3_11[6]:'octopus.bind' undefined in: _18:bat[:oid,:int] := octopus.bind(_2:str, _8:str, _9:str, _13:str, _19:int, _16:int, _17:int, version:int) Would you be able to offer some guidance on how to get this running? I was considering setting up application level partitioning and run multiple MonetDB instances directly, but if octopus does a lot of that work I would prefer to save myself the development and debugging time. Thanks for any help you could provide. -Andrei

3 2

[MonetDB-users] Problem with Create Tables
by Lowterm 03 Jun '11

03 Jun '11

Hi, what's the meaning of this syntax error? Syntax error, unexpected CREATE, expecting SCOLON in: "create ..." I get this message with the "create table" if I load a Files with "sql>\< filename", which contains this statement. The create-statement works, if I insert it directly. The second error message ist: syntax error, unexpected ON, expecting ')' or ',' in: "create ..." in the following create statement: CREATE TABLE dim_warehouse ( is_current smallint NOT NULL, last_modified timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP ) but I can not see wath is wrong. Thanks -- View this message in context: http://old.nabble.com/Problem-with-Create-Tables-tp31757411p31757411.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 6

[MonetDB-users] Aggregating by a computed field?
by Synaptic Axon 01 Jun '11

01 Jun '11

Hello, I recently install MonetDB and have been trying to get a feel for its speed and storage. I ran into an issue with SELECT queries trying to GROUP BY a computed field. For example, imagine a table like this: CREATE TABLE foo ( ts TIMESTAMP NOT NULL, category VARCHAR(32) NOT NULL ); I want to count the number of categories by year, so I want to craft a query like this: sql> SELECT EXTRACT( year from ts ) as year, category, COUNT(category) FROM foo GROUP BY 1, 2 ORDER BY 3 DESC; syntax error, unexpected sqlINT in: "SELECT EXTRACT( year from ts ) as year, category, COUNT(category) FROM foo GROUP BY 1, 2 ORDER BY 3 DESC" Unfortunately, MonetDB does not appear to support positional fields in the GROUP BY clause. So I tried rewriting it like this: sql> SELECT EXTRACT( year from ts ) as year, category, COUNT(category) FROM foo GROUP BY EXTRACT( year from ts ), category ORDER BY 3 DESC; syntax error, unexpected EXTRACT in: "SELECT EXTRACT( year from ts ) as year, category, COUNT(category) FROM foo GROUP BY EXTRACT( year from ts ), category ORDER BY 3 DESC" Ok, how about using "year" instead of the extract( .. ) bit? sql> SELECT EXTRACT( year from ts ) as year, category, COUNT(category) FROM foo GROUP BY year, category ORDER BY 3 DESC; syntax error, unexpected YEAR in: "SELECT EXTRACT( year from ts ) as year, category, COUNT(category) FROM foo GROUP BY year, category ORDER BY 3 DESC" What is the correct way to group by the computed field? Should I create a view? Is there a positional syntax for the GROUP BY clause I am overlooking? $ monetdb --version MonetDB Database Server Toolkit v0.8 (Apr2011) $ monetdbd --version MonetDB Database Server v1.4 (Apr2011) $ mclient --version mclient, the MonetDB interactive terminal (Apr2011) support for command-line editing compiled-in Thank you.

3 5

[MonetDB-users] Announcement: New Apr2011-SP1 bugfix release
by Fabian Groffen 31 May '11

31 May '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2011-SP1 bugfix release of the MonetDB suite of programs. At the time of this release, we also officially changed to our new website at http://www.monetdb.org/. This new website is our central source of information, and comes with our new logo and style. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>.

1 0

[MonetDB-users] Syntax for column tables
by Michael K 25 May '11

25 May '11

Hi, is there a "CREATE COLUMN TABLE xyz" like statement in MonetDB or will every "CREATE TABLE" create a column table? Can I create "normal" row store tables in MonetDB at all? I have read http://monetdb.cwi.nl/SQL/Documentation/Table-Definition.htmlbut did not found the information I was looking for. Kind regards. michael

3 2

[MonetDB-users] mclient credentials after install?
by Philippe Hanrigou 22 May '11

22 May '11

Hi, I am trying to get started with MonetDB in order to assess its response time for an in-house dataset. I managed to configure/make/install it on my MacBook Pro (I have joined the custom Portfile I used for this purpose at the end of this email in case it matters). Nevertheless, when I go through the "Getting started" instructions, mclient is asking for credentials that I never had a chance to explicitly set: ph7:~$ monetdbd start ph7:~$ monetdbd get all property value hostname Solaris.local dbfarm /opt/local/var/monetdb5/dbfarm mserver /opt/local/bin/mserver5 logfile merovingian.log pidfile merovingian.pid sockdir /tmp port 50000 controlport 50001 discoveryport 50000 exittimeout 60 forward proxy discoveryttl 600 passphrase <unknown> mapisock /tmp/.s.monetdb.50000 controlsock /tmp/.s.merovingian.50001 ph7:~$ monetdb create demo created database in maintenance mode: demo ph7:~$ monetdb status name state uptime health last crash demo locked ph7:~$ monetdb release demo taken database out of maintenance mode: demo ph7:~$ monetdb status name state uptime health last crash demo stopped ph7:~$ mclient -d demo user(ph7): password: InvalidCredentialsException:checkCredentials:invalid credentials for user 'ph7' Which credentials am supposed to use to launch mclient? I am willing to try any workaround as, at this point, I am not concerned with security at all. Thanks in advance, - Philippe --------------------------------------------------------- # ph7's custom Portfile # $Id PortSystem 1.0 name MonetDB version 11.3.1 categories databases platforms darwin maintainers nomaintainer description MonetDB is an open-source database system for high-performance applications in data mining, OLAP, GIS, XML Query, text and multimedia retrieval. MonetDB often achieves a significant speed improvement for SQL and XQuery over other open-source systems. homepage http://www.monetdb.org master_sites http://dev.monetdb.org/downloads/sources/Latest/ distfiles MonetDB-${version}.tar.bz2 use_bzip2 yes checksums md5 f48869c87a56604d1c06f7d8243554f9 \ sha1 03a29a5e00a90d5294443ffb849167a2c108302e \ rmd160 c9bcdff333a60aad42dc8504c458335494e749f1 configure.args --enable-debug=no --enable-assert=no --enable-optimize=yes

3 3

[MonetDB-users] using MonetDB in embedded mode - mapi interface
by Ramak 21 May '11

21 May '11

Hi, I am trying to use monetDB in embedded mode. The sample at "http://monetdb.cwi.nl/SQL/Documentation/A-Simple-Example.html#A-Simple-Exam…" suggests that we can interface to the data base using "embedded_sql()" method in combination with "embeddedclient.h". I dont seem to find this method in the library "libmapi.so.3"; neither the include file in the code base. Is "embedded mode" still supported? is there an alternate way of connecting to embedded monetDB server? PS: I am using monedDB5 - April 2011 release. Thanks, Ram -- View this message in context: http://old.nabble.com/using-MonetDB-in-embedded-mode---mapi-interface-tp316… Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Does a primary key based on a single column + numeric improve performance in MonetDB?
by Philippe Hanrigou 20 May '11

20 May '11

Hi, I am designing the schema of our brand new MonetDB database, and I am not sure whether my previous data modeling instincts are still relevant ;-). In particular, I am wondering: A) Does the definition of primary keys / foreign keys improve MonetDB performance? B) I need to do joins with a table whose natural key which is the combination of (bigint, varchar(51)). Is it best to: 1) just add these 2 columns to the dependent tables, and join on these 2 columns. 2) create a single varchar column that would be the concatenation of the bigint + varchar(51) and join on this single varchar column 3) Introduce a numeric+sequential surrogate key (presumably generated from a sequence), and join on the surrogate key. I am especially interested in the performance / acidental complexity tradeoffs in this situation. Thanks in advance, - Philippe

1 0

[MonetDB-users] MonetDB support for Upserts / On duplicate key ...
by Philippe Hanrigou 20 May '11

20 May '11

Hi, I am trying to operationalize our MonetDB usage and need some advice. What is the best way to do upserts in MonetDB? (https://secure.wikimedia.org/wikipedia/en/wiki/Upsert) Our application stores its data in a reference database (not MonetDB) and need to export some of its (derived) data in MonetDB for our real-time analytics needs. This "population" of MonetDB can happen in a couple of ways: A) In "real-time", on a "one row at a time" basis, when the application detects that it would be valuable to update the analytics information. B) Nightly in a batch oriented basis, to make sure that no information was lost due to a glitch/downtime during the day. C) We also periodically "refresh" all historical data in a batch oriented mode when we find bugs in the way we are deriving the data or what to derive new analytics from the same reference data. And whatever triggered the update, the rule is that the last update "wins" (and contains the full data). I am explaining this to illustrate why upserts would be very convenient. They would allow me to use the same code in all situations (A, B, C) and in "row" mode as well as in "batch" mode. If, as I suspect there is no out-of-the-box support for upserts, what is the best way to achieve them efficiently in batch mode. I am open to any (performant) solution, it does not have to be SQL based. Thanks in advance, - Philippe PS: Thanks a lot for MonetDB, I am completely amazed by its performance for our analytics queries.

1 0

Re: [MonetDB-users] MonetDB-11.3.1 on OpenBSD 4.9 i386 fails during make
by Michael Sioutis 20 May '11

20 May '11

It seems that in OpenBSD sys/sysctl.h requires sys/param.h (http://lists.gnu.org/archive/html/bug-autoconf/2005-08/msg00015.html) I've found this workaround, but I won't test it until tomorrow probably: http://lists.gnu.org/archive/html/bug-autoconf/2005-08/msg00015.html I also don't know exactly how to apply the above workaround (or any other) in the MonetDB Makefile(s ?). On Wed, May 4, 2011 at 6:27 PM, Michael Sioutis <sioutis(a)di.uoa.gr> wrote: > It seems that in OpenBSD sys/sysctl.h requires sys/param.h > (http://lists.gnu.org/archive/html/bug-autoconf/2005-08/msg00015.html) > I've found this workaround, but I won't test it until tomorrow > probably: http://lists.gnu.org/archive/html/bug-autoconf/2005-08/msg00015.html > > I also don't know exactly how to apply the above workaround (or any > other) in the MonetDB Makefile(s ?). > > On Wed, May 4, 2011 at 6:20 PM, Michael Sioutis <papito.dit(a)gmail.com> wrote: >> I can't find any of HAVE_SYS_SYSCTL_H and KERN_PROC_PATHNAME defined >> in my config.log. >> Also, I have a warning about sysctl.h, which is in the >> configuration.log I posted earlier (my first post). >> >> The important part is this: >> configure: WARNING: sys/sysctl.h: present but cannot be compiled >> configure: WARNING: sys/sysctl.h: check for missing prerequisite headers? >> configure: WARNING: sys/sysctl.h: see the Autoconf documentation >> configure: WARNING: sys/sysctl.h: section "Present But Cannot Be Compiled" >> configure: WARNING: sys/sysctl.h: proceeding with the compiler's result >> configure: WARNING: ## ------------------------------- ## >> configure: WARNING: ## Report this to info(a)monetdb.org ## >> configure: WARNING: ## ------------------------------- ## >> >> On Wed, May 4, 2011 at 6:06 PM, Fabian Groffen <Fabian.Groffen(a)cwi.nl> wrote: >>> Thanks for the summarisation! >>> >>> On 04-05-2011 17:56:54 +0300, Michael Sioutis wrote: >>>> To sum it up.. >>>> >>>> add >>>> #include <pthread.h> >>>> #include <pthread_np.h> >>> >>> hmm, do you really need pthread_np.h? That seems like a non-standard >>> header to me. >>> >>>> and... >>>> 2011-05-04 17:44:29 MSG mydb[19795]: !MALException:malInclude:could >>>> not open file: mal_init >>> >>> This indicates some bigger problem... >>> >>>> 2011-05-04 17:44:29 MSG mydb[19795]: #warning: unable to determine binary location, please use --set monet_mod_path=/path/to/... to allow finding modules >>> >>> ... and this causes it. >>> >>> Can you tell me if HAVE_SYS_SYSCTL_H and KERN_PROC_PATHNAME are defined >>> for you? >>> common/utils/mutils.c is apparently unable to find the executable path >>> on OpenBSD. >>> Perhaps KERN_PROC_PATHNAME is not in sys/sysctl.h on OpenBSD? >>> >>> ------------------------------------------------------------------------------ >>> WhatsUp Gold - Download Free Network Management Software >>> The most intuitive, comprehensive, and cost-effective network >>> management toolset available today. Delivers lowest initial >>> acquisition cost and overall TCO of any competing solution. >>> http://p.sf.net/sfu/whatsupgold-sd >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >> >

5 45

[MonetDB-users] Where [on file] does MonetDB persists the dB?
by Me Anonimous 19 May '11

19 May '11

I uploaded a 1M records, 67M file but can't find anything in the directory indicating dB storage... Thanks!

2 1

[MonetDB-users] feature request (srid support )
by George Garmpis 19 May '11

19 May '11

Hi, I try to use geometries in different reference systems but i don't get the expected results. I create a table with geometries and insert 2 points with different SRID. sql>create table geom(id int, g geometry); operation successful sql>insert into geom values(1, GeomFromText('POINT(100 100)', 4326)); operation successful sql>insert into geom values(2, GeomFromText('POINT(100 100)', 27700)); operation successful When i ask for these SRID i get the same (0) for every point. sql>select id, srid(g) from geom; +------+--------+ | id | srid_g | +======+=+ | 2 | 0 | | 1 | 0 | +------+--------+ Also, if i ask whether these points overlap i don't get the right result. sql>select equals(g1.g, g2.g) from geom g1, geom g2 where g1.id=1 and g2.id =2; +-----------------+ | overlaps_g | +========+ | true | +-----------------+ 1 tuple (4.119ms) I have noticed that function wkbFromText (in file geom/monetdb5/geom.mx), which is used in SQL functions GeomFromText, PointFromText e.t.c., returns a wkb datatype (defined in geom/lib/libgeom.mx) which has not any field for srid. Is srid stored somewhere else (e.g. as metadata of a geometry collumn)? Generally, does monetdb supports different reference systems? If no, will this feature be added ? -- George Garbis

1 0

[MonetDB-users] 'mkey.bulk_rotate_xor_hash' exception
by Steve Elder 18 May '11

18 May '11

With a very simple test, I get the 'mkey.bulk_rotate_xor_hash' exception. After this error, the only way I have found to recover is to drop the table, re-create and repopulate it. * System parameters: o MonetDB-11.3.1 o x86-64 ( note: build was missing a shared library that was subsequently ) o Solaris 10. o 96 GB memory * I create a simple table. create table fluid ( fluid bigint not null, userID int not null default 0, relativeUser bigint not null, typeID tinyint default 127, updateTime timestamp(0) default now(), constraint fluid_pk primary key( fluid, userID ), constraint user_fk1 foreign key ( userID ) references fuser ( userID ) ); create index rel_user_idx1 on fluid ( relativeUser ); create index userID_idx2 on fluid ( userID ); create index rel_user_user_idx1 on fluid ( relativeUser, userID ); create index update_idx1 on fluid ( updateTime ); * Populate it with records ( Note: the number of records does not seem to matter ). * Delete records. It does not appear to matter how many records I delete. Deleting 1 record or deleting all records seems to have the same effect. * Drop the primary key * Do "set optimizer='nov2009_pipe';" * Add the primary key. I always get the mkey.bulk_rotate_xor_hash exception. Here is a snapshot including a trace of a successful 'alter table drop constraint ...' command after NO records have been deleted. When I delete 1 ( or more records ), I get the exception with no trace output. sql>alter table fluid drop constraint fluid_pk; operation successful sql>set optimizer='nov2009_pipe'; sql>trace alter table fluid add constraint fluid_pk primary key( fluid, userID ); operation successful +------+--------------------------------------------------------------------------------------------------------------+ | tick | stmt | : s : : +======+==============================================================================================================+ | 7 | _2 := sql.mvc(); | | 451 | sql.catalog(29,"FLDB",42302696,0); | | 32 | _16:bat[:oid,:lng] <tmp_4741501>[52] := sql.bind(_2=48362568,"FLDB","fluid","fluid",0); | | 22 | _13:bat[:oid,:int] <tmp_4741500>[52] := sql.bind(_2=48362568,"FLDB","fluid","userID",0); | | 15 | _10:bat[:oid,:oid] <tmp_4741364>[0] := sql.bind_dbat(_2=48362568,"FLDB","fluid",1); | | 10 | _12<tmpr_4741364>[0] := bat.reverse(_10=nil:bat[:oid,:oid]); | | 20 | _15<tmp_4741350>[52] := algebra.kdifference(_13=nil:bat[:oid,:int],_12=<tmpr_4741364>[0]); | | 22 | _17<tmp_4741012>[52] := algebra.kdifference(_16=nil:bat[:oid,:lng],_12=nil); | | 11 | _19:bat[:oid,:wrd] <tmp_4741265>[52] := batcalc.hash(_17=<tmp_4741012>[52]); | | 29 | _22<tmp_4741636>[52] := bat.mirror(_17=<tmp_4741012>[52]); | | 16 | _21:bat[:oid,:wrd] <tmp_4741417>[52] := mkey.bulk_rotate_xor_hash(_19=nil:bat[:oid,:wrd],22,_15=<tmp_4741350 | : : >[52]); : | 8 | _23<tmpr_4741636>[52] := bat.reverse(_22=nil); | | 52 | _24<tmp_4741265>[52] := algebra.join(_23=nil,_21=nil:bat[:oid,:wrd]); | | 15 | (ext60<tmp_4741417>[52],grp58<tmp_4741636>[52]) := group.new(_24=<tmp_4741265>[52]); | | 19 | (ext63<tmp_4741417>[52],grp61<tmp_4741636>[52]) := group.derive(ext60=nil,grp58=nil,_17=nil); | | 10 | (ext66<tmp_4741417>[52],grp64<tmp_4741636>[52]) := group.done(ext63=nil,grp61=nil,_15=nil); | | 9 | _31<tmp_4740027>[52] := bat.mirror(ext66=nil); | | 10 | _32:bat[:oid,:wrd] <tmp_4741417>[52] := aggr.count(grp64=nil,grp64=nil,_31=nil); | | 17 | return sum := aggr.sum(b=<tmp_4741417>[52],true); | | 44 | _33:wrd := aggr.sum(_32=<tmp_4741417>:bat[:oid,:wrd][52]); | | 5 | _34 := calc.isnil(_33=52:wrd); | | 7 | _37 := aggr.count(_32=nil:bat[:oid,:wrd]); | | 6 | _36 := calc.ifthenelse(_34=false,0:wrd,_33=52:wrd); | | 5 | _38 := calc.!=(_37=52:wrd,_36=52:wrd); | | 2827 | barrier _53 := language.dataflow(); | | 6 | sql.assert(_38=false,"UPDATE: PRIMARY KEY constraint 'fluid.fluid_pk' violated"); | | 27 | sql.update(_2=48362568,"FLDB","fluid","fluid_pk",_24=nil); | | 5 | sql.exportOperation(""); | | 3724 | user.s1_1(); | +------+--------------------------------------------------------------------------------------------------------------+ 29 tuples (50.262ms) sql>set optimizer='default_pipe'; sql>alter table fluid drop constraint fluid_pk; operation successful sql>delete from fluid where relativeUser = 11111; 1 affected row (1.557ms) sql>trace alter table fluid add constraint fluid_pk primary key( fluid, userID ); MALException:mkey.bulk_rotate_xor_hash:operation failed Operands not synced on head I need to periodically do bulk loads ( 'copy into' ), but also need to be able to periodically delete records from the same table. Is there a resolution to this problem? Thanks Steve

2 4

[MonetDB-users] mclient credentials after install?
by Philippe Hanrigou 17 May '11

17 May '11

Hi, I am trying to get started with MonetDB in order to assess its response time for an in-house dataset. I managed to configure/make/install it on my MacBook Pro (I have joined the custom Portfile I used for this purpose at the end of this email in case it matters). Nevertheless, when I go through the "Getting started" instructions, mclient is asking for credentials that I never had a chance to explicitly set: ph7:~$ monetdbd start ph7:~$ monetdbd get all property value hostname Solaris.local dbfarm /opt/local/var/monetdb5/dbfarm mserver /opt/local/bin/mserver5 logfile merovingian.log pidfile merovingian.pid sockdir /tmp port 50000 controlport 50001 discoveryport 50000 exittimeout 60 forward proxy discoveryttl 600 passphrase <unknown> mapisock /tmp/.s.monetdb.50000 controlsock /tmp/.s.merovingian.50001 ph7:~$ monetdb create demo created database in maintenance mode: demo ph7:~$ monetdb status name state uptime health last crash demo locked ph7:~$ monetdb release demo taken database out of maintenance mode: demo ph7:~$ monetdb status name state uptime health last crash demo stopped ph7:~$ mclient -d demo user(ph7): password: InvalidCredentialsException:checkCredentials:invalid credentials for user 'ph7' Which credentials I am supposed to use to launch mclient? I am willing to try any workaround as, at this point, I am not concerned with security at all. Thanks in advance, - Philippe --------------------------------------------------------- # ph7's custom Portfile # $Id PortSystem 1.0 name MonetDB version 11.3.1 categories databases platforms darwin maintainers nomaintainer description MonetDB is an open-source database system for high-performance applications in data mining, OLAP, GIS, XML Query, text and multimedia retrieval. MonetDB often achieves a significant speed improvement for SQL and XQuery over other open-source systems. homepage http://www.monetdb.org master_sites http://dev.monetdb.org/downloads/sources/Latest/ distfiles MonetDB-${version}.tar.bz2 use_bzip2 yes checksums md5 f48869c87a56604d1c06f7d8243554f9 \ sha1 03a29a5e00a90d5294443ffb849167a2c108302e \ rmd160 c9bcdff333a60aad42dc8504c458335494e749f1 configure.args --enable-debug=no --enable-assert=no --enable-optimize=yes

1 0

[MonetDB-users] Problem with optimizer ?
by Georgios Garmpis 15 May '11

15 May '11

Hi, Please find attached a query (and corresponding schema) that results in mserver5 using more than 1GB of memory and terminating after 6 mins with this output: MAPI = (monetdb) /tmp/.s.monetdb.50000 ACTION= read_line QUERY = PLAN SELECT DISTINCT h3.obj, CASE WHEN u_ENDPOINT.value IS NOT NULL THEN u_ENDPOINT.value WHEN l_ENDPOINT.value IS NOT NULL THEN l_ENDPOINT.value WHEN ll_ENDPOINT.value IS NOT NULL THEN ll_ENDPOINT.value WHEN lu_ENDPOINT.value IS NOT NULL THEN lu_ENDPOINT.value END, NULL FROM triples t0 INNER JOIN triples h1 ON (h1.pred = 21 AND h1.obj = 42 AND h1.subj = t0.subj) INNER JOIN triples h2 ON (h2.pred = 5 AND h2.subj = t0.subj) INNER JOIN triples h3 ON (h3.pred = 20 AND h3.subj = t0.s ERROR = !Connection terminated Is it a problem of optimizer? Any guidelines to deal with it will be appreciated. I also attached merovigian.log. -- George Garbis

4 10

[MonetDB-users] Repeated mmap/munmap calls
by Steve Elder 13 May '11

13 May '11

We are running the April '11 version of MonetDB on multi-core x86-64 hardware running Solaris 10 OS. We are running on a machine with 96GB of memory. The ulimit for memory shows 'unlimited'. We have an application that we believe should be appropriate for MonetDB fast response-time characteristics. We have a very simple schema that includes the following table: create table tuid ( TUID bigint not null, clientID int not null default 0, mafID bigint not null, addrMatch tinyint default 127, lastUsed timestamp(0) default now(), constraint tuid_pk primary key( TUID, clientID ), constraint client_fk1 foreign key ( clientID ) references client ( clientID ) ); The above table currently contains 13.65 million records ( mostly bulk loaded via 'copy into' ). The primary key was dropped and re-added via appropriate 'alter table' statements. There are other 5 other tables in the schema - all with less than 100 records. All 5 tables have fewer than 5 columns. The total size of the database directory in the dbfarm directory is 1.3 GB. Ultimately, the application could have between 500 million and 1 billion records in the above table. Except for one of the other tables, the number of records will be small ( less than 1000 ). We are running a Java application that uses the JDBC interface to query and update the table. Files of 1-3 million records are processed by the application in a given run. Less than 5% of the records require an update to above table ( no other tables are currently updated or inserted to ). When the application is processing these records, we run 'prstat' against 'mserver5' and see via the 'prstat' command the processing load appears to be distributed fairly evenly across many threads. However, occasionally at some point, one thread appears to be using most of the CPU. During these intervals, 'truss' shows 'mserver5' making repeated calls to mmap and munmap. (The calls to mmap have the same value for the mmap size parameter.) These intervals appear to last for many minutes. During these intervals, query response times slow to a crawl. As a result, the overall throughput of the application is disappointing. Given that we are actually accessing a fairly small number of columns all of which should be in memory ( we believe ), it would appear that MonetDB is not functioning properly. It is our belief that MonetDB should be a good fit for this particular application. I should note that we have 3 different MonetDB databases running on the machine ( thus requiring 3 instances of mserver5 ). However, two of the databases are idle when we run the above application. Given the amount of memory available the size of the database, the number of columns being queries ( and/or updated ), and the amount of memory on the machine, the behavior we are observing does not appear to be consistent with descriptions of MonetDB's expected behavior. We have 3 questions: * Is what we are observing concerning the frequency of mmap and munmap calls typical where the mmap size parameter has the same value? * Why would we be mapping columns into memory so frequently when we are not adding records? * Is it typical to have periods where one thread appears to be handling the entire load? * What should we look for concerning the tuning of the x86-64 platform or MonetDB to address these issues? Thanks Steve

1 0

[MonetDB-users] simple load?
by Zoltan Tomory 11 May '11

11 May '11

Sorry to be a bother, but I wonder how hard it is to load a schema or table from ascii text or an .mdb "access" file. I am using the convenient simple console client that is included. Upon searching the list archives, I found some leads which lead me to panic and reflexively consider using Java's Robot class to perform row by row insert operations from the text file. I'm hopeful that there is another way which is both elegant and understandable to a newbie. Thanks in advance. I have loved using MonetDB to organize my information Zoltan -- View this message in context: http://old.nabble.com/simple-load--tp31587056p31587056.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] Alter Table
by Henning Grosch 11 May '11

11 May '11

Is it somehow possible to to add a column with alter table behind an specific column? Example: ALTER TABLE xxx ADD COLUMN newCol int AFTER oldCol; Henning

2 1

[MonetDB-users] wrong results for group by queries
by Daniel Boesswetter 05 May '11

05 May '11

Hi, we are using MonetDB (Oct-2010) for analyzing a 250G data warehouse and have found at least one "thing" which might be a serious bug. The simplest scenario that I have found to describe it, is as follows: SELECT comp.id as compid, comp.name, affnets.name as affid FROM toc INNER JOIN app ON toc.application_id=app.id INNER JOIN comp ON app.company_id = comp.id INNER JOIN affnets ON toc.affiliate_network_id=affnets.id GROUP BY compid, comp.name, affid This query is supposed to return all unique company/affnet pairs which occur in the fact table "toc". However, it returns only the unique company-names, each with a random (the first?) affnet. (Of course, this could be accomplished with DISTINCT, but our aim to to aggregate other fields). If I change the query to select app.company_id as compid, comp.name, affnets.name as affid ... or select comp.id as compid, comp.name, affnets.name as affid ... i.e. semantically the same but only with fields from 2 directly joined tables in the GROUP BY, then the result is correct. Question: Is this a bug or are we missing something? Thank you in advance! Daniel

4 10

[MonetDB-users] Records deleted still appear in: select count(*) from table
by Osmin Castillo 05 May '11

05 May '11

Hi guys: After delete some records from a table using a "delete from table where field=value" criteria, why the "select cont(0) from table" still shows the same records count? Maybe the records are only "marked" and not really removed from the table? Thanks in advance Osmin

3 8

[MonetDB-users] System-wide installation as of April11 release
by brien colwell 05 May '11

05 May '11

Hi all, I recently upgraded one machine to the April11 release of MonetDB. In the April11 release, what is the recommended way to start monetdb with the system, as it used to be done with the rc*.d scripts? For me, maybe a bug, the /etc/init.d/monetdb-sql script is broken (Starting MonetDB SQL server: unable to open 'merovingian.log': Permission denied ... and other missing file, permission errors). Currently I can start the server using 'sudo monetdbd', but of course I want the system-wide installation to start with the system. Thanks, Brien

3 2

[MonetDB-users] Inserted table in triggers
by Vassilis Papakonstantinou 04 May '11

04 May '11

Hello, I want to use a trigger to get the last inserted row of a table so i can insert some fields of this row to another table. Is there something like the INSERTED table in sql server so i can get the row? Thanks Vassilis

1 0

[MonetDB-users] MonetDB-11.3.1 on OpenBSD 4.9 i386 fails during make
by Michael Sioutis 04 May '11

04 May '11

Hello, I didn't now whether I should send a bug report about this, because I am not sure whether the problem occurs because of a bug in the MonetDB source code or my current unfamiliarity with OpenBSD. I downloaded the April2011 tarball from here: http://dev.monetdb.org/downloads/sources/Apr2011/ I installed all required libraries/programs and configuration went fine (except some warning parts which I will attach). The configurations is as follows: MonetDB is configured as follows: * Compilation specifics: Host: i386-unknown-openbsd4.9 Compiler: gcc CFLAGS: -g -O2 $(X_CFLAGS) -Wno-declaration-after-statement X_CFLAGS: LDFLAGS: * Enabled/disabled components: common is enabled monetdb5 is enabled sql is enabled geom is disabled (geos library required for geom module) fits is enabled rdf is disabled (by default) unixodbc is enabled java is enabled jdbc is enabled control is enabled testing is enabled * Important options: OID size: 32 bits When I run "make" is crashes giving me these erros (they are also attached): gcc: ../../../common/options/monet_options.c: No such file or directory gcc: no input files *** Error code 1 ... *** Error code 1 ... Mike

2 9

[MonetDB-users] Fwd: wkb representation of a geometry
by Georgios Garmpis 04 May '11

04 May '11

Hi, I need to get the Well Known Binary representation of a geometry but I didn't find any function in documentation to do this. Any ideas? -- George Garbis

4 11

[MonetDB-users] How does MonetDB work
by anjenny 04 May '11

04 May '11

Hey, my name is anjenny and im doin a little research on MonetDB for a project. My Question is How does MonetDB actually work? Greetings anjenny -- View this message in context: http://old.nabble.com/How-does-MonetDB-work-tp31440282p31440282.html Sent from the monetdb-users mailing list archive at Nabble.com.

5 7

[MonetDB-users] Time Interval Overlap
by brien colwell 29 Apr '11

29 Apr '11

Hi all, I have a table that stores a time interval as two timestamps, and want to construct the most efficient query to check whether two intervals in the table overlap. The naive way, using two range conditions [1], seems to choke for large tables. The M5 SQL manual mentions an OVERLAPS operator that, as far as I've read [2], seems to use an optimized interval join algorithm. Can anyone clarify: 1. How to convert a start timestamp and an end timestamp into an interval type 2. How to use the OVERLAPS operator Many thanks. Brien [0] create table intervals as ( id bigint, series_id bigint, start_time timestamp, end_time timestamp ) [1] select a.id as a_id b.id as b_id from intervals a inner join intervals b on b.series_id = a.series_id and b.id < a.id and b.start_time < a.end_time and a.start_time < b.end_time [2] http://monetdb.cwi.nl/XQuery/Documentation/StandOff-Extension.html

1 0

[MonetDB-users] rowcount in monetdb
by Vassilis Papakonstantinou 29 Apr '11

29 Apr '11

Hello all, Is there a way to take the number of rows affected by the last statement just like @@ROWCOUNT in SQL Server or SQL%ROWCOUNT in Oracle? Thanks, Vassilis

3 8

Re: [MonetDB-users] Discontinuation of MonetDB XQuery
by Peter Boncz 29 Apr '11

29 Apr '11

Hi Ryad, There have been plans to port XQuery to MonetDB5, but there has been lack of time and manpower to make this possible. This shortage of resources might change if the database research community regains interest in XQuery in a future. At CWI, a research project, engineering effort must always be justified by ongoing research projects that profit from it. For the moment, it is hard to predict when there will be XML database research projects again, but I would not rule it out. For the moment, you can also just continue to use MonetDB/XQuery. It had not changed much anyway in the past two years, so the change is not dramatic. The software is still available in open source in this April branch of the MonetDB software. So one can still download it in binary, check out the source, change it, compile it and even redistribute it according to the license. CWI for the moment just will not create new releases for it. Peter From: Ryad Ben-El-Kezadri [mailto:ryad.bek@gmail.com] Sent: Friday, April 29, 2011 9:49 AM To: monetdb-users(a)lists.sourceforge.net Subject: [MonetDB-users] Discontinuation of MonetDB XQuery Hello I am a MonetDB XQuery supporter (http://permalink.gmane.org/gmane.comp.db.monetdb.user/3824) and I am sad that it is discontinued. I was very enthusiastic about the software so I was rather excepting an upgrade to MonetDB5. However I am confident that it will be easy to turn my application to MonetDB5. As a basic XQuery user, my questions are threefold: 1)Are there some explanations for this change (flaws in the design of XQuery 2.0, lack of users, performance limitation)? 2)Is MonetDB5 SQL able to store GB of data (as MonetDB XQuery)? Will we observe performance gains for (time ordered) one dimensional data? 3)Is there an equivalent to the XRPC API? Warm regards Ryad

1 0

[MonetDB-users] Discontinuation of MonetDB XQuery
by Ryad Ben-El-Kezadri 29 Apr '11

29 Apr '11

Hello I am a MonetDB XQuery supporter ( http://permalink.gmane.org/gmane.comp.db.monetdb.user/3824) and I am sad that it is discontinued. I was very enthusiastic about the software so I was rather excepting an upgrade to MonetDB5. However I am confident that it will be easy to turn my application to MonetDB5. As a basic XQuery user, my questions are threefold: 1)Are there some explanations for this change (flaws in the design of XQuery 2.0, lack of users, performance limitation)? 2)Is MonetDB5 SQL able to store GB of data (as MonetDB XQuery)? Will we observe performance gains for (time ordered) one dimensional data? 3)Is there an equivalent to the XRPC API? Warm regards Ryad

2 1

[MonetDB-users] Announcement: New Apr2011 feature release of MonetDB suite
by Sjoerd Mullender 28 Apr '11

28 Apr '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2011 feature release of the MonetDB suite of programs. As announced one month ago to the day (March 28, 2011), this release marks the start of a new era: development of MonetDB4 and MonetDB/XQuery has ceased. Development of the Pathfinder compiler continues unabated at the University of Tübingen [1], but there is no longer a link being maintained with MonetDB. For details on this release, please see the release notes at <http://monetdb.cwi.nl/Development/Releases/Apr2011/>. [1] http://www.pathfinder-xquery.org/ -- Sjoerd Mullender

1 0

[MonetDB-users] invisible join in Monetdb
by Souman Mandal 26 Apr '11

26 Apr '11

Hi, I am a M-tech student at IIT Bombay. Currently I am doing a project to compare the performance of a column store database with implementation of invisible join and without implementation of invisible join. Now I want to use MonetDB for my project. But I have couple of doubts: > Does invisible join is already implemented in MonetDB? > Where can I get the documentation of the source code? > Though I am running mserver5 process, still if I try to execute the command 'monetdb create test' I am getting a error "monetdb: cannot perform: MonetDB Database Server (merovingian) is not running". How to solve this problem. I really need some help. Waiting for some reply. Thanks & Regards, Souman Mandal

2 1

[MonetDB-users] values of type char(n) not always blank padded?
by Ying Zhang 24 Apr '11

24 Apr '11

Hi, I'm wondering if the following results of querying values of the type char(n) (here n = 50), is expected or an error, since char(n) defines a fixed-length, blank padded string: sql>create table a2 (c1 char(5), c2 char(50)); operation successful sql>insert into a2 values ('abc', 'abc'), ('defg', 'defg'); 2 affected rows (2.848ms) sql>select c1, c2 from a2; +-------+----------------------------------------------------+ | c1 | c2 | +=======+====================================================+ | abc | abc | | defg | defg | +-------+----------------------------------------------------+ 2 tuples (1.655ms) sql>select '#' || c1 || '#' || c2 || '#' from a2; +------------------------------------------------------------+ | concat_single_value | +============================================================+ | #abc#abc# | | #defg#defg# | +------------------------------------------------------------+ 2 tuples (2.242ms) LQL The result of the first SELECT shows that c1 and c2 are blank padded (according to the SQL standard). The result of the second SELECT looks a bit strange, are the padding space characters removed or added after the last '#'? Is this the expected result or an error? Thanks, Jennie

2 1

[MonetDB-users] Using COPY with php_mapi and php_monetdb.php
by Henning Grosch 15 Apr '11

15 Apr '11

Hi, at the moment I use the odbc_drive to connect to my monetdb database. In one script I use COPY to get lots of data in my database. Everything works fine. Yesterday is downloaded the latest source files. There I found the php_mapi.inc and the php_monetdb.php with these two files I can now use monetdb in php without odbc. Now I am trying to use COPY and what happens is I get an error that the remote address canceled the socket connection. This only seems to happen with copy anything else works as excpected. So is it possible to use copy with a socket connection? Thanks. Henning

1 0