- users-list - monetdb.org

new gdk bug (on windows) that might've been introduced by Feb2013-SP4
by Anthony Damico 30 Sep '13

30 Sep '13

if you run this R code (with MonetDB.R), you will automatically load a series of brazilian census bureau data sets onto your computer. after all tables have been downloaded and imported, the R script will run a very simple CREATE TABLE command that will cause mserver to shut down when it should not. library(downloader) setwd( "C:/My Directory/CENSO/" ) source_url( " https://raw.github.com/ajdamico/usgsd/master/Censo%20Demografico/download%2…" , prompt = FALSE , echo = TRUE ) after mserver.exe crashes, even if i re-load mserver.exe manually and then connect to it with mclient.exe, and run this block of code: create table c10_dom_pre_fpc as (SELECT * FROM ac_dom) UNION ALL (SELECT * FROM al_dom) UNION ALL (SELECT * FROM am_dom) UNION ALL (SELECT * FROM ap_dom) UNION ALL (SELECT * FROM ba_dom) UNION ALL (SELECT * FROM ce_dom) UNION ALL (SELECT * FROM df_dom) UNION ALL (SELECT * FROM es_dom) UNION ALL (SELECT * FROM go_dom) UNION ALL (SELECT * FROM ma_dom) UNION ALL (SELECT * FROM mg_dom) UNION ALL (SELECT * FROM ms_dom) UNION ALL (SELECT * FROM mt_dom) UNION ALL (SELECT * FROM pa_dom) UNION ALL (SELECT * FROM pb_dom) UNION ALL (SELECT * FROM pe_dom) UNION ALL (SELECT * FROM pi_dom) UNION ALL (SELECT * FROM pr_dom) UNION ALL (SELECT * FROM rj_dom) UNION ALL (SELECT * FROM rn_dom) UNION ALL (SELECT * FROM ro_dom) UNION ALL (SELECT * FROM rr_dom) UNION ALL (SELECT * FROM rs_dom) UNION ALL (SELECT * FROM sc_dom) UNION ALL (SELECT * FROM se_dom) UNION ALL (SELECT * FROM sp1_dom) UNION ALL (SELECT * FROM sp2_rm_dom) UNION ALL (SELECT * FROM to_dom ) WITH DATA; the exact same crash occurs, so this is a new mserver.exe bug not a MonetDB.R bug. should i file a bug report? thanks!!! in mserver---- # MonetDB 5 server v11.15.15 "Feb2013-SP4" # Serving database 'censo_demografico', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 7.860 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50011/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded >= .\..\..\gdk\gdk_posix.c:800: MT_mremap(bat\02\65\26501.tail,468c50000,1889785 61024,755914244096): GDKextend() failed in mclient---- C:\Windows\System32>cd .. C:\Windows>cd .. C:\>cd "Program Files" C:\Program Files>cd MonetDB C:\Program Files\MonetDB>mclient censo_demografico -p 50011 'mclient' is not recognized as an internal or external command, operable program or batch file. C:\Program Files\MonetDB>mclient 'mclient' is not recognized as an internal or external command, operable program or batch file. C:\Program Files\MonetDB>ls 'ls' is not recognized as an internal or external command, operable program or batch file. C:\Program Files\MonetDB>dir Volume in drive C has no label. Volume Serial Number is 882E-FD8A Directory of C:\Program Files\MonetDB 09/06/2013 04:34 PM <DIR> . 09/06/2013 04:34 PM <DIR> .. 09/27/2013 12:27 PM <DIR> MonetDB5 0 File(s) 0 bytes 3 Dir(s) 451,568,218,112 bytes free C:\Program Files\MonetDB>cd MonetDB5 C:\Program Files\MonetDB\MonetDB5>mclient censo_demografico -p 50011 user(win32):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Feb2013-SP4) Database: MonetDB v11.15.15 (Feb2013-SP4), 'censo_demografico' Type \q to quit, \? for a list of available commands auto commit mode: on sql>create table c10_dom_pre_fpc as (SELECT * FROM ac_dom) UNION ALL (SELECT * F ROM al_dom) UNION ALL (SELECT * FROM am_dom) UNION ALL (SELECT * FROM ap_dom) UN ION ALL (SELECT * FROM ba_dom) UNION ALL (SELECT * FROM ce_dom) UNION ALL (SELEC T * FROM df_dom) UNION ALL (SELECT * FROM es_dom) UNION ALL (SELECT * FROM go_do m) UNION ALL (SELECT * FROM ma_dom) UNION ALL (SELECT * FROM mg_dom) UNION ALL ( SELECT * FROM ms_dom) UNION ALL (SELECT * FROM mt_dom) UNION ALL (SELECT * FROM pa_dom) UNION ALL (SELECT * FROM pb_dom) UNION ALL (SELECT * FROM pe_dom) UNION ALL (SELECT * FROM pi_dom) UNION ALL (SELECT * FROM pr_dom) UNION ALL (SELECT * FROM rj_dom) UNION ALL (SELECT * FROM rn_dom) UNION ALL (SELECT * FROM ro_dom) U NION ALL (SELECT * FROM rr_dom) UNION ALL (SELECT * FROM rs_dom) UNION ALL (SELE CT * FROM sc_dom) UNION ALL (SELECT * FROM se_dom) UNION ALL (SELECT * FROM sp1_ dom) UNION ALL (SELECT * FROM sp2_rm_dom) UNION ALL (SELECT * FROM to_dom ) WITH DATA; GDK reported error. HEAPextend: failed to extend to 755914244096 for 02\65\26501.tail: MT_mremap() f ailed sql> sql>

3 4

MonetDB compile error 11.15.15
by Brian Hood 28 Sep '13

28 Sep '13

Hi Guys, I just trying to update my Arch Linux package and when building came up with this compile error the package passed the sha1sum after download. Hope someone can help. In file included from /usr/include/string.h:25:0, from /var/abs/AUR/monetdb/src/MonetDB-11.15.15/monetdb5/extras/jaql/parser/jaql.y:23: /usr/include/features.h:327:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp] # warning _FORTIFY_SOURCE requires compiling with optimization (-O) ^ y.tab.c: In function 'jaqlparse': y.tab.c:1665:32: error: 'scanner' undeclared (first use in this function) y.tab.c:1665:32: note: each undeclared identifier is reported only once for each function it appears in make[9]: *** [libjaqlp_la-jaql.tab.lo] Error 1 Regards, Brian Hood

1 0

regular expression - substring
by Budulinku Dejmihrasku 27 Sep '13

27 Sep '13

Hi, i am looking for any idea/suggestion how to do something like Oracle has, i.e. REGEXP_SUBSTR(source, pattern[, position [, occurrence[, match_parameter]]]) which returns the desired substring. why i need this: i have vast amount of data to parse - i.e. i need to get all of the many possible [parameter=value] pairs from an url access log in order to feed the dimension tables w/parameter-value from the URL by parsing the string directly in MonetDB when i write a SQL function to do this (or any string parsing as a matter of fact), it is rather slow when parsing millions of records. i know that there is the PCRE.MAL and i already use with some success functions INDEX, MATCH or REPLACE but i was not able to find anything similar to the REGEXP_SUBSTR functionality to get the desired substring in one go would most likely help me to get better performance any suggestion? please? thanks milan

2 1

Stopping queries, deleting tuples and other general use questions
by Ruben Silva 27 Sep '13

27 Sep '13

Hi, after a while using MonetDB some general use questions pops up. I've collected some which answers I think will be of great help for any MonetDB user. * * *#1:* In the monetdb manpage we can read: *“The kill command immediately sends a SIGKILL and should only be used as last resort for a database that doesn't respond any more. Killing a database may result in (partial) data loss.”* * * 1.1 If kill is the last resort, what are the other options? * * *#2:* System monitor (Oct 2013)<http://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/monitor> 2.1 Will this be available in October or is there a new estimate? 2.2 Does the current development version include this feature yet so we can test it? I'm interested mainly in the methods sys.pause(qtag), sys.resume(qtag) and sys.stop(qtag), *specially sys.stop(qtag)*. 2.4 Is there some alternative method already available in the Feb2013 release to achieve this action? *#3:* In the monetdb manpage we can read about the *nclients* property: *“Sets the maximum amount of clients that can connect to this database at the same time. Setting this to a high value is discouraged.” * 3.1 What do you consider as a high value? Fabian Groffen stated in a similar question *"Concurrency is always a problem for MonetDB, since in the ideal case it means the processes are fighting for resources with each other." * 3.2 Does this mean (theoretically) that if I can run Query A in 10s in a system with resources R (Ram, Disk, Processor), with Rx2 I can run Query A and B (equivalent in effort to A) in the same 10s? *#4:* www.monetdb.org/Documentation/Manuals/SQLreference/Transactions: *"WARNING. The tuples being deleted are only marked as such. They do not reduce the table size. It calls for a vacuum cleaning algorithm."* * * I've found the /usr/lib/monetdb5/createdb/20_vacuum.sql script and after some googling I found the following post* * http://mail.monetdb.org/pipermail/users-list/2011-August/005051.html without an answer. 4.1 Does the following sequence frees the space of the deleted rows? sql>delete from sys.transfer_ip ; 651950 affected rows (117.620ms) sql>call vacuum('sys','transfer_ip'); Best regards, Ruben Silva

2 1

Create Sorted View in monetdb
by Khaled Dehdouh 27 Sep '13

27 Sep '13

Please can you help me. I want to create a view sorted according to attributes like Oracle, my query is as follows: *Create view V1 as Select Name, First_Name, Age from **People O**redr by Name**;* but Monetdb shows me the following error "CREATE VIEW: ORDER BY not supported." How should I do to create a sorted view ?.

2 1

Announcement: New Feb2013-SP4 Bugfix release of MonetDB suite
by Sjoerd Mullender 27 Sep '13

27 Sep '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013-SP4 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Feb 2013-SP4 bugfix release Merovingian * monetdb now no longer compresses output when not connected to a terminal, bug #3292 Bug Fixes * 2781: SQL localtime() returns GMT time * 2962: client timezone information needs to be handled also for modifications to timestamp values * 3232: The function weekofyear returns results not according to the iso 8601 standard * 3255: M5 tests inlineFunction & ifthencst fail since changeset 207ed7d7e03a * 3292: monetdb status should print full line * 3307: distinct multicolumn performance and implementation suggestion for count distinct * 3310: Cast value to integer gives wrong result * 3311: strLength assertion on join and concat * 3313: Assertion in rel2bin_project * 3315: Incorrect behavior with subqueries in DELETE * 3317: correlated subselect does not work * 3318: UPDATE does not work with DECLEAREd TABLE in FUNCTION * 3319: UNIQUE column in DECLAREd TABLE in FUNCTION results in segfault * 3329: Function AVG fails during runtime in a procedure * 3330: Corrupt mitosis plan with group-by query * 3332: MEDIAN: no such operator 'median(decimal)' * 3336: DB files not removed if all rows are deleted, even after restart * 3337: Monet ODBC gives wrong schema information * 3339: Segvault in complex function call * 3340: crash on seeming incorrect SQL query * 3341: segfault caused by select rank() on a newly-loaded table * 3342: incorrect automatic cast from decimal to tinyint * 3345: copy binary into fails with the wrong error message * 3346: Deadlock in mal_dataflow's q_dequeue() (SciQL-2 branch) * 3348: SQL: LIKE incorrect with _ * 3349: When selecting everything from a joined result, column contents are incorrect in one of my use cases. * 3351: gzip'ed files cannot be bulkloaded into DB * 3352: Median function with decimal type doesn't work * 3353: Readonly tables call for UNSET * 3354: Introduce query time-out * 3358: Remote client connection stays in CLOSE_WAIT state * 3360: uninitialized memory causing bad output in query plans * 3362: ALTER TABLE operations silently remove previously set READ ONLY property * 3363: a join causes SIGSEGV after creation of Unique Key * 3367: Fully qualified order by column gives "relational query without result" * 3369: \0 Byte appears in MAPI

1 0

Re: Contents of users-list Digest, Vol 13, Issue 20
by Edgar Mejia 25 Sep '13

25 Sep '13

1. I have MonetDB Database Server Toolkit v1.1 (Feb2013-SP3) (i install so: sudo apt-get install monetdb5-sql monetdb-client) shell> monetdbd create /path/to/mydbfarm shell> monetdbd start /path/to/mydbfarm shell> monetdb create voc shell> monetdb release voc 2. Ubuntu 12.04.3 LTS 3. Schema and data: Schema: CREATE TABLE timedimension ( id integer NOT NULL, yeart int DEFAULT NULL, quarter varchar(2) DEFAULT NULL, weekyear int DEFAULT NULL, dayt int DEFAULT NULL, namemonth varchar(10) DEFAULT NULL, nameday varchar(20) DEFAULT NULL, montht int DEFAULT NULL, day_of_week varchar(15) DEFAULT NULL, date date DEFAULT NULL, month2 char(2) DEFAULT NULL ) CREATE TABLE bi_con_cdr_diario ( ano INT , facmesfac VARCHAR(6) , comcodigo INT , succodigo VARCHAR(3) , procodigo INT , pronombre VARCHAR(20) , prycodigo VARCHAR(20) , fecllamada timestamp , clicodigo INT , clinombre VARCHAR(80) , vencodigo INT , vennombre VARCHAR(50) , sercodigo VARCHAR(3) , pryservicio VARCHAR(4) , sertipo VARCHAR(3) , serclase VARCHAR(4) , trdtipotelefono CHAR(10) , cfacodigo INT , cfanombre VARCHAR(30) , trddurllamada INT , minutosreales DOUBLE , minutosenteros INT , minutosfacturados DOUBLE , costo DOUBLE , costohit DOUBLE , valorbruto DOUBLE , valorneto DOUBLE , trdvalorlocalneto DOUBLE , trdvalorlocalimpu DOUBLE , valorlocal DOUBLE , codpaisori INT , nompaisori VARCHAR(80) , codpaisdes INT , nompaisdes VARCHAR(80) , comnombre2 VARCHAR(15) , linprnombre VARCHAR(40) , trdcodciuori BIGINT , trdnomciuori VARCHAR(80) , sernombre VARCHAR(50) , idtimedimension integer , linprcodigo INT , trdcodciudes INT , trdnomciudes VARCHAR(80) ); Timedimension data:20080101,2008,"1",1,1,"Ene","",1,"Martes","2008-01-01","01" bi_con_cdr_diario data:2013,"092013",4,"CTA",4,"EMBRATEL","HITBRAEMB","2013-09-04 00:00:00",3608,"Air Liquide do Brasil Ltda - Canoas",420,"Flavio Eugenio/Caroline Carneiro","E-N","E-N","ddd","PSTN","Fijo",89,"Consumo",459,7.65,10,10,0.66924,0.66924,1.43199992,1.1456000328063967,3.437,0.471,3.9080000000000004,55,"Brasil",55,"Brazil","HitBrasil","HIT VOICE",0,"0","HIT CALLSERVICE EMB NAL",20130904,1,0,"0",\N But the error is when i do group by as (select sum(minutosfacturados),succodigo,montht,comnombre2 from bi_con_cdr_diario as cdr inner join timedimension as td on cdr.idtimedimension=td.id where comcodigo=4 group by succodigo,comnombre2,montht;) and have 10.000.000 of rows. I appreciate very much your help. I believe this is one great product. Best Regards, Edgar Mejia

1 0

Re: Contents of users-list Digest, Vol 13, Issue 20
by Edgar Mejia 25 Sep '13

25 Sep '13

Thank you Jennie Zhang. 1. I have MonetDB Database Server Toolkit v1.1 (Feb2013-SP3) (i install so: sudo apt-get install monetdb5-sql monetdb-client) shell> monetdbd create /path/to/mydbfarm shell> monetdbd start /path/to/mydbfarm shell> monetdb create voc shell> monetdb release voc 2. Ubuntu 12.04.3 LTS 3. Schema view schemadatamonetdb.sql file 4. Data: timedimension view datatimediM.csv bi_con_cdr_diario: cdr.csv But the error is when i do group by as (select sum(minutosfacturados),succodigo,montht,comnombre2 from bi_con_cdr_diario as cdr inner join timedimension as td on cdr.idtimedimension=td.id where comcodigo=4 group by succodigo,comnombre2,montht;) and have 10.000.000 of rows. I appreciate very much your help. I believe this is one great product. Best Regards, Edgar Mejia

1 0

Monetdb crashed as group by
by Edgar Mejia 24 Sep '13

24 Sep '13

select sum(minutosfacturados),succodigo,montht,comnombre2 from bi_con_cdr_diario as cdr inner join timedimension as td on cdr.idtimedimension=td.id where comcodigo=4 group by succodigo,comnombre2,montht; but for example as this query work fine: select sum(minutosfacturados),succodigo,montht from bi_con_cdr_diario as cdr inner join timedimension as td on cdr.idtimedimension=td.id where comcodigo=4 group by succodigo,montht; What i need to do for to fix this? Thanks Edgar M

3 2

Whats coming in upcoming build monetdb release
by Ashish Kumar Singh 21 Sep '13

21 Sep '13

Hi , During my periodic search for "Is a new monetdb build is going to be released soon" , I found one is placed under testing recently. Is there a place where it can be seen and reviewed what kind of features/fixes are expected in upcoming release? That will be a great help for some of the folks like me to plan ahead! Regards, Ashish

1 0

Re: users-list Digest, Vol 13, Issue 16
by Rose Beck 21 Sep '13

21 Sep '13

Thanks a lot for replying :) But its really unclear to me as to how can I load RDF data into monetdb. As I have a 1000 property values and I can not manually create table for each of the property values. TSQL allows something like SELECT * INTO someTable FROM sourceTable. RDF query engines like RDF3X, BitMat (Medha Atre, et.al.) have all employed monetdb. So, if there is some way out can you please help me with it. On Sat, Sep 21, 2013 at 12:00 PM, <users-list-request(a)monetdb.org> wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. Re: How to automate creation of property tables in monetdb? > (Rose Beck) > 2. Re: How to automate creation of property tables in monetdb? > (Martin Kersten) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 21 Sep 2013 11:49:10 +0530 > From: Rose Beck <rosebeck.82(a)gmail.com> > To: users-list(a)monetdb.org > Subject: Re: How to automate creation of property tables in monetdb? > Message-ID: > <CAPx9ze9DR0bmy9joSZewwrSL_UNQGP=HefZLHax= > WZAAfDK3Cw(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > A gentle reminder. Can someone please help me with this? > > > > > On Fri, Sep 20, 2013 at 7:58 PM, Rose Beck <rosebeck.82(a)gmail.com> wrote: > > > I am trying to load data (from myfile, the format of myfile is explained > > below) into monetdb using itsSQL interface. mytable format: > > > > 1 2 3 > > 5 7 8 > > 8 2 3..... > > > > The commands which I am using for performing the same are given below: > > > > sql> create table mytable(subject integer, property integer, object > integer); > > > > sql> copy 10000000 records into mytable from myfile delimiter ' ','\n'; > > > > For each property value (i.e. middle column) I create a separate table: > > > > create table prop1(subject integer, object integer); > > create table prop2(subject integer, object integer); > > create table prop3(subject integer, object integer); ... > > > > and then fill these tables using the following commands: > > > > INSERT into prop1 SELECT subject, object FROM triples_table where > property=prop1 ORDER BY subject, object; > > > > Is there some way to automate this process. i.e. right now I am manually > > creating a separate table for all the distinct properties (using create > > table prop1(subject integer, object integer); ) and then filling this > > table using insert command (INSERT into prop1 SELECT subject, object FROM > > triples_table where property=prop1 ORDER BY subject, object; ). Is there > > some way by which I may automate this procedure. > > > > One of the methods which I know of is to create a C++/python program and > > then call all functions like "create table" by using shell script. But is > > there a faster method to achieve the same. > > > > Also can you please guide as to how can I create and integrate dictionary > > (RDF: id to string mapping) to sql queries? > > <users-list(a)monetdb.org> > > >

2 2

How to automate creation of property tables in monetdb?
by Rose Beck 21 Sep '13

21 Sep '13

I am trying to load data (from myfile, the format of myfile is explained below) into monetdb using itsSQL interface. mytable format: 1 2 3 5 7 8 8 2 3..... The commands which I am using for performing the same are given below: sql> create table mytable(subject integer, property integer, object integer); sql> copy 10000000 records into mytable from myfile delimiter ' ','\n'; For each property value (i.e. middle column) I create a separate table: create table prop1(subject integer, object integer); create table prop2(subject integer, object integer); create table prop3(subject integer, object integer); ... and then fill these tables using the following commands: INSERT into prop1 SELECT subject, object FROM triples_table where property=prop1 ORDER BY subject, object; Is there some way to automate this process. i.e. right now I am manually creating a separate table for all the distinct properties (using create table prop1(subject integer, object integer); ) and then filling this table using insert command (INSERT into prop1 SELECT subject, object FROM triples_table where property=prop1 ORDER BY subject, object; ). Is there some way by which I may automate this procedure. One of the methods which I know of is to create a C++/python program and then call all functions like "create table" by using shell script. But is there a faster method to achieve the same. Also can you please guide as to how can I create and integrate dictionary (RDF: id to string mapping) to sql queries? <users-list(a)monetdb.org>

2 2

Shouldn't multiple identical bind calls be impossible?
by Roberto Cornacchia 19 Sep '13

19 Sep '13

I'd like to ask a simple question before reporting what I suspect to be a bug. On a Feb2013, using default_pipe (same holds for no_mitosis and sequential pipes), no matter what the query and the data schema look like (really, that shouldn't matter here), is it for any reason possible to have a MAL plan with repeated, identical bind operations? Like this: function user.s4_1():void; ... X_40 := sql.bind(X_1,"spinque","dict","idstr",0); ... X_126 := sql.bind(X_1,"spinque","dict","idstr",0); ... X_298 := sql.bind(X_1,"spinque","dict","idstr",0); .. X_384 := sql.bind(X_1,"spinque","dict","idstr",0); ... ... and so on My understanding is: with commonTerms optimizer enabled (I'm just using the standard pipes), there should be no way to see those calls repeated: - they have no side-effects - they do not depend on any user data Why isn't the first bind reused in the rest of the plan? Of course my worries are not much about those calls per se, which are super fast, but about my suspicion that entire subqueries built on top of such binds are replicated unnecessarily. I don't want to focus on the actual query here, because if I am correct (please correct me otherwise), the situation above is already enough to submit a more complete bug report. By the way, another suspicious instruction that I see in my MAL plan is: X_25 := X_24; Shouldn't this be removed by the aliases optimizer? Best, Roberto

3 4

overflow in calculation
by Hassan, Ahmad 19 Sep '13

19 Sep '13

Hi, I am trying latest MonetDB-11.15.11 release on Ubuntu 64bit with TPCH dataset of size 120GB (scale factor 100). I am testing on a system with 512Gb RAM. Out of 22 TPCH queries, query #1 always fail and thrown the following error: overflow in calculation. Command exited with non-zero status 1 113.04 Rest of the queries 2-22 runs fine for 120GB data. If I try with 200GB of data then more queries fail. Any suggestions to work around this problem please? Thanks. Kind Regards, Ahmad

2 1

tomograph not working with remote function?
by Mengmeng Chen 18 Sep '13

18 Sep '13

Hi, We have a MAL function and we are trying to use tomograph to profile the execution of it when it is executed on a remote node. However, the tomograph fails right after the function is executed. We are just wondering if we did something wrong and if tomograph can work with a remote function call. Here's what we want to do: >From node A, we run a MAL function, which will execute a MAL function on both node A and node B using remote.exec() method. Here's what we did and what happened: 1. in A start mclient 2. in A and B, start tomograph "tomograph -d db1 -h localhost" 3. in A, start the driver function, which will call remote.exec for the MAL function on both A and B Then, in A and B, tomograpg exits promptly ubuntu@MC-MoDB1:~->tomograph -d db1 -h localhost -- opened UDP profile stream MC-MoDB1:50010 for localhost Created tomogram 'tomograph' Run: 'gnuplot tomograph.gpl' to create the 'tomograph.pdf' file The memory map is stored in 'tomograph.dat' The trace is saved in 'tomograph.trace' for use with --trace option Looking at the tomograph trace file generated on node B, we have something like: 9 1430 18111 17039 71 71 2 0 0 remote.connect "X_22 := remote.connect(\"mapi:monetdb:// 192.168.4.4:50000/db1\ <http://192.168.4.4:50000/db1%5C>",\"monetdb\",\"monetdb\",\"msql\");", ] 11 1007 10501 9496 70 71 2 0 0 remote.connect "X_24 := remote.connect(\"mapi:monetdb:// 192.168.5.4:50000/db1\ <http://192.168.5.4:50000/db1%5C>",\"monetdb\",\"monetdb\",\"msql\");", ] 69 23408 23605 198 72 72 2 0 0 remote.bintype "remote.bintype();", ] 11 10867 10867 0 71 71 5 0 0 profiler.wait 0.00 0.04 0.75 0.00 0.00 0.00 0.00 0.17 0.17 0.17 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.25 0.01 0.25 0.00 0.00 0.00 0.00 0.00 9 18497 18497 0 71 71 5 0 0 profiler.wait 0.00 0.04 0.75 0.00 0.00 0.00 0.00 0.17 0.17 0.17 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.25 0.01 0.25 0.00 0.00 0.00 0.00 0.00 69 23978 23978 0 72 72 4 0 0 profiler.ping 0.00 0.04 0.75 0.00 0.00 0.00 0.00 0.17 0.17 0.17 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.25 0.01 0.25 0.00 0.00 0.00 0.00 0.00 ================================ on node A, the tomograph trace file looks like: 4 327 453 127 66 66 2 0 0 remote.bintype "remote.bintype();", ] 4 684 684 0 66 66 4 0 0 profiler.ping 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.17 0.15 0.15 0.15 0.15 0.15 0.17 0.15 0.25 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.20 0.17 0.15 0.15 0.15 0.15 0.15 ================================ Any help would be appreciated! Thanks, Mengmeng

2 1

Re: Need information about culstering monetdb
by Hannes Mühleisen 16 Sep '13

16 Sep '13

Hello Luis, first, please direct such general questions to the mailing list. On 09/04/2013 01:01 AM, Luis Moreno wrote: > I want to know which way works a cluster system MonetDB > > This works distributing information in each node or replicated the > information in each node and distributing query processing or duplicated > information in each node and distributes the connexions as a load balancer > > I hope you can help me > > Im testing monetdb with the Pentaho business intelligence suit. To answer your questions, there are multiple ways of running a clustered MonetDB setup. There is some support for clustered operations in monetdbd, see the section on "MULTIPLEX-FUNNELS" [1]. Apart from that, some virtual duct tape is required to set things up, but this nothing to be afraid of and the load balancing scenario is certainly possible. Best, Hannes [1] http://www.monetdb.org/Documentation/monetdbd-man-page

1 0

Bulk Import
by Thomas Johann 15 Sep '13

15 Sep '13

Hi everybody, I have a problem inserting a large amount of data into a monetdb database using bulk import. I'm running the following commands in a loop: connection.execute("COPY *19000000* OFFSET 2 RECORDS INTO XXX FROM '" + csv + "' USING DELIMITERS ';','\n' ;") connection.commit() , where csv is a different csv-file in each round of the loop, but always containing 18000001 rows of data. To be sure that enough memory is allocated I chose 19000000 in the execute command. I now have two questions: 1. Should the number of records (here 19000000) represent the number of lines per .csv-file or the number of lines of the final database (number of csv-files * 18Mio.)??? 2. Can you think of any reason why monetdb would stop reading one specific variable, while continuing to read the others? Let's say my csv has 8 columns and 18000000 rows with no missing values in the raw data. Until Row 16537472 the total data is read-in, but for the following lines variable 3 is missing until line 18000000 while variable 1 as well as 3-8 are perfectly fine. Can this be due to memory or harddisk speed constraints? Why is no error message raised? It would be great if someone could help me. Thanks, Thomas

4 4

Monetdb blocked
by Edgar Mejia 14 Sep '13

14 Sep '13

Hi, please, i have on server as 3.750 mb as ubuntu 12.04, one processor and 2.000 mhz. I install here monetdb and have 12.000,000 millions of rows. I had one problem as one query. For example this query block database: select "timedimension"."namemonth" as "c0", "bi_con_cdr_diario"."comnombre2" as "c1", "bi_con_cdr_diario"."succodigo" as "c2", sum("bi_con_cdr_diario"."minutosfacturados") as "m0" from "timedimension" as "timedimension", "bi_con_cdr_diario" as "bi_con_cdr_diario" where "bi_con_cdr_diario"."idtimedimension" = "timedimension"."id" and "bi_con_cdr_diario"."comcodigo" = '4' group by "timedimension"."namemonth", "bi_con_cdr_diario"."comnombre2", "bi_con_cdr_diario"."succodigo" if i delete (and "bi_con_cdr_diario"."comcodigo" = '4' ) this work fine. Too i im testing very others querys and work fine, but why this not work and block database? Thaks for your help. Edgar M

1 0

Query Problem with LIKE clause
by Hancel Gonzalez 11 Sep '13

11 Sep '13

Hi, finally i installed MonetDB. Now i get the next problem =S When i execute one query, i get error and i dont know how to solve. Please, help me. =( Hancel. PD: Error is: *MAPI = (monetdb) /tmp/.s.monetdb.50000* *ACTION= read_line* *QUERY = select s0.value,s1.value* * from (* * (select f0.subject as r0,f0.object as r1* * from p1 f0,p0 f1* * where f1.subject=f0.subject and f1.object=58758 and ((select value from strings where id = f0.object) like '%ambilateral%'))* * )facts, strings s0, strings s1* * where s0.id=facts.r0 and s1.id=facts.r1;* *ERROR = !Connection terminated*

4 6

Compiling problem
by Hancel Gonzalez 11 Sep '13

11 Sep '13

Hi again, i have a problem installing MonetDB 11.9.7 from source code. When i do the * make* , get the next error in the process =S Can i help me? Many thanks. Regards, Hancel. PD: Error here: *fits.c:54:20: error: fitsio.h: No such file or directory* *fits.c:55:21: error: fitsio2.h: No such file or directory* *fits.c:56:21: error: longnam.h: No such file or directory* *fits.c: In function 'fits2mtype':* *fits.c:126: error: 'TBIT' undeclared (first use in this function)* *fits.c:126: error: (Each undeclared identifier is reported only once* *fits.c:126: error: for each function it appears in.)* *fits.c:127: error: 'TLOGICAL' undeclared (first use in this function)* *fits.c:129: error: 'TBYTE' undeclared (first use in this function)* *fits.c:130: error: 'TSBYTE' undeclared (first use in this function)* *fits.c:132: error: 'TSTRING' undeclared (first use in this function)* *fits.c:134: error: 'TUSHORT' undeclared (first use in this function)* *fits.c:135: error: 'TSHORT' undeclared (first use in this function)* *fits.c:137: error: 'TUINT' undeclared (first use in this function)* *fits.c:138: error: 'TINT' undeclared (first use in this function)* *fits.c:140: error: 'TLONG' undeclared (first use in this function)* *fits.c:141: error: 'TULONG' undeclared (first use in this function)* *fits.c:142: error: 'TLONGLONG' undeclared (first use in this function)* *fits.c:144: error: 'TFLOAT' undeclared (first use in this function)* *fits.c:146: error: 'TDOUBLE' undeclared (first use in this function)* *fits.c:149: error: 'TCOMPLEX' undeclared (first use in this function)* *fits.c:150: error: 'TDBLCOMPLEX' undeclared (first use in this function)* *fits.c: In function 'fits2subtype':* *fits.c:161: error: 'TBIT' undeclared (first use in this function)* *fits.c:162: error: 'TLOGICAL' undeclared (first use in this function)* *fits.c:165: error: 'TBYTE' undeclared (first use in this function)* *fits.c:166: error: 'TSBYTE' undeclared (first use in this function)* *fits.c:169: error: 'TSTRING' undeclared (first use in this function)* *fits.c:172: error: 'TUSHORT' undeclared (first use in this function)* *fits.c:173: error: 'TSHORT' undeclared (first use in this function)* *fits.c:176: error: 'TUINT' undeclared (first use in this function)* *fits.c:177: error: 'TINT' undeclared (first use in this function)* *fits.c:180: error: 'TULONG' undeclared (first use in this function)* *fits.c:181: error: 'TLONG' undeclared (first use in this function)* *fits.c:182: error: 'TLONGLONG' undeclared (first use in this function)* *fits.c:185: error: 'TFLOAT' undeclared (first use in this function)* *fits.c:188: error: 'TDOUBLE' undeclared (first use in this function)* *fits.c:192: error: 'TCOMPLEX' undeclared (first use in this function)* *fits.c:193: error: 'TDBLCOMPLEX' undeclared (first use in this function)* *fits.c: In function 'FITSexportTable':* *fits.c:209: error: 'fitsfile' undeclared (first use in this function)* *fits.c:209: error: 'fptr' undeclared (first use in this function)* *fits.c:303: error: 'USHORT_IMG' undeclared (first use in this function)* *fits.c:305: error: 'READWRITE' undeclared (first use in this function)* *fits.c:308: error: 'BINARY_TBL' undeclared (first use in this function)* *fits.c:332: error: 'TLOGICAL' undeclared (first use in this function)* *fits.c:361: error: 'TBYTE' undeclared (first use in this function)* *fits.c:419: error: 'TSHORT' undeclared (first use in this function)* *fits.c:448: error: 'TINT' undeclared (first use in this function)* *fits.c:477: error: 'TLONG' undeclared (first use in this function)* *fits.c:506: error: 'TFLOAT' undeclared (first use in this function)* *fits.c:535: error: 'TDOUBLE' undeclared (first use in this function)* *fits.c: In function 'FITSdir':* *fits.c:573: error: 'fitsfile' undeclared (first use in this function)* *fits.c:573: error: 'fptr' undeclared (first use in this function)* *fits.c:588: error: 'READONLY' undeclared (first use in this function)* *fits.c: In function 'FITSdirpat':* *fits.c:607: error: 'fitsfile' undeclared (first use in this function)* *fits.c:607: error: 'fptr' undeclared (first use in this function)* *fits.c:631: error: 'READONLY' undeclared (first use in this function)* *fits.c: In function 'FITStest':* *fits.c:646: error: 'fitsfile' undeclared (first use in this function)* *fits.c:646: error: 'fptr' undeclared (first use in this function)* *fits.c:651: error: 'READONLY' undeclared (first use in this function)* *fits.c: In function 'FITSattach':* *fits.c:670: error: 'fitsfile' undeclared (first use in this function)* *fits.c:670: error: 'fptr' undeclared (first use in this function)* *fits.c:684: error: 'READONLY' undeclared (first use in this function)* *fits.c:731: error: 'ASCII_TBL' undeclared (first use in this function)* *fits.c:731: error: 'BINARY_TBL' undeclared (first use in this function)* *fits.c:735: error: 'TSTRING' undeclared (first use in this function)* *fits.c: In function 'FITSloadTable':* *fits.c:834: error: 'fitsfile' undeclared (first use in this function)* *fits.c:834: error: 'fptr' undeclared (first use in this function)* *fits.c:842: error: 'FLEN_VALUE' undeclared (first use in this function)* *fits.c:878: error: 'READONLY' undeclared (first use in this function)* *fits.c:886: error: 'ASCII_TBL' undeclared (first use in this function)* *fits.c:886: error: 'BINARY_TBL' undeclared (first use in this function)* *fits.c:906: error: 'TSTRING' undeclared (first use in this function)* *fits.c:970: error: 'FLEN_ERRMSG' undeclared (first use in this function)* *make[9]: *** [lib_fits_la-fits.lo] Error 1* *make[9]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql/backends/monet5/vaults'* *make[8]: *** [all] Error 2* *make[8]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql/backends/monet5/vaults'* *make[7]: *** [all-recursive] Error 1* *make[7]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql/backends/monet5'* *make[6]: *** [all] Error 2* *make[6]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql/backends/monet5'* *make[5]: *** [all-recursive] Error 1* *make[5]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql/backends'* *make[4]: *** [all] Error 2* *make[4]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql/backends'* *make[3]: *** [all-recursive] Error 1* *make[3]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql'* *make[2]: *** [all] Error 2* *make[2]: Leaving directory `/home/hancel/MonetDB-11.9.7/sql'* *make[1]: *** [all-recursive] Error 1* *make[1]: Leaving directory `/home/hancel/MonetDB-11.9.7'* *make: *** [all] Error 2*

4 11

Setting connection time-out?
by Dennis Pallett 10 Sep '13

10 Sep '13

Hello, Is it possible to set a time-out in MonetDB for connections? On our test instance of MonetDB we are running into connection limits because our developers are starting/stopping our application all day long without properly closing the connections. On production this is not a problem because there the connections are closed properly. Best regards, Dennis Pallett

1 0

GeoSpatial
by Lee Hachadoorian 05 Sep '13

05 Sep '13

I found out about MonetDB a few months ago and am currently giving it a whirl. I am a geographer and am interested in the GeoSpatial extension. I have found limited documentation [1][2], which (I think) has changed since I first saw it a few months ago. I remember something about a shapefile import application that was based on shp2pgsql used by PostGIS, and also something about using OpenJump to visualize the data stored in the MonetDB database, but I cannot find this information any more. I have successfully created a spatial table by copying a CREATE TABLE statement from PostGIS, then using COPY ... TO in PostGIS and COPY ... FROM in MonetDB. But I would like to know if a shp2pgsql equivalent exists for MonetDB, or if there is any work on or plans for OGR functionality. I would also like to know how users visualize MonetDB spatial data in OpenJump or any other GIS, which incidentally also leads into the question of OGR functionality since so many FOSS GISes (e.g. QGIS) use OGR for data loading. Best, --Lee [1] http://www.monetdb.org/Documentation/Extensions/GIS [2] http://www.monetdb.org/Documentation/Extensions/GIS/SpatialTypes -- Lee Hachadoorian Assistant Professor in Geography, Dartmouth College http://freecity.commons.gc.cuny.edu

2 1

Database crash; won't start anymore
by Ralph Holz 30 Aug '13

30 Aug '13

Hi, I've tried to load a good chunk of the Internet census into our MonetDB - about 100M rows. Although the import scripts completed, it seems it shot down Monet: 2013-08-28 04:10:35 MSG merovingian[9857]: database 'census' (10773) was killed by signal SIGBUS 2013-08-28 10:47:59 MSG merovingian[9857]: database 'census' has crashed after start on 2013-08-28 01:14:07, attempting restart, up min/avg/max: 4m/4d/3w, crash average: 1.00 0.10 0.03 (8-7=1) This is then followed by 2013-08-28 10:48:32 MSG census[19161]: # still reading write-ahead log "/mnt/monetdb/census/sql_logs/sql/log.990" (0% done) [...] 2013-08-28 10:53:12 MSG census[19161]: # still reading write-ahead log "/mnt/monetdb/census/sql_logs/sql/log.990" (4% done) 2013-08-28 10:53:34 MSG merovingian[9857]: database 'census' (19161) has crashed (dumped core) Restarting it, yielded the same results for a few times. Once or twice it got to 35% in the WAL 2013-08-28 15:34:27 MSG census[14420]: # still reading write-ahead log "/mnt/monetdb/census/sql_logs/sql/log.990" (35% done) 2013-08-28 15:34:59 MSG merovingian[9857]: database 'census' (14420) has crashed (dumped core) 2013-08-28 15:34:59 ERR merovingian[9857]: client error: database 'census' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-08-28 15:34:59 ERR merovingian[9857]: client error: client (local) sent challenge in incomplete block: 2013-08-28 15:34:59 ERR merovingian[9857]: client error: client (local) sent challenge in incomplete block: 2013-08-28 15:34:59 MSG merovingian[9857]: database 'census' has crashed after start on 2013-08-28 14:43:34, attempting restart, up min/avg/max: 4m/4d/3w, crash average: 1.00 0.90 0.30 (16-7=9) Since then, it won't start (I don't know what caused the client error BTW). Data loss is not a concern, but we'd sure like to know if this can be fixed or if we should start looking at another DB... Thanks for any help, Ralph -- Ralph Holz I8 - Network Architectures and Services Technische Universität München http://www.net.in.tum.de/de/mitarbeiter/holz/ Phone +49.89.289.18043 PGP: A805 D19C E23E 6BBB E0C4 86DC 520E 0C83 69B0 03EF

3 7

Archlinux AUR Package
by Brian Hood 29 Aug '13

29 Aug '13

Hi All, I maintain a package build in the user repository which i update every new release. aur.archlinux.org - just search Regards, Brian Hood

1 0

Clustered Indexes with monetdb
by Ashish Kumar Singh 28 Aug '13

28 Aug '13

HI All, Can somebody point me to relevant information on how to do clustered indexes with monetdb? Regards, Ashish

3 7