- users-list - monetdb.org

[MonetDB-users] mclient credentials after install?
by Philippe Hanrigou 17 May '11

17 May '11

Hi, I am trying to get started with MonetDB in order to assess its response time for an in-house dataset. I managed to configure/make/install it on my MacBook Pro (I have joined the custom Portfile I used for this purpose at the end of this email in case it matters). Nevertheless, when I go through the "Getting started" instructions, mclient is asking for credentials that I never had a chance to explicitly set: ph7:~$ monetdbd start ph7:~$ monetdbd get all property value hostname Solaris.local dbfarm /opt/local/var/monetdb5/dbfarm mserver /opt/local/bin/mserver5 logfile merovingian.log pidfile merovingian.pid sockdir /tmp port 50000 controlport 50001 discoveryport 50000 exittimeout 60 forward proxy discoveryttl 600 passphrase <unknown> mapisock /tmp/.s.monetdb.50000 controlsock /tmp/.s.merovingian.50001 ph7:~$ monetdb create demo created database in maintenance mode: demo ph7:~$ monetdb status name state uptime health last crash demo locked ph7:~$ monetdb release demo taken database out of maintenance mode: demo ph7:~$ monetdb status name state uptime health last crash demo stopped ph7:~$ mclient -d demo user(ph7): password: InvalidCredentialsException:checkCredentials:invalid credentials for user 'ph7' Which credentials I am supposed to use to launch mclient? I am willing to try any workaround as, at this point, I am not concerned with security at all. Thanks in advance, - Philippe --------------------------------------------------------- # ph7's custom Portfile # $Id PortSystem 1.0 name MonetDB version 11.3.1 categories databases platforms darwin maintainers nomaintainer description MonetDB is an open-source database system for high-performance applications in data mining, OLAP, GIS, XML Query, text and multimedia retrieval. MonetDB often achieves a significant speed improvement for SQL and XQuery over other open-source systems. homepage http://www.monetdb.org master_sites http://dev.monetdb.org/downloads/sources/Latest/ distfiles MonetDB-${version}.tar.bz2 use_bzip2 yes checksums md5 f48869c87a56604d1c06f7d8243554f9 \ sha1 03a29a5e00a90d5294443ffb849167a2c108302e \ rmd160 c9bcdff333a60aad42dc8504c458335494e749f1 configure.args --enable-debug=no --enable-assert=no --enable-optimize=yes

1 0

[MonetDB-users] Problem with optimizer ?
by Georgios Garmpis 15 May '11

15 May '11

Hi, Please find attached a query (and corresponding schema) that results in mserver5 using more than 1GB of memory and terminating after 6 mins with this output: MAPI = (monetdb) /tmp/.s.monetdb.50000 ACTION= read_line QUERY = PLAN SELECT DISTINCT h3.obj, CASE WHEN u_ENDPOINT.value IS NOT NULL THEN u_ENDPOINT.value WHEN l_ENDPOINT.value IS NOT NULL THEN l_ENDPOINT.value WHEN ll_ENDPOINT.value IS NOT NULL THEN ll_ENDPOINT.value WHEN lu_ENDPOINT.value IS NOT NULL THEN lu_ENDPOINT.value END, NULL FROM triples t0 INNER JOIN triples h1 ON (h1.pred = 21 AND h1.obj = 42 AND h1.subj = t0.subj) INNER JOIN triples h2 ON (h2.pred = 5 AND h2.subj = t0.subj) INNER JOIN triples h3 ON (h3.pred = 20 AND h3.subj = t0.s ERROR = !Connection terminated Is it a problem of optimizer? Any guidelines to deal with it will be appreciated. I also attached merovigian.log. -- George Garbis

4 10

[MonetDB-users] Repeated mmap/munmap calls
by Steve Elder 13 May '11

13 May '11

We are running the April '11 version of MonetDB on multi-core x86-64 hardware running Solaris 10 OS. We are running on a machine with 96GB of memory. The ulimit for memory shows 'unlimited'. We have an application that we believe should be appropriate for MonetDB fast response-time characteristics. We have a very simple schema that includes the following table: create table tuid ( TUID bigint not null, clientID int not null default 0, mafID bigint not null, addrMatch tinyint default 127, lastUsed timestamp(0) default now(), constraint tuid_pk primary key( TUID, clientID ), constraint client_fk1 foreign key ( clientID ) references client ( clientID ) ); The above table currently contains 13.65 million records ( mostly bulk loaded via 'copy into' ). The primary key was dropped and re-added via appropriate 'alter table' statements. There are other 5 other tables in the schema - all with less than 100 records. All 5 tables have fewer than 5 columns. The total size of the database directory in the dbfarm directory is 1.3 GB. Ultimately, the application could have between 500 million and 1 billion records in the above table. Except for one of the other tables, the number of records will be small ( less than 1000 ). We are running a Java application that uses the JDBC interface to query and update the table. Files of 1-3 million records are processed by the application in a given run. Less than 5% of the records require an update to above table ( no other tables are currently updated or inserted to ). When the application is processing these records, we run 'prstat' against 'mserver5' and see via the 'prstat' command the processing load appears to be distributed fairly evenly across many threads. However, occasionally at some point, one thread appears to be using most of the CPU. During these intervals, 'truss' shows 'mserver5' making repeated calls to mmap and munmap. (The calls to mmap have the same value for the mmap size parameter.) These intervals appear to last for many minutes. During these intervals, query response times slow to a crawl. As a result, the overall throughput of the application is disappointing. Given that we are actually accessing a fairly small number of columns all of which should be in memory ( we believe ), it would appear that MonetDB is not functioning properly. It is our belief that MonetDB should be a good fit for this particular application. I should note that we have 3 different MonetDB databases running on the machine ( thus requiring 3 instances of mserver5 ). However, two of the databases are idle when we run the above application. Given the amount of memory available the size of the database, the number of columns being queries ( and/or updated ), and the amount of memory on the machine, the behavior we are observing does not appear to be consistent with descriptions of MonetDB's expected behavior. We have 3 questions: * Is what we are observing concerning the frequency of mmap and munmap calls typical where the mmap size parameter has the same value? * Why would we be mapping columns into memory so frequently when we are not adding records? * Is it typical to have periods where one thread appears to be handling the entire load? * What should we look for concerning the tuning of the x86-64 platform or MonetDB to address these issues? Thanks Steve

1 0

[MonetDB-users] simple load?
by Zoltan Tomory 11 May '11

11 May '11

Sorry to be a bother, but I wonder how hard it is to load a schema or table from ascii text or an .mdb "access" file. I am using the convenient simple console client that is included. Upon searching the list archives, I found some leads which lead me to panic and reflexively consider using Java's Robot class to perform row by row insert operations from the text file. I'm hopeful that there is another way which is both elegant and understandable to a newbie. Thanks in advance. I have loved using MonetDB to organize my information Zoltan -- View this message in context: http://old.nabble.com/simple-load--tp31587056p31587056.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] Alter Table
by Henning Grosch 11 May '11

11 May '11

Is it somehow possible to to add a column with alter table behind an specific column? Example: ALTER TABLE xxx ADD COLUMN newCol int AFTER oldCol; Henning

2 1

[MonetDB-users] wrong results for group by queries
by Daniel Boesswetter 05 May '11

05 May '11

Hi, we are using MonetDB (Oct-2010) for analyzing a 250G data warehouse and have found at least one "thing" which might be a serious bug. The simplest scenario that I have found to describe it, is as follows: SELECT comp.id as compid, comp.name, affnets.name as affid FROM toc INNER JOIN app ON toc.application_id=app.id INNER JOIN comp ON app.company_id = comp.id INNER JOIN affnets ON toc.affiliate_network_id=affnets.id GROUP BY compid, comp.name, affid This query is supposed to return all unique company/affnet pairs which occur in the fact table "toc". However, it returns only the unique company-names, each with a random (the first?) affnet. (Of course, this could be accomplished with DISTINCT, but our aim to to aggregate other fields). If I change the query to select app.company_id as compid, comp.name, affnets.name as affid ... or select comp.id as compid, comp.name, affnets.name as affid ... i.e. semantically the same but only with fields from 2 directly joined tables in the GROUP BY, then the result is correct. Question: Is this a bug or are we missing something? Thank you in advance! Daniel

4 10

[MonetDB-users] Records deleted still appear in: select count(*) from table
by Osmin Castillo 05 May '11

05 May '11

Hi guys: After delete some records from a table using a "delete from table where field=value" criteria, why the "select cont(0) from table" still shows the same records count? Maybe the records are only "marked" and not really removed from the table? Thanks in advance Osmin

3 8

[MonetDB-users] System-wide installation as of April11 release
by brien colwell 05 May '11

05 May '11

Hi all, I recently upgraded one machine to the April11 release of MonetDB. In the April11 release, what is the recommended way to start monetdb with the system, as it used to be done with the rc*.d scripts? For me, maybe a bug, the /etc/init.d/monetdb-sql script is broken (Starting MonetDB SQL server: unable to open 'merovingian.log': Permission denied ... and other missing file, permission errors). Currently I can start the server using 'sudo monetdbd', but of course I want the system-wide installation to start with the system. Thanks, Brien

3 2

[MonetDB-users] Inserted table in triggers
by Vassilis Papakonstantinou 04 May '11

04 May '11

Hello, I want to use a trigger to get the last inserted row of a table so i can insert some fields of this row to another table. Is there something like the INSERTED table in sql server so i can get the row? Thanks Vassilis

1 0

[MonetDB-users] MonetDB-11.3.1 on OpenBSD 4.9 i386 fails during make
by Michael Sioutis 04 May '11

04 May '11

Hello, I didn't now whether I should send a bug report about this, because I am not sure whether the problem occurs because of a bug in the MonetDB source code or my current unfamiliarity with OpenBSD. I downloaded the April2011 tarball from here: http://dev.monetdb.org/downloads/sources/Apr2011/ I installed all required libraries/programs and configuration went fine (except some warning parts which I will attach). The configurations is as follows: MonetDB is configured as follows: * Compilation specifics: Host: i386-unknown-openbsd4.9 Compiler: gcc CFLAGS: -g -O2 $(X_CFLAGS) -Wno-declaration-after-statement X_CFLAGS: LDFLAGS: * Enabled/disabled components: common is enabled monetdb5 is enabled sql is enabled geom is disabled (geos library required for geom module) fits is enabled rdf is disabled (by default) unixodbc is enabled java is enabled jdbc is enabled control is enabled testing is enabled * Important options: OID size: 32 bits When I run "make" is crashes giving me these erros (they are also attached): gcc: ../../../common/options/monet_options.c: No such file or directory gcc: no input files *** Error code 1 ... *** Error code 1 ... Mike

2 9