- users-list - monetdb.org

Database corruption
by Pedro Salgueiro 04 Nov '13

04 Nov '13

Hi, We are using MonetDB combined with Mondrian for a while ago, and ever since we started put more load on it, with several users using the same instance, we are experiencing some serious data corruption with no reason, leading lost data of some tables and some times loosing the entire database. This weekend we had such an issue, for no apparent reason, the data got corrupted and apparently all data of a table got lost. We have no way of seeing what was being done before the data got corrupted, but since we were not loading data into it, the only queries that could be running were queries generated from Mondrian. In the logs, the only thing that we can find related to this issue is the following: 2013-11-04 11:17:56 MSG merovingian[22802]: database 'normal' (16384) was > killed by signal SIGSEGV which, results in a "connection terminated" After alot of debugging, we found out that issuing a simple query as "select count(*) from table" would cause this issue: sql>select count(*) FROM table; > Connection terminated Under the sql_logs in the database directory, there is a file named "log" with the following contents: 052001 9516 and a file named "log.9516" which is empty. When I do "\d table", I get the following result. It seems the table lost all its columns: sql>\d "table" > CREATE TABLE "sys"."table" ( > ); It's also important to note that this only happened only some tables. The rest of tables are ok. Don't know if it's important, the entire database uses about 390MB, so it's quite small. This seems to be quite a big issue leading to data loss, and worst, corrupted databases preventing its usage, unless the entire database is rebuilt. I have some questions regarding this issue which I never saw them answered, and might be very useful to any users of MonetDB: In a situation like this, is there any way to understand what happened to MonetDB in order to avoid such situation in the future? What are the "sql_logs" for? I see there are some files there, but can't find any meaning of the contents. Could these files be used to recover lost data? In previous mails to the mailing list Martin stated that the system logs all entries added to each table. Where are these logs located? Also, could they be used to recover lost data? If you want, we can provide the entire database so you can debug and try to understand what happened in order to prevent such situations in the future. Best regards, Pedro Salgueiro

1 0

Login Help
by Nitin Gautam 04 Nov '13

04 Nov '13

Hi I need to exectue some batch jobs on Monet which are supposed to run as cron jobs. Since the process runs in background I was hoping to use mclient to execute the queries. But when I use mclient I am prompted to enter a password. I could not see any option to specify the password too as an argument to mclient. Is there something I am missing or is there another way to do what I am trying to do i.e. not use mclient. Thanks Nitin

2 2

Char limitation for ruby-monetdb-sql
by Christian Kleinke 01 Nov '13

01 Nov '13

Hi, I have problem regarding executing a script with ruby-monetdb-sql. The connection works and simple statements are executed as expected. However, when I try to send a large script, nearly 20000 characters, nothing happens. Neither an exception is thrown nor a SQL error is produced by MonetDB. However, I use "\<" with the same 20000 character script, it works as expected. Two questions: 1. Is there a char limitation for ruby-monetdb-sql? 2. Is there a MonetDB-SQL command to load scripts, something like sql>load("my_script.sql")? Thank you very much in advance. Christian

2 1

MonetDB using all availaible disk space
by Ruben Silva 01 Nov '13

01 Nov '13

Hello, I'm performing some bulk inserts into some tables (COPY INTO). Meanwhile some JOINs between those tables are executed and also some DELETEs. During this JOIN's, the disk begins to fill up and only stops after there is no more space available (60GB in my case). Same for RAM. After a (long) while the space gets freed up. The COPY INTO's are done without any significant delay. The files that use that space are some .tail and .head in the database bat folder. If the queries are not concurrent there aren't any problems. The JOIN it's not using any PK or FK. Have some one noticed this before? Should I open a bug report? Ruben Silva * *

2 2

Using GeoSpatial functionality
by Dennis Pallett 30 Oct '13

30 Oct '13

Hello, I want to start using the GeoSpatial features of MonetDB and according to http://www.monetdb.org/Documentation/Extensions/GIS I need to rebuild MonetDB with the --enable-geom argument to do so. Does this mean that all the existing binaries published do not include the GeoSpatial functionality? If so, I need to follow the guide at http://www.monetdb.org/Developers/SourceCompile to build my own version of MonetDB, correct? Best regards, Dennis Pallett

2 2

Re: Are joins really this expensive?
by Bryan Senseman 30 Oct '13

30 Oct '13

Forwarding to user-list as this seems to have gotten lost Actually, I'm running a few different versions of MonetDB on different environments. July2012SP2, Feb2013SP4 & SP5. July2012SP2 on Windows and Linux (Ubuntu); Feb2013SP5 on Windows and Linux, and Feb2013SP4 on Linux. All were installed using pre-built packages/installers. The trace files themselves are from a Linux Feb2013SP4 installation. The Feb2013SP5 install was an inplace install on top of July2012SP2 and wasn't sure if different "data" versions would cloud the trace. To Martins questions (which I never got in my inbox...or junk) yes, date_hour_id is a unique key for the dim_date_hour table, but obviously duplicated within the fact_utilization_hourly table. My question was more generalized, but if a specific solution can be found, fantastic. In general I am seeing joins drastically reduce performance overall. All are running on EC2; m1.xlarge with High I/O Thanx, Bryan On Mon, Oct 28, 2013 at 9:10 AM, Bryan Senseman <monetdb(a)openbi.com> wrote: > Forwarding to user-list as this seems to have gotten lost > > Actually, I'm running a few different versions of MonetDB on different > environments. July2012SP2, Feb2013SP4 & SP5. July2012SP2 on Windows and > Linux (Ubuntu); Feb2013SP5 on Windows and Linux, and Feb2013SP4 on Linux. > All were installed using pre-built packages/installers. The trace files > themselves are from a Linux Feb2013SP4 installation. The Feb2013SP5 > install was an inplace install on top of July2012SP2 and wasn't sure if > different "data" versions would cloud the trace. > > To Martins questions (which I never got in my inbox...or junk) yes, > date_hour_id is a unique key for the dim_date_hour table, but obviously > duplicated within the fact_utilization_hourly table. > > My question was more generalized, but if a specific solution can be found, > fantastic. In general I am seeing joins drastically reduce performance > overall. > > All are running on EC2; m1.xlarge with High I/O > > Thanx, > Bryan > > ---------- Forwarded message ---------- > From: Bryan Senseman <monetdb(a)openbi.com> > Date: Sat, Oct 19, 2013 at 8:51 PM > Subject: Re: Are joins really this expensive? > To: Stefan Manegold <Stefan.Manegold(a)cwi.nl> > > > Ask, and ye shall receive, let me know if you need anything else; and I > can't share the data, nor would you want a many GB data bomb. > > Bryan > > > On Sat, Oct 19, 2013 at 3:17 PM, Stefan Manegold <Stefan.Manegold(a)cwi.nl>wrote: > >> Dear Bryan, >> >> unless you could share your data such that we could reproduce the problem, >> could you please profile both queries using TRACE, stethoscope or >> tomograph; >> cf., >> http://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/QueryTiming >> and share the respective output? >> >> Then we might be able analyze what exactly causes the performance >> difference between both queries. >> >> Thanks, >> Stefan >> >> >> Bryan Senseman <monetdb(a)openbi.com> wrote: >>> >>> I have created a traditional star-schema in MonetDB, using both Linux >>> and Windows. We are using Mondrian OLAP on top so it generates the >>> standard OLAP-y type of query using group-bys of dimensional values. What >>> I’m finding is that doing joins in MonetDB is extremely expensive, for >>> example: >>> >>> >>> >>> The following query against a fact table with 261 million rows returns >>> in 9-11seconds consistently >>> >>> select date_hour_id/10000 as date_month_id, avg(seconds) from >>> fact_utilization_hourly group by date_month_id; >>> >>> returning 41 rows, the query includes a “Trick” to get to a month_year >>> value because the time_ids are smart keys… yyyyMMddhh. >>> >>> >>> >>> With the following query, using standard OLAP-y syntax, the query takes >>> over 5 mins consistently. >>> >>> select year_month_value, avg(seconds) from fact_utilization_hourly f >>> join dim_date_hour d on f.date_hour_id = d.date_hour_id group by >>> year_month_value; >>> >>> returning the same 41 rows. >>> >>> >>> >>> I’m at a loss to figure out why this join causes a 3200% percent >>> increase in response time. And this seems to have gotten worse in SP5 from >>> anecdotal evidence. >>> >>> >>> >>> Bryan >>> >>> ------------------------------ >>> >>> This transmission is confidential and intended solely for the use of the >>> recipient named above. It may contain confidential, proprietary, or legally >>> privileged information. If you are not the intended recipient, you are >>> hereby notified that any unauthorized review, use, disclosure or >>> distribution is strictly prohibited. If you have received this transmission >>> in error, please contact the sender by reply e-mail and delete the original >>> transmission and all copies from your system. >>> >>> ------------------------------ >>> >>> users-list mailing list >>> users-list(a)monetdb.org >>> >>> https://www.monetdb.org/mailman/listinfo/users-list >>> >>> >> -- >> | Stefan.Manegold(a)CWI.nl | Database Architectures (DA) | >> | www.CWI.nl/~manegold | Science Park 123 (L321) | >> | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >> > > > -- ------------------------------ This transmission is confidential and intended solely for the use of the recipient named above. It may contain confidential, proprietary, or legally privileged information. If you are not the intended recipient, you are hereby notified that any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this transmission in error, please contact the sender by reply e-mail and delete the original transmission and all copies from your system.

1 0

Dump database with other than admin-user - access denied
by Klaus Kruse 30 Oct '13

30 Oct '13

Hello, we will run MonetDB on two servers and while installing and so on went fine, I struggle with the backup solution. For this, we normally add a separate user with full rights. The dump seems to fail on the sys.db_user_info : mclient --format=sql --user=backup > /dump-file MAPI = backup@localhost:50000 QUERY = SELECT "name" FROM "sys"."auths" WHERE "name" NOT IN (SELECT "name" FROM "sys"."db_user_info") AND "grantor" <> 0 ORDER BY "name" ERROR = !SELECT: access denied for backup to table 'sys.db_user_info' Ok, let's give it a try: GRANT ALL ON sys.db_user_info TO "backup" ; operation successful (1.874ms) MAPI = backup@localhost:50000 QUERY = SELECT "ui"."name", "ui"."fullname", "password_hash"("ui"."name"), "s"."name" FROM "sys"."db_user_info" "ui", "sys"."schemas" "s" WHERE "ui"."default_schema" = "s"."id" AND "ui"."name" <> 'monetdb' ORDER BY "ui"."name" ERROR = !access denied for user 'backup' So, my question is: How to give another user than 'monetdb' full rights to dump the whole db? Thanks in advance und best regards Klaus Kruse -- Corpex Internet GmbH * Schauenburgerstrasse 6 * D-20095 Hamburg Tel: +49 40 822268-0 * Fax: +49 40 822268-100 * http://www.corpex.de/ HRB 78752, Amtsgericht Hamburg, Sitz: Hamburg, USt.-Id: DE 213001412 Geschäftsführer: Daniel Hanelt

1 0

Accessing MonetDB form Oracle DB
by baraa Mohamad 26 Oct '13

26 Oct '13

Dear MonetDB users, I have a question about accessing MonetDB via Oracle using ODBC. I have oracle on machine1 (ubuntu 64bit) and MonetDB on machine2(ubuntu 64bit)I installed Monetdb odbc driver using apt-get install libmonetdb-client-odbc I have done all the needed configurations for the listner, tnsnames, sqlnet, initservice.ora and I used UnixODBC. After creating the database link, I get this error when I try to connect MonetDB: ORA-28500: connection from ORACLE to a non-Oracle system returned this message:[unixODBC][MonetDB][ODBC Driver 11.11.11]Client unable to establish connection{08001}ORA-02063: preceding 2 lines from MONETDB I don't know if the use of (64bit) needs some special configurations?? or its du to something else. So I need your help to figure out what is the problem.... Best regards,Baraa

3 4

Primary and Foreign keys
by Hilario Fernandes 22 Oct '13

22 Oct '13

Hi, I've been trying to understand if there is any performance benefit in using primary and foreign key constraints when performing join queries. I've tested joining a table with 1 million rows with one with 60k. Tryed the same queries with and without constraints and couldn't see a significant change in execution time. The tables are not that big but they are a sample of what we'll be working with in average. Should i be seeing any difference? Hows do PK and FK constraints affect query performance in Monetdb? Will it only be noticeable with larger datasets? Thank you Hilario Fernandes

2 1

Are joins really this expensive?
by Bryan Senseman 20 Oct '13

20 Oct '13

I have created a traditional star-schema in MonetDB, using both Linux and Windows. We are using Mondrian OLAP on top so it generates the standard OLAP-y type of query using group-bys of dimensional values. What I’m finding is that doing joins in MonetDB is extremely expensive, for example: The following query against a fact table with 261 million rows returns in 9-11seconds consistently select date_hour_id/10000 as date_month_id, avg(seconds) from fact_utilization_hourly group by date_month_id; returning 41 rows, the query includes a “Trick” to get to a month_year value because the time_ids are smart keys… yyyyMMddhh. With the following query, using standard OLAP-y syntax, the query takes over 5 mins consistently. select year_month_value, avg(seconds) from fact_utilization_hourly f join dim_date_hour d on f.date_hour_id = d.date_hour_id group by year_month_value; returning the same 41 rows. I’m at a loss to figure out why this join causes a 3200% percent increase in response time. And this seems to have gotten worse in SP5 from anecdotal evidence. Bryan -- ------------------------------ This transmission is confidential and intended solely for the use of the recipient named above. It may contain confidential, proprietary, or legally privileged information. If you are not the intended recipient, you are hereby notified that any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this transmission in error, please contact the sender by reply e-mail and delete the original transmission and all copies from your system.

3 3

Monetdb python error in connection
by Rose Beck 19 Oct '13

19 Oct '13

I am using python-monetdb 11.16.0.7. I created my database farm and database according to instructions given below (source: http://www.monetdb.org/Documentation/monetdbd) % monetdbd start /home/my-dbfarm % monetdb create my-first-db Then I tried to connect to the database using the below mentioned command in python(https://pypi.python.org/pypi/python-monetdb/). Upon doing so I am getting the below mentioned error: >import monetdb.sql >connection=monetdb.sql.connect(username="monetdb",password="monetdb",hostname="localhost",database="my-first-db"); File "/usr/local/lib/python2.7/dist-packages/monetdb/sql/__init__.py", line 28, in connect return Connection(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/monetdb/sql/connections.py", line 58, in __init__ unix_socket=unix_socket) File "/usr/local/lib/python2.7/dist-packages/monetdb/mapi.py", line 93, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Can someone please help me with this? Also my my-first-db resides inside my db-farm? I am not getting as to what should I do?

1 0

Re: nulls last
by Klaudiusz 18 Oct '13

18 Oct '13

On 18.10.2013 12:05, Hilario Fernandes wrote: > O made the same test in SP4 and the results were the same, sum(null) kept > returning 0 and no results for the case. My tests in SP5 were done with a > clean database with no data imported, so according to your tests i should > at least have the correct value for the sum... > > These are the queries i've done and what i believe would be the expected > results: > > create table test (number int, name varchar(50)); > > insert into test values(1,'jonh'); > insert into test values(2,'mike'); > insert into test values(3,'mary'); > insert into test values(null,'austin'); > insert into test values(null,'jonh'); > insert into test values(null,'mary'); > > select name, sum(number) > from test > group by name; > > Result: > austin | 0 > jonh | 1 > mary | 3 > mike | 2 > > Expected: > austin | null > jonh | 1 > mary | 3 > mike | 2 > Hi Hilario, I've tested your query using default branch (11.16.0) and I can confirm that this bug also exist. So probably it depends on type of value by which you group. Probably if you try my example from http://bugs.monetdb.org/show_bug.cgi?id=3388 on fresh database, results should be as expected. So group by int works, but group by varchar doesn't :-(. I'm not sure if this is another bug than 3388 or it is the same. It's rather another one. Klaudiusz

1 0

nulls last
by Hilario Fernandes 18 Oct '13

18 Oct '13

Hello, I'm trying to build a query with order by that puts nulls last always. I tryed to use the "NULLS LAST" expression but apparently it's not supported by monetdb. So i came up with something like this: select name, sum(number) from test group by name order by case when sum(number) is null then 1 else 0 end, sum(number) ASC Thing is, this works when the ordering column (number) is not aggretated, as soon as i throw the sum in there it returns no results. The sum of null values seems to be returning 0, is it possible to change that behaviour trough configuration? Is there a better way to achieve this? thanks

2 1

Announcement: New Feb2013-SP5 Bugfix release of MonetDB suite
by Hannes Mühleisen 16 Oct '13

16 Oct '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013-SP5 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Feb 2013-SP5 bugfix release Client Package * ODBC: Implemented retrieving variable-length data in parts with SQLGetData. See bug 3381. MonetDB Common * Removed the heap cache. Since the fix for bug 3323 which made that the cache was actually getting used, bug 3376 made clear that it didn't work very well. In addition, on Linux at least, the heap cache slows things down. Java Module * Rearranged order of returned columns of certain metadata functions to comply with the JDBC documentation. See bug 3356. Bug Fixes * 3323: Heapcache bugs/performance issues * 3331: SAMPLE will return same result every time. * 3356: DatabaseMetaData.getColumns() doesn't work correctly when using index-based getters * 3367: Fully qualified order by column gives "relational query without result" * 3368: BAT sortedness info ignored on ORDER BY and TOPN * 3370: SQL environment settings are updated even when the set statement fails * 3373: Setting table to read-only (Bug 3362) fails on big-endian * 3375: LIKE join: BATfetchjoin does not hit always * 3376: COPY INTO fails with HEAPextend: failed to extend: MT_mremap() failed * 3377: Query interfering with next query in same transaction, after SP4 * 3380: Python DBAPI driver throws exception when fetching EXPLAIN results * 3381: Windows ODBC driver hangs or crashes on simple queries

1 0

Fwd: Query execution time
by baraa Mohamad 13 Oct '13

13 Oct '13

>> Hello dear MontDB users, >> >> I have a question please, I just need to get the query execution time >> without showing the results of the query (for performance comparaison) . >> Something which corresponds to the "traceonly" in Oracle. that will be >> greate if I can see the query plan but for the moment the most important >> for me is to get the query execution time. >> >> for example: I want to know the query exection time of this query >> (select * from source where a>10) knowing that I have more than one >> hundred million records which correspond to this predicate. >> >> Thank in advace for your appreciated help. >> >> Baraa

4 16

Re: How to transfer monetdb database to another machine
by Rose Beck 13 Oct '13

13 Oct '13

Thanks a lot for helping. I am unable to figure as to where does * <dbfarm>/<dbname>* directory reside within ubuntu 12.04. Can you please help me with this. On Sun, Oct 13, 2013 at 6:06 PM, <users-list-request(a)monetdb.org> wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. Re: Query execution time (baraa Mohamad) > 2. Re: Query execution time (Martin Kersten) > 3. Re: How to transfer monetdb database to another machine > (Martin Kersten) > 4. Re: Query execution time (Martin Kersten) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 13 Oct 2013 12:31:35 +0200 > From: baraa Mohamad <bem85(a)hotmail.com> > To: Communication channel for MonetDB users <users-list(a)monetdb.org> > Cc: Communication channel for MonetDB users <users-list(a)monetdb.org> > Subject: Re: Query execution time > Message-ID: <BAY406-EAS1557A9804AABD08B94A7905A8190(a)phx.gbl> > Content-Type: text/plain; charset="utf-8" > > That's great! Thank you very much for your appreciated help. > > I just could't really understand the output of this command. > > This is my output: > Real: 1m23 > User: 0m24s > Sys: 0m2.3s > > I'm a little bit confused because when I execute the same query directly > without the time option i get 733,28ms. > > So i couldn't know what is the real execution time. > > > > > Le 13 oct. 2013 ? 11:29, "Martin Kersten" <Martin.Kersten(a)cwi.nl> a > ?crit : > > > > How about (time mclient -d db <inputfile >/dev/null) > > > >> On 10/13/13 11:24 AM, baraa Mohamad wrote: > >> Thank you for your answer. > >> But in general I would like to know how to get the query exexution time > >> without printing the results. It's for performance comparaison. I want > >> to evaluate MonetDB for my research and i'm working with large tables. > >> > >> Please any idea?? > >> > >> Greetings, > >> Baraa > >> > >> > >> > >> > >> Le 13 oct. 2013 ? 11:02, "Martin Kersten" <Martin.Kersten(a)cwi.nl > >> <mailto:Martin.Kersten@cwi.nl>> a ?crit : > >> > >>> Hi > >>> > >>> > >>>> On 10/13/13 10:55 AM, baraa Mohamad wrote: > >>>> > >>>>>> Hello dear MontDB users, > >>>>>> > >>>>>> I have a question please, I just need to get the query execution > time > >>>>>> without showing the results of the query (for performance > >>>>>> comparaison) . > >>>>>> Something which corresponds to the "traceonly" in Oracle. that will > be > >>>>>> greate if I can see the query plan but for the moment the most > >>>>>> important > >>>>>> for me is to get the query execution time. > >>>>>> > >>>>>> for example: I want to know the query exection time of this query > >>>>>> (select * from source where a>10) knowing that I have more than one > >>> This is not the ind of query you fire when you expect a large outcome. > >>> Better use SELECT COUNT(*).... > >>> > >>>>>> hundred million records which correspond to this predicate. > >>>>>> > >>>>>> Thank in advace for your appreciated help. > >>>>>> > >>>>>> Baraa > >>>> _______________________________________________ > >>>> users-list mailing list > >>>> users-list(a)monetdb.org <mailto:users-list@monetdb.org> > >>>> https://www.monetdb.org/mailman/listinfo/users-list > >>> _______________________________________________ > >>> users-list mailing list > >>> users-list(a)monetdb.org <mailto:users-list@monetdb.org> > >>> https://www.monetdb.org/mailman/listinfo/users-list > >> > >> > >> _______________________________________________ > >> users-list mailing list > >> users-list(a)monetdb.org > >> https://www.monetdb.org/mailman/listinfo/users-list > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org > > https://www.monetdb.org/mailman/listinfo/users-list >

2 1

How to transfer monetdb database to another machine
by Rose Beck 13 Oct '13

13 Oct '13

I created monetdb database using the python library https://pypi.python.org/pypi/python-monetdb/11.16.0.7 on my ubuntu 12.04 lts server. Now I want to transfer this database to another ubuntu 12.04 lts server. However, I am unable to understand as to where(directory name) does monetdb create its database (within ubuntu 12.04) and what is the procedure which I must follow to transfer this database to another machine (so that i may query the database on another machine)?

2 1

Crashed DB - BBPextend: trying to extend BAT pool beyond the limit
by Pedro Salgueiro 09 Oct '13

09 Oct '13

Hi, We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database: 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend > BAT pool beyond the limit (16384000) > 2013-10-07 10:35:27 ERR testbd[12111]: > 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has > exited with exit status 1 > 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database > 'testbd' has crashed after starting, manual intervention needed, check > monetdbd's logfile for details > 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed > after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: > 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25) The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users. I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue: 1 - Is there any way to see a more detailed log about the error? 2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success. 3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have? Thank you, Pedro Salgueiro

3 8

ORDER BY in sub-query
by Hadley Wickham 08 Oct '13

08 Oct '13

Currently, monetdb fails on this query: SELECT * FROM (SELECT * FROM "Batting" ORDER BY "playerID") with syntax error, unexpected ORDER, expecting INTERSECT or EXCEPT or UNION or ')' in: "select * from (select * (obviously that query isn't very useful, but it makes it harder to automatically generate sql) In case it matters, that query (with minor escaping variations) works in sqlite, postgres and mysql. Hadley -- Chief Scientist, RStudio http://had.co.nz/

5 17

Create database other then "demo" in windows 8
by Prashant Patel 07 Oct '13

07 Oct '13

Hi all, I am using MonetDB on my windows 8 platform every time open monetdb client and server , it automatically connects to "demo" database by default. I am very new to these technology I have searched but couldn't find how to create our own database in monetdb (other then "demo"). Can anyone tell me how to create database and connect that database in monetdb server and client. I know this is very dump question but help will be really appreciated. ( If i can get some documents that would also work) Thanks in advance, Prashant Rajkotiya Research Aspirant

2 2

How to take COLD RUNs Execution time
by Prashant Patel 07 Oct '13

07 Oct '13

as a part of my project, i have to check the performance of monetdb column-store database. I am very new to this technology, so i dont know where the monetdb caches its previous query results.. I want to know how can i take results for cold runs on monetdb ( i have tried closing monetdb client and server every time i fire a query , but doesnt seem to clear the cache and i always get hot runs result). I am using window 8 core i5 2.5GHz machine. so please can anyone help me regarding these.... suggestions are appreciated.. thanks in advance.. Prashant Rajkotiya M.tech. (ICT) DA-IICT ,Gandhinagar

2 2

Window functions
by Hadley Wickham 05 Oct '13

05 Oct '13

Hi all, Does monetdb support window functions? I found https://www.monetdb.org/pipermail/users-list/2011-June/004992.html, which claims that OVER is supported, but: * I can't see where support is listed on http://www.monetdb.org/print/book/export/html/63 * I get errors when I try and use window functions: sql> SELECT *, MAX("AB") OVER (PARTITION BY "playerID") AS "_W4", MAX("G") OVER (PARTITION BY "playerID") FROM "Batting"; SELECT: function 'max' not found But that might be because I don't have the syntax right (but that query works in Postgres) Are there are any plans to add standard window functions like rank, ntile, lead, lag etc to monetdb? They are pretty useful for analysis. Hadley -- Chief Scientist, RStudio http://had.co.nz/

2 1

new "too many digits" bug in SP-4
by Anthony Damico 03 Oct '13

03 Oct '13

this issue is new for SP-4 ..i have been using this syntax for a long time and never encountered this problem until the upgrade :( > dbGetQuery( db , "select ( sum( ( bene_sex_ident_cd = 1 ) ) ) / ( count( * ) ) as pct_male from x11" ) pct_male 1 0 > dbGetQuery( db , "select ( sum( ( bene_sex_ident_cd = 1 ) )*1.000 ) / ( count( * )*1.000 ) as pct_male from x11" ) Error in .local(conn, statement, ...) : Unable to execute statement 'select ( sum( ( bene_sex_ident_cd = 1 ) )*1.000 ) / ( count( * )*1.000 ) as pct_male from x11'. Server says 'too many digits (19 > 18)' [#22003]. > what diagnostics can i run for you to troubleshoot this? thanks!! :)

2 7

Bug with fetching EXPLAIN results in Python DBAPI driver.
by Pete Hollobon 03 Oct '13

03 Oct '13

Hi, I spotted a bug in the Python DBAPI driver when fetching results from an EXPLAIN. >>> import monetdb.sql >>> db = monetdb.sql.connect(database="demo") >>> cursor = db.cursor() >>> cursor.execute("explain select count(*) from sys.tables") 31 >>> cursor.fetchone() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "monetdb/sql/cursors.py", line 203, in fetchone result = self.__rows[self.rownumber - self.__offset] IndexError: list index out of range >>> I've logged a bug with patches for Python 2 and 3 here: http://bugs.monetdb.org/show_bug.cgi?id=3380 thanks, Pete.

1 0

Static binary compilation
by Hassan, Ahmad 01 Oct '13

01 Oct '13

Hi, May be someone has tried compiling monetdb as a static binary please? I tried compiling the binary using -static flag but the resulting binary is still 'Dynamically linked' binary. I think it is due to glibc and dl libraries. If I pass libglibc.a and libdl.a statically then I get errors like: ============= /tmp/mserver5-rkIbiC.o: In function `mdlopen': /data/ahmad/test/MonetDB/gdk/gdk_posix.c:454: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/x86_64-linux-gnu/libreadline.a(complete.o): In function `rl_username_completion_function': (.text+0x2bc9): warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ... /data/ahmad/test/MonetDB/common/stream/stream.c:1765: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ============= Does anyone know the workaround of creating static binary of monetdb executable please? Cheers, Ahmad

2 1