- users-list - monetdb.org

[MonetDB-users] ERROR GDKload failed....CMDthetajoin; operation failed
by George H 10 Oct '06

10 Oct '06

Hello again kind people. This is an sql self join on TableA which contains a serial primary key column called 'seq'. The select is limited to less than 400 rows of output by 'a1.seq<=400'. sql->select a1.*, sum(a2.num)/17 from TableA a1, TableA a2 where a1.seq<=400 and a1.seq>=17 and a1.seq<a2.seq+17 group by a1.seq, a1.num order by a1.seq asc; This select statement results in 384 rows, as expected, with a newly calculated column for sum(a2.num)/17 which yields a running average of 'num' between any given value for 'seq' and 'seq+17'. This next sql statement is exactly the same as above except the limitation of 384 rows of output imposed by 'a1.seq<=400' has been removed. Note that TableA contained 40 million plus rows. sql->select a1.*, sum(a2.num)/17 from TableA a1, TableA a2 where and a1.seq>=17 and a1.seq<a2.seq+17 group by a1.seq, a1.num order by a1.seq asc; Error: ERROR: GDKload failed: name=14/1457, ext=buns.priv ERROR: CMDthetajoin: operation failed. Perhaps there is a better way to accomplish a running aggregate on 40 million rows? Thanks.........George

3 4

[MonetDB-users] MonetSQLdb python API
by Ronald Oussoren 10 Oct '06

10 Oct '06

Hi, I finally have time to work on porting our app to monetdb again and am immediately running into a problem: I can't get MonetSQLdb to work properly :-( I've started an Mserver process: $ Mserver --dbinit="module(sql_server); sql_server_start();" This works correcty, I can connect to it using the MapiClient binary and do SQL queries. However, this python code fails: import MonetSQLdb c = MonetSQLdb.connect() It says: !ERROR mapi_reconnect: connect: Connection refused Traceback (most recent call last): File "t.py", line 2, in ? c = MonetSQLdb.connect() File "/opt/monet-4.10/opt/python2.4/lib/python2.4/site-packages/ MonetSQLdb/__init__.py", line 51, in Connect return Connection(**kwargs) File "/opt/monet-4.10/opt/python2.4/lib/python2.4/site-packages/ MonetSQLdb/__init__.py", line 117, in __init__ kwargs['user'], kwargs['password'], kwargs['lang']) File "/opt/monet-4.10/opt/python2.4/lib/python2.4/site-packages/ CMapi.py", line 26, in __init__ raise IOError(self.error_str()) IOError: Setup connection failed Looking into the source code for the python client it should try to connect to port 45123, which the server is listening to. What it actually seems to do is connect to port 42312, but I have no idea why it does so (and hunting this down through several layers of generated code, one of which is swig, is no fun at all). What am I doing wrong? Ronald

4 23

[MonetDB-users] algebra?
by George H 05 Oct '06

05 Oct '06

Hello Again: I'm trying to do some clinical research on large data sets and I ran into a problem represented in the following simplified example: geo-> select * from test2; +-----------+-----------+ | a | b | +===========+===========+ | 3.05 | 3.07 | | 3.06 | 3.04 | | 3.13 | 3.16 | | 3.22 | 3.22 | | 3.05 | 3.05 | | 3.07 | 3.06 | | 3.08 | 3.05 | | 3.12 | 3.09 | | 3.20 | 3.17 | | 3.17 | 3.20 | | 3.09 | 3.11 | | 2.74 | 2.73 | | 2.79 | 2.78 | | 2.80 | 2.81 | +-----------+-----------+ 14 rows geo-> select * from test2 where a=b; +-----------+-----------+ | a | b | +===========+===========+ | 3.22 | 3.22 | | 3.05 | 3.05 | +-----------+-----------+ 2 rows geo-> select * from test2 where (a/b)-1=0; +-----------+-----------+ | a | b | +===========+===========+ | 3.06 | 3.04 | | 3.22 | 3.22 | | 3.05 | 3.05 | | 3.07 | 3.06 | | 3.08 | 3.05 | | 3.12 | 3.09 | | 3.20 | 3.17 | | 2.74 | 2.73 | | 2.79 | 2.78 | +-----------+-----------+ <>9 rows geo-> select * from test2 where (b/a)-1=0; +-----------+-----------+ | a | b | +===========+===========+ | 3.05 | 3.07 | | 3.13 | 3.16 | | 3.22 | 3.22 | | 3.05 | 3.05 | | 3.17 | 3.20 | | 3.09 | 3.11 | | 2.80 | 2.81 | +-----------+-----------+ <>7 rows I thought that if a=b, then (a/b)-1 and (b/a)-1 must both also be equal to 0. Columns a and b are type decimal(7,2) and the data was entered to two decimal places by hand using insert statements. Thanks and best.......George ge I

2 1

Re: [MonetDB-users] algebra?
by George H 04 Oct '06

04 Oct '06

Hello: Sorry to have troubled you with the basic sql question below. Of course, it has no place here. The algebra will, of course, yield results in the same precision as the decimal types used in the calculation. George Hello Again: I'm trying to do some clinical research on large data sets and I ran into a problem represented in the following simplified example: geo-> select * from test2; +-----------+-----------+ | a | b | +===========+===========+ | 3.05 | 3.07 | | 3.06 | 3.04 | | 3.13 | 3.16 | | 3.22 | 3.22 | | 3.05 | 3.05 | | 3.07 | 3.06 | | 3.08 | 3.05 | | 3.12 | 3.09 | | 3.20 | 3.17 | | 3.17 | 3.20 | | 3.09 | 3.11 | | 2.74 | 2.73 | | 2.79 | 2.78 | | 2.80 | 2.81 | +-----------+-----------+ 14 rows geo-> select * from test2 where a=b; +-----------+-----------+ | a | b | +===========+===========+ | 3.22 | 3.22 | | 3.05 | 3.05 | +-----------+-----------+ 2 rows geo-> select * from test2 where (a/b)-1=0; +-----------+-----------+ | a | b | +===========+===========+ | 3.06 | 3.04 | | 3.22 | 3.22 | | 3.05 | 3.05 | | 3.07 | 3.06 | | 3.08 | 3.05 | | 3.12 | 3.09 | | 3.20 | 3.17 | | 2.74 | 2.73 | | 2.79 | 2.78 | +-----------+-----------+ <>9 rows geo-> select * from test2 where (b/a)-1=0; +-----------+-----------+ | a | b | +===========+===========+ | 3.05 | 3.07 | | 3.13 | 3.16 | | 3.22 | 3.22 | | 3.05 | 3.05 | | 3.17 | 3.20 | | 3.09 | 3.11 | | 2.80 | 2.81 | +-----------+-----------+ <>7 rows I thought that if a=b, then (a/b)-1 and (b/a)-1 must both also be equal to 0. Columns a and b are type decimal(7,2) and the data was entered to two decimal places by hand using insert statements. Thanks and best.......George ge I

1 0

[MonetDB-users] Relieve
by classes: 04 Oct '06

04 Oct '06

Money Good Will Public People are spelled Try different keywords. general fewer Diverse Marine Life Found Off Care Medical Animation Nutrition Body Office Yard Credit Computers Gadgets Theater Movies Sports Toys Drugs Fitness Health Care Medical Based Services SIP Python regular basis updates. Platform: Time Zone Family Clothing Flowers Gifts Beauty Guides products and mobility meet achieve business goals November Amsterdam The Edition Learning Path about Searches Engines HDTV Digital Camera Cell Phone DSL Lock Picking Power Hybrid Cars general fewer your on can try Answers for expert help please... T.One mackdaddy thumb drives. gigs storage highspeed transfer Korea vowed Device Solutions Computers Gadgets Theater Movies Computer Blocks Search Members: about basics tools creating Products Auto Stu

1 0

[MonetDB-users] more sql blob
by Ronald Oussoren 03 Oct '06

03 Oct '06

Another problem with SQL Blob's is that they cannot be NULL :-(. (using the same testblob table as my previous mail) sql>insert into testblob(id, b) VALUES (NULL, NULL); MAPI = monetdb@localhost:50000 QUERY = insert into testblob(id, b) VALUES (NULL, NULL); ERROR = !ERROR: interpret_params s0_0(param 2): invalid sqlblob. I can probably work around this, but using empty blobs is less elegant than NULL values. Ronald

1 0

[MonetDB-users] SQL blob's
by Ronald Oussoren 03 Oct '06

03 Oct '06

My database scheme requires usage of blobs, but something seems wrong there: CREATE TABLE testblob (id INT, b BLOB); INSERT INTO testblob (id, b) VALUES ( 1, BLOB '1234'); SELECT * FROM testblob; # sys.testblob, sys.testblob # table_name # id, b # name # int, blob # type # 1, 0 # length [ 1, 0203 ] DROP TABLE testblob; It turns out that the conversion of a SQL blob literal to an array of bytes is buggy, the patch below fixes that. Ronald $ diff -u ../MonetDB-4.12.1/src/modules/plain/blob.c.org ../ MonetDB-4.12.1/src/modules/plain/blob.c --- ../MonetDB-4.12.1/src/modules/plain/blob.c.org 2006-10-03 16:04:05.000000000 +0200 +++ ../MonetDB-4.12.1/src/modules/plain/blob.c 2006-10-03 16:15:04.691486914 +0200 @@ -370,14 +370,15 @@ s++; res <<= 4; if (*s >= '0' && *s <= '9') { - res = *s - '0'; + res |= *s - '0'; } else if (*s >= 'A' && *s <= 'F') { - res = 10 + *s - 'A'; + res |= 10 + *s - 'A'; } else if (*s >= 'a' && *s <= 'f') { - res = 10 + *s - 'a'; + res |= 10 + *s - 'a'; } else { GDKerror("sqlblob_fromstr: Illegal char '%c' in blob\n", *s); } + s++; result->data[i] = res; } Ronald

1 0

[MonetDB-users] sql/jdbcclient...Exception in thread "main' java.lang.OutOfMemoryError
by George H 02 Oct '06

02 Oct '06

Fabian: I'm way out of you guys' league. I wouldn't know a stack trace from my elbow, but I'll do what I can to help. Just say so and tell me how. As far as whether I can tell if the MapiClient displayed all the rows or just a subset? Well, I actually did the MapiClient runs with 40million rows. It took awhile for it to role through them all, and of course, without the exception. Query results with less rows took less time. I can't say for sure if it actually showed all the rows or a subset, but I just assumed it was all of them because of how long it took to work through it. I'm afraid I am unaware of how to know if it showed all the rows or a subset. I only saw the beginning and the end of the runs, and that much of it seemed right. If there is any way I can help, just let me know. George

2 2

Re: [MonetDB-users] Recent bugs and question about modules
by Stefan Manegold 29 Sep '06

29 Sep '06

Maurice, finally, here's my reply --- at least as far as I Think I can answers your question right now... On Fri, Sep 29, 2006 at 10:02:51AM +0200, Maurice van Keulen wrote: > Hi Stefan, > > Thanks for resolving some of my bugs. I'm trying to develop an XQuery > module for probabilistic XML. This involves highly recursive functions > and XML fragments being constructed and then queried upon again in the > next function. I'm impressed that I got so far already (322 lines of > really complex XQuery code). Every time I do stumble onto a bug, I try > to isolate it and report it nicely. Currently, the bug that handers my > progress most is "1562868: Complex XQuery fails with error in > "batbat_lng_add_inplace". It is hard to already give the correct answer > to the test you created. I suggest to simply approve the result when the > error message does not occur anymore. And, if you are ready to pick one > of my bugs I reported, please have a look at this one if you will. Well, Jan spent quite some time on this one (see the bug report; Thanks, Jan!) and it appears to be a tircky one --- I'll consider applying Jan's "fast and ugly" solution for the time being, and *try* to find a better one as soon as time permits... > I also have a question. Eventually, my functions need to become part of > a module for Probabilistic XML. Besides XQuery functions, I also have a > few fundamental functions that are much better dealt with in MIL. I > have, for example, a function product that, given a sequence of numbers, > multiplies all these numbers: > > declare function product($seq as xs:decimal*) as xs:decimal > { if (count($seq) gt 0) then exactly-one($seq[1]) * > product($seq[position() gt 1]) else 1.0 }; > > This recursive definition of the function is for sure much less > efficient than something I could write in MIL. > > My question is: how can I develop a module with additional functions > that are implemented in MIL and usable in XQuery? Well, obviously(?), there are to ways to go. The first is to indeed add a new built-in function and provide a MIL implementation as we do for all other built-in functions --- I guess, this is not what you intended, right? The second one is to provide an XQuery module. In fact, MonetDB/XQuery allows both "plain" XQuery module definitions and MIL module definitions --- as far as I know, the best (only? for now?) way to create a MIL modules (template) is to have the pathfinder compiler translate an XQuery module to MIL (basically, Peter added the support for MIL modules to have pre-compiled modules, saving the XQuery to MIL translation...). Here's my approach to your example: ======== $ cat /tmp/mvk.xq module namespace mvk = "http://wwwhome.cs.utwente.nl/~keulen/"; declare function mvk:product($seq as xs:decimal*) as xs:decimal { if (count($seq) gt 0) then exactly-one($seq[1]) * mvk:product($seq[position() gt 1]) else 1.0 }; -------- $ pf /tmp/mvk.xq > /tmp/mvk.mil -------- $ echo 'import module namespace mvkXQ = "http://wwwhome.cs.utwente.nl/~keulen/" at "/tmp/mvk.xq"; mvkXQ:product((1,2,3,4))' | MapiClient -lx <?xml version="1.0" encoding="utf-8"?> <XQueryResult>24.000000</XQueryResult> -------- $ echo 'import module namespace mvkMIL = "http://wwwhome.cs.utwente.nl/~keulen/" at "/tmp/mvk.mil"; mvkMIL:product((1,2,3,4))' | MapiClient -lx <?xml version="1.0" encoding="utf-8"?> <XQueryResult>24.000000</XQueryResult> ======== well, if you look at the attached mvk.mil, you'll see that it looks quite complicated, and using it as a template to replace the recursive product implementation by a more efficient version will be quite tricky. Hence, here's another approach: While there is (AFAIK) not product aggregation in XQUery (yet?), there is one in MIL; and there is a sum aggregation in both. Hence, simply take a fn:sum in XQuery have it translated to MIL, which results in usage of {sum}() in MIL, and replace the latter by {prod}(): ======== $ cat /tmp/stm.xq module namespace stm = "http://www.cwi.nl/~manegold/"; declare function stm:product($seq as xs:decimal*) as xs:decimal { sum($seq) }; -------- $ pf /tmp/stm.xq > /tmp/stm.mil -------- $ egrep '{sum}|{prod}' /tmp/stm.mil var iter_aggr := {sum}(item.leftfetchjoin(dec_values), iter_grp, reverse(loop000)).tmark(0@0); -------- $ perl -i -p -e 's|{sum}|{prod}|' /tmp/stm.mil -------- $ egrep '{sum}|{prod}' /tmp/stm.mil var iter_aggr := {prod}(item.leftfetchjoin(dec_values), iter_grp, reverse(loop000)).tmark(0@0); -------- $ echo 'import module namespace stmXQ = "http://www.cwi.nl/~manegold/" at "/tmp/stm.xq"; stmXQ:product((1,2,3,4))' | MapiClient -lx <?xml version="1.0" encoding="utf-8"?> <XQueryResult>10.000000</XQueryResult> -------- $ echo 'import module namespace stmMIL = "http://www.cwi.nl/~manegold/" at "/tmp/stm.mil"; stmMIL:product((1,2,3,4))' | MapiClient -lx <?xml version="1.0" encoding="utf-8"?> <XQueryResult>24.000000</XQueryResult> ======== Seem to work well ;-) As said, I don't know a better way to produce a MonetDB/XQuery conforming MIL module by hand, but this way, you can create a template and modify it even further than I did here. There's one(?) caveat, though: Modules created this way are not necessairyly protable from one version of MonetDB/XQuery to another one, as the requires/expected signature of the respective MIL procedure and/or format/content of its arguments might change... Hope this helps you further, anyway... More suggestions are of course welcome (that's why I moved this discussion to this mailing list ;-)) Stefan > Thanks again for your effort! > Maurice. > > -- > ---------------------------------------------------------------------- > Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology > Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands > Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 > Room: INF3039, WWW: http://www.cs.utwente.nl/~keulen -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [MonetDB-users] sql sequence
by George H 27 Sep '06

27 Sep '06

Hey Guys: I hate to bother you, but I just downloaded and installed the updated builds presently (Sept 27) posted on http://monetdb.cwi.nl/testing/projects/monetdb/Stable/.DailyBuilds./RPMs/Su… I still get: monetdb-> create table test (name varchar(10), seq serial); Operation successful monetdb-> create table test1 (name varchar(10)); Operation successful monetdb-> insert into test1 values ('a'), ('b'), ('c'); 3 affected rows monetdb-> insert into test (name) select (name) from test1; Error: ERROR: INSERT INTO: PRIMARY KEY constraint 'test.test_seq_pkey' violated What do you think? George

1 0

[MonetDB-users] Obtain the career you have always wanted with the University Degree you deserve. Bo6p8HFZV7ZcX
by monet＠isp4mail.com 27 Sep '06

27 Sep '06

1 0

[MonetDB-users] sql/jdbcclient...Exception in thread "main' java.lang.OutOfMemoryError
by George H 26 Sep '06

26 Sep '06

I'm using Suse 10.1 with an AMD dual core processor and 850MB memory (note: There's 1000MB of ram but 150MB is shared with the video card thus leaving the system to recognize only 850MB) MonetDB softwares used are the packages presently posted on: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/.DailyBuilds./RPMs/Su… With the table, "test" having 4 million rows using sql and jdbcclient: geo->select column1 from test; It displays the column1 query results for about 10 seconds and then the jdbcclient-1.4.jar crashes leaving the following message before it bugs out: Exception in thread "main" java.lang.OutOfMemeoryError I tried many many combinations of java options concerning memory and gc to try to fix it but in my hands, no luck. I worked around this by switching to the MapiClient which works just fine. Just thought there might be some interest in these results. Thanks to everyone again.....George

2 1

Re: [MonetDB-users] sql sequence
by George H 26 Sep '06

26 Sep '06

Thanks Fabian and Niels. A fix of this bug is greatly welcomed. That was quick action. I'm using SUSE10.1 and I just installed the build presently posted on: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/.DailyBuilds./RPMs/Su… However, this build still carries the problem. Will the fix appear on this web page and if so when? Or is there some other way I should get this fix? Thanks again..........George

2 1

Re: [MonetDB-users] sql sequence
by George H 26 Sep '06

26 Sep '06

Thanks Fabian and Niels. A fix of this bug is greatly welcomed. That was quick action. I'm using SUSE10.1 and I just installed the build presently posted on: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/.DailyBuilds./RPMs/Su… However, this build still carries the problem. Will the fix appear on this web page and if so when? Or is there some other way I should get this fix? Thanks again..........George

1 0

Re: [MonetDB-users] sql sequence
by George H 26 Sep '06

26 Sep '06

Hey Fabian: Thanks so much for your response. I really appreciate the help. I don't think I can use the idea of creating a table with a sequence column and later doctoring that column with an ALTER SEQUENCE statement because I use a COPY INTO command to populate the table with a text file. The table must have the same number of columns as the test file for the COPY INTO statement to work and the text file has no sequence column. Do you know of some other way to populate a table from a text file that might get around this? I tried to use an INSERT statement to populate a table that already had a sequence column, but that didn't work either. Here's what I did. sql->create table test (name varchar(10), seq serial); sql->create table test2 (name varchar(10)); sql->insert into test2 values ('a), ('b'), ('c'); [ 3 ] sql->insert into test (name) select (name) from test2; ERROR = !ERROR: INSERT INTO: PRIMARY KEY constraint 'test.test_seq_pkey' violated So if I use the COPY INTO statement to populate a table from my text file, I can't use the resulting table in an INSERT/SELECT statement to populate another table that has a sequence column. Is there some course of action you can think of to get a sequenced table from a text file? With 40million+ rows it sure can't be done by hand. Again, thanks for your help and also thanks for all your postings on this list. I've found them interesting and instructive. Best......George

3 2

[MonetDB-users] sql sequence
by George H 25 Sep '06

25 Sep '06

Hello: I have a large table (40 million+ rows) that I generated with "copy into" statements using several 1g txt files. The table was subsequently subjected to an "order asc" statement and now I need to add a sequence column to it and there I have a problem. A simple test table illustrates the problem as follows: sql-> create table test (name varchar(10)); sql->alter table test add column seq serial; Error: ALTER TABLE ADD COLUMN: adding column with key constraint not implemented I have tried several different ways to generate a sequence that work from me in postgres but I continuously run into an error like "primary key constraint violated". The only way I can get it to work is by adding one row at a time. Sorry about my ignorance, but can someone lend a hand? Thanks, George H

2 1

[MonetDB-users] serial/auto_increment
by George H 25 Sep '06

25 Sep '06

Thanks: I have a large table (40 million+ rows) that I generated with "copy into" statements using several 1g txt files. The table was subsequently subjected to an "order asc" statement and now I need to have a sequence put on it and there I have a problem. A small simple test table illustrates the problem as follows: sql-> create table test (name varchar(10)); sql->alter table test add column seq serial; Error: ALTER TABLE ADD COLUMN: adding column with key constraint not implemented I have tried several different ways to generate a sequence in the table without success, including insert/select and auto_increment. I have encountered the error "primary key constraint violated". The only way I can get it to work is by adding one row at a time and with 40million rows....well. Sorry about my naivete, but can someone tell me what I am missing. Thanks, George H

1 0

Re: [MonetDB-users] Shredding DBLP XML document
by Stefan Manegold 22 Sep '06

22 Sep '06

Hi Erwin, shredding dblp.xml works fine for me with MonetDB/XQuery 0.13.1 (latest CVS version) on an Althon64 X2 with 2 GB RAM under Fedora Core 4: ======== # Monet Database Server V4.13.1 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. MonetDB>shred_doc("/tmp/dblp.xml","dblp.xml"); # Elapsed time = 02m 05s 557ms 075us [005us/node] # Shredded 1 XML documents, total time after commit=133.708s ======== Hence, could you please be more verbose about your setup? Which version of MonetDB/XQuery are you using? What kind of "PC" are you using (operating system, hardware)? You say "the shredding process was not finished". Does this mean that MonetDB/XQuery was still busy (with shredding)? Stefan ps: I felt free to cc this to the MonetDB-users lists. On Thu, Sep 21, 2006 at 02:02:05PM +0800, Erwin Leonardi wrote: > Hi all, > > I tried to shred DBLP XML document (around 300MB), but I found the following > situations. I left my PC over the weekend, and the shredding process was not > finish and there is no error message. I tried to check my free disk space, > and I found out that the free disk space remains the same. That is, it seems > that MonetDB has not written any data to disk. Do you why this happened? Can > you try to shred DBLP XML document ? > > Thanks > Erwin -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 6

Re: [MonetDB-users] Comparing TPC-H benchmark results?
by moredata＠fastmail.net 21 Sep '06

21 Sep '06

Hello, I too would be interested in TPC-H comparisons of MonetDB versus other open source databases like MySQL and PostgreSQL, in addition to commercial databases like Oracle. If anyone has done such comparisons, please let me know. Martin also mentioned that the plans are not currently optimized but that it is targeted for V5. Are any of these optimizations already in the V5 CVS code (correlated subqueries?)? Thanks in advance. > -----Original Message----- > From: monetdb-users-bounces(a)lists.sourceforge.net > [mailto:monetdb-users-bounces@lists.sourceforge.net] On > Behalf Of Martin Kersten > Sent: Saturday, September 16, 2006 7:39 AM > To: monetdb-users(a)lists.sourceforge.net > Subject: Re: [MonetDB-users] Comparing TPC-H benchmark results? > > > If I a remember correctly the scale factor is 0.01 in the > testweb. Beware that some queries in TPC-H require a lot of > intermediate space and the plans produced for MIL are not > optimized. (It is the target for the MonetDB V5 release) > > Actually, we would be interested in experiences in the > upcoming TPC-DS, which is more modern and not well explored > by any system. > > Fabian Groffen wrote: > > Hi, > > > > No need to resubmit your question. It was received, but I'm sorry > > that we've been too busy to answer this question. > > > > On 15-09-2006 21:25:15 +0200, Wouter Scherphof wrote: > >> Hi, > >> > >> The TPC-H benchmark (http://www.tpc.org/tpch/default.asp) > is part of > >> the MonetDB SQL source and the benchmark is run during > weekends, as I > >> can see on the TestWeb > >> > (http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests03 > >> /times.html). > >> How > >> should I interpret the listed results? According to TPC, > results should be > >> stated as a "QphH@Size" value, where "QphH" is the number > of queries per > >> hour for the H benchmark, and "Size" is a "scaling > factor". On the TestWeb, > >> the results are stated as seconds of execution time. > > > > The testweb tpc-h is very very very small, to keep both > testweb small > > in diskspace, and to keep the test time reasonable on our slower > > testing machines. > > > >> Is it correct to compute the QphH value as follows: > >> Queries 01-22 took 1.985 seconds, so QphH is (3600 / 1.985) * 22 = > >> 39.899 ? > > > > Well, the testweb framework does way more than just executing the > > test, so the performance time of the test might not be accurate. > > Also, you probably want to test with a higher workload, to > get a more > > accurate result. > > > >> What scaling factor is used in the benchmark runs; is this > 1? Is it > >> possible to let the TestWeb run with scaling factor 100? Because > >> that's the smallest Size for which results are published > on the TPC > >> web site. > > > > It's very small. I don't recall how small, but very very small. > > > >> Did anyone yet try to determine the Price of the test platforms, > >> following the TPC Pricing Spec? TPC-H results normally include a > >> Price/Performance metric in the form of $/QphH@Size, and I guess > >> MonetDB could score particularly well on this metric. > > > > We never really did this. But, feel free to try and setup > such test, > > we are very interested in the outcomes, of course. > > > > > > I hope this answers your questions. > > > > Regards > > > > > ---------------------------------------------------------------------- > > --- > > Using Tomcat but need to do more? Need to support web > services, security? > > Get stuff done quickly with pre-integrated technology to > make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& dat=121642 > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users -- moredata(a)fastmail.net -- http://www.fastmail.fm - IMAP accessible web-mail

1 0

[MonetDB-users] schema reconstruction
by Ronald Oussoren 19 Sep '06

19 Sep '06

Hi, I'm trying to build a driver for monetdb for sqlalchemy (a python ORM layer). One of the things I need to do for that is provide a way to reconstruct the (some of) database schema on the fly. It seems to be easily possible to reconstruct most of the table definition from the 'table' and 'columns' tables, but how do I retrieve key contraints? Specifically, given this table definition: create table test ( id integer primairy key, name varchar(10) ) , I get some data in the keycolums table, but don't know how to relate that to the tables and colums tables. I also need something simular for foreign keys, but haven't looked into that yet. Ronald P.S. I'm using the head of the CVS trunk.

4 3

[MonetDB-users] Comparing TPC-H benchmark results?
by Wouter Scherphof 16 Sep '06

16 Sep '06

Hi, The TPC-H benchmark (http://www.tpc.org/tpch/default.asp) is part of the MonetDB SQL source and the benchmark is run during weekends, as I can see on the TestWeb ( http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests03/times.…) How should I interpret the listed results? According to TPC, results should be stated as a "QphH@Size" value, where "QphH" is the number of queries per hour for the H benchmark, and "Size" is a "scaling factor". On the TestWeb, the results are stated as seconds of execution time. Is it correct to compute the QphH value as follows: Queries 01-22 took 1.985 seconds, so QphH is (3600 / 1.985) * 22 = 39.899 ? What scaling factor is used in the benchmark runs; is this 1? Is it possible to let the TestWeb run with scaling factor 100? Because that's the smallest Size for which results are published on the TPC web site. Did anyone yet try to determine the Price of the test platforms, following the TPC Pricing Spec? TPC-H results normally include a Price/Performance metric in the form of $/QphH@Size, and I guess MonetDB could score particularly well on this metric. Regards, Wouter Scherphof. Utrecht Professional School, The Netherlands.

3 3

[MonetDB-users] packages for YUM (fedora 5) and PORT (os x intel)?
by Shahbaz 15 Sep '06

15 Sep '06

Hi, I would like to use Monet DB, however the RPMs provided on your webpage don't seem to work for Fedora Core 5. Despite having these libraries installed, rpm -Uvh and rpm -ivh tell me that libMapi, libalgebra, lib_arith, etc. are needed by MonetDB-QL-server-2.12.0-1.x86_64.rpm. (I have AMD 64). Could someone please provide a yum repository so users can' install monetdb just by doing the following: sudo yum install monetdb I don't have an immediate need for a darwin-ports distribution, but it will be very helpful for those who would like to try monetdb on osx. Thanks

2 1

Re: [MonetDB-users] Fedora core 5 64 bit horribly slow compilation
by Fabian Groffen 11 Sep '06

11 Sep '06

On 11-09-2006 18:03:17 +0200, Tomas Brambora wrote: > Hello, Mr. Grofen, Hi Tomas, > I would like to ask whether there have been attempts to compile MonetDB on AMD Athlon 64 mobile achitecture OS Fedora Core 5 (64 bit version). > I ask because I was trying to compile it on my fresh installation of this OS and it just gets stuck on the file "gdk_scanselect.c". Or perhaps it doesn't get stuck, but I let it compile this file for 20 minutes and it didn't make any progress (it keeps on working with the harddisc though), then I lost patience. I know this file is BIG, but anyway, 20 minutes... On mobile chips we haven't tested, but on our own systems with athlon64's and Fedora Core 5 it takes a while to compile this file, especially with full optimisations on. 20 minutes sounds a bit long though. What's your CPU speed? And what is the active CPU speed when compiling MonetDB? I cc monetdb-users list, as maybe someone else has more experience with compiling on your CPU. Regards

2 1

[MonetDB-users] XML Query Test Suite 1.0 released
by Martijn Faassen 30 Aug '06

30 Aug '06

Hi there, The developers of MonetDB are probably already more than aware of this, but just in case: The w3c has released version 1.0 of the XML Query Test suite. They ask for comments and display reports on the conformance of various systems on their page: http://www.w3.org/XML/Query/test-suite/ http://www.w3.org/XML/Query/test-suite/XQTSReportSimple.html http://www.w3.org/XML/Query/test-suite/XQTSReport.html Perhaps it'd be nice to be included in those reports? Regards, Martijn

2 2

[MonetDB-users] Could not bootstrap buildtools on SunOS 5.10
by Ying Zhang 29 Aug '06

29 Aug '06

Hello, I'm trying to install MonetDB (CVS HEAD) on a SunOS 5.10 32 bits machine (electra at CWI), but I'm already stuck by bootstrapping the buildtools. I've attached the whole output of the bootstrap. Please note the message at the end of the output. I have no idea what is going wrong here, so any help will be appreciated! Kind regards, Jennie ========================================== $ ./bootstrap + rm -f configure + cp configure.ac configure + chmod +x configure + cd Mx + ./bootstrap /opt/sfw/share/aclocal/xmms.m4:17: warning: underquoted definition of XMMS_TEST_ VERSION run info '(automake)Extending aclocal' or see http://sources.redhat.com/automake/automake.html#Extending%20aclocal /opt/sfw/share/aclocal/xmms.m4:62: warning: underquoted definition of AM_PATH_XM MS /opt/sfw/share/aclocal/xdelta.m4:7: warning: underquoted definition of AM_PATH_X DELTA /opt/sfw/share/aclocal/vorbis.m4:9: warning: underquoted definition of XIPH_PATH _VORBIS /opt/sfw/share/aclocal/sdl.m4:11: warning: underquoted definition of AM_PATH_SDL /opt/sfw/share/aclocal/progtest.m4:13: warning: underquoted definition of AM_PAT H_PROG_WITH_TEST /opt/sfw/share/aclocal/ogg.m4:8: warning: underquoted definition of XIPH_PATH_OG G /opt/sfw/share/aclocal/lcmessage.m4:11: warning: underquoted definition of AM_LC _MESSAGES /opt/sfw/share/aclocal/imlib.m4:9: warning: underquoted definition of AM_PATH_IM LIB /opt/sfw/share/aclocal/imlib.m4:167: warning: underquoted definition of AM_PATH_ GDK_IMLIB /opt/sfw/share/aclocal/gimpprint.m4:8: warning: underquoted definition of AM_PAT H_GIMPPRINT /opt/sfw/share/aclocal/gimp.m4:8: warning: underquoted definition of AM_PATH_GIM P /opt/sfw/share/aclocal/gettext.m4:11: warning: underquoted definition of AM_WITH _NLS /opt/sfw/share/aclocal/gettext.m4:196: warning: underquoted definition of AM_GNU _GETTEXT /opt/sfw/share/aclocal/ao.m4:9: warning: underquoted definition of XIPH_PATH_AO /usr/share/aclocal/vorbis.m4:9: warning: underquoted definition of XIPH_PATH_VOR BIS /usr/share/aclocal/vorbis-old.m4:9: warning: underquoted definition of AM_PATH_V ORBIS /usr/share/aclocal/pkg.m4:5: warning: underquoted definition of PKG_CHECK_MODULE S /usr/share/aclocal/ogg.m4:8: warning: underquoted definition of XIPH_PATH_OGG /usr/share/aclocal/ogg-old.m4:8: warning: underquoted definition of AM_PATH_OGG /usr/share/aclocal/esd.m4:10: warning: underquoted definition of AM_PATH_ESD /usr/share/aclocal/esd.m4:175: warning: underquoted definition of AM_ESD_SUPPORT S_MULTIPLE_RECORD /usr/share/aclocal/audiofile.m4:12: warning: underquoted definition of AM_PATH_A UDIOFILE /usr/sfw/share/aclocal/gtk.m4:7: warning: underquoted definition of AM_PATH_GTK /usr/sfw/share/aclocal/glib.m4:8: warning: underquoted definition of AM_PATH_GLI B /usr/sfw/share/aclocal/gimpprint.m4:8: warning: underquoted definition of AM_PAT H_GIMPPRINT ./bootstrap: test: argument expected + exit 1

3 7