May 2007 - developers-list

[Monetdb-developers] usleep not portable
by Sjoerd Mullender 22 May '07

22 May '07

Jennie, you introduced a call to usleep in xrpc_client.mx. How important is that? The problem is, usleep is not portable. For one, it is not a Posix call (use nanosleep instead), and worse (in practice), it's not available on Windows (but neither is nanosleep). On Windows there is Sleep() which takes milliseconds as an argument (unlike sleep which takes seconds, usleep which takes microseconds, and nanosleep which has nanosecond precision). -- Sjoerd Mullender

2 1

[Monetdb-developers] MonetDB5 SQL installer for windows 32....
by Romulo Goncalves 21 May '07

21 May '07

During the installation of MonetDB5 SQL for windows 32 I got an error. The print screen is in the following link: http://homepages.cwi.nl/~goncalve/spool/Picture1.png Regards, Romulo

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pf_support.mx, 1.233, 1.234
by Stefan Manegold 16 May '07

16 May '07

Peter, triggered by --enable-optimize compilation warning about ======== .../pathfinder/runtime/pf_support.mx: In function 'CMDws_bat': .../pathfinder/runtime/pf_support.mx:7122: warning: 'v' may be used uninitialized in this function ======== I changed BAT *v, *b = BBPdescriptor(bid); into BAT *v = 0, *b = BBPdescriptor(bid); in CMDws_bat(), and noticed that CMDws_bat() returns GDK_SUCCEED in any case, now, although *ret is initialized/set only in case b = BBPdescriptor(bid) is non-null. Is this --- i.e., returning GDK_SUCCEED and not initializing *ret in case b = BBPdescriptor(bid) return NULL --- intended / OK? Stefan On Wed, May 16, 2007 at 10:08:57AM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16:/tmp/cvs-serv3794 > > Modified Files: > pf_support.mx > Log Message: > [ 1718066 ] PF: put() to /tmp crashes Mserver > - error in error message formatting > > > > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.233 > retrieving revision 1.234 > diff -u -d -r1.233 -r1.234 > --- pf_support.mx 16 May 2007 07:55:53 -0000 1.233 > +++ pf_support.mx 16 May 2007 10:08:54 -0000 1.234 > @@ -699,7 +699,7 @@ > } > var absolute := [or](filepath.[startsWith]("/"), filepath.[startsWith]("\\")).uselect(true).mirror().leftfetchjoin(sel); > if (count(absolute) > 0) { > - ERROR("fn_put: file URI '%s' must be a relative path (%d such errors).", backwards.fetch(0), count(backwards)); > + ERROR("fn_put: file URI '%s' must be a relative path (%d such errors).", absolute.fetch(0), count(absolute)); > } > var backwards := filepath.[search]("..").uselect(0,int_nil).mirror().leftfetchjoin(sel); > if (count(backwards) > 0) { > @@ -7118,13 +7118,19 @@ > int CMDws_bat(BAT **ret, lng* wsid) > { > bat bid = *wsid & 1073741823LL; > - BAT *v = NULL; > if (BBPfix(bid)) { > - BAT *b = BBPdescriptor(bid); > + BAT *v, *b = BBPdescriptor(bid); > if (b) *ret = v = VIEWcreate(b); > BBPunfix(bid); > + if (v) { > + /* give the view a name such that lng(bbname(v)) = lng(bbpname(b)) */ > + long_str buf; > + snprintf(buf, sizeof(buf), "%s-%d", BBP_logical(bid), ABS(b->batCacheid)); > + BBPrename(v->batCacheid, buf); > + return GDK_SUCCEED; > + } > } > - return v?GDK_SUCCEED:GDK_FAIL; > + return GDK_SUCCEED; > } > > #include "serialize.h" > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] MonetDB with Perl in windows
by Steffen Goeldner 16 May '07

16 May '07

Romulo Goncalves wrote: > Hello, > > My name is Romulo (INS-1) and I am trying to create a windows installer > with Perl. My colleagues told me that you are the Perl expert in MonetDB... > > To compile PERL I did: > > install perl in C:\perl_5\ > and unzip the sources in same directory as MonetDB SQL MONETDB5 .... > (The same steps as we should do for php} > > In the MonetDB\conf\rules.msc I added: > > PERL_SRCDIR = $(TOPDIR)\..\..\perl-5.8.8 > PERL_INSTDIR = C:\perl-5 > # for these, also see configure.py > PERL_EXTENSIONDIR = lib\perl5 > > Note: I am compiling for M5 and sql from cvs branch. > > For the compilation I set the flag HAVE_PERL=1. > > Everything compiled. > > Now I need to know which files I have to add to the installer. > I was looking for a perl lib in cliens\NT\lib\ and It does not have such > lib. > > Could you help me a bit please? First and foremost, let me admit that I did not find time to implement the new protocol (8) yet - thus the perl modules may not work for M5 (at least the pure perl (MapiPP) client). Anyway, this has nothing to do with the installation procedure. In the spirit of Perl’s motto, "There’s More Than One Way To Do It" (TMTOWTDI), there are many perl modules in clients/src/perl: - Mapi.pm is a simple pure perl client, maintained by CWI and works (AFAIK) with the current protocol - MapiLib (in Cimpl) is a wrapper around libMapi and you need SWIG to build it - The modules in MonetDB-CLI* are maintained by me: - MonetDB::CLI::MapiPP is pure perl - MonetDB::CLI::MapiLib is pure perl but uses MapiLib - MonetDB::CLI::MapiXS is a XS (that is: C) wrapper around libMapi - MonetDB::CLI is a simple loader for one of the MonetDB::CLI::* modules above - DBD::monetdb (in DBD) is a DBI client on top of MonetDB::CLI In general, you install a perl module in the following way: perl Makefile.PL make make test make install In this way, perl saves you from all the drudgery. Installing by hand is easy for pure perl modules (though not recommended): simple copy all *.pm into your perl lib directory. Example (MonetDB::CLI::MapiPP): MonetDB-CLI-MapiPP/MapiPP.pm -> /perl/site/lib/MonetDB/CLI/MapiPP.pm (assuming /perl/site/lib is in the lib search path - try 'perl -V' and look for @INC and PERL5LIB. Later, if you try to load MonetDB::CLI::MapiPP, perl replaces '::' with '/', adds .pm and looks in every path from @INC.) However, installing an extension module (XS or SWIG) is difficult! Perl builds additional files (e.g. .dll) which have to be installed into different places. Best you call the standard 'make install' from the installer or at least watch what 'make install' would do. Regards, Steffen

2 5

Re: [Monetdb-developers] Cubulus OLAP tuning advice & building Monet on OS X fails
by Niels Nes 14 May '07

14 May '07

On Mon, May 14, 2007 at 12:24:49AM +0300, Alexandru Toth wrote: > Yes: > > > swig -version > > SWIG Version 1.3.27 > Copyright (c) 1995-1998 > University of Utah and the Regents of the University of California > Copyright (c) 1998-2005 > University of Chicago > Compiled with /usr/bin/g++-4.0 [powerpc-apple-darwin8.8.0] That should do fine. In my linux version if python and swig are found by configure it builds the MonetSQLdb python module. It however doesn't install in the python path (as that would require root rights, which we cannot assume). Therefor we install it under $prefix, in a directory lib64/python2.4/site-packages/. I set my PYTHONPATH using export PYTHONPATH=$prefix/`python -c 'import distutils.sysconfig; print distutils.sysconfig.get_python_lib(0,0,"")'` Niels > > > ----- > > which swig > /opt/local/bin//swig > > > BR Alex > > On May 14, 2007, at 12:19 AM, Niels Nes wrote: > > >On Mon, May 14, 2007 at 12:07:04AM +0300, Alexandru Toth wrote: > >> > >> Hello, > >> > >> After removing the extra Python versions from the PATH, > >> monetdb-install.sh runs without errors. But there is no > >>MonetSQLdb , > >> so I started ./configure;make;sudo make install in > >> MonetDB-Venus-SuperBall-SR1/MonetDB-client/clients > > > >Do you also have swig installed? > > > >Niels > >> > >> Result is the same: > >> > >> -MonetSQLdb is not deployed to Mac OS X 10.4 default python's > >> site-packages > >> > >> -in > >> MonetDB-Venus-SuperBall-SR1/MonetDB-client/clients/src/python/ > >>Cimpl > >> there is a folder called import MonetSQLdb, so start python there, > >> type "import MonetSQLdb". Error message is: > >> > >> macu:~/Desktop/clients/src/python/Cimpl alex$ python > >> > >> Python 2.3.5 (#1, Mar 20 2005, 20:38:20) > >> > >> [GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin > >> > >> Type "help", "copyright", "credits" or "license" for more > >> information. > >> > >>>>>import MonetSQLdb > >> > >> Traceback (most recent call last): > >> > >> File "<stdin>", line 1, in ? > >> > >> File "MonetSQLdb/__init__.py", line 45, in ? > >> > >> from CMapi import * > >> > >> File "CMapi.py", line 17, in ? > >> > >> import MapiLib > >> > >> File "MapiLib.py", line 5, in ? > >> > >> import _MapiLib > >> > >> ImportError: No module named _MapiLib > >> > >>>>>^D > >> > >> macu:~/Desktop/clients/src/python/Cimpl alex$ ls > >> > >> CMapi.py Makefile.am MapiLib.py MapiLib.pyc > >> > >> CMapi.pyc Makefile.in MapiLib.py.c MonetSQLdb > >> > >> Makefile Makefile.msc MapiLib.py.i README > >> > >> macu:~/Desktop/clients/src/python/Cimpl alex$ ls MonetSQLdb/ > >> > >> Makefile __init__.py monetexceptions.py > >> > >> Makefile.am __init__.pyc times.py > >> > >> Makefile.in converters.py > >> > >> Makefile.msc cursors.py > >> > >> On Windows I did the same: copy manuallyMonetSQLdb to site- > >>packages, > >> and add the missing *.py and *.pyc files from all folders into > >> MonetSQLdb until import stopped complaining. However, on Mac can't > >> figure all the _ files, so I've got a Monet server + SQL > >>client, but > >> no python library. Please advice. > >> > >> BR Alexandru > >> > >> On May 9, 2007, at 10:33 AM, Fabian Groffen wrote: > >> > >> On 09-05-2007 10:28:02 +0300, Alexandru Toth wrote: > >> > >> Hello, > >> > >> True, Python 2.4 and 2.5 were installed from Darwinports. > >> > >> Where to configure which Python to use in MonetDB build? > >>Builtin > >> > >> Python2.3 would be ok. Or, just remove extra Pythons from > >>PATH ? > >> > >> Only the latest install script has support for giving configure > >> extra > >> > >> options. In your case I would opt for removing DP from your > >> > >> environment, however, then the build will fail because you miss > >>the > >> > >> functionality provided by DP. In my opinion, DP fails here, > >>because > >> it > >> > >> doesn't help you here, like it should. Anyway. In my own work > >>this > >> > >> whould have worked, here you probably need some LDFLAGS hackery to > >> point > >> > >> to your DP offset. LDFLAGS="-L/path/to/lib". Maybe that helps > >> here. > >> > >> There was another error of missing gawk in the main build > >>script > >> > >> (Python client makefile looks for awk gawk nawk..), and sed > >> > >> complaining about unsupported --version > >> > >> The error handler calls various tools with --version. If that > >>fails > >> we > >> > >> basically already know enough. In your case here, this is not > >> > >> important. > > > >-- > > > >Niels Nes, Centre for Mathematics and Computer Science (CWI) > >Kruislaan 413, 1098 SJ Amsterdam, The Netherlands > >room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 > >url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

[Monetdb-developers] Fwd: Re: Fwd: pf/tijah startNodes: node* or string*
by Jan Flokstra 10 May '07

10 May '07

---------- Forwarded Message ---------- Subject: Re: Fwd: pf/tijah startNodes: node* or string* Date: Tuesday 08 May 2007 17:13 From: Jens Teubner <jens.teubner(a)in.tum.de> To: Jan Flokstra <flokstra(a)cs.utwente.nl> Cc: monetdb-developers(a)lists.sourceforge.net On Tue, May 08, 2007 at 03:39:03PM +0200, Jan Flokstra wrote: > Hi Jens, > > Hope you are still in Munich:-) We have a little problem here which we > do not known how to debug. It is described extensively in the the > e-mail at the end of this mail. But the basic problem is we do not > know how to debug a typing problem. We have the following query: > > tijah:query(fn:collection("c"),"//LINE[about(.,foul)]") > > And the signatures of the function involved are: > > (0) fn:collection (string) as node* > (1) tijah:query(string) as node* > (2) tijah:query(item*, string) as node* > (3) tijah:query(string, item) as node* > (4) tijah:query(item*, string, item) as node* Yes, I am still here. First of all, the function signatures that you listed here are the ones that appear in the *comments* in xquery_fo.c. Unfortunately, the code says something different! Very dangerous! I have corrected this some minutes ago. So here's what happens during Core generation and type checking and why your example makes trouble. (It boils down to a conflict between the signatures (2) and (3).) Out of the possible implementations for a function (tijah:query()), Pathfinder's function scope checker assigns those to the function application that have the matching *number of parameters*. This is in the spirit of the W3C specs: user-defined functions may be overloaded, but must differ in the number of arguments (and not just be distinguishable by their types). Only built-in functions may be overloaded in the sense that two functions may have the same number of arguments, but different argument types. Then, Core generation in fs.brg looks at the required type for each of the arguments. If the function expects an atomic value, the function argument is *atomized* (i.e., nodes are converted into something atomic, typically xs:untypedAtomic). Pathfinder's type checker finally selects the most specific implementation if there is more than one for the given number of argument (only really relevant for built-ins). The problem that you are experiencing is a combination of the two foregoing paragraphs. The function scope checker assigns those function implementations with two arguments to the `apply' node (signatures (2) and (3)). Core generation then (randomly) picks on of these two to determine whether atomization is required. Type checking finally throws the error. This behavior is perfectly okay for all functions we had so far and for all functions in the W3C specs. The combination of (a) two function signatures with the same number of arguments and (b) arguments having a node type in one signature and an atomic type in the other is what bites you. You probably should rethink your two function signatures (2) and (3). Note that they also introduce some ambiguity. If I call the function as `tijah:query (node, node)', should the first or the second argument be casted to an atomic value? Note that this implicit atomization happens everywhere in XQuery (if necessary) and it is perfectly consistent to do this here as well. A clean solution would work without such two conflicting signatures. A hack could be to treat tijah:query() explicitly in the involved parts in the Pathfinder code and do something magic to make things work. I would not recommend to do so, though, as it primarily would introduce inconsistency. Don't hesitate to ask again if my answers weren't clear. Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Please avoid sending me MS Word, MS Excel or MS PowerPoint files. Why? See http://www.fsf.org/philosophy/no-word-attachments.html -------------------------------------------------------

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, 1.181, 1.182
by Sjoerd Mullender 09 May '07

09 May '07

On 05/09/2007 01:58 PM, Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16:/tmp/cvs-serv2304 > > Modified Files: > gdk_utils.mx > Log Message: > bug with terabyte memory usage was introduced in 1.172 (sjoerd) > - GDKmallocmax(size, *maxsize) => > this routine is also used for realloc and maxsize is an input parameter, > so must be initialized. Did you look at the *reason* for that change? I think not. The reason was that the size value afterwards was wrong for the call to memset. Now you have reintroduced that bug. What you *should* have done was to initialize maxsize with size. > > Index: gdk_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > retrieving revision 1.181 > retrieving revision 1.182 > diff -u -d -r1.181 -r1.182 > --- gdk_utils.mx 2 May 2007 16:17:04 -0000 1.181 > +++ gdk_utils.mx 9 May 2007 11:58:06 -0000 1.182 > @@ -1070,11 +1070,8 @@ > void * > GDKzalloc(size_t size) > { > - size_t maxsize; > - void *p = GDKmallocmax(size, &maxsize, 1); > - > - if (p) > - memset(p, 0, size); > + void *p = GDKmallocmax(size, &size, 1); > + if (p) memset(p, 0, size); > return p; > } > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

[Monetdb-developers] SERIAL column,
by Romulo Goncalves 09 May '07

09 May '07

Good afternoon, sql>create table t1 (id SERIAL, PRIMARY KEY(id)); MAPI = monetdb@localhost:50000 QUERY = create table t1 (id SERIAL, PRIMARY KEY(id)); ERROR = !CONSTRAINT PRIMARY KEY: a table can have only one PRIMARY KEY sql>create table t1 (id SERIAL); sql>insert into t1 values(5); [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; MAPI = monetdb@localhost:50000 QUERY = insert into t1; ERROR = !SQLException:assert:INSERT INTO: PRIMARY KEY constraint 't1.t1_id_pkey' violated sql> The constraint check is correct, but should the first insertion be allowed? I checked the parser to know what SERIAL means: | column SERIAL { /* SERIAL = INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY */ Then I check what means GENERATED ALWAYS means: http://troels.arvin.dk/db/rdbms/#mix-identity "When ALWAYS is specified, the user cannot specify a value for the column which means that the DBMS can guarantee successful insertion of a unique value on each table insert." Who is right? The web-page or MonetDB? I tried to find something in the standard book and nothing (maybe I did not check correctly). Regards, Romulo

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_server.mx, 1.44, 1.45
by Fabian Groffen 09 May '07

09 May '07

On 09-05-2007 08:40:10 +0000, Stefan Manegold wrote: > In any case, this once again proves how important and "handy" > multi-platform testing is to detect and eventually fix bugs > --- provided someone does care at all ... Yeah, but not for any price. While you can get it for free, you appear to *insist* on paying a very high price for it.

1 0

[Monetdb-developers] Optimizers...
by Romulo Goncalves 08 May '07

08 May '07

Please, when you change an optimizer or you insert a new optimizer check the tests before and after your changes. After 1000 times it is time to learn. The test sql/src/test/Triggers/trigger_activation.sql is red. However, if I do set optimizer='off'; on the begin of the test it starts to work. Sorry, but this time I will not fix this bug. Regards, Romulo

2 4