- developers-list - monetdb.org

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder MonetDB-XQuery.spec.in, , 1.42, 1.43
by Sjoerd Mullender 13 Nov '07

13 Nov '07

Please spell check. Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv27856 > > Modified Files: > MonetDB-XQuery.spec.in > Log Message: > - added a separate JAR package for the XRPC java API files > - added the new file for 'make rpm' as well > > > > > Index: MonetDB-XQuery.spec.in > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/MonetDB-XQuery.spec.in,v > retrieving revision 1.42 > retrieving revision 1.43 > diff -u -d -r1.42 -r1.43 > --- MonetDB-XQuery.spec.in 15 Oct 2007 15:39:06 -0000 1.42 > +++ MonetDB-XQuery.spec.in 13 Nov 2007 13:56:50 -0000 1.43 > @@ -90,14 +90,21 @@ > automatic index management, extensibility of data types and search > accelerators, SQL- and XQuery- frontends. > > -This package contains an XRPC Wrapper (xrpcwrapper.jar), a stand-alone > -Java program that can be pluged in on top of an XQuery engine to handle > -non-nested XRPC requests (i.e. requests that do not trigger another XRPC > -request). > +This package contains three Java Archieves > +- xrpcwrapper.jar: the XRPC Wrapper, which is a stand-alone Java program > + that can be pluged in on top of an XQuery engine to handle non-nested > + XRPC requests (i.e. requests that do not trigger another XRPC > + request). > > -This package also contains an example client program > -(xrpcwrapper-test.jar), which (i) can be used to test the XRPC Wrapper, > -and (ii) shows how to write an XRPC Java client. > +- xrpcwrapper-test.jar: an example client program, which (i) can be used > + to test the XRPC Wrapper, and (ii) shows how to write an XRPC Java > + client. > + > +- xrpcutil.jar: the XRPC Java API, which includes funcionts for > + -- creating XRPC request messages, > + -- parsing XRPC response messages, > + -- send and/or receive XRPC messages using HTTP connection, > + -- XRPC specific exception classes > > %endif > > @@ -162,7 +169,7 @@ > %files xrpcwrapper > %defattr(-,root,root) > %dir %{_datadir}/MonetDB/lib > -%{_datadir}/MonetDB/lib/xrpcwrapper*.jar > +%{_datadir}/MonetDB/lib/xrpc*.jar > %endif > > %files devel > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] Physical stack space
by Maurice van Keulen 12 Nov '07

12 Nov '07

Hi, When writing recursive xqueries that that manipulate somewhat larger documents, MonetDB frequently quits with the error message !ERROR: interpret: running out of physical stack space! Is there a way to increase the physical stack space? I found no parameter to set in MonetDB.conf, perhaps in the source somewhere? Maurice. -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing Mtest.py.in, , 1.320, 1.321
by Stefan Manegold 07 Nov '07

07 Nov '07

On Wed, Nov 07, 2007 at 03:44:26PM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv21513 > > Modified Files: > Mtest.py.in > Log Message: > > added pythonlibdir (obtained from monetdb-config and monetdb-clients-config) > into the PYTHONPATH (in an attempt to fix the problems with tests using > MonetSQLdb) should the respective test do this locally? I recall addint something like this to (some of) the MonetSQLdb tests ... Stefan > > > > Index: Mtest.py.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/testing/Mtest.py.in,v > retrieving revision 1.320 > retrieving revision 1.321 > diff -u -d -r1.320 -r1.321 > --- Mtest.py.in 2 Nov 2007 07:31:36 -0000 1.320 > +++ Mtest.py.in 7 Nov 2007 15:44:24 -0000 1.321 > @@ -102,6 +102,17 @@ > import distutils.sysconfig > p = _configure(os.path.join('@QXprefix@',distutils.sysconfig.get_python_lib(1,0,""))) > sys.path.insert(0, p) > + try: > + q = GetConfig('monetdb-config', '--pythonlibdir') > + p = p + os.pathsep + q > + except: > + None > + try: > + q = GetConfig('monetdb-clients-config', '--pythonlibdir') > + p = p + os.pathsep + q > + p = p + os.pathsep + q + os.sep + 'MonetDB' > + except: > + None > from MonetDB import monet_options > if os.environ.has_key('PYTHONPATH'): > p = p + os.pathsep + os.environ['PYTHONPATH'] > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Does Monetdb support Chinese?
by ding_guangfu＠topsec.com.cn 07 Nov '07

07 Nov '07

As Title. Thanks.

2 1

[Monetdb-developers] MAPI wire protocol?
by Tim Gebhardt 07 Nov '07

07 Nov '07

Hello, I'm working on a C#/.NET client library to connect to a MonetDB server. I've mapped most of the libMapi.dll api calls so that I can call the C library from .NET but this presents a few problems: 1. There's a performance penalty to interface native code libraries to .NET 2. There's a lot of overhead of marshaling UTF8 strings between .NET and the C library, and potential problems with NULL parameters. 3. There's a portability problem with using the native libraries (trouble deploying to the Mono runtime) I think the client library would be better off implementing the MAPI wire protocol rather than interfacing with the C library, but I cannot find any documents that detail the protocol. Is there somewhere this is published, or did I just not Google the right stuff? Also if anyone is interested in giving me a hand you can find the Google Code workspace of the aforementioned project at this link: http://code.google.com/p/monetdb-net/ -Tim Gebhardt tim(a)gebhardtcomputing.com

3 2

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/php/Cimpl Makefile.ag, Clients_1-20, 1.4.4.1, 1.4.4.2
by Fabian Groffen 06 Nov '07

06 Nov '07

On 06-11-2007 11:55:48 +0000, Stefan Manegold wrote: > since compilation with strict compiler settings have (once again) > proved to be bery useful to locate bugs in the code, > we nolonger simply disable all of the strict "X_CFLAGS" > in directories that (also) contain swig-code, > but rather selectively disable only those extra warnings/errors > that do indeed occur in code that is beyond our control. > > (This affects only compilation with --enable-strict.) ... > +CFLAGS = $(GCC_SWIG_CFLAGS) $(ICC_SWIG_CFLAGS) -DCOMPILE_DL_MONETDB=1 -D_GNU_SOURCE Unless I misunderstood the code in the .m4, this looks like the user specified CFLAGS are overriden here, which is an undesirable effect.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/java build.properties, XQuery_0-20, 1.1.2.3, 1.1.2.4
by Fabian Groffen 01 Nov '07

01 Nov '07

On 01-11-2007 10:48:18 +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/src/tools/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5671 > > Modified Files: > Tag: XQuery_0-20 > build.properties > Log Message: > Corrected value of the "compiler" attribute of <javac> [snip] > # which Java compiler to use, default: javac > -build_compiler=javac > +build_compiler=javac1.5 While the comment doesn't reflect a valid state here, this isn't too flexible. Why don't you just drop the whole build_compiler variable you introduced, and use the completely transparent and well supported and usually expected build.compiler variable only once you really want to control what compiler is being used, as I suggested to you earlier?

4 6

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/java build.properties, XQuery_0-20, 1.1.2.4, 1.1.2.5 build.xml, XQuery_0-20, 1.1.2.4, 1.1.2.5
by Sjoerd Mullender 01 Nov '07

01 Nov '07

Ying Zhang wrote: > diff -u -d -r1.1.2.4 -r1.1.2.5 > --- build.properties 1 Nov 2007 10:48:16 -0000 1.1.2.4 > +++ build.properties 1 Nov 2007 12:49:25 -0000 1.1.2.5 > @@ -14,4 +14,5 @@ > javac_version=1.5 > > # which Java compiler to use, default: javac > -build_compiler=javac1.5 > +build.compiler=javac1.5 > + > Do we really need this property? Isn't the default good enough? Won't still this require Sun Java (since you mentioned you tried it with value gcj to compile using gcj)? If so, then that isn't good enough. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/java build.properties, XQuery_0-20, 1.1.2.2, 1.1.2.3 build.xml, XQuery_0-20, 1.1.2.3, 1.1.2.4
by Fabian Groffen 31 Oct '07

31 Oct '07

On 30-10-2007 22:33:56 +0000, Ying Zhang wrote: > Added a new variable in build.properties (build_compiler) to make > compiler switching easier. http://ant.apache.org/manual/CoreTasks/javac.html <quote> It is possible to use different compilers. This can be specified by either setting the global build.compiler property, which will affect all <javac> tasks throughout the build, or by setting the compiler attribute, specific to the current <javac> task. Valid values for either the build.compiler property or the compiler attribute are: </quote> In other words, don't do what you just did. Just rely on standard behaviour, and set build.compiler in your build.properties to gcj if you want to build with gcj.

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra intro_rec_border.c, XQuery_0-20, 1.5, 1.5.4.1
by Stefan Manegold 30 Oct '07

30 Oct '07

On Mon, Oct 29, 2007 at 10:00:19AM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15130/algebra > > Modified Files: > Tag: XQuery_0-20 > intro_rec_border.c > Log Message: > -- Fix a bug for recursive queries detected by Maurice. which one? and should it (the bug, resp. its fix) be tested? Stefan > Index: intro_rec_border.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/intro_rec_border.c,v > retrieving revision 1.5 > retrieving revision 1.5.4.1 > diff -u -d -r1.5 -r1.5.4.1 > --- intro_rec_border.c 27 Jun 2007 13:46:26 -0000 1.5 > +++ intro_rec_border.c 29 Oct 2007 10:00:16 -0000 1.5.4.1 > @@ -95,6 +95,13 @@ > } > break; > > + case pa_fcns: > + /* this also skips the introduction of a rec_border > + operator for the content of an empty elements: > + elem (fcns (nil, nil)). */ > + if (R(n)->kind == pa_nil) > + break; > + /* else fall through */ > default: > /* follow the children until a base or a leaf is reached */ > for (unsigned int i = 0; i < PFPA_OP_MAXCHILD && n->child[i]; i++) > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > > > > > > > > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_builder.mx, MonetDB_5-2, 1.26, 1.26.2.1
by Stefan Manegold 29 Oct '07

29 Oct '07

On Mon, Oct 29, 2007 at 07:09:42AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18604/src/mal > > Modified Files: > Tag: MonetDB_5-2 > mal_builder.mx > Log Message: > pushStr shouldn't do a GDKstrdup, as it may return an equal string from > the constant cache, ie only if the returned variable holds the given string > (poniters are equal) we do a string dup. This solves another memory leak. > > > Index: mal_builder.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_builder.mx,v > retrieving revision 1.26 > retrieving revision 1.26.2.1 > diff -u -d -r1.26 -r1.26.2.1 > --- mal_builder.mx 28 Aug 2007 19:56:41 -0000 1.26 > +++ mal_builder.mx 29 Oct 2007 07:09:40 -0000 1.26.2.1 > @@ -260,10 +260,12 @@ > ValRecord cst; > > cst.vtype= TYPE_str; > - cst.val.sval= GDKstrdup(val); > + cst.val.sval= val; gcc -DHAVE_CONFIG_H -I. -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal -I../.. -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal -I../optimizer -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/../optimizer -I../scheduler -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/../scheduler -I../modules/kernel -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/../modules/kernel -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/include/MonetDB -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/include/MonetDB/common -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/include/MonetDB/gdk -DLIBMAL -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -D_REENTRANT -c mal_builder.c -fPIC -DPIC -o .libs/libmal_la-mal_builder.o cc1: warnings being treated as errors /ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/mal_builder.mx: In function 'pushStr': /ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/mal_builder.mx:263: warning: assignment discards qualifiers from pointer target type make[5]: *** [libmal_la-mal_builder.lo] Error 1 Stefan > cst.len= strlen(cst.val.sval); > _t = defConstant(mb,TYPE_str,&cst); > > + if (getConstant(mb,_t).val.sval == val) > + getConstant(mb,_t).val.sval = GDKstrdup(val); > return pushArgument(mb, q, _t); > } > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 4

[Monetdb-developers] MonetDB(5) and modules
by Fabian Groffen 26 Oct '07

26 Oct '07

I noticed that MonetDB5 uses modules, but compiles them as shared libraries. Maybe other targets do the same here. autogen appears to have (had) some partial support for modules, and the attached patch fixes/re-enables that support. In short it passes -module to libtool during link phase, such that the object is generated as loadable module. Why did I look at this? Mainly because on Darwin there is a difference between those objects that are loaded by dlopen(3) (loadable modules) and those that are used by the link editor ld(1) (shared libraries). The latter have type DYLIB whereas the former have type MH_BUNDLE. A so-called "bundle" doesn't have an install_name, as it can't be used by the link editor. Now initially all went well during compilation until I hit this error: /Library/Gentoo/usr/bin/ld: ../atoms/.libs/lib_color.so is input for the dynamic link editor, is not relocatable by the static link editor again This message is correct on Darwin, as batcolor.mx links against lib_color.so. libtool already detected this case: *** Warning: Linking the shared library lib_batcolor.la against the loadable module *** lib_color.so is not portable! ** Warning, lib lib_color.so is a module, not a shared library ** And there doesn't seem to be a static archive available ** The link will probably fail, sorry I don't recall ever seeing problems with these loads (apart from on Panther which didn't grok it at all) on my (as of today out-of-date) OSX 10.4, so I'm not sure how relevant it is to actually fix this. It looks like support to load shared libraries via dlopen actually was added, so we need not to worry about it at all. Actually, the man-page says it all: "dlopen() can load dynamic libraries and bundles." "In Mac OS X 10.4, dlopen was rewritten to be a native part of dyld." So far for sharing my experiences in this area. As far as I know, 10.3 support for MonetDB was shot already long ago (as it should), so maybe we should remove the remaining bits that could/can trigger -module libtool invocation, as it *will* cause trouble.

1 0

[Monetdb-developers] problems with XQuery aggregation
by Henning Rode 25 Oct '07

25 Oct '07

hej monetdb developers, i tried to write a simple aggregation query: let $cands := doc("x.xml")//c for $eid in distinct-values($cands/@id) return <candidate id="{$eid}" count="{count($cands[@id eq $eid])}"/> however, executing the above query turns out to be rather difficult. i always killed the process after waiting for more than half an hour without getting any result. x.xml is 17MB large and contains 300000 candidates nodes c. i run it on Monet Database Server V4.19.0 where the query crashed ERROR = !ERROR: GDKload: cannot mmap(): name=10/1030, ext=buns.priv !OS: Cannot allocate memory !ERROR: GDKload failed: name=10/1030, ext=buns.priv !ERROR: CMDleftjoin: operation failed. and on the newer stable release MonetDB Server v4.20.0 on a machine with 16GB memory where the query just runs endless. apparently the query compiler does not recognise the necessary join and aggregation here. is that a known problem? btw. sorting the candidates by @id value is done in seconds: let $cands := doc("x.xml")//c for $c in $cands order by $c/@id return $c best -henning

3 4

[Monetdb-developers] Segmentation Fault upon sql.start()
by Stefan de Konink 22 Oct '07

22 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 After loading in a bunch of data the monetdb server was Killed due to memory starvation. With swap MonetDB performed the huge amount of one insert in 10 seconds, so that wasn't an option too. I restarted it, and tried to 'load the data again'. Backtrace: gdb ./mserver5 GNU gdb 6.7 Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu"... Using host libthread_db library "/lib/libthread_db.so.1". (gdb) run Starting program: /opt/monetdb/bin/mserver5 warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff275fe000 [Thread debugging using libthread_db enabled] [New Thread 0x2afd85f0f2a0 (LWP 23929)] # MonetDB Server v5.0.0 # Copyright (c) 1993-2007 CWI, all rights reserved # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # dbname:osm # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /opt/monetdb/etc/monetdb5.conf) [New Thread 0x40800950 (LWP 23932)] >include sql; >sql.start(); Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2afd85f0f2a0 (LWP 23929)] 0x00002aaaaab40a8f in load_column () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so (gdb) bt #0 0x00002aaaaab40a8f in load_column () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #1 0x00002aaaaab41a71 in load_table () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #2 0x00002aaaaab433cc in load_trans () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #3 0x00002aaaaab46e15 in store_init () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #4 0x00002aaaaab1b95b in mvc_init () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #5 0x00002aaaaaad0b50 in SQLinit () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #6 0x00002afd8389f476 in initScenario () from /opt/monetdb/lib/libmal.so.0 #7 0x00002aaaaaad0ac8 in SQLsession () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #8 0x00002afd8387842d in runMALsequence () from /opt/monetdb/lib/libmal.so.0 #9 0x00002afd83879dfd in runMAL () from /opt/monetdb/lib/libmal.so.0 #10 0x00002afd83871dc8 in MALengine () from /opt/monetdb/lib/libmal.so.0 #11 0x00002afd8389f082 in runScenarioBody () from /opt/monetdb/lib/libmal.so.0 #12 0x00002afd8389f0d2 in runScenario () from /opt/monetdb/lib/libmal.so.0 #13 0x00002afd838719e0 in MSserveClient () from /opt/monetdb/lib/libmal.so.0 #14 0x0000000000402c14 in main () (gdb) It work doesn't anymore... I can conclude, inserting a lot of data start to become a huge waste of time. If required I can recompile with the -g option and a lower amount of optimization. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHHQTsYH1+F2Rqwn0RCjWKAJ9u/OEu8Sk6HTSY+v431bLjmylrCwCffsNN Rpf0Fs4XjrakYkdyEwkknWQ= =0UHS -----END PGP SIGNATURE-----

2 2

[Monetdb-developers] python development
by Sjoerd Mullender 22 Oct '07

22 Oct '07

For all you Python developers, a new rule is being enforced from now on: Python source files are not allowed to contain TAB characters or trailing white space. This is enforced at cvs commit time. -- Sjoerd Mullender

1 0

[Monetdb-developers] man-pages
by Fabian Groffen 22 Oct '07

22 Oct '07

We should put or html-ified (autogenerated) man-pages for merovingian, monetdb and mclient on our website, as easy to read documentation.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing Mtest.py.in, MonetDB_1-20, 1.309.2.8, 1.309.2.9
by Fabian Groffen 22 Oct '07

22 Oct '07

On 21-10-2007 16:36:26 +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv16749/src/testing > > Modified Files: > Tag: MonetDB_1-20 > Mtest.py.in > Log Message: > export TST_FIVE, we sometimes need to know which server we test > > > Index: Mtest.py.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/testing/Mtest.py.in,v > retrieving revision 1.309.2.8 > retrieving revision 1.309.2.9 > diff -u -d -r1.309.2.8 -r1.309.2.9 > --- Mtest.py.in 19 Oct 2007 13:41:24 -0000 1.309.2.8 > +++ Mtest.py.in 21 Oct 2007 16:36:21 -0000 1.309.2.9 > @@ -1237,6 +1237,7 @@ > os.environ['TST_MODS'] = str(env['TST_MODS']) > if par['M5']: > env['TST_FIVE'] = "Five" > + os.environ['TST_FIVE'] = "Five" This looks like an upcoming IndentationError to me. > return rtrn > ### GetBitsAndOIDsAndModsAndStatic(env) #

2 2

[Monetdb-developers] Mserver5 issues / Perl patches
by Stefan de Konink 21 Oct '07

21 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, [If the first part should be in monetdb-users please inform me] Currently I am still using the same setup, because I do not have the time to reimport my data using after the nightly builds. After my first big import in SQL resulting in 4*10^7 rows, I decided to move on. And continue my work. I have started mserver5 again, include sql; [no problems] sql.start(); [this command took 10 minutes to be processed] After this 10 minutes, the command works instantly when running the server again. Is this some kind of integrity checking? In my last e-mail I asked about the Perl issues, Niels pointed that the Perl sources were out of sync. I took the liberty to try to come up with a fix, please check as attached. I think no copyright can be claimed on 3 changes, so if this is the right solution :) Please apply :) Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHGqTsYH1+F2Rqwn0RCnZqAJ9l4IMVeGOCMm7JJN0BijnCvEIrAQCdEDd0 8zEe0HfX1ygvsJdawvOC63Q= =RqZ8 -----END PGP SIGNATURE-----

2 1

[Monetdb-developers] Perl documentation and connectivity
by Stefan de Konink 20 Oct '07

20 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, I'm trying the first lines of the Perl example presented: http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/The-Perl-Library.h… # connect to the database: my $dsn = 'dbi:monetdb:database=test;host=localhost;port=50000;language=mil'; my $dbh = DBI->connect( $dsn, undef, undef, # no authentication in MIL { PrintError => 0, RaiseError => 1 } # turn on exception handling ); skinkie@xen01 ~/private $ perl test.pl Start a simple Monet MIL interaction data sources: dbi:monetdb: DBI connect('database=test;host=localhost;port=50000;language=mil','',...) failed: RuntimeError Usage: mapi_connect(host,port,username,password,lang,dbname); at test.pl line 15 (I have traced this back to: MonetDB/CLI/MapiLib.pm) Is there some trivial thing I have missed? (I'm definitely not a Perl hero so forgive me for my incompetence.) Yours Sincerely, Stefan de Konink ps. Running: # MonetDB Server v5.0.0 # Copyright (c) 1993-2007 CWI, all rights reserved # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # dbname:demo ps2. I know test!=demo -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHGU0jYH1+F2Rqwn0RCvlvAJ0XbNzyV8ySOyoNcKbmitNWUSwH1gCfaDma /kMe7PQ4nl9cRhMcKpun2Uw= =BP78 -----END PGP SIGNATURE-----

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/debug logdebug.c, XQuery_0-20, 1.72, 1.72.2.1
by Stefan Manegold 18 Oct '07

18 Oct '07

Sjoerd, not that I want to "worry" you, but just out of curiosity: did this one make it into the release? If not, I would not recomment to re-build the release once again --- unless Jan has very strong arguments that this is a strickt show-stopped (don't think so, since most tests work fine) ---, but just keep it for a potential bug-fix release. Stefan On Wed, Oct 17, 2007 at 04:29:31PM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/debug > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26773/debug > > Modified Files: > Tag: XQuery_0-20 > logdebug.c > Log Message: > -- Bugfix: Added missing 'break;'. > > > Index: logdebug.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/debug/logdebug.c,v > retrieving revision 1.72 > retrieving revision 1.72.2.1 > diff -u -d -r1.72 -r1.72.2.1 > --- logdebug.c 18 Sep 2007 11:46:58 -0000 1.72 > +++ logdebug.c 17 Oct 2007 16:29:28 -0000 1.72.2.1 > @@ -1494,6 +1494,7 @@ > PFatt_str (n->sem.pos_sel.part)); > > PFarray_printf (xml, " </content>\n"); > + break; > > case la_fun_1to1: > PFarray_printf (xml, > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred encoding.c, XQuery_0-20, 1.8.4.3, 1.8.4.4
by Sjoerd Mullender 18 Oct '07

18 Oct '07

Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19535 > > Modified Files: > Tag: XQuery_0-20 > encoding.c > Log Message: > -- Bugfix: Added missing quotes. > > > Index: encoding.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/encoding.c,v > retrieving revision 1.8.4.3 > retrieving revision 1.8.4.4 > diff -u -d -r1.8.4.3 -r1.8.4.4 > --- encoding.c 9 Oct 2007 19:54:07 -0000 1.8.4.3 > +++ encoding.c 18 Oct 2007 08:02:23 -0000 1.8.4.4 > @@ -146,7 +146,7 @@ > if (shredstate.names_separate) > fprintf(out, "%i", tuple.name_id); > else > - fprintf(out, "%s", (char*)tuple.name); > + fprintf(out, "\"%s\"", (char*)tuple.name); > } > break; > case 't': > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins Too late. I have built the installers, RPMs, and tar balls. This change is not in it. Unless it is a show stopper, in which case I will be extremely unhappy, but seeing that it is in the sqlhelpers directory, I don't believe that. -- Sjoerd Mullender

2 1

[Monetdb-developers] next release: mps or algebra?
by Peter Boncz 17 Oct '07

17 Oct '07

Stefan suggested rightly to move this discussion to the mailing list. I also changed the language into english for that purpose. Maurice' original questions were: (a) whether the new release would be algebra-only (short answer: hopefully). (b) whether our talk of a new release also refers to submodules, such as probxml and pftijah (short answer: yes) (c) whether all issues on the wiki, see: http://www.pathfinder-xquery.org/wiki/index.php/Jupiter:_The_Algebra_Release needed for this action, have already been resolved (short answer: no) On Tue, Oct 16, 2007 at 05:23:43PM +0200, Peter Boncz wrote: > Hi Maurice, et al, > > The release of (hopefully) this week will still be mps based. I also > anticipate at least one bug-fix release (ie 0.20.2), or even more. However, > the next major release (0.22) should be algebra based, in my opinion. I am > certainly reluctant to keep investing any more effort in mps. I would aim > for an mps-free release, yes. > > The time-goal of that release is "XMAS 2007", as far as we are concerned. > For the work of built-in functions I have good hope that with the help of > Lefteris we can come a long way. > > I hope and think updates can also be ported by joerd in that time-frame. As > for pftijah, if Jan Flokstra has time, it should be possible to port that as > well in 2-3 months. > > There are some hairy issues, one of them being free-form recursion (needed > by you), another one being precompiled modules (needed by XRPC). The > resolution of these needs assistance from Munich, and maybe related. It needs > a method to compile certain functions into scripted procedures (here MIL) > and calling them. One algorithm is to do this for all functions in modules, > as well as for all recursive functions. The issue to resolve is the > representation of the parameters. > > Depending on how much time Jan Rittinger has available for this, we will > have to find creative solutions for this. Indeed, a backup solution would be > to keep handling certain queries for the time being with mps, but that > surely is ugly. > > Peter > > -----Original Message----- > From: Keulen, M. van (Maurice) [mailto:m.vankeulen@utwente.nl] > Sent: dinsdag 16 oktober 2007 16:38 > To: Peter Boncz [CWI]; Stefan Manegold [CWI] > Cc: Djoerd Hiemstra; Jan Flokstra; Henning Rode; Abdel Kader, R. (Riham) > Subject: Nieuwe releases en mps-backend > > > Hoi, > > Sorry dat ik niet bij de Skype-sessie van vandaag kon zijn, maar Riham > vertelde me dat het onderwerp van nieuwe releases en de mps-backend ter > sprake is geweest. Het riep wat specifieke vragen bij me op die ze niet > allemaal kon beantwoorden: > > a.. Is het de bedoeling dat een nieuwe release de mps-backend helemaal > niet meer bevat of alleen maar de algebra-backend als default heeft? > b.. Als jullie praten over "nieuwe release", bedoel je dan alleen een > package van alle modules van MonetDB/XQuery? Maw zegt het niets over > eventuele `subreleases' van individuele modules? > > c.. Op de wiki hebben we een pagina met open issues voor de > algebra-release. Wat is de status van die open issues? Ik zie namelijk op de > pagina nergens dat bepaalde issues al opgelost/geïmplementeerd zijn. > Ik Cc deze e-mail even aan de PF/Tijah-mensen, want hoewel er levendige > discussies zijn hoe PF/Tijah naar de algebra-backend to porten, ik heb niet > het idee dat er oplossingen zijn die op korte termijn realiseerbaar zijn. > Evenzo het recursie-vraagstuk. Het lijkt me bijvoorbeeld voor de > zichtbaarheid naar buiten toe onwenselijk als deze features ineens zouden > wegvallen voor non-insiders. Of andere onwenselijke consequenties van het > niet meer (sub)releasen van modules die gebaseerd zijn op de mps-backend. > > Groetjes, > Maurice. > > -- > ---------------------------------------------------------------------- > Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology > Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands > Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 > Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

5 8

[Monetdb-developers] FW: [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, 1.143, 1.143.2.1
by Peter Boncz 17 Oct '07

17 Oct '07

Thanks Stefan, > I'll leave the further interpretation of the above results to > the interested recipient / reader. I will give it a try, then. > S08-64: > SR QR SU QU[1] QU[2] > mh 659m34s 1m09s 81m06s ERROR - > mH - - 383m17s ERROR - > Mh - - 77m03s 344m32s 1m34s > MH 644m01s 0m49s 390m42s 342m47s 1m36s These h/SU results are strange, because a 5 x faster time due to a hash function with more collisions is.. puzzling. In any case, with 64-bits oids on a 8GB machine, the shredding generates very intense swapping, and Linux performance under swapping is known to be highly variable. Difficult to draw conclusions. I am still investigating the behavior of MT_mmap under load and it may well be that shredding performance can be improved by better memory management (less swapping). From the result below we know that it can be done in 25m. The not-yet checked in improvements in index construction will strongly reduce the QU[1] time by taking less memory; in the order of the 100m seen below. > S16-32: > SR QR SU QU[1] QU[2] > mh 127m59s 0m17s 43m14s ERROR - > mH 110m33s 0m16s 26m26s ERROR - > Mh 128m11s 0m18s 44m00s 100m50s 1m15s > MH 191m42s 0m17s 25m43s 101m37s 1m21s These numbers show that MonetDB prefers to operate on RAM resident data. I am glad you reproduced my 25 minutes h/SU result. Must be a coicidence that the mini hash self-join benchmark on 50M string numbers ran in 26s with the new hash and 43s with the old hash (we see the same numbers in minutes here repeated for shredding). In my current version, QU[2] with the ordered indices is back at 17s. Reason for 1m15s is that you tend to loose the hash-table on the index bats due to swapping. So that includes hash table creation. Whether QU[1] is acceptable is highly questionable. Starting the database takes 1.5 hours!! I am seriously considering to switch to persistent ordered indices for the updatable case. For that to happen, the worksing set needs to be extended with persistent delta-bats on those indices, changes to these deltas need to be logged (in order to make the indices recoverable). > -----Original Message----- > From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] > Sent: Wednesday, October 17, 2007 1:54 PM > To: monetdb-developers(a)lists.sourceforge.net > Cc: Peter Boncz; Peter Boncz > Subject: Re: [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, > MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, > 1.143, 1.143.2.1 > > > Just for the records: > > I finally managed to finsh my experiments regarding > [ 1811229 ] [ADT] Adding large document, with update support > http://sourceforge.net/tracker/index.php?func=detail&aid=18112 > 29&group_id=56967&atid=482468 > and the related code changes. For those interested, here's > the detailed > story: > > > "S08-64" System (beo-24): > - 2x 64-bit Dual-Core Opteron270 @ 2 Ghz > - 8 GB memory > - MonetDB/XQuery 0.20, 64-bit, 64-bit OIDs, --enable-optimize > (gcc 4.1.2) > > "S16-32" System (core-1): > - 4x 64-bit Dual-Core Opteron870 @ 2 Ghz > - 16 GB memory > - MonetDB/XQuery 0.20, 64-bit, 32-bit OIDs, --enable-optimize > (gcc 4.1.2) > > Document: > http://mirror.openstreetmap.nl/planet/planet-071003.osm.bz2 > (extracted: 19 GB XML file) > > "SR" Shredding read-only: > pf:add-doc(".../planet-071003.osm","planet-071003.osm") > > "SU" Shredding updateable: > pf:add-doc(".../planet-071003.osm","planet-071003.osm","planet > -071003.osm",5) > > "QR"/"QU" Count query: > count(doc("planet-071003.osm")//*) > > Configurations: > m: without Peter's mmap fix in gdk_posix.mx > (i.e., using rev. 1.143 of gdk_posix.mx) > M: with Peter's mmap fix in gdk_posix.mx > (i.e., using rev. 1.143.2.1 of gdk_posix.mx) > h: without Peter's new string hash function in gdk_atoms.mx > (i.e., using rev. 1.134 of gdk_atoms.mx) > H: with Peter's new string hash function in gdk_atoms.mx > (i.e., using rev. 1.134.6.1 of gdk_atoms.mx) > > > Results (wall-clock times): > > S08-64: > SR QR SU QU[1] QU[2] > mh 659m34s 1m09s 81m06s ERROR - > mH - - 383m17s ERROR - > Mh - - 77m03s 344m32s 1m34s > MH 644m01s 0m49s 390m42s 342m47s 1m36s > > S16-32: > SR QR SU QU[1] QU[2] > mh 127m59s 0m17s 43m14s ERROR - > mH 110m33s 0m16s 26m26s ERROR - > Mh 128m11s 0m18s 44m00s 100m50s 1m15s > MH 191m42s 0m17s 25m43s 101m37s 1m21s > > (NB: "SR" includes building of indices, while "SU" does not; > consequently, "QR" can exploit the indices built during > "SR", while "QU[1]" > has to build the indices first, and only "QU[2]" can exploit them.) > > > Apparently, the mmap fix in gdk_posix.mx seems to be > sufficient to prevent > the remap-ERROR reported (for a system & configuration > similar to "S08-64") > in > [ 1811229 ] [ADT] Adding large document, with update support > http://sourceforge.net/tracker/index.php?func=detail&aid=18112 > 29&group_id=56967&atid=482468 > > I'll leave the further interpretation of the above results to > the interested > recipient / reader. > > > Stefan > >

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, 1.143, 1.143.2.1
by Stefan Manegold 17 Oct '07

17 Oct '07

Just for the records: I finally managed to finsh my experiments regarding [ 1811229 ] [ADT] Adding large document, with update support http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=5… and the related code changes. For those interested, here's the detailed story: "S08-64" System (beo-24): - 2x 64-bit Dual-Core Opteron270 @ 2 Ghz - 8 GB memory - MonetDB/XQuery 0.20, 64-bit, 64-bit OIDs, --enable-optimize (gcc 4.1.2) "S16-32" System (core-1): - 4x 64-bit Dual-Core Opteron870 @ 2 Ghz - 16 GB memory - MonetDB/XQuery 0.20, 64-bit, 32-bit OIDs, --enable-optimize (gcc 4.1.2) Document: http://mirror.openstreetmap.nl/planet/planet-071003.osm.bz2 (extracted: 19 GB XML file) "SR" Shredding read-only: pf:add-doc(".../planet-071003.osm","planet-071003.osm") "SU" Shredding updateable: pf:add-doc(".../planet-071003.osm","planet-071003.osm","planet-071003.osm",5) "QR"/"QU" Count query: count(doc("planet-071003.osm")//*) Configurations: m: without Peter's mmap fix in gdk_posix.mx (i.e., using rev. 1.143 of gdk_posix.mx) M: with Peter's mmap fix in gdk_posix.mx (i.e., using rev. 1.143.2.1 of gdk_posix.mx) h: without Peter's new string hash function in gdk_atoms.mx (i.e., using rev. 1.134 of gdk_atoms.mx) H: with Peter's new string hash function in gdk_atoms.mx (i.e., using rev. 1.134.6.1 of gdk_atoms.mx) Results (wall-clock times): S08-64: SR QR SU QU[1] QU[2] mh 659m34s 1m09s 81m06s ERROR - mH - - 383m17s ERROR - Mh - - 77m03s 344m32s 1m34s MH 644m01s 0m49s 390m42s 342m47s 1m36s S16-32: SR QR SU QU[1] QU[2] mh 127m59s 0m17s 43m14s ERROR - mH 110m33s 0m16s 26m26s ERROR - Mh 128m11s 0m18s 44m00s 100m50s 1m15s MH 191m42s 0m17s 25m43s 101m37s 1m21s (NB: "SR" includes building of indices, while "SU" does not; consequently, "QR" can exploit the indices built during "SR", while "QU[1]" has to build the indices first, and only "QU[2]" can exploit them.) Apparently, the mmap fix in gdk_posix.mx seems to be sufficient to prevent the remap-ERROR reported (for a system & configuration similar to "S08-64") in [ 1811229 ] [ADT] Adding large document, with update support http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=5… I'll leave the further interpretation of the above results to the interested recipient / reader. Stefan On Tue, Oct 16, 2007 at 01:46:47PM +0200, Peter Boncz wrote: > Hi, > > Hm, I cannot really understand the purpose of the question. And what is > wrong with performance fixes? > > both fixes are related to the same bug: > - the remap failing is addressed by the gdk_posix fix > - the shredding in the bug report taking excessively long is addressed by > the gdk_atoms fix > > indeed, any hash function can have collisions.. it all depends on the > distribution. > > Peter > > PS most probably, this mail (sent from my home account) will be rejected by > the sourceforge mailing list -- and I cannot sent through CWI from home as > the secure mail sending is not supported by CWI staff for microsoft > emailers. > > > -----Original Message----- > From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] > Sent: dinsdag 16 oktober 2007 12:01 > To: Peter Boncz > Cc: monetdb-developers(a)lists.sourceforge.net > Subject: Re: [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, > MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, 1.143, > 1.143.2.1 > > > Peter, > > which part of your changes do fix the problem with updatedable shredding of > large XML documents as reporten in > [ 1811229 ] [ADT] Adding large document, with update support > http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=56 > 967&atid=482468 > ? > > The new has string function in gdk_atoms.mx or the file descriptor fixes in > gdk_posix.mx? > > The former looks for like a performance fix to me --- too many collisions > should only slows the system down, but not copromize its > fucntionallity/correctness, right? > Also with the new string has functions ("too") many collisions can still > occur with certain datasets ... > > Stefan > > > On Sun, Oct 14, 2007 at 08:31:36PM +0000, Stefan Manegold wrote: > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15103 > > > > Modified Files: > > Tag: MonetDB_1-20 > > gdk_atoms.mx gdk_posix.mx > > Log Message: > > > > [checkin on behalf of Peter] > > > > fixing XQuery bug > > [ 1811229 ] [ADT] Adding large document, with update support > > > http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=56 > 967&atid=482468 > > > > gdk_atoms.mx: > > - hash collisions in strings that consists of digits only (a common case!) > > we now use a fast derivative of the Bob Jenkins function from now on > > > > Really bad collisions, in case of the 20GB document of the bug report, > > shredding took 8 hours before, 1 hour after this change. > > > > NOTE: this change affects the binary format (string heaps) and all > product > > families, as the hash function is a compiled-in macro! > > In particular, lookup operations and joins on SQL (Monet4/5) > columns > > consisting of digits only, but stored in a VARCHAR, should be > faster > > after this check-in. > > > > gdk_posix.mx > > - we lost track of the file descriptor for large heaps (the file desc is > given > > to the mmap-monitoring-thread to close later), such that the remap > function > > could fail (when it was given the illegal file descriptor 0) > > > > NOTE: this change only affects xquery it only uses remap() -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] Compiling MonetDB5 (Current) failed on GNU.32.32.d-Fedora6
by Fabian Groffen 17 Oct '07

17 Oct '07

Maybe it is better to use fork/exec here instead of system as well when fixing this? You could "hardcode" the mcc path in your call, using what you got from autoconf so you more probably get the tool you expect. On 17-10-2007 08:53:11 +0200, Testweb Compilation wrote: > 852 char buf[1024]; > 853 snprintf(buf,1024,"mcc %s",*fname); > 854 system(buf); > 855 (void) ret; > 856 return MAL_SUCCEED; > > /export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/src/compiler/mal_compiler.mx: In function 'MCmcc': > /export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/src/compiler/mal_compiler.mx:854: warning: ignoring return value of 'system', declared with attribute warn_unused_result > make[5]: *** [lib_mal_compiler_la-mal_compiler.lo] Error 1 > make[5]: Leaving directory `/export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/Fedora6/src/compiler' > make[4]: *** [all] Error 2 > make[4]: Leaving directory `/export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/Fedora6/src/compiler' > make[3]: *** [all-recursive] Error 1 > make[3]: Leaving directory `/export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/Fedora6/src' > make[2]: *** [all] Error 2 > make[2]: Leaving directory `/export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/Fedora6/src' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/export/scratch1/monet/monet.GNU.32.32.d.8963/MonetDB5/Fedora6' > make: *** [all] Error 2 > grep: /export/scratch1/monet/monet.GNU.32.32.d.8963/TestTools/ALL.MonetDB5: No such file or directory

2 1

[Monetdb-developers] PhD Thesis Steenhagen Optimization of object query languages
by darabi＠web.de 16 Oct '07

16 Oct '07

Hello, sorry for this OT question. I was reading the literature relevant to what I would like to do and found the papers of H. Steenhagen. A thesis is referenced very often which I could order a dead-tree copy of. But that would probably take very long: Title: Optimization of object query languages Author: Hendrika Janna Steenhagen (1954-) Year: 1995 Note: Proefschrift Universiteit Twente Met lit. opg., index. - Met een samenvatting in het Nederlands ISBN: 90-9008745-1 Does anybody on the list have a digital copy of it or knows where one is available? Many thanks Kambiz

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/kernel algebra.mx, , 1.184, 1.185 bat5.mx, , 1.26, 1.27 mmath.mx, , 1.35, 1.36
by Romulo Goncalves 16 Oct '07

16 Oct '07

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/kernel > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28642 > > Modified Files: > algebra.mx bat5.mx mmath.mx > Log Message: > Move all signatures to the .h section to ease compilation of > programs using the interface directly. More files to do. > > > Index: mmath.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/mmath.mx,v > retrieving revision 1.35 > retrieving revision 1.36 > diff -u -d -r1.35 -r1.36 > --- mmath.mx 13 Sep 2007 13:45:13 -0000 1.35 > +++ mmath.mx 14 Oct 2007 08:41:47 -0000 1.36 > @@ -206,16 +206,9 @@ > @h > #ifndef __MMATH_H__ > #define __MMATH_H__ > - > -#include <gdk.h> > - > -#endif /* __MMATH_H__ */ > - > -@+ Implementation Code > -@c > #include "mal_config.h" > - > -#include "mmath.h" > +#include "mal.h" > +#include "mal_exception.h" > #include <math.h> > > extern double sqrt(double x); > @@ -234,29 +227,6 @@ > #define mmath_export extern > #endif > > -#define acos_unary(x, z) *z = acos(*x) > -#define asin_unary(x, z) *z = asin(*x) > -#define atan_unary(x, z) *z = atan(*x) > -#define atan2_binary(x, y, z) *z = atan2(*x,*y) > -#define cos_unary(x, z) *z = cos(*x) > -#define sin_unary(x, z) *z = sin(*x) > -#define tan_unary(x, z) *z = tan(*x) > - > -#define cosh_unary(x, z) *z = cosh(*x) > -#define sinh_unary(x, z) *z = sinh(*x) > -#define tanh_unary(x, z) *z = tanh(*x) > - > -#define exp_unary(x, z) *z = exp(*x) > -#define log_unary(x, z) *z = log(*x) > -#define log10_unary(x, z) *z = log10(*x) > - > -#define pow_binary(x, y, z) *z = pow(*x,*y) > -#define sqrt_unary(x, z) *z = sqrt(*x) > - > -#define ceil_unary(x, z) *z = ((-1.0<*x)&&(*x<0.0))?0.0:ceil(*x) > -#define fabs_unary(x, z) *z = fabs(*x) > -#define floor_unary(x, z) *z = floor(*x) > -#define fmod_binary(x, y, z) *z = fmod(*x,*y) > > #ifdef _MSC_VER > #include <float.h> > @@ -306,11 +276,78 @@ > #endif /* HAVE_ISINF */ > #endif /* HAVE_FPCLASSIFY */ > > -#include "mal.h" > -#include "mal_exception.h" > -mmath_export str math_unary_FINITE(bit *res, dbl *a); > -mmath_export str math_unary_ISNAN(bit *res, dbl *a); > -mmath_export str math_unary_ISINF(int *res, dbl *a); > +#define acos_unary(x, z) *z = acos(*x) > +#define asin_unary(x, z) *z = asin(*x) > +#define atan_unary(x, z) *z = atan(*x) > +#define atan2_binary(x, y, z) *z = atan2(*x,*y) > +#define cos_unary(x, z) *z = cos(*x) > +#define sin_unary(x, z) *z = sin(*x) > +#define tan_unary(x, z) *z = tan(*x) > + > +#define cosh_unary(x, z) *z = cosh(*x) > +#define sinh_unary(x, z) *z = sinh(*x) > +#define tanh_unary(x, z) *z = tanh(*x) > + > +#define exp_unary(x, z) *z = exp(*x) > +#define log_unary(x, z) *z = log(*x) > +#define log10_unary(x, z) *z = log10(*x) > + > +#define pow_binary(x, y, z) *z = pow(*x,*y) > +#define sqrt_unary(x, z) *z = sqrt(*x) > + > +#define ceil_unary(x, z) *z = ((-1.0<*x)&&(*x<0.0))?0.0:ceil(*x) > +#define fabs_unary(x, z) *z = fabs(*x) > +#define floor_unary(x, z) *z = floor(*x) > +#define fmod_binary(x, y, z) *z = fmod(*x,*y) > + > +@= unopbaseM5_export > +mmath_export str MATHunary@1@3(@3 *res , @3 *a ); > +@= unopM5_export > + @:unopbaseM5_export(@1,@2,dbl)@ > + @:unopbaseM5_export(@1,@2,flt)@ > +@= binopbaseM5_export > +mmath_export str MATHbinary@1@3(@3 *res, @3 *a, @3 *b ); > +@= binopM5_export > + @:binopbaseM5_export(@1,@2,dbl)@ > + @:binopbaseM5_export(@1,@2,flt)@ > + > +@h > +@:unopM5_export(_ACOS,acos)@ > +@:unopM5_export(_ASIN,asin)@ > +@:unopM5_export(_ATAN,atan)@ > +@:binopM5_export(_ATAN2,atan2)@ > +@:unopM5_export(_COS,cos)@ > +@:unopM5_export(_SIN,sin)@ > +@:unopM5_export(_TAN,tan)@ > + > +@:unopM5_export(_COSH,cosh)@ > +@:unopM5_export(_SINH,sinh)@ > +@:unopM5_export(_TANH,tanh)@ > + > +@:unopM5_export(_EXP,exp)@ > +@:unopM5_export(_LOG,log)@ > +@:unopM5_export(_LOG10,log10)@ > + > +@:binopM5_export(_POW,pow)@ > +@:unopM5_export(_SQRT,sqrt)@ > + > +@:unopM5_export(_CEIL,ceil)@ > +@:unopbaseM5_export(_FABS,fabs,dbl)@ > +@:unopM5_export(_FLOOR,floor)@ > +@:binopM5_export(_FMOD,fmod)@ > + > + > +mmath_export str MATHunary_ISNAN(bit *res, dbl *a); > +mmath_export str MATHunary_ISINF(int *res, dbl *a); > +mmath_export str MATHunary_FINITE(bit *res, dbl *a); > +mmath_export str MATHrandint(int *res); > +mmath_export str MATHsrandint(int *seed); > +mmath_export str MATHpi(dbl *pi); > +#endif /* __MMATH_H__ */ > + > +@+ Implementation Code > +@c > +#include "mmath.h" > > @:unop(_ACOS,acos)@ > @:unop(_ASIN,asin)@ > @@ -375,7 +412,6 @@ > } > return MAL_SUCCEED; > } > -@ > > @c > str > @@ -453,7 +489,6 @@ > Wrapping around the M4 code base > @- > @= unopbaseM5 > -mmath_export str MATHunary@1@3(@3 *res , @3 *a ); > str MATHunary@1@3(@3 *res , @3 *a ) { > #ifdef DEBUG > printf( "MATHunary@1@3\n"); > @@ -476,7 +511,6 @@ > @:unopbaseM5(@1,@2,flt)@ > > @= binopbaseM5 > -mmath_export str MATHbinary@1@3(@3 *res, @3 *a, @3 *b ); > str MATHbinary@1@3(@3 *res, @3 *a, @3 *b ) { > #ifdef DEBUG > printf( "MATHbinary@1\n"); > @@ -495,7 +529,6 @@ > @:binopbaseM5(@1,@2,flt)@ > @ > @= roundM5 > -mmath_export str MATHbinary_ROUND@1(@1 *res, @1 *x, int *y); > str MATHbinary_ROUND@1(@1 *res, @1 *x, int *y) { If you remove the export, you will break the compilation on windows. Maybe the following Mserver5 start can tell you which functions should be exported, but unfortunately they aren't. ;) E:\MonetDB\Current\sql\NT>RunMserver5.bat > # MonetDB server v5.3.0, based on kernel v1.21.0 > # Serving database 'demo' > # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked > # Copyright (c) 1993-2007 CWI, all rights reserved > # Visit http://monetdb.cwi.nl/ for further information > #warning: please don't forget to set your vault key! > #(see E:\MonetDB\Current\MonetDB5\NT\etc\monetdb5.conf) > !MALException:MAL.getAddress:address of 'mmath.MATHbinary_ROUNDflt' not > found > !MALException:MAL.getAddress:address of 'mmath.MATHbinary_ROUNDdbl' not > found > !MALException:MAL.getAddress:address of 'mmath.math_unary_ISNAN' not found > !MALException:MAL.getAddress:address of 'mmath.math_unary_ISINF' not found > !MALException:MAL.getAddress:address of 'mmath.math_unary_FINITE' not found > !MALException:MAL.getAddress:address of 'algebra.ALGselectNotNil' not found > !MALException:MAL.getAddress:address of 'algebra.ALGuhashsplit' not found > !MALException:MAL.getAddress:address of 'group.GRPmin' not found > !MALException:MAL.getAddress:address of 'group.GRPmax' not found > # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ > > >\q Regards, Romulo

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, 1.143, 1.143.2.1
by Stefan Manegold 16 Oct '07

16 Oct '07

Peter, the purpose and/or reason of/for my question is the fact that we cannot release the binary packages that Sjoerd built yesterday, because the binary database format has been modified without increasing the GDK version number. We simply forgot the latter. Hence, a new Mserver cannot notice in case it is started on an pld database, and instead of complaining properly, it will just behave strangely, crash and/or corrup the database. This is not good. Consequently, Sjoerd needs to re-do the building once again, and we have to think of how to solve the above problem. Obviously, one solution is th "simply" increase the GDK version number. But then, users have to abandon or dump/retore their databases now and with the next release (GDK-2). An other alternative would be to undo the string hash function changes in the release branch, keeping the same binary format for now, and include both binary format changes in one release. In particular if the string hash function changes are "only" for performance reasons, both Sjoerd, Niels, and I would favor the second solution. After all, we and our users have been able to live with the "old" string hash function for years. Those that really require the new one can switch to the development version. I'm busy with testing --- on a 8GB dual-dual-core 2GHz Opteron running 64-bit FedoraCore 6 and and optimized 64-bit Mserver with 64-bit OIDs, shredding the 19 GB file read-only took 11 hours, both with the old and the new hash functions (Mserver grew to 10 GB res, >50 GB virt). Shredding updateable without your two fixes took only 2 hours, but the resulting DB is currupt as reported in [ 1811229 ] [ADT] Adding large document, with update support http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=56 . Shredding updateable with your two fixes is still busy, running for more than 6 hours, now... Stefan On Tue, Oct 16, 2007 at 01:46:47PM +0200, Peter Boncz wrote: > Hi, > > Hm, I cannot really understand the purpose of the question. And what is > wrong with performance fixes? > > both fixes are related to the same bug: > - the remap failing is addressed by the gdk_posix fix > - the shredding in the bug report taking excessively long is addressed by > the gdk_atoms fix > > indeed, any hash function can have collisions.. it all depends on the > distribution. > > Peter > > PS most probably, this mail (sent from my home account) will be rejected by > the sourceforge mailing list -- and I cannot sent through CWI from home as > the secure mail sending is not supported by CWI staff for microsoft > emailers. > > > -----Original Message----- > From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] > Sent: dinsdag 16 oktober 2007 12:01 > To: Peter Boncz > Cc: monetdb-developers(a)lists.sourceforge.net > Subject: Re: [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, > MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, 1.143, > 1.143.2.1 > > > Peter, > > which part of your changes do fix the problem with updatedable shredding of > large XML documents as reporten in > [ 1811229 ] [ADT] Adding large document, with update support > http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=56 > 967&atid=482468 > ? > > The new has string function in gdk_atoms.mx or the file descriptor fixes in > gdk_posix.mx? > > The former looks for like a performance fix to me --- too many collisions > should only slows the system down, but not copromize its > fucntionallity/correctness, right? > Also with the new string has functions ("too") many collisions can still > occur with certain datasets ... > > Stefan > > > On Sun, Oct 14, 2007 at 08:31:36PM +0000, Stefan Manegold wrote: > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15103 > > > > Modified Files: > > Tag: MonetDB_1-20 > > gdk_atoms.mx gdk_posix.mx > > Log Message: > > > > [checkin on behalf of Peter] > > > > fixing XQuery bug > > [ 1811229 ] [ADT] Adding large document, with update support > > > http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=56 > 967&atid=482468 > > > > gdk_atoms.mx: > > - hash collisions in strings that consists of digits only (a common case!) > > we now use a fast derivative of the Bob Jenkins function from now on > > > > Really bad collisions, in case of the 20GB document of the bug report, > > shredding took 8 hours before, 1 hour after this change. > > > > NOTE: this change affects the binary format (string heaps) and all > product > > families, as the hash function is a compiled-in macro! > > In particular, lookup operations and joins on SQL (Monet4/5) > columns > > consisting of digits only, but stored in a VARCHAR, should be > faster > > after this check-in. > > > > gdk_posix.mx > > - we lost track of the file descriptor for large heaps (the file desc is > given > > to the mmap-monitoring-thread to close later), such that the remap > function > > could fail (when it was given the illegal file descriptor 0) > > > > NOTE: this change only affects xquery it only uses remap() > > > > > > Index: gdk_posix.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_posix.mx,v > > retrieving revision 1.143 > > retrieving revision 1.143.2.1 > > diff -u -d -r1.143 -r1.143.2.1 > > --- gdk_posix.mx 4 Sep 2007 17:55:20 -0000 1.143 > > +++ gdk_posix.mx 14 Oct 2007 20:31:33 -0000 1.143.2.1 > > @@ -615,7 +615,7 @@ > > MT_mmap_tab[i].writable = writable; > > MT_mmap_tab[i].fd = fd; > > MT_mmap_tab[i].pincnt = 0; > > - fd = -1; > > + fd = -fd; > > } > > (void) pthread_mutex_unlock(&MT_mmap_lock); > > return fd; > > @@ -1051,9 +1051,7 @@ > > } > > if (ret != (void *) -1L) { > > hdl->fixed = ret; > > - fd = MT_mmap_new(path, ret, len, fd, (mode & MMAP_WRITABLE)); > > - if (fd <= 0) > > - hdl->hdl = (void *) 0; /* MT_mmap_new keeps the fd */ > > + hdl->hdl = (void*) (ssize_t) MT_mmap_new(path, ret, len, fd, (mode & > MMAP_WRITABLE)); > > } > > return ret; > > } > > @@ -1061,13 +1059,12 @@ > > void * > > MT_mmap_remap(MT_mmap_hdl *hdl, off_t off, size_t len) > > { > > - void *ret; > > - > > - ret = mmap(hdl->fixed, > > + int fd = (int) (ssize_t) hdl->hdl; > > + void *ret = mmap(hdl->fixed, > > len, > > ((hdl->mode & MMAP_WRITABLE) ? PROT_WRITE : 0) | PROT_READ, > > ((hdl->mode & MMAP_COPY) ? (MAP_PRIVATE | MAP_NORESERVE) : > MAP_SHARED) | (hdl->fixed ? MAP_FIXED : 0), > > - (int) (ssize_t) hdl->hdl, > > + (fd < 0)?-fd:fd, > > off); > > > > if (ret != (void *) -1L) { > > @@ -1083,9 +1080,7 @@ > > MT_mmap_close(MT_mmap_hdl *hdl) > > { > > int fd = (int) (ssize_t) hdl->hdl; > > - > > - if (fd) > > - close(fd); > > + if (fd > 0) close(fd); > > hdl->hdl = NULL; > > } > > > > > > Index: gdk_atoms.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_atoms.mx,v > > retrieving revision 1.134 > > retrieving revision 1.134.6.1 > > diff -u -d -r1.134 -r1.134.6.1 > > --- gdk_atoms.mx 2 May 2007 16:16:58 -0000 1.134 > > +++ gdk_atoms.mx 14 Oct 2007 20:31:32 -0000 1.134.6.1 > > @@ -1878,13 +1878,19 @@ > > rotates all characters together. It is optimized to process 2 characters > > at a time (adding 16-bits to the hash value each iteration). > > @h > > -#define GDK_STRHASH(x,y) { \ > > - str _c = (str) (x); \ > > - for((y)=0; _c[0] && _c[1]; _c+=2) { \ > > - (y) = ((y) << 3) ^ ((y) >> 11) ^ ((y) >> 17) ^ (_c[1] << > 8) ^ _c[0];\ > > - } \ > > - (y) ^= _c[0]; \ > > +#define GDK_STRHASH(x,y) {\ > > + str _key = (str) (x);\ > > + int _i;\ > > + for (_i = y = 0; _key[_i]; _i++) {\ > > + y += _key[_i];\ > > + y += (y << 10);\ > > + y ^= (y >> 6);\ > > + }\ > > + y += (y << 3);\ > > + y ^= (y >> 11);\ > > + y += (y << 15);\ > > } > > + > > @c > > hash_t > > strHash(str s) > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Monetdb-checkins mailing list > > Monetdb-checkins(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > | The Netherlands | Fax : +31 (20) 592-4312 | -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_atoms.mx, MonetDB_1-20, 1.134, 1.134.6.1 gdk_posix.mx, MonetDB_1-20, 1.143, 1.143.2.1
by Stefan Manegold 16 Oct '07

16 Oct '07

Peter, which part of your changes do fix the problem with updatedable shredding of large XML documents as reporten in [ 1811229 ] [ADT] Adding large document, with update support http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=5… ? The new has string function in gdk_atoms.mx or the file descriptor fixes in gdk_posix.mx? The former looks for like a performance fix to me --- too many collisions should only slows the system down, but not copromize its fucntionallity/correctness, right? Also with the new string has functions ("too") many collisions can still occur with certain datasets ... Stefan On Sun, Oct 14, 2007 at 08:31:36PM +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15103 > > Modified Files: > Tag: MonetDB_1-20 > gdk_atoms.mx gdk_posix.mx > Log Message: > > [checkin on behalf of Peter] > > fixing XQuery bug > [ 1811229 ] [ADT] Adding large document, with update support > http://sourceforge.net/tracker/index.php?func=detail&aid=1811229&group_id=5… > > gdk_atoms.mx: > - hash collisions in strings that consists of digits only (a common case!) > we now use a fast derivative of the Bob Jenkins function from now on > > Really bad collisions, in case of the 20GB document of the bug report, > shredding took 8 hours before, 1 hour after this change. > > NOTE: this change affects the binary format (string heaps) and all product > families, as the hash function is a compiled-in macro! > In particular, lookup operations and joins on SQL (Monet4/5) columns > consisting of digits only, but stored in a VARCHAR, should be faster > after this check-in. > > gdk_posix.mx > - we lost track of the file descriptor for large heaps (the file desc is given > to the mmap-monitoring-thread to close later), such that the remap function > could fail (when it was given the illegal file descriptor 0) > > NOTE: this change only affects xquery it only uses remap() > > > Index: gdk_posix.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_posix.mx,v > retrieving revision 1.143 > retrieving revision 1.143.2.1 > diff -u -d -r1.143 -r1.143.2.1 > --- gdk_posix.mx 4 Sep 2007 17:55:20 -0000 1.143 > +++ gdk_posix.mx 14 Oct 2007 20:31:33 -0000 1.143.2.1 > @@ -615,7 +615,7 @@ > MT_mmap_tab[i].writable = writable; > MT_mmap_tab[i].fd = fd; > MT_mmap_tab[i].pincnt = 0; > - fd = -1; > + fd = -fd; > } > (void) pthread_mutex_unlock(&MT_mmap_lock); > return fd; > @@ -1051,9 +1051,7 @@ > } > if (ret != (void *) -1L) { > hdl->fixed = ret; > - fd = MT_mmap_new(path, ret, len, fd, (mode & MMAP_WRITABLE)); > - if (fd <= 0) > - hdl->hdl = (void *) 0; /* MT_mmap_new keeps the fd */ > + hdl->hdl = (void*) (ssize_t) MT_mmap_new(path, ret, len, fd, (mode & MMAP_WRITABLE)); > } > return ret; > } > @@ -1061,13 +1059,12 @@ > void * > MT_mmap_remap(MT_mmap_hdl *hdl, off_t off, size_t len) > { > - void *ret; > - > - ret = mmap(hdl->fixed, > + int fd = (int) (ssize_t) hdl->hdl; > + void *ret = mmap(hdl->fixed, > len, > ((hdl->mode & MMAP_WRITABLE) ? PROT_WRITE : 0) | PROT_READ, > ((hdl->mode & MMAP_COPY) ? (MAP_PRIVATE | MAP_NORESERVE) : MAP_SHARED) | (hdl->fixed ? MAP_FIXED : 0), > - (int) (ssize_t) hdl->hdl, > + (fd < 0)?-fd:fd, > off); > > if (ret != (void *) -1L) { > @@ -1083,9 +1080,7 @@ > MT_mmap_close(MT_mmap_hdl *hdl) > { > int fd = (int) (ssize_t) hdl->hdl; > - > - if (fd) > - close(fd); > + if (fd > 0) close(fd); > hdl->hdl = NULL; > } > > > Index: gdk_atoms.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_atoms.mx,v > retrieving revision 1.134 > retrieving revision 1.134.6.1 > diff -u -d -r1.134 -r1.134.6.1 > --- gdk_atoms.mx 2 May 2007 16:16:58 -0000 1.134 > +++ gdk_atoms.mx 14 Oct 2007 20:31:32 -0000 1.134.6.1 > @@ -1878,13 +1878,19 @@ > rotates all characters together. It is optimized to process 2 characters > at a time (adding 16-bits to the hash value each iteration). > @h > -#define GDK_STRHASH(x,y) { \ > - str _c = (str) (x); \ > - for((y)=0; _c[0] && _c[1]; _c+=2) { \ > - (y) = ((y) << 3) ^ ((y) >> 11) ^ ((y) >> 17) ^ (_c[1] << 8) ^ _c[0];\ > - } \ > - (y) ^= _c[0]; \ > +#define GDK_STRHASH(x,y) {\ > + str _key = (str) (x);\ > + int _i;\ > + for (_i = y = 0; _key[_i]; _i++) {\ > + y += _key[_i];\ > + y += (y << 10);\ > + y ^= (y >> 6);\ > + }\ > + y += (y << 3);\ > + y ^= (y >> 11);\ > + y += (y << 15);\ > } > + > @c > hash_t > strHash(str s) > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] XRPCwrapper fails to compile with java 1.4
by Stefan Manegold 16 Oct '07

16 Oct '07

Hoi, apparently, the new XPRCwrapper code fails to compile with java 1.4 (our RedHat 4 WS machine, titan, is the only testing machine that still has only java 1.4, all others have java 1.5); for details see below and/or http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.ntv.64.64… Question: Assuming that there is no easy way to make the new XRPC warpper code java 1.4 compliant: should we modify (tighten) our requirements accordingly, i.e., requiring (only) java 1.5 instead of (at least) java 1.4 to build MonetDB* (java 1.6 is not (yet?) supported)? Stefan ======== "/usr/local/bin/ant" -f "`readlink -f /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/build.xml" -Dbuilddir="`readlink -f /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/RedHat4WS/src/tools/java" -Djardir="`readlink -f /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/RedHat4WS/src/tools/java" -Dbasedir="`readlink -f /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java" distall Buildfile: /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/build.xml distall: prepare: [echo] Debug is true, optimise is true compile_util: [echo] Compiling Utilities [javac] Compiling 9 source files to /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/RedHat4WS/src/tools/java [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/NamespaceContextImpl.java:22: package javax.xml.namespace does not exist [javac] import javax.xml.namespace.*; [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/NamespaceContextImpl.java:29: cannot resolve symbol [javac] symbol : class NamespaceContext [javac] location: class nl.cwi.monetdb.xquery.util.NamespaceContextImpl [javac] public class NamespaceContextImpl implements NamespaceContext{ [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:23: package javax.xml.namespace does not exist [javac] import javax.xml.namespace.*; [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:24: package javax.xml.xpath does not exist [javac] import javax.xml.xpath.*; [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:28: package com.sun.org.apache.xml.internal.dtm.ref does not exist [javac] import com.sun.org.apache.xml.internal.dtm.ref.DTMNodeList; [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:372: cannot resolve symbol [javac] symbol : class XPath [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] XPath xPath, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:409: cannot resolve symbol [javac] symbol : class XPath [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] public static NamedNodeMap getNodeAttributes(XPath xPath, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:432: cannot resolve symbol [javac] symbol : class XPath [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] public static String[] getNodeListAttribute(XPath xPath, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:415: cannot resolve symbol [javac] symbol : class DTMNodeList [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] DTMNodeList nodeList = (DTMNodeList) xPath.evaluate(xPathExpr, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:415: cannot resolve symbol [javac] symbol : class DTMNodeList [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] DTMNodeList nodeList = (DTMNodeList) xPath.evaluate(xPathExpr, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:416: cannot resolve symbol [javac] symbol : variable XPathConstants [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] inputSource, XPathConstants.NODESET); [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:439: cannot resolve symbol [javac] symbol : class DTMNodeList [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] DTMNodeList nodeList = (DTMNodeList) xPath.evaluate(xPathExpr, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:439: cannot resolve symbol [javac] symbol : class DTMNodeList [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] DTMNodeList nodeList = (DTMNodeList) xPath.evaluate(xPathExpr, [javac] ^ [javac] /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util/XRPCMessage.java:440: cannot resolve symbol [javac] symbol : variable XPathConstants [javac] location: class nl.cwi.monetdb.xquery.util.XRPCMessage [javac] inputSource, XPathConstants.NODESET); [javac] ^ [javac] 14 errors BUILD FAILED /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/build.xml:55: The following error occurred while executing this line: /export/scratch1/monet/monet.ntv.64.64.d.8915/pathfinder/src/tools/java/build.xml:107: Compile failed; see the compiler error output for details. Total time: 6 seconds make[7]: *** [distall_ant_target] Error 1 ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

5 12

Re: [Monetdb-developers] BAT sizes
by Stefan Manegold 15 Oct '07

15 Oct '07

[I once again felt free to share this with the community ...] Henning, in case BAT capacities are significantly larger than their actual content (count), this might indeed have a negative influence on performance. (1) in case the BAT is memory-mapped when loaded, it "only" blocks some more address space than strictly necessary (no problem on 64-bit systems, potentially a problem on 32-bit systems); (2) in case the BAT is malloced when loaded, it also occupied some more memory than strictly necessary (potential problem on but 64- & 32-bit systems). However, unless there is some accurate estimation, it is often hard (or virually impossible) to "guess" a BATs size before filling it; hence, a "generous" initial size allocation is good to avoid expensive BAT extents. In your case, I'm lost concerning which BATs your taling about. The shredder-generated pre_* (actually rid_*) BATs need to be allocated before reading the document; hence, there is know knowledge about the number of nodes in the document, and as far as I can tell no trivial way to estimate this accurately. Hence, the shredder needs to guess something --- JanF can tell more, I guess... In case of the TIJAH indices, I have no clue at all, how/where/when they are built and whether there might be better information available to not overallocate but allocate only just enough space. You or some of your colleagues in Twente should know all the details. Finally, is there any concrete case where you actually experiences any problems due to "over-allocation", or are you just wondering? Stefan ps: in the case given below, the batsize just fits the BAT's capacity; only the count is smaller than the capacity (obviously, it cannot be larger) --- if you want/need to know why, you better ask him/her who allocated/created/filled the "tj_DFLT_FT_INDEX_size1" BAT ... On Mon, Oct 15, 2007 at 02:36:46PM +0200, Henning Rode wrote: > hej stefan, > > sorry, that i did not answer earlier. i justed wanted to report the > actual sizes of pf/tijah indices in a paper. so that is done now. > > still, i was asking myself, whether it might have any kind of > performance influences, that BAT capacities are so much higher than the > actual BAT counts. This is of course handy, when we still want to add > new entries, but once we indexed a collection, we usually only query it. > > in case of our "pre_size" BAT this difference between BATsize and > BATdsksize can easily be 250MB or more. > > best -henning > > mil>var t := bat("tj_DFLT_FT_INDEX_size1"); > mil>t.count().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |203091470 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>t.capacity().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |260898816 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>t.batsize().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |1043599360 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>t.batdsksize().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |812366848 > > > > mil>var x := t.copy(); > mil>x.count().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |203091470 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>x.capacity().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |260898816 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>x.batsize().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |1043599360 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>x.batdsksize().print(); > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > |812366848 > > | > +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > mil>t.access(BAT_READ); > > > Stefan Manegold wrote: > > Henning, > > > > you should also check & report b.capacity(), i.e., > > > > b.count(); > > b.capacity(); > > b.info().reverse().like("batBuns").like("size").print(); > > b.batsize(); > > b.batdsksize(); > > > > var c := b.copy(); > > > > c.count(); > > c.capacity(); > > c.info().reverse().like("batBuns").like("size").print(); > > c.batsize(); > > c.batdsksize(); > > > > Stefan > > > > > > On Sat, Oct 06, 2007 at 07:43:28PM +0200, Stefan Manegold wrote: > >> [felt free to cc the monetdb-developers list as more people might be > >> interested or want to contribute] > >> > >> Henning, > >> > >> are you just "concerned" or are you having concrete problems with the bat > >> sizes? > >> > >> In cany case, to give any reasonable answer we'd need to know more about the > >> details. In particular how large is the BAT your talking about. > >> > >> I.e., with "b" being your BAT and "c := b.copy()", please check & report > >> > >> b.count(); > >> b.info().reverse().like("batBuns").like("size").print(); > >> b.batsize(); > >> b.batdsksize(); > >> > >> c.count(); > >> c.info().reverse().like("batBuns").like("size").print(); > >> c.batsize(); > >> c.batdsksize(); > >> > >> Stefan > >> > >> > >> On Fri, Oct 05, 2007 at 01:47:01PM +0200, Henning Rode wrote: > >>> hej stefan, > >>> > >>> thanks for the answer. so in conclusion, the over-allocation of memory > >>> is quite normal, and nothing to worry about. > >>> > >>> i was more surprised that the copied BAT still has this considerable > >>> over-allocation of memory, though it exactly knows how many entries it > >>> needs to hold. > >>> > >>> groeten -henning > >> -- > >> | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > >> | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > >> | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > >> | The Netherlands | Fax : +31 (20) 592-4312 | > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/tools monet_version.c.in, MonetDB_5-2, NONE, 1.1.2.1 monet_version.h, MonetDB_5-2, NONE, 1.1.2.1 mserver5.mx, MonetDB_5-2, NONE, 1.29.2.2 Makefile.ag, MonetDB_5-2, 1.54.2.1, 1.54.2.2 mserver5.mx.in, MonetDB_5-2, 1.33, NONE
by Stefan Manegold 14 Oct '07

14 Oct '07

Niels, thanks for the fix --- unfortunately a compile from scratch fails as follows: ======== make[5]: Entering directory /net/corona.ins.cwi.nl/export/scratch0/manegold/Monet/Testing/Stable/build.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/MonetDB5/src/tools' /bin/sh ../../libtool --tag=CC --mode=link gcc -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -D_REENTRANT -o mserver5 -export-dynamic mserver5.o ../mal/libmal.la -L/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64 -lbat -lmutils -lstream -lz -lbz2 -lpthread -ldl -lpcre -lssl -lcrypto -lcrypt gcc -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -D_REENTRANT -o .libs/mserver5 mserver5.o -Wl,--export-dynamic ../mal/.libs/libmal.so -L/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64 -lreadline -ltermcap /ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/libbat.so /ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/libstream.so /ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/libmutils.so -lz -lbz2 -lpthread -ldl -lpcre -lssl -lcrypto -lcrypt -Wl,--rpath -Wl,/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64 mserver5.o: In function `main': /ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/tools/mserver5.mx:452: undefined reference to `monet_version' collect2: ld returned 1 exit status make[5]: *** [mserver5] Error 1 ======== It seems that autogen does not handle .c.in files properly, at least not in "SOURCE =" lists ... Stefan On Sun, Oct 14, 2007 at 08:20:25AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv25912/src/tools > > Modified Files: > Tag: MonetDB_5-2 > Makefile.ag > Added Files: > Tag: MonetDB_5-2 > monet_version.c.in monet_version.h mserver5.mx > Removed Files: > Tag: MonetDB_5-2 > mserver5.mx.in > Log Message: > tablet fixed null pointer dereference > split out the monet_version code as it needs configure to set variables. > (mx.in's are now not allowed anymore) > > > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/tools/Makefile.ag,v > retrieving revision 1.54.2.1 > retrieving revision 1.54.2.2 > diff -u -d -r1.54.2.1 -r1.54.2.2 > --- Makefile.ag 3 Oct 2007 09:45:59 -0000 1.54.2.1 > +++ Makefile.ag 14 Oct 2007 08:20:22 -0000 1.54.2.2 > @@ -19,7 +19,7 @@ > INCLUDES = -I../mal $(CLIENTS_INCS) $(MONETDB_INCS) $(OPENSSL_INCS) $(PCRE_CFLAGS) > > bin_mserver5 = { > - SOURCES = mserver5.mx.in > + SOURCES = mserver5.mx monet_version.h monet_version.c.in > LIBS = ../mal/libmal @SHARED_LIBS@ $(MONETDB_LIBS) \ > -lbat -lmutils -lstream \ > $(SOCKET_LIBS) $(Z_LIBS) $(BZ_LIBS) \ > > --- NEW FILE: monet_version.h --- > #ifndef MONETDB_VERSION_H > #define MONETDB_VERSION_H > > extern void monet_version(void); > > #endif /* MONETDB_VERSION_H */ > > --- NEW FILE: mserver5.mx --- > @' The contents of this file are subject to the MonetDB Public License > @' Version 1.1 (the "License"); you may not use this file except in > @' compliance with the License. You may obtain a copy of the License at > @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html > @' > @' Software distributed under the License is distributed on an "AS IS" > @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > @' License for the specific language governing rights and limitations > @' under the License. > @' > @' The Original Code is the MonetDB Database System. > @' > @' The Initial Developer of the Original Code is CWI. > @' Portions created by CWI are Copyright (C) 1997-2007 CWI. > @' All Rights Reserved. > > @f mserver5 > @a M.L. Kersten, P. Boncz, Niels Nes, Stefan Manegold, Sjoerd Mullender > @v 5.0 > @* The Monet Server > The program @code{ mserver5} is the MonetDB server. It is started by the database > administrator. The console is primarilly meant to further initialize > the server, such as starting internet listeners. > > @+ Manual Page > The server is a multithreaded program. There is one system thread, and for > each service, e.g. a client session or transaction service, > there is one worker thread. > As a default, the server also starts the internet listener thread, > on the port number specified in the parameter file. > > @- Usage > @verbatim > usage: mserver5 [options] [scripts] > --dbname=<database_name> > --dbfarm=<directory> > --dbinit=<stmt> Server prepare statement > --config=<config_file> > --debug=<number> trace server actions[0] > --daemon=yes|no run in background [no] > --set <option>=<value> set environment value > --help this list of options > --version version and compile time info > @end verbatim > > The server options have the following meaning: > @table @code > @item --dbname <db-name> > open the database <db-name>. > @item --config <config-file> > where to find the environment settings > @item --dbinit <stmt> > execute the statement first. > @end itemize > > @{ > @+ Implementation > @h > #ifndef _MONET_GLOBAL_H_ > #define _MONET_GLOBAL_H_ > #include "mal_config.h" > #include "monet_options.h" > #include "mal.h" > #include "mal_session.h" > #include "mal_import.h" > #include "mal_client.h" > #include "mal_function.h" > > /* #define MONET_GLOBAL_DEBUG */ > #endif /* _MONET_GLOBAL_H_ */ > @c > #include "mal_config.h" > #include "mserver5.h" > #include "mal_authorize.h" > #include "mal_sabaoth.h" > #include <stdio.h> > #include <errno.h> > #include <string.h> /* strerror */ > #include <locale.h> > #include "monet_version.h" > > #ifdef _CRTDBG_MAP_ALLOC > /* Windows only: > our definition of new and delete clashes with the one if > _CRTDBG_MAP_ALLOC is defined. > */ > #undef _CRTDBG_MAP_ALLOC > #endif > > #if defined(_MSC_VER) && _MSC_VER >= 1400 > #define getcwd _getcwd > #endif > @- > The architecture is setup to handle multiple streams of requests. > The first thread started represents the server. It reads from standard input > and writes to standard input. This is also a way to recognize the server > actions. > To start the server in the background one should use the argument -background. > This closes standard input. Direct execution in the background may cause > the server to hang in stdio for input from the terminal. > @ > The server thread started remains in existence until all other threads die. > The server is stopped by cntrl-D or receiving the quit command. > @ > > @c > static int malloc_init = 1; > /* NEEDED? */ > #if defined(_MSC_VER) && defined(__cplusplus) > #include <eh.h> > void > mserver_abort() > { > fprintf(stderr, "\n! mserver_abort() was called by terminate(). !\n"); > fflush(stderr); > MT_global_exit(0); > } > #endif > > void > usage(char *prog) > { > fprintf(stderr, "Usage: %s [options] [scripts]\n", prog); > fprintf(stderr, " --dbname=<database_name> \n"); > fprintf(stderr, " --dbfarm=<database_directory> \n"); > fprintf(stderr," --version \n"); > fprintf(stderr," --help for more details\n"); > exit(0); > } > void > usage2(char *prog) > { > fprintf(stderr, "Usage: %s [options] [scripts]\n", prog); > fprintf(stderr, " --dbname=<database_name> \n"); > fprintf(stderr, " --dbfarm=<directory> \n"); > fprintf(stderr, " --dbinit=<stmt> Server prepare statement\n"); > fprintf(stderr, " --config=<config_file> \n"); > fprintf(stderr, " --daemon=yes|no run in background [no]\n"); > fprintf(stderr, " --set <option>=<value> set environment value\n"); > fprintf(stderr, " --help this list of options \n"); > fprintf(stderr, " --version version and compile time info\n"); > > fprintf(stderr,"The debug options:\n"); > fprintf(stderr," --threads\n"); > fprintf(stderr," --memory\n"); > fprintf(stderr," --properties\n"); > fprintf(stderr," --io\n"); > fprintf(stderr," --transactions\n"); > fprintf(stderr," --modules\n"); > fprintf(stderr," --algorithms\n"); > fprintf(stderr," --xproperties\n"); > fprintf(stderr," --performance\n"); > fprintf(stderr," --debug=<bitmask>\n"); > > exit(0); > } > > @- > A welcoming message is displayed to inform the user about recent > changes. > @c > void > monet_hello(opt *set, int setlen) > { > #ifdef STATIC > char *linkinfo = "statically"; > #else > char *linkinfo = "dynamically"; > #endif > char *msg = mo_find_option(set, setlen, "monet_welcome"); > > if (msg && strcmp(msg, "yes") == 0) { > printf("# MonetDB server v" VERSION ", based on kernel v%s\n", > GDKversion()); > printf("# Serving database '%s'\n", GDKgetenv("gdk_dbname")); > printf("# Compiled for %s/" SZFMT "bit with " SZFMT "bit OIDs %s linked\n", HOST, (size_t) (sizeof(ptr) * 8), (size_t) (sizeof(oid) * 8), linkinfo); > #ifdef MONET_GLOBAL_DEBUG > printf("# Configuration:%s\n", GDKgetenv("config")); > printf("# Database farm:%s\n", GDKgetenv("gdk_dbfarm")); > #endif > printf("# Copyright (c) 1993-2007 CWI, all rights reserved\n"); > printf("# Visit http://monetdb.cwi.nl/ for further information\n"); > } > } > > @- > Version information, and compile time options. > @c > #ifdef HAVE_LIBPCRE > #include <pcre.h> > #endif > #ifdef HAVE_OPENSSL > #include <openssl/opensslv.h> > #endif > > str > absolute_path(str s) > { > if (!MT_path_absolute(s)) { > str ret = (str) GDKmalloc(strlen(s) + strlen(monet_cwd) + 2); > > sprintf(ret, "%s%c%s", monet_cwd, DIR_SEP, s); > return ret; > } > return GDKstrdup(s); > } > > @- > The options obtained during initialization should be maintained as > a global structure for other components to extract information. > @c > #define BSIZE 8192 > > int > monet_init(opt *set, int setlen) > { > char *p; > opt *n = (opt *) malloc(setlen * sizeof(opt)); > int i, j, nlen = 0; > > char *dbname = mo_find_option(set, setlen, "gdk_dbname"); > char *dbfarm = mo_find_option(set, setlen, "gdk_dbfarm"); > char *alloc_map = mo_find_option(set, setlen, "gdk_alloc_map"); > > if (n == NULL || dbname == NULL || dbfarm == NULL || alloc_map == NULL) { > fprintf(stderr, "Error, no database name or directory\n"); > if (n) > free(n); > return 0; > } > > dbfarm = mo_substitute(set, setlen, dbfarm); > > if ((p = mo_find_option(set, setlen, "gdk_debug"))) > GDKdebug = strtol(p, NULL, 10); > > if ((p = mo_find_option(set, setlen, "gdk_mem_pagebits"))) > GDK_mem_pagebits = strtol(p, NULL, 10); > > if ((p = mo_find_option(set, setlen, "gdk_vmtrim"))) > GDK_vm_trim = strcasecmp(p, "yes") == 0; > > /* determine Monet's kernel settings */ > if (!GDKinit(dbname, dbfarm, strcasecmp(alloc_map, "yes") == 0) ) { > free(dbfarm); > free(n); > return 0; > } > free(dbfarm); > > @- > Find duplicate entries in the property list and move > them to the front. Actually, this should be done in > monet_options.mx.in > @c > for (i = 0; i < setlen; i++) { > int done = 0; > > for (j = 0; j < nlen; j++) { > if (strcmp(n[j].name, set[i].name) == 0) { > if (n[j].kind < set[i].kind) { > n[j] = set[i]; > } > done = 1; > break; > } > } > if (!done) { > n[nlen] = set[i]; > nlen++; > } > } > for (i = 0; i < nlen; i++) { > char *value; > > value = mo_substitute(n, nlen, n[i].value); > GDKsetenv(n[i].name, value); > free(value); > } > free(n); > > if ((p = GDKgetenv("gdk_mem_bigsize"))) { > /* when allocating >6% of all RAM; do so using vmalloc() iso malloc() */ > lng max_mem_bigsize = GDK_mem_maxsize/16; > > /* sanity check to avoid memory fragmentation */ > GDK_mem_bigsize = (size_t) MIN(max_mem_bigsize, strtol(p, NULL, 10)); > } > if (GDKgetenv_isyes("gdk_embedded") || GDKgetenv_isyes("embedded")) { > GDKembedded = 1; > } > > if (GDKgetenv_isyes("monet_daemon") || GDKgetenv_isyes("daemon")) { > monet_daemon = 1; > #ifdef HAVE_SETSID > setsid(); > #endif > } > > monet_hello(set, setlen); > /* you don't need the commandline arguments anymore */ > mo_free_options(set, setlen); > return 1; > } > > void emergencyBreakpoint(){ > /* just a handle to break after system initialization for GDB */ > } > > static void > handler(int sig) > { > (void)sig; > mal_exit(); > } > > int > main(int argc, char **av) > { > char *prog = *av; > opt *set = NULL; > int idx = 0, debug = 0, setlen = 0, listing = 0, i = 0; > str dbinit = NULL; > str err = MAL_SUCCEED; > > static struct option long_options[] = { > {"config", 1, 0, 'c'}, > {"dbname", 1, 0, 0}, > {"dbfarm", 1, 0, 0}, > {"dbinit", 1, 0, 0}, > {"daemon", 1, 0, 0}, > {"debug", 2, 0, 'd'}, > {"help", 0, 0, 'h'}, > {"version", 0, 0, 0}, > {"set", 1, 0, 's'}, > {"trace", 0, 0, 't'}, > {"threads",0,0,0}, > {"memory",0,0,0}, > {"properties",0,0,0}, > {"io",0,0,0}, > {"transaction",0,0,0}, > {"modules",0,0,0}, > {"algorithms",0,0,0}, > {"performance",0,0,0}, > {"xproperties",0,0,0}, > {0, 0, 0, 0} > }; > > @- > We give malloc advice here. Main goal: prevent fragmentation. > We do this by declaring everything below 2K as 'small'. These > values will be drawn from a fixed pools of 400K. > A grain size of 128 bytes is used to keep overhead low. > > We do this by declaring everything below 2K as 'small'. These > values will be drawn from a fixed pools of 400K. > A grain size of 128 bytes is used to keep overhead low. > Trivial remark: for dynamically linked executables the mallopt > capabilities depend on the malloc implementation used at run time. > > Unlike V4 we ignore the alloc_map advice, which leads to a much > faster system start. > @= mallopt > if (malloc_init) { > /* for (Red Hat) Linux (6.2) unused and ignored at least as of glibc-2.1.3-15 */ > /* for (Red Hat) Linux (8) used at least as of glibc-2.2.93-5 */ > if (mallopt(M_MXFAST, 192)) { > fprintf(stderr, "!monet: mallopt(M_MXFAST,192) fails.\n"); > } > #ifdef M_BLKSZ > if (mallopt(M_BLKSZ, 8*1024)) { > fprintf(stderr, "!monet: mallopt(M_BLKSZ,8*1024) fails.\n"); > } > #endif > } > malloc_init=0; > @c > > #if defined(_MSC_VER) && defined(__cplusplus) > set_terminate(mserver_abort); > #endif > if (setlocale(LC_CTYPE, "") == NULL) { > GDKfatal("cannot set locale\n"); > } > > #ifdef HAVE_MALLOPT > @:mallopt@ > #else > (void) malloc_init; /* still unused */ > #endif > > if (getcwd(monet_cwd, PATHLENGTH - 1) == NULL) { > perror("pwd"); > GDKfatal("monet_init: could not determine current directory\n"); > } > > if (!(setlen = mo_builtin_settings(&set))) > usage(prog); > > setlen = mo_add_option(&set, setlen, opt_config, "prefix", MONETDBPREFIX); > setlen = mo_add_option(&set, setlen, opt_config, "config", MONETDBCONFIG); > > for (;;) { > int option_index = 0; > > int c = getopt_long(argc, av, "c:d::t:r:h?s:m:i:a:e:x", > long_options, &option_index); > > if (c == -1) > break; > > switch (c) { > case 0: > if (strcmp(long_options[option_index].name, "dbname") == 0) { > setlen = mo_add_option(&set, setlen, opt_cmdline, "gdk_dbname", optarg); > break; > } > if (strcmp(long_options[option_index].name, "dbfarm") == 0) { > setlen = mo_add_option(&set, setlen, opt_cmdline, "gdk_dbfarm", optarg); > break; > } > if (strcmp(long_options[option_index].name, "dbinit") == 0) { > if (dbinit) > fprintf(stderr, "#warning: ignoring multiple --dbinit argument\n"); > else > dbinit = optarg; > break; > } > if (strcmp(long_options[option_index].name, "daemon") == 0) { > setlen = mo_add_option(&set, setlen, opt_cmdline, "gdk_daemon", optarg); > break; > } > if (strcmp(long_options[option_index].name, "version") == 0) { > monet_version(); > break; > } > /* debugging options */ > if (strcmp(long_options[option_index].name, "properties") == 0) { > debug |=GRPproperties; > break; > } > if (strcmp(long_options[option_index].name, "algorithms") == 0) { > debug|= GRPalgorithms; > break; > } > if (strcmp(long_options[option_index].name, "xproperties") == 0) { > debug|= GRPxproperties; > break; > } > if (strcmp(long_options[option_index].name, "performance") == 0) { > debug|= GRPperformance; > break; > } > if (strcmp(long_options[option_index].name, "io") == 0) { > debug|= GRPio; > break; > } > if (strcmp(long_options[option_index].name, "memory") == 0) { > debug|= GRPmemory; > break; > } > if (strcmp(long_options[option_index].name, "modules") == 0) { > debug|= GRPmodules; > break; > } > if (strcmp(long_options[option_index].name, "transactions") == 0) { > debug|= GRPtransactions; > break; > } > if (strcmp(long_options[option_index].name, "threads") == 0) { > debug|= GRPthreads; > break; > } > usage(prog); > break; > case 'c': > setlen = mo_add_option(&set, setlen, opt_cmdline, "config", optarg); > break; > case 'd': > if (optarg) { > setlen = mo_add_option(&set, setlen, opt_cmdline, "gdk_debug", optarg); > } else { > debug = 1; > } > break; > case 's':{ > /* should add option to a list */ > char *tmp = strchr(optarg, '='); > > if (tmp) { > *tmp = '\0'; > setlen = mo_add_option(&set, setlen, opt_cmdline, optarg, tmp + 1); > } else { > fprintf(stderr, "ERROR: wrong format %s\n", optarg); > } > } > break; > case 't': /* trace option, ignored to reduce testweb complaints > fprintf(stderr, "#warning: trace option not yet supported\n"); > */ > break; > case 'h': > if(strcmp(long_options[option_index].name, "help") == 0) { > usage2(prog); > break; > } > case '?': > usage(prog); > default: > fprintf(stderr, "ERROR: getopt returned character " > "code '%c' 0%o\n",c, c); > usage(prog); > } > } > > if (!(setlen = mo_system_config(&set, setlen))) > usage(prog); > > monet_script = (str *) GDKmalloc(sizeof(str) * (argc + 1)); > monet_script[idx] = NULL; > while (optind < argc) { > monet_script[idx] = absolute_path(av[optind]); > monet_script[idx + 1] = NULL; > optind++; > idx++; > } > > if (debug) > mo_print_options(set, setlen); > if (monet_init(set, setlen) == 0 ) > return 0; > /* propagate the debug flag to the environment table */ > if( debug){ > char buf[BUFSIZ]; > snprintf(buf,BUFSIZ,"%d",debug); > GDKsetenv("gdk_debug", buf); > GDKdebug=debug; /* overruled by command line */ > } > > GDKsetenv("monet_version", VERSION); > > /* configure sabaoth to use the right dbfarm and active database */ > SABAOTHinit(GDKgetenv("gdk_dbfarm"), GDKgetenv("gdk_dbname")); > /* wipe out all cruft, if left over */ > if ((err = SABAOTHwildRetreat(&i)) != MAL_SUCCEED) { > /* just swallow the error */ > GDKfree(err); > } > /* From this point, the server should exit cleanly. Discussion: > * even earlier? Sabaoth here registers the server has started up. */ > if ((err = SABAOTHregisterStart(&i)) != MAL_SUCCEED) { > /* throw the error at the user, but don't die */ > fprintf(stderr, "!%s\n", err); > GDKfree(err); > } > > #ifdef HAVE_SIGACTION > { > struct sigaction sa; > > sigemptyset(&sa.sa_mask); > sa.sa_flags = 0; > sa.sa_handler = handler; > if ( > sigaction(SIGINT, &sa, NULL) == -1 || > sigaction(SIGQUIT, &sa, NULL) == -1 || > sigaction(SIGTERM, &sa, NULL) == -1) > { > fprintf(stderr, "!unable to create signal handlers\n"); > } > } > #else > signal(SIGINT, handler); > #ifdef SIGQUIT > signal(SIGQUIT, handler); > #endif > signal(SIGTERM, handler); > #endif > > { > str lang = "mal"; > /* we inited mal before, so publish its existence */ > if ((err = SABAOTHmarchScenario(&i, &lang)) != MAL_SUCCEED) { > /* throw the error at the user, but don't die */ > fprintf(stderr, "!%s\n", err); > GDKfree(err); > } > } > > { > /* unlock the vault, first see if we can find the file which > * holds the secret */ > char* secret = alloca(sizeof(char) * 1024); > FILE* secretf; > size_t len; > > if (GDKgetenv("monet_vault_key") == NULL) { > /* use a default (hard coded, non safe) key */ > secret = "Xas632jsi2whjds8"; > fprintf(stderr, "#warning: please don't forget to set your " > "vault key!\n#(see %s)\n", GDKgetenv("config")); > } else { > if ((secretf = fopen(GDKgetenv("monet_vault_key"), "r")) == NULL) { > snprintf(secret, 1023, "unable to open vault_key_file %s: %s", > GDKgetenv("monet_vault_key"), strerror(errno)); > secret[1023] = '\0'; > GDKfatal(secret); > } > len = fread(secret, 1, 1023, secretf); > secret[len] = '\0'; > len = strlen(secret); /* secret can contain null-bytes */ > if (len < 5) { > fprintf(stderr, "#warning: your vault key is too short " > "(" SZFMT "), enlarge your vault key!\n", > len); /* penis */ > } > fclose(secretf); > } > if ((err = AUTHunlockVault(&secret)) != MAL_SUCCEED) > GDKfatal(err); > } > /* make sure the authorisation BATs are loaded */ > if ((err = AUTHinitTables()) != MAL_SUCCEED) > GDKfatal(err); > if (mal_init()) > return 0; > > @- > Time to execute the script files. > Start with the init command[todo], which may be defined in > the configuration file. > @c > MSinitClientPrg(mal_clients, "user","main"); > if (dbinit == NULL) > dbinit = GDKgetenv("dbinit"); > if (dbinit) { > str input = GDKstrdup(dbinit); > > callString(mal_clients, input, listing); > GDKfree(input); > } > > if (GDKgetenv("mal_listing")) > sscanf(GDKgetenv("mal_listing"), "%d", &listing); > > emergencyBreakpoint(); > for (i = 0; monet_script[i]; i++) { > str msg=evalFile(mal_clients, monet_script[i], listing); > /* check for internal exception message to terminate */ > if(msg && strcmp(msg,"MALException:client.quit:Server stopped.")==0) > mal_exit(); > if( msg) GDKfree(msg); > GDKfree(monet_script[i]); > monet_script[i] = 0; > } > GDKfree(monet_script); > if (monet_daemon) { > while(1) > MT_sleep_ms(5000); > } else > MSserveClient(mal_clients); > > /* mal_exit calls MT_global_exit, so statements after this call will > * never get reached */ > mal_exit(); > > return 0; > } > @} > > --- NEW FILE: monet_version.c.in --- > > #include "monet_version.h" > > void > monet_version() > { > printf("MonetDB server v" VERSION " (" SZFMT "-bit), " > "based on kernel v%s (" SZFMT "-bit oids)\n", > (size_t) (sizeof(ptr) * 8), GDKversion(), > (size_t) (sizeof(oid) * 8)); > printf("Copyright (c) 1993-2007 CWI, all rights reserved\n"); > printf("Visit http://monetdb.cwi.nl/ for further information\n"); > printf("Configured for prefix: " MONETDBPREFIX "\n"); > printf("Libraries:\n"); > #ifdef HAVE_LIBPCRE > printf(" libpcre: %s (%s)\n", pcre_version(), "@pcreversion@"); > #endif > #ifdef HAVE_OPENSSL > printf(" openssl: %s (%s)\n", OPENSSL_VERSION_TEXT, "@opensslversion@"); > #endif > printf("Compiled by: %s\n", "@builtby@"); > printf("Compilation: %s\n", "@compilercall@"); > #ifdef STATIC > printf("Linking : %s (static)\n", "@linkercall@"); > #else > printf("Linking : %s\n", "@linkercall@"); > #endif > exit(0); > } > > --- mserver5.mx.in DELETED --- -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] MAL access to tables created with SQL
by darabi＠web.de 13 Oct '07

13 Oct '07

Niels.Nes(a)cwi.nl wrote: > Well embedded_mal could still load the sql module and then extra > threads maybe created. Its more a warning, sql should be the one > making changes to the catalog. If you think of you application/extention > as do some sql to create tables, then exit sql, do querying with your > own frontend, it should work out just fine. Just to make sure, I understand correctly what you mean: - I can open an sql session with mclient and modify the tables with sql - then I open a mal session and query the bats/create the additional bats? My question is: can I open a mal session and somehow query/modify the sql catalog with the functions in the sql module? For example, is it legal to do this: $ mclient -l mal -d demo -H mal>include sql; mal>sql.start(); mal>bat1 := sql.bind("sys","_tables","id",1); The last statement hangs the client. If this is not allowed/possible in mclient, would I be able to do that with an embedded server, which is started with embedded_mal? > Anyway this means you'll have to get in depth knowledge on m5 and sql. > Which afcourse you can only obtain by reading all the manuals etc and > playing with the code, we won't have the time to give you this by email. Yes, this is what I'm trying to do and I didn't expect you to give me this be email. You (well, and the colleagues your organisation :) did enough to a) create this amazing piece of software b) make it open source, so everyone can learn from it c) write some extensive documentation (which is not usual for OSS) and I appreciate your doing so! Unfortunately, the first days are the hardest. So, bear with me if I ask stupid questions. I hope I could clarify what I would like to do: 1) use module sql (function in MonetDB-SQL/sql/src/backends/monet5/sql.mal) to do most of the work 2) use some additional bats to store information related to the objects If this is a bad idea, I would appreciate any hint. Thanks a lot Kambiz

1 1

[Monetdb-developers] MAL access to tables created with SQL
by darabi＠web.de 13 Oct '07

13 Oct '07

Hello, how is it possible to access in a MAL client session (mclient -l mal) the BATs of tables which were created previously in sql? I have tried different versions of bat.new, and bbp.bind. I tried different naming schemes (e.g. "tablename_columnname" which was mentioned somewhere in the docs), but I always get Referenced BAT not found And when I try to use sql.bind, I get SQLException:mvc:MAL_SCENARIO_PARSER not initialized Another question: is there a way of seeing the MAL translation of sql queries? EXPLAIN works great for SELECT statements, but it doesn't translate DDL expressions like CREATE TABLE. Many thanks Kambiz

4 8

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/BugDay_2005-10-06_2.8/Tests MapiClient-dump.SF-905851.SQL.py, SQL_2-20, 1.1.2.1, 1.1.2.2
by Stefan Manegold 13 Oct '07

13 Oct '07

Hi, this any all other tests that were ported from .sh/.bat to .py yesterday now fail on platforms that have a pyhton installed without subprocess; cf., http://monetdb.cwi.nl/testing/projects/monetdb/Stable/sql/.mTests103/index_… To solve the problem, in all newly added .py scripts, ======== import subprocess ======== should be replaced by ======== try: import subprocess except ImportError: # use private copy for old Python versions import MonetDB.subprocess26 as subprocess ======== as in Mtest.py[.in] to fall-back to the subprocess that we provide with MonetDB in case there is none available on the respective system. However, before checking in, please coordinate with Sjoerd, first! Stefan On Fri, Oct 12, 2007 at 09:53:41AM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/test/BugDay_2005-10-06_2.8/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31626 > > Modified Files: > Tag: SQL_2-20 > MapiClient-dump.SF-905851.SQL.py > Log Message: > On windows the stdout and stderr of a python subprocess is not automatically redirect to > main process stdout and stderr. > The proper way is to use Popen and then use a PIPE to read the stderr > and stdout from the subprocess. > > NOTE: This script was tested on windows and Linux, and the test became green on both platforms. ;) > > > > Index: MapiClient-dump.SF-905851.SQL.py > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/BugDay_2005-10-06_2.8/Tests/MapiClient-dump.SF-905851.SQL.py,v > retrieving revision 1.1.2.1 > retrieving revision 1.1.2.2 > diff -u -d -r1.1.2.1 -r1.1.2.2 > --- MapiClient-dump.SF-905851.SQL.py 11 Oct 2007 14:40:50 -0000 1.1.2.1 > +++ MapiClient-dump.SF-905851.SQL.py 12 Oct 2007 09:53:39 -0000 1.1.2.2 > @@ -1,15 +1,21 @@ > -import os, sys > +import os, sys, subprocess > + > + > +def client(cmd): > + clt = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) > + sys.stdout.write(clt.stdout.read()) > + clt.stdout.close() > + sys.stderr.write(clt.stderr.read()) > + clt.stderr.close() > + > > > def main(): > clcmd = ("%s/bin/Mlog -x " % os.getenv('MONETDB_PREFIX')) + str(os.getenv('SQL_CLIENT')) + "< %s" % ('%s/JdbcClient_create_tables.sql' % os.getenv('RELSRCDIR')) > clcmd1 = ("%s/bin/Mlog -x " % os.getenv('MONETDB_PREFIX')) + str(os.getenv('SQL_CLIENT')) + "< %s" % ('%s/JdbcClient_inserts_selects.sql' % os.getenv('RELSRCDIR')) > clcmd2 = ("%s/bin/Mlog -x " % os.getenv('MONETDB_PREFIX')) + str(os.getenv('SQL_DUMP')) > - clt = os.popen(clcmd, 'w') > - clt.close() > - clt1 = os.popen(clcmd1, 'w') > - clt1.close() > - clt2 = os.popen(clcmd2, 'w') > - clt2.close() > + client(clcmd) > + client(clcmd1) > + client(clcmd2) > > main() > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] clients on windows 32...
by Romulo Goncalves 13 Oct '07

13 Oct '07

Why the option \h - show the readline history mclient.exe -lsql --help Is not available on windows? Is an design feature or a bug? Regards, Romulo

6 16

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/tools mcheckpoint, , 1.2, 1.3
by Sjoerd Mullender 12 Oct '07

12 Oct '07

How does this checkin message cover the change made here? On 2007-10-12 19:40, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv9755/tools > > Modified Files: > mcheckpoint > Log Message: > Use a stricter runtime test to trap possible BAT bind mismatchs. > > > Index: mcheckpoint > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/tools/mcheckpoint,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- mcheckpoint 9 Sep 2007 10:00:17 -0000 1.2 > +++ mcheckpoint 12 Oct 2007 17:40:25 -0000 1.3 > @@ -20,9 +20,8 @@ > function usage() > { > echo "Usage: mcheckpoint [options] " > - echo " --dbname=<database_name>" > + echo " --database=<database_name>" > echo " --ascii=<yes/no> default no" > - echo " --tool=<mapi/jdbc> default mapi" > echo " --dir=<directory>" > echo " --help" > exit 0 > @@ -99,8 +98,8 @@ > die "--tool: '${1#--tool=}' not understood, use 'mapi' or 'jdbc'" > esac > ;; > - --dbname=*) > - DBNAME="${1#--dbname=}" > + --database=*) > + DBNAME="${1#--database=}" > ;; > --dir=*) > CHKPNTDIR="${1#--dir=}" > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] Simple bat operations
by darabi＠web.de 12 Oct '07

12 Oct '07

Hello Nils, OK, then something doesn't work. When I try your suggestion, I get this: user@host$ mclient -d demo -l mal mal>b := bat.new(:int,:int); mal>io.print(b); #-----------------# # h t # name # int int # type #-----------------# mal>bat.setName(b,"test"); mal>bat.setPersistent(b,true); mal>transaction.commit(); mal>\q user@host$ mclient -d demo -l mal mal>b:= bbp.bind("test"); mal>io.print(b); MAPI = monetdb@127.0.0.1:50001 QUERY = io.print(b); ERROR = !TypeException:user.main[1]:'b' may not be used before being initialized mal>\q I have downloaded and compiled MonetDB-Mars-SuperBall-SR3.tar.bz2 on an Ubuntu i686 single cpu system running a 2.6.20 kernel. user@host$ ps auxwww | grep mserver user 12072 0.0 4.3 133232 67268 ? Ssl Oct10 0:01 /opt/monetdb/bin/mserver5 --dbfarm=/opt/monetdb/var/MonetDB5/dbfarm --set daemon yes --set monet_welcome no --dbname=voc --config=/opt/monetdb/var/MonetDB5/dbfarm/voc/.monetdb.conf /opt/monetdb/var/MonetDB5/dbfarm/voc/.mserver5.dbinit user 12479 0.0 3.9 109320 61760 ? Ssl Oct10 0:00 /opt/monetdb/bin/mserver5 --dbfarm=/opt/monetdb/var/MonetDB5/dbfarm --set daemon yes --set monet_welcome no --dbname=demo --config=/opt/monetdb/var/MonetDB5/dbfarm/demo/.monetdb.conf user@host$ mserver5 --version MonetDB Server v5.0.0 Copyright (c) 1993-2007 CWI, all rights reserved Compiled by: sjoerd(a)localhost.localdomain Compilation: gcc -O2 -std=c99 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt Linking: ld -IPA -m elf_x86_64 I already tried 'mclient -l sql' without any apparent problems, so it appeared unlikely that there were bigger issues. I will try to run the test suite(s). Thank you very much Kambiz ----- Original Message ----- From: "Niels Nes" <Niels.Nes(a)cwi.nl> To: darabi(a)web.de Sent: Donnerstag, 11. Oktober 2007 16.24 Uhr (GMT+0100) Europe/Berlin Subject: Re: [Monetdb-developers] Simple bat operations On Thu, Oct 11, 2007 at 02:34:15PM +0200, darabi(a)web.de wrote: > In the meantime, I have also tried all the bbp functions which > I could think of: > > bbp.find > bbp.bind > bbp.take > > But none of them gives me a bat which I can use. Strangely, none of > them says anything about whether the function call succeeded > or not. okay here a small example then b := bat.new(:int,:int); io.print(b); bat.setName(b,"test"); bat.setPersistent(b,true); transaction.commit(); # after server restart the following should work. b:= bbp.bind("test"); io.print(b); Niels > > > Kambiz > > ----- Original Message ----- > From: darabi(a)web.de > To: monetdb-developers(a)lists.sourceforge.net > Cc: "Niels Nes" <Niels.Nes(a)cwi.nl> > Sent: Donnerstag, 11. Oktober 2007 9.47 Uhr (GMT+0100) Europe/Berlin > Subject: Re: [Monetdb-developers] Simple bat operations > > Niels, > > thank you very much for your help. > > How can I access this bat, when I start a new session? > > bat.load("person_name"); > > doesn't seem to work. io.print(n): > > ERROR = !TypeException:user.main[1]:'n' may not be used before being initialized > > > Thanks > > Kambiz > > ----- Original Message ----- > From: "Niels Nes" > To: darabi(a)web.de > Cc: monetdb-developers(a)lists.sourceforge.net > Sent: Donnerstag, 11. Oktober 2007 9.38 Uhr (GMT+0100) Europe/Berlin > Subject: Re: [Monetdb-developers] Simple bat operations > > On Thu, Oct 11, 2007 at 08:00:36AM +0200, darabi... wrote: > > Hello, > > > > sorry for asking so many dumb questions, but I don't seem > > to be able to find the answers in the docs. > > > > I try to create and persist a bat: > > > > (mclient -d demo -l mal) > > > > n := bat.new(:oid,:str); > > bat.setName(n, "person_name"); > > bat.setPersistent(n, true); > > > > bat.append(n, "John", true); > > bat.append(n, "James", true); > > > > replace this save > > bat.save(n); > by > > transaction.commit(); > > Niels > > (ps later you probably want to switch to subcommit, ie only commit those > bat that changed, for this see the documenation in the transaction module > (src/modules/mal/transaction.mx) > > > > But this doesn't seem to be sufficient. > > > > Or at least I then don't know how to load this bat the next > > time I start mclient. > > > > Is there a file (one of the tests or sth.) which contains > > such basic tests? I tried to find-grep over them, but didn't > > succeed in finding what I was looking for. > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

6 20

Re: [Monetdb-developers] Monetdb-developers Digest, Vol 17, Issue 6
by Peter Boncz 11 Oct '07

11 Oct '07

Dear Stefan, Thanks for your question, but I regret to inform you that your way of communicating technical problems severely falls short of what is required by technical etiquette: (1) ambiguous software version description: you cannot be running MonetDB4 and MonetDB5 at the same time. Even more, I want to know the exact version number of the software, whether it was a 32-bits or 64-bits build (and if so, whether it used 64-bits or 32-bits oids). (2) major omissions in your platform description. I presume you have installed an operating system on your Xeon. Would you care to tell us which operating system that is? (3) ill-described reproduction path. Your bug report suggests being about updates, but as far as I read your email, I doubt you did any updates yet. This is again is crucial information. Thus, my main answer is that you have to retry describing your problem, paying better attention to the above three points. The error you see is the "remap" call failing -- the most probable cause of error is VM space shortage. For MonetDB to work on 20GB size datasets you must use a 64-bits operating system and MonetDB binary, and may even 64-bits oids (because your volume of text nodes is likely to be significant). Peter Boncz

2 3

[Monetdb-developers] Simple bat operations
by darabi＠web.de 11 Oct '07

11 Oct '07

Hello, sorry for asking so many dumb questions, but I don't seem to be able to find the answers in the docs. I try to create and persist a bat: (mclient -d demo -l mal) n := bat.new(:oid,:str); bat.setName(n, "person_name"); bat.setPersistent(n, true); bat.append(n, "John", true); bat.append(n, "James", true); bat.save(n); But this doesn't seem to be sufficient. Or at least I then don't know how to load this bat the next time I start mclient. Is there a file (one of the tests or sth.) which contains such basic tests? I tried to find-grep over them, but didn't succeed in finding what I was looking for. Thanks you for your patience. Kambiz

2 3

[Monetdb-developers] Simple bat operations
by Kambiz Darabi 11 Oct '07

11 Oct '07

Hello, sorry for asking so many dumb questions, but I don't seem to be able to find the answers in the docs. I try to create and persist a bat: (mclient -d demo -l mal) n := bat.new(:oid,:str); bat.setName(n, "person_name"); bat.setPersistent(n, true); bat.append(n, "John", true); bat.append(n, "James", true); bat.save(n); But this doesn't seem to be sufficient. Or at least I then don't know how to load this bat the next time I start mclient. Is there a file (one of the tests or sth.) which contains such basic tests? I tried to find-grep over them, but didn't succeed in finding what I was looking for. Thanks you for your patience. Kambiz

1 0

[Monetdb-developers] Fixing the update issue for Large XML documents
by Stefan de Konink 11 Oct '07

11 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, A short introduction. I'm Stefan de Konink, one of the students in the ADT course at the University of Amsterdam. I have a degree in Advanced Computer Science, so I know what C looks like ;) Currently I can use MonetDB on a rather heavy machine, basically to benchmark the storage of this machine. And look for bottle necks. I am also a participant of OpenStreetMap NL, as datacollector from already existing GIS data. OpenStreetMap runs on MySQL in the UK, and I wanted to find out if I could run it more efficiently on MonetDB. For compatibility reasons OSM publishes an XML file containing the current view of the database daily. My 'job' to see if I could get the same results on this XML file (using XQuery) as the active MySQL/Ruby implementation. Importing a 20GB XML file was great, but now the 'lessons' progressed we should update the data. So I took a recent version of the document (they migrated from one format to another this week), allocated 5% slackspace and run the pf:add-doc. This went ok, but the trick to count all nodes ended up in: > xquery> > more> fn:count(doc("planet.osm")//*) > more>MAPI = monetdb@localhost:50000 > QUERY = fn:count(doc("planet.osm")//*) > ERROR = !ERROR: [remap]: 5 times inserted nil due to errors at tuples > 1@0, 2@0, 3@0, 4@0, 5@0. > !ERROR: [remap]: first error was: > !ERROR: CMDremap: operation failed. > !ERROR: interpret_unpin: [remap] bat=492,stamp=-729 OVERWRITTEN > !ERROR: BBPdecref: 1000000001_rid_nid does not have pointer fixes. > !ERROR: interpret_params: leftfetchjoin(param 2): evaluation error. (As posted to the bugtracker and privately to the lecturer.) My first guess was that the 1..5 maybe had something to do with the 'slackspace' that was five also. Today I imported the same document readonly without any problems. To confirm 'import-updating' in general works I took the data from only The Netherlands and imported this. (143MB) Which worked like a charm. I'm running MonetDB4/5 (SR3) on a Quad Xeon, 8GB memory. Storage is a bit variable ;) But currently on local raid5. Are there any suggestions from the developers to catch this bug? Upgrading to CVS could be an option. Trying again with CVS is an option, trying again with a smaller document too, or if someone instantly comes up with a solution that could be nice too ;) Yours Sincerely, Stefan de Konink -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHDXXIYH1+F2Rqwn0RCk/VAJwLpXp0Yl1ENToPMRFgIJy2No/BMgCdFv7C dGQzJgoGmwM9ucPxrohIkFk= =sHEr -----END PGP SIGNATURE-----

1 0

Re: [Monetdb-developers] Object persistence using MonetDB
by darabi＠web.de 10 Oct '07

10 Oct '07

Thank you for the information. So, does that mean that you generally do not recommend the use of MonetDB beyond the realms of > datawarehouses, information retrieval, and data mining ? >From the result of the web application contest in the ct magazine, I had the impression that 'normal' applications could also benefit from the architecture of MonetDB: http://sourceforge.net/forum/forum.php?forum_id=584739 What I would like to do is to create a persistence engine for a Prolog like language which has a type system. Which means that data is to be stored in tuples. To experiment with MAL, I would have to use the MapiClient, right? I think I will try to play with it. Thank you Kambiz ----- Original Message ----- From: "Martin Kersten" <Martin.Kersten(a)cwi.nl> To: darabi(a)web.de, monetdb-developers-bounces(a)lists.sourceforge.net Sent: Mittwoch, 10. Oktober 2007 11.01 Uhr (GMT+0100) Europe/Berlin Subject: Re: [Monetdb-developers] Object persistence using MonetDB darabi(a)web.de wrote: > Hello, > > I'm a developer located in Germany with a current interest in > object persistence for a dynamically typed language. > Hello, thank you for your interest in the MonetDB software. > I was experimenting with something along the lines of the > decomposed storage model when I found out that it was (of course) > decades old and found the papers about it and MonetDB. > > Many thanks for your efforts and for this great piece of software. > > I was reading the 1995 CWI report 'High Performance support for > OO traversals in monet' and was wondering whether the ideas > in it are still present in today's MonetDB. And if not, how > much effort would it take to implement OO persistence using > MAL. I would also appreciate any pointer to papers, code samples > etc. in this direction. > Between 1995 and 2003 MonetDB was used in the company Data Distilleries (now part of SPSS) where the interaction between application and DBMS was based on persistent objects. However, the thrust of the development is not on individual object access, but bulk processing as it emerges from datawarehouses, information retrieval, and data mining. > If I understand correctly, in the beginning, MonetDB was rather > heading towards a OODBMS. Was there any reason why you didn't > pursue that goal? Is there any road block? > There are (have been) attempts to hook up Hibernate and MonetDB. I don't know their status. If ultra-high performance is your target, I question if a DBMS is the right way to go. However, I also think that most o-o applications might better benefit from relational view on data. regards, Martin > > Thank you very much for your help. > > > Kambiz Darabi >

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util CmdLineOpts.java, XQuery_0-20, NONE, 1.1.2.1 OptionsException.java, XQuery_0-20, NONE, 1.1.2.1
by Fabian Groffen 10 Oct '07

10 Oct '07

On 10-10-2007 11:01:20 +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8103/src/tools/java/src/nl/cwi/monetdb/xquery/util > > Added Files: > Tag: XQuery_0-20 > CmdLineOpts.java OptionsException.java > Log Message: > Moved the XRPC Wrapper stuff into pathfinder/src/tools/java, because > XRPC Wrapper is not a client program and it is XQuery related. > > The XRPC Wrapper is build in a separate RPM package > (MonetDB4-XQuery-xrpcwrapper), so that the > pathfinder package is still Java-independent. Please note that this is code replication. (which IMHO is BAD).

2 3

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/NT monetdb_winconfig.h, , 1.2, 1.3
by Niels Nes 10 Oct '07

10 Oct '07

On Wed, Oct 10, 2007 at 09:05:45AM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB/NT > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14659 > > Modified Files: > monetdb_winconfig.h > Log Message: > Move the definition of ssize_t as int into the win32 else clause > Please check if it is correct like this. The compilation works, but it gives warnnings > about probable lost of data witht the convertion from ssize_t to BUN. Shouldn't this also be done on the stable? The move of the line is fine. BUN in cvs head is still a 'size_t' (same as oid), ie ssize_t (signed) indeed may lose info. Niels > > > Index: monetdb_winconfig.h > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/NT/monetdb_winconfig.h,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- monetdb_winconfig.h 3 Oct 2007 13:59:59 -0000 1.2 > +++ monetdb_winconfig.h 10 Oct 2007 09:05:43 -0000 1.3 > @@ -668,13 +668,14 @@ > > /* The size of a `void *', as computed by sizeof. */ > #define SIZEOF_VOID_P 4 > -#endif > + > > /* The size of a `ssize_t', as computed by sizeof. */ > #define SIZEOF_SSIZE_T SIZEOF_SIZE_T > typedef int ssize_t; > #define HAVE_SSIZE_T 1 > > +#endif > /* The size of a `__int64', as computed by sizeof. */ > #define SIZEOF___INT64 8 > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

3 2

[Monetdb-developers] Object persistence using MonetDB
by darabi＠web.de 10 Oct '07

10 Oct '07

Hello, I'm a developer located in Germany with a current interest in object persistence for a dynamically typed language. I was experimenting with something along the lines of the decomposed storage model when I found out that it was (of course) decades old and found the papers about it and MonetDB. Many thanks for your efforts and for this great piece of software. I was reading the 1995 CWI report 'High Performance support for OO traversals in monet' and was wondering whether the ideas in it are still present in today's MonetDB. And if not, how much effort would it take to implement OO persistence using MAL. I would also appreciate any pointer to papers, code samples etc. in this direction. If I understand correctly, in the beginning, MonetDB was rather heading towards a OODBMS. Was there any reason why you didn't pursue that goal? Is there any road block? Thank you very much for your help. Kambiz Darabi

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/compiler/Tests mal01.stable.out, , 1.1, 1.2 sql01.mal, , 1.1, 1.2 sql01.stable.out, , 1.1, 1.2
by Stefan Manegold 10 Oct '07

10 Oct '07

On Tue, Oct 09, 2007 at 08:09:05PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/compiler/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28543/Tests > > Modified Files: > mal01.stable.out sql01.mal sql01.stable.out > Log Message: > Make the test self-contained, i.e. not dependent on sql libraries. Just as a double-check: Are the warnings below expected/correct, and hence their approval intended? Stefan > Index: mal01.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/compiler/Tests/mal01.stable.out,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- mal01.stable.out 7 Oct 2007 06:44:53 -0000 1.1 > +++ mal01.stable.out 9 Oct 2007 20:09:01 -0000 1.2 [...] > +!WARNING: call to bat.new can not be handled correctly [...] > > Index: sql01.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/compiler/Tests/sql01.stable.out,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- sql01.stable.out 7 Oct 2007 06:44:55 -0000 1.1 > +++ sql01.stable.out 9 Oct 2007 20:09:02 -0000 1.2 > +MCdynamicCompiler: sql.qry > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/compiler/Tests All, , NONE, 1.1 mal01.mal, , NONE, 1.1 mal01.stable.err, , NONE, 1.1 mal01.stable.out, , NONE, 1.1 sql01.mal, , NONE, 1.1 sql01.stable.err, , NONE, 1.1 sql01.stable.out, , NONE, 1.1
by Stefan Manegold 09 Oct '07

09 Oct '07

On Sun, Oct 07, 2007 at 06:44:57AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/compiler/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1743/Tests > > Added Files: > All mal01.mal mal01.stable.err mal01.stable.out sql01.mal > sql01.stable.err sql01.stable.out > Log Message: > Patching some very old code to bring it closer to reality. > The examples roughly indicate the code to be considered for inclusion > in a compiled plan. The expected gain is only 25-50% interpreter overhead, > which result from known argument locations and stack initialization actions. > A real optimizing MAL ->C compiler should weave instructions to > more efficient ones, e.g. dropping intermediates. > > The SQL version highlights, however, the calls needed against the BBP > due to the shared bat assumption. > > [...] > --- NEW FILE: sql01.mal --- > include sql; ^^^ This test is bound to fail here (i.e., in MonetDB5) as there is no module "sql" in MonetDB5; module "sql" is only provided by sql. Stefan > #sql.compiler("sql01","select * from tables;"); > include mal_compiler; > function user.qry():void; > T1:bat[:oid,:int]{notnil=true,rows=0:lng,bid=113} := sql.bind("tmp","Ttables","id",0); > constraints.emptySet(T1); > T10:bat[:oid,:sht]{notnil=true,rows=0:lng,bid=117} := sql.bind("tmp","Ttables","type",0); > T13:bat[:oid,:int]{notnil=true,rows=30:lng,bid=67} := sql.bind("sys","Ttables","id",0); > T15:bat[:oid,:int]{notnil=true,rows=0:lng,bid=307} := sql.bind("sys","Ttables","id",1); > constraints.emptySet(T15); > T15:bat[:oid,:int]{notnil=true,rows=0:lng,bid=307} := nil; > T16:bat[:oid,:int]{notnil=true,rows=0:lng,bid=309} := sql.bind("sys","Ttables","id",2); > constraints.emptySet(T16); > T16:bat[:oid,:int]{notnil=true,rows=0:lng,bid=309} := nil; > T20:bat[:oid,:sht]{notnil=true,rows=30:lng,bid=71} := sql.bind("sys","Ttables","type",0); > T39:bat[:oid,:str]{notnil=true,rows=0:lng,bid=114} := sql.bind("tmp","Ttables","name",0); > T42:bat[:oid,:str]{notnil=true,rows=30:lng,bid=68} := sql.bind("sys","Ttables","name",0); > T47:bat[:oid,:int]{notnil=true,rows=0:lng,bid=115} := sql.bind("tmp","Ttables","schemaTid",0); > T50:bat[:oid,:int]{notnil=true,rows=30:lng,bid=69} := sql.bind("sys","Ttables","schemaTid",0); > T55:bat[:oid,:str]{notnil=true,rows=0:lng,bid=116} := sql.bind("tmp","Ttables","query",0); > T58:bat[:oid,:str]{notnil=true,rows=30:lng,bid=70} := sql.bind("sys","Ttables","query",0); > T64:bat[:oid,:bit]{notnil=true,rows=0:lng,bid=118} := sql.bind("tmp","Ttables","system",0); > T67:bat[:oid,:bit]{notnil=true,rows=30:lng,bid=72} := sql.bind("sys","Ttables","system",0); > T72:bat[:oid,:sht]{notnil=true,rows=0:lng,bid=119} := sql.bind("tmp","Ttables","commitTaction",0); > T75:bat[:oid,:sht]{notnil=true,rows=30:lng,bid=73} := sql.bind("sys","Ttables","commitTaction",0); > T8{rows=0:lng} := algebra.markT(T1,0@0); > T9{rows=0:lng} := bat.reverse(T8); > T8{rows=0:lng} := nil; > T12{rows=0:lng} := algebra.join(T9,T10); > T10:bat[:oid,:sht]{notnil=true,rows=0:lng,bid=117} := nil; > T18{rows=30:lng} := algebra.markT(T13,0@0); > T19{rows=30:lng} := bat.reverse(T18); > T18{rows=30:lng} := nil; > T21{rows=30:lng} := algebra.join(T19,T20); > T20:bat[:oid,:sht]{notnil=true,rows=30:lng,bid=71} := nil; > T22{rows=31:lng} := bat.setWriteMode(T21); > T21{rows=30:lng} := nil; > bat.append(T22,T12,true); > T12{rows=0:lng} := nil; > T26{rows=31:lng} := algebra.uselect(T22,nil:sht,2:sht,false,false); > T29{rows=31:lng} := algebra.markT(T26,0@0); > T26{rows=31:lng} := nil; > T30{rows=31:lng} := bat.reverse(T29); > T29{rows=31:lng} := nil; > T31{rows=0:lng} := algebra.join(T9,T1); > T1:bat[:oid,:int]{notnil=true,rows=0:lng,bid=113} := nil; > T32{rows=30:lng} := algebra.join(T19,T13); > T13:bat[:oid,:int]{notnil=true,rows=30:lng,bid=67} := nil; > T33{rows=31:lng} := bat.setWriteMode(T32); > T32{rows=30:lng} := nil; > bat.append(T33,T31,true); > T31{rows=0:lng} := nil; > T35{rows=31:lng} := algebra.join(T30,T33); > T36{rows=31:lng} := algebra.markT(T35,0@0); > T35{rows=31:lng} := nil; > T37{rows=31:lng} := bat.reverse(T36); > T36{rows=31:lng} := nil; > T38{rows=31:lng} := algebra.joinPath(T37,T30,T33); > T33{rows=31:lng} := nil; > T41{rows=0:lng} := algebra.join(T9,T39); > T39:bat[:oid,:str]{notnil=true,rows=0:lng,bid=114} := nil; > T43{rows=30:lng} := algebra.join(T19,T42); > T42:bat[:oid,:str]{notnil=true,rows=30:lng,bid=68} := nil; > T44{rows=31:lng} := bat.setWriteMode(T43); > T43{rows=30:lng} := nil; > bat.append(T44,T41,true); > T41{rows=0:lng} := nil; > T46{rows=31:lng} := algebra.joinPath(T37,T30,T44); > T44{rows=31:lng} := nil; > T49{rows=0:lng} := algebra.join(T9,T47); > T47:bat[:oid,:int]{notnil=true,rows=0:lng,bid=115} := nil; > T51{rows=30:lng} := algebra.join(T19,T50); > T50:bat[:oid,:int]{notnil=true,rows=30:lng,bid=69} := nil; > T52{rows=31:lng} := bat.setWriteMode(T51); > T51{rows=30:lng} := nil; > bat.append(T52,T49,true); > T49{rows=0:lng} := nil; > T54{rows=31:lng} := algebra.joinPath(T37,T30,T52); > T52{rows=31:lng} := nil; > T57{rows=0:lng} := algebra.join(T9,T55); > T55:bat[:oid,:str]{notnil=true,rows=0:lng,bid=116} := nil; > T59{rows=30:lng} := algebra.join(T19,T58); > T58:bat[:oid,:str]{notnil=true,rows=30:lng,bid=70} := nil; > T60{rows=31:lng} := bat.setWriteMode(T59); > T59{rows=30:lng} := nil; > bat.append(T60,T57,true); > T57{rows=0:lng} := nil; > T62{rows=31:lng} := algebra.joinPath(T37,T30,T60); > T60{rows=31:lng} := nil; > T63{rows=31:lng} := algebra.joinPath(T37,T30,T22); > T22{rows=31:lng} := nil; > T66{rows=0:lng} := algebra.join(T9,T64); > T64:bat[:oid,:bit]{notnil=true,rows=0:lng,bid=118} := nil; > T68{rows=30:lng} := algebra.join(T19,T67); > T67:bat[:oid,:bit]{notnil=true,rows=30:lng,bid=72} := nil; > T69{rows=31:lng} := bat.setWriteMode(T68); > T68{rows=30:lng} := nil; > bat.append(T69,T66,true); > T66{rows=0:lng} := nil; > T71{rows=31:lng} := algebra.joinPath(T37,T30,T69); > T69{rows=31:lng} := nil; > T74{rows=0:lng} := algebra.join(T9,T72); > T72:bat[:oid,:sht]{notnil=true,rows=0:lng,bid=119} := nil; > T76{rows=30:lng} := algebra.join(T19,T75); > T75:bat[:oid,:sht]{notnil=true,rows=30:lng,bid=73} := nil; > T77{rows=31:lng} := bat.setWriteMode(T76); > T76{rows=30:lng} := nil; > bat.append(T77,T74,true); > T74{rows=0:lng} := nil; > T79{rows=31:lng} := algebra.joinPath(T37,T30,T77); > T77{rows=31:lng} := nil; > T81{rows=0:lng} := algebra.project(T9,1); > T9{rows=0:lng} := nil; > T83{rows=30:lng} := algebra.project(T19,0); > T19{rows=30:lng} := nil; > T84{rows=31:lng} := bat.setWriteMode(T83); > T83{rows=30:lng} := nil; > bat.append(T84,T81,true); > T81{rows=0:lng} := nil; > T86{rows=31:lng} := algebra.joinPath(T37,T30,T84); > T37{rows=31:lng} := nil; > T30{rows=31:lng} := nil; > T84{rows=31:lng} := nil; > T87 := sql.resultSet(8,1,T38); > sql.rsColumn(T87,"sys.tables","id","int",32,0,T38); > T38{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","name","varchar",1024,0,T46); > T46{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","schemaTid","int",32,0,T54); > T54{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","query","varchar",2048,0,T62); > T62{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","type","smallint",16,0,T63); > T63{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","system","boolean",1,0,T71); > T71{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","commitTaction","smallint",16,0,T79); > T79{rows=31:lng} := nil; > sql.rsColumn(T87,"sys.tables","temporary","tinyint",8,0,T86); > T86{rows=31:lng} := nil; > sql.exportResult(T87,""); > end qry; > compiler.MALtoC("user","qry","sql01"); -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] BAT sizes
by Stefan Manegold 07 Oct '07

07 Oct '07

[felt free to cc the monetdb-developers list as more people might be interested or want to contribute] Henning, are you just "concerned" or are you having concrete problems with the bat sizes? In cany case, to give any reasonable answer we'd need to know more about the details. In particular how large is the BAT your talking about. I.e., with "b" being your BAT and "c := b.copy()", please check & report b.count(); b.info().reverse().like("batBuns").like("size").print(); b.batsize(); b.batdsksize(); c.count(); c.info().reverse().like("batBuns").like("size").print(); c.batsize(); c.batdsksize(); Stefan On Fri, Oct 05, 2007 at 01:47:01PM +0200, Henning Rode wrote: > hej stefan, > > thanks for the answer. so in conclusion, the over-allocation of memory > is quite normal, and nothing to worry about. > > i was more surprised that the copied BAT still has this considerable > over-allocation of memory, though it exactly knows how many entries it > needs to hold. > > groeten -henning -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [Monetdb-developers] [Monetdb-bugs] JDBC: Negative Integer in Prepared Statement
by Stefan Manegold 06 Oct '07

06 Oct '07

Hi Jan, thanks for using MonetDB, and sorry for the inconvenience. We'll investigate the reported problem shortly. However, the monetdb-bugs mailing list is actually not meant to be used directly (admittedly our fault, in case we didn't communicate this clearly and/or haven't prevented direct postings, yet). The monetdb-bugs mailing list is mainly meant notify developers of newly files bug reports and their follow-up comments. Hence, could you please file your bug using the SF bug tracker at http://sourceforge.net/tracker/?atid=482468&group_id=56967&func=browse ? Thanks, Stefan On Sat, Oct 06, 2007 at 08:08:57PM +0200, Schaffner, Jan wrote: > Hi, > > When using prepared statements in Java, I am unable to pass negative > integer values to the database: > > String APPEND_JOURNAL = "INSERT INTO journal VALUES (?, ?);"; > st = con.prepareStatement(APPEND_JOURNAL); > st.setInt(1, fromAcc); > st.setInt(2, -amount); > st.executeUpdate(); > > The table "journal" has two columns: account (int) and amount (bigint). > > The insert statement executes without error, but inserts the positive > value (+amount, cf. line 4). The debugger, in contrast, says something > like "exec 3(200, -30)". Note that is has a minus in front of the > second parameter. > > Without prepared statements, however, inserting negative values works > perfactly fine. > > Thanks, > Jan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/tests/BugTracker/Tests noroot.SF-1452237.XQUERY.bat, XQuery_0-20, NONE, 1.1.2.1
by Sjoerd Mullender 05 Oct '07

05 Oct '07

Stefan Manegold wrote: > Update of /cvsroot/monetdb/pathfinder/tests/BugTracker/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv23002 > > Added Files: > Tag: XQuery_0-20 > noroot.SF-1452237.XQUERY.bat > Log Message: > > added Windows version --- still needs to be tested/checked! > > > --- NEW FILE: noroot.SF-1452237.XQUERY.bat --- > @XQUERY_CLIENT@ -s1 -f xml-noroot %...% -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB4/tests/BugDay_2005-10-06_4.8/Tests multiple_servers.SF-914113.MIL.bat, MonetDB_4-20, NONE, 1.1.2.1
by Sjoerd Mullender 05 Oct '07

05 Oct '07

Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB4/tests/BugDay_2005-10-06_4.8/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15739/MonetDB4/tests/BugDay_2005-10-06_4.8/Tests > > Added Files: > Tag: MonetDB_4-20 > multiple_servers.SF-914113.MIL.bat > Log Message: > > added Windows version --- still needs to be tested/checked! > > > --- NEW FILE: multiple_servers.SF-914113.MIL.bat --- > @call @MSERVER@ --dbname=@TSTDB@ I can tell you right away that this won't fly. Either it has to be a .in file so that the @...@ stuff gets expanded, or if they are supposed to be environment variables, they should be %...%. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah nexi_generate_mil.c, XQuery_0-18, 1.33.2.3, 1.33.2.4
by Sjoerd Mullender 05 Oct '07

05 Oct '07

Djoerd Hiemstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1427 > > Modified Files: > Tag: XQuery_0-18 > nexi_generate_mil.c > Log Message: > The ir-model="LM" option was translated to "LMs" in the algebra. You do realize that the XQuery_0-18 branch is the *old* stable branch and that we're now using XQuery_0-20? > > Index: nexi_generate_mil.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/nexi_generate_mil.c,v > retrieving revision 1.33.2.3 > retrieving revision 1.33.2.4 > diff -u -d -r1.33.2.3 -r1.33.2.4 > --- nexi_generate_mil.c 15 Jun 2007 08:53:00 -0000 1.33.2.3 > +++ nexi_generate_mil.c 4 Oct 2007 13:11:35 -0000 1.33.2.4 > @@ -711,7 +711,7 @@ > case MODEL_LM : > > // MILPRINTF(MILOUT, "R%d := R%d.p_containing_q_LM(terms, qenv);\n", com_num, com_nr_left); > - MILPRINTF(MILOUT, "R%d := R%d.p_containing_q_LMs%s(terms%s,qenv);\n", com_num, com_nr_left, parserCtx->ffPfx, parserCtx->flastPfx); > + MILPRINTF(MILOUT, "R%d := R%d.p_containing_q_LM%s(terms%s,qenv);\n", com_num, com_nr_left, parserCtx->ffPfx, parserCtx->flastPfx); > > break; > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

5 5

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/tests/gdkTests/Tests casts.mal, MonetDB_5-0, 1.4, 1.4.4.1
by Stefan Manegold 02 Oct '07

02 Oct '07

On Sat, Sep 29, 2007 at 09:07:39PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/tests/gdkTests/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv20428/Tests > > Modified Files: > Tag: MonetDB_5-0 > casts.mal > Log Message: > Better handle to detect the differences. > No need to roll forward. Since this checkin was to the abondoned MonetDB_5-0 branch, there will not even be any attempts to propagate it. For info: Since Sep 18 2007, we have and use release branch MonetDB_5-2 for MonetDB5; see also http://sourceforge.net/mailarchive/forum.php?thread_name=20070918223806.GA2… Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Mserver is aborting
by Riham Abdel Kader 02 Oct '07

02 Oct '07

Forgot the attachments! --------------------------------------------------------------------------- Hi all, I checked out today the MonetDB/XQuery code and Mserver is crashing whenever I try to execute MIL statements using the mclient command. More details follow. I first compile an XQuery to MIL statements using the command "pf -A" Then I execute the MIL statements with "mclient -t -l" Whenever I do so, I get the correct results but Mserver aborts after giving the following error message: Mserver: monet_tbl.mx:1743: TBL_M_decref: Assertion `p' failed. Aborted I have attached the XQuery file and the generated MIL file. Thank you and best regards, Riham -- Address: Database Group, Dept of EEMCS, University of Twente, POBox 217, 7500 AE Enschede, The Netherlands Phone: +31 (0)53 489 4628 Fax: +31 (0)53 489 2927 Email: r.abdelkader(a)ewi.utwente.nl for $p in doc("doc_100000_10ab_1000pr.xml")//a/p, $r in doc("doc_100000_10ab_1000pr.xml")//b/r where $p/text() = $r/text() return $r module ("pathfinder"); module ("mkey"); var time_load; var time_query; var time_print; var a0000; var a0001; var a0002; var a0003; var a0004; var a0005; var a0006; var a0007; var a0008; var a0009; var a0010; var a0011; var a0012; var a0013; var a0014; var unused; var ws; unused := nil; # trace structure var trace_outer; var trace_inner; var trace_iter; var trace_msg; var trace_item; var trace_type; var trace_rel; # Begin rule: "Query: serialize(FragList, Rel)" a0001 := CATCH ({ ws := ws_create (); # Begin rule: "Rel: lit_tbl" # a0000: iter(nat) a0000 := new (void, oid).seqbase (0@0); a0000.append (1@0); a0000.access (BAT_READ); # a0001: item(str) a0001 := new (void, str).seqbase (0@0); a0001.append ("doc_100000_10ab_1000pr.xml"); a0001.access (BAT_READ); # End rule: "Rel: lit_tbl" # Begin rule: "FragRel: doc_tbl(Rel)" # a0000: iter(nat) # a0005: item(pre) # a0004: item(pfrag) a0003 := doc_tbl (ws, a0001); a0005 := a0003.fetch (1); a0004 := a0003.fetch (2); ws := a0003.fetch (0); a0003 := unused; a0002 := unused; a0001 := unused; # End rule: "FragRel: doc_tbl(Rel)" # Begin rule: "Rel: roots(FragRel)" # a0000: iter(nat) # a0005: item(pre) # a0004: item(pfrag) # End rule: "Rel: roots(FragRel)" # Begin rule: "Frag: fragment(FragRel)" # End rule: "Frag: fragment(FragRel)" # Begin rule: "FragList: empty_frag" # End rule: "FragList: empty_frag" # Begin rule: "FragList: frag_union(FragList, Frag)" # End rule: "FragList: frag_union(FragList, Frag)" # Begin rule: "Rel: llscj_desc(FragList, Rel)" a0001 := loop_lifted_descendant_step_with_nsloc_test (a0000, a0005, a0004, ws, 2, "", "b"); a0002 := a0001.fetch (1); a0003 := a0001.fetch (2).materialize (a0002); # a0002: item(pre) # a0003: item(pfrag) a0006 := a0001.fetch (0).materialize (a0001.fetch (1)); # a0006: iter(nat) a0001 := unused; # End rule: "Rel: llscj_desc(FragList, Rel)" # Begin rule: "Rel: llscj_child(FragList, Rel)" a0001 := loop_lifted_child_step_with_nsloc_test (a0006, a0002, a0003, ws, 3, "", "r"); a0007 := a0001.fetch (1); a0008 := a0001.fetch (2).materialize (a0007); # a0007: item(pre) # a0008: item(pfrag) # a0009: iter(nat) a0001 := unused; a0002 := unused; a0003 := unused; a0006 := unused; # End rule: "Rel: llscj_child(FragList, Rel)" # Begin rule: "Rel: project(Rel)" # a0008: item(pfrag) # a0007: item(pre) a0009 := unused; # End rule: "Rel: project(Rel)" # Begin rule: "Rel: number(Rel)" # a0008: item(pfrag) # a0007: item(pre) a0001 := a0007.mark (1@0); # a0001: iter1(nat) # End rule: "Rel: number(Rel)" # Begin rule: "Rel: project(Rel)" # a0008: item(pfrag) # a0007: item(pre) # a0001: iter(nat) # End rule: "Rel: project(Rel)" # Begin rule: "Rel: llscj_child(FragList, Rel)" a0002 := loop_lifted_child_step_with_kind_test (a0001, a0007, a0008, ws, 3, TEXT); a0003 := a0002.fetch (1); a0006 := a0002.fetch (2); # a0003: item(pre) # a0006: item(pfrag) a0009 := a0002.fetch (0).materialize (a0002.fetch (1)); # a0009: iter(nat) a0002 := unused; # End rule: "Rel: llscj_child(FragList, Rel)" # Begin rule: "Rel: doc_access(FragList, Rel)" # a0003: item(pre) # a0006: item(pfrag) # a0009: iter(nat) a0002 := mposjoin (mposjoin (a0003, a0006, ws.fetch (PRE_PROP)), mposjoin (a0003, a0006, ws.fetch (PRE_CONT)), ws.fetch (PROP_TEXT)); # a0002: item1(str) # End rule: "Rel: doc_access(FragList, Rel)" # Begin rule: "Rel: project(Rel)" # a0009: iter(nat) # a0002: item1(str) a0003 := unused; a0006 := unused; # End rule: "Rel: project(Rel)" # Begin rule: "Rel: project(Rel)" # a0000: iter(nat) # a0004: item1(pfrag) # a0005: item1(pre) # End rule: "Rel: project(Rel)" # Begin rule: "Rel: llscj_desc(FragList, Rel)" a0003 := loop_lifted_descendant_step_with_nsloc_test (a0000, a0005, a0004, ws, 2, "", "a"); a0006 := a0003.fetch (1); a0010 := a0003.fetch (2).materialize (a0006); # a0006: item1(pre) # a0010: item1(pfrag) a0011 := a0003.fetch (0).materialize (a0003.fetch (1)); # a0011: iter(nat) a0003 := unused; a0000 := unused; a0004 := unused; a0005 := unused; # End rule: "Rel: llscj_desc(FragList, Rel)" # Begin rule: "Rel: llscj_child(FragList, Rel)" a0000 := loop_lifted_child_step_with_nsloc_test (a0011, a0006, a0010, ws, 3, "", "p"); a0003 := a0000.fetch (1); a0004 := a0000.fetch (2).materialize (a0003); # a0003: item1(pre) # a0004: item1(pfrag) # a0005: iter(nat) a0000 := unused; a0006 := unused; a0010 := unused; a0011 := unused; # End rule: "Rel: llscj_child(FragList, Rel)" # Begin rule: "Rel: project(Rel)" # a0004: item1(pfrag) # a0003: item1(pre) a0005 := unused; # End rule: "Rel: project(Rel)" # Begin rule: "Rel: number(Rel)" # a0004: item1(pfrag) # a0003: item1(pre) a0000 := a0003.mark (1@0); # a0000: iter2(nat) # End rule: "Rel: number(Rel)" # Begin rule: "Rel: llscj_child(FragList, Rel)" a0005 := loop_lifted_child_step_with_kind_test (a0000, a0003, a0004, ws, 3, TEXT); a0006 := a0005.fetch (1); a0010 := a0005.fetch (2); # a0006: item1(pre) # a0010: item1(pfrag) a0011 := a0005.fetch (0).materialize (a0005.fetch (1)); # a0011: iter2(nat) a0005 := unused; # End rule: "Rel: llscj_child(FragList, Rel)" # Begin rule: "Rel: doc_access(FragList, Rel)" # a0006: item1(pre) # a0010: item1(pfrag) # a0011: iter2(nat) a0005 := mposjoin (mposjoin (a0006, a0010, ws.fetch (PRE_PROP)), mposjoin (a0006, a0010, ws.fetch (PRE_CONT)), ws.fetch (PROP_TEXT)); # a0005: item(str) # End rule: "Rel: doc_access(FragList, Rel)" # Begin rule: "Rel: project(Rel)" # a0011: iter2(nat) # a0005: item(str) a0006 := unused; a0010 := unused; # End rule: "Rel: project(Rel)" # Begin rule: "Rel: unq2_thetajoin(Rel, Rel)" # a0010: iter2(nat) # a0012: iter(nat) a0006 := htordered_unique_thetajoin (EQ, a0011.reverse ().leftjoin (a0005), a0009.reverse ().leftjoin (a0002), nil, nil); a0010 := a0006.mark (0(a)0).reverse (); a0010 := a0010.assert_order (); a0012 := a0006.reverse ().mark (0(a)0).reverse (); a0006 := unused; a0011 := unused; a0005 := unused; a0009 := unused; a0002 := unused; # End rule: "Rel: unq2_thetajoin(Rel, Rel)" # Begin rule: "Rel: project(Rel)" # a0010: iter2(nat) # a0012: iter(nat) # End rule: "Rel: project(Rel)" # Begin rule: "Rel: leftjoin(Rel, Rel)" a0002 := a0012.leftjoin (a0001.reverse ()); a0005 := a0002.mark (0(a)0).reverse (); a0006 := a0002.reverse ().mark (0(a)0).reverse (); a0002 := unused; a0002 := a0005.leftjoin (a0010); a0002 := a0002.reverse ().mark (0(a)0).reverse (); # a0002: iter2(nat) # a0009: iter(nat) a0011 := a0006.leftjoin (a0008); a0011 := a0011.reverse ().mark (0(a)0).reverse (); # a0011: item(pfrag) a0013 := a0006.leftjoin (a0007); a0013 := a0013.reverse ().mark (0(a)0).reverse (); # a0013: item(pre) # a0014: iter1(nat) a0005 := unused; a0006 := unused; a0010 := unused; a0012 := unused; a0008 := unused; a0007 := unused; a0001 := unused; # End rule: "Rel: leftjoin(Rel, Rel)" # Begin rule: "Rel: project(Rel)" # a0011: item(pfrag) # a0013: item(pre) # a0002: iter2(nat) a0009 := unused; a0014 := unused; # End rule: "Rel: project(Rel)" # Begin rule: "Rel: project(Rel)" # a0004: item1(pfrag) # a0003: item1(pre) # a0000: iter(nat) # End rule: "Rel: project(Rel)" # Begin rule: "Rel: leftjoin(Rel, Rel)" a0001 := a0000.leftjoin (a0002.reverse ()); a0005 := a0001.mark (0(a)0).reverse (); a0006 := a0001.reverse ().mark (0(a)0).reverse (); a0001 := unused; a0001 := a0005.leftjoin (a0004); a0001 := a0001.reverse ().mark (0(a)0).reverse (); # a0001: item1(pfrag) a0007 := a0005.leftjoin (a0003); a0007 := a0007.reverse ().mark (0(a)0).reverse (); # a0007: item1(pre) # a0008: iter(nat) a0009 := a0006.leftjoin (a0011); a0009 := a0009.reverse ().mark (0(a)0).reverse (); # a0009: item(pfrag) a0010 := a0006.leftjoin (a0013); a0010 := a0010.reverse ().mark (0(a)0).reverse (); # a0010: item(pre) # a0012: iter2(nat) a0005 := unused; a0006 := unused; a0004 := unused; a0003 := unused; a0000 := unused; a0011 := unused; a0013 := unused; a0002 := unused; # End rule: "Rel: leftjoin(Rel, Rel)" # Begin rule: "Rel: project(Rel)" # a0009: item(pfrag) # a0010: item(pre) # a0001: item1(pfrag) # a0007: item1(pre) a0008 := unused; a0012 := unused; # End rule: "Rel: project(Rel)" # Begin rule: "Rel: refine_sort(Rel)" a0000 := a0001; a0000 := a0000.CTrefine (a0007); a0000 := a0000.CTrefine (a0009); a0000 := a0000.CTrefine (a0010); a0000 := a0000.mark (0(a)0).reverse (); a0002 := a0000.leftjoin (a0009); a0002 := a0002.reverse ().mark (0(a)0).reverse (); # a0002: item(pfrag) a0003 := a0000.leftjoin (a0010); a0003 := a0003.reverse ().mark (0(a)0).reverse (); # a0003: item(pre) # a0001: item1(pfrag) # a0007: item1(pre) a0000 := unused; a0009 := unused; a0010 := unused; # End rule: "Rel: refine_sort(Rel)" # Begin rule: "Rel: number(Rel)" # a0002: item(pfrag) # a0003: item(pre) # a0001: item1(pfrag) # a0007: item1(pre) # a0000: pos(nat) # End rule: "Rel: number(Rel)" # Begin rule: "Rel: project(Rel)" # a0002: item(pfrag) # a0003: item(pre) # a0000: pos(nat) a0001 := unused; a0007 := unused; # End rule: "Rel: project(Rel)" a0004 := new (lng, void).key (true).reverse ().seqbase (0@0); a0005 := new (dbl, void).key (true).reverse ().seqbase (0@0); a0006 := new (str, void).key (true).reverse ().seqbase (0@0); a0007 := a0003; a0008 := a0002.set_kind (ELEM); a0007 := a0007.reverse ().mark (0(a)0).reverse (); a0008 := a0008.reverse ().mark (0(a)0).reverse (); print_result ("xml", ws, a0007, a0008, a0004, a0005, a0005, a0006); a0004 := unused; a0005 := unused; a0006 := unused; a0007 := unused; a0008 := unused; a0001 := unused; }); if (not(isnil(ws))) { ws_destroy (ws); } else { } if (not(isnil(a0001))) { ERROR (a0001); } else { } a0001 := unused; a0002 := unused; a0003 := unused; a0000 := unused; # End rule: "Query: serialize(FragList, Rel)"

2 2

[Monetdb-developers] Mserver is aborting
by Riham Abdel Kader 02 Oct '07

02 Oct '07

Hi all, I checked out today the MonetDB/XQuery code and Mserver is crashing whenever I try to execute MIL statements using the mclient command. More details follow. I first compile an XQuery to MIL statements using the command "pf -A" Then I execute the MIL statements with "mclient -t -l" Whenever I do so, I get the correct results but Mserver aborts after giving the following error message: Mserver: monet_tbl.mx:1743: TBL_M_decref: Assertion `p' failed. Aborted I have attached the XQuery file and the generated MIL file. Thank you and best regards, Riham -- Address: Database Group, Dept of EEMCS, University of Twente, POBox 217, 7500 AE Enschede, The Netherlands Phone: +31 (0)53 489 4628 Fax: +31 (0)53 489 2927 Email: r.abdelkader(a)ewi.utwente.nl

1 0

[Monetdb-developers] sourceforge outage
by Sjoerd Mullender 01 Oct '07

01 Oct '07

To forestall any questions, the Sourceforge stite status has this today: ( 2007-09-30 14:42:31 - Project CVS Service ) As of 2007-09-29 at 10:00PM Pacific, we have a CVS outage due to unplanned maintenance. This affects developer CVS access (including ViewVC) for projects starting with the letters m, n and q. Site status will be updated when the system becomes available again. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.198, 1.199
by Stefan Manegold 28 Sep '07

28 Sep '07

On Fri, Sep 28, 2007 at 08:10:06PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18777 > > Modified Files: > mal_interpreter.mx > Log Message: > Avoid complaints by compilers by switching to the preprocessor. Martin, before you wonder why the SLOW ans FAST cases don't show any differences any more --- neither performance nor in speed ---: Since your code defines both macros SLOW (as 1) and FAST (as 0), directives #ifdef @1 with @1 being substituted with either SLOW or FAST by Mx, always evaluate to true, regardsless of whether @1 is SLOW (== 1) or FAST (== 0). As initially suggested by both Sjoerd and me, you might want to use #if @1 directives, instead, to re-establish your (supposely) intended behaviour. Stefan > Index: mal_interpreter.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > retrieving revision 1.198 > retrieving revision 1.199 > diff -u -d -r1.198 -r1.199 > --- mal_interpreter.mx 28 Sep 2007 16:00:57 -0000 1.198 > +++ mal_interpreter.mx 28 Sep 2007 20:10:04 -0000 1.199 > @@ -743,13 +743,15 @@ > case ENDsymbol: > if( getInstrPtr(mb,0)->token == FACTORYsymbol) > ret= shutdownFactory(mb); > - if( @1 && oldtimer) > +#ifdef @1 > + if( oldtimer) > cntxt->timer= oldtimer; > +#endif > if( pcicaller && garbageControl(getInstrPtr(mb,0)) ) > garbageCollector(mb, stk, TRUE); > - if(@1){ > - @:endProfile@ > - } > +#ifdef @1 > + @:endProfile@ > +#endif > stkpc= mb->stop; > continue; > default: > @@ -759,9 +761,9 @@ > } > ret = createScriptException(mb, stkpc,MAL, > NULL, "unkown operation"); > - if( @1){ > - @:endProfile@ > - } > +#ifdef @1 > + @:endProfile@ > +#endif > stkpc= mb->stop; > continue; > } > @@ -1027,7 +1029,8 @@ > } > } > /* Provide debugging support */ > - if( @1 && (GDKdebug & 10) ){ > +#ifdef @1 > + if( GDKdebug & 10 ){ > BAT *b; > str oldmsg =0; > > @@ -1061,6 +1064,7 @@ > } > } > } > +#endif > @- > @= commandcall > { > @@ -1599,7 +1603,8 @@ > #endif > > @= timingHndlr > -if( @1 && cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ > +#ifdef @1 > +if( cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ > if( cntxt->flags & timerFlag) > stream_printf(cntxt->fdout,"[%6d usec ",GDKusec()-cntxt->timer); > #ifdef HAVE_SYS_RESOURCE_H > @@ -1641,6 +1646,7 @@ > if( cntxt->flags & timerFlag) > cntxt->timer = GDKusec(); > } > +#endif > @- > The bigfoot memory tracker keeps track on the space > occupancy of BATs. It ignores for the time being the heaps. > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.197, 1.198
by Sjoerd Mullender 28 Sep '07

28 Sep '07

On 2007-09-28 18:01, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5547 > > Modified Files: > mal_interpreter.mx > Log Message: > Split the interpreter in two versions. A FAST and SLOW version. > If there is no request for debugging or performance measurement > outstanding, we can go the fast path. However, given the fact > that at any time we may enter the debugger, a jump to the slow > line is always possible. > The code relies on compiler optimization to get rid of blocks > with constant arguments. Unclear if all compilers are that smart. This may fail for some compilers. Some compilers warn about constant expressions in conditionals, and we turn warnings into errors. Be prepared to fix this if there are compilers that have a problem with this usage. One way to solve the problem is to use the preprocessor: #if @1 ... #endif instead of if (@1) { ... } > > Index: mal_interpreter.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > retrieving revision 1.197 > retrieving revision 1.198 > diff -u -d -r1.197 -r1.198 > --- mal_interpreter.mx 24 Sep 2007 08:24:22 -0000 1.197 > +++ mal_interpreter.mx 28 Sep 2007 16:00:57 -0000 1.198 > @@ -70,6 +70,9 @@ > #include "mal_debugger.h" /* for mdbStep() */ > #include "mal_type.h" > > +#define SLOW 1 > +#define FAST 0 > + > static str runMALsequence( Client cntxt, MalBlkPtr mb, int startpc, > int stoppc, MalStkPtr stk, MalStkPtr env, InstrPtr pcicaller); > static str setDynamicType(MalBlkPtr mb, VarPtr v, int tpe, int pc); > @@ -421,8 +424,27 @@ > stkpc = startpc; > exceptionVar = exceptionPC = -1; > (void)exceptionPC; /* TODO maybe we should use this variable somewhere*/ > +@- > +From this point onwards we should differentiate fast processing > +against monitored processing. Fast processing is possible if there is > +no call to the debugger statically/dynamically set. Same holds for > +performance control statements. > +The code currently does not statically checks the mode change. > +Preferrably we should introduce a itrace flag PROFILE > +We rely on optimizing compilers to remove the redundant code. > +@c > + if( malProfileMode == 0 && cntxt->itrace==0){ > + while(stkpc < mb->stop && stkpc != stoppc ){ > + pci = getInstrPtr(mb,stkpc); > + if( malProfileMode + cntxt->itrace) > + goto workslow; > + @:MALinterpret(FAST)@ > + @:MALflowofcontrol(FAST)@ > + } > + } else > while(stkpc < mb->stop && stkpc != stoppc ){ > pci = getInstrPtr(mb,stkpc); > +workslow: > if( cntxt->itrace ) { > lng t=0; > if( stk->cmd== 0) stk->cmd= cntxt->itrace; > @@ -445,8 +467,8 @@ > } > > @:beginProfile@ > - @:MALinterpret@ > - @:MALflowofcontrol@ > + @:MALinterpret(SLOW)@ > + @:MALflowofcontrol(SLOW)@ > @:endProfile@ > } > @:MALwrapup@ > @@ -639,11 +661,11 @@ > limited. > @c > switch( pci->token){ > - case ASSIGNsymbol: @:assignStmt@ break; > - case PATcall: @:patterncall@ break; > - case CMDcall: @:commandcall@ break; > - case FACcall: @:factorycall@ break; > - case FCNcall: @:functioncall@ break; > + case ASSIGNsymbol: @:assignStmt(SLOW)@ break; > + case PATcall: @:patterncall(SLOW)@ break; > + case CMDcall: @:commandcall(SLOW)@ break; > + case FACcall: @:factorycall(SLOW)@ break; > + case FCNcall: @:functioncall(SLOW)@ break; > } > /* we don't allow sequential flow control here */ > dropMALflowStep(flow,stkpc,n,pending,mb,stk); > @@ -710,22 +732,24 @@ > @= MALinterpret > ret = 0; > switch( pci->token){ > - case ASSIGNsymbol: @:assignStmt@ break; > - case PATcall: @:patterncall@ break; > - case CMDcall: @:commandcall@ break; > - case FACcall: @:factorycall@ break; > - case FCNcall: @:functioncall@ break; > + case ASSIGNsymbol: @:assignStmt(@1)@ break; > + case PATcall: @:patterncall(@1)@ break; > + case CMDcall: @:commandcall(@1)@ break; > + case FACcall: @:factorycall(@1)@ break; > + case FCNcall: @:functioncall(@1)@ break; > case NOOPsymbol: > case REMsymbol: > break; > case ENDsymbol: > if( getInstrPtr(mb,0)->token == FACTORYsymbol) > ret= shutdownFactory(mb); > - if( oldtimer) > + if( @1 && oldtimer) > cntxt->timer= oldtimer; > if( pcicaller && garbageControl(getInstrPtr(mb,0)) ) > garbageCollector(mb, stk, TRUE); > - @:endProfile@ > + if(@1){ > + @:endProfile@ > + } > stkpc= mb->stop; > continue; > default: > @@ -735,7 +759,9 @@ > } > ret = createScriptException(mb, stkpc,MAL, > NULL, "unkown operation"); > - @:endProfile@ > + if( @1){ > + @:endProfile@ > + } > stkpc= mb->stop; > continue; > } > @@ -886,9 +912,10 @@ > if( lhs->vtype == TYPE_bat) > BBPincref(lhs->val.br.id, TRUE); > } > - @:restoreTarget@ > + @:restoreTarget(@1)@ > ret = 0; > - @:exceptionHndlr@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @} > @- > @@ -1000,7 +1027,7 @@ > } > } > /* Provide debugging support */ > - if( GDKdebug & 10 ){ > + if( @1 && (GDKdebug & 10) ){ > BAT *b; > str oldmsg =0; > > @@ -1146,8 +1173,9 @@ > ret = createScriptException(mb, stkpc, MAL, NULL, > "too many arguments for command call"); > } > - @:restoreTarget@ > - @:exceptionHndlr@ > + @:restoreTarget(@1)@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @- > @= patterncall > @@ -1157,8 +1185,9 @@ > else { > @:safeTarget@ > ret = (str) (*pci->fcn)(mb,stk,pci); > - @:restoreTarget@ > - @:exceptionHndlr@ > + @:restoreTarget(@1)@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @- > MAL function calls are relatively expensive, because they have to assemble > @@ -1171,8 +1200,9 @@ > stk->pcup = stkpc; > @:safeTarget@ > ret= runMAL(cntxt,pci->blk,1,mb,stk,pci); > - @:restoreTarget@ > - @:exceptionHndlr@ > + @:restoreTarget(@1)@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @- > Factory calls are more involved. At this stage it is a synchrononous > @@ -1184,7 +1214,8 @@ > "reference to MAL function missing"); > else > ret= runFactory(cntxt,pci->blk,mb,stk,pci); > - @:exceptionHndlr@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(SLOW)@ > @- > The type dispatching table in getArgValue can be removed if we > determine at compile time the address offset within a ValRecord. > @@ -1323,7 +1354,6 @@ > @{ > > @= exceptionHndlr > - @:timingHndlr@ > if( ret != MAL_SUCCEED ) { > str msg = 0, nxt; > if( stk->cmd ) { > @@ -1569,7 +1599,7 @@ > #endif > > @= timingHndlr > -if( cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ > +if( @1 && cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ > if( cntxt->flags & timerFlag) > stream_printf(cntxt->fdout,"[%6d usec ",GDKusec()-cntxt->timer); > #ifdef HAVE_SYS_RESOURCE_H > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.197, 1.198
by Stefan Manegold 28 Sep '07

28 Sep '07

On Fri, Sep 28, 2007 at 04:01:00PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5547 > > Modified Files: > mal_interpreter.mx > Log Message: > Split the interpreter in two versions. A FAST and SLOW version. > If there is no request for debugging or performance measurement > outstanding, we can go the fast path. However, given the fact > that at any time we may enter the debugger, a jump to the slow > line is always possible. > The code relies on compiler optimization to get rid of blocks > with constant arguments. Unclear if all compilers are that smart. All C-compilers (respetively their C preprocessors) are expected to respect C preprocessor directives (e.g., #if ... #endif), "even" with optimization disabled. Hence, why not use those compile-time conditionals instead "pseudo"-runtime conditionals with constant conditions? (Would also speed-up the compilation process...) Stefan > > > Index: mal_interpreter.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > retrieving revision 1.197 > retrieving revision 1.198 > diff -u -d -r1.197 -r1.198 > --- mal_interpreter.mx 24 Sep 2007 08:24:22 -0000 1.197 > +++ mal_interpreter.mx 28 Sep 2007 16:00:57 -0000 1.198 > @@ -70,6 +70,9 @@ > #include "mal_debugger.h" /* for mdbStep() */ > #include "mal_type.h" > > +#define SLOW 1 > +#define FAST 0 > + > static str runMALsequence( Client cntxt, MalBlkPtr mb, int startpc, > int stoppc, MalStkPtr stk, MalStkPtr env, InstrPtr pcicaller); > static str setDynamicType(MalBlkPtr mb, VarPtr v, int tpe, int pc); > @@ -421,8 +424,27 @@ > stkpc = startpc; > exceptionVar = exceptionPC = -1; > (void)exceptionPC; /* TODO maybe we should use this variable somewhere*/ > +@- > +From this point onwards we should differentiate fast processing > +against monitored processing. Fast processing is possible if there is > +no call to the debugger statically/dynamically set. Same holds for > +performance control statements. > +The code currently does not statically checks the mode change. > +Preferrably we should introduce a itrace flag PROFILE > +We rely on optimizing compilers to remove the redundant code. > +@c > + if( malProfileMode == 0 && cntxt->itrace==0){ > + while(stkpc < mb->stop && stkpc != stoppc ){ > + pci = getInstrPtr(mb,stkpc); > + if( malProfileMode + cntxt->itrace) > + goto workslow; > + @:MALinterpret(FAST)@ > + @:MALflowofcontrol(FAST)@ > + } > + } else > while(stkpc < mb->stop && stkpc != stoppc ){ > pci = getInstrPtr(mb,stkpc); > +workslow: > if( cntxt->itrace ) { > lng t=0; > if( stk->cmd== 0) stk->cmd= cntxt->itrace; > @@ -445,8 +467,8 @@ > } > > @:beginProfile@ > - @:MALinterpret@ > - @:MALflowofcontrol@ > + @:MALinterpret(SLOW)@ > + @:MALflowofcontrol(SLOW)@ > @:endProfile@ > } > @:MALwrapup@ > @@ -639,11 +661,11 @@ > limited. > @c > switch( pci->token){ > - case ASSIGNsymbol: @:assignStmt@ break; > - case PATcall: @:patterncall@ break; > - case CMDcall: @:commandcall@ break; > - case FACcall: @:factorycall@ break; > - case FCNcall: @:functioncall@ break; > + case ASSIGNsymbol: @:assignStmt(SLOW)@ break; > + case PATcall: @:patterncall(SLOW)@ break; > + case CMDcall: @:commandcall(SLOW)@ break; > + case FACcall: @:factorycall(SLOW)@ break; > + case FCNcall: @:functioncall(SLOW)@ break; > } > /* we don't allow sequential flow control here */ > dropMALflowStep(flow,stkpc,n,pending,mb,stk); > @@ -710,22 +732,24 @@ > @= MALinterpret > ret = 0; > switch( pci->token){ > - case ASSIGNsymbol: @:assignStmt@ break; > - case PATcall: @:patterncall@ break; > - case CMDcall: @:commandcall@ break; > - case FACcall: @:factorycall@ break; > - case FCNcall: @:functioncall@ break; > + case ASSIGNsymbol: @:assignStmt(@1)@ break; > + case PATcall: @:patterncall(@1)@ break; > + case CMDcall: @:commandcall(@1)@ break; > + case FACcall: @:factorycall(@1)@ break; > + case FCNcall: @:functioncall(@1)@ break; > case NOOPsymbol: > case REMsymbol: > break; > case ENDsymbol: > if( getInstrPtr(mb,0)->token == FACTORYsymbol) > ret= shutdownFactory(mb); > - if( oldtimer) > + if( @1 && oldtimer) > cntxt->timer= oldtimer; > if( pcicaller && garbageControl(getInstrPtr(mb,0)) ) > garbageCollector(mb, stk, TRUE); > - @:endProfile@ > + if(@1){ > + @:endProfile@ > + } > stkpc= mb->stop; > continue; > default: > @@ -735,7 +759,9 @@ > } > ret = createScriptException(mb, stkpc,MAL, > NULL, "unkown operation"); > - @:endProfile@ > + if( @1){ > + @:endProfile@ > + } > stkpc= mb->stop; > continue; > } > @@ -886,9 +912,10 @@ > if( lhs->vtype == TYPE_bat) > BBPincref(lhs->val.br.id, TRUE); > } > - @:restoreTarget@ > + @:restoreTarget(@1)@ > ret = 0; > - @:exceptionHndlr@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @} > @- > @@ -1000,7 +1027,7 @@ > } > } > /* Provide debugging support */ > - if( GDKdebug & 10 ){ > + if( @1 && (GDKdebug & 10) ){ > BAT *b; > str oldmsg =0; > > @@ -1146,8 +1173,9 @@ > ret = createScriptException(mb, stkpc, MAL, NULL, > "too many arguments for command call"); > } > - @:restoreTarget@ > - @:exceptionHndlr@ > + @:restoreTarget(@1)@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @- > @= patterncall > @@ -1157,8 +1185,9 @@ > else { > @:safeTarget@ > ret = (str) (*pci->fcn)(mb,stk,pci); > - @:restoreTarget@ > - @:exceptionHndlr@ > + @:restoreTarget(@1)@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @- > MAL function calls are relatively expensive, because they have to assemble > @@ -1171,8 +1200,9 @@ > stk->pcup = stkpc; > @:safeTarget@ > ret= runMAL(cntxt,pci->blk,1,mb,stk,pci); > - @:restoreTarget@ > - @:exceptionHndlr@ > + @:restoreTarget(@1)@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(@1)@ > } > @- > Factory calls are more involved. At this stage it is a synchrononous > @@ -1184,7 +1214,8 @@ > "reference to MAL function missing"); > else > ret= runFactory(cntxt,pci->blk,mb,stk,pci); > - @:exceptionHndlr@ > + @:exceptionHndlr(@1)@ > + @:timingHndlr(SLOW)@ > @- > The type dispatching table in getArgValue can be removed if we > determine at compile time the address offset within a ValRecord. > @@ -1323,7 +1354,6 @@ > @{ > > @= exceptionHndlr > - @:timingHndlr@ > if( ret != MAL_SUCCEED ) { > str msg = 0, nxt; > if( stk->cmd ) { > @@ -1569,7 +1599,7 @@ > #endif > > @= timingHndlr > -if( cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ > +if( @1 && cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ > if( cntxt->flags & timerFlag) > stream_printf(cntxt->fdout,"[%6d usec ",GDKusec()-cntxt->timer); > #ifdef HAVE_SYS_RESOURCE_H > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.197, 1.198
by Fabian Groffen 28 Sep '07

28 Sep '07

On 28-09-2007 18:07:48 +0200, Martin Kersten wrote: > Fabian Groffen wrote: >> On 28-09-2007 16:01:00 +0000, Martin Kersten wrote: >> >>> +#define SLOW 1 >>> +#define FAST 0 >>> >> >> How about an enum here? >> > He, ze zijn weer terug ;-) > Niet nodig is allemaal locaal duidelijk. The idea is that the compiler is able to help you (bail out). Local or not is irrellevant IMO.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.197, 1.198
by Fabian Groffen 28 Sep '07

28 Sep '07

On 28-09-2007 16:01:00 +0000, Martin Kersten wrote: > +#define SLOW 1 > +#define FAST 0 How about an enum here?

1 0

[Monetdb-developers] MonetDB4 server (Mserver) crash on solaris 10
by xxiao 27 Sep '07

27 Sep '07

hi, I built MonetDB4 server on a solaris 10 SPARC machine. When I try to execute it, the server applciation crashed and posted the following error message: >Mserver # Monet Database Server V4.18.2 # Copyright (c) 1993-2007, CWI. All rights reserved. # Compiled for sparc-sun-solaris2.10/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: BATSIGcrash: Mserver internal error (Bus error), please restart. !ERROR: (One potential cause could be that your disk might be full...) the machine configuration is: >uname -a SunOS vlsi 5.10 Generic_118833-33 sun4u sparc SUNW,Sun-Fire-480R Solaris Is the crash really due to that the disk is full? And can someone give me a hint how I can trace my problems? Thanks a lot. -- View this message in context: http://www.nabble.com/MonetDB4-server-%28Mserver%29-crash-on-solaris-10-tf4… Sent from the monetdb-developers mailing list archive at Nabble.com.

4 5

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/db2tools common.inc, XQuery_0-20, NONE, 1.1.2.1 load, XQuery_0-20, NONE, 1.1.2.1 table, XQuery_0-20, NONE, 1.1.2.1
by Stefan Manegold 24 Sep '07

24 Sep '07

To ensure a smooth finalization of our combined effort to get the upcoming release out "as fast as possible" and without any self-made hurdles, I assume that adding these new "feature(s)" in the *release branch* has been coordinated with Sjoerd, right? (In particular wrt. how/where they need to be added to the source and in particular binary packages, documentation, and which platforms they are targeted for (Windows??).) Greetings from Vienna, Stefan On Mon, Sep 24, 2007 at 09:10:33AM +0000, Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/db2tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18874 > > Added Files: > Tag: XQuery_0-20 > common.inc load table > Log Message: > > > Shell scripts to create tables, and load content for SQL-backend > > > --- NEW FILE: load --- > #!/bin/sh > > THIS=${0} > THISDIR=${0%/*} > THISFILE=${0##*/} > DB2=`which db2` > > source common.inc > > ################################################ > # FUNCTIONS > > function load_table { > #check parameter list > if [ ${#} -ne 3 ]; then > error "Usage: ${0} <file> <schema> <table>" > fi > > SCHEMA=${2} > FILE=${1} > TABLE=${3} > > # set schema to ours > ${DB2} "set current schema ${SCHEMA}" > > # create the names-table > ${DB2} -t<<EOT > LOAD FROM ${FILE} OF DEL > MODIFIED BY DELPRIORITYCHAR > REPLACE INTO ${TABLE} > NONRECOVERABLE > INDEXING MODE rebuild; > EOT > > ${DB2} "SET INTEGRITY FOR ${TABLE} IMMEDIATE CHECKED" > EFLAG=${?} > return ${EFLAG} > } > > function print_help () { > out "Pathfinder XQuery" > out "(c) Database Group, Technische Universitaet Muenchen" > out "" > out "Usage: ${THISFILE} (create|drop) <db> <schema> <file>" > out "" > out -e "\t<db>:\t\tName of the Database the content should be placed/removed." > out -e "\t<schema>:\tName of the Schema the content should be placed/removed." > out -e "\t<names_file>:\tNames file" > out -e "\t<file>:\tEncoded document" > } > > ################################################ > # MAIN > > if [ $# -ne 4 ]; then > print_help; > exit 0; > fi > > DATABASE=${1} > SCHEMA=${2} > NFILE=${3} > LFILE=${4} > > ${DB2} "connect to ${DATABASE}" > > EFLAG=${?} > out -ne "Connecting to database ${DATABASE} ... "; fail $EFLAG; > > test ${EFLAG} -ne 0 && exit 1; > > load_table "${NFILE}" "${SCHEMA}" "${TAB_NAMES}"; > EFLAG=${?}; > out -ne "Loading ${SCHEMA}.${TAB_NAMES} ... "; fail ${EFLAG}; > > load_table ${LFILE} "${SCHEMA}" "${TAB_DOC}"; > EFLAG=${?}; > out -ne "Loading ${SCHEMA}.${TAB_DOC} ... "; fail ${EFLAG}; > > ${DB2} "terminate" > > --- NEW FILE: table --- > #!/bin/sh > > THIS=${0} > THISDIR=${0%/*} > THISFILE=${0##*/} > DB2=`which db2` > > source common.inc > > ################################################ > # FUNCTIONS > > function create_names { > #check parameter list > if [ ${#} -ne 2 ]; then > error "Usage: ${0} <schema> <tabname>" > fi > > SCHEMA=${1} > TABNAME=${2} > > # set schema to ours > ${DB2} "set current schema ${SCHEMA}" > > # create the names-table > ${DB2} -t<<EOT > CREATE TABLE ${TABNAME}(nameid INT NOT NULL PRIMARY KEY, > name VARCHAR(32) NOT NULL); > EOT > EFLAG=${?} > return ${EFLAG} > } > > function create_doc { > #check parameter list > if [ ${#} -ne 3 ]; then > error "Usage: ${0} <schema> <tabname> <namesname>" > fi > > SCHEMA=${1} > TABNAME=${2} > NAMESNAME=${3} > > ${DB2} "set current schema ${SCHEMA}" > > ${DB2} -t<<EOT > CREATE TABLE ${TABNAME}(pre INT NOT NULL PRIMARY KEY, > size INT NOT NULL, > level SMALLINT NOT NULL, > kind SMALLINT, > nameid INT, > value VARCHAR(100), > guide INT NOT NULL, > FOREIGN KEY (nameid) references ${NAMESNAME}(nameid)); > EOT > EFLAG=${?} > return ${EFLAG} > } > > function create_xmldoc { > #check parameter list > if [ ${#} -ne 4 ]; then > error "Usage: ${0} <schema> <tabname> <docname> <namesname>" > fi > > SCHEMA=${1} > TABNAME=${2} > DOCNAME=${3} > NAMENAME=${4} > > ${DB2} "set current schema ${SCHEMA}" > > ${DB2} -t<<EOT > CREATE VIEW ${TABNAME}(pre, size, level, kind, guide, > nameid, value, name) AS > (SELECT pre, size, level, kind, guide, > doc.nameid, value, name > FROM ${DOCNAME} AS doc > LEFT OUTER JOIN > ${NAMESNAME} AS names > ON doc.nameid = names.nameid); > EOT > EFLAG=${?} > return ${EFLAG} > } > > function drop_table { > #check parameter list > if [ ${#} -ne 1 ]; then > error "Usage: ${0} <table>" > fi > > TABLE=${1} > > ${DB2} "drop table ${TABLE}" > } > > function drop_names { > #check parameter list > if [ ${#} -ne 1 ]; then > error "Usage: ${0} <schema>" > fi > > SCHEMA=${1} > > ${DB2} "set current schema ${SCHEMA}" > drop_table "${TAB_NAMES}" > EFLAG=${?} > return ${EFLAG} > } > > function drop_doc { > #check parameter list > if [ ${#} -ne 1 ]; then > error "Usage: ${0} <schema>" > fi > > SCHEMA=${1} > > ${DB2} "set current schema ${SCHEMA}" > drop_table "${TAB_DOC}" > EFLAG=${?} > return ${EFLAG} > } > > function drop_xmldoc { > #check parameter list > if [ ${#} -ne 1 ]; then > error "Usage: ${0} <schema>" > fi > > SCHEMA=${1} > > ${DB2} "set current schema ${SCHEMA}" > ${DB2} "drop view xmldoc" > EFLAG=${?} > return ${EFLAG} > } > > function print_help () { > out "Pathfinder XQuery" > out "(c) Database Group, Technische Universitaet Muenchen" > out "" > out "Usage: $THISFILE (create|drop) <db> <schema>" > out "" > out -e "\t(create|drop):\teither create or drop the tables " > out -e "\t<db>:\t\tName of the Database the tables should be placed/removed." > out -e "\t<schema>:\tName of the Schema the tables should be placed/removed." > } > > ################################################ > # MAIN > > if [ $# -ne 3 ]; then > print_help; > exit 0; > fi > > OPTION=${1} > DATABASE=${2} > SCHEMA=${3} > > ${DB2} "connect to ${DATABASE}" > EFLAG=$? > out -ne "Connecting to database ${DATABASE} ... "; fail $EFLAG; > test $EFLAG -ne 0 && exit 1; > > case ${OPTION} in > create) create_names ${SCHEMA} ${TAB_NAMES}; > EFLAG=${?}; > out -ne "Creating ${SCHEMA}.${TAB_NAMES} ... "; fail ${EFLAG}; > > create_doc ${SCHEMA} ${TAB_DOC} ${TAB_NAMES}; > EFLAG=${?}; > out -ne "Creating ${SCHEMA}.${TAB_DOC} ... "; fail ${EFLAG}; > > create_xmldoc ${SCHEMA} ${TAB_XMLDOC} ${TAB_DOC} ${TAB_NAMES}; > EFLAG=${?}; > out -ne "Creating ${SCHEMA}.${TAB_XMLDOC} ... "; fail ${EFLAG}; > ;; > > drop) drop_xmldoc ${SCHEMA} > EFLAG=${?} > out -ne "Dropping ${SCHEMA}.${TAB_XMLDOC} ... "; fail ${EFLAG}; > > drop_doc ${SCHEMA} > EFLAG=${?} > out -ne "Dropping ${SCHEMA}.${TAB_DOC} ... "; fail ${EFLAG}; > > drop_names ${SCHEMA} > EFLAG=${?} > out -ne "Dropping ${SCHEMA}.${TAB_NAMES} ... "; fail ${EFLAG}; > ;; > *) print_help; > exit 1; > ;; > esac > ${DB2} "terminate" > > --- NEW FILE: common.inc --- > DEBUG=0 > > TAB_NAMES="names" > TAB_DOC="doc" > TAB_XMLDOC="xmldoc" > > DB2=`which db2` > > function fail { > if [ ${#} -ne 1 ]; then > error "Wrong use of fail" > fi > > if [ ${1} -eq 0 ]; then > out "Success" > return 0; > else > out "Failed" > return 0; > fi > } > > function error { > if [ ${#} -ne 1 ]; then > echo "Wrong use of error" > exit 1; > fi > echo -e "Error: ${1}" > exit 1 > } > > function warning > { > if [ ${#} -ne 1 ]; then > error "Wrong use of debug" > fi > echo -e "Warning: ${1}" > } > > function out { > if [ ${#} -eq 1 ]; then > echo -e "${1}" > return 0; > fi > if [ ${#} -eq 2 ]; then > echo "${1}" "${2}" > else > error "wrong use of out" > fi > } > > function debug { > if [ ${#} -ne 1 ]; then > error "Wrong use of debug" > fi > test ${DEBUG} -eq 1 && echo -e "Debug: ${1}" > } > > function LOG1 { > prompt="\n# `date +%H:%M:%S` > "; { echo -en "$prompt$prompt" ; echo -n "$*" ; echo -en "$prompt\n\n" ; } >&2; > } > > function getAbsolutePath { > if [ -z ${1} ]; then > error "Wrong use of ${0}"; > fi > > PATH_=${1%/*} > FILE_=${1##*/} > > echo `cd ${PATH_} 2> /dev/null && pwd || echo ${PATH_}`/${FILE_} > } > > test -z $DB2 && error "DB2 not available"; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 3

[Monetdb-developers] hi
by Mark Polo 20 Sep '07

20 Sep '07

hi, Monet DB is in-memory database only ? is it disk-based just like timesten contains both? thnaks n regards mark polo -- View this message in context: http://www.nabble.com/hi-tf4486230.html#a12793347 Sent from the monetdb-developers mailing list archive at Nabble.com.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/sql lalg2sql.brg, , 1.73, 1.74
by Stefan Manegold 19 Sep '07

19 Sep '07

On Wed, Sep 19, 2007 at 07:19:52AM +0000, Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/sql > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17096/sql > > Modified Files: > lalg2sql.brg > Log Message: > > > Fixing failure indicated by testweb Thanks! BUT: http://sourceforge.net/mailarchive/forum.php?thread_name=20070918223806.GA2… says: " [...] IMPORTANT NOTES: 1) As usually, as of NOW, a) ALL bug fixes (and ONLY those) MUST be checked in to the above release branches (and ONLY there)! b) ALL new features (and ONLY those) MUST be checked in to the development trunk (HEAD) (and ONLY there)! (In case you need assistance please feel free to contact MonetDB-developers@li... .) [...] " Stefan > Index: lalg2sql.brg > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/sql/lalg2sql.brg,v > retrieving revision 1.73 > retrieving revision 1.74 > diff -u -d -r1.73 -r1.74 > --- lalg2sql.brg 18 Sep 2007 11:47:20 -0000 1.73 > +++ lalg2sql.brg 19 Sep 2007 07:19:48 -0000 1.74 > @@ -482,8 +482,6 @@ > #define EXT_COLUMN_NAME(alias,att,ty) (ext_column_name ( \ > (alias), new_col ((att),(ty)))) > > - > - > /* .......... Column Environment Helper Functions .......... */ > > struct sql_column_env_t > @@ -1523,14 +1521,18 @@ > * twigs */ > PFla_op_t *twig; > unsigned int i; > - for (i = 0; i < PFarray_last (frags); i++) { > - twig = *(PFla_op_t **)PFarray_at (frags, i); > - if (twig == p) > - break; > - } > - if (i < PFarray_last (frags) && twig) { > - SER_REPORT(p) = ser_tc; > - return; > + > + if (p->kind == la_twig) { > + for (i = 0; i < PFarray_last (frags); i++) { > + twig = *(PFla_op_t **)PFarray_at (frags, i); > + if (twig == p) > + break; > + } > + > + if (i < PFarray_last (frags) && twig) { > + SER_REPORT(p) = ser_tc; > + return; > + } > } > > /* if we find a leaf return yes */ > @@ -1773,10 +1775,8 @@ > alias_bind(document, alias (docalias1))), > eq (PRE(docalias1), item_pre)), > alias_bind (document, alias (docalias2))), > - and_ ( > - gteq (PRE(docalias2), PRE(docalias1)), > - gteq (add (PRE(docalias1), SIZE(docalias1)), > - PRE(docalias2))))), > + between (PRE(docalias2), PRE(docalias1), > + add (PRE(docalias1), SIZE(docalias1))))), > NULL, > (IS_LITERAL(cpos))? > stmt_list (PRE(docalias2)): > @@ -2211,12 +2211,14 @@ > case 6: > /* Rel: thetajoin (Rel, Rel) */ > case 8: > + { > assert (kids[0] && nts[0]); > assert (kids[1] && nts[1]); > /* reduce the first child */ > reduce (kids[0], nts[0]); > > PFarray_t *left_args = PFarray (sizeof(PFsql_t*)); > + /* left join predicate for the equijoin */ > if (p->kind == la_eqjoin) { > PFalg_simple_type_t l_ty; > > @@ -2229,7 +2231,7 @@ > p->sem.eqjoin.att1, > l_ty); > } > - /* join predicates for the theta-join operator */ > + /* left join predicates for the theta-join operator */ > else if (p->kind == la_thetajoin) { > PFalg_att_t left; > PFalg_simple_type_t l_ty; > @@ -2256,6 +2258,7 @@ > reduce (kids[1], nts[1]); > > PFarray_t *right_args = PFarray (sizeof(PFsql_t*)); > + /* left join predicate for the equi-join operator */ > if (p->kind == la_eqjoin) { > PFalg_simple_type_t r_ty; > > @@ -2382,7 +2385,7 @@ > where_list_add (WHERELIST(p), cond); > } > } > - break; > + } break; > > /* Rel: semijoin (Rel, Rel) */ > case 7: > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Release preparations: Please refrain from checkins!
by Stefan Manegold 19 Sep '07

19 Sep '07

Dear MonetDB developers, Sjoerd and me are busing preparing for the upcoming release. The next step will be to create the new release branches. To ensure a smooth progress of the next step(s), and to avoid conflicts, we kindly ask you to refrain from ANY checkins to ANY MonetDB CVS repositories as of NOW until we notify you that checkins are allowed, again. Thank you very much for your cooperation! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/Tests pf-A_XMark.stable.err, , 1.1, 1.2 pf-A_XMark.stable.out, , 1.1, 1.2 pf-M_XMark.stable.err, , 1.6, 1.7 pf-M_XMark.stable.out, , 1.6, 1.7 usage.stable.out, , 1.15, 1.16 usageH.stable.out, , 1.34, 1.35
by Stefan Manegold 18 Sep '07

18 Sep '07

Oops! Peter, was this approval of error messages intended??? ( see also [ 1760459 ] PF/alg: tests fails since 2007/07/11 http://sourceforge.net/tracker/index.php?func=detail&aid=1760459&group_id=5… ) If not, please undo your (unintended) changes. In many cases, conveniences of `cvs diff` have proven very useful to check/verify one's changes before checking them in... Stefan On Tue, Sep 18, 2007 at 01:32:29PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12888/compiler/Tests > > Modified Files: > pf-A_XMark.stable.err pf-A_XMark.stable.out > pf-M_XMark.stable.err pf-M_XMark.stable.out usage.stable.out > usageH.stable.out > Log Message: > - approving correct test results > > > > Index: pf-A_XMark.stable.err > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/Tests/pf-A_XMark.stable.err,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- pf-A_XMark.stable.err 28 Dec 2005 17:04:06 -0000 1.1 > +++ pf-A_XMark.stable.err 18 Sep 2007 13:32:25 -0000 1.2 > @@ -5,6 +5,16 @@ > # 16:36:51 > Mtimeout -timeout 60 ./pf-A_XMark pf-A_XMark > # 16:36:51 > > > +"pf -A ff.xq >/dev/null" > +error in function application: at (13,64-13,89): reference to undefined function `pf:hash' > +# halted in /data/home/boncz/monet4/src/pathfinder/compiler/semantics/functions.c (check_fun_usage), line 296 > +"pf -A f.xq >/dev/null" > +"pf -A g.xq >/dev/null" > +"pf -A id01.xq >/dev/null" > +"pf -A id04.xq >/dev/null" > +"pf -A or01.xq >/dev/null" > +"pf -A or04.xq >/dev/null" > +"pf -A or10.xq >/dev/null" > "pf -A q01.xq >/dev/null" > "pf -A q02.xq >/dev/null" > "pf -A q03.xq >/dev/null" > @@ -25,6 +35,13 @@ > "pf -A q18.xq >/dev/null" > "pf -A q19.xq >/dev/null" > "pf -A q20.xq >/dev/null" > +"pf -A tst2.xq >/dev/null" > +error in function application: at (1,3-1,22): reference to undefined function `pf:hash' > +# halted in /data/home/boncz/monet4/src/pathfinder/compiler/semantics/functions.c (check_fun_usage), line 296 > +"pf -A tst.xq >/dev/null" > +error in function application: at (1,71-1,92): reference to undefined function `pf:hash' > +# halted in /data/home/boncz/monet4/src/pathfinder/compiler/semantics/functions.c (check_fun_usage), line 296 > +"pf -A t.xq >/dev/null" > "pf -A upd.xq >/dev/null" > > # 16:36:55 > > > Index: pf-A_XMark.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/Tests/pf-A_XMark.stable.out,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- pf-A_XMark.stable.out 28 Dec 2005 17:04:06 -0000 1.1 > +++ pf-A_XMark.stable.out 18 Sep 2007 13:32:26 -0000 1.2 > @@ -5,6 +5,14 @@ > # 16:36:51 > Mtimeout -timeout 60 ./pf-A_XMark pf-A_XMark > # 16:36:51 > > > +"pf -A ff.xq >/dev/null" > +"pf -A f.xq >/dev/null" > +"pf -A g.xq >/dev/null" > +"pf -A id01.xq >/dev/null" > +"pf -A id04.xq >/dev/null" > +"pf -A or01.xq >/dev/null" > +"pf -A or04.xq >/dev/null" > +"pf -A or10.xq >/dev/null" > "pf -A q01.xq >/dev/null" > "pf -A q02.xq >/dev/null" > "pf -A q03.xq >/dev/null" > @@ -25,6 +33,9 @@ > "pf -A q18.xq >/dev/null" > "pf -A q19.xq >/dev/null" > "pf -A q20.xq >/dev/null" > +"pf -A tst2.xq >/dev/null" > +"pf -A tst.xq >/dev/null" > +"pf -A t.xq >/dev/null" > "pf -A upd.xq >/dev/null" > > # 16:36:55 > > > > Index: pf-M_XMark.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/Tests/pf-M_XMark.stable.out,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- pf-M_XMark.stable.out 23 May 2005 22:14:32 -0000 1.6 > +++ pf-M_XMark.stable.out 18 Sep 2007 13:32:26 -0000 1.7 > @@ -5,6 +5,14 @@ > # 16:36:51 > Mtimeout -timeout 60 ./pf-M_XMark pf-M_XMark > # 16:36:51 > > > +"pf -M ff.xq >/dev/null" > +"pf -M f.xq >/dev/null" > +"pf -M g.xq >/dev/null" > +"pf -M id01.xq >/dev/null" > +"pf -M id04.xq >/dev/null" > +"pf -M or01.xq >/dev/null" > +"pf -M or04.xq >/dev/null" > +"pf -M or10.xq >/dev/null" > "pf -M q01.xq >/dev/null" > "pf -M q02.xq >/dev/null" > "pf -M q03.xq >/dev/null" > @@ -25,6 +33,9 @@ > "pf -M q18.xq >/dev/null" > "pf -M q19.xq >/dev/null" > "pf -M q20.xq >/dev/null" > +"pf -M tst2.xq >/dev/null" > +"pf -M tst.xq >/dev/null" > +"pf -M t.xq >/dev/null" > "pf -M upd.xq >/dev/null" > > # 16:36:55 > > > > Index: pf-M_XMark.stable.err > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/Tests/pf-M_XMark.stable.err,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- pf-M_XMark.stable.err 23 May 2005 22:14:32 -0000 1.6 > +++ pf-M_XMark.stable.err 18 Sep 2007 13:32:26 -0000 1.7 > @@ -5,6 +5,16 @@ > # 16:36:51 > Mtimeout -timeout 60 ./pf-M_XMark pf-M_XMark > # 16:36:51 > > > +"pf -M ff.xq >/dev/null" > +error in function application: at (13,64-13,89): reference to undefined function `pf:hash' > +# halted in /data/home/boncz/monet4/src/pathfinder/compiler/semantics/functions.c (check_fun_usage), line 296 > +"pf -M f.xq >/dev/null" > +"pf -M g.xq >/dev/null" > +"pf -M id01.xq >/dev/null" > +"pf -M id04.xq >/dev/null" > +"pf -M or01.xq >/dev/null" > +"pf -M or04.xq >/dev/null" > +"pf -M or10.xq >/dev/null" > "pf -M q01.xq >/dev/null" > "pf -M q02.xq >/dev/null" > "pf -M q03.xq >/dev/null" > @@ -25,6 +35,13 @@ > "pf -M q18.xq >/dev/null" > "pf -M q19.xq >/dev/null" > "pf -M q20.xq >/dev/null" > +"pf -M tst2.xq >/dev/null" > +error in function application: at (1,3-1,22): reference to undefined function `pf:hash' > +# halted in /data/home/boncz/monet4/src/pathfinder/compiler/semantics/functions.c (check_fun_usage), line 296 > +"pf -M tst.xq >/dev/null" > +error in function application: at (1,71-1,92): reference to undefined function `pf:hash' > +# halted in /data/home/boncz/monet4/src/pathfinder/compiler/semantics/functions.c (check_fun_usage), line 296 > +"pf -M t.xq >/dev/null" > "pf -M upd.xq >/dev/null" > > # 16:36:55 > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] SQL serializer
by Manuel Mayr 18 Sep '07

18 Sep '07

Hi guys, how can i adjust the Makefile.ag to copy the serialize.pl script to the binary-directory. The following approach Makefile.ag>> scripts_pl = { DIR = bindir SOURCES = serialize.pl } doesn't work. Best regards, Manuel -- Manuel Mayr, Dipl. Inf. Institut fuer Informatik, TU Muenchen Tel.: +49 89 289-17259

2 2

[Monetdb-developers] Hi
by Mark Polo 18 Sep '07

18 Sep '07

Hi Im installed MonetDB in windows XP. I dont find the mapi.h.header file.. from where should i get this ? -- View this message in context: http://www.nabble.com/Hi-tf4440370.html#a12669112 Sent from the monetdb-developers mailing list archive at Nabble.com.

4 11

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing Mtest.py.in, , 1.307, 1.308
by Sjoerd Mullender 18 Sep '07

18 Sep '07

Please revert and update your clients. The -o option for mclient is gone. Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv20410 > > Modified Files: > Mtest.py.in > Log Message: > mclient -lx -oxml -fxml > > produces very oddly escaped and pre-abled results (correct/bug?) > in any case, such output is not what the testweb expects > > therefore: removing -fxml > > > > Index: Mtest.py.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/testing/Mtest.py.in,v > retrieving revision 1.307 > retrieving revision 1.308 > diff -u -d -r1.307 -r1.308 > --- Mtest.py.in 14 Sep 2007 16:09:12 -0000 1.307 > +++ Mtest.py.in 17 Sep 2007 23:12:34 -0000 1.308 > @@ -2700,7 +2700,7 @@ > dft['MALCLIENT'] = "'mclient -lmal'" > dft['SQLCLIENT'] = "'mclient -lsql'" > dft['SQLDUMP'] = "'msqldump'" > - dft['XQUERYCLIENT'] = "'mclient -lxquery -fxml'" > + dft['XQUERYCLIENT'] = "'mclient -lxquery -oxml'" > dft['X100CLIENT'] = "'x100run_mapi'" > > #par = {} > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/benchmarks/MBench/Tests qj01.timeout, , 1.1, 1.2
by Stefan Manegold 18 Sep '07

18 Sep '07

Peter, just for info: This test/query used to run in 20-50 secs (depending on the platform). Now is seems to take at least 104 secs even on our fastest platforms. See also grep -w qj01 /ufs/monet/Testing/Current/pathfinder/.CheckLists/Mtest10..2007.0[89].*.time Stefan On Mon, Sep 17, 2007 at 09:24:23PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/benchmarks/MBench/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv20637 > > Modified Files: > qj01.timeout > Log Message: > > > Index: qj01.timeout > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/benchmarks/MBench/Tests/qj01.timeout,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- qj01.timeout 13 May 2007 09:43:08 -0000 1.1 > +++ qj01.timeout 17 Sep 2007 21:24:20 -0000 1.2 > @@ -1 +1 @@ > -2 > +4 > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] SQL temporary table not working?
by Agustin Schapira 14 Sep '07

14 Sep '07

Hi, I am having problems with SQL TEMPORARY TABLES: even when I insert rows into them, they remain empty. See for example: monetdb-> create local temporary table d (id int); Operation successful monetdb-> insert into d values(1); 1 affected row monetdb-> select * from d; +--------+ | id | +========+ +--------+ 0 rows If I create a TEMPORARY TABLE from a SELECT ... WITH DATA statement, it gets the correct structure but no data. Also, if I later insert rows into the temporary table, it still remains empty. Here's a sample session that demonstrates the problem: monetdb-> create table a (id int); Operation successful monetdb-> insert into a values(1); 1 affected row monetdb-> select * from a; +--------+ | id | +========+ | 1 | +--------+ 1 row monetdb-> create temporary table b as select * from a with data; Operation successful monetdb-> select * from b; +--------+ | id | +========+ +--------+ 0 rows monetdb-> insert into b values(5); 1 affected row monetdb-> select * from b; +--------+ | id | +========+ +--------+ 0 rows monetdb-> insert into b select * from a; 1 affected row monetdb-> select * from b; +--------+ | id | +========+ +--------+ 0 rows This happens with the latest release, compiled yesterday with the -- nightly='stable' switch: # MonetDB Server v5.0.1 # Copyright (c) 1993-2007 CWI, all rights reserved # Compiled for i686-apple-darwin8.10.1/32bit with 32bit OIDs dynamically linked Database: MonetDB 1.18.3 Driver: MonetDB Native Driver 1.6 (Steadfast/MCL-1.0 20070913) auto commit mode: on By the way, this used to work with MonetDB Server v5.0.0_beta1_2 (although the WITH DATA option didn't load the data: I had to create the structure and then INSERT into the table to get the rows). Any ideas? Am I doing something wrong? Are TEMPORARY TABLES not supposed to be used like this? Thanks for your help, -- Agustin

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB4/src/mapi/Tests manyMapiClients.MIL.py, , 1.2, 1.3
by Fabian Groffen 14 Sep '07

14 Sep '07

On 13-09-2007 23:09:33 +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB4/src/mapi/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1511/MonetDB4/src/mapi/Tests > > Modified Files: > manyMapiClients.MIL.py > Log Message: > > clean-up: > do not play any tricks to find where Mapi.py is installed; > rather assume that it is in Python's default search path, > or that `monetdb-clients-config --pythonlibdir` is in PYTHONPATH > +# assume module MonetDB is in Python's default search path, > +# or `monetdb-clients-config --pythonlibdir` is in PYTHONPATH > from MonetDB import Mapi Alternatively, maybe you can do something like try: from MonetDB import Mapi except ImportError: sys.path.insert(0, "autoconf-path-to-mapi") from MonetDB import Mapi ? Of course using the correct python commands for this so it works.

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred xmlshred.c, , 1.19, 1.20
by Sjoerd Mullender 13 Sep '07

13 Sep '07

Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12897/src/sqlhelpers/xmlshred > > Modified Files: > xmlshred.c > Log Message: > > > -- snprintf "bug" > -- fn:number implemented > > > Index: xmlshred.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/xmlshred.c,v > retrieving revision 1.19 > retrieving revision 1.20 > diff -u -d -r1.19 -r1.20 > --- xmlshred.c 13 Sep 2007 11:46:08 -0000 1.19 > +++ xmlshred.c 13 Sep 2007 13:40:21 -0000 1.20 > @@ -281,8 +281,10 @@ > guideout = SHopen_write (guideoutfile); > } > > + > + char outfile[FILENAME_MAX]; > snprintf (status.outfile, FILENAME_MAX, "%s.csv", status.outfile); Shouldn't you chante status.outfile to outfile here? > - shout = SHopen_write (status.outfile); > + shout = SHopen_write (outfile); > } > else > shout = stdout; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred/include shred_helper.h, , 1.9, 1.10
by Manuel Mayr 13 Sep '07

13 Sep '07

Thanks a lot. Best regards, Manuel Stefan Manegold wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/include > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv32155/src/sqlhelpers/xmlshred/include > > Modified Files: > shred_helper.h > Log Message: > > for the sake of compatibility and portability: > > To ensure that all C files equally benefit from what configure finds out > we need to make pf_config.h the *first* include in each C file > (either explicitly, or implicitly via some other common header file; > since the xmlshredded C files do not seem to share a single common > header file, we go for the explicit solution). > > > Index: shred_helper.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/include/shred_helper.h,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- shred_helper.h 12 Sep 2007 20:07:36 -0000 1.9 > +++ shred_helper.h 12 Sep 2007 20:12:37 -0000 1.10 > @@ -28,7 +28,6 @@ > #ifndef SHRED_HELPER_H__ > #define SHRED_HELPER_H__ > > -#include "pf_config.h" > #include <unistd.h> > #include <stdlib.h> > #include <stdio.h> > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- Manuel Mayr, Dipl. Inf. Institut fuer Informatik, TU Muenchen Tel.: +49 89 289-17259

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/semantics heuristic.c
by Jan Rittinger 12 Sep '07

12 Sep '07

Hi Peter, I have two small comments to your changes: 1. Is there a reason for including monet_utils.h here? -- In my eyes it introduces an unnecessary dependency to MonetDB (as we want the SQL code generation to compile even without MonetDB). Index: milprint_summer.c =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint_summer.c,v retrieving revision 1.404 retrieving revision 1.405 diff -u -d -r1.404 -r1.405 --- milprint_summer.c 5 Sep 2007 17:06:54 -0000 1.404 +++ milprint_summer.c 11 Sep 2007 22:15:29 -0000 1.405 @@ -56,6 +56,7 @@ #define USE_DEPRECATED_ACCESS_TO_TYPE_SYSTEM 1 #include "pathfinder.h" +#include "monet_utils.h" #include <stdio.h> #include <assert.h> 2. I think you should use "#pf" (PFns_pf) as prefix for variables names instead of "pf" (PFns_lib). Namespace PFns_lib can be used in the query which allows a user to create pf:heur* variables that might conflict lateron (whereas PFns_pf can only be created inside pathfinder). --- NEW FILE: heuristic.c --- ... static PFpnode_t* var_(PFloc_t loc, int varnum) { PFpnode_t *r = p_leaf(p_varref, loc); if (r) { r->sem.qname_raw.prefix = "pf"; r->sem.qname_raw.loc = (char*) PFmalloc(8); if (r->sem.qname_raw.loc) snprintf(r->sem.qname_raw.loc, 8, "heur%03d", varnum); } return r; } On 09/12/2007 12:15 AM, Peter Boncz wrote with possible deletions: > Update of /cvsroot/monetdb/pathfinder/compiler/semantics > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24884/compiler/semantics > > Modified Files: > Makefile.ag xquery_fo.c > Added Files: > heuristic.c > Log Message: > The infamous push-selections-down heuristic rewrite > =================================================== > > the good: equi comparisons on attributes, texts and elements in MXQ can now > be done in sublinear time thanks to indices. This involves the > "reversing"of multi-step XPath paths if required. > > the bad: it is a heuristic so no performance improvement assured, though much > of the performance pitfalls are mitigated by a loop-lifted run-time > decision whether to use the index plan or the original plan. > > the ugly: well, since I can't (or couldn't) stomach XQuery core plan, it is all > implemeted on the absssyn level. > > there are quite a few comments/explanations in compiler/semantics/heuristic.c > > runtime/pathfinder.mx > - major changes to the indexing scheme, supporting the pf:text/pf:attribute functions > - added run-time behavior to vx_lookup: for those iterations where index lookup is > not beneficial, it returns a bogus node (1@0, recognizable as a docnode) so > the original (non index-pushdown) plan is used for those iterations > > runtime/pf_support.mx > - new xquery hash function, with less collisions and the ability to preserve equality > semantics for any XQuery type (currently supported..). > - added a new dbl(byte1,...byte8) constructor that allows a IEEE double to be > constructed from its 8 binary bytes (per byte to avoid little/big endian issues) > without any precision loss. > - optimization for the ll_htordered_thetajoin in case we have constant join columns > > compiler/compile.c > compiler/include/Makefile.ag > compiler/semantics/Makefile.ag > - add heuristic.[ch] > - added heuristic index pushdown opt (currently only in mps, but alg/M4 should follow) > > compiler/semantics/xquery_fo.c > compiler/mil/milprint_summer.c > - rewrote pf:atttribute and pf:text to give approximate answers > - new pf:supernode function > - pass parsed decimals/doubles using the new byte-wise MIL dbl() constructor > (to avoid precision loss in the pf -> MonetDB transition) > - bugfix in empty ipik opt for the for-rule > > compiler/debug/abssynprint.c > - PFabssyn_stdout for debugging sanity > compiler/debug/prettyp.c > - margin=6 for better debug output readability > -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

2 1

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.86, 1.87
by Fabian Groffen 12 Sep '07

12 Sep '07

On 11-09-2007 15:51:28 +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv2455/src/mapiclient > > Modified Files: > MapiClient.mx > Log Message: > Coalesced mclient's -o and -f options into a single -f (output format) > option. > > The -o option was used to specify output formatting for XQuery, > whereas -f was used to specify output formatting for the other > languages. Now the same option is used for both (similar) functions. > > This also frees up -o for a future, possible -o output file option. Is there a chance the two could be switched? -f traditionally is in use for [f]ile. JdbcClient also happens to have implemented the -f flag like that.

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_client.mx, , 1.157, 1.158
by Martin Kersten 12 Sep '07

12 Sep '07

Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15887 > > Modified Files: > mal_client.mx > Log Message: > dirty fix for block mode (from end of block find new line, shift all > characters up to new line add EOS and increment length of read data UGH > possible problem new line in string) Not dirty. They should be escaped to be acceptable in MAL. The new line terminates the instruction.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing Makefile.ag, , 1.20, 1.21 Mtest.py.in, , 1.300, 1.301
by Stefan Manegold 11 Sep '07

11 Sep '07

On Tue, Sep 11, 2007 at 11:11:49AM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17756/src/testing > > Modified Files: > Makefile.ag Mtest.py.in > Log Message: > Put Python files in their own Python module MonetDB. > In the RPMs, put Python files in the Python hierarchy. > Created a new RPM MonetDB-python which contains monet_options.py and other such files. > Added merovingian man page to RPM. > Fixed MonetDB5 spec file. > Removed Python ODBC interface: it was not directly related to MonetDB. These changes requires to set PYTHONPATH="MONETDB_PREFIX/`python -c 'import distutils.sysconfig; print distutils.sysconfig.get_python_lib(0,0,"")'`" before calling Mtest.py, unless MONETDB_PREFIX is /usr . Should we consider making Mtest.py search automatically for modules in the above path (as it IMHO did before), or shall we just document the above requirement? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 3

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred/include shred_helper.h, , 1.2, 1.3
by Stefan Manegold 11 Sep '07

11 Sep '07

On Tue, Sep 11, 2007 at 10:01:22AM +0000, Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/include > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17796/include > > Modified Files: > shred_helper.h > Log Message: > > > Adding definitions for SSZFMT and fixing some bugs > > > Index: shred_helper.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/include/shred_helper.h,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- shred_helper.h 10 Sep 2007 10:09:37 -0000 1.2 > +++ shred_helper.h 11 Sep 2007 10:01:20 -0000 1.3 [...] > #if !(HAVE_STRING_H && HAVE_STRDUP) ^^ > +char * strndup (const char * s); ^^^ one 'n' too much? (or one 'N' missing??) Stefan > +#endif > + > +#if !(HAVE_STRING_H && HAVE_STRNDUP) > char * strndup (const char * s, size_t n); > #endif -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/tools merovingian.mx, , 1.46, 1.47
by Martin Kersten 11 Sep '07

11 Sep '07

Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv3294 > > Modified Files: > merovingian.mx > Log Message: > Again, Merovingian is not the one to create directories, monetdb does > that. as reported. it doesn't

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred xmlshred.c, , 1.2, 1.3
by Sjoerd Mullender 10 Sep '07

10 Sep '07

When printing a value of type size_t, use SZFMT. When printing a value of type ssize_t, use SSZFMT. The format needed to print these values varies from system to system, so that difference is hidden in a macro. Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv13392 > > Modified Files: > xmlshred.c > Log Message: > > > Thx for noting. > > > Index: xmlshred.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/xmlshred.c,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- xmlshred.c 9 Sep 2007 08:27:39 -0000 1.2 > +++ xmlshred.c 10 Sep 2007 06:55:38 -0000 1.3 > @@ -51,8 +51,7 @@ > > #define NAME_ID 0 > > -typedef long unsigned int nat; > -typedef int pre_t; > +typedef ssize_t nat; > > enum kind_t { > elem > @@ -138,8 +137,8 @@ > > typedef struct node_t node_t; > struct node_t { > - pre_t pre; > - pre_t apre; > + nat pre; > + nat apre; > nat post; > nat pre_stretched; > nat post_stretched; > @@ -155,7 +154,7 @@ > static node_t stack[STACK_MAX]; > static int level; > static int max_level; > -static pre_t pre; > +static nat pre; > static nat post; > static nat rank; > static nat att_id; > @@ -346,7 +345,7 @@ > attr_guide_node = insert_guide_node((char*)atts_back[0], > current_guide_node, attr); > > - fprintf (out_attr, "%lu, %i, \"%s\", \"%s\", %lu\n", > + fprintf (out_attr, "%i, %i, \"%s\", \"%s\", %i\n", > att_id++, pre, (char*)atts[0], (char*)atts[1], attr_guide_node->guide); > > atts += 2; > @@ -602,10 +601,10 @@ > case 'e': if(tuple.pre != -1) > fprintf (out, "%i", tuple.pre); > break; > - case 'o': fprintf (out, "%lu", tuple.post); break; > - case 'E': fprintf (out, "%lu", tuple.pre_stretched); break; > - case 'O': fprintf (out, "%lu", tuple.post_stretched); break; > - case 's': fprintf (out, "%lu", tuple.size); break; > + case 'o': fprintf (out, "%i", tuple.post); break; > + case 'E': fprintf (out, "%i", tuple.pre_stretched); break; > + case 'O': fprintf (out, "%i", tuple.post_stretched); break; > + case 's': fprintf (out, "%i", tuple.size); break; > case 'l': fprintf (out, "%u", tuple.level); break; > > case 'p': > @@ -616,7 +615,7 @@ > break; > case 'P': > if (tuple.parent) > - fprintf (out, "%lu",tuple.parent->pre_stretched); > + fprintf (out, "%i",tuple.parent->pre_stretched); > else > fprintf (out, "NULL"); > break; > @@ -653,7 +652,7 @@ > fprintf(out, "NULL"); > } > break; > - case 'g': fprintf (out, "%lu", tuple.guide); break; > + case 'g': fprintf (out, "%i", tuple.guide); break; > > default: putc (format[i], out); break; > } > @@ -966,7 +965,7 @@ > /* print the node self */ > fprintf ( > guide_out, > - "<node guide=\"%lu\" count=\"%lu\" kind=\"", > + "<node guide=\"%i\" count=\"%i\" kind=\"", > root->guide, > root->count); > print_kind (guide_out, root->kind); > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

4 5

[Monetdb-developers] conf.bash still used?
by Stefan Manegold 10 Sep '07

10 Sep '07

Hi, I'm about to remove */conf/conf.bash, the script that "once" was provided to "conveniently" set your environment for building MonetDB, but --- thanks to improved configure setup and system installations --- should not be required any more. Please let me know a.s.a.p. in case there are any objections and/or anybody is still using this script. Thanks, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/tools merovingian.mx, , 1.46, 1.47
by Martin Kersten 10 Sep '07

10 Sep '07

That creates a deadlock. monetdb can not work without the mero and mero needs the dbfarm. Try testing your system without a dbfarm and var/log Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv3294 > > Modified Files: > merovingian.mx > Log Message: > Again, Merovingian is not the one to create directories, monetdb does > that. > > > Index: merovingian.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/tools/merovingian.mx,v > retrieving revision 1.46 > retrieving revision 1.47 > diff -u -d -r1.46 -r1.47 > --- merovingian.mx 10 Sep 2007 07:44:09 -0000 1.46 > +++ merovingian.mx 10 Sep 2007 07:46:38 -0000 1.47 > @@ -1099,11 +1099,6 @@ > argp = dup(1); > topdp->fout = fdopen(argp, "w"); > } else { > - /* check the log directory */ > - if (access(msglog,F_OK) == -1 ){ > - GDKcreatedir(msglog); /* it doesn't necessarily create all */ > - fprintf(stderr, "log directory created '%s'\n", msglog); > - } > /* write to the given file */ > topdp->fout = fopen(msglog, "a"); > if (topdp->fout == NULL) { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing difflib.c, , 1.46, 1.47
by Fabian Groffen 10 Sep '07

10 Sep '07

On 09-09-2007 11:36:26 +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv4669 > > Modified Files: > difflib.c > Log Message: > > no need to call "touch" via "system()" to create an empty file; > "fclose(fopen())" should do the job within pure C. Because you use the stream versions here, fclose(fopen()) can crash you tool if fopen() returns NULL, as fclose doesn't grok it. > Index: difflib.c > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/testing/difflib.c,v > retrieving revision 1.46 > retrieving revision 1.47 > diff -u -d -r1.46 -r1.47 > --- difflib.c 5 Sep 2007 17:05:58 -0000 1.46 > +++ difflib.c 9 Sep 2007 11:36:22 -0000 1.47 > @@ -922,7 +922,7 @@ > > if (!u_diff2lwc_diff(mindiff, LWC, u_diff_fn, lwc_diff_fn)) > /* { UNLINK(lwc_diff_fn); ERRHNDL(0,"u_diff2lwc_diff returns 0 in u_diff2html","",1); } */ > - SYSTEM(strconcat("touch ", lwc_diff_fn)); > + fclose(Wfopen(lwc_diff_fn)); Unless Wfopen does something fancy with the pointer, I suggest to check if it's non NULL.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred xmlshred.c, , 1.1, 1.2
by Stefan Manegold 10 Sep '07

10 Sep '07

On Sun, Sep 09, 2007 at 08:27:41AM +0000, Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17315 > > Modified Files: > xmlshred.c > Log Message: > > Fixing compiler errors. > > > Index: xmlshred.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/xmlshred.c,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- xmlshred.c 8 Sep 2007 08:39:59 -0000 1.1 > +++ xmlshred.c 9 Sep 2007 08:27:39 -0000 1.2 > @@ -51,7 +51,8 @@ > > #define NAME_ID 0 > > -typedef long int nat; > +typedef long unsigned int nat; > +typedef int pre_t; What's the reason for introducing yet another type, here? Should "nat" be 32-bit on 32-bit systems and 64-bit on 64-bit systems? Well, that won't work for Windows: x64 Windows uses the P64 model, i.e., "int" and "long int" are 32 bits long, but pointers are 64 bits long. x86-64 Linux uses the LP64 model, when "long int" is also 64 bits while "int" remains 32 bits. In contrast, 32-bit Windows and Linux are both ILP32, meaning that "int", "long int" and pointers are all 32 bits. Hence, "long int" is a considered "evil" and a "portability killer". Why not use standard types "size_t" (unsigned) and "ssize_t" (signed) if you need a system word-size integer? In fact, "*pre*" and "*post*" --- assuming the names suggest that they represent pre- & post-orders --- are actually of type "oid" in the remainder of the pathfinder (MonetDB/XQUuery) code base (enabling 32-bit space-saving pre/post storage also on 64-bit systems); likiwise, "size" is "int" and "level" is "chr". Is there any reason not to use the same, here? Stefan > enum kind_t { > elem > @@ -133,17 +134,17 @@ > void free_hash(bucket_t **hash_table); > > /* return a brand new name_id */ > -int new_nameid(); > +int new_nameid(void); > > typedef struct node_t node_t; > struct node_t { > - nat pre; > - nat apre; > + pre_t pre; > + pre_t apre; > nat post; > nat pre_stretched; > nat post_stretched; > nat size; > - int level; > + unsigned int level; > int name_id; > node_t *parent; > kind_t kind; -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime/Tests serialization.milS, , 1.9, 1.10 serialization.stable.out, , 1.13, 1.14
by Stefan Manegold 09 Sep '07

09 Sep '07

Jan (R.), could you please check whether the output of the "text" serialization mode at http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests10… http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests10… http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests10… --- in particular the error message with top-level (root-node) attributes and the lack of whitespace(s) --- is correct / as desired? Thanks in advance! Stefan > Update of /cvsroot/monetdb/pathfinder/runtime/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv10673 > > Modified Files: > serialization.milS serialization.stable.out > Log Message: > > Extended serialization test with (not that) new "text" mode. > > Correct/desired output still needs to be verified by Jan Rittinger. > > > Index: serialization.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/Tests/serialization.stable.out,v > retrieving revision 1.13 > retrieving revision 1.14 > diff -u -d -r1.13 -r1.14 > --- serialization.stable.out 1 Sep 2007 17:06:53 -0000 1.13 > +++ serialization.stable.out 1 Sep 2007 17:28:05 -0000 1.14 > @@ -13,6 +13,7 @@ > > # query for all output modes (contains all different kinds) > var query := "(1, 2.3, attribute attr { \"val\" }, 3E1, \"string\", doc(\"nodekinds.xml\")/descendant-or-self::node())"; > +var query_no_attr := "(1, 2.3, 3E1, \"string\", doc(\"nodekinds.xml\")/descendant-or-self::node())"; > > # current different XML modes > # > @@ -103,6 +104,39 @@ > [ "&1 0 10 1 10\n% xquery.result\t# table_name\n% xml\t# name\n% varchar\t# type\n% 60\t# length\n[ \"1\"\t]\n[ \"2.300000\"\t]\n[ \"attribute attr { \"val\" }\"\t]\n[ \"30.000000\"\t]\n[ \"\\\"string\\\"\"\t]\n[ \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\\n<elem attr=\\\"val\\\">text<?pi_tgt pi_data?></elem>\\n\"\t]\n[ \"<elem attr=\\\"val\\\">text<?pi_tgt pi_data?></elem>\"\t]\n[ \"text {\\\"text\\\"}\"\t]\n[ \"\"\t]\n[ \"<?pi_tgt pi_data?>\"\t]\n" ] > > > +# text mode w/ top-level attribute > +# > +xquery("text" ,query , 0).printf(); > += Please remove this line once the below error message has been verified as correct/expected. > +!ERROR: 12.300000!ERROR: serialize_dflt.mx: unable to print top level attribute nodes in text mode > +!ERROR: xquery_print_result_loop: operation failed. > +!ERROR: interpret: no matching MIL operator to 'printf(void)'. > +!MAYBE YOU MEAN: > +! printf(str, ..any..) : void > += Please remove this line once the above error message has been verified as correct/expected. > + > +xquery("mapi-text" ,query , 0).printf(); > += Please remove this line once the below error message has been verified as correct/expected. > +!ERROR: =12.300000!ERROR: serialize_dflt.mx: unable to print top level attribute nodes in text mode > +!ERROR: xquery_print_result_loop: operation failed. > +!ERROR: interpret: no matching MIL operator to 'printf(void)'. > +!MAYBE YOU MEAN: > +! printf(str, ..any..) : void > += Please remove this line once the above error message has been verified as correct/expected. > + > +# text mode w/o top-level attribute > +# > +xquery("text" ,query_no_attr , 0).printf(); > += Please remove this line once the below text-mode output has been verified as correct/expected. > +12.30000030.000000string<elem attr="val">text<?pi_tgt pi_data?></elem><elem attr="val">text<?pi_tgt pi_data?></elem>text<?pi_tgt pi_data?> > += Please remove this line once the above text-mode output has been verified as correct/expected. > + > +xquery("mapi-text" ,query_no_attr , 0).printf(); > += Please remove this line once the below text-mode output has been verified as correct/expected. > +=12.30000030.000000string<elem attr="val">text<?pi_tgt pi_data?></elem><elem attr="val">text<?pi_tgt pi_data?></elem>text<?pi_tgt pi_data?> > += Please remove this line once the above text-mode output has been verified as correct/expected. > + > + > # default mode is xml mode (it also works with mapi > # -- all other mode arguments are ignored) > # > > Index: serialization.milS > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/Tests/serialization.milS,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- serialization.milS 1 Sep 2007 17:06:53 -0000 1.9 > +++ serialization.milS 1 Sep 2007 17:28:05 -0000 1.10 > @@ -2,6 +2,7 @@ > > # query for all output modes (contains all different kinds) > var query := "(1, 2.3, attribute attr { \"val\" }, 3E1, \"string\", doc(\"nodekinds.xml\")/descendant-or-self::node())"; > +var query_no_attr := "(1, 2.3, 3E1, \"string\", doc(\"nodekinds.xml\")/descendant-or-self::node())"; > > # current different XML modes > # > @@ -39,6 +40,19 @@ > xquery("seq" ,query , 0).print(); > > > +# text mode w/ top-level attribute > +# > +xquery("text" ,query , 0).printf(); > + > +xquery("mapi-text" ,query , 0).printf(); > + > +# text mode w/o top-level attribute > +# > +xquery("text" ,query_no_attr , 0).printf(); > + > +xquery("mapi-text" ,query_no_attr , 0).printf(); > + > + > # default mode is xml mode (it also works with mapi > # -- all other mode arguments are ignored) > # > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Super Source Ball release update
by Martin Kersten 07 Sep '07

07 Sep '07

A glitch in maintaining the monetdb website led to an outdated link for the super source ball on the download page. This tarball is a convenience package containing all source tars and a script (for Linux) to build components. The link has been updated to the latest version: http://downloads.sourceforge.net/monetdb/MonetDB-Mars-SuperBall-SR3.tar.bz2 The MonetDB Development team

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal Makefile.ag, , 1.103, 1.104 batxml.mx, , 1.14, 1.15
by Romulo Goncalves 06 Sep '07

06 Sep '07

Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29918/mal > > Modified Files: > Makefile.ag batxml.mx > Log Message: > Initialize variable to resolve problems on windows (32bit with VS) compilation Add a missing lib (lib_xml.lib) to be able to compile batxml.mx > > > > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/Makefile.ag,v > retrieving revision 1.103 > retrieving revision 1.104 > diff -u -d -r1.103 -r1.104 > --- Makefile.ag 21 Aug 2007 20:54:59 -0000 1.103 > +++ Makefile.ag 6 Sep 2007 11:04:32 -0000 1.104 > @@ -35,7 +35,7 @@ > #datacell.mx > > HEADERS = h > - LIBS = ../../mal/libmal $(MONETDB_LIBS) -lbat -lstream $(MATH_LIBS) \ > + LIBS = ../../mal/libmal ../atoms/lib_xml $(MONETDB_LIBS) -lbat -lstream $(MATH_LIBS) \ > $(CLIENTS_LIBS) -lMapi \ > $(SOCKET_LIBS) $(Z_LIBS) $(BZ_LIBS) \ > $(MALLOC_LIBS) $(PTHREAD_LIBS) $(DL_LIBS) > > Index: batxml.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/batxml.mx,v > retrieving revision 1.14 > retrieving revision 1.15 > diff -u -d -r1.14 -r1.15 > --- batxml.mx 1 Sep 2007 12:11:12 -0000 1.14 > +++ batxml.mx 6 Sep 2007 11:04:32 -0000 1.15 > @@ -729,7 +729,7 @@ > BAT *j, *r, *g, *b, *bn; > BUN p,q; > int xx; > - oid gid,o=0; > + oid gid = 0,o=0; > int first= 1; > str t; > str buf= GDKmalloc(BUFSIZ); > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal batxml.mx, GDK-2, 1.4.2.2, 1.4.2.3
by Romulo Goncalves 06 Sep '07

06 Sep '07

Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15018/src/modules/mal > > Modified Files: > Tag: GDK-2 > batxml.mx > Log Message: > propagated changes of Friday Aug 24 2007 - Tuesday Aug 28 2007 > from the development trunk to the GDK-2 branch > > There were inexplicable conflicts in src/modules/mal/batxml.mx and > src/modules/mal/Tests/inspect05.stable.out, neither of which had been > changed on the branch. I just copied the HEAD version to the branch > to resolve the conflicts. > ...... > str > -BATXMLagg(MalBlkPtr mb, MalStkPtr stk, InstrPtr p) > +BATXMLagg(int *ret, int *grp,int *bid) > { > - (void) mb; > - (void) stk; > - (void) p; > - throw(MAL,"xml.agg","Not yet implemented"); > + BAT *j, *r, *g, *b, *bn; > + BUN p,q; > + int xx; > + oid gid,o=0; The variable is not initialized. I will take the freedom to initialize it with value 0; Do I have green light or the values should be another one? Note: this is a propagation and the this code comes from the Martin checking on 08/25/2007 9:48 am > + int first= 1; > + str t; > + str buf= GDKmalloc(BUFSIZ); > + int len, size=BUFSIZ,offset; > + > + @:getBATdescriptor(grp,g,"xml.agg")@ > + @:getBATdescriptor(bid,b,"xml.agg",BBPunfix(g->batCacheid))@ > + > + bn= BATnew(g->htype, b->ttype,BATcount(g)); > + if( bn== NULL){ > + BBPunfix(b->batCacheid); > + BBPunfix(g->batCacheid); > + throw(MAL,"xml.agg","Could not create BAT"); > + } > + > + j= BATjoin(g,b,oid_nil); > + r= BATsort(j); > + /* now we can perform a simple scan and emit the group*/ > + > + buf[0]=0; > + offset=0; > + first=1; > + BATloopFast(r,p,q,xx){ > + o= *(oid*) BUNhead(r,p); > + t= (str) BUNtail(r,p); > + if( gid != o && first==0){ It is used here for the first time without being initialized. Regards, Romulo > + /* flush */ > + bunfastins(bn,&gid,buf); > + buf[0]=0; > + offset =0; > + } > + gid=o; > + first=0; .......

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_client.mx, , 1.156, 1.157 mal_scenario.mx, , 1.111, 1.112
by Romulo Goncalves 06 Sep '07

06 Sep '07

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18380/mal > > Modified Files: > mal_client.mx mal_scenario.mx > Log Message: > Add a few statistics for tracking client behavior. > > > Index: mal_client.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_client.mx,v > retrieving revision 1.156 > retrieving revision 1.157 > diff -u -d -r1.156 -r1.157 > --- mal_client.mx 30 Aug 2007 07:18:54 -0000 1.156 > +++ mal_client.mx 5 Sep 2007 18:13:16 -0000 1.157 > @@ -89,6 +89,10 @@ > # include <sys/resource.h> > #endif > > +#ifdef HAVE_SYS_TIMES_H > +# include <sys/times.h> > +#endif > + > @- > The prompt structure is designed to simplify recognition > of the language framework for interaction. For direct console > @@ -226,6 +230,18 @@ > Symbol curprg; /* focus of parser */ > Symbol backup; /* save parsing context */ > MalStkPtr glb; /* global variable stack */ > +@- > +Some statistics on client behavior becomes relevant > +for server maintenance. The scenario loop is used as > +a frame of reference. We measure the elapsed time after > +a request has been received and we have to wait for > +the next one. > +@h > + int actions; > + lng totaltime; /* sum of elapsed processing times */ > +#ifdef HAVE_TIMES > + struct tms workload; > +#endif > } *Client, ClientRec; > > mal_export ClientRec mal_clients[MAL_MAXCLIENTS+1]; > @@ -446,6 +462,9 @@ > prompt = !fin? GDKgetenv("monet_prompt"): PROMPT1; > c->prompt= GDKstrdup(prompt); > c->promptlength= strlen(prompt); > + > + c->actions =0; > + c->totaltime= 0; > return c; > } > @- > > Index: mal_scenario.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_scenario.mx,v > retrieving revision 1.111 > retrieving revision 1.112 > diff -u -d -r1.111 -r1.112 > --- mal_scenario.mx 30 Aug 2007 07:18:54 -0000 1.111 > +++ mal_scenario.mx 5 Sep 2007 18:13:16 -0000 1.112 > @@ -183,6 +183,10 @@ > #include "mal_authorize.h" > #include "mal_exception.h" > > +#ifdef HAVE_SYS_TIMES_H > +# include <sys/times.h> > +#endif > + > struct SCENARIO scenarioRec[MAXSCEN] = { > {"mal", "mal", > 0, 0, /* hardwired MALinit*/ > @@ -576,15 +580,31 @@ > runScenarioBody(Client c) > { > str msg= MAL_SUCCEED; > + lng start; > + struct tms t0,t1; I think here you need to do like this: > #ifdef HAVE_TIMES > struct tms t0,t1; > #endif If not the windows compilation does not work. I tried and now it compiles on windows. To checkin I am just waiting for green light because I am not really sure of the impact of my change in your semantics. Regards, Romulo > > while (c->mode > FINISHING) { > msg = MAL_SUCCEED; > @:runPhase(MAL_SCENARIO_READER)@ > c->lastcmd= time(0); > + start= GDKusec(); > +#ifdef HAVE_TIMES > + times(&t0); > +#endif > @:runPhase(MAL_SCENARIO_PARSER)@ > @:runPhase(MAL_SCENARIO_OPTIMIZE)@ > @:runPhase(MAL_SCENARIO_SCHEDULER)@ > @:runPhase(MAL_SCENARIO_ENGINE)@ > + start= GDKusec()-start; > + c->actions++; > + c->totaltime += start; > +#ifdef HAVE_TIMES > + times(&t1); > + c->workload.tms_utime += t1.tms_utime-t0.tms_utime; > + c->workload.tms_cutime += t1.tms_cutime-t0.tms_cutime; > + c->workload.tms_stime += t1.tms_stime-t0.tms_stime; > + c->workload.tms_cstime += t1.tms_cstime-t0.tms_cstime; > +#endif > } > return msg; > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah/Tests procs.stable.out, , 1.29, 1.30
by Stefan Manegold 05 Sep '07

05 Sep '07

Henning, thanks for being aware of stable output and maitaining int accoring to your changes. However, did you change the signatures of PROCs client(), descendant(), descendant_or_self(), ll_child(), ll_descendant(), ll_descendant_or_self(), preceding(), preceding_sibling() in addition to vx_maintain()? I cannot see any comment in your checkin message, nor did I find any hint in the diffs. Hence, why did you change the respective stable output, then? Well, to solve the "riddle" (addressing all of you, not only Henning!): when updating stable output, please make sure that (1) either your code base is indeed up-to-date, or that (2) you change only those parts of stable output which actually relate to your code changes. Stable output is even more important and sensitive than the actual code --- it is used to test the correctness of the actual code. Hence, please treat it with the respective concentration and care. Thanks. Stefan On Wed, Sep 05, 2007 at 03:41:57PM +0000, Henning Rode wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12978/Tests > > Modified Files: > procs.stable.out > Log Message: > bug-fix in multiplicative retrieval models > (thanks djoerd for finding it. > in those models, the number a term occurs in a query > becomes an exponent in the score function (not a factor)). > > > Index: procs.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/Tests/procs.stable.out,v > retrieving revision 1.29 > retrieving revision 1.30 > diff -u -d -r1.29 -r1.30 > --- procs.stable.out 1 Sep 2007 11:34:18 -0000 1.29 > +++ procs.stable.out 5 Sep 2007 15:41:55 -0000 1.30 > @@ -137,7 +137,6 @@ > [ "ceil(flt) : flt" ] > [ "checkpoint_protect(BAT[void,BAT]) : void" ] > [ "checkpoint_unprotect(BAT[void,BAT]) : void" ] > -[ "child(BAT[oid,oid], BAT[oid,oid], BAT[oid,oid], oid, BAT[oid,BAT], int, BAT[void,any], chr) : BAT[void,BAT]" ] > [ "child(BAT[oid,oid], BAT[oid,oid], oid, BAT[oid,BAT], int, BAT[void,any], chr) : BAT[void,BAT]" ] > [ "clear(BAT[any::1,any::2]) : BAT[any::1,any::2]" ] > [ "clients() : void" ] > @@ -173,9 +172,7 @@ > [ "delete_doc(BAT[void,str]) : void" ] > [ "delete_doc(str) : void" ] > [ "delete_pfc_fti(str) : void" ] > -[ "descendant(BAT[oid,oid], BAT[oid,oid], BAT[oid,oid], oid, BAT[oid,BAT], int, BAT[void,any], chr) : BAT[void,BAT]" ] > [ "descendant(BAT[oid,oid], BAT[oid,oid], oid, BAT[oid,BAT], int, BAT[void,any], chr) : BAT[void,BAT]" ] > -[ "descendant_or_self(BAT[oid,oid], BAT[oid,oid], BAT[oid,oid], oid, BAT[oid,BAT], int, BAT[void,any], chr) : BAT[void,BAT]" ] > [ "descendant_or_self(BAT[oid,oid], BAT[oid,oid], oid, BAT[oid,BAT], int, BAT[void,any], chr) : BAT[void,BAT]" ] > [ "destroy(BAT[any,any]) : bit" ] > [ "diff(BAT[any::1,any::2], BAT[any::1,any::2]) : BAT[any::1,any::2]" ] > @@ -271,9 +268,6 @@ > [ "listen(int, int) : void" ] > [ "listen(int, int, bit) : void" ] > [ "listen(int, str, int) : void" ] > -[ "ll_child(BAT[oid,oid], BAT[oid,oid], BAT[oid,int], BAT[void,any], bit, bit, oid, oid, bit, chr) : BAT[oid,oid]" ] > -[ "ll_descendant(BAT[oid,oid], BAT[oid,oid], BAT[oid,int], BAT[void,any], bit, bit, oid, oid, bit, chr) : BAT[oid,oid]" ] > -[ "ll_descendant_or_self(BAT[oid,oid], BAT[oid,oid], BAT[oid,int], BAT[void,any], bit, bit, oid, oid, bit, chr) : BAT[oid,oid]" ] > [ "ll_htordered_unique_thetajoin(int, BAT[oid,any::2], BAT[oid,any::2], BAT[void,oid], BAT[void,oid], any, any) : BAT[any,any]" ] > [ "ll_standoff_select_narrow(BAT[void,oid], BAT[oid,oid], BAT[oid,oid], BAT[oid,oid], BAT[oid,any], BAT[oid,any]) : BAT[oid,oid]" ] > [ "ll_standoff_select_wide(BAT[void,oid], BAT[oid,oid], BAT[oid,oid], BAT[oid,oid], BAT[oid,any], BAT[oid,any]) : BAT[oid,oid]" ] > @@ -490,8 +484,9 @@ > [ "pi() : dbl" ] > [ "play_doc_tape(BAT[void,BAT], BAT[void,oid], BAT[void,int], BAT[void,lng], BAT[void,str]) : void" ] > [ "play_update_tape(BAT[void,BAT], BAT[void,oid], BAT[void,int], BAT[void,lng], BAT[void,str]) : void" ] > +[ "preceding(BAT[void,oid], BAT[void,oid], oid, BAT[void,BAT], int) : BAT[void,BAT]" ] > [ "preceding(BAT[void,oid], BAT[void,oid], oid, BAT[void,BAT], int, chr) : BAT[void,BAT]" ] > -[ "preceding_sibling(BAT[void,oid], BAT[void,oid], oid, BAT[void,BAT], int, chr) : BAT[void,BAT]" ] > +[ "preceding_sibling(BAT[void,oid], BAT[void,oid], oid, BAT[void,BAT], int) : BAT[void,BAT]" ] > [ "print(str, BAT[any::1,any], ..BAT[any::1,any]..) : void" ] > [ "prior_logls(BAT[any,any], int, BAT[oid,str]) : BAT[any,any]" ] > [ "prior_ls(BAT[any,any], int, BAT[oid,str]) : BAT[any,any]" ] > @@ -652,7 +647,7 @@ > [ "vm_usage(int) : BAT[str,lng]" ] > [ "vx_lookup(BAT[void,BAT], BAT[void,oid], BAT[void,int], BAT[void,str], str, str, bit) : BAT[oid,oid]" ] > [ "vx_lookup(BAT[void,BAT], oid, BAT[void,str], str, str, bit) : BAT[oid,oid]" ] > -[ "vx_maintain(BAT[oid,oid], BAT[oid,oid], BAT[oid,oid], BAT[oid,str]) : BAT[int,oid]" ] > +[ "vx_maintain(BAT[oid,oid], BAT[oid,oid], BAT[oid,str]) : BAT[int,oid]" ] > [ "vx_maintain(BAT[oid,oid], BAT[oid,str]) : BAT[int,oid]" ] > [ "vx_reduce(BAT[oid,int], BAT[void,oid]) : BAT[oid,oid]" ] > [ "ws_collection(BAT[void,BAT], BAT[oid,str], BAT[void,oid]) : BAT[oid,oid]" ] > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/autogen/autogen am.py, , 1.45, 1.46 msc.py, , 1.34, 1.35
by Stefan Manegold 04 Sep '07

04 Sep '07

On Tue, Sep 04, 2007 at 12:30:16PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/buildtools/autogen/autogen > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv27558/autogen/autogen > > Modified Files: > am.py msc.py > Log Message: > Make it possible to have conditional symlinks. Great. THANKS! I missed that one the other day, but then forgot about it ... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler compile.c, , 1.120, 1.121
by Sjoerd Mullender 04 Sep '07

04 Sep '07

Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19410 > > Modified Files: > compile.c > Log Message: > -- Cleaned up the SQL code generation. > > -- Completed Manuels checkin in compile.c from 08/21. > > > Index: compile.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/compile.c,v > retrieving revision 1.120 > retrieving revision 1.121 > diff -u -d -r1.120 -r1.121 > --- compile.c 21 Aug 2007 10:17:29 -0000 1.120 > +++ compile.c 4 Sep 2007 10:01:30 -0000 1.121 > @@ -28,6 +28,7 @@ > * Universitaet Muenchen, respectively. All Rights Reserved. > * > * $Id$ > + * $Id$ > */ > > Isn't this overdoing it just a little bit? ;-) -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/tools monetdb.mx, , 1.19, 1.20
by Sjoerd Mullender 31 Aug '07

31 Aug '07

On 2007-08-31 19:14, Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv4192 > > Modified Files: > monetdb.mx > Log Message: > reduce overkill > > > Index: monetdb.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/tools/monetdb.mx,v > retrieving revision 1.19 > retrieving revision 1.20 > diff -u -d -r1.19 -r1.20 > --- monetdb.mx 31 Aug 2007 17:09:13 -0000 1.19 > +++ monetdb.mx 31 Aug 2007 17:14:21 -0000 1.20 > @@ -610,7 +610,6 @@ > } > > if (stats != NULL) { > - char path[8096]; > err e; > > if (stats->state == SABdbRunning) { > @@ -627,11 +626,12 @@ > } > > if (force == 0) { > + char answ; > printf("you are about to remove database %s\n", dbname); > printf("ALL data in this database will get lost, " > "are you sure? [y/N] "); > - scanf("%c", path); > - if (path[0] == 'y' || path[0] == 'Y') { > + scanf("%c", &answ); > + if (answ == 'y' || answ == 'Y') { > /* do it! */ > } else { > printf("battle control terminated\n"); > > Even better would be to use getchar(). scanf is still considerable overkill for reading a character. But also remember to read the rest of the line (also when using scanf). -- Sjoerd Mullender

1 0

[Monetdb-developers] Some help to understand this error output from m5...
by Romulo Goncalves 31 Aug '07

31 Aug '07

I am using monetBD5 with datacell module and I got the following crash. I know it is related with memory map and maybe a valgrind execution would be a nice thing to do. However, I would like more feedback from the experts: insert 1188567930287233: line :[0,3] /ufs/goncalve/scratch/MonetDB/MonetDB5/lib/libmal.so.0[0x2aaaaaace64f] /ufs/goncalve/scratch/MonetDB/MonetDB5/lib/libmal.so.0(reenterMAL+0x7a)[0x2aaaaaac94d5] /ufs/goncalve/scratch/MonetDB/MonetDB5/lib/MonetDB5/lib/lib_datacell.so[0x2aaabccb2207] /lib64/libpthread.so.0[0x33dba062f7] /lib64/libc.so.6(clone+0x6d)[0x33daace86d] ======= Memory map: ======== 00400000-00405000 r-xp 00000000 08:13 9634823 /net/amelia/export/scratch1/goncalve/MonetDB/MonetDB5/bin/mserver5 00604000-00605000 rw-p 00004000 08:13 9634823 /net/amelia/export/scratch1/goncalve/MonetDB/MonetDB5/bin/mserver5 00605000-00d4e000 rw-p 00605000 00:00 0 [heap] 40000000-40001000 ---p 40000000 00:00 0 40001000-40801000 rw-p 40001000 00:00 0 40801000-40802000 ---p 40801000 00:00 0 40802000-41002000 rw-p 40802000 00:00 0 41002000-41003000 ---p 41002000 00:00 0 41003000-41803000 rw-p 41003000 00:00 0 41803000-41804000 ---p 41803000 00:00 0 41804000-42004000 rw-p 41804000 00:00 0 42004000-42005000 ---p 42004000 00:00 0 42005000-42805000 rw-p 42005000 00:00 0 42805000-42806000 ---p 42805000 00:00 0 42806000-43006000 rw-p 42806000 00:00 0 43006000-43007000 ---p 43006000 00:00 0 43007000-43807000 rw-p 43007000 00:00 0 33da600000-33da61a000 r-xp 00000000 08:15 2578621 /lib64/ld-2.5.so 33da819000-33da81a000 r--p 00019000 08:15 2578621 /lib64/ld-2.5.so 33da81a000-33da81b000 rw-p 0001a000 08:15 2578621 /lib64/ld-2.5.so 33daa00000-33dab46000 r-xp 00000000 08:15 2578761 /lib64/libc-2.5.so 33dab46000-33dad46000 ---p 00146000 08:15 2578761 /lib64/libc-2.5.so 33dad46000-33dad4a000 r--p 00146000 08:15 2578761 /lib64/libc-2.5.so 33dad4a000-33dad4b000 rw-p 0014a000 08:15 2578761 /lib64/libc-2.5.so 33dad4b000-33dad50000 rw-p 33dad4b000 00:00 0 33dae00000-33dae82000 r-xp 00000000 08:15 2578841 /lib64/libm-2.5.so 33dae82000-33db081000 ---p 00082000 08:15 2578841 /lib64/libm-2.5.so 33db081000-33db082000 r--p 00081000 08:15 2578841 /lib64/libm-2.5.so 33db082000-33db083000 rw-p 00082000 08:15 2578841 /lib64/libm-2.5.so 33db200000-33db202000 r-xp 00000000 08:15 2578887 /lib64/libdl-2.5.so 33db202000-33db402000 ---p 00002000 08:15 2578887 /lib64/libdl-2.5.so 33db402000-33db403000 r--p 00002000 08:15 2578887 /lib64/libdl-2.5.so 33db403000-33db404000 rw-p 00003000 08:15 2578887 /lib64/libdl-2.5.so 33db600000-33db614000 r-xp 00000000 08:15 4234966 /usr/lib64/libz.so.1.2.3 33db614000-33db813000 ---p 00014000 08:15 4234966 /usr/lib64/libz.so.1.2.3 33db813000-33db814000 rw-p 00013000 08:15 4234966 /usr/lib64/libz.so.1.2.3 33dba00000-33dba15000 r-xp 00000000 08:15 2578924 /lib64/libpthread-2.5.so 33dba15000-33dbc14000 ---p 00015000 08:15 2578924 /lib64/libpthread-2.5.so 33dbc14000-33dbc15000 r--p 00014000 08:15 2578924 /lib64/libpthread-2.5.so 33dbc15000-33dbc16000 rw-p 00015000 08:15 2578924 /lib64/libpthread-2.5.so 33dbc16000-33dbc1a000 rw-p 33dbc16000 00:00 0 33dce00000-33dce35000 r-xp 00000000 08:15 4225510 /usr/lib64/libreadline.so.5.1 33dce35000-33dd034000 ---p 00035000 08:15 4225510 /usr/lib64/libreadline.so.5.1 33dd034000-33dd03c000 rw-p 00034000 08:15 4225510 /usr/lib64/libreadline.so.5.1 33dd03c000-33dd03d000 rw-p 33dd03c000 00:00 0 33de200000-33de203000 r-xp 00000000 08:15 2578939 /lib64/libtermcap.so.2.0.8 33de203000-33de402000 ---p 00003000 08:15 2578939 /lib64/libtermcap.so.2.0.8 33de402000-33de403000 rw-p 00002000 08 Program received signal SIGABRT, Aborted. [Switching to Thread 1124096320 (LWP 24972)] 0x00000033daa30065 in raise () from /lib64/libc.so.6 (gdb) up #1 0x00000033daa31b00 in abort () from /lib64/libc.so.6 (gdb) up #2 0x00000033daa6825b in __libc_message () from /lib64/libc.so.6 (gdb) up #3 0x00000033daa6f504 in _int_free () from /lib64/libc.so.6 (gdb) up #4 0x00000033daa72b2c in free () from /lib64/libc.so.6 (gdb) up #5 0x00002aaaab40b504 in GDKfree (blk=0xa9e948) at /ufs/goncalve/MonetDB/MonetDB/src/gdk/gdk_utils.mx:1140 1140 free(((char *) s) - 8); (gdb) up #6 0x00002aaaab36e349 in HEAPfree (h=0xac36a0) at /ufs/goncalve/MonetDB/MonetDB/src/gdk/gdk_heap.mx:265 265 GDKfree(h->base); (gdb) up #7 0x00002aaaab4a907a in heapfree (src=0xac36a0, dst=0x43004290) at /ufs/goncalve/MonetDB/MonetDB/src/gdk/gdk_bat.mx:630 630 HEAPfree(src); Regards, Romulo

2 2

[Monetdb-developers] Release plans
by Sjoerd Mullender 29 Aug '07

29 Aug '07

We are planning to start a new "Stable" branch in about one to two weeks time in preparation for a new release. This new release will consist of all packages. I would like to ask all developers to be conservative with your checkins until that time. This does not mean you're not allowed to check in any fixes or new features, just that I would like to urge you to make extra sure that your changes work. Also, by checking in any changes before we create this new branch, you commit yourself to making sure that the features work flawlessly in two weeks' time. This new release will not contain many new features. It is mostly a bug-fix release with some new stuff. We wanted to do a partial release, but can't because of slight incompatibilities between the current Development and Stable branches, so we need to release either nothing or everything. We've chosen everything. By the way, this is independent from the SQL bug fix release which is going to be released today (and hence will be superseded in a few weeks time). -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapilib Mapi.mx, , 1.19, 1.20
by Sjoerd Mullender 28 Aug '07

28 Aug '07

Shouldn't you compile *before* you check in? Romulo Goncalves wrote: > Update of /cvsroot/monetdb/clients/src/mapilib > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv7913 > > Modified Files: > Mapi.mx > Log Message: > Include missing library (sys/timeb) to resolve compilation problems on windows. > > > Index: Mapi.mx > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapilib/Mapi.mx,v > retrieving revision 1.19 > retrieving revision 1.20 > diff -u -d -r1.19 -r1.20 > --- Mapi.mx 24 Aug 2007 15:18:29 -0000 1.19 > +++ Mapi.mx 28 Aug 2007 08:50:07 -0000 1.20 > @@ -701,6 +701,11 @@ > #include <string.h> > #include <memory.h> > > +/*additional definitions for date and time/* > +#ifdef HAVE_FTIME > +#include <sys/timeb.h> > +#endif > + > #if TIME_WITH_SYS_TIME > # include <sys/time.h> > # include <time.h> > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

3 4

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.73, 1.74
by Sjoerd Mullender 28 Aug '07

28 Aug '07

Wrong on (almost) all counts. It's amazing. Martin Kersten wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv10235 > > Modified Files: > MapiClient.mx > Log Message: > Move the mode change into one place. Set the default rendering of MAL to This I had already done in the previous checkin, but better because more complete. > align with the console. Best option still is to make mclient part of This is debatable. > the server too. This has *nothing* to do with this checkin (and is debatable anyway). > > > Index: MapiClient.mx > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapiclient/MapiClient.mx,v > retrieving revision 1.73 > retrieving revision 1.74 > diff -u -d -r1.73 -r1.74 > --- MapiClient.mx 28 Aug 2007 07:43:07 -0000 1.73 > +++ MapiClient.mx 28 Aug 2007 08:03:57 -0000 1.74 > @@ -243,6 +243,15 @@ > } > > static void > +modeChange(char *reply){ > + if (strstr(reply, "mdb>#EOD")){ > + setPrompt(); > + } else > + if(strncmp(reply,"mdb>",4)==0) > + sprintf(promptbuf,"mdb>"); > +} > + > +static void > SQLsetSpecial(const char *command) > { > if (mode == SQL && command) { > @@ -371,6 +380,7 @@ > do { > if ((reply = fetch_line(hdl)) == NULL) > return 0; > + modeChange(reply); > } while (*reply != '[' && *reply != '='); > return mapi_split_line(hdl); > } Now the change is checked twice. > @@ -412,7 +422,8 @@ > { > char *line; > > - while ((line = fetch_line(hdl)) != 0) { > + while ((line = mapi_fetch_line(hdl)) != 0) { > + modeChange(line); > if (*line == '=') > line++; > fprintf(toConsole, "%s\n", line); Why did you undo my change which did this already? > @@ -1310,7 +1321,7 @@ > continue; > } > case '?': > - if (!isspace((int) line[2]) && mode == MAL) { > + if (mode==MAL ||debugMode() ){ > strcpy(line, line + 1); > break; > } This is the only part of the checkin that has merit, except a comment that the line with ? is sent to the server might be useful. > @@ -1756,7 +1767,8 @@ > > /* default formatter depends on whether we're interactive */ > if (formatter == NOformatter) > - formatter = interactive && interactive_stdin && mode != XQUERY ? TABLEformatter : RAWformatter; > + formatter = interactive && interactive_stdin && mode != XQUERY ? > + (mode==MAL?RAWformatter: TABLEformatter) : RAWformatter; > > if (command) { > /* execute from command-line */ > > If you really insist that in MAL mode the default should be RAW, then write it like this: formatter = interactive && interactive_stdin && mode != XQUERY && mode != MAL ? TABLEformatter : RAWformatter; > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 2

Re: [Monetdb-developers] C-Store as MySQL storage engine
by Daniel J Abadi 26 Aug '07

26 Aug '07

Hi Dossy, There is no source repository/patch submission system in C-Store like you'd see in a mature open source community project. C-Store is only designed to be used as tool by academic researchers to look at the properties of column-oriented database query execution. It is far from a complete database system and parts of the code are quite rudimentary. Further, it has been neglected recently as many of the people who used to code for C-Store are now coding for Vertica. However, I like the idea of using a column-oriented database as a storage engine for MySQL. The thought actually crossed my mind as well recently when I read about Solid using their database as a storage engine for MySQL. My recommendation is that you look into the Monet instead of C-Store, however (http://monetdb.cwi.nl/) They have a far more complete open source column-oriented database. BTW: I came across your blog entry on Vertica recently. IMO, Vertica is much more than Sybase IQ, with far more advanced compression schemes, and the query executer is better able to take advantage of the column-oriented layout (data is kept in columns as long as possible and operated on as such). I think Vertica is about 10 times as fast as IQ. Your blog entry says you have been in contact recently with the director of sales at Vertica who can probably give you more information. D On Fri, 24 Aug 2007, Dossy Shiobara wrote: > Hi! > > Thanks for the work on C-Store and releasing it as open source under the > BSD license. > > Is there a source repository for the project? What's the process for > submitting patches, etc.? > > I'm currently working on using/adapting C-Store as a storage engine for > MySQL. I anticipate needing to make a series of changes to the C-Store > 0.2 code to make this possible and I'd like to try and get those into > the official source tree. > > Thanks, > > -- Dossy > > -- > Dossy Shiobara | dossy(a)panoptic.com | http://dossy.org/ > Panoptic Computer Network | http://panoptic.com/ > "He realized the fastest way to change is to laugh at your own > folly -- then you can let go and quickly move on." (p. 70) > >

1 0

Re: [Monetdb-developers] Monet crashes with {sum} and {avg}
by Stefan Manegold 24 Aug '07

24 Aug '07

On Fri, Aug 24, 2007 at 11:00:05AM -0400, Agustin Schapira wrote: > Hi Niels, > > Sorry for the late reply --I have been travelling. > > Thanks a lot for the fix. I'm glad that the bug was easy to find and > correct. Which file should I change in the Mars-SuperBall > distribution? Is it > MonetDB4-server/MonetDB4/src/monet/monet_multiplex.mx yep, that's the file. Please let us konw, in case there are any problems with the patch --- I'm not sure, whether there might have been other changes to this file in CVS since the tarbaal was made. > And then should I re-run the monetdb-install.sh script? By the way, > is it possible to can have the script do all the make in a directory > other than /tmp, so I can keep the intermediate files around and I > don't have to recompile EVERYTHING whenever I make a change? I'm not familiar at all with the monetdb-install.sh script, but Niels and/or Fabian can probably help. Stefan > Thanks for the quick fix! > > -- Agustin > > > > >Hi Agustin, > > > >You found an interesting bug. The interpret_setaggr implementation has > >indeed many cases. One set of cases uses an intermediate bat in > >a strait forward (normal inserts) way. This case is the one you > >trigger. > >As soon as many values exist in your input bat the intermediate may > >need extending which happens. > >Now many of the other cases don't used the nested bat in such a > >clean way, but do 'dirty' tricks, such as reusing the heaps of your > >input > >bat. Therefor at the end of interpret_setaggr the nested (bats) are > >cleared by restoring the initial contend. > > > >As you can see restoring the 'initial' contend of the normal > >inserts with > >extends is a problem. I now have a fix for this which I hope to > >check in > >soon. > > > > > >Index: monet_multiplex.mx > >=================================================================== > >RCS file: /cvsroot/monetdb/MonetDB4/src/monet/monet_multiplex.mx,v > >retrieving revision 1.4 > >diff -u -r1.4 monet_multiplex.mx > >--- monet_multiplex.mx 20 Feb 2007 11:32:16 -0000 1.4 > >+++ monet_multiplex.mx 22 Aug 2007 19:06:44 -0000 > >@@ -1430,7 +1430,7 @@ > > BAT *b, *nested, *nested_rev, *histo = 0, *extent = 0, *ret > >= 0; > > size_t minpos; > > int ret_val = -1, varsize, head_type, hidden_vartype = 0; > >- int prop_key, prop_sorted = 0; > >+ int prop_key, prop_sorted = 0, restore = 1; > > GDKfcn direct_call = NULL, packed_call = NULL; > > ValRecord argv_bak; > > BUN p, q, r, s; > >@@ -1688,6 +1688,7 @@ > > assert(batbuns->base!=(ptr)1); > > nested->batDeleted = nested->batInserted = batbuns- > >>base; > > > >+ restore = 0; > > if (direct_call == NULL) { > > if (GDKdebug & 131072) > > THRprintf(GDKout, "setaggr impl: > >non-optimized hash\n"); > >@@ -1814,8 +1815,16 @@ > > argv[1] = argv_bak; > > > > /* undo all our hacks in nested before freeing it */ > >- *(BATstore*) nested = nested_bak; > >- *(BAT*) nested_rev = nested_rev_bak; > >+ if (restore) { > >+ *(BATstore*) nested = nested_bak; > >+ *(BAT*) nested_rev = nested_rev_bak; > >+ } else { > >+ BATstore *n = (BATstore*)nested; > >+ nested->H = &n->H; > >+ nested->T = &n->T; > >+ nested_rev->H = &n->T; > >+ nested_rev->T = &n->H; > >+ } > > BBPreclaim(nested); > > > > if (ret_val < 0) { > > > > > > > >Niels > >> > >>I am having more difficulties with {sum} and {avg}. The problems > >>happen both on 4.16.2 and 4.18.0, on Linux and OS X, compiled with > >>32- and 64- bits, with and without optimizations. > >> > >>I have been able to isolate the problem and write a simple test case > >>that shows the error. The attached file has the contents of a BAT; if > >>you save it and then run the following script: > >> > >>module(ascii_io); > >>var b:=bat(str,int).import(<filename>); > >>var x:={sum}(b); > >> > >>then you will get the following error: > >> > >>*** glibc detected *** double free or corruption (!prev): > >>0x00000000006a6360 *** > >> > >>and Monet will crash. > >> > >>The failing free is called from line 1819 of MonetDB4/src/monet/ > >>monet_multiplex.mx. It's inside the 'interpret_setaggr' function, and > >>it happens only with the 'non-optimized hash' implementation of > >>setaggr starting on line 1694 (in fact, if you sort the BAT before > >>calling {sum}, there will be no error. The same is true for example > >>when you call {count}, which uses a different implementation of > >>setaggr to process the {}). > >> > >>The failing free appears within the 'bunins_failed' label (line > >>1800), which in turn is called by the loop that processes the > >>aggregate when, I think, its call to 'bunins_unary' fails. Here's the > >>failing code, starting on line 1694: > >> > >> BATloopFast(extent, p, q, yy) { > >> [...] > >> if (BATprepareHash(b)) > >> goto bunins_failed; > >> HASHloop(b, b->hhash, hh, last) { > >> r = BUNptr(b, hh); > >> @:bunins_unary(r,hh)@ > >> } > >> [...] > >> } > >> > >>I don't understand why @:bunins_unary(r,hh)@ should fail. I have > >>noticed, however, that if I remove the last couple of lines from the > >>attached file then the call to {sum} doesn't crash. These last two > >>lines are exactly the same as others, so it's not their particular > >>value that matters, but rather the number of rows in the BAT with the > >>"Other" head value. > >> > >>Do you have any suggestions? Any other information that I can provide > >>to help you fix the problem? > >> > >>Thanks a lot again, > >> > >>-- Agustin > >> > >> > >>PS: On OS X, instead of the crash you get an error message "malloc: > >>*** Deallocation of a pointer not malloced: 0x281ee00; This could be > >>a double free(), or free() called with the middle of an allocated > >>block", but for all practical purposes it's the same: the free() is > >>called from the same place in interpret_setaggr. The difference is > >>just in the way the Mac equivalent of glibc handles the double free. > >> > > > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Course",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"Faculty",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"ResearchProject",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Staff",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Student",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > >>"Other",1 > > > >> > >> > >> > > > >>--------------------------------------------------------------------- > >>---- > >>This SF.net email is sponsored by: Splunk Inc. > >>Still grepping through log files to find problems? Stop. > >>Now Search log events and configuration files using AJAX and a > >>browser. > >>Download your FREE copy of Splunk now >> http://get.splunk.com/ > >>_______________________________________________ > >>Monetdb-developers mailing list > >>Monetdb-developers(a)lists.sourceforge.net > >>https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > > >-- > > > >Niels Nes, Centre for Mathematics and Computer Science (CWI) > >Kruislaan 413, 1098 SJ Amsterdam, The Netherlands > >room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 > >url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Monet crashes with {sum} and {avg}
by Agustin Schapira 22 Aug '07

22 Aug '07

Hi, I am having more difficulties with {sum} and {avg}. The problems happen both on 4.16.2 and 4.18.0, on Linux and OS X, compiled with 32- and 64- bits, with and without optimizations. I have been able to isolate the problem and write a simple test case that shows the error. The attached file has the contents of a BAT; if you save it and then run the following script: module(ascii_io); var b:=bat(str,int).import(<filename>); var x:={sum}(b); then you will get the following error: *** glibc detected *** double free or corruption (!prev): 0x00000000006a6360 *** and Monet will crash. The failing free is called from line 1819 of MonetDB4/src/monet/ monet_multiplex.mx. It's inside the 'interpret_setaggr' function, and it happens only with the 'non-optimized hash' implementation of setaggr starting on line 1694 (in fact, if you sort the BAT before calling {sum}, there will be no error. The same is true for example when you call {count}, which uses a different implementation of setaggr to process the {}). The failing free appears within the 'bunins_failed' label (line 1800), which in turn is called by the loop that processes the aggregate when, I think, its call to 'bunins_unary' fails. Here's the failing code, starting on line 1694: BATloopFast(extent, p, q, yy) { [...] if (BATprepareHash(b)) goto bunins_failed; HASHloop(b, b->hhash, hh, last) { r = BUNptr(b, hh); @:bunins_unary(r,hh)@ } [...] } I don't understand why @:bunins_unary(r,hh)@ should fail. I have noticed, however, that if I remove the last couple of lines from the attached file then the call to {sum} doesn't crash. These last two lines are exactly the same as others, so it's not their particular value that matters, but rather the number of rows in the BAT with the "Other" head value. Do you have any suggestions? Any other information that I can provide to help you fix the problem? Thanks a lot again, -- Agustin PS: On OS X, instead of the crash you get an error message "malloc: *** Deallocation of a pointer not malloced: 0x281ee00; This could be a double free(), or free() called with the middle of an allocated block", but for all practical purposes it's the same: the free() is called from the same place in interpret_setaggr. The difference is just in the way the Mac equivalent of glibc handles the double free.

2 1

[Monetdb-developers] (no subject)
by Chantola Kit 22 Aug '07

22 Aug '07

Hi MonetDB-developers: I have tried to install Pathfinder on my Fedora7 many times and used all the kind of installations from both Tarball and sourceforge, but I fail to install it and always get error as bellow: grep: /home/sjoerd/Monet-Stable/lib/libbat.la: No such file or directory /bin/sed: can't read /home/sjoerd/Monet-Stable/lib/libbat.la: No such file or directory libtool: link: `/home/sjoerd/Monet-Stable/lib/libbat.la' is not a valid libtool archive make[4]: *** [lib_pf_support.la] Error 1 make[4]: Leaving directory `/root/Desktop/MonetDBInstall/runtime' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/root/Desktop/MonetDBInstall/runtime' make[2]: *** [all] Error 2 make[2]: Leaving directory `/root/Desktop/MonetDBInstall/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/Desktop/MonetDBInstall' make: *** [all] Error 2 It would be grateful if you can help me to solve this problem. I am looking forward to hearing from you. Tola -- Department of Computer Science Graduate School of Systems and Information Engineering University of Tsukuba Ibaraki, Tsukuba city, Japan

4 4

[Monetdb-developers] SQL Stable branch is closed: it is being prepared for bug-fix release
by Sjoerd Mullender 21 Aug '07

21 Aug '07

From today, the Stable branch of sql is closed for all checkins unless approved by myself or Stefan. We are going to prepare a bug-fix release. Note that the development branch (HEAD branch) is open for normal business. -- Sjoerd Mullender

2 1

[Monetdb-developers] Pathfinder installation error
by Chantola Kit 21 Aug '07

21 Aug '07

Dear MonetDBs: I have problem of installing pathfinder. Even though I try to install all MonetDBs such as MonetDB, MonetDB-client, MonetDB4-server, MonetDB5-server, MonetDB-SQL, I cannot install MonetDB4-XQuery. Every time, I use command "make" for Pathfinder, I got some errors which unable me to install Pathfinder as follow: grep: /home/sjoerd/Monet-Stable/lib/libbat.la: No such file or directory /bin/sed: can't read /home/sjoerd/Monet-Stable/lib/libbat.la: No such file or directory libtool: link: `/home/sjoerd/Monet-Stable/lib/libbat.la' is not a valid libtool archive make[4]: *** [lib_pf_support.la] Error 1 make[4]: Leaving directory `/root/Desktop/MonetDBInstall/runtime' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/root/Desktop/MonetDBInstall/runtime' make[2]: *** [all] Error 2 make[2]: Leaving directory `/root/Desktop/MonetDBInstall/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/Desktop/MonetDBInstall' make: *** [all] Error 2 I am looking forward to hearing from you. Best regards, Tola -- Department of Computer Science Graduate School of Systems and Information Engineering University of Tsukuba Ibaraki, Tsukuba city, Japan

2 4

[Monetdb-developers] Installation problem
by Chantola Kit 17 Aug '07

17 Aug '07

Dear monetdb developers: I try to install MonetDB many times on Fedora7 using MonetDB-Mars-SuperBall, but I have the same problem which unables me to connect to server. 1. When I go to Mserver it was: [root@localhost ~]# Mserver # Monet Database Server V4.18.0 # Copyright (c) 1993-2007, CWI. All rights reserved. # Compiled for i686-redhat-linux-gnu/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. MonetDB> But, then, I open MapiClient, there is an error like below: [root@localhost ~]# MapiClient -q MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !MALException:setScenario:Scenario 'mil' not initialized 2. When I go to mserver5, it was: MonetDB>[root@localhost ~]# mserver5 # MonetDB Server v5.0.0 # Copyright (c) 1993-2007 CWI, all rights reserved # Compiled for i686-redhat-linux-gnu/32bit with 32bit OIDs dynamically linked # dbname:demo # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /usr/local/etc/monetdb5.conf) > Then, I open MapiClient: [root@localhost ~]# MapiClient -q MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !MALException:setScenario:Scenario 'mil' not initialized [root@localhost ~]# Could you help me to solve the problems? I am looking forward to hearing from you. Tola -- Department of Computer Science Graduate School of Systems and Information Engineering University of Tsukuba Ibaraki, Tsukuba city, Japan

3 3

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.54, 1.55
by Fabian Groffen 16 Aug '07

16 Aug '07

On 16-08-2007 18:49:48 +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14690 > > Modified Files: > MapiClient.mx > Log Message: > > > Index: MapiClient.mx This commit looks pretty dangerous. Can you elaborate/explain?

2 1

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java/scripts Makefile.ag, Clients_1-18, 1.3, 1.3.6.1 mjclient.bat.in, Clients_1-18, 1.1, 1.1.6.1 mjclient.in, Clients_1-18, 1.3, 1.3.6.1
by Sjoerd Mullender 16 Aug '07

16 Aug '07

Fabian wrote: > Update of /cvsroot/monetdb/clients/src/java/scripts > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31067/scripts > > Modified Files: > Tag: Clients_1-18 > Makefile.ag mjclient.bat.in mjclient.in > Log Message: > Merged HEAD into Clients_1-18 These checkins look like *incompatible* changes to the stable branch. That is not allowed. The whole idea of the stable branch is that you don't have to worry about changes, you can just use the old code with the new version. If you change the name of a file, you can't do that. > > Index: mjclient.bat.in > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/scripts/mjclient.bat.in,v > retrieving revision 1.1 > retrieving revision 1.1.6.1 > diff -u -d -r1.1 -r1.1.6.1 > --- mjclient.bat.in 8 Jan 2007 14:03:39 -0000 1.1 > +++ mjclient.bat.in 16 Aug 2007 10:52:44 -0000 1.1.6.1 > @@ -21,4 +21,4 @@ > @rem Future versions might inspect a database of eligible servers > @rem to direct the interaction to the proper ports. > > -call java -jar @datadir@\MonetDB\lib\@JDBCCLIENT_JAR@ %* > +call java -jar @datadir@\MonetDB\lib\jdbcclient.jar %* > > Index: mjclient.in > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/scripts/mjclient.in,v > retrieving revision 1.3 > retrieving revision 1.3.6.1 > diff -u -d -r1.3 -r1.3.6.1 > --- mjclient.in 3 Jan 2007 12:17:27 -0000 1.3 > +++ mjclient.in 16 Aug 2007 10:52:44 -0000 1.3.6.1 > @@ -23,4 +23,4 @@ > # to direct the interaction to the proper ports. > > prefix=@prefix@ > -exec java -jar @datadir@/MonetDB/lib/@JDBCCLIENT_JAR@ $* > +exec java -jar @datadir@/MonetDB/lib/jdbcclient.jar $* > > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/scripts/Makefile.ag,v > retrieving revision 1.3 > retrieving revision 1.3.6.1 > diff -u -d -r1.3 -r1.3.6.1 > --- Makefile.ag 8 Jan 2007 14:03:39 -0000 1.3 > +++ Makefile.ag 16 Aug 2007 10:52:43 -0000 1.3.6.1 > @@ -17,12 +17,14 @@ > MTSAFE > > scripts_sh = { > - NOINST > + COND = NOT_WIN32 > + DIR = bindir > EXT = "" > SOURCES = mjclient.in > } > > scripts_bat = { > - NOINST > + COND = NATIVE_WIN32 > + DIR = bindir > SOURCES = mjclient.bat.in > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.41, 1.42
by Martin Kersten 15 Aug '07

15 Aug '07

Fabian Groffen wrote: > On 15-08-2007 11:24:47 +0000, Martin Kersten wrote: >> - case 'e': >> + case 'E': >> echoquery = 1; >> continue; >> case 'f': >> @@ -1253,7 +1243,7 @@ >> setFormatter(s); >> continue; >> } >> - case 'x': >> + case 'e': >> exit_on_error = 1; > > I suggest haveing -e for echo. > > exit_on_error should not be necessary, if you assume the following: > 1) the language being used has control for errors itself, e.g. SQL's > auto-commit, or > 2) an error should not happen in batches, thus > 3) the client continues executing when reading from stdin, but dies on > the first error when reading from file > > Don't know how much this would affect testweb, but somewhere I feel it > is only in an interactive session sensible to continue after an error. According to Stefan this does not affect the test web. The feature will be dropped on the next commit

3 5

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.40, 1.41
by Martin Kersten 15 Aug '07

15 Aug '07

Fabian Groffen wrote: > On 15-08-2007 08:23:20 +0000, Martin Kersten wrote: >> Update of /cvsroot/monetdb/clients/src/mapiclient >> In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26440 >> >> Modified Files: >> MapiClient.mx >> Log Message: >> Blockmode is an interaction feature for MAL. It allows you to enter multiple >> lines before it is being set for execution. Sending is triggered by an empty >> line. > > Ehm... I thought we killed linemode some releases ago, and only support > blockmode these days. Yes. But the option was still there and not working. The feature was relevant for MIL, because its reader wasn't too clever. Then postponing execution was an option. In MAL you can now enter multiple lines > mal>\b mal>io.print(1); more>io.print(2); more> [ 1 ] [ 2 ] I am not sure if this really useful, because it raises the issue that you might want to call an editor on the buffer to correct it. > So maybe you should call it "Batchmode" instead of "Blockmode" to avoid > confusion. good point. But perhaps we should drop it completely. > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapilib Mapi.mx, , 1.13, 1.14
by Martin Kersten 13 Aug '07

13 Aug '07

Fabian Groffen wrote: > On 13-08-2007 14:16:28 +0000, Martin Kersten wrote: >> Update of /cvsroot/monetdb/clients/src/mapilib >> In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv20770 >> >> Modified Files: >> Mapi.mx >> Log Message: >> It is handy to gain access to the query from the handler. The MapiClient program is in flux. It functionality is quickly made on par with what is available for other systems. Rendering a tabular display is probably the most important feature, but also help functionality for MAL programmers is worth mentioning. > > The -e switch in JdbcClient does not print the full query per se, it > echos for each line read, just that very line, before it sends it to the > server, if it does at all. This way you can see results appearing right > after a query, which in multi-statment queries may differ from the full > query. There will/should be a simple version of \t that will do the job on a line by line basis. The statement boundaries are not easy to detect. > > I guess the difference is not that important though. For test scripts > (Mtest in particular) it is an extremely useful feature to be able to > see to what query the given result corresponds. yes. Given the fact that Mtest ignores lines starting with # makes it possible to realize this feature. A few more steps are needed to make MapiClient the default textual interface for all. The request list includes: CVS and TSV output. Or even XML and HTML. Inclusion of an easy logging. So, a few more things to do during this summer activity. > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient dump.c, , 1.7, 1.8
by Sjoerd Mullender 12 Aug '07

12 Aug '07

On 08/12/2007 08:39 PM, Martin Kersten wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv25741 > > Modified Files: > dump.c > Log Message: > Remove duplicate rows. This solves the duplicates seen > when typing '\d idxs'. Is this the right thing to do? The problem, it seems to me, is that idxs occurs twice in the tables table (select * from tables where name = 'idxs'; returns two entries). Is that correct? > Index: dump.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapiclient/dump.c,v > retrieving revision 1.7 > retrieving revision 1.8 > diff -u -d -r1.7 -r1.8 > --- dump.c 10 Aug 2007 11:48:59 -0000 1.7 > +++ dump.c 12 Aug 2007 18:39:52 -0000 1.8 > @@ -79,7 +79,7 @@ > fprintf(toConsole, " (\n"); > > snprintf(query, maxquerylen, > - "SELECT \"c\".\"name\"," /* 0 */ > + "SELECT DISTINCT \"c\".\"name\"," /* 0 */ > "\"c\".\"type\"," /* 1 */ > "\"c\".\"type_digits\"," /* 2 */ > "\"c\".\"type_scale\"," /* 3 */ > @@ -212,7 +212,7 @@ > } > mapi_close_handle(hdl); > snprintf(query, maxquerylen, > - "SELECT \"kc\".\"column\"," /* 0 */ > + "SELECT DISTINCT \"kc\".\"column\"," /* 0 */ > "\"kc\".\"nr\", " /* 1 */ > "\"k\".\"name\" " /* 2 */ > "FROM \"keycolumns\" \"kc\", " > @@ -267,7 +267,7 @@ > mapi_close_handle(hdl); > > snprintf(query, maxquerylen, > - "SELECT \"kc\".\"column\"," /* 0 */ > + "SELECT DISTINCT \"kc\".\"column\"," /* 0 */ > "\"kc\".\"nr\", " /* 1 */ > "\"k\".\"name\" " /* 2 */ > "FROM \"keycolumns\" \"kc\", " > @@ -325,7 +325,7 @@ > mapi_close_handle(hdl); > > snprintf(query, maxquerylen, > - "SELECT \"pkt\".\"name\"," /* 0 */ > + "SELECT DISTINCT \"pkt\".\"name\"," /* 0 */ > "\"pkkc\".\"column\"," /* 1 */ > "\"fkkc\".\"column\"," /* 2 */ > "\"fkkc\".\"nr\"," /* 3 */ > @@ -432,7 +432,7 @@ > fprintf(toConsole, ");\n"); > > snprintf(query, maxquerylen, > - "SELECT \"i\".\"name\", " /* 0 */ > + "SELECT DISTINCT \"i\".\"name\", " /* 0 */ > "\"k\".\"name\", " /* 1 */ > "\"kc\".\"nr\", " /* 2 */ > "\"c\".\"name\" " /* 3 */ > @@ -576,13 +576,13 @@ > { > const char *start = "START TRANSACTION"; > const char *end = "COMMIT"; > - const char *sequences1 = "SELECT \"name\" FROM \"sequences\""; > - const char *sequences2 = "SELECT seq.\"name\",get_value_for(s.\"name\",seq.\"name\"),\"minvalue\",\"maxvalue\",\"increment\",\"cycle\" FROM \"sequences\" seq, \"schemas\" s WHERE s.id = seq.schema_id"; > + const char *sequences1 = "SELECT DISTINCT \"name\" FROM \"sequences\""; > + const char *sequences2 = "SELECT DISTINCT seq.\"name\",get_value_for(s.\"name\",seq.\"name\"),\"minvalue\",\"maxvalue\",\"increment\",\"cycle\" FROM \"sequences\" seq, \"schemas\" s WHERE s.id = seq.schema_id"; > const char *tables = "SELECT \"name\" FROM \"_tables\" WHERE " > "\"type\" = 0 AND \"system\" = FALSE"; > - const char *views = "SELECT \"name\",\"query\" FROM \"_tables\" WHERE " > + const char *views = "SELECT DISTINCT \"name\",\"query\" FROM \"_tables\" WHERE " > "\"type\" = 1 AND \"system\" = FALSE"; > - const char *functions = "SELECT \"func\" FROM \"functions\" WHERE " > + const char *functions = "SELECT DISTINCT \"func\" FROM \"functions\" WHERE " > "\"sql\" = TRUE"; > MapiHdl hdl; > int rc = 0; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.8, 1.9
by Romulo Goncalves 09 Aug '07

09 Aug '07

Martin Kersten wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19893 > > Modified Files: > MapiClient.mx > Log Message: > MapiClient has been extended with a SQL rendering option based on > a predefined columnwidth and pagesize. > A columnwidth= -1 means old display structure. > Columnwidth=0 indicates to respect the sizes in the table headers. > Columnwidth>0 is an upperbound. Broad columns are pushed to make > all fit. The long values are displayed over multiple lines. If columnwidth is bigger than the table headers it equivalent to set it to zero. For columnwidth=5 and columnswith=10 I do not see any differences in the output, why? The error output is different. [goncalve@amelia src]$ MapiClient -lsql --columnwidth=-1 --pagesize=-1 sql>select * from table; MAPI = monetdb@localhost:50000 QUERY = select * from table; ERROR = !syntax error, unexpected TABLE in: "select * from table" sql>\q [goncalve@amelia src]$ MapiClient -lsql --columnwidth=0 --pagesize=-1 sql>select * from table; !syntax error, unexpected TABLE in: "select * from table" Timer 0.427 msec sql>\q [goncalve@amelia src]$ Another interesting point is the response time. If I use: columnwidth=0 the response time is around 8 ms columnwidth=10 the response time is around 13 ms Note: I am looking to the response time printed in the MapiClient. There are not special measures made by me. > > The pagesize default is -1 to indicate the old situation. A pagesize > = 0 indicates no pagination structure, and a pagesize>0 produces > a paginated result, which can be stopped. Well if I use columnwidth=-1 the pagesize does not have any effect. If I load a file the options do not work. MapiClient -lsql --columnwidth=0 --pagesize=10 romulo.sql Where romulo.sql contains the query select * from tables; > > The styling works with the debugger, which now also better indicates > the mode. I will test this after you fix the previous problems. ;) Regards, Romulo The trace mode requires some extra work (for columnwidth>=0) > > The style conventions for the other languages should not have been > changed. (checking is needed) (and multi platform compilation) >

2 2

[Monetdb-developers] glib - double free or corruption
by Agustin Schapira 08 Aug '07

08 Aug '07

Dear Monet developers, We're having trouble with a script that uses the {} operator to compute the avg of a large (4M rows) table: we get a 'double free' error from glib, and Monet crashes. This is using Monet 4.16.2 compiled for 64-bits and 32-bit oids on Linux. (BTW, it also happens if we use {sum}, but not if we use {count}) The code takes two tables, links and attr, and does the equivalent of SELECT avg(attr.value) FROM links, attr WHERE links.id = attr.id GROUP BY links.from Here's the MIL code: # Get the BATs var var_attr:=bat(bat("prox_link_attr").fetch(2)).find(oid(0)); var var_attr_id:=bat(bat(var_attr).fetch(0)); var var_attr_val:=bat(bat(var_attr).fetch(1)); var var_link_id:=bat(bat("prox_link").fetch(0)); var var_link_from:=bat(bat("prox_link").fetch(1)); # Join ATTR x LINK, keep LINK.from, ATTR.val var var_1:=var_link_id.join(var_attr_id.reverse()); var var_2:=var_1.mark(0@0); var var_3:=var_1.reverse().mark(0@0); var var_5:=var_2.reverse().join(var_link_from); var var_8:=var_3.reverse().join(var_attr_val); # GROUP BY LINK.from and compute AVG(ATTR.val) var var_9:=var_5.reverse().join(var_8); var_5.info().print(); var_8.info().print(); var_9.info().print(); var var_10:={avg}(var_9); At the end of the last statement, Monet crashes with the error: *** glibc detected *** double free or corruption (!prev): 0x0000000000632080 *** I've run this using gdb; below is the trace, including the information printed by the .info() calls at the end of the script, and some extra info that is printed with debugmask(32 and 131072). Any thoughts? Any more information that you would need to debug this problem? Thanks a lot, -- Agustin ------------ $ gdb --args /usr/local/Monet-venus-debug/bin/Mserver --dbname xxx ~/ test.mil GNU gdb Red Hat Linux (6.3.0.0-1.143.el4rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db library "/lib64/tls/libthread_db.so.1". (gdb) run Starting program: /usr/local/Monet-venus-debug/bin/Mserver --dbname xxx /home/aschapira/test.mil [Thread debugging using libthread_db enabled] [New Thread 182918188384 (LWP 14037)] [New Thread 1082132832 (LWP 14040)] # Monet Database Server V4.16.2 # Copyright (c) 1993-2007, CWI. All rights reserved. # Compiled for x86_64-redhat-linux-gnu/64bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. #-----------------------------------------# # h t # name # str str # type #-----------------------------------------# [ "version", "25105" ] [ "batId", "tmp_41" ] [ "batCacheid", "-33" ] [ "batParentid", "0" ] [ "batSharecnt", "0" ] [ "head", "oid" ] [ "tail", "oid" ] [ "batPersistence", "transient" ] [ "batRestricted", "read-only" ] [ "batRefcnt", "1" ] [ "batLRefcnt", "1" ] [ "batDirty", "dirty" ] [ "batSet", "0" ] [ "void_tid", "0" ] [ "void_cnt", "0" ] [ "hsorted", "65" ] [ "hident", "t" ] [ "hdense", "1" ] [ "hseqbase", "0@0" ] [ "hkey", "1" ] [ "hloc", "4" ] [ "hvarsized", "0" ] [ "halign", "1008981" ] [ "hnosorted", "0" ] [ "hnosorted_rev", "0" ] [ "hnodense", "0" ] [ "hnokey[0]", "0" ] [ "hnokey[1]", "0" ] [ "tident", "h" ] [ "tdense", "0" ] [ "tseqbase", "nil" ] [ "tsorted", "0" ] [ "tkey", "0" ] [ "tloc", "0" ] [ "tvarsized", "0" ] [ "talign", "1005793" ] [ "tnosorted", "58570" ] [ "tnosorted_rev", "0" ] [ "tnodense", "0" ] [ "tnokey[0]", "0" ] [ "tnokey[1]", "1" ] [ "batInserted", "0" ] [ "batDeleted", "0" ] [ "batFirst", "0" ] [ "top", "4180239" ] [ "batStamp", "-73" ] [ "lastUsed", "24012" ] [ "curStamp", "86" ] [ "batCopiedtodisk", "0" ] [ "batDirtydesc", "dirty" ] [ "batDirtybuns", "clean" ] [ "batBuns.free", "33441912" ] [ "batBuns.size", "33441912" ] [ "batBuns.maxsize", "40173552" ] [ "batBuns.storage", "malloced" ] [ "batBuns.filename", "41.buns" ] [ "hheapdirty", "clean" ] [ "theapdirty", "clean" ] #-----------------------------------------# # h t # name # str str # type #-----------------------------------------# [ "version", "25105" ] [ "batId", "tmp_42" ] [ "batCacheid", "34" ] [ "batParentid", "0" ] [ "batSharecnt", "0" ] [ "head", "oid" ] [ "tail", "int" ] [ "batPersistence", "transient" ] [ "batRestricted", "read-only" ] [ "batRefcnt", "1" ] [ "batLRefcnt", "1" ] [ "batDirty", "dirty" ] [ "batSet", "0" ] [ "void_tid", "-1" ] [ "void_cnt", "0" ] [ "hsorted", "65" ] [ "hident", "h" ] [ "hdense", "1" ] [ "hseqbase", "0@0" ] [ "hkey", "1" ] [ "hloc", "0" ] [ "hvarsized", "0" ] [ "halign", "1009002" ] [ "hnosorted", "0" ] [ "hnosorted_rev", "0" ] [ "hnodense", "0" ] [ "hnokey[0]", "0" ] [ "hnokey[1]", "0" ] [ "tident", "t" ] [ "tdense", "0" ] [ "tseqbase", "0@0" ] [ "tsorted", "0" ] [ "tkey", "0" ] [ "tloc", "4" ] [ "tvarsized", "0" ] [ "talign", "1009003" ] [ "tnosorted", "0" ] [ "tnosorted_rev", "0" ] [ "tnodense", "0" ] [ "tnokey[0]", "0" ] [ "tnokey[1]", "0" ] [ "batInserted", "0" ] [ "batDeleted", "0" ] [ "batFirst", "0" ] [ "top", "4180239" ] [ "batStamp", "-84" ] [ "lastUsed", "24028" ] [ "curStamp", "86" ] [ "batCopiedtodisk", "0" ] [ "batDirtydesc", "dirty" ] [ "batDirtybuns", "clean" ] [ "batBuns.free", "33441912" ] [ "batBuns.size", "35202008" ] [ "batBuns.maxsize", "42270704" ] [ "batBuns.storage", "malloced" ] [ "batBuns.filename", "42.buns" ] [ "hheapdirty", "clean" ] [ "theapdirty", "clean" ] #-----------------------------------------# # h t # name # str str # type #-----------------------------------------# [ "version", "25105" ] [ "batId", "tmp_43" ] [ "batCacheid", "-35" ] [ "batParentid", "0" ] [ "batSharecnt", "0" ] [ "head", "oid" ] [ "tail", "int" ] [ "batPersistence", "transient" ] [ "batRestricted", "read-only" ] [ "batRefcnt", "1" ] [ "batLRefcnt", "1" ] [ "batDirty", "dirty" ] [ "batSet", "0" ] [ "void_tid", "6122832" ] [ "void_cnt", "0" ] [ "hsorted", "0" ] [ "hident", "t" ] [ "hdense", "0" ] [ "hseqbase", "nil" ] [ "hkey", "0" ] [ "hloc", "4" ] [ "hvarsized", "0" ] [ "halign", "1005793" ] [ "hnosorted", "58570" ] [ "hnosorted_rev", "0" ] [ "hnodense", "0" ] [ "hnokey[0]", "0" ] [ "hnokey[1]", "1" ] [ "tident", "h" ] [ "tdense", "0" ] [ "tseqbase", "0@0" ] [ "tsorted", "0" ] [ "tkey", "0" ] [ "tloc", "0" ] [ "tvarsized", "0" ] [ "talign", "1009003" ] [ "tnosorted", "0" ] [ "tnosorted_rev", "0" ] [ "tnodense", "0" ] [ "tnokey[0]", "0" ] [ "tnokey[1]", "0" ] [ "batInserted", "0" ] [ "batDeleted", "0" ] [ "batFirst", "0" ] [ "top", "4180239" ] [ "batStamp", "-85" ] [ "lastUsed", "24043" ] [ "curStamp", "86" ] [ "batCopiedtodisk", "0" ] [ "batDirtydesc", "dirty" ] [ "batDirtybuns", "clean" ] [ "batBuns.free", "33441912" ] [ "batBuns.size", "33441912" ] [ "batBuns.maxsize", "40173552" ] [ "batBuns.storage", "malloced" ] [ "batBuns.filename", "43.buns" ] [ "hheapdirty", "clean" ] [ "theapdirty", "clean" ] *** double free or corruption (!prev): 0x0000000000632080 *** Program received signal SIGABRT, Aborted. [Switching to Thread 182918188384 (LWP 14037)] 0x0000003a7b72e21d in raise () from /lib64/tls/libc.so.6 (gdb) bt #0 0x0000003a7b72e21d in raise () from /lib64/tls/libc.so.6 #1 0x0000003a7b72fa1e in abort () from /lib64/tls/libc.so.6 #2 0x0000003a7b763451 in __libc_message () from /lib64/tls/libc.so.6 #3 0x0000003a7b76906e in _int_free () from /lib64/tls/libc.so.6 #4 0x0000003a7b7693b6 in free () from /lib64/tls/libc.so.6 #5 0x0000002a96665c43 in GDKfree (blk=0x632088) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB/src/ gdk/gdk_utils.mx:1121 #6 0x0000002a965ea1cf in HEAPfree (h=0x631940) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB/src/ gdk/gdk_heap.mx:264 #7 0x0000002a966dc693 in BATdelete (b=0x631828) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB/src/ gdk/gdk_storage.mx:716 #8 0x0000002a965e5e6d in BBPaddtobin (b=0x631828) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB/src/ gdk/gdk_bbp.mx:2366 #9 0x0000002a965e252a in BBPdestroy (b=0x631828) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB/src/ gdk/gdk_bbp.mx:1665 #10 0x0000002a965e12b1 in BBPreclaim (b=0x631828) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB/src/ gdk/gdk_bbp.mx:1514 #11 0x0000002a95ef5e0f in interpret_setaggr (name=0x62a038 "{avg}", argc=2, argv=0x620c18, res=0x7fbffff790, tt=0x631078, stk=1) at /export/scratch0/monet/monet.GNU.64.64.d.14791/ MonetDB4/src/monet/monet_multiplex.mx:1819 #12 0x0000002a95e95c57 in interpret (stk=1, lt=0x625090, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:1202 #13 0x0000002a95e9da04 in interpret_assignment (stk=1, lt=0x625040, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:1842 #14 0x0000002a95e97349 in interpret_var (stk=1, lt=0x625018, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:1329 #15 0x0000002a95e93e30 in interpret (stk=1, lt=0x625018, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:832 #16 0x0000002a95e9dca3 in interpret_seqblock (stk=1, lt=0x5ca8b8, res=0x7fbffff790, scope=0) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:1892 #17 0x0000002a95e93606 in interpret (stk=1, lt=0x629478, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:770 #18 0x0000002a95e9173d in interpret_str (stk=0, buf=0x629908 "# Get the BATs\nvar var_attr:=bat(bat (\"prox_link_attr\").fetch(2)).find(oid(0));\nvar var_attr_id:=bat(bat (var_attr).fetch(0));\nvar var_attr_val:=bat(bat(var_attr).fetch(1)); \n\nvar var_link_id:=bat(bat(\"p"..., res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d. 14791/MonetDB4/src/monet/monet_interpreter.mx:246 #19 0x0000002a95e91bf0 in interpret_file (stk=0, lt=0x5411c8, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:291 #20 0x0000002a95e93f39 in interpret (stk=0, lt=0x5411c8, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_interpreter.mx:858 #21 0x0000002a95ebf65e in handleRequest (t=0x2a96a46840, q=0x64d5a8, res=0x7fbffff790) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_queue.mx:537 ---Type <return> to continue, or q <return> to quit--- #22 0x0000002a95ebfadb in doRequest (t=0x2a96a46840, preference=0x0) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_queue.mx:563 #23 0x0000002a95f1108c in monetInterpreter (status=0x7fbffff7f8) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ monet/monet_process.mx:112 #24 0x0000000000402674 in main (argc=4, av=0x7fbffff908) at /export/scratch0/monet/monet.GNU.64.64.d.14791/MonetDB4/src/ tools/Mserver.mx:403 When I run this with debugmask(32 + 131072), I get the following debug output as well. This is right after the calls to .info(), during the processing of {avg}: #interpret_unpin(print) on bat(59) refcnt = 0 ##60 = new tmp_74(int,int) ##BBPreclaim: bat(60) view=0 lrefs=0 ref=1 stat=1 #interpret_pin({avg}) on bat(-35) refcnt = 1 ##61 = new tmp_75(oid,int) ##62 = new tmp_76(oid,void) ##65 = new tmp_101(oid,void) ##66 = new tmp_102(oid,void) ##67 = new tmp_103(oid,void) ##BBPreclaim: bat(67) view=0 lrefs=0 ref=1 stat=1 ##clear 67 (tmp_103) ##uncache 67 (tmp_103) ##BBPreclaim: bat(66) view=0 lrefs=0 ref=1 stat=1 ##clear 66 (tmp_102) ##uncache 66 (tmp_102) ##BBPreclaim: bat(65) view=0 lrefs=0 ref=1 stat=1 ##clear 65 (tmp_101) ##uncache 65 (tmp_101) ##BBPreclaim: bat(62) view=0 lrefs=0 ref=1 stat=1 ##clear 62 (tmp_76) ##uncache 62 (tmp_76) ##BBPreclaim: bat(61) view=0 lrefs=0 ref=1 stat=1 ##clear 61 (tmp_75) ##uncache 61 (tmp_75) #setaggr impl: non-optimized hash #interpret_pin(count) on bat(61) refcnt = 2 #interpret_unpin(count) on bat(61) refcnt = 1 #interpret_pin(sum_lng) on bat(61) refcnt = 2 #interpret_unpin(sum_lng) on bat(61) refcnt = 1 ... [these 4 lines repeat 20,000 times, which corresponds to the number of unique values in the HEAD of var_9] ... ##BBPreclaim: bat(65) view=0 lrefs=0 ref=1 stat=1 ##clear 65 (tmp_101) ##uncache 65 (tmp_101) ##BBPreclaim: bat(61) view=0 lrefs=0 ref=1 stat=1 I hope this helps. Thanks again.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_bbp.mx, , 1.201, 1.202
by Sjoerd Mullender 06 Aug '07

06 Aug '07

Should this be back-ported to the stable branch? Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv30287/src/gdk > > Modified Files: > gdk_bbp.mx > Log Message: > - race condition in BBOclear > > > > Index: gdk_bbp.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bbp.mx,v > retrieving revision 1.201 > retrieving revision 1.202 > diff -u -d -r1.201 -r1.202 > --- gdk_bbp.mx 22 Jun 2007 06:46:53 -0000 1.201 > +++ gdk_bbp.mx 5 Aug 2007 23:00:50 -0000 1.202 > @@ -1116,7 +1116,7 @@ > THRprintf(GDKout, "#clear %d (%s)\n", (int) i, BBPname(i)); > } > BBPuncacheit_(i, TRUE); > - BBP_status_set(i, 0, "BBPclear"); > + BBP_status_set(i, BBPUNLOADING, "BBPclear"); > BBP_refs(i) = 0; > BBP_lrefs(i) = 0; > if (lock) > @@ -1137,6 +1137,7 @@ > } > if (BBP_logical(i) != BBP[ABS(i)].bak[(i) < 0]) > GDKfree(BBP_logical(i)); > + BBP_status_set(i, 0, "BBPclear"); > BBP_logical(i) = NULL; > BBP_next(i) = BBP_free(idx); > BBP_free(idx) = i; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] Unimplemented XQuery standard functions
by Djoerd Hiemstra 03 Aug '07

03 Aug '07

Dear all, For demo purposes we are in need for implementations of some XQuery standard functions, especially fn:tokenize() and fn:base-uri(). Are there plans for implementing more (or all) standard functions? Best, Djoerd.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/tests/BugsViaSourgeforce/Tests ID.1766414.stable.err, , NONE, 1.1 ID.1766414.stable.out, , NONE, 1.1 ID.1766414.xml, , NONE, 1.1 ID.1766414.xq, , NONE, 1.1 All, , 1.34, 1.35
by Stefan Manegold 02 Aug '07

02 Aug '07

On Thu, Aug 02, 2007 at 08:42:23PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/tests/BugsViaSourgeforce/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv13362 > > Modified Files: > All > Added Files: > ID.1766414.stable.err ID.1766414.stable.out ID.1766414.xml > ID.1766414.xq > Log Message: > adding stable output, produced on branch -- will still fail in head until Why hasn't this test been added to the release branch? (Apparently, the bug occured and was fixed there...) > I propagate the fix Is there an urgend need for individual "by hand" propagation (with all possible conflicts it might cause for the ustandard (semi-)automa[tgn]ic propagation by Sjoerd and/or me or does the head-fix require more than the usual standard (semi-)automa[tgn]ic propagation by Sjoerd and/or me? If 2x no, I'd prefer to have this done during / via the usual standard (semi-)automa[tgn]ic propagation by Sjoerd and/or me. (Please feel free to request a usual standard (semi-)automa[tgn]ic propagation by Sjoerd and/or me whenever you "need" to have changes propagated from the latest release branch to the development head "urgendly".) Thanks. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Any efforts on using bitmap indices?
by Gabriel Welsche 02 Aug '07

02 Aug '07

Hello, does anybody consider supporting bitmap indices? An indexing software called FastBit was integrated to MonetDB (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) and some benchmarks show that there were serious improvements concerning combined numerical and text queries (offers up to 10 times better performance then MonetDB without FastBit). Gabriel

5 6

[Monetdb-developers] Key constraints and optimizers...
by Romulo Goncalves 31 Jul '07

31 Jul '07

Dear developers.... I was checking the status of the SQL test web-site (cvs branch) and I found some red ones. In the test src/test/BugConstraint/delete_primarykey_1643845.sql If I run the normal Mtest the test is red (the err output). The constraint error given is for the foreign key and not for the primary key (as in the stable output). If I ran with M4 the test is green the same happens if I set the optimizers off. So we can conclude that the reason for this test being red are the optimizers and nothing else in SQL module. I tried to give a look to see which optimizer is breaking the test, but no results. Tomorrow I will continue. I also think that all the tests that are red (the ones related with primary keys) are due this problem. Should I open a bug or this is enough? Because it is an existent test that is red ;) Regards, Romulo

4 5

[Monetdb-developers] ANSI Characters
by Bharani 30 Jul '07

30 Jul '07

Hi, I am trying to insert ansi characters and it doesn't seem to be working in monetdb/sql insert into reason values(100,'SAVE 25¢ ON ICE CREAM'); reason is a table with Id integer message string The client thinks the query is not complete and prompts '>more'. What should i configure to get this working Thanks Bharani -- View this message in context: http://www.nabble.com/ANSI-Characters-tf4169853.html#a11862871 Sent from the monetdb-developers mailing list archive at Nabble.com.

3 6

[Monetdb-developers] Oops
by Sjoerd Mullender 30 Jul '07

30 Jul '07

I forgot to add a file to the list of files to be checked out in the CVSROOT directory when I added a commit test there. This means that it is now impossible to check anything in on Sourceforge. I have already submitted a support request, so we just have to wait until they can handle that. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/Tests bpmUpdate.mal, , NONE, 1.1 bpmUpdate.stable.err, , NONE, 1.1 bpmUpdate.stable.out, , NONE, 1.1 All, , 1.36, 1.37 inspect05.stable.out, , 1.318, 1.319 bpt00.mal, , 1.8, NONE bpt00.stable.err, , 1.2, NONE bpt00.stable.out, , 1.21, NONE bpt03.mal, , 1.7, NONE bpt03.stable.err, , 1.2, NONE bpt03.stable.out, , 1.21, NONE bpt04.mal, , 1.7, NONE bpt04.stable.err, , 1.2, NONE bpt04.stable.out, , 1.21, NONE bpt05.mal, , 1.5, NONE bpt05.stable
by Stefan Manegold 28 Jul '07

28 Jul '07

On Sat, Jul 28, 2007 at 01:48:06AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv7852 > > Modified Files: > All inspect05.stable.out > Added Files: > bpmUpdate.mal bpmUpdate.stable.err bpmUpdate.stable.out > Removed Files: > bpt00.mal bpt00.stable.err bpt00.stable.out bpt03.mal > bpt03.stable.err bpt03.stable.out bpt04.mal bpt04.stable.err > bpt04.stable.out bpt05.mal bpt05.stable.err bpt05.stable.out > bpt06.mal bpt06.stable.err bpt06.stable.out > Log Message: > Removing old tests that are not relevant anymore. > Added a basic test for bpm [...] > --- NEW FILE: bpmUpdate.stable.err --- > stderr of test 'bpmUpdate` in directory 'src/modules/mal` itself: > > > # 03:37:48 > > # 03:37:48 > Mtimeout -timeout 60 mserver5 "--config=/ufs/mk/monet5/Linux/etc/monetdb5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin" --set "gdk_dbfarm=/ufs/mk/monet5//Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/sql_logs" --set "xquery_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/xquery_logs" --set mapi_open=true --set mapi_port=39694 --set xrpc_port=49963 --set monet_prompt= --trace --dbname=mTests_src_modules_mal bpmUpdate.mal > # 03:37:48 > > > #warning: please don't forget to set your vault key! > #(see /ufs/mk/monet5/Linux/etc/monetdb5.conf) > mserver5: /ufs/mk/monet5//package/MonetDB5/src/modules/mal/bpm.mx:726: newPartition: Assertion `bpmcat[i] == ((void *)0)' failed. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > !Mtimeout: signal 6: mserver5 --config=/ufs/mk/monet5/Linux/etc/monetdb5.conf --debug=10 --set monet_mod_path=/ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin --set gdk_dbfarm=/ufs/mk/monet5//Linux/var/MonetDB5/dbfarm --set sql_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/sql_logs --set xquery_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/xquery_logs --set mapi_open=true --set mapi_port=39694 --set xrpc_port=49963 --set monet_prompt= --trace --dbname=mTests_src_modules_mal bpmUpdate.mal ^^^^^^^^^^^^^^^^^^^^ Have the above error messages been approved intentionally i.e. is this test supposed to fail like this? Or have they been approved "by accident" (explicitely(!) using `Mapprove -f ...`)? If the latter please fix the stable output. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_system.mx, , 1.103, 1.104
by Stefan Manegold 24 Jul '07

24 Jul '07

On Thu, Jul 19, 2007 at 03:55:14PM +0000, Fabian wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv4085 > > Modified Files: > gdk_system.mx > Log Message: > - added MT_join_thread to wait for a thread to exit > - don't rely on the Apple compiler, but rely on the Mach-O system > instead when trying to get around missing semaphore functionalities OK, but shouldn't it be "__MACH__" instead of "__MACH_O__": ======== monet@horizon::/tmp> uname -a -------- Darwin horizon.cs.umass.edu 8.10.0 Darwin Kernel Version 8.10.0: Wed May 23 16:50:59 PDT 2007; root:xnu-792.21.3~1/RELEASE_PPC Power Macintosh powerpc ======== monet@horizon::/tmp> cat x.c -------- #include <stdlib.h> #include <stdio.h> int main () { #ifdef __APPLE_CC__ printf("__APPLE_CC__ = %d\n", __APPLE_CC__); #endif #ifdef __MACH__ printf("__MACH__ = %d\n", __MACH__); #endif #ifdef __MACH_O__ printf("__MACH_O__ = %d\n", __MACH_O__); #endif return 0; } ======== monet@horizon::/tmp> gcc x.c -o x && ./x -------- __APPLE_CC__ = 1809 __MACH__ = 1 ======== monet@horizon::/tmp> find /usr/include/ -type f | xargs grep --color -w __MACH_O__ ======== monet@horizon::/tmp> find /usr/include/ -type f | xargs grep --color -w __MACH__ -------- /usr/include//com_err.h:#if defined(macintosh) || (defined(__MACH__) && defined(__APPLE__)) /usr/include//des.h:#if defined(__MACH__) && defined(__APPLE__) /usr/include//gcc/darwin/3.3/c++/cctype:#if defined(__APPLE__) && defined(__MACH__) Binary file /usr/include//gcc/darwin/4.0/c++/i686-apple-darwin8/bits/stdc++.h.gch/O0g.gch matches Binary file /usr/include//gcc/darwin/4.0/c++/i686-apple-darwin8/bits/stdc++.h.gch/O2g.gch matches Binary file /usr/include//gcc/darwin/4.0/c++/powerpc-apple-darwin8/bits/stdc++.h.gch/O0g.gch matches Binary file /usr/include//gcc/darwin/4.0/c++/powerpc-apple-darwin8/bits/stdc++.h.gch/O2g.gch matches /usr/include//gssapi/gssapi.h:#if defined(__MACH__) && defined(__APPLE__) /usr/include//kerberosIV/des.h:#if defined(__MACH__) && defined(__APPLE__) /usr/include//kerberosIV/krb.h:#if defined(__MACH__) && defined(__APPLE__) /usr/include//krb.h:#if defined(__MACH__) && defined(__APPLE__) /usr/include//krb5.h:#if defined(__MACH__) && defined(__APPLE__) Binary file /usr/include//libc-gcc3.p matches Binary file /usr/include//mach/mach-gcc3.p matches /usr/include//profile.h:#if defined(__MACH__) && defined(__APPLE__) /usr/include//TargetConditionals.h: #ifdef __MACH__ /usr/include//TargetConditionals.h: #ifdef __MACH__ /usr/include//tidy/platform.h:#elif defined(__APPLE__) && defined(__MACH__) Binary file /usr/include//unistd-gcc3.p matches /usr/include//wx-2.5/wx/defs.h: #if defined(__MACH__) && !defined(SIZEOF_WCHAR_T) /usr/include//wx-2.5/wx/filefn.h: #ifdef __MACH__ /usr/include//wx-2.5/wx/filefn.h: #if !defined(__WXMAC__) || defined(__UNIX__) || defined(__MACH__) /usr/include//wx-2.5/wx/filefn.h:#elif defined(__MWERKS__) && !defined(__INTEL__) && !defined(__MACH__) /usr/include//wx-2.5/wx/platform.h: #if defined(__MACH__) /usr/include//wx-2.5/wx/wxchar.h:#if defined(__MWERKS__) && !defined(__MACH__) /usr/include//wx-2.5/wx/wxchar.h: #elif defined(__MWERKS__) && ( defined(__MSL__) || defined(__MACH__) ) /usr/include//wx-2.5/wx/wxchar.h: #ifdef __MACH__ /usr/include//wx-2.5/wx/wxchar.h: #if defined(__MWERKS__) && !defined(__MACH__) && (__MSL__ < 0x00008000) ======== ?? *Maybe*, that would help to get client tests on MacOS X / Darwin (on PowerPC) working, again ... Stefan > > Index: gdk_system.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_system.mx,v > retrieving revision 1.103 > retrieving revision 1.104 > diff -u -d -r1.103 -r1.104 > --- gdk_system.mx 2 May 2007 16:17:00 -0000 1.103 > +++ gdk_system.mx 19 Jul 2007 15:55:11 -0000 1.104 > @@ -143,6 +143,7 @@ > gdk_export void MT_exit_thread(int status); > gdk_export void MT_global_exit(int status); > gdk_export MT_Id MT_getpid(void); > +gdk_export int MT_join_thread(MT_Id t); > gdk_export int MT_kill_thread(MT_Id t); > > #ifdef HAVE_PTHREAD_SIGMASK > @@ -187,7 +188,7 @@ > > @- MT Semaphore API > @h > -#if defined(_AIX) || defined(__APPLE_CC__) > +#if defined(_AIX) || defined(__MACH_O__) > typedef struct { > int cnt; > pthread_mutex_t mutex; > @@ -425,6 +426,12 @@ > pthread_exit(&st); > } > > +int > +MT_join_thread(MT_Id t) > +{ > + return pthread_join((pthread_t)(t - 1), NULL); > +} > + > > int > MT_kill_thread(MT_Id t) > @@ -439,7 +446,7 @@ > #endif > } > > -#if defined(_AIX) || defined(__APPLE_CC__) > +#if defined(_AIX) || defined(__MACH_O__) > void > pthread_sema_init(pthread_sema_t *s, int flag, int nresources) > { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Prepared Statement Error
by Bharani 23 Jul '07

23 Jul '07

Hi, I am trying to execute this sql as prepared statement update reason set failurereason=? where ID=? reason is a table with two columns ID is integer failurereason is a string During the batch execution i see exec 3('whatever',2); in the batch and then i get MALException:(unknown):wrong argument for prepared statement (expected int instead of char) Looks like expected types are getting swapped. Thanks Bharani -- View this message in context: http://www.nabble.com/Prepared-Statement-Error-tf4116290.html#a11705803 Sent from the monetdb-developers mailing list archive at Nabble.com.

2 3

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pf_support.mx, , 1.250, 1.251
by Jan Rittinger 21 Jul '07

21 Jul '07

Stefan, thanks a lot! Your change speeds up XMark Q10 (on a 100MB sf 1.0 variant) by a second (4958 vs. 3828 -- measured on my laptop; compiled without optimization). If I only look at the times for the multi_merged_union call I can even see an improvement of factor 2.5 (1821 vs. 726 ms). Jan On 07/20/2007 11:12 PM, Stefan Manegold wrote with possible deletions: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv23220/runtime > > Modified Files: > pf_support.mx > Log Message: > > finally, I managed to fullfil JanR's "small wish": > > replaced > > PROC multi_merged_union (BAT[void, BAT] b) : BAT[void,BAT] > > by > > .COMMAND multi_merged_union (BAT[void, BAT] b) > : BAT[void,BAT] = CMDmulti_merged_union; > "PARAMETERS: > BAT[void,BAT t[void,BAT c[VoID,any]]]: > a void-headed BAT of n void-headed BATs t such that each BAT t represents a > relational table t of m columns c, each represented by a dense-headed BAT c; > the first column of each table must be sorted. > DESCRIPTION: > Merges the n relational tables t according to the value-order as defined by the > first columns of all tables." > > At a slight performance decrease with only 2 or 3 tables, > it yields a performance gain of 25% - 63% with 4-9 tables > (each table has 9 columns and 1000000 tuples): > > # Old New New-Old (N-O)/O > 2 289745 339914 50169 17% > 3 719059 904862 185803 25% > 4 1302676 955985 -346691 -26% > 5 2023403 1082348 -941055 -46% > 6 2922657 1359887 -1562770 -53% > 7 3757045 1597463 -2159582 -57% > 8 5151854 1878731 -3273123 -63% > 9 6028759 2177258 -3851501 -63% > > > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.250 > retrieving revision 1.251 > diff -u -d -r1.250 -r1.251 > --- pf_support.mx 12 Jul 2007 10:23:29 -0000 1.250 > +++ pf_support.mx 20 Jul 2007 21:12:46 -0000 1.251 > @@ -133,6 +133,17 @@ > input." > @) > > +.COMMAND multi_merged_union (BAT[void, BAT] b) > + : BAT[void,BAT] = CMDmulti_merged_union; > +"PARAMETERS: > +BAT[void,BAT t[void,BAT c[VoID,any]]]: > +a void-headed BAT of n void-headed BATs t such that each BAT t represents a > +relational table t of m columns c, each represented by a dense-headed BAT c; > +the first column of each table must be sorted. > +DESCRIPTION: > +Merges the n relational tables t according to the value-order as defined by the > +first columns of all tables." > + > .COMMAND ll_tokenize( BAT[void,str] strs, BAT[void,str] seps ) > : BAT[oid,str] = CMDll_strSplit; > "PARAMETERS: > @@ -5042,266 +5053,353 @@ > } > > static int > -merged_union( BAT** res, int nbats, BAT **b) > +merged_union( BAT** res, int ntabs, int ncols, BAT ***b) > { > - BAT *bn[MAXPARAMS>>1], *BN; > - BUN cur[MAXPARAMS], dst[MAXPARAMS>>1], DST; > - int bs[MAXPARAMS], bns[MAXPARAMS>>1], BS; > - size_t idx[MAXPARAMS], cnt[MAXPARAMS]; > - int npairs = 1, i, j, k, any, b0 = -1, b1 = -1; > - bit concat = FALSE; > - chr *w = NULL; > - size_t sze = 0, h = 0; > + BAT *bn[ncols], *BN, *temp, *tmp[2], *tgt; > + BUN cur[ntabs][ncols], dst[ncols], DST; > + bte bs[ntabs][ncols], bns[ncols], BS; > + bit mat[ntabs]; > + int ttpe[ncols]; /* sht / bte ? */ > + oid hsqb[ntabs]; > + size_t Bcnt[ntabs], inc[2]; > + int t, c, any, non_empty[ntabs+1]; > + bit concat01 = FALSE, concat10 = FALSE; > + bte *w = NULL, *ww[2], *wt, *ws[2]; > + size_t res_count = 0, h = 0; > + > + BAT *src[2]; > + BUN csr[2]; > + bte sze[2]; > + size_t idx[2], cnt[2], hs[2], ht; > > + assert(ncols > 0); > + assert(ntabs > 1); > + assert(ntabs <= GDK_bte_max); /* allows to use type bte for w[] */ > + > *res = NULL; > > /* check arguments */ > - ERRORcheck(BATcount(b[0])>1 && !(BATtordered(b[0])&1), "merged_union: tail of first BAT must be sorted.\n"); > - ERRORcheck(BATcount(b[1])>1 && !(BATtordered(b[1])&1), "merged_union: tail of second BAT must be sorted.\n"); > - if (nbats&1) { > - GDKerror("merged_union: uneven number of BATs: %d.\n", nbats); > - return GDK_FAIL; > - } > - npairs = nbats>>1; > - > - for (i=0; i<nbats; i+=2) { > - bit ci, cj; > - j = i+1; > - ci = !is_fake_project(b[i]); > - cj = !is_fake_project(b[j]); > - if (ci && b0 < 0) { > - b0 = i; > - } > - if (cj && b1 < 0) { > - b1 = j; > - } > - if (ci && !BAThdense(b[i])) { > - GDKerror("merged_union: BAT %d must have a dense head.\n", i+1); > - return GDK_FAIL; > - } > - if (cj && !BAThdense(b[j])) { > - GDKerror("merged_union: BAT %d must have a dense head.\n", j+1); > - return GDK_FAIL; > - } > - if (ATOMtype(b[i]->ttype) != ATOMtype(b[j]->ttype)) { > - GDKerror("merged_union: BATs %d (ttype=%d) & %d (ttype=%d) must have the same tail types.\n", > - i+1, b[i]->ttype, j+1, ATOMtype(b[j]->ttype)); > - return GDK_FAIL; > - } > - if (ci && b0 >= 0 && b[i]->hseqbase != b[b0]->hseqbase) { > - GDKerror("merged_union: BAT %d (hseqbase="OIDFMT") must have the same hseqbase as BAT %d (hseqbase="OIDFMT").\n", > - i+1, b[i]->hseqbase, b0+1, b[b0]->hseqbase); > + for (t=0; t<ntabs; t++) { > + if (BATcount(b[t][0])>1 && !(BATtordered(b[t][0])&1)) { > + GDKerror("merged_union: tail of first BAT/column of table %d must be sorted.\n", t+1); > return GDK_FAIL; > } > - if (cj && b1 >= 0 && b[j]->hseqbase != b[b1]->hseqbase) { > - GDKerror("merged_union: BAT %d (hseqbase="OIDFMT") must have the same hseqbase as BAT %d (hseqbase="OIDFMT").\n", > - j+1, b[j]->hseqbase, b1+1, b[b1]->hseqbase); > - return GDK_FAIL; > + } > + > + for (t=0; t<ntabs; t++) { > + mat[t] = FALSE; > + hsqb[t] = oid_nil; > + Bcnt[t] = (size_t)-1; > + } > + for (c=0; c<ncols; c++) { > + ttpe[c] = int_nil; > + } > + > + non_empty[ntabs] = 0; > + for (t=0; t<ntabs; t++) { > + for (c=0; c<ncols; c++) { > + if (ttpe[c] == int_nil || ( ttpe[c] == TYPE_oid && b[t][c]->ttype == TYPE_void ) ) { > + /* get column type and record whether any OID column was non-materialized (VoID) */ > + ttpe[c] = b[t][c]->ttype; > + } else { > + /* check column type (VoID matches with OID) */ > + if (ATOMtype(ttpe[c]) != ATOMtype(b[t][c]->ttype)) { > + GDKerror("merged_union: BAT/column %d of table %d (ttype=%d) must have the same tail type as BAT/column %d of all other tables (ttype=%d).\n", > + c+1, t+1, b[t][c]->ttype, c+1, ATOMtype(ttpe[c])); > + return GDK_FAIL; > + } > } > - if (ci && b0 >= 0 && BATcount(b[i]) != BATcount(b[b0])) { > - GDKerror("merged_union: BAT %d ("SZFMT" BUNs) must have the same size as BAT %d ("SZFMT" BUNs).\n", > - i+1, BATcount(b[i]), b0+1, BATcount(b[b0])); > - return GDK_FAIL; > + if (!is_fake_project(b[t][c])) { > + mat[t] |= TRUE; > + if (!BAThdense(b[t][c])) { > + GDKerror("merged_union: BAT/column %d of table %d must have a dense head.\n", c+1, t+1); > + return GDK_FAIL; > + } > + if (hsqb[t] == oid_nil) { > + hsqb[t] = b[t][c]->hseqbase; > + } else { > + if (hsqb[t] != b[t][c]->hseqbase) { > + GDKerror("merged_union: BAT/column %d of table %d (hseqbase="OIDFMT") must have the same hseqbase as all other BATs/columns of table %d (hseqbase="OIDFMT").\n", > + c+1, t+1, b[t][c]->hseqbase, t+1, hsqb[t]); > + return GDK_FAIL; > + } > + } > + if (Bcnt[t] == (size_t)-1) { > + Bcnt[t] = BATcount(b[t][c]); > + } else { > + if (Bcnt[t] != BATcount(b[t][c])) { > + GDKerror("merged_union: BAT/column %d of table %d ("SZFMT" BUNs) must have the same size as all other BATs/columns of table %d ("SZFMT" BUNs).\n", > + c+1, t+1, BATcount(b[t][c]), t+1, Bcnt[t]); > + return GDK_FAIL; > + } > + } > } > - if (cj && b1 >= 0 && BATcount(b[j]) != BATcount(b[b1])) { > - GDKerror("merged_union: BAT %d ("SZFMT" BUNs) must have the same size as BAT %d ("SZFMT" BUNs).\n", > - j+1, BATcount(b[j]), b1+1, BATcount(b[b1])); > + } > + if (!mat[t]) { > + GDKerror("merged_union: at least one BAT/column of table %d must be materialized, i.e., no 'fake_project'.\n", t+1); > return GDK_FAIL; > - } > - } > - if (b0 < 0) { > - GDKerror("merged_union: at least one of the 'odd' BATs must be materialized, i.e., no 'fake_project'.\n"); > - return GDK_FAIL; > - } > - if (b1 < 0) { > - GDKerror("merged_union: at least one of the 'even' BATs must be materialized, i.e., no 'fake_project'.\n"); > - return GDK_FAIL; > + } > + res_count += Bcnt[t]; > + non_empty[non_empty[ntabs]] = t; > + non_empty[ntabs] += (Bcnt[t] > 0); > } > > /* create result BATs */ > > - sze = BATcount(b[b0]) + BATcount(b[b1]); > - BN = BATnew(TYPE_void, TYPE_bat, npairs); > + BN = BATnew(TYPE_void, TYPE_bat, ncols); > if (BN == NULL) { > - GDKerror("merged_union: BATnew(TYPE_void, TYPE_bat, %d) failed.\n", npairs); > + GDKerror("merged_union: BATnew(TYPE_void, TYPE_bat, %d) failed.\n", ncols); > return GDK_FAIL; > } > - for (k=0; k<npairs; k++) { > - i = k<<1; > - bn[k] = BATnew(TYPE_void, ATOMtype(b[i]->ttype), sze); > - if (bn[k] == NULL) { > - GDKerror("merged_union: BATnew(TYPE_void, %s, " SZFMT ") failed.\n", ATOMname(ATOMtype(b[i]->ttype)), sze); > - while (k>0) { > - BBPreclaim(bn[--k]); > + temp = BATnew(TYPE_void, ATOMtype(ttpe[0]), res_count); > + if (temp == NULL) { > + GDKerror("merged_union: BATnew(TYPE_void, %s, " SZFMT ") failed.\n", ATOMname(ATOMtype(ttpe[0])), res_count); > + BBPreclaim(BN); > + return GDK_FAIL; > + } > + for (c=0; c<ncols; c++) { > + bn[c] = BATnew(TYPE_void, ATOMtype(ttpe[c]), res_count); > + if (bn[c] == NULL) { > + GDKerror("merged_union: BATnew(TYPE_void, %s, " SZFMT ") failed.\n", ATOMname(ATOMtype(ttpe[c])), res_count); > + while (c>0) { > + BBPreclaim(bn[--c]); > } > + BBPreclaim(temp); > BBPreclaim(BN); > return GDK_FAIL; > } > } > - if (sze > 0) { > - w = (chr*)GDKmalloc(sze); > - if (w == NULL) { > - GDKerror("merged_union: GDKmalloc(" SZFMT ") failed.\n", sze); > - goto cleanup; > - } > + w = (bte*)GDKmalloc(2*res_count + 1); > + if (w == NULL) { > + GDKerror("merged_union: GDKmalloc(" SZFMT ") failed.\n", 2*res_count + 1); > + goto cleanup; > } > > /* do the merged_union */ > > - for (k=0; k<npairs; k++) { > - bns[k] = BUNsize(bn[k]); > - dst[k] = BUNlast(bn[k]); > + for (c=0; c<ncols; c++) { > + bns[c] = (bte)BUNsize(bn[c]); > + dst[c] = BUNlast(bn[c]); > } > - for (i=0; i<nbats; i++) { > - if (is_fake_project(b[i])) { > - bs[i] = 0; > - } else { > - bs[i] = BUNsize(b[i]); > - } > - cur[i] = BUNfirst(b[i]); > - idx[i] = 0; > - if (i&1) { > - cnt[i] = BATcount(b[b1]); > + for (t=0; t<ntabs; t++) { > + for (c=0; c<ncols; c++) { > + if (is_fake_project(b[t][c])) { > + bs[t][c] = (bte)0; > } else { > - cnt[i] = BATcount(b[b0]); > + bs[t][c] = (bte)BUNsize(b[t][c]); > } > + cur[t][c] = BUNfirst(b[t][c]); > + } > } > +@ > @= merged_union_0 > - /* @1: ATOMstorage(b[@3]->ttype) (chr, sht, int, flt, lng, dbl, any=b[@3]->ttype) > + /* @1: ATOMstorage(ttpe[0]) (chr, bte, sht, int, flt, lng, dbl, any=ttpe[0]) > * @2: tloc, tvar, tail > - * @3: 0, 1 > - * @5: tail value comparison, > - e.g., simple_LE(BUN@2(b[0],cur[0]), BUN@2(b[1],cur[1]), @1) > - or atom_GT(BUN@2(b[0],cur[0]), BUN@2(b[1],cur[1]), @1) > + * @3: 0 1 > + * @4: tail value comparison, > + e.g., simple_LE(BUN@2(src[0],csr[0]), BUN@2(src[1],csr[1]), @1) > + or atom_GT(BUN@2(src[0],csr[0]), BUN@2(src[1],csr[1]), @1) > */ > /* copy tails from BAT @3 to the results; > for each BUN, remember in w, whether it came from BAT 0 or BAT 1 */ > while ((idx[@3] < cnt[@3]) && (@4)) { > - void@1_bunfastins_nocheck_noinc(bn[0],dst[0],0,BUN@2(b[@3],cur[@3])); > + void@1_bunfastins_nocheck_noinc(tgt,dst[0],0,BUN@2(src[@3],csr[@3])); > + wt[ht] = ws[@3][hs[@3]]; > idx[@3]++; > - cur[@3] += bs[@3]; > + csr[@3] += sze[@3]; > dst[0] += bns[0]; > - w[h++] = (chr)@3; > + hs[@3] += inc[@3]; > + ht++; > } > + > + /* sucessively pair-wise merge-union the first BAT/column of all tables */ > +@ > @= merged_union_1 > - /* @1: ATOMstorage(b[0]->ttype) (chr, sht, int, flt, lng, dbl, any=b[0]->ttype) > + /* @1: ATOMstorage(ttpe[0]) (chr, bte, sht, int, flt, lng, dbl, any=ttpe[0]) > * @2: tloc, tvar, tail (for BAT 0) > * @3: tloc, tvar, tail (for BAT 1) > * @4: simple, atom > */ > - /* merge-union the first two BATs; regard and preserve tail-order */ > - h = 0; > - concat = (BATcount(b[0])==0 || BATcount(b[1])==0); > - if (!concat) { > - concat = ( BATtordered(b[0])&BATtordered(b[1])&1 && \ > - @4_LE(BUN@2(b[0],BUNlast(b[0])-BUNsize(b[0])),BUN@3(b[1],cur[1]),@1) ); > + /* merge-union the first BATs/columns; regard and preserve tail-order */ > + concat01 = (cnt[0]==0 || cnt[1]==0); > + concat10 = FALSE; > + if (BATtordered(src[0])&BATtordered(src[1])&1 || (BATtordered(src[0])&1 && cnt[1]==1) || (cnt[0]==1 && BATtordered(src[1])&1) ) { > + if (!concat01) { > + concat01 = @4_LE(BUN@2(src[0],BUNlast(src[0])-BUNsize(src[0])),BUN@3(src[1],csr[1]),@1); > + } > + if (!concat01) { > + concat10 = @4_LT(BUN@3(src[1],BUNlast(src[1])-BUNsize(src[1])),BUN@2(src[0],csr[0]),@1); > + } > } > - if (!concat) { > + if (!concat01 && !concat10) { > while ((idx[0] < cnt[0]) && (idx[1] < cnt[1])) { > - @:merged_union_0(@1,@2,0,@4_LE(BUN@2(b[0],cur[0]),BUN@3(b[1],cur[1]),@1))@ > + @:merged_union_0(@1,@2,0,@4_LE(BUN@2(src[0],csr[0]),BUN@3(src[1],csr[1]),@1))@ > if (idx[0] < cnt[0]) { > - @:merged_union_0(@1,@3,1,@4_GT(BUN@2(b[0],cur[0]),BUN@3(b[1],cur[1]),@1))@ > + @:merged_union_0(@1,@3,1,@4_GT(BUN@2(src[0],csr[0]),BUN@3(src[1],csr[1]),@1))@ > } > } > } > /* get remaining BUNs */ > - @:merged_union_0(@1,@2,0,TRUE)@ > + if (!concat10) { > + @:merged_union_0(@1,@2,0,TRUE)@ > + } > @:merged_union_0(@1,@3,1,TRUE)@ > + @:merged_union_0(@1,@2,0,TRUE)@ > +@ > @= merged_union_2 > - /* @1: ATOMstorage(b[0]->ttype) (chr, sht, int, flt, lng, dbl, any=b[0]->ttype) > + /* @1: ATOMstorage(ttpe[0]) (chr, bte, sht, int, flt, lng, dbl, any=ttpe[0]) > * @2: tloc, tvar, tail (for BAT 0) > * @3: simple, atom > */ > @:merged_union_1(@1,@2,@2,@3)@ > break; > +@ > @= merged_union_3 > - /* @1: ATOMstorage(b[@3]->ttype) (chr, sht, int, flt, lng, dbl, any=b[0]->ttype) > - * @2: tloc, tvar, tail > - */ > - /* merge-union each of the remaining BAT-pairs; > - w tell us, from which BAT we need to get the next BUN */ > - for (h=0; h<sze; h++) { > - j = i + (int)w[h]; > - void@1_bunfastins_nocheck_noinc(bn[k],dst[k],0,BUN@2(b[j],cur[j])); > - idx[j]++; > - cur[j] += bs[j]; > - dst[k] += bns[k]; > - } > -@= merged_union_4 > - /* @1: ATOMstorage(b[@3]->ttype) (chr, sht, int, flt, lng, dbl, any=b[0]->ttype) > - * @2: tloc, tvar, tail > - */ > - @:merged_union_3(@1,@2)@ > - break; > -@c > - /* merge-union the first two BATs */ > + sze[0] = BUNsize(src[0]); > + sze[1] = BUNsize(src[1]); > + csr[0] = BUNfirst(src[0]); > + csr[1] = BUNfirst(src[1]); > + cnt[0] = BATcount(src[0]); > + cnt[1] = BATcount(src[1]); > + dst[0] = BUNfirst(tgt); > + idx[0] = idx[1] = 0; > + hs[0] = hs[1] = ht = 0; > + > /* HACK(?): compare [v]oid (unsigned) as int/lng (signed) to get nil's first... */ > #if SIZEOF_OID == SIZEOF_INT > - if (b[0]->ttype==TYPE_void && b[1]->ttype==TYPE_void) { > - @:merged_union_1(int,tvar,tvar,simple)@ > - } else if (b[0]->ttype==TYPE_void && b[1]->ttype==TYPE_oid) { > - @:merged_union_1(int,tvar,tloc,simple)@ > - } else if (b[0]->ttype==TYPE_oid && b[1]->ttype==TYPE_void) { > - @:merged_union_1(int,tloc,tvar,simple)@ > - } else if (b[0]->ttype==TYPE_oid && b[1]->ttype==TYPE_oid) { > - @:merged_union_1(int,tloc,tloc,simple)@ > + if (src[0]->ttype==TYPE_void && src[1]->ttype==TYPE_void) { > + @:merged_union_1(int,tvar,tvar,simple)@ > + } else if (src[0]->ttype==TYPE_void && src[1]->ttype==TYPE_oid) { > + @:merged_union_1(int,tvar,tloc,simple)@ > + } else if (src[0]->ttype==TYPE_oid && src[1]->ttype==TYPE_void) { > + @:merged_union_1(int,tloc,tvar,simple)@ > + } else if (src[0]->ttype==TYPE_oid && src[1]->ttype==TYPE_oid) { > + @:merged_union_1(int,tloc,tloc,simple)@ > #else > - if (b[0]->ttype==TYPE_void && b[1]->ttype==TYPE_void) { > - @:merged_union_1(lng,tvar,tvar,simple)@ > - } else if (b[0]->ttype==TYPE_void && b[1]->ttype==TYPE_oid) { > - @:merged_union_1(lng,tvar,tloc,simple)@ > - } else if (b[0]->ttype==TYPE_oid && b[1]->ttype==TYPE_void) { > - @:merged_union_1(lng,tloc,tvar,simple)@ > - } else if (b[0]->ttype==TYPE_oid && b[1]->ttype==TYPE_oid) { > - @:merged_union_1(lng,tloc,tloc,simple)@ > + if (src[0]->ttype==TYPE_void && src[1]->ttype==TYPE_void) { > + @:merged_union_1(lng,tvar,tvar,simple)@ > + } else if (src[0]->ttype==TYPE_void && src[1]->ttype==TYPE_oid) { > + @:merged_union_1(lng,tvar,tloc,simple)@ > + } else if (src[0]->ttype==TYPE_oid && src[1]->ttype==TYPE_void) { > + @:merged_union_1(lng,tloc,tvar,simple)@ > + } else if (src[0]->ttype==TYPE_oid && src[1]->ttype==TYPE_oid) { > + @:merged_union_1(lng,tloc,tloc,simple)@ > #endif > - } else { > - any = b[0]->ttype; > - switch(ATOMstorage(b[0]->ttype)) { > - case TYPE_chr: @:merged_union_2(chr,tloc,simple)@ > - case TYPE_sht: @:merged_union_2(sht,tloc,simple)@ > - case TYPE_int: @:merged_union_2(int,tloc,simple)@ > - case TYPE_flt: @:merged_union_2(flt,tloc,simple)@ > - case TYPE_lng: @:merged_union_2(lng,tloc,simple)@ > - case TYPE_dbl: @:merged_union_2(dbl,tloc,simple)@ > - default: > - if (b[0]->tvarsized) { > - @:merged_union_2(any,tvar,atom)@ > - } else { > - @:merged_union_2(any,tloc,atom)@ > + } else { > + any = src[0]->ttype; > + switch(ATOMstorage(any)) { > + case TYPE_chr: @:merged_union_2(chr,tloc,simple)@ > + case TYPE_bte: @:merged_union_2(bte,tloc,simple)@ > + case TYPE_sht: @:merged_union_2(sht,tloc,simple)@ > + case TYPE_int: @:merged_union_2(int,tloc,simple)@ > + case TYPE_flt: @:merged_union_2(flt,tloc,simple)@ > + case TYPE_lng: @:merged_union_2(lng,tloc,simple)@ > + case TYPE_dbl: @:merged_union_2(dbl,tloc,simple)@ > + default: > + if (src[0]->tvarsized) { > + @:merged_union_2(any,tvar,atom)@ > + } else { > + @:merged_union_2(any,tloc,atom)@ > + } > } > } > + > + tgt->batBuns->free = dst[0] - tgt->batBuns->base; > + BATsetcount(tgt, tgt->batBuns->free/bns[0]); > + if (!tgt->batDirty) tgt->batDirty = TRUE; > + BATkey(BATmirror(tgt),FALSE); > + tgt->tsorted = GDK_SORTED; > + tgt->tdense = FALSE; > +@ > +@c > + tgt = bn[0]; > + wt = w; > + ws[0] = w + res_count; > + ws[1] = w + res_count + 1; > + inc[0] = inc [1] = 0; > + > + if (non_empty[ntabs] == 1) { > + src[0] = b[non_empty[0]][0]; > + src[1] = b[non_empty[0] == 0][0]; > + ws[0][0] = non_empty[0]; > + ws[1][0] = (non_empty[0] == 0); > + > + @:merged_union_3@ > + } else > + if (non_empty[ntabs] == 2) { > + src[0] = b[non_empty[0]][0]; > + src[1] = b[non_empty[1]][0]; > + ws[0][0] = non_empty[0]; > + ws[1][0] = non_empty[1]; > + > + @:merged_union_3@ > + } else { > + t = ntabs - 1; > + tmp[0] = bn[0]; > + tmp[1] = temp; > + ww[0] = w; > + ww[1] = ws[1]; > + wt = ww[t&1]; > + wt[0] = t; > + tgt = b[t--][0]; > + for (; t >= 0; t--) { > + src[0] = b[t][0]; > + src[1] = tgt; > + tgt = tmp[t&1]; > + ws[0][0] = t; > + ws[1] = wt; > + wt = ww[t&1]; > + > + @:merged_union_3@ > + > + inc[1] = 1; > + } > } > - /* merge-union each of the remaining BAT-pairs */ > - for (k=1; k<npairs; k++) { > - i = (k<<1); > - j = i+1; > -/* HACK(?): compare [v]oid (unsigned) at int/lng (signed) to get nil's first... */ > -#if SIZEOF_OID == SIZEOF_INT > - if (b[i]->ttype==TYPE_void || b[j]->ttype==TYPE_void) { > - @:merged_union_3(int,tail,simple)@ > - } else if (b[i]->ttype==TYPE_oid && b[j]->ttype==TYPE_oid) { > - @:merged_union_3(int,tloc,simple)@ > -#else > - if (b[i]->ttype==TYPE_void || b[j]->ttype==TYPE_void) { > - @:merged_union_3(lng,tail,simple)@ > - } else if (b[i]->ttype==TYPE_oid && b[j]->ttype==TYPE_oid) { > - @:merged_union_3(lng,tloc,simple)@ > -#endif > + > + /* merge-union the remaining BATs/columns of all tables */ > +@ > +@= merged_union_4 > + /* @1: ATOMstorage(ttpe[c]) (chr, bte, sht, int, flt, lng, dbl, any) > + * @2: tloc, tvar, tail > + */ > + /* merge-union the remaining BATs/columns of all tables: > + w tell us, from which table we need to get the next BUN */ > + for (h=0; h<res_count; h++) { > + t = w[h]; > + void@1_bunfastins_nocheck_noinc(bn[c],dst[c],0,BUN@2(b[t][c],cur[t][c])); > + cur[t][c] += bs[t][c]; > + dst[c] += bns[c]; > + } > +@ > +@= merged_union_5 > + /* @1: ATOMstorage(ttpe[c]) (chr, bte, sht, int, flt, lng, dbl, any) > + * @2: tloc, tvar, tail > + */ > + @:merged_union_4(@1,@2)@ > + break; > +@ > +@c > + for (c=1; c<ncols; c++) { > + if (ttpe[c]==TYPE_void) { > + /* at least one OID column is non-materialized (VoID) */ > + @:merged_union_4(oid,tail)@ > + } else if (ttpe[c]==TYPE_oid) { > + /* all OID columns are materialized */ > + @:merged_union_4(oid,tloc)@ > } else { > - any = b[i]->ttype; > - switch(ATOMstorage(b[i]->ttype)) { > - case TYPE_chr: @:merged_union_4(chr,tloc)@ > - case TYPE_sht: @:merged_union_4(sht,tloc)@ > - case TYPE_int: @:merged_union_4(int,tloc)@ > - case TYPE_flt: @:merged_union_4(flt,tloc)@ > - case TYPE_lng: @:merged_union_4(lng,tloc)@ > - case TYPE_dbl: @:merged_union_4(dbl,tloc)@ > + switch(ATOMstorage(ttpe[c])) { > + case TYPE_chr: @:merged_union_5(chr,tloc)@ > + case TYPE_bte: @:merged_union_5(bte,tloc)@ > + case TYPE_sht: @:merged_union_5(sht,tloc)@ > + case TYPE_int: @:merged_union_5(int,tloc)@ > + case TYPE_flt: @:merged_union_5(flt,tloc)@ > + case TYPE_lng: @:merged_union_5(lng,tloc)@ > + case TYPE_dbl: @:merged_union_5(dbl,tloc)@ > default: > - if (b[i]->tvarsized) { > - @:merged_union_4(any,tvar)@ > + if (b[0][c]->tvarsized) { > + @:merged_union_5(any,tvar)@ > } else { > - @:merged_union_4(any,tloc)@ > + @:merged_union_5(any,tloc)@ > } > } > } > @@ -5309,42 +5407,59 @@ > > /* set BAT properties */ > > - for (k=0; k<npairs; k++) { > - BAT *b0 = b[2*k], *b1 = b[(2*k)+1]; > - BATseqbase(bn[k], (oid)0); > - bn[k]->batBuns->free = dst[k] - bn[k]->batBuns->base; > - BATsetcount(bn[k], bn[k]->batBuns->free/BUNsize(bn[k])); > - if (!bn[k]->batDirty) bn[k]->batDirty = TRUE; > - BATkey(bn[k],TRUE); > - BATkey(BATmirror(bn[k]),FALSE); > - bn[k]->hsorted = GDK_SORTED; > - if (k==0 || (BATcount(b0)==0 && BATcount(b1)==0)) { > - bn[k]->tsorted = GDK_SORTED; > +@( > + /* not required as concat01 & concat10 are inherited from above */ > + if (non_empty[ntabs] == 2) { > + BAT *b0 = b[non_empty[0]][0]; > + BAT *b1 = b[non_empty[1]][0]; > + concat01 = concat10 = FALSE; > + if (BATtordered(b0)&BATtordered(b1)&1 || (BATtordered(b0)&1 && BATcount(b1)==1) || (BATcount(b0)==1 && BATtordered(b1)&1) ) { > + concat01 = atom_LE(BUNtail(b0,BUNlast(b0)-BUNsize(b0)),BUNtail(b1,BUNfirst(b1)),ATOMstorage(ATOMtype(ttpe[0]))); > + concat10 = !concat01 && \ > + atom_LT(BUNtail(b1,BUNlast(b1)-BUNsize(b1)),BUNtail(b0,BUNfirst(b0)),ATOMstorage(ATOMtype(ttpe[0]))); > + } > + } > +@) > + for (c=0; c<ncols; c++) { > + BATseqbase(bn[c], (oid)0); > + bn[c]->batBuns->free = dst[c] - bn[c]->batBuns->base; > + BATsetcount(bn[c], bn[c]->batBuns->free/bns[c]); > + assert(BATcount(bn[c]) == res_count); > + if (!bn[c]->batDirty) bn[c]->batDirty = TRUE; > + BATkey(bn[c],TRUE); > + BATkey(BATmirror(bn[c]),(res_count < 2)); > + bn[c]->hsorted = GDK_SORTED; > + if (c == 0 || res_count < 2) { > + bn[c]->tsorted = GDK_SORTED; > } else > - if (BATcount(b0)==0) { > - bn[k]->tsorted = (BATtordered(b1)&1 ? GDK_SORTED : FALSE); > + if (non_empty[ntabs] == 1) { > + bn[c]->tsorted = (BATtordered(b[non_empty[0]][c])&1 ? GDK_SORTED : FALSE); > } else > - if (BATcount(b1)==0) { > - bn[k]->tsorted = (BATtordered(b0)&1 ? GDK_SORTED : FALSE); > + if (non_empty[ntabs] == 2) { > + BAT *b0 = b[non_empty[0]][c]; > + BAT *b1 = b[non_empty[1]][c]; > + if ( ( BATtordered(b0)&BATtordered(b1)&1 || (BATtordered(b0)&1 && BATcount(b1)==1) || (BATcount(b0)==1 && BATtordered(b1)&1) ) && \ > + ( ( concat01 && atom_LE(BUNtail(b0,BUNlast(b0)-BUNsize(b0)),BUNtail(b1,BUNfirst(b1)),ATOMstorage(ATOMtype(ttpe[c]))) ) || \ > + ( concat10 && atom_LE(BUNtail(b1,BUNlast(b1)-BUNsize(b1)),BUNtail(b0,BUNfirst(b0)),ATOMstorage(ATOMtype(ttpe[c]))) ) ) ) { > + bn[c]->tsorted = GDK_SORTED; > + } else { > + bn[c]->tsorted = FALSE; > + } > } else { > - bn[k]->tsorted = ( ( concat && \ > - BATtordered(b0)&BATtordered(b1)&1 && \ > - atom_LE(BUNtail(b0,BUNlast(b0)-BUNsize(b0)),BUNtail(b1,BUNfirst(b1)),b0->ttype) ) \ > - ? GDK_SORTED \ > - : FALSE ); > + bn[c]->tsorted = FALSE; > } > - bn[k]->hdense = TRUE; > - bn[k]->tdense = FALSE; > + bn[c]->hdense = TRUE; > + bn[c]->tdense = FALSE; > } > > /* insert bn[] BATs in BN BAT */ > > DST = BUNlast(BN); > - BS = BUNsize(BN); > + BS = (bte)BUNsize(BN); > BATseqbase(BN, (oid)0); > - for (k=0; k<npairs; k++) { > - voidany_bunfastins_nocheck_noinc(BN,DST,0,&bn[k]->batCacheid); > - BBPunfix(bn[k]->batCacheid); > + for (c=0; c<ncols; c++) { > + voidany_bunfastins_nocheck_noinc(BN,DST,0,&bn[c]->batCacheid); > + BBPunfix(bn[c]->batCacheid); > DST += BS; > } > BN->batBuns->free = DST - BN->batBuns->base; > @@ -5360,34 +5475,67 @@ > *res = BN; > > GDKfree(w); > + BBPreclaim(temp); > > return GDK_SUCCEED; > bunins_failed: > GDKerror("merged_union: bunins failed.\n"); > cleanup: > BBPreclaim(BN); > - for (k=0; k<npairs; k++) { > - BBPreclaim(bn[k]); > + BBPreclaim(temp); > + for (c=0; c<ncols; c++) { > + BBPreclaim(bn[c]); > } > return GDK_FAIL; > } > > int CMDmerged_union( BAT** res, ptr L, int ltpe, ptr R, int rtpe, ... ) > { > - int i, tpe, nbats = 0, ret = GDK_SUCCEED; > - BAT *b[MAXPARAMS]; > + int tpe, nbats, t, ntabs = 2, ncols, ret = GDK_SUCCEED; > + BAT ***b; > va_list ap; > ptr p; > > + nbats = 2; > + va_start(ap,rtpe); > + while((p = va_arg(ap, ptr)) != NULL) { > + tpe = va_arg(ap, int); > + nbats++; > + } > + va_end(ap); > + if (nbats&1) { > + GDKerror("merged_union: uneven number of BATs: %d.\n", nbats); > + return GDK_FAIL; > + } > + ncols = nbats>>1; > + > + b = (BAT***)GDKmalloc(ntabs*sizeof(BAT**)); > + if (b == NULL) { > + GDKerror("merged_union: GDKmalloc(" SZFMT ") failed.\n", ntabs*sizeof(BAT**)); > + return GDK_FAIL; > + } > + for (t=0; t<ntabs; t++) { > + b[t] = (BAT**)GDKmalloc(ncols*sizeof(BAT*)); > + if (b[t] == NULL) { > + GDKerror("merged_union: GDKmalloc(" SZFMT ") failed.\n", ncols*sizeof(BAT*)); > + while (t>0) { > + GDKfree(b[--t]); > + } > + GDKfree(b); > + return GDK_FAIL; > + } > + } > + > + nbats = 0; > /* first convert any constant parameters to fake projects */ > - if (CMDfakeProject(b+nbats, L, ltpe) == GDK_SUCCEED) { > + if (CMDfakeProject(&b[nbats&1][nbats>>1], L, ltpe) == GDK_SUCCEED) { > nbats++; > - if (CMDfakeProject(b+nbats, R, rtpe) == GDK_SUCCEED) { > + if (CMDfakeProject(&b[nbats&1][nbats>>1], R, rtpe) == GDK_SUCCEED) { > nbats++; > va_start(ap,rtpe); > while((p = va_arg(ap, ptr)) != NULL) { > tpe = va_arg(ap, int); > - if (CMDfakeProject(b+nbats, p, tpe) == GDK_SUCCEED) { > + if (CMDfakeProject(&b[nbats&1][nbats>>1], p, tpe) == GDK_SUCCEED) { > nbats++; > } else { > ret = GDK_FAIL; > @@ -5398,11 +5546,104 @@ > } > } > if (ret == GDK_SUCCEED) { > - ret = merged_union(res, nbats, b); > + ret = merged_union(res, ntabs, ncols, b); > } > + > /* unfix all bats; this destroys any created fake projects */ > - for(i=0; i<nbats; i++) > - BBPunfix(b[i]->batCacheid); > + while (--nbats >= 0) { > + BBPunfix(b[nbats&1][nbats>>1]->batCacheid); > + } > + for(t=0; t<ntabs; t++) { > + GDKfree(b[t]); > + } > + GDKfree(b); > + return ret; > +} > + > +int CMDmulti_merged_union( BAT** res, BAT *Bt) > +{ > + int t, ntabs, c, ncols = 0, ret = GDK_SUCCEED; > + BAT **Bc, ***b; > + > + BATcheck(Bt, "multi_merged_union"); > + > + if (Bt->ttype != TYPE_bat) { > + GDKerror("multi_merged_union: tail-type must be BAT, not %s.\n", ATOMname(Bt->ttype)); > + return GDK_FAIL; > + } > + > + if ((ntabs = BATcount(Bt)) < 2) { > + GDKerror("multi_merged_union: input must contain at least 2 tables/BUNs.\n"); > + return GDK_FAIL; > + } > + > + Bc = (BAT**)GDKmalloc(ntabs*sizeof(BAT*)); > + if (Bc == NULL) { > + GDKerror("multi_merged_union: GDKmalloc(" SZFMT ") failed.\n", ntabs*sizeof(BAT*)); > + return GDK_FAIL; > + } > + > + for (t=0; t<ntabs; t++) { > + Bc[t] = BATdescriptor(*(bat*)BUNtloc(Bt, BUNptr(Bt,t))); > + if (Bc[t]->ttype != TYPE_bat) { > + GDKerror("multi_merged_union: tail-type of BAT holding table %d must be BAT, not %s.\n", t+1, ATOMname(Bc[t]->ttype)); > + ret = GDK_FAIL; > + } else if (t == 0) { > + assert(BATcount(Bc[t]) <= (size_t)GDK_int_max); > + if ((ncols = (int)BATcount(Bc[t])) < 1) { > + GDKerror("multi_merged_union: first table must consist of at least 1 column.\n"); > + ret = GDK_FAIL; > + } > + } else if (ncols != (int)BATcount(Bc[t])) { > + GDKerror("multi_merged_union: table %d (%d columns) must have as many columns as all other tables (%d columns).\n", > + t+1, (int)BATcount(Bc[t]), ncols); > + ret = GDK_FAIL; > + } > + if (ret == GDK_FAIL) { > + while (t >= 0) { > + BBPunfix(Bc[t--]->batCacheid); > + } > + GDKfree(Bc); > + return GDK_FAIL; > + } > + } > + > + b = (BAT***)GDKmalloc(ntabs*sizeof(BAT**)); > + if (b == NULL) { > + GDKerror("multi_merged_union: GDKmalloc(" SZFMT ") failed.\n", ntabs*sizeof(BAT**)); > + for (t=0; t<ntabs; t++) BBPunfix(Bc[t]->batCacheid); > + GDKfree(Bc); > + return GDK_FAIL; > + } > + for (t=0; t<ntabs; t++) { > + b[t] = (BAT**)GDKmalloc(ncols*sizeof(BAT*)); > + if (b[t] == NULL) { > + GDKerror("multi_merged_union: GDKmalloc(" SZFMT ") failed.\n", ncols*sizeof(BAT*)); > + while (t>0) { > + GDKfree(b[--t]); > + } > + GDKfree(b); > + for (t=0; t<ntabs; t++) BBPunfix(Bc[t]->batCacheid); > + GDKfree(Bc); > + return GDK_FAIL; > + } > + } > + > + for (t=0; t<ntabs; t++) { > + for (c=0; c<ncols; c++) { > + b[t][c] = BATdescriptor(*(bat*)BUNtloc(Bc[t], BUNptr(Bc[t],c))); > + } > + BBPunfix(Bc[t]->batCacheid); > + } > + GDKfree(Bc); > + > + ret = merged_union(res, ntabs, ncols, b); > + > + for (t=0; t<ntabs; t++) { > + for (c=0; c<ncols; c++) { > + BBPunfix(b[t][c]->batCacheid); > + } > + } > return ret; > } > > @@ -7561,33 +7802,6 @@ > return newProp; > } > > -PROC multi_merged_union (BAT[void, BAT] b) : BAT[void,BAT] > -{ > - var res := b.fetch(0); > - > - b.slice(1,count(b))@batloop() { > - var iter := res.fetch(0); > - var pre := res.fetch(1); > - var size := res.fetch(2); > - var level := res.fetch(3); > - var kind := res.fetch(4); > - var prop := res.fetch(5); > - var cont := res.fetch(6); > - var old_pre := res.fetch(7); > - var old_cont := res.fetch(8); > - res := merged_union (iter, $t.fetch(0), > - pre, $t.fetch(1), > - size, $t.fetch(2), > - level, $t.fetch(3), > - kind, $t.fetch(4), > - prop, $t.fetch(5), > - cont, $t.fetch(6), > - old_pre, $t.fetch(7), > - old_cont, $t.fetch(8)); > - } > - return res; > -} > - > > @- !!! THE FOLLOWING PROCs (*_constr) ARE DEPRECATED !!! > They are however not removed yet to support comparison > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

1 0

[Monetdb-developers] Query Timeout
by Bharani 18 Jul '07

18 Jul '07

Hi I am trying to make monetdb work with hibernate using existing mysql dialect. When i do a simple select it actually returns the value but i get an exception saying Method not supported,Sorry! for nl.cwi.monetdb.jdbc.MonetStatement.getQueryTimeout any reason why it is being thrown Thanks Bharani -- View this message in context: http://www.nabble.com/Query-Timeout-tf4102673.html#a11666902 Sent from the monetdb-developers mailing list archive at Nabble.com.

2 6

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/tools mserver5.mx.in, , 1.20, 1.21
by Stefan Manegold 17 Jul '07

17 Jul '07

On Tue, Jul 17, 2007 at 04:07:11AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/tools > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv10293/tools > > Modified Files: > mserver5.mx.in > Log Message: > My penny to cleanup the welcome message. see my change below. Stefan > Index: mserver5.mx.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/tools/mserver5.mx.in,v > retrieving revision 1.20 > retrieving revision 1.21 > diff -u -d -r1.20 -r1.21 > --- mserver5.mx.in 16 Jul 2007 11:51:44 -0000 1.20 > +++ mserver5.mx.in 17 Jul 2007 04:07:07 -0000 1.21 > @@ -174,15 +174,15 @@ > char *msg = mo_find_option(set, setlen, "monet_welcome"); > > if (msg && strcmp(msg, "yes") == 0) { > - printf("# MonetDB Server v" VERSION "\n"); > - printf("# based on GDK v%s\n", GDKversion()); > - printf("# Copyright (c) 1993-2007 CWI, all rights reserved\n"); > + printf("# MonetDB server v" VERSION "\n"); > + printf("# Based on kernel v%s\n", GDKversion()); > + printf("# Database serviced:%s\n", GDKgetenv("gdk_dbname")); ^^ shouldn't there be a space for sake of (human) readability? > printf("# Compiled for %s/" SZFMT "bit with " SZFMT "bit OIDs %s linked\n", HOST, (size_t) (sizeof(ptr) * 8), (size_t) (sizeof(oid) * 8), linkinfo); > #ifdef MONET_GLOBAL_DEBUG > - printf("# config:%s\n", GDKgetenv("config")); > - printf("# dbfarm:%s\n", GDKgetenv("gdk_dbfarm")); > + printf("# Configuration:%s\n", GDKgetenv("config")); ^^ shouldn't there be a space for sake of (human) readability? > + printf("# Database farm:%s\n", GDKgetenv("gdk_dbfarm")); ^^ shouldn't there be a space for sake of (human) readability? > #endif > - printf("# dbname:%s\n", GDKgetenv("gdk_dbname")); > + printf("# Copyright (c) 1993-2007 CWI, all rights reserved\n"); > printf("# Visit http://monetdb.cwi.nl/ for further information\n"); > } > } -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java build.properties, , 1.6, 1.7 build.xml, , 1.20, 1.21 release.txt, , 1.4, 1.5 version.sh, , 1.7, 1.8
by Stefan Manegold 16 Jul '07

16 Jul '07

On Mon, Jul 16, 2007 at 12:16:14PM +0000, Fabian wrote: > Update of /cvsroot/monetdb/clients/src/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18385/src/java > > Modified Files: > build.properties build.xml release.txt version.sh > Log Message: > For ease of maintenance, make jdbcclient a versionless jar-file. > [...] > > Index: version.sh > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/version.sh,v > retrieving revision 1.7 > retrieving revision 1.8 > diff -u -d -r1.7 -r1.8 > --- version.sh 16 Jul 2007 10:38:21 -0000 1.7 > +++ version.sh 16 Jul 2007 12:16:08 -0000 1.8 > @@ -113,13 +113,7 @@ > > for file in \ > Makefile.ag \ > - ../../../*/NT/MonetDB4-SQL/MonetDB4-Installer.vdproj \ > - ../../../*/NT/MonetDB5-SQL/MonetDB5-Installer.vdproj \ > - ../../../*/src/jdbc/tests/Tests/Test_JdbcClient.SQL.bat \ > ../../../*/src/jdbc/tests/Tests/Test.SQL.bat \ > - ../../../*/NT/MonetDB-XQuery/MonetDB-XQuery.vdproj \ > - ../../../*/NT/MonetDB4-XQuery/MonetDB4-Installer.vdproj \ ../../../*/NT/MonetDB4-SQL/MonetDB4-Installer.vdproj ../../../*/NT/MonetDB5-SQL/MonetDB5-Installer.vdproj ../../../*/NT/MonetDB-XQuery/MonetDB-XQuery.vdproj ../../../*/NT/MonetDB4-XQuery/MonetDB4-Installer.vdproj still contain "monetdb-1.6-jdbc.jar"; hence, this version number is now no longer maintained by clients/src/java/version.sh ... Stefan > - ../../../*/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.bat \ > ; do > if [[ -f ${file} ]] ; then > for f in $FILES ; do > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_box.mx, , 1.78, 1.79
by Sjoerd Mullender 16 Jul '07

16 Jul '07

Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12770/mal > > Modified Files: > mal_box.mx > Log Message: > insertToBox copies its 2nd and 3rd arguments, so it can declare them as > const. Nevertheless we still need a cast, because char * != str. > > > Index: mal_box.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_box.mx,v > retrieving revision 1.78 > retrieving revision 1.79 > diff -u -d -r1.78 -r1.79 > --- mal_box.mx 12 Jun 2007 17:21:17 -0000 1.78 > +++ mal_box.mx 16 Jul 2007 12:45:25 -0000 1.79 > @@ -226,7 +226,7 @@ > mal_export int releaseAllBox(Box box); > > mal_export int depositBox(Box box, str name, ValPtr val); > -mal_export void insertToBox(Box box, str name, str val); > +mal_export void insertToBox(Box box, const str name, const str val); > mal_export int takeBox(Box box, str name, ValPtr val, int tpe); > mal_export int bindBAT(Box box, str name, str location); > mal_export int releaseBox(Box box, str name); > @@ -418,7 +418,7 @@ > } > > void > -insertToBox(Box box, str nme, str val) > +insertToBox(Box box, const str nme, const str val) > { > ValRecord vr; > > > This is actually pretty useless. You've declared val as a const, not what val point to (i.e. the chars that make up the string). Consider this: typedef char *str; void foo(const str x) { *x = 0; } this is valid code and will be accepted by both gcc and icc, but this is exactly what you try to prevent from happening when you declare a parameter const char *. In other words, val should be declared as const char *val, and then you shouldn't need the case when calling the function. -- Sjoerd Mullender

2 2

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java build.properties, , 1.5, 1.6 build.xml, , 1.19, 1.20 release.txt, , 1.3, 1.4 version.sh, , 1.5, 1.6
by Stefan Manegold 16 Jul '07

16 Jul '07

On Mon, Jul 16, 2007 at 09:42:16AM +0000, Fabian wrote: > Update of /cvsroot/monetdb/clients/src/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5504 > > Modified Files: > build.properties build.xml release.txt version.sh > Log Message: > Fix for bug #1754452 > Store the given connection in the SendThread instead of ignoring it. > > Since we changed the code after the release, tag the version suffix > [...] > Index: version.sh > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/version.sh,v > retrieving revision 1.5 > retrieving revision 1.6 > diff -u -d -r1.5 -r1.6 > --- version.sh 6 Jun 2007 12:00:55 -0000 1.5 > +++ version.sh 16 Jul 2007 09:42:12 -0000 1.6 > @@ -106,10 +106,13 @@ > > file="release.txt" > sed \ > - -e "s|version ${ESC_MAJOR}\.${ESC_MINOR} (${ESC_SUFFIX})|version ${NEW_MAJOR}.${NEW_MINOR} $${NEW_SUFFIX}$|g" \ > + -e "s|version ${ESC_MAJOR}\.${ESC_MINOR} (${ESC_SUFFIX}|version ${NEW_MAJOR}.${NEW_MINOR} \(${NEW_SUFFIX}|g" \ > + -e "s|${TYPE}-${ESC_MAJOR}\.${ESC_MINOR}|${TYPE}-${NEW_MAJOR}.${NEW_MINOR}|g" \ > -e "s|Release date: 20[0-9][0-9]-[01][0-9]-[0-3][0-9]|Release date: `date +%F`|" \ > ${file} | ${diff} ${file} - | ${patch} > > +# not found: > +# ../../../*/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.bat \ ======== $ l */tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.bat -------- -rwxr-x--- 1 manegold ins1 376 Jun 6 14:16 pathfinder/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.bat* ======== $ cat */tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.bat -------- @echo off set JAR="%CLIENTS_PREFIX%\share\MonetDB\lib\jdbcclient-1.6.jar" echo user=monetdb> .monetdb echo password=monetdb>> .monetdb set LANG=en_US.UTF-8 call Mlog.bat -x java -jar "%JAR%" -h %HOST% -p %MAPIPORT% -d %TSTDB% -l xquery -f "%TST%.250.xq" call Mlog.bat -x java -jar "%JAR%" -h %HOST% -p %MAPIPORT% -d %TSTDB% -l xquery -f "%TST%.251.xq" del .monetdb ======== maybe, we should find some way to use the "latest" jdbcclient.jar on both unix and windows platforms without knowing the exact version number ... Stefan > for file in \ > Makefile.ag \ > ../../../*/NT/MonetDB4-SQL/MonetDB4-Installer.vdproj \ > @@ -118,7 +121,6 @@ > ../../../*/src/jdbc/tests/Tests/Test.SQL.bat \ > ../../../*/NT/MonetDB-XQuery/MonetDB-XQuery.vdproj \ > ../../../*/NT/MonetDB4-XQuery/MonetDB4-Installer.vdproj \ > - ../../../*/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.bat \ > ; do > if [[ -f ${file} ]] ; then > for f in $FILES ; do > @@ -138,5 +140,4 @@ > -e "s|${TYPE}_MAJOR=${ESC_MAJOR}|${TYPE}_MAJOR=${NEW_MAJOR}|g" \ > -e "s|${TYPE}_MINOR=${ESC_MINOR}|${TYPE}_MINOR=${NEW_MINOR}|g" \ > -e "s|${TYPE}_VER_SUFFIX=${ESC_SUFFIX}|${TYPE}_VER_SUFFIX=${NEW_SUFFIX}|g" \ > - -e "s|${TYPE}-${ESC_MAJOR}\.${ESC_MINOR}|${TYPE}-${NEW_MAJOR}.${NEW_MINOR}|g" \ > ${file} | ${diff} ${file} - | ${patch} > [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk.mx, , 1.234, 1.235 gdk_value.mx, , 1.37, 1.38
by Stefan Manegold 16 Jul '07

16 Jul '07

On Mon, Jul 16, 2007 at 07:20:22AM +0000, Fabian wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19433 > > Modified Files: > gdk.mx gdk_value.mx > Log Message: > Added GDKversion function. I do not know what the best place for the > implementation is, so I just chose a file. what about gdk_utils.mx ? seems more intuitive tham gdk_value.mx to me ... > Putting it in gdk.mx itself does not seem to work. (symbol not exported) gdk.mx only contains the gdk.h header file (cf., `grep '^@[a-z]$' MonetDB/src/gdk/gdk.mx`). Stefan > > > Index: gdk.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk.mx,v > retrieving revision 1.234 > retrieving revision 1.235 > diff -u -d -r1.234 -r1.235 > --- gdk.mx 9 Jul 2007 08:36:32 -0000 1.234 > +++ gdk.mx 16 Jul 2007 07:20:18 -0000 1.235 > @@ -1612,7 +1612,6 @@ > gdk_export int GDKsave(str nme, str ext, void *buf, size_t size, int mode); > gdk_export int GDKunlink(str dir, str nme, str extension); > gdk_export int GDKmove(str dir1, str nme1, str ext1, str dir2, str nme2, str ext2); > - > @ > @} > > @@ -3275,6 +3274,16 @@ > @ > > @h > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +gdk_export char *GDKversion(); > + > +#ifdef __cplusplus > +} > +#endif > + > @{ > #define ILLEGALVALUE ((ptr)-1L) > #define MAXPARAMS 32 > > Index: gdk_value.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_value.mx,v > retrieving revision 1.37 > retrieving revision 1.38 > diff -u -d -r1.37 -r1.38 > --- gdk_value.mx 2 May 2007 16:17:06 -0000 1.37 > +++ gdk_value.mx 16 Jul 2007 07:20:19 -0000 1.38 > @@ -697,5 +697,11 @@ > return p == ILLEGALVALUE ? p : VALptr(t); > } > > + > +static char *_gdk_version_string = VERSION; > +char *GDKversion() { > + return(_gdk_version_string); > +} > + > @} > @ > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet4 sql_server.mx, , 1.169, 1.170
by Stefan Manegold 14 Jul '07

14 Jul '07

Ok, before we start changing these things back and forth: Anyone else who has an opinion on how M4 (server), M5 (server), SQL4, SQL5, XQuery(4) should reveal their version numbers at startup-/module-loading time, and whether the differences between M4 & M5 should/need to be increased by having different/non-aligned/"inconsistend" welcome messages? Stefan On Sat, Jul 14, 2007 at 08:49:25AM +0200, Fabian Groffen wrote: > On 13-07-2007 21:33:50 +0000, Stefan Manegold wrote: > > Update of /cvsroot/monetdb/sql/src/backends/monet4 > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15825/sql/src/backends/monet4 > > > > Modified Files: > > sql_server.mx > > Log Message: > > > > I do not argue about the exact wording, > > but let's at least (try to) keep it aligned/consistent > > Can you please undo this commit? On purpose I do NOT want it to be > aligned, because I want to see a clear distinction between MonetDB 5 and > MonetDB 4. I'd like it when M4 would annouce itself as MonetDB Database > Server for that same reason. (i.e. reverting that earlier commit as > well) > > There is absolutely no need to make it harder to see what engine you're > dealing with. MonetDB 5 should and must (IMO) look different, newer, > better and more advanced. (Because it is.) > > So please, don't align M4 with M5. > > > > Index: sql_server.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/sql/src/backends/monet4/sql_server.mx,v > > retrieving revision 1.169 > > retrieving revision 1.170 > > diff -u -d -r1.169 -r1.170 > > --- sql_server.mx 13 Jul 2007 15:35:58 -0000 1.169 > > +++ sql_server.mx 13 Jul 2007 21:33:48 -0000 1.170 > > @@ -1579,5 +1579,4 @@ > > > > sql_server_start(); > > if (monet_environment.find("monet_welcome") = "yes") > > - printf("%c MonetDB/SQL Server v2.19.0 on port %s\n", > > - int(35), monet_environment.find("mapi_port")); > > + printf("%c MonetDB/SQL module v2.19.0 loaded\n", int(35)); > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > _______________________________________________ > > Monetdb-sql-checkins mailing list > > Monetdb-sql-checkins(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 4

[Monetdb-developers] Q: pathfinder - cdata
by Arjen P. de Vries 13 Jul '07

13 Jul '07

While <a /> is accepted as a query, <![CDATA[<]]> is not. !parse error: syntax error, unexpected <![CDATA[ on line 1, column 9 (next token is `<![CDATA[') !parse error: XQuery parsing failed Is that the expected behaviour? Context: I try to output email addresses as "test <test(a)my.org>" and failing Arjen -- ==================================================================== CWI, room C1.16 Centre for Mathematics and Computer Science Kruislaan 413 Email: Arjen.de.Vries(a)cwi.nl 1098 SJ Amsterdam tel: +31-(0)20-5924306 The Netherlands fax: +31-(0)20-5924312 ===================== http://www.cwi.nl/~arjen/ ====================

3 5

[Monetdb-developers] [M5] RFC: function shipping semantics
by Fabian Groffen 13 Jul '07

13 Jul '07

Hi all, I'd like some input on an issue that I can't get a clear vision on for now. Recently I added the remote module in MonetDB 5 which basically has 3 functions which perform actions with a remote (MonetDB 5) server. The functions get, put and exec retrieve, store and execute an object from, to or on a remote host. get and put allow for retrieval and storage of basically any type, scalars, strings, BATs, etc. exec only executes functions available on the remote host. The idea is to have one to "put()" a function to the remote host, and then execute it, retrieving its return value. exec is currently implemented that way, however, get and put of functions is not because it is unclear to me how this should exactly happen. For the case of put() I can think of the following which seems quite intuitive and ok to me: function user.func(v:int):str; r := "something"; return(r); end func; m := remote.put("host", "user.func(:int):str"); t := 5; q := remote.exec("host", m, t); io.print(q); The example puts function func to the remote host and executes it with argument t which is 5. The return string kept in q will contain the string "something". Because the identifier of the function on the remote host is returned and assigned to m, the function can easily be executed, since exec requires the function name as string. Currently that function name is just the function without its signature. The signature (which is not required, even not desired currently) can be deduced from the arguments given, resulting in a normal function resolution and (slightly misleading) error reporting if that fails. The other way around (thus to get a remote function) is still unclear to me how to do it. I could imagine the following that should work: v := remote.get("host", m); w := remote.get("host", "user.func(:int):str"); Problem with v is that m is not a full signature, so a conflict may arise if m is overloaded. This is not necessarily a problem, since m is a unique identifier, so no overloading for it exists. However, it is a requirement that a get after a put should always succeed and result in the same object that was put. The bigger problem here, however, is how to deal with the function being get. Since a function cannot be stored in a variable in MAL, it has to be stored on the local machine, or its contents as a string returned. In the first case the get() function would return its local identifier. In the latter case it would return a (big) string. In both cases, the function cannot be "used" as function in the program. A variable containing a function name cannot be used as in e.g.: k := v(t); where v, a variable, would be used as function, "dereferencing" v's value or something. Of course with a human factor one can do: io.print(v); and use the value of v to construct a call to the locally stored function. In the case of a string, the function has to be made, which I don't know how to do that, but would give explicit control over its name to the program. I don't think the concept of eval is clearing up things, as well as possible at all (if you think about the k := statement where only v should be expanded, not t neither k). Ideas?

3 3

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/Tests remote00.mal, , 1.3, 1.4 remote00.stable.out, , 1.4, 1.5
by Martin Kersten 10 Jul '07

10 Jul '07

Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv4482/Tests > > Modified Files: > remote00.mal remote00.stable.out > Log Message: > Be a little bit smarter when dealing with functions. Allow dotted > notation for <module>.<func> next to just <func> for the default/current > module. (currently I guess this is "user", but should be retrieved > somehow to be on the safe site...) Yes, the default is the user module. I have been 'struggling' with arguments for reflective functions. I settled down to using two arguments where needed e.g. f(mod:str,fcn:str,...) It is not optimal, but avoid string construction/analysis

3 5

[Monetdb-developers] Swallowed logmessage of /cvsroot/monetdb/pathfinder/compiler/Tests/usageH.stable.out; /cvsroot/monetdb/pathfinder/compiler/main.c
by Manuel Mayr 09 Jul '07

09 Jul '07

Sorry, somehow SF accomplished to swallow my Log-Messages for the last two checkins. I set them afterwards with 'cvs admin' to "Pathfinders SQL support is now available with -S." Regards, Manuel -- Manuel Mayr, Dipl. Inf. Institut fuer Informatik, TU Muenchen Tel.: +49 89 289-17288

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_statistics.mx, , 1.3, 1.4
by Martin Kersten 04 Jul '07

04 Jul '07

Thanks! It is not yet working. The test shows wrong bats. Looking into it.

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra load_stats.c, , 1.2, 1.3
by Stefan Manegold 04 Jul '07

04 Jul '07

On Tue, Jul 03, 2007 at 02:20:25PM +0000, Alexander Gafriller wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26332 > > Modified Files: > load_stats.c > Log Message: > * compiler error fixed > > /scratch/monet.GNU.64.64.s.25128/pathfinder/compiler/algebra/load_stats.c:84: warning: pointer targets in passing argument 1 of 'atoi' differ in signedness > /scratch/monet.GNU.64.64.s.25128/pathfinder/compiler/algebra/load_stats.c:86: warning: pointer targets in passing argument 1 of 'atoi' differ in signedness > /scratch/monet.GNU.64.64.s.25128/pathfinder/compiler/algebra/load_stats.c:88: warning: pointer targets in passing argument 1 of 'atoi' differ in signedness Hi Alexander, thanks for noticing and fixing it instantly! A perfect start as new MonetDB developer!! ;-) > and > > /scratch/monet.ntv.64.32.d.25128/pathfinder/compiler/debug/../include/load_stats.h(71): error #310: old-style parameter list (anachronism) > PFguide_tree_t* PFguide_tree(); > ^ this one still failes; I guess, your forgot to checkin your changes in pathfinder/compiler/include/load_stats.h ... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/benchmarks/tpch/Tests 01-22.stable.out, , 1.8, 1.9 01.stable.out, , 1.10, 1.11 18.stable.out, , 1.9, 1.10
by Martin Kersten 29 Jun '07

29 Jun '07

Gaan we Venks en Colin separaat benadering om te testen?

1 0

[Monetdb-developers] build problem for java jdbc driver
by Frans Verster 23 Jun '07

23 Jun '07

Hi monetdb-developers, I just build the MonetDB-Venus-SuperBall-SR1, the build of the c code went ok (nice!) except for the java jdbc driver I do find checking for javac itself in the build log but not the actual use of it. Do i build correctly? /var/tmp/MonetDB-1182546749/log : Fri Jun 22 23:12:29 CEST 2007 called with arguments: --prefix=/tmp/frans/MonetDB --enable-sql --enable-xquery Calling javac (1.6.0) with all/most files does not help: javac nl/cwi/monetdb/jdbc/*.java nl/cwi/monetdb/jdbc/MonetDataSource.java:57: cannot find symbol symbol : class MonetDriver location: class nl.cwi.monetdb.jdbc.MonetDataSource private MonetDriver driver; ^ etc... The only file with that name is /var/tmp/MonetDB-1182546749/MonetDB-client/clients/src/java/src/nl/cwi/monetdb/jdbc/MonetDriver.java.in but renaming doesnt help. and for an other file: nl/cwi/monetdb/jdbc/MonetClob.java:35: nl.cwi.monetdb.jdbc.MonetClob is not abstract and does not override abstract method getCharacterStream(long,long) in java.sql.Clob public class MonetClob implements Clob { ^ Did the jdbc changed with java 1.6, is the code compatible? Howto proceed? Thanks in advance Frans Verster

2 1

[Monetdb-developers] problem building monetdb from cvs stable
by Colin Foss 22 Jun '07

22 Jun '07

I've read some recent emails about stable having issues so forgive me if this is expected behaviour at the moment. When I run monetdb-install.sh --enable-sql --prefix=/var/lib/MonetDB --enable-optimise --cvs=stable I receive the following cvs error: cvs [export aborted]: no such tag stable Please refer to the full log at /var/tmp/MonetDB-1182455148/log If you believe this is an error in the script or the software, file a bug and attach the logfile. exporting MonetDB from CVS failed! monetdb-install.sh: line 204: glibtool: command not found Am I doing something wrong?

2 1

Re: [Monetdb-developers] [MonetDB-users] SQL server startup time problem (undo/redo problem?)
by Martin Kersten 20 Jun '07

20 Jun '07

Dear James, Can you specify the system you are working on, e.g. hardware/os. You are really beating the system. I don't know a system that can handle a TPC-H SF100 out of the box. This size often require a multistage process and careful setting of the system parameters. But, let's see what you have done and learn from your experience. James Laken wrote: > Dear MonetDB Developers and Users, > > My TPC-H SF100 test is still producing interesting problems. After > several hours of work I have managed to import near 420 million record > to the lineitem table (7x 60 million record slice). Accidentally I Ok, you perform a sliced base load. i.e. 7 x a SF-10 > killed the import process, and stopped the server process. I have tried Did you stop and restart the server between the loads? If not, then from a recovery point of view all 420M are stored in a single log file and become the target of a single reload. It behaves as if you loaded the 7x60M as a single batch. Killing a database process is of course hard. In that case, the recovery process has to reload the data and enters a really expensive part of TPC-H: ensure correctness of the integrity relationships. Protection against this is hard, because it requires that integrity rules enforcement should either be disabled (the method persued in MySQL). > to restart the server process but after three hours of intensive > processing the sql module still not started. Please note that the > initialization process allocated nearly all memory and swap. This is what we expect. Your tables require a lot of space, because MonetDB does not automatically partitioning it. (That's scheduled for an upcoming release ;-)) > > I have attached a gdb to the server process and the execution stacks > looks like this: > > Program received signal SIGINT, Interrupt. > 0x00002b240cb1c2a4 in file_read () from /usr/lib/libstream.so.0 > (gdb) where > #0 0x00002b240cb1c2a4 in file_read () from /usr/lib/libstream.so.0 > #1 0x00002b240cb1b744 in stream_readLngArray () from > /usr/lib/libstream.so.0 > #2 0x00002b240c67a27f in lngRead () from /usr/lib/libbat.so.0 > #3 0x00002b240c788e90 in logger_readlog () from /usr/lib/libbat.so.0 > #4 0x00002b240c789b0c in logger_create () from /usr/lib/libbat.so.0 > #5 0x00002aaaaab42658 in store_init () from > /usr/lib/MonetDB5/lib/lib_sql.so > #6 0x00002aaaaab179b3 in mvc_init () from /usr/lib/MonetDB5/lib/lib_sql.so > #7 0x00002aaaaaacf9f4 in SQLinit () from /usr/lib/MonetDB5/lib/lib_sql.so > #8 0x00002b240bf89e3a in initScenario () from /usr/lib/libmal.so.0 > #9 0x00002aaaaaacf968 in SQLsession () from > /usr/lib/MonetDB5/lib/lib_sql.so > #10 0x00002b240bf6536e in runMALsequence () from /usr/lib/libmal.so.0 > #11 0x00002b240bf6697b in runMAL () from /usr/lib/libmal.so.0 > #12 0x00002b240bf5f3f3 in MALengine () from /usr/lib/libmal.so.0 > #13 0x00002b240bf5e3ab in callString () from /usr/lib/libmal.so.0 > #14 0x0000000000402a65 in main () > (gdb) c > Continuing. > > Program received signal SIGINT, Interrupt. > 0x00002b240d6c31a0 in malloc () from /lib/libc.so.6 > (gdb) where > #0 0x00002b240d6c31a0 in malloc () from /lib/libc.so.6 > #1 0x00002b240c675c04 in GDKmallocmax () from /usr/lib/libbat.so.0 > #2 0x00002b240c675da9 in GDKmalloc () from /usr/lib/libbat.so.0 > #3 0x00002b240c6787ee in strRead () from /usr/lib/libbat.so.0 > #4 0x00002b240c788e90 in logger_readlog () from /usr/lib/libbat.so.0 > #5 0x00002b240c789b0c in logger_create () from /usr/lib/libbat.so.0 > #6 0x00002aaaaab42658 in store_init () from > /usr/lib/MonetDB5/lib/lib_sql.so > #7 0x00002aaaaab179b3 in mvc_init () from /usr/lib/MonetDB5/lib/lib_sql.so > #8 0x00002aaaaaacf9f4 in SQLinit () from /usr/lib/MonetDB5/lib/lib_sql.so > #9 0x00002b240bf89e3a in initScenario () from /usr/lib/libmal.so.0 > #10 0x00002aaaaaacf968 in SQLsession () from > /usr/lib/MonetDB5/lib/lib_sql.so > #11 0x00002b240bf6536e in runMALsequence () from /usr/lib/libmal.so.0 > #12 0x00002b240bf6697b in runMAL () from /usr/lib/libmal.so.0 > #13 0x00002b240bf5f3f3 in MALengine () from /usr/lib/libmal.so.0 > #14 0x00002b240bf5e3ab in callString () from /usr/lib/libmal.so.0 > #15 0x0000000000402a65 in main () > > Any idea? > > Regards, > J. > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users

2 2

[Monetdb-developers] MonetDB XQuery 0.18.2
by Sjoerd Mullender 19 Jun '07

19 Jun '07

A bug fix release for MonetDB XQuery 0.18.0 has been done. Two smallish bugs have been fixed in this release: - compiling an XQuery query took much longer than warranted due to a configuration error; - pftijah was accidentally omitted from the Windows builds. The promised MonetDB/SQL release will remain a promise. The current head of the Stable branch is still not yet stable, unfortunately. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal tablet.mx, , 1.93, 1.94
by Romulo Goncalves 19 Jun '07

19 Jun '07

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14370 > > Modified Files: > tablet.mx > Log Message: > There is no guarantee that the table descriptor has a BAT > assigned to the first field. So you may not perform a BATcount. After this checkin (I still have to check if I did not change the code). The crash in the mserver with the skyserver data is gone. I will run more experiments to check it. Regards, Romulo > > > Index: tablet.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet.mx,v > retrieving revision 1.93 > retrieving revision 1.94 > diff -u -d -r1.93 -r1.94 > --- tablet.mx 16 Jun 2007 07:54:02 -0000 1.93 > +++ tablet.mx 19 Jun 2007 05:54:56 -0000 1.94 > @@ -1495,7 +1495,7 @@ > TABLETload_file(Tablet * as, bstream *b, stream *out) > { > int res = 0, done = 0; > - size_t i = 0; > + size_t i = 0, tuples=0; > char *sep = as->format[as->nr_attrs - 1].sep; > int seplen = as->format[as->nr_attrs - 1].seplen; > > @@ -1551,9 +1551,10 @@ > as->error=0; > GDKerror("TABLETload_file: read error " > "(after loading %d records)\n", > - BATcount(as->format[0].c)); > + tuples); > res = -1; > } > + tuples++; > break; > } > end = b->buf + b->len; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 1

Re: [Monetdb-developers] [MonetDB-users] Importing large dataset (TPC-H SF100)
by Martin Kersten 18 Jun '07

18 Jun '07

Dear All, The problems reported on loading have our full attention. They are hard to reproduce. It takes a long time to reach the point were it breaks. BUT, we can reproduce (hopefully) the bug (It seems a memory overwrite). Due to a local science meeting and priority (they pay for the development of MonetDB), the concerted frontal attack on the bug will start upcoming Wednesday. Please stay with us. regards, Martin James Laken wrote: > I forgot to mention that the load script use the following formula: > COPY N RECORDS INTO table from 'path/to/importfile' ... where N is the > number of records in the inputfile. > > Regards, > Zoltan > > > > Colin Foss wrote: >> James, >> >> I've had success avoiding "swap-of-death" with COPY >> INTO by specifying the number of records on each >> statement. > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users

1 0

[Monetdb-developers] Importing large dataset (TPC-H SF100)
by James Laken 18 Jun '07

18 Jun '07

Hi, I am trying to import the TPC-H dataset (SF100) to the database without success. The import method is the same as in the benchmark/tpch, except the number of expected records in the load script and the load script execution (executed line-by-line in the console). The machine is an dual core AMD64 (64bit OS), with 4G ram and 8 disk raid0 for storage. The import process consume nearly all memory and nearly all the swap (please note that the number of expected records is specified after the COPY command). In fact I have to restart the server process after each table import to prevent the "swap to death" state. However it seems that the lineitem table import will fail, no matter what I do. I have tried it with different client (eg.: mjclient with -Xbatching mode), sliced lineitem.tlb file, with the same result. I have noticed that two or three hours after the issued lineitem COPY command the mserver5 process does not consume CPU any more. The attached strace shows me the following: [pid 4843] select(6, [5], NULL, NULL, {0, 500}) = 0 (Timeout) [pid 4843] select(6, [5], NULL, NULL, {0, 500}) = 0 (Timeout) [pid 4843] select(6, [5], NULL, NULL, {0, 500}) = 0 (Timeout) Any idea? Is there any way to import huge datasets (TB) in bulk mode to the database? For example the postgresql database has a feature that it can import the data without write ahead logging, nearly at disk speed. The dataset can be sliced up per column, so a direct column copy would be possible. Regards, J.

1 0

[Monetdb-developers] new release
by Sjoerd Mullender 15 Jun '07

15 Jun '07

Some problems have been found in the X.18.0 release that we've created in both SQL and XQuery. Because of that we're planning to do a minor release of just those two packages on Tuesday. In SQL, the error was a crash of the server when adding a foreign key to an existing table. In XQuery, the problem was mainly a performance issue (and missing pftijah functionality in the Windows builds). -- Sjoerd Mullender

1 0

[Monetdb-developers] Release is done
by Sjoerd Mullender 14 Jun '07

14 Jun '07

The new release (X.18.0 for various values of X, and 5.0.0) is available on Sourceforge. The CVS Stable branch is once again open for bug fixing (and *only* bug fixing). As soon as the web site has been updated, you can see the full release notes at http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Version4.18/ind… Niels, can you update the web site? Martin, can you send the announcements? -- Sjoerd Mullender

2 1

[Monetdb-developers] Dealing with large tables
by Peter Boncz 14 Jun '07

14 Jun '07

Hi Augustin, The crash is a bug. You have not told us about your platform (Darwin, I presume) and whether you have compiled with 64-bits enabled. Considering the data is 'large', it may be an out-of-memory problem but then in any case it is a bug, because lack of resources should lead to the query failing rather than crashing. So, first provide us the missing platform info (is it 64 bits????), and second, recompile with debugging enabled (configure with: --enable-debug=yes --enable-optimize=no). Since this problem seems highly reproducable, then run again following my instructions: (0) start Mserver in gdb (gdb --args Mserver ...args''') (1) modify the MIL script to set the debugmask 19 (2) also insert some extra info() calls (3) when it crashes, type in the gdb console: bt (4) send us all the output Your modified MIL script should resemble: debugmask(19); var var_0 := [=](col1,col2).select(true).mirror(); var_0.info().print(); col1.info().print(); col1.delete(var_0); commit(); col2.info().print(); col2.delete(var_0); commit(); col3.info().print(); col3.delete(var_0); commit(); thanks in advance, Peter ------------------------------ Message: 6 Date: Thu, 14 Jun 2007 11:59:24 -0400 From: Agustin Schapira <schapira(a)cs.umass.edu> Subject: [Monetdb-developers] Dealing with large tables To: monetdb-developers(a)lists.sourceforge.net Message-ID: <9FAD9406-07DC-454A-9AB0-A9C9DFD05153(a)cs.umass.edu> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Dear Monet developers, Once again, here I am, writing for help ;-) We are working with a 'large' table, with 250 million rows, 11 columns. This takes up a lot of memory space, as you can imagine: a single column of 250 million 'ints' requires almost a gigabyte, and there's several of those; in fact, I think that the entire table takes up a bit over 6GB in memory (some columns are strings, and can be compressed). The problem is that we have a machine with 8GB, and every time we try to do something with that table, Monet (4.16.2) crashes. For example, I've been trying to delete rows from the table where two columns have the same value: var var_0 := [=](col1,col2).select(true).mirror(); col1.delete(var_0); commit(); col2.delete(var_0); commit(); col3.delete(var_0); commit(); ... After deleting from a few of those columns, Monet dies: !ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), please restart. !ERROR: (One potential cause could be that your disk might be full...) Do you know if there a fix for this? In principle, Monet should be able to unload the BATs that it has already deleted from, to release memory. Also, would doing a semijoin() instead of a delete() help? (I though that a semijoin would need to create a new BAT, thereby using even more memory!). And, in general: is Monet 5 capable of accessing BATs that don't entirely fit in memory? Would the new version fix this problem? Maybe it's time for us to adapt it... Thanks again for all your help, and regards from Amherst, -- Agustin ------------------------------ Message: 7 Date: Thu, 14 Jun 2007 18:29:05 +0200 From: Martin Kersten <Martin.Kersten(a)cwi.nl> Subject: Re: [Monetdb-developers] Dealing with large tables To: Agustin Schapira <schapira(a)cs.umass.edu> Cc: monetdb-developers(a)lists.sourceforge.net Message-ID: <46716CD1.8080104(a)cwi.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Agustin Schapira wrote: > Dear Monet developers, > > Once again, here I am, writing for help ;-) > > We are working with a 'large' table, with 250 million rows, 11 > columns. This takes up a lot of memory space, as you can imagine: a > single column of 250 million 'ints' requires almost a gigabyte, and I would say 1GB is a minimum. What if you are on a 64bit machine? And also, if this table requires a hash index, then add another 1GB/table. > there's several of those; in fact, I think that the entire table > takes up a bit over 6GB in memory (some columns are strings, and can > be compressed). The problem is that we have a machine with 8GB, and > every time we try to do something with that table, Monet (4.16.2) > crashes. For example, I've been trying to delete rows from the table > where two columns have the same value: > > var var_0 := [=](col1,col2).select(true).mirror(); Hard to deduce what's happening. > col1.delete(var_0); > commit(); > col2.delete(var_0); > commit(); > col3.delete(var_0); > commit(); I would first try to make sure if the commit is not sitting in the way. Do you need to commit here ? > ... > > After deleting from a few of those columns, Monet dies: > > !ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), > please restart. > !ERROR: (One potential cause could be that your disk might be full...) > > > Do you know if there a fix for this? In principle, Monet should be > able to unload the BATs that it has already deleted from, to release > memory. Also, would doing a semijoin() instead of a delete() help? (I > though that a semijoin would need to create a new BAT, thereby using > even more memory!). And, in general: is Monet 5 capable of accessing > BATs that don't entirely fit in memory? Would the new version fix > this problem? Maybe it's time for us to adapt it... > > Thanks again for all your help, and regards from Amherst, > > -- Agustin > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers ------------------------------ ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ------------------------------ _______________________________________________ Monetdb-developers mailing list Monetdb-developers(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers End of Monetdb-developers Digest, Vol 13, Issue 6 *************************************************

1 0

[Monetdb-developers] Dealing with large tables
by Agustin Schapira 14 Jun '07

14 Jun '07

Dear Monet developers, Once again, here I am, writing for help ;-) We are working with a 'large' table, with 250 million rows, 11 columns. This takes up a lot of memory space, as you can imagine: a single column of 250 million 'ints' requires almost a gigabyte, and there's several of those; in fact, I think that the entire table takes up a bit over 6GB in memory (some columns are strings, and can be compressed). The problem is that we have a machine with 8GB, and every time we try to do something with that table, Monet (4.16.2) crashes. For example, I've been trying to delete rows from the table where two columns have the same value: var var_0 := [=](col1,col2).select(true).mirror(); col1.delete(var_0); commit(); col2.delete(var_0); commit(); col3.delete(var_0); commit(); ... After deleting from a few of those columns, Monet dies: !ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), please restart. !ERROR: (One potential cause could be that your disk might be full...) Do you know if there a fix for this? In principle, Monet should be able to unload the BATs that it has already deleted from, to release memory. Also, would doing a semijoin() instead of a delete() help? (I though that a semijoin would need to create a new BAT, thereby using even more memory!). And, in general: is Monet 5 capable of accessing BATs that don't entirely fit in memory? Would the new version fix this problem? Maybe it's time for us to adapt it... Thanks again for all your help, and regards from Amherst, -- Agustin

3 3

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder MonetDB-XQuery.spec.in, XQuery_0-18, 1.21, 1.21.2.1
by Stefan Manegold 13 Jun '07

13 Jun '07

On Mon, Jun 11, 2007 at 01:37:14PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv2101 > > Modified Files: > Tag: XQuery_0-18 > MonetDB-XQuery.spec.in > Log Message: > There is no need to specify that the version of MonetDB4-server needs > to be below 5.0 because it is by definition. > Also, make the minimum requirement 4.18. good point --- I apperantly overlooked that one when updateing the vertoo configuration --- should we also handle this via vertoo? Stefan > Index: MonetDB-XQuery.spec.in > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/MonetDB-XQuery.spec.in,v > retrieving revision 1.21 > retrieving revision 1.21.2.1 > diff -u -d -r1.21 -r1.21.2.1 > --- MonetDB-XQuery.spec.in 11 Feb 2007 22:04:35 -0000 1.21 > +++ MonetDB-XQuery.spec.in 11 Jun 2007 13:37:11 -0000 1.21.2.1 > @@ -39,7 +39,7 @@ > %package devel > Summary: MonetDB XQuery development package > Group: System > -Requires: MonetDB4-server>=4.15<5.0 > +Requires: MonetDB4-server >= 4.18 > > %description devel > Add the MonetDB devel description here > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 3

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/conf MonetDB.m4, MonetDB_1-18, 1.16.2.6, 1.16.2.7
by Stefan Manegold 13 Jun '07

13 Jun '07

On Fri, Jun 08, 2007 at 02:03:41PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/buildtools/conf > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19133 > > Modified Files: > Tag: MonetDB_1-18 > MonetDB.m4 > Log Message: > Ported to Fedora (no Core) 7. Thanks, Sjoerd! ... did you also update TestTools/.Mconfig.rc & TestTools/MdoServer accordingly? (sorry, due bad network and/but good and intensive networking, I haven't followed all check in message...) It's not curcial, until we have an F67 machine in our testing pool, but we shuld not "forget" it ... Greetings from a hot, busy, foggy Beijing! Stefan > > Index: MonetDB.m4 > =================================================================== > RCS file: /cvsroot/monetdb/buildtools/conf/MonetDB.m4,v > retrieving revision 1.16.2.6 > retrieving revision 1.16.2.7 > diff -u -d -r1.16.2.6 -r1.16.2.7 > --- MonetDB.m4 8 Jun 2007 08:19:19 -0000 1.16.2.6 > +++ MonetDB.m4 8 Jun 2007 14:03:38 -0000 1.16.2.7 > @@ -412,7 +412,7 @@ > dnl Please keep this aligned / in sync with TestTools/.Mconfig.rc & TestTools/MdoServer ! > if test -s /etc/fedora-release ; then > LINUX_DIST="`cat /etc/fedora-release | head -n1 \ > - | sed 's|^.*$Fedora$ Core.* release $[[0-9]][[^ \n]]*$$ .*$*$|\1:\2|'`" > + | sed 's|^.*$Fedora$.* release $[[0-9]][[^ \n]]*$$ .*$*$|\1:\2|'`" > elif test -s /etc/centos-release ; then > LINUX_DIST="`cat /etc/centos-release | head -n1 \ > | sed 's|^$CentOS$ release $[[0-9]][[^ \n]]*$$ .*$*$|\1:\2|'`" > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Compilation of M5 stable in windows
by Romulo Goncalves 11 Jun '07

11 Jun '07

During the compilation I got: copy ".\..\..\src\mal\Makefile.msc" "mal\Makefile" 1 file(s) copied. cd "mal" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\Stable\MonetDB5\NT" all cl -GF -W3 -wd4273 -wd4102 -MD -nologo -Zi -Ox -I. -I.\..\.. "-IC:\Pthreads\include" -I.\..\..\..\src\mal -I"C:\cygwi \home\goncalve\MonetDB\MonetDB_SQL\Stable\MonetDB\NT\include" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\Stable\MonetDB\N \include\common" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\Stable\MonetDB\NT\include\gdk" -I"C:\cygwin\home\goncalve\Mon tDB\MonetDB_SQL\Stable\MonetDB\NT\include\monet" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\Stable\MonetDB\NT\include\pla n" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\Stable\MonetDB\NT\include\mapi" "-I..\optimizer" "-I..\scheduler" "-I..\m dules\kernel" -DLIBMAL -DCOMPILE_DL_MAL -Fo"mal.obj" -c "mal.c" mal.c ..\..\mal_config.h(23) : fatal error C1083: Cannot open include file: 'monetdb_winconfig.h': No such file or directory NMAKE : fatal error U1077: 'cl' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: '"E:\Visual Studio .NET 2003\VC7\BIN\nmake.exe"' : return code '0x2' I am using the same environment as the one used for the cvs branch. Any idea in how to resolve this problem? Regards, Romulo

3 6

Re: [Monetdb-developers] Compilation of M5 stable in windows
by Romulo Goncalves 11 Jun '07

11 Jun '07

Niels Nes wrote: > On Mon, Jun 11, 2007 at 09:50:37PM +0200, Romulo Goncalves wrote: >> Sjoerd Mullender wrote: >>> On 06/11/2007 08:43 PM, Romulo Goncalves wrote: >>>> Sjoerd Mullender wrote: >>>>> On 06/11/2007 06:46 PM, Romulo Goncalves wrote: >>>> I just checked out everything. I think it is a PATH problem. >>> Did you also re-bootstrap? Another standard requirement if there has >>> been a change in buildtools. >> When you check out a clean version you also need a re-boostraap? I do >> not think so. > If new files are added (ie Makefile.ag changed) or buildtools > is changed it is needed. When I say checkout clean source I mean everything (buildtools, clients, MonetDB, MonetDB5, sql). Then bootstraap, configure make and make install ;) As I said the problem it is for sure some wrong PATH that I set. ;) It is a stupid error from my part. So discussion is closed .... :) Regards, Romulo > > Niels >> Regards, >> Romulo >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by DB2 Express >> Download DB2 Express C - the FREE version of DB2 express and take >> control of your XML. No limits. Just data. Click to get it now. >> http://sourceforge.net/powerbar/db2/ >> _______________________________________________ >> Monetdb-developers mailing list >> Monetdb-developers(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

1 0

[Monetdb-developers] M5 strict and optimized compilation...
by Romulo Goncalves 11 Jun '07

11 Jun '07

During the compilation I got the following error: cc1: warnings being treated as errors /ufs/goncalve/MonetDB/MonetDB5/src/modules/mal/bpm.mx: In function 'BPMadapt': /ufs/goncalve/MonetDB/MonetDB5/src/modules/mal/bpm.mx:1587: warning: 'vp' may be used uninitialized in this function make[7]: *** [lib_bpm_la-bpm.lo] Error 1 make[7]: Leaving directory `/net/amelia/export/scratch1/goncalve/MonetDB/Stable/MonetDB5/build/src/modules/mal' The function head is here: static str BPMadaptImpl(int *ret, bat *bid, ptr low, ptr hgh, bat *rs, int policy ) { BAT *b,*bs, *bs1, *bs2; Partition ps, p, p2; int (*cmp) (ptr, ptr); ptr nilptr; ValPtr vp; Can I initialize it to NULL? If someone disagree please fix it with the correct value. Regards, Romulo

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/Triggers cascaded_triggers.sql, SQL_2-18, 1.2, 1.2.4.1 recursive_triggers.sql, SQL_2-18, 1.2, 1.2.4.1 referencing_clause_new.sql, SQL_2-18, 1.4, 1.4.4.1 referencing_clause_old.sql, SQL_2-18, 1.4, 1.4.4.1
by Romulo Goncalves 08 Jun '07

08 Jun '07

Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/test/Triggers > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv21172 > > Modified Files: > Tag: SQL_2-18 > cascaded_triggers.sql recursive_triggers.sql > referencing_clause_new.sql referencing_clause_old.sql > Log Message: > The new tests for UPDATE/DELETE CASCADE The new tests for the Trigger tests... Sorry for the wrong log message > > > > > Index: referencing_clause_new.sql > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Triggers/referencing_clause_new.sql,v > retrieving revision 1.4 > retrieving revision 1.4.4.1 > diff -u -d -r1.4 -r1.4.4.1 > --- referencing_clause_new.sql 7 Dec 2006 15:26:06 -0000 1.4 > +++ referencing_clause_new.sql 8 Jun 2007 12:47:50 -0000 1.4.4.1 > @@ -1,101 +1,199 @@ > +--this test only tests the sintax > --the semantic should also be tested after the syntax test > > create table t1 (id int, name varchar(1024)); > +create table t2 (id int, name varchar(1024)); > > ---test when the trigger event is UPDATE > +--test when trigger event is UPDATE > insert into t1 values(10, 'monetdb'); > insert into t1 values(20, 'monet'); > > > create trigger test1 > after update on t1 referencing new row as new_row > - for each row insert into t1 values(0, 'update_new_row'); > + for each row insert into t2 values(0, 'update_new_row'); > > create trigger test2 > after update on t1 referencing new row new_row > - for each row insert into t1 values(1, 'update_new_row'); > + for each row insert into t2 values(1, 'update_new_row'); > > create trigger test3 > after update on t1 referencing new as new_row > - for each row insert into t1 values(2, 'update_new_row'); > + for each row insert into t2 values(2, 'update_new_row'); > > create trigger test4 > after update on t1 referencing new new_row > - for each row insert into t1 values(3, 'update_new_row'); > + for each row insert into t2 values(3, 'update_new_row'); > > > update t1 set name = 'mo' where id = 10; > > select * from t1; > +select * from t2; > + > +delete from t1 where id > -1; > +delete from t2 where id > -1; > + > +drop trigger test1; > +drop trigger test2; > +drop trigger test3; > +drop trigger test4; > + > +--test when trigger event is DELETE > +insert into t1 values(10, 'monetdb'); > +insert into t1 values(20, 'monet'); > + > +create trigger test1 > + after delete on t1 referencing new row as new_row > + for each row insert into t2 values(0, 'delete_new_row'); > + > +create trigger test2 > + after delete on t1 referencing new row new_row > + for each row insert into t2 values(1, 'delete_new_row'); > + > +create trigger test3 > + after delete on t1 referencing new as new_row > + for each row insert into t2 values(2, 'delete_new_row'); > + > +create trigger test4 > + after delete on t1 referencing new new_row > + for each row insert into t2 values(3, 'delete_new_row'); > + > > delete from t1 where id >-1; > > +select * from t1; > +select * from t2; > + > drop trigger test1; > drop trigger test2; > drop trigger test3; > drop trigger test4; > > ---test when the trigger event is INSERT > +delete from t2 where id >-1; > + > +--test error messages > +--new row and new table are not allowed if the Trigger event is INSERT > + > insert into t1 values(10, 'monetdb'); > > create trigger test1 > after insert on t1 referencing new row as new_row > - for each row insert into t1 values(0, 'insert_new_row'); > + for each row insert into t2 values(0, 'insert_new_row'); > > create trigger test2 > after insert on t1 referencing new row new_row > - for each row insert into t1 values(1, 'insert_new_row'); > + for each row insert into t2 values(1, 'insert_new_row'); > > create trigger test3 > after insert on t1 referencing new as new_row > - for each row insert into t1 values(2, 'insert_new_row'); > + for each row insert into t2 values(2, 'insert_new_row'); > > create trigger test4 > after insert on t1 referencing new new_row > - for each row insert into t1 values(3, 'insert_new_row'); > + for each row insert into t2 values(3, 'insert_new_row'); > + > > insert into t1 values(20, 'monet'); > > select * from t1; > +select * from t2; > > -delete from t1 where id > -1; > +delete from t1 where id >-1; > +delete from t2 where id >-1; > > drop trigger test1; > drop trigger test2; > drop trigger test3; > drop trigger test4; > > ---test error messages > ---new row and new table are not allowed if the Trigger event is DELETE > +--test with new row and new table and mixed > > insert into t1 values(10, 'monetdb'); > insert into t1 values(20, 'monet'); > > create trigger test1 > - after delete on t1 referencing new row as new_row > - for each row insert into t1 values(0, 'delete_new_row'); > + after update on t1 referencing new row as new_row new table as new_table > + for each row insert into t2 values(0, 'insert_new_row_table'); > > create trigger test2 > - after delete on t1 referencing new row new_row > - for each row insert into t1 values(1, 'delete_new_row'); > + after update on t1 referencing new row new_row new row as new_row > + for each row insert into t2 values(1, 'insert_new_new_row'); > > create trigger test3 > - after delete on t1 referencing new as new_row > - for each row insert into t1 values(2, 'delete_new_row'); > + after update on t1 referencing new table as new_table new table as new_table > + for each row insert into t2 values(2, 'insert_new__new_table'); > > create trigger test4 > - after delete on t1 referencing new new_row > - for each row insert into t1 values(3, 'delete_new_row'); > + after update on t1 referencing new row as new_row new table as new_table > + for each row insert into t2 values(3, 'insert_new_row_new_table'); > > +create trigger test5 > + after update on t1 referencing new table as new_table new row as new_row > + for each row insert into t2 values(4, 'insert_new_table_new_row'); > > -delete from t1 where id >1; > + > +update t1 set name = 'mo' where id = 10; > > select * from t1; > +select * from t2; > + > +delete from t1 where id >-1; > +delete from t2 where id >-1; > + > > drop trigger test1; > drop trigger test2; > drop trigger test3; > drop trigger test4; > +drop trigger test5; > + > +--test stanger combinations > + > +insert into t1 values(10, 'monetdb'); > + > +create trigger test1 > + after update on t1 referencing new row as new_row new table as new_table > + for each row insert into t2 values(0, 'update_new_row_new_table'); > + > +create trigger test2 > + after insert on t1 referencing new row new_row new row as new_row > + for each row insert into t2 values(1, 'insert_new_new_row'); > + > +create trigger test3 > + after delete on t1 referencing new row new_row new row as new_row > + for each row insert into t2 values(2, 'delete_new_new_row'); > + > +create trigger test4 > + after delete on t1 referencing new row as new_row new table as new_table > + for each row insert into t2 values(3, 'delete_new_row_new_table'); > + > +create trigger test5 > + after insert on t1 referencing new table as new_table new row as new_row > + for each row insert into t2 values(4, 'insert_new_table_new_row'); > + > +insert into t1 values(20, 'monet'); > +select * from t1; > +select * from t2; > + > +update t1 set name = 'mo' where id = 10; > +select * from t1; > +select * from t2; > + > +delete from t1 where id >5; > +select * from t1; > +select * from t2; > + > + > +drop trigger test1; > +drop trigger test2; > +drop trigger test3; > +drop trigger test4; > +drop trigger test5; > + > +delete from t1 where id >-1; > +delete from t2 where id >-1; > > --Cleanup > drop table t1; > +drop table t2; > > Index: cascaded_triggers.sql > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Triggers/cascaded_triggers.sql,v > retrieving revision 1.2 > retrieving revision 1.2.4.1 > diff -u -d -r1.2 -r1.2.4.1 > --- cascaded_triggers.sql 6 Dec 2006 10:27:38 -0000 1.2 > +++ cascaded_triggers.sql 8 Jun 2007 12:47:49 -0000 1.2.4.1 > @@ -1,53 +1,31 @@ > ---test the semantic of cascaded triggers > create table t1(id int, name varchar(1024), age int); > - > create table t2(id int, age int); > > create trigger test_0 after insert on t1 > - update t1 set id = 1, name = 'monetdb', age = 24 where id =1; > - > -insert into t1 values(1, 'mo', 25); > - > -select * from t1; > - > -select * from t2; > - > -create trigger test_1 after insert on t1 > insert into t2 select id,age from t1; > > -insert into t1 values(2, 'mo', 26); > - > -select * from t1; > - > -select * from t2; > - > -create trigger test_2 after insert on t1 > - insert into t2 values(1,23); > - > -insert into t1 values(3, 'mo', 27); > +insert into t1 values(1, 'mo', 25); > > select * from t1; > - > select * from t2; > > -create trigger test_3 after delete on t1 > - delete from t1 where id =3; > +create trigger test_1 after delete on t1 > + insert into t1 values(3, 'mo', 27); > > delete from t1 where id = 1; > > select * from t1; > - > select * from t2; > > -create trigger test_4 after update on t1 > - update t1 > - set age = 27 > - where id = 2; > +create trigger test_2 after update on t1 > + delete from t2; > + > +create trigger test_3 after delete on t2 > + insert into t1 values(1, 'mo', 25); > > update t1 set name = 'monet' where id = 2; > > select * from t1; > - > select * from t2; > > drop trigger test_0; > @@ -58,8 +36,6 @@ > > drop trigger test_3; > > -drop trigger test_4; > - > drop table t1; > > drop table t2; > > Index: referencing_clause_old.sql > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Triggers/referencing_clause_old.sql,v > retrieving revision 1.4 > retrieving revision 1.4.4.1 > diff -u -d -r1.4 -r1.4.4.1 > --- referencing_clause_old.sql 8 Dec 2006 12:17:42 -0000 1.4 > +++ referencing_clause_old.sql 8 Jun 2007 12:47:50 -0000 1.4.4.1 > @@ -2,6 +2,7 @@ > --the semantic should also be tested after the syntax test > > create table t1 (id int, name varchar(1024)); > +create table t2 (id int, name varchar(1024)); > > --test when trigger event is UPDATE > insert into t1 values(10, 'monetdb'); > @@ -10,26 +11,28 @@ > > create trigger test1 > after update on t1 referencing old row as old_row > - for each row insert into t1 values(0, 'update_old_row'); > + for each row insert into t2 values(0, 'update_old_row'); > > create trigger test2 > after update on t1 referencing old row old_row > - for each row insert into t1 values(1, 'update_old_row'); > + for each row insert into t2 values(1, 'update_old_row'); > > create trigger test3 > after update on t1 referencing old as old_row > - for each row insert into t1 values(2, 'update_old_row'); > + for each row insert into t2 values(2, 'update_old_row'); > > create trigger test4 > after update on t1 referencing old old_row > - for each row insert into t1 values(3, 'update_old_row'); > + for each row insert into t2 values(3, 'update_old_row'); > > > update t1 set name = 'mo' where id = 10; > > select * from t1; > +select * from t2; > > delete from t1 where id > -1; > +delete from t2 where id > -1; > > drop trigger test1; > drop trigger test2; > @@ -42,31 +45,32 @@ > > create trigger test1 > after delete on t1 referencing old row as old_row > - for each row insert into t1 values(0, 'delete_old_row'); > + for each row insert into t2 values(0, 'delete_old_row'); > > create trigger test2 > after delete on t1 referencing old row old_row > - for each row insert into t1 values(1, 'delete_old_row'); > + for each row insert into t2 values(1, 'delete_old_row'); > > create trigger test3 > after delete on t1 referencing old as old_row > - for each row insert into t1 values(2, 'delete_old_row'); > + for each row insert into t2 values(2, 'delete_old_row'); > > create trigger test4 > after delete on t1 referencing old old_row > - for each row insert into t1 values(3, 'delete_old_row'); > + for each row insert into t2 values(3, 'delete_old_row'); > > > delete from t1 where id >-1; > > select * from t1; > +select * from t2; > > drop trigger test1; > drop trigger test2; > drop trigger test3; > drop trigger test4; > > -delete from t1 where id >-1; > +delete from t2 where id >-1; > > --test error messages > --old row and old table are not allowed if the Trigger event is INSERT > @@ -75,26 +79,28 @@ > > create trigger test1 > after insert on t1 referencing old row as old_row > - for each row insert into t1 values(0, 'insert_old_row'); > + for each row insert into t2 values(0, 'insert_old_row'); > > create trigger test2 > after insert on t1 referencing old row old_row > - for each row insert into t1 values(1, 'insert_old_row'); > + for each row insert into t2 values(1, 'insert_old_row'); > > create trigger test3 > after insert on t1 referencing old as old_row > - for each row insert into t1 values(2, 'insert_old_row'); > + for each row insert into t2 values(2, 'insert_old_row'); > > create trigger test4 > after insert on t1 referencing old old_row > - for each row insert into t1 values(3, 'insert_old_row'); > + for each row insert into t2 values(3, 'insert_old_row'); > > > insert into t1 values(20, 'monet'); > > select * from t1; > +select * from t2; > > delete from t1 where id >-1; > +delete from t2 where id >-1; > > drop trigger test1; > drop trigger test2; > @@ -108,30 +114,33 @@ > > create trigger test1 > after update on t1 referencing old row as old_row old table as old_table > - for each row insert into t1 values(0, 'insert_old_row_table'); > + for each row insert into t2 values(0, 'insert_old_row_table'); > > create trigger test2 > after update on t1 referencing old row old_row new row as new_row > - for each row insert into t1 values(1, 'insert_old_new_row'); > + for each row insert into t2 values(1, 'insert_old_new_row'); > > create trigger test3 > after update on t1 referencing old table as old_table new table as new_table > - for each row insert into t1 values(2, 'insert_old__new_table'); > + for each row insert into t2 values(2, 'insert_old__new_table'); > > create trigger test4 > after update on t1 referencing old row as old_row new table as new_table > - for each row insert into t1 values(3, 'insert_old_row_new_table'); > + for each row insert into t2 values(3, 'insert_old_row_new_table'); > > create trigger test5 > after update on t1 referencing old table as old_table new row as new_row > - for each row insert into t1 values(3, 'insert_old_table_new_row'); > + for each row insert into t2 values(4, 'insert_old_table_new_row'); > > > update t1 set name = 'mo' where id = 10; > > select * from t1; > +select * from t2; > > delete from t1 where id >-1; > +delete from t2 where id >-1; > + > > drop trigger test1; > drop trigger test2; > @@ -145,32 +154,35 @@ > > create trigger test1 > after update on t1 referencing old row as old_row new table as new_table > - for each row insert into t1 values(0, 'update_old_row__new_table'); > + for each row insert into t2 values(0, 'update_old_row_new_table'); > > create trigger test2 > after insert on t1 referencing old row old_row new row as new_row > - for each row insert into t1 values(1, 'insert_old_new_row'); > + for each row insert into t2 values(1, 'insert_old_new_row'); > > create trigger test3 > after delete on t1 referencing old row old_row new row as new_row > - for each row insert into t1 values(1, 'delete_old_new_row'); > + for each row insert into t2 values(2, 'delete_old_new_row'); > > create trigger test4 > after delete on t1 referencing old row as old_row new table as new_table > - for each row insert into t1 values(3, 'delete_old_row_new_table'); > + for each row insert into t2 values(3, 'delete_old_row_new_table'); > > create trigger test5 > after insert on t1 referencing old table as old_table new row as new_row > - for each row insert into t1 values(3, 'insert_old_table_new_row'); > + for each row insert into t2 values(4, 'insert_old_table_new_row'); > > insert into t1 values(20, 'monet'); > select * from t1; > +select * from t2; > > update t1 set name = 'mo' where id = 10; > select * from t1; > +select * from t2; > > delete from t1 where id >5; > select * from t1; > +select * from t2; > > > drop trigger test1; > @@ -180,6 +192,8 @@ > drop trigger test5; > > delete from t1 where id >-1; > +delete from t2 where id >-1; > > --Cleanup > drop table t1; > +drop table t2; > > Index: recursive_triggers.sql > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Triggers/recursive_triggers.sql,v > retrieving revision 1.2 > retrieving revision 1.2.4.1 > diff -u -d -r1.2 -r1.2.4.1 > --- recursive_triggers.sql 6 Dec 2006 10:27:38 -0000 1.2 > +++ recursive_triggers.sql 8 Jun 2007 12:47:49 -0000 1.2.4.1 > @@ -1,57 +1,62 @@ > create table t1(id int, name varchar(1024), age int); > create table t2(id int, age int); > > +--the trigger calls itself > create trigger test_0 after insert on t1 > - insert into t1 values(1, 'monetdb', 24); > - > -insert into t1 values(1, 'mo', 25); > + insert into t1 values(3, 'mo', 27); > > -select * from t1; > -select * from t2; > +drop trigger test_0; > > -create trigger test_1 after insert on t1 > +--recursivity of 2 levels > +create trigger test_0 after insert on t1 > insert into t2 select id,age from t1; > +create trigger test_1 after insert on t2 > + insert into t1 values(3, 'mo', 27); > > -insert into t1 values(2, 'mo', 26); > > -select * from t1; > -select * from t2; > +drop trigger test_0; > +drop trigger test_1; > > -create trigger test_2 before insert on t1 > - insert into t2 values(1,23); > +--recursivity of n levels > +create trigger test_0 after insert on t1 > + insert into t2 select id,age from t1; > > -insert into t1 values(3, 'mo', 27); > +create trigger test_1 after insert on t2 > + delete from t2; > > -select * from t1; > -select * from t2; > +create trigger test_2 after delete on t2 > + insert into t1 values(3, 'mo', 27); > > -create trigger test_3 after delete on t1 > - delete from t1 where id =3; > > -delete from t1 where id = 1; > +drop trigger test_0; > +drop trigger test_1; > +drop trigger test_2; > > -select * from t1; > -select * from t2; > +--recursivity with procedure calls > +create PROCEDURE p1(id int, age int) > +BEGIN > + insert into t2 values(id, age); > +END; > > -create trigger test_4 after update on t1 > - update t1 > - set age = 27 > - where id = 2; > +create PROCEDURE p1() > +BEGIN > + declare id int, age int; > + set id = 1; > + set age = 23; > + call p1(id, age); > +END; > > -update t1 set name = 'monet' where id = 2; > +create trigger test_0 after insert on t2 > +BEGIN ATOMIC > + insert into t1 values(1, 'monetdb', 24); > + call p1(); > +END; > > -select * from t1; > -select * from t2; > +--insert into t2 values(0, 24); > > drop trigger test_0; > > -drop trigger test_1; > - > -drop trigger test_2; > - > -drop trigger test_3; > - > -drop trigger test_4; > +drop ALL procedure p1; > > drop table t1; > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/semantics xquery_fo.c, XQuery_0-18, 1.129.2.1, 1.129.2.2
by Stefan Manegold 08 Jun '07

08 Jun '07

On Thu, Jun 07, 2007 at 08:01:52PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/semantics > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31053 > > Modified Files: > Tag: XQuery_0-18 > xquery_fo.c > Log Message: > Experiment: split up declaration of xquery_fo into smaller pieces in > multiple functions to see whether it then compiles with icc on Linux. Sjoerd, thanks for yoiur efforts! Unfortunately, it still does not help/work: /net/corona.ins.cwi.nl/export/scratch0/manegold/Monet/Testing/Stable/source/pathfinder/compiler/semantics/xquery_fo.c(1664): (col. 1)internal error: 0_0 Maybe, we should finally consider upgrading icc --- provided there is a newer (free) version available... Stefan > Index: xquery_fo.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/semantics/xquery_fo.c,v > retrieving revision 1.129.2.1 > retrieving revision 1.129.2.2 > diff -u -d -r1.129.2.1 -r1.129.2.2 > --- xquery_fo.c 7 Jun 2007 14:07:00 -0000 1.129.2.1 > +++ xquery_fo.c 7 Jun 2007 20:01:49 -0000 1.129.2.2 > @@ -97,8 +97,8 @@ > } > } > > -void > -PFfun_xquery_fo (void) > +static void > +PFfun_xquery_fo1 (void) > { > struct xquery_fo xquery_fo1[] = > /** > @@ -956,7 +956,15 @@ > .ret_ty = PFty_xs_double () } }, > .alg = PFbui_fn_floor_dbl } > , { .loc = 0 } > - }, xquery_fo2[] = { > + }; > + > + xquery_fo_initiaze(xquery_fo1); > +} > + > +static void > +PFfun_xquery_fo2 (void) > +{ > + struct xquery_fo xquery_fo2[] = { > /* fn:floor (integer?) as integer? */ > { .ns = PFns_fn, .loc = "floor", > .arity = 1, .sig_count = 1, .sigs = { { > @@ -1646,7 +1654,15 @@ > .ret_ty = PFty_opt (PFty_doc (PFty_xs_anyNode ())) } }, > .alg = PFbui_fn_doc } > , { .loc = 0 } > - }, xquery_fo3[] = { > + }; > + > + xquery_fo_initiaze(xquery_fo2); > +} > + > +static void > +PFfun_xquery_fo3 (void) > +{ > + struct xquery_fo xquery_fo3[] = { > /* fn:collection (string) as node* */ > { .ns = PFns_fn, .loc = "collection", > .arity = 1, .sig_count = 1, .sigs = { { > @@ -2241,11 +2257,17 @@ > , { .loc = 0 } > }; > > + xquery_fo_initiaze(xquery_fo3); > +} > + > +void > +PFfun_xquery_fo (void) > +{ > PFfun_env = PFenv (); > > - xquery_fo_initiaze(xquery_fo1); > - xquery_fo_initiaze(xquery_fo2); > - xquery_fo_initiaze(xquery_fo3); > + PFfun_xquery_fo1 (); > + PFfun_xquery_fo2 (); > + PFfun_xquery_fo3 (); > } > > /* vim:set shiftwidth=4 expandtab: */ > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah serialize_pftijah.mx, XQuery_0-18, 1.49, 1.49.2.1
by Stefan Manegold 07 Jun '07

07 Jun '07

Jan, property checking is quite expensive, hence, you might want to do it for debugging purposes, only, e.g, when property checking is enabled in Mserver via --debug=8 / --debug=10. To do so, just replace "if ( 1 )" by "PROPDEBUG" (which is defined as "if (GDKdebug & PROPMASK)" in MonetDB/src/gdk/gdk.mx) Stefan On Thu, Jun 07, 2007 at 12:50:55PM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12149 > > Modified Files: > Tag: XQuery_0-18 > serialize_pftijah.mx > Log Message: > - add a propcheck to our "C" generated ft-index BATs to see if they are correct > > > > Index: serialize_pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/serialize_pftijah.mx,v > retrieving revision 1.49 > retrieving revision 1.49.2.1 > diff -u -d -r1.49 -r1.49.2.1 > --- serialize_pftijah.mx 1 Jun 2007 15:35:00 -0000 1.49 > +++ serialize_pftijah.mx 7 Jun 2007 12:50:49 -0000 1.49.2.1 > @@ -689,6 +689,16 @@ > BBPunfix(BBPcacheid(tjctx->b_collPre)); > BBPunfix(BBPcacheid(tjctx->b_collSize)); > BBPunfix(BBPcacheid(tjctx->b_collPfPre)); > + if ( 1 ) { > + BATpropcheck(tjctx->b_globalTerm,BATPROPS_ALL); > + BATpropcheck(tjctx->b_globalTag,BATPROPS_ALL); > + BATpropcheck(tjctx->b_docName,BATPROPS_ALL); > + BATpropcheck(tjctx->b_docFirstPre,BATPROPS_ALL); > + BATpropcheck(tjctx->b_collParam,BATPROPS_ALL); > + BATpropcheck(tjctx->b_collPre,BATPROPS_ALL); > + BATpropcheck(tjctx->b_collSize,BATPROPS_ALL); > + BATpropcheck(tjctx->b_collPfPre,BATPROPS_ALL); > + } > if ( tjctx->stemCtx->clear && !tjctx->stemCtx->clear(tjctx->stemCtx) ) > return GDK_FAIL; > free(tjctx); > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, XQuery_0-18, 1.134.2.1, 1.134.2.2
by Stefan Manegold 07 Jun '07

07 Jun '07

Jan, IMHO, this changes is not correct. Though being of type BIT (which is physically a signed char, hence 8 bit wide), the ordered flags use all 8 bits, only the lowest one (1) indicates sortedness, the highest one (1<<7=128) indicates reverse sortedness, and the remaining ones are "mis-"used for radix clustering. Hence, checking for sortedness shoudl alwasy be done with BAT[ht]ordered(b)&1 --- regardless of the endianess, this should alwasy check for the lowest bit, now matter whether this is located left or right --- anyway, endianess is about byte orde, not about bit order, right?? Hence, if the your change does fix the Itanium problems, than the bug is IMHO caused somewhere else (wrong setting of the ordered flag?), and only appears here... Stefan On Thu, Jun 07, 2007 at 09:28:20AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29459 > > Modified Files: > Tag: XQuery_0-18 > pftijah.mx > Log Message: > - Try to fix Itanium bug [ 1732596 ] PFtijah tests fail on Itanium > Remove the bitwise AND in the check because it could be at the wrong side > of the word? > > > > Index: pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.134.2.1 > retrieving revision 1.134.2.2 > diff -u -d -r1.134.2.1 -r1.134.2.2 > --- pftijah.mx 4 Jun 2007 17:42:06 -0000 1.134.2.1 > +++ pftijah.mx 7 Jun 2007 09:28:18 -0000 1.134.2.2 > @@ -3745,8 +3745,10 @@ > /* --------------------------- checks ---------------------------------- */ > > BATcheck(tid, name); > - if (!(BATtordered(tid) & 1)) > + /* if (!(BATtordered(tid) & 1)) */ > + if ( !(BATtordered(tid) ) > { > + /* BUG#1732596 this test currently fails on Itanium */ > GDKerror("%s: index-bat must be ordered on tail.\n", name); > return GDK_FAIL; > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime shredder.mx, XQuery_0-18, 1.126, 1.126.2.1
by Jens Teubner 07 Jun '07

07 Jun '07

Hi Jan F, thanks for looking into our shredder's entity handling. This has been bugging us for quite a while now. It feels a bit strange, though, that we really have to implement getEntity() ourselves. After all, this is exactly what I would expect to be handled automatically by an XML parsing library. I have just glanced briefly over the libxml2 documentation. And I saw that there is a replaceEntities field in libxml2's xmlParserCtxt struct. As usual, the documentation is very poor here, but this sounds to me like exactly what we need. Have you tried whether simply enabling this flag would handle entities automatically? (Sorry, I don't have the time to test this myself right now.) You find the documentation for xmlParserCtxt at http://xmlsoft.org/html/libxml-tree.html#xmlParserCtxt . Just my 2¢ (still Euro-Cents) Jens On Tue, Jun 05, 2007 at 07:30:50AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv2405 > > Modified Files: > Tag: XQuery_0-18 > shredder.mx > Log Message: > - A first attempt at making ENTITIES work. After long searching and > trying the final solution was very simple. The libxml2 package already > maintains a hashtable with defined entities. The only thing missing was > the lookup function which it mysteriously does not use. > I implemented the "getEntity()" function in the xmlSAXHandler structure and > now simple ENTITIES defined in the internal subset work. > This solves bug#1185932 XML: Entities by Wouter Alink > > - Change error handling. I noticed for some time that there were no error > messages generated anymore on my system. The xmlSetGenericErrorFunc() does > not work on my machine. I added the "error()" function in the xmlSAXHandler > structure and now I get descriptive error messages again. This may screw up > some testset output but I think this is a big system improvement. > > > > Index: shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v > retrieving revision 1.126 > retrieving revision 1.126.2.1 > diff -u -d -r1.126 -r1.126.2.1 > --- shredder.mx 24 Apr 2007 15:04:42 -0000 1.126 > +++ shredder.mx 5 Jun 2007 07:30:46 -0000 1.126.2.1 > @@ -1103,6 +1103,35 @@ > stream_printf(GDKerr, "%s", buf); > } > > +static void > +shred_warning(void *ctx, > + const char *msg, ...) > +{ > + /* IMPORTANT this function may be called multiple times for one error > + * message so it is not possible to use GDKerror() here. > + * Instead, we "mis-use" the ctx pointer to remember whether a newline > + * has occured in the error message, and thus be able to prefix each > + *(line of an) error message with GDKERROR("!Error: "), which is > + * required to properly get the error message through the MAPI > + * protocol... > + */ > + va_list args; > + int *print_error_newline =(int*)ctx; > + int len = 0; > + char buf[PFSHRED_BUFLEN]; > + > + if (*print_error_newline) { > + len += snprintf(buf+len, PFSHRED_BUFLEN-len-1, GDKERROR); > + } > + > + va_start(args, msg); > + len += vsnprintf(buf+len, PFSHRED_BUFLEN-len-1, msg, args); > + va_end(args); > + > + *print_error_newline =(strchr(buf,(int)'\n') != NULL); > + stream_printf(GDKerr, "!WARNING: %s", buf); > +} > + > /** > * The shred_attribute_defAttributeDef() handles the DTD attribute definition callbacks > * in the from the header of the XML file. This is used for the ID/IDREF > @@ -1163,6 +1192,44 @@ > } > } > > +static xmlEntityPtr > +shred_getEntity(void *xmlCtx, const xmlChar *name) > +{ > + /* the shredder is now able to handle ENTITY's from the internal > + * subset. I do not really understand yet why this had to be done > + * here and why it was not handled automagically > + * The functions used are defined in $LIBXML2INCLUDES/entities.h > + */ > +#if 0 > + stream_printf(GDKerr,"shred_getEntity(ctx,\"%s\") CALLED\n",name); > +#endif > + xmlParserCtxtPtr ctx = ((shredCtxStruct*) xmlCtx)->xmlCtx; > + /* lookup the entity in the document entity hash table */ > + return xmlGetDocEntity(ctx->myDoc,name); > + /* QUESTION: xmlGetDtdEntity() and xmlGetParameterEntity() were also > + * possible, whats the diff between the doc/dtd versions, they both > + * seem to work. */ > +} > + > +#if 0 > +/* My first try at building an entity table but this one was not necessary > + * because the internal subset table was already build. > + */ > +static void > +shred_entityDecl(void *xmlCtx, > + const xmlChar *name, > + int type, > + const xmlChar *publicId, > + const xmlChar *systemId, > + xmlChar *content) > +{ > + xmlParserCtxtPtr ctx = ((shredCtxStruct*) xmlCtx)->xmlCtx; > + if ( ! xmlAddDtdEntity(ctx->myDoc,name,type,publicId,systemId,content) ) > + stream_printf(GDKerr,"shred_entityDecl(ctx,\"%s\") FAIL\n",name); > +} > +#endif > + > + > /* ==================================================================================== > * the shredder and its data structures > * - shredder_create() create all data structures > @@ -1183,14 +1250,14 @@ > , .characters = shred_characters > , .processingInstruction = shred_pi > , .comment = shred_comment > - , .error = 0 > + , .error = shred_error > , .cdataBlock = shred_cdata > , .internalSubset = 0 > , .isStandalone = 0 > , .hasInternalSubset = 0 > , .hasExternalSubset = 0 > , .resolveEntity = 0 > - , .getEntity = 0 > + , .getEntity = shred_getEntity > , .entityDecl = 0 > , .notationDecl = 0 > , .attributeDecl = shred_attribute_def > @@ -1199,7 +1266,7 @@ > , .setDocumentLocator = 0 > , .reference = 0 > , .ignorableWhitespace = 0 > - , .warning = 0 > + , .warning = shred_warning > , .fatalError = 0 > , .getParameterEntity = 0 > , .externalSubset = shred_external_subset > @@ -1229,6 +1296,10 @@ > char buf[XMLCHUNK+1]; > > /* reset libxml2 error handling */ > + /* note JF: this does not have any effect on SuSe9.3. No error messages > + * are printed. I assigned the 'error' field in the xmlSAXHandler and > + * this works fine. > + */ > xmlSetGenericErrorFunc((void*)&print_error_newline, shred_error); > > /* parse XML input(receive SAX events) */ > @@ -1237,7 +1308,30 @@ > } else if (buffer) { > xmlCtx = xmlCreateMemoryParserCtxt(buffer, shredCtx->fileSize); > } else { > - xmlCtx = xmlCreateURLParserCtxt(location, XML_PARSE_XINCLUDE|XML_PARSE_NOXINCNODE); > + /* Possible options for the second arg are: > + * XML_PARSE_RECOVER = recover on errors > + * XML_PARSE_NOENT = substitute entities > + * XML_PARSE_DTDLOAD = load the external subset > + * XML_PARSE_DTDATTR = default DTD attributes > + * XML_PARSE_DTDVALID = validate with the DTD > + * XML_PARSE_NOERROR = suppress error reports > + * XML_PARSE_NOWARNING = suppress warning reports > + * XML_PARSE_PEDANTIC = pedantic error reporting > + * XML_PARSE_NOBLANKS = remove blank nodes > + * XML_PARSE_SAX1 = use the SAX1 interface internally > + * XML_PARSE_XINCLUDE = Implement XInclude substitition > + * XML_PARSE_NONET = Forbid network access > + * XML_PARSE_NODICT = Do not reuse the context dictionnary > + * XML_PARSE_NSCLEAN = rm redundant namespaces declarations > + * XML_PARSE_NOCDATA = merge CDATA as text nodes > + * XML_PARSE_NOXINCNODE= do not generate XINCLUDE START/END nodes > + */ > + /* > + * TODO: how to prevent expansion of entities? > + */ > + xmlCtx = xmlCreateURLParserCtxt(location, > + XML_PARSE_XINCLUDE| > + XML_PARSE_NOXINCNODE); > } > if (!xmlCtx) { > GDKerror("shredder_parse: libxml2 could not initialize a parser.\n"); > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 >From /usr/src/linux/include/linux/kernel.h: #define STACK_MAGIC 0xdeadbeef

2 1

[Monetdb-developers] Release Branding?
by Stefan Manegold 05 Jun '07

05 Jun '07

Dear fellow developers, some time ago, we intorduced "nicknames" to be able to refer to our "mega-releases" (in particular MonetDB/XQuery, but in general all MonetDB related packages released together. Hence, what should be the name of this (final?) "mega-release"? (Future releases will preferably be release of individual packages such that a new release of a package work with the most recently released version of all related packages.) My suggestion: Starting the journey with (on) Valentijn('s day 2005), and after stop-overs at Mercury, Earth, Ceres, (being lost in Star Dust), and Venus, Pathfinder finally arrived at Mars (only to leave our own solar system hereafter to explore outer space...) Comments? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/conf MonetDB.m4, MonetDB_1-18, 1.16.2.2, 1.16.2.3
by Stefan Manegold 05 Jun '07

05 Jun '07

On Tue, Jun 05, 2007 at 09:24:10AM +0200, Fabian Groffen wrote: > On 05-06-2007 06:13:22 +0000, Stefan Manegold wrote: > > Update of /cvsroot/monetdb/buildtools/conf > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17106/conf > > > > Modified Files: > > Tag: MonetDB_1-18 > > MonetDB.m4 > > Log Message: > > > > Java 1.6 seems to be (partly) incompatible with Java 1.4 & 1.5; > > we do not support Java 1.6, yet. > > Stefan, > > Thanks for doing this. It was on my todo-list, but I ended up > forgetting about it every time. no problem. same holds/held for me, just happened to recall it "by accident", and took the "early bird's" opportunity ;-) Stefan > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] Data type of the text() nodes
by Peter Boncz 04 Jun '07

04 Jun '07

Hi Jennie + Jens, Jennie, the idea in XRPC is to put the burden of function resolution and type promotion at the client -- server does nothing, in order to make it easy to implement XRPC compliant web services in environments that lack any XPath/XQuery (type analysis) support, such as simple wrapper code. The client side (i.e. pf) already performs type promotion, otherwise the function call would have given the resolution error at client side. So, the only thing that needs to be done is *use* the types of the function signature when generating the XRPC request. ..Fix around line 1000 in xrpc_client.mx.. 1067 cur_kind = XTRACT_KIND(cmbn_cont_kind); 1068 1069 switch (cur_kind) { But, Jens, there is also static vs. dynamic typing issue here. Of course, dynamic types are often more specific than static types. An example that excploits this is: declare function isInt($ctx as item()* as xs:boolean { typeswitch($ctx) case xs:integer: return true() default: return false() } For this reason my proposed fix is to use the function signature type *only* if it is a subtype of the runtime type (= if it is more specific). While the above probably fixes Jennie's issue, I do think there is a problem in pathfinder with function parameter promotion. This can surface when the typeswitch() is used inside the function body. Basically, when types are promoted, the runtime representation should be forced to reflect this. AFAIK this is not done in mps -- maybe the new type annotation system (in progress) handles this? With such a system in place, the runtime type would already be xs:string and the hack I propose above would not be needed. As an aside, I also think that there is something wrong in pathfinder with type promotion in general comparisons. Specifically, while the spec says that in case of a comparision between untyped data and a numeric, the untyped should be cast to xs:double. Pathfinder seems to apply a direct cast to the specific numeric data type (e.g. xs:integer) instead. Example: <a t='1.1'/>/@t = 1 I stubled on this in my Friday exercise to implement selection pushdown (not yet finished). Peter ------------------------------ Message: 4 Date: Fri, 1 Jun 2007 17:14:46 +0200 From: Ying Zhang <Y.Zhang(a)cwi.nl> Subject: [Monetdb-developers] Data type of the text() nodes To: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070601151446.GV11079(a)cwi.nl> Content-Type: text/plain; charset=us-ascii Hello, I'm having a problem with the data type of text() nodes in XRPC implementation. --------------------------- module function definition: --------------------------- module namespace foo = "xrpc-test-function"; declare function foo:q2($strs as xs:string*) as node()* { for $s in $strs return <bar>{$s}</bar> }; ----------- test query: ----------- import module namespace test = "xrpc-test-function" at "/ufs/zhang/max-nr-params/xrpc-mod.xq"; let $hello := <hello> <lang>en</lang> <lang>nl</lang> </hello> let $lang := $hello/lang/text() return execute at {"localhost"} {test:q2($lang)} ------------------------------------------------------------- I got the error "function could not be resolved" when I run the XRPC query above. The reason is that all items in the sequence $lang have the type U_A (xs:untypedAtomic), and they are marshalled in the XRPC req. msg as, for example: <xrpc:atomic-value xsi:type="xs:untypedAtomic">en</xrpc:atomic-value> At the server side, the values are unmarshalled and assigned the type U_A again. So far, the values are handled correctly, IMHO. But then problem happens, namely, when the server calls the function 'xquery_sig_match' (in pathfinder.mx), it concludes that xs:untypedAtomic is not a sub-type of xs:string, hence, the function signature can not be resolved. This error does not occur if the function 'q2' is called without XRPC. So, what is the trick of pathfinder to match the signature? Is xs:untypedAtomic just treated the same as xs:string? The easiest solution of my problem is to just marshall an xs:untypedAtomic as an xs:string, but if someone has better suggestions, I would like to know. Thanks a lot! Regards, Jennie ------------------------------ Message: 5 Date: Fri, 1 Jun 2007 17:27:03 +0200 From: Jens Teubner <jens.teubner(a)in.tum.de> Subject: Re: [Monetdb-developers] Data type of the text() nodes To: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070601152703.GB8539(a)notekemper14.informatik.tu-muenchen.de> Content-Type: text/plain; charset=us-ascii On Fri, Jun 01, 2007 at 05:14:46PM +0200, Ying Zhang wrote: > [...] > > But then problem happens, namely, when the server calls the function > 'xquery_sig_match' (in pathfinder.mx), it concludes that > xs:untypedAtomic is not a sub-type of xs:string, hence, the function > signature can not be resolved. > > This error does not occur if the function 'q2' is called without XRPC. > So, what is the trick of pathfinder to match the signature? Is > xs:untypedAtomic just treated the same as xs:string? Hi Jennie, the keywords to look for in the Formal Semantics are "type promotion" and "function call". And there are the function conversion rules as implemented in core/fs.brg:function_conversion(). Sorry for my very brief answer, but these pointers should bring you quite far. Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 I invented <Ctrl><Alt><Delete>, but Bill Gates made it famous. -- David Bradley, member of the original IBM PC design team ------------------------------ Message: 6 Date: Fri, 1 Jun 2007 21:00:27 +0200 From: Jens Teubner <jens.teubner(a)in.tum.de> Subject: Re: [Monetdb-developers] [MonetDB-users] XRPC problem: too many parameters To: Ying Zhang <Y.Zhang(a)cwi.nl> Cc: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070601190027.GF8539(a)notekemper14.informatik.tu-muenchen.de> Content-Type: text/plain; charset=us-ascii On Fri, Jun 01, 2007 at 06:50:41PM +0200, Ying Zhang wrote: > [...] > > I forgot to say that you need to wrap "$gb-hit/Hit_gb/text()" into the > fn:data() function before those values can be passed to "test:q1". This > is a restriction of the current implementation of XRPC. > > The values you get by selecting the "text()" child node are of the type > "untypedAtomic". XRPC currently does not cast eash untypedAtomic to the > expected atomic type, hence, you need to add a fn:data() to do this. I > will add this cast later in XRPC's implementation. This sounds odd. Applied to (untyped) nodes, fn:data() returns xs:untypedAtomic. Only if a value is used in a function call, the items are cast to the respective atomic type ("function conversion rules"). (Actually, I just saw that fn:data() is not allowed at all on untyped nodes. Maybe this changed in the latest revisions of the specs.) Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Linus Torvalds is a lot like Bill Gates. Both are about the same height. -- USA Today ------------------------------ Message: 8 Date: Sat, 2 Jun 2007 22:14:44 +0200 From: Ying Zhang <Y.Zhang(a)cwi.nl> Subject: Re: [Monetdb-developers] [MonetDB-users] XRPC problem: too many parameters To: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070602201444.GC11079(a)cwi.nl> Content-Type: text/plain; charset=us-ascii On Fri, Jun 01, 2007 at 09:00:27PM +0200, Jens Teubner wrote: > On Fri, Jun 01, 2007 at 06:50:41PM +0200, Ying Zhang wrote: > > > [...] > > > > I forgot to say that you need to wrap "$gb-hit/Hit_gb/text()" into the > > fn:data() function before those values can be passed to "test:q1". This > > is a restriction of the current implementation of XRPC. > > > > The values you get by selecting the "text()" child node are of the type > > "untypedAtomic". XRPC currently does not cast eash untypedAtomic to the > > expected atomic type, hence, you need to add a fn:data() to do this. I > > will add this cast later in XRPC's implementation. > > This sounds odd. Applied to (untyped) nodes, fn:data() returns > xs:untypedAtomic. Only if a value is used in a function call, the items > are cast to the respective atomic type ("function conversion rules"). > > (Actually, I just saw that fn:data() is not allowed at all on untyped > nodes. Maybe this changed in the latest revisions of the specs.) > > Jens Hi Jens, You are right about that fn:data() does not do what I need, namely take an untypedAtomic and return an atomic value in the right type. But according to the latest revision, I think fn:data() does allow untypedAtomic as its parameter: fn:data($arg as item()*) as xs:anyAtomicType* But I need to find another way to solve this problem. Jennie > > -- > Jens Teubner > Technische Universitaet Muenchen, Department of Informatics > D-85748 Garching, Germany > Tel: +49 89 289-17259 Fax: +49 89 289-17263 > > Linus Torvalds is a lot like Bill Gates. Both are > about the same height. > -- USA Today

2 1

Re: [Monetdb-developers] xs:docmgmt in XQuery user defined functions?
by Peter Boncz 04 Jun '07

04 Jun '07

Hi Djoerd, All document management functions return the docmgt type, as a mechanism to ensure that such queries do not perform any document updates. For MonetDB, this separation is handy, as document management and XQUF updates rely on two different mechanisms (checkpointing vs WAL) and if queries would rely on both, it would be hard to make them atomic. Pathfinder supports the 'declare document management function' syntax extension (imilar to the xQUF notation 'declare updating function') to allow user-defined functions that perform document management. Peter

1 0

[Monetdb-developers] Feature freeze & new release branches
by Stefan Manegold 04 Jun '07

04 Jun '07

Dear all, as decided and announced last week, I'll "enforce" the feature freeze for the upcoming *.18 release today, Sunday June 3 2007 --- well, the origin plan was for Friday June 1 2007 but other obligations and lack of time forced my to post-pone this action by two days --- hopefully, SF will be less busy on a weekend... Hence, from now on, please do NOT checkin to ANY CVS modules releated to MonetDB until I'm done later today. I'll let you know, once I'm done and you can checkin, again. Thank you very much for your cooperation. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [Monetdb-developers] [MonetDB-users] XRPC problem: too many parameters
by Jens Teubner 02 Jun '07

02 Jun '07

On Fri, Jun 01, 2007 at 06:50:41PM +0200, Ying Zhang wrote: > [...] > > I forgot to say that you need to wrap "$gb-hit/Hit_gb/text()" into the > fn:data() function before those values can be passed to "test:q1". This > is a restriction of the current implementation of XRPC. > > The values you get by selecting the "text()" child node are of the type > "untypedAtomic". XRPC currently does not cast eash untypedAtomic to the > expected atomic type, hence, you need to add a fn:data() to do this. I > will add this cast later in XRPC's implementation. This sounds odd. Applied to (untyped) nodes, fn:data() returns xs:untypedAtomic. Only if a value is used in a function call, the items are cast to the respective atomic type ("function conversion rules"). (Actually, I just saw that fn:data() is not allowed at all on untyped nodes. Maybe this changed in the latest revisions of the specs.) Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Linus Torvalds is a lot like Bill Gates. Both are about the same height. -- USA Today

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal.mx, 1.125, 1.126
by Martin Kersten 01 Jun '07

01 Jun '07

Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1997/src/mal > > Modified Files: > mal.mx > Log Message: > corrected checks for busy clients when exiting Nice bug. > > > Index: mal.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal.mx,v > retrieving revision 1.125 > retrieving revision 1.126 > diff -u -d -r1.125 -r1.126 > --- mal.mx 1 Jun 2007 06:48:28 -0000 1.125 > +++ mal.mx 1 Jun 2007 09:48:20 -0000 1.126 > @@ -391,10 +391,10 @@ > do{ > freeclient=0; finishing=0; claimed=0; awaiting=0; > for(cntxt= mal_clients+1; cntxt<mal_clients+MAL_MAXCLIENTS; cntxt++){ > - freeclient += cntxt->mode & FREECLIENT; > - finishing += cntxt->mode & FINISHING; > - claimed += cntxt->mode & CLAIMED; > - awaiting += cntxt->mode & AWAITING; > + freeclient += (cntxt->mode == FREECLIENT); > + finishing += (cntxt->mode == FINISHING); > + claimed += (cntxt->mode == CLAIMED); > + awaiting += (cntxt->mode == AWAITING); > if( cntxt->mode & FINISHING) > printf("#CLient %d %d\n",(int)(cntxt - mal_clients), cntxt->idx); > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

[Monetdb-developers] Data type of the text() nodes
by Ying Zhang 01 Jun '07

01 Jun '07

Hello, I'm having a problem with the data type of text() nodes in XRPC implementation. --------------------------- module function definition: --------------------------- module namespace foo = "xrpc-test-function"; declare function foo:q2($strs as xs:string*) as node()* { for $s in $strs return <bar>{$s}</bar> }; ----------- test query: ----------- import module namespace test = "xrpc-test-function" at "/ufs/zhang/max-nr-params/xrpc-mod.xq"; let $hello := <hello> <lang>en</lang> <lang>nl</lang> </hello> let $lang := $hello/lang/text() return execute at {"localhost"} {test:q2($lang)} ------------------------------------------------------------- I got the error "function could not be resolved" when I run the XRPC query above. The reason is that all items in the sequence $lang have the type U_A (xs:untypedAtomic), and they are marshalled in the XRPC req. msg as, for example: <xrpc:atomic-value xsi:type="xs:untypedAtomic">en</xrpc:atomic-value> At the server side, the values are unmarshalled and assigned the type U_A again. So far, the values are handled correctly, IMHO. But then problem happens, namely, when the server calls the function 'xquery_sig_match' (in pathfinder.mx), it concludes that xs:untypedAtomic is not a sub-type of xs:string, hence, the function signature can not be resolved. This error does not occur if the function 'q2' is called without XRPC. So, what is the trick of pathfinder to match the signature? Is xs:untypedAtomic just treated the same as xs:string? The easiest solution of my problem is to just marshall an xs:untypedAtomic as an xs:string, but if someone has better suggestions, I would like to know. Thanks a lot! Regards, Jennie

2 1

[Monetdb-developers] xs:docmgmt in XQuery user defined functions?
by Djoerd Hiemstra 01 Jun '07

01 Jun '07

Dear all, Just a small suggestion for the release version. New functions like pf:add-doc() return an item of type docmgmt. However, this type does not seem to exist if I try to use user defined functions such as declare function local:weird( ) as xs:docmgmt { pf:add-doc("weird.xml", "weird.xml") }; gives > use of undefined type: at (14,38-14,47): ``xs:docmgmt'' Similarly with "docmgmt" or "pf:docmgmt", etc. Or if I use "item()" > type error: body of local:weird() evaluates to `docmgmt' which is not > a subtype of `item' Best, Djoerd.

1 0

[Monetdb-developers] MonetDB Release Preparation
by Martin Kersten 29 May '07

29 May '07

Dear fellow MonetDB developers/users, we plan to have a new release of the MonetDB software family within the next two weeks, i.e., Feature Freeze: Friday Jun 01 2007 Release: Monday Jun 11 2007 This release cycle is used to reduce the component dependencies, such that we are better prepared to release individual components more frequently. A synopsis of what to expect: MonetDB 5 beta status is dropped and it becomes the sole back-end for the SQL front-end. SQL now supports Persistent Stored Modules. XQuery Update Facility has undergone a large series of stress tests to improve correctness, performance and robustness against concurrent updates. Many bug-fixes based on highly appreciated feedback from our user community. The priorities set for post-release actions as agreed upon by the development team are: shortterm:(summer) SQL GIS geometry model testing and release SQL/XML support development and release PF/SQL exploration (research focus) Skyserver demo website (target 2.7TB online demo) midterm: (fall) Remote SQL execution in a cluster setting SQL adaptive partitioning of large databases XQuery Algebra -> MIL compiler Date/time support in XQuery MonetDB Private BATs policy (kernel modification) lngterm: (fall/spring) XQuery Algebra -> MAL compiler public release of StreetTivo for collaborative media analysis Of course, as always, progress in each of these areas depend on availability of resources and their contribution to our research agenda. We are looking forward to messages from projects based on/exploitation of the MonetDB platform. We plan to add a section in the website to make this work more broadly visible. regards, Martin

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, 1.123, 1.124
by Stefan Manegold 29 May '07

29 May '07

On Tue, May 29, 2007 at 12:20:21PM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv10231 > > Modified Files: > pftijah.mx > Log Message: > - try to repair pftijah "testcoll2" bug on 64bit archs > > > > Index: pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.123 > retrieving revision 1.124 > diff -u -d -r1.123 -r1.124 > --- pftijah.mx 25 May 2007 12:34:24 -0000 1.123 > +++ pftijah.mx 29 May 2007 12:20:18 -0000 1.124 > @@ -3267,15 +3267,16 @@ > return GDK_FAIL; > } > > - doc_start = *(oid*)BUNtail(doc_firstpre,r); > - oid tj_nextIndex = tj_docIndex + 1; > + // 64bit ERROR?? doc_start = *(oid*)BUNtail(doc_firstpre,r); ^^^^^^^^^^^^^ No, a bug in the code: BAT doc_firstpre is [oid,int]., i.e., its tail is int, not oid, and int != oid; ======== pathfinder/modules/pftijah/pftijah.mx-73-.COMMAND pf2tijah_node( pathfinder/modules/pftijah/pftijah.mx-74- BAT[oid,str] doc_name, pathfinder/modules/pftijah/pftijah.mx:75: BAT[oid,int] doc_firstpre, ^^^^^^^^^^^^^^^^^^^^^^^^^ pathfinder/modules/pftijah/pftijah.mx-76- BAT[oid,oid] pfpre, pathfinder/modules/pftijah/pftijah.mx-77- BAT[oid,oid] item, pathfinder/modules/pftijah/pftijah.mx-78- BAT[oid,int] kind, pathfinder/modules/pftijah/pftijah.mx-79- BAT[oid,str] doc_loaded) pathfinder/modules/pftijah/pftijah.mx-80- : BAT[void,oid] = CMDpf2tijah_node; pathfinder/modules/pftijah/pftijah.mx-81- "Translate Pathfinder node sequence to tijah node sequence" ======== hence, to access the tail in C, you (obviously!) need to use int doc_start = *(int*) BUNtail(doc_firstpre,r); ^^^ ^^^ if (for what ever reason) doc_start needs to be of type oid instead of int, you need to use oid doc_start = (oid) *(int*) BUNtail(doc_firstpre,r); ^^^ ^^^ ^^^ (as you do below); in fact, you should first check, whether the respective tail value of doc_firstpre is not negative, before you caat it to oid! > + doc_start = (oid)*(int*)BUNtail(doc_firstpre,r); > + oid tj_nextIndex = tj_docIndex + (oid)1; > if ( BATcount(doc_firstpre) > tj_nextIndex ) { > r = BUNfnd(doc_firstpre,&tj_nextIndex); > if ( !r ) { > stream_printf(GDKout,"Cannot do range for tijah-firstpre @ %d.\n",tj_docIndex); > return GDK_FAIL; > } > - doc_end = *(oid*)BUNtail(doc_firstpre,r) - 1; > + doc_end = *(oid*)BUNtail(doc_firstpre,r) - (oid)1; NOPE! either int doc_end = *(int*) BUNtail(doc_firstpre,r) - 1; or oid doc_end = (oid) (*(int*) BUNtail(doc_firstpre,r) - 1); plus a check, whether *(int*) BUNtail(doc_firstpre,r) is > 0 ! Stefan > } else { > doc_end = oid_nil; > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/conf monetdb5.conf.in, 1.11, 1.12
by Stefan Manegold 29 May '07

29 May '07

Another M4 -> M5 "incompatibility" that should be documented properly in the M4 -> M5 transition documentation: ` While with M4, "mapi_open=true"/"mapi_open=false" works fine, M5 (seems to) require(s) "mapi_open=1"/"mapi_open=0". ` Stefan On Sun, May 27, 2007 at 05:27:33PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/conf > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31984 > > Modified Files: > monetdb5.conf.in > Log Message: > Use the proper values for mapi_open > > > Index: monetdb5.conf.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/conf/monetdb5.conf.in,v > retrieving revision 1.11 > retrieving revision 1.12 > diff -u -d -r1.11 -r1.12 > --- monetdb5.conf.in 17 May 2007 09:51:09 -0000 1.11 > +++ monetdb5.conf.in 27 May 2007 17:27:31 -0000 1.12 > @@ -126,7 +126,7 @@ > # Monet Application Interface Section > #==================================== > mapi_port=0 #default port to address a mserver > -mapi_open=false #should be set to 'true' to allow for > +mapi_open=0 #should be set to '1' to allow for > #remote access to a server > > # SQL Interface Section > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/sql random.sql, 1.1, NONE
by Romulo Goncalves 24 May '07

24 May '07

This means that it has to be removed from the Makefile (the compilation crash due this problem). I will remove it, if none disagree. Regards, Romulo Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/sql > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24021 > > Removed Files: > random.sql > Log Message: > removed as random functions moved into the default set (as they have > side effects, which currently cannot be specified in sql) > > > --- random.sql DELETED --- > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins

2 1

[Monetdb-developers] Debian/Ubuntu not in testweb?
by Jan Flokstra 24 May '07

24 May '07

Hi All, Is there a specific reason why there is no Debian Linux version in the testweb or was there just no machine available. We are experiencing a couple of very strange compilation and runtime errors with student projects running Ubuntu Linux and wonder if the Pathfinder HEAD is even supposed to work on Ubuntu? Regards, JanF.

2 1

[Monetdb-developers] nightly current target missing
by Colin Foss 24 May '07

24 May '07

It appears something bad happened to the overnight builds. i.e. --22:27:57-- http://www.ins.cwi.nl/~fabian/monetdb/tars/latest/current/MonetDB.tar.gz => `/var/tmp/MonetDB-1179984476/MonetDB/MonetDB.tar.gz' Resolving www.ins.cwi.nl... 192.16.196.21 Connecting to www.ins.cwi.nl|192.16.196.21|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 22:27:58 ERROR 403: Forbidden. also, http://monetdb.cwi.nl/testing/projects/monetdb/Current/.DailyBuilds./200705… is empty.

2 1

[Monetdb-developers] Infinite loop...
by Romulo Goncalves 23 May '07

23 May '07

Dear developers, I opened a bug related to an infinite loop created by this example: create table t1(id int, name varchar(1024), age int); create table t2(id int, age int); create PROCEDURE p1(id int, age int) BEGIN insert into t2 values(id, age); END; create PROCEDURE p1() BEGIN declare id int, age int; set id = 1; set age = 23; call p1(id, age); END; create trigger test_0 after insert on t2 BEGIN ATOMIC insert into t1 values(1, 'monetdb', 24); call p1(); END; insert into t2 values(0, 24); AS you can see the procedure p1() will do an insertion in t2 which will fire the trigger test_0 again and all the process will be repeated again. I decided to check what happen in another DBMS (I checked postgres because it is used as a reference). Looking to this example http://archives.postgresql.org/pgsql-general/2003-07/msg01530.php It seems postgres has the same problem, but should we consider this a problem or bug? SQL with procedures (function...) can almost be considered a programming language so it is not our responsibility to kill loops created by users SQL code. If an user see a infinite loop he has to construct the schema in such way that the infinite loops are avoided. Another point is the restriction that we created for recursive triggers (we only allow at least one call for each trigger in each statement execution) which I think it is not correct. Imagine that the user has an "if clause" to recursively fire a trigger 10 times and after that 10 times the trigger is not fired anymore. Should we restrict this example in our system? I do not think so. However, If the clause check is wrong and an infinite loop is created it is the user who has to correct his schema. The big conclusion of the day is: More programing language features do we offer to the user less control we have over his SQL code. If everyone agree I will close the bug infinite loop and remove the loop check for triggers. Regards, Romulo

2 1

[Monetdb-developers] nightly target contents
by Colin Foss 23 May '07

23 May '07

Sorry if this question is borderline user. Using monetdb-install.sh, what does --nightly=stable actually contain? What does --nightly=current contain? Do the nightly tar files contain source from a sucessful build or simply a CVS checkout? Is a cross-reference between bug number and --nightly=<target> maintained?

3 4

Re: [Monetdb-developers] possible bug regarding auto-commit mixing with start transaction?
by Martin Kersten 23 May '07

23 May '07

Hi, Alexandru Toth wrote: > Hi, > > Not in console. But if you create a file , it will show up with command > line bellow. > > mclient -lsql -umonetdb 1.sql Confirmed. > > Or, is Mac OS -speciffic ? No, also on Linux PC. You assume here that auto-commit is re-enabled when the statements fall outside the transaction brackets. The mode change and the consequences should be clearly documented in Mapi. It may indeed be useful to fall-back to auto-commit outside explicit transaction boundaries. regards, Martin > > BR, Alex > > On May 22, 2007, at 11:43 PM, Martin Kersten wrote: > >> Hi, >> >> Bug not confirmed on my Linux PC using >> direct execution at the console, nor >> using MapiClient. >> >> % sys.tbl_xy, sys.tbl_xy # table_name >> % x, y # name >> % int, int # type >> % 2, 4 # length >> [ 0, 0 ] >> [ 1, 1 ] >> [ 2, 2 ] >> [ 3, 3 ] >> [ 4, 4 ] >> [ 5, 5 ] >> [ 6, 6 ] >> [ 7, 7 ] >> [ -1, -100 ] >> [ 8, 8 ] >> [ 9, 9 ] >> [ 10, 10 ] >> >> >> >> Alexandru Toth wrote: >>> Hi, >>> Test script bellow has few transactions, and statements outside any >>> transaction which are not executed (INSERT negative numbers, easy to >>> spot). Is this a bug? Both MySQL and SQLite inserts the row with >>> negative numbers (after replacing "start transaction" with "begin"). >>> Save the script bellow as 1.sql >>> ----------------- >>> start transaction; >>> create table tbl_xy (x int , y int); >>> insert into tbl_xy values (0,0); >>> insert into tbl_xy values (1,1); >>> commit; >>> start transaction; >>> insert into tbl_xy values (2,2); >>> insert into tbl_xy values (3,3); >>> insert into tbl_xy values (4,4); >>> commit; >>> start transaction; >>> insert into tbl_xy values (5,5); >>> insert into tbl_xy values (6,6); >>> insert into tbl_xy values (7,7); >>> commit; >>> insert into tbl_xy values (-1, -100); >>> start transaction; >>> insert into tbl_xy values (8,8); >>> insert into tbl_xy values (9,9); >>> insert into tbl_xy values (10,10); >>> commit; >>> ---------------- >>> Execution of script: >>> macu:~/Desktop alex$ mclient -lsql -umonetdb ~/Desktop/1.sql >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> [ 1 ] >>> -------------------- >>> Looking for negative numbers, none found: >>> macu:~/Desktop alex$ mclient -lsql -umonetdb >>> sql>select * from tbl_xy; >>> % sys.tbl_xy, sys.tbl_xy # table_name >>> % x, y # name >>> % int, int # type >>> % 2, 2 # length >>> [ 0, 0 ] >>> [ 1, 1 ] >>> [ 2, 2 ] >>> [ 3, 3 ] >>> [ 4, 4 ] >>> [ 5, 5 ] >>> [ 6, 6 ] >>> [ 7, 7 ] >>> [ 8, 8 ] >>> [ 9, 9 ] >>> [ 10, 10 ] >>> Version: MonetDB Server v5.0.0_beta2_1 >>> BR, Alex >> >

3 2

[Monetdb-developers] usleep not portable
by Sjoerd Mullender 22 May '07

22 May '07

Jennie, you introduced a call to usleep in xrpc_client.mx. How important is that? The problem is, usleep is not portable. For one, it is not a Posix call (use nanosleep instead), and worse (in practice), it's not available on Windows (but neither is nanosleep). On Windows there is Sleep() which takes milliseconds as an argument (unlike sleep which takes seconds, usleep which takes microseconds, and nanosleep which has nanosecond precision). -- Sjoerd Mullender

2 1

[Monetdb-developers] MonetDB5 SQL installer for windows 32....
by Romulo Goncalves 21 May '07

21 May '07

During the installation of MonetDB5 SQL for windows 32 I got an error. The print screen is in the following link: http://homepages.cwi.nl/~goncalve/spool/Picture1.png Regards, Romulo

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pf_support.mx, 1.233, 1.234
by Stefan Manegold 16 May '07

16 May '07

Peter, triggered by --enable-optimize compilation warning about ======== .../pathfinder/runtime/pf_support.mx: In function 'CMDws_bat': .../pathfinder/runtime/pf_support.mx:7122: warning: 'v' may be used uninitialized in this function ======== I changed BAT *v, *b = BBPdescriptor(bid); into BAT *v = 0, *b = BBPdescriptor(bid); in CMDws_bat(), and noticed that CMDws_bat() returns GDK_SUCCEED in any case, now, although *ret is initialized/set only in case b = BBPdescriptor(bid) is non-null. Is this --- i.e., returning GDK_SUCCEED and not initializing *ret in case b = BBPdescriptor(bid) return NULL --- intended / OK? Stefan On Wed, May 16, 2007 at 10:08:57AM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16:/tmp/cvs-serv3794 > > Modified Files: > pf_support.mx > Log Message: > [ 1718066 ] PF: put() to /tmp crashes Mserver > - error in error message formatting > > > > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.233 > retrieving revision 1.234 > diff -u -d -r1.233 -r1.234 > --- pf_support.mx 16 May 2007 07:55:53 -0000 1.233 > +++ pf_support.mx 16 May 2007 10:08:54 -0000 1.234 > @@ -699,7 +699,7 @@ > } > var absolute := [or](filepath.[startsWith]("/"), filepath.[startsWith]("\\")).uselect(true).mirror().leftfetchjoin(sel); > if (count(absolute) > 0) { > - ERROR("fn_put: file URI '%s' must be a relative path (%d such errors).", backwards.fetch(0), count(backwards)); > + ERROR("fn_put: file URI '%s' must be a relative path (%d such errors).", absolute.fetch(0), count(absolute)); > } > var backwards := filepath.[search]("..").uselect(0,int_nil).mirror().leftfetchjoin(sel); > if (count(backwards) > 0) { > @@ -7118,13 +7118,19 @@ > int CMDws_bat(BAT **ret, lng* wsid) > { > bat bid = *wsid & 1073741823LL; > - BAT *v = NULL; > if (BBPfix(bid)) { > - BAT *b = BBPdescriptor(bid); > + BAT *v, *b = BBPdescriptor(bid); > if (b) *ret = v = VIEWcreate(b); > BBPunfix(bid); > + if (v) { > + /* give the view a name such that lng(bbname(v)) = lng(bbpname(b)) */ > + long_str buf; > + snprintf(buf, sizeof(buf), "%s-%d", BBP_logical(bid), ABS(b->batCacheid)); > + BBPrename(v->batCacheid, buf); > + return GDK_SUCCEED; > + } > } > - return v?GDK_SUCCEED:GDK_FAIL; > + return GDK_SUCCEED; > } > > #include "serialize.h" > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] MonetDB with Perl in windows
by Steffen Goeldner 16 May '07

16 May '07

Romulo Goncalves wrote: > Hello, > > My name is Romulo (INS-1) and I am trying to create a windows installer > with Perl. My colleagues told me that you are the Perl expert in MonetDB... > > To compile PERL I did: > > install perl in C:\perl_5\ > and unzip the sources in same directory as MonetDB SQL MONETDB5 .... > (The same steps as we should do for php} > > In the MonetDB\conf\rules.msc I added: > > PERL_SRCDIR = $(TOPDIR)\..\..\perl-5.8.8 > PERL_INSTDIR = C:\perl-5 > # for these, also see configure.py > PERL_EXTENSIONDIR = lib\perl5 > > Note: I am compiling for M5 and sql from cvs branch. > > For the compilation I set the flag HAVE_PERL=1. > > Everything compiled. > > Now I need to know which files I have to add to the installer. > I was looking for a perl lib in cliens\NT\lib\ and It does not have such > lib. > > Could you help me a bit please? First and foremost, let me admit that I did not find time to implement the new protocol (8) yet - thus the perl modules may not work for M5 (at least the pure perl (MapiPP) client). Anyway, this has nothing to do with the installation procedure. In the spirit of Perl’s motto, "There’s More Than One Way To Do It" (TMTOWTDI), there are many perl modules in clients/src/perl: - Mapi.pm is a simple pure perl client, maintained by CWI and works (AFAIK) with the current protocol - MapiLib (in Cimpl) is a wrapper around libMapi and you need SWIG to build it - The modules in MonetDB-CLI* are maintained by me: - MonetDB::CLI::MapiPP is pure perl - MonetDB::CLI::MapiLib is pure perl but uses MapiLib - MonetDB::CLI::MapiXS is a XS (that is: C) wrapper around libMapi - MonetDB::CLI is a simple loader for one of the MonetDB::CLI::* modules above - DBD::monetdb (in DBD) is a DBI client on top of MonetDB::CLI In general, you install a perl module in the following way: perl Makefile.PL make make test make install In this way, perl saves you from all the drudgery. Installing by hand is easy for pure perl modules (though not recommended): simple copy all *.pm into your perl lib directory. Example (MonetDB::CLI::MapiPP): MonetDB-CLI-MapiPP/MapiPP.pm -> /perl/site/lib/MonetDB/CLI/MapiPP.pm (assuming /perl/site/lib is in the lib search path - try 'perl -V' and look for @INC and PERL5LIB. Later, if you try to load MonetDB::CLI::MapiPP, perl replaces '::' with '/', adds .pm and looks in every path from @INC.) However, installing an extension module (XS or SWIG) is difficult! Perl builds additional files (e.g. .dll) which have to be installed into different places. Best you call the standard 'make install' from the installer or at least watch what 'make install' would do. Regards, Steffen

2 5

Re: [Monetdb-developers] Cubulus OLAP tuning advice & building Monet on OS X fails
by Niels Nes 14 May '07

14 May '07

On Mon, May 14, 2007 at 12:24:49AM +0300, Alexandru Toth wrote: > Yes: > > > swig -version > > SWIG Version 1.3.27 > Copyright (c) 1995-1998 > University of Utah and the Regents of the University of California > Copyright (c) 1998-2005 > University of Chicago > Compiled with /usr/bin/g++-4.0 [powerpc-apple-darwin8.8.0] That should do fine. In my linux version if python and swig are found by configure it builds the MonetSQLdb python module. It however doesn't install in the python path (as that would require root rights, which we cannot assume). Therefor we install it under $prefix, in a directory lib64/python2.4/site-packages/. I set my PYTHONPATH using export PYTHONPATH=$prefix/`python -c 'import distutils.sysconfig; print distutils.sysconfig.get_python_lib(0,0,"")'` Niels > > > ----- > > which swig > /opt/local/bin//swig > > > BR Alex > > On May 14, 2007, at 12:19 AM, Niels Nes wrote: > > >On Mon, May 14, 2007 at 12:07:04AM +0300, Alexandru Toth wrote: > >> > >> Hello, > >> > >> After removing the extra Python versions from the PATH, > >> monetdb-install.sh runs without errors. But there is no > >>MonetSQLdb , > >> so I started ./configure;make;sudo make install in > >> MonetDB-Venus-SuperBall-SR1/MonetDB-client/clients > > > >Do you also have swig installed? > > > >Niels > >> > >> Result is the same: > >> > >> -MonetSQLdb is not deployed to Mac OS X 10.4 default python's > >> site-packages > >> > >> -in > >> MonetDB-Venus-SuperBall-SR1/MonetDB-client/clients/src/python/ > >>Cimpl > >> there is a folder called import MonetSQLdb, so start python there, > >> type "import MonetSQLdb". Error message is: > >> > >> macu:~/Desktop/clients/src/python/Cimpl alex$ python > >> > >> Python 2.3.5 (#1, Mar 20 2005, 20:38:20) > >> > >> [GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin > >> > >> Type "help", "copyright", "credits" or "license" for more > >> information. > >> > >>>>>import MonetSQLdb > >> > >> Traceback (most recent call last): > >> > >> File "<stdin>", line 1, in ? > >> > >> File "MonetSQLdb/__init__.py", line 45, in ? > >> > >> from CMapi import * > >> > >> File "CMapi.py", line 17, in ? > >> > >> import MapiLib > >> > >> File "MapiLib.py", line 5, in ? > >> > >> import _MapiLib > >> > >> ImportError: No module named _MapiLib > >> > >>>>>^D > >> > >> macu:~/Desktop/clients/src/python/Cimpl alex$ ls > >> > >> CMapi.py Makefile.am MapiLib.py MapiLib.pyc > >> > >> CMapi.pyc Makefile.in MapiLib.py.c MonetSQLdb > >> > >> Makefile Makefile.msc MapiLib.py.i README > >> > >> macu:~/Desktop/clients/src/python/Cimpl alex$ ls MonetSQLdb/ > >> > >> Makefile __init__.py monetexceptions.py > >> > >> Makefile.am __init__.pyc times.py > >> > >> Makefile.in converters.py > >> > >> Makefile.msc cursors.py > >> > >> On Windows I did the same: copy manuallyMonetSQLdb to site- > >>packages, > >> and add the missing *.py and *.pyc files from all folders into > >> MonetSQLdb until import stopped complaining. However, on Mac can't > >> figure all the _ files, so I've got a Monet server + SQL > >>client, but > >> no python library. Please advice. > >> > >> BR Alexandru > >> > >> On May 9, 2007, at 10:33 AM, Fabian Groffen wrote: > >> > >> On 09-05-2007 10:28:02 +0300, Alexandru Toth wrote: > >> > >> Hello, > >> > >> True, Python 2.4 and 2.5 were installed from Darwinports. > >> > >> Where to configure which Python to use in MonetDB build? > >>Builtin > >> > >> Python2.3 would be ok. Or, just remove extra Pythons from > >>PATH ? > >> > >> Only the latest install script has support for giving configure > >> extra > >> > >> options. In your case I would opt for removing DP from your > >> > >> environment, however, then the build will fail because you miss > >>the > >> > >> functionality provided by DP. In my opinion, DP fails here, > >>because > >> it > >> > >> doesn't help you here, like it should. Anyway. In my own work > >>this > >> > >> whould have worked, here you probably need some LDFLAGS hackery to > >> point > >> > >> to your DP offset. LDFLAGS="-L/path/to/lib". Maybe that helps > >> here. > >> > >> There was another error of missing gawk in the main build > >>script > >> > >> (Python client makefile looks for awk gawk nawk..), and sed > >> > >> complaining about unsupported --version > >> > >> The error handler calls various tools with --version. If that > >>fails > >> we > >> > >> basically already know enough. In your case here, this is not > >> > >> important. > > > >-- > > > >Niels Nes, Centre for Mathematics and Computer Science (CWI) > >Kruislaan 413, 1098 SJ Amsterdam, The Netherlands > >room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 > >url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

[Monetdb-developers] Fwd: Re: Fwd: pf/tijah startNodes: node* or string*
by Jan Flokstra 10 May '07

10 May '07

---------- Forwarded Message ---------- Subject: Re: Fwd: pf/tijah startNodes: node* or string* Date: Tuesday 08 May 2007 17:13 From: Jens Teubner <jens.teubner(a)in.tum.de> To: Jan Flokstra <flokstra(a)cs.utwente.nl> Cc: monetdb-developers(a)lists.sourceforge.net On Tue, May 08, 2007 at 03:39:03PM +0200, Jan Flokstra wrote: > Hi Jens, > > Hope you are still in Munich:-) We have a little problem here which we > do not known how to debug. It is described extensively in the the > e-mail at the end of this mail. But the basic problem is we do not > know how to debug a typing problem. We have the following query: > > tijah:query(fn:collection("c"),"//LINE[about(.,foul)]") > > And the signatures of the function involved are: > > (0) fn:collection (string) as node* > (1) tijah:query(string) as node* > (2) tijah:query(item*, string) as node* > (3) tijah:query(string, item) as node* > (4) tijah:query(item*, string, item) as node* Yes, I am still here. First of all, the function signatures that you listed here are the ones that appear in the *comments* in xquery_fo.c. Unfortunately, the code says something different! Very dangerous! I have corrected this some minutes ago. So here's what happens during Core generation and type checking and why your example makes trouble. (It boils down to a conflict between the signatures (2) and (3).) Out of the possible implementations for a function (tijah:query()), Pathfinder's function scope checker assigns those to the function application that have the matching *number of parameters*. This is in the spirit of the W3C specs: user-defined functions may be overloaded, but must differ in the number of arguments (and not just be distinguishable by their types). Only built-in functions may be overloaded in the sense that two functions may have the same number of arguments, but different argument types. Then, Core generation in fs.brg looks at the required type for each of the arguments. If the function expects an atomic value, the function argument is *atomized* (i.e., nodes are converted into something atomic, typically xs:untypedAtomic). Pathfinder's type checker finally selects the most specific implementation if there is more than one for the given number of argument (only really relevant for built-ins). The problem that you are experiencing is a combination of the two foregoing paragraphs. The function scope checker assigns those function implementations with two arguments to the `apply' node (signatures (2) and (3)). Core generation then (randomly) picks on of these two to determine whether atomization is required. Type checking finally throws the error. This behavior is perfectly okay for all functions we had so far and for all functions in the W3C specs. The combination of (a) two function signatures with the same number of arguments and (b) arguments having a node type in one signature and an atomic type in the other is what bites you. You probably should rethink your two function signatures (2) and (3). Note that they also introduce some ambiguity. If I call the function as `tijah:query (node, node)', should the first or the second argument be casted to an atomic value? Note that this implicit atomization happens everywhere in XQuery (if necessary) and it is perfectly consistent to do this here as well. A clean solution would work without such two conflicting signatures. A hack could be to treat tijah:query() explicitly in the involved parts in the Pathfinder code and do something magic to make things work. I would not recommend to do so, though, as it primarily would introduce inconsistency. Don't hesitate to ask again if my answers weren't clear. Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Please avoid sending me MS Word, MS Excel or MS PowerPoint files. Why? See http://www.fsf.org/philosophy/no-word-attachments.html -------------------------------------------------------

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, 1.181, 1.182
by Sjoerd Mullender 09 May '07

09 May '07

On 05/09/2007 01:58 PM, Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16:/tmp/cvs-serv2304 > > Modified Files: > gdk_utils.mx > Log Message: > bug with terabyte memory usage was introduced in 1.172 (sjoerd) > - GDKmallocmax(size, *maxsize) => > this routine is also used for realloc and maxsize is an input parameter, > so must be initialized. Did you look at the *reason* for that change? I think not. The reason was that the size value afterwards was wrong for the call to memset. Now you have reintroduced that bug. What you *should* have done was to initialize maxsize with size. > > Index: gdk_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > retrieving revision 1.181 > retrieving revision 1.182 > diff -u -d -r1.181 -r1.182 > --- gdk_utils.mx 2 May 2007 16:17:04 -0000 1.181 > +++ gdk_utils.mx 9 May 2007 11:58:06 -0000 1.182 > @@ -1070,11 +1070,8 @@ > void * > GDKzalloc(size_t size) > { > - size_t maxsize; > - void *p = GDKmallocmax(size, &maxsize, 1); > - > - if (p) > - memset(p, 0, size); > + void *p = GDKmallocmax(size, &size, 1); > + if (p) memset(p, 0, size); > return p; > } > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

[Monetdb-developers] SERIAL column,
by Romulo Goncalves 09 May '07

09 May '07

Good afternoon, sql>create table t1 (id SERIAL, PRIMARY KEY(id)); MAPI = monetdb@localhost:50000 QUERY = create table t1 (id SERIAL, PRIMARY KEY(id)); ERROR = !CONSTRAINT PRIMARY KEY: a table can have only one PRIMARY KEY sql>create table t1 (id SERIAL); sql>insert into t1 values(5); [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; [ 1 ] sql>insert into t1; MAPI = monetdb@localhost:50000 QUERY = insert into t1; ERROR = !SQLException:assert:INSERT INTO: PRIMARY KEY constraint 't1.t1_id_pkey' violated sql> The constraint check is correct, but should the first insertion be allowed? I checked the parser to know what SERIAL means: | column SERIAL { /* SERIAL = INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY */ Then I check what means GENERATED ALWAYS means: http://troels.arvin.dk/db/rdbms/#mix-identity "When ALWAYS is specified, the user cannot specify a value for the column which means that the DBMS can guarantee successful insertion of a unique value on each table insert." Who is right? The web-page or MonetDB? I tried to find something in the standard book and nothing (maybe I did not check correctly). Regards, Romulo

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_server.mx, 1.44, 1.45
by Fabian Groffen 09 May '07

09 May '07

On 09-05-2007 08:40:10 +0000, Stefan Manegold wrote: > In any case, this once again proves how important and "handy" > multi-platform testing is to detect and eventually fix bugs > --- provided someone does care at all ... Yeah, but not for any price. While you can get it for free, you appear to *insist* on paying a very high price for it.

1 0

[Monetdb-developers] Optimizers...
by Romulo Goncalves 08 May '07

08 May '07

Please, when you change an optimizer or you insert a new optimizer check the tests before and after your changes. After 1000 times it is time to learn. The test sql/src/test/Triggers/trigger_activation.sql is red. However, if I do set optimizer='off'; on the begin of the test it starts to work. Sorry, but this time I will not fix this bug. Regards, Romulo

2 4

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/Views/Tests All, 1.1, 1.2
by Romulo Goncalves 08 May '07

08 May '07

Martin Kersten wrote: > Update of /cvsroot/monetdb/sql/src/test/Views/Tests > In directory sc8-pr-cvs16:/tmp/cvs-serv2956/Tests > > Modified Files: > All > Log Message: > Added test file for bug --[ 1714814 ] View creation using a function.... > > Update of /cvsroot/monetdb/sql/src/test/BugTracker/Tests In directory sc8-pr-cvs16:/tmp/cvs-serv23652 Modified Files: All create_column.SF-1643542.sql Added Files: infinite_loop.SF-1714809.sql view_creation_using_func.SF-1714814.sql This checkin was done at 13:21 by me ;) the test for the bug was added to the BugTracker with the name of the bug_where_bug_was_reported_bug_number In views I only add normal tests created by me to test the views to detect future crash due modifications on the code. The tests that are there are do not come from bugs reports. Regards, Romulo > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Views/Tests/All,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- All 2 Nov 2006 15:04:43 -0000 1.1 > +++ All 8 May 2007 15:33:42 -0000 1.2 > @@ -1 +1,2 @@ > Views > +view_functioncall > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins

1 0

Re: [Monetdb-developers] Fwd: pf/tijah startNodes: node* or string*
by Jens Teubner 08 May '07

08 May '07

On Tue, May 08, 2007 at 03:39:03PM +0200, Jan Flokstra wrote: > Hi Jens, > > Hope you are still in Munich:-) We have a little problem here which we > do not known how to debug. It is described extensively in the the > e-mail at the end of this mail. But the basic problem is we do not > know how to debug a typing problem. We have the following query: > > tijah:query(fn:collection("c"),"//LINE[about(.,foul)]") > > And the signatures of the function involved are: > > (0) fn:collection (string) as node* > (1) tijah:query(string) as node* > (2) tijah:query(item*, string) as node* > (3) tijah:query(string, item) as node* > (4) tijah:query(item*, string, item) as node* Yes, I am still here. First of all, the function signatures that you listed here are the ones that appear in the *comments* in xquery_fo.c. Unfortunately, the code says something different! Very dangerous! I have corrected this some minutes ago. So here's what happens during Core generation and type checking and why your example makes trouble. (It boils down to a conflict between the signatures (2) and (3).) Out of the possible implementations for a function (tijah:query()), Pathfinder's function scope checker assigns those to the function application that have the matching *number of parameters*. This is in the spirit of the W3C specs: user-defined functions may be overloaded, but must differ in the number of arguments (and not just be distinguishable by their types). Only built-in functions may be overloaded in the sense that two functions may have the same number of arguments, but different argument types. Then, Core generation in fs.brg looks at the required type for each of the arguments. If the function expects an atomic value, the function argument is *atomized* (i.e., nodes are converted into something atomic, typically xs:untypedAtomic). Pathfinder's type checker finally selects the most specific implementation if there is more than one for the given number of argument (only really relevant for built-ins). The problem that you are experiencing is a combination of the two foregoing paragraphs. The function scope checker assigns those function implementations with two arguments to the `apply' node (signatures (2) and (3)). Core generation then (randomly) picks on of these two to determine whether atomization is required. Type checking finally throws the error. This behavior is perfectly okay for all functions we had so far and for all functions in the W3C specs. The combination of (a) two function signatures with the same number of arguments and (b) arguments having a node type in one signature and an atomic type in the other is what bites you. You probably should rethink your two function signatures (2) and (3). Note that they also introduce some ambiguity. If I call the function as `tijah:query (node, node)', should the first or the second argument be casted to an atomic value? Note that this implicit atomization happens everywhere in XQuery (if necessary) and it is perfectly consistent to do this here as well. A clean solution would work without such two conflicting signatures. A hack could be to treat tijah:query() explicitly in the involved parts in the Pathfinder code and do something magic to make things work. I would not recommend to do so, though, as it primarily would introduce inconsistency. Don't hesitate to ask again if my answers weren't clear. Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Please avoid sending me MS Word, MS Excel or MS PowerPoint files. Why? See http://www.fsf.org/philosophy/no-word-attachments.html

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_partitions.mx, 1.6, 1.7
by Stefan Manegold 07 May '07

07 May '07

On Mon, May 07, 2007 at 07:48:22AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16:/tmp/cvs-serv3297 > > Modified Files: > opt_partitions.mx > Log Message: > The first version of the partition code generator. > The code produced looks reasonable. > The necessary list module is not yet implemented. Nevertheless, checked-in code should always compile --- at least on one platform (and with the most conservative settings); however: ======== gcc -DHAVE_CONFIG_H -I. -I/ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer -I../.. -I/ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer -I../mal -I/ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer/../mal -I../modules/mal -I/ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer/../modules/mal -I/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-optimize_--enable-debug/include/MonetDB -I/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-optimize_--enable-debug/include/MonetDB/common -I/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-optimize_--enable-debug/include/MonetDB/gdk -DLIBOPTIMIZER -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -D_REENTRANT -c opt_partitions.c -fPIC -DPIC -o .libs/lib_optimizer_la-opt_partitions.o /ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer/opt_partitions.mx: In function 'OPTpartitionStrategy': /ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer/opt_partitions.mx:269: error: 'TYPE_list' undeclared (first use in this function) /ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer/opt_partitions.mx:269: error: (Each undeclared identifier is reported only once /ufs/manegold/_/scratch0/Monet/Testing/Current/source/MonetDB5/src/optimizer/opt_partitions.mx:269: error: for each function it appears in.) make[5]: *** [lib_optimizer_la-opt_partitions.lo] Error 1 ======== Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] An XQuery question
by Ying Zhang 07 May '07

07 May '07

Dear developers, If I run the follwoing XQuery query with Galax: ========== test.xq ========== declare namespace t = "test"; declare function t:echoVoid() { () }; t:echoVoid() ========== Galax gives a warning: $ galax-run ~/test.xq [WARNING] Expression '???' at File "/ufs/zhang/test.xq", line 3, characters 8-37 has empty type, but is not the empty expression () Does any one have an idea how I can correct/modify my query to get rid of this warning? Thanks in advance! Jennie

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah nexi.c, 1.57, 1.58 nexi_generate_mil.c, 1.27, 1.28 pftijah.mx, 1.116, 1.117
by Stefan Manegold 07 May '07

07 May '07

On Mon, May 07, 2007 at 07:01:43AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16:/tmp/cvs-serv20990/modules/pftijah > > Modified Files: > nexi.c nexi_generate_mil.c pftijah.mx > Log Message: > * Implement private term- and tag dictionaries. Default is still the global > version so nothing changed yet in the format. I'm having terrible trouble > with the Mtest's again. This version could not be tested by our Mtest tests > because the first query executes OK but all queries after that seem to > 'forget' the instance variables of module pftijah. I will look if the same > happens on other machine and post a bugreport for it. does the same happen if you run the test(s) by hand i.e., without Mtest? How do the queries and the test script(s) look like? How do you (successfully?) run them by hand? How do you (unsuccesfully) run them via Mtest? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] Mtest_4_sql
by Romulo Goncalves 04 May '07

04 May '07

I am running Mtest_4_SQL and I am getting strange error messages from MAPIPORT: ! Socket-Check failed for MAPIserver on <localhost:37631> with #98; 'Address already in use' ! ! MAPIPORT was not properly released by Mserver ! * (zones.test.err.FILTERED) significantly ... * (rank.test.out.FILTERED) significantly . ! Socket-Check failed for MAPIserver on <localhost:30932> with #98; 'Address already in use' ! ! MAPIPORT was not properly released by Mserver ! * (function_syntax.test.out.FILTERED) significantly * (function_syntax.test.err.FILTERED) significantly . ! Socket-Check failed for MAPIserver on <localhost:37107> with #98; 'Address already in use' ! ! MAPIPORT was not properly released by Mserver ! Any idea? Regards, Romulo

3 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/mil milprint_summer.c, 1.371, 1.372
by Stefan Manegold 02 May '07

02 May '07

Sjoerd, didn't like my fix? ;-) Well, actually, I'm afraid you just re-introduced the problem: in short ======== size_t maxbufsize = 0; char *nme = NULL; while (...) { ... if (maxbufsize < strlen(tpe) + 4) { maxbufsize = strlen(tpe) + 4; nme = nme ? realloc(nme, maxbufsize) : malloc(maxbufsize); } ... if (...) { *nme++ = 'x'; ... } } if (nme) free(nme); ... ======== IMHO does not work, since the nme++ changes nme, causing later realloc(nme, maxbufsize) and/or free(nme) to "behave strangly" --- or simply segfault ... Stefan On Wed, May 02, 2007 at 09:39:32PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/mil > In directory sc8-pr-cvs16:/tmp/cvs-serv18640 > > Modified Files: > milprint_summer.c > Log Message: > Fix bug introduced by replacing call to alloca. > > Index: milprint_summer.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint_summer.c,v > retrieving revision 1.371 > retrieving revision 1.372 > diff -u -d -r1.371 -r1.372 > --- milprint_summer.c 2 May 2007 21:15:18 -0000 1.371 > +++ milprint_summer.c 2 May 2007 21:39:30 -0000 1.372 > @@ -11142,17 +11142,17 @@ > /* initialize function variables */ > /* ============================= */ > size_t maxbufsize = 0; > - char *buf = NULL; > + char *nme = NULL; > while (args->kind != c_nil) > { > /* get the type name, and assure there *is* a namespace */ > char *tpe = PFty_str(TY(LR(args))); > - char *nme; > + char *buf; > if (maxbufsize < strlen(tpe) + 4) { > maxbufsize = strlen(tpe) + 4; > - buf = buf ? realloc(buf, maxbufsize) : malloc(maxbufsize); > + nme = nme ? realloc(nme, maxbufsize) : malloc(maxbufsize); > } > - nme = buf; > + buf = nme; > if (strchr(tpe, ':') == NULL) { > *nme++ = 'x'; > *nme++ = 's'; > @@ -11178,8 +11178,8 @@ > > args = R(args); > } > - if (buf) > - free(buf); > + if (nme) > + free(nme); > /* create the full signature that also is a valid MIL identifier */ > c->sem.fun->sig = PFmalloc(12+3*(strlen(sig)+strlen(p))); > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/tools embeddedclient.mx, 1.23, 1.24 monetdb.py.i, 1.5, 1.6 prog.c, 1.17, 1.18
by Jan Rittinger 02 May '07

02 May '07

perhaps its triggered by Niels newest ``any type'' changes in bat_store.mx -- just guessing. On 04/27/2007 03:47 PM, Jan Rittinger wrote with possible deletions: > I just managed to install gdb as well and it seems like it is memory > related (BTW pathfinder works). > > gdb M4: > MonetDB>module(sql_server); > [New Thread 1082132832 (LWP 896)] > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 182925371136 (LWP 887)] > 0x0000002a97183c1f in memcpy () from /lib64/tls/libc.so.6 > (gdb) bt > #0 0x0000002a97183c1f in memcpy () from /lib64/tls/libc.so.6 > #1 0x0000002a9f960718 in ?? () > #2 0x0000002a965eff6c in HEAPcopy () from > /home/ntap/rittinge/work/MonetDB/Linux/lib64/libbat.so.0 > #3 0x0000002a966b340e in BATcopy () from > /home/ntap/rittinge/work/MonetDB/Linux/lib64/libbat.so.0 > #4 0x0000002a965f0a34 in BATkdiff () from > /home/ntap/rittinge/work/MonetDB/Linux/lib64/libbat.so.0 > #5 0x0000002a9e8713f9 in load_trans () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/MonetDB4/lib/lib_sql_server.so > > #6 0x0000002a9e873b65 in store_init () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/MonetDB4/lib/lib_sql_server.so > > #7 0x0000002a9e846b1a in mvc_init () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/MonetDB4/lib/lib_sql_server.so > > #8 0x0000002a9e80276a in sql_frontend_wrap () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/MonetDB4/lib/lib_sql_server.so > > #9 0x0000002a95f8bd48 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #10 0x0000002a95f9c0aa in interpret_assignment () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #11 0x0000002a95f8bfb6 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #12 0x0000002a95f9bb29 in interpret_seqblock () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #13 0x0000002a95f8bf98 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #14 0x0000002a95a32d44 in CMDcatch () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/MonetDB4/lib/lib_builtin.so.0 > #15 0x0000002a95f8bd48 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #16 0x0000002a95f9bebb in interpret_assignment () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #17 0x0000002a95f9c2af in interpret_var () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #18 0x0000002a95f8c118 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #19 0x0000002a95f9bb29 in interpret_seqblock () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #20 0x0000002a95f8bf98 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #21 0x0000002a95f8c4ec in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #22 0x0000002a95f9bb29 in interpret_seqblock () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #23 0x0000002a95f8bf98 in interpret () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #24 0x0000002a95f9e289 in handleRequest () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #25 0x0000002a95f9efc9 in doRequest () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #26 0x0000002a95fe280e in monetInterpreter () from > /home/ntap/rittinge/work/MonetDB4/Linux/lib64/libmonet.so.0 > #27 0x00000000004020f1 in main () > > gdb M5: > > (gdb) run --dbinit="include sql;" > Starting program: /home/ntap/rittinge/work/MonetDB4/Linux/bin/mserver5 > --dbinit="include sql;" > [Thread debugging using libthread_db enabled] > [New Thread 182918905952 (LWP 938)] > # MonetDB Server v5.0.0_beta2_1 > # Copyright (c) 1993-2007 CWI, all rights reserved > # Compiled for x86_64-suse-linux/64bit with 32bit OIDs dynamically linked > # dbname:demo > # Visit http://monetdb.cwi.nl/ for further information > [New Thread 1082132832 (LWP 939)] > #warning: please don't forget to set your vault key! > #(see /home/ntap/rittinge/work/MonetDB4/Linux//etc/monetdb5.conf) > > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 1082132832 (LWP 939)] > 0x0000002a969f8c1f in memcpy () from /lib64/tls/libc.so.6 > (gdb) bt > #0 0x0000002a969f8c1f in memcpy () from /lib64/tls/libc.so.6 > #1 0x0000000000b373d8 in ?? () > #2 0x0000002a95d2bf6c in HEAPcopy () from > /home/ntap/rittinge/work/MonetDB/Linux/lib64/libbat.so.0 > #3 0x0000002a95def40e in BATcopy () from > /home/ntap/rittinge/work/MonetDB/Linux/lib64/libbat.so.0 > #4 0x0000002a95d2ca34 in BATkdiff () from > /home/ntap/rittinge/work/MonetDB/Linux/lib64/libbat.so.0 > #5 0x0000002aa0403c59 in load_trans () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/MonetDB5/lib/lib_sql.so > #6 0x0000002aa04063c5 in store_init () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/MonetDB5/lib/lib_sql.so > #7 0x0000002aa03d937a in mvc_init () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/MonetDB5/lib/lib_sql.so > #8 0x0000002aa039507b in SQLinit () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/MonetDB5/lib/lib_sql.so > #9 0x0000002a956ad57b in initScenario () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/libmal.so.0 > #10 0x0000002a956ad81f in setScenario () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/libmal.so.0 > #11 0x0000002a9568528b in MSscheduleClient () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/libmal.so.0 > #12 0x0000002a9f80e371 in doChallenge () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/MonetDB5/lib/lib_mserver.so > #13 0x0000002a9f80ece9 in SERVERlistenThread () from > /home/ntap/rittinge/work/MonetDB4/Linux//lib64/MonetDB5/lib/lib_mserver.so > #14 0x0000002a9663db8f in start_thread () from /lib64/tls/libpthread.so.0 > #15 0x0000002a96a414b3 in clone () from /lib64/tls/libc.so.6 > -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

2 3

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/mil milprint_summer.c, 1.367, 1.368
by Jan Flokstra 02 May '07

02 May '07

I have great compilation troubles with the last lesson from the master:-) The compiler doesn not know the realloc() and the malloc() function() in the current setting. When I change them into PFrealloc() and PFmalloc() it compiles. But then I still have problems with the "free(buf);" line. There is no PFfree(); and I think GDKfree() will not always work, JanF. On Tuesday 01 May 2007 17:58, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/mil > In directory sc8-pr-cvs16:/tmp/cvs-serv17830 > > Modified Files: > milprint_summer.c > Log Message: > When you want to use a library function, make sure the appropriate > include file is included. For alloca there is an extra twist: for the > way to include alloca.h see e.g. monet_utils.mx. Do not hide the lack > of a declaration with a cast. The compiler then thinks the function > returns an int which may well be smaller in space than the actual > value, and hence some bytes of the value may get lost. > > Having said this, alloca should *not* be used in a loop. It allocates > memory in each iteration which is only freed at the end of the > function. If the loop loops many times, that can be a lot of memory > which is allocated on the stack (which is a very finite resource!). > > Here ends today's lesson. > > > Index: milprint_summer.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint_summer.c,v > retrieving revision 1.367 > retrieving revision 1.368 > diff -u -d -r1.367 -r1.368 > --- milprint_summer.c 27 Apr 2007 18:22:40 -0000 1.367 > +++ milprint_summer.c 1 May 2007 15:57:59 -0000 1.368 > @@ -11140,12 +11140,18 @@ > /* ============================= */ > /* initialize function variables */ > /* ============================= */ > + size_t maxbufsize = 0; > + char *buf = NULL; > while (args->kind != c_nil) > { > /* get the type name, and assure there *is* a namespace */ > char *tpe = PFty_str(TY(LR(args))); > - char *buf = (char*) alloca(4+strlen(tpe)); > - char *nme = buf; > + char *nme; > + if (maxbufsize < strlen(tpe) + 4) { > + maxbufsize = strlen(tpe) + 4; > + buf = buf ? realloc(buf, maxbufsize) : malloc(maxbufsize); > + } > + nme = buf; > if (strchr(tpe, ':') == NULL) { > *buf++ = 'x'; > *buf++ = 's'; > @@ -11171,6 +11177,8 @@ > > args = R(args); > } > + if (buf) > + free(buf); > /* create the full signature that also is a valid MIL identifier > */ c->sem.fun->sig = PFmalloc(12+3*(strlen(sig)+strlen(p))); > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

3 3

Re: [Monetdb-developers] ! Testing 'Current' sql: Results DIFFER from 2007.04.28_04-03-41's (GDKdebug=10) !
by Romulo Goncalves 29 Apr '07

29 Apr '07

Project account wrote: > Testing results of sql in /export/scratch2/monet/Testing/Current with GDKdebug=10 DIFFER from those of 2007.04.28_04-03-41 ! > > Maybe YOUR checkins since 28.04.2007, 04:03:41 caused some of the problems: > > buildtools > MonetDB > clients > MonetDB4 > MonetDB5 mk(a)cwi.nl > sql mk(a)cwi.nl niels(a)cwi.nl romulog(a)users.sourceforge.net > > Check http://monetdb.cwi.nl/Development/TestWeb/Current/sql/ for details ! > > > Old file: Mtest10..2007.04.28_04-03-41..out > New file: Mtest10..2007.04.29_04-16-39..out > =========================================== > > > Summary: > ======== > > * 1 New systems > !! 3 Systems now DON'T produce output !! I compiled my sources before the commit with optimization on: --enable-strict --enable-assert --enable-optimize --enable-bits=64 Why the variable initialized, but never used is not detected by this compilation? Which compilation must be done to detect these kind of errors? Regards, Romulo

3 3

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/tools embeddedclient.mx, 1.23, 1.24 monetdb.py.i, 1.5, 1.6 prog.c, 1.17, 1.18
by Jan Rittinger 27 Apr '07

27 Apr '07

Compilation worked fine, but M4 crashes when I start with Mserver --dbinit="module(sql_server);" and M5 segfaults as soon as I start a mclient with mclient -lsql. As I compiled the whole suite from scratch after the propagation I think there is some problem in the code. On 04/27/2007 11:46 AM, Sjoerd Mullender wrote with possible deletions: > Update of /cvsroot/monetdb/sql/src/tools > In directory sc8-pr-cvs16:/tmp/cvs-serv3115/src/tools > > Modified Files: > embeddedclient.mx monetdb.py.i prog.c > Log Message: > propagated changes of Thursday Apr 12 2007 - Friday Apr 27 2007 > from the SQL_2-16 branch to the development trunk > > > Index: embeddedclient.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/tools/embeddedclient.mx,v > retrieving revision 1.23 > retrieving revision 1.24 > diff -u -d -r1.23 -r1.24 > --- embeddedclient.mx 9 Mar 2007 14:22:32 -0000 1.23 > +++ embeddedclient.mx 27 Apr 2007 09:46:00 -0000 1.24 > @@ -174,7 +174,7 @@ > out = ((stream **) arg)[1]; > free(arg); > snprintf(buf, sizeof(buf), "client(Stream(\"" PTRFMT "\"), Stream(\"" PTRFMT "\"));", PTRFMTCAST in, PTRFMTCAST out); > - monet_exec("module(sql_server);mapi_register(sql_frontend());"); > + monet_exec("module(sql_server);"); > monet_exec(buf); > return NULL; > } > > Index: prog.c > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/tools/prog.c,v > retrieving revision 1.17 > retrieving revision 1.18 > diff -u -d -r1.17 -r1.18 > --- prog.c 7 Jan 2007 10:18:04 -0000 1.17 > +++ prog.c 27 Apr 2007 09:46:00 -0000 1.18 > @@ -16,7 +16,7 @@ > * All Rights Reserved. > */ > > -#include <sql_config.h> > +#include <monetdb4_config.h> > #include <monet_options.h> > #include "embeddedclient.h" > > @@ -101,6 +101,11 @@ > if (!(setlen = mo_builtin_settings(&set))) > usage(prog); > > + /* needed, to prevent the MonetDB config file to be used */ > + setlen = mo_add_option(&set, setlen, opt_config, "prefix", MONETDBPREFIX); > + setlen = mo_add_option(&set, setlen, opt_config, "config", MONETDBCONFIG); > + setlen = mo_system_config(&set, setlen); > + > for (;;) { > int option_index = 0; > > > Index: monetdb.py.i > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/tools/monetdb.py.i,v > retrieving revision 1.5 > retrieving revision 1.6 > diff -u -d -r1.5 -r1.6 > --- monetdb.py.i 3 Jan 2007 12:39:54 -0000 1.5 > +++ monetdb.py.i 27 Apr 2007 09:46:00 -0000 1.6 > @@ -23,6 +23,8 @@ > #undef HAVE_FSTAT > #endif > #include "Mapi.h" > + > +extern Mapi monetdb_sql(char *dbfarm, char *dbname); > %} > > extern Mapi monetdb_sql(char *dbfarm, char *dbname); > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

3 3