- developers-list - monetdb.org

[Monetdb-developers] PF: (Failing) algebra tests in the XQuery_0-14 release branch
by Stefan Manegold 18 Dec '06

18 Dec '06

Dear fellow developers, triggered by the discussion in http://sourceforge.net/tracker/index.php?func=detail&aid=1543871&group_id=5… my question: Would anybody mind if we disable all currently failing pathfinder-algebra tests in the XQuery_0-14 release branch? (look for "alg" at http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103… ) Since the algebra version will not officially be supported in the XQuery_0-14 release, the respective problems will not be fixed in the branch, but only in the development trunk. Thanks in advance, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] CVS "stable" branch considerations
by Fabian Groffen 14 Dec '06

14 Dec '06

Hi all, A while ago I was discussing the "stable" branch for M5 with Niels, and we came up with the idea to have a "continuous", "configuration free" stable branch, to ease the life of developers/team members that prefer not to live on the (bleeding) edge of novel and exciting technologies. Basically, what the ground idea is, is that when you check out the "stable" branch, that you will always get the latest stable branch. In ASCII-grafix: HEAD o=======================o============================~ \ 1.0 \ 2.0 stable \_______+______+_____ \______+______+______+__ rc_1 rc_2 rc_3 rc_1 rc_2 rc_3 This image is not very different from the current state, but for the 1.0 and the 2.0 release branches, two separate branch names are used: stable-1_0 and stable-2_0. This makes that people who checked out stable-1_0 never get the "next" stable, but instead stay frozen with stable-1_0. The basic idea we had is that people get "upgraded" to the latest stable if they use the "stable" branch, such that they automatically get the latest and greatest, but stable whatever product. For assuring correct tracability a number of tags can be set on the branch, of course. Technically speaking, CVS allows to "force" a branch to be created that already existed, thereby achieving the idea of having a "stable" branch as described above. I'm in favour of such branch, as I think it solves a number of problems with the ever moving branch, and eases CVS use for many of us. It is also easier to remember, as I personally always mix up the captitals, hyphens and underscores. Now we are at the edge of a big CVS move (are we still?), I thought may be it's a good idea to use this strategy for all our modules now. Any thoughts on or complaints against such branch usage?

2 1

[Monetdb-developers] Pegatoo/Gentoo with GCC 4.3
by Fabian Groffen 14 Dec '06

14 Dec '06

Hi all, This is a heads up, that "tonights" testing will be confronted to a GCC 4.3 compiler on Pegatoo, the Gentoo Linux node. GCC 4.3 will most probably be the next compiler that gets stable on Gentoo (like 4.0 was never stable either), hence early testing might be nice. If it results in too many problems, I'll switch it down to GCC 4.2 instead afterwards.

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/leaks/Tests check0.stable.out, 1.1.2.2, 1.1.2.3 check1.stable.out, 1.1.2.2, 1.1.2.3 check2.stable.out, 1.1.2.2, 1.1.2.3 check3.stable.out, 1.1.2.2, 1.1.2.3 check4.stable.out, 1.1.2.1, 1.1.2.2 check5.stable.out, 1.1.2.1, 1.1.2.2
by Martin Kersten 13 Dec '06

13 Dec '06

ziek ?:-) of alleen teveeel gedanst.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx,
by p.a.boncz＠chello.nl 13 Dec '06

13 Dec '06

Hoi Sjoerd, In de log message ben ik denk ik vergeten om op te schrijven dat ik consecutive text node coalescing uit had gezegd; daarentegen heb ik je een uitgebreide email geschreven waarin ik dit verteld heb dat en waarom ik dat gedaan heb. De fix_consecutive_text code heeft een groot performance probleem. Het maakt in feite deletes onbruikbaar. Op de 1GB document test heb ik hem na enige minuten afgebroken. Omdat ik niet wist hoe die code sneller gemaakt kan worden heb ik het uitgezet. Hoewel technisch niet correct, is het voor weinig applicaties een ramp als er twee text nodes na elkaar komen. De pathfinder gegenereerde code breekt daar niet van, voor zover ik weet. Maar ik begrijp dat jij het hier niet mee eens bent.. Het beste zou zijn om fix-consecutive text te beperken tot de range van nodes direct om het delete punt heen. Dan zou performance geen probleem hoeven te zijn. Wat ik zag was dat er een CTrefine over het hele document heen ging. Peter > > van: Sjoerd Mullender <sjoerd(a)acm.org> > datum: 2006/12/12 Tue PM 04:11:49 CET > aan: monetdb-developers(a)lists.sourceforge.net > cc: monetdb-pf-checkins(a)lists.sourceforge.net > onderwerp: Re: [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, > 1.269.2.14, 1.269.2.15 pf_support.mx, 1.172.2.11, 1.172.2.12 > >

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, 1.269.2.14, 1.269.2.15 pf_support.mx, 1.172.2.11, 1.172.2.12
by Sjoerd Mullender 12 Dec '06

12 Dec '06

On 2006-12-09 16:19, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv28149 > > Modified Files: > Tag: XQuery_0-14 > pathfinder.mx pf_support.mx > Log Message: > various performance fixes; small inserts and deletes are now reasonably fast > > - index new qnames in qnloc index > - better handling of the hash tables on the qn_loc index > - shortened the collection lock time by moving most of the logging > out of the critical period > > > > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.172.2.11 > retrieving revision 1.172.2.12 > diff -u -d -r1.172.2.11 -r1.172.2.12 > --- pf_support.mx 8 Dec 2006 17:48:43 -0000 1.172.2.11 > +++ pf_support.mx 9 Dec 2006 15:19:52 -0000 1.172.2.12 > @@ -2096,47 +2097,22 @@ > } > > var texts := ws.fetch(UPDATED_TEXT); > - fix_consecutive_texts(ws, texts); > + #fix_consecutive_texts(ws, texts); > > # get containers in the order in which we must commit them ([COLL_ID,CONT]) > var cont_order := affected_conts.reverse().mirror().leftfetchjoin(ws.fetch(CONT_COLL)).reverse().sort(); > 1. what does this change have to do with the log message? 2. this change breaks the document: if a node between two text nodes is deleted, the two text nodes have to be coalesced. That is what this function does. It is a necessary step to keep subsequent queries working correctly. -- Sjoerd Mullender

1 0

[Monetdb-developers] MonetDB as MySQL storage engine
by Arman Bostani 12 Dec '06

12 Dec '06

Hello all, Has anyone considered the possibility of using MonetDB as a MySQL storage engine. This idea is appealing since, theoretically, you'd immediately have access to MySQL features such as clustering, replication, etc. I would guess, however, that you would lose much of MonetDB performance advantages. Any thoughts? -arman

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder MonetDB-XQuery.spec.in, 1.16, 1.17 Makefile.ag, 1.22, 1.23 extra.mk, 1.18, 1.19
by Stefan Manegold 09 Dec '06

09 Dec '06

Niels, these changes are not meant for the release? Stefan On Sat, Dec 09, 2006 at 11:13:52AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/pathfinder > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv22364 > > Modified Files: > MonetDB-XQuery.spec.in Makefile.ag extra.mk > Log Message: > install howtos is done within the monetweb tools > > > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/Makefile.ag,v > retrieving revision 1.22 > retrieving revision 1.23 > diff -u -d -r1.22 -r1.23 > --- Makefile.ag 30 Nov 2006 11:56:57 -0000 1.22 > +++ Makefile.ag 9 Dec 2006 11:13:50 -0000 1.23 > @@ -26,11 +26,21 @@ > > EXTRA_DIST = rules.mk extra.mk RunMtest.in RunMtest.bat.in \ > RunMapprove.in RunMapprove.bat.in \ > - MonetDB-XQuery.spec.in MonetDB-XQuery.spec rpm.mk.in \ > - HowToStart-PF HowToStart-MinGW.txt > + MonetDB-XQuery.spec.in MonetDB-XQuery.spec rpm.mk.in > EXTRA_DIST_DIR = NT tests > > EXTRA_HEADERS = pf_config.h > > +scripts_txt = { > + SOURCES = HowToStart-MinGW.txt > + DIR = datadir/doc/MonetDB-XQuery-${VERSION} > +} > + > +scripts_docs = { > + EXT = "" > + SOURCES = HowToStart-PF > + DIR = datadir/doc/MonetDB-XQuery-${VERSION} > +} > + > TESTS = RunMtest > TESTS_ENVIRONMENT = $(SHELL) > > Index: extra.mk > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/extra.mk,v > retrieving revision 1.18 > retrieving revision 1.19 > diff -u -d -r1.18 -r1.19 > --- extra.mk 8 Nov 2006 13:16:44 -0000 1.18 > +++ extra.mk 9 Dec 2006 11:13:50 -0000 1.19 > @@ -68,19 +68,3 @@ > # > doc: > cd $(top_srcdir) ; doc/gen_doc.sh > - > -# > -# Stefan's automated test system uses `make docs', not `make doc'. > -# Make him happy as well. > -# > -docs: \ > - $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix/index.html \ > - $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows/index.html > - > -$(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix/index.html: $(top_srcdir)/HowToStart-PF > - mkdir -p $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix > - rst2html.py --stylesheet http://monetdb.cwi.nl/MonetDB.css --link-stylesheet $(top_srcdir)/HowToStart-PF > $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Unix/index.html > - > -$(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows/index.html: $(top_srcdir)/HowToStart-MinGW.txt > - mkdir -p $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows > - rst2html.py --stylesheet http://monetdb.cwi.nl/MonetDB.css --link-stylesheet $(top_srcdir)/HowToStart-MinGW.txt > $(prefix)/doc/MonetDB/GetGoing/Setup/XQuery/Windows/index.html > > Index: MonetDB-XQuery.spec.in > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/MonetDB-XQuery.spec.in,v > retrieving revision 1.16 > retrieving revision 1.17 > diff -u -d -r1.16 -r1.17 > --- MonetDB-XQuery.spec.in 1 Nov 2006 17:48:37 -0000 1.16 > +++ MonetDB-XQuery.spec.in 9 Dec 2006 11:13:50 -0000 1.17 > @@ -21,8 +21,6 @@ > > %define comp_cc @CC@ > > -%define builddoc 0 > - > # groups of related archs > %define all_x86 i386 i586 i686 > > @@ -47,15 +45,6 @@ > Requires: %{name}-server > Requires: MonetDB-devel > > -%if %{builddoc} > -%package docu > -Summary: MonetDB XQuery documentation package > -Group: System > - > -%description docu > -Add the MonetDB documentaion description here > -%endif > - > %prep > rm -rf $RPM_BUILD_ROOT > > @@ -95,18 +84,8 @@ > %{_libdir}/MonetDB/*.mil > %{prefix}/bin/Mbeddedxq > %{_libdir}/libembeddedxq.* > +%{_datadir}/doc/%{name}-%{version}/* > > %files devel > %defattr(-,monetdb,monetdb) > %{prefix}/include/pathfinder/*.h > - > -%if %{builddoc} > -%files docu > -%defattr(-,monetdb,monetdb) > -%{prefix}/share/MonetDB/docs/pathfinder/*.html > -%{prefix}/share/MonetDB/docs/pathfinder/*.pdf > -%{prefix}/share/MonetDB/docs/pathfinder/*.ps > -%{prefix}/share/MonetDB/docs/pathfinder/*/*.html > -%{prefix}/share/MonetDB/docs/pathfinder/*/*.pdf > -%{prefix}/share/MonetDB/docs/pathfinder/*/*.ps > -%endif > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/doc mxdoc.tex, 1.5, NONE
by Stefan Manegold 08 Dec '06

08 Dec '06

Martin, just two observations I made when seein these checkins: - the changes were made on the development trunk, and hence will not be in the release. - `make docs` for MonetDB will fail, now as MonetDB/html.mk still refers to the no longer existing MonetDB/doc/mxdoc.tex Stefan (yes, I am German and will always be!) On Fri, Dec 08, 2006 at 09:08:48AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB/doc > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv16022 > > Removed Files: > mxdoc.tex > Log Message: > Mx documentation is moved to the buildtools. > > > --- mxdoc.tex DELETED --- > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/mil milprint_summer.c, 1.298.2.4, 1.298.2.5
by Stefan Manegold 07 Dec '06

07 Dec '06

Peter, the changes below seem to make 3 of the 6 Music/limit tests fail with " !ERROR: interpret: no matching MIL operator to 'seqbase(int, oid)'. !MAYBE YOU MEAN: ! seqbase(BAT[void,any::1], oid) : BAT[void,any::1] ! seqbase(BAT[void,any::1]) : oid ! seqbase(BAT[oid,any::1]) : oid " What kind of solution/fix do you recommend? (1) set the seqbase only if the type is BAT, or (2) add a "fake" constant-seqbase variant to the constant module ? Thanks! Stefan On Tue, Nov 28, 2006 at 02:18:09PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/mil > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv14236 > > Modified Files: > Tag: XQuery_0-14 > milprint_summer.c > Log Message: > - [ 1600704 ] XQuery: subsequence > 1 fails > > we require the seqbase of our iter|pos|item relations to be 0@0 > > > > Index: milprint_summer.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint_summer.c,v > retrieving revision 1.298.2.4 > retrieving revision 1.298.2.5 > diff -u -d -r1.298.2.4 -r1.298.2.5 > --- milprint_summer.c 16 Nov 2006 17:52:56 -0000 1.298.2.4 > +++ milprint_summer.c 28 Nov 2006 14:18:06 -0000 1.298.2.5 > @@ -7641,10 +7641,10 @@ > > milprintf(f, "\n" > " # select a slice\n" > - " ipik := ipik.slice(int(lo),hi);\n" > - " iter := iter.slice(int(lo),hi);\n" > - " kind := kind.slice(int(lo),hi);\n" > - " item%s := item%s.slice(int(lo), hi);\n" > + " ipik := ipik.slice(int(lo),hi).seqbase(0@0);\n" > + " iter := iter.slice(int(lo),hi).seqbase(0@0);\n" > + " kind := kind.slice(int(lo),hi).seqbase(0@0);\n" > + " item%s := item%s.slice(int(lo), hi).seqbase(0@0);\n" > " pos := ipik.mark(1@0);\n" > "} else {\n" > " # evaluate selection tuple by tuple (note: fully constant-resistant code)\n" > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1