- developers-list - monetdb.org

Re: [Monetdb-developers] MonetDB: default - Revert the changes of May 04 to recover perfo...
by Stefan Manegold 09 May '10

09 May '10

In case the changes of May 04 indeed caused the reported performance degradation, I assume this checkin should then also be back-ported to the Jun2010 branch that was created on May 06 and hence also contains the problematic changes of May 04, right? Once confirmed and I can back-port this checkin. Stefan On Sat, May 08, 2010 at 08:30:35PM +0200, Martin.Kersten(a)cwi.nl wrote: > Changeset: e9c1206c5008 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e9c1206c5008 > Modified Files: > MonetDB5/src/mal/mal_interpreter.mx > clients/src/python/test/capabilities.py > clients/src/python/test/dbapi20.py > geom/src/sql/geom.sql > Branch: default > Log Message: > > Revert the changes of May 04 to recover performance. > > > diffs (76 lines): > > diff -r dd777c76a3f9 -r e9c1206c5008 MonetDB5/src/mal/mal_interpreter.mx > --- a/MonetDB5/src/mal/mal_interpreter.mx Sat May 08 09:16:12 2010 +0200 > +++ b/MonetDB5/src/mal/mal_interpreter.mx Sat May 08 20:30:23 2010 +0200 > @@ -341,7 +341,7 @@ > > /* pass the new debug mode to the caller */ > if (stk->cmd && env && stk->cmd!='f') env->cmd = stk->cmd; > - if ( garbageControl(getInstrPtr(mb,0)) ) > + if ( !stk->keepAlive && garbageControl(getInstrPtr(mb,0)) ) > garbageCollector(cntxt, mb,stk, env != stk); > @:endProfile(stk)@ > return ret; > @@ -361,15 +361,19 @@ > MalStkPtr stk, MalStkPtr env, InstrPtr pcicaller) > { > str ret; > + int keepAlive; > > if (stk == NULL) > throw(MAL, "mal.interpreter", MAL_STACK_FAIL); > + keepAlive = stk->keepAlive; > if(env && stk && stk->cmd!='f') stk->cmd = env->cmd ; > > ret = runMALsequence(cntxt, mb, startpc, stoppc, stk, env, pcicaller); > > /* pass the new debug mode to the caller */ > if (env && stk->cmd!='f') env->cmd = stk->cmd; > + if ( keepAlive == 0 && garbageControl(getInstrPtr(mb,0)) ) > + garbageCollector(cntxt, mb,stk,env!=stk); > return ret; > } > @- > @@ -1336,7 +1340,6 @@ > { > DataFlow flow = NULL; > str ret = MAL_SUCCEED; > - int keepalive; > > #ifdef DEBUG_FLOW > stream_printf(GDKstdout, "runMALdataflow for block %d - %d\n", startpc,stoppc); > @@ -1387,9 +1390,6 @@ > flow->stop = stoppc; > flow->mb = mb; > flow->stk = stk; > - /* avoid the garbage collection call at the end of interpreter loop */ > - keepalive = flow->stk->keepAlive; > - flow->stk->keepAlive= TRUE; > > flow->status = (char*) GDKzalloc((stoppc-startpc+1)); > flow->blocked = (char*) GDKzalloc(sizeof(char)*mb->vtop); > @@ -1397,7 +1397,6 @@ > mal_unset_lock(mal_contextLock, "runMALdataflow"); > > ret = DFLOWscheduler(flow); > - flow->stk->keepAlive= keepalive; > GDKfree(flow->status); > GDKfree(flow->blocked); > GDKfree(flow->inuse); > @@ -1764,7 +1763,7 @@ > sbackup[i]= 0; > backup[i]= 0; > garbage[i]= -1; > - if (stk->stk[getArg(pci,i)].vtype == TYPE_bat && getEndOfLife(mb,getArg(pci,i)) == stkpc && isNotUsedIn(pci,i+1,getArg(pci,i)) && stk->keepAlive == 0){ > + if (stk->stk[getArg(pci,i)].vtype == TYPE_bat && getEndOfLife(mb,getArg(pci,i)) == stkpc && isNotUsedIn(pci,i+1,getArg(pci,i)) ){ > garbage[i]= getArg(pci,i); > #ifdef DEBUG_GC > stream_printf(GDKstdout,"GC %d %s prep\n",getArg(pci,i), getArgName(mb,pci,i)); > @@ -2345,8 +2344,6 @@ > int k; > ValPtr v; > > - if ( stk->keepAlive) > - return; > #ifdef STACKTRACE > stream_printf(cntxt->fdout,"#--->stack before garbage collector\n"); > printStack(cntxt->fdout,mb,stk,0); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

3 2

Re: [Monetdb-developers] commit mail subject
by Fabian Groffen 07 May '10

07 May '10

On 07-05-2010 18:38:42 +0200, Stefan Manegold wrote: > On Fri, May 07, 2010 at 06:31:21PM +0200, Jan Rittinger wrote: > > Hi all, > > > > I think the subject format of the checkin message mail could be improved by changing the format from 'MonetDB: <log-message> (<branch>)' or sometimes 'MonetDB: <partial log-message>...' to 'MonetDB (<branch>): <log-message>' or 'MonetDB (<branch>): <partial log-message>...'. > > FYI: > > Mercurial uses the first line of checkin messages as summary in (e.g.) logs, > etc. and as email subject. > > It depends on the committer how useful this first line (or "partial > log-message") in isolation, and hence, the summary and email subject is > --- in particular for other --- but that holds for the complete checkin > message, too ... If the first line is too long (like a full page) the line is truncated by the script, guess that's what Jan refers too. Makes sense from a mail point of view. > (Sjoerd pointed this out in his Mercurial intorduction during MADAM and in > his MonetDB-Mercirial document.) > > > In short I vote for switching the order of <log-message> and <branch> in the subject. > > Fabian patched the Mercurial setup to add the brnach to the checkin email > subject. I guess, he can also prepend instead of append it. > > Would be fine by me. (In fact, I like your suggestion ;-)) Ok, that makes the score already 2 then :)

2 3

[Monetdb-developers] commit mail subject
by Fabian Groffen 07 May '10

07 May '10

On 07-05-2010 18:31:21 +0200, Jan Rittinger wrote: > Hi all, > > I think the subject format of the checkin message mail could be improved by changing the format from 'MonetDB: <log-message> (<branch>)' or sometimes 'MonetDB: <partial log-message>...' to 'MonetDB (<branch>): <log-message>' or 'MonetDB (<branch>): <partial log-message>...'. > > In short I vote for switching the order of <log-message> and <branch> in the subject. I had that first, but it looked somewhat awkward. I have no problems with changing it back though. Other people with an opinion on this one? Also, there is a X-Hg-Branch header in the mail, in case you want to easily filter them.

1 0

[Monetdb-developers] Move of SourceForge bugtracker to MonetDB Bugzilla
by Fabian Groffen 07 May '10

07 May '10

This message is important for all that used to report bugs on our SourceForge tracker, or are planning to report bugs in the future. With the move of our CVS repository to a new Mercurial repository, we have also migrated the bugtracker from SourceForge to a Bugzilla instance. The old SourceForge trackers are no longer in use. Please DO NOT file any bugs there, as they will get lost. We have removed their presence as far as possible to avoid any confusion here. The new location to search, read and file new bugs is here: http://dev.monetdb.org/bugzilla/ (Please bear with us while we try to get the interface/skin well suited for our project.) All bugs that were previously on the SourceForge trackers have been migrated to the new Bugzilla instance. The bug numbers could not be retained, hence all have been renumbered. See [1] for dealing with this in Bugzilla. All users that have ever reported a bug, commented on a bug, or been an assignee for a bug on SourceForge have been imported in Bugzilla. This means that for all those users a <sf-nickname>@users.sourceforge.net account has been created. Exception to this are users who work at CWI, whose addresses have been mapped to their CWI address. If you want to use your migrated account, simply click the "Forgot Password" link on the Bugzilla main page. It will send an email to your (SourceForge) email address with a token to allow you to set a password. If you are a new user, you can simply sign up on our Bugzilla using the "Open a New Account" button. In both cases, make sure you look through the preferences, in particular the "Email Preferences". For migrated accounts these preferences are set to exclude sending any bug email, which may or may not be what you prefer. Feel free to ask for any help if problems arise. [1] Searching an old bug using its bugnumber: If you have an old SourceForge bug number and need to find tha associated bug in Bugzilla, open up an advanced search, and enter the old bug number in the URL field. Make sure that you select the appropriate bug resolutions (e.g. include RESOLVED) and search. Each migrated bug has an URL set to the original bug on SourceForge, hence is easily found this way.

4 6

[Monetdb-developers] New Jun2010 stable branch created
by Sjoerd Mullender 07 May '10

07 May '10

I have created new stable branches for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. All packages use the same name for the branch: Jun2010. So in each of your stable checkout directories you can do hg update -rJun2010 to update to this new branch. -- Sjoerd Mullender

2 1

[Monetdb-developers] MonetDB testing temporarily suspended
by Stefan Manegold 06 May '10

06 May '10

Dear fellow developers, it appears that the directory structure used by MonetDB testing (read: the "TestTools"), in particular the setup that adminstrative data and testing results (incl. archives of the last 30 days) end up "polluting" the source tree, is not suitable for Mercurial --- at least the not totally up-to-date Mercurial 1.1 on our testing server that is still running Fedora 8 cannot cope well with the lots of extra "unknown" files and directories in the source tree. I have not yet tested, whether version of Mercurial (then running from another machine via NFS) would cope better. I guess, it's finally time to revise the design of TestTools --- at least the directory structure it uses --- after all, the priciple design is still from the last century. I will start doing so (later) today, but cannot promise that I'll finish the liberation from this legacy today. Hence, MonetDB testing (which did not run for the given reasons this night) will remain suspended until I finish the re-design and implementaion of a revised directory structure (or at least a temporary work around. I'll keep you posted. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 2

Re: [Monetdb-developers] SQL main BAT fallback creates collision
by Isidor Zeuner 05 May '10

05 May '10

> > Where can I find the new bug tracker? > > We're still working on it. We aim to get it online sometime today. > Thanks. The new tracker is working fine. > > By slightly changing the test submitted with bug #2995671, I was able > > to trigger an unrelated bug (see the attached test file .. > > I have attached a patch (no-fallback-ibid.diff) which avoids this > > problem by disabling the iBAT fallback, creating an empty main BAT > > instead. I have submitted the issue on the new tracker ([1]). I slightly changed the patch so the empty main BAT is not saved to bat->bid, as this caused inserts done inside a transaction not to be committed. Is there some documentation on the meaning of 0 == bat->bid with respect to the stored data? > > > > To improve the subset of possible interactions between components > > which is tested, have you considered a fuzz-testing-like approach? > > Best regards, Isidor [1] http://bugs.monetdb.org/show_bug.cgi?id=2529

1 0

[Monetdb-developers] IMPORTANT: move of MonetDB repository imminent
by Sjoerd Mullender 04 May '10

04 May '10

On Monday, May 3, 2010, we are going to move the MonetDB repository over from the Sourceforge-hosted CVS repository to a Mercurial repository hosted on dev.monetdb.org. On March 18 I sent a pre announcement to the monetdb-developers list (IMPORTANT: changes are afoot). [1] In particular, the following CVS modules will be converted and moved: MonetDB, MonetDB4, MonetDB5, buildtools, clients, geom, java, pathfinder, sql, template, testing. The other CVS modules that are hosted at Sourceforge (pf-haskell, pf-tools, xml) will remain there, at least for now. This means that as of Monday, the affected modules in the CVS repository will be made read only (it will remain readable for a long time, just not writable). When the move is complete, the repository will open at the following location: http://dev.monetdb.org/hg/MonetDB/ This URL can be used to browse the repository using a web browser, and it can be used as the URL to make a Mercurial clone (hg clone http://dev.monetdb.org/hg/MonetDB/) This will be a read-only version of the repository. Core developers will get read-write access at the URL ssh://hg@dev.monetdb.org/MonetDB/ I wrote a document to help with the conversion to Mercurial, both describing the way to use Mercurial, and how to convert your CVS working set to a Mercurial clone. For now (until we find a better place), this document can be accessed at: http://homepages.cwi.nl/~sjoerd/downloads/monetdb/MonetDB-Mercurial.html http://homepages.cwi.nl/~sjoerd/downloads/monetdb/MonetDB-Mercurial.pdf As part of the move, I will subscribe subscribers to the various monetdb checkin mailing lists to the new mailing list checkin-list(a)monetdb.org. If you are subscribed to one or more of monetdb-checkins(a)lists.sourceforge.net monetdb-sql-checkins(a)lists.sourceforge.net monetdb-pf-checkins(a)lists.sourcefoge.net I will subscribe you to the list checkin-list(a)monetdb.org. This I will do on Monday as well, so you should get a message to that effect on Monday. [1] http://sourceforge.net/mailarchive/forum.php?thread_name=4BA239BF.4090307%4… -- Sjoerd Mullender

3 2

[Monetdb-developers] bug report on trigger statement
by Huabing 03 May '10

03 May '10

I run following sql statements on MonetDB server v5.18.5 (32-bit), based on kernel v1.36.5 (32-bit oids) create table a (a int, b int); create table ta(a int, b int); insert into ta values(1,2); insert into ta values(3,4); insert into ta values(5,6); create trigger d_ta after delete on ta referencing old row as o for each row insert into a values(o.a, o.b); delete from ta where a=1; select * from a; drop trigger d_ta; drop table ta; drop table a; The output is: +---+-----+ | a | b | +==+==+ | 3 | 4 | | 3 | 4 | | 5 | 6 | | 5 | 6 | +---+----+ Obviously, the trigger functions wrong.

1 0

[Monetdb-developers] regarding clustering in monetdb
by Puneet Birwa 02 May '10

02 May '10

hello, i have the MonetDB-Nov2009-SuperBall-SP2.tar.bz2 file after extracting this i install the monetdb . i want to know whether the above version of monetdb which i an using, uses clustering for doing join operations. if no please tell me upto what version of monetdb clustering is used for doing join. please reply as soon as possible. thank you

1 0