- developers-list - monetdb.org

[Monetdb-developers] Testing code
by Charalampos Nikolaou 11 Jan '12

11 Jan '12

Hi, I have written some code and now I would like to make changes and still be sure that the new changes are correct. So, I have written some scenarios in mal and put it in a Tests directory. I have read the README file under testing/ directory, but still haven't understood how I could generate the <scenario>.stable.out file. How could that file be generated? Even if I run Mtest.py <mytest>, then Mapprove.py the previously "wrong" output, and then again Mtest.py the test fails. Thanks, Babis

3 3

Re: [Monetdb-developers] MonetDB: default - Fix for bug 2965
by Sjoerd Mullender 05 Jan '12

05 Jan '12

Shouldn't this have gone to the Dec2011 branch? On 2012-01-04 22:30, Martin Kersten wrote: > Changeset: fa4034576f02 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fa4034576f02 > Modified Files: > monetdb5/optimizer/opt_joinpath.mx > Branch: default > Log Message: > > Fix for bug 2965 > Typechecking should be included as extra defense line. > Furthermore, we need to remove reference counts during > error handling. > > > diffs (74 lines): > > diff --git a/monetdb5/optimizer/opt_joinpath.mx b/monetdb5/optimizer/opt_joinpath.mx > --- a/monetdb5/optimizer/opt_joinpath.mx > +++ b/monetdb5/optimizer/opt_joinpath.mx > @@ -580,7 +580,7 @@ ALGjoinPathBody(Client cntxt, int top, B > * BEWARE. you may not use a size estimation, because it > * may fire a BATproperty check in a few cases. > * In case a join fails, we may try another order first before > - * abandoning the task. > + * abandoning the task. It can handle cases where a Cartesian product emerges. > * > * A left-join sequence only requires the result to be sorted > * against the first operand. For all others operand pairs, the cheapest join suffice. > @@ -599,12 +599,20 @@ ALGjoinPathBody(Client cntxt, int top, B > b = BATsemijoin(joins[j], joins[j + 1]); > } > if (b==NULL){ > + if ( postpone[j] && postpone[j+1]){ > + for( --top; top>=0; top--) > + BBPreleaseref(joins[top]->batCacheid); > + GDKfree(postpone); > + return NULL; > + } > postpone[j] = TRUE; > postpone[j+1] = TRUE; > postponed = 0; > for( k=0; k<top; k++) > postponed += postpone[k]== TRUE; > if ( postponed == top){ > + for( --top; top>=0; top--) > + BBPreleaseref(joins[top]->batCacheid); > GDKfree(postpone); > return NULL; > } > @@ -624,6 +632,12 @@ ALGjoinPathBody(Client cntxt, int top, B > mnstr_printf(cntxt->fdout, "#joinPath %d:= join(%d,%d)\n", > b->batCacheid, joins[j]->batCacheid, joins[j + 1]->batCacheid); > > + if ( b == 0 ){ > + for( --top; top>=0; top--) > + BBPreleaseref(joins[top]->batCacheid); > + GDKfree(postpone); > + return 0; > + } > BBPdecref(joins[j]->batCacheid, FALSE); > BBPdecref(joins[j+1]->batCacheid, FALSE); > joins[j] = b; > @@ -645,17 +659,25 @@ ALGjoinPath(Client cntxt, MalBlkPtr mb, > BAT *b, **joins = (BAT**)GDKmalloc(pci->argc*sizeof(BAT*)); > str joinPathRef = putName("joinPath",8); > str leftjoinPathRef = putName("leftjoinPath",12); > + int error = 0; > > if ( joins == NULL) > throw(MAL, "algebra.joinPath", MAL_MALLOC_FAIL); > (void)mb; > for (i = pci->retc; i < pci->argc; i++) { > bid = (int *) getArgReference(stk, pci, i); > - if ((b = BATdescriptor(*bid)) == NULL) { > + b = BATdescriptor(*bid); > + if ( b && top ) { > + if (TYPEerror(joins[top-1]->ttype, b->htype) ) { > + b= NULL; > + error = 1; > + } > + } > + if ( b == NULL) { > for( --top; top>=0; top--) > BBPreleaseref(joins[top]->batCacheid); > GDKfree(joins); > - throw(MAL, "algebra.joinPath", INTERNAL_BAT_ACCESS); > + throw(MAL, "algebra.joinPath", error? SEMANTIC_TYPE_MISMATCH: INTERNAL_BAT_ACCESS); > } > joins[top++] = b; > } > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

1 0

[Monetdb-developers] group by on BATs
by Charalampos Nikolaou 04 Jan '12

04 Jan '12

Hi, I have three BATs and I would like to create groups of pairs from the first two BATs and take the bitwise and for the tail values of each group according to the third BAT. I saw that there is a module for grouping operations in monetdb5/modules/kernel/group.mx. Is it the right one for my purpose? For the case of bitwise and is there any function that I could use or I should write my own like for example the GRPaggr_count() function in that file. Thanks, Babis

1 1

[Monetdb-developers] /usr/local/bin/mserver5: error while loading shared libraries: libmonetdb5.so.8: cannot open shared object file: No such file or directory
by Mark Kerzner 04 Jan '12

04 Jan '12

Hi, sorry to bother, but in building Dec2011 branch, I get the libmonetdb5.so.8 not found error, and in building the default branch, I get a crash. Both errors are attached. Thank you. Sincerely, Mark

2 1

[Monetdb-developers] MX files
by David Pennell 02 Jan '12

02 Jan '12

Where can I learn more about mx files? In particular, in the following code, it looks like @2 is derived from the type expands this into a multitude of type specific functions. I can't figure out what @3 is. Is there a list of these? SORTfndlast_@2(BAT *b, ptr v) { BUN diff, end = BUNlast(b), cur = end; BUN lo = BUNfirst(b), hi = end; int cmp = 1; BATiter bi = bat_iterator(b); if (lo >= hi || @3_CMP(BUNt@1(bi, cur - 1), v, @4)<=0) { /* shortcut: if BAT is empty or last (and hence all) tail * value is <= v, we're done. */ return cur; } while (lo < hi) { BUN mid = (lo + hi) >> 1; cur = mid; cmp = @3_CMP(BUNt@1(bi, cur), v, @4); if (cmp < 0) { lo = ++mid; cur++; } else if (cmp > 0) { hi = mid; } else { break; } } if (cmp == 0 && b->tkey == 0) { /* shift over multiple equals */ for(diff = (end-cur)>>1; diff; diff >>= 1) { while ((cur+diff) < end && @3_EQ(BUNt@1(bi, cur+diff), v, @4)) cur += diff; } } if (cmp == 0) cur++; return cur; }

2 1

[Monetdb-developers] Extending TimeStamp and Time to support 24+ hour days
by Stefan de Konink 15 Dec '11

15 Dec '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 In some fields the usage of 32 or 36 hours in a day are common. The extra time after midnight is used to indicate that a certain operation is linear in time. My particular use case is public transport. Is there any other way that using TIMESTAMP to get the 'features' of TIME, but than in a 32h or 36h day? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk7pYboACgkQYH1+F2Rqwn3CQgCfU0CCgnJgxJj0lFW9RC+qiyz+ Pj0An1Vn8oZETQTCCeIFv+LgWOAx7ERQ =wSgl -----END PGP SIGNATURE-----

2 1

[Monetdb-developers] monetdbd takes too long to start
by Charalampos Nikolaou 12 Dec '11

12 Dec '11

Hi, in Ubuntu 10.04 (as far as I remember it happens in other releases too) monetdbd takes about 10 seconds to start as opposed to mac os x 10.6 and 10.7 in which it starts instantaneously. Inspecting connections.c, the delay is due to the call on gethostbyaddr(). Reading the respective man page, getting the hostname involves a number of steps ranging from looking at /etc/hosts.conf, /etc/hosts, as well as consulting named for resolving the ip address. What's more, the man page states explicitly that gethostbyaddr() is obsolete and suggests the use of getnameinfo() instead. Correspondingly, the man page in os x recommends the use of getnameinfo() as well, but without stating any explanation for this. Have you investigated this issue? Do you know what causes such a delay? Is it present in other OSs too? Thanks, Babis

5 13

[Monetdb-developers] 3D gis support
by luca paganotti 09 Dec '11

09 Dec '11

Hi, I've read in the docs that one of the planned future features of monetdb will be 'support for 3D types'. - Do you mean full 3D support or 2.5D ? - Will be these 3D spatial extensions also integrated in sciQL? - Have you scheduled this feature release? Thanks in advance for any answer. Luca Paganotti

1 0

[Monetdb-developers] bat.inplace() strange exception
by Charalampos Nikolaou 07 Dec '11

07 Dec '11

Hello, considering the following interaction $ mclient -lmal -d test Welcome to mclient, the MonetDB interactive terminal (unreleased) Type \q to quit, \? for a list of available commands mal>sql.init(); mal>_mvc := sql.mvc(); mal>abat:bat[:int,:int] := bat.new(:int, :int); mal>bat.insert(abat, 1, 4); mal>bat.insert(abat, 1, 4); mal>bat.insert(abat, 1, 1); mal>bat.insert(abat, 1, 1); mal>bat.insert(abat, 0, 4); mal>bat.inplace(abat, 0, 5); MAPI = (monetdb) /tmp/.s.monetdb.50000 ACTION= read_line QUERY = bat.inplace(abat, 0, 5); ERROR = !Connection terminated Is this a bug or not? The log of merovingian is the following: 2011-12-07 13:01:51 ERR test[15564]: mserver5: ../../MonetDB/gdk/gdk_bat.mx:1917: void_inplace: Assertion `b->H->type == 0' failed. 2011-12-07 13:01:51 MSG merovingian[14343]: database 'test' (15564) was killed by signal SIGABRT However, the bat5.mx declares the following prototype: command inplace( o:bat[:any_1,:any_2], id:any_1, t:any_2) :bat[:any_1,:any_2] address BKCbun_inplace comment "inplace replace values on the given locations"; Regards, Babis

1 0

[Monetdb-developers] Problem (and possible fix!) with monetdbd.conf path as specified in Makefile.in
by Eric G 29 Nov '11

29 Nov '11

Hello list, I'm currently trying to use the OpenSUSE build service (http://build.openeuse.org) to build packages RPMs for for CentOS and RedHat distributions, and i think I've found a bug in a Makefile for MonetDB. Essentially it looks like the explicit path /etc/tmpfiles.d/monetdbd.conf is used in the Makefile located here: MonetDB-11.5.9/buildtools/conf/Makefile.in ...and the RPM build process doesn't like that. OpenSUSE essentially implicitly enables the "norootdir" option in the spec file (regardless of if it's actually present or not) because RPMs are built is special purpose-built VMs that are provisioned when a "job" to build an RPM is submitted, and the build process doesn't run as root. See here and here for more discussion on this problem (it's a common problem apparently): http://wiki.sipfoundry.org/display/sipXecs/Building+RPM+tips http://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#Permission_de… I think the "fix" is as simple as inserting "$(prefix)" into the line in Makefile.in, so that the build path is used to create the temporary folder instead of attempting to writing to the system-wide /etc folder (the build fails on the Makefile's install phase with permission denied errors because of this static path). Am I on to something here? If I'm right, can the Makefile.in file be patched please, adding $(prefix)? -- Thanks, Eric

3 3