- developers-list - monetdb.org

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_const.mx, , NONE, 1.1 Makefile.ag, , 1.89, 1.90 serialize_dflt.mx, , 1.49, 1.50 xrpc_client.mx, , 1.49, 1.50 xrpc_common.mx, , 1.6, 1.7 xrpc_server.mx, , 1.80, 1.81
by Stefan Manegold 17 Jul '08

17 Jul '08

On Thu, Jul 17, 2008 at 04:42:42PM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1165 > > Modified Files: > Makefile.ag serialize_dflt.mx xrpc_client.mx xrpc_common.mx > xrpc_server.mx > Added Files: > xrpc_const.mx > Log Message: > Since xrpc_server and xrpc_client are not independent of each other (the > server needs the client to handle nested xrpc queries), better put them > into one lib. This way, we also get rid of the problem that > xrpc_common.mx is included in two libs. The "xrpc_server" & "xrpc_client" used to be not only "libs" but also *modules*. Replacing two modules xrpc_server & xrpc_client by one module "xrpc" requires some more work: ======== $ Mserver --dbinit='module(pathfinder);' !WARNING: GDKlockHome: created directory /ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/var/MonetDB4/dbfarm/demo/ !WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !WARNING: BBPdir: initializing BBP. # MonetDB Server v4.25.0 # based on GDK v1.25.0 # Copyright (c) 1993-2008, CWI. All rights reserved. # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: TBL_openmod: DL_open: library not found. !ERROR: TBL_loadmod: module load _xrpc_server cancelled. !ERROR: moduleClient: module(_xrpc_server) load error. !ERROR: TBL_openmod: DL_open: library not found. !ERROR: TBL_loadmod: module load _xrpc_client cancelled. !ERROR: moduleClient: module(_xrpc_client) load error. # PF/Tijah module v0.5.0 loaded. http://dbappl.cs.utwente.nl/pftijah !ERROR: interpret_params: append(param 1): invalid BAT. MonetDB> ======== Stefan > Also separate xrpc constant strings into a new header file, so that > serialize_dflt kan include it. > > > > U xrpc_common.mx > Index: xrpc_common.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_common.mx,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- xrpc_common.mx 17 Jul 2008 14:56:19 -0000 1.6 > +++ xrpc_common.mx 17 Jul 2008 16:42:38 -0000 1.7 > @@ -24,134 +24,12 @@ > @' $Id$ > @f xrpc_common > @a Ying Zhang > -@t Includes header files, definitions shared by the XRPC server, XRPC client and serializer > +@t Includes struct and functions shared by the XRPC server and XRPC client > > @h > #ifndef XRPC_COMMON_H > #define XRPC_COMMON_H > > -/* HTTP defines */ > -#define XRPC_DOC_CALLBACK "/xrpc/doc" > -#define XRPC_ADM_CALLBACK "/xrpc/admin" > -#define XRPC_WSAT_CALLBACK "/xrpc/wsat" > -#define XRPC_REQ_CALLBACK "/xrpc" > - > -#define ERR403 "403 Forbidden" > -#define ERR404 "404 Bad Request" > -#define ERR408 "408 Request Timeout" > -#define ERR500 "500 Internal Server Error" > -#define ERR504 "504 Gateway Timeout" > - > -#define OUT_OF_MEM "Internal Receiver Error: out-of memory" > -#define NOT_WELL_FORMED "Request XML message not well-formed" > -#define MAX_NR_PARAMS 1024 > - > -#define HTTP_200_OK\ > - "HTTP/1.1 200 OK\r\n"\ > - "Content-type: text/xml; charset=\"utf-8\"\r\n\r\n" > - > -/* Namespaces */ > -#define MXQ_ADMIN "http://monetdb.cwi.nl/XQuery/admin/" > -#define SOAP_NS "http://www.w3.org/2003/05/soap-envelope" > -#define XDT_NS "http://www.w3.org/2005/xpath-datatypes" > -#define XS_NS "http://www.w3.org/2001/XMLSchema" > -#define XSI_NS "http://www.w3.org/2001/XMLSchema-instance" > -#define XRPC_NS "http://monetdb.cwi.nl/XQuery" > -#define XRPC_LOC "http://monetdb.cwi.nl/XQuery/XRPC.xsd" > -#define WSCOOR_NS "http://docs.oasis-open.org/ws-tx/wscoor/2006/06" > -#define WSAT_NS "http://docs.oasis-open.org/ws-tx/wsat/2006/06" > - > -#define XRPC_REQ_QN XRPC_NS"|request" > -#define XRPC_RES_QN XRPC_NS"|response" > - > -/* XRPC SOAP snippets */ > -#define SOAP_ENVELOPE\ > - "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"\ > - "<env:Envelope\n"\ > - " xmlns:env=\"" SOAP_NS "\"\n"\ > - " xmlns:xrpc=\"" XRPC_NS "\"\n"\ > - " xmlns:xdt=\"" XDT_NS "\" \n"\ > - " xmlns:xs=\"" XS_NS "\"\n"\ > - " xmlns:xsi=\"" XSI_NS "\"\n"\ > - " xsi:schemaLocation=\"" XRPC_NS " " XRPC_LOC "\">\n" > - > -#define XRPC_WS_QID\ > - " <xrpc:header xrpc:mode=\"%s\">\n"\ > - " <wscoor:CoordinationContext\n"\ > - " xmlns:wscoor=\""WSCOOR_NS"\"\n"\ > - " env:mustUnderstand=\"true\">\n"\ > - " <wscoor:Identifier>%s</wscoor:Identifier>\n"\ > - " <wscoor:Expires>"LLFMT"</wscoor:Expires>\n"\ > - " <wscoor:CoordinationType>" WSAT_NS "</wscoor:CoordinationType>\n"\ > - " </wscoor:CoordinationContext>\n"\ > - " </xrpc:header>\n" > - > -#define XRPC_REQ_BODY_OPEN\ > - "<env:Body>\n"\ > - " <xrpc:request\n"\ > - " xrpc:module=\"%s\"\n"\ > - " xrpc:location=\"%s\"\n"\ > - " xrpc:method=\"%s\"\n"\ > - " xrpc:arity=\""LLFMT"\"\n"\ > - " xrpc:iter-count=\""LLFMT"\"\n"\ > - " xrpc:updCall=\"%s\"\n"\ > - " xrpc:caller=\"" > -#define XRPC_REQ_BODY\ > - XRPC_REQ_BODY_OPEN "%s:"LLFMT"\">\n" > -#define XRPC_REQ_BODY_FIRST_CALLER\ > - XRPC_REQ_BODY_OPEN "query\">\n" > - > -#define XRPC_WSAT_REQ SOAP_ENVELOPE\ > - "<env:Header>"\ > - XRPC_WS_QID \ > - "</env:Header>\n"\ > - "<env:Body><wsat:%s/></env:Body>\n"\ > - "</env:Envelope>" > - > -#define XRPC_WSAT_RES SOAP_ENVELOPE\ > - "<env:Header>\n"\ > - XRPC_WS_QID \ > - " <xrpc:partipant>%s,%s:%s:%s,"LLFMT",%d</xrpc:participant>\n"\ > - "</env:Header>\n"\ > - "<env:Body><wsat:%s/></env:Body>\n"\ > - "</env:Envelope>" > - > -#define XRPC_RES_BODY\ > - "<env:Body>\n"\ > - " <xrpc:response\n"\ > - " xrpc:module=\"%s\"\n"\ > - " xrpc:method=\"%s\">\n" > - > -#define XRPC_HTTP_CALL\ > - " <xrpc:call>"\ > - " <xrpc:sequence>"\ > - " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > - " </xrpc:sequence>"\ > - " </xrpc:call>" > - > -#define XRPC_PUT_CALL\ > - " <xrpc:call>"\ > - " <xrpc:sequence>"\ > - " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > - " </xrpc:sequence>"\ > - " <xrpc:sequence>"\ > - " <xrpc:element>%s</xrpc:element>"\ > - " </xrpc:sequence>"\ > - " </xrpc:call>" > - > -#define XRPC_REQ_FOOTER\ > - " </xrpc:request>\n"\ > - "</env:Body>\n"\ > - "</env:Envelope>" > - > -#ifndef HOST_NAME_MAX > -#ifdef _POSIX_HOST_NAME_MAX > -#define HOST_NAME_MAX _POSIX_HOST_NAME_MAX > -#else > -#define HOST_NAME_MAX 255 > -#endif > -#endif > - > typedef struct { > char *qid; > char *caller; > @@ -188,6 +66,7 @@ > #include "pf_config.h" > #include "pathfinder.h" > #include "xrpc_common.h" > +#include "xrpc_const.h" > > XRPCreq_t * > XRPCreq_new( > > U xrpc_client.mx > Index: xrpc_client.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_client.mx,v > retrieving revision 1.49 > retrieving revision 1.50 > diff -u -d -r1.49 -r1.50 > --- xrpc_client.mx 17 Jul 2008 14:56:19 -0000 1.49 > +++ xrpc_client.mx 17 Jul 2008 16:42:38 -0000 1.50 > @@ -398,6 +398,7 @@ > #include "shredder.h" > #include "serialize.h" > #include "xrpc_common.h" > +#include "xrpc_const.h" > #include "xrpc_client.h" > > int > > U serialize_dflt.mx > Index: serialize_dflt.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/serialize_dflt.mx,v > retrieving revision 1.49 > retrieving revision 1.50 > diff -u -d -r1.49 -r1.50 > --- serialize_dflt.mx 17 Jul 2008 14:56:19 -0000 1.49 > +++ serialize_dflt.mx 17 Jul 2008 16:42:38 -0000 1.50 > @@ -41,7 +41,7 @@ > #include "serialize_null.h" > #include "pathfinder.h" > #include "pf_support.h" > -#include "xrpc_common.h" /* *_NS defs */ > +#include "xrpc_const.h" > > /* a lot of characters, static strings, and their sizes > (the idea is to reuse constant character pointers during > > U Makefile.ag > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/Makefile.ag,v > retrieving revision 1.89 > retrieving revision 1.90 > diff -u -d -r1.89 -r1.90 > --- Makefile.ag 15 Jul 2008 14:14:15 -0000 1.89 > +++ Makefile.ag 17 Jul 2008 16:42:38 -0000 1.90 > @@ -49,11 +49,6 @@ > ll_sibling.mx > } > > -lib_xrpc_common = { > - NOINST > - SOURCES = xrpc_common.mx > -} > - > lib__pf_support = { > DIR = libdir/MonetDB4 > SOURCES = \ > @@ -103,25 +98,16 @@ > $(MONETDB4_MODS) -l_logger -l_streams -l_builtin -l_ascii_io -l_algebra -l_sys -l_constant -l_mapi > } > > -lib__xrpc_client = { > - DIR = libdir/MonetDB4 > - SOURCES = xrpc_client.mx > - LIBS = \ > - libserialize libxrpc_common \ > - ./lib_pf_support ./lib_pathfinder \ > - $(PF_LIBS) $(SOCKET_LIBS) \ > - $(MONETDB_LIBS) -lbat -lstream $(MONETDB4_LIBS) -lmonet \ > - $(MONETDB4_MODS) -l_streams -l_builtin -l_ascii_io > -} > - > -lib__xrpc_server = { > +lib__xrpc = { > DIR = libdir/MonetDB4 > SOURCES = \ > + xrpc_const.mx \ > + xrpc_common.mx \ > shttpd.c shttpd.h \ > - xrpc_server.mx > + xrpc_server.mx \ > + xrpc_client.mx > LIBS = \ > - libxrpc_common \ > - ./lib_pf_support ./lib_pathfinder \ > + libserialize ./lib_pf_support ./lib_pathfinder \ > $(PF_LIBS) $(SOCKET_LIBS) \ > $(MONETDB_LIBS) -lbat -lstream $(MONETDB4_LIBS) -lmonet $(PTHREAD_LIBS) \ > $(MONETDB4_MODS) -l_mapi -l_streams -l_builtin -l_ascii_io > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.80 > retrieving revision 1.81 > diff -u -d -r1.80 -r1.81 > --- xrpc_server.mx 17 Jul 2008 15:54:23 -0000 1.80 > +++ xrpc_server.mx 17 Jul 2008 16:42:39 -0000 1.81 > @@ -177,6 +177,7 @@ > #include "serialize.h" > #include "shttpd.h" > #include "xrpc_common.h" > +#include "xrpc_const.h" > > static int rpcd_running = 0; > static int timing = 0; > > --- NEW FILE: xrpc_const.mx --- > @' Copyright Notice: > @' ----------------- > @' > @' The contents of this file are subject to the Pathfinder Public License > @' Version 1.1 (the "License"); you may not use this file except in > @' compliance with the License. You may obtain a copy of the License at > @' http://monetdb.cwi.nl/Legal/PathfinderLicense-1.1.html > @' > @' Software distributed under the License is distributed on an "AS IS" > @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See > @' the License for the specific language governing rights and limitations > @' under the License. > @' > @' The Original Code is the Pathfinder system. > @' > @' The Original Code has initially been developed by the Database & > @' Information Systems Group at the University of Konstanz, Germany and > @' is now maintained by the Database Systems Group at the Technische > @' Universitaet Muenchen, Germany. Portions created by the University of > @' Konstanz and the Technische Universitaet Muenchen are Copyright (C) > @' 2000-2005 University of Konstanz and (C) 2005-2008 Technische > @' Universitaet Muenchen, respectively. All Rights Reserved. > @' > @' $Id: xrpc_const.mx,v 1.1 2008/07/17 16:42:39 yingying Exp $ > @f xrpc_const > @a Ying Zhang > @t Defines XRPC string constants > > @h > #ifndef XRPC_CONST_H > #define XRPC_CONST_H > > /* HTTP defines */ > #define XRPC_DOC_CALLBACK "/xrpc/doc" > #define XRPC_ADM_CALLBACK "/xrpc/admin" > #define XRPC_WSAT_CALLBACK "/xrpc/wsat" > #define XRPC_REQ_CALLBACK "/xrpc" > > #define ERR403 "403 Forbidden" > #define ERR404 "404 Bad Request" > #define ERR408 "408 Request Timeout" > #define ERR500 "500 Internal Server Error" > #define ERR504 "504 Gateway Timeout" > > #define OUT_OF_MEM "Internal Receiver Error: out-of memory" > #define NOT_WELL_FORMED "Request XML message not well-formed" > #define MAX_NR_PARAMS 1024 > > #define HTTP_200_OK\ > "HTTP/1.1 200 OK\r\n"\ > "Content-type: text/xml; charset=\"utf-8\"\r\n\r\n" > > /* Namespaces */ > #define MXQ_ADMIN "http://monetdb.cwi.nl/XQuery/admin/" > #define SOAP_NS "http://www.w3.org/2003/05/soap-envelope" > #define XDT_NS "http://www.w3.org/2005/xpath-datatypes" > #define XS_NS "http://www.w3.org/2001/XMLSchema" > #define XSI_NS "http://www.w3.org/2001/XMLSchema-instance" > #define XRPC_NS "http://monetdb.cwi.nl/XQuery" > #define XRPC_LOC "http://monetdb.cwi.nl/XQuery/XRPC.xsd" > #define WSCOOR_NS "http://docs.oasis-open.org/ws-tx/wscoor/2006/06" > #define WSAT_NS "http://docs.oasis-open.org/ws-tx/wsat/2006/06" > > #define XRPC_REQ_QN XRPC_NS"|request" > #define XRPC_RES_QN XRPC_NS"|response" > > /* XRPC SOAP snippets */ > #define SOAP_ENVELOPE\ > "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"\ > "<env:Envelope\n"\ > " xmlns:env=\"" SOAP_NS "\"\n"\ > " xmlns:xrpc=\"" XRPC_NS "\"\n"\ > " xmlns:xdt=\"" XDT_NS "\" \n"\ > " xmlns:xs=\"" XS_NS "\"\n"\ > " xmlns:xsi=\"" XSI_NS "\"\n"\ > " xsi:schemaLocation=\"" XRPC_NS " " XRPC_LOC "\">\n" > > #define XRPC_WS_QID\ > " <xrpc:header xrpc:mode=\"%s\">\n"\ > " <wscoor:CoordinationContext\n"\ > " xmlns:wscoor=\""WSCOOR_NS"\"\n"\ > " env:mustUnderstand=\"true\">\n"\ > " <wscoor:Identifier>%s</wscoor:Identifier>\n"\ > " <wscoor:Expires>"LLFMT"</wscoor:Expires>\n"\ > " <wscoor:CoordinationType>" WSAT_NS "</wscoor:CoordinationType>\n"\ > " </wscoor:CoordinationContext>\n"\ > " </xrpc:header>\n" > > #define XRPC_REQ_BODY_OPEN\ > "<env:Body>\n"\ > " <xrpc:request\n"\ > " xrpc:module=\"%s\"\n"\ > " xrpc:location=\"%s\"\n"\ > " xrpc:method=\"%s\"\n"\ > " xrpc:arity=\""LLFMT"\"\n"\ > " xrpc:iter-count=\""LLFMT"\"\n"\ > " xrpc:updCall=\"%s\"\n"\ > " xrpc:caller=\"" > #define XRPC_REQ_BODY\ > XRPC_REQ_BODY_OPEN "%s:"LLFMT"\">\n" > #define XRPC_REQ_BODY_FIRST_CALLER\ > XRPC_REQ_BODY_OPEN "query\">\n" > > #define XRPC_WSAT_REQ SOAP_ENVELOPE\ > "<env:Header>"\ > XRPC_WS_QID \ > "</env:Header>\n"\ > "<env:Body><wsat:%s/></env:Body>\n"\ > "</env:Envelope>" > > #define XRPC_WSAT_RES SOAP_ENVELOPE\ > "<env:Header>\n"\ > XRPC_WS_QID \ > " <xrpc:partipant>%s,%s:%s:%s,"LLFMT",%d</xrpc:participant>\n"\ > "</env:Header>\n"\ > "<env:Body><wsat:%s/></env:Body>\n"\ > "</env:Envelope>" > > #define XRPC_RES_BODY\ > "<env:Body>\n"\ > " <xrpc:response\n"\ > " xrpc:module=\"%s\"\n"\ > " xrpc:method=\"%s\">\n" > > #define XRPC_HTTP_CALL\ > " <xrpc:call>"\ > " <xrpc:sequence>"\ > " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > " </xrpc:sequence>"\ > " </xrpc:call>" > > #define XRPC_PUT_CALL\ > " <xrpc:call>"\ > " <xrpc:sequence>"\ > " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > " </xrpc:sequence>"\ > " <xrpc:sequence>"\ > " <xrpc:element>%s</xrpc:element>"\ > " </xrpc:sequence>"\ > " </xrpc:call>" > > #define XRPC_REQ_FOOTER\ > " </xrpc:request>\n"\ > "</env:Body>\n"\ > "</env:Envelope>" > > #ifndef HOST_NAME_MAX > #ifdef _POSIX_HOST_NAME_MAX > #define HOST_NAME_MAX _POSIX_HOST_NAME_MAX > #else > #define HOST_NAME_MAX 255 > #endif > #endif > > #endif /* XRPC_CONST_H */ > > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/common stream.mx, , 1.166, 1.167
by Ying Zhang 17 Jul '08

17 Jul '08

On Jul 15, 2008, at 08:58 , Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/common > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8300 > > Modified Files: > stream.mx > Log Message: > hopefully fixing the mapi hangs on macos Hi Niels, Unfortunately, the hanging problem still exists, on Mac OS 10.5 By the way, 'mclient -lmil' doesn't hang like 'mclient -lx'. I don't know how 'mclient -lsql' behaves. Regards, Jennie

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_common.mx, , 1.4, 1.5 xrpc_server.mx, , 1.77, 1.78
by Stefan Manegold 17 Jul '08

17 Jul '08

Jennie, in fact, only xprc_admin and xrpc_trusted are currently(?) used/required as global C variable in xrpc_server.c , all other BATs are indeed share as (global) MIL variables and/or via the BAT names. Hence, the attached patch would IMHO clean-up the code even more. (I.e., reduce use of / need for global variables as much as possible.) Stefan On Thu, Jul 17, 2008 at 10:51:38AM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24940 > > Modified Files: > xrpc_common.mx xrpc_server.mx > Log Message: > Try to fix bug > > [ 2020269 ] PF/runtime fails to link on Darwin (MacOS X) > > remove declaratin of BATs xrpc_* from xrpc_common.mx > xrpc_trusted and xrpc_admin are not shared BATs (only used by > xrpc_server), the others are shared via MIL, so should no be (re)defined > in xrpc_common.mx > > > > U xrpc_common.mx > Index: xrpc_common.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_common.mx,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- xrpc_common.mx 15 Jul 2008 14:14:21 -0000 1.4 > +++ xrpc_common.mx 17 Jul 2008 10:51:33 -0000 1.5 > @@ -182,14 +182,6 @@ > XRPCreq_t *xrpc_parse_message(char *err, int errlen, BAT *shredBAT, > BAT *participants); > > -BAT *xrpc_qids; > -BAT *xrpc_timeouts; > -BAT *xrpc_statuses; > -BAT *xrpc_locks; > -BAT *xrpc_wsbats; > -BAT *xrpc_trusted; > -BAT *xrpc_admin; > - > #endif /* XRPC_COMMON_H */ > > @c > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.77 > retrieving revision 1.78 > diff -u -d -r1.77 -r1.78 > --- xrpc_server.mx 15 Jul 2008 14:14:23 -0000 1.77 > +++ xrpc_server.mx 17 Jul 2008 10:51:33 -0000 1.78 > @@ -194,6 +194,12 @@ > static MT_Sema xrpc_commit_sema; > static xrpc_commit_t *xrpc_commit_active = NULL; > > +/* shared BATs */ > +BAT *xrpc_qids = NULL, *xrpc_statuses = NULL, *xrpc_timeouts = NULL; > +BAT *xrpc_locks = NULL, *xrpc_wsbats = NULL; > +/* only used by xrpc_server */ > +BAT *xrpc_trusted = NULL, *xrpc_admin = NULL, *xrpc_user = NULL; > + > int CMDmy_hostname(char **res) > { > int ret = 0, len = HOST_NAME_MAX > 255 ? HOST_NAME_MAX : 255; > @@ -871,9 +877,6 @@ > BATseqbase(xrpc_@1, 1); > BBPrename(xrpc_@1->batCacheid, "xrpc_@1"); > @c > -BAT *xrpc_qids = NULL, *xrpc_statuses = NULL, *xrpc_timeouts = NULL, *xrpc_locks = NULL, *xrpc_wsbats = NULL; > -BAT *xrpc_trusted = NULL, *xrpc_admin = NULL, *xrpc_user = NULL; > - > > bat* xrpc_prelude(void) { > @:xrpc_bat(qids,str)@ > @@ -881,6 +884,7 @@ > @:xrpc_bat(timeouts,lng)@ > @:xrpc_bat(locks,lock)@ > @:xrpc_bat(wsbats,bat)@ > + > @:xrpc_bat(trusted,str)@ > @:xrpc_bat(admin,str)@ > @:xrpc_bat(user,str)@ > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_server.mx, XQuery_0-24, 1.68.2.4, 1.68.2.5
by Sjoerd Mullender 14 Jul '08

14 Jul '08

Propagation prevented. On 2008-07-14 17:52, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv9021 > > Modified Files: > Tag: XQuery_0-24 > xrpc_server.mx > Log Message: > only in stable: fixes in dealing with whitespace characters between > elements. > > Please *do not* propagate this change > > > > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.68.2.4 > retrieving revision 1.68.2.5 > diff -u -d -r1.68.2.4 -r1.68.2.5 > --- xrpc_server.mx 4 Jun 2008 11:57:58 -0000 1.68.2.4 > +++ xrpc_server.mx 14 Jul 2008 15:52:24 -0000 1.68.2.5 > @@ -756,8 +756,11 @@ > } > argcnt[i][j]++; > nr_args++; > + next_tpe_node_pre += (pre_kindT[next_tpe_node_pre] == TEXT ? 1 : 0); > } /* end loop 'xrpc:<type>' */ > + next_seq_node_pre += (pre_kindT[next_seq_node_pre] == TEXT ? 1 : 0); > } /* end loop 'xrpc:sequence' */ > + next_call_node_pre += (pre_kindT[next_call_node_pre] == TEXT ? 1 : 0); > } /* end loop 'xrpc:call' */ > frag_root = BATsetaccess(frag_root, BAT_READ); > > > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql_gencode.mx, SQL_2-24, 1.275.2.3, 1.275.2.4
by Sjoerd Mullender 14 Jul '08

14 Jul '08

Propagation prevented. Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv16011/src/backends/monet5 > > Modified Files: > Tag: SQL_2-24 > sql_gencode.mx > Log Message: > incompatible backport of a fix. SQL functions returning tables return bat[:str,:BAT]. We now correctly return this full type vs only bat before. > > Next to this we now print errors (they are in this part of the code > unexpected). > > This should not be propagated to the main branch > > > U sql_gencode.mx > Index: sql_gencode.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/sql_gencode.mx,v > retrieving revision 1.275.2.3 > retrieving revision 1.275.2.4 > diff -u -d -r1.275.2.3 -r1.275.2.4 > --- sql_gencode.mx 17 Jun 2008 08:37:28 -0000 1.275.2.3 > +++ sql_gencode.mx 13 Jul 2008 14:43:08 -0000 1.275.2.4 > @@ -2060,7 +2060,10 @@ > return; > f->sql++; > s = scope_sqlparse(m, NULL, f->imp, m_instantiate); > - if (s) { > + if (!s) { > + fprintf(stderr, m->errstr); > + return; > + } else { > stmt *opt; > opt = optimize(m, s); > stmt_destroy(s); > @@ -2078,7 +2081,10 @@ > varSetProp(curBlk, getArg(curInstr, 0), inlineProp, op_eq, NULL); > > if (f->res.type) { > - setVarType(curBlk, 0, f->res.type->localtype); > + if (f->res.type->localtype == TYPE_bat) > + setVarType(curBlk, 0, newBatType(TYPE_str, TYPE_bat)); > + else > + setVarType(curBlk, 0, f->res.type->localtype); > setVarUDFtype(curBlk,0); > } else { > setVarType(curBlk, 0, TYPE_void); > > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] MonetDB4 TMsubcommit test
by Sjoerd Mullender 11 Jul '08

11 Jul '08

After my checkin today, the TMsubcommit test in MonetDB4 will fail. This is due to the fact that the BAT alignment information is now stored in the BBP.dir file which is printed by the test. This alignment information is a random OID, and so it can (and does) differ from run to run. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, , 1.427, 1.428
by Sjoerd Mullender 11 Jul '08

11 Jul '08

Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5919 > > Modified Files: > pathfinder.mx > Log Message: > check 'seqnr' before write into it. > 'seqnr' is only used by xrpc. > > This might be the cause of failing tests due to segfault. > > > > U pathfinder.mx > Index: pathfinder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v > retrieving revision 1.427 > retrieving revision 1.428 > diff -u -d -r1.427 -r1.428 > --- pathfinder.mx 8 Jul 2008 17:51:30 -0000 1.427 > +++ pathfinder.mx 11 Jul 2008 10:45:31 -0000 1.428 > @@ -5070,7 +5070,7 @@ > *(ctx->time_compile) = GDKusec() - usec; > > /* Done preparing the query. Time to (re-)execute the MIL tree */ > - if (xquery_tree_exec(ctx, prepfun->lt, 1)) { > + if (xquery_tree_exec(ctx, prepfun->lt, 1) && seqnr) { > *seqnr = (*ctx->xrpc_seqnr); > return NULL; > } Is this really correct? Shouldn't this be: if (xquery_tree_exec(ctx, prepfun->lt, 1)) { if (seqnr) *seqnr = (*ctx->xrpc_seqnr); return NULL; } And you really don't need those parentheses around *ctx->xrpc_seqnr. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra/opt opt_algebra_cse.c, , 1.33, 1.34
by Stefan Manegold 10 Jul '08

10 Jul '08

Hi Teggy, is this a bug fix that should also (have) be(en) applied in the XQuery_0-24 release branch? Stefan On Mon, Jul 07, 2008 at 07:19:00AM +0000, Torsten Grust wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra/opt > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv6385 > > Modified Files: > opt_algebra_cse.c > Log Message: > -- Fixed attribute list allocation bug in structural algebra CSE. > > > > U opt_algebra_cse.c > Index: opt_algebra_cse.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/opt/opt_algebra_cse.c,v > retrieving revision 1.33 > retrieving revision 1.34 > diff -u -d -r1.33 -r1.34 > --- opt_algebra_cse.c 18 Jun 2008 11:24:23 -0000 1.33 > +++ opt_algebra_cse.c 7 Jul 2008 07:18:57 -0000 1.34 > @@ -1330,21 +1330,23 @@ > ACTATT(R(n), n->sem.ser_seq.item)); > > case la_serialize_rel: > - { > - PFalg_attlist_t items = (PFalg_attlist_t) { > - .count = n->sem.ser_rel.items.count, > - .atts = PFmalloc (sizeof(PFalg_att_t)) > - }; > - > - for (unsigned int i = 0; i < items.count; i++) { > - items.atts[i] = ACTATT (L(n), n->sem.ser_rel.items.atts[i]); > - } > - > + { > + PFalg_attlist_t items = > + (PFalg_attlist_t) { > + .count = n->sem.ser_rel.items.count, > + .atts = PFmalloc (n->sem.ser_rel.items.count * > + sizeof(PFalg_att_t)) > + }; > + > + for (unsigned int i = 0; i < items.count; i++) { > + items.atts[i] = ACTATT (L(n), n->sem.ser_rel.items.atts[i]); > + } > + > return PFla_serialize_rel (CSE(L(n)), > ACTATT (L(n), n->sem.ser_rel.iter), > ACTATT (L(n), n->sem.ser_rel.pos), > items); > - } > + } > case la_lit_tbl: > return PFla_lit_tbl_ (create_attlist (n->schema), > n->sem.lit_tbl.count, > > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Not yet packages precompiled tests
by Stefan Manegold 04 Jul '08

04 Jul '08

Dear all, the MAPI & JDBC tests implemented in C and Java in MonetDB/Clients and MonetDB/Java (installed in <prefix>/lib/MonetDB/Tests/ & <prefix>/share/MonetDB/Tests/, respectively) are not yet distributed in any binary package. For convenience of running Mtest.py on binary installations, should we concider to package the C-MAPI tests from MonetDB/Clients in, say, a MonetDB-client-testing RPM and the Java-JDBC tests from MonetDB/Java in, say, a MonetDB-java-testing.jar? Opinions and/or (other) suggestions? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] monetdb-java-config
by Stefan Manegold 04 Jul '08

04 Jul '08

Dear all, Mtest.oy and several tests (see below) (still) rely on monetdb-java-config to find out where MonetDB/Java (i.e., the *.jar's and the Test_*.class examples/tests) are installed. However, we do not distribute monetdb-java-config in any (binary) package (only with the sources). I see the followin options: 1) Leave everything as it is. 2) Keep monetdb-java-config in the source (for conveniencce with Mtest.py on source-base installations), but extend Mtest.py with command line options as an alternative manner to provide the requires information in case of binary installation (i.e., w/o monetdb-java-config). 3) Abandon monetdb-java-config completely and have Mtest.py rely only in the (to be added) command line options Any preferences and/or other suggestions? Stefan ======== ./MonetDB/src/testing/Mtest.py.in: if CheckExec('monetdb-java-config'): ./MonetDB/src/testing/Mtest.py.in: CONDITIONALS['HAVE_MONETDB_JAVAJDBC'] = GetConfig('monetdb-java-config', '--HAVE_MONETDB_JAVAJDBC') ./MonetDB/src/testing/Mtest.py.in: CONDITIONALS['HAVE_MONETDB_JAVAXRPC'] = GetConfig('monetdb-java-config', '--HAVE_MONETDB_JAVAXRPC') ./pathfinder/tests/BugTracker/Tests/JDBC_0_results.SF-1886326.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./pathfinder/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./pf_rox/tests/BugTracker/Tests/JDBC_0_results.SF-1886326.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./pf_rox/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./sql/src/jdbc/tests/Tests/Test_JdbcClient.SQL.sh:pdd="`monetdb-java-config --pkgdatadir`" ./sql/src/jdbc/tests/Tests/Test.SQL.sh:pdd="`monetdb-java-config --pkgdatadir`" ./sql/src/test/snodgrass/Tests/cast_select.SQL.sh:pdd="`monetdb-java-config --pkgdatadir`" ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

[Monetdb-developers] mjclient
by Stefan Manegold 04 Jul '08

04 Jul '08

Dear all, I was wondering whether MonetDB/Java's "mjclient" wrapper script for jdbcclient.jar is still "maintained", "supported" and/or used? We do only distribute it via MonetDB/Java's CVS sources, source tarball and the SuperSourceBall, but not in any binary package or as individual file like the *.jar's. However, we (still) use it in some tests (see below). I see the follwoing options: 1) Leave everything as it is. 2) Leave and distribute mjclient in/with the MonetDB/Java source, but make the tests independed of it by "inlining" its functionality 3) Abandon mjclient completely and make the tests independed of it by "inlining" its functionality Any preferences and/or other suggestions? Stefan ======== ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/alter_table_describe.SF-1146092.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\alter_table_describe.SF-1146092-src.sql" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/alter_table_describe.SF-1146092.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d $TSTDB -f \"$RELSRCDIR/alter_table_describe.SF-1146092-src.sql\"" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/not_null.SF-933194.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\not_null.SF-933194-src.sql" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/not_null.SF-933194.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d $TSTDB -f \"$RELSRCDIR/not_null.SF-933194-src.sql\"" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/type_dump_test.SF-989257.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\type_dump_test.SF-989257-src.sql" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/type_dump_test.SF-989257.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d $TSTDB -f \"$RELSRCDIR/type_dump_test.SF-989257-src.sql\"" ./sql/src/test/bugs/Tests/except-union-intersect-bug-sf-1146079.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\except-union-intersect-bug-sf-1146079.sql" ./sql/src/test/bugs/Tests/except-union-intersect-bug-sf-1146079.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../except-union-intersect-bug-sf-1146079.sql\"" ./sql/src/test/bugs/Tests/unicode_varchar-bug-sf-1041324_JdbcClient.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\..\unicode_varchar-bug-sf-1041324.sql" ./sql/src/test/bugs/Tests/unicode_varchar-bug-sf-1041324_JdbcClient.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/../unicode_varchar-bug-sf-1041324.sql\"" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\authorization.SF-1430616-data.sql" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -q ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\authorization.SF-1430616-drop_user.sql" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/authorization.SF-1430616-data.sql\"" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -q" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/authorization.SF-1430616-drop_user.sql\"" ./sql/src/test/BugTracker/Tests/drop_schema_crash.SF-1504794.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\drop_schema_crash.SF-1504794-data.sql" ./sql/src/test/BugTracker/Tests/drop_schema_crash.SF-1504794.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/drop_schema_crash.SF-1504794-data.sql\"" ./sql/src/test/BugTracker/Tests/explain.SF-1739353.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\explain.SF-1739353-data.sql" ./sql/src/test/BugTracker/Tests/explain.SF-1739353.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/explain.SF-1739353-data.sql\"" ./sql/src/test/BugTracker/Tests/jdbc_no_debug.SF-1739356.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\jdbc_no_debug.SF-1739356-data.sql" ./sql/src/test/BugTracker/Tests/jdbc_no_debug.SF-1739356.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/jdbc_no_debug.SF-1739356-data.sql\"" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCcreate_user.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCschema.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCinsert.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCquery.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCmanual_examples.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCdrop.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCdrop_user.sql" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCcreate_user.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCschema.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCinsert.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCquery.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCmanual_examples.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCdrop.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCdrop_user.sql\" 2>&1" ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

[Monetdb-developers] Tests for XQuery admin GUI (-functionality)
by Stefan Manegold 03 Jul '08

03 Jul '08

Peter, Jennie, we should consider adding tests (or identifying them clearly if they already exist) that mimic the functionality required by the XQuery admin GUI. If you could provide me with input what is required and needs to be tested, I could help designing/implementing the actual tests. Please see also the following bug reports: fixed: [ 1995879 ] algebra kills admin GUI https://sourceforge.net/tracker/index.php?func=detail&aid=1995879&group_id=… OPEN: [ 1899258 ] PF: Error message with Administrative Console https://sourceforge.net/tracker/index.php?func=detail&aid=1899258&group_id=… [ 1989573 ] no collections in db while collections do exist https://sourceforge.net/tracker/index.php?func=detail&aid=1989573&group_id=… [ 1990815 ] backup creates 8.5GB data from 78MB database https://sourceforge.net/tracker/index.php?func=detail&aid=1990815&group_id=… Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

[Monetdb-developers] Why has a BAT no more the same order after a bat.delete(...)
by Simon Berther 01 Jul '08

01 Jul '08

Hello, I'm developing in MAL accessing commands and functions from C++ over the MAPI-library. My task is to delete some BUNs from a BAT. The problem: The BAT with some deleted BUNs has a different order than before. I do it the following way: c:bat[:oid,:int] := bbp.bind("joe"); slice:bat[:oid,:int] := algebra.slice(c,4,6); bat.delete(c,slice); The original BAT "joe": #-----------------# # h t # name # oid int # type #-----------------# [ 0@0, -1 ] [ 1@0, 2 ] [ 2@0, 3 ] [ 3@0, -1 ] [ 4@0, -2 ] <- to be deleted [ 5@0, 5 ] <- to be deleted [ 6@0, -2 ] <- to be deleted After delete: The original BAT "joe": #-----------------# # h t # name # oid int # type #-----------------# [ 3@0, -1 ] <- not in right order, why? [ 0@0, -1 ] [ 1@0, 2 ] [ 2@0, 3 ] Why is this so and how can I prevent MonetDB to change the order, while deleting BUNs. The order is important, because I'd like to work with position and slices to read my BUNs. Thanks for helping!

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/VOC/Tests All, SQL_2-24, 1.4, 1.4.4.1 VOC.SQL.bat, SQL_2-24, 1.3, 1.3.4.1 VOC.SQL.sh, SQL_2-24, 1.4, 1.4.4.1
by Romulo Goncalves 01 Jul '08

01 Jul '08

Stefan Manegold wrote: > Update of /cvsroot/monetdb/sql/src/test/VOC/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14066/sql/src/test/VOC/Tests > > Modified Files: > Tag: SQL_2-24 > All VOC.SQL.bat VOC.SQL.sh > Log Message: > > Clean-up of buildtools-, configure-, and testing-legacy in MonetDB-Java. > ( > Considered as "bug-fix" by Sjoerd and me. > Has no impact on binary distributions and/or packaging. > Wrapped in tags "JavaTestingCleanUp1_before" & "JavaTestingCleanUp1_after". > Will fix or undo tomorrow in case unexpected problems occur. > ) > > - Removed "mjclient" as is was not packaged/distributed, anyway. > > - Changed test that used "mjclient" to use "jdbcclient.jar" directly. > (Note: "jdbcclient.jar" is not packaged and/or distributed, either, but > only used internally for testing and "convenience".) What about change all these tests to use mclient? It is a slave job, but it will make the things a lot easier because not every one has Java. So me and other users are not obliged to install Java package to check if all the tests are Green. It just a suggestion... Regards, Romulo

2 1

[Monetdb-developers] New MonetDB Release
by Stefan Manegold 28 Jun '08

28 Jun '08

Dear MonetDB users and developers, The MonetDB team at CWI is pleased to announce the new "Jun2008" release of the MonetDB suite of programs. For MonetDB/XQuery, this release marks the roll forward from the old "milprint_summer"-based Pathfinder compiler to the new "Algebra"-based Pathfinder compiler. MonetDB/SQL has been extended with initial support for OpenGIS via the "Geom[etry]" extension module, aka. MonetDB/SQL/GIS. Additionally, the configure defaults for compilation from sources on Unix(-like) systems as well as the Windows installers have been cleaned-up. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ IMPORTANT NOTE: Before upgrading to the latest "Jun2008" release from any previous release of MonetDB, please make a dump of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ The new release consists of the following packages: MonetDB Common 1.24.0 MonetDB Clients 1.24.0 MonetDB4 Server 4.24.0 MonetDB4 XQuery 0.24.0 MonetDB5 Server 5.6.0 MonetDB5 SQL 2.24.0 MonetDB5 Geom 0.4.0 monetdb-1.8-jdbc.jar There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. Enjoy! The MonetDB development team. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Question about subquery to calculate median
by Rt Ibmer 20 Jun '08

20 Jun '08

Hello, I am trying to write a subquery that can serve as a poor man's median function. However I am running into issues every angle I try and cannot find any details about this in the docs or list archives. Question 1: Currently I can calculate the median manually as two steps: sql> select count(*)/2 from mytable where myIntVal is not null; +----------------+ | sql_div_count_ | +================+ | 10 | +----------------+ sql> select myIntVal as median from mytable order by myIntVal limit 1 offset 10; +--------+ | median | +========+ | 1172 | +--------+ However I cannot figure out how to combine these two queries into one query so that the count/2 can be done together with the query, so that I don't have to manually first get the count and then hard code it into the second query that does the median. How can these be combined? For instance I tried this which does not work: sql> select myIntVal as median from mytable order by myIntVal limit 1 offset (select count(*)/2 from mytable where myIntVal is not null); but this gives me: !syntax error, unexpected '(', expecting IDENT or sqlINT So I assume you can not dynamically substitute a subquery for an OFFSET value. But is there then another way to combine these statements into one statement? Question 2: Once I get the above resolved, I need to be able to get the median as an aggregate function, so that I can get medians in a group by query for each element in the group. For instance, if there really was a median() aggregate function I would do this: select median(myIntval), customerid from mytable group by customerid; However of course there is no median function. So I understand the work around for not having a median function is to get the total rows, sort the rows and then take the row in the middle. But how can I do that if it is in a group by statement like the above? What I am trying desperately to avoid is having to make a query to get all the group by results, and then for each result having to do a separate individual query to get its median separately. Please tell me this is possible! :) And if so, how? Thank you!!

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal batxml.mx, , 1.25, 1.26 clients.mx, , 1.75, 1.76 mal_init.mx, , 1.131, 1.132 tablet.mx, , 1.121, 1.122 txtsim.mx, , 1.4, 1.5
by Romulo Goncalves 19 Jun '08

19 Jun '08

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26834/mal > > Modified Files: > batxml.mx clients.mx mal_init.mx tablet.mx txtsim.mx > Log Message: > Just move comments out of the way so they don't end up in the main program > while loading the modules. This gives a polluted and confusing listing > when e.g. the socket binding fails during the startup. > > > U batxml.mx > Index: batxml.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/batxml.mx,v > retrieving revision 1.25 > retrieving revision 1.26 > diff -u -d -r1.25 -r1.26 > --- batxml.mx 10 Feb 2008 23:48:46 -0000 1.25 > +++ batxml.mx 18 Jun 2008 08:54:58 -0000 1.26 > @@ -71,10 +71,12 @@ > address BATXMLxquery > comment "Execute the XQuery against the elements."; > > -#todo > -#command xml.table(val:bat[:oid,:str],expr:str):bat[:oid,:xml] > -#address BATXMLquery > -#comment "Execute the XQuery against the elements"; > +@- > +todo > +command xml.table(val:bat[:oid,:str],expr:str):bat[:oid,:xml] > +address BATXMLquery > +comment "Execute the XQuery against the elements"; > +@mal > > command xml.pi(operator:str, ret:bat[:oid,:xml]):bat[:oid,:xml] > address XMLpi > > U clients.mx > Index: clients.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/clients.mx,v > retrieving revision 1.75 > retrieving revision 1.76 > diff -u -d -r1.75 -r1.76 > --- clients.mx 26 Apr 2008 22:39:44 -0000 1.75 > +++ clients.mx 18 Jun 2008 08:54:58 -0000 1.76 > @@ -26,7 +26,6 @@ > @mal > module clients; > > -#Operations for all clients > pattern setListing(flag:int):int > address CLTsetListing > comment "Turn on/off echo of MAL instructions: > @@ -62,7 +61,9 @@ > address CLTexit > comment "Terminate the session for a single client using a soft error."; > > -#Administrator operations > +@- > +Administrator operations > +@mal > command getLogins( ):bat[:int,:str] > address CLTLogin > comment "Pseudo bat of client login time."; > > U txtsim.mx > Index: txtsim.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/txtsim.mx,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- txtsim.mx 11 Jan 2008 10:41:38 -0000 1.4 > +++ txtsim.mx 18 Jun 2008 08:54:59 -0000 1.5 > @@ -1002,11 +1002,9 @@ > throw(MAL, "txtsim.qgramselfjoin", "could not realloc\n"); > } > > -@mal > - > +@- > #mil implementation > > -#mil implementation > #proc str2qgrams(str s) : bat[oid,str] > #{ > # s := qgramnormalize(s); > @@ -1015,7 +1013,25 @@ > # var b := bat(str,void,len+4); > # var i := 0; > # var last := " "; > +# > +# b.insert("#",nil); > +# b.insert("#",nil); > +# while(i < len) { > +# b.insert(s.string(i,1), nil); > +# if (b.count() > 32764) break; # ignore suffix of >32KB strings > +# i :+= 1; > +# } > +# b.insert("$",nil); > +# b.insert("$",nil); > +# b.reverse().seqbase(0@0); > +# > +# var c0 := b.select(0@0, oid(b.count() - 3)).mark(0(a)0).reverse() > +# var c1 := b.select(1@0, oid(b.count() - 2)).mark(0(a)0).reverse() > +# var c2 := b.select(2@0, oid(b.count() - 1)).mark(0(a)0).reverse() > +# return [+](c0, [+](c1, c2)); > +#} > #mal implementation > +@mal > function str2qgrams(s:str):bat[:oid,:str]; > s := qgramnormalize(s); > > @@ -1024,16 +1040,8 @@ > b := bat.new(:str,:oid,len); > i := 0; > last := " "; > -# > -# b.insert("#",nil); > -# b.insert("#",nil); > bat.insert(b,"#",nil:oid); > bat.insert(b,"#",nil:oid); > -# while(i < len) { > -# b.insert(s.string(i,1), nil); > -# if (b.count() > 32764) break; # ignore suffix of >32KB strings > -# i :+= 1; > -# } > cnt:= 0:int; > id := calc.oid(cnt); > barrier B:= i < len; > @@ -1044,29 +1052,22 @@ > i := calc.+(i,1); > redo B:= i < len; > exit B; > -# b.insert("$",nil); > -# b.insert("$",nil); > -# b.reverse().seqbase(0@0); > bat.insert(b,"$",nil:oid); > bat.insert(b,"$",nil:oid); > rev := bat.reverse (b); > alg := algebra.markH(rev, 0@0); > -# > -# var c0 := b.select(0@0, oid(b.count() - 3)).mark(0(a)0).reverse() > cnt := aggr.count(alg); > cnt := calc.-(cnt,3); > sel := algebra.select(b, 0@0, id); > mrk := algebra.markT(sel, 0@0); > c0 := bat.reverse(mrk); > > -# var c1 := b.select(1@0, oid(b.count() - 2)).mark(0(a)0).reverse() > cnt := aggr.count(alg); > cnt := calc.-(cnt,2); > id := calc.oid(cnt); > sel := algebra.select(b, 1@0, id); > mrk := algebra.markT(sel, 0@0); > c1 := bat.reverse(mrk); > -# var c2 := b.select(2@0, oid(b.count() - 1)).mark(0(a)0).reverse() > cnt := aggr.count(alg); > cnt := calc.-(cnt,1); > id := calc.oid(cnt); > @@ -1074,8 +1075,6 @@ > mrk := algebra.markT(sel, 0@0); > c2 := bat.reverse(mrk); > > -# return [+](c0, [+](c1, c2)); > -#} > res := batcalc.+(c1, c2); > res := batcalc.+(c0, res); > > > U tablet.mx > Index: tablet.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet.mx,v > retrieving revision 1.121 > retrieving revision 1.122 > diff -u -d -r1.121 -r1.122 > --- tablet.mx 17 Jun 2008 21:58:18 -0000 1.121 > +++ tablet.mx 18 Jun 2008 08:54:58 -0000 1.122 > @@ -235,7 +235,6 @@ > The dump_format scripts dump a format file for a given set of > to be dumped bats. These bats can be dumped with dump_data. > @mal > -# the major tablet shuffling routines > module tablet; > > command load( names:bat[:oid,:str], seps:bat[:oid,:str], > @@ -303,7 +302,9 @@ > address TABsetRowBracket > comment "Format the brackets around a row"; > > -# Set the column properties > +@- > +Set the column properties > +@mal > pattern setColumn(idx:int, v:any_1) > address TABsetColumn > comment "Bind i-th output column to a variable"; > @@ -443,7 +444,7 @@ > Column columns[1]; /* at least one column, enlarged upon need */ > } Tablet; > > -tablet_export BAT *TABLETload(BAT *names, BAT *seps, BAT *types, char *datafile, size_t nr); > +tablet_export BAT *TABLETload(Tablet *as, char *datafile); > tablet_export BAT *TABLETinput(BAT *names, BAT *seps, BAT *types, bstream *s, stream *out, size_t nr, size_t est); > tablet_export size_t TABLEToutput(BAT *order, BAT *seps, BAT *bats, stream *s); > tablet_export void TABLETdump(BAT *names, BAT *seps, BAT *bats, char *datafile, size_t nr); > @@ -451,7 +452,7 @@ > /* The low level routines are primarilly used by the SQL front-end.*/ > tablet_export int TABLETcreate_bats(Tablet * as, size_t est); > tablet_export size_t TABLETassign_BATs(Tablet * as, BAT *bats); > -tablet_export ssize_t TABLETload_file(Tablet * as, bstream *b, stream *out); > +tablet_export ssize_t TABLETload_file(Tablet * as, bstream *b, stream *out, size_t col1, size_t col2); > tablet_export BAT *TABLETcollect_bats(Tablet * as); > tablet_export void TABLETdestroy_format(Tablet * as); > tablet_export int TABLEToutput_file(Tablet * as, BAT *order, stream *s); > @@ -459,7 +460,7 @@ > tablet_export ptr *TABLETstrFrStr(Column *c, char *s, char *e); > tablet_export ptr *TABLETadt_frStr(void *extra, int type, char *s, char *e, char quote); > tablet_export int TABLETadt_toStr(void *extra, char **buf, int *len, int type, ptr a); > -tablet_export int insert_line(Tablet * as, char *line, ptr key); > +tablet_export int insert_line(Tablet * as, char *line, ptr key, size_t col1, size_t col2 ); I do not see any changes on SQL to support these new function signatures. It seems SQL compilation will be broken tomorrow morning: r-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -c sql_result.c -fPIC -DPIC -o .libs/lib_sql_la-sql_result.o /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_result.mx: In function 'mvc_import_table': /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_result.mx:478: error: too few arguments to function 'TABLETload_file' make[7]: *** [lib_sql_la-sql_result.lo] Error 1 Regards, Romulo > tablet_export int output_file_dense(Tablet * as, stream *fd); > tablet_export int has_whitespace(char *sep); > > @@ -1376,7 +1377,7 @@ > } > > INLINE int > -insert_line(Tablet * as, char *line, ptr key) > +insert_line(Tablet * as, char *line, ptr key, size_t col1, size_t col2 ) > { > int res = 0; > Column *fmt = as->format; > @@ -1422,7 +1423,8 @@ > if (!e && i == (as->nr_attrs-1)) > e = line; > if (e) { > - res |= insert_val(&fmt[i], s, e, quote, key, &as->error, i); > + if (i>= col1 && i< col2) > + res |= insert_val(&fmt[i], s, e, quote, key, &as->error, i); > quote = 0; > line = e + fmt[i].seplen; > if (as->error && as->complaints) > @@ -1606,7 +1608,7 @@ > > > ssize_t > -TABLETload_file(Tablet * as, bstream *b, stream *out) > +TABLETload_file(Tablet * as, bstream *b, stream *out, size_t col1, size_t col2) > { > int res = 0, done = 0; > size_t i = 0; > @@ -1640,7 +1642,7 @@ > > if (e) { > *e = '\0'; > - if (i>=offset && insert_line(as, s, NULL) < 0) { > + if (i>=offset && insert_line(as, s, NULL, col1,col2) < 0) { > s = e + seplen; > b->pos = (s - b->buf); > res = -1; > @@ -1661,7 +1663,7 @@ > res = 1; > if (b->len > b->pos && > i>=offset && > - insert_line(as, s, NULL) < 0 && !as->tryall) { > + insert_line(as, s, NULL, col1,col2) < 0 && !as->tryall) { > GDKerror(as->error); > as->error=0; > GDKerror("TABLETload_file: read error " > @@ -1810,26 +1812,45 @@ > } > > BAT * > -TABLETload(BAT *names, BAT *seps, BAT *types, char *datafile, size_t nr) > +TABLETload(Tablet *as,char *datafile) > { > - BAT *res; > - stream *s = open_rastream(datafile); > + BAT *res= NULL; > + stream *s = NULL; > bstream *b = NULL; > - size_t est = nr; > + size_t est = as->nr; > + size_t col,stride=1; > + int error= FALSE; > > - if (s == NULL || stream_errnr(s)) { > - GDKerror("could not open file %s\n", datafile); > - if (s) { > - stream_destroy(s); > + if ( TABLETcreate_bats(as, est) < 0) > + return NULL; > + > + if (as->nr == ~(size_t) 0) > + est = estimator(datafile); > +@- > +Determine a good Stride for the table at hand. It should be based > +on the number of records to be expected and the general system resources. > +@c > + for (col=0; col <as->nr_attrs; col += stride){ > + s = open_rastream(datafile); > + if (s == NULL || stream_errnr(s)) { > + GDKerror("could not open file %s\n", datafile); > + if (s) { > + stream_destroy(s); > + } > + return NULL; > } > - return NULL; > + if (TABLETload_file(as, b = bstream_create(s, SIZE), NULL ,col,col+stride) < 0){ > + error= TRUE; > + break; > + } > + > + bstream_destroy(b); > + stream_close(s); > + stream_destroy(s); > } > - if (nr == ~(size_t) 0) > - est = estimator(datafile); > - res = TABLETinput(names, seps, types, b = bstream_create(s, SIZE), NULL, nr, est); > - bstream_destroy(b); > - stream_close(s); > - stream_destroy(s); > + if( error == FALSE) > + res = TABLETcollect_bats(as); > + TABLETdestroy_format(as); > return res; > } > > @@ -1841,8 +1862,9 @@ > > as.nr_attrs = 0; > as.nr = nr; > - if (create_loadformat(&as, names, seps, types) != ~(size_t) 0 && TABLETcreate_bats(&as, est) >= 0) { > - if (TABLETload_file(&as, s, out) >= 0) > + if (create_loadformat(&as, names, seps, types) != ~(size_t) 0 && > + TABLETcreate_bats(&as, est) >= 0) { > + if (TABLETload_file(&as, s, out, 0, as.nr_attrs) >= 0) > bats = TABLETcollect_bats(&as); > } > TABLETdestroy_format(&as); > @@ -1917,10 +1939,15 @@ > tablet_load(BAT **bats, BAT *names, BAT *seps, BAT *types, str datafile, int *N) > { > size_t nr = ~(size_t)0; > + Tablet as; > > if (*N >= 0) > nr = *N; > - *bats = TABLETload(names, seps, types, datafile, nr); > + as.nr_attrs = 0; > + as.nr = nr; > + > + if (create_loadformat(&as, names, seps, types) != ~(size_t) 0 ) > + *bats = TABLETload(&as, datafile); > } > static void > tablet_input(BAT **bats, BAT *names, BAT *seps, BAT *types, void **s, int *N) > > U mal_init.mx > Index: mal_init.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/mal_init.mx,v > retrieving revision 1.131 > retrieving revision 1.132 > diff -u -d -r1.131 -r1.132 > --- mal_init.mx 2 May 2008 07:34:48 -0000 1.131 > +++ mal_init.mx 18 Jun 2008 08:54:58 -0000 1.132 > @@ -89,7 +89,9 @@ > library chopper; > library txtsim; > > -#library logger; > +@- > +library logger; > +@mal > library transaction; > > library mal_isolation; > @@ -157,9 +159,13 @@ > include factory; > include recycle; > > -# experimental > +@- > +experimental > +@mal > include mat; > -# the real partition handler > +@- > +the real partition handler > +@mal > include bpm; > include inspect; > include manual; > @@ -203,14 +209,20 @@ > include chopper; > include txtsim; > > -#include logger; > +@- > +include logger; > +@mal > include transaction; > > -#scheduler components > +@- > +scheduler components > +@mal > include mal_isolation; > include mal_memorun; > > -# Cracker schemes > +@- > + Cracker schemes > +@mal > include opt_crack; > include opt_joinselect; > include opt_remoteQueries; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, , 1.421, 1.422 pf_support.mx, , 1.301, 1.302
by Stefan Manegold 18 Jun '08

18 Jun '08

On Tue, Jun 17, 2008 at 05:07:29PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv2937 > > Modified Files: > pathfinder.mx pf_support.mx > Log Message: > - BUG 1995879: only use algebra for normal queries > > cachable queries and the XRPC server still use mps Peter, just for clarification and the records: am I right that this means that now all XQuery modules are compiled with MPS, but when a (non-XRPC?) query uses an XQuery module, this XQuery itself is compiled with ALG? Or are such queries compiled with MPS, too? Thanks in advance! Stefan > > > > U pathfinder.mx > Index: pathfinder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v > retrieving revision 1.421 > retrieving revision 1.422 > diff -u -d -r1.421 -r1.422 > --- pathfinder.mx 16 Jun 2008 23:38:41 -0000 1.421 > +++ pathfinder.mx 17 Jun 2008 17:07:26 -0000 1.422 > @@ -4246,7 +4246,7 @@ > * - execute MIL buffer (parse & execute) > * > * int > - * xquery_compile_exec(xquery_client *ctx, char* xquery, int is_url, > + * xquery_compile_exec(xquery_client *ctx, int options, char* xquery, int is_url, > * char** prologue, char** query, char** epilogue, char* nsurl) > * - translate xquery to MIL and execute > * > @@ -4345,6 +4345,7 @@ > extern char* PFmaxstack; > static int > xquery_compile_exec(xquery_client *ctx, > + int options, > char* url, > int is_url, > char** prologue, > @@ -4356,22 +4357,9 @@ > char *xquery = url; > char *err = NULL; > > - int options = 0; > /* Setting the StandOff flag based on runtime settings */ > if ((GDKgetenv("standoff") != NULL) && (strcmp(GDKgetenv("standoff"),"enabled") == 0)) > options |= COMPILE_OPTION_STANDOFF; > - /* Setting the Algebra flag; server setting overules compile-time default; client choice overrules server setting */ > -#if MILPRINT_SUMMER_IS_DEFAULT > - if (((ctx->mode & XQ_ALGEBRA) != 0) || \ > - (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > - (GDKgetenv("xquery_backend") != NULL) && (strcmp(GDKgetenv("xquery_backend"),"algebra") == 0))) > - options |= COMPILE_OPTION_ALGEBRA; > -#else /* ALGEBRA_IS_DEFAULT */ > - if (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > - (((ctx->mode & XQ_ALGEBRA) != 0) || \ > - (GDKgetenv("xquery_backend") == NULL) || (strcmp(GDKgetenv("xquery_backend"),"milprint_summer") != 0))) > - options |= COMPILE_OPTION_ALGEBRA; > -#endif > > MT_set_lock(pf_compiler_lock, "xquery_compile_exec"); > if (is_url) { > @@ -4910,7 +4898,7 @@ > strcpy(mod->url, url); > mod->nsurl = mod->url + url_len; > > - ret = xquery_compile_exec(ctx, url, 1, &mod->prologue, NULL, &mod->epilogue, mod->nsurl); > + ret = xquery_compile_exec(ctx, 0, url, 1, &mod->prologue, NULL, &mod->epilogue, mod->nsurl); > if (!ret) { > xquery_module_free(mod); > return NULL; > @@ -5745,9 +5733,22 @@ > char *sec1 = NULL; > char *sec2 = NULL; > char *sec3 = NULL; > + int options = 0; > /* compile and execute the query (minus module imports) */ > err = xquery_nondescriptive_error; > - if (xquery_compile_exec(ctx, query, 0, &sec1, &sec2, &sec3, NULL)) > + /* Setting the Algebra flag; server setting overules compile-time default; client choice overrules server setting */ > +#if MILPRINT_SUMMER_IS_DEFAULT > + if (((ctx->mode & XQ_ALGEBRA) != 0) || \ > + (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > + (GDKgetenv("xquery_backend") != NULL) && (strcmp(GDKgetenv("xquery_backend"),"algebra") == 0))) > + options = COMPILE_OPTION_ALGEBRA; > +#else /* ALGEBRA_IS_DEFAULT */ > + if (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > + (((ctx->mode & XQ_ALGEBRA) != 0) || \ > + (GDKgetenv("xquery_backend") == NULL) || (strcmp(GDKgetenv("xquery_backend"),"milprint_summer") != 0))) > + options = COMPILE_OPTION_ALGEBRA; > +#endif > + if (xquery_compile_exec(ctx, options, query, 0, &sec1, &sec2, &sec3, NULL)) > err = NULL; > if (sec1) free(sec1); > if (sec2) free(sec2); > > U pf_support.mx > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.301 > retrieving revision 1.302 > diff -u -d -r1.301 -r1.302 > --- pf_support.mx 10 Jun 2008 09:22:17 -0000 1.301 > +++ pf_support.mx 17 Jun 2008 17:07:27 -0000 1.302 > @@ -8373,7 +8373,7 @@ > BUNins(*res, &docpre, &docid[j], FALSE); > > /* skip all nodes of this document */ > - docpre = docpre + size[docpre]; > + docpre = docpre + size[docpre] + 1; > for (delta = (n-i) >> 4; delta > 40; delta >>= 4) > while (i+delta < n && * (oid *) BUNtail(inputi, i+delta) <= docpre) > i += delta; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] database 'mydbname' does not allow connections error
by Rt Ibmer 18 Jun '08

18 Jun '08

Hello, I have a start up script as part of a server image that runs the first time a new instance of the server is started. It performs the following: # start the monetdb server /usr/bin/merovingian # create and start the db monetdb create mydb monetdb start mydb # create tables mclient -lsql --time --database=mydb < /db/schemas.sql However the mclient fails to execute the schemas. The consoles shows the following: successfully created database 'mydb' database 'bhtg' does not allow connections However, if I execute the same exact commands from a bash shell once the system is booted, then it works fine and the mclient executes the .sql script and creates the tables as expected. What could then be causing the error about not allowing connections? The startup script is running with this: # chkconfig: 345 15 95 So I think the network is fully initialized and operational by the time this runs. Any advice please? Thank you.

2 5

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime/xrpc/export xrpc.html, XQuery_0-24, 1.4.6.2, 1.4.6.3
by Stefan Manegold 18 Jun '08

18 Jun '08

On Tue, Jun 17, 2008 at 11:18:42PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime/xrpc/export > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28334 > > Modified Files: > Tag: XQuery_0-24 > xrpc.html > Log Message: > - fix link in demo page FYI: http://monetdb.cwi.nl/XQuery/Documentation/XRPC-Extension.html and http://monetdb.cwi.nl/projects/monetdb/XQuery/Documentation/XRPC-Extension.… in fact point to the same page; in general http://monetdb.cwi.nl/ and http://monetdb.cwi.nl/projects/monetdb/ are equivalent. Stefan > > > > U xrpc.html > Index: xrpc.html > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc/export/xrpc.html,v > retrieving revision 1.4.6.2 > retrieving revision 1.4.6.3 > diff -u -d -r1.4.6.2 -r1.4.6.3 > --- xrpc.html 31 May 2008 21:26:50 -0000 1.4.6.2 > +++ xrpc.html 17 Jun 2008 23:18:40 -0000 1.4.6.3 > @@ -49,7 +49,7 @@ > For more information on XRPC, check the following sources: > <ul> > <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/QuickTour/XRPC">tutorial</a> > - <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/Documentation/XRPC-Extension.html">reference manual</a> > + <li> the XRPC <a href="http://monetdb.cwi.nl/projects/monetdb/XQuery/Documentation/XRPC-Extension.…">reference manual</a> > <li> the XRPC <a href="http://www.cwi.nl/htbin/ins1/publications?request=intabstract&key=ZhBo:VLDB…">VLDB Paper</a> > </ul> > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, XQuery_0-24, 1.416.2.8, 1.416.2.9
by Stefan Manegold 18 Jun '08

18 Jun '08

On Tue, Jun 17, 2008 at 09:56:41PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8192 > > Modified Files: > Tag: XQuery_0-24 > pathfinder.mx > Log Message: > check in correct branch (just in time for testing?) Yes. Thanks! One open questions: Was the change in pf_support that you committed with the same checkin on the development trunk (see below) related to the bug fix or not? If not, what was it for? In either case: should it be back-ported as well> Stefan ======== Index: pf_support.mx =================================================================== RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v retrieving revision 1.301 retrieving revision 1.302 diff -u -d -r1.301 -r1.302 --- pf_support.mx 10 Jun 2008 09:22:17 -0000 1.301 +++ pf_support.mx 17 Jun 2008 17:07:27 -0000 1.302 @@ -8373,7 +8373,7 @@ BUNins(*res, &docpre, &docid[j], FALSE); /* skip all nodes of this document */ - docpre = docpre + size[docpre]; + docpre = docpre + size[docpre] + 1; for (delta = (n-i) >> 4; delta > 40; delta >>= 4) while (i+delta < n && * (oid *) BUNtail(inputi, i+delta) <= docpre) i += delta; ======== > > > U pathfinder.mx > Index: pathfinder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v > retrieving revision 1.416.2.8 > retrieving revision 1.416.2.9 > diff -u -d -r1.416.2.8 -r1.416.2.9 > --- pathfinder.mx 16 Jun 2008 21:23:51 -0000 1.416.2.8 > +++ pathfinder.mx 17 Jun 2008 21:56:37 -0000 1.416.2.9 > @@ -4246,7 +4246,7 @@ > * - execute MIL buffer (parse & execute) > * > * int > - * xquery_compile_exec(xquery_client *ctx, char* xquery, int is_url, > + * xquery_compile_exec(xquery_client *ctx, int options, char* xquery, int is_url, > * char** prologue, char** query, char** epilogue, char* nsurl) > * - translate xquery to MIL and execute > * > @@ -4345,6 +4345,7 @@ > extern char* PFmaxstack; > static int > xquery_compile_exec(xquery_client *ctx, > + int options, > char* url, > int is_url, > char** prologue, > @@ -4356,22 +4357,9 @@ > char *xquery = url; > char *err = NULL; > > - int options = 0; > /* Setting the StandOff flag based on runtime settings */ > if ((GDKgetenv("standoff") != NULL) && (strcmp(GDKgetenv("standoff"),"enabled") == 0)) > options |= COMPILE_OPTION_STANDOFF; > - /* Setting the Algebra flag; server setting overules compile-time default; client choice overrules server setting */ > -#if MILPRINT_SUMMER_IS_DEFAULT > - if (((ctx->mode & XQ_ALGEBRA) != 0) || \ > - (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > - (GDKgetenv("xquery_backend") != NULL) && (strcmp(GDKgetenv("xquery_backend"),"algebra") == 0))) > - options |= COMPILE_OPTION_ALGEBRA; > -#else /* ALGEBRA_IS_DEFAULT */ > - if (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > - (((ctx->mode & XQ_ALGEBRA) != 0) || \ > - (GDKgetenv("xquery_backend") == NULL) || (strcmp(GDKgetenv("xquery_backend"),"milprint_summer") != 0))) > - options |= COMPILE_OPTION_ALGEBRA; > -#endif > > MT_set_lock(pf_compiler_lock, "xquery_compile_exec"); > if (is_url) { > @@ -4910,7 +4898,7 @@ > strcpy(mod->url, url); > mod->nsurl = mod->url + url_len; > > - ret = xquery_compile_exec(ctx, url, 1, &mod->prologue, NULL, &mod->epilogue, mod->nsurl); > + ret = xquery_compile_exec(ctx, 0, url, 1, &mod->prologue, NULL, &mod->epilogue, mod->nsurl); > if (!ret) { > xquery_module_free(mod); > return NULL; > @@ -5745,9 +5733,22 @@ > char *sec1 = NULL; > char *sec2 = NULL; > char *sec3 = NULL; > + int options = 0; > /* compile and execute the query (minus module imports) */ > err = xquery_nondescriptive_error; > - if (xquery_compile_exec(ctx, query, 0, &sec1, &sec2, &sec3, NULL)) > + /* Setting the Algebra flag; server setting overules compile-time default; client choice overrules server setting */ > +#if MILPRINT_SUMMER_IS_DEFAULT > + if (((ctx->mode & XQ_ALGEBRA) != 0) || \ > + (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > + (GDKgetenv("xquery_backend") != NULL) && (strcmp(GDKgetenv("xquery_backend"),"algebra") == 0))) > + options = COMPILE_OPTION_ALGEBRA; > +#else /* ALGEBRA_IS_DEFAULT */ > + if (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > + (((ctx->mode & XQ_ALGEBRA) != 0) || \ > + (GDKgetenv("xquery_backend") == NULL) || (strcmp(GDKgetenv("xquery_backend"),"milprint_summer") != 0))) > + options = COMPILE_OPTION_ALGEBRA; > +#endif > + if (xquery_compile_exec(ctx, options, query, 0, &sec1, &sec2, &sec3, NULL)) > err = NULL; > if (sec1) free(sec1); > if (sec2) free(sec2); > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Pathfinder (MonetDB/XQuery) testing results
by Stefan Manegold 17 Jun '08

17 Jun '08

Dear all, after I finally switched the compile time default for Pathfinder from using the old MPS ("milprint_summer") back-end to using the new ALG ("Algebra") back-end last night, the nightly testing result for Pathfinder (aka. MonetDB/XQuery) are available at (`make check` using default ALG back-end): http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103… http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests10… (`RunMtest -g` explicitly using the old MPS back-end): http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTestsg10… http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTestsg1… The TestWeb pages at http://monetdb.cwi.nl/Development/TestWeb/Stable/pathfinder/index.html and http://monetdb.cwi.nl/Development/TestWeb/Current/pathfinder/index.html will be updated accordingly as soon as we manage to re-buld the web-site ... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Question on monetdb PHP support
by Rt Ibmer 17 Jun '08

17 Jun '08

I am building Monetdb using the stable nightly builds. I would like to write queries against Monetdb using PHP. I came across this in the docs and have two questions about it: ---------------- The unix configure process normally tries to detect if you have PHP including developer packages installed and builds the PHP module only if you have it. With the --with-php option you could tell 'configure' where to find the PHP installation. ---------------- 1) Regarding "detect if you have PHP including developer packages" - can you please tell me what packages exactly MonetDB will look for. Obviously it wants the main php package. But I am not sure what development packages I need to install so that Monetdb will build in the PHP support and make me the DB/monetdb.php file. I am using Fedora 8. What yum packages (or equiv) should I have installed for this? 2) Regarding the "--with-php" compile option - which of the PHP base directories would I want to point the build .sh script for MonetDb at? Thanks!

2 1

[Monetdb-developers] Cannot get perl talking to monetdb5 nightly stable
by Rt Ibmer 17 Jun '08

17 Jun '08

I am using the nightly stable build of monetdb5. All outside of perl->monetdb seems to be working just fine. I just can't get perl to talk to the db. Here's what I did: - installed from nightly stable source on Fedora 8 - copied the source from http://homepages.cwi.nl/~mk/MonetDB/The-Perl-Library.html into mytest.pl - tried running: perl /tmp/mytest.pl but it came back complaining about missing libs in @INC. So then I added these steps: - /var/tmp/MonetDB-1213632062/MonetDB-client/clients/src/perl - make install and also ran make instlal in the various subdirs like MonetDB-CLI and others (not sure if this step was necessar) Then I tried running the script and this time it executed without the @INC errors. So this was progress. However all I get is this: ------- Start a simple Monet MIL interaction data sources: dbi:monetdb: -------- and then nothing. It just sits there indefinitely. In playing with the source of the .pl I see that it is the "my $dbh = DBI->connect( $dsn, ..." that execution stops at. I have merovingian running and the db is started. I also tried with merovigan stopped and just using mserver5 --dbinit="include sql;" but I had the same result. Am I doing something wrong? Thanks.

1 0

[Monetdb-developers] arrow keys not working in mclient in stable nightly
by Rt Ibmer 16 Jun '08

16 Jun '08

Hi - I use a SSH client to access my Fedora 8 core box. When I was using the RELEASE built I was able to use left/right/up/down/home/end keys within mclient to edit the line. For instance I could push up arrow to have it automatically cycle through the previous lines, and then use left arrow to move over to a part of the line and process to change it. That all worked fine under the RELEASE build. However now that I have built from source the nightly stable (as of a couple nights ago) I can no longer use these keys. For instance when I press UP arrow it simply adds "^[[A" to the line. Other arrow keys / home /end do something similar. Not sure but I think something broke regarding this at some point between the released build and the stable nightly. Thanks.

2 1

[Monetdb-developers] Does monetdb support limit x,y syntax?
by Rt Ibmer 16 Jun '08

16 Jun '08

To get a median I want to use this: select myIntCol from mytable order by myIntCol limit half_table_size,1; This works in mySQL. However I get an error when I use limit with x,y params. If I just use limit x then it limits the return results to just x rows. But what I need to do is tell it to start with row x and only give me 1 row (i.e. "... limit 5000, 1;" when I have 10000 rows in the table. Does MonetDB support this format? If not is there a work around?? Thanks.

2 1

[Monetdb-developers] JDBC connection error: Max number of redirects reached
by Rt Ibmer 15 Jun '08

15 Jun '08

OK I have a strange problem that I have FINALLY narrowed down and hoping someone can help me resolve please. I am using the latest stable nightly MonetDB5 with SQL on Fedora core 8. If I run merovingian and then try to connect to my database using JDBC through my Java app, I get this error returned by JDBC: "Maximum number of redirects reached, aborting connection attempt. Sorry." However, if instead of running merovingian I run this from the shell: mserver5 --dbinit="include sql;" Then my JDBC connection works perfectly!! However, I need to start MonetDB in the background and use merovingian at boot time, so I am stuck. What does this 'maximum number of redirects reached' mean exactly? Why would I get this problem when using merovingian but not when using MonetDB? How can I resolve this or troubleshoot things further? I tried searching the web for this error message but didn't come up with anything related to monetdb. Thank you!!

3 6

[Monetdb-developers] Bad SQL query causes mclient to terminate
by Rt Ibmer 14 Jun '08

14 Jun '08

When working with RELEASE version of MonetDB5 every once in a while I would enter in a bad query into mclient and it would crash the entire server hard - it would not even reboot. I did notice, however, that in the nightly stable release this same issue is handled better, in that it no longer crashes the server (very good news!). However it does result in mclient terminating the connection and closing. Here is how you can reproduce this: 1) run merovingian 2) mclient -lsql --time --database=demo 3) enter this query: sql>select avg(1); and you will get this: MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = select avg(1); ERROR = Connection terminated Timer 3.049 msec <mclient exits> [bash ~] # Am I correct to assume this is not expected behavior? Thanks.

1 0

[Monetdb-developers] Support for stdev() and var() ?
by Rt Ibmer 14 Jun '08

14 Jun '08

Does the very latest version of MonetDB support built-in queries for the mathematical standard deviation and variance functions stdev() and var()? When I try these queries it returns "no such unary operator". I was under the impression that these were fairly standard in the SQL syntax so I am surprised to find this. If this is not supported, can you please let me know if there are any plans to add this and if so in what approximate timeframe? These functions are critical to our use of the db. Thank you.

1 0

Re: [Monetdb-developers] Works fine once, then get Invalid credentials for user monetdb
by Rt Ibmer 14 Jun '08

14 Jun '08

>Probably you aren't but the binpkgs you use are. You'll probably have >less frustrations at this point if you just compile from source, using >the monetdb-install.sh script. What seems wrong here is that the >default path is set to the builddir instead of the target dir. Thanks for the suggestion. My results were mixed. The good news is that in the latest nightly stable build the problems with the intermitant bad passwords and with the server failing to reboot have been fixed! So that is great! However the not-so-good news is that now my JDBC connection is no longer working properly. Do I need to use a newer version of the JDBC driver in conjunction with this latest stable nightly build? If so I could not find it - where do I get it? The problem with the JDBC driver is that when running with merovingian I get: Maximum number of redirects reached, aborting connection attempt. Sorry. then instead of running withmerovingian, as a test I ran with: mserver5 --dbinit="include sql;" and when running that way I get InvalidCredentialsException:checkCredentials:Invalid credentials for user 'monetdb' However I am using the default username and password of monetdb for both and haven't changed this. Furthermore If I run mclient with -umonetdb and specify monetdb as the password it accepts it there just fine. So I assume monetdb is still the default valid username/password for this stable nightly. But for some reason when I pass this as the user name and password in my JDBC connection it won't accept it. As mentioned earlier, this part was working just fine under the released version of MonetDB5. Can someone please help? I am running out of time to get this implemented so I am hoping to close this loop today. Thank you so much!!

2 17

[Monetdb-developers] Did something with Copy Into and nulls change in nightly stable?
by Rt Ibmer 14 Jun '08

14 Jun '08

I found that the nightly stable build of MonetDB5/sql apparently does not like empty values in tbl text records for importing when using Copy Into, even though the field is defined to allow nulls. This is different behavior compared to the RELEASE version, which did allow this. Please allow me to explain... I have a text file like this: field1|field2|| My db has defined field 1 and 2 as varchar null, and field 3 as INT null. When I run the Copy Into command on the nightly stable, it errors out with this: ' from line 0 field 3 not inserted, expecting type int !SQLException:importTable:failed to import table As a quick test I edited the first line of my text file to insert a fake integer into the 3rd field like this: field1|field2|12345| Then I reran the Copy Into command, this time it errored out on the SECOND line (even those it says line 1 it is really 0 index based so its line 2): ' from line 1 field 3 not inserted, expecting type int !SQLException:importTable:failed to import table So it was happy when I inserted the fake data for the field and does not like rows that are empty. Again, this column is defined as int NULL. Was this change intentional? If not I think there is a bug there. How can I important my text file that has many nulls in it, short of going back to the released version, which I cannot do due to problems with that earlier version. Thank you.

2 1

[Monetdb-developers] server locks up on reboot attempt anytime after running mclient
by Rt Ibmer 13 Jun '08

13 Jun '08

I am running a server instance under Amazon EC2 with MonetDB 5 (latest released version) installed. If I boot up the instance and do NOT start or use Monetdb, then I can issue a reboot command at any time, and the system reboots just fine and comes back up. However if I issue the commands: mkdir -p /mnt/MonetDB5/dbfarm merovingian monetdb create demo monetdb start demo mclient -lsql --time --database=demo sql> \q Then when I go to reboot I get the following output in the console and then things hang, and the instance never reboots. Here is the console output I get (more notes on the issue following this output): INIT: Switching to runlevel: 6 INIT: Sending processes the TERM signal Stopping ConsoleKit: [ OK ] Stopping sshd: [ OK ] Stopping crond: [ OK ] Stopping system message bus: [ OK ] Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] Shutting down interface eth0: [ OK ] Shutting down loopback interface: [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Unloading modules: [ OK ] Starting killall: [ OK ] Sending all processes the TERM signal... ------------[ cut here ]------------ kernel BUG at include/linux/tracehook.h:369! invalid opcode: 0000 [#1] SMP last sysfs file: /class/net/sit0/address Modules linked in: sit(U) tunnel4(U) fuse(U) ipv6(U) dm_mirror(U) dm_multipath(U) dm_mod(U) pcspkr(U) ext3(U) jbd(U) mbcache(U) uhci_hcd(U) ohci_hcd(U) ehci_hcd(U) xenblk(U) xennet(U) CPU: 0 EIP: 0061:[<c10254df>] Not tainted VLI EFLAGS: 00210087 (2.6.21.7-2.fc8xen #1) EIP is at release_task+0x38/0x2f1 eax: c12ff000 ebx: c2b9f450 ecx: f5416000 edx: 00000000 esi: c2b9f450 edi: 00000000 ebp: 00000001 esp: ed207e68 ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0069 Process mserver5 (pid: 1124, ti=ed207000 task=c1404910 task.ti=ed207000) Stack: 00000020 c1404910 00000000 ed7c15b0 c1026918 00000009 00000008 0007b6c0 c1404d64 c1404d64 c14049c4 00000000 c14f30e4 c188bac0 00000000 ed207fb8 c10269fb c14f30e4 00000009 b438a364 c102dcb2 00000000 00000000 00000000 Call Trace: [<c1026918>] do_exit+0x6d6/0x730 [<c10269fb>] sys_exit_group+0x0/0xd [<c102dcb2>] get_signal_to_deliver+0x3d2/0x414 [<c1004be7>] do_notify_resume+0x8c/0x6e6 [<c12038ae>] do_page_fault+0x7a1/0xc24 [<c10e48a8>] copy_to_user+0x3c/0x50 [<c107d766>] sys_select+0x161/0x187 [<c1005765>] work_notifysig+0x13/0x1a ======================= Code: 98 04 00 00 00 74 07 89 f0 e8 e6 98 02 00 8b 86 80 01 00 00 90 ff 48 04 b8 00 f0 2f c1 e8 c5 c7 1d 00 83 be 90 00 00 00 20 74 04 <0f> 0b eb fe 83 be 98 04 00 00 00 74 27 8d 86 a8 04 00 00 e8 ee EIP: [<c10254df>] release_task+0x38/0x2f1 SS:ESP 0069:ed207e68 Fixing recursive fault but reboot is needed! --------------------------------------------------- Note that if I run these same commands mkdir -p /mnt/MonetDB5/dbfarm merovingian monetdb create demo monetdb start demo # OMIT THIS TIME mclient -lsql --time --database=demo But if I DO NOT run mclient, then the problem does not occur and it will reboot fine. However once I run mclient then I am guaranteed to lock up on a reboot with the console output as shown above. What is really bad about this issue in particular is that on EC2 if an instance will not reboot it needs to be terminated. And when terminated all data on the instance is completely detroyed! So I do not get a second chance - once the server is locked out like this it has to be destroyed and a new one built. This is running on 32 bit Fedora Core 8. What is causing this and how can I fix it? Thanks.

2 2

Re: [Monetdb-developers] [MonetDB-users] MonetDB Server dies on connect
by Martin Kersten 13 Jun '08

13 Jun '08

Sjoerd, I was planning to try out the installers on my Vista machine. The previous version is v5.4.0 installed Uninstalling it gives the following error: There is a problem with this Windows Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor. You want specific information, or should i manually remove the C:\Program Files\CWI/MonetDB5 Sjoerd Mullender wrote: > Niels Nes wrote: > >> On Fri, Jun 13, 2008 at 09:41:50AM +0200, Robert Schumacher wrote: >> >>> I am using MonetDB server v5.4.0, based on kernel v1.22.0. >>> >>> Deleting those directories worked, thanks. >>> >> We are very interested in learning if this bug is fixed in >> the (tobe released) version (MonetDB v5.6.0 / sql 2.24 / kernel 1.24). >> >> Sjoerd are there allready windows installers for XP x64 to be tested? >> > > The following files are available for download. Note that these > installers are *completely untested*. > > Since the 64-bit version has a new so-called Upgrade Code (the code that > Windows uses to determine whether a package is an upgrade of an already > installed package), you need to first uninstall the old version and then > install the new. > > Note that the ODBC driver is no longer packaged in the MonetDB5-SQL > installer but only as a separate installer. Also note that you should > install both 32-bit and 64-bit versions of the ODBC installer on a > 64-bit platform. > > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB5-SQL-Installer-x86_64-200… > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB5-SQL-Installer-i386-20080… > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB-ODBC-Installer-x86_64-200… > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB-ODBC-Installer-i386-20080… > > The files with x86_64 in the names are 64-bit, the files with i386 in > the names are 32 bit. > > >> Niels >> >>> Robert >>> >>> -----Ursprüngliche Nachricht----- >>> Von: monetdb-users-bounces(a)lists.sourceforge.net [mailto:monetdb-users-bounces@lists.sourceforge.net] Im Auftrag von Niels Nes >>> Gesendet: Freitag, 13. Juni 2008 09:37 >>> An: Communication channel for MonetDB users >>> Betreff: Re: [MonetDB-users] MonetDB Server dies on connect >>> >>> On Tue, Jun 10, 2008 at 02:51:22PM +0200, Robert Schumacher wrote: >>> >>>> Hello, >>>> >>>> i was evaluating MonetDB. Got the Server running finr on Windows XP x64. I created about 12 Tabled by using a JDBC Client, then the Server died instantly. Since then when I start the Server it dies asap when I either start the Windows Client or perform a JDBC connection. I get no error log, no message, nothing. Has anybody an idea how to get MonetDB to stop crashing on connect? >>>> >>>> >>> Which version of MonetDB are you using? As you seem to have hit a bug, >>> the only solution out of the crash seems to be to remove the dbfarm (and >>> xquery_logs/sql_logs) directories. >>> >>> Niels >>> >>>> Thanks, >>>> Robert >>>> >>>> Robert Schumacher, Senior Software Developer >>>> >>>> prevero Systems GmbH >>>> Landsberger Straße 154 >>>> 80339 München >>>> >>>> Tel +49 89 455 77 - 0 >>>> Fax +49 89 455 77 - 200 >>>> >>>> Registergericht: Amtsgericht München, HRB 143809 -- Geschäftsführer: Matthias Thurner >>>> >>>> Bitte beachten Sie unsere Veranstaltungshinweise unter www.prevero.de >>>> >>>> >>>> >>>> ------------------------------------------------------------------------- >>>> Check out the new SourceForge.net Marketplace. >>>> It's the best place to buy or sell services for >>>> just about anything Open Source. >>>> http://sourceforge.net/services/buy/index.php >>>> _______________________________________________ >>>> MonetDB-users mailing list >>>> MonetDB-users(a)lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>>> >>> -- >>> >>> Niels Nes, Centre for Mathematics and Computer Science (CWI) >>> Kruislaan 413, 1098 SJ Amsterdam, The Netherlands >>> room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 >>> url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl >>> >>> ------------------------------------------------------------------------- >>> Check out the new SourceForge.net Marketplace. >>> It's the best place to buy or sell services for >>> just about anything Open Source. >>> http://sourceforge.net/services/buy/index.php >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >>> ------------------------------------------------------------------------- >>> Check out the new SourceForge.net Marketplace. >>> It's the best place to buy or sell services for >>> just about anything Open Source. >>> http://sourceforge.net/services/buy/index.php >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > ------------------------------------------------------------------------ > > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users >

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql_optimizer.mx, , 1.183, 1.184
by Romulo Goncalves 12 Jun '08

12 Jun '08

Milena Ivanova wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29777 > > Modified Files: > sql_optimizer.mx > Log Message: > Switch off the recycler. Why? We activated it to test it during night on other platforms. It helps to see if there is any problem from one day to another and it also helps to detect problems on the code. If you made a change and you broke tests, you should not commit until until you make sure that the existent tests (SQL or M5) are green. It is with these steps that we can get stable code and avoid bugs... Romulo > > > U sql_optimizer.mx > Index: sql_optimizer.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/sql_optimizer.mx,v > retrieving revision 1.183 > retrieving revision 1.184 > diff -u -d -r1.183 -r1.184 > --- sql_optimizer.mx 6 Jun 2008 20:25:15 -0000 1.183 > +++ sql_optimizer.mx 12 Jun 2008 15:33:12 -0000 1.184 > @@ -455,9 +455,10 @@ > "joinPath," > "mergetable," > "commonTerms," > +/* "replicator," */ > "accumulators," > "deadcode," > - "recycle," > +/* "recycle," */ > "reduce," > "garbageCollector," > "multiplex"; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins

2 1

Re: [Monetdb-developers] Works fine once, then get Invalid credentials for user monetdb
by Rt Ibmer 12 Jun '08

12 Jun '08

Its been a day of great frustration trying to get this going so I hope someone will have some ideas here. I decided to take a shot at upgrading just by running yum again pointing at the stable night - the upgrade process appeared to run just fine. However it was all downhill from there. Once I had that installed, I then tried to start merovingian and upon doing so received the following error message: merovingian cannot open config file /var/tmp/monet.GNU.32.32.d.5761/MonetDB5/.GNU.32.32.d-Fedora8/etc/monetdb5.conf I have no idea why it would be looking at that path. So anyway just to try and make progress I then created the above directory and put a copy of monetdb5.conf in there - and then the error message went away. However, when I try to run mclient I am now getting this 'sql' not initialized mclient -lsql --time --database=demo !MALException:setScenario:Scenario 'sql' not initialized Before all I had to do was run merowingian and then I could use the above mclient command ok, but not any longer. Then I decided to try with mserver5 and it cannot find the sql libraries so it seems: # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'demo' # Compiled for i686-redhat-linux-gnu/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ #WARNING: LoaderException:loadLibrary:sql: loader error lib_logger.so.0: cannot open shared object file: No such file or directory !MALException:MAL.getAddress:address of 'sql.SQLsession' not found !MALException:MAL.getAddress:address of 'sql.SQLsession2' not found .... snip (many more error lines like the above) So then I decided to try everything from scratch. I built a new server, installed the stable nightly from scratch, and then I wound up with the exact same as everything I wrote above. What am I doing wrong? Thanks!

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/common stream.mx, , 1.161, 1.162
by Sjoerd Mullender 12 Jun '08

12 Jun '08

Was this in the correct branch? Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/common > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31463/common > > Modified Files: > stream.mx > Log Message: > fix compilation > > > U stream.mx > Index: stream.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/common/stream.mx,v > retrieving revision 1.161 > retrieving revision 1.162 > diff -u -d -r1.161 -r1.162 > --- stream.mx 9 Jun 2008 05:25:25 -0000 1.161 > +++ stream.mx 12 Jun 2008 11:02:14 -0000 1.162 > @@ -2964,7 +2964,7 @@ > ns->errnr = OPEN_ERROR; > /* blocksizes have a fixed little endian byteorder */ > #ifdef WORDS_BIGENDIAN > - s->byteorder = 3412; /* simply != 1234 */ > + ns->byteorder = 3412; /* simply != 1234 */ > #endif > ns->type = s->type; > ns->access = s->access; > @@ -3057,7 +3057,7 @@ > return NULL; > /* blocksizes have a fixed little endian byteorder */ > #ifdef WORDS_BIGENDIAN > - s->byteorder = 3412; /* simply != 1234 */ > + ns->byteorder = 3412; /* simply != 1234 */ > #endif > ns->type = orig->type; > ns->access = orig->access; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] Works fine once, then get Invalid credentials for user monetdb
by Rt Ibmer 12 Jun '08

12 Jun '08

OK this is very odd... I do this: mkdir -p /mnt/MonetDB5/dbfarm merovingian monetdb create demo monetdb start demo and then start mclient: mclient -lsql --time --database=demo at that point everything is just fine and I can enter sql commands and all works great. I am using the default monetdb/monetdb for uname/pass. However once I exit out of this (\q) and try to go back in using the same command: mclient -lsql --time --database=demo It then comes back with: InvalidCredentialsException:checkCredentials:Invalid credentials for user 'monetdb' no matter how many times I try. So it works the very first time, but then not again. It seems like the very first onc to get a connection to the monetdb server is ok, but then any time after that it gets locked out somehow. What can be causing this? Also on a related note what is the monetdb5.secret file actually used for? I am also confused by this: # If you use Merovingian, a per-database vault key is used and this # setting ignored. #monet_vault_key=/etc/monetdb.secret So where then does the per-database vault key reside? And do I even need to mess with this? Thanks!

2 1

[Monetdb-developers] My little adventure using Sun Studio 12
by Fabian Groffen 11 Jun '08

11 Jun '08

With Sun Studio 12 cc compiler you' won't get far on MonetDB's sources. I had to unset all flags. Just a FYI: /opt/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/ufs/fabian/scratch/monetdb/stable/buildtools/burg -c gram.c "/ufs/fabian/scratch/monetdb/stable/buildtools/burg/zalloc.c", line 14: warning: statement not reached /opt/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/ufs/fabian/scratch/monetdb/stable/common/src/gdk -I../.. -I/ufs/fabian/scratch/monetdb/stable/common/src/gdk -I../common -I/ufs/fabian/scratch/monetdb/stable/common/src/gdk/../common -DLIBGDK -g -D_REENTRANT -c gdk_bbp.c -KPIC -DPIC -o .libs/libbat_la-gdk_bbp.o "gdk_posix.h", line 102: warning: macro redefined: __USE_BSD "/ufs/fabian/scratch/monetdb/stable/common/src/gdk/gdk_bbp.mx", line 1497: identifier redeclared: unshare current : static function(int) returning void previous: function(int) returning int : "/usr/include/bits/sched.h", line 79 (The Linux installation is somewhere on volund (/tmp/Linux), and the CDROM is on my desk, if someone is interested.)

1 0

[Monetdb-developers] crackers on the stable branch
by Romulo Goncalves 11 Jun '08

11 Jun '08

Lately I had to compile the stable branch several times in the optimized mode. Crackers is not a feature to be used in the stable branch, so why do we have crackers there? Its compilation takes too long. Could we remove it? Stratos you are the one who should decide... ;) Thanks, Romulo

2 2

[Monetdb-developers] Increasing speed of inserts?
by Rt Ibmer 10 Jun '08

10 Jun '08

I have a Java app that needs to write single inserts into the db in an extremely fast manner (it is receiving data in real time, and buffering it, then doing single inserts to feed the data into the db as fast as it will insert it). I am doing this using JDBC and this resides on the same physical box as MonetDB 5 latest version. Short of increasing/changing my hardware or changing my code around to bulk write inserts (which I assume would help?) is there anything I can do to improve the speed at which MonetDB can insert records into the db? For this one particular table it is about 160 columns. Maybe only about 25% of the columns are written to in any particular insert. This is literraly the type of thing where the requests are coming in so fast that it needs the db to keep up. I'm sure there is room for improvement in terms of the hardware and even the way we are writing the single inserts (i.e. maybe writing like 10 at a time or something). But short of that, is there any configuration type of options or optimization settings within MonetDB that could help increase the speed over what I am getting now? Thanks!

2 1

[Monetdb-developers] New default configure options for MonetDB
by Stefan Manegold 09 Jun '08

09 Jun '08

Dear MonetDB developer and users, For convenience of both developers and users as well as to comply even more with open source standards, we now set/use the following defaults for the configure options --enable-strict, --enable-assert, --enable-debug, --enable-optimize (Unless specified differently, the same defaults are used for both the "Stable" release branches and the "Current" development trunk.) When compiling from CVS sources (as mainly done by developers): strict=yes assert=yes debug=yes optimize=no (*) When compiling from packaged/distributed sources (i.e., tarballs) (as mainly done by users): strict=no assert=no debug=no optimize=no (*) For building binary distributions (RPMs, MSIs): strict=no assert=no debug=no optimize=yes With nightly testing for the "Stable" release branches: strict=yes assert=no debug=no optimize=yes With nightly testing for the "Current" development trunk: strict=yes assert=yes debug=no optimize=yes (*) IMPORTANT NOTE: Since "--enable-optimize=yes" is no longer the default for any case except binary packages, it is *strongly recommended* to (re)compile everything from scratch, *explicitly configured* with "--enable-debug=no --enable-optimize=yes", in case you want/need to run any performance experiments with MonetDB! (Please note: "--enable-X=yes" is equivalent to "--enable-X", and "--enable-X=no" is equivalent to "--disable-X".) Don't hesitate to ask in case you have any questions! Enjoy! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] script postgresSQL to Monetdb
by Roberto De Virgilio 09 Jun '08

09 Jun '08

Hi i have this problem.. i have to convert a dump file (.sql) of a PostgreSQL Db into a dump file for MonetDB to create the same DB of postgres into MonetDB... do you help me with a script? thanks a lot ____________________________________________________________________________________________________________ Roberto De Virgilio Ph.D Università Roma Tre Dipartimento di Informatica e Automazione Via della Vasca Navale 79, 00146 Roma Phone: +39-06-55173229 Fax: +39-06-5573030 E-mail: rde79(a)yahoo.com Home Page: http://mais.dia.uniroma3.it/rodevirg Room: 219

1 0

[Monetdb-developers] windows problem...
by Romulo Goncalves 09 Jun '08

09 Jun '08

I removed all the dbfarm, sql_logs and all the lib_* files.... Clean installation and I continue to get: E:\MonetDB\Current\sql\NT>RunMserver5.bat !WARNING: GDKlockHome: created directory E:\MonetDB\Current\\sql\NT\dbfarm\demo\ !WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !WARNING: BBPdir: initializing BBP. # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'demo' # Compiled for i686-pc-win32/64bit with 64bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see E:\MonetDB\Current\MonetDB5\NT\etc\monetdb5.conf) !FATAL: MALException:addUser:user 'monetdb' already exists E:\MonetDB\Current\sql\NT> It seems the bats are somewhere, the problem is where. The search of windows does not detect any dbfarm... Any suggestion? Romulo

2 1

[Monetdb-developers] Fastest way to calculate median?
by Rt Ibmer 09 Jun '08

09 Jun '08

Hi - I have an int column with several million rows of data. What is the fastest way to write a query to get the median value of the column? Apparently there is no median() function in SQL. A web search revealed lots of various talk about using stored procedures and fancy functions. None of this looked like it would be very efficient. I searched these archives but didn't see a single mention of median. Thank you!

2 1

[Monetdb-developers] Bug report - COPY RECORDS and encoded data
by Rt Ibmer 09 Jun '08

09 Jun '08

I was going crazy trying to figure out why my COPY RECORDS command wasn't working. Turns out it is caused from funny (encoded?) characters in the string for one of the fields. For instance one of the fields contained this "?a?as?e??, 18 ?a??? 2001 10:24:00 pµ" How did the data get this way? Well I exported it from SQL Server and I guess that db is set up to store encoded characters. When I did the export I told SQL Server to make the file ASCII since I figured that is what COPY RECORDS expects. So apparently SQL Server is writing the characters this way into the string. Then MonetDB (latest version) cannot handle this. So what happens is that the COPY x RECORDS command just returns immediately after I try this with this data in it. After that all my SQL statements in mclient return immediately. So I can say "select count(*) from foo;" and it just comes immediately back with the sql> prompt. So something with these funny characters sends mclient into a funk. I have to exit and restart it to get it going again. Any advice on how to handle this? Thank you.

3 2

[Monetdb-developers] Calculate disk space for table?
by Rt Ibmer 09 Jun '08

09 Jun '08

Hi - how can I tell how much disk storage space on of my tables is using? I tried one rough way but not sure if it is accurate. Basically I did a linux du command and looked at the farmdb dir root, then imported about 2M rows, then looked a tthe du number again. But it only went up about 1.5MB for 2M rows of data? There are about 150 columns but only about 30-40 columns for any given row are populated. Certainly this would be fantastic if the compression is really this good! Seems possible, but I wasn't sure if this technique was accurate and if there was a better way. Also on a related note - is there any detailed information on what types of compression and encoding MonetDB uses. For instance does it use Run Length Encoding or deltas or other such methods? And are there query plans / optimizers of any sort that you can feed MonetDB and it will optimize the layout or create internal subtables to help speed the types of queries we will be making? Thank you!

3 3

[Monetdb-developers] Division by zero on this query - why?
by Rt Ibmer 08 Jun '08

08 Jun '08

Any ideas why the following query would result in a division by zero exception? sql> select prodcode, avg(price) from sales group by prodcode; !ArithmeticException:batcalc./:Division by zero Timer 24326.296 msec 0 rows Thank you.

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pf_support.mx, XQuery_0-24, 1.299.2.3, 1.299.2.4
by Stefan Manegold 07 Jun '08

07 Jun '08

On Fri, Jun 06, 2008 at 09:00:53PM +0000, Lefteris Sidirourgos wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv9671/runtime > > Modified Files: > Tag: XQuery_0-24 > pf_support.mx > Log Message: > A rather aggresive fix for the order of attributes and nodes in updates. > > Now we reverse in algebra the order of insert as last and insert before at runtime like the MPS. This fix the bug [1981844] PF/alg: result order differs from mps. Also it fix the test Update/update.xq (which makes unnecessary the algebra specific output). Also other failing tests of updates are fixed in tests/BugTrack. > > However, now the test insert_test_order_seq fails:( However, this is a different error that should be addressed with a diffrent fix. > > To be continued... > > > U pf_support.mx > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.299.2.3 > retrieving revision 1.299.2.4 > diff -u -d -r1.299.2.3 -r1.299.2.4 > --- pf_support.mx 29 May 2008 16:45:15 -0000 1.299.2.3 > +++ pf_support.mx 6 Jun 2008 21:00:46 -0000 1.299.2.4 > @@ -3321,9 +3321,27 @@ > { > var inserts := command.ord_select(UPDATE_INSERT_FIRST, UPDATE_REPLACENODE); # [i,insertcommand] > if (inserts.count() > 0) { > + > + # UPDATE_INSERT_LAST and UPDATE_INSERT_BEFORE commands must be executed in reverse order > + var order := inserts.ord_uselect(UPDATE_INSERT_LAST, UPDATE_INSERT_BEFORE).copy().mark(0@0); > + if (order.count() > 1) { > + pre_tgt.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(pre_tgt).access(BAT_WRITE).revert().tmark(0@0))); > + pre_cont_tgt.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(pre_cont_tgt).access(BAT_WRITE).revert().tmark(0@0))); > + attr_tgt.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(attr_tgt).access(BAT_WRITE).revert().tmark(0@0))); > + attr_cont_tgt.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(attr_cont_tgt).access(BAT_WRITE).revert().tmark(0@0))); > + pre_ins.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(pre_ins).access(BAT_WRITE).revert().tmark(0@0))); > + pre_cont_ins.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(pre_cont_ins).access(BAT_WRITE).revert().tmark(0@0))); > + attr_ins.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(attr_ins).access(BAT_WRITE).revert().tmark(0@0))); > + attr_cont_ins.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(attr_cont_ins).access(BAT_WRITE).revert().tmark(0@0))); > + inserts.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(inserts).access(BAT_WRITE).revert().tmark(0@0))); > + command.access(BAT_WRITE).replace(order.leftjoin(order.mirror().leftjoin(command).access(BAT_WRITE).revert().tmark(0@0))); > + } Lefteris, thanks for the fix! I'm not completely sure, whether I understand what your above code does, or better, whether I understand how is does what *I think* it is supposed to do, i.e. revert the order of BUNs that are related to UPDATE_INSERT_LAST and UPDATE_INSERT_BEFORE commands in all BATs. However given my idea of what it is supposed to do, wouldn't the following solution look "simpler" and be more efficient? ======== var order := inserts.ord_uselect(UPDATE_INSERT_LAST, UPDATE_INSERT_BEFORE); if (order.count() > 1) { var redro := order.copy().access(BAT_WRITE).revert().access(BAT_READ); var order_redro := order.mark(0@0).leftfetchjoin(redro.tmark(0@0)); pre_tgt .access(BAT_WRITE).replace( order_redro.leftjoin(pre_tgt) ).access(BAT_READ); pre_cont_tgt .access(BAT_WRITE).replace( order_redro.leftjoin(pre_cont_tgt) ).access(BAT_READ); attr_tgt .access(BAT_WRITE).replace( order_redro.leftjoin(attr_tgt) ).access(BAT_READ); attr_cont_tgt.access(BAT_WRITE).replace( order_redro.leftjoin(attr_cont_tgt) ).access(BAT_READ); pre_ins .access(BAT_WRITE).replace( order_redro.leftjoin(pre_ins) ).access(BAT_READ); pre_cont_ins .access(BAT_WRITE).replace( order_redro.leftjoin(pre_cont_ins) ).access(BAT_READ); attr_ins .access(BAT_WRITE).replace( order_redro.leftjoin(attr_ins) ).access(BAT_READ); attr_cont_ins.access(BAT_WRITE).replace( order_redro.leftjoin(attr_cont_ins) ).access(BAT_READ); inserts .access(BAT_WRITE).replace( order_redro.leftjoin(inserts) ).access(BAT_READ); command .access(BAT_WRITE).replace( order_redro.leftjoin(command) ).access(BAT_READ); } ======== ... just an idea; might be completely wrong ... let's think about anc check it tomorrow or "live" on Monday ... Good night > + > var inserts_pre_ins := inserts.mirror().leftfetchjoin(pre_ins); # [i,PRE] > var inserts_elems := inserts_pre_ins.ord_uselect(oid_nil, oid_nil).hmark(0@0); # [j,i] inserts on elements > var inserts_attrs := inserts_pre_ins.ord_uselect(oid_nil).hmark(0@0); # [k,i] inserts on attributes > + > + > if (inserts_attrs.count() > 0) { > do_update_insert_attributes(ws, inserts_attrs.leftjoin(pre_tgt), inserts_attrs.leftjoin(pre_cont_tgt), inserts_attrs.leftjoin(attr_ins), inserts_attrs.leftjoin(attr_cont_ins)); > } > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql_optimizer.mx, , 1.182, 1.183 sql_scenario.mx, , 1.308, 1.309
by Stefan Manegold 06 Jun '08

06 Jun '08

On Fri, Jun 06, 2008 at 08:25:18PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv27738 > > Modified Files: > sql_optimizer.mx sql_scenario.mx > Log Message: > The first nightly test of the recycler ... will have to wait until Sunday: To have more time for (hopefully) migrating MonetDB testing to a new "server", I decided to run only Stable testing tonight, and "skip" Current testing for one night, assuming that Stable testing is currently more "mission-critical" than Current testing. Sorry for the inconveniences this might cause ... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal recycle.mx, , 1.27, 1.28
by Romulo Goncalves 06 Jun '08

06 Jun '08

Milena Ivanova wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv27013/mal > > Modified Files: > recycle.mx > Log Message: > Reset is called after an update instruction to clean the recycle cache. > > > U recycle.mx > Index: recycle.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/recycle.mx,v > retrieving revision 1.27 > retrieving revision 1.28 > diff -u -d -r1.27 -r1.28 > --- recycle.mx 5 Jun 2008 08:37:43 -0000 1.27 > +++ recycle.mx 6 Jun 2008 14:00:22 -0000 1.28 > @@ -70,7 +70,7 @@ > command getCachePolicy():int > address RECYCLEgetCache; > > -pattern reset():void > +pattern reset(bid:bat):void Is the optimized aware of this change? It might call for the command with wrong number of arguments and it might break several sql tests... Romulo > address RECYCLEresetCMD > comment "Reset off all recycled variables"; > > @@ -78,11 +78,6 @@ > address RECYCLEshutdownWrap > comment "Clear the recycle cache"; > > -command restore():void > -address RECYCLErestore > -comment "Restore the recycle cache to the start of the call"; > - > - > command monitor(f:int):void > address RECYCLEmonitor > comment "start/stop the monitoring (printing) of the recycler info (storage size used and number of statements retained)"; > @@ -207,7 +202,7 @@ > > /* and dump the statistics */ > if (recycleBlk) > - for(i=0; i< recycleBlk->stop; i++){ > + for(i=1; i< recycleBlk->stop; i++){ > stream_printf(cntxt->fdout,"# %4d lru= %8d cnt= %8d ticks= %9d rd= %9d wr= %9d %s\n", i, > recycleBlk->profiler[i].clk, > recycleBlk->profiler[i].counter, > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

[Monetdb-developers] Default of "--enable-strict/--disable-strict" in release branch
by Stefan Manegold 06 Jun '08

06 Jun '08

Dear all, I do see the point that we do not necessarily distribute released source tarballs with "--enable-strict" as default, as this might "unnecessarily" hinder compilation with compilers and/or environments that "we don't know yet". However, nightly testing does explicitly use --enable-strict both both the development trunk ("Current") and the release branch ("Stable") to ensure "correctness" of the code. This leads to the "inconvenient" situation that, during bug fixing, developers that do not explicitly use "--enable-strict" compile with the default "--disable-strict" and consequently might miss small (but "nasty") problems that then cause nightly testing to fail. To solve this "dilemma", I came up with the idea to make the default setting "more flexible" and "automatically detect" whether a developer is compiling from CVS or a user is compiling from source tarball(s). In the former case, the default is "--enable-strict", it the latter it is "--disable-strict". Here's my patch: ======== --- buildtools/conf/MonetDB.m4 24 May 2008 17:13:41 -0000 1.75.2.1 +++ buildtools/conf/MonetDB.m4 6 Jun 2008 06:14:32 -0000 @@ -17,7 +17,15 @@ dnl Defaults that differ between development trunk and release branch: AC_DEFUN([AM_MONETDB_DEFAULTS], [ -dft_strict=no +if test -f "$srcdir"/vertoo.data; then + dnl Developers compiling from CVS: + dnl default is --enable-strict + dft_strict=yes +else + dnl Users compiling from source tarball(s): + dnl default is --disable-strict + dft_strict=no +fi dft_assert=no dft_optimi=yes dft_warning=no ======== In case there are no (strong) objections, I'd like to check this in later today. Open question: shall we only do this on the release branch and keep "--enable-strict" as default in all cases in the development trunk, or shall we apply the same strategy o the development trunk, too? Yours, Stefan On Fri, Jun 06, 2008 at 06:45:45AM +0200, Testweb Compilation wrote: > opt_icol.c:376: warning: enumeration value 'la_dummy' not handled in switch > > 374 if (L(p)->kind == la_twig && > 375 !PFprop_icol (p->prop, L(p)->sem.iter_item.item)) > 376 switch (LL(p)->kind) { > 377 > 378 case la_docnode: > > /home/monet/testing/monet.GNU.32.32.d.2271/pathfinder/compiler/algebra/opt/opt_icol.c:376: warning: enumeration value 'la_string_join' not handled in switch > /home/monet/testing/monet.GNU.32.32.d.2271/pathfinder/compiler/algebra/opt/opt_icol.c:376: warning: enumeration value 'la_dummy' not handled in switch > make[8]: *** [libopt_la-opt_icol.lo] Error 1 > > http://www.ins.cwi.nl/~monet/permastore/2008-06-06/Stable/make-pathfinder-G… > > CVS committers: boncz(a)cwi.nl jan.rittinger(a)in.tum.de lsidir(a)cwi.nl manegold(a)cwi.nl sjoerd(a)cwi.nl zhang(a)cwi.nl > > An identical error message was detected on the following systems: > GNU.32.32.d-Debian4.0 GNU.32.32.d-Fedora6 GNU.32.32.d-Fedora8 GNU.64.32.d-Fedora8 GNU.64.64.d-Fedora8 GNU.64.64.s-Fedora8 > > -- > At this time you cannot configure Testing mail, sorry > You are receiving this mail because: > - you checked in some code on pathfinder (Stable) > - you are a fixed recipient > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 8

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_debugger.mx, , 1.227, 1.228
by Niels Nes 05 Jun '08

05 Jun '08

On Thu, Jun 05, 2008 at 11:44:25AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14824 > > Modified Files: > mal_debugger.mx > Log Message: > void double properties display bug fixes go ..... Niels > > > U mal_debugger.mx > Index: mal_debugger.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_debugger.mx,v > retrieving revision 1.227 > retrieving revision 1.228 > diff -u -d -r1.227 -r1.228 > --- mal_debugger.mx 20 May 2008 07:33:38 -0000 1.227 > +++ mal_debugger.mx 5 Jun 2008 11:44:21 -0000 1.228 > @@ -1674,10 +1674,8 @@ > } else if (isaBatType(n->type) && v) { > BAT *b = BATdescriptor(v->val.ival); > > - if (b) { > - printBATproperties(f,b); > + if (b) > BBPunfix(b->batCacheid); > - } > } > } > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

[Monetdb-developers] MAPI cache bug: explained
by Stefan de Konink 05 Jun '08

05 Jun '08

Hi, Already two days I am very annoyed with the MAPI interface, having 'random' fallout in mapi_fetch_field. With high confidence of trust in my own code I decided to walk the road of debugging the Mapiclient library and I'm reporting here. Please point your editors at clients/src/mapilib/Mapi.mx Around line 4800 you will find the function mapi_fetch_row. This is a public function and gets borked randomly but there seems to be a correlation between the amount of results fetched, lets move on. I was able to find the source of the problem in the while loop, where mapi_fetch_line is accessed. This function is the first place this function can return 0. This seems to be consistent with my logs. So what is this mapi_fetch_line all about? Around line 4260, you can find it. The idea of this function would be not to return NULL. After some debugging it seems that the function on the 'bug' occasion always returns 0. Why? If we explore the if statement something interesting can be found: mapi_fetch_line bypass: 709 + 91 < 800 = 0 result->cache.first + result->cache.tuplecount < result->row_count That seems to make sense, but it also prevents the mapi_fetch_line function to be processed. So what is not doing its job? The logical answer would be the function above mapi_fetch_line_internal. In this cute function there is an interesting check: if (mid->active != hdl || hdl->needmore) I was a bit distracted here, but when I explicitly have set a check on both constraints it seems the mid->active != hdl is the failure here. And I guessed it would be possible that mid->active (or actually hdl->active) was in fact equal to NULL. When I read about the MapiHdl active; MapiHdl active; /* set when not all rows have been received */ Isn't it strange that needmore is actually 0, but we would still require an active handle? I thought this would suggest the replacement of || with &&, but even with the assert in read_into_cache removed, this comes out with even MORE failed data. I hope I have outlined the problem in sufficient details so that one of the core developers can look where possible failures of hdl->mid->active takes place. If someone wants to see the live failures he is welcome to ask a private login. Yours Sincerely, Stefan de Konink

2 1

[Monetdb-developers] BAT-leak in gdk_logger.mx ?
by Stefan Manegold 04 Jun '08

04 Jun '08

Niels, Sjoerd, Peter, after (quite) some time of debugging for [ 1976341 ] XQ: leftovers after deleting document http://sourceforge.net/tracker/index.php?func=detail&aid=1976341&group_id=5… I found the --- or at least one possible --- cause for this bug. In pathfinder, newly shredded read-only documents are added to the logger using logger_add_bat() --- let's ignore for the time being, that I am a bit puzzled by this, since I though / was told, pathfinder uses check-pointing instead of the logger (WAL) for shredding documents. Obviously, logger_add_bat() increments the ref-count of the logged BATs (BBPincref()). However, logger_del_bat() does not seem to decrements the ref-count, again, once a BAT is removed from the logger. This way, document BATs keep the logger_add_bat()-increased ref-count even after a pf:del-doc() and stay around until the Mserver is shutdown. The below patch seem to "fix" this problem. However, since I'm everything but familiar with the logger or the usage of it in either XQuery or SQL, I'd rather have you double-check the patch before I'd check it in. (Of course I'll also run "private" testing of the whole suite overnight to see whether my patch causes any (obvious) harm. Thank you very much in advance! Stefan ======== Index: MonetDB/src/gdk/gdk_logger.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_logger.mx,v retrieving revision 1.17.2.1 diff -u -r1.17.2.1 gdk_logger.mx --- MonetDB/src/gdk/gdk_logger.mx 27 May 2008 20:33:39 -0000 1.17.2.1 +++ MonetDB/src/gdk/gdk_logger.mx 3 Jun 2008 23:27:15 -0000 @@ -1721,6 +1721,7 @@ void logger_del_bat(logger *lg, log_bid bid) { + BBPdecref(bid, TRUE); BUNdelHead(lg->catalog, &bid, FALSE); } ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/BugDay_2005-10-06_2.9.3/Tests All, SQL_2-24, 1.10, 1.10.4.1 having.SF-922614.stable.out, SQL_2-24, 1.3, 1.3.14.1
by Stefan Manegold 03 Jun '08

03 Jun '08

On Tue, Jun 03, 2008 at 07:03:54AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/test/BugDay_2005-10-06_2.9.3/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28722 > > Modified Files: > Tag: SQL_2-24 > All having.SF-922614.stable.out > Log Message: > approved new output > > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/BugDay_2005-10-06_2.9.3/Tests/All,v > retrieving revision 1.10 > retrieving revision 1.10.4.1 > diff -u -d -r1.10 -r1.10.4.1 > --- All 11 Dec 2007 22:19:06 -0000 1.10 > +++ All 3 Jun 2008 07:03:52 -0000 1.10.4.1 > @@ -1,7 +1,7 @@ > jdbc_crashes_on_getTables.SF_944316 > variable_arg_substring.SF_942981 > UNION_JOIN_vs_plain_set_UNION-2.SF-920585 > -huge_expression_and_column_name.SF-921173 > +#huge_expression_and_column_name.SF-921173 will not be fixed in the MonetDB 5.6.0, SQL 2.24 release > CrashMe_SQL_server_crash-2.SF-921673 > CrashMe_SQL_server_crash-1.SF-921673 > inserts_with_defaults.SF-921977 I just prevented the propagation of this change ;-) Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/snodgrass/Tests All, SQL_2-24, 1.4, 1.4.10.1
by Stefan Manegold 03 Jun '08

03 Jun '08

On Tue, Jun 03, 2008 at 07:13:37AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/test/snodgrass/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv559/src/test/snodgrass/Tests > > Modified Files: > Tag: SQL_2-24 > All > Log Message: > disabled test (will not be fixed in this release) > > THIS SHOULD NOT BE PROPAGATED!! propagation has been prevented. Stefan > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/snodgrass/Tests/All,v > retrieving revision 1.4 > retrieving revision 1.4.10.1 > diff -u -d -r1.4 -r1.4.10.1 > --- All 25 Jan 2007 01:28:36 -0000 1.4 > +++ All 3 Jun 2008 07:13:35 -0000 1.4.10.1 > @@ -1 +1 @@ > -cast_select > +#cast_select will not be fixed in the MonetDB 5.6.0, SQL 2.24 release > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_recycler.mx, MonetDB_5-6, 1.23.2.1, 1.23.2.2
by Romulo Goncalves 02 Jun '08

02 Jun '08

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24665 > > Modified Files: > Tag: MonetDB_5-6 > opt_recycler.mx > Log Message: > Since we may introduce new instructions and variables, > the recycled[] object may have been too small. The recycler is feature and it is used in the current branch. It is not tested in the stable and it might stop to be compiled in the stable. All the implementation related to the recycler should be checked in at the current one and not in the stable one. This avoids problems in the stable and saves time in the recycler development (I do not need to wait for the propagations to have an actualized code)... I am going to revert this change, and do it in the current Note: Martin is aware of this... :) Regards, Romulo > > > U opt_recycler.mx > Index: opt_recycler.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_recycler.mx,v > retrieving revision 1.23.2.1 > retrieving revision 1.23.2.2 > diff -u -d -r1.23.2.1 -r1.23.2.2 > --- opt_recycler.mx 31 May 2008 16:29:54 -0000 1.23.2.1 > +++ opt_recycler.mx 2 Jun 2008 20:26:08 -0000 1.23.2.2 > @@ -107,7 +107,8 @@ > > (void) cntxt; > (void) stk; > - recycled= GDKzalloc(sizeof(char)*mb->vtop); > + /* watch out, instructions may introduce new variables */ > + recycled= GDKzalloc(sizeof(char)*mb->vtop*2); > setLifespan(mb,span); > limit= mb->stop; > old = mb->stmt; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_recycle.mx, MonetDB_5-6, 1.75.2.2, 1.75.2.3
by Romulo Goncalves 02 Jun '08

02 Jun '08

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28477 > > Modified Files: > Tag: MonetDB_5-6 > mal_recycle.mx > Log Message: > Protect the garbage collector against removing NULL bats. The recycler is feature and it is used in the current branch. It is not tested in the stable and it might stop to be compiled in the stable. All the implementation related to the recycler should be checked in at the current one and not in the stable one. This avoids problems in the stable and saves time in the recycler development (I do not need to wait for the propagations to have an actualized code)... I am going to revert this change, and do it in the current Note: Martin is aware of this... :) Regards, Romulo > > > U mal_recycle.mx > Index: mal_recycle.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_recycle.mx,v > retrieving revision 1.75.2.2 > retrieving revision 1.75.2.3 > diff -u -d -r1.75.2.2 -r1.75.2.3 > --- mal_recycle.mx 31 May 2008 20:06:48 -0000 1.75.2.2 > +++ mal_recycle.mx 2 Jun 2008 17:55:04 -0000 1.75.2.3 > @@ -273,7 +273,8 @@ > for(j=0; j< q->argc; j++){ > v= &getVarConstant(mb,getArg(q,j)); > if(getArgType(mb,q,j)==TYPE_bat || isaBatType(getArgType(mb, q,j)) ){ > - BBPdecref(ABS(v->val.bval), TRUE); > + if( v->val.bval || !BBP_lrefs(v->val.bval)) > + BBPdecref(ABS(v->val.bval), TRUE); > v->val.bval= 0; > } > if( v->vtype == TYPE_str) { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/BugTracker/Tests authorization.SF-1430616-drop_user.sql, SQL_2-24, NONE, 1.1.2.1 authorization.SF-1430616.SQL.bat, SQL_2-24, 1.3, 1.3.4.1 authorization.SF-1430616.SQL.sh, SQL_2-24, 1.2, 1.2.4.1
by Stefan Manegold 02 Jun '08

02 Jun '08

On Mon, Jun 02, 2008 at 03:17:02PM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/test/BugTracker/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8577 > > Modified Files: > Tag: SQL_2-24 > authorization.SF-1430616.SQL.bat > authorization.SF-1430616.SQL.sh > Added Files: > Tag: SQL_2-24 > authorization.SF-1430616-drop_user.sql > Log Message: > If you create the user (voc) do not forget to drop it. > > This was the reason to break some following tests which were trying to create user voc... That's indeed the better solution than my renaming user "voc" to "voc2" in the later test. Thanks! --- I was simply "too lazy" to check all the subsequent test that output some catalog statistics, and hence, might need to have their output approved, too. Did you check that? Stefan > Thanks Stefan for dectecting the problem... > > > --- NEW FILE: authorization.SF-1430616-drop_user.sql --- > START TRANSACTION; > > DROP USER "voc"; > DROP SCHEMA "voc"; > > commit; > > U authorization.SF-1430616.SQL.sh > Index: authorization.SF-1430616.SQL.sh > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh,v > retrieving revision 1.2 > retrieving revision 1.2.4.1 > diff -u -d -r1.2 -r1.2.4.1 > --- authorization.SF-1430616.SQL.sh 29 Oct 2007 20:11:51 -0000 1.2 > +++ authorization.SF-1430616.SQL.sh 2 Jun 2008 15:16:59 -0000 1.2.4.1 > @@ -17,4 +17,11 @@ > > Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -q" > > +cat << EOF > .monetdb > +user=monetdb > +password=monetdb > +EOF > + > +Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/authorization.SF-1430616-drop_user.sql\"" > + > rm -f .monetdb > > U authorization.SF-1430616.SQL.bat > Index: authorization.SF-1430616.SQL.bat > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat,v > retrieving revision 1.3 > retrieving revision 1.3.4.1 > diff -u -d -r1.3 -r1.3.4.1 > --- authorization.SF-1430616.SQL.bat 29 Oct 2007 20:11:51 -0000 1.3 > +++ authorization.SF-1430616.SQL.bat 2 Jun 2008 15:16:58 -0000 1.3.4.1 > @@ -15,4 +15,9 @@ > > call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -q > > +echo user=monetdb> .monetdb > +echo password=monetdb>> .monetdb > + > +call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\authorization.SF-1430616-drop_user.sql" > + > @del .monetdb > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra/opt opt_join_pd.c, XQuery_0-24, 1.46.2.1, 1.46.2.2
by Stefan Manegold 02 Jun '08

02 Jun '08

Jan, Sjoerd, I'll take care of these. Sjoerd, please do not propagate until I'm done. Stefan On Mon, Jun 02, 2008 at 08:44:32AM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra/opt > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1643/algebra/opt > > Modified Files: > Tag: XQuery_0-24 > opt_join_pd.c > Log Message: > -- circumvent bugs #1981491 and #1908822 for the release. > > This 'hack' does not fix the real problem (that we have to get rid > of bit-encoded column names), but just disables some optimizations > in case we might get into trouble. Thus we can apply less rewrites > and get 'suboptimal' plans. > > THIS IS A RELEASE ONLY CHECKIN --- PLEASE DO NOT PROPAGATE! > > > U opt_join_pd.c > Index: opt_join_pd.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/opt/opt_join_pd.c,v > retrieving revision 1.46.2.1 > retrieving revision 1.46.2.2 > diff -u -d -r1.46.2.1 -r1.46.2.2 > --- opt_join_pd.c 28 May 2008 11:37:25 -0000 1.46.2.1 > +++ opt_join_pd.c 2 Jun 2008 08:44:30 -0000 1.46.2.2 > @@ -580,6 +580,11 @@ > } break; > > case la_project: > + /* disable rewrite in case we might get wider relations > + than we can map back to the bit-encoded (ori) column > + names */ > + if (PFarray_last (rproj) + L(lp)->schema.count < 30) > + > /* Arbitrary projections are pushed through the join operator > by: > 1.) merging the projection lists of the projection > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] geom/src/monetdb5 geom.mx, Geom_0-4, 1.13.4.1, 1.13.4.2
by Sjoerd Mullender 30 May '08

30 May '08

Stefan Manegold wrote: > Update of /cvsroot/monetdb/geom/src/monetdb5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv30443/src/monetdb5 > > Modified Files: > Tag: Geom_0-4 > geom.mx > Log Message: > > attempt to fix compilation problem > " > geom/src/monetdb5/geom.mx:791: error: initialization discards qualifiers from pointer target type > " > with gcc 4.3.0 on Gentoo > > > U geom.mx > Index: geom.mx > =================================================================== > RCS file: /cvsroot/monetdb/geom/src/monetdb5/geom.mx,v > retrieving revision 1.13.4.1 > retrieving revision 1.13.4.2 > diff -u -d -r1.13.4.1 -r1.13.4.2 > --- geom.mx 28 May 2008 09:43:48 -0000 1.13.4.1 > +++ geom.mx 30 May 2008 04:58:29 -0000 1.13.4.2 > @@ -788,7 +788,7 @@ > GEOSGeom geosGeometry = wkb2geos(*geom); > > if (geosGeometry) { > - GEOSCoordSeq gcs = GEOSGeom_getCoordSeq(geosGeometry); > + GEOSCoordSeq gcs = (GEOSCoordSeq)GEOSGeom_getCoordSeq(geosGeometry); > > /* we could also check if geom is a > LineString, LinearRing or Point */ > > It seems to me a better fix is to keep the qualifier and to declare gcs as const. -- Sjoerd Mullender

1 0

[Monetdb-developers] HOST_NAME_MAX op mac os 10.5
by Ying Zhang 29 May '08

29 May '08

Dear all, I use HOST_NAME_MAX in my code. Compiling the code on a CWI machine does not have any problem, however, compilation on Mac OS 10.5 complains that HOST_NAME_MAX is not defined. fgrep on mac 10.5 says: $fgrep -r HOST_NAME_MAX /usr/include/* /usr/include/limits.h:#define _POSIX_HOST_NAME_MAX 255 /usr/include/unistd.h:#define _SC_HOST_NAME_MAX 72 How (and where) can I add proper check to define HOST_NAME_MAX for darwin 9? Thanks! Jennie

3 3

Re: [Monetdb-developers] Error in compilation of PF-ROX
by Peter Boncz 29 May '08

29 May '08

Hi Stefan, Riham PF_ROX has the new indexing code, that Lefteris has tried to describe in his recent paper. This code was only checked for correct operation from MIL in the code generated by the pf-rox java program of Riham. The correct operation of the new indexing code when activated automatically from mps was not tested yet. It may well be the source of trouble. i will pick this up once I have some time again (..). Peter Dear all, I checked all (modified) files in the PF_ROX branch by hand, and all but runtime/pathfinder.mx & runtime/pf_support.mx (this means in particular the compiler/mil/milprint_summer.c I fixed the other day) are OK, i.e., they contain all (intended) changes applied in the PF_ROX branch as well as all chnages that were made on the development trunk since the creation of the PF_ROX branch (and hence properly propagated). For runtime/pathfinder.mx & runtime/pf_support.mx, I am not yet sure, whether everything is OK. There have been numerous "interleaved" (and partly "redundant" or "conflicting") changes both genuine on the PF_ROX branch and propagated from the development trunk, and the (very tedious) analysis requires more time than I could spent today. More to come "soon", I hope ... Unfortunately, while only 50 (100) tests fail in the new stable branch (and in the development trunk), more than 500 tests seem to fail in the PF_ROX branch --- also this requires "some" more ananlysis ... Stefan

1 0

[Monetdb-developers] Error in compilation of PF-ROX
by Riham Abdel Kader 28 May '08

28 May '08

Hi all, I'm getting the following error while compiling PF-ROX: make[5]: Entering directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler/mil' /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I. -I../include -I./../include -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/common -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/gdk -DLIBMIL -g -O2 -Wall -Wextra -std=c99 -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -Wno-declaration-after-statement -c -o libmil_la-milprint_summer.lo `test -f 'milprint_summer.c' || echo './'`milprint_summer.c gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I. -I../include -I./../include -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/common -I/local/kader/home/workspace_pathfinder/MonetDB_XQuery/include/MonetDB/gdk -DLIBMIL -g -O2 -Wall -Wextra -std=c99 -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -Wno-declaration-after-statement -c milprint_summer.c -fPIC -DPIC -o .libs/libmil_la-milprint_summer.o milprint_summer.c: In function 'translateFunction': milprint_summer.c:8518: error: expected expression before 'else' make[5]: *** [libmil_la-milprint_summer.lo] Error 1 make[5]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler/mil' make[4]: *** [install] Error 2 make[4]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler/mil' make[3]: *** [install-recursive] Error 1 make[3]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler' make[2]: *** [install] Error 2 make[2]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder/compiler' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/local/kader/home/workspace_pathfinder/MonetDB_XQuery/pathfinder' make: *** [install] Error 2 I checked out all needed modules yesterday night. Best regards, Riham -- Address: Database Group, Dept of EEMCS, University of Twente, POBox 217, 7500 AE Enschede, The Netherlands Phone: +31 (0)53 489 4628 Fax: +31 (0)53 489 2927 Email: r.abdelkader(a)ewi.utwente.nl

3 4

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal/Tests All, MonetDB_5-6, 1.97, 1.97.2.1
by Stefan Manegold 27 May '08

27 May '08

Propagation has been prevented. Stefan On Tue, May 27, 2008 at 04:51:01PM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv7156 > > Modified Files: > Tag: MonetDB_5-6 > All > Log Message: > If the recycle is still in developing, so the tests should be deactivated on the stable testing... > > Note:Do not progate this change to the current branch > > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/Tests/All,v > retrieving revision 1.97 > retrieving revision 1.97.2.1 > diff -u -d -r1.97 -r1.97.2.1 > --- All 22 Mar 2008 08:55:24 -0000 1.97 > +++ All 27 May 2008 16:50:59 -0000 1.97.2.1 > @@ -222,10 +222,10 @@ > tst901b > # > > -recycle00 > -recycle01 > -recycle02 > -recycle03 > +#recycle00 > +#recycle01 > +#recycle02 > +#recycle03 > venks > > saveload > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/Tests All, SQL_2-24, 1.51, 1.51.2.1
by Stefan Manegold 27 May '08

27 May '08

Propagation has been prevented. Stefan On Tue, May 27, 2008 at 04:48:00PM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/test/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv5851 > > Modified Files: > Tag: SQL_2-24 > All > Log Message: > Deactivated > crashme > 50ways > > These are tests to be tested on the current branch and not in the stable. > In the previous release we had the same procedure... > > Note: Do not propagated this change to the current one... > > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/Tests/All,v > retrieving revision 1.51 > retrieving revision 1.51.2.1 > diff -u -d -r1.51 -r1.51.2.1 > --- All 6 Mar 2008 08:33:16 -0000 1.51 > +++ All 27 May 2008 16:47:58 -0000 1.51.2.1 > @@ -88,8 +88,8 @@ > setoptimizer > load_dec_as_int > string > -NOT_WIN32?crashme > -NOT_WIN32?50ways > +#NOT_WIN32?crashme > +#NOT_WIN32?50ways > load_with_offset > copy_into > antiselect > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_recycler.mx, , 1.23, 1.24
by Stefan Manegold 27 May '08

27 May '08

Milena, is this a (bug-?) "fix" that should (have) go(ne) into the upcoming release (and hence release branch)? Stefan On Tue, May 27, 2008 at 09:37:31AM +0000, Milena Ivanova wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17027/optimizer > > Modified Files: > opt_recycler.mx > Log Message: > Fixed chain of recycled instructions. > > > U opt_recycler.mx > Index: opt_recycler.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_recycler.mx,v > retrieving revision 1.23 > retrieving revision 1.24 > diff -u -d -r1.23 -r1.24 > --- opt_recycler.mx 20 May 2008 07:20:19 -0000 1.23 > +++ opt_recycler.mx 27 May 2008 09:37:29 -0000 1.24 > @@ -160,17 +160,18 @@ > if( getFunctionId(p)== selectRef && recycled[getArg(p,1)] ) > p->recycle = -1; > if( getFunctionId(p)== uselectRef && recycled[getArg(p,1)] && > - !ATOMvarsized( getGDKType( getArgType(mb,p,1))) > - ){ > - q= copyInstruction(p); > - getArg(q,0)= newTmpVariable(mb,TYPE_any); > - setFunctionId(q, selectRef); > + !ATOMvarsized( getGDKType( getArgType(mb,p,1)))){ > + q = copyInstruction(p); > + getArg(q,0)= newTmpVariable(mb,TYPE_any); > + setFunctionId(q, selectRef); > q->recycle = -1; > + recycled[getArg(q,0)] = 1; > pushInstruction(mb,q); > - getArg(p,1)= getArg(q,0); > + getArg(p,1) = getArg(q,0); > setFunctionId(p,markTRef); > - p->argc=2; > - p->recycle = -1; > + p->argc = 2; > + p->recycle = -1; > + recycled[getArg(p,0)] = 1; > } > @- > The sql.bind instructions should be handled carefully > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 2

[Monetdb-developers] best practices with connection pool
by Stefan de Konink 27 May '08

27 May '08

Hi, Maybe I should move this to the users part, if so, please inform me. What is currently the best practice of connecting to MonetDB5/SQL? Is there much overhead in the connection phase (thus: cache the connection) or can a developer just reconnect each time the database connection is required without much overhead? The target platform is extremely demanding in requests and memory, so I would like to design it good from the start. Stefan

2 1

[Monetdb-developers] MonetDB release planning
by Stefan Manegold 25 May '08

25 May '08

Dear fellow MonetDB developers, this is to keep you up-to-date about the short-term release plans for the MonetDB product family as they have recently been discussed here at CWI in A'dam. Please read until the end. Thank you very much in advance for your patience ans cooperation! Triggered by (1) the major progress in extending the functionality of the new "Algebra back-end" based version of Pathfinder (MonetDB/XQuery) to (almost) meet the functionality as provided by the (til date) default "milprint_summer back-end" based version (see details below), and (2) the desire of at least one distinct client to use the new "Algebra back-end" based version of Pathfinder (MonetDB/XQuery), we plan to have a new release of Pathfinder (MonetDB/XQuery), soon. Unfortunately, the functionality of the new "Algebra back-end" based version of Pathfinder (MonetDB/XQuery) is not yet identical with that of the (til date) default "milprint_summer back-end" based version. First of all, extensions beyond the XQuery and XQuery Updates standards, such as XRpc, StandOff, and pre-compiled XQuery modules ("MIL modules") are not supported, as well as recursive user-defined functions are not supported, yet. (The not yet working id/idref functionality is currently being implemented.) Moreover, the XQuery Update functionality is not yet complete/correct, as it cannot yet handle inserts of node who's type is not statically known at query compile time. In particular the latter shortcoming currently keeps us from our original plans of making the new "Algebra back-end" the default in the upcoming release. Hence, most probably, the upcoming release will still come with the "milprint_summer back-end" set as default. However, as already in the latest release, runtime options allow users to easily enable the new "Algebra back-end". Some (internal) API changes on the development trunk since the last release (mainly the signatures of GDK functions BAT_select(), BATmmap() & BATmadvice()), prohibit a new "feature release" of pathfinder, only, as this is incompatible with currently available releases of MonetDB4 (Server) and MonetDB (Common). Hence, also this upcoming release will be a "complete" one, i.e., all MonetDB related packages (MonetDB Common, Clients, MonetDB4 Server, MonetDB5 Server, MonetDB/SQL, MonetDB/Geom & MonetDB/XQuery will (have to be) released. Given this, our plan is as follows: Within the next week, we will fork new "Stable" branches of the current development trunks. Please let us know at your earliest convenience, in case you have new feature developments "in the pipe", that should go into the new release. In case you have just started new "cutting edge" developments that are still far from mature to be released soon, please refrain from checking them in until the new release branches have been created. Once the new release branches are created, we plan to spend about one month on thoroughly testing and stabilizing the new release and its packaging. Any help from all of you is highly appreciated! In particular, we count on the support for all "first users" of the new Pathfinder (MonetDB/XQuery) "Algebra back-end' (even if it will not be the default choice). The envisioned release date is "End of June". Please let us know, in case you have any questions, comments, suggestions, etc. Yours Sincerely, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 20

[Monetdb-developers] Independant SQL engine?
by Sylvain Bertrand 22 May '08

22 May '08

Hi, I would like to know if your SQL engine is independant enough from MonetDB in order to let me reasonably write my own SQL engine storage backend? (Not I dislike MonetDB, just that for the moment I use a custom disk storage and would try to put a SQL engine on top of it) Best regards, Sylvain B. -- use single GPL licensed software, use Linux and secure your digital freedom!

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah serialize_pftijah.mx, , 1.60, 1.61
by Ying Zhang 15 May '08

15 May '08

Hi Jan, Your checkin breaks the compilation due to the following error: $SRCBASE_DIR/pathfinder/modules/pftijah/serialize_pftijah.mx: In function 'dbat_extend': $SRCBASE_DIR/pathfinder/modules/pftijah/serialize_pftijah.mx:175: error: too few arguments to function 'BATmmap' BATmmap has recently got an extra argument (please see Niels' checkin yesterday), so you might want to update your MonetDB modules firstly :) Regards, Jennie On May 15, 2008, at 11:59 , Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12778 > > Modified Files: > serialize_pftijah.mx > Log Message: > + repair pftijah direct memory acces problem which caused Henning's > vitalis > collection problems. > > > > U serialize_pftijah.mx > Index: serialize_pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/ > serialize_pftijah.mx,v > retrieving revision 1.60 > retrieving revision 1.61 > diff -u -d -r1.60 -r1.61 > --- serialize_pftijah.mx 17 Apr 2008 10:29:10 -0000 1.60 > +++ serialize_pftijah.mx 15 May 2008 09:59:43 -0000 1.61 > @@ -135,9 +135,22 @@ > #define MINCHUNK 8192 > #define MAXCHUNK 67108864 > > -int dbat_extend(dbat* dbat, oid min_i /*raw-index*/, size_t > forced_size) { > +static void > +dbat_setcount(dbat *dbat, oid n) > +{ > + BAT *b = dbat->bat; > + BATsetcount(b, n); > + if (b->ttype) { > + b->tsorted = 0; > + b->tdense = 0; > + } > +} > + > +static int > +dbat_extend(dbat* dbat, oid min_i /*raw-index*/, size_t > forced_size) { > size_t newsize; > > + dbat_setcount(dbat, dbat->raw_max); > if ( forced_size ) { > newsize = forced_size; > } else { > @@ -156,10 +169,19 @@ > return -1; > } > } > +#if 1 > + /* copied this form shredder.mx, keep it here ??? */ > + if (newsize+newsize > (size_t) REMAP_PAGE_MAXSIZE) { /* try to > use mmap() */ > + BATmmap(dbat->bat, STORE_MMAP, STORE_MMAP, STORE_MMAP); > + } > + > +#endif > + > #if 0 > stream_printf(GDKout,"dbat_extend[%s](%d -> %d)\n",dbat- > >name,dbat->raw_max,newsize); > #endif > - dbat->raw_max= newsize; > + > + dbat->raw_max = newsize; > dbat->seq_max = dbat->raw_max + dbat->seqbase; > if ( !(dbat->bat = BATextend(dbat->bat,newsize)) ) { > GDKerror("dbat_extend: BATextend[\"%s\"](to %d) fails > \n","incomplete",newsize); > @@ -169,17 +191,30 @@ > return 1; > } > > -int dbat_sizeHint(dbat* dbat, int sizeHint_mark) { > +#if 0 > +static int > +dbat_sizeHint(dbat* dbat, int sizeHint_mark) { > int sizeHint = sizeHint_mark - dbat->seqbase; > int estimate = dbat->raw_max + sizeHint; > > return dbat_extend(dbat, estimate, 0); > } > +#endif > > #define dbat_set_oid(DBAT,I,V) (DBAT)->cast.oidCAST[I-(DBAT)- > >seqbase] = V > > #define dbat_set_int(DBAT,I,V) (DBAT)->cast.intCAST[I-(DBAT)- > >seqbase] = V > > +#if 0 > +static void > +dbat_int_debug(char* t, dbat* dbat) { > + stream_printf(GDKout,"+ DEBUGGING INT DBAT[sz=%d]@=[%s]:\n",dbat- > >seq_max,t); > + for(int i=0; i<10; i++) { > + stream_printf(GDKout,"+ dbat[%d] = %d\n",i,dbat- > >cast.intCAST[i]); > + } > +} > +#endif > + > /************************************************ > * > * The Tijah-Pathfinder index creation context > @@ -391,7 +426,7 @@ > if ( (tjPre = tj_newPre(tjctx) ) == oid_nil ) > return 0; > dbat_set_oid(&tjctx->dbat_collPre, tjPre, termOid); > - dbat_set_int(&tjctx->dbat_collSize,(int)tjPre,0); > + dbat_set_int(&tjctx->dbat_collSize,tjPre,0); > #ifdef TJ_TRACE > if ( TJ_TRACE ) stream_printf(GDKout,"C[%s]:TERM: \"%s > \", termoid=%d, tjPre=%d\n",tjctx->name,term,termOid,tjPre); > #endif > @@ -1230,7 +1265,7 @@ > return GDK_FAIL; > if ( dbat_finalize(&tjctx->dbat_collSize, tjctx->tijahPre) < 0 ) > return GDK_FAIL; > - > + > tjctx->b_collPre->batDirty = TRUE; > tjctx->b_collSize->batDirty = TRUE; > tjctx->b_collPfPre->batDirty = TRUE; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

1 0

Re: [Monetdb-developers] php5 interface
by Martin Kersten 09 May '08

09 May '08

Michiel Kamermans wrote: > Martin, > > A test using windows XP with PHP 5.2.6 and MonetDB seems to work just > fine. I suspect that this means that in Vista monetDB won't be able to > interface with PHP either, as the core is essentially the same as the > one used in windows server 2008... I am not a windows expert myself > when it comes to figuring out API differences, so I wouldn't know > where to look... but I do have one improvement to the php file in the > /share/php dir: > > Right now it auto-assumes running on *nix, automatically calling > dl('monetdb.so'). A change for this line to the following would make > the script usable on both *nix and windows: > > $ext="monetdb"; > if (!extension_loaded($ext)) { > $prefix = (PHP_SHLIB_SUFFIX === 'dll') ? 'php_' : ''; > dl($prefix . $ext . '.' . PHP_SHLIB_SUFFIX); } > > Regards, Thank you very much for the information provided, we will take this suggestion into consideration > > Mike Kamermans > (who will now no doubt spend a nice bit of time in XP doing > benchmarking =)

1 0

[Monetdb-developers] [Fwd: Re: php5 interface]
by Martin Kersten 09 May '08

09 May '08

-------- Original Message -------- Subject: Re: php5 interface Date: Fri, 09 May 2008 19:13:40 +0200 From: Michiel Kamermans <mkamerma(a)science.uva.nl> To: Martin Kersten <Martin.Kersten(a)cwi.nl> References: <48248084.5060306(a)science.uva.nl> <4824835A.10101(a)cwi.nl> Hi Martin, I've tested this on windows server 2008 (which is basically vista, without all the vista bits stripped off), with the MonetDB installed using "MonetDB5-SQL-Installer-i386-20080214.msi". While technically the setup also involves Apache 2.2.8, the fact that PHP as standalone engine will crash means it doesn't really enter into the picture. I will see if I have the same problems in windows XP with PHP 5.2.6 and report back on the issue. Regards, -- Michiel Kamermans ,-------------------------------------------------------------- | Masters student AI at the University of Amsterdam [0024538] | http://www.nihongoresources.com `--------------------------------------------------------------

1 0

[Monetdb-developers] Reporting a bug
by Nadeem Iftikhar 07 May '08

07 May '08

Hi I tried the following link to report the bug, but it does not work http://monetdb.wiki.sourceforge.net/ Bug..... I need to insert some null values into a table via a procedure. I created the table and did insert two rows. One with (insert into) command and second with (insert into ..select) command. Both commands work well and I could see two rows added in result. When I used the same (insert into.. select) command in a procedure. It behaves in a strange fashion. It shows that a row has been affected, but does not show the newly added row. If the rows are counted, it gives the correct count which is (3) in this case but still I could not see the last row which is added via a procedure. ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' create sequence "ff" as integer start with 1; CREATE TABLE t1(timeid INT DEFAULT NEXT VALUE FOR "ff" PRIMARY KEY, a INT, b INT, c INT, d INT, e INT, f VARCHAR(20), g VARCHAR(20), h INT, i VARCHAR(10), j VARCHAR(10), k VARCHAR(10), l INT, m INT, n VARCHAR(10), o VARCHAR(10), p VARCHAR(10), q VARCHAR(15), r INT ); ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' And then insert a row ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' insert into t1(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r) values(1, 1, 1, 1, 1, 'first row', 'first row',1, 'first row', 'first row', 'first row',1,1, 'first row', 'first row', 'first row', 'first row', 1); select * from t1;sql>sql>sql>sql>more>more>more>more>more>more>more>more>more>more>more>more>more>more>more>more>more>more>more>sql> sql>Rows affected 1 sql> +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ | time |a |b |c |d |e |f |g |h |i |j |k |l |m |n |o |p |q |r | : id | | | | | | | | | | | | | | | | | | | +======+==+==+==+==+==+=====+=====+==+=====+======+======+==+==+======+======+======+======+==+ | 1 |1 |1 |1 |1 |1 |firs |firs |1 |firs |first |first |1 |1 |first |first |first |first |1 | : | | | | | |t |t | |t |row |row | | |row |row |row |row | | : | | | | | |row |row | |row | | | | | | | | | | +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Every thing is fine so far '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' sql>insert into t1(a, b, c, d, e, f, g, h) select a, b, c, d, e, f, g, h from t1 where timeid = 1; Rows affected 1 sql>select * from t1; +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ | time |a |b |c |d |e |f |g |h |i |j |k |l |m |n |o |p |q |r | : id | | | | | | | | | | | | | | | | | | | +======+==+==+==+==+==+=====+=====+==+=====+======+======+==+==+======+======+======+======+==+ | 1 |1 |1 |1 |1 |1 |firs |firs |1 |firs |first |first |1 |1 |first |first |first |first |1 | : | | | | | |t |t | |t |row |row | | |row |row |row |row | | : | | | | | |row |row | |row | | | | | | | | | | | 2 |1 |1 |1 |1 |1 |firs |firs |1 |null |null |null |n |n |null |null |null |null |n | : | | | | | |t |t | | | | |u |u | | | | |u | : | | | | | |row |row | | | | |l |l | | | | |l | : | | | | | | | | | | | |l |l | | | | |l | +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' inserted another row and it works fine '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' sql>create procedure tt() begin insert into t1(a, b, c, d, e, f, g, h) select a, b, c, d, e, f, g, h from t1 where timeid = 1; end; call tt(); select * from t1; more>more>more> sql>Rows affected 1 sql>+------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ | time |a |b |c |d |e |f |g |h |i |j |k |l |m |n |o |p |q |r | : id | | | | | | | | | | | | | | | | | | | +======+==+==+==+==+==+=====+=====+==+=====+======+======+==+==+======+======+======+======+==+ | 1 |1 |1 |1 |1 |1 |firs |firs |1 |firs |first |first |1 |1 |first |first |first |first |1 | : | | | | | |t |t | |t |row |row | | |row |row |row |row | | : | | | | | |row |row | |row | | | | | | | | | | | 2 |1 |1 |1 |1 |1 |firs |firs |1 |null |null |null |n |n |null |null |null |null |n | : | | | | | |t |t | | | | |u |u | | | | |u | : | | | | | |row |row | | | | |l |l | | | | |l | : | | | | | | | | | | | |l |l | | | | |l | +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' The problem starts now: when I try to insert the row via a procedure it shows that row effected 1 but does not show the row. ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' sql>select count(*) from t1; +--------+ | count_ | +========+ | 3 | +--------+ sql>select timeid from t1; +--------+ | timeid | +========+ | 1 | | 2 | | 3 | +--------+ sql>select * from t1; +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ | time |a |b |c |d |e |f |g |h |i |j |k |l |m |n |o |p |q |r | : id | | | | | | | | | | | | | | | | | | | +======+==+==+==+==+==+=====+=====+==+=====+======+======+==+==+======+======+======+======+==+ | 1 |1 |1 |1 |1 |1 |firs |firs |1 |firs |first |first |1 |1 |first |first |first |first |1 | : | | | | | |t |t | |t |row |row | | |row |row |row |row | | : | | | | | |row |row | |row | | | | | | | | | | | 2 |1 |1 |1 |1 |1 |firs |firs |1 |null |null |null |n |n |null |null |null |null |n | : | | | | | |t |t | | | | |u |u | | | | |u | : | | | | | |row |row | | | | |l |l | | | | |l | : | | | | | | | | | | | |l |l | | | | |l | +------+--+--+--+--+--+-----+-----+--+-----+------+------+--+--+------+------+------+------+--+ sql> ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' I could not see the 3rd row *********************************** Regards, Nadeem Iftikhar, Ph.D. Fellow Department of Computer Science, Aalborg University Selma Lagerlöfs Vej 300, DK-9220 Aalborg Øst, Denmark Phone: (+45) 9940 9830, fax: (+45) 9940 9798 E-mail: nadeem(a)cs.aau.dk; nadeem(a)ieee.org Home Page: http://www.cs.aau.dk/~nadeem ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, , 1.411, 1.412
by Lefteris 07 May '08

07 May '08

That was drastic:) On Tue, May 6, 2008 at 7:20 PM, Jan Rittinger <tsheyar(a)users.sourceforge.net> wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14794 > > Modified Files: > pathfinder.mx > Log Message: > -- Check whether Lefteris changes to ws_create() are complete :) > > > U pathfinder.mx > Index: pathfinder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v > retrieving revision 1.411 > retrieving revision 1.412 > diff -u -d -r1.411 -r1.412 > --- pathfinder.mx 6 May 2008 11:36:10 -0000 1.411 > +++ pathfinder.mx 6 May 2008 16:20:00 -0000 1.412 > @@ -1348,7 +1348,6 @@ > return bat(void,tpe).seqbase(PRE_BASE); # a constant bat > } > > -PROC ws_create() : BAT[void,bat] { return ws_create(0); } # still exists for algebra > PROC ws_create(int update) : BAT[void,bat] > { > # wsid = 1bit(0) | 31bits(unique-id) | 2bits(update) | 30bits(batid) > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java… > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins >

2 2

[Monetdb-developers] check-in "killed" pf/tijah
by Henning Rode 07 May '08

07 May '08

hej, one of the check-ins yesterday more or less "killed" pf/tijah. the test-set is now completely red for our module. since i also checked in one line of code yesterday, i tested already whether it was the cause of the trouble, but undoing my check-in does not solve anything. so, i suppose there must be some other reason. -henning

3 5

[Monetdb-developers] How to debug MonetDB project on Windows OS?
by 廉小伟 06 May '08

06 May '08

Hi.. I want to know about the method of debugging MonetDB on Windows OS. After compiering MonetDB use "namke" , how can I debug it. Is special debug tool needed?

1 0

[Monetdb-developers] RDF
by Bellekens, P.A.E. 18 Apr '08

18 Apr '08

Dear MonetDB developers, I have a question related to RDF. In you PDF online, I see that it is possible to import RDF in the MonetDB system, although there is not much further information concerning RDF input. Could you please inform me a bit more about: - is it possible to upload RDF data? - is there any kind of RDF specific querying? - is it possible to run MonetDB as a backend in a triple store like for example in Sesame? - is there any performance measuring done for RDF querying? It is known that column stores have a good performance (c-store benchmarks) for RDF data as well... With kind regards, Pieter

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal rdf.mx, , 1.2, 1.3
by Lefteris 17 Apr '08

17 Apr '08

On Thu, Apr 17, 2008 at 5:24 PM, Marco Antonelli <marcoantonelli(a)users.sourceforge.net> wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26814/src/modules/mal > > Modified Files: > rdf.mx > Log Message: > Added a T bat that stores the types of the RDF objects (IRI, integer, double, plain literal...). It still has to be sorted together with the S, P, O bats in all their possible combinations > > U rdf.mx > Index: rdf.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf.mx,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- rdf.mx 1 Apr 2008 14:48:08 -0000 1.2 > +++ rdf.mx 17 Apr 2008 15:24:15 -0000 1.3 > +static rdf_type RDFparseXsdDataType(char *uri) > +{ > + rdf_type ret; > + if (uri == NULL) > + ret = PLAIN; > + else if ( strcmp(uri, "http://www.w3.org/2001/XMLSchema#int") || > + strcmp(uri, "http://www.w3.org/2001/XMLSchema#integer")) > + ret = INTEGER; > + else if (strcmp(uri, "http://www.w3.org/2001/XMLSchema#decimal")) > + ret = DECIMAL; > + else if (strcmp(uri, "http://www.w3.org/2001/XMLSchema#double")) > + ret = DOUBLE; > + else if (strcmp(uri, "http://www.w3.org/2001/XMLSchema#boolean")) > + ret = BOOLEAN; > + else if (strcmp(uri, "http://www.w3.org/2001/XMLSchema#float")) > + ret = FLOAT; > + else if (strcmp(uri, "http://www.w3.org/2001/XMLSchema#string")) > + ret = STRING; > + else if (strcmp(uri, "http://www.w3.org/2001/XMLSchema#dateTime")) > + ret = DATETIME; > + else > + ret = UNKNOWN; > + > + raptor_free_memory(uri); > + return ret; > +} I am not sure if this is the correct way to identify the type, what happens if 2001 changes to 1998 or 2003? or if they use prefix instead of the uri? Does raptor gives you always the full uri back? it seems way to much hard wired. > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java… > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

2 3

[Monetdb-developers] MonetDB clients asks for MonetDB common 1.22.0
by Marco Antonelli 17 Apr '08

17 Apr '08

Hello, I'm building Monet on my laptop, but the clients module during configure stops saying: checking whether MonetDB Common version 1.22.0 or newer is installed... no: found only version 1.20.0 configure: error: Clients requires at least version 1.22.0 of MonetDB, but only version 1.20.0 was found. I checked out from cvs as told here on the sourceforce page: http://sourceforge.net/cvs/?group_id=56967 how may I fix it? Marco

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/mil mil_dce.c, , 1.19, 1.20 milgen.brg, , 1.142, 1.143
by Jan Rittinger 17 Apr '08

17 Apr '08

The answer is simple: kind=165 is your new tape operator and its printing is missing in print_statement(). Your tape is no expression with a return value but a statement with side effects. Thus the MIL grammar in milprint.c expects the handling of your tape in print_statement() instead of print_expression() -- see also the grammar at the top of milprint.c (as the error message suggests :)). Jan On 04/17/2008 09:27 AM, Jan Flokstra wrote with possible deletions: > Hi Jan, > > When I remove the if_() around my tj_docmgmt_tape() function in milgen.brg I > get the following error message. [...] > > Jan. > > # End rule: "Rel: std_sort(Rel)" > a0001 := a0006; > a0004 := a0001.project (str(nil)); > a0002 := a0007; > a0003 := a0001.project (str(nil)); > DocmgmTape (ws, a0004, a0002, a0003, a0001); > fatal error: I encountered problems while generating MIL output. > fatal error: This is possibly due to an illegal MIL tree, not conforming to > the grammar in milprint.c > fatal error: This is how far I was able to generate the script: > notice: node: ALG_tj_docmgmt_tape > fatal error: Illegal MIL tree. MIL printer screwed up (kind: 165). > # halted in milprint.c (print_statement), line 596 -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_scanselect.mx, , 1.50, 1.51
by Sjoerd Mullender 14 Apr '08

14 Apr '08

On 2008-04-14 21:08, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv3941 > > Modified Files: > gdk_scanselect.mx > Log Message: > don't typemap dbl/flt to lng/int in scanselect bunfastins > (this is an attempt to fix the compilation problems with gcc 4.3 and up > which have very strict c99 aliasing rules ) > > > U gdk_scanselect.mx > Index: gdk_scanselect.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_scanselect.mx,v > retrieving revision 1.50 > retrieving revision 1.51 > diff -u -d -r1.50 -r1.51 > --- gdk_scanselect.mx 6 Mar 2008 08:25:38 -0000 1.50 > +++ gdk_scanselect.mx 14 Apr 2008 19:08:34 -0000 1.51 > @@ -271,6 +271,8 @@ > @:T_BUNfastins(@1,@2,@3,sht,SIMPLE,sht)@ > @:T_BUNfastins(@1,@2,@3,int,SIMPLE,int)@ > @:T_BUNfastins(@1,@2,@3,lng,SIMPLE,lng)@ > + @:T_BUNfastins(@1,@2,@3,flt,SIMPLE,flt)@ > + @:T_BUNfastins(@1,@2,@3,dbl,SIMPLE,dbl)@ > @:T_BUNfastins(@1,@2,@3,fix,FIXEDATOM,(b)->ttype)@ > @:T_BUNfastins(@1,@2,@3,var,VARATOM,(b)->ttype)@ > @- > @@ -282,6 +284,8 @@ > @:H_BUNfastins(sht,SIMPLE,sht)@ > @:H_BUNfastins(int,SIMPLE,int)@ > @:H_BUNfastins(lng,SIMPLE,lng)@ > +@:H_BUNfastins(flt,SIMPLE,flt)@ > +@:H_BUNfastins(dbl,SIMPLE,dbl)@ > @:H_BUNfastins(vid,VID,vid)@ > @:H_BUNfastins(fix,FIXEDATOM,(b)->htype)@ > @:H_BUNfastins(var,VARATOM,(b)->htype)@ > @@ -296,21 +300,6 @@ > #define @2@1_bunfastins_nocheck(b, p, h, t, hs, ts) @3@1_bunfastins_nocheck(b, p, h, t, hs, ts) > #define @2@1_bunfastins(b, h, t) @3@1_bunfastins(b, h, t) > @h > -@:HT_bunfastins_typemap(void,flt,int)@ > -@:HT_bunfastins_typemap(chr,flt,int)@ > -@:HT_bunfastins_typemap(sht,flt,int)@ > -@:HT_bunfastins_typemap(int,flt,int)@ > -@:HT_bunfastins_typemap(lng,flt,int)@ > -@:HT_bunfastins_typemap(fix,flt,int)@ > -@:HT_bunfastins_typemap(var,flt,int)@ > -@:HT_bunfastins_typemap(void,dbl,lng)@ > -@:HT_bunfastins_typemap(chr,dbl,lng)@ > -@:HT_bunfastins_typemap(sht,dbl,lng)@ > -@:HT_bunfastins_typemap(int,dbl,lng)@ > -@:HT_bunfastins_typemap(lng,dbl,lng)@ > -@:HT_bunfastins_typemap(fix,dbl,lng)@ > -@:HT_bunfastins_typemap(var,dbl,lng)@ > - > #if SIZEOF_OID == SIZEOF_INT > @:HT_bunfastins_typemap(void,oid,int)@ > @:HT_bunfastins_typemap(chr,oid,int)@ > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java… > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins I came up with the same fix today which I hadn't checked in yet. But I went one further: I also did oid and wrd. They may well cause the same warning/error. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra builtins.c, , 1.84, 1.85 logical.c, , 1.93, 1.94
by Jan Rittinger 10 Apr '08

10 Apr '08

Hi Jan, your code looks quite good. Picky as I am I however have some comments :) Jan On 04/10/2008 03:38 PM, Jan Flokstra wrote with possible deletions: > [...] > U builtins.c > Index: builtins.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/builtins.c,v > retrieving revision 1.84 > retrieving revision 1.85 > diff -u -d -r1.84 -r1.85 > --- builtins.c 3 Apr 2008 09:42:11 -0000 1.84 > +++ builtins.c 10 Apr 2008 13:38:38 -0000 1.85 > @@ -44,6 +44,7 @@ > #include <assert.h> > #include <stdio.h> > > +#include "mem.h" ^^^^^ what do you need that for? > + > +/* > + * PFTIJAH defines. I decided not make a seperate include file for > + * pftijah so ensure that these defines are exactly the same as in > + * ../mil/milgen.brg > + */ that is ugly in my eyes! -- You could place your stuff in algebra.h like e.g., the axis steps instead. > + > +#define MYNODEKIND aat_pnode > +#define DOCMGMTTYPE aat_docmgmt > + > +#define PFT_FUN(F) (strncmp(F,"pftijah_",8)==0) > + > +#define PFT_QUERY_N_XX "pftijah_query_n_xx" > +#define PFT_QUERY_N_SX "pftijah_query_n_sx" > +#define PFT_QUERY_N_XO "pftijah_query_n_xo" > +#define PFT_QUERY_N_SO "pftijah_query_n_so" > +#define PFT_QUERY_I_XX "pftijah_query_i_xx" > +#define PFT_QUERY_I_SX "pftijah_query_i_sx" > +#define PFT_QUERY_I_XO "pftijah_query_i_xo" > +#define PFT_QUERY_I_SO "pftijah_query_i_so" > + > +#define PTF_QUERY_NODES(N) (N[14]=='n') > +#define PTF_QUERY_STARTNODES(N) (N[16]=='s') > +#define PTF_QUERY_OPTIONS(N) (N[17]=='o') Encoding all information in a single string is even more ugly!!! You have a context (ctx) where you can store arbitrary information. Just build your own struct (in algebra.h) and make it less error-prone. > + > +PFalg_schema_t pft_empty_schema(PFalg_simple_type_t item_t) { > + PFalg_schema_t schema; > + schema.count = 3; > + // INCOMPLETE, what about freeing this space ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ we use garbage collection. > +struct PFla_pair_t pft_query_param2(struct PFla_pair_t *p1, PFalg_simple_type_t itemType1, struct PFla_pair_t *p2, PFalg_simple_type_t itemType2) { > [...] > +struct PFla_pair_t pft_query_param3(struct PFla_pair_t *p1, PFalg_simple_type_t itemType1, struct PFla_pair_t *p2, PFalg_simple_type_t itemType2, struct PFla_pair_t *p3, PFalg_simple_type_t itemType3) { textwidth > 80 -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

1 0

Re: [Monetdb-developers] RDF
by Marco Antonelli 10 Apr '08

10 Apr '08

Bellekens, P.A.E. wrote: > The link to the PDF is: > > monetdb.cwi.nl/Assets/m5manual.pdf > > There (if you search on 'rdf') you find a picture in which RDF is mentioned as input. Although I do not exactly know what the context there is. As you may see from the picture, the RDF box is dashed, since is still not there... > Cheers and good luck, > Pieter Thank you Pieter, Marco > -----Original Message----- > From: Marco Antonelli [mailto:M.Antonelli@cwi.nl] > Sent: woensdag 9 april 2008 15:56 > To: Bellekens, P.A.E. > Cc: monetdb-developers(a)lists.sourceforge.net > Subject: Re: [Monetdb-developers] RDF > > Hello Pieter, > > I'm the master thesis student who started the SPARQL front-end for > MonerDB this January. > > Bellekens, P.A.E. ha scritto: >> Dear MonetDB developers, >> >> I have a question related to RDF. >> In you PDF online, I see that it is possible to import RDF in the MonetDB system, >> although there is not much further information concerning RDF input. > > may you send a link to the PDF? > >> Could you please inform me a bit more about: >> - is it possible to upload RDF data? > > Actually it is possible to import and export RDF in MonetDB, but this > functionality will be under heavy development starting from the next week. > If you want to get a look at the code, the import-export functions are > defined in MonetDB5/src/modules/mal/rdf.mx. > It is not compiled by default, to getting it compiled you have to modify > the Makefile.ag file in the same directory, then run 'bootstrap', > 'configure', 'make' and 'make install'. > >> - is there any kind of RDF specific querying? > > Still not, the SPARQL compiler is under development too > >> - is it possible to run MonetDB as a backend in a triple store like for example in Sesame? > > Not at the moment, but MonetDB will be a backend for Sesame in the next > future > >> - is there any performance measuring done for RDF querying? >> It is known that column stores have a good performance (c-store benchmarks) for RDF data as well... > > Lefteris and Romulo can answer you on these topics more in detail than me > Anyway MonetDB/SQL performed well in querying RDF data :) > > Best regards, > Marco Antonelli > >> With kind regards, >> Pieter >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >> Don't miss this year's exciting event. There's still time to save $100. >> Use priority code J8TL2D2. >> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java… >> _______________________________________________ >> Monetdb-developers mailing list >> Monetdb-developers(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > No virus found in this incoming message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.22.11/1368 - Release Date: 9-4-2008 16:20 > > > No virus found in this outgoing message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.22.11/1368 - Release Date: 9-4-2008 16:20 >

1 0

[Monetdb-developers] dbname....
by Romulo Goncalves 08 Apr '08

08 Apr '08

I did a mistake in my scripts to start my mserver and I set my dbname like this: /scratch5/romulo/var/dbfarm/ When I start my mserver I get this information: -bash-3.1$ ../server.sh optimized # MonetDB server v5.5.0, based on kernel v1.23.0 # Serving database '/scratch5/romulo/var/dbfarm' # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved Until now no problem. The problem comes when I connect a mclient: !SABAOTHgetMyStatus: MALException:sabaoth.assert:SABAOTHgetMyStatus couldn't find a database for /scratch5/romulo/var/dbfarm! I decided to debug the code and there I decided to print the following variables: printf("%s , %s \n ", GDKgetenv("gdk_dbfarm"), GDKgetenv("gdk_dbname")); Result: /scratch5/romulo/var/dbfarm , /scratch5/romulo/var/dbfarm Until now no problem, so what is the reason for the crash? Well, I decide to check my dbfarm: /scratch5/romulo/var/dbfarm There I found out that the directory with name "/scratch5/romulo/var/dbfarm" does not exist, but instead of that sub-directories were created: /scratch5/romulo/var/dbfarm/scratch5/romulo/var/dbfarm/ My question is: Is the location of the database correct just because in the dbname I defined the name like a path? Or should that name be interpreted as a single string? Regards, Romulo PS: Next time I will be more be careful with my scripts :)

2 1

[Monetdb-developers] embedded mserver...
by Romulo Goncalves 08 Apr '08

08 Apr '08

I start the mserver with the following command: ./mserver5 --set embedded=yes --dbinit="include sql;" & mclient connection is refused. The I decided to set the mapi_port: --set mapi_port=50000 & Mclient connection is again refused for port 50000: [goncalve@amelia bin]$ mclient -lsql !ERROR mapi_reconnect: connect: Connection refused Setup connection failed [goncalve@amelia bin]$ and for port 33177 isn't: [goncalve@amelia bin]$ mclient -lsql -P33177 User (goncalve): [goncalve@amelia bin]$ As you can see the above example the user used for the connection is goncalve and not monetdb. Why the default user is not monetdb? And why this port? This example (the first command to start mserver) was taken from our web-site... http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Embedded-Server.ht… Regards, Romulo

2 6

[Monetdb-developers] [Fwd: ! Compiling 'Current' pathfinder: Results DIFFER from 2008.04.02_00-09-03's !]
by Jan Rittinger 03 Apr '08

03 Apr '08

How do I have to interpret this result? -- Testing was not possible due to ... (e.g., sf)? Jan -------- Original-Nachricht -------- Betreff: ! Compiling 'Current' pathfinder: Results DIFFER from 2008.04.02_00-09-03's ! Datum: Thu, 3 Apr 2008 02:20:39 +0200 (CEST) Von: monet(a)cwi.nl (Project account) An: jan.rittinger(a)in.tum.de, E.Sidirourgos(a)cwi.nl, Stefan.Manegold(a)cwi.nl, Sjoerd.Mullender(a)cwi.nl Compilation results of pathfinder in /ufs/monet/Testing/Current DIFFER from those of 2008.04.02_00-09-03 ! Summary: ======== * 2 Systems no longer relevant !! 12 Systems now DON'T produce output !! * 4 Systems still don't produce output * 55 Tests no longer relevant Maybe YOUR checkins since 02.04.2008, 00:09:03 caused some of the problems: buildtools MonetDB java clients MonetDB4 MonetDB5 pathfinder jan.rittinger(a)in.tum.de lsidir(a)cwi.nl Check http://monetdb.cwi.nl/Development/TestWeb/Current/pathfinder/ for details ! Details: ======== 2 Systems no longer relevant: F6-I64/32d, F6-I64/64d 12 Systems now DON'T produce output: D4.0-G32/32d, F6-G32/32d, F8-G32/32d, F8-G64/32d, F8-G64/64d, F8-G64/64s, F8-I64/32d, F8-I64/64d, G1.12.11.1-G32/32d, R4-I64/64d, S5.10-G32/32d, W5.1-I32/32d 4 Systems still don't produce output: D8.11.0-G32/32d, W5.1-M32/32d, W5.2-I64/64d, W5.2-M64/64d 55 Tests no longer relevant: conf/bin/monetdb-xquery-config, conf/bin/monetdb-xquery-config.bat, compiler/bin/pfsql, compiler/bin/pf, compiler/bin/pfopt, runtime/lib/MonetDB4/lib/lib_pf_standoff.so, runtime/lib/MonetDB4/lib/lib_pf_support.so, runtime/lib/MonetDB4/pf_standoff.mil, runtime/lib/MonetDB4/pathfinder.mil, runtime/lib/MonetDB4/xrpc_client.mil, runtime/lib/MonetDB4/pf_support.mil, runtime/lib/MonetDB4/xrpc_server.mil, runtime/lib/MonetDB4/lib/lib_xrpc_server.so, runtime/lib/MonetDB4/lib/lib_xrpc_client.so, runtime/lib/MonetDB4/lib/lib_pathfinder.so, runtime/xrpc/admin/share/MonetDB/xrpc/admin/content.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/ctit.png, runtime/xrpc/admin/share/MonetDB/xrpc/admin/sf.bmp, runtime/xrpc/admin/share/MonetDB/xrpc/admin/index.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/monetdb_med_abstract.png, runtime/xrpc/admin/share/MonetDB/xrpc/admin/admin.js, runtime/xrpc/admin/share/MonetDB/xrpc/admin/title.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/cwi_logo.p ng, runtime/xrpc/admin/share/MonetDB/xrpc/admin/xrpcwebclient.js, runtime/xrpc/admin/share/MonetDB/xrpc/admin/admin.xq, runtime/xrpc/admin/share/MonetDB/xrpc/admin/ctit.gif, runtime/xrpc/admin/share/MonetDB/xrpc/admin/multimedian_logo.png, runtime/xrpc/admin/share/MonetDB/xrpc/admin/tum-logo.gif, runtime/xrpc/admin/share/MonetDB/xrpc/admin/menu.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/cwi.bmp, runtime/xrpc/export/share/MonetDB/xrpc/export/index.html, runtime/xrpc/export/share/MonetDB/xrpc/export/xrpc.html, runtime/xrpc/export/share/MonetDB/xrpc/export/export.xq, runtime/xrpc/export/share/MonetDB/xrpc/export/xrpc-mod.xq, runtime5/lib/MonetDB5/xquery.mal, runtime5/lib/MonetDB5/lib/lib_xquery.so, modules/pftijah/lib/MonetDB4/pftijah.mil, modules/pftijah/lib/MonetDB4/lib/lib_pftijah.so, modules/probxml/lib/MonetDB4/lib/lib_probxml.so, modules/probxml/lib/MonetDB4/probxml.mil, src/tools/bin/Mserver, src/tools/bin/Mbeddedxq, src/tools/lib/libembeddedxq.so, src/sqlhelpers/ xmlshred/bin/pfshred, src/sqlhelpers/serializer/bin/pfserialize.pl, src/sqlhelpers/db2tools/bin/pfcreatetables.sh, src/sqlhelpers/db2tools/bin/pfloadtables.sh, dist/MonetDB-XQuery-0.23.0.tar.gz, rpm/rpm/rpmmacros, rpm/rpm/rpmrc, rpm/rpm/RPMS/MonetDB4-XQuery-0.23.0-1_.rpm, rpm/rpm/RPMS/MonetDB4-XQuery-debuginfo-0.23.0-1_.rpm, rpm/rpm/RPMS/MonetDB4-XQuery-devel-0.23.0-1_.rpm, rpm/rpm/SRPMS/MonetDB4-XQuery-0.23.0-1.src.rpm, UNKNOWN/rpm/SPECS/MonetDB-XQuery.spec -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

3 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra algopt.c, , 1.30, 1.31 planner.c, , 1.56, 1.57 properties.c, , 1.34, 1.35
by Keulen, M. van (Maurice) 02 Apr '08

02 Apr '08

Hi Jan, This one is likely to speed up my use of MXQ a lot! Does "references" also mean that a lot less elements will be constructed? One of the scalability bottlenecks that I frequently encounter in my experiments is not that it is slow, but that I create more than 2^32 elements (which doesn't fit in a bat with 32-bit oids). Avoiding subtree copies would probably improve the scalability of my experiments enormously. I must definitely push the move to the algebra backend :-) Maurice. Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv13092/algebra > > Modified Files: > algopt.c planner.c properties.c > Log Message: > -- Added node based properties that check (top-down) if ... > > ... the node may be used (indirectly) for serialization > > ... the node id may be used > > ... the node order may be used > > ... the node values may be accessed > > ... a downward axis is applied lateron > (child, descendant, descendant-or-self) > > ... a side-way axis is applied lateron > (following, following-sibling, preceding, preceding-sibling) > > ... an upward axis is applied lateron > (parent, ancestor, ancestor-or-self) > > ... a self axis is applied lateron > (self, ancestor-or-self, descendant-or-self) > > ... the node may be used as input to node construction > > -- Exploited the node base property that checks for the downward axis > property: > > For every content constructor whose input is not queried we can > use the physical shallow content constructor that makes use of > references instead of creating a subtree copy. > > (This sped up the execution time of XMark Q10 (scale factor 1) > by 60% and the execution time of XMark Q15 by 150%.) > > > U algopt.c > Index: algopt.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/algopt.c,v > retrieving revision 1.30 > retrieving revision 1.31 > diff -u -d -r1.30 -r1.31 > --- algopt.c 25 Feb 2008 15:37:04 -0000 1.30 > +++ algopt.c 1 Apr 2008 16:37:11 -0000 1.31 > @@ -339,6 +339,7 @@ > false /* composite key */, > true /* key */, > false /* ocols */, > + true /* req_node */, > false /* reqval */, > true /* level */, > true /* refctr */, > @@ -356,6 +357,7 @@ > true /* composite key */, > true /* key */, > true /* ocols */, > + true /* req_node */, > true /* reqval */, > true /* level */, > true /* refctr */, > > U planner.c > Index: planner.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/planner.c,v > retrieving revision 1.56 > retrieving revision 1.57 > diff -u -d -r1.56 -r1.57 > --- planner.c 26 Mar 2008 11:20:47 -0000 1.56 > +++ planner.c 1 Apr 2008 16:37:13 -0000 1.57 > @@ -1784,7 +1784,7 @@ > *(plan_t **) PFarray_at (R(n)->plans, i), > sortby (iter, pos))); > > - if (false /* magic flag to enable shallow content constructors */) > + if (!PFprop_node_content_queried (n->prop, att_item)) > /* for each plan, generate a constructor */ > for (unsigned int i = 0; i < PFarray_last (ordered_in); i++) > add_plan (ret, > > U properties.c > Index: properties.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/properties.c,v > retrieving revision 1.34 > retrieving revision 1.35 > diff -u -d -r1.34 -r1.35 > --- properties.c 14 Jan 2008 14:05:24 -0000 1.34 > +++ properties.c 1 Apr 2008 16:37:14 -0000 1.35 > @@ -122,9 +122,9 @@ > void > PFprop_infer (bool card, bool const_, bool set, > bool dom, bool icol, bool ckey, > - bool key, bool ocols, bool reqval, > - bool level, bool refctr, bool guides, > - bool ori_names, bool unq_names, > + bool key, bool ocols, bool req_node, > + bool reqval, bool level, bool refctr, > + bool guides, bool ori_names, bool unq_names, > PFla_op_t *root, PFguide_tree_t *guide) > { > PFprop_create_prop (root); > @@ -145,6 +145,8 @@ > PFprop_infer_guide (root, guide); > if (const_) > PFprop_infer_const (root); > + if (req_node) > + PFprop_infer_req_node (root); > if (reqval) > PFprop_infer_reqval (root); > if (dom) > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen

2 1

[Monetdb-developers] rpm conflict, monetdb and fedora 8
by Bob McKay 02 Apr '08

02 Apr '08

Not sure if this is the right place to report this, but it seems there is a conflict between the current version of the monetdb fedora rpm install and an up-to-date fedora 8: Transaction Check Error: file /usr/lib/libmal.so.0 from install of MonetDB5-server-5.4.0-1.fc8.oid32.i386 conflicts with file from package libmal-0.31-7.fc8.i386 Any suggestions? (I'm not too keen on uninstalling libmal, as that entails also uninstalling significant parts of kde). When I first got the message I thought it might be that my fedora 8 was out-of-date, but I performed a full update and still get the problem.

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/mapi/Tests php_monetdb.stable.out, , 1.6, 1.7 php_monetdb.stable.out.Five, , 1.6, 1.7
by Martin Kersten 02 Apr '08

02 Apr '08

Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/test/mapi/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17226/src/test/mapi/Tests > > Modified Files: > php_monetdb.stable.out php_monetdb.stable.out.Five > Log Message: > removed 'explain' variable, ie no more set explain etc > but always use the statement modifiers debug, explain, profile > > profile now only outputs the real, user and clk ticks per sec. > > The old profile functionality is taken over by a new statement modifier > 'record' > > Don't forget to update the website accordingly

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal rdf.mx, , 1.1, 1.2
by Romulo Goncalves 01 Apr '08

01 Apr '08

Marco Antonelli wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv9457/src/modules/mal > > Modified Files: > rdf.mx > Log Message: The log message is missing... It is always nice to have a summary of what you are check in... It also helps the developers in future debugging tasks.... Romulo > > > U rdf.mx > Index: rdf.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf.mx,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- rdf.mx 26 Mar 2008 12:51:06 -0000 1.1 > +++ rdf.mx 1 Apr 2008 14:48:08 -0000 1.2 > @@ -41,7 +41,7 @@ > address RDFimportGraph > comment "import a graph in a named rdf db"; > > -command exportGraph( url:str, db:str, graph:str) : void > +command exportGraph( url:str, db:str, graph:str, order:str, syntax:str) : void > address RDFexportGraph > comment "export a graph in a named rdf db"; > > @@ -62,6 +62,7 @@ > #include <raptor.h> > > #define rdf_export extern > +#define RET_MAXLEN 2048 > > typedef struct { > lng *cur; > @@ -73,7 +74,7 @@ > oid *o; > oid *p; > oid *s; > - str ret; > + str *ret; > } triple_handler_vars; > //TODO > //static lng RDFestimate(str url) { > @@ -91,34 +92,31 @@ > > void message_handler(void *user_data, raptor_locator *locator, const char *message) > { > - unsigned int maxlength = 2048; > - char finalmessage[maxlength]; > size_t length = 0; > - char *msg = finalmessage; > str *ret = (str *)user_data; > + str msg = *ret; > if (locator->uri) > { > - snprintf(msg, maxlength, "In URI location \"%s\": ", raptor_uri_as_string(locator->uri)); > - length = strlen(finalmessage); > - msg = finalmessage + length*(sizeof(char)); > + snprintf(msg, RET_MAXLEN, "In URI location \"%s\": ", raptor_uri_as_string(locator->uri)); > + length = strlen(*ret); > + msg = *ret + length * (sizeof(char)); > } > - if (locator->line >=0 && length < maxlength-1) > + if (locator->line >=0 && length < RET_MAXLEN-1) > { > - snprintf(msg, maxlength-length, "at line %d: ", locator->line); > - length = strlen(finalmessage); > - msg = finalmessage + length*(sizeof(char)); > + snprintf(msg, RET_MAXLEN-length, "at line %d: ", locator->line); > + length = strlen(*ret); > + msg = *ret + length * (sizeof(char)); > } > - if (locator->column >=0 && length < maxlength-1) > + if (locator->column >=0 && length < RET_MAXLEN-1) > { > - snprintf(msg, maxlength-length, "at column %d: ", locator->column); > - length = strlen(finalmessage); > - msg = finalmessage + length*(sizeof(char)); > + snprintf(msg, RET_MAXLEN-length, "at column %d: ", locator->column); > + length = strlen(*ret); > + msg = *ret + length * (sizeof(char)); > } > - if (message && length < maxlength-1) > + if (message && length < RET_MAXLEN-1) > { > - snprintf(msg, maxlength-length, "%s\n", message); > + snprintf(msg, RET_MAXLEN-length, "%s\n", message); > } > - *ret=finalmessage; > } > > @= RDFins > @@ -133,7 +131,7 @@ > bun = (BUN) map->batCount; > if (!BUNappend(map,(ptr)@1v, TRUE)) { > GDKerror("RDFimport: @1[" LLFMT "] = %s, BUNappend fails\n", cur, @1v); > - data->ret = RDFimport_fail(map,ob,pb,sb); > + *(data->ret) = RDFimport_fail(map,ob,pb,sb); > } > } > /* s,p,o[cur] are set to the virtual oid which is bun s p and o are arrays of oid */ > @@ -153,9 +151,10 @@ > oid *o = data->o; > oid *p = data->p; > oid *s = data->s; > + str *ret = data->ret; > BATiter mi = bat_iterator(BATmirror(map)); > BUN bun; > - if (data->ret!=MAL_SUCCEED) > + if (strlen(*ret)!=0) > return; > /* get the subject */ > if (triple->subject_type == RAPTOR_IDENTIFIER_TYPE_RESOURCE) > @@ -164,7 +163,7 @@ > sv = (str)triple->subject; > else > { > - data->ret="Unexpected subject type"; > + *ret="Unexpected subject type"; > return; > } > > @@ -173,7 +172,7 @@ > pv = (str)raptor_uri_as_string((raptor_uri *)triple->predicate); > else > { > - data->ret="Unexpected predicate type"; > + *ret="Unexpected predicate type"; > return; > } > > @@ -190,18 +189,18 @@ > ov = (str)raptor_uri_as_string((raptor_uri*)triple->object); > else > { > - data->ret="Unexpected object type"; > + *ret="Unexpected object type"; > return; > } > > @:RDFins(o)@ > - if (data->ret == MAL_SUCCEED) > + if (strlen(*ret)==0) > { > @:RDFins(p)@ > - if (data->ret == MAL_SUCCEED) > + if (strlen(*ret)==0) > { > @:RDFins(s)@ > - if (data->ret == MAL_SUCCEED) > + if (strlen(*ret)==0) > { > if (++(*cur) > *lim) { > *lim *= 2; > @@ -209,7 +208,7 @@ > pb = BATextend(pb, *lim); > sb = BATextend(sb, *lim); > if (ob == NULL || pb == NULL || sb == NULL) > - data->ret = RDFimport_fail(map,ob,pb,sb); > + *ret = RDFimport_fail(map,ob,pb,sb); > o = (oid *) Tloc(ob, BUNfirst(ob)); > p = (oid *) Tloc(pb, BUNfirst(pb)); > s = (oid *) Tloc(sb, BUNfirst(sb)); > @@ -223,6 +222,7 @@ > RDFimport_(BAT** S, BAT** P, BAT **O, str url, BAT *map) { > raptor_parser *parser; > raptor_uri *rdf_file; > + str ret; > raptor_init(); > parser = raptor_new_parser("rdfxml"); > if (parser) > @@ -244,41 +244,15 @@ > p = (oid *) Tloc(pb, BUNfirst(pb)); > s = (oid *) Tloc(sb, BUNfirst(sb)); > > - //char obuf[20],*op, pbuf[20],*pp, sbuf[20],*sp; > - //str ov, pv, sv; > - > if (map == NULL || ob == NULL || pb == NULL || sb == NULL) > return RDFimport_fail(map,ob,pb,sb); > - > - > - > -// strcpy(obuf, "object"); for(ov=op=obuf; *op; op++); op[1]=0; > -// strcpy(pbuf, "predicate"); for(pv=pp=pbuf; *pp; pp++); pp[1]=0; > -// strcpy(sbuf, "subject"); for(sv=sp=sbuf; *sp; sp++); sp[1]=0; > -// while (cur < 8 /*redland gives me (ov,pv,sv)*/ ) { > -// *op = (cur & 4)?'1':'0'; > -// *pp = (cur & 2)?'1':'0'; > -// *sp = (cur & 1)?'1':'0'; > -// RDFins(o) /* mx macros*/ > -// RDFins(p) > -// RDFins(s) > -// > -// if (++cur > lim) { > -// lim *= 2; > -// ob = BATextend(ob, lim); > -// pb = BATextend(pb, lim); > -// sb = BATextend(sb, lim); > -// if (ob == NULL || pb == NULL || sb == NULL) > -// return RDFimport_fail(map,ob,pb,sb); > -// o = (oid *) Tloc(ob, BUNfirst(ob)); > -// p = (oid *) Tloc(pb, BUNfirst(pb)); > -// s = (oid *) Tloc(sb, BUNfirst(sb)); > -// } > -// } > > - data = malloc(sizeof(triple_handler_vars)); > + data = GDKmalloc(sizeof(triple_handler_vars)); > if (data) > { > + char ret_array[RET_MAXLEN]; > + str ret_ptr = ret_array; > + ret_array[0] = '\0'; > data->cur = cur; > data->lim = lim; > data->map =map; > @@ -288,18 +262,18 @@ > data->s = s; > data->p = p; > data->o = o; > - data->ret = MAL_SUCCEED; > + data->ret = &ret_ptr; > raptor_set_statement_handler(parser, data, triples_handler); > - raptor_set_fatal_error_handler(parser, &data->ret, message_handler); > - raptor_set_error_handler(parser, &data->ret, message_handler); > - raptor_set_warning_handler(parser, &data->ret, message_handler); > - //raptor_set_parser_strict(parser, TRUE); > + raptor_set_fatal_error_handler(parser, &ret_ptr, message_handler); > + raptor_set_error_handler(parser, &ret_ptr, message_handler); > + raptor_set_warning_handler(parser, &ret_ptr, message_handler); > + raptor_set_parser_strict(parser, TRUE); > > if (raptor_parse_file(parser, rdf_file, NULL) == 0) > { > - /* do something with redland */ > - if (data->ret == MAL_SUCCEED) > + if(strlen(ret_ptr)==0) > { > + ret = MAL_SUCCEED; > BATseqbase(ob, 0); > BATseqbase(pb, 0); > BATseqbase(sb, 0); > @@ -312,30 +286,31 @@ > BATkey(BATmirror(ob), FALSE); > BATkey(BATmirror(sb), FALSE); > BATkey(BATmirror(pb), FALSE); > - > - free(data); > - raptor_free_parser(parser); > - raptor_free_uri(rdf_file); > - raptor_finish(); > - return MAL_SUCCEED; > } > - else > - throw(MAL, "rdf.import", data->ret); > } > else > - throw(MAL, "rdf.import", "RDF parsing error"); > + ret = "RDF parsing error"; > + GDKfree(data); > } > else > - throw(MAL, "rdf.import", "triple_handler_vars malloc failed"); > + ret = "triple_handler_vars malloc failed"; > + raptor_free_uri(rdf_file); > } > else > - throw(MAL, "rdf.import", "Unexistent file: \"%s\"", url); > + ret = "Unexistent url"; > + raptor_free_parser(parser); > } > else > - throw(MAL, "rdf.import", "Could not create a Redland parser"); > + ret = "Could not create a Redland parser"; > + raptor_finish(); > + if (ret != MAL_SUCCEED) > + throw(MAL, "rdf.import", ret); > + > + return MAL_SUCCEED; > } > > -rdf_export str RDFimport(int* S, int* P, int* O, str *url, bat m) > +rdf_export > +str RDFimport(int* S, int* P, int* O, str *url, bat m) > { > BAT *sb = NULL, *ob=NULL, *pb = NULL, *map = BATdescriptor(m); > str ret; > @@ -352,45 +327,109 @@ > } > > static str > -RDFexport_(str url, bat M, bat S, bat P, bat O) { > - str ret = "RDFexport failed to obtain bats"; > +RDFexport_(str url, bat M, bat S, bat P, bat O, str syntax) > +{ > + str ret = MAL_SUCCEED; > BAT *map = BATdescriptor(M); > - if (map) { > + (void)url; > + if (map) > + { > BAT *sb = BATdescriptor(S); > - if (sb) { > + if (sb) > + { > BAT *pb = BATdescriptor(P); > - if (pb) { > + if (pb) > + { > BAT *ob = BATdescriptor(O); > - if (ob) { > - lng i, cnt = BATcount(ob); > + if (ob) > + { > + raptor_serializer* rdf_serializer; > + const char *filename="/ufs/antonell/Desktop/output.rdf"; > + lng i, cnt = BATcount(ob); > oid *o = (oid *) Tloc(ob, BUNfirst(ob)); > oid *p = (oid *) Tloc(pb, BUNfirst(pb)); > oid *s = (oid *) Tloc(sb, BUNfirst(sb)); > //var_t position in heap > var_t *off = (var_t *) Tloc(map, BUNfirst(map)); > str base = (str) map->T->vheap->base;//heap first position > - > - for(i=0; i <cnt; i++) { > - str ov = base + off[o[i]]; > - str pv = base + off[p[i]]; > - str sv = base + off[s[i]]; > - stream_printf(GDKout, "'%s' '%s' '%s' '%s'\n", ov, pv, sv, url); > + > + raptor_init(); > + rdf_serializer = raptor_new_serializer(syntax); > + > + if (rdf_serializer) > + { > + char ret_array[RET_MAXLEN]; > + str ret_ptr = ret_array; > + ret_array[0] = '\0'; > + raptor_serializer_set_error_handler(rdf_serializer, &ret_ptr, message_handler); > + raptor_serializer_set_warning_handler(rdf_serializer, &ret_ptr, message_handler); > + if (raptor_serialize_start_to_filename(rdf_serializer, filename) == 0) > + { > + for(i=0; i <cnt; i++) > + { > + if (strlen(ret_ptr)==0) > + { > + raptor_statement *triple; > + str ov = base + off[o[i]]; > + str pv = base + off[p[i]]; > + str sv = base + off[s[i]]; > + //stream_printf(GDKout, "'%s' '%s' '%s' '%s'\n", ov, pv, sv, url); > + /* Make a triple with URI subject, URI predicate, literal object */ > + triple=GDKmalloc(sizeof(raptor_statement)); > + > + triple->subject=(void*)raptor_new_uri((const unsigned char*)sv); > + triple->subject_type=RAPTOR_IDENTIFIER_TYPE_RESOURCE; > + > + triple->predicate=(void*)raptor_new_uri((const unsigned char*)pv); > + triple->predicate_type=RAPTOR_IDENTIFIER_TYPE_RESOURCE; > + > + triple->object=ov; > + triple->object_type=RAPTOR_IDENTIFIER_TYPE_LITERAL; > + //triple->object_literal_language=(const unsigned char*)"en"; > + triple->object_literal_language=NULL; > + triple->object_literal_datatype = NULL; > + /* Write the triple */ > + if (raptor_serialize_statement(rdf_serializer, triple)!=0) > + ret="failed to serialize a statement"; > + > + /* Delete the triple */ > + raptor_free_uri((raptor_uri*)triple->subject); > + raptor_free_uri((raptor_uri*)triple->predicate); > + GDKfree(triple); > + } > + } > + ret = MAL_SUCCEED; > + raptor_serialize_end(rdf_serializer); > + } > + else > + ret = "raptor_serialize_start failed"; > + raptor_free_serializer(rdf_serializer); > } > - ret = MAL_SUCCEED; > + else > + ret = "raptor_new_serializer failed"; > + raptor_finish(); > BBPreleaseref(ob->batCacheid); > - } > + } > + else > + ret = "Failed to obtain O bat"; > BBPreleaseref(pb->batCacheid); > - } > + } > + else > + ret = "Failed to obtain P bat"; > BBPreleaseref(sb->batCacheid); > - } > + } > + else > + ret = "Failed to obtain S bat"; > BBPreleaseref(map->batCacheid); > - } > + } > + else > + ret = "Failed to obtain MAP bat"; > return ret; > } > > str > RDFexport(str *url, bat M, bat S, bat P, bat O) { > - str ret = RDFexport_(*url, M, S, P, O); > + str ret = RDFexport_(*url, M, S, P, O, "dummy syntax"); > if (ret != MAL_SUCCEED) > throw(MAL, "RDFexport", ret); > return MAL_SUCCEED; > @@ -513,23 +552,23 @@ > > ret = "RDFimportGraph: sorting failed"; > if (sorted) { > - ret = @:sort(sb, pb, ob, Spo, sPo, spO)@ > + ret = @:sort(sb,pb,ob,Spo,sPo,spO)@ > if (ret == MAL_SUCCEED) { > - ret = @:sort(sb, ob, pb, Sop, sOp, soP)@ > + ret = @:sort(sb,ob,pb,Sop,sOp,soP)@ > if (ret == MAL_SUCCEED) { > BBPreclaim(sorted); > sorted = BATmirror(BATsort(BATmirror(pb))); > if (sorted) { > - ret = @:sort(pb, ob,sb,Pos,pOs,poS)@ > + ret = @:sort(pb,ob,sb,Pos,pOs,poS)@ > if (ret == MAL_SUCCEED) { > - ret = @:sort(pb, sb,ob,Pso,pSo,psO)@ > + ret = @:sort(pb,sb,ob,Pso,pSo,psO)@ > if (ret == MAL_SUCCEED) { > BBPreclaim(sorted); > sorted = BATmirror(BATsort(BATmirror(ob))); > if (sorted) { > - ret = @:sort(ob, pb, sb, Ops, oPs, opS)@ > + ret = @:sort(ob,pb,sb,Ops,oPs,opS)@ > if (ret == MAL_SUCCEED) { > - ret = @:sort(ob, sb, pb,Osp,oSp,osP)@ > + ret = @:sort(ob,sb,pb,Osp,oSp,osP)@ > if (ret == MAL_SUCCEED) { > ret = MAL_SUCCEED; > goto succeed; > @@ -572,15 +611,84 @@ > throw(MAL, "RDFimportGraph", ret); > return MAL_SUCCEED; > } > -str RDFexportGraph(str *url, str *db, str *graph) { > + > +static str getBatNames(str db, str graph, char *s_buf,char *p_buf,char *o_buf, str order) > +{ > + str err = "order may be only one among \"spo\",\"sop\",\"pso\",\"pos\",\"osp\",\"ops\""; > + if (strlen(order)>3) > + return err; > + if (strcmp(order, "spo")==0) > + { > + snprintf(s_buf, 1024, "%s_%s_Spo", db, graph); > + snprintf(p_buf, 1024, "%s_%s_sPo", db, graph); > + snprintf(o_buf, 1024, "%s_%s_spO", db, graph); > + } > + else if (strcmp(order, "sop")==0) > + { > + snprintf(s_buf, 1024, "%s_%s_Sop", db, graph); > + snprintf(o_buf, 1024, "%s_%s_sOp", db, graph); > + snprintf(p_buf, 1024, "%s_%s_soP", db, graph); > + } > + else if (strcmp(order, "pos")==0) > + { > + snprintf(p_buf, 1024, "%s_%s_Pos", db, graph); > + snprintf(o_buf, 1024, "%s_%s_pOs", db, graph); > + snprintf(s_buf, 1024, "%s_%s_poS", db, graph); > + } > + else if (strcmp(order, "pso")==0) > + { > + snprintf(p_buf, 1024, "%s_%s_Pso", db, graph); > + snprintf(s_buf, 1024, "%s_%s_pSo", db, graph); > + snprintf(o_buf, 1024, "%s_%s_psO", db, graph); > + } > + else if (strcmp(order, "osp")==0) > + { > + snprintf(o_buf, 1024, "%s_%s_Osp", db, graph); > + snprintf(s_buf, 1024, "%s_%s_oSp", db, graph); > + snprintf(p_buf, 1024, "%s_%s_osP", db, graph); > + } > + else if (strcmp(order, "ops")==0) > + { > + snprintf(o_buf, 1024, "%s_%s_Ops", db, graph); > + snprintf(p_buf, 1024, "%s_%s_oPs", db, graph); > + snprintf(s_buf, 1024, "%s_%s_opS", db, graph); > + } > + else > + return err; > + return MAL_SUCCEED; > +} > + > +static str checkSyntax(str syntax) > +{ > + if ( //!strcmp(syntax, "atom") || > + !strcmp(syntax, "ntriples") || > + !strcmp(syntax, "rdfxml") || > + !strcmp(syntax, "rdfxml-abbrev") || > + !strcmp(syntax, "rdfxml-xmp") || > + !strcmp(syntax, "turtle") || > + //!strcmp(syntax, "rdfxml-xmp") || > + !strcmp(syntax, "rss-1.0") || > + !strcmp(syntax, "dot")) > + { > + return MAL_SUCCEED; > + } > + else > + return "syntax may be only one among \"atom\",\"ntriples\",\"rdfxml\",\"rdfxml-abbrev\",\"rdfxml-xmp\",\"turtle\",\"rdfxml-xmp\",\"rss-1.0\",\"dot\""; > +} > + > +str RDFexportGraph(int *d, str *url, str *db, str *graph, str *order, str *syntax) > +{ > char m_buf[1024], o_buf[1024], p_buf[1024], s_buf[1024]; > bat map, ob, pb, sb; > - str ret = "RDFexportGraph failed to obtain bats"; > - > + str ret; > + (void)d; > + ret = checkSyntax(*syntax); > + if (ret!= MAL_SUCCEED) > + throw(MAL, "RDFexportGraph", ret); > + ret = getBatNames(*db, *graph, s_buf, p_buf, o_buf, *order); > + if (ret!= MAL_SUCCEED) > + throw(MAL, "RDFexportGraph", ret); > snprintf(m_buf, 1024, "%s_%s_map", *db, *graph); > - snprintf(p_buf, 1024, "%s_%s_Ops", *db, *graph); > - snprintf(o_buf, 1024, "%s_%s_oPs", *db, *graph); > - snprintf(s_buf, 1024, "%s_%s_opS", *db, *graph); > map = BBPindex(m_buf); > if (map) { > sb = BBPindex(s_buf); > @@ -589,11 +697,19 @@ > if (pb) { > ob = BBPindex(o_buf); > if (ob) { > - ret = RDFexport_(*url, map, sb, pb, ob); > + ret = RDFexport_(*url, map, sb, pb, ob, *syntax); > } > + else > + ret = "Failed to obtain O bat"; > } > + else > + ret = "Failed to obtain P bat"; > } > + else > + ret = "Failed to obtain S bat"; > } > + else > + ret = "Failed to obtain map bat"; > if (ret != MAL_SUCCEED) > throw(MAL, "RDFexportGraph", ret); > return MAL_SUCCEED; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

2 1

[Monetdb-developers] blobs performance
by Henrik Vendelbo 30 Mar '08

30 Mar '08

I am working through the options for clustered file management. Normally I would expect fairly poor performance of binary file storage in a database. Hence I would favor using the file system, and sync across the machines in the cluster. Being quite different from common relational implementations, how well does Monet handle blobs of 1-10Mb? As a side question is there a support for multi-master replication? The technology looks really impressive, I hope to be able to base my future projects on MonetDB Thanks, Henrik

2 1

Re: [Monetdb-developers] Monetdb-developers Digest, Vol 22, Issue 4
by Peter Boncz 28 Mar '08

28 Mar '08

Hi, I read the digests of the monetdb mailing list, which in this case took more than a month to send me this discussion. That's pretty useless.. I will switch to immediate mails. Anyway, Jennie is currently working on getting the repeatable reads transactions working. This means that by passing a request-ID you will be guaranteed that all queries and updates execute on the same database image, i.e. a transaction. In case of update requests in the same transaction, these will be executed atomically using a 2PC protocol. While this functionality is designed to be triggerered from XQuery queries, you client API could also generate the appropriate messages. It may not answer you needs regarding performance (you would send a read-request, update-request, prepare-request and commit-request), but it would answer your functional requirements AFAIS. Peter > Message: 1 > Date: Wed, 19 Mar 2008 12:30:20 +0100 > From: "Keulen, M. van (Maurice)" <m.vankeulen(a)utwente.nl> > Subject: Re: [Monetdb-developers] [XQ] Mixing update/docmgmt/query > To: monetdb-developers(a)lists.sourceforge.net > Message-ID: <47E0F94C.4000709(a)utwente.nl> > Content-Type: text/plain; charset=windows-1252; format=flowed > > Ying Zhang wrote: > > > >> Jennie: my SOAP-messages do not come from MonetDB/XQuery, but from > >> outside (i.e., some scripting language). > > > > Then, an order can only be guaranteed if you, in your > scripts, send a > > message, wait for result, and send the next message. In all other > > cases, unfortunately, no guarantee, as an xrpc server immediately > > start handling a request when it's received. > > > > Jennie > > I thought so already. This is why I don't like to have to > send multiple > requests to do one thing (transaction). I'd rather be able to > send one > request to both do, say, an update and return a result, or add a > document and return a result. The system so far doesn't allow > me to do > this, because I cannot define one query (or function in a > module) that > does both because of typing restrictions, and I also cannot send two > separate queries in one request because of restrictions in the XRPC > protocol. > > Any thoughts about achieving what I want? > > Maurice. > > -- > ---------------------------------------------------------------------- > Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology > Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, > Netherlands > Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 > 534892927 > Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen *********************************

3 8

[Monetdb-developers] MonetDB testing problems
by Stefan Manegold 27 Mar '08

27 Mar '08

Dear fellow MonetDB developers, the recently introduced archiving of testing result summaries seems to require more storage than expected --- I don't know, yet, whether this is indeed inherent in the approach, or "just" some bug --- filling up the disk space on our testing server, and thus preventing testing from running properly. I will not have any time to investigate (let alone fix) the problem this week --- in fact, given other obligations, I'm not sure whether I will find any time before May 1st ... Hence, there might be some time ahead without proper/reliable MonetDB testing. I'll keep you posted, in case/once I managed to solve the problem... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] [PF] Question about timings
by Keulen, M. van (Maurice) 25 Mar '08

25 Mar '08

Hi, A student of mine is trying to speed up some queries and is puzzled about the timing-readings he gets, for example: Trans 19681.811 msec Shred 0.000 msec Query 16384.143 msec Print 81.180 msec Timer 55087.438 msec The total of the first four 19.7+0+16.4+0.08 is approximately 36.2 seconds. "Timer" gives 55.1 seconds. Where are the remaining 18.9 seconds going? By the way, what do these measures precisely indicate? "Query", for example, can be interpreted as query compilation or query execution. Maurice. -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen

2 3

[Monetdb-developers] Experimental testweb history available
by Fabian Groffen 20 Mar '08

20 Mar '08

Hi all, This is just to inform you all of a recently added "feature" (to both Stable and Current ;) ) in our beloved TestWeb. We added a "history" function to parts of TestWeb's output, which can come in handy if you're chasing pavem^Wbugs, or just interested in (since) when happened what. First off, a nice, nifty and sexy GUI is missing for sure, but who knows what the future will bring :D The MonetDB PermaStore (it is *not* powered by any MonetDB technology) http://www.ins.cwi.nl/~monet/permastore/ At this URL you will find all the output of the compilation check script, that some of you may already have been confronted with. In particular, here you can find per day the skeletons of the email messages sent out, and the annotated HTML versions of the build logs. Improvements in this area are low hanging fruit, but they just require time. Note that the aforementioned URL is still sort of experimental only. The MonetDB TestWeb 30-day archive http://monetdb.cwi.nl/testing/projects/monetdb/<branch>/<module>/.<thing>..<german_date>/ where <german_date> = yyyy.mm.dd If you know what branch (Stable, Current), module (buildtools, MonetDB, sql, ...) and thing (Makes3, Int.32.32.d-Fedora8, etc.) should be then you can inspect the TestWeb output till 30 days back in time. Note that the full 30 days are not yet generated, so only partially available.

3 2

[Monetdb-developers] [XQ] Mixing update/docmgmt/query
by Keulen, M. van (Maurice) 19 Mar '08

19 Mar '08

Hi, I know that the XQUF prohibits it, but for writing robust functions, it is often necessary to mix query and update/docmgmt functionality. Some examples: * Expression that queries/updates document, but if the document does not exist, adds an empty version of it: let $name := "foo.xml" let $emptydoc := <root/> let $doc := if (empty(pf:documents()[.=$name])) then (put($emptydoc,$name),$emptydoc) else doc("foo.xml") return $doc//bar * Expression that adds to a log-file: let $log := doc("log.xml") let $expr := ...some expression... return (do insert <entry>...{$expr}...<entry> as last into $log,$expr) Each of these patterns gives you a "!type error: err:XUST0101: illegal mix of updating and non-updating expressions" error. Isn't is possible to somehow allow such expressions? It is against the standard, so it probably requires some server configuration parameter, but it would be really really useful to have it. Mixing queries with update/docmgmt is known to cause many semantical problems, but only if these updates are visible within the query. Looking at the implementation based on pending update lists, then each of these patterns is harmless. If you declare that no updates are visible within the transaction (which is against the standard), then allowing these queries is I guess not hard: it simply means that with updating expressions and document management expressions you do return a result. Another thing I've been thinking about in this context: If I send two SOAP-messages to a server on its XRPC-port, am I guaranteed that they are served in the same order? I could perhaps, with a lot of duplication of code, separate update/docmgmt from a query, i.e., implement these things with two requests to the server. I really dislike such a solution, though. Looking forward to comments :-) Maurice. -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen

3 5

[Monetdb-developers] Bug in Python interface converters.py Feb2008-SuperBall
by Andrei Tovtchigretchko 15 Mar '08

15 Mar '08

MonetSQLdb.converters.escape(x, conv=None) should return 'tuple' for any input sequence type (currently it returns tuple only if it receives tuple, and returns list if it receives list). The return result is used as a second argument in expression 'query = query % self.connection.literal(args)' in cursors.py. The '%' operator raises an exception on anything but tuple. Also, when 'escape()' cannot convert some argument, it should immediately raise a descriptive exception. Currently it silently drops the argument from the result sequence, which causes an exception to be raised later in the calling code. That exception reads like this: """ProgrammingError: not enough arguments for format string""" This unnecessarily complicates debugging. Thank you, Andrey _________________________________________________________________ Watch “Cause Effect,” a show about real people making a real difference. Learn more. http://im.live.com/Messenger/IM/MTV/?source=text_watchcause

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_recycle.mx, , 1.49, 1.50
by Stefan Manegold 15 Mar '08

15 Mar '08

On Sat, Mar 15, 2008 at 08:41:35AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17899 > > Modified Files: > mal_recycle.mx > Log Message: > Silence Windows compiler FYI: `man bzero` says: ======== CONFORMING TO 4.3BSD. This function is deprecated (marked as LEGACY in POSIX.1-2001): use memset(3) in new programs. ======== Stefan > Index: mal_recycle.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_recycle.mx,v > retrieving revision 1.49 > retrieving revision 1.50 > diff -u -d -r1.49 -r1.50 > --- mal_recycle.mx 14 Mar 2008 09:43:07 -0000 1.49 > +++ mal_recycle.mx 15 Mar 2008 08:41:33 -0000 1.50 > @@ -260,7 +260,7 @@ > newMalBlkStmt(recycleBlk,recycleBlk->ssize); > > used= alloca(recycleBlk->vtop); > - bzero(used, recycleBlk->vtop); > + memset(used,0, recycleBlk->vtop); > > /* set all used variables */ > for (i = 0; i < pc; i++){ > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal recycle.mx, , 1.9, 1.10
by Stefan Manegold 10 Mar '08

10 Mar '08

On Mon, Mar 10, 2008 at 11:21:59AM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24521 > > Modified Files: > recycle.mx > Log Message: > Check in the MALblk exist to avoid a mserver crash when we to > recycle.dump() before any query execution why do you check the loop-invariant condition with every iteration of the loop? Stefan > Index: recycle.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/recycle.mx,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- recycle.mx 9 Mar 2008 15:47:30 -0000 1.9 > +++ recycle.mx 10 Mar 2008 11:21:47 -0000 1.10 > @@ -147,7 +147,7 @@ > stream_printf(GDKout,"Recycler catalog \n"); > > /* and dump the statistics */ > - for(i=0; i< recycleBlk->stop; i++){ > + for(i=0; recycleBlk && i< recycleBlk->stop; i++){ > stream_printf(GDKout,"#%4d count=%3d ticks=%4d %s\n", i, > recycleBlk->profiler[i].counter, > recycleBlk->profiler[i].ticks, > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_client.mx, xquery-decomposition, 1.41.2.8, 1.41.2.9
by Stefan Manegold 10 Mar '08

10 Mar '08

On Mon, Mar 10, 2008 at 02:01:16AM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv3350 > > Modified Files: > Tag: xquery-decomposition > xrpc_client.mx > Log Message: > - adopt Stefan's tips: used rangesplit to split item|kind per container > (Thanks!). > > - a lot of corrections to get this thing working > > > > Index: xrpc_client.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_client.mx,v > retrieving revision 1.41.2.8 > retrieving revision 1.41.2.9 > diff -u -d -r1.41.2.8 -r1.41.2.9 > --- xrpc_client.mx 8 Mar 2008 21:17:27 -0000 1.41.2.8 > +++ xrpc_client.mx 10 Mar 2008 02:01:13 -0000 1.41.2.9 > @@ -94,94 +94,40 @@ > BAT[void, dbl] dbl_values, > BAT[void, dbl] dec_values, > BAT[void, str] str_values) : oid { > - > var maxnconts := ws.fetch(0).count(); > # split used_<item|kind> into multiple BATs according to their container IDs vvvvvvvvv > var uitems := bat(oid, bat, maxnconts); # container ID in head > var ukinds := bat(oid, bat, maxnconts); ^^^^^^^^^ These two BATs are IMHO overwritten below (see "vvvvvvvvv"/"^^^^^^^^^"); hence, no (more) need to make them, here!?? > - var conts := used_kind.[int]().[>>](6).tsort(); # [oid,int]: container IDs in tail > - var nconts := conts.tunique().count(); > - if(nconts = 1){ # all items are from the same container > - uitems.insert(conts.fetch(0).oid(), used_item); > - ukinds.insert(conts.fetch(0).oid(), used_kind); > - } else { > - # sort used_<item|kind> on container ID > - var itemcont := used_item.reverse().join(conts).reverse().tmark(0@0); > - var kindcont := used_kind.reverse().join(conts).reverse().tmark(0@0); > - > - var curcont := conts.fetch(0); > - var tmpitem := bat(void, oid).seqbase(0@0); > - var tmpkind := bat(void, int).seqbase(0@0); > - > - conts := conts.tmark(0@0); # conts[void,int] > - conts@batloop(){ > - # start of next container, save items|kinds of previous container > - if($t != curcont) { > - curcont := $t; > - uitems.insert(curcont, tmpitem); > - ukinds.insert(curcont, tmpkind); > - tmpitem := bat(void, oid).seqbase(0@0); > - tmpkind := bat(void, int).seqbase(0@0); > - } > - > - tmpitem.append(itemcont.fetch($h)); > - tmpkind.append(kindcont.fetch($h)); > - } > - uitems.insert(curcont, tmpitem); > - ukinds.insert(curcont, tmpkind); > - } > + var conts := get_container([int](used_kind)); # [oid,int]: container IDs in tail > + var unq_conts := tunique(conts).hmark(0@0); > + var nconts := count(unq_conts); > + var rs := rangesplit(conts, nconts).tmark(0(a)0).reverse().join(unq_conts).reverse() This will do want you want (have rs's head contain the conts from the BATs in its tail) ONLY if tunique(conts) & rangesplit(conts, nconts) happen to produce the same order! To be sure, you'd need to sort the unq_conts before marking them (result of rangesplit shoiuld hopefully be sorted): var unq_conts := tunique(conts).sort().hmark(0@0); Was there something wrong with my proposal: var rs := rangesplit(conts, nconts); rs := rs.reverse().mirror().[fetch](0).reverse(); ? vvvvvvvvv > + uitems := rs.[mirror]().[leftfetchjoin](const used_item); > + uitems := [tmark](uitems, const 0@0); > + ukinds := rs.[mirror]().[leftfetchjoin](const used_kind); > + ukinds := [tmark](ukinds, const 0@0); ^^^^^^^^^ (see "vvvvvvvvv"/^^^^^^^^^" above) Plus: It might be more efficient to get rid of the head before the join, since you "overwrite" them afterwards, anyway; i.e.: uitems := rs.[tmark](oid(nil)).[leftfetchjoin](const used_item).[tmark](uitems, const 0@0); > > # split returned_<item|kind> into multiple BATs according to their container IDs vvvvvvvvv > var ritems := bat(oid, bat, maxnconts); # container ID in head > var rkinds := bat(oid, bat, maxnconts); ^^^^^^^^^ Same as with uitems/ukinds above. > - conts := returned_kind.[int]().[>>](6).tsort(); # [oid,int]: container IDs in tail > - nconts := conts.tunique().count(); > - if(nconts = 1){ # all items are from the same container > - ritems.insert(conts.fetch(0).oid(), returned_item); > - rkinds.insert(conts.fetch(0).oid(), returned_kind); > - } else { > - # sort used_<item|kind> on container ID > - var itemcont := returned_item.reverse().join(conts).reverse().tmark(0@0); > - var kindcont := returned_kind.reverse().join(conts).reverse().tmark(0@0); > - > - var curcont := conts.fetch(0); > - var tmpitem := bat(void, oid).seqbase(0@0); > - var tmpkind := bat(void, int).seqbase(0@0); > - > - conts := conts.tmark(0@0); # conts[void,int] > - conts@batloop(){ > - # start of next container, save items|kinds of previous container > - if($t != curcont) { > - curcont := $t; > - ritems.insert(curcont, tmpitem); > - rkinds.insert(curcont, tmpkind); > - tmpitem := bat(void, oid).seqbase(0@0); > - tmpkind := bat(void, int).seqbase(0@0); > - } > - > - tmpitem.append(itemcont.fetch($h)); > - tmpkind.append(kindcont.fetch($h)); > - } > - ritems.insert(curcont, tmpitem); > - rkinds.insert(curcont, tmpkind); > - } > + conts := get_container([int](returned_kind)); # [oid,int]: container IDs in tail > + unq_conts := tunique(conts).hmark(0@0); > + nconts := count(unq_conts); > + rs := rangesplit(conts, nconts).tmark(0(a)0).reverse().join(unq_conts).reverse() (see above) vvvvvvvvv > + ritems := rs.[mirror]().[leftfetchjoin](const returned_item); > + ritems := [tmark](ritems, const 0@0); > + rkinds := rs.[mirror]().[leftfetchjoin](const returned_kind); > + rkinds := [tmark](rkinds, const 0@0); ^^^^^^^^^ (see above) > > # align uitems|kinds and ritems|kinds, > # fill missing container with emtpybat > var emptybat := bat(void,void,0); > - uitems@batloop(){ > - var err := CATCH({ritems.find($h);}); > - if(err.search("not found") >= 0){ > - ritems.insert($h, emptybat); > - } > - } > - ritems@batloop(){ > - var err := CATCH({uitems.find($h);}); > - if(err.search("not found") >= 0){ > - uitems.insert($h, emptybat); > - } > - } > - > + var usedonly := kdiff(uitems,ritems).project(emptybat); > + ritems.access(BAT_WRITE).insert(usedonly); > + rkinds.access(BAT_WRITE).insert(usedonly); > + var returnedonly := kdiff(ritems,uitems).project(emptybat); > + uitems.access(BAT_WRITE).insert(returnedonly); > + ukinds.access(BAT_WRITE).insert(returnedonly); a) Do you need to modify these BATs later on? If not, you might want the set them read-only, again, after the insert: ritems.access(BAT_WRITE).insert(usedonly).access(BAT_READ); b) To be sure that your variables till point to the correct BATs (access(BAT_WRITE) *might* return a copy!), you should asign the result: ritems := ritems.access(BAT_WRITE).insert(usedonly)[.access(BAT_READ)]; Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_replicator.mx, , 1.30, 1.31
by Stefan Manegold 03 Mar '08

03 Mar '08

On Mon, Mar 03, 2008 at 02:35:30PM +0000, Milena Ivanova wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv4109/src/optimizer > > Modified Files: > opt_replicator.mx > Log Message: > Make sure the constant exists in var area of replicator. Just a simple question out of curiosity: It seems, the new variable v1 is only use in the new if-statement's body, right? Hence, why don't you also declare the variable v1 as "local" variable in the if-body? The is no need to have it as "global" variable in function newReplica(), is there? Stefan > Index: opt_replicator.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_replicator.mx,v > retrieving revision 1.30 > retrieving revision 1.31 > diff -u -d -r1.30 -r1.31 > --- opt_replicator.mx 2 Mar 2008 12:08:46 -0000 1.30 > +++ opt_replicator.mx 3 Mar 2008 14:35:27 -0000 1.31 > @@ -438,7 +438,7 @@ > static int newReplica(MalStkPtr s, InstrPtr p, bit keep) > { > int i, j, k, c, bid, r; > - ValPtr v; > + ValPtr v, v1; > InstrPtr p1; > > assureSpace(); > @@ -446,7 +446,12 @@ > for(i = 0; i< p->argc; i++){ > j = p->argv[i]; > v = &s->stk[j]; > - c = defConstant(repl, v->vtype, v); > + c = fndConstant(repl, v); > + if ( c < 0 ){ > + v1 = VALnew(); > + VALcopy(v1,v); > + c = defConstant(repl, v1->vtype, v1); > + } > setArg(p1,i,c); > > if ((i > 0) || > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Visual Studio Solution/Project
by mobigital1 03 Mar '08

03 Mar '08

Does anyone know if it's possible to include a capability to generate a set of Visual Studio (2005 or 2008) project files for all components, together with solution file it would be really great to have. Because it would make it much easier to debug it on windows. perhaps someone already made such projects (even if they are outdated is ok too, I can modify to bring them up to date). -- View this message in context: http://www.nabble.com/Visual-Studio-Solution-Project-tp15640939p15640939.ht… Sent from the monetdb-developers mailing list archive at Nabble.com.

2 3

[Monetdb-developers] suggestions on debugging/tracing
by mobigital-nbl＠yahoo.com 02 Mar '08

02 Mar '08

Hi, I am trying to investigate this bug (https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1905467&group_…) which i submitted earlier. I was importing records with COPY TO a staging table in batches of 250K rows. Then copying them into final table. After completing 4,750,000 rows and importing a new batch thebelow exception occurs in GDKreallocmax at the last line of the snippetbelow: oldsize = GDK_MEM_BLKSIZE(s); with "s"=0x00000000100e0000" GDK_MEM_BLKSIZE macro tries to read 8 bytes prior to "s", which is 0x00000000100dfff8 and turns out to be unallocated. I am thinking the h->base of the heap is either free'ed at some point before being used or may be it is set to incorrect value? in any case i wanted to add some tracing to Heap to maintain some history of base change (and who did the change), so that when I hit another access violation during debugging I can use that to pinpoint why Heap base points to unallocated space. i wanted to see if anyone has already used something like that so that I can get a headstart quicker. if anyone have any suggestions on fixing it would be appreciated. Cheers.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_replicator.mx, , 1.28, 1.29
by Milena.Ivanova＠cwi.nl 02 Mar '08

02 Mar '08

> Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv3358 > > Modified Files: > opt_replicator.mx > Log Message: > defConstant already performs a search for the constant value. > > This was my first version too, but defConstant sets a pointer and does not copy the value. The next call finds whatever has left in memory and breaks. Milena > Index: opt_replicator.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_replicator.mx,v > retrieving revision 1.28 > retrieving revision 1.29 > diff -u -d -r1.28 -r1.29 > --- opt_replicator.mx 29 Feb 2008 15:43:20 -0000 1.28 > +++ opt_replicator.mx 2 Mar 2008 11:46:49 -0000 1.29 > @@ -438,20 +438,15 @@ > static int newReplica(MalStkPtr s, InstrPtr p, bit keep) > { > int i, j, k, c, bid, r; > - ValPtr v, v1; > + ValPtr v; > InstrPtr p1; > > assureSpace(); > p1 = copyInstruction(p); > for(i = 0; i< p->argc; i++){ > j = p->argv[i]; > - v = &s->stk[j]; > - c = fndConstant(repl, v); > - if ( c < 0 ){ > - v1 = VALnew(); > - VALcopy(v1,v); > - c = defConstant(repl, v1->vtype, v1); > - } > + v = &s->stk[j]; > + c = defConstant(repl, v->vtype, v); > setArg(p1,i,c); > > if ((i > 0) || > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_replicator.mx, , 1.23, 1.24
by Milena Ivanova 29 Feb '08

29 Feb '08

Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18911 > > Modified Files: > opt_replicator.mx > Log Message: > If you stop you should free the structure that you allocked in REPstart... > > Compilation works and also the replicator test... > > > Index: opt_replicator.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_replicator.mx,v > retrieving revision 1.23 > retrieving revision 1.24 > diff -u -d -r1.23 -r1.24 > --- opt_replicator.mx 29 Feb 2008 07:29:21 -0000 1.23 > +++ opt_replicator.mx 29 Feb 2008 13:13:20 -0000 1.24 > @@ -132,7 +132,7 @@ > opt_export str REPsemijoin(MalBlkPtr mb, MalStkPtr stk, InstrPtr pci); > opt_export str REPdump(int *ret); > opt_export str REPstart(MalBlkPtr mb, MalStkPtr stk, InstrPtr pci); > -opt_export str REPstop(void); > +opt_export str REPstop(MalStkPtr stk); > opt_export str REPsetRetain(int *ret, sht *p); > opt_export str REPsetReuse(int *ret, sht *p); > opt_export str REPsetCache(int *ret, sht *p); > @@ -242,8 +242,10 @@ > } > > > -str REPstop(void) > +str REPstop(MalStkPtr s) > { > + if( s->blk->replica ) > + GDKfree(((ReplicaRec*)s->blk->replica)->vmap); > replref--; > return MAL_SUCCEED; > } > In principle yes, but in this case I don't think so. The vmap is the map to replicated variables, that can be used next time the same query is executed, which is after rep.stop. Pls, revert Milena > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal tablet.mx, , 1.114, 1.115
by Romulo Goncalves 29 Feb '08

29 Feb '08

Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv3888/src/modules/mal > > Modified Files: > tablet.mx > Log Message: > correctly check for null values > > > Index: tablet.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet.mx,v > retrieving revision 1.114 > retrieving revision 1.115 > diff -u -d -r1.114 -r1.115 > --- tablet.mx 18 Feb 2008 19:29:07 -0000 1.114 > +++ tablet.mx 18 Feb 2008 19:57:37 -0000 1.115 > @@ -1264,9 +1264,8 @@ > e = s; > if ( s == NULL || > (s == e && fmt->nullstr[0] == 0) || Would not be better here to do only fmt->nullstr because if fmt->nullstr was initialized with NULL this if clause will crash the mserver... Romulo > - (!quote && strcasecmp(s,fmt->nullstr) == 0) || > - (quote && strncasecmp(s+1, fmt->nullstr, e-s) == 0) > - ) { > + (s != e && !quote && strcasecmp(s,fmt->nullstr) == 0) || > + (s != e && quote && strncasecmp(s+1, fmt->nullstr, e-s) == 0)) { > adt = fmt->data; > memcpy(adt, ATOMnilptr(fmt->adt), fmt->nillen); > } else { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] load runtime incorrectly
by mobigital-nbl＠yahoo.com 28 Feb '08

28 Feb '08

FYI, this issue turned out to be due to the linking to non-debug MSVCR90.lib in: 1. iconv project. - had to make sure that MDFlags in iconv makefile also changed to /MDd when debug was requested. 2. pthreads - changed /MD to /MDd in flags to each .c compilation to .obj, also added /MDd in final linking and removed explicit linking with libcrt.lib - because /MDd already does it with debug crt. > additional info: > > it seems even after DEBUG build, my executable is looking for non-DEBUG version > of MSVCR90.dll > > --------------------------- > mserver5.exe - Unable To Locate Component > --------------------------- > This application has failed to start because MSVCR90.dll was not found. > Re-installing the application may fix this problem. > --------------------------- > > when i copy the set of MSVCR90.dll and manifest into debug directory, i get the > error shown below.(my prior message) > > here are my build commands that i use for all modules: > > nmake /nologo BITS64=1 NEED_MX=1 DEBUG=1 HAVE_ICONV=1 HAVE_MONETDB4=1 > HAVE_MONETDB5=1 HAVE_PYTHON=1 "prefix=%BUILDTOOLS_PREFIX%" > > nmake /nologo BITS64=1 NEED_MX=1 DEBUG=1 HAVE_ICONV=1 HAVE_MONETDB4=1 > HAVE_MONETDB5=1 HAVE_PYTHON=1 "prefix=%BUILDTOOLS_PREFIX%" install > > > > > Does anyone know why i may be getting this error: > > > > Runtime Error! > > > > Program: C:\monetdb\build\bin\mserver5.exe > > > > R6034 > > > > An application has made an attempt to load the C runtime library incorrectly. > > Please contact the application's support team for more information. > > > > ----------- > > also is there a 'clean' target for make? > > i don't see it anywhere. > > > >

1 0

[Monetdb-developers] FYI: reversed nightly testing order: Current, Stable
by Stefan Manegold 27 Feb '08

27 Feb '08

Dear all, just to keep you posted: As of tonight, nightly MonetDB testing will (again) first run the "Current" development head and then the "Stable" release branch. (With the release out for just over a week, now, "urgent" bug-fixing and TestWeb clean-up has settled, making early-morning "Stable" testing results less needed. Instead, increased feature development activities ask for early-morning "Current" testing results.) Best regards from A'dam, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] load runtime incorrectly
by mobigital-nbl＠yahoo.com 27 Feb '08

27 Feb '08

additional info: it seems even after DEBUG build, my executable is looking for non-DEBUG version of MSVCR90.dll --------------------------- mserver5.exe - Unable To Locate Component --------------------------- This application has failed to start because MSVCR90.dll was not found. Re-installing the application may fix this problem. --------------------------- when i copy the set of MSVCR90.dll and manifest into debug directory, i get the error shown below.(my prior message) here are my build commands that i use for all modules: nmake /nologo BITS64=1 NEED_MX=1 DEBUG=1 HAVE_ICONV=1 HAVE_MONETDB4=1 HAVE_MONETDB5=1 HAVE_PYTHON=1 "prefix=%BUILDTOOLS_PREFIX%" nmake /nologo BITS64=1 NEED_MX=1 DEBUG=1 HAVE_ICONV=1 HAVE_MONETDB4=1 HAVE_MONETDB5=1 HAVE_PYTHON=1 "prefix=%BUILDTOOLS_PREFIX%" install > > Does anyone know why i may be getting this error: > > Runtime Error! > > Program: C:\monetdb\build\bin\mserver5.exe > > R6034 > > An application has made an attempt to load the C runtime library incorrectly. > Please contact the application's support team for more information. > > ----------- > also is there a 'clean' target for make? > i don't see it anywhere. > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

1 0

[Monetdb-developers] load runtime incorrectly
by mobigital-nbl＠yahoo.com 27 Feb '08

27 Feb '08

Does anyone know why i may be getting this error: Runtime Error! Program: C:\monetdb\build\bin\mserver5.exe R6034 An application has made an attempt to load the C runtime library incorrectly. Please contact the application's support team for more information. ----------- also is there a 'clean' target for make? i don't see it anywhere.

1 0

Re: [Monetdb-developers] iconv.dll DEBUG version
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

you're right the iconvd.lib and iconvd.dll were actually created, i just didn't see them. many thanks > >> Edit lib/Makefile.msvc and change -Feiconv.dll to -Feiconvd.dll, and > >> compile using > >> > >> nmake -f Makefile.msvc NO_NLS=1 DLL=1 MFLAGS=-MDd DEBUG=1 > >> PREFIX=C:\iconv-1.11.win64 > >> > >> This will fail to compile the programs, but should create the library. > > Did you read this last statement? Once you have the files iconvd.dll > and iconvd.lib, you don't need to go any further compiling the iconv > stuff. You only need these two files. Copy them to where you have your > iconv.dll and iconv.lib files and go back to compiling MonetDB. > > -- > Sjoerd Mullender

1 0

Re: [Monetdb-developers] pathfinder __func___ not declared
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

Is there a free version of intel compiler that will work? it looked to me that Intel compiler must be purchased? Stefan Manegold wrote: > Pathfinder requires C99 features that are not supported by the Microsoft > Visual C/C++ compiler; hence, on Windows, Pathfinder can only with the Intel > C/C++ compiler. See > http://monetdb.cwi.nl/projects/monetdb//MonetDB/Documentation/Windows-Insta… > for details. > > Stefan >

3 2

Re: [Monetdb-developers] iconv.dll DEBUG version
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

it ran for a while, but couldn't make iconv.lib, do I need to change that target to iconvd.lib somewhere in the makefile as well? NMAKE : fatal error U1073: don't know how to make '../lib/iconv.lib' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\ VC\BIN\amd64\nmake.exe"' : return code '0x2' Stop. Sjoerd Mullender wrote: > > Edit lib/Makefile.msvc and change -Feiconv.dll to -Feiconvd.dll, and > compile using > > nmake -f Makefile.msvc NO_NLS=1 DLL=1 MFLAGS=-MDd DEBUG=1 > PREFIX=C:\iconv-1.11.win64 > > This will fail to compile the programs, but should create the library. >

2 1

[Monetdb-developers] iconv.dll DEBUG version
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

can someone advise how to build the iconvd.dll (debug version) of iconv. on windows 64 bit platform.

2 1

[Monetdb-developers] pathfinder __func___ not declared
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

does anyone know what may be the issue causing this compilation error: this is when buildng "pathfinder" on win64. other components have built successfully - buildtoold, MonetDB, clients, MonetDB4, MonetDB5, sql. C:\monetdb\pathfinder\NT>nmake /nologo BITS64=1 NEED_MX=1 DEBUG=1 HAVE_ICONV=1 HAVE_MONETDB4=1 HAVE_MONETDB5=1 HAVE_PYTHON=1 "prefix=c:\monetdb\build" "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\nmake.exe" /nologo /f ".\..\Makefile.msc" "prefix=c:\monetdb\build" "bits=64" all if not exist "conf" mkdir "conf" copy ".\..\conf\Makefile.msc" "conf\Makefile" 1 file(s) copied. cd "conf" && "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\nmake.exe" /nologo DEBUG=1 "prefix=c:\monetdb\build" "bits=64" all if not exist "compiler" mkdir "compiler" copy ".\..\compiler\Makefile.msc" "compiler\Makefile" 1 file(s) copied. cd "compiler" && "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\nmake.exe" /nologo DEBUG=1 "prefix=c:\monetdb\build" "bits=64" all if not exist "utils" mkdir "utils" copy ".\..\..\compiler\utils\Makefile.msc" "utils\Makefile" 1 file(s) copied. cd "utils" && "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\nmake.exe" /nologo DEBUG=1 "prefix=c:\monetdb\build" "bits=64" all if not exist "schema" mkdir "schema" copy ".\..\..\compiler\schema\Makefile.msc" "schema\Makefile" 1 file(s) copied. cd "schema" && "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\nmake.exe" /nologo DEBUG=1 "prefix=c:\monetdb\build" "bits=64" all cl -GF -W3 -wd4273 -wd4102 -MDd -nologo -Zi -Od -D_DEBUG -RTC1 -ZI -I. -I.\..\.. "-IC:\Pthreads\include" -I.\..\..\..\compiler\schema "-I..\include" -I.\..\..\..\compiler\schema\..\include "-Ic:\monetdb\build\include\MonetDB" "-Ic:\monetdb\build\include\MonetDB\common" "-Ic:\monetdb\build\include\MonetDB\gdk" -Wp64 -DMILPRINT_SUMMER_IS_DEFAULT=1 -DALGEBRA_IS_DEFAULT=0 -DSQL_IS_DEFAULT=0 -DHAVE_LIBXML2 "-IC:\libxml2-2.6.30.win64\include" "-IC:\zlib-1.2.3.win64\include" "-IC:\iconv-1.11.win64\include" -DLIBSCHEMA -DCOMPILE_DL_SCHEMA -Fo"import.obj" -c ".\..\..\..\compiler\schema\import.c" cl : Command line warning D9038 : /ZI is not supported on this platform; enabling /Zi instead cl : Command line warning D9025 : overriding '/Zi' with '/ZI' cl : Command line warning D9035 : option 'Wp64' has been deprecated and will be removed in a future release import.c .\..\..\..\compiler\schema\import.c(392) : error C2065: '__func__' : undeclared identifier .\..\..\..\compiler\schema\import.c(392) : warning C4047: 'function' : 'const char *' differs in levels of indirection from 'int' .\..\..\..\compiler\schema\import.c(392) : warning C4024: 'PFmalloc_' : different types for formal and actual parameter 3 .\..\..\..\compiler\schema\import.c(397) : error C2059: syntax error : '{' .\..\..\..\compiler\schema\import.c(468) : error C2065: '__func__' : undeclared identifier .\..\..\..\compiler\schema\import.c(468) : warning C4047: 'function' : 'const char *' differs in levels of indirection from 'int' .\..\..\..\compiler\schema\import.c(468) : warning C4024: 'PFoops_' : different types for formal and actual parameter 3 .\..\..\..\compiler\schema\import.c(518) : error C2065: '__func__' : undeclared identifier .\..\..\..\compiler\schema\import.c(518) : warning C4047: 'function' : 'const char *' differs in levels of indirection from 'int' .\..\..\..\compiler\schema\import.c(518) : warning C4024: 'PFoops_' : different types for formal and actual parameter 3 .\..\..\..\compiler\schema\import.c(525) : error C2065: '__func__' : undeclared identifier .\..\..\..\compiler\schema\import.c(525) : warning C4047: 'function' : 'const char *' differs in levels of indirection from 'int' .\..\..\..\compiler\schema\import.c(525) : warning C4024: 'PFoops_' : different types for formal and actual parameter 3 .\..\..\..\compiler\schema\import.c(535) : warning C4244: 'return' : conversion from '__int64' to 'int', possible loss of data .\..\..\..\compiler\schema\import.c(607) : error C2065: '__func__' : undeclared identifier .\..\..\..\compiler\schema\import.c(607) : warning C4047: 'function' : 'const char *' differs in levels of indirection from 'int' .\..\..\..\compiler\schema\import.c(607) : warning C4024: 'PFoops_' : different types for formal and actual parameter 3 .\..\..\..\compiler\schema\import.c(663) : error C2065: '__func__' : undeclared identifier .\..\..\..\compiler\schema\import.c(663) : warning C4047: 'function' : 'const char *' differs in levels of indirection from 'int' .\..\..\..\compiler\schema\import.c(663) : warning C4024: 'PFoops_' : different types for formal and actual parameter 3 .\..\..\..\compiler\schema\import.c(674) : error C2065: '__func__' : undeclared identifier

2 1

Re: [Monetdb-developers] libxml2 LINK error on win64.
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

i found this reference to libxml2 issue: http://osdir.com/ml/lang.perl.xml/2004-03/msg00082.html it suggested to put undefs in the specific source files nanohttp.c and nanoftp.c i added to nanohttp.c: #undef SUPPORT_IP6 #undef HAVE_GETADDRINFO to nanoftp.c: #undef SUPPORT_IP6 after that it compiled and got built ok. > acconfig.h says: > /* Define if IPV6 support is there */ > #undef SUPPORT_IP6 > /* Define if getaddrinfo is there */ > #undef HAVE_GETADDRINFO > > > config.h.in says this (in 2 places) > /* Define if IPV6 support is there */ > #undef SUPPORT_IP6 > > > configure has this (not sure if it's used on windows) > ... > [line 26531] #define SUPPORT_IP6 > _ACEOF > > it seems IP6 support is turned off everywhere, do I need to pass a command line > argument to turn off IP6 support? > > here is what configure.in says > [line 498] > dnl ***********************Checking for availability of IPv6******************* > > AC_MSG_CHECKING([whether to enable IPv6]) > AC_ARG_ENABLE(ipv6, [ --enable-ipv6[[=yes/no]] enables compilation of IPv6 > code [[default=yes]]],, enable_ipv6=yes) > > what command line switch should I use to turn off IP6? > > thanks in advance.

1 0

Re: [Monetdb-developers] libxml2 LINK error on win64.
by mobigital-nbl＠yahoo.com 26 Feb '08

26 Feb '08

Sjoerd Mullender wrote: > The functions the linker complains about are freeaddrinfo and > getaddrinfo. These functions are only used in sections inside #ifdef > SUPPORT_IP6/#endif. So it seems you have the symbol SUPPORT_IP6 defined > somewhere. You need to find out where that is defined. My guess would > be: the file config.h. One possibility might be that you used the same > sources on a non-Windows system where the file config.h was created? > You may be able to remove the file config.h and rerun the cscript > command. That should recreate the file. config.h is in the main > directory (i.e. one up from the win32 directory where you need to build). I deleted config.h, it was re-created by cscript, but problem still persists. acconfig.h says: /* Define if IPV6 support is there */ #undef SUPPORT_IP6 /* Define if getaddrinfo is there */ #undef HAVE_GETADDRINFO config.h.in says this (in 2 places) /* Define if IPV6 support is there */ #undef SUPPORT_IP6 configure has this (not sure if it's used on windows) ... [line 26531] #define SUPPORT_IP6 _ACEOF it seems IP6 support is turned off everywhere, do I need to pass a command line argument to turn off IP6 support? here is what configure.in says [line 498] dnl ***********************Checking for availability of IPv6******************* AC_MSG_CHECKING([whether to enable IPv6]) AC_ARG_ENABLE(ipv6, [ --enable-ipv6[[=yes/no]] enables compilation of IPv6 code [[default=yes]]],, enable_ipv6=yes) what command line switch should I use to turn off IP6? thanks in advance.

1 0

[Monetdb-developers] libxml2 LINK error on win64.
by mobigital1 25 Feb '08

25 Feb '08

Hi, I am trying to build on windows 2003 x64, and was able to build a few prerequisites, however, libxml2 is not linking due to this error: nanohttp.obj : error LNK2019: unresolved external symbol __imp_freeaddrinfo referenced in function xmlNanoHTTPConnectHost nanohttp.obj : error LNK2019: unresolved external symbol __imp_getaddrinfo referenced in function xmlNanoHTTPConnectHost bin.msvc\libxml2.dll : fatal error LNK1120: 2 unresolved externals NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe"' : return code '0x460' Stop. I am enclosing the output of /verbose:lib Microsoft (R) Program Maintenance Utility Version 9.00.21022.08 Copyright (C) Microsoft Corporation. All rights reserved. link.exe /nologo /VERBOSE:lib /VERSION:2.6 /LIBPATH:bin.msvc /LIBPATH:C:\libxml2-2.6.30.win64\lib /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\ATLMFC\LIB\amd64" /OPT:NOWIN98 /DLL /IMPLIB:bin.msvc\libxml2.lib /OUT:bin.msvc\libxml2.dll int.msvc\c14n.obj int.msvc\catalog.obj int.msvc\chvalid.obj int.msvc\debugXML.obj int.msvc\dict.obj int.msvc\DOCBparser.obj int.msvc\encoding.obj int.msvc\entities.obj int.msvc\error.obj int.msvc\globals.obj int.msvc\hash.obj int.msvc\HTMLparser.obj int.msvc\HTMLtree.obj int.msvc\legacy.obj int.msvc\list.obj int.msvc\nanoftp.obj int.msvc\nanohttp.obj int.msvc\parser.obj int.msvc\parserInternals.obj int.msvc\pattern.obj int.msvc\relaxng.obj int.msvc\SAX2.obj int.msvc\SAX.obj int.msvc\schematron.obj int.msvc\threads.obj int.msvc\tree.obj int.msvc\uri.obj int.msvc\valid.obj int.msvc\xinclude.obj int.msvc\xlink.obj int.msvc\xmlIO.obj int.msvc\xmlmemory.obj int.msvc\xmlreader.obj int.msvc\xmlregexp.obj int.msvc\xmlmodule.obj int.msvc\xmlsave.obj int.msvc\xmlschemas.obj int.msvc\xmlschemastypes.obj int.msvc\xmlunicode.obj int.msvc\xmlwriter.obj int.msvc\xpath.obj int.msvc\xpointer.obj int.msvc\xmlstring.obj wsock32.lib iconv.lib zlib1.lib kernel32.lib LINK : warning LNK4224: /OPT:NOWIN98 is no longer supported; ignored Searching libraries Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\wsock32.lib: Searching C:\iconv-1.11.win64\lib\iconv.lib: Searching C:\zlib-1.2.3.win64\lib\zlib1.lib: Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\kernel32.lib: Searching C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64\MSVCRT.lib: Searching C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64\OLDNAMES.lib: Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\uuid.lib: Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\wsock32.lib: Searching C:\iconv-1.11.win64\lib\iconv.lib: Searching C:\zlib-1.2.3.win64\lib\zlib1.lib: Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\kernel32.lib: Searching C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64\MSVCRT.lib: Finished searching libraries Creating library bin.msvc\libxml2.lib and object bin.msvc\libxml2.exp Searching libraries Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\wsock32.lib: Searching C:\iconv-1.11.win64\lib\iconv.lib: Searching C:\zlib-1.2.3.win64\lib\zlib1.lib: Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\kernel32.lib: Searching C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64\MSVCRT.lib: Searching C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\LIB\amd64\OLDNAMES.lib: Searching C:\Program Files\\Microsoft SDKs\Windows\v6.0A\lib\x64\uuid.lib: Finished searching libraries nanohttp.obj : error LNK2019: unresolved external symbol __imp_freeaddrinfo referenced in function xmlNanoHTTPConnectHost nanohttp.obj : error LNK2019: unresolved external symbol __imp_getaddrinfo referenced in function xmlNanoHTTPConnectHost bin.msvc\libxml2.dll : fatal error LNK1120: 2 unresolved externals NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe"' : return code '0x460' Stop. Any advice on how to address this would be appreciated! -- View this message in context: http://www.nabble.com/libxml2-LINK-error-on-win64.-tp15673701p15673701.html Sent from the monetdb-developers mailing list archive at Nabble.com.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra algebra_cse.c, XQuery_0-22, 1.66, 1.66.2.1
by Stefan Manegold 22 Feb '08

22 Feb '08

On Fri, Feb 22, 2008 at 08:35:12PM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15561/algebra > > Modified Files: > Tag: XQuery_0-22 > algebra_cse.c > Log Message: > -- Properly initialize the subexpression list > > (Please do not propagate to the CVS HEAD. > A better solution has to be found.) propagation has been prevented. Stefan > > Index: algebra_cse.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/algebra_cse.c,v > retrieving revision 1.66 > retrieving revision 1.66.2.1 > diff -u -d -r1.66 -r1.66.2.1 > --- algebra_cse.c 1 Feb 2008 13:31:59 -0000 1.66 > +++ algebra_cse.c 22 Feb 2008 20:35:10 -0000 1.66.2.1 > @@ -582,6 +582,9 @@ > PFla_op_t *res; > > subexps = PFarray (sizeof (PFarray_t *)); > + /* properly initialize list of subexpressions */ > + for (unsigned int i = 0; i < la_dummy /* the last op */; i++) > + *(PFarray_t **) PFarray_at (subexps, i) = NULL; > > res = la_cse (n); > PFla_dag_reset (res); -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] java/src/nl/cwi/monetdb/jdbc MonetConnection.java, , 1.2, 1.3
by Stefan Manegold 20 Feb '08

20 Feb '08

Maybe a bit late for this time, but might help next time: history of moved files is still in CVS at their original location, see cvs -d<user>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb rlog -N clients/src/java/src/nl/cwi/monetdb/jdbc/MonetConnection.java Stefan On Wed, Feb 20, 2008 at 07:50:26PM +0000, Fabian wrote: > Update of /cvsroot/monetdb/java/src/nl/cwi/monetdb/jdbc > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12362/src/nl/cwi/monetdb/jdbc > > Modified Files: > MonetConnection.java > Log Message: > Fix for bug #1886326: > MonetDB JDBC driver exception for XQuery with no results > > 2 and a half hour later ... > Due to missing history (cvs moves) and missing documentation (on my > end) it took quite some while to figure out what was the "correct" way > to solve this bug. In the end it al boiled down on the limitation of > the XQuery backend to only support full-results-at-once, which means the > resource allocation of JDBC needs to allocate the entire result at once. > If the result is 0 rows big, however, then things went wrong when > calculating the current cache block. Anyway, for the future we now know > why we did this cache block calulation thing like this again... > > > > Index: MonetConnection.java > =================================================================== > RCS file: /cvsroot/monetdb/java/src/nl/cwi/monetdb/jdbc/MonetConnection.java,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- MonetConnection.java 11 Jan 2008 10:36:16 -0000 1.2 > +++ MonetConnection.java 20 Feb 2008 19:50:24 -0000 1.3 > @@ -1164,7 +1164,17 @@ > isSet = new boolean[7]; > this.parent = parent; > if (parent.cachesize == 0) { > - cacheSize = lang == LANG_SQL ? MonetConnection.DEF_FETCHSIZE : tuplecount; > + /* Below we have to calculate how many "chunks" we need > + * to allocate to store the entire result. However, if > + * the user didn't set a cache size, as in this case, we > + * need to stick to our defaults. So far, so good. Now > + * the problem with XQuery is, that it doesn't support > + * any block fetching, so we need to always fetch > + * everything at once. For that reason, the cache size > + * is here set to the tuplecount, such that we do a full > + * fetch at once. To avoid a division by zero lateron, > + * we make sure the cache size is not 0 */ > + cacheSize = lang == LANG_SQL ? MonetConnection.DEF_FETCHSIZE : (tuplecount + 1); > cacheSizeSetExplicitly = false; > } else { > cacheSize = parent.cachesize; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 3

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra physical.c, XQuery_0-22, 1.55, 1.55.2.1
by Stefan Manegold 20 Feb '08

20 Feb '08

On Wed, Feb 20, 2008 at 03:05:25PM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15240/algebra > > Modified Files: > Tag: XQuery_0-22 > physical.c > Log Message: > -- Testing (XMach-1/q8) correctly complains about the wrong input order. > > kintersect seems to respect the order of the right operand instead of the > order of the left one. AFAIK, kintersect does not guarantee any order at all! (Just recalling from the back of my head; did not check the code, again...) By default, it will scan the left input and look-up into the right input, thus "by coincedence" maintain the order of the left input. However, in case it "thinks" that scanning the right and looking up into the left is more efficient (e.g., in case the is already a hash index on the left head, or the left head is a dense OID column, allowing positional lookup), it might switch the scanning the right and looking-up into the left. We call this "operational optimization" ... In case pathfinder indeed relies on any particular order-preserving kintersect, we should carefully double-check the above, and if necessary consider adding an explicit "left_kintersect" (and/or "right_kintersect") primitive analogously to the left*join and ord_*select primitives ... Stefan > > > Index: physical.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/physical.c,v > retrieving revision 1.55 > retrieving revision 1.55.2.1 > diff -u -d -r1.55 -r1.55.2.1 > --- physical.c 31 Jan 2008 21:43:33 -0000 1.55 > +++ physical.c 20 Feb 2008 15:05:22 -0000 1.55.2.1 > @@ -718,9 +718,10 @@ > /* ---- SemiJoin: orderings ---- */ > > /* > - * We preserve any ordering of the left operand. > + * kintersect (the MIL implementation) preserves the > + * ordering of the right operand. > */ > - ret->orderings = PFord_unique (n1->orderings); > + ret->orderings = PFord_unique (n2->orderings); > > /* ---- SemiJoin: costs ---- */ > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/sql lalg2sql.brg, , 1.119, 1.120
by Manuel Mayr 20 Feb '08

20 Feb '08

Sorry, I omitted the Log message here. Bugfix: missing "break" added, after the action code to translate attribtues. Manuel Mayr wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/sql > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv32182/compiler/sql > > Modified Files: > lalg2sql.brg > Log Message: > > > Index: lalg2sql.brg > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/sql/lalg2sql.brg,v > retrieving revision 1.119 > retrieving revision 1.120 > diff -u -d -r1.119 -r1.120 > --- lalg2sql.brg 20 Feb 2008 12:26:56 -0000 1.119 > +++ lalg2sql.brg 20 Feb 2008 13:08:35 -0000 1.120 > @@ -4366,7 +4366,7 @@ > col_env_lookup (COLMAP(L(p)), item2, type_of (L(p), item2)), > col_env_lookup (COLMAP(L(p)), item1, aat_qname_loc), > col_env_lookup (COLMAP(L(p)), item1, aat_qname_uri)); > - } > + } break; > /* Twig: processi (Rel) */ > case 87: > { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Manuel Mayr, Dipl.-Inf. | | | | EMail: manuel.mayr(a)in.tum.de | | Web: http://www3.in.tum.de/cms/members/mayrm | | Phone: +49 89 289-17259 | | Fax: +49 89 289-17263 | | | | Database Systems - Technische Universität München (Germany) |

1 0

[Monetdb-developers] preceding-sibling
by Djoerd Hiemstra 20 Feb '08

20 Feb '08

Hi there MonetDBs, The results for the following query changed between v0.18 and v0.22: let $list := <list> <item>1</item> <item>2</item> <item>3</item> <item>4</item> </list> return $list//item[. = "4"]/preceding-sibling::*[1] It now gives item 3, which I think is wrong since the results of any axis step (also preceding-sibling), should be in document order. The result in version 0.18 is item 1. Best, Djoerd.

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra/opt opt_algebra_cse.c, XQuery_0-22, 1.15.2.4, 1.15.2.5
by Manuel Mayr 19 Feb '08

19 Feb '08

No that's fine, I forgot to remove this one. Thx and best regards, Manuel On Feb 19, 2008, at 9:14 PM, Stefan Manegold wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra/opt > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15698/ > compiler/algebra/opt > > Modified Files: > Tag: XQuery_0-22 > opt_algebra_cse.c > Log Message: > > making pathfinder compile with "--enable-strict", again: > removed unused variable. > > Manuel, > > please check, that I did not break anything ... > > > > > Index: opt_algebra_cse.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/opt/ > opt_algebra_cse.c,v > retrieving revision 1.15.2.4 > retrieving revision 1.15.2.5 > diff -u -d -r1.15.2.4 -r1.15.2.5 > --- opt_algebra_cse.c 19 Feb 2008 17:39:11 -0000 1.15.2.4 > +++ opt_algebra_cse.c 19 Feb 2008 20:14:42 -0000 1.15.2.5 > @@ -1964,7 +1964,6 @@ > PFla_op_t *seed = CSE(L(n)); > PFla_op_t *result = CSE(R(n)); > > - PFalg_proj_t *p = NULL; > PFarray_t *seed_actmap = create_actatt_map (); > PFarray_t *res_actmap = create_actatt_map (); > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins | Manuel Mayr, Dipl.- Inf . | | | | EMail: manuel .mayr @in.tum.de | | Web: http://www3.in.tum.de/cms/members/ mayrm | | Phone: +49 89 289 -17259 | | Fax: +49 89 289 -17263 | | | | Database Systems - Technische Universität München (Germany) |

1 0

[Monetdb-developers] New MonetDB Release
by Stefan Manegold 19 Feb '08

19 Feb '08

The MonetDB team at CWI is pleased to announce the new "Feb2008" release of the MonetDB suite of programs. This release marks the roll forward of the binary storage layout to a single column storage layout. This involved a major overhaul of the MonetDB Server code. IMPORTANT NOTE: The database format of this release is not compatible with that of any previous release. Before installing this release, make a dump of your database, and do a restore afterwards. Further changes include: SQL: The functional extensions to SQL are limited. The mostly concern minor bugs and some of the optimizers have been improved to better exploit re-occurring joins. The core cracker code has been cleaned up for better inspection and sharing in the community. XQuery: Next to the original Pathfinder compiler (nick-named "milprint_summer" or simply "MPS"), the development of the new improved and enhanced Algebra-based Pathfinder compiler ("ALG") is progressing quickly. While the original MPS version remains the default in this release of MonetDB/XQuery, a preview of the new ALG version is now available without recompilation. JDBC & XRPC: The Java code (JDBC driver & XRPC wrapper) has been moved from the clients and pathfinder source trees to a dedicated java source tree in CVS, which can simply be compiled using "ant". Apart from this, many bugs have been fixed, and many speed improvements have been made. See http://monetdb.cwi.nl/Development/Releases/Version4.22/ for more details. The new release consists of the following packages: MonetDB Common 1.22.0 MonetDB Clients 1.22.0 MonetDB4 Server 4.22.0 MonetDB XQuery 0.22.0 MonetDB5 Server 5.4.0 MonetDB SQL 2.22.0 monetdb-1.7-jdbc.jar There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] PF: 334 tests fail with ALG that work fine with MPS
by Stefan Manegold 16 Feb '08

16 Feb '08

Hi, those of you involved it Pathfinder development but not following the Monetdb-bugs mailing list closely, might want to have a look at this one: [ 1894891 ] PF: 334 tests fail with ALG that work fine with MPS http://sourceforge.net/tracker/index.php?func=detail&aid=1894891&group_id=5… Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Releasing: Please do not check-in to Stable release branches!
by Stefan Manegold 15 Feb '08

15 Feb '08

Dear all, we finally plan to produce a new release tomorrow (Wednesday) or Thursday. To ensure a smooth process, please refrain as of now from any check-ins to the Stable release branches until we're done and notify you accordingly. In case there are any (unexpected) urgend changes to be checked in, please check with Niels, Sjoerd and/or me, first! Thank you very much for your cooperation! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra builtins.c, , 1.79, 1.80
by Lefteris 15 Feb '08

15 Feb '08

Jan, is this the correct way to cope with empty input? The xquery function specs say that if input is ommited then it defaults to the context item (.) That also is expresed in xquery_fo.c with the .arity = 0 version of fn:name etc. If that is true, shouldn't be dealed at the core level? lefteris On Wed, Feb 13, 2008 at 9:54 PM, Jan Rittinger <tsheyar(a)users.sourceforge.net> wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18492/algebra > > Modified Files: > builtins.c > Log Message: > -- Fix the translation of fn:name, fn:local-name, and fn:namespace-uri. > (All three have to cope with empty input.) > > > Index: builtins.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/builtins.c,v > retrieving revision 1.79 > retrieving revision 1.80 > diff -u -d -r1.79 -r1.80 > --- builtins.c 6 Feb 2008 16:36:28 -0000 1.79 > +++ builtins.c 13 Feb 2008 20:54:17 -0000 1.80 > @@ -2350,6 +2350,10 @@ > * is not even the right function to use there, because the constructors > * should do their job based on _statically-known namespace declarations_, > * not in-scope declarations. > + * > + * Furthermore a cast from string to xs:QName is only allowed at compile > + * time on constant strings. At runtime a cast to xs:QName is only allowed > + * for QNames inputs. > */ > struct PFla_pair_t > PFbui_fn_resolve_qname (const PFla_op_t *loop, bool ordering, > @@ -2416,22 +2420,34 @@ > struct PFla_pair_t > PFbui_fn_name (const PFla_op_t *loop, bool ordering, struct PFla_pair_t *args) > { > - (void) loop; (void) ordering; > + (void) ordering; > > - return (struct PFla_pair_t) { > - .rel = project ( > - fun_1to1 ( > - project (args[0].rel, > - proj (att_iter, att_iter), > - proj (att_pos, att_pos), > - proj (att_item, att_item)), > - alg_fun_fn_name, > - att_res, > - attlist(att_item)), > - proj (att_iter, att_iter), > - proj (att_pos, att_pos), > - proj (att_item, att_res)), > - .frag = PFla_empty_set() }; > + PFla_op_t *strings = project ( > + fun_1to1 ( > + project (args[0].rel, > + proj (att_iter, att_iter), > + proj (att_pos, att_pos), > + proj (att_item, att_item)), > + alg_fun_fn_name, > + att_res, > + attlist(att_item)), > + proj (att_iter, att_iter), > + proj (att_pos, att_pos), > + proj (att_item, att_res)); > + > + PFla_op_t *res = disjunion ( > + strings, > + attach ( > + attach ( > + difference ( > + loop, > + project ( > + strings, > + proj (att_iter, att_iter))), > + att_pos, lit_nat (1)), > + att_item, lit_str (""))); > + > + return (struct PFla_pair_t) { .rel = res, .frag = PFla_empty_set() }; > } > > /* ------------------- */ > @@ -2442,22 +2458,34 @@ > PFbui_fn_local_name (const PFla_op_t *loop, bool ordering, > struct PFla_pair_t *args) > { > - (void) loop; (void) ordering; > + (void) ordering; > > - return (struct PFla_pair_t) { > - .rel = project ( > - fun_1to1 ( > - project (args[0].rel, > - proj (att_iter, att_iter), > - proj (att_pos, att_pos), > - proj (att_item, att_item)), > - alg_fun_fn_local_name, > - att_res, > - attlist(att_item)), > - proj (att_iter, att_iter), > - proj (att_pos, att_pos), > - proj (att_item, att_res)), > - .frag = args[0].frag }; > + PFla_op_t *strings = project ( > + fun_1to1 ( > + project (args[0].rel, > + proj (att_iter, att_iter), > + proj (att_pos, att_pos), > + proj (att_item, att_item)), > + alg_fun_fn_local_name, > + att_res, > + attlist(att_item)), > + proj (att_iter, att_iter), > + proj (att_pos, att_pos), > + proj (att_item, att_res)); > + > + PFla_op_t *res = disjunion ( > + strings, > + attach ( > + attach ( > + difference ( > + loop, > + project ( > + strings, > + proj (att_iter, att_iter))), > + att_pos, lit_nat (1)), > + att_item, lit_str (""))); > + > + return (struct PFla_pair_t) { .rel = res, .frag = PFla_empty_set() }; > } > > /* --------------------- */ > @@ -2468,22 +2496,34 @@ > PFbui_fn_namespace_uri (const PFla_op_t *loop, bool ordering, > struct PFla_pair_t *args) > { > - (void) loop; (void) ordering; > + (void) ordering; > > - return (struct PFla_pair_t) { > - .rel = project ( > - fun_1to1 ( > - project (args[0].rel, > - proj (att_iter, att_iter), > - proj (att_pos, att_pos), > - proj (att_item, att_item)), > - alg_fun_fn_namespace_uri, > - att_res, > - attlist(att_item)), > - proj (att_iter, att_iter), > - proj (att_pos, att_pos), > - proj (att_item, att_res)), > - .frag = args[0].frag }; > + PFla_op_t *strings = project ( > + fun_1to1 ( > + project (args[0].rel, > + proj (att_iter, att_iter), > + proj (att_pos, att_pos), > + proj (att_item, att_item)), > + alg_fun_fn_namespace_uri, > + att_res, > + attlist(att_item)), > + proj (att_iter, att_iter), > + proj (att_pos, att_pos), > + proj (att_item, att_res)); > + > + PFla_op_t *res = disjunion ( > + strings, > + attach ( > + attach ( > + difference ( > + loop, > + project ( > + strings, > + proj (att_iter, att_iter))), > + att_pos, lit_nat (1)), > + att_item, lit_str (""))); > + > + return (struct PFla_pair_t) { .rel = res, .frag = PFla_empty_set() }; > } > > /* --------------- */ > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins >

3 3

[Monetdb-developers] python error during bootstrap
by Ying Zhang 14 Feb '08

14 Feb '08

Hi all, after cvs update all modules, rebuild monet from scratch (i.e. boostrap, confgure, make, make install). all other modules (i.e. buildtools, MonetDB, clients, java, MonetDB4, MonetDB5, sql) can be build without problem, however bootstrap pathfinder results in some python errors: [andorea:/ufs/zhang/monet/dev/pathfinder] $ ./bootstrap automake 1.10 is 1.7 or newer. Good. autoconf 2.61 is 2.57 or newer. Good. libtool 1.5.24 is 1.4 or newer. Good. Python 2.5.1 is 2.0.0 or newer. Good. /ufs/zhang/monet/dev/pathfinder/conf /ufs/zhang/monet/dev/pathfinder/compiler /ufs/zhang/monet/dev/pathfinder/compiler/utils /ufs/zhang/monet/dev/pathfinder/compiler/schema /ufs/zhang/monet/dev/pathfinder/compiler/debug /ufs/zhang/monet/dev/pathfinder/compiler/mem /ufs/zhang/monet/dev/pathfinder/compiler/parser /ufs/zhang/monet/dev/pathfinder/compiler/semantics /ufs/zhang/monet/dev/pathfinder/compiler/core /ufs/zhang/monet/dev/pathfinder/compiler/algebra /ufs/zhang/monet/dev/pathfinder/compiler/algebra/opt /ufs/zhang/monet/dev/pathfinder/compiler/algebra/prop /ufs/zhang/monet/dev/pathfinder/compiler/algebra/map /ufs/zhang/monet/dev/pathfinder/compiler/mil /ufs/zhang/monet/dev/pathfinder/compiler/include /ufs/zhang/monet/dev/pathfinder/compiler/sql /ufs/zhang/monet/dev/pathfinder/compiler/xmlimport /ufs/zhang/monet/dev/pathfinder/runtime ('error', '}', '=') Traceback (most recent call last): File "/export/scratch1/zhang/monet-install/dev/debug/bin/ autogen.py", line 163, in <module> (InstallList, DocList, OutList) = main(topdir, topdir, automake, []) File "/export/scratch1/zhang/monet-install/dev/debug/bin/ autogen.py", line 146, in main main(d, topdir, automake, incdirsmap, conditional + cond) File "/export/scratch1/zhang/monet-install/dev/debug/bin/ autogen.py", line 133, in main (InstallList, DocList, OutList) = am.output(p.curvar, cwd, topdir, automake, conditional) File "/export/scratch1/zhang/monet-install/dev/debug/lib/python2.5/ site-packages/autogen/am.py", line 1097, in output output_funcs[i](fd, i, v, am) File "/export/scratch1/zhang/monet-install/dev/debug/lib/python2.5/ site-packages/autogen/am.py", line 809, in am_libs if (libsmap.has_key("DIR")): AttributeError: 'list' object has no attribute 'has_key' Does someone has a idea what's happening here? I'm using the current branch. OS info: $ uname -a Linux andorea.ins.cwi.nl 2.6.23.15-137.fc8 #1 SMP Sun Feb 10 17:03:13 EST 2008 x86_64 x86_64 x86_64 GNU/Linux Thanks, Jennie

1 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/tests/XRpc/Tests add_trusted.mil, XQuery_0-22, 1.2, 1.2.12.1
by Stefan Manegold 13 Feb '08

13 Feb '08

On Wed, Feb 13, 2008 at 02:50:03PM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/tests/XRpc/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv23952/tests/XRpc/Tests > > Modified Files: > Tag: XQuery_0-22 > add_trusted.mil > Log Message: > [ monetdb-Bugs-1892582 ] PF: XRpc does not work on Windows > > Don't use hard coded directory name, '/' as directory separator will not > work on Windows. Thanks --- although I have NOT released the CVS check-in lock for the branches, yet ... Stefan > Index: add_trusted.mil > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/tests/XRpc/Tests/add_trusted.mil,v > retrieving revision 1.2 > retrieving revision 1.2.12.1 > diff -u -d -r1.2 -r1.2.12.1 > --- add_trusted.mil 22 Feb 2007 11:34:56 -0000 1.2 > +++ add_trusted.mil 13 Feb 2008 14:50:00 -0000 1.2.12.1 > @@ -1,7 +1,7 @@ > module("pathfinder"); > module("unix"); > > -var TSTSRCDIR := getenv("TSTSRCBASE") + "/tests/XRpc/Tests"; > +var TSTSRCDIR := getenv("TSTSRCDIR"); > var TSTTRGDIR := getenv("TSTTRGDIR"); > > add_xrpc_trusted(TSTSRCDIR); > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] monetdb/SQL development plan
by Marcin Zukowski 12 Feb '08

12 Feb '08

Hi Michael, Thank you for your interest in MonetDB. Since I'm not involved with the core MonetDB development, I forward this email to the monetdb-developers list. I'm sure the guys over there will be able to help you. All the best, Marcin Michael Yang wrote: > Hi Marcin, > > I saw that you're one of Peter's PhD students. I just got introduced > to MonetDB as a possible column-store solution for some time series I > work with. I'm in financial mathematics here at Princeton in the > Operations Research department. > > I am a heavy user of MS SQL Server and I use aggregate ranking > functions a lot - row_number(), rank(), etc. Could you give me a > rough estimate as to when MonetDB might add support for these > functions - LEAD(), LAG(), and X() over (partition by ... order by > ...). I see that it's in the development plan, but will it take 1 > year? 1 month? many years? > > Also, in the meantime is there a way you know to emulate these > aggregate functions using local variables in monetdb? > > thanks, > Michael Yang > > Princeton University Operations Research and Financial Engineering

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB4/src/modules/plain bat_arith.mx, MonetDB_4-22, 1.4.2.2, 1.4.2.3
by Stefan Manegold 12 Feb '08

12 Feb '08

my below fixes in MonetDB4/src/modules/plain/bat_arith.mx would need to be propagated / applied to MonetDB5/src/modules/kernel/batcalc.mx too, but a simple diff|patch does not work, and I haven't found time to do it by hand, yet, sorry ... Stefan On Sun, Feb 10, 2008 at 10:46:53PM +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB4/src/modules/plain > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv13856 > > Modified Files: > Tag: MonetDB_4-22 > bat_arith.mx > Log Message: > > fixed bug > [ 1890131 ] PF: property error: incorrectly marked sorted! > http://sourceforge.net/tracker/index.php?func=detail&aid=1890131&group_id=5… > > fixed sortedness propagation for "const [op] BAT" & "BAT [op] const", > in particular three cases: > - substracting a BAT from a constant > yields a result with the input's sort order reversed > - dividing a BAT by a negative constant > yields a result with the input's sort order reversed > - for modulo (%) (also on) float & double > we cannot predict any result order > > > Index: bat_arith.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB4/src/modules/plain/bat_arith.mx,v > retrieving revision 1.4.2.2 > retrieving revision 1.4.2.3 > diff -u -d -r1.4.2.2 -r1.4.2.3 > --- bat_arith.mx 8 Feb 2008 15:02:14 -0000 1.4.2.2 > +++ bat_arith.mx 10 Feb 2008 22:46:51 -0000 1.4.2.3 > @@ -57,42 +57,42 @@ > .END bat_arith; > > @= operations > -@:@1_operation(bte,+,add,,0,2)@ > -@:@1_operation(bte,-,sub,,0,2)@ > -@:@1_operation(bte,*,mul,1,0,3)@ > -@:@1_operation(bte,/,div,1,1,1)@ > +@:@1_operation(bte,+,add,,0,(1|2))@ > +@:@1_operation(bte,-,sub,,0,(1|8))@ > +@:@1_operation(bte,*,mul,1,0,(16|32))@ > +@:@1_operation(bte,/,div,1,1,16)@ > @:@1_operation(bte,%,mod,1,1,0)@ > -@:@1_operation(sht,+,add,,0,2)@ > -@:@1_operation(sht,-,sub,,0,2)@ > -@:@1_operation(sht,*,mul,1,0,3)@ > -@:@1_operation(sht,/,div,1,1,1)@ > +@:@1_operation(sht,+,add,,0,(1|2))@ > +@:@1_operation(sht,-,sub,,0,(1|8))@ > +@:@1_operation(sht,*,mul,1,0,(16|32))@ > +@:@1_operation(sht,/,div,1,1,16)@ > @:@1_operation(sht,%,mod,1,1,0)@ > -@:@1_operation(int,+,add,,0,2)@ > -@:@1_operation(int,-,sub,,0,2)@ > -@:@1_operation(int,*,mul,1,0,3)@ > -@:@1_operation(int,/,div,1,1,1)@ > +@:@1_operation(int,+,add,,0,(1|2))@ > +@:@1_operation(int,-,sub,,0,(1|8))@ > +@:@1_operation(int,*,mul,1,0,(16|32))@ > +@:@1_operation(int,/,div,1,1,16)@ > @:@1_operation(int,%,mod,1,1,0)@ > -@:@1_operation(wrd,+,add,,0,2)@ > -@:@1_operation(wrd,-,sub,,0,2)@ > -@:@1_operation(wrd,*,mul,1,0,3)@ > -@:@1_operation(wrd,/,div,1,1,1)@ > +@:@1_operation(wrd,+,add,,0,(1|2))@ > +@:@1_operation(wrd,-,sub,,0,(1|8))@ > +@:@1_operation(wrd,*,mul,1,0,(16|32))@ > +@:@1_operation(wrd,/,div,1,1,16)@ > @:@1_operation(wrd,%,mod,1,1,0)@ > -@:@1_operation(lng,+,add,,0,2)@ > -@:@1_operation(lng,-,sub,,0,2)@ > -@:@1_operation(lng,*,mul,1,0,3)@ > -@:@1_operation(lng,/,div,1,1,1)@ > +@:@1_operation(lng,+,add,,0,(1|2))@ > +@:@1_operation(lng,-,sub,,0,(1|8))@ > +@:@1_operation(lng,*,mul,1,0,(16|32))@ > +@:@1_operation(lng,/,div,1,1,16)@ > @:@1_operation(lng,%,mod,1,1,0)@ > -@:@1_operation(flt,+,add,,0,2)@ > -@:@1_operation(flt,-,sub,,0,2)@ > -@:@1_operation(flt,*,mul,1,0,3)@ > -@:@1_operation(flt,/,div,1,1,1)@ > -@:@1_operation(dbl,+,add,,0,2)@ > -@:@1_operation(dbl,-,sub,,0,2)@ > -@:@1_operation(dbl,*,mul,1,0,3)@ > -@:@1_operation(dbl,/,div,1,1,1)@ > +@:@1_operation(flt,+,add,,0,(1|2))@ > +@:@1_operation(flt,-,sub,,0,(1|8))@ > +@:@1_operation(flt,*,mul,1,0,(16|32))@ > +@:@1_operation(flt,/,div,1,1,16)@ > +@:@1_operation(dbl,+,add,,0,(1|2))@ > +@:@1_operation(dbl,-,sub,,0,(1|8))@ > +@:@1_operation(dbl,*,mul,1,0,(16|32))@ > +@:@1_operation(dbl,/,div,1,1,16)@ > @:@1_redefine_OP()@ > -@:@1_operation(flt,%,mod,1,1,1,%)@ > -@:@1_operation(dbl,%,mod,1,1,1,%)@ > +@:@1_operation(flt,%,mod,1,1,0,%)@ > +@:@1_operation(dbl,%,mod,1,1,0,%)@ > > @* Implementation > @c > @@ -108,9 +108,13 @@ > @3: add, sub, mul, div, mod basic arithmetic operator name > @4: not used > @5: 0 / 1 disable/enable division by zero check > - @6: 0 / 1 / 2 disable/enable sorting propagation > - 1 propagate on bat,const > - 2 propagate also on const,bat > + @6: (bit pattern) disable/enable sorting propagation > + 1 propagate as-is on bat,const > + 2 propagate as-is on const,bat > + 4 propagate reverted on bat,const (not required/used!) > + 8 propagate reverted on const,bat > + 16 propagate as-is / reverted on bat,const if const >=0 / <0 > + 32 propagate as-is / reverted on const,bat if const >=0 / <0 > */ > @= c_operation > int > @@ -201,7 +205,6 @@ > if (bn == NULL) > return GDK_FAIL; > BATseqbase(bn,b->hseqbase); > - bn->tsorted = 0; > bq = (@1*)Tloc(b,BUNlast(b)); > bp = (@1*)Tloc(b,BUNfirst(b)); > s = bnp = (@1*)Tloc(bn,BUNlast(bn)); > @@ -225,13 +228,19 @@ > BATsetcount(bn, bnp-s); > if (!bn->batDirty) bn->batDirty = TRUE; > > - if (@6 == 2) > + if (@6 & 2) { > bn->tsorted = BATtordered(b); > - if (@6 == 3) { > - if (*v > 0) > + } else > + if (@6 & 8) { > + bn->tsorted = REVERT_SORTED(BATtordered(b)); > + } else > + if (@6 & 32) { > + if (*v >= 0) > bn->tsorted = BATtordered(b); > else > bn->tsorted = REVERT_SORTED(BATtordered(b)); > + } else { > + bn->tsorted = 0; > } > > *ret = bn; > @@ -264,15 +273,20 @@ > } > if (!b->batDirty) b->batDirty = TRUE; > > - if (@6 == 2) > + if (@6 & 2) { > b->tsorted = BATtordered(b); > - if (@6 == 3) { > - if (*v > 0) > + } else > + if (@6 & 8) { > + b->tsorted = REVERT_SORTED(BATtordered(b)); > + } else > + if (@6 & 32) { > + if (*v >= 0) > b->tsorted = BATtordered(b); > else > b->tsorted = REVERT_SORTED(BATtordered(b)); > - } else > + } else { > b->tsorted = 0; > + } > > BBPfix(b->batCacheid); > *ret = b; > @@ -294,7 +308,6 @@ > if( bn == NULL) > return GDK_FAIL; > BATseqbase(bn,b->hseqbase); > - bn->tsorted = 0; > bq = (@1*)Tloc(b,BUNlast(b)); > bp = (@1*)Tloc(b,BUNfirst(b)); > s = bnp = (@1*)Tloc(bn,BUNlast(bn)); > @@ -317,14 +330,21 @@ > } > BATsetcount(bn, bnp-s); > if (!bn->batDirty) bn->batDirty = TRUE; > - if (@6 == 2) > + > + if (@6 & 1) { > bn->tsorted = BATtordered(b); > - if (@6 == 3) { > - if (*v > 0) > + } else > + if (@6 & 4) { > + bn->tsorted = REVERT_SORTED(BATtordered(b)); > + } else > + if (@6 & 16) { > + if (*v >= 0) > bn->tsorted = BATtordered(b); > else > bn->tsorted = REVERT_SORTED(BATtordered(b)); > - } > + } else { > + bn->tsorted = 0; > + } > > *ret = bn; > return GDK_SUCCEED; > @@ -356,15 +376,21 @@ > } > } > if (!b->batDirty) b->batDirty = TRUE; > - if (@6 == 2) > + > + if (@6 & 1) { > b->tsorted = BATtordered(b); > - if (@6 == 3) { > - if (*v > 0) > + } else > + if (@6 & 4) { > + b->tsorted = REVERT_SORTED(BATtordered(b)); > + } else > + if (@6 & 16) { > + if (*v >= 0) > b->tsorted = BATtordered(b); > else > b->tsorted = REVERT_SORTED(BATtordered(b)); > - } else > + } else { > b->tsorted = 0; > + } > > BBPfix(b->batCacheid); > *ret = b; -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins]MonetDB/src/gdkgdk_utils.mx, MonetDB_1-22, 1.206, 1.206.2.1
by Peter Boncz 11 Feb '08

11 Feb '08

Hi Stefan Thanks for your response. However, as I have said, but let me say it as clear as possble: I am *sure* there is a bug still unfound in the way re-allocations are done when mem_bigsize redirects it into virtual memory. For this reason, at this stage, with the goal of a stable release, it is just my recommendation *not* to re-enable it. In M5 nor in M5. Peter Dear all, On Mon, Feb 11, 2008 at 02:54:59PM +0100, Peter Boncz wrote: > Hi Stefan, > > The pftijah bug was unrelated to vmalloc(), a relationship that I never > assumed. Indeed, that was (only) my mis-interpretation. Sorry. > The vmalloc() troubles occured, as you may recall, during the data > preparation of the sorted 100GB TPC-H. > > As it is entirely foreseeable that we at some point again will re-encounter an > OS with a malloc implementation that is prone to fragmentation, I'd keep the > vmalloc() code in for the moment (though in disabled state). Linux and Windows > do not need it. > > So, my advise is to for the monet let vmalloc() be disabled, until the issue > is properly investigated and a bugfix is found. Let's call it option (0). Ok. Just to sum up the current state. Please check, whether this is OK and consistent: a) Documentation in MonetDB4/conf/MonetDB.conf.in & MonetDB5/conf/monetdb5.conf.in says: " # gdk_mem_bigsize & gdk_vm_minsize will be set/limited to # 1/2 of the physically available amount of main-memory # during start-up in src/tools/Mserver.mx # memory chunks of size >= gdk_mem_bigsize (in bytes) will be mmaped anonymously #gdk_mem_bigsize=262144 # memory chunks of size >= gdk_vm_minsize (in bytes) will be mmaped; #gdk_vm_minsize=137438953472 " b) Documentation in ./monetweb/Docs/XQuery/MonetDB.conf.texi says " @item @code{gdk_mem_bigsize}: minimum size for memory-mapped columns, e.g. @code{262144}. @item @code{gdk_vm_minsize}: column size above which memory mapped files are used always, e.g. @code{1749291171} " c) MonetDB/src/common/monet_options.mx sets " set[i].kind = opt_builtin; set[i].name = strdup("gdk_mem_bigsize"); set[i].value = strdup("262144"); " d) MonetDB/src/common/monet_options.py.in sets " # gdk_mem_bigsize & gdk_vm_minsize will be set/limited to # 1/2 of the physically available amount of main-memory # during start-up in src/tools/Mserver.mx gdk_mem_bigsize = '256K' gdk_vm_minsize = '128G' " e) GDKinit() in MonetDB/src/gdk/gdk_utils.mx sets " if ((p = GDKgetenv("gdk_mem_bigsize"))) { /* when allocating >6% of all RAM; do so using vmalloc() iso malloc() */ lng max_mem_bigsize = GDK_mem_maxsize/16; /* sanity check to avoid memory fragmentation */ GDK_mem_bigsize = (size_t) MIN(max_mem_bigsize, strtol(p, NULL, 10)); } " f) GDKmallocmax() in MonetDB/src/gdk/gdk_utils.mx ignores GDK_mem_bigsize g) HEAPalloc() in MonetDB/src/gdk/gdk_heap.mx uses GDK_mem_bigsize as follows: " /* when using anonymous vm we malloc we need 64K chunks, also we * 20% extra malloc */ if (h->size > GDK_mem_bigsize) { h->maxsize = (size_t) ((double) h->maxsize * BATMARGIN) - 1; h->maxsize = (1 + (h->maxsize >> 16)) << 16; } " h) HEAPextend() in MonetDB/src/gdk/gdk_heap.mx uses GDK_mem_bigsize as follows: " /* extend a malloced heap, possibly switching over to file-mapped storage */ Heap bak = *h; int can_mmap = (h->filename && size >= GDK_mem_bigsize); int must_mmap = can_mmap && (size >= GDK_vm_minsize || (h->newstorage != STORE_MEM)); [...] if (can_mmap) { /* in anonymous vm, if have to realloc anyway, we reserve some extra space */ if (size > h->maxsize) { h->maxsize = (size_t) ((double) size * BATMARGIN); } /* when using anonymous vm we malloc we need 64K chunks */ h->maxsize = (1 + ((h->maxsize - 1) >> 16)) << 16; [...] /* too big: convert it to a disk-based temporary heap */ if (can_mmap) { char privext[PATHLENGTH], *of = h->filename; FILE *fp; h->filename = NULL; sprintf(privext, "%s.priv", ext); fp = GDKfilelocate(nme, "wb", privext); if (fp != NULL) { fclose(fp); /* a non-persistent heap: we create a .priv but *not* MMAP_PRIV !!! */ h->storage = STORE_MMAP; h->base = NULL; if (HEAPload(h, nme, privext, FALSE) >= 0) { memcpy(h->base, bak.base, bak.free); HEAPfree(&bak); return 0; } } GDKfree(of); } " i) MIL offers mem_bigsize() & mem_bigsize(lng) to get / set mem_bigsize in MonetDB4/src/modules/plain/sys.mx j) both MIL & MAL offer commands to set mem_maxsize and vm_minsize, however, only the MIL variants (still) related mem_maxsize & vm_minsize with mem_bigsize, while the MAL variants no longer do so: MonetDB4/src/modules/plain/sys.mx: " int set_mem_maxsize(lng *num) { @:num2sze(mem_maxsize)@ if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze); return GDK_SUCCEED; } int set_vm_minsize(lng *num) { @:num2sze(vm_minsize)@ if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_vm_minsize = MAX(GDK_mem_bigsize, sze); return GDK_SUCCEED; } " MonetDB5/src/modules/kernel/status.mx " int set_mem_maxsize(lng *num) { @:num2sze(mem_maxsize)@ #if 0 if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze); #endif GDK_mem_maxsize = sze; return GDK_SUCCEED; } int set_vm_minsize(lng *num) { @:num2sze(vm_minsize)@ #if 0 if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_vm_minsize = MAX(GDK_mem_bigsize, sze); #endif GDK_vm_minsize = sze; return GDK_SUCCEED; } " If all this is considered OK and consistent, we're done. Otherwise, we need to do what ever is required to get the code and documentation consistent, preferably before the release. Stefan > Peter > > > > > > > > > > > > > > > > > Dear all, > > since Roberto's bug appears to be fixed by Peter's ".priv" (vm_minsize > related) fixes (Thanks!), and disabling of vmalloc() (mem_bigsize related) > does not appear have any impact on this bug (just tested successfully both > with vmalloc() disabled and vmalloc() enabled), there are two ways to finish > this release (more or less) quickly: > > (1) finish the disabling of vmalloc() in all parts of the code as indicated > below (or better: remove all remains completely) --- this might need > some extra testing to ensure that it does not have any (significant) > impact on performance ... > > (2) for now (i.e., in the release branch) re-enable vmalloc(), and then > finish the removal of it (incl. extensive testing) in the development > trunk for th next release. > > Since I'm not completely aware of the impact of vmalloc(), I cannot really > make this decision --- the "save" back-up would IMHO be (2). > > Any other comments, opinions, suggestions, expertise? > > Thanks in advance! > > Stefan > > > On Sun, Feb 10, 2008 at 01:33:04PM +0100, Stefan Manegold wrote: > > Peter: Thanks for disabling mem_bigsize / vmalloc(). > > > > All: > > (a) > > Unfortunately, this does not (yet) solve > > [ 1872685 ] PF(tijah): HEAPsetmmap: Assertion `p' failed. > > http://sourceforge.net/tracker/index.php?func=detail&aid=1872685&group_id=5… > > > > But this appears to be more vm_minsize related than mem_bigsize related. > > > > > > (b) > > We should check all occurances of mem_bigsize and remove / "hide" those that > > have become irrelevant with the deactivation of mem_bigsize / vmalloc(): > > ======== > > $ cvsfiles * | egrep -v ' |/Tests/' | xargs grep -n --color 'mem_bigsize' > > -------- > > MonetDB4/conf/MonetDB.conf.in:29:# gdk_mem_bigsize & gdk_vm_minsize will be > set/limited to > > MonetDB4/conf/MonetDB.conf.in:32:# memory chunks of size >= gdk_mem_bigsize > (in bytes) will be mmaped anonymously > > MonetDB4/conf/MonetDB.conf.in:33:#gdk_mem_bigsize=262144 > > MonetDB5/conf/monetdb5.conf.in:33:# gdk_mem_bigsize & gdk_vm_minsize will be > set/limited to > > MonetDB5/conf/monetdb5.conf.in:36:# memory chunks of size >= gdk_mem_bigsize > (in bytes) will be mmaped anonymously > > MonetDB5/conf/monetdb5.conf.in:37:#gdk_mem_bigsize=262144 > > MonetDB/src/common/monet_options.mx:410: set[i].name = > strdup("gdk_mem_bigsize"); > > MonetDB/src/common/monet_options.py.in:39: # gdk_mem_bigsize & > gdk_vm_minsize will be set/limited to > > MonetDB/src/common/monet_options.py.in:42: gdk_mem_bigsize = '256K' > > MonetDB/src/gdk/gdk_heap.mx:121: if (h->size > GDK_mem_bigsize) { > > MonetDB/src/gdk/gdk_heap.mx:189: int can_mmap = (h->filename > && size >= GDK_mem_bigsize); > > MonetDB/src/gdk/gdk.mx:2210:gdk_export size_t GDK_mem_bigsize; /* size > after which we use VM rather than heap */ > > MonetDB/src/gdk/gdk_utils.mx:396:size_t GDK_mem_bigsize = 1 << 20; > > MonetDB/src/gdk/gdk_utils.mx:964:larger than GDK_mem_bigsize) to anonymous > virtual memory. The seamless > > MonetDB/src/gdk/gdk_utils.mx:1054: if (size > GDK_mem_bigsize) { > > MonetDB/src/gdk/gdk_utils.mx:1191: if (size <= GDK_mem_bigsize) { > > MonetDB/src/gdk/gdk_utils.mx:1529: if ((p = > GDKgetenv("gdk_mem_bigsize"))) { > > MonetDB/src/gdk/gdk_utils.mx:1531: lng max_mem_bigsize = > GDK_mem_maxsize/16; > > MonetDB/src/gdk/gdk_utils.mx:1534: GDK_mem_bigsize = (size_t) > MIN(max_mem_bigsize, strtol(p, NULL, 10)); > > monetweb/Docs/XQuery/MonetDB.conf.texi:115:@item @code{gdk_mem_bigsize}: > minimum size for memory-mapped columns, e.g. @code{262144}. > > MonetDB4/src/modules/plain/sys.mx:96:.COMMAND mem_bigsize() : lng = > get_mem_bigsize; > > MonetDB4/src/modules/plain/sys.mx:98:.COMMAND mem_bigsize(lng) : void = > set_mem_bigsize; > > MonetDB4/src/modules/plain/sys.mx:1430:get_mem_bigsize(lng *num) > > MonetDB4/src/modules/plain/sys.mx:1432: *num = GDK_mem_bigsize; > > MonetDB4/src/modules/plain/sys.mx:1437:set_mem_bigsize(lng *num) > > MonetDB4/src/modules/plain/sys.mx:1439: @:num2sze(mem_bigsize)@ > > MonetDB4/src/modules/plain/sys.mx:1440: GDK_mem_bigsize = MAX(32768, sze); > > MonetDB4/src/modules/plain/sys.mx:1462: if (sze < GDK_mem_bigsize) > > MonetDB4/src/modules/plain/sys.mx:1463: set_mem_bigsize(num); > > MonetDB4/src/modules/plain/sys.mx:1464: GDK_mem_maxsize = > MAX(GDK_mem_bigsize, sze); > > MonetDB4/src/modules/plain/sys.mx:1486: if (sze < GDK_mem_bigsize) > > MonetDB4/src/modules/plain/sys.mx:1487: set_mem_bigsize(num); > > MonetDB4/src/modules/plain/sys.mx:1488: GDK_vm_minsize = > MAX(GDK_mem_bigsize, sze); > > MonetDB5/src/modules/kernel/status.mx:405:get_mem_bigsize(lng *num) > > MonetDB5/src/modules/kernel/status.mx:407: *num = GDK_mem_bigsize; > > MonetDB5/src/modules/kernel/status.mx:412:set_mem_bigsize(lng *num) > > MonetDB5/src/modules/kernel/status.mx:414: @:num2sze(mem_bigsize)@ > > MonetDB5/src/modules/kernel/status.mx:415: GDK_mem_bigsize = MAX(32768, > sze); > > MonetDB5/src/modules/kernel/status.mx:439: if (sze < GDK_mem_bigsize) > > MonetDB5/src/modules/kernel/status.mx:440: set_mem_bigsize(num); > > MonetDB5/src/modules/kernel/status.mx:441: GDK_mem_maxsize = > MAX(GDK_mem_bigsize, sze); > > MonetDB5/src/modules/kernel/status.mx:466: if (sze < GDK_mem_bigsize) > > MonetDB5/src/modules/kernel/status.mx:467: set_mem_bigsize(num); > > MonetDB5/src/modules/kernel/status.mx:468: GDK_vm_minsize = > MAX(GDK_mem_bigsize, sze); > > monetweb/MonetDB/QuickTour/HelloWorld/content.shtml:42:[ "gdk_mem_bigsize", > "262144" > > ======== > > > > more tomorrow during MADAM ... > > > > Stefan > > > > > > On Sun, Feb 10, 2008 at 10:37:54AM +0100, Peter Boncz wrote: > > > no consequences. > > > > > > > > > > > > > > > Peter, > > > > > > What about the consequences in: > > > build/MonetDB/src/gdk/gdk_heap.c > > > build/MonetDB5/src/modules/kernel/status.c > > > build/MonetDB/src/gdk/gdk.h > > > > > > Peter Boncz wrote: > > > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651 > > > > > > > > Modified Files: > > > > Tag: MonetDB_1-22 > > > > gdk_utils.mx > > > > Log Message: > > > > disable use of vmalloc() > > > > > > > > > > > > > > > > Index: gdk_utils.mx > > > > =================================================================== > > > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > > > > retrieving revision 1.206 > > > > retrieving revision 1.206.2.1 > > > > diff -u -d -r1.206 -r1.206.2.1 > > > > --- gdk_utils.mx 14 Jan 2008 10:48:58 -0000 1.206 > > > > +++ gdk_utils.mx 9 Feb 2008 15:38:11 -0000 1.206.2.1 > > > > @@ -1050,6 +1050,7 @@ > > > > #endif > > > > } > > > > size = (size + 7) & ~7; /* round up to a multiple of eight */ > > > > +#if 0 > > > > if (size > GDK_mem_bigsize) { > > > > size_t newsize = size + sizeof(size_t) + sizeof(size_t); > > > > size_t newmax = MAX(*maxsize, newsize); > > > > @@ -1064,6 +1065,7 @@ > > > > *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); > > > > return (void *) s; > > > > } > > > > +#endif > > > > CHKMEM(size, 0); > > > > GDKmalloc_prefixsize(s, size); > > > > if (s == NULL) { > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > This SF.net email is sponsored by: Microsoft > > > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > > _______________________________________________ > > > > Monetdb-checkins mailing list > > > > Monetdb-checkins(a)lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > This SF.net email is sponsored by: Microsoft > > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > _______________________________________________ > > > Monetdb-developers mailing list > > > Monetdb-developers(a)lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > > > > > > > > > > > > > > > > -- > > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > > | The Netherlands | Fax : +31 (20) 592-4312 | > > -- > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > | The Netherlands | Fax : +31 (20) 592-4312 | > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 2

[Monetdb-developers] mem_bigsize
by Peter Boncz 11 Feb '08

11 Feb '08

Hi Stefan, Thanks for so much Grundlichkeit. It is appreciated. It seems that the comment in the XQuery documentation describing mem_bigsize is not correct. Given the current disabled status of mem_bigsize I will remove it. The behavior to limit mem_bigsize to the values of both mem_maxsize and vm_minsize I did not recall; but it probably makes sense. The rest is.. indeed correct. The repitition of configuration values in the python and monet_options has always bothered me, but I recall there was some reason why it cannot be avoided (though not what it was). However, I have spent the entire night&day, re-thinking the mmap-heap situation; and there are still some problems. I think it can be fixed once and for all, but it requires abandoning the X.priv approach to STORE_PRIV copy-on-write heaps in favor of always mapping the X file and writing into X.new files (which take preference over X files if encountered). But that also affects the BBP, the commit protocol. Peter Dear all, On Mon, Feb 11, 2008 at 02:54:59PM +0100, Peter Boncz wrote: > Hi Stefan, > > The pftijah bug was unrelated to vmalloc(), a relationship that I never > assumed. Indeed, that was (only) my mis-interpretation. Sorry. > The vmalloc() troubles occured, as you may recall, during the data > preparation of the sorted 100GB TPC-H. > > As it is entirely foreseeable that we at some point again will re-encounter an > OS with a malloc implementation that is prone to fragmentation, I'd keep the > vmalloc() code in for the moment (though in disabled state). Linux and Windows > do not need it. > > So, my advise is to for the monet let vmalloc() be disabled, until the issue > is properly investigated and a bugfix is found. Let's call it option (0). Ok. Just to sum up the current state. Please check, whether this is OK and consistent: a) Documentation in MonetDB4/conf/MonetDB.conf.in & MonetDB5/conf/monetdb5.conf.in says: " # gdk_mem_bigsize & gdk_vm_minsize will be set/limited to # 1/2 of the physically available amount of main-memory # during start-up in src/tools/Mserver.mx # memory chunks of size >= gdk_mem_bigsize (in bytes) will be mmaped anonymously #gdk_mem_bigsize=262144 # memory chunks of size >= gdk_vm_minsize (in bytes) will be mmaped; #gdk_vm_minsize=137438953472 " b) Documentation in ./monetweb/Docs/XQuery/MonetDB.conf.texi says " @item @code{gdk_mem_bigsize}: minimum size for memory-mapped columns, e.g. @code{262144}. @item @code{gdk_vm_minsize}: column size above which memory mapped files are used always, e.g. @code{1749291171} " c) MonetDB/src/common/monet_options.mx sets " set[i].kind = opt_builtin; set[i].name = strdup("gdk_mem_bigsize"); set[i].value = strdup("262144"); " d) MonetDB/src/common/monet_options.py.in sets " # gdk_mem_bigsize & gdk_vm_minsize will be set/limited to # 1/2 of the physically available amount of main-memory # during start-up in src/tools/Mserver.mx gdk_mem_bigsize = '256K' gdk_vm_minsize = '128G' " e) GDKinit() in MonetDB/src/gdk/gdk_utils.mx sets " if ((p = GDKgetenv("gdk_mem_bigsize"))) { /* when allocating >6% of all RAM; do so using vmalloc() iso malloc() */ lng max_mem_bigsize = GDK_mem_maxsize/16; /* sanity check to avoid memory fragmentation */ GDK_mem_bigsize = (size_t) MIN(max_mem_bigsize, strtol(p, NULL, 10)); } " f) GDKmallocmax() in MonetDB/src/gdk/gdk_utils.mx ignores GDK_mem_bigsize g) HEAPalloc() in MonetDB/src/gdk/gdk_heap.mx uses GDK_mem_bigsize as follows: " /* when using anonymous vm we malloc we need 64K chunks, also we * 20% extra malloc */ if (h->size > GDK_mem_bigsize) { h->maxsize = (size_t) ((double) h->maxsize * BATMARGIN) - 1; h->maxsize = (1 + (h->maxsize >> 16)) << 16; } " h) HEAPextend() in MonetDB/src/gdk/gdk_heap.mx uses GDK_mem_bigsize as follows: " /* extend a malloced heap, possibly switching over to file-mapped storage */ Heap bak = *h; int can_mmap = (h->filename && size >= GDK_mem_bigsize); int must_mmap = can_mmap && (size >= GDK_vm_minsize || (h->newstorage != STORE_MEM)); [...] if (can_mmap) { /* in anonymous vm, if have to realloc anyway, we reserve some extra space */ if (size > h->maxsize) { h->maxsize = (size_t) ((double) size * BATMARGIN); } /* when using anonymous vm we malloc we need 64K chunks */ h->maxsize = (1 + ((h->maxsize - 1) >> 16)) << 16; [...] /* too big: convert it to a disk-based temporary heap */ if (can_mmap) { char privext[PATHLENGTH], *of = h->filename; FILE *fp; h->filename = NULL; sprintf(privext, "%s.priv", ext); fp = GDKfilelocate(nme, "wb", privext); if (fp != NULL) { fclose(fp); /* a non-persistent heap: we create a .priv but *not* MMAP_PRIV !!! */ h->storage = STORE_MMAP; h->base = NULL; if (HEAPload(h, nme, privext, FALSE) >= 0) { memcpy(h->base, bak.base, bak.free); HEAPfree(&bak); return 0; } } GDKfree(of); } " i) MIL offers mem_bigsize() & mem_bigsize(lng) to get / set mem_bigsize in MonetDB4/src/modules/plain/sys.mx j) both MIL & MAL offer commands to set mem_maxsize and vm_minsize, however, only the MIL variants (still) related mem_maxsize & vm_minsize with mem_bigsize, while the MAL variants no longer do so: MonetDB4/src/modules/plain/sys.mx: " int set_mem_maxsize(lng *num) { @:num2sze(mem_maxsize)@ if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze); return GDK_SUCCEED; } int set_vm_minsize(lng *num) { @:num2sze(vm_minsize)@ if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_vm_minsize = MAX(GDK_mem_bigsize, sze); return GDK_SUCCEED; } " MonetDB5/src/modules/kernel/status.mx " int set_mem_maxsize(lng *num) { @:num2sze(mem_maxsize)@ #if 0 if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze); #endif GDK_mem_maxsize = sze; return GDK_SUCCEED; } int set_vm_minsize(lng *num) { @:num2sze(vm_minsize)@ #if 0 if (sze < GDK_mem_bigsize) set_mem_bigsize(num); GDK_vm_minsize = MAX(GDK_mem_bigsize, sze); #endif GDK_vm_minsize = sze; return GDK_SUCCEED; } " If all this is considered OK and consistent, we're done. Otherwise, we need to do what ever is required to get the code and documentation consistent, preferably before the release. Stefan > Peter > > > > > > > > > > > > > > > > > Dear all, > > since Roberto's bug appears to be fixed by Peter's ".priv" (vm_minsize > related) fixes (Thanks!), and disabling of vmalloc() (mem_bigsize related) > does not appear have any impact on this bug (just tested successfully both > with vmalloc() disabled and vmalloc() enabled), there are two ways to finish > this release (more or less) quickly: > > (1) finish the disabling of vmalloc() in all parts of the code as indicated > below (or better: remove all remains completely) --- this might need > some extra testing to ensure that it does not have any (significant) > impact on performance ... > > (2) for now (i.e., in the release branch) re-enable vmalloc(), and then > finish the removal of it (incl. extensive testing) in the development > trunk for th next release. > > Since I'm not completely aware of the impact of vmalloc(), I cannot really > make this decision --- the "save" back-up would IMHO be (2). > > Any other comments, opinions, suggestions, expertise? > > Thanks in advance! > > Stefan > > > On Sun, Feb 10, 2008 at 01:33:04PM +0100, Stefan Manegold wrote: > > Peter: Thanks for disabling mem_bigsize / vmalloc(). > > > > All: > > (a) > > Unfortunately, this does not (yet) solve > > [ 1872685 ] PF(tijah): HEAPsetmmap: Assertion `p' failed. > > http://sourceforge.net/tracker/index.php?func=detail&aid=1872685&group_id=5… > > > > But this appears to be more vm_minsize related than mem_bigsize related. > > > > > > (b) > > We should check all occurances of mem_bigsize and remove / "hide" those that > > have become irrelevant with the deactivation of mem_bigsize / vmalloc(): > > ======== > > $ cvsfiles * | egrep -v ' |/Tests/' | xargs grep -n --color 'mem_bigsize' > > -------- > > MonetDB4/conf/MonetDB.conf.in:29:# gdk_mem_bigsize & gdk_vm_minsize will be > set/limited to > > MonetDB4/conf/MonetDB.conf.in:32:# memory chunks of size >= gdk_mem_bigsize > (in bytes) will be mmaped anonymously > > MonetDB4/conf/MonetDB.conf.in:33:#gdk_mem_bigsize=262144 > > MonetDB5/conf/monetdb5.conf.in:33:# gdk_mem_bigsize & gdk_vm_minsize will be > set/limited to > > MonetDB5/conf/monetdb5.conf.in:36:# memory chunks of size >= gdk_mem_bigsize > (in bytes) will be mmaped anonymously > > MonetDB5/conf/monetdb5.conf.in:37:#gdk_mem_bigsize=262144 > > MonetDB/src/common/monet_options.mx:410: set[i].name = > strdup("gdk_mem_bigsize"); > > MonetDB/src/common/monet_options.py.in:39: # gdk_mem_bigsize & > gdk_vm_minsize will be set/limited to > > MonetDB/src/common/monet_options.py.in:42: gdk_mem_bigsize = '256K' > > MonetDB/src/gdk/gdk_heap.mx:121: if (h->size > GDK_mem_bigsize) { > > MonetDB/src/gdk/gdk_heap.mx:189: int can_mmap = (h->filename > && size >= GDK_mem_bigsize); > > MonetDB/src/gdk/gdk.mx:2210:gdk_export size_t GDK_mem_bigsize; /* size > after which we use VM rather than heap */ > > MonetDB/src/gdk/gdk_utils.mx:396:size_t GDK_mem_bigsize = 1 << 20; > > MonetDB/src/gdk/gdk_utils.mx:964:larger than GDK_mem_bigsize) to anonymous > virtual memory. The seamless > > MonetDB/src/gdk/gdk_utils.mx:1054: if (size > GDK_mem_bigsize) { > > MonetDB/src/gdk/gdk_utils.mx:1191: if (size <= GDK_mem_bigsize) { > > MonetDB/src/gdk/gdk_utils.mx:1529: if ((p = > GDKgetenv("gdk_mem_bigsize"))) { > > MonetDB/src/gdk/gdk_utils.mx:1531: lng max_mem_bigsize = > GDK_mem_maxsize/16; > > MonetDB/src/gdk/gdk_utils.mx:1534: GDK_mem_bigsize = (size_t) > MIN(max_mem_bigsize, strtol(p, NULL, 10)); > > monetweb/Docs/XQuery/MonetDB.conf.texi:115:@item @code{gdk_mem_bigsize}: > minimum size for memory-mapped columns, e.g. @code{262144}. > > MonetDB4/src/modules/plain/sys.mx:96:.COMMAND mem_bigsize() : lng = > get_mem_bigsize; > > MonetDB4/src/modules/plain/sys.mx:98:.COMMAND mem_bigsize(lng) : void = > set_mem_bigsize; > > MonetDB4/src/modules/plain/sys.mx:1430:get_mem_bigsize(lng *num) > > MonetDB4/src/modules/plain/sys.mx:1432: *num = GDK_mem_bigsize; > > MonetDB4/src/modules/plain/sys.mx:1437:set_mem_bigsize(lng *num) > > MonetDB4/src/modules/plain/sys.mx:1439: @:num2sze(mem_bigsize)@ > > MonetDB4/src/modules/plain/sys.mx:1440: GDK_mem_bigsize = MAX(32768, sze); > > MonetDB4/src/modules/plain/sys.mx:1462: if (sze < GDK_mem_bigsize) > > MonetDB4/src/modules/plain/sys.mx:1463: set_mem_bigsize(num); > > MonetDB4/src/modules/plain/sys.mx:1464: GDK_mem_maxsize = > MAX(GDK_mem_bigsize, sze); > > MonetDB4/src/modules/plain/sys.mx:1486: if (sze < GDK_mem_bigsize) > > MonetDB4/src/modules/plain/sys.mx:1487: set_mem_bigsize(num); > > MonetDB4/src/modules/plain/sys.mx:1488: GDK_vm_minsize = > MAX(GDK_mem_bigsize, sze); > > MonetDB5/src/modules/kernel/status.mx:405:get_mem_bigsize(lng *num) > > MonetDB5/src/modules/kernel/status.mx:407: *num = GDK_mem_bigsize; > > MonetDB5/src/modules/kernel/status.mx:412:set_mem_bigsize(lng *num) > > MonetDB5/src/modules/kernel/status.mx:414: @:num2sze(mem_bigsize)@ > > MonetDB5/src/modules/kernel/status.mx:415: GDK_mem_bigsize = MAX(32768, > sze); > > MonetDB5/src/modules/kernel/status.mx:439: if (sze < GDK_mem_bigsize) > > MonetDB5/src/modules/kernel/status.mx:440: set_mem_bigsize(num); > > MonetDB5/src/modules/kernel/status.mx:441: GDK_mem_maxsize = > MAX(GDK_mem_bigsize, sze); > > MonetDB5/src/modules/kernel/status.mx:466: if (sze < GDK_mem_bigsize) > > MonetDB5/src/modules/kernel/status.mx:467: set_mem_bigsize(num); > > MonetDB5/src/modules/kernel/status.mx:468: GDK_vm_minsize = > MAX(GDK_mem_bigsize, sze); > > monetweb/MonetDB/QuickTour/HelloWorld/content.shtml:42:[ "gdk_mem_bigsize", > "262144" > > ======== > > > > more tomorrow during MADAM ... > > > > Stefan > > > > > > On Sun, Feb 10, 2008 at 10:37:54AM +0100, Peter Boncz wrote: > > > no consequences. > > > > > > > > > > > > > > > Peter, > > > > > > What about the consequences in: > > > build/MonetDB/src/gdk/gdk_heap.c > > > build/MonetDB5/src/modules/kernel/status.c > > > build/MonetDB/src/gdk/gdk.h > > > > > > Peter Boncz wrote: > > > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651 > > > > > > > > Modified Files: > > > > Tag: MonetDB_1-22 > > > > gdk_utils.mx > > > > Log Message: > > > > disable use of vmalloc() > > > > > > > > > > > > > > > > Index: gdk_utils.mx > > > > =================================================================== > > > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > > > > retrieving revision 1.206 > > > > retrieving revision 1.206.2.1 > > > > diff -u -d -r1.206 -r1.206.2.1 > > > > --- gdk_utils.mx 14 Jan 2008 10:48:58 -0000 1.206 > > > > +++ gdk_utils.mx 9 Feb 2008 15:38:11 -0000 1.206.2.1 > > > > @@ -1050,6 +1050,7 @@ > > > > #endif > > > > } > > > > size = (size + 7) & ~7; /* round up to a multiple of eight */ > > > > +#if 0 > > > > if (size > GDK_mem_bigsize) { > > > > size_t newsize = size + sizeof(size_t) + sizeof(size_t); > > > > size_t newmax = MAX(*maxsize, newsize); > > > > @@ -1064,6 +1065,7 @@ > > > > *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); > > > > return (void *) s; > > > > } > > > > +#endif > > > > CHKMEM(size, 0); > > > > GDKmalloc_prefixsize(s, size); > > > > if (s == NULL) { > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > This SF.net email is sponsored by: Microsoft > > > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > > _______________________________________________ > > > > Monetdb-checkins mailing list > > > > Monetdb-checkins(a)lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > This SF.net email is sponsored by: Microsoft > > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > _______________________________________________ > > > Monetdb-developers mailing list > > > Monetdb-developers(a)lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > > > > > > > > > > > > > > > > -- > > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > > | The Netherlands | Fax : +31 (20) 592-4312 | > > -- > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > | The Netherlands | Fax : +31 (20) 592-4312 | > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdkgdk_utils.mx, MonetDB_1-22, 1.206, 1.206.2.1
by Peter Boncz 11 Feb '08

11 Feb '08

Hi Stefan, The pftijah bug was unrelated to vmalloc(), a relationship that I never assumed. The vmalloc() troubles occured, as you may recall, during the data preparation of the sorted 100GB TPC-H. As it is entirely foreseeable that we at some point again will re-encounter an OS with a malloc implementation that is prone to fragmentation, I'd keep the vmalloc() code in for the moment (though in disabled state). Linux and Windows do not need it. So, my advise is to for the monet let vmalloc() be disabled, until the issue is properly investigated and a bugfix is found. Let's call it option (0). Peter Dear all, since Roberto's bug appears to be fixed by Peter's ".priv" (vm_minsize related) fixes (Thanks!), and disabling of vmalloc() (mem_bigsize related) does not appear have any impact on this bug (just tested successfully both with vmalloc() disabled and vmalloc() enabled), there are two ways to finish this release (more or less) quickly: (1) finish the disabling of vmalloc() in all parts of the code as indicated below (or better: remove all remains completely) --- this might need some extra testing to ensure that it does not have any (significant) impact on performance ... (2) for now (i.e., in the release branch) re-enable vmalloc(), and then finish the removal of it (incl. extensive testing) in the development trunk for th next release. Since I'm not completely aware of the impact of vmalloc(), I cannot really make this decision --- the "save" back-up would IMHO be (2). Any other comments, opinions, suggestions, expertise? Thanks in advance! Stefan On Sun, Feb 10, 2008 at 01:33:04PM +0100, Stefan Manegold wrote: > Peter: Thanks for disabling mem_bigsize / vmalloc(). > > All: > (a) > Unfortunately, this does not (yet) solve > [ 1872685 ] PF(tijah): HEAPsetmmap: Assertion `p' failed. > http://sourceforge.net/tracker/index.php?func=detail&aid=1872685&group_id=5… > > But this appears to be more vm_minsize related than mem_bigsize related. > > > (b) > We should check all occurances of mem_bigsize and remove / "hide" those that > have become irrelevant with the deactivation of mem_bigsize / vmalloc(): > ======== > $ cvsfiles * | egrep -v ' |/Tests/' | xargs grep -n --color 'mem_bigsize' > -------- > MonetDB4/conf/MonetDB.conf.in:29:# gdk_mem_bigsize & gdk_vm_minsize will be set/limited to > MonetDB4/conf/MonetDB.conf.in:32:# memory chunks of size >= gdk_mem_bigsize (in bytes) will be mmaped anonymously > MonetDB4/conf/MonetDB.conf.in:33:#gdk_mem_bigsize=262144 > MonetDB5/conf/monetdb5.conf.in:33:# gdk_mem_bigsize & gdk_vm_minsize will be set/limited to > MonetDB5/conf/monetdb5.conf.in:36:# memory chunks of size >= gdk_mem_bigsize (in bytes) will be mmaped anonymously > MonetDB5/conf/monetdb5.conf.in:37:#gdk_mem_bigsize=262144 > MonetDB/src/common/monet_options.mx:410: set[i].name = strdup("gdk_mem_bigsize"); > MonetDB/src/common/monet_options.py.in:39: # gdk_mem_bigsize & gdk_vm_minsize will be set/limited to > MonetDB/src/common/monet_options.py.in:42: gdk_mem_bigsize = '256K' > MonetDB/src/gdk/gdk_heap.mx:121: if (h->size > GDK_mem_bigsize) { > MonetDB/src/gdk/gdk_heap.mx:189: int can_mmap = (h->filename && size >= GDK_mem_bigsize); > MonetDB/src/gdk/gdk.mx:2210:gdk_export size_t GDK_mem_bigsize; /* size after which we use VM rather than heap */ > MonetDB/src/gdk/gdk_utils.mx:396:size_t GDK_mem_bigsize = 1 << 20; > MonetDB/src/gdk/gdk_utils.mx:964:larger than GDK_mem_bigsize) to anonymous virtual memory. The seamless > MonetDB/src/gdk/gdk_utils.mx:1054: if (size > GDK_mem_bigsize) { > MonetDB/src/gdk/gdk_utils.mx:1191: if (size <= GDK_mem_bigsize) { > MonetDB/src/gdk/gdk_utils.mx:1529: if ((p = GDKgetenv("gdk_mem_bigsize"))) { > MonetDB/src/gdk/gdk_utils.mx:1531: lng max_mem_bigsize = GDK_mem_maxsize/16; > MonetDB/src/gdk/gdk_utils.mx:1534: GDK_mem_bigsize = (size_t) MIN(max_mem_bigsize, strtol(p, NULL, 10)); > monetweb/Docs/XQuery/MonetDB.conf.texi:115:@item @code{gdk_mem_bigsize}: minimum size for memory-mapped columns, e.g. @code{262144}. > MonetDB4/src/modules/plain/sys.mx:96:.COMMAND mem_bigsize() : lng = get_mem_bigsize; > MonetDB4/src/modules/plain/sys.mx:98:.COMMAND mem_bigsize(lng) : void = set_mem_bigsize; > MonetDB4/src/modules/plain/sys.mx:1430:get_mem_bigsize(lng *num) > MonetDB4/src/modules/plain/sys.mx:1432: *num = GDK_mem_bigsize; > MonetDB4/src/modules/plain/sys.mx:1437:set_mem_bigsize(lng *num) > MonetDB4/src/modules/plain/sys.mx:1439: @:num2sze(mem_bigsize)@ > MonetDB4/src/modules/plain/sys.mx:1440: GDK_mem_bigsize = MAX(32768, sze); > MonetDB4/src/modules/plain/sys.mx:1462: if (sze < GDK_mem_bigsize) > MonetDB4/src/modules/plain/sys.mx:1463: set_mem_bigsize(num); > MonetDB4/src/modules/plain/sys.mx:1464: GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze); > MonetDB4/src/modules/plain/sys.mx:1486: if (sze < GDK_mem_bigsize) > MonetDB4/src/modules/plain/sys.mx:1487: set_mem_bigsize(num); > MonetDB4/src/modules/plain/sys.mx:1488: GDK_vm_minsize = MAX(GDK_mem_bigsize, sze); > MonetDB5/src/modules/kernel/status.mx:405:get_mem_bigsize(lng *num) > MonetDB5/src/modules/kernel/status.mx:407: *num = GDK_mem_bigsize; > MonetDB5/src/modules/kernel/status.mx:412:set_mem_bigsize(lng *num) > MonetDB5/src/modules/kernel/status.mx:414: @:num2sze(mem_bigsize)@ > MonetDB5/src/modules/kernel/status.mx:415: GDK_mem_bigsize = MAX(32768, sze); > MonetDB5/src/modules/kernel/status.mx:439: if (sze < GDK_mem_bigsize) > MonetDB5/src/modules/kernel/status.mx:440: set_mem_bigsize(num); > MonetDB5/src/modules/kernel/status.mx:441: GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze); > MonetDB5/src/modules/kernel/status.mx:466: if (sze < GDK_mem_bigsize) > MonetDB5/src/modules/kernel/status.mx:467: set_mem_bigsize(num); > MonetDB5/src/modules/kernel/status.mx:468: GDK_vm_minsize = MAX(GDK_mem_bigsize, sze); > monetweb/MonetDB/QuickTour/HelloWorld/content.shtml:42:[ "gdk_mem_bigsize", "262144" > ======== > > more tomorrow during MADAM ... > > Stefan > > > On Sun, Feb 10, 2008 at 10:37:54AM +0100, Peter Boncz wrote: > > no consequences. > > > > > > > > > > Peter, > > > > What about the consequences in: > > build/MonetDB/src/gdk/gdk_heap.c > > build/MonetDB5/src/modules/kernel/status.c > > build/MonetDB/src/gdk/gdk.h > > > > Peter Boncz wrote: > > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651 > > > > > > Modified Files: > > > Tag: MonetDB_1-22 > > > gdk_utils.mx > > > Log Message: > > > disable use of vmalloc() > > > > > > > > > > > > Index: gdk_utils.mx > > > =================================================================== > > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > > > retrieving revision 1.206 > > > retrieving revision 1.206.2.1 > > > diff -u -d -r1.206 -r1.206.2.1 > > > --- gdk_utils.mx 14 Jan 2008 10:48:58 -0000 1.206 > > > +++ gdk_utils.mx 9 Feb 2008 15:38:11 -0000 1.206.2.1 > > > @@ -1050,6 +1050,7 @@ > > > #endif > > > } > > > size = (size + 7) & ~7; /* round up to a multiple of eight */ > > > +#if 0 > > > if (size > GDK_mem_bigsize) { > > > size_t newsize = size + sizeof(size_t) + sizeof(size_t); > > > size_t newmax = MAX(*maxsize, newsize); > > > @@ -1064,6 +1065,7 @@ > > > *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); > > > return (void *) s; > > > } > > > +#endif > > > CHKMEM(size, 0); > > > GDKmalloc_prefixsize(s, size); > > > if (s == NULL) { > > > > > > > > > ------------------------------------------------------------------------- > > > This SF.net email is sponsored by: Microsoft > > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > _______________________________________________ > > > Monetdb-checkins mailing list > > > Monetdb-checkins(a)lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Microsoft > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > _______________________________________________ > > Monetdb-developers mailing list > > Monetdb-developers(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > > > > > > > > > -- > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > | The Netherlands | Fax : +31 (20) 592-4312 | -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, MonetDB_1-22, 1.206, 1.206.2.1
by Peter Boncz 11 Feb '08

11 Feb '08

no consequences. Peter, What about the consequences in: build/MonetDB/src/gdk/gdk_heap.c build/MonetDB5/src/modules/kernel/status.c build/MonetDB/src/gdk/gdk.h Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651 > > Modified Files: > Tag: MonetDB_1-22 > gdk_utils.mx > Log Message: > disable use of vmalloc() > > > > Index: gdk_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > retrieving revision 1.206 > retrieving revision 1.206.2.1 > diff -u -d -r1.206 -r1.206.2.1 > --- gdk_utils.mx 14 Jan 2008 10:48:58 -0000 1.206 > +++ gdk_utils.mx 9 Feb 2008 15:38:11 -0000 1.206.2.1 > @@ -1050,6 +1050,7 @@ > #endif > } > size = (size + 7) & ~7; /* round up to a multiple of eight */ > +#if 0 > if (size > GDK_mem_bigsize) { > size_t newsize = size + sizeof(size_t) + sizeof(size_t); > size_t newmax = MAX(*maxsize, newsize); > @@ -1064,6 +1065,7 @@ > *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); > return (void *) s; > } > +#endif > CHKMEM(size, 0); > GDKmalloc_prefixsize(s, size); > if (s == NULL) { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, MonetDB_1-22, 1.206, 1.206.2.1
by Martin Kersten 10 Feb '08

10 Feb '08

Peter, What about the consequences in: build/MonetDB/src/gdk/gdk_heap.c build/MonetDB5/src/modules/kernel/status.c build/MonetDB/src/gdk/gdk.h Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651 > > Modified Files: > Tag: MonetDB_1-22 > gdk_utils.mx > Log Message: > disable use of vmalloc() > > > > Index: gdk_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > retrieving revision 1.206 > retrieving revision 1.206.2.1 > diff -u -d -r1.206 -r1.206.2.1 > --- gdk_utils.mx 14 Jan 2008 10:48:58 -0000 1.206 > +++ gdk_utils.mx 9 Feb 2008 15:38:11 -0000 1.206.2.1 > @@ -1050,6 +1050,7 @@ > #endif > } > size = (size + 7) & ~7; /* round up to a multiple of eight */ > +#if 0 > if (size > GDK_mem_bigsize) { > size_t newsize = size + sizeof(size_t) + sizeof(size_t); > size_t newmax = MAX(*maxsize, newsize); > @@ -1064,6 +1065,7 @@ > *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); > return (void *) s; > } > +#endif > CHKMEM(size, 0); > GDKmalloc_prefixsize(s, size); > if (s == NULL) { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, MonetDB_1-22, 1.206, 1.206.2.1
by Martin Kersten 10 Feb '08

10 Feb '08

Shouldn't then also monetdb.conf be adjusted Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651 > > Modified Files: > Tag: MonetDB_1-22 > gdk_utils.mx > Log Message: > disable use of vmalloc() > > > > Index: gdk_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > retrieving revision 1.206 > retrieving revision 1.206.2.1 > diff -u -d -r1.206 -r1.206.2.1 > --- gdk_utils.mx 14 Jan 2008 10:48:58 -0000 1.206 > +++ gdk_utils.mx 9 Feb 2008 15:38:11 -0000 1.206.2.1 > @@ -1050,6 +1050,7 @@ > #endif > } > size = (size + 7) & ~7; /* round up to a multiple of eight */ > +#if 0 > if (size > GDK_mem_bigsize) { > size_t newsize = size + sizeof(size_t) + sizeof(size_t); > size_t newmax = MAX(*maxsize, newsize); > @@ -1064,6 +1065,7 @@ > *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); > return (void *) s; > } > +#endif > CHKMEM(size, 0); > GDKmalloc_prefixsize(s, size); > if (s == NULL) { > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_replicator.mx, , 1.10, 1.11
by Stefan Manegold 09 Feb '08

09 Feb '08

On Sat, Feb 09, 2008 at 03:49:10PM +0000, Milena Ivanova wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv450 > > Modified Files: > opt_replicator.mx > Log Message: > Fix make warning. Why didn't it show with my flags set to > CONFIGURE_OPTIONS="--enable-debug --disable-optimize --enable-strict"? `man gcc` says ======== [...] -Wuninitialized Warn if an automatic variable is used without first being initialized or if a variable may be clobbered by a "setjmp" call. These warnings are possible only in optimizing compilation, because they require data flow information that is computed only when optimizing. If you don’t specify -O, you simply won’t get these warnings. [...] ======== hence, these warnings (=> error) only occur with optimization enabled. Stefan > > Index: opt_replicator.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_replicator.mx,v > retrieving revision 1.10 > retrieving revision 1.11 > diff -u -d -r1.10 -r1.11 > --- opt_replicator.mx 8 Feb 2008 16:44:33 -0000 1.10 > +++ opt_replicator.mx 9 Feb 2008 15:49:08 -0000 1.11 > @@ -242,7 +242,7 @@ > Replica r; > int (*cmp) (ptr, ptr); > ptr nilptr; > - int i, minr; > + int i, minr = -1; > size_t minc = (size_t) GDK_lng_max; > bit leq, lsub, heq, hsub; > int rs[replnr], ri=0; /* subset replicas */ > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/doc windowsbuild.rst, MonetDB_1-22, 1.10, 1.10.2.1
by Niels Nes 08 Feb '08

08 Feb '08

On Fri, Feb 08, 2008 at 10:12:44AM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/buildtools/doc > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv11533 > > Modified Files: > Tag: MonetDB_1-22 > windowsbuild.rst > Log Message: > Updated for ICC 1.10. > Added CC and CXX variables for ICC. Small request. Could the line cscript configure.js compiler=msvc prefix=C:\libxml2-2.6.30.win64 include=C:\iconv-1.11.win64\include;C:\zlib-1.2.3.win64\include lib=C:\iconv-1.11.win64\lib;C:\zlib-1.2.3.win64\lib iconv=yes zlib=yes somehow be split correctly (ie for windows and rst correct that is) over say 3 lines? Niels > > > Index: windowsbuild.rst > =================================================================== > RCS file: /cvsroot/monetdb/buildtools/doc/windowsbuild.rst,v > retrieving revision 1.10 > retrieving revision 1.10.2.1 > diff -u -d -r1.10 -r1.10.2.1 > --- windowsbuild.rst 15 Jan 2008 15:29:27 -0000 1.10 > +++ windowsbuild.rst 8 Feb 2008 10:12:42 -0000 1.10.2.1 > @@ -114,7 +114,8 @@ > > - Microsoft Visual Studio .NET 2003 (also known as Microsoft Visual Studio 7); > - Microsoft Visual Studio 2005 (also known as Microsoft Visual Studio 8); > -- Intel(R) C++ Compiler 9.1 (which actually needs one of the above). > +- Intel(R) C++ Compiler 9.1 (which actually needs one of the above); > +- Intel(R) C++ Compiler 10.1 (which also needs one of the Microsoft compilers). > > Note that the pathfinder component can currently not be compiled with > any of the Microsoft compilers. It can be compiled with the Intel > @@ -124,7 +125,7 @@ > Compiler gcc under Cygwin__. Using that it (probably still) is possible > to build a version that runs using the Cygwin DLLs, but also a version > that uses the MinGW__ (Minimalist GNU for Windows) package. This is > -not further described here. > +not supported and not further described here. > > __ http://www.cygwin.com/ > __ http://www.mingw.org/ > @@ -534,9 +535,18 @@ > > call "%ProgramFiles%\Microsoft Visual Studio 8\Common7\Tools\vsvars32.bat" > > -- Intel(R) C++ Compiler 9.1:: > +- Intel(R) C++ Compiler 10.1.013:: > > - call "%ProgramFiles%\Intel\Compiler\C++\9.1\IA32\Bin\iclvars.bat" > + call "C:%ProgramFiles%\Intel\Compiler\C++\10.1.013\IA32\Bin\iclvars.bat" > + > +When using the Intel compiler, you also need to set the ``CC`` and > +``CXX`` variables:: > + > + set CC=icl -Qstd=c99 -GR- -Qsafeseh- > + set CXX=icl -Qstd=c99 -GR- -Qsafeseh- > + > +(These are the values for the 10.1 version, for 9.1 replace > +``-Qstd=c99`` with ``-Qc99``.) > > Internal Variables > ~~~~~~~~~~~~~~~~~~ > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_joinpath.mx, , 1.21, 1.22
by Sjoerd Mullender 07 Feb '08

07 Feb '08

On 2008-02-07 18:45, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937 > > Modified Files: > opt_joinpath.mx > Log Message: > Revert Martin code because it does not work. The code will remain, but commented > for future improvements as Martin suggested... A better way to comment larger sections of code is to use #if 0/#endif. That nests. > > > > Index: opt_joinpath.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v > retrieving revision 1.21 > retrieving revision 1.22 > diff -u -d -r1.21 -r1.22 > --- opt_joinpath.mx 7 Feb 2008 16:57:18 -0000 1.21 > +++ opt_joinpath.mx 7 Feb 2008 17:44:59 -0000 1.22 > @@ -231,22 +231,30 @@ > The join path optimizer takes a join sequence and > attempts to minimize the intermediate result. > The choice depends on a good estimate of intermediate > -results.SAMPLE_THRESHOLD_lOG > +results. > For the time being, we use a simplistic model, based > on the assumption that most joins are foreign key joins anyway. > + > We use a sample based approach for sizeable tables. > -The model is derived from the select statement. > +The model is derived from the select statement. However, we did not succeed. > +The code is now commented for future improvement. > +Final conclusion from this exercise is: > +The difference between the join input size and the join output size is not > +the correct (or unique) metric which should be used to decide which order > +should be used in the joinPath. > + > @c > -#define SAMPLE_THRESHOLD_lOG 17 > +/*#define SAMPLE_THRESHOLD_lOG 17*/ > size_t > ALGjoinCost(BAT *l, BAT *r) > { > - size_t lc, rc, lsize,rsize; > - BAT *lsample, *rsample, *j; > + size_t lc, rc; > +/* size_t lsize,rsize; > + BAT *lsample, *rsample, *j; */ > > lc = BATcount(l); > rc = BATcount(r); > - if( lc > 100000 || rc > 100000){ > +/* if( lc > 100000 || rc > 100000){ > lsize= MIN(lc/100, (1<<SAMPLE_THRESHOLD_lOG)/3); > lsample= BATsample(l,lsize); > BBPreclaim(lsample); > @@ -257,7 +265,7 @@ > lsize= BATcount(j); > BBPreclaim(j); > return lsize; > - } > + }*/ > if( l->ttype== TYPE_oid || r->htype== TYPE_oid ) > return MIN(lc,rc); > return lc * rc; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

4 6

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal bpm.mx, , 1.123, 1.124
by Stefan Manegold 07 Feb '08

07 Feb '08

Martin I guess this should have gone into the MonetDB_5-4 branch, right? Please sync with Niels before back-porting/fixing! Stefan On Thu, Feb 07, 2008 at 08:18:17AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8081 > > Modified Files: > bpm.mx > Log Message: > typos > > > Index: bpm.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/bpm.mx,v > retrieving revision 1.123 > retrieving revision 1.124 > diff -u -d -r1.123 -r1.124 > --- bpm.mx 11 Jan 2008 10:41:37 -0000 1.123 > +++ bpm.mx 7 Feb 2008 08:18:12 -0000 1.124 > @@ -183,13 +183,13 @@ > @example > bpm.open(); > Ralias:= bpm.take("myversion",:bat[:oid,:int]); # get the partition alias > - R0{hlow=nil:oid,hhigh=nil:oid,tlow=nil:int,thigh=100}:= bpm.take(Ralias, 0); > + R0@{hlow=nil:oid,hhigh=nil:oid,tlow=nil:int,thigh=100@}:= bpm.take(Ralias, 0); > T0:=algebra.select(R0,0,100); > > - R1{hlow=nil:oid,hhigh=nil:oid,tlow=101,thigh=200}:= bpm.take(Ralias, 1); > + R1@{hlow=nil:oid,hhigh=nil:oid,tlow=101,thigh=200@}:= bpm.take(Ralias, 1); > T1:= algebra.select(R1,0,100); > > - R2{hlow=nil:oid,hhigh=nil:oid,tlow=201,thigh=nil:int}:= bpm.take(Ralias, 2); > + R2@{hlow=nil:oid,hhigh=nil:oid,tlow=201,thigh=nil:int@}:= bpm.take(Ralias, 2); > R:= mat.new(T0,T1,T2); > T2:=algebra.select(R2,0,100); > @end example > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.209, 1.210 mal_profiler.mx, , 1.102, 1.103 mal_properties.mx, , 1.47, 1.48
by Stefan Manegold 07 Feb '08

07 Feb '08

Martin I guess this should have gone into the MonetDB_5-4 branch, right? Please sync with Niels before back-porting/fixing! Stefan On Thu, Feb 07, 2008 at 08:12:49AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv4344 > > Modified Files: > mal_interpreter.mx mal_profiler.mx mal_properties.mx > Log Message: > fix reference > > > Index: mal_profiler.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_profiler.mx,v > retrieving revision 1.102 > retrieving revision 1.103 > diff -u -d -r1.102 -r1.103 > --- mal_profiler.mx 11 Jan 2008 10:41:33 -0000 1.102 > +++ mal_profiler.mx 7 Feb 2008 08:12:44 -0000 1.103 > @@ -104,7 +104,7 @@ > @node Event Caching, Stethoscope, Event Filtering, The MAL Profiler > @+ Event Caching > Aside from shipping events to a separate process, the profiler > -can keep the events in a local @sc{BAT} group. > +can keep the events in a local @sc{bat} group. > It is the default when no target file has been opened > to collect the information. > > > Index: mal_interpreter.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > retrieving revision 1.209 > retrieving revision 1.210 > diff -u -d -r1.209 -r1.210 > --- mal_interpreter.mx 11 Jan 2008 10:41:32 -0000 1.209 > +++ mal_interpreter.mx 7 Feb 2008 08:12:44 -0000 1.210 > @@ -1581,7 +1581,7 @@ > activated using the compile option MALprofiler. Activation can lead to > a significant performance degradation, because for all traced functions > we have to keep track of essential system counter information. > -See @url{The MAL Profiler} for more details. > +See @ref{The MAL Profiler} for more details. > > @= performanceVariables > #ifdef MALprofiler > > Index: mal_properties.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_properties.mx,v > retrieving revision 1.47 > retrieving revision 1.48 > diff -u -d -r1.47 -r1.48 > --- mal_properties.mx 11 Jan 2008 10:41:33 -0000 1.47 > +++ mal_properties.mx 7 Feb 2008 08:12:44 -0000 1.48 > @@ -17,7 +17,6 @@ > @f mal_properties > @a M. Kersten > @+ Property Management > - > Properties come in several classes, those linked with the symbol table and > those linked with the runtime environment. The former are determined > once upon parsing or catalog lookup. The runtime properties have two > @@ -282,12 +281,14 @@ > @tab data can be appended > @end multitable > @- > +@{ > @h > #ifndef _MAL_PROPERTIES_ > #define _MAL_PROPERTIES_ > #include "mal.h" > #include "mal_namespace.h" > > +@{ > typedef struct MalProp { > bte idx; > bte op; > @@ -302,6 +303,7 @@ > op_gt = 4, > op_ne = 5 > } prop_op_t; > +@} > > mal_export sht PropertyIndex(str name); > mal_export str PropertyName(sht idx); > @@ -310,7 +312,6 @@ > > #endif > > -@{ > @c > #include "mal_config.h" > #include "mal_properties.h" > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] New release branches
by Stefan Manegold 03 Feb '08

03 Feb '08

Dear fellow MonetDB developers, to initiate the usual release process for the upcoming "GDK-2" release of the whole MonetDB software family, I plan to create new release branches in CVS during this weekend (most probably on Sunday Feb 03 2008). Please let me know, in case there are any (strong) reasons to postpone the creation of the new release branches. As usually, please note that once the release branches have been created, only bug fixes may be checked in to the release branches (in fact, all bug fixes have to be check in to the release branches, only!), while any new feature may be checked in to the development trunk, only. In case you have (minor!) features to add or finish, that should be in the release, please do so today, i.e., before I'll start creating the new release branches --- but make sure (*before* checking them in!) that they work flawlessly in all cases! In case you plan to add new features, please postpone your check-in until Monday Feb 04, i.e., until after I have created the new release branches. In case you have any question, please do not hesitate to ask, preferably using one of the public communication channels (read: mailing lists). Thank you very much for your cooperation! Have a nice (and free) weekend. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 2

[Monetdb-developers] what about the jdbc driver ?
by Jean-Paul Le Fèvre 01 Feb '08

01 Feb '08

Hello, I'm unable to compile the jdbc driver : some Monet classes do not implement abstract methods. I've read a previous thread about this problem in the archive from april 2007. The cause of the problem seems to be the version of java I'm using (1.6). In my opinion jdk 1.5 is obsolete at this point and I no longer have this version on any of my machines. I tried the installation of the MonetDB-Mars-SuperBall-SR5 tarball, then of the last snapshot available but without success. Is there any already compiled jar files somewhere ? -- ___________________________________________________________________ Jean-Paul Le Fèvre * Mail : LeFevre(a)fonteny.org

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing Mtest.py.in, , 1.333, 1.334
by Romulo Goncalves 31 Jan '08

31 Jan '08

Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26786/MonetDB/src/testing > > Modified Files: > Mtest.py.in > Log Message: > > Added "-a / --algebra" switch to Mtest.py to enable the Algebra version of the SQL compiler > (once we have a proper switch for Mserver / mclient, also Pathfinder can use this switch). Well, the option -a does not work while the option --algebra works fine. the Error: Traceback (most recent call last): File "/ufs/goncalve/scratch/MonetDB/debug/MonetDB/bin/Mtest.py", line 3742, in <module> main(sys.argv) File "/ufs/goncalve/scratch/MonetDB/debug/MonetDB/bin/Mtest.py", line 2912, in main opts, args = monet_options.parse_options(argv[1:], options, Usage, False) File "/ufs/goncalve/scratch/MonetDB/debug/MonetDB/lib64/python2.5/site-packages/MonetDB/monet_options.py", line 211, in parse_options opts, args = getopt.getopt(argv, getopt_short, getopt_long) File "/usr/lib64/python2.5/getopt.py", line 91, in getopt opts, args = do_shorts(opts, args[0][1:], shortopts, args[1:]) File "/usr/lib64/python2.5/getopt.py", line 191, in do_shorts if short_has_arg(opt, shortopts): File "/usr/lib64/python2.5/getopt.py", line 207, in short_has_arg raise GetoptError('option -%s not recognized' % opt, opt) getopt.GetoptError: option -a not recognized Romulo > > Via `Mapprove.py -S.Algebra ...`, algebra-specific stable output can be approved. > > Also changed the default for template, sql, & geom to use M5 server iso. M4 server. > > > Index: Mtest.py.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/testing/Mtest.py.in,v > retrieving revision 1.333 > retrieving revision 1.334 > diff -u -d -r1.333 -r1.334 > --- Mtest.py.in 11 Jan 2008 10:37:02 -0000 1.333 > +++ Mtest.py.in 28 Jan 2008 10:34:14 -0000 1.334 > @@ -1211,6 +1211,7 @@ > env['TST_OIDS'] = "" > env['TST_FIVE'] = "" > env['TST_STATIC'] = "" > + env['TST_ALGEBRA'] = "" > if qOut: > tbs = re.compile("^# Compiled for .*/([63][42]bit) with ([63][42])bit OIDs;? (.*) linked", re.MULTILINE) > tm = re.compile("^Modules: (.*)$", re.MULTILINE) > @@ -1262,6 +1263,9 @@ > if par['M5']: > env['TST_FIVE'] = "Five" > os.environ['TST_FIVE'] = "Five" > + if par['ALGEBRA']: > + env['TST_ALGEBRA'] = "Algebra" > + os.environ['TST_ALGEBRA'] = "Algebra" > return rtrn > ### GetBitsAndOIDsAndModsAndStatic(env) # > > @@ -1375,12 +1379,15 @@ > OIDS = env['TST_OIDS'] > FIVE = env['TST_FIVE'] > STATIC = env['TST_STATIC'] > + ALGEBRA = env['TST_ALGEBRA'] > if FIVE: > FIVE = "(\.Five)?" > if STATIC: > STATIC = "(\.STATIC)?" > + if ALGEBRA: > + ALGEBRA = "(\.Algebra)?" > dir,file = os.path.split(TST) > - outre = re.compile('^'+re.escape(file)+'\.stable\.(?P<tp>out|err)(\.(%s(%s)?|%s(%s)?))?(\.%s)?(\.%s)?%s%s$' % (re.escape(OS), re.escape(RELEASE), re.escape(DIST), re.escape(VERSION), BITS, OIDS, FIVE, STATIC)) > + outre = re.compile('^'+re.escape(file)+'\.stable\.(?P<tp>out|err)(\.(%s(%s)?|%s(%s)?))?(\.%s)?(\.%s)?%s%s%s$' % (re.escape(OS), re.escape(RELEASE), re.escape(DIST), re.escape(VERSION), BITS, OIDS, FIVE, STATIC, ALGEBRA)) > bestout = besterr = '' > for f in os.listdir(dir or os.curdir): > res = outre.match(f) > @@ -2206,6 +2213,8 @@ > Srvr = '%s "--dbinit=%s"' % (Srvr, DBINIT) > if SERVER == "SQL": > lang="sql" > + if par['ALGEBRA']: > + Srvr = '%s --set sql_debug=16384' % Srvr > if par['M5']: > Srvr = '%s --set mal_listing=0 "--dbinit=%s include sql;"' % (Srvr, DBINIT) > else: > @@ -2534,6 +2543,7 @@ > v['OIDS'] = '' > v['FIVE'] = '' > v['STATIC'] = '' > + v['ALGEBRA'] = '' > if os.path.isfile(f): > r = re.compile('^([A-Z]+) = "(.*)".*$') > for l in open(f): > @@ -2709,7 +2719,7 @@ > } > > par = {} > -# the default; only used for usage information in case -p/---package has not been given > +# the default; only used for usage information in case -p/--package has not been given > par['PACKAGE'] = 'monetdb' > # try to be "smart" > if os.path.exists(os.path.join('CVS', 'Repository')): > @@ -2808,6 +2818,8 @@ > "use Mserver of MonetDB Version 4 (default with <package> not in {monetdb5, monetdb-datacell})"), > ('MonetDB5', '5', 'MonetDB5', None, > "use mserver5 of MonetDB Version 5 (default with <package> in {monetdb5, monetdb-datacell})"), > + ('algebra', 'a', 'algebra', None, > + "use the algebra version of the SQL compiler"), > ('recursive', 'r', 'recursive', None, > "recurse into subdirectories (implies 'All')"), > ('MONETDB_SOURCE', None, 'MONETDB_SOURCE', '<path>', > @@ -2872,7 +2884,7 @@ > elif THISFILE == 'Mapprove.py': > f = os.path.join(GetMonetConfig(par['PACKAGE'],'--prefix'),dftTSTPREF,'.Mapprove.rc') > v = ReadMapproveRc(f) > - for i in 'BITS', 'OIDS', 'FIVE', 'STATIC': > + for i in 'BITS', 'OIDS', 'FIVE', 'STATIC', 'ALGEBRA': > if v[i]: > v[i] = '[.%s]' % v[i] > options = cmd_options + [ > @@ -2881,9 +2893,9 @@ > "(default: <ext> = 'out' & 'err')"), > (None, 'S', 'sys', '<sys>', > "approve specific output *.<ext><sys>\n" > - "(<sys> = '[.(<OS>[<RELEASE>]|<DIST>[<VERSION>])][.(32|64)bit][.oid(32|64)][.Five][.STATIC]',\n" > - "(default: longest match for <sys> = '[.(%s[%s]|%s[%s])]%s%s%s%s')" > - % (v['OS'], v['RELEASE'], v['DIST'], v['VERSION'], v['BITS'], v['OIDS'], v['FIVE'], v['STATIC'])), > + "(<sys> = '[.(<OS>[<RELEASE>]|<DIST>[<VERSION>])][.(32|64)bit][.oid(32|64)][.Five][.STATIC][.Algebra]',\n" > + "(default: longest match for <sys> = '[.(%s[%s]|%s[%s])]%s%s%s%s%s')" > + % (v['OS'], v['RELEASE'], v['DIST'], v['VERSION'], v['BITS'], v['OIDS'], v['FIVE'], v['STATIC'], v['ALGEBRA'])), > (None, 'f', 'force', None, > "force approval of error messages (i.e., lines starting with '!')"), > ] > @@ -2934,8 +2946,8 @@ > if par['M5'] and par['PACKAGE'] == 'monetdb4': > ErrXit('"-5/--MonetDB5" conflicts with "--package=monetdb4"!') > if not par['M4'] and not par['M5']: > - par['M4'] = par['PACKAGE'] not in ['monetdb5', 'monetdb-datacell'] > - par['M5'] = par['PACKAGE'] in ['monetdb5', 'monetdb-datacell'] > + par['M4'] = par['PACKAGE'] not in ['monetdb5', 'monetdb-datacell', 'monetdb-template', 'monetdb-sql', 'monetdb-geom'] > + par['M5'] = par['PACKAGE'] in ['monetdb5', 'monetdb-datacell', 'monetdb-template', 'monetdb-sql', 'monetdb-geom'] > > config = opts.get('config') > if config is None and par['PACKAGE'] not in ('monetdb', 'monetdb-java', 'monetdb-clients'): > @@ -2950,10 +2962,15 @@ > except monet_options.Error: > sys.exit(1) > > + par['ALGEBRA'] = opts.get('algebra', 0) > par['RECURSIVE'] = opts.get('recursive', 0) > par['QUIET'] = opts.get('quiet', 0) > if THISFILE == "Mtest.py": > - par['IGNORE'] = opts.get('ignore', dftIGNORE) > + if par['ALGEBRA'] and par['PACKAGE'] == 'monetdb-sql': > + _IGNORE = '^[#%]' > + else: > + _IGNORE = dftIGNORE > + par['IGNORE'] = opts.get('ignore', _IGNORE) > par['CONTEXT'] = '-C%d' % int(opts.get('context', 1)) > a = int(opts.get('accuracy', 1)) > if a not in (-1,0,1,2): > @@ -3132,17 +3149,17 @@ > os.environ['DISTVER'] = DISTVER > os.environ['VERSION'] = VERSION > w = {} > - for i in 'OS', 'RELEASE', 'DIST', 'VERSION', 'BITS', 'OIDS', 'FIVE', 'STATIC': > + for i in 'OS', 'RELEASE', 'DIST', 'VERSION', 'BITS', 'OIDS', 'FIVE', 'STATIC', 'ALGEBRA': > w[i] = re.escape(v[i]) > - for i in 'BITS', 'OIDS', 'FIVE', 'STATIC': > + for i in 'BITS', 'OIDS', 'FIVE', 'STATIC', 'ALGEBRA': > j = 'TST_'+i > env[j] = v[i] > os.environ[j] = v[i] > if v[i]: > v[i] = '(.%s)?' % v[i] > w[i] = '(\.%s)?' % w[i] > - sv = '^(.(%s(%s)?|%s(%s)?))?%s%s%s%s$' % (v['OS'], v['RELEASE'], v['DIST'], v['VERSION'], v['BITS'], v['OIDS'], v['FIVE'], v['STATIC']) > - sw = '^(\.(%s(%s)?|%s(%s)?))?%s%s%s%s$' % (w['OS'], w['RELEASE'], w['DIST'], w['VERSION'], w['BITS'], w['OIDS'], w['FIVE'], w['STATIC']) > + sv = '^(.(%s(%s)?|%s(%s)?))?%s%s%s%s%s$' % (v['OS'], v['RELEASE'], v['DIST'], v['VERSION'], v['BITS'], v['OIDS'], v['FIVE'], v['STATIC'], v['ALGEBRA']) > + sw = '^(\.(%s(%s)?|%s(%s)?))?%s%s%s%s%s$' % (w['OS'], w['RELEASE'], w['DIST'], w['VERSION'], w['BITS'], w['OIDS'], w['FIVE'], w['STATIC'], w['ALGEBRA']) > r = re.compile(sw) > if not r.match(par['SYSTEM']): > ErrXit("System (-S) must match '"+sv+"' !") > @@ -3396,6 +3413,7 @@ > env['TST_OIDS'] = "" > env['TST_FIVE'] = "" > env['TST_STATIC'] = "" > + env['TST_ALGEBRA'] = "" > > is_sql = par['PACKAGE'] == 'monetdb-sql' > if not is_sql: > @@ -3491,7 +3509,7 @@ > env['VERSION'] = os.environ['VERSION'] > n = os.path.join(env['TSTTRGBASE'],TSTPREF,'.Mapprove.rc') > f = open(n, 'w') > - for v in 'OS', 'RELEASE', 'DIST', 'VERSION', 'TST_BITS', 'TST_OIDS', 'TST_FIVE', 'TST_STATIC': > + for v in 'OS', 'RELEASE', 'DIST', 'VERSION', 'TST_BITS', 'TST_OIDS', 'TST_FIVE', 'TST_STATIC', 'TST_ALGEBRA': > w = v.replace('TST_','') > f.write('%s = "%s"\n' % (w, env[v])) > f.close() > @@ -3537,6 +3555,7 @@ > product = os.path.split(env['TSTSRCBASE'])[-1] > if product == 'sql': > product += env['TST_FIVE'] > + product += env['TST_ALGEBRA'] > compiler = '' > if os.name == 'nt': > compiler = 'Mic' # Microsoft Visual Studio C > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/Tests mclient-lxquery--help.stable.out, , 1.5, 1.6 mclient-lxquery--help.stable.out.Darwin, , 1.4, 1.5
by Martin Kersten 28 Jan '08

28 Jan '08

Using this kind of switches is not in line with the other switches - the short-cut version sofar has been aligned with the first letter of the command - \ commands may optionally contain arguments, which seems that it is more in line to use e.g. '\g true' and '\g false' Same holds for '\A' and '\a' - \g is in Postgresql (and mysql) used as a \go shortcut - \g switch does not apply to mil and mal and should therefore be placed in the language specific help sections Stefan Manegold wrote: > Update of /cvsroot/monetdb/pathfinder/src/tools/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv30003 > > Modified Files: > mclient-lxquery--help.stable.out > mclient-lxquery--help.stable.out.Darwin > Log Message: > > approved new \G / \g switches > > > Index: mclient-lxquery--help.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/tools/Tests/mclient-lxquery--help.stable.out,v > retrieving revision 1.5 > retrieving revision 1.6 > diff -u -d -r1.5 -r1.6 > --- mclient-lxquery--help.stable.out 19 Dec 2007 22:52:11 -0000 1.5 > +++ mclient-lxquery--help.stable.out 28 Jan 2008 18:35:44 -0000 1.6 > @@ -1,12 +1,14 @@ > stdout of test 'mclient-lxquery--help` in directory 'src/tools` itself: > > > -# 23:46:23 > > -# 23:46:23 > Mserver "--config=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/etc/MonetDB.conf" --debug=10 --set "monet_mod_path=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/lib:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/bin" --set "gdk_dbfarm=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/dbfarm" --set "sql_logdir=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/sql_logs" --set "xquery_logdir=/ufs/manegold/_/scratch0/Monet > /Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/xquery_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=34548 --set xrpc_port=49429 --set monet_prompt= --trace "--dbname=mTests_src_tools" "--dbinit= module(pathfinder);" ; echo ; echo Over.. > -# 23:46:23 > > +# 19:33:05 > > +# 19:33:05 > Mserver "--config=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/etc/MonetDB.conf" --debug=10 --set "monet_mod_path=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4:/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/lib:/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/bin" --set "gdk_dbfarm=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/dbfarm" --set "sql_logdir=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/sql_logs" --set "xquery_logdir=/ufs/manegold/_/scratch0 > /Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/xquery_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=31795 --set xrpc_port=42301 --set monet_prompt= --trace "--dbname=mTests_src_tools" "--dbinit= module(pathfinder);" ; echo ; echo Over.. > +# 19:33:05 > > > -# MonetDB Server v4.20.1 > -# based on GDK v1.20.1 > + > + > +# MonetDB Server v4.21.0 > +# based on GDK v1.21.0 > # Copyright (c) 1993-2007, CWI. All rights reserved. > # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs; dynamically linked. > # Visit http://monetdb.cwi.nl/ for further information. > @@ -47,6 +49,8 @@ > \|cmd - pipe result to process, or stop when no command is given > \h - show the readline history > \t - toggle timer > +\G - switch to algebra frontend > +\g - switch to old frontend > \f - result format: dm or xml[-noheader][-typed|-noroot|-root-FOOBAR] > \L file - save client/server interaction > \X - trace mclient code > > Index: mclient-lxquery--help.stable.out.Darwin > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/tools/Tests/mclient-lxquery--help.stable.out.Darwin,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- mclient-lxquery--help.stable.out.Darwin 19 Dec 2007 22:52:11 -0000 1.4 > +++ mclient-lxquery--help.stable.out.Darwin 28 Jan 2008 18:35:44 -0000 1.5 > @@ -1,12 +1,14 @@ > stdout of test 'mclient-lxquery--help` in directory 'src/tools` itself: > > > -# 23:46:23 > > -# 23:46:23 > Mserver "--config=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/etc/MonetDB.conf" --debug=10 --set "monet_mod_path=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/lib:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/bin" --set "gdk_dbfarm=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/dbfarm" --set "sql_logdir=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/sql_logs" --set "xquery_logdir=/ufs/manegold/_/scratch0/Monet > /Testing/Stable/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/xquery_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=34548 --set xrpc_port=49429 --set monet_prompt= --trace "--dbname=mTests_src_tools" "--dbinit= module(pathfinder);" ; echo ; echo Over.. > -# 23:46:23 > > +# 19:33:05 > > +# 19:33:05 > Mserver "--config=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/etc/MonetDB.conf" --debug=10 --set "monet_mod_path=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4:/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/lib:/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/lib64/MonetDB4/bin" --set "gdk_dbfarm=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/dbfarm" --set "sql_logdir=/ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/sql_logs" --set "xquery_logdir=/ufs/manegold/_/scratch0 > /Monet/Testing/Current/prefix.--enable-strict_--disable-debug_--enable-optimize_--disable-assert/var/MonetDB4/xquery_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=31795 --set xrpc_port=42301 --set monet_prompt= --trace "--dbname=mTests_src_tools" "--dbinit= module(pathfinder);" ; echo ; echo Over.. > +# 19:33:05 > > > -# MonetDB Server v4.20.1 > -# based on GDK v1.20.1 > + > + > +# MonetDB Server v4.21.0 > +# based on GDK v1.21.0 > # Copyright (c) 1993-2007, CWI. All rights reserved. > # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs; dynamically linked. > # Visit http://monetdb.cwi.nl/ for further information. > @@ -46,6 +48,8 @@ > \>file - save response in file, or stdout if no file is given > \|cmd - pipe result to process, or stop when no command is given > \t - toggle timer > +\G - switch to algebra frontend > +\g - switch to old frontend > \f - result format: dm or xml[-noheader][-typed|-noroot|-root-FOOBAR] > \L file - save client/server interaction > \X - trace mclient code > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins >

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer/Tests remote00.stable.err, , 1.4, 1.5 remote00.stable.out, , 1.8, 1.9 remote01.stable.err, , 1.4, 1.5 remote01.stable.out, , 1.10, 1.11
by Stefan Manegold 27 Jan '08

27 Jan '08

Martin, please double check the results you approved. I hvaeno idea what the test (are supposed to) do, but a simple layman's sanity check reveals the following: - Test "remote00" contains six "io.print()", one with a literal "1", two with "mapi_port", and three with a BAT argument; however, the new stable output only contains a "[ 1 ]", the two mapi_ports and two BATs (there was no third?) have been removed by your checkin ... ? - Test "remote01" prints 4 BATs and one count; however, your changes of the stable (corrct?) output replace the last two BATs and the count by error messages ... ? Thanks! Stefan On Sun, Jan 27, 2008 at 08:36:49AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8290/Tests > > Modified Files: > remote00.stable.err remote00.stable.out remote01.stable.err > remote01.stable.out > Log Message: > Approve test output > > > Index: remote00.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Tests/remote00.stable.out,v > retrieving revision 1.8 > retrieving revision 1.9 > diff -u -d -r1.8 -r1.9 > --- remote00.stable.out 21 Aug 2006 14:40:36 -0000 1.8 > +++ remote00.stable.out 27 Jan 2008 08:36:46 -0000 1.9 > @@ -1,41 +1,25 @@ > stdout of test 'remote00` in directory 'src/optimizer` itself: > > > -# 23:06:19 > > -# 23:06:19 > Mtimeout -timeout 60 mserver5 "--config=/ufs/mk/monet5/Linux/etc/MonetDB5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5/Linux/lib/MonetDB5:/ufs/mk/opensource/MonetDB/Linux/lib/MonetDB" --set "gdk_dbfarm=/ufs/mk/monet5/Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5/Linux/var/MonetDB5/dblogs" --set mapi_port=30126 --set monet_prompt= --trace --dbname=mTests_src_optimizer remote00.mal < /dev/null > -# 23:06:19 > > +# 09:35:05 > > +# 09:35:05 > /usr/bin/python remote00.py remote00 > +# 09:35:05 > > > -# MonetDB Server v4.99.19 > -# Copyright (c) 1993-2006 CWI, all rights reserved > -# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked > -# config:/ufs/mk/monet5/Linux/etc/MonetDB5.conf > -# dbfarm:/ufs/mk/monet5/Linux/var/MonetDB5/dbfarm > -# dbname:mTests_src_optimizer > +# MonetDB server v5.3.0, based on kernel v1.21.0 > +# Serving database 'db1' > +# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > +# Copyright (c) 1993-2007 CWI, all rights reserved > # Visit http://monetdb.cwi.nl/ for further information > +# Listening for connection requests on mapi:monetdb://gio.ins.cwi.nl:12345/ > [ 1 ] > -[ "12345" ] > -# MonetDB Server v4.99.19 > -# Copyright (c) 1993-2006 CWI, all rights reserved > -# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked > -# config:/ufs/niels/scratch/Linux-x86_64/etc/MonetDB5.conf > -# dbfarm:/ufs/niels/scratch/Linux-x86_64/var/MonetDB5/dbfarm > -# dbname:db2 > +# MonetDB server v5.3.0, based on kernel v1.21.0 > +# Serving database 'db2' > +# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > +# Copyright (c) 1993-2007 CWI, all rights reserved > # Visit http://monetdb.cwi.nl/ for further information > -[ "12346" ] > -#-----------------# > -# h t # name > -# int int # type > -#-----------------# > -[ 1, 1 ] > -[ 2, 7 ] > -[ 1 ] > -#-----------------# > -# h t # name > -# int int # type > -#-----------------# > -[ 2, 7 ] > +# Listening for connection requests on mapi:monetdb://gio.ins.cwi.nl:12346/ > > -# 16:36:12 > > -# 16:36:12 > Done. > -# 16:36:12 > > +# 09:35:07 > > +# 09:35:07 > Done. > +# 09:35:07 > > > > Index: remote00.stable.err > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Tests/remote00.stable.err,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- remote00.stable.err 26 Jan 2008 13:34:23 -0000 1.4 > +++ remote00.stable.err 27 Jan 2008 08:36:46 -0000 1.5 > @@ -1,12 +1,16 @@ > stderr of test 'remote00` in directory 'src/optimizer` itself: > > > -# 23:06:19 > > -# 23:06:19 > Mtimeout -timeout 60 mserver5 "--config=/ufs/mk/monet5/Linux/etc/MonetDB5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5/Linux/lib/MonetDB5:/ufs/mk/opensource/MonetDB/Linux/lib/MonetDB" --set "gdk_dbfarm=/ufs/mk/monet5/Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5/Linux/var/MonetDB5/dblogs" --set mapi_port=30126 --set monet_prompt= --trace --dbname=mTests_src_optimizer remote00.mal < /dev/null > -# 23:06:19 > > +# 09:35:05 > > +# 09:35:05 > /usr/bin/python remote00.py remote00 > +# 09:35:05 > > > +#warning: please don't forget to set your vault key! > +#(see /ufs/mk/monet5/Linux/etc/monetdb5.conf) > +#warning: please don't forget to set your vault key! > +#(see /ufs/mk/monet5/Linux/etc/monetdb5.conf) > > -# 23:06:19 > > -# 23:06:19 > Done. > -# 23:06:19 > > +# 09:35:07 > > +# 09:35:07 > Done. > +# 09:35:07 > > > > Index: remote01.stable.err > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Tests/remote01.stable.err,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- remote01.stable.err 26 Jan 2008 13:34:23 -0000 1.4 > +++ remote01.stable.err 27 Jan 2008 08:36:46 -0000 1.5 > @@ -1,12 +1,14 @@ > stderr of test 'remote01` in directory 'src/optimizer` itself: > > > -# 23:06:19 > > -# 23:06:19 > Mtimeout -timeout 60 mserver5 "--config=/ufs/mk/monet5/Linux/etc/MonetDB5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5/Linux/lib/MonetDB5:/ufs/mk/opensource/MonetDB/Linux/lib/MonetDB" --set "gdk_dbfarm=/ufs/mk/monet5/Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5/Linux/var/MonetDB5/dblogs" --set mapi_port=30126 --set monet_prompt= --trace --dbname=mTests_src_optimizer remote01.mal < /dev/null > -# 23:06:19 > > +# 09:35:05 > > +# 09:35:05 > mserver5 "--config=/ufs/mk/monet5/Linux/etc/monetdb5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin" --set "gdk_dbfarm=/ufs/mk/monet5//Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/sql_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=39715 --set xrpc_port=43844 --set monet_prompt= --trace --dbname=mTests_src_optimizer remote01.mal > +# 09:35:05 > > > +#warning: please don't forget to set your vault key! > +#(see /ufs/mk/monet5/Linux/etc/monetdb5.conf) > > -# 23:06:19 > > -# 23:06:19 > Done. > -# 23:06:19 > > +# 09:35:05 > > +# 09:35:05 > Done. > +# 09:35:05 > > > > Index: remote01.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Tests/remote01.stable.out,v > retrieving revision 1.10 > retrieving revision 1.11 > diff -u -d -r1.10 -r1.11 > --- remote01.stable.out 8 Jun 2007 08:47:31 -0000 1.10 > +++ remote01.stable.out 27 Jan 2008 08:36:46 -0000 1.11 > @@ -1,36 +1,53 @@ > stdout of test 'remote01` in directory 'src/optimizer` itself: > > > -# 23:06:19 > > -# 23:06:19 > Mtimeout -timeout 60 mserver5 "--config=/ufs/mk/monet5/Linux/etc/MonetDB5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5/Linux/lib/MonetDB5:/ufs/mk/opensource/MonetDB/Linux/lib/MonetDB" --set "gdk_dbfarm=/ufs/mk/monet5/Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5/Linux/var/MonetDB5/dblogs" --set mapi_port=30126 --set monet_prompt= --trace --dbname=mTests_src_optimizer remote01.mal < /dev/null > -# 23:06:19 > > +# 09:35:05 > > +# 09:35:05 > mserver5 "--config=/ufs/mk/monet5/Linux/etc/monetdb5.conf" --debug=10 --set "monet_mod_path=/ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin" --set "gdk_dbfarm=/ufs/mk/monet5//Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/sql_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=39715 --set xrpc_port=43844 --set monet_prompt= --trace --dbname=mTests_src_optimizer remote01.mal > +# 09:35:05 > > > -# MonetDB Server v4.99.19 > -# Copyright (c) 1993-2006 CWI, all rights reserved > -# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked > -# config:/ufs/mk/monet5/Linux/etc/MonetDB5.conf > -# dbfarm:/ufs/mk/monet5/Linux/var/MonetDB5/dbfarm > -# dbname:mTests_src_optimizer > +# MonetDB server v5.3.0, based on kernel v1.21.0 > +# Serving database 'mTests_src_optimizer' > +# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > +# Copyright (c) 1993-2007 CWI, all rights reserved > # Visit http://monetdb.cwi.nl/ for further information > +# Listening for connection requests on mapi:monetdb://gio.ins.cwi.nl:39715/ > ## First test of the remoteQueries optimizer > ## we create a remote bat and fake a bind operation. > # > -#mid:= mserver.reconnect("s0_0","localhost",50000,"monetdb","monetdb","mal"); > -#mserver.rpc(mid,"rb:= bat.new(:int,:int); bat.setName(rb,\"rbat\"); bat.insert(rb,1,1);"); > +#cat:= inspect.getEnvironment(); > +#fs := algebra.find(cat,"mapi_port"); > +#port := calc.int(fs); > +##io.printf("port %d\n",port); > # > -#b:bat[:int,:int]:= mserver.bind(mid,"rbat"); > -#mserver.rpc(mid,"c:=algebra.select(b,0,12);"); > -#c:bat[:int,:int]:= mserver.rpc(mid, "io.print(c);"); > +#mid:= mapi.reconnect("localhost",port,"s0_0","monetdb","monetdb","mal"); > +#mapi.rpc(mid,"rb:= bat.new(:int,:int); bat.setName(rb,\"rbat\");"); > +#mapi.rpc(mid,"bat.insert(rb,1,1);"); > +#mapi.rpc(mid,"bat.insert(rb,3,3);"); > +#mapi.rpc(mid,"bat.insert(rb,5,5);"); > +#mapi.rpc(mid,"bat.insert(rb,6,6);"); > +#mapi.rpc(mid,"bat.insert(rb,7,7);"); > +#mapi.rpc(mid,"c:=algebra.select(rb,0,12);"); > +#c:bat[:int,:int]:= mapi.rpc(mid, "io.print(c);"); > #io.print(c); > -#mserver.rpc(mid,"d:=algebra.select(b,5,10);"); > +#mapi.rpc(mid,"c:=algebra.select(rb,0,4);"); > +#c:bat[:int,:int]:= mapi.rpc(mid, "io.print(c);"); > +#io.print(c); > +# > +#b:bat[:int,:int]:= mapi.bind(mid,"rbat"); > +#io.print(b); > +# > +## see if we can do some remote selections. > +#mapi.rpc(mid,"d:=algebra.select(b,5,10);"); > #low:= 5+1; > -#mserver.put(mid,"low",low); > -#mserver.rpc(mid,"e:=algebra.select(d,low,7);"); > -#mserver.rpc(mid,"i:=aggr.count(d);"); > -#i:lng:= mserver.rpc(mid,"io.print(i);"); > +#mapi.put(mid,"low",low); > +#mapi.rpc(mid,"e:=algebra.select(d,low,7);"); > +#mapi.rpc(mid,"i:=aggr.count(d);"); > +#i:lng:= mapi.rpc(mid,"io.print(i);"); > #io.printf(" count %d\n",i); > -#d:bat[:int,:int]:= mserver.rpc(mid,"io.print(d);"); > +#d:bat[:int,:int]:= mapi.rpc(mid,"io.print(d);"); > #io.print(d); > +#mapi.disconnect(mid); > +#optimizer.remoteQueries(); > # > function user.main():void; # 0 (main:void) > # First test of the remoteQueries optimizer # 1 (_1:str) > @@ -39,34 +56,33 @@ > fs := algebra.find(cat,"mapi_port"); # 4 ALGfind (fs:str)<-(cat:bat[:str,:str])(_5:str) > port := calc.int(fs); # 5 CALCstr2int (port:int)<-(fs:str) > #io.printf("port %d\n",port); # 6 (_7:str) > - mid := mserver.reconnect("localhost",port,"s0_0","monetdb","monetdb","mal"); # 7 SERVERreconnectAlias (mid:int)<-(_9:str)(port:int)(_10:str)(_11:str)(_11:str)(_12:str) > - mserver.rpc(mid,"rb:= bat.new(:int,:int); bat.setName(rb,\"rbat\");"); # 8 SERVERquery (_13:void)<-(mid:int)(_14:str) > - mserver.rpc(mid,"bat.insert(rb,1,1);"); # 9 SERVERquery (_15:void)<-(mid:int)(_16:str) > - mserver.rpc(mid,"bat.insert(rb,3,3);"); # 10 SERVERquery (_17:void)<-(mid:int)(_18:str) > - mserver.rpc(mid,"bat.insert(rb,5,5);"); # 11 SERVERquery (_19:void)<-(mid:int)(_20:str) > - mserver.rpc(mid,"bat.insert(rb,6,6);"); # 12 SERVERquery (_21:void)<-(mid:int)(_22:str) > - mserver.rpc(mid,"bat.insert(rb,7,7);"); # 13 SERVERquery (_23:void)<-(mid:int)(_24:str) > - mserver.rpc(mid,"c:=algebra.select(rb,0,12);"); # 14 SERVERquery (_25:void)<-(mid:int)(_26:str) > - c:bat[:int,:int] := mserver.rpc(mid,"io.print(c);"); # 15 SERVERmapi_rpc_bat (c:bat[:int,:int])<-(mid:int)(_28:str) > + mid := mapi.reconnect("localhost",port,"s0_0","monetdb","monetdb","mal"); # 7 SERVERreconnectAlias (mid:int)<-(_9:str)(port:int)(_10:str)(_11:str)(_11:str)(_12:str) > + mapi.rpc(mid,"rb:= bat.new(:int,:int); bat.setName(rb,\"rbat\");"); # 8 SERVERquery (_13:void)<-(mid:int)(_14:str) > + mapi.rpc(mid,"bat.insert(rb,1,1);"); # 9 SERVERquery (_15:void)<-(mid:int)(_16:str) > + mapi.rpc(mid,"bat.insert(rb,3,3);"); # 10 SERVERquery (_17:void)<-(mid:int)(_18:str) > + mapi.rpc(mid,"bat.insert(rb,5,5);"); # 11 SERVERquery (_19:void)<-(mid:int)(_20:str) > + mapi.rpc(mid,"bat.insert(rb,6,6);"); # 12 SERVERquery (_21:void)<-(mid:int)(_22:str) > + mapi.rpc(mid,"bat.insert(rb,7,7);"); # 13 SERVERquery (_23:void)<-(mid:int)(_24:str) > + mapi.rpc(mid,"c:=algebra.select(rb,0,12);"); # 14 SERVERquery (_25:void)<-(mid:int)(_26:str) > + c:bat[:int,:int] := mapi.rpc(mid,"io.print(c);"); # 15 SERVERmapi_rpc_bat (c:bat[:int,:int])<-(mid:int)(_28:str) > io.print(c); # 16 IOprint_val (_29:void)<-(c:bat[:int,:int]) > - mserver.rpc(mid,"c:=algebra.select(rb,0,4);"); # 17 SERVERquery (_30:void)<-(mid:int)(_31:str) > - c:bat[:int,:int] := mserver.rpc(mid,"io.print(c);"); # 18 SERVERmapi_rpc_bat (c:bat[:int,:int])<-(mid:int)(_28:str) > + mapi.rpc(mid,"c:=algebra.select(rb,0,4);"); # 17 SERVERquery (_30:void)<-(mid:int)(_31:str) > + c:bat[:int,:int] := mapi.rpc(mid,"io.print(c);"); # 18 SERVERmapi_rpc_bat (c:bat[:int,:int])<-(mid:int)(_28:str) > io.print(c); # 19 IOprint_val (_32:void)<-(c:bat[:int,:int]) > - mserver.rpc(mid,"b:bat[:int,:int] := bbp.bind(\"rbat\":str);"); # 20 SERVERmapi_rpc_bat (_57:bat[:int,:int])<-(mid:int)(_58:str) > - b:bat[:int,:int] := mserver.rpc(mid,"io.print(b);"); # 21 SERVERmapi_rpc_bat (b:bat[:int,:int])<-(mid:int)(_60:str) > - io.print(b); # 22 IOprint_val (_35:void)<-(b:bat[:int,:int]) > -# see if we can do some remote selections. # 23 (_36:str) > - mserver.rpc(mid,"d:=algebra.select(b,5,10);"); # 24 SERVERquery (_37:void)<-(mid:int)(_38:str) > - low := calc.+(5,1); # 25 CALCbinaryADDintint (low:int)<-(_40:int)(_41:int) > - mserver.put(mid,"low",low); # 26 SERVERput (_42:void)<-(mid:int)(_43:str)(low:int) > - mserver.rpc(mid,"e:=algebra.select(d,low,7);"); # 27 SERVERquery (_44:void)<-(mid:int)(_45:str) > - mserver.rpc(mid,"i:=aggr.count(d);"); # 28 SERVERquery (_46:void)<-(mid:int)(_47:str) > - i:lng := mserver.rpc(mid,"io.print(i);"); # 29 SERVERmapi_rpc_single_row (i:lng)<-(mid:int)(_49:str) > - io.printf(" count %d\n",i); # 30 IOprint_formatted_lng (_50:void)<-(_51:str)(i:lng) > - d:bat[:int,:int] := mserver.rpc(mid,"io.print(d);"); # 31 SERVERmapi_rpc_bat (d:bat[:int,:int])<-(mid:int)(_53:str) > - io.print(d); # 32 IOprint_val (_54:void)<-(d:bat[:int,:int]) > - mserver.disconnect(mid); # 33 SERVERdisconnect (_55:void)<-(mid:int) > -end main; # 34 > + b:bat[:int,:int] := mapi.bind(mid,"rbat"); # 20 SERVERbindBAT (b:bat[:int,:int])<-(mid:int)(_34:str) > + io.print(b); # 21 IOprint_val (_35:void)<-(b:bat[:int,:int]) > +# see if we can do some remote selections. # 22 (_36:str) > + mapi.rpc(mid,"d:=algebra.select(b,5,10);"); # 23 SERVERquery (_37:void)<-(mid:int)(_38:str) > + low := calc.+(5,1); # 24 CALCbinaryADDintint (low:int)<-(_40:int)(_41:int) > + mapi.put(mid,"low",low); # 25 SERVERput (_42:void)<-(mid:int)(_43:str)(low:int) > + mapi.rpc(mid,"e:=algebra.select(d,low,7);"); # 26 SERVERquery (_44:void)<-(mid:int)(_45:str) > + mapi.rpc(mid,"i:=aggr.count(d);"); # 27 SERVERquery (_46:void)<-(mid:int)(_47:str) > + i:lng := mapi.rpc(mid,"io.print(i);"); # 28 SERVERmapi_rpc_single_row (i:lng)<-(mid:int)(_49:str) > + io.printf(" count %d\n",i); # 29 IOprint_formatted_lng (_50:void)<-(_51:str)(i:lng) > + d:bat[:int,:int] := mapi.rpc(mid,"io.print(d);"); # 30 SERVERmapi_rpc_bat (d:bat[:int,:int])<-(mid:int)(_53:str) > + io.print(d); # 31 IOprint_val (_54:void)<-(d:bat[:int,:int]) > + mapi.disconnect(mid); # 32 SERVERdisconnect (_55:void)<-(mid:int) > +end main; # 33 > #-----------------# > # h t # name > # int int # type > @@ -82,25 +98,12 @@ > #-----------------# > [ 1, 1 ] > [ 3, 3 ] > -#-----------------# > -# h t # name > -# int int # type > -#-----------------# > -[ 1, 1 ] > -[ 3, 3 ] > -[ 5, 5 ] > -[ 6, 6 ] > -[ 7, 7 ] > - count 3 > -#-----------------# > -# h t # name > -# int int # type > -#-----------------# > -[ 5, 5 ] > -[ 6, 6 ] > -[ 7, 7 ] > +!MALException:mapi.bind:remote error: SyntaxException:parseError:b:bat[int,int]:=bbp.bind("rbat"); > +!MALException:mapi.bind:remote error:SyntaxException:parseError: ^',' expected > +!MALException:mapi.bind:remote error:SyntaxException:parseError: > +!MALException:mapi.bind:remote error:';' expected > > -# 23:06:19 > > -# 23:06:19 > Done. > -# 23:06:19 > > +# 09:35:05 > > +# 09:35:05 > Done. > +# 09:35:05 > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] re compiling on darwin9
by Stefan Manegold 22 Jan '08

22 Jan '08

Hi, I just made a very simple attempt to fix the problem by ensuring that --enable-oid32 is handled and MONET_OID32 defined only with 64-bit builds (but ignored/unset otherwise); see also [ 1876580 ] Compile MonetDB on Darwin 9: failed by --enable-oid32 https://sourceforge.net/tracker/index.php?func=detail&aid=1876580&group_id=… If this works, we're fine. If not, we can go back to the previous situtation (before Fabian's changes in buildtools/conf/MonetDB.m4), i.e., --enable-bits is only handled on platform where it is known to work (incl. Darwin 8), while on other platforms (e.g., Darwin 9; or when using the Intel compiler), configure stops with the following advice: "case "$GCC-$CC-$host_os-$host-$bits" not handled with --enable-bits, yet; please use CC, CFLAGS and/or LDFLAGS (instead of --enable-bits=$bits) to make "$CC" produce $bits-bit code" (obviously with the variables replaced with the respective values...) Stefan On Tue, Jan 22, 2008 at 10:25:37AM +0100, Fabian Groffen wrote: > I just noticed > http://article.gmane.org/gmane.comp.db.monetdb.devel/937 > > > > (3) Indeed, even though it does not make sense, we should not crash on > > > "--enable-bits=32 --enable-oid32", but either (preferred) handle it > > > correctly (e.g., by ignoring the latter in case the former holds > > > (regardless of whether requested explicitly or simply by being the > > > default)), or (at the very least) stop with a proper message > > > saying that > > > it is not required to specify the redundant information. > > > > I would prefer to give a message that the redundant "--enable-oid32" > > has been discarded and continue. > > I think the only "redundant" option here is --enable-bits. Simply > because --enable-bits only works for GNU multilib compilers, but it > doesn't check them, and also assumes every compiler to be a multilib > compiler. I noticed so when I changed MonetDB.m4 for Jennie. > > The correct way to deal with multilib compilers is to leave it up to the > user, e.g. set CC="gcc -m64" or CC="gcc64" or CHOST/CTARGET=xxx64-xx-xx. > Having configure have a flag that switches on the -mX per magic is > rather misleading, and from the auto* world's point of view something > that should have happened outside. I guess we run the test for the flag > early enough in the process such that all AC_COMPILE tests use the -mX > flag, but still I think it would clean things up when the flag would > disappear. > > (I cannot reply on the thread, because I am not subscribed to the > list/didn't receive the message.) -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Compiling on Darwin 7
by Fabian Groffen 21 Jan '08

21 Jan '08

Is there anyone that can think of a solution for this problem? cc1: warnings being treated as errors /Volumes/Scratch/monetdb/stable/monetdb/src/modules/plain/bat.mx: In function 'local_itoa': /Volumes/Scratch/monetdb/stable/monetdb/src/modules/plain/bat.mx:1253: warning: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t' I mean... is there a difference between "signed size_t" and "ssize_t"? Ehrm. [Orion:src/modules/plain] fabian% uname -a Darwin Orion.local 7.9.0 Darwin Kernel Version 7.9.0: Wed Mar 30 20:11:17 PST 2005; root:xnu/xnu-517.12.7.obj~1/RELEASE_PPC Power Macintosh powerpc PowerBook4,3 Darwin [Orion:src/modules/plain] fabian% gcc --version gcc (GCC) 4.0.1 (Apple Computer, Inc. build 5341) Copyright (C) 2005 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Probably just need to compile without -Werror.

4 5

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/conf MonetDB.m4, MonetDB_1-20, 1.37.2.18, 1.37.2.19
by Ying Zhang 21 Jan '08

21 Jan '08

helaas, als ik de stable brach compileer, krijg ik steeds de selfde error, misschien is de string "+yes-*-darwin[89]*-powerpc*-* | yes-*- darwin[89]*-i386*-* | yes-*-darwin[89]*-x86_64*-*)" waar het script naar kijkt niet correct, mijn gcc zegt namelijk zo iets: $ gcc i686-apple-darwin9-gcc-4.0.1: no input files ik kan dit zelf wel uitzoeken, denk ik, maar niet nu:) Als ik "--enable-bits" helemaal niet gebruikt, compileert het als een speer:) Ik kan alleen de tijah module net laden in mserver, maar dat is een ander verhaal. Jennie On Jan 18, 2008, at 10:34 , Fabian wrote: > Update of /cvsroot/monetdb/buildtools/conf > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv25825 > > Modified Files: > Tag: MonetDB_1-20 > MonetDB.m4 > Log Message: > Add support for 64-bits with GCC 4 on Solaris and on Darwin 9 and > Darwin on Intel > > Index: MonetDB.m4 > =================================================================== > RCS file: /cvsroot/monetdb/buildtools/conf/MonetDB.m4,v > retrieving revision 1.37.2.18 > retrieving revision 1.37.2.19 > diff -u -d -r1.37.2.18 -r1.37.2.19 > --- MonetDB.m4 22 Nov 2007 15:44:17 -0000 1.37.2.18 > +++ MonetDB.m4 18 Jan 2008 09:34:48 -0000 1.37.2.19 > @@ -490,8 +490,8 @@ > ;; > yes-*-solaris*-*-*) > case `$bits-$CC -v 2>&1` in > - 32-*|*-*'gcc version 3.'*) ;; > - *) AC_MSG_ERROR([need GCC version 3.X for 64 bits]);; > + 32-*|*-*'gcc version '[34]'.'*) ;; > + *) AC_MSG_ERROR([need GCC version >=3.X for 64 bits]);; > esac > CC="$CC -m$bits" > ;; > @@ -520,7 +520,8 @@ > -*pgcc*-linux*-x86_64*-*) > CC="$CC -tp=k8-$bits" > ;; > -yes-*-darwin8*-powerpc*-*) > +yes-*-darwin[89]*-powerpc*-* | yes-*-darwin[89]*-i386*-* | yes-*- > darwin[89]*-x86_64*-*) > + # Darwin/OSX cases > CC="$CC -m$bits" > ;; > *) > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

2 8

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/Tests remote04.mal, , NONE, 1.1 remote05.mal, , NONE, 1.1
by Romulo Goncalves 20 Jan '08

20 Jan '08

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31308 > > Added Files: > remote04.mal remote05.mal > Log Message: > Examples of poor-mans bat transport. > > > --- NEW FILE: remote04.mal --- > #this example tests recieving tuple values as part of the > #MAL interface. > > b:bat[:oid,:int]:=mserver.receive(); > [ 0@0, 1] > [ 0@1, 2] > > c:bat[:oid,:str]:=mserver.receive(); > [ 0@0, "hello"] > [ 0@1, "world"] > > io.print(b); > io.print(c); > > --- NEW FILE: remote05.mal --- > #this example tests recieving tuple values as part of the > #MAL interface. > # now in a remote setting > > include remote; > # for testweb, we need to ask the server itself how it is available, use > # sabaoth for that > include sabaoth; > > h := sabaoth.getLocalConnectionHost(); > p := sabaoth.getLocalConnectionPort(); > > # create a connecton to ourself > remote.create("test", h, p, nil:str, "monetdb", "monetdb"); > > b:= bat.new(:oid,:str); > bat.insert( 0@0, "hello"); > bat.insert( 0@1, "world"); Shouldn't here be: bat.insert( b, 0@0, 1); bat.insert( b, 0@1, 2); Because your remote site is expecting to receive b:bat[:oid,:int]:=mserver.receive(); Romulo > > # store the values > remote.send("test",b); > c:= remote.query("test","io.print(b);") > io.print(c); > # shutdown > remote.destroy("test"); > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime/xrpc/export xrpc.html, xquery-decomposition, 1.3, 1.3.2.1
by Sjoerd Mullender 17 Jan '08

17 Jan '08

Should this also be fixed on the main branch? Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime/xrpc/export > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26528 > > Modified Files: > Tag: xquery-decomposition > xrpc.html > Log Message: > fixed wrong link > > > > Index: xrpc.html > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc/export/xrpc.html,v > retrieving revision 1.3 > retrieving revision 1.3.2.1 > diff -u -d -r1.3 -r1.3.2.1 > --- xrpc.html 18 Dec 2007 21:02:22 -0000 1.3 > +++ xrpc.html 17 Jan 2008 12:11:33 -0000 1.3.2.1 > @@ -40,7 +40,7 @@ > For more information on XRPC, check the following sources: > <ul> > <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/QuickTour/XRPC">tutorial</a> > - <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/Documentation/XRPC/">reference manual</a> > + <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/Documentation/XRPC-Extension.html">reference manual</a> > <li> the XRPC <a href="http://www.cwi.nl/htbin/ins1/publications?request=intabstract&key=ZhBo:VLDB…">VLDB Paper</a> > </ul> > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/sqlhelpers/xmlshred encoding.c, , 1.15, 1.16 hash.c, , 1.7, 1.8
by Sjoerd Mullender 10 Jan '08

10 Jan '08

Why did you move the include of pf_config.h? Generally, the *config.h files need to be *first*. Torsten Grust wrote: > Update of /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv10014 > > Modified Files: > encoding.c hash.c > Log Message: > -- pfshred now requires SAX2 (namespace URI processing) > > -- Local name and URI hash tables operate with xmlChar* values > > > > Index: encoding.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/encoding.c,v > retrieving revision 1.15 > retrieving revision 1.16 > diff -u -d -r1.15 -r1.16 > --- encoding.c 9 Jan 2008 09:16:05 -0000 1.15 > +++ encoding.c 10 Jan 2008 09:02:16 -0000 1.16 > @@ -25,13 +25,14 @@ > * $Id$ > */ > > -#include "pf_config.h" > #include <stdio.h> > #include <string.h> > > /* libxml SAX2 parser internals */ > #include "libxml/parserInternals.h" > > +#include "pf_config.h" > + > #include "encoding.h" > #include "guides.h" > #include "oops.h" > @@ -40,6 +41,9 @@ > > #include <assert.h> > > +#ifndef HAVE_SAX2 > + #error "libxml2 SAX2 interface required to compile the XML shredder `pfshred'" > +#endif > > FILE *out; > FILE *out_attr; > @@ -256,15 +260,14 @@ > > if (!localname) > return -1; > + > + localname_id = hashtable_find (localname_hash, localname); > > - localname_id = hashtable_find (localname_hash, (char *) localname); > - > - /* key not found */ > if (NOKEY (localname_id)) { > - /* create a new name id */ > + /* key not found, create a new name id */ > localname_id = global_localname_id++; > - /* add the pair into the hashtable */ > - hashtable_insert (localname_hash, (char *) localname, localname_id); > + /* add the (localname, localname_id) pair into the hash table */ > + hashtable_insert (localname_hash, localname, localname_id); > /* print the name binding if necessary */ > if (shredstate.names_separate) > fprintf (out_names, "%i, \"%s\"\n", localname_id, (char*) localname); > @@ -282,14 +285,13 @@ > if (!URI) > return -1; > > - uri_id = hashtable_find (uris_hash, (char *) URI); > + uri_id = hashtable_find (uris_hash, URI); > > - /* key not found */ > if (NOKEY (uri_id)) { > - /* create a new URI id */ > + /* key not found, create a new URI id */ > uri_id = global_uri_id++; > - /* add the pair into the hashtable */ > - hashtable_insert (uris_hash, (char *) URI, uri_id); > + /* add the (URI, uri_id) pair to the hash table */ > + hashtable_insert (uris_hash, URI, uri_id); > /* print the URI binding if necessary */ > if (shredstate.names_separate) > fprintf (out_uris, "%i, \"%s\"\n", uri_id, (char*) URI); > @@ -313,7 +315,8 @@ > > /* check if tagname is larger than TAG_SIZE characters */ > if (localname && xmlStrlen (localname) > TAG_SIZE) > - BAILOUT ("attribute local name `%s' exceeds %u characters", localname, TAG_SIZE); > + BAILOUT ("attribute local name `%s' exceeds %u characters", > + localname, TAG_SIZE); > > if (URI && xmlStrlen (URI) > TAG_SIZE) > BAILOUT ("namespace URI `%s' exceeds length of %u characters", > @@ -686,7 +689,7 @@ > report (void) > { > if (text_stripped > 0) { > - fprintf (err, "%u values were stripped to %u " > + fprintf (err, "%u text node/attribute values were stripped to %u " > "character(s).\n", text_stripped, text_size); > } > } > > Index: hash.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/src/sqlhelpers/xmlshred/hash.c,v > retrieving revision 1.7 > retrieving revision 1.8 > diff -u -d -r1.7 -r1.8 > --- hash.c 8 Jan 2008 10:00:07 -0000 1.7 > +++ hash.c 10 Jan 2008 09:02:16 -0000 1.8 > @@ -25,39 +25,39 @@ > * $Id$ > */ > > -#include "pf_config.h" > -#include "hash.h" > -#include "shred_helper.h" > #include <string.h> > #include <unistd.h> > #include <assert.h> > > +#include "pf_config.h" > +#include "hash.h" > +#include "shred_helper.h" > + > /* We use a seperate chaining strategy to > - * mantain our hash_table, > - * So each bucket is a chained list itself, > - * to handle possible collisions. > + * mantain our hash table, so each bucket is a chained list itself > + * to handle collisions. > */ > struct bucket_t { > - char *key; /**< key as string */ > - int id; /**< name_id */ > - bucket_t* next; /**< next bucket in our list */ > + xmlChar *key; /**< key (elem/attr name or namespace URI) */ > + int id; /**< name_id */ > + bucket_t *next; /**< next bucket in overflow chain */ > }; > > -/* size of the hashtable */ > +/* hashtable size */ > #define PRIME 113 > > /** > - * Lookup an id in a given bucket using it associated key. > + * Lookup an id in a given bucket using its associated key. > */ > static int > -find_id (bucket_t *bucket, char *key) > +find_id (bucket_t *bucket, const xmlChar *key) > { > bucket_t *cur_bucket = bucket; > > assert (key); > > while (cur_bucket) > - if (strcmp (cur_bucket->key, key) == 0) > + if (xmlStrcmp (cur_bucket->key, key) == 0) > return cur_bucket->id; > else > cur_bucket = cur_bucket->next; > @@ -69,16 +69,16 @@ > * Attach an (id, key) pair to a given bucket list. > */ > static bucket_t * > -bucket_insert (bucket_t *bucket, char *key, int id) > +bucket_insert (bucket_t *bucket, const xmlChar *key, int id) > { > int ident = find_id (bucket, key); > > - /* no key found */ > if (NOKEY (ident)) { > + /* no key found */ > bucket_t *newbucket = (bucket_t*) malloc (sizeof (bucket_t)); > > newbucket->id = id; > - newbucket->key = strndup (key, strlen(key)); > + newbucket->key = xmlStrdup (key); > > /* add new bucket to the front of list */ > newbucket->next = bucket; > @@ -95,13 +95,12 @@ > * Create the hash value for a given key. > */ > static int > -find_hash_bucket (char *key) > +find_hash_bucket (const xmlChar *key) > { > assert (key); > > - size_t len = strlen (key); > - /* keys have at least length 1 */ > - /* assert (len > 0); */ > + size_t len = xmlStrlen (key); > + > /* build a hash out of the first and the last character > and the length of the key */ > return (key[0] * key[MAX(0,len-1)] * len) % PRIME; > @@ -113,7 +112,9 @@ > hashtable_t > new_hashtable (void) > { > - hashtable_t ht = malloc (PRIME * sizeof (bucket_t)); > + hashtable_t ht; > + > + ht = (hashtable_t) malloc (PRIME * sizeof (bucket_t)); > > /* initialize the hash table */ > for (unsigned int i = 0; i < PRIME; i++) > @@ -126,24 +127,25 @@ > * Insert key and id into hashtable. > */ > void > -hashtable_insert (hashtable_t hash_table, char *key, int id) > +hashtable_insert (hashtable_t hash_table, const xmlChar *key, int id) > { > int hashkey; > > - assert (hash_table && key); > - > + assert (hash_table); > + assert (key); > + > hashkey = find_hash_bucket (key); > hash_table[hashkey] = bucket_insert (hash_table[hashkey], key, id); > - return; > } > > /** > * Find element in hashtable. > */ > int > -hashtable_find (hashtable_t hash_table, char *key) > +hashtable_find (hashtable_t hash_table, const xmlChar *key) > { > - assert (key); > + assert (key); > + > return find_id (hash_table[find_hash_bucket (key)], key); > } > > @@ -156,7 +158,6 @@ > bucket_t *bucket, *free_bucket; > > assert (hash_table); > - if (!hash_table) return; > > for (int i = 0; i < PRIME; i++) { > bucket = hash_table[i]; > @@ -165,9 +166,11 @@ > free_bucket = bucket; > bucket = bucket->next; > /* free the copied hash key */ > - if (free_bucket->key) free (free_bucket->key); > + if (free_bucket->key) > + xmlFree (free_bucket->key); > free (free_bucket); > } > } > + > free(hash_table); > } > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] Run Mtest on windows...
by Romulo Goncalves 08 Jan '08

08 Jan '08

I am running Mtest on windows 64 to fix a test that I submitted yesterday and I am getting the following error when I start my Mtest: E:\MonetDB\Current\sql\NT>RunMtest.bat -rq Mtest.py: ERROR: ' mserver5 "--config=E:\MonetDB\Current\MonetDB5\NT\etc\monetdb5.conf" --debug=10 --set "monet_mod_pat h=E:\MonetDB\Current\\sql\NT\src\backends\monet5\.libs;E:\MonetDB\Current\\sql\NT\src\backends\monet5;E:\MonetDB\Current\ MonetDB5\NT\lib\MonetDB5" --set "gdk_dbfarm=E:\MonetDB\Current\\sql\NT\dbfarm" --set "sql_logdir=E:\MonetDB\Current\\sql\ NT\sql_logs" --set "xquery_logdir=E:\MonetDB\Current\\sql\NT\xquery_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=38998 --set xrpc_port=43861 --set monet_prompt= --trace --dbname=mTests` failed: !WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !WARNING: BBPdir: initializing BBP. # MonetDB server v5.3.0, based on kernel v1.21.0 # Serving database 'mTests' # Compiled for i686-pc-win32/64bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2007 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see E:\MonetDB\Current\MonetDB5\NT\etc\monetdb5.conf) !FATAL: MALException:addUser:user 'monetdb' already exists ! Exit 1 Exception in thread Thread-12 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "C:\Python25\lib\threading.py", line 460, in __bootstrap File "C:\Python25\lib\threading.py", line 623, in run File "C:\Python25\lib\threading.py", line 364, in wait File "C:\Python25\lib\threading.py", line 238, in wait <type 'exceptions.ValueError'>: list.remove(x): x not in list E:\MonetDB\Current\sql\NT> I already cleaned my dbfarm was you can see... Any idea? Regards, Romulo

2 2

[Monetdb-developers] find temp tables....
by Romulo Goncalves 05 Jan '08

05 Jan '08

In sql we have some problems in locate temp tables. If I create the following table: create global temporary t1(id int); If I do: select * from t1; or select * from tmp.t1; I can see the table. However, When I do alter table, copy into, create trigger, create index statements I cannot detect the table even if I use tmp.t1 Should I open a bug? Or there is a special reason for this? Regards, Romulo

2 1

[Monetdb-developers] commit options for create table statement....
by Romulo Goncalves 05 Jan '08

05 Jan '08

Reading the MonetDB code seems that commit option ON COMMIT DROP ON COMMIT PRESERVE ROWS ON COMMIT DELETE ROWS should not be used for normal tables (the ones which are not temporary). If that is the rule, the commit option should be allowed in the create table statement. If it is not the rule, the commit option is not being used at all for the normal tables. Look to the following example (auto-commit mode is on): >CREATE TABLE test1(id int) ON COMMIT DELETE ROWS; &3 >insert into test1 values(1); &2 1 -1 >select * from test1; &1 3 1 1 1 % sys.test1 # table_name % id # name % int # type % 1 # length [ 1 ] > So what is the rule for MonetDB? Regards, Romulo

2 1

[Monetdb-developers] Geom II
by Stefan de Konink 04 Jan '08

04 Jan '08

And I probably should take the other part of the documentation out there more seriously. I guess this is 'suppose' to happen if you do *not* include geom in your (jfu) database. gdb --args /opt/monetdb/bin/mserver5 --dbinit="include sql;" --dbname=osm --dbfarm=/mnt/media/MonetDB5/dbfarm GNU gdb 6.7.1 Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu"... Using host libthread_db library "/lib/libthread_db.so.1". (gdb) run Starting program: /opt/monetdb/bin/mserver5 --dbinit=include\ sql\; --dbname=osm --dbfarm=/mnt/media/MonetDB5/dbfarm warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff30ffd000 [Thread debugging using libthread_db enabled] [New Thread 0x2b887c85d860 (LWP 2919)] [New Thread 0x40800950 (LWP 2922)] # MonetDB server v5.3.0, based on kernel v1.21.0 # Serving database 'osm' # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # Copyright (c) 1993-2007 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /opt/monetdb/etc/monetdb5.conf) [New Thread 0x41001950 (LWP 2923)] # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.21.0 loaded >!ERROR: BATnew:tt error Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x41001950 (LWP 2923)] 0x00002aaab26c8d3c in temp_create (b=0x0) at bat_utils.mx:130 130 temp_dup(b->batCacheid); (gdb) bt #0 0x00002aaab26c8d3c in temp_create (b=0x0) at bat_utils.mx:130 #1 0x00002aaab26c4d76 in e_bat (type=-31) at bat_storage.mx:84 #2 0x00002aaab26c62ee in load_bat (sname=0xba3028 "sys", tname=0x10607d8 "nodes_geom", bname=0x107d118 "geometry", type=-31, sz=1024, cnt=0x1060700) at bat_storage.mx:454 #3 0x00002aaab26c6660 in create_col (tr=0xba2ee8, c=0x107d088) at bat_storage.mx:518 #4 0x00002aaab26b639b in load_column (tr=0xba2ee8, t=0x10606c8, rid=193) at store.mx:516 #5 0x00002aaab26b691c in load_table (tr=0xba2ee8, s=0xba2f58, rid=193) at store.mx:583 #6 0x00002aaab26b7dd3 in load_schema (tr=0xba2ee8, id=1128, rid=38) at store.mx:850 #7 0x00002aaab26b819e in load_trans (tr=0xba2ee8, id=1128) at store.mx:902 #8 0x00002aaab26ba16d in store_init (debug=0, store=store_bat, logdir=0x63cfc0 "/opt/monetdb/var/MonetDB5/sql_logs", dbname=0x63ce90 "osm", stk=0) at store.mx:1328 #9 0x00002aaab269707d in mvc_init (debug=0, store=store_bat, stk=0) at sql_mvc.mx:247 #10 0x00002aaab26367bb in SQLinit (c=0x6046c8) at sql_scenario.mx:227 #11 0x00002b8879e003af in initScenario (c=0x6046c8, s=0x2b887a01b070) at mal_scenario.mx:263 #12 0x00002b8879e0159c in setScenario (c=0x6046c8, nme=0xad8123 "sql") at mal_scenario.mx:471 #13 0x00002b8879dc9951 in MSscheduleClient (command=0xad80e8 "LIT", challenge=0x41000fc0 "Roh8IvYDJa", fin=0xacdf40, fout=0xacdea0) at mal_session.mx:342 #14 0x00002aaab0f8527b in doChallenge (cmd=0x0, in=0xacdd60, out=0xacdcc0) at mserver.mx:481 #15 0x00002aaab0f85533 in SERVERlistenThread (Sock=0x9f0938) at mserver.mx:563 #16 0x00002b887b8d5067 in start_thread () from /lib/libpthread.so.0 #17 0x00002b887c5d9bed in clone () from /lib/libc.so.6 People may wonder why I tried anyway... The simple query "select * from nodes LIMIT 1;" also took far more time (22s) to complete than expected. Stefan

1 0

[Monetdb-developers] Geom and its examples.
by Stefan de Konink 04 Jan '08

04 Jan '08

Hi, So I decided to try out Geom, and after copying in a small set of data. I decided to test if it was any faster than my own made up SQL fantasy, which greatly seems to depend on the state MonetDB is in, I had some nice results before my Geom test, but I guess I just have overloaded the database again. select * from nodes, node_tags where latitude >= 52.375 AND latitude <= 52.379 AND longitude >= 4.89 AND longitude <= 4.90 AND nodes.id = node_tags.node AND name='amenity' and value='atm'; Took before 8 seconds... now it is ending nowhere. (In the over 10 minutes range.) Next to this now is 27.8GB in virtual memory (my entire dbfarm directory, while the data that is required for this query BY FAR not that amount of data.) So I suspect a problem in the Geom code :) read on :) I'm using the very latest CVS HEAD. sql>SELECT id, geometry FROM nodes_geom LIMIT 1; +----+---------------------------------------------------------------------+ | id |geometry | +====+=====================================================================+ | 2 |POINT (50.1360073999999969 8.3023717000000001) | +----+---------------------------------------------------------------------+ Looks shining. Lets do a basic operation: sql>SELECT id, X(geometry) FROM nodes_geom LIMIT 1; !types point(0,0) (wkb) and geometry(0,0) (wkb) are not equal Hum. Was it my poor choose of column name? So I decided to try out the example: http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/SQL_002fGIS.html sql>SELECT forests.name,buildings.name more>FROM forests,buildings more>WHERE forests.name = 'Green Forest' and more> Overlaps(forests.shape, buildings.outline) = true; !types multipolygon(0,0) (wkb) and geometry(0,0) (wkb) are not equal So what went so terribly wrong? Or better: what can be done to fix it. Yours Sincerely, Stefan de Konink

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/NT monetdb_winconfig.h, , 1.6, 1.7 winrules.msc, , 1.16, 1.17
by Romulo Goncalves 03 Jan '08

03 Jan '08

Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB/NT > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15904/NT > > Modified Files: > monetdb_winconfig.h winrules.msc > Log Message: > > made OID size (command-line-) configurable (also) on 64-bit Windows: > As opposed to Unix, where the default OID size is identical to the > native machine word size (in particular 64-bit OIDs on 64-bit systems), > the default OID size on 64-bit Windows has been and remains 32-bit > (see NT/monetdb_winconfig.h for details). > This default can be overruled by compiling with `nmake MONET_OID64=1`. > (In case the default in NT/monetdb_winconfig.h is changed to 64-bit > OIDs on 64-bit Windows, 32-bit OIDs on 64-bit Windows can be enabled > by compiling with `nmake MONET_OID32=1`.) Following these rules I decided to compile MonetDB on windows like this: E:\MonetDB\Current\MonetDB\NT>nmake MONET_OID64=1 Microsoft (R) Program Maintenance Utility Version 8.00.50727.42 Copyright (C) Microsoft Corporation. All rights reserved. "E:\Microsoft Visual Studio 8\VC\BIN\amd64\nmake.exe" /nologo /f ".\..\Makefile.msc" "prefix=E:\MonetDB\Current\M onetDB\NT" "bits=32" all if not exist "src" mkdir "src" copy ".\..\src\Makefile.msc" "src\Makefile" 1 file(s) copied. cd "src" && "E:\Microsoft Visual Studio 8\VC\BIN\amd64\nmake.exe" /nologo "prefix=E:\MonetDB\Current\MonetDB\NT" "bits=32" all if not exist "common" mkdir "common" copy ".\..\..\src\common\Makefile.msc" "common\Makefile" 1 file(s) copied. cd "common" && "E:\Microsoft Visual Studio 8\VC\BIN\amd64\nmake.exe" /nologo "prefix=E:\MonetDB\Current\MonetDB\N T" "bits=32" all cl -GF -W3 -wd4273 -wd4102 -MD -nologo -Ox -I. -I.\..\.. "-IC:\Pthreads\include" -I.\..\..\..\src\common -DHAVE_L IBZ "-IC:\zlib-1.2.3.win32\include" -DHAVE_ICONV "-IC:\iconv-1.9.2.win32\include" -DLIBSTREAM -DCOMPILE_DL_STREAM - Fo"stream.obj" -c "stream.c" stream.c As you can see the compilation is done for 32 bits and I would like to do it for 64 bits. Could someone give a help here please. Thanks in advance, Romulo

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, , 1.157, 1.158
by Stefan Manegold 02 Jan '08

02 Jan '08

On Tue, Dec 11, 2007 at 10:08:13AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18823 > > Modified Files: > pftijah.mx > Log Message: > - add an optional extra commit after document indexing to survey a bug > where a server restart leads to a zero filled index bat. Just a stupid(?) question: In the code below, the "extra_commit" is initialized to "false", and never changed --- hence, the extra subcommit() in if ( extra_commit ) subcommit(_tj_collection_str(ftiName)); is never called, right? Hence, what is/was the intention/need of this checkin? Stefan > Index: pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.157 > retrieving revision 1.158 > diff -u -d -r1.157 -r1.158 > --- pftijah.mx 4 Oct 2007 10:41:12 -0000 1.157 > +++ pftijah.mx 11 Dec 2007 10:08:09 -0000 1.158 > @@ -315,7 +315,9 @@ > # # > ##################################################################### > > -var verbose := false; > +var verbose := false; > + > +var extra_commit := false; > > const dflt_ft_index := "DFLT_FT_INDEX"; > const dflt_bg_index := "DFLT_FT_INDEX"; > @@ -997,6 +999,7 @@ > _tj_add2collection(ftiName, collBat, uri_loc, uri_name, store); > _tj_finalize_collection(ftiName, collBat, FALSE); > _tj_commit(collBat); > + if ( extra_commit ) subcommit(_tj_collection_str(ftiName)); > if ( timing ) { > var ms := (usec()-t_start)/1000; > printf("#C[%s]:tj_add2collection(): + aggregate time = %lld.%03llds.\n",ftiName,/(ms,1000),%(ms,1000)); > @@ -1038,6 +1041,7 @@ > printf("#C[%s]:tj_add2collection(BAT): + aggregate time = %lld.%03llds.\n",ftiName,/(ms,1000),%(ms,1000)); > } > _tj_commit(collBat); > + if ( extra_commit ) subcommit(_tj_collection_str(ftiName)); > }); > lock_unset(coll_lock); > if (not(isnil(err))) ERROR(err); > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] external function for XQuery
by Andrea Romei 24 Dec '07

24 Dec '07

Dear MonetDB developers, I'am trying to use MonetDB XQuery for a programmative purpose and I need to invoke external functions into the XQuery engine (e.g. declare function random() as xs:double external). Does MonetDB XQuery support external C/C++/Java functions? If yes, how I can use them? If not, this feature will be implemented in the future? Thanks in advance, Andrea

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, Clients_1-20, 1.88.2.4, 1.88.2.5
by Peter Boncz 21 Dec '07

21 Dec '07

Hi Sjoerd, Thanks for fixing some things I was unaware of (empty lines in other languages?) > The timer is again started when the first line of a query is typed. > This is the case for *all* languages. I cannot image cases where it is useful for mclient to keeps stays on someone's typing speed, while I can image cases where one would like to have information about query performance. Do others agree? (I maybe unjustly assumed that timing query performance was the intended behavior) Peter > -----Original Message----- > From: monetdb-checkins-bounces(a)lists.sourceforge.net > [mailto:monetdb-checkins-bounces@lists.sourceforge.net] On > Behalf Of Sjoerd Mullender > Sent: Wednesday, December 19, 2007 3:04 PM > To: monetdb-checkins(a)lists.sourceforge.net > Subject: [Monetdb-checkins] clients/src/mapiclient > MapiClient.mx,Clients_1-20, 1.88.2.4, 1.88.2.5 > > > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv6918 > > Modified Files: > Tag: Clients_1-20 > MapiClient.mx > Log Message: > Did Peter's checkin and Niels' fix correctly: In XQuery mode, when > typing <> when there is no query yet will result in just asking for > more data, and empty lines in other modes will not end the session. > Also fixed \f processing in that the new value is now remembered for a > subsequent bare \f. > The timer is again started when the first line of a query is typed. > This is the case for *all* languages. > > > Index: MapiClient.mx > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapiclient/MapiClient.mx,v > retrieving revision 1.88.2.4 > retrieving revision 1.88.2.5 > diff -u -d -r1.88.2.4 -r1.88.2.5 > --- MapiClient.mx 19 Dec 2007 08:35:21 -0000 1.88.2.4 > +++ MapiClient.mx 19 Dec 2007 14:04:09 -0000 1.88.2.5 > @@ -152,6 +152,7 @@ > XMLformatter > }; > static enum formatters formatter = NOformatter; > +char *output = NULL; /* output format as string */ > > #define DEFWIDTH 80 > > @@ -247,8 +248,6 @@ > } > } > > -char *output = NULL; /* output format is a global variable */ > - > static void > SQLsetSpecial(const char *command) > { > @@ -796,9 +795,14 @@ > } > > static void > -setFormatter(char *s) > +setFormatter(Mapi mid, char *s) > { > - if (strcmp(s, "sql") == 0) > + if (mode == XQUERY) { > + mapi_output(mid, s); > + if (output != NULL) > + free(output); > + output = strdup(s); > + } else if (strcmp(s, "sql") == 0) > formatter = TABLEformatter; > else if (strcmp(s, "csv") == 0) > formatter = CSVformatter; > @@ -1123,13 +1127,13 @@ > fprintf(toConsole, "\\a - disable auto commit\n"); > } > if (mode == XQUERY) { > - fprintf(toConsole, "\\f - result format: > dm or xml[-noheader][-typed|-noroot|-root-FOOBAR] \n"); > - } else { > + fprintf(toConsole, "\\f - result format: > dm or xml[-noheader][-typed|-noroot|-root-FOOBAR]\n"); > + } else { > fprintf(toConsole, "\\e - echo the query > in sql formatting mode\n"); > - fprintf(toConsole, "\\f - format using a > built-in renderer {csv,tab,raw,sql,xml} \n"); > + fprintf(toConsole, "\\f - format using a > built-in renderer {csv,tab,raw,sql,xml}\n"); > fprintf(toConsole, "\\w# - set maximal page > width (-1=raw,0=no limit, >0 max char)\n"); > - fprintf(toConsole, "\\r# - set maximum rows > per page (-1=raw)\n "); > - } > + fprintf(toConsole, "\\r# - set maximum rows > per page (-1=raw)\n"); > + } > fprintf(toConsole, "\\L file - save client/server > interaction\n"); > fprintf(toConsole, "\\X - trace mclient code\n"); > fprintf(toConsole, "\\q - terminate session\n"); > @@ -1140,9 +1144,10 @@ > { > char *line = NULL; > char *buf = NULL; > - ssize_t length; > + size_t length; > MapiHdl hdl = mapi_get_active(mid); > MapiMsg rc = MOK; > + int sent = 0; /* whether we sent any data to > the server */ > > #ifdef HAVE_LIBREADLINE > if (prompt == NULL) > @@ -1185,13 +1190,17 @@ > line = fgets(buf, BUFSIZ, fp); > } > if (line == NULL || (mode == XQUERY && line[0] > == '<' && line[1] == '>')) { > - length = 0; > - if (line) > - length = -1; > - else if (hdl == NULL) > - return 0; /* EOF: nothing more to do */ > + /* end of file */ > + if (hdl == NULL) { > + if (line != NULL) > + continue; > + /* nothing more to do */ > + return 0; > + } > + > /* hdl != NULL, we should finish the > current query */ > line = NULL; > + length = 0; > } else > length = strlen(line); > if (hdl == NULL && length > 0 && line[length - > 1] == '\n') { > @@ -1217,7 +1226,7 @@ > if (mark2) > free(mark2); > mark2 = strdup(line + 2); > - if (mode == XQUERY) > + if (mode == XQUERY) > > mapi_profile(mid, mark != NULL); > continue; > case 'X': > @@ -1378,31 +1387,31 @@ > if (*line == 0) { > > fprintf(toConsole, "Current formatter: "); > if (mode == XQUERY) > - > fprintf(toConsole, "%s\n", output); > - else switch > (formatter) { > - case RAWformatter: > - > fprintf(toConsole, "raw\n"); > - break; > - case TABLEformatter: > - > fprintf(toConsole, "sql\n"); > - break; > - case CSVformatter: > - > fprintf(toConsole, "csv\n"); > - break; > - case TABformatter: > - > fprintf(toConsole, "tab\n"); > - break; > - case XMLformatter: > - > fprintf(toConsole, "xml\n"); > - break; > - default: > - > fprintf(toConsole, "none\n"); > - break; > + > fprintf(toConsole, "%s\n", output == NULL ? "dm" : output); > + else { > + switch > (formatter) { > + case > RAWformatter: > + > fprintf(toConsole, "raw\n"); > + break; > + case > TABLEformatter: > + > fprintf(toConsole, "sql\n"); > + break; > + case > CSVformatter: > + > fprintf(toConsole, "csv\n"); > + break; > + case > TABformatter: > + > fprintf(toConsole, "tab\n"); > + break; > + case > XMLformatter: > + > fprintf(toConsole, "xml\n"); > + break; > + default: > + > fprintf(toConsole, "none\n"); > + break; > + } > } > - } else if (mode == XQUERY) > - mapi_output(mid, line); > - else > - setFormatter(line); > + } else > + setFormatter(mid, line); > continue; > default: > showCommands(); > @@ -1412,34 +1421,37 @@ > } > > if (hdl == NULL) { > + timerStart(); > hdl = mapi_query_prep(mid); > CHECK_RESULT(mid, hdl, buf, continue); > } > assert(hdl != NULL); > > if (length > 0) { > + sent = 1; > SQLsetSpecial(line); > mapi_query_part(hdl, line, length); > CHECK_RESULT(mid, hdl, buf, continue); > } > > - if (mode == XQUERY && line) > - continue; /* XQuery always wants more data */ > - > /* If the server wants more but we're at the > end of file (line == NULL), notify the > server that we don't have anything more. > If the server still wants more (shouldn't > happen according to the protocol) we break > out of the loop (via the continue). The > - assertion at the end will then go off. > - */ > - timerStart(); > - if (mapi_query_done(hdl) == MMORE) { > - assert(mode != XQUERY); /* XQuery never > sends MMORE */ > - if (line != NULL) continue; /* get more data */ > - timerStart(); > - if (mapi_query_done(hdl) == MMORE) { > + assertion at the end will then go off. > + > + Note that XQuery is weird: we continue > + sending more until we reach end-of-file, > + and *then* we send the mapi_query_done. To > + exit, you need to send an end-of-file > + again. */ > + if (mode == XQUERY || mapi_query_done(hdl) == MMORE) { > + if (line != NULL) { > + continue; /* get more data */ > + } else if (mapi_query_done(hdl) == MMORE) { > + assert(mode != XQUERY); /* > XQuery never sends MMORE */ > hdl = NULL; > continue; /* done */ > } > @@ -1459,7 +1471,7 @@ > > /* for XQuery, only exit when end-of-file and we > didn't send any data */ > - } while (hdl != NULL); > + } while (line != NULL || (mode == XQUERY && sent)); > /* reached on end of file */ > assert(hdl == NULL); > return 0; > @@ -1603,7 +1615,6 @@ > strcmp(optarg, "x") == 0) { > language = "xquery"; > mode = XQUERY; > - if (!output) output = "dm"; > } else { > fprintf(stderr, "language > option needs to be one of sql, mil, mal, or xquery\n"); > exit(-1); > @@ -1618,7 +1629,7 @@ > passwd = optarg; /* can be NULL */ > break; > case 'f': > - output = optarg; /* output format */ > + output = strdup(optarg); /* output format */ > break; > case 'I': > input = optarg; > @@ -1726,13 +1737,8 @@ > > mapi_profile(mid, mark != NULL); > mapi_trace(mid, trace); > - > - if (output) { > - if (mode == XQUERY) > - mapi_output(mid, output); > - else > - setFormatter(output); > - } > + if (output) > + setFormatter(mid, output); > > c = 0; > > > > -------------------------------------------------------------- > ----------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services > for just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.n et/marketplace _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

5 6

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_posix.mx, , 1.150, 1.151
by Sjoerd Mullender 20 Dec '07

20 Dec '07

Peter Boncz wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19538 > > Modified Files: > gdk_posix.mx > Log Message: > - try making the file writable if CreateFile fails I doubt that this will work since Windows reports: "The process cannot access the file because it is being used by another process." > > > Index: gdk_posix.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_posix.mx,v > retrieving revision 1.150 > retrieving revision 1.151 > diff -u -d -r1.150 -r1.151 > --- gdk_posix.mx 19 Dec 2007 18:58:33 -0000 1.150 > +++ gdk_posix.mx 20 Dec 2007 08:45:23 -0000 1.151 > @@ -1290,8 +1290,12 @@ > > h1 = CreateFile(path, mode0, mode1, &sa, OPEN_ALWAYS, mode2, NULL); > if (h1 == INVALID_HANDLE_VALUE) { > - GDKsyserror("MT_mmap: CreateFile('%s', %lu, %lu, &sa, %lu, %lu, NULL) failed\n", path, mode0, mode1, (DWORD) OPEN_ALWAYS, mode2); > - return (void *) -1; > + (void) SetFileAttributes(path, FILE_ATTRIBUTE_NORMAL); > + h1 = CreateFile(path, mode0, mode1, &sa, OPEN_ALWAYS, mode2, NULL); > + if (h1 == INVALID_HANDLE_VALUE) { > + GDKsyserror("MT_mmap: CreateFile('%s', %lu, %lu, &sa, %lu, %lu, NULL) failed\n", path, mode0, mode1, (DWORD) OPEN_ALWAYS, mode2); > + return (void *) -1; > + } > } > > h2 = CreateFileMapping(h1, &sa, mode3, (DWORD) ((((gdk_int64) off + (gdk_int64) len) >> 32) & LL_CONSTANT(0xFFFFFFFF)), (DWORD) ((off + len) & LL_CONSTANT(0xFFFFFFFF)), NULL); > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services > for just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] Replication services
by Aaron Craelius 19 Dec '07

19 Dec '07

I see a mention of replication services being planned for the Winter of 2007 for SQL in the roadmap. Does anyone have any idea as to when they will actually be available and what the development status/plan is? Thanks, Aaron

1 0

[Monetdb-developers] Inconsistent code in pftijah.mx?
by Stefan Manegold 17 Dec '07

17 Dec '07

Hoi, I have no idea what the below code snippets from pathfinder/modules/pftijah/pftijah.mx are supposed to do, but to me, they look (at least) "inconsistent". Maybe, an expert might want to look at them ... Stefan ======== PROC tj_delete_collection(str ftiName, bit doAdmLock) : void { [...] bat("tj_" + ftiName + "_fragments")@batloop() { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ var fpfx := str(int($h)); ^^ bat("tj_" + ftiName + "_tid"+fpfx).persists(false); bat("tj_" + ftiName + "_size"+fpfx).persists(false); } [...] } ======== PROC _tj_collection_str(str ftiName) : BAT[void,bat] { [...] bat("tj_" + ftiName + "_fragments")@batloop() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ { tjCollBat.append("tj_" + ftiName + "_tid" + str(int($t))); ^^ tjCollBat.append("tj_" + ftiName + "_size"+ str(int($t))); ^^ } [...] ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra/opt opt_const.c, , 1.24, 1.25
by Stefan Manegold 14 Dec '07

14 Dec '07

Intel's icc (IMHO correctly) says: ======== icc -no-gcc -DHAVE_CONFIG_H -I. -I/scratch/monet.Int.64.64.d.29718/pathfinder/compiler/algebra/opt -I../../.. -I/scratch/monet.Int.64.64.d.29718/pathfinder/compiler/algebra/opt -I../../include -I/scratch/monet.Int.64.64.d.29718/pathfinder/compiler/algebra/opt/../../include -I/scratch/monet.Int.64.64.d.29718/MonetDB/.Int.64.64.d-Fedora6/include/MonetDB -I/scratch/monet.Int.64.64.d.29718/MonetDB/.Int.64.64.d-Fedora6/include/MonetDB/common -I/scratch/monet.Int.64.64.d.29718/MonetDB/.Int.64.64.d-Fedora6/include/MonetDB/gdk -DLIBOPT -we140 -wd1418 -Wall -w2 -c99 -mp1 -O3 -restrict -unroll -tpp7 -axWP -we266 -Werror -wd1418,1419,279,981,810,193,111,1357,1572,1599 -c /scratch/monet.Int.64.64.d.29718/pathfinder/compiler/algebra/opt/opt_const.c -KPIC -DPIC -o .libs/libopt_la-opt_const.o /scratch/monet.Int.64.64.d.29718/pathfinder/compiler/algebra/opt/opt_const.c(125): error #589: transfer of control bypasses initialization of: variable "items" (declared at line 168) variable "count" (declared at line 169) switch (p->kind) { ^ compilation aborted for /scratch/monet.Int.64.64.d.29718/pathfinder/compiler/algebra/opt/opt_const.c (code 2) make[8]: *** [libopt_la-opt_const.lo] Error 1 ======== cf., http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.Int.64.6… Stefan On Thu, Dec 13, 2007 at 10:08:57PM +0000, Tom Schreiber wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra/opt > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv11953/compiler/algebra/opt > > Modified Files: > opt_const.c > Log Message: > -- Introduced a rewrite for the la_serialize_rel operator. > -- For plans that use the la_serialize_rel operator, we ensure (analogous to the existing rewrite for la_serialize_seq) that attributes with constant values are introduced at the latest possible point in the plan. > > Index: opt_const.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/opt/opt_const.c,v > retrieving revision 1.24 > retrieving revision 1.25 > diff -u -d -r1.24 -r1.25 > --- opt_const.c 10 Dec 2007 15:10:00 -0000 1.24 > +++ opt_const.c 13 Dec 2007 22:08:55 -0000 1.25 > @@ -139,6 +139,48 @@ > } > break; > > + > + case la_serialize_rel: > + > + /* Introduce (superfluous) attach-ops for constant attributes. > + This rewrite ensures, that constants are introduced at > + the latest possible point in the plan. > + */ > + > + /* Rewrite for the iter-attribute. */ > + if (PFprop_const_left (p->prop, p->sem.ser_rel.iter)) { > + L(p) = add_attach (L(p), p->sem.ser_rel.iter, > + PFprop_const_val_left ( > + p->prop, > + p->sem.ser_rel.iter)); > + > + } > + > + /* Rewrite for the pos-attribute. */ > + if (PFprop_const_left (p->prop, p->sem.ser_rel.pos)) { > + L(p) = add_attach (L(p), p->sem.ser_rel.pos, > + PFprop_const_val_left ( > + p->prop, > + p->sem.ser_rel.pos)); > + } > + > + /* Rewrite for the item-attributes. */ > + PFalg_attlist_t items = p->sem.ser_rel.items; > + unsigned int count = items.count; > + for (unsigned int i = 0; i < count; i++) > + { > + PFalg_att_t item = items.atts[i]; > + if (PFprop_const_left (p->prop, item)) { > + L(p) = add_attach (L(p), item, > + PFprop_const_val_left ( > + p->prop, > + item)); > + } > + } > + > + break; > + > + > case la_eqjoin: > { /** > * If both join columns are constant > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services > for just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.96, 1.97
by Sjoerd Mullender 13 Dec '07

13 Dec '07

On 2007-12-13 19:35, Niels Nes wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17600/src/mapiclient > > Modified Files: > MapiClient.mx > Log Message: > don't leak memory > > > Index: MapiClient.mx > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapiclient/MapiClient.mx,v > retrieving revision 1.96 > retrieving revision 1.97 > diff -u -d -r1.96 -r1.97 > --- MapiClient.mx 13 Dec 2007 15:44:24 -0000 1.96 > +++ MapiClient.mx 13 Dec 2007 18:35:21 -0000 1.97 > @@ -1224,6 +1224,8 @@ > fflush(stdout); > } > #endif > + if (buf != oldbuf) > + free(buf); > buf = oldbuf; > line = fgets(buf, BUFSIZ, fp); > } > @@ -1237,7 +1239,7 @@ > line = to; > iconv(cd_in, &from, &fromlen, &to, &tolen); > *to = 0; > - if (oldbuf) > + if (!oldbuf) I prefer oldbuf == NULL. oldbuf is not a Boolean. > free(buf); > buf = line; > } > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services > for just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] mclient query output....
by Romulo Goncalves 13 Dec '07

13 Dec '07

I have a file with several queries: if I run mclient like this: mclient -lsql file.sql I get output only for the first 5 queries.. If run mclient -lsql < file.sql I get the complete output. Should this be reported as a bug or is there a logical explanation? Regards, Romulo

2 1

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/autogen README, MonetDB_1-20, 1.5, 1.5.8.1
by Fabian Groffen 13 Dec '07

13 Dec '07

On 13-12-2007 15:22:43 +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/buildtools/autogen > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv12301/autogen > > Modified Files: > Tag: MonetDB_1-20 > README > Log Message: > > documenting a Mac-peculiarity we recently encountered Keep in mind that Apple builds their Python, Perl, etc. etc. in very strange non-standard ways which break in many ways for other applications. (And it indeed changes inbetween major releases of the OS.) I agree with the noting of this difference, but would like to note that I think we should encourage people to use non-Apple built versions of software when developing on MonetDB. The amount of patches that Apple applies (and rejected upstream) is not good for our bugfixabilities, though easily solvable by using the respective non-changed versions via any of the widely known providers of "Unix" software for Mac OS X. > Index: README > =================================================================== > RCS file: /cvsroot/monetdb/buildtools/autogen/README,v > retrieving revision 1.5 > retrieving revision 1.5.8.1 > diff -u -d -r1.5 -r1.5.8.1 > --- README 17 Oct 2006 08:59:45 -0000 1.5 > +++ README 13 Dec 2007 15:22:41 -0000 1.5.8.1 > @@ -14,6 +14,16 @@ > <python-version> is the version of Python used (e.g. 2.4). > You can automatically get the respective directory via > "<install-dir-prefix>/`python -c 'import distutils.sysconfig; print distutils.sysconfig.get_python_lib(0,0,"")'`". > +( > +In case your are using Apple-built Python 2.5.1 on Mac OS X 10.5.1 (Darwin 9.1.0), > +it can be that the above command yields > +"<install-dir-prefix>/Library/Python/2.5/site-packages", > +while Python's distutils.core.setup() actually installs the python modules into > +"<install-dir-prefix>/lib/python2.5/site-packages". > +In that case, replacing the above command by > +"<install-dir-prefix>/`python -c 'import distutils.sysconfig; print distutils.sysconfig.get_python_lib(0,1,"")'`/site-packages" > +might help to overcome this inconsistency. > +) > > On Windows, to follow these instructions, Python must of course be > found in your PATH. The default location where autogen.py is

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB4/src/modules/plain arith.mx, MonetDB_4-20, 1.3, 1.3.6.1
by Stefan Manegold 13 Dec '07

13 Dec '07

On Thu, Dec 13, 2007 at 11:40:49AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB4/src/modules/plain > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24024/src/modules/plain > > Modified Files: > Tag: MonetDB_4-20 > arith.mx > Log Message: > added function sign (sql allready expected this to exist) > > returns 1 for positive numbers > 0 for 0 > -1 for negative > > > Index: arith.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB4/src/modules/plain/arith.mx,v > retrieving revision 1.3 > retrieving revision 1.3.6.1 > diff -u -d -r1.3 -r1.3.6.1 > --- arith.mx 20 Feb 2007 11:32:12 -0000 1.3 > +++ arith.mx 13 Dec 2007 11:40:47 -0000 1.3.6.1 > @@ -251,6 +251,7 @@ > @= mel_unary_ops > .COMMAND abs(@1) : @1 = @1_unary_ABS; "absolute value" > .OPERATOR "-" (@1) : @1 = @1_unary_NEG; "negative value" > + .COMMAND sign(@1) : @1 = @1_unary_SIGN; "Returns +1, 0, -1 based on the sign of the given expression" > @= unary_ops > @:@1_unary_ops(chr)@ > @:@1_unary_ops(bte)@ > @@ -411,9 +412,11 @@ > > #define arith_abs(s) ((s)<0)?-(s):(s) > #define arith_neg(s) (-(s)) > +#define arith_sign(s) ((s)<0)?-1:(s==0)?0:1 ^^^ ^ should(n't) the second "s" be in parentheses, too? Stefan > @= c_unary_ops > @:arith_unop(_ABS,arith_abs,@1)@ > @:arith_unop(_NEG,arith_neg,@1)@ > +@:arith_unop(_SIGN,arith_sign,@1)@ > @c > @:unary_ops(c)@ -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Nightly Testing: ("Stable", "Current") -> ("Current", Stable")
by Stefan Manegold 13 Dec '07

13 Dec '07

Dear fellow MonetDB developers, for your information: To improve overlap between the actual testing (on the "clients") and the "aftermath" (results analysis/assessment & TestWeb generation) on the "server", I'll switch the order of nightly MonetDB testing (as usually starting just after midnight CET) from "first Stable, then Current" to "first Current, then Stable" as of tonight/tomorrow morning. I plan to keep the "first Current, then Stable" order (at least) until we start preparing a new release (i.e., create new release branches), or until there is any good reason/need/request to return to the "first Stable, then Current" order. Kind regards, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/kernel batcalc.mx, , 1.145, 1.146 batifthen.mx, , 1.10, 1.11 group.mx, , 1.99, 1.100 kprelude.mx, , 1.7, 1.8
by Stefan Manegold 12 Dec '07

12 Dec '07

On Tue, Dec 11, 2007 at 05:32:04PM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/kernel > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28347/src/modules/kernel > > Modified Files: > batcalc.mx batifthen.mx group.mx kprelude.mx > Log Message: > batifthen.mx: use (fast) direct assignments instead of bunfastins for > ifthenelse (not for ifthen as it can be small as the input bat) > group.mx: return read only bats. Intel's icc says: ======== icc -no-gcc -DHAVE_CONFIG_H -I. -I/scratch/monet.Int.64.64.d.23892/MonetDB5/src/modules/kernel -I../../.. -I/scratch/monet.Int.64.64.d.23892/MonetDB5/src/modules/kernel -I../atoms -I/scratch/monet.Int.64.64.d.23892/MonetDB5/src/modules/kernel/../ atoms -I../../mal -I/scratch/monet.Int.64.64.d.23892/MonetDB5/src/modules/kernel/../../mal -I/scratch/monet.Int.64.64.d.23892/MonetDB/.Int.64.64.d-Fedora6/include/MonetDB -I/scratch/monet.Int.64.64.d.23892/MonetDB/.Int.64.64.d-Fedora6/include/Mone tDB/common -I/scratch/monet.Int.64.64.d.23892/MonetDB/.Int.64.64.d-Fedora6/include/MonetDB/gdk -DLIBBATIFTHEN -we140 -wd1418 -Wall -w2 -c99 -mp1 -O3 -restrict -unroll -tpp7 -axWP -we266 -Werror -wd1418,1419,279,981,810,193,111,1357,1572,1599 -D_RE ENTRANT -c batifthen.c -KPIC -DPIC -o .libs/lib_batifthen_la-batifthen.o /scratch/monet.Int.64.64.d.23892/MonetDB5/src/modules/kernel/batifthen.mx(430): error #593: variable "bi" was set but never used BATiter bi, tbi; ^ /scratch/monet.Int.64.64.d.23892/MonetDB5/src/modules/kernel/batifthen.mx(430): error #593: variable "bi" was set but never used BATiter bi, tbi; ^ compilation aborted for batifthen.c (code 2) make[7]: *** [lib_batifthen_la-batifthen.lo] Error 1 ======== basic questions: is it correct that the input's heads are no longer used in CMDifThenElseCst2_@1(), while they are (still) used in CMDifThenElseCst_@1(), CMDifThenElse_@1() & CMDifThenElseCst1_@1() ? Stefan [...] > Index: batifthen.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/batifthen.mx,v > retrieving revision 1.10 > retrieving revision 1.11 > diff -u -d -r1.10 -r1.11 > --- batifthen.mx 14 Nov 2007 12:52:15 -0000 1.10 > +++ batifthen.mx 11 Dec 2007 17:32:02 -0000 1.11 [...] > @@ -289,38 +436,31 @@ > @:getBATdescriptor(bid,b,"batcalc.ifThenElse")@ > @:getBATdescriptor(tid,tb,"batcalc.ifThenElse",BBPreleaseref(b->batCacheid);)@ > @:chkSize(b,tb,ifThenElse)@ > - @:resBAT(@1,"batcalc.ifThenElse")@ > + @:resultBAT(@1,"batcalc.ifThenElse")@ > + bn->tsorted = FALSE; > + BATkey(BATmirror(bn), FALSE); > > - if (ATOMvarsized(TYPE_@1)) > - val = *(@1**)val; > bi = bat_iterator(b); > tbi = bat_iterator(tb); > > t = (bit*) Tloc(b,BUNfirst(b)); > - > + > + if (ATOMvarsized(TYPE_@1)) > + val = *(@1**)val; > BATloop(b, p, q) { > if (*t == bit_nil) > - BUNfastins(bn, BUNhead(bi,p), (ptr) &nilval); > + BUNfastins(bn, NULL, (ptr) &nilval); > else if (*t) > - BUNfastins(bn, BUNhead(bi,p), BUNtail(tbi, p)); > + BUNfastins(bn, NULL, BUNtail(tbi, p)); > else > - BUNfastins(bn, BUNhead(bi,p), val); > + BUNfastins(bn, NULL, val); > t++; > } > BBPreleaseref(tb->batCacheid); > @:wrapup@ > } > +@:ifthenImpl(@1)@ > @c > -#include "mal_config.h" > -#include "batifthen.h" > - > -@:ifthencstImpl(int)@ > -@:ifthencstImpl(sht)@ > -@:ifthencstImpl(lng)@ > -@:ifthencstImpl(oid)@ > -@:ifthencstImpl(flt)@ > -@:ifthencstImpl(dbl)@ > -@:ifthencstImpl(str)@ > -@:ifthencstImpl(chr)@ > -@:ifthencstImpl(bit)@ > +@:ifthenelseDefault(oid)@ > +@:ifthenelseDefault(str)@ > @} > [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 merovingian.mx, , 1.20, 1.21
by Sjoerd Mullender 11 Dec '07

11 Dec '07

On 2007-12-11 20:16, Fabian wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv7608 > > Modified Files: > merovingian.mx > Log Message: > FreeBSD is kind of b0rkened, or my code is completely wrong. It appears > that FreeBSD doesn't set the pid of the child that sends the SIGCHLD to > its parent, leaving merovingian clueless when it receives such signal. > This "bug" is kind of limiting merovingian's abilities on FreeBSD in a way. In the signal handler for SIGCHILD, you're supposed to call one of the variants of wait to get the PID of the child that caused the signal to happen. In certain older systems (System V), you would keep getting the signal when the signal handler returned if you didn't. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] java/scripts/Tests All, , 1.1.1.1, 1.2
by Stefan Manegold 11 Dec '07

11 Dec '07

On Tue, Dec 11, 2007 at 01:36:50PM +0000, Fabian wrote: > Update of /cvsroot/monetdb/java/scripts/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22088 > > Modified Files: > All > Log Message: > Don't run an absolutely pointless test What is "pointless" in testing the minimal functionality of a tool that we build? Stefan > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/java/scripts/Tests/All,v > retrieving revision 1.1.1.1 > retrieving revision 1.2 > diff -u -d -r1.1.1.1 -r1.2 > --- All 28 Nov 2007 14:54:05 -0000 1.1.1.1 > +++ All 11 Dec 2007 13:36:48 -0000 1.2 > @@ -1 +1 @@ > -HAVE_JAVA?mjclient--help > +#HAVE_JAVA?mjclient--help > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 12

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/common monet_options.mx, , 1.20, 1.21 monet_utils.mx, , 1.33, 1.34
by Sjoerd Mullender 11 Dec '07

11 Dec '07

Fabian wrote: > Update of /cvsroot/monetdb/MonetDB/src/common > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26515 > > Modified Files: > monet_options.mx monet_utils.mx > Log Message: > GCC on FreeBSD chokes on #if without a comparison, so use #ifdef/#ifndef instead > > Index: monet_options.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/common/monet_options.mx,v > retrieving revision 1.20 > retrieving revision 1.21 > diff -u -d -r1.20 -r1.21 > --- monet_options.mx 29 Oct 2007 20:09:08 -0000 1.20 > +++ monet_options.mx 11 Dec 2007 12:30:25 -0000 1.21 > @@ -95,7 +95,7 @@ > #include <string.h> > #include <ctype.h> > > -#if HAVE_UNISTD_H > +#ifdef HAVE_UNISTD_H > #include <unistd.h> > #endif > > @@ -104,7 +104,7 @@ > # include "getopt1.c" > #endif > > -#if !HAVE_DECL_STRDUP > +#ifndef HAVE_DECL_STRDUP This is incorrect. Please revert. (And read the manual.) > #ifdef HAVE_STRDUP > extern char *strdup(const char *); > #else > > Index: monet_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/common/monet_utils.mx,v > retrieving revision 1.33 > retrieving revision 1.34 > diff -u -d -r1.33 -r1.34 > --- monet_utils.mx 3 Apr 2007 08:54:37 -0000 1.33 > +++ monet_utils.mx 11 Dec 2007 12:30:25 -0000 1.34 > @@ -49,7 +49,7 @@ > #include <stdarg.h> /* va_alist.. */ > > /* AIX requires this to be the first thing in the file. */ > -#if HAVE_ALLOCA_H > +#ifdef HAVE_ALLOCA_H > # include <alloca.h> > #else > # ifdef _AIX > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

[Monetdb-developers] MonetDB/XQuery on FreeBSD
by Djoerd Hiemstra 11 Dec '07

11 Dec '07

Here are some remarks from Tristan Pothoven here in Twente for installing the stable MonetDB/XQuery on FreeBSD (FreeBSD 6.2-RELEASE-p7). He did the following: 1. 'make' on FreeBSD cannot cope with the wildcards in some of the Makefiles. No problem if you use gmake instead. 2. Changes in sourcefiles: Package MonetDB: file: monet_utils.h original: #if HAVE_ALLOCA_H changed to: #ifdef HAVE_ALLOCA_H Package Pathfinder: file: pathfinder.h original: #if HAVE_ALLOCA_H changed to: #ifdef HAVE_ALLOCA_H 3. Furthermore, it seems that the files parser.tab.h and parser.tab.c, are not generated (are they generated?) correctly. Tristan took them from the tarball on the monetdb cwi site. Similarly, in the pftijah module with nexi.tab.c and nexi.tab.h. Best, Djoerd. -------- Original Message -------- Subject: Re: [IR] Vraag met betrekking tot PFTijah / MonetDB Date: Fri, 23 Nov 2007 17:06:05 +0100 From: Tristan Pothoven <t.pothoven(a)student.utwente.nl> To: Djoerd Hiemstra <hiemstra(a)cs.utwente.nl> CC: Vliet,W.M.van(stud-INF) <w.m.vanvliet(a)student.utwente.nl> Hallo Djoerd (en Marijn, de jongen van de grote xml), Over de installatie van de stable release (zoals op de dbapps site, onder CVS) op FreeBSD (FreeBSD 6.2-RELEASE-p7) heb ik het volgende te melden: Je moet per definitie gmake gebruiken ipv make. De make op FreeBSD kan niet overweg met de wildcards die in sommige Makefile's staan. Gmake kan dit wel. Wijzigingen in de sourcefiles: Package MonetDB: file: monet_utils.h origineel: #if HAVE_ALLOCA_H vervanging: #ifdef HAVE_ALLOCA_H Package Pathfinder: file: pathfinder.h origineel: #if HAVE_ALLOCA_H vervanging: #ifdef HAVE_ALLOCA_H Verder heb ik geen wijzigingen in de code moeten aanbrengen. Wel misten er nog wat files in de pathfinder module. Ergens in de parser (vergeef me dat ik de exacte locatie niet meer weet) missen de files parser.tab.h en parser.tab.c, deze heb ik gehaald uit de tarball beschikbaar op de monetdb cwi site. In de pftijah module missen nexi.tab.c en nexi.tab.h. (Op dezelfde manier achterhaald en gekopieerd). Ik geef toe, het was nachtwerk voor mij, dus mogelijke fouten die ik per ongeluk verzonnen heb: mijn excuses daarvoor, maar iig doet hij het nu. :) Groeten, Tristan

6 8

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, , 1.157, 1.158
by Stefan Manegold 11 Dec '07

11 Dec '07

Roberto, does this fix your problem? Stefan On Tue, Dec 11, 2007 at 10:08:13AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18823 > > Modified Files: > pftijah.mx > Log Message: > - add an optional extra commit after document indexing to survey a bug > where a server restart leads to a zero filled index bat. > > > > Index: pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.157 > retrieving revision 1.158 > diff -u -d -r1.157 -r1.158 > --- pftijah.mx 4 Oct 2007 10:41:12 -0000 1.157 > +++ pftijah.mx 11 Dec 2007 10:08:09 -0000 1.158 > @@ -315,7 +315,9 @@ > # # > ##################################################################### > > -var verbose := false; > +var verbose := false; > + > +var extra_commit := false; > > const dflt_ft_index := "DFLT_FT_INDEX"; > const dflt_bg_index := "DFLT_FT_INDEX"; > @@ -997,6 +999,7 @@ > _tj_add2collection(ftiName, collBat, uri_loc, uri_name, store); > _tj_finalize_collection(ftiName, collBat, FALSE); > _tj_commit(collBat); > + if ( extra_commit ) subcommit(_tj_collection_str(ftiName)); > if ( timing ) { > var ms := (usec()-t_start)/1000; > printf("#C[%s]:tj_add2collection(): + aggregate time = %lld.%03llds.\n",ftiName,/(ms,1000),%(ms,1000)); > @@ -1038,6 +1041,7 @@ > printf("#C[%s]:tj_add2collection(BAT): + aggregate time = %lld.%03llds.\n",ftiName,/(ms,1000),%(ms,1000)); > } > _tj_commit(collBat); > + if ( extra_commit ) subcommit(_tj_collection_str(ftiName)); > }); > lock_unset(coll_lock); > if (not(isnil(err))) ERROR(err); > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pf_support.mx, XQuery_0-20, 1.267.2.3, 1.267.2.4
by Lefteris 11 Dec '07

11 Dec '07

Join the Group, they said. See the Code, they said... On Dec 11, 2007 2:54 PM, Sjoerd Mullender <sjoerd(a)users.sourceforge.net> wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv30198 > > Modified Files: > Tag: XQuery_0-20 > pf_support.mx > Log Message: > Added an appropriate comment. > This comment actully comes from the display code of Gosling Emacs > ca. 1981, but is also very appropriate here. > > > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.267.2.3 > retrieving revision 1.267.2.4 > diff -u -d -r1.267.2.3 -r1.267.2.4 > --- pf_support.mx 7 Dec 2007 14:16:29 -0000 1.267.2.3 > +++ pf_support.mx 11 Dec 2007 13:54:20 -0000 1.267.2.4 > @@ -2060,6 +2060,53 @@ > @- update primitives > @mil > > +################################################################# > +# > +# > +# > +# /-------------\ > +# / \ > +# / \ > +# / \ > +# | XXXX XXXX | > +# | XXXX XXXX | > +# | XXX XXX | > +# \ X / > +# --\ XXX /-- > +# | | XXX | | > +# | | | | > +# | I I I I I I I | > +# | I I I I I I | > +# \ / > +# -- -- > +# \-------/ > +# XXX XXX > +# XXXXX XXXXX > +# XXXXXXXXX XXXXXXXXXX > +# XXXXX XXXXX > +# XXXXXXX > +# XXXXX XXXXX > +# XXXXXXXXX XXXXXXXXXX > +# XXXXX XXXXX > +# XXX XXX > +# > +# ************** > +# * BEWARE!! * > +# ************** > +# > +# All ye who enter here: > +# Most of the code in this module > +# is twisted beyond belief! > +# > +# Tread carefully. > +# > +# If you think you understand it, > +# You Don't, > +# So Look Again. > +# > +################################################################# > + > +# > # the UPDATE_INSERT_*, UPDATE_REPLACENODE and UPDATE_REPLACECONTENT commands MUST be > # consecutive, the UPDATE_INSERT_LAST and UPDATE_INSERT_BEFORE > # commands also MUST be consecutive > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins >

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet4 sql_result.mx, , 1.70, 1.71
by Fabian Groffen 09 Dec '07

09 Dec '07

On 09-12-2007 13:14:22 +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet4 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1301/src/backends/monet4 > > Modified Files: > sql_result.mx > Log Message: > output the last_id (the last inserted value in a auto_increment column) Shouldn't we upgrade the protocol version for this? If I "fix" JDBC to use it now, it will crash on a server not sending this extra value... > Index: sql_result.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet4/sql_result.mx,v > retrieving revision 1.70 > retrieving revision 1.71 > diff -u -d -r1.70 -r1.71 > --- sql_result.mx 3 Dec 2007 08:51:13 -0000 1.70 > +++ sql_result.mx 9 Dec 2007 13:14:20 -0000 1.71 > @@ -956,12 +956,12 @@ > */ > if (!s) return 0; > > -#ifdef NDEBUG > - (void) m; /* pacify compiler in case asserts are disabled */ > -#endif > assert(m->type == Q_UPDATE); > - if (stream_write(s, "&2 ", 3, 1) != 1 || !mvc_send_lng(s, val) || > - stream_write(s, "\n", 1, 1) != 1) > + if (stream_write(s, "&2 ", 3, 1) != 1 || > + !mvc_send_lng(s, val) || > + stream_write(s, " ", 1, 1) != 1 || > + !mvc_send_lng(s, m->last_id) || > + stream_write(s, "\n", 1, 1) != 1) > return -1; > if (mvc_export_warning(s, w) != 1) > return -1; > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins

2 3

[Monetdb-developers] 7. yum install does not work (John Baxter)
by John Baxter 06 Dec '07

06 Dec '07

3 8

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder configure.ag, , 1.122, 1.123
by Ying Zhang 06 Dec '07

06 Dec '07

On Dec 06, 2007, at 00:55 , Stefan Manegold wrote: > Update of /cvsroot/monetdb/pathfinder > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18880 > > Modified Files: > configure.ag > Log Message: > > Since Jennie 'Moved the XRPC Wrapper to the new "java" module', > Pathfinder does not require Java any more. So I still missed several files. Thanks for the final clean up! > > Index: configure.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/configure.ag,v > retrieving revision 1.122 > retrieving revision 1.123 > diff -u -d -r1.122 -r1.123 > --- configure.ag 3 Dec 2007 08:24:54 -0000 1.122 > +++ configure.ag 5 Dec 2007 23:55:05 -0000 1.123 > @@ -78,8 +78,6 @@ > AC_MSG_ERROR([Pathfinder requires a C99 compliant C compiler, but > your C compiler does not seem to be C99 compliant.]) > fi > > -req_java_ver_min=1.5 # >= $req_java_ver_min required > -AM_MONETDB_ANT_JAVA($req_java_ver_min) > req_monetdb_java_ver='1.7.0' > # ^^^^^^ > # Maintained via vertoo. Please don't modify by hand! > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: The Future of Linux Business White Paper > from Novell. From the desktop to the data center, Linux is going > mainstream. Let it simplify your IT future. > http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

1 0

[Monetdb-developers] yum install does not work
by John Baxter 05 Dec '07

05 Dec '07

hi, I'm trying to install monetdb from yum using the supplied repo files. It seems that the url with the repomd.xml file does not exist anymore. Can anyone tell me what url it should point to so i can amend my repo files please? Thank you John -- Co: Baxter Designs Web: http://www.baxterdesigns.co.uk Mail: john(a)baxterdesigns.co.uk Tel: 07968754215

2 1

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java version.sh, Clients_1-20, 1.10.2.6, 1.10.2.7
by Sjoerd Mullender 05 Dec '07

05 Dec '07

Fabian wrote: > Update of /cvsroot/monetdb/clients/src/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15410 > > Modified Files: > Tag: Clients_1-20 > version.sh > Log Message: > de-complicate. > > ( > actually: > - use shell globbing > - avoid needless fork/execs > - use bashisms for consistency > - fix "$[f}" > - don't do it difficult if it can be done simple > ) > > > > Index: version.sh > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/Attic/version.sh,v > retrieving revision 1.10.2.6 > retrieving revision 1.10.2.7 > diff -u -d -r1.10.2.6 -r1.10.2.7 > --- version.sh 5 Dec 2007 10:35:28 -0000 1.10.2.6 > +++ version.sh 5 Dec 2007 10:55:00 -0000 1.10.2.7 > @@ -112,15 +112,9 @@ > ../../../sql/src/jdbc/tests/Tests/Test.SQL.bat \ > ../../../sql/NT/MonetDB4-SQL/MonetDB4-SQL-Installer.vdproj \ > ../../../sql/NT/MonetDB5-SQL/MonetDB5-SQL-Installer.vdproj \ > - ../../../pathfinder/NT/MonetDB4-XQuery/MonetDB4-XQuery-Installer.vdproj \ > + ../../../"*"/NT/MonetDB4-XQuery/MonetDB4-XQuery-Installer.vdproj \ > ; do > - if [[ ! -f ${file} ]] ; then > - f="`echo ${file} | sed 's=\.\./$sql\|pathfinder$/=../*/='`" > - f="`ls ${f} 2>/dev/null`" > - if [[ -f $[f} ]] ; then > - file=${f} > - fi > - fi > + file=$(eval echo ${file}) This does not work if $file contains "funny" characters. It will also collapse multiple spaces in $file to a single space. > if [[ -f ${file} ]] ; then > for f in $FILES ; do > fr=${f//XXX/${NEW_MAJOR}.${NEW_MINOR}} > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: The Future of Linux Business White Paper > from Novell. From the desktop to the data center, Linux is going > mainstream. Let it simplify your IT future. > http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java version.sh, Clients_1-20, 1.10.2.6, 1.10.2.7
by Stefan Manegold 05 Dec '07

05 Dec '07

Fabian took care of applying these changes to java/version.sh; hence, I took care of preventing the automa[tgn]ic propagation. Stefan On Wed, Dec 05, 2007 at 10:55:02AM +0000, Fabian wrote: > Update of /cvsroot/monetdb/clients/src/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15410 > > Modified Files: > Tag: Clients_1-20 > version.sh > Log Message: > de-complicate. > > ( > actually: > - use shell globbing > - avoid needless fork/execs > - use bashisms for consistency > - fix "$[f}" > - don't do it difficult if it can be done simple > ) > > > > Index: version.sh > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/Attic/version.sh,v > retrieving revision 1.10.2.6 > retrieving revision 1.10.2.7 > diff -u -d -r1.10.2.6 -r1.10.2.7 > --- version.sh 5 Dec 2007 10:35:28 -0000 1.10.2.6 > +++ version.sh 5 Dec 2007 10:55:00 -0000 1.10.2.7 > @@ -112,15 +112,9 @@ > ../../../sql/src/jdbc/tests/Tests/Test.SQL.bat \ > ../../../sql/NT/MonetDB4-SQL/MonetDB4-SQL-Installer.vdproj \ > ../../../sql/NT/MonetDB5-SQL/MonetDB5-SQL-Installer.vdproj \ > - ../../../pathfinder/NT/MonetDB4-XQuery/MonetDB4-XQuery-Installer.vdproj \ > + ../../../"*"/NT/MonetDB4-XQuery/MonetDB4-XQuery-Installer.vdproj \ > ; do > - if [[ ! -f ${file} ]] ; then > - f="`echo ${file} | sed 's=\.\./$sql\|pathfinder$/=../*/='`" > - f="`ls ${f} 2>/dev/null`" > - if [[ -f $[f} ]] ; then > - file=${f} > - fi > - fi > + file=$(eval echo ${file}) > if [[ -f ${file} ]] ; then > for f in $FILES ; do > fr=${f//XXX/${NEW_MAJOR}.${NEW_MINOR}} > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: The Future of Linux Business White Paper > from Novell. From the desktop to the data center, Linux is going > mainstream. Let it simplify your IT future. > http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java version.sh, Clients_1-20, 1.10.2.2, 1.10.2.3
by Fabian Groffen 05 Dec '07

05 Dec '07

On 05-12-2007 09:53:05 +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/clients/src/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17263/clients/src/java > > Modified Files: > Tag: Clients_1-20 > version.sh > Log Message: > > make explicte, which files (also outside clients) > contain the version numbers of monetdb-*-mcl.jar & monetdb-*-jdbc.jar > and hence need to be updated when/after these version numbers changes. > (propagation to java is handled by hand.) > > for file in \ > Makefile.ag \ > - ../../../*/src/jdbc/tests/Tests/Test.SQL.bat \ > + ../../../sql/src/jdbc/tests/Tests/Test.SQL.bat \ > + ../../../sql/NT/MonetDB4-SQL/MonetDB4-SQL-Installer.vdproj \ > + ../../../sql/NT/MonetDB5-SQL/MonetDB5-SQL-Installer.vdproj \ > + ../..../pathfinder/NT/MonetDB4-XQuery/MonetDB4-XQuery-Installer.vdproj \ The problem with this is that it doesn't work for me, as my directory layout is different. However, I'm usually the one to bump the version numbers...

3 7

[Monetdb-developers] Questions about MIL....
by Romulo Goncalves 04 Dec '07

04 Dec '07

Hello, Firs question: b.select(0@0, oid(b.count() - 3)).mark(0(a)0).reverse() What is the meaning of such MIL code line? I translate it to MAL like this: cnt := aggr.count(base); cnt := cnt - 3; oid := calc.oid(cnt); sel := algebra.select(b, 0@0, oid); mrk := bat.markT(sel,0@0); c0 := bat.reverse(mrk); Is the order correct? Because I am getting the following type Error: !TypeException:user.str2qgrams[26]:'algebra.select' undefined in: sel:any := algebra.select(b:bat[:str,:void], _31:oid, oid:oid) Second question: Which function I should call for: b.reverse().seqbase(0@0); In MAL I did rev := bat.reverse(b); base := bat.setBase(rev,0@0); But the bat.setBase is comment in bat5.mal, so what function should I use? Third question: !SyntaxException:parseError: res := [+](c1, c2); !SyntaxException:parseError: ^operator expected How to resolve this error? I do not have any clue what is this about. Regards, Romulo

4 8

[Monetdb-developers] Mil to MAL....
by Romulo Goncalves 02 Dec '07

02 Dec '07

Dear developers, I need to convert some MIL to MAL; however, I never used MIL in my life. Is there any document which give some tips in how to do it? Or what should be my steps to convert MIL to MAL? Thanks in advance Regards, Romulo

1 0

Re: [Monetdb-developers] [Monetdb-checkins] java configure.ag, , 1.2, 1.3 rpm.mk.in, , 1.2, 1.3
by Stefan Manegold 29 Nov '07

29 Nov '07

On Thu, Nov 29, 2007 at 12:01:39PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/java > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28680 > > Modified Files: > configure.ag rpm.mk.in > Log Message: > Added more rpmrc files to the --rcfile flag of rpmbuild. This is > required for rpmbuild to recognize that in can build for a noarch CPU > architecture. > Removed references to host type from configure. This makes that the > configure which is called by rpmbuild doesn't choke on the noarch CPU > architecture. > > Together these two changes make that we can build the rpms. > > As a bonus: > Removed AC_DEFINE calls from configure. We don't actually include > monetdb_java_config.h anywhere, so we don't need to add anything extra > to it. Sjoerd said: "Just check it in so more people can have a look at it" THANKS! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] "MonetDB Java" extracted from "MonetDB Clients"
by Stefan Manegold 28 Nov '07

28 Nov '07

Dear fellow MonetDB developers, today, I separated MonetDB-related Java code from "MonetDB Clients" clients/src/java into its own new "MonetDB Java" CVS module java (obviously in the "Current" development trunk only). "MonetDB Java" compiles just like the remaining MonetDB-related code, and requires buildtools only when compiled from CVS source. "MonetDB Java" does not require any other MonetDB package. No other MonetDB package requires "MonetDB Java". Hence, "MonetDB Java" can be built independently of the other MonetDB packages. Background & Details: --------------------- To allow implementation of versioning-, packaging-, & release-schemes for MonetDB'\''s Java/JDBC Support & Interface that are decoupled from the respective schemes used for the (mainly C-based) remainder of the MonetDB product family, I implemented a move of the code from "clients/src/java/" into "java/", a new separate CVS module for "MonetDB Java". It comes with all the infrastructure of the usual MonetDB building process (stripped-down, though, to the limited requirements), including `./bootstrap`, `configure`, `make`, `make check`, `sh RunMtest`, `make dist`, `make rpm`, `make install`, & .`Mtest.py -rq`. Version number maintenance within "java/" is done exclusively by java/version.sh, no vertoo involved, here. To build, the new "MonetDB Java" requires only the buildtools (and that only when building straight from CVS). To run `make check` and/or `Mtest.py -rq`, Mtest.py from MonetDB Common must be available in the PATH, though. sql and pathfinder now check for the availability of MonetDB Java before running their Java/JDBC tests. On the live web site, the related Current-TestWeb pages have been updated. Updating the build documentation in the development web site still has to be done. Caveat: ------- `make rpm` requesting "noarch" does not work, yet. ToDo: ----- The "xrpcwrapper" Java code from pathfinder/src/tools/java/ should also be moved into the new "MonetDB Java" package. (Note, though, that "MonetDB Java" currently requires Java >= 1.4 & < 1.6, while "xrpcwrapper" requires Java >= 1.5 .) Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, , 1.196, 1.197
by Sjoerd Mullender 28 Nov '07

28 Nov '07

Please revert. This is not the solution. It is also *blatantly* untrue that nobody cares. We (at least Niels, Stefan, and I--we didn't discuss this with Peter) do care, but we want a proper solution, and this is not it. We agree that there should not be a call to umask in the code, but we should also check the modes that are used in calls to creat, open, mkdir, etc. If you think this umask thing is important for you, say so in the bug report and increase the priority accordingly. You can also *talk* to people if something is important to you. Fabian wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv7068 > > Modified Files: > gdk_utils.mx > Log Message: > Because nobody cares but me. > Don't wipe out the user's umask if gdk_dont_unset_umask is set in the > environment. Can't use GDKgetenv as that stuff is not yet initialised > at this point. > > > Index: gdk_utils.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v > retrieving revision 1.196 > retrieving revision 1.197 > diff -u -d -r1.196 -r1.197 > --- gdk_utils.mx 14 Nov 2007 13:22:59 -0000 1.196 > +++ gdk_utils.mx 28 Nov 2007 11:59:39 -0000 1.197 > @@ -1636,7 +1636,8 @@ > if (GDKrecovery && unlink(GDKLOCK) < 0) { > GDKfatal("GDKlockHome: unlock DB failed\n"); > } > - umask(0000); > + if (getenv("gdk_dont_unset_umask") == NULL) > + umask(0000); > if (MT_lockf(GDKLOCK, F_TLOCK, 4, 1) < 0) { > GDKlockFile = 0; > GDKfatal("GDKlockHome: Database lock '%s' denied\n", GDKLOCK); > > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: The Future of Linux Business White Paper > from Novell. From the desktop to the data center, Linux is going > mainstream. Let it simplify your IT future. > http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] geom/src/sql/Tests geom_surface.stable.err, , 1.2, 1.3 geom_surface.stable.err.Five, , 1.1, 1.2 spatial.stable.err, , 1.2, 1.3 spatial.stable.err.Five, , 1.1, 1.2 spatial.stable.out, , 1.2, 1.3
by Fabian Groffen 27 Nov '07

27 Nov '07

On 27-11-2007 13:39:32 +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/geom/src/sql/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24450/src/sql/Tests > > Modified Files: > geom_surface.stable.err geom_surface.stable.err.Five > spatial.stable.err spatial.stable.err.Five spatial.stable.out > Log Message: > > "down"dated output from goes 3.0.0 to geos 2.2.3. > The latter seems to be working just fine, and is no available as RPM > on our Fedora testing machines (cluster & crux-farm); > and will hopefully soon also be available on our desktops... Hmmm... I have 3.0 installed (not using it really...) but doesn't this approval mean configure should require 2.2 then?

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 merovingian.mx, , 1.9, 1.10
by Stefan Manegold 25 Nov '07

25 Nov '07

On Sun, Nov 25, 2007 at 10:18:26AM +0000, Fabian wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26685 > > Modified Files: > merovingian.mx > Log Message: > Attempt to fix some compiler warnings. I cannot test myself, as I don't > get them, and if I use TestWeb's CFLAGS the code doesn't compile at all > at a much earlier stage. "TestWeb" does not use any (particular) CFLAGS, but just the default that (our) configure uses/provides (with --enable-strict, which is default for the development trunk). In case you have problems with this (or other) settings, please help us fixing them by reporting them ans bug report. The compiler warning is triggered by the following setting in lines 742,3 of buildtools/conf/MonetDB.m4 ======== dnl The default configure invocation when doing an rpmbuild also uses this X_CFLAGS="$X_CFLAGS -Wp,-D_FORTIFY_SOURCE=2" ======== Some functions (system calls) do return, whether they were successful, or not, and the calling ocde should check the respective return values. Ignoring them via (e.g.) (void)write(...); will (most probably) not help. Rather, the code should check the results, e.g., if (write(...) < 0) { ... } or at least pretend to do so, e.g., ssize_t wrt_res = 0; wrt_res = write(...); (void)wrt_res; /* deliberately irgnore portential errors */ Stefan > Index: merovingian.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/merovingian.mx,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- merovingian.mx 25 Nov 2007 09:55:05 -0000 1.9 > +++ merovingian.mx 25 Nov 2007 10:18:24 -0000 1.10 > @@ -1203,10 +1203,10 @@ > case -1: > /* oops, forking went wrong, cheat a bit in order to write to > * the console */ > - write(oerr, "unable to fork into background: ", > + (void)write(oerr, "unable to fork into background: ", > sizeof("unable to fork into background: ")); > - write(oerr, strerror(errno), strlen(strerror(errno))); > - write(oerr, "\n", 1); > + (void)write(oerr, strerror(errno), strlen(strerror(errno))); > + (void)write(oerr, "\n", 1); > return(1); > case 0: > /* the child, we want to keep it, try to lock, we can only > @@ -1217,17 +1217,17 @@ > * remain available, otherwise we loose the lock */ > if ((ret = MT_lockf(buf, F_TLOCK, 4, 1)) == -1) { > /* locking failed */ > - write(oerr, "another merovingian is already running\n", > + (void)write(oerr, "another merovingian is already running\n", > sizeof("another merovingian is already running\n")); > exit(1); > } else if (ret == -2) { > /* directory or something doesn't exist */ > - write(oerr, "unable to create .merovingian_lock file in ", > + (void)write(oerr, "unable to create .merovingian_lock file in ", > sizeof("unable to create .merovingian_lock file in ")); > - write(oerr, dbfarm, strlen(dbfarm)); > - write(oerr, ": ", 2); > - write(oerr, strerror(errno), strlen(strerror(errno))); > - write(oerr, "\n", 1); > + (void)write(oerr, dbfarm, strlen(dbfarm)); > + (void)write(oerr, ": ", 2); > + (void)write(oerr, strerror(errno), strlen(strerror(errno))); > + (void)write(oerr, "\n", 1); > exit(1); > } > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] New MonetDB Bug Fix Release
by Sjoerd Mullender 24 Nov '07

24 Nov '07

The MonetDB team at CWI is pleased to announce a new bug fix release of the MonetDB suite of programs. The following packages have been released: MonetDB Clients 1.20.2 MonetDB5 Server 5.2.2 MonetDB SQL 2.20.2 There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. See http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Version4.20/ind… for a list of fixes. -- Sjoerd Mullender

1 0

[Monetdb-developers] Start studying MonetDB code
by Marco Antonelli 23 Nov '07

23 Nov '07

Hello everyone, I am an Italian student, and I will work at MonetDB/SPARQL for my master thesis starting from next January, in Amsterdam. Prof. Martin Kersten told to have a look at the code, in particular the SQL->MAL translator. So I downloaded the code, and managed to compile and install it correctly (on Linux). Then I opened the code in Eclipse (do you use it too? do you use some better C IDE?) and started to look at MonetDB5-server. Prof. Kersten told not to worry... but that's impressive! So where do you suggest me to start? Which c files and functions first? Thanks in advance and kind regards, Marco Antonelli

2 2

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/conf MonetDB.m4, MonetDB_1-20, 1.37.2.14, 1.37.2.15
by Fabian Groffen 21 Nov '07

21 Nov '07

On 21-11-2007 15:07:04 +0000, Stefan Manegold wrote: > +case "$host-$GCC-$CC-$enableval" in > +ia64*-*-*-32) > + AC_MSG_ERROR([we do not support 32 bits on $host]);; So HPUX's 32-bits support on ia64 is not supported :( > +i?86*-*-*-64) > + AC_MSG_ERROR([$host does not support 64 bits]);; So Darwin's multilib compiler (i386-apple-darwin[89]) is assumed not to handle 64-bits code here :( > +*--icc*-32|*--icc*-64) > + AC_MSG_ERROR([icc does not support a command line option to switch between 32- & 64-bit compilation; use CC & PATH (instead of --enable-bits=$enableval) to point to the $enableval-bit version of icc]);; > +*-*-*-32|*-*-*-64) > ;; > -*) AC_MSG_ERROR(--enable-bits argument must be either 32 or 64);; > +*) > + AC_MSG_ERROR(--enable-bits argument must be either 32 or 64);; Clever one ;)

2 3

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/mapi/Tests All, SQL_2-20, 1.1.2.6, 1.1.2.7
by Stefan Manegold 21 Nov '07

21 Nov '07

On Wed, Nov 21, 2007 at 11:07:47AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/test/mapi/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv25856 > > Modified Files: > Tag: SQL_2-20 > All > Log Message: > disabled python_dbapi_embedded as fixing this problem requires an embedded > api change (ie we should pass more configuration options, ie monetdb_mod_path > too find sql_server/sql in weird places) > > !!! this should not be forwarded to the current branch !!! propagation of this change has been prevented. Stefan > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/test/mapi/Tests/All,v > retrieving revision 1.1.2.6 > retrieving revision 1.1.2.7 > diff -u -d -r1.1.2.6 -r1.1.2.7 > --- All 22 Oct 2007 19:58:08 -0000 1.1.2.6 > +++ All 21 Nov 2007 11:07:44 -0000 1.1.2.7 > @@ -8,6 +8,6 @@ > mclient-lsql--help > HAVE_PERL_SWIG?perl_dbi > HAVE_PYTHON_SWIG?python_dbapi > -HAVE_PYTHON_SWIG?python_dbapi_embedded > +#HAVE_PYTHON_SWIG?python_dbapi_embedded > HAVE_PHP?php_monetdb > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/BugTracker/Tests auto_increment.SF-1834820.sql, SQL_2-20, NONE, 1.1.2.1
by Stefan Manegold 20 Nov '07

20 Nov '07

Please note that this new test will not be executed, unless also added to the repstective "All" file. Stefan On Tue, Nov 20, 2007 at 06:50:31AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/sql/src/test/BugTracker/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29561 > > Added Files: > Tag: SQL_2-20 > auto_increment.SF-1834820.sql > Log Message: > Added the test file. Output needs to be produced. > > > --- NEW FILE: auto_increment.SF-1834820.sql --- > CREATE TABLE testusers ( > id INT NOT NULL AUTO_INCREMENT, > name VARCHAR(40), > fullname VARCHAR(100), > PRIMARY KEY (id) > ); > > > INSERT INTO testusers (id, name, fullname) VALUES (1, 'wendy', 'Wendy > Wones'); > select * from testusers; > > -- This will fail the first time > INSERT INTO testusers (name, fullname) VALUES ('fred', 'Fred Flintstone'); > select * from testusers; > > -- Now it will succeed > INSERT INTO testusers (name, fullname) VALUES ('fred', 'Fred Flintstone'); > select * from testusers; > > drop table testusers; > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] bat.new undefined
by darabi＠web.de 15 Nov '07

15 Nov '07

Hello, I am trying to make a small function for creating some persistent bats. Could you point me to my (probably stupid) error? Many thanks Kambiz $ mclient --language=mal -d demo mal> mal>function init_schema():void; mal> mal> bbp.bind("schema_version"); mal> mal> # create schema, if it doesn't exist mal> catch MALException:str; mal> mal> s := bat.new(:void,:oid); mal> bat.setName(s, "schema"); mal> bat.setPersistent(s); mal> mal> s := bat.new(:void,:void); mal> bat.setName(s, "version"); mal> bat.setPersistent(s); mal> mal> sv := bat.new(:oid,:int); mal> bat.setName(sv, "schema_version"); mal> bat.setPersistent(sv); mal> mal> cl := bat.new(:oid,:oid); mal> bat.setName(sv, "schema_class"); mal> bat.setPersistent(sv); mal> mal># etc. etc. etc. mal> mal> transaction.commit(); mal> mal> exit; MAPI = monetdb@localhost:50000 QUERY = exit; ERROR = !TypeException:user.init_schema[7]:'bat.new' undefined in: s:bat[:void,:oid] := bat.new(_6:void, _6:void) mal> MAPI = monetdb@localhost:50000 QUERY = ERROR = !TypeException:user.init_schema[7]:'bat.new' undefined in: s:bat[:void,:oid] := bat.new(_6:void, _6:void) !SyntaxException:user.init_schema[18]:begin 'MALException' without exit in init_schema[19] mal> io.print("Schema tables seem to exist. Will do nothing\n"); MAPI = monetdb@localhost:50000 QUERY = io.print("Schema tables seem to exist. Will do nothing\n"); ERROR = !TypeException:user.init_schema[7]:'bat.new' undefined in: s:bat[:void,:oid] := bat.new(_6:void, _6:void) !SyntaxException:user.init_schema[19]:begin 'MALException' without exit in init_schema[20] mal>end init_schema; MAPI = monetdb@localhost:50000 QUERY = end init_schema; ERROR = !TypeException:user.init_schema[7]:'bat.new' undefined in: s:bat[:void,:oid] := bat.new(_6:void, _6:void) !SyntaxException:user.init_schema[18]:begin 'MALException' without exit in init_schema[19] mal>

3 5