- developers-list - monetdb.org

Buffer Manager - MonetDB
by Baskar Duraikannu 20 Jun '13

20 Jun '13

Good morning. I recently installed and tested monetdb with a sample dataset and it performed really well. Then I came across the below posthttp://stackoverflow.com/questions/7435235/is-it-worth-trying-monetdb Above post talks about non-availability of buffer manager in MonetDB. Has this changed?

2 1

how to speed up the loading of data into a table
by Bouslah Ahmed 18 Jun '13

18 Jun '13

Good afternoon, I have 2 questions : - 1st question I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows :? mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl ?for information I use : MonetDB5 Windows Memory : 16 Go There are two constraints : primary key and foreign key on the table prior to loading data? 2nd question : how inner joins are made in MongoDB? cordialy.

3 4

Re: MonetDB: DVframework_bam - Added bam folder under sql/backends/m...
by Stefan Manegold 15 Jun '13

15 Jun '13

Thanks Robin! Eventually, we should of course add a proper configure check and make conditional for the samtools ... ;-) Stefan Robin Cijvat <commits(a)monetdb.org> wrote: >Changeset: d87b8aafa5b7 for MonetDB >URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d87b8aafa5b7 >Added Files: > sql/backends/monet5/bam/85_bam.mal > sql/backends/monet5/bam/85_bam.sql > sql/backends/monet5/bam/Makefile.ag > sql/backends/monet5/bam/bam.mal > sql/backends/monet5/bam/bam_clear.sql > sql/backends/monet5/bam/bam_schema.sql > sql/backends/monet5/bam/bamloader.c > sql/backends/monet5/bam/bamloader.h >Modified Files: > sql/backends/monet5/Makefile.ag >Branch: DVframework_bam >Log Message: > >Added bam folder under sql/backends/monet5/ containing UDF that >provides functionality for loading BAM file contents into temporary >BATs. To run code, samtools has to be installed and referred to in the >Makefile of the directory. > > >diffs (truncated from 1260 to 300 lines): > >diff --git a/sql/backends/monet5/Makefile.ag >b/sql/backends/monet5/Makefile.ag >--- a/sql/backends/monet5/Makefile.ag >+++ b/sql/backends/monet5/Makefile.ag >@@ -15,7 +15,7 @@ > # Copyright August 2008-2013 MonetDB B.V. > # All Rights Reserved. > >-SUBDIRS = NOT_WIN32?vaults UDF LSST ENABLE_DATACELL?datacell >HAVE_MSEED?miniseed >+SUBDIRS = NOT_WIN32?vaults UDF bam LSST ENABLE_DATACELL?datacell >HAVE_MSEED?miniseed > > INCLUDES = ../../include ../../common ../../storage ../../server \ > ../../../monetdb5/modules/atoms \ >diff --git a/sql/backends/monet5/bam/85_bam.mal >b/sql/backends/monet5/bam/85_bam.mal >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/85_bam.mal >@@ -0,0 +1,2 @@ >+library bam; >+include bam; >\ No newline at end of file >diff --git a/sql/backends/monet5/bam/85_bam.sql >b/sql/backends/monet5/bam/85_bam.sql >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/85_bam.sql >@@ -0,0 +1,2 @@ >+CREATE PROCEDURE bamloader(repo string, mode int, num_threads int) >+external name bam.bamloader; >\ No newline at end of file >diff --git a/sql/backends/monet5/bam/Makefile.ag >b/sql/backends/monet5/bam/Makefile.ag >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/Makefile.ag >@@ -0,0 +1,62 @@ >+# The contents of this file are subject to the MonetDB Public License >+# Version 1.1 (the "License"); you may not use this file except in >+# compliance with the License. You may obtain a copy of the License at >+# http://www.monetdb.org/Legal/MonetDBLicense >+# >+# Software distributed under the License is distributed on an "AS IS" >+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See >the >+# License for the specific language governing rights and limitations >+# under the License. >+# >+# The Original Code is the MonetDB Database System. >+# >+# The Initial Developer of the Original Code is CWI. >+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI. >+# Copyright August 2008-2013 MonetDB B.V. >+# All Rights Reserved. >+ >+INCLUDES = .. \ >+ ../../../include \ >+ ../../../common \ >+ ../../../storage \ >+ ../../../server \ >+ ../../../../monetdb5/modules/atoms \ >+ ../../../../monetdb5/modules/kernel \ >+ ../../../../monetdb5/mal \ >+ ../../../../monetdb5/modules/mal \ >+ ../../../../monetdb5/optimizer \ >+ ../../../../clients/mapilib \ >+ ../../../../common/options \ >+ ../../../../common/stream \ >+ ../../../../gdk \ >+ -I$(HOME)/Documents/BAM/samtools/include >+ >+lib__bam = { >+ MODULE >+ DIR = libdir/monetdb5 >+ SOURCES = bamloader.c bamloader.h >+ LIBS = ../../../../monetdb5/tools/libmonetdb5 \ >+ ../../../../gdk/libbat \ >+ -L$(HOME)/Documents/BAM/samtools/lib -lbam >+ >+} >+ >+headers_mal = { >+ HEADERS = mal >+ DIR = libdir/monetdb5 >+ SOURCES = bam.mal >+} >+ >+headers_sql = { >+ HEADERS = sql >+ DIR = libdir/monetdb5/createdb >+ SOURCES = 85_bam.sql >+} >+ >+headers_autoload = { >+ HEADERS = mal >+ DIR = libdir/monetdb5/autoload >+ SOURCES = 85_bam.mal >+} >+ >+EXTRA_DIST_DIR = Tests >diff --git a/sql/backends/monet5/bam/bam.mal >b/sql/backends/monet5/bam/bam.mal >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/bam.mal >@@ -0,0 +1,5 @@ >+module bam; >+ >+pattern bamloader(entry:str, mode:int, num_threads:int):void >+address bamloader >+comment "Read the files in the BAM repository, fill and return a >temp_container accordingly."; >\ No newline at end of file >diff --git a/sql/backends/monet5/bam/bam_clear.sql >b/sql/backends/monet5/bam/bam_clear.sql >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/bam_clear.sql >@@ -0,0 +1,7 @@ >+DROP TABLE bam.alignments_extra; >+DROP TABLE bam.alignments; >+DROP TABLE bam.pg; >+DROP TABLE bam.rg; >+DROP TABLE bam.sq; >+DROP TABLE bam.files; >+DROP SCHEMA bam; >diff --git a/sql/backends/monet5/bam/bam_schema.sql >b/sql/backends/monet5/bam/bam_schema.sql >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/bam_schema.sql >@@ -0,0 +1,91 @@ >+CREATE SCHEMA bam; >+ >+CREATE TABLE "bam"."files" ( >+ "file_location" STRING NOT NULL UNIQUE, >+ "format_version" REAL, >+ "sorting_order" VARCHAR(10), >+ "comments" STRING, >+ CONSTRAINT "files_pkey_file_location" PRIMARY KEY (file_location) >+); >+ >+CREATE TABLE "bam"."sq" ( >+ "sn" STRING NOT NULL, >+ "file_location" STRING NOT NULL, >+ "ln" INT NOT NULL, >+ "as" INT, >+ "m5" STRING, >+ "sp" STRING, >+ "ur" STRING, >+ CONSTRAINT "sq_pkey_sn_file_location" PRIMARY KEY (sn, >file_location), >+ CONSTRAINT "sq_fkey_file_location" FOREIGN KEY (file_location) >REFERENCES bam.files (file_location) >+); >+ >+CREATE TABLE "bam"."rg" ( >+ "id" STRING NOT NULL UNIQUE, >+ "file_location" STRING NOT NULL, >+ "cn" STRING, >+ "ds" STRING, >+ "dt" TIMESTAMP, >+ "fo" STRING, >+ "ks" STRING, >+ "lb" STRING, >+ "pg" STRING, >+ "pi" INT, >+ "PL" STRING, >+ "PU" STRING, >+ "SM" STRING, >+ CONSTRAINT "rg_pkey_id_file_location" PRIMARY KEY (id, >file_location), >+ CONSTRAINT "rg_fkey_file_location" FOREIGN KEY (file_location) >REFERENCES bam.files (file_location) >+); >+ >+CREATE TABLE "bam"."pg" ( >+ "id" STRING NOT NULL UNIQUE, >+ "file_location" STRING NOT NULL, >+ "pn" STRING, >+ "cl" STRING, >+ "pp" STRING, >+ "vn" REAL, >+ CONSTRAINT "pg_pkey_id_file_location" PRIMARY KEY (id, >file_location), >+ CONSTRAINT "pg_fkey_file_location" FOREIGN KEY (file_location) >REFERENCES bam.files (file_location) >+); >+ >+CREATE TABLE "bam"."alignments" ( >+ "file_location" STRING NOT NULL, >+ "virtual_offset" INT NOT NULL, >+ "qname" STRING NOT NULL, >+ "flag_temp_mult_segm" BOOLEAN NOT NULL, >+ "flag_each_segm_prop_alig" BOOLEAN NOT NULL, >+ "flag_segm_unma" BOOLEAN NOT NULL, >+ "flag_next_segm_unma" BOOLEAN NOT NULL, >+ "flag_seq_reve_comp" BOOLEAN NOT NULL, >+ "flag_seq_next_segm_reve" BOOLEAN NOT NULL, >+ "flag_first_segm" BOOLEAN NOT NULL, >+ "flag_last_segm" BOOLEAN NOT NULL, >+ "flag_seco_alig" BOOLEAN NOT NULL, >+ "flag_not_pass_qual_cont" BOOLEAN NOT NULL, >+ "flag_pcr_opti_dupl" BOOLEAN NOT NULL, >+ "rname" STRING NOT NULL, >+ "pos" INT NOT NULL, >+ "mapq" INT NOT NULL, >+ "cigar" STRING NOT NULL, >+ "rnext" STRING NOT NULL, >+ "pnext" INT NOT NULL, >+ "tlen" INT NOT NULL, >+ "seq" STRING NOT NULL, >+ "qual" STRING NOT NULL, >+ CONSTRAINT "alignments_pkey_file_location_virtual_offset" PRIMARY >KEY (file_location, virtual_offset), >+ CONSTRAINT "alignments_fkey_file_location" FOREIGN KEY >(file_location) REFERENCES bam.files (file_location) >+); >+ >+CREATE TABLE "bam"."alignments_extra" ( >+ "tag" CHAR(2) NOT NULL, >+ "file_location" STRING NOT NULL, >+ "virtual_offset" INT NOT NULL, >+ "type" CHAR(1) NOT NULL, >+ "value" STRING, >+ CONSTRAINT >"alignments_extra_pkey_tag_file_location_virtual_offset" PRIMARY KEY >(tag, file_location, virtual_offset), >+ CONSTRAINT "alignments_extra_fkey_file_location_virtual_offset" >FOREIGN KEY (file_location, virtual_offset) REFERENCES bam.alignments >(file_location, virtual_offset) >+); >+ >+ >+ >diff --git a/sql/backends/monet5/bam/bamloader.c >b/sql/backends/monet5/bam/bamloader.c >new file mode 100644 >--- /dev/null >+++ b/sql/backends/monet5/bam/bamloader.c >@@ -0,0 +1,1013 @@ >+#include <stdio.h> >+#include "monetdb_config.h" >+#include "bamloader.h" >+ >+ >+/* >+ * NOTE: Copied directly from miniseed/registrar.c >+ * TODO: Make both miniseed/registrar.c and this file include some >generic library for these kind of structures >+ * keeps BAT and other properties of columns of a table. >+ */ >+typedef struct { >+ bat *column_bats; /* keeps bats of the columns: lower array */ >+ str *column_names; /* names of columns that are kept in the higher >array */ >+ str *column_types_strs; /* type strings of columns */ >+} temp_subcontainer; >+ >+/* >+ * NOTE: Copied directly from miniseed/registrar.c >+ * TODO: Make both miniseed/registrar.c and this file include some >generic library for these kind of structures >+ * keeps (some) tables of a schema. >+ */ >+typedef struct { >+ str schema_name; /* schema or vault name */ >+ temp_subcontainer *tables_columns; /* keeps tables: higher array */ >+ str *table_names; /* names of tables that are kept in the higher >array */ >+ int *num_columns; /* number of columns in each table in the higher >array */ >+ int num_tables; >+} temp_container; >+ >+/* >+* File format specific structures >+*/ >+typedef struct { >+ str tag; >+ str value; >+} bam_header_option; >+ >+typedef struct { >+ str header_tag; >+ bam_header_option *options; >+ int num_options; >+} bam_header_line; >+ >+ >+/* Global vars */ >+FILE *logfile = NULL; //keep logfile file opened in this global var >while the bam code runs, >+ //since opening and closing every time >something has to be written turned out to be very slow >+ >+ >+ >+/* File format specific functions */ >+str init_temp_container(temp_container *ret_tc); >+str loadfile(str filepath, temp_container *ret_tc); //load file and >add contents to tc >+str process_bam_header(str filepath, str header, temp_container >*ret_tc); >+int append_to_bat_cond(temp_container *ret_tc, bam_header_option *opt, >str cmp, int table, int col, int *appendErr, int *flag); >+str read_bam_header_line(str *header, bam_header_line *ret_hl, int >*eof); >+void free_bam_header_line(bam_header_line *hl); >+str process_bam_alignment(str filepath, int virtual_offset, >bam_header_t *header, bam1_t *alignment, temp_container *ret_tc); >+int parse_alignment_str(str *sam_alig, str *dest); >+int parse_alignment_long(str *sam_alig, long int *dest); >+ >+/* Generic functions */ >+str init_temp_subcontainer(temp_subcontainer *ret_tsc, >+ str *col_names, str *col_types_strs, int *col_types, int >num_cols); >+str append_to_bat(bat cb, ptr val); >+int concatenate_strs(str* words_to_concat, int num_words_to_concat, >str* ret_concatenated); >+str prepare_insertion(Client cntxt, temp_container* tc); >+str insert_into_vault(Client cntxt, temp_container* tc); >+str register_table(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr >pci); >+int read_string_until_delim(str *src, str *ret, char *delims, int >num_delims); >+int parse_long(str *src, long int *i); >+int get_kth_bit(unsigned int i, int k); >+void append_to_log(str mssg); >+void free_temp_container(temp_container* tc); >+ >+/* External functions */ >+str SQLstatementIntern(Client c, str *expr, str nme, int execute, bit >output); >+ >+ >+ >+/* File format specific functions */ >_______________________________________________ >checkin-list mailing list >checkin-list(a)monetdb.org >http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | Database Architectures (DA) | | www.CWI.nl/~manegold | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Re: MonetDB: sciql - for now, disbale imprints in sciql branch,
by Ying Zhang 14 Jun '13

14 Jun '13

Hai Stefan, What problems is imprints causing in the sciql branch? Jennie On Jun 14, 2013, at 10:29, Stefan Manegold wrote: > Changeset: 3841d3bd5fd7 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=3841d3bd5fd7 > Modified Files: > gdk/gdk_select.c > Branch: sciql > Log Message: > > for now, disbale imprints in sciql branch, > until we find the time to debug suspicious errors > > > diffs (12 lines): > > diff --git a/gdk/gdk_select.c b/gdk/gdk_select.c > --- a/gdk/gdk_select.c > +++ b/gdk/gdk_select.c > @@ -1344,7 +1344,7 @@ BATsubselect(BAT *b, BAT *s, const void > * ii) it is not an equi-select, and > * iii) is not var-sized. > */ > - use_imprints = 1; > + use_imprints = 0; /* temporarily disabled in sciql branch */ > } > bn = BAT_scanselect(b, s, bn, tl, th, li, hi, equi, anti, > lval, hval, maximum, use_imprints); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

2 1

System infomation!//Re: After plenty of querys the database use 40 times more disk space than the original data, How can I avoid this?
by liyuesen 00108910 06 Jun '13

06 Jun '13

Hi Stefan: Thanks for your help! Here is the system infomation: [root@host-28-for-monet monetdb]# mserver5 --version MonetDB 5 server v11.15.3 "Feb2013-SP1" (64-bit, 64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 94.4GiB available memory, 24 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 1.0.0 29 Mar 2010 (compiled with OpenSSL 1.0.0-fips 29 Mar 2010) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: root@host-28-for-monet (x86_64-unknown-linux-gnu) Compilation: gcc -g//When testing , debug was disabled. --by Jason Linking : /usr/bin/ld -m elf_x86_64 OS:Linux host-28-for-monet 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux CPU: 24 cores INTEL cpu Memory info, in Million Bytes: total used free shared buffers cached Mem: 99030412 3717976 95312436 0 658496 250616 -/+ buffers/cache: 2808864 96221548 Swap: 101269496 23768 101245728 No extra schema was used, we have seven tables in schema SYS, 155 colums of each table. Every miniute, 30 thousands rows were imported into each table, and then 40 threads execute the followwing SQL. The case run more than 90 times, that means the test last longer than one and an half hours! SQL query:select count(*), bkpi.GGSN_ID, bkpi.SGSN_ID, CASE WHEN CAST(SUM(bkpi.DNS_DELAY_NUM) AS BIGINT) = 0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.DNS_DELAY_NUM) AS BIGINT))/CAST(SUM(bkpi.DNS_DELAY_NUM) AS BIGINT) * 100 END as KQIDELAY, CASE WHEN CAST(SUM(bkpi.COUNTS) AS BIGINT)=0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.DNS_DELAY_MSEL) AS BIGINT))/CAST(SUM(bkpi.COUNTS) AS BIGINT) END as DNS_DELAY, CASE WHEN CAST(SUM(bkpi.TCP_DELAY_NUM) AS BIGINT) = 0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.TCP_DELAY_MSEL) AS BIGINT))/CAST(SUM(bkpi.TCP_DELAY_NUM) AS BIGINT) * 100 END as TCPDELAY , CASE WHEN CAST(SUM(bkpi.GET_DELAY_NUM) AS BIGINT)=0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.GET_DELAY_MSEL) AS BIGINT))/CAST(SUM(bkpi.GET_DELAY_NUM) AS BIGINT) END as TCPGETDELAY, CASE WHEN CAST(SUM(bkpi.TCP_DELAY_NUM) AS BIGINT) = 0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.TCP_DELAY_MSEL) AS BIGINT))/CAST(S UM(bkpi.TCP_DELAY_NUM) AS BIGINT) * 100 END AS TCP_DELAY_AVGMSEL, CASE WHEN (CAST(SUM(bkpi.POST_DELAY_NUM) AS BIGINT)*1024)=0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.POST_DELAY_MSEL) AS BIGINT)*1000*8)/(CAST(SUM(bkpi.POST_DELAY_NUM) AS BIGINT)*1024) END as TCPPOSTDELAY, CASE WHEN CAST(SUM(bkpi.GET_DELAY_NUM) AS BIGINT)=0 THEN -1 ELSE FLOOR(CAST(SUM(bkpi.GET_DELAY_MSEL) AS BIGINT))/CAST(SUM(bkpi.GET_DELAY_NUM) AS BIGINT) END AS GET_DELAY_MSE L_AVG from (select GENTIME, APN, GGSN_ID,SGSN_ID, COUNTS, KQI_DELAY_MSEL,DNS_DELAY_MSEL, DNS_DELAY_NUM, TCP_DELAY_MSEL,TCP_DELAY_NUM, GET_DELAY_MSEL,GET_DELAY_NUM,POST_DELAY_MSEL,POST_DELAY_NUM from bkpi_web_dspdelay_cisai_15min_0 where sgsn_id = 0 and GENTIME >= '2013-06-06 07:51:41' and GENTIME <= '2013-06-06 07:52:27' union all select GENTIME, APN, GGSN_ID,SGSN_ID, COUNTS, KQI_DELAY_MSEL,DNS_DELAY_MSEL, DNS_DELAY_NUM, TCP_ DELAY_MSEL,TCP_DELAY_NUM, GET_DELAY_MSEL,GET_DELAY_NUM,POST_DELAY_MSEL,POST_DELAY_NUM from bkpi_web_dspdelay_cisai_15min_1 where ggsn_id = 0 and GENTIME >= '2013-06-06 07:51:41' and GENTIME <= '2013-06-06 07:52:27' union all select GENTIME, APN, GGSN_ID,SGSN_ID, COUNTS, KQI_DELAY_MSEL,DNS_DELAY_MSEL, DNS_DELAY_NUM, TCP_DELAY_MSEL,TCP_DELAY_NUM, GET_DELAY_MSEL,GET_DELAY_NUM,POST_DELAY_MSEL,POST_DELAY_NUM from bkpi_web_dspdelay_ cisai_15min_0 where sgsn_id = 0 and GENTIME >= '2013-06-06 07:51:41' and GENTIME <= '2013-06-06 07:52:27' union all select GENTIME, APN, GGSN_ID,SGSN_ID, COUNTS, KQI_DELAY_MSEL,DNS_DELAY_MSEL, DNS_DELAY_NUM, TCP_DELAY_MSEL,TCP_DELAY_NUM, GET_DELAY_MSEL,GET_DELAY_NUM,POST_DELAY_MSEL,POST_DELAY_NUM from bkpi_web_dspdelay_cisai_15min_1 where ggsn_id = 0 and GENTIME >= '2013-06-06 07:51:41' and GENTIME <= '2013-06-06 07:52:27' ) b kpi group by bkpi.GGSN_ID, bkpi.SGSN_ID; Explain result of the query: +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mal | +=============================================================================================================================================================================================================+ | function user.s1_4{autoCommit=true}(A0:lng,A1:flt,A2:flt,A3:lng,A4:flt,A5:lng,A6:flt,A7:flt,A8:lng,A9:flt,A10:lng,A11:flt,A12:flt,A13:lng,A14:lng,A15:flt,A16:lng,A17:lng,A18:lng,A19:lng,A20:flt,A21:lng,A | : 22:str,A23:str,A24:lng,A25:str,A26:str,A27:lng,A28:str,A29:str,A30:lng,A31:str,A32:str):void; : | X_36:bat[:oid,:wrd] := bat.new(nil:oid,nil:wrd); | | X_39:bat[:oid,:wrd] := bat.new(nil:oid,nil:wrd); | | X_40:bat[:oid,:wrd] := bat.new(nil:oid,nil:wrd); | | X_35 := sql.mvc(); | | X_44 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","gentime",0); | | X_55 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","sgsn_id",0); | | X_41:bat[:oid,:oid] := sql.tid(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0"); | | X_962 := algebra.subselect(X_55,X_41,A21,A21,true,true,false); | | (X_57,r1_63) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","sgsn_id",2); | | X_963 := algebra.subselect(r1_63,A21,A21,true,true,false); | | X_59 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","sgsn_id",1); | | X_964 := algebra.subselect(X_59,X_41,A21,A21,true,true,false); | | X_60 := sql.subdelta(X_962,X_57,X_963,X_964); | | X_52 := calc.timestamp(A22,7); | | X_54 := calc.timestamp(A23,7); | | X_965 := algebra.subselect(X_44,X_60,X_52,X_54,true,true,false); | | (X_47,r1_47) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","gentime",2); | | X_966 := algebra.subselect(r1_47,X_52,X_54,true,true,false); | | X_50 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","gentime",1); | | X_967 := algebra.subselect(X_50,X_60,X_52,X_54,true,true,false); | | X_63 := sql.subdelta(X_965,X_47,X_966,X_967); | | X_64 := sql.projectdelta(X_63,X_55,X_57,r1_63,X_59); | | X_65 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","ggsn_id",0); | | (X_70,r1_84) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","ggsn_id",2); | | X_73 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","ggsn_id",1); | | X_75 := sql.projectdelta(X_63,X_65,X_70,r1_84,X_73); | | (X_76,r1_96,r2_96) := group.subgroup(X_75); | | (X_79,r1_99,r2_99) := group.subgroupdone(X_64,X_76); | | X_82:bat[:oid,:wrd] := aggr.subcount(X_79,X_79,r1_99,false); | | X_128 := algebra.leftfetchjoin(r1_99,X_64); | | X_134 := algebra.leftfetchjoin(r1_99,X_75); | | X_283 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","dns_delay_num",0); | | (X_288,r1_503) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","dns_delay_num",2); | | X_291 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","dns_delay_num",1); | | X_293 := sql.projectdelta(X_63,X_283,X_288,r1_503,X_291); | | X_294:bat[:oid,:lng] := aggr.subsum(X_293,X_79,r1_99,true,true); | | X_363 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","counts",0); | | (X_368,r1_749) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","counts",2); | | X_371 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","counts",1); | | X_373 := sql.projectdelta(X_63,X_363,X_368,r1_749,X_371); | | X_374:bat[:oid,:lng] := aggr.subsum(X_373,X_79,r1_99,true,true); | | X_406 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","dns_delay_msel",0); | | (X_411,r1_875) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","dns_delay_msel",2); | | X_414 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","dns_delay_msel",1); | | X_416 := sql.projectdelta(X_63,X_406,X_411,r1_875,X_414); | | X_417:bat[:oid,:lng] := aggr.subsum(X_416,X_79,r1_99,true,true); | | X_467 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","tcp_delay_num",0); | | (X_472,r1_1061) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","tcp_delay_num",2); | | X_475 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","tcp_delay_num",1); | | X_477 := sql.projectdelta(X_63,X_467,X_472,r1_1061,X_475); | | X_478:bat[:oid,:lng] := aggr.subsum(X_477,X_79,r1_99,true,true); | | X_510 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","tcp_delay_msel",0); | | (X_515,r1_1187) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","tcp_delay_msel",2); | | X_518 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","tcp_delay_msel",1); | | X_520 := sql.projectdelta(X_63,X_510,X_515,r1_1187,X_518); | | X_521:bat[:oid,:lng] := aggr.subsum(X_520,X_79,r1_99,true,true); | | X_572 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","get_delay_num",0); | | (X_577,r1_1376) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","get_delay_num",2); | | X_580 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","get_delay_num",1); | | X_582 := sql.projectdelta(X_63,X_572,X_577,r1_1376,X_580); | | X_583:bat[:oid,:lng] := aggr.subsum(X_582,X_79,r1_99,true,true); | | X_615 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","get_delay_msel",0); | | (X_620,r1_1502) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","get_delay_msel",2); | | X_623 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","get_delay_msel",1); | | X_625 := sql.projectdelta(X_63,X_615,X_620,r1_1502,X_623); | | X_626:bat[:oid,:lng] := aggr.subsum(X_625,X_79,r1_99,true,true); | | X_731 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","post_delay_num",0); | | (X_736,r1_1865) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","post_delay_num",2); | | X_739 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","post_delay_num",1); | | X_741 := sql.projectdelta(X_63,X_731,X_736,r1_1865,X_739); | | X_742:bat[:oid,:lng] := aggr.subsum(X_741,X_79,r1_99,true,true); | | X_775 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","post_delay_msel",0); | | (X_780,r1_1996) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","post_delay_msel",2); | | X_783 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","post_delay_msel",1); | | X_785 := sql.projectdelta(X_63,X_775,X_780,r1_1996,X_783); | | X_786:bat[:oid,:lng] := aggr.subsum(X_785,X_79,r1_99,true,true); | | X_83 := bat.append(X_40,X_82,true); | | X_87 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","gentime",0); | | X_96 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","ggsn_id",0); | | X_85:bat[:oid,:oid] := sql.tid(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1"); | | X_968 := algebra.subselect(X_96,X_85,A24,A24,true,true,false); | | (X_99,r1_128) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","ggsn_id",2); | | X_969 := algebra.subselect(r1_128,A24,A24,true,true,false); | | X_102 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","ggsn_id",1); | | X_970 := algebra.subselect(X_102,X_85,A24,A24,true,true,false); | | X_104 := sql.subdelta(X_968,X_99,X_969,X_970); | | X_93 := calc.timestamp(A25,7); | | X_95 := calc.timestamp(A26,7); | | X_971 := algebra.subselect(X_87,X_104,X_93,X_95,true,true,false); | | (X_89,r1_112) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","gentime",2); | | X_972 := algebra.subselect(r1_112,X_93,X_95,true,true,false); | | X_92 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","gentime",1); | | X_973 := algebra.subselect(X_92,X_104,X_93,X_95,true,true,false); | | X_106 := sql.subdelta(X_971,X_89,X_972,X_973); | | X_108 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","sgsn_id",0); | | (X_111,r1_145) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","sgsn_id",2); | | X_114 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","sgsn_id",1); | | X_115 := sql.projectdelta(X_106,X_108,X_111,r1_145,X_114); | | X_116 := sql.projectdelta(X_106,X_96,X_99,r1_128,X_102); | | (X_117,r1_161,r2_161) := group.subgroup(X_116); | | (X_120,r1_164,r2_164) := group.subgroupdone(X_115,X_117); | | X_123:bat[:oid,:wrd] := aggr.subcount(X_120,X_120,r1_164,false); | | X_131 := algebra.leftfetchjoin(r1_164,X_115); | | X_136 := algebra.leftfetchjoin(r1_164,X_116); | | X_297 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","dns_delay_num",0); | | (X_299,r1_524) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","dns_delay_num",2); | | X_301 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","dns_delay_num",1); | | X_302 := sql.projectdelta(X_106,X_297,X_299,r1_524,X_301); | | X_303:bat[:oid,:lng] := aggr.subsum(X_302,X_120,r1_164,true,true); | | X_377 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","counts",0); | | (X_379,r1_770) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","counts",2); | | X_381 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","counts",1); | | X_382 := sql.projectdelta(X_106,X_377,X_379,r1_770,X_381); | | X_383:bat[:oid,:lng] := aggr.subsum(X_382,X_120,r1_164,true,true); | | X_420 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","dns_delay_msel",0); | | (X_422,r1_896) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","dns_delay_msel",2); | | X_424 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","dns_delay_msel",1); | | X_425 := sql.projectdelta(X_106,X_420,X_422,r1_896,X_424); | | X_426:bat[:oid,:lng] := aggr.subsum(X_425,X_120,r1_164,true,true); | | X_481 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","tcp_delay_num",0); | | (X_483,r1_1082) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","tcp_delay_num",2); | | X_485 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","tcp_delay_num",1); | | X_486 := sql.projectdelta(X_106,X_481,X_483,r1_1082,X_485); | | X_487:bat[:oid,:lng] := aggr.subsum(X_486,X_120,r1_164,true,true); | | X_524 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","tcp_delay_msel",0); | | (X_526,r1_1208) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","tcp_delay_msel",2); | | X_528 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","tcp_delay_msel",1); | | X_529 := sql.projectdelta(X_106,X_524,X_526,r1_1208,X_528); | | X_530:bat[:oid,:lng] := aggr.subsum(X_529,X_120,r1_164,true,true); | | X_586 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","get_delay_num",0); | | (X_588,r1_1397) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","get_delay_num",2); | | X_590 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","get_delay_num",1); | | X_591 := sql.projectdelta(X_106,X_586,X_588,r1_1397,X_590); | | X_592:bat[:oid,:lng] := aggr.subsum(X_591,X_120,r1_164,true,true); | | X_629 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","get_delay_msel",0); | | (X_631,r1_1523) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","get_delay_msel",2); | | X_633 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","get_delay_msel",1); | | X_634 := sql.projectdelta(X_106,X_629,X_631,r1_1523,X_633); | | X_635:bat[:oid,:lng] := aggr.subsum(X_634,X_120,r1_164,true,true); | | X_745 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","post_delay_num",0); | | (X_747,r1_1886) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","post_delay_num",2); | | X_749 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","post_delay_num",1); | | X_750 := sql.projectdelta(X_106,X_745,X_747,r1_1886,X_749); | | X_751:bat[:oid,:lng] := aggr.subsum(X_750,X_120,r1_164,true,true); | | X_789 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","post_delay_msel",0); | | (X_791,r1_2017) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","post_delay_msel",2); | | X_793 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","post_delay_msel",1); | | X_794 := sql.projectdelta(X_106,X_789,X_791,r1_2017,X_793); | | X_795:bat[:oid,:lng] := aggr.subsum(X_794,X_120,r1_164,true,true); | | X_124 := bat.append(X_83,X_123,true); | | X_125:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_129 := bat.append(X_125,X_128,true); | | X_132 := bat.append(X_129,X_131,true); | | X_133:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_135 := bat.append(X_133,X_134,true); | | X_137 := bat.append(X_135,X_136,true); | | (X_138,r1_216,r2_216) := group.subgroup(X_137); | | (X_141,r1_219,r2_219) := group.subgroupdone(X_132,X_138); | | X_144:bat[:oid,:wrd] := aggr.subsum(X_124,X_141,r1_219,true,true); | | X_194 := algebra.leftfetchjoin(r1_219,X_132); | | X_199 := algebra.leftfetchjoin(r1_219,X_137); | | X_282:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_296 := bat.append(X_282,X_294,true); | | X_304 := bat.append(X_296,X_303,true); | | X_305:bat[:oid,:lng] := aggr.subsum(X_304,X_141,r1_219,true,true); | | X_325:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_326 := bat.append(X_325,X_294,true); | | X_327 := bat.append(X_326,X_303,true); | | X_328 := algebra.leftfetchjoin(r1_219,X_327); | | X_342:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_343 := bat.append(X_342,X_294,true); | | X_345 := bat.append(X_343,X_303,true); | | X_346 := algebra.leftfetchjoin(r1_219,X_345); | | X_362:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_376 := bat.append(X_362,X_374,true); | | X_384 := bat.append(X_376,X_383,true); | | X_385:bat[:oid,:lng] := aggr.subsum(X_384,X_141,r1_219,true,true); | | X_405:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_419 := bat.append(X_405,X_417,true); | | X_427 := bat.append(X_419,X_426,true); | | X_428:bat[:oid,:lng] := aggr.subsum(X_427,X_141,r1_219,true,true); | | X_447:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_448 := bat.append(X_447,X_374,true); | | X_449 := bat.append(X_448,X_383,true); | | X_451 := algebra.leftfetchjoin(r1_219,X_449); | | X_466:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_480 := bat.append(X_466,X_478,true); | | X_488 := bat.append(X_480,X_487,true); | | X_489:bat[:oid,:lng] := aggr.subsum(X_488,X_141,r1_219,true,true); | | X_509:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_523 := bat.append(X_509,X_521,true); | | X_531 := bat.append(X_523,X_530,true); | | X_532:bat[:oid,:lng] := aggr.subsum(X_531,X_141,r1_219,true,true); | | X_551:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_552 := bat.append(X_551,X_478,true); | | X_553 := bat.append(X_552,X_487,true); | | X_555 := algebra.leftfetchjoin(r1_219,X_553); | | X_571:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_585 := bat.append(X_571,X_583,true); | | X_593 := bat.append(X_585,X_592,true); | | X_594:bat[:oid,:lng] := aggr.subsum(X_593,X_141,r1_219,true,true); | | X_614:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_628 := bat.append(X_614,X_626,true); | | X_636 := bat.append(X_628,X_635,true); | | X_637:bat[:oid,:lng] := aggr.subsum(X_636,X_141,r1_219,true,true); | | X_656:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_657 := bat.append(X_656,X_583,true); | | X_658 := bat.append(X_657,X_592,true); | | X_660 := algebra.leftfetchjoin(r1_219,X_658); | | X_675:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_676 := bat.append(X_675,X_478,true); | | X_678 := bat.append(X_676,X_487,true); | | X_679 := algebra.leftfetchjoin(r1_219,X_678); | | X_693:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_694 := bat.append(X_693,X_521,true); | | X_696 := bat.append(X_694,X_530,true); | | X_697 := algebra.leftfetchjoin(r1_219,X_696); | | X_710:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_711 := bat.append(X_710,X_478,true); | | X_713 := bat.append(X_711,X_487,true); | | X_714 := algebra.leftfetchjoin(r1_219,X_713); | | X_730:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_744 := bat.append(X_730,X_742,true); | | X_752 := bat.append(X_744,X_751,true); | | X_753:bat[:oid,:lng] := aggr.subsum(X_752,X_141,r1_219,true,true); | | X_774:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_788 := bat.append(X_774,X_786,true); | | X_796 := bat.append(X_788,X_795,true); | | X_797:bat[:oid,:lng] := aggr.subsum(X_796,X_141,r1_219,true,true); | | X_818:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_819 := bat.append(X_818,X_742,true); | | X_821 := bat.append(X_819,X_751,true); | | X_822 := algebra.leftfetchjoin(r1_219,X_821); | | X_838:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_839 := bat.append(X_838,X_583,true); | | X_841 := bat.append(X_839,X_592,true); | | X_842 := algebra.leftfetchjoin(r1_219,X_841); | | X_856:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_857 := bat.append(X_856,X_626,true); | | X_859 := bat.append(X_857,X_635,true); | | X_860 := algebra.leftfetchjoin(r1_219,X_859); | | X_873:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_874 := bat.append(X_873,X_583,true); | | X_876 := bat.append(X_874,X_592,true); | | X_877 := algebra.leftfetchjoin(r1_219,X_876); | | X_146 := bat.append(X_39,X_144,true); | | X_150 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","gentime",0); | | X_161 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","sgsn_id",0); | | X_147:bat[:oid,:oid] := sql.tid(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0"); | | X_974 := algebra.subselect(X_161,X_147,A27,A27,true,true,false); | | (X_163,r1_251) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","sgsn_id",2); | | X_975 := algebra.subselect(r1_251,A27,A27,true,true,false); | | X_165 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","sgsn_id",1); | | X_976 := algebra.subselect(X_165,X_147,A27,A27,true,true,false); | | X_166 := sql.subdelta(X_974,X_163,X_975,X_976); | | X_158 := calc.timestamp(A28,7); | | X_160 := calc.timestamp(A29,7); | | X_977 := algebra.subselect(X_150,X_166,X_158,X_160,true,true,false); | | (X_153,r1_235) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","gentime",2); | | X_978 := algebra.subselect(r1_235,X_158,X_160,true,true,false); | | X_156 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","gentime",1); | | X_979 := algebra.subselect(X_156,X_166,X_158,X_160,true,true,false); | | X_169 := sql.subdelta(X_977,X_153,X_978,X_979); | | X_170 := sql.projectdelta(X_169,X_161,X_163,r1_251,X_165); | | X_171 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","ggsn_id",0); | | (X_176,r1_272) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","ggsn_id",2); | | X_179 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_0","ggsn_id",1); | | X_181 := sql.projectdelta(X_169,X_171,X_176,r1_272,X_179); | | (X_182,r1_284,r2_284) := group.subgroup(X_181); | | (X_185,r1_287,r2_287) := group.subgroupdone(X_170,X_182); | | X_188:bat[:oid,:wrd] := aggr.subcount(X_185,X_185,r1_287,false); | | X_196 := algebra.leftfetchjoin(r1_287,X_170); | | X_202 := algebra.leftfetchjoin(r1_287,X_181); | | X_307 := sql.projectdelta(X_169,X_283,X_288,r1_503,X_291); | | X_308:bat[:oid,:lng] := aggr.subsum(X_307,X_185,r1_287,true,true); | | X_387 := sql.projectdelta(X_169,X_363,X_368,r1_749,X_371); | | X_388:bat[:oid,:lng] := aggr.subsum(X_387,X_185,r1_287,true,true); | | X_430 := sql.projectdelta(X_169,X_406,X_411,r1_875,X_414); | | X_431:bat[:oid,:lng] := aggr.subsum(X_430,X_185,r1_287,true,true); | | X_491 := sql.projectdelta(X_169,X_467,X_472,r1_1061,X_475); | | X_492:bat[:oid,:lng] := aggr.subsum(X_491,X_185,r1_287,true,true); | | X_534 := sql.projectdelta(X_169,X_510,X_515,r1_1187,X_518); | | X_535:bat[:oid,:lng] := aggr.subsum(X_534,X_185,r1_287,true,true); | | X_596 := sql.projectdelta(X_169,X_572,X_577,r1_1376,X_580); | | X_597:bat[:oid,:lng] := aggr.subsum(X_596,X_185,r1_287,true,true); | | X_639 := sql.projectdelta(X_169,X_615,X_620,r1_1502,X_623); | | X_640:bat[:oid,:lng] := aggr.subsum(X_639,X_185,r1_287,true,true); | | X_755 := sql.projectdelta(X_169,X_731,X_736,r1_1865,X_739); | | X_756:bat[:oid,:lng] := aggr.subsum(X_755,X_185,r1_287,true,true); | | X_799 := sql.projectdelta(X_169,X_775,X_780,r1_1996,X_783); | | X_800:bat[:oid,:lng] := aggr.subsum(X_799,X_185,r1_287,true,true); | | X_189 := bat.append(X_146,X_188,true); | | X_191:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_195 := bat.append(X_191,X_194,true); | | X_197 := bat.append(X_195,X_196,true); | | X_198:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_200 := bat.append(X_198,X_199,true); | | X_203 := bat.append(X_200,X_202,true); | | (X_204,r1_341,r2_341) := group.subgroup(X_203); | | (X_207,r1_344,r2_344) := group.subgroupdone(X_197,X_204); | | X_210:bat[:oid,:wrd] := aggr.subsum(X_189,X_207,r1_344,true,true); | | X_259 := algebra.leftfetchjoin(r1_344,X_197); | | X_264 := algebra.leftfetchjoin(r1_344,X_203); | | X_281:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_306 := bat.append(X_281,X_305,true); | | X_310 := bat.append(X_306,X_308,true); | | X_311:bat[:oid,:lng] := aggr.subsum(X_310,X_207,r1_344,true,true); | | X_324:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_329 := bat.append(X_324,X_328,true); | | X_331 := bat.append(X_329,X_308,true); | | X_332 := algebra.leftfetchjoin(r1_344,X_331); | | X_341:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_347 := bat.append(X_341,X_346,true); | | X_348 := bat.append(X_347,X_308,true); | | X_349 := algebra.leftfetchjoin(r1_344,X_348); | | X_361:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_386 := bat.append(X_361,X_385,true); | | X_390 := bat.append(X_386,X_388,true); | | X_391:bat[:oid,:lng] := aggr.subsum(X_390,X_207,r1_344,true,true); | | X_404:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_429 := bat.append(X_404,X_428,true); | | X_433 := bat.append(X_429,X_431,true); | | X_434:bat[:oid,:lng] := aggr.subsum(X_433,X_207,r1_344,true,true); | | X_446:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_452 := bat.append(X_446,X_451,true); | | X_453 := bat.append(X_452,X_388,true); | | X_454 := algebra.leftfetchjoin(r1_344,X_453); | | X_465:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_490 := bat.append(X_465,X_489,true); | | X_494 := bat.append(X_490,X_492,true); | | X_495:bat[:oid,:lng] := aggr.subsum(X_494,X_207,r1_344,true,true); | | X_508:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_533 := bat.append(X_508,X_532,true); | | X_537 := bat.append(X_533,X_535,true); | | X_538:bat[:oid,:lng] := aggr.subsum(X_537,X_207,r1_344,true,true); | | X_550:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_556 := bat.append(X_550,X_555,true); | | X_557 := bat.append(X_556,X_492,true); | | X_558 := algebra.leftfetchjoin(r1_344,X_557); | | X_570:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_595 := bat.append(X_570,X_594,true); | | X_599 := bat.append(X_595,X_597,true); | | X_600:bat[:oid,:lng] := aggr.subsum(X_599,X_207,r1_344,true,true); | | X_613:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_638 := bat.append(X_613,X_637,true); | | X_642 := bat.append(X_638,X_640,true); | | X_643:bat[:oid,:lng] := aggr.subsum(X_642,X_207,r1_344,true,true); | | X_655:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_661 := bat.append(X_655,X_660,true); | | X_662 := bat.append(X_661,X_597,true); | | X_663 := algebra.leftfetchjoin(r1_344,X_662); | | X_674:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_680 := bat.append(X_674,X_679,true); | | X_681 := bat.append(X_680,X_492,true); | | X_682 := algebra.leftfetchjoin(r1_344,X_681); | | X_692:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_698 := bat.append(X_692,X_697,true); | | X_699 := bat.append(X_698,X_535,true); | | X_700 := algebra.leftfetchjoin(r1_344,X_699); | | X_709:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_715 := bat.append(X_709,X_714,true); | | X_716 := bat.append(X_715,X_492,true); | | X_717 := algebra.leftfetchjoin(r1_344,X_716); | | X_729:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_754 := bat.append(X_729,X_753,true); | | X_758 := bat.append(X_754,X_756,true); | | X_759:bat[:oid,:lng] := aggr.subsum(X_758,X_207,r1_344,true,true); | | X_773:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_798 := bat.append(X_773,X_797,true); | | X_802 := bat.append(X_798,X_800,true); | | X_803:bat[:oid,:lng] := aggr.subsum(X_802,X_207,r1_344,true,true); | | X_817:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_823 := bat.append(X_817,X_822,true); | | X_824 := bat.append(X_823,X_756,true); | | X_825 := algebra.leftfetchjoin(r1_344,X_824); | | X_837:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_843 := bat.append(X_837,X_842,true); | | X_844 := bat.append(X_843,X_597,true); | | X_845 := algebra.leftfetchjoin(r1_344,X_844); | | X_855:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_861 := bat.append(X_855,X_860,true); | | X_862 := bat.append(X_861,X_640,true); | | X_863 := algebra.leftfetchjoin(r1_344,X_862); | | X_872:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_878 := bat.append(X_872,X_877,true); | | X_879 := bat.append(X_878,X_597,true); | | X_880 := algebra.leftfetchjoin(r1_344,X_879); | | X_211 := bat.append(X_36,X_210,true); | | X_215 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","gentime",0); | | X_226 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","ggsn_id",0); | | X_212:bat[:oid,:oid] := sql.tid(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1"); | | X_980 := algebra.subselect(X_226,X_212,A30,A30,true,true,false); | | (X_228,r1_374) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","ggsn_id",2); | | X_981 := algebra.subselect(r1_374,A30,A30,true,true,false); | | X_230 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","ggsn_id",1); | | X_982 := algebra.subselect(X_230,X_212,A30,A30,true,true,false); | | X_231 := sql.subdelta(X_980,X_228,X_981,X_982); | | X_223 := calc.timestamp(A31,7); | | X_225 := calc.timestamp(A32,7); | | X_983 := algebra.subselect(X_215,X_231,X_223,X_225,true,true,false); | | (X_218,r1_358) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","gentime",2); | | X_984 := algebra.subselect(r1_358,X_223,X_225,true,true,false); | | X_221 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","gentime",1); | | X_985 := algebra.subselect(X_221,X_231,X_223,X_225,true,true,false); | | X_234 := sql.subdelta(X_983,X_218,X_984,X_985); | | X_235 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","sgsn_id",0); | | (X_240,r1_391) := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","sgsn_id",2); | | X_243 := sql.bind(X_35,"sys","bkpi_web_dspdelay_cisai_15min_1","sgsn_id",1); | | X_245 := sql.projectdelta(X_234,X_235,X_240,r1_391,X_243); | | X_246 := sql.projectdelta(X_234,X_226,X_228,r1_374,X_230); | | (X_247,r1_408,r2_408) := group.subgroup(X_246); | | (X_250,r1_411,r2_411) := group.subgroupdone(X_245,X_247); | | X_253:bat[:oid,:wrd] := aggr.subcount(X_250,X_250,r1_411,false); | | X_261 := algebra.leftfetchjoin(r1_411,X_245); | | X_267 := algebra.leftfetchjoin(r1_411,X_246); | | X_313 := sql.projectdelta(X_234,X_297,X_299,r1_524,X_301); | | X_314:bat[:oid,:lng] := aggr.subsum(X_313,X_250,r1_411,true,true); | | X_393 := sql.projectdelta(X_234,X_377,X_379,r1_770,X_381); | | X_394:bat[:oid,:lng] := aggr.subsum(X_393,X_250,r1_411,true,true); | | X_436 := sql.projectdelta(X_234,X_420,X_422,r1_896,X_424); | | X_437:bat[:oid,:lng] := aggr.subsum(X_436,X_250,r1_411,true,true); | | X_497 := sql.projectdelta(X_234,X_481,X_483,r1_1082,X_485); | | X_498:bat[:oid,:lng] := aggr.subsum(X_497,X_250,r1_411,true,true); | | X_540 := sql.projectdelta(X_234,X_524,X_526,r1_1208,X_528); | | X_541:bat[:oid,:lng] := aggr.subsum(X_540,X_250,r1_411,true,true); | | X_602 := sql.projectdelta(X_234,X_586,X_588,r1_1397,X_590); | | X_603:bat[:oid,:lng] := aggr.subsum(X_602,X_250,r1_411,true,true); | | X_645 := sql.projectdelta(X_234,X_629,X_631,r1_1523,X_633); | | X_646:bat[:oid,:lng] := aggr.subsum(X_645,X_250,r1_411,true,true); | | X_761 := sql.projectdelta(X_234,X_745,X_747,r1_1886,X_749); | | X_762:bat[:oid,:lng] := aggr.subsum(X_761,X_250,r1_411,true,true); | | X_805 := sql.projectdelta(X_234,X_789,X_791,r1_2017,X_793); | | X_806:bat[:oid,:lng] := aggr.subsum(X_805,X_250,r1_411,true,true); | | X_254 := bat.append(X_211,X_253,true); | | X_256:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_260 := bat.append(X_256,X_259,true); | | X_262 := bat.append(X_260,X_261,true); | | X_263:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_265 := bat.append(X_263,X_264,true); | | X_268 := bat.append(X_265,X_267,true); | | (X_269,r1_464,r2_464) := group.subgroup(X_268); | | (X_272,r1_467,r2_467) := group.subgroupdone(X_262,X_269); | | X_275:bat[:oid,:wrd] := aggr.subsum(X_254,X_272,r1_467,true,true); | | X_276 := algebra.leftfetchjoin(r1_467,X_268); | | X_277 := algebra.leftfetchjoin(r1_467,X_262); | | X_278:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_312 := bat.append(X_278,X_311,true); | | X_315 := bat.append(X_312,X_314,true); | | X_316:bat[:oid,:lng] := aggr.subsum(X_315,X_272,r1_467,true,true); | | X_321:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_333 := bat.append(X_321,X_332,true); | | X_334 := bat.append(X_333,X_314,true); | | X_335 := algebra.leftfetchjoin(r1_467,X_334); | | X_338:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_350 := bat.append(X_338,X_349,true); | | X_351 := bat.append(X_350,X_314,true); | | X_352 := algebra.leftfetchjoin(r1_467,X_351); | | X_358:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_392 := bat.append(X_358,X_391,true); | | X_395 := bat.append(X_392,X_394,true); | | X_396:bat[:oid,:lng] := aggr.subsum(X_395,X_272,r1_467,true,true); | | X_401:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_435 := bat.append(X_401,X_434,true); | | X_438 := bat.append(X_435,X_437,true); | | X_439:bat[:oid,:lng] := aggr.subsum(X_438,X_272,r1_467,true,true); | | X_443:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_455 := bat.append(X_443,X_454,true); | | X_456 := bat.append(X_455,X_394,true); | | X_457 := algebra.leftfetchjoin(r1_467,X_456); | | X_462:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_496 := bat.append(X_462,X_495,true); | | X_499 := bat.append(X_496,X_498,true); | | X_500:bat[:oid,:lng] := aggr.subsum(X_499,X_272,r1_467,true,true); | | X_505:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_539 := bat.append(X_505,X_538,true); | | X_542 := bat.append(X_539,X_541,true); | | X_543:bat[:oid,:lng] := aggr.subsum(X_542,X_272,r1_467,true,true); | | X_547:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_559 := bat.append(X_547,X_558,true); | | X_560 := bat.append(X_559,X_498,true); | | X_561 := algebra.leftfetchjoin(r1_467,X_560); | | X_567:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_601 := bat.append(X_567,X_600,true); | | X_604 := bat.append(X_601,X_603,true); | | X_605:bat[:oid,:lng] := aggr.subsum(X_604,X_272,r1_467,true,true); | | X_610:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_644 := bat.append(X_610,X_643,true); | | X_647 := bat.append(X_644,X_646,true); | | X_648:bat[:oid,:lng] := aggr.subsum(X_647,X_272,r1_467,true,true); | | X_652:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_664 := bat.append(X_652,X_663,true); | | X_665 := bat.append(X_664,X_603,true); | | X_666 := algebra.leftfetchjoin(r1_467,X_665); | | X_671:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_683 := bat.append(X_671,X_682,true); | | X_684 := bat.append(X_683,X_498,true); | | X_685 := algebra.leftfetchjoin(r1_467,X_684); | | X_689:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_701 := bat.append(X_689,X_700,true); | | X_702 := bat.append(X_701,X_541,true); | | X_703 := algebra.leftfetchjoin(r1_467,X_702); | | X_706:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_718 := bat.append(X_706,X_717,true); | | X_719 := bat.append(X_718,X_498,true); | | X_720 := algebra.leftfetchjoin(r1_467,X_719); | | X_726:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_760 := bat.append(X_726,X_759,true); | | X_763 := bat.append(X_760,X_762,true); | | X_764:bat[:oid,:lng] := aggr.subsum(X_763,X_272,r1_467,true,true); | | X_770:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_804 := bat.append(X_770,X_803,true); | | X_807 := bat.append(X_804,X_806,true); | | X_808:bat[:oid,:lng] := aggr.subsum(X_807,X_272,r1_467,true,true); | | X_814:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_826 := bat.append(X_814,X_825,true); | | X_827 := bat.append(X_826,X_762,true); | | X_828 := algebra.leftfetchjoin(r1_467,X_827); | | X_834:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_846 := bat.append(X_834,X_845,true); | | X_847 := bat.append(X_846,X_603,true); | | X_848 := algebra.leftfetchjoin(r1_467,X_847); | | X_852:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_864 := bat.append(X_852,X_863,true); | | X_865 := bat.append(X_864,X_646,true); | | X_866 := algebra.leftfetchjoin(r1_467,X_865); | | X_869:bat[:oid,:lng] := bat.new(nil:oid,nil:lng); | | X_881 := bat.append(X_869,X_880,true); | | X_882 := bat.append(X_881,X_603,true); | | X_883 := algebra.leftfetchjoin(r1_467,X_882); | | X_849:bat[:oid,:bit] := batcalc.==(X_848,A19); | | X_850:bat[:oid,:bit] := batcalc.isnil(X_849); | | X_851:bat[:oid,:bit] := batcalc.ifthenelse(X_850,false:bit,X_849); | | X_867 := batcalc.flt(X_866); | | X_868:bat[:oid,:flt] := batmmath.floor(X_867); | | X_884 := batcalc.flt(X_883); | | X_885:bat[:oid,:flt] := batcalc.ifthenelse(X_851,1:flt,X_884); | | X_886:bat[:oid,:flt] := batcalc./(X_868,X_885); | | X_887:bat[:oid,:flt] := batcalc.ifthenelse(X_851,A20,X_886); | | X_766:bat[:oid,:lng] := batcalc.*(X_764,A13); | | X_767:bat[:oid,:bit] := batcalc.==(X_766,A14); | | X_768:bat[:oid,:bit] := batcalc.isnil(X_767); | | X_769:bat[:oid,:bit] := batcalc.ifthenelse(X_768,false:bit,X_767); | | X_810:bat[:oid,:lng] := batcalc.*(X_808,A16); | | X_811:bat[:oid,:lng] := batcalc.*(X_810,A17); | | X_812 := batcalc.flt(X_811); | | X_813:bat[:oid,:flt] := batmmath.floor(X_812); | | X_829:bat[:oid,:lng] := batcalc.*(X_828,A18); | | X_830 := batcalc.flt(X_829); | | X_831:bat[:oid,:flt] := batcalc.ifthenelse(X_769,1:flt,X_830); | | X_832:bat[:oid,:flt] := batcalc./(X_813,X_831); | | X_833:bat[:oid,:flt] := batcalc.ifthenelse(X_769,A15,X_832); | | X_686:bat[:oid,:bit] := batcalc.==(X_685,A10); | | X_687:bat[:oid,:bit] := batcalc.isnil(X_686); | | X_688:bat[:oid,:bit] := batcalc.ifthenelse(X_687,false:bit,X_686); | | X_704 := batcalc.flt(X_703); | | X_705:bat[:oid,:flt] := batmmath.floor(X_704); | | X_721 := batcalc.flt(X_720); | | X_722:bat[:oid,:flt] := batcalc.ifthenelse(X_688,1:flt,X_721); | | X_723:bat[:oid,:flt] := batcalc./(X_705,X_722); | | X_724:bat[:oid,:flt] := batcalc.*(X_723,A12); | | X_725:bat[:oid,:flt] := batcalc.ifthenelse(X_688,A11,X_724); | | X_607:bat[:oid,:bit] := batcalc.==(X_605,A8); | | X_608:bat[:oid,:bit] := batcalc.isnil(X_607); | | X_609:bat[:oid,:bit] := batcalc.ifthenelse(X_608,false:bit,X_607); | | X_650 := batcalc.flt(X_648); | | X_651:bat[:oid,:flt] := batmmath.floor(X_650); | | X_667 := batcalc.flt(X_666); | | X_668:bat[:oid,:flt] := batcalc.ifthenelse(X_609,1:flt,X_667); | | X_669:bat[:oid,:flt] := batcalc./(X_651,X_668); | | X_670:bat[:oid,:flt] := batcalc.ifthenelse(X_609,A9,X_669); | | X_502:bat[:oid,:bit] := batcalc.==(X_500,A5); | | X_503:bat[:oid,:bit] := batcalc.isnil(X_502); | | X_504:bat[:oid,:bit] := batcalc.ifthenelse(X_503,false:bit,X_502); | | X_545 := batcalc.flt(X_543); | | X_546:bat[:oid,:flt] := batmmath.floor(X_545); | | X_562 := batcalc.flt(X_561); | | X_563:bat[:oid,:flt] := batcalc.ifthenelse(X_504,1:flt,X_562); | | X_564:bat[:oid,:flt] := batcalc./(X_546,X_563); | | X_565:bat[:oid,:flt] := batcalc.*(X_564,A7); | | X_566:bat[:oid,:flt] := batcalc.ifthenelse(X_504,A6,X_565); | | X_398:bat[:oid,:bit] := batcalc.==(X_396,A3); | | X_399:bat[:oid,:bit] := batcalc.isnil(X_398); | | X_400:bat[:oid,:bit] := batcalc.ifthenelse(X_399,false:bit,X_398); | | X_441 := batcalc.flt(X_439); | | X_442:bat[:oid,:flt] := batmmath.floor(X_441); | | X_458 := batcalc.flt(X_457); | | X_459:bat[:oid,:flt] := batcalc.ifthenelse(X_400,1:flt,X_458); | | X_460:bat[:oid,:flt] := batcalc./(X_442,X_459); | | X_461:bat[:oid,:flt] := batcalc.ifthenelse(X_400,A4,X_460); | | X_318:bat[:oid,:bit] := batcalc.==(X_316,A0); | | X_319:bat[:oid,:bit] := batcalc.isnil(X_318); | | X_320:bat[:oid,:bit] := batcalc.ifthenelse(X_319,false:bit,X_318); | | X_336 := batcalc.flt(X_335); | | X_337:bat[:oid,:flt] := batmmath.floor(X_336); | | X_353 := batcalc.flt(X_352); | | X_354:bat[:oid,:flt] := batcalc.ifthenelse(X_320,1:flt,X_353); | | X_355:bat[:oid,:flt] := batcalc./(X_337,X_354); | | X_356:bat[:oid,:flt] := batcalc.*(X_355,A2); | | X_357:bat[:oid,:flt] := batcalc.ifthenelse(X_320,A1,X_356); | | X_889 := sql.resultSet(10,1,X_275); | | sql.rsColumn(X_889,".bkpi","L1","wrd",64,0,X_275); | | sql.rsColumn(X_889,".bkpi","ggsn_id","bigint",64,0,X_276); | | sql.rsColumn(X_889,".bkpi","sgsn_id","bigint",64,0,X_277); | | sql.rsColumn(X_889,".","kqidelay","real",24,0,X_357); | | sql.rsColumn(X_889,".","dns_delay","real",24,0,X_461); | | sql.rsColumn(X_889,".","tcpdelay","real",24,0,X_566); | | sql.rsColumn(X_889,".","tcpgetdelay","real",24,0,X_670); | | sql.rsColumn(X_889,".","tcp_delay_avgmsel","real",24,0,X_725); | | sql.rsColumn(X_889,".","tcppostdelay","real",24,0,X_833); | | sql.rsColumn(X_889,".","get_delay_msel_avg","real",24,0,X_887); | | X_919 := io.stdout(); | | sql.exportResult(X_919,X_889); | | end s1_4; | | # optimizer.mitosis() | | # optimizer.dataflow() | +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 616 tuples (20.058ms) Thanks, Jason Hi Jason, to be able to analyze the problem, we'd need all the information to be able to reproduce it; this includes at least - exact version of MonetDB - operating system - hardware, in particular #hardware threads and size of RAM (the output of `mserver5 --version` should contain all these) - database schema and queries - data; or at least EXPLAIN's and TRACE's of your queries Preferably, file a bug report with all this information. Best, Stefan ----- Original Message ----- > Hi guys: > In my case , original data 8G bytes, but after plenty of > query, the disk space usage is 361G bytes. The query is > simply, first five UNION ALL, then fetch the result. When the > query is running, I use vmstat to watch the IO status. When > the database becomes larger, the BI and BO number increase > very large too, for example bo 355224, bi 27392. And the > query cost ten times longer than before! > After rebooting the database, the database disk space was > reduced to its original data size, 8G. It seems too much > runtime data was produced. How can I solve the problem or > avoid the problem! > > Thanks,Jason ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

1 0

After plenty of querys the database use 40 times more disk space than the original data, How can I avoid this?
by liyuesen 00108910 06 Jun '13

06 Jun '13

Hi guys: In my case , original data 8G bytes, but after plenty of query, the disk space usage is 361G bytes. The query is simply, first five UNION ALL, then fetch the result. When the query is running, I use vmstat to watch the IO status. When the database becomes larger, the BI and BO number increase very large too, for example bo 355224, bi 27392. And the query cost ten times longer than before! After rebooting the database, the database disk space was reduced to its original data size, 8G. It seems too much runtime data was produced. How can I solve the problem or avoid the problem! Thanks,Jason ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

2 1

about select of GDK
by 张磊 03 Jun '13

03 Jun '13

hi all, I am tracing the select of GDK, it's so complex, and In every type, there is a nocheck variable, who can help me explain the nocheck variable, what's the meaning of it? :) -- Best Wishes! zhanglei

2 1

Need help with geometry buffer function
by Casey Gum 29 May '13

29 May '13

In the docs about the geospatial extension, it mentions a buffer function: CREATE FUNCTION Buffer(a Geometry, distance FLOAT) RETURNS Geometry Does anyone know what a good value would be to use for distance if I wanted to buffer a point, line, etc. X number of meters? Or does anyone have any ideas for what would be a good approach for this problem - where you want the user to be able to specify how close two things can be to one another to be a match? Thanks a lot, Casey

2 2

Inserting geometry using JDBC PreparedStatement
by Casey Gum 27 May '13

27 May '13

Is there a way to use JDBC PreparedStatements to insert geospatial data? Here's what my prepared statement looks like: PreparedStatement statement = sqlRunner.prepareStatement(insertStatement); statement.setString(1, id); statement.setString(2, feature.getName()); statement.setString(3, feature.getDescription()); statement.setString(4, feature.getType().toString()); statement.setString(5, feature.getGeometry()); statement.executeUpdate(); In this case, feature.getGeometry() returns a string in WKT format. Here's the error I get: EXEC: wrong type for argument 5 of prepared statement: char, expected geometry Thoughts? Thanks, Casey

2 1

Is there a pre-compiled version of MonetDB with geospatial support for OSX?
by Casey Gum 22 May '13

22 May '13

If so, does anyone have a link? Thanks, Casey

1 0

Re: MonetDB: default - Use normal join code for thetajoin with == op...
by Stefan Manegold 21 May '13

21 May '13

On Fri, May 17, 2013 at 11:33:39AM +0200, Sjoerd Mullender wrote: > Changeset: f2c52e636835 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f2c52e636835 > Modified Files: > gdk/gdk_join.c > Branch: default > Log Message: > > Use normal join code for thetajoin with == operator. > > > diffs (22 lines): > > diff --git a/gdk/gdk_join.c b/gdk/gdk_join.c > --- a/gdk/gdk_join.c > +++ b/gdk/gdk_join.c > @@ -1720,11 +1720,17 @@ BATsubthetajoin(BAT **r1p, BAT **r2p, BA > { > BAT *r1, *r2; > > + if (op[0] == '=' && ((op[1] == '=' && op[2] == 0) || op[2] == 0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ~~~~~~~~~~ Sjoerd, given the last predicate ("~"), the second ("^") seems obsolete to me. Should the last one rather read "op[1] == 0", i.e., both op == "==" and op == "=" indicate an equality predicate? Or should the last predicate be opmitted completely to accept only op = "==" as euality predicate? Stefan > + return BATsubjoin(r1p, r2p, l, r, sl, sr, estimate); > + > *r1p = NULL; > *r2p = NULL; > if (joinparamcheck(l, r, sl, sr, "BATsubthetajoin") == GDK_FAIL) > return GDK_FAIL; > - if (joininitresults(&r1, &r2, estimate != BUN_NONE ? estimate : sl ? BATcount(sl) : BATcount(l), "BATsubthetajoin") == GDK_FAIL) > + if (joininitresults(&r1, &r2, > + estimate != BUN_NONE ? estimate : > + (sl ? BATcount(sl) : BATcount(l)) * (sr ? BATcount(sr) : BATcount(r)), > + "BATsubthetajoin") == GDK_FAIL) > return GDK_FAIL; > *r1p = r1; > *r2p = r2; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 1

Smart top-k querying: sorted access
by Maarten Duren 16 May '13

16 May '13

Hi all, We are two Computer Science Master students at Utrecht University. As part of our study, we are doing an experimentation project involving MonetDB. We would like to enrich MonetDB by implementing some smart techniques to handle top-k queries. So far, we have successfully implemented the Threshold Algorithm [Fagin] in MAL for a given top-k selection SQL query. It already outperforms MonetDB's strategy (a naive strategy) if the dataset is large enough :) The problem we are facing now is that almost all top-k techniques require sorted access on the attributes used in the score/ranking function. And sorting a BAT in the MAL program is the real performance bottleneck in our MAL program right now. We have had no success using indexes for sorted access because of the way MonetDB deals with these. We already thought of creating an extra table for each attribute used and insert every value in sorted order. This is a way to achieve sorted access on that attribute. But, obviously, that is not the way to go. It would be feasible for one given SQL query and given data, but not as a general solution. So the real question boils down to: (if possible) how can we get sorted access to an arbitrary attribute of a SQL table or a BAT without sorting it in MAL / MonetDB? Thanks, Maarten van Duren & Nik Schuiling

1 0

Building RPMs with geospatial module
by Casey Gum 15 May '13

15 May '13

I need to build all of the RPMs for MonetDB. I don't really have a choice in this - building from scratch on the machine it will be deployed on is (contractually) not an option in this case. I can build everything fine. For instance, from source directory: *mkdir build; cd build* *../configure strict=no assert=no debug=no optimize=yes* *make* *make install* *make rpm* On the machine I build on, I can go to mclient and execute something like: *sql>CREATE TABLE forests(id INT,name TEXT,shape MULTIPOLYGON);* *operation successful (20.620ms)* On the machine that I install RPMs on, I get this: * sql>CREATE TABLE forests(id INT,name TEXT,shape MULTIPOLYGON); 22000!type (multipolygon) unknown in: "create table forests(id int,name text,shape multipolygon)" * Anyone know how to get the RPMs to build with geospatial support?? Thanks, Casey

3 5

Re: In SQL functions, how to fetch table field value to a variable and how to execute dynamic statement?
by liyuesen 00108910 15 May '13

15 May '13

Oh, I got it, the first time there is one record, after that because more than one records are inserted, so select into can't not work! Thanks, Jason On Wed, May 15, 2013 at 06:46:29PM +0800, liyuesen 00108910 wrote: > I use select into as follow, but it does not work as I want. Can you help me fix it? > step1 create table: > CREATE TABLE table_test(f1 integer, f2 integer); > > step2 create inner function: > CREATE FUNCTION MY_Test() > RETURNS int > BEGIN > DECLARE num integer; > SET num = 100; > INSERT INTO table_test values(1, 2); > RETURN num; > END; > step3 create outer function: > CREATE FUNCTION MY_Test1() > RETURNS integer > BEGIN > DECLARE num integer; > DECLARE num1 integer; > DECLARE num2 integer; > SET num = MY_Test(); > SELECT * INTO num1, num2 from table_test; > SET num = num1 + num2; > RETURN num; > END; > step4 call outer function: > > sql>select MY_Test1(); > cardinality violation (2>1) > > I get the result, what's wrong with the statements? how many rows are already in your table_test? Niels > > Thanks, Jason > > On Wed, May 15, 2013 at 06:21:26PM +0800, liyuesen 00108910 wrote: > > What about the SELECT INTO or CURSOR in SQL functions? Can them be used? > CURSORs don't exist, ie you should select into a variable. > > Niels > > > > Jason > > On Wed, May 15, 2013 at 05:16:38PM +0800, liyuesen 00108910 wrote: > > > Hi guys: > > > I am using Moentdb 11.15.3 on Redhat linux on Inter CPU. > > > First issue, I try to write a SQL function , contains statements fetching table values into local variable, but the SELECT INTO and CURSOR statements are not supported, how can I do that? > > Select into is supported. Cursor is and will not be supported. > > > Second issue, can I execute dynamic statements in SQL function? If that is the case, can you show me how to do it? > > Dynamic sql also isn't supported and very unlikely to be supported. > > > > Niels > > > > > > Thanks, Jason ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

2 1

Re: In SQL functions, how to fetch table field value to a variable and how to execute dynamic statement?
by liyuesen 00108910 15 May '13

15 May '13

I use select into as follow, but it does not work as I want. Can you help me fix it? step1 create table: CREATE TABLE table_test(f1 integer, f2 integer); step2 create inner function: CREATE FUNCTION MY_Test() RETURNS int BEGIN DECLARE num integer; SET num = 100; INSERT INTO table_test values(1, 2); RETURN num; END; step3 create outer function: CREATE FUNCTION MY_Test1() RETURNS integer BEGIN DECLARE num integer; DECLARE num1 integer; DECLARE num2 integer; SET num = MY_Test(); SELECT * INTO num1, num2 from table_test; SET num = num1 + num2; RETURN num; END; step4 call outer function: sql>select MY_Test1(); cardinality violation (2>1) I get the result, what's wrong with the statements? Thanks, Jason On Wed, May 15, 2013 at 06:21:26PM +0800, liyuesen 00108910 wrote: > What about the SELECT INTO or CURSOR in SQL functions? Can them be used? CURSORs don't exist, ie you should select into a variable. Niels > > Jason > On Wed, May 15, 2013 at 05:16:38PM +0800, liyuesen 00108910 wrote: > > Hi guys: > > I am using Moentdb 11.15.3 on Redhat linux on Inter CPU. > > First issue, I try to write a SQL function , contains statements fetching table values into local variable, but the SELECT INTO and CURSOR statements are not supported, how can I do that? > Select into is supported. Cursor is and will not be supported. > > Second issue, can I execute dynamic statements in SQL function? If that is the case, can you show me how to do it? > Dynamic sql also isn't supported and very unlikely to be supported. > > Niels > > > > Thanks, Jason > > ****************************************************************************************** > > This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or e ma > il > > immediately and delete it! > > ***************************************************************************************** > > _______________________________________________ > > developers-list mailing list > > developers-list(a)monetdb.org > > http://mail.monetdb.org/mailman/listinfo/developers-list > > -- > Niels Nes, Centrum Wiskunde & Informatica (CWI) > Science Park 123, 1098 XG Amsterdam, The Netherlands > room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl > url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl > _______________________________________________ > developers-list mailing list > developers-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/developers-list > ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

2 1

Re: In SQL functions, how to fetch table field value to a variable and how to execute dynamic statement?
by liyuesen 00108910 15 May '13

15 May '13

What about the SELECT INTO or CURSOR in SQL functions? Can them be used? Jason On Wed, May 15, 2013 at 05:16:38PM +0800, liyuesen 00108910 wrote: > Hi guys: > I am using Moentdb 11.15.3 on Redhat linux on Inter CPU. > First issue, I try to write a SQL function , contains statements fetching table values into local variable, but the SELECT INTO and CURSOR statements are not supported, how can I do that? Select into is supported. Cursor is and will not be supported. > Second issue, can I execute dynamic statements in SQL function? If that is the case, can you show me how to do it? Dynamic sql also isn't supported and very unlikely to be supported. Niels > > Thanks, Jason > ****************************************************************************************** > This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or ema il > immediately and delete it! > ***************************************************************************************** > _______________________________________________ > developers-list mailing list > developers-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/developers-list -- Niels Nes, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

2 1

Re: developers-list Digest, Vol 9, Issue 6
by kelianh 15 May '13

15 May '13

hello, The tables do not have any constrains. And the disk space used only 5%,the total size is 1.2TB. This error relates to the mmap function , when I type 'ulimit -a': I find the virtual memory was set only 20GB by default. So I change it to unlimited. Then I load sf100 again. It works. Thx, keliang 在 2013-5-15，18:00，developers-list-request@monetdb.org 写道： > Send developers-list mailing list submissions to > developers-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.monetdb.org/mailman/listinfo/developers-list > or, via email, send a message with subject or body 'help' to > developers-list-request(a)monetdb.org > > You can reach the person managing the list at > developers-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of developers-list digest..." > > > Today's Topics: > > 1. The crash of loading data (??) > 2. Re: The crash of loading data (Sjoerd Mullender) > 3. Re: The crash of loading data (Stefan Manegold) > 4. In SQL functions, how to fetch table field value to a > variable and how to execute dynamic statement? (liyuesen 00108910) > 5. Re: In SQL functions, how to fetch table field value to a > variable and how to execute dynamic statement? (Niels Nes) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 14 May 2013 20:29:08 +0800 > From: ?? <hskeliang(a)gmail.com> > To: developers-list(a)monetdb.org > Subject: The crash of loading data > Message-ID: > <CAKzNtiqTBDtN+gNHOznZHRH9PrjsXtD7kegGHvOXbGv9jjtLDQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hello, > I'm trying to load SF100 data into MonetDB but it crashed. My server: > > 64bit openSUSE 11.3 > mem: 47GB > swap: 1GB > disk: 1.2T > monetDB: 11.15.3 > > I use DBgen to generate my data. > when I'm in a client I execute a script, after a few minutes, the monetDB > was crashed. > > get some error msg like this: HEAPextend failed to extend to XXX for XXXX > > my script file : > COPY records into from nation.tbl using ...; > COPY records into from region.tbl using ...; > COPY records into from customer.tbl using ...; > COPY records into from part.tbl using ...; > COPY records into from ...; > COPY records into from ...; > COPY records into from ...; > > Does anybody know what's reason for this error ? > and how can i overcome this error? > Any help would be most welcome:) > > Thx, > keliang >

1 0

In SQL functions, how to fetch table field value to a variable and how to execute dynamic statement?
by liyuesen 00108910 15 May '13

15 May '13

Hi guys: I am using Moentdb 11.15.3 on Redhat linux on Inter CPU. First issue, I try to write a SQL function , contains statements fetching table values into local variable, but the SELECT INTO and CURSOR statements are not supported, how can I do that? Second issue, can I execute dynamic statements in SQL function? If that is the case, can you show me how to do it? Thanks, Jason ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

2 1

The crash of loading data
by 柯亮 14 May '13

14 May '13

Hello, I'm trying to load SF100 data into MonetDB but it crashed. My server: 64bit openSUSE 11.3 mem: 47GB swap: 1GB disk: 1.2T monetDB: 11.15.3 I use DBgen to generate my data. when I'm in a client I execute a script, after a few minutes, the monetDB was crashed. get some error msg like this: HEAPextend failed to extend to XXX for XXXX my script file : COPY records into from nation.tbl using ...; COPY records into from region.tbl using ...; COPY records into from customer.tbl using ...; COPY records into from part.tbl using ...; COPY records into from ...; COPY records into from ...; COPY records into from ...; Does anybody know what's reason for this error ? and how can i overcome this error? Any help would be most welcome:) Thx, keliang

3 2

How to Capture Data with Stethoscope / Tomograph
by Stephen P. Morgan 14 May '13

14 May '13

Can you give a quick explanation on how to get stethoscope or tomograph to capture data? The online documentation is a bit sparse. As I understand it, I can start stethoscope with the following command: "stethoscope -d tpch -h localhost +ts algebra.* bat.* group.* sql.* aggr.*" It does seem to start, but it never seems to progress. I see the following line: "-- opened UDP profile stream StephenMorgan:50010 for localhost" But I never get any more output. As apparently suggested, I ran a query in another window, but got no obvious trace events or any other output. I eventually just killed it with ^C, at which point it said, "-- connection with server localhost closed". It didn't seem to leave behind a file. I also tried running tomograph, using "tomograph -d tpch -h localhost" and got a similar startup message. I ran a query in another window, but nothing happened. When I finally interrupted it, it told me to "Run: 'gnuplot tomograph.gpl'". Aha! Success: however, when I tried that command, it complained that it was "Skipping data file with no valid points". Clearly, I'm missing how to capture data. Thanks, Steve Morgan

1 0

When trying to create a MAL function, an error occurs, can you guys tell me what should I do to create a MAL function.
by liyuesen 00108910 12 May '13

12 May '13

By the way, I am using the newest Monetdb source package,version 11.5.3. And the platfrom is Redhat linux on intel CPU. The error message: mal>function test(i:int):str; mal>io.print(i); mal>i:= i*2; mal> b:= bat.new(:int,:int); mal>bat.insert(b,1,i); mal>io.print(b); mal>return test:= "ok"; mal>end test; MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = end test; ERROR = !TypeException:BAT.test[3]:'bat.new' undefined in: b:any := bat.new(_5:int, _6:int) !TypeException:BAT.test[4]:'bat.insert' undefined in: _7:any := bat.insert(b:any, _8:int, i:int) Can you help me to fix the error and give me some advice about how to create a MAL function. The other issue, about the binary copy command. Use explain we can get a MAL function like this. Can the append operations be executed parallel by using language.dataflow(). If that is the case, can you show me how to do this? explain copy binary into sys.DETAIL_UFDR_OTHER from ('/srv/monetdb/o/SID.dat','/srv/monetdb/o/PROBEID.dat','/srv/monetdb/o/GROUPID.dat','/srv/monetdb/o/GGSN_ID.dat','/srv/monetdb/o/SGSN_ID.dat','/srv/monetdb/o/SESSION_INDICATOR.dat','/srv/monetdb/o/BEGIN_TIME.dat','/srv/monetdb/o/BEGIN_TIME_MSEL.dat','/srv/monetdb/o/END_TIME.dat','/srv/monetdb/o/END_TIME_MSEL.dat','/srv/monetdb/o/PROT_CATEGORY.dat','/srv/monetdb/o/PROT_TYPE.dat','/srv/mone tdb/o/L7_CARRIER_PROT.dat','/srv/monetdb/o/MSISDN.dat','/srv/monetdb/o/IMSI.dat','/srv/monetdb/o/IMEI.dat','/srv/monetdb/o/ENCRYPT_VERSION.dat','/srv/monetdb/o/ROAMING_TYPE.dat','/srv/monetdb/o/ROAM_DIRECTION.dat','/srv/monetdb/o/MS_IP.dat','/srv/monetdb/o/SERVER_IP.dat','/srv/monetdb/o/MS_PORT.dat','/srv/monetdb/o/SERVER_PORT.dat','/srv/monetdb/o/APN.dat','/srv/monetdb/o/SGSN_SIG_IP.dat','/srv/monetdb/o/GGSN_SIG_IP.dat','/srv/monetdb/o/SGSN _USER_IP.dat','/srv/monetdb/o/GGSN_USER_IP.dat','/srv/monetdb/o/MCC.dat','/srv/monetdb/o/MNC.dat','/srv/monetdb/o/RAT.dat','/srv/monetdb/o/LAC.dat','/srv/monetdb/o/RAC.dat','/srv/monetdb/o/SAC.dat','/srv/monetdb/o/CI.dat','/srv/monetdb/o/L4_TYPE.dat','/srv/monetdb/o/L4_UL_THROUGHPUT.dat','/srv/monetdb/o/L4_DW_THROUGHPUT.dat','/srv/monetdb/o/L4_UL_GOODPUT.dat','/srv/monetdb/o/L4_DW_GOODPUT.dat','/srv/monetdb/o/NETWORK_UL_TRAFFIC.dat','/srv/mo netdb/o/NETWORK_DL_TRAFFIC.dat','/srv/monetdb/o/L4_UL_PACKETS.dat','/srv/monetdb/o/L4_DW_PACKETS.dat','/srv/monetdb/o/TCP_CONN_STATES.dat','/srv/monetdb/o/TCP_CONN_TIMES.dat','/srv/monetdb/o/TCP_CONN_2_FAILED_TIMES.dat','/srv/monetdb/o/TCP_CONN_3_FAILED_TIMES.dat','/srv/monetdb/o/FLAG.dat','/srv/monetdb/o/FLOW_SAMPLE_RATIO.dat','/srv/monetdb/o/TAC.dat','/srv/monetdb/o/ECI.dat','/srv/monetdb/o/TCP_SYN_TIME.dat','/srv/monetdb/o/TCP_SYN_TIME_MS EL.dat','/srv/monetdb/o/TCP_RTT_STEP1.dat','/srv/monetdb/o/TCP_UL_RETRANS_WITHPL.dat','/srv/monetdb/o/TCP_DW_RETRANS_WITHPL.dat','/srv/monetdb/o/TCP_UL_RETRANS_WITHOUTPL.dat','/srv/monetdb/o/TCP_DW_RETRANS_WITHOUTPL.dat','/srv/monetdb/o/TCP_UL_PACKAGES_WITHPL.dat','/srv/monetdb/o/TCP_DW_PACKAGES_WITHPL.dat','/srv/monetdb/o/TCP_UL_PACKAGES_WITHOUTPL.dat','/srv/monetdb/o/TCP_DW_PACKAGES_WITHOUTPL.dat','/srv/monetdb/o/UPPERLAYER_IP_UL_PACKETS.d at','/srv/monetdb/o/UPPERLAYER_IP_DL_PACKETS.dat','/srv/monetdb/o/DOWNLAYER_IP_UL_PACKETS.dat','/srv/monetdb/o/DOWNLAYER_IP_DL_PACKETS.dat','/srv/monetdb/o/UPPERLAYER_IP_UL_FRAGMENTS.dat','/srv/monetdb/o/UPPERLAYER_IP_DL_FRAGMENTS.dat','/srv/monetdb/o/DOWNLAYER_IP_UL_FRAGMENTS.dat','/srv/monetdb/o/DOWNLAYER_IP_DL_FRAGMENTS.dat','/srv/monetdb/o/SERVICE_DALAY_MSEC.dat','/srv/monetdb/o/VALID_TRANS_DURATION.dat','/srv/monetdb/o/AIR_PORT_DURATION .dat','/srv/monetdb/o/RADIO_CONN_TIMES.dat','/srv/monetdb/o/TCP_FST_SYN_DIRECTION.dat','/srv/monetdb/o/RAN_NE_ID.dat','/srv/monetdb/o/RAN_NE_USER_IP.dat','/srv/monetdb/o/TCP_STATES.dat','/srv/monetdb/o/TCP_RTT.dat','/srv/monetdb/o/TCP_UL_OUTOFSEQU.dat','/srv/monetdb/o/TCP_DW_OUTOFSEQU.dat','/srv/monetdb/o/TCP_WIN_SIZE.dat','/srv/monetdb/o/TCP_MSS.dat','/srv/monetdb/o/TRANS_TYPE.dat','/srv/monetdb/o/GET_NUM.dat','/srv/monetdb/o/GET_SUCCEED_NU M.dat','/srv/monetdb/o/GET_RETRANS_NUM.dat','/srv/monetdb/o/GET_TIMEOUT_NUM.dat','/srv/monetdb/o/POST_NUM.dat','/srv/monetdb/o/POST_SUCCEED_NUM.dat','/srv/monetdb/o/POST_RETRANS_NUM.dat','/srv/monetdb/o/POST_TO_NUM.dat','/srv/monetdb/o/SERVICE_SUCCEED_FLAG.dat','/srv/monetdb/o/INTERFACEID.dat','/srv/monetdb/o/HOMEMCC.dat','/srv/monetdb/o/HOMEMNC.dat','/srv/monetdb/o/HOMEPROVINCE.dat','/srv/monetdb/o/HOMECITY.dat','/srv/monetdb/o/HOST.dat','/ srv/monetdb/o/SERVICE_ID.dat','/srv/monetdb/o/CHARGING_CLASS.dat','/srv/monetdb/o/CHARGE_FLAG.dat','/srv/monetdb/o/Prepaid_Flag.dat'); +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mal | +=============================================================================================================================================================================================================+ | function user.s1_1{autoCommit=true}():void; | | X_2 := sql.mvc(); | | (X_3:bat[:oid,:lng] ,X_4:bat[:oid,:lng] ,X_5:bat[:oid,:int] ,X_6:bat[:oid,:lng] ,X_7:bat[:oid,:lng] ,X_8:bat[:oid,:bte] ,X_9:bat[:oid,:lng] ,X_10:bat[:oid,:sht] ,X_11:bat[:oid,:lng] ,X_12:bat[:oid,:s | : ht] ,X_13:bat[:oid,:int] ,X_14:bat[:oid,:int] ,X_15:bat[:oid,:sht] ,X_16:bat[:oid,:str] ,X_17:bat[:oid,:str] ,X_18:bat[:oid,:str] ,X_19:bat[:oid,:int] ,X_20:bat[:oid,:bte] ,X_21:bat[:oid,:bte] ,X_22:bat[ : : :oid,:str] ,X_23:bat[:oid,:str] ,X_24:bat[:oid,:int] ,X_25:bat[:oid,:int] ,X_26:bat[:oid,:str] ,X_27:bat[:oid,:str] ,X_28:bat[:oid,:str] ,X_29:bat[:oid,:str] ,X_30:bat[:oid,:str] ,X_31:bat[:oid,:str] ,X_ : : 32:bat[:oid,:str] ,X_33:bat[:oid,:bte] ,X_34:bat[:oid,:str] ,X_35:bat[:oid,:str] ,X_36:bat[:oid,:str] ,X_37:bat[:oid,:str] ,X_38:bat[:oid,:bte] ,X_39:bat[:oid,:lng] ,X_40:bat[:oid,:lng] ,X_41:bat[:oid,:l : : ng] ,X_42:bat[:oid,:lng] ,X_43:bat[:oid,:lng] ,X_44:bat[:oid,:lng] ,X_45:bat[:oid,:lng] ,X_46:bat[:oid,:lng] ,X_47:bat[:oid,:bte] ,X_48:bat[:oid,:int] ,X_49:bat[:oid,:int] ,X_50:bat[:oid,:int] ,X_51:bat[ : : :oid,:sht] ,X_52:bat[:oid,:int] ,X_53:bat[:oid,:str] ,X_54:bat[:oid,:str] ,X_55:bat[:oid,:int] ,X_56:bat[:oid,:sht] ,X_57:bat[:oid,:lng] ,X_58:bat[:oid,:lng] ,X_59:bat[:oid,:lng] ,X_60:bat[:oid,:lng] ,X_ : : 61:bat[:oid,:lng] ,X_62:bat[:oid,:lng] ,X_63:bat[:oid,:lng] ,X_64:bat[:oid,:lng] ,X_65:bat[:oid,:lng] ,X_66:bat[:oid,:lng] ,X_67:bat[:oid,:lng] ,X_68:bat[:oid,:lng] ,X_69:bat[:oid,:lng] ,X_70:bat[:oid,:l : : ng] ,X_71:bat[:oid,:lng] ,X_72:bat[:oid,:lng] ,X_73:bat[:oid,:lng] ,X_74:bat[:oid,:lng] ,X_75:bat[:oid,:lng] ,X_76:bat[:oid,:lng] ,X_77:bat[:oid,:lng] ,X_78:bat[:oid,:bte] ,X_79:bat[:oid,:lng] ,X_80:bat[ : : :oid,:str] ,X_81:bat[:oid,:bte] ,X_82:bat[:oid,:lng] ,X_83:bat[:oid,:lng] ,X_84:bat[:oid,:lng] ,X_85:bat[:oid,:int] ,X_86:bat[:oid,:int] ,X_87:bat[:oid,:int] ,X_88:bat[:oid,:int] ,X_89:bat[:oid,:int] ,X_ : : 90:bat[:oid,:int] ,X_91:bat[:oid,:int] ,X_92:bat[:oid,:int] ,X_93:bat[:oid,:int] ,X_94:bat[:oid,:int] ,X_95:bat[:oid,:int] ,X_96:bat[:oid,:bte] ,X_97:bat[:oid,:bte] ,X_98:bat[:oid,:str] ,X_99:bat[:oid,:s : : tr] ,X_100:bat[:oid,:lng] ,X_101:bat[:oid,:lng] ,X_102:bat[:oid,:str] ,X_103:bat[:oid,:lng] ,X_104:bat[:oid,:lng] ,X_105:bat[:oid,:bte] ,X_106:bat[:oid,:bte] ) := sql.importTable("sys":str,"detail_ufdr_o : : ther":str,"/srv/monetdb/o/SID.dat":str,"/srv/monetdb/o/PROBEID.dat":str,"/srv/monetdb/o/GROUPID.dat":str,"/srv/monetdb/o/GGSN_ID.dat":str,"/srv/monetdb/o/SGSN_ID.dat":str,"/srv/monetdb/o/SESSION_INDICATO : : R.dat":str,"/srv/monetdb/o/BEGIN_TIME.dat":str,"/srv/monetdb/o/BEGIN_TIME_MSEL.dat":str,"/srv/monetdb/o/END_TIME.dat":str,"/srv/monetdb/o/END_TIME_MSEL.dat":str,"/srv/monetdb/o/PROT_CATEGORY.dat":str,"/s : : rv/monetdb/o/PROT_TYPE.dat":str,"/srv/monetdb/o/L7_CARRIER_PROT.dat":str,"/srv/monetdb/o/MSISDN.dat":str,"/srv/monetdb/o/IMSI.dat":str,"/srv/monetdb/o/IMEI.dat":str,"/srv/monetdb/o/ENCRYPT_VERSION.dat":s : : tr,"/srv/monetdb/o/ROAMING_TYPE.dat":str,"/srv/monetdb/o/ROAM_DIRECTION.dat":str,"/srv/monetdb/o/MS_IP.dat":str,"/srv/monetdb/o/SERVER_IP.dat":str,"/srv/monetdb/o/MS_PORT.dat":str,"/srv/monetdb/o/SERVER_ : : PORT.dat":str,"/srv/monetdb/o/APN.dat":str,"/srv/monetdb/o/SGSN_SIG_IP.dat":str,"/srv/monetdb/o/GGSN_SIG_IP.dat":str,"/srv/monetdb/o/SGSN_USER_IP.dat":str,"/srv/monetdb/o/GGSN_USER_IP.dat":str,"/srv/mone : : tdb/o/MCC.dat":str,"/srv/monetdb/o/MNC.dat":str,"/srv/monetdb/o/RAT.dat":str,"/srv/monetdb/o/LAC.dat":str,"/srv/monetdb/o/RAC.dat":str,"/srv/monetdb/o/SAC.dat":str,"/srv/monetdb/o/CI.dat":str,"/srv/monet : : db/o/L4_TYPE.dat":str,"/srv/monetdb/o/L4_UL_THROUGHPUT.dat":str,"/srv/monetdb/o/L4_DW_THROUGHPUT.dat":str,"/srv/monetdb/o/L4_UL_GOODPUT.dat":str,"/srv/monetdb/o/L4_DW_GOODPUT.dat":str,"/srv/monetdb/o/NET : : WORK_UL_TRAFFIC.dat":str,"/srv/monetdb/o/NETWORK_DL_TRAFFIC.dat":str,"/srv/monetdb/o/L4_UL_PACKETS.dat":str,"/srv/monetdb/o/L4_DW_PACKETS.dat":str,"/srv/monetdb/o/TCP_CONN_STATES.dat":str,"/srv/monetdb/o : : /TCP_CONN_TIMES.dat":str,"/srv/monetdb/o/TCP_CONN_2_FAILED_TIMES.dat":str,"/srv/monetdb/o/TCP_CONN_3_FAILED_TIMES.dat":str,"/srv/monetdb/o/FLAG.dat":str,"/srv/monetdb/o/FLOW_SAMPLE_RATIO.dat":str,"/srv/m : : onetdb/o/TAC.dat":str,"/srv/monetdb/o/ECI.dat":str,"/srv/monetdb/o/TCP_SYN_TIME.dat":str,"/srv/monetdb/o/TCP_SYN_TIME_MSEL.dat":str,"/srv/monetdb/o/TCP_RTT_STEP1.dat":str,"/srv/monetdb/o/TCP_UL_RETRANS_W : : ITHPL.dat":str,"/srv/monetdb/o/TCP_DW_RETRANS_WITHPL.dat":str,"/srv/monetdb/o/TCP_UL_RETRANS_WITHOUTPL.dat":str,"/srv/monetdb/o/TCP_DW_RETRANS_WITHOUTPL.dat":str,"/srv/monetdb/o/TCP_UL_PACKAGES_WITHPL.da : : t":str,"/srv/monetdb/o/TCP_DW_PACKAGES_WITHPL.dat":str,"/srv/monetdb/o/TCP_UL_PACKAGES_WITHOUTPL.dat":str,"/srv/monetdb/o/TCP_DW_PACKAGES_WITHOUTPL.dat":str,"/srv/monetdb/o/UPPERLAYER_IP_UL_PACKETS.dat": : : str,"/srv/monetdb/o/UPPERLAYER_IP_DL_PACKETS.dat":str,"/srv/monetdb/o/DOWNLAYER_IP_UL_PACKETS.dat":str,"/srv/monetdb/o/DOWNLAYER_IP_DL_PACKETS.dat":str,"/srv/monetdb/o/UPPERLAYER_IP_UL_FRAGMENTS.dat":str : : ,"/srv/monetdb/o/UPPERLAYER_IP_DL_FRAGMENTS.dat":str,"/srv/monetdb/o/DOWNLAYER_IP_UL_FRAGMENTS.dat":str,"/srv/monetdb/o/DOWNLAYER_IP_DL_FRAGMENTS.dat":str,"/srv/monetdb/o/SERVICE_DALAY_MSEC.dat":str,"/sr : : v/monetdb/o/VALID_TRANS_DURATION.dat":str,"/srv/monetdb/o/AIR_PORT_DURATION.dat":str,"/srv/monetdb/o/RADIO_CONN_TIMES.dat":str,"/srv/monetdb/o/TCP_FST_SYN_DIRECTION.dat":str,"/srv/monetdb/o/RAN_NE_ID.dat : : ":str,"/srv/monetdb/o/RAN_NE_USER_IP.dat":str,"/srv/monetdb/o/TCP_STATES.dat":str,"/srv/monetdb/o/TCP_RTT.dat":str,"/srv/monetdb/o/TCP_UL_OUTOFSEQU.dat":str,"/srv/monetdb/o/TCP_DW_OUTOFSEQU.dat":str,"/sr : : v/monetdb/o/TCP_WIN_SIZE.dat":str,"/srv/monetdb/o/TCP_MSS.dat":str,"/srv/monetdb/o/TRANS_TYPE.dat":str,"/srv/monetdb/o/GET_NUM.dat":str,"/srv/monetdb/o/GET_SUCCEED_NUM.dat":str,"/srv/monetdb/o/GET_RETRAN : : S_NUM.dat":str,"/srv/monetdb/o/GET_TIMEOUT_NUM.dat":str,"/srv/monetdb/o/POST_NUM.dat":str,"/srv/monetdb/o/POST_SUCCEED_NUM.dat":str,"/srv/monetdb/o/POST_RETRANS_NUM.dat":str,"/srv/monetdb/o/POST_TO_NUM.d : : at":str,"/srv/monetdb/o/SERVICE_SUCCEED_FLAG.dat":str,"/srv/monetdb/o/INTERFACEID.dat":str,"/srv/monetdb/o/HOMEMCC.dat":str,"/srv/monetdb/o/HOMEMNC.dat":str,"/srv/monetdb/o/HOMEPROVINCE.dat":str,"/srv/mo : : netdb/o/HOMECITY.dat":str,"/srv/monetdb/o/HOST.dat":str,"/srv/monetdb/o/SERVICE_ID.dat":str,"/srv/monetdb/o/CHARGING_CLASS.dat":str,"/srv/monetdb/o/CHARGE_FLAG.dat":str,"/srv/monetdb/o/Prepaid_Flag.dat": : : str); : | X_109 := sql.append(X_2,"sys","detail_ufdr_other","sid",X_3); | | X_112 := sql.append(X_109,"sys","detail_ufdr_other","probeid",X_4); | | X_114 := sql.append(X_112,"sys","detail_ufdr_other","groupid",X_5); | | X_116 := sql.append(X_114,"sys","detail_ufdr_other","ggsn_id",X_6); | | X_118 := sql.append(X_116,"sys","detail_ufdr_other","sgsn_id",X_7); | | X_120 := sql.append(X_118,"sys","detail_ufdr_other","session_indicator",X_8); | | X_123 := sql.append(X_120,"sys","detail_ufdr_other","begin_time",X_9); | | X_126 := sql.append(X_123,"sys","detail_ufdr_other","begin_time_msel",X_10); | | X_128 := sql.append(X_126,"sys","detail_ufdr_other","end_time",X_11); | | X_130 := sql.append(X_128,"sys","detail_ufdr_other","end_time_msel",X_12); | | X_132 := sql.append(X_130,"sys","detail_ufdr_other","prot_category",X_13); | | X_135 := sql.append(X_132,"sys","detail_ufdr_other","prot_type",X_14); | | X_137 := sql.append(X_135,"sys","detail_ufdr_other","l7_carrier_prot",X_15); | | X_140 := sql.append(X_137,"sys","detail_ufdr_other","msisdn",X_16); | | X_142 := sql.append(X_140,"sys","detail_ufdr_other","imsi",X_17); | | X_144 := sql.append(X_142,"sys","detail_ufdr_other","imei",X_18); | | X_146 := sql.append(X_144,"sys","detail_ufdr_other","encrypt_version",X_19); | | X_149 := sql.append(X_146,"sys","detail_ufdr_other","roaming_type",X_20); | | X_151 := sql.append(X_149,"sys","detail_ufdr_other","roam_direction",X_21); | | X_154 := sql.append(X_151,"sys","detail_ufdr_other","ms_ip",X_22); | | X_156 := sql.append(X_154,"sys","detail_ufdr_other","server_ip",X_23); | | X_158 := sql.append(X_156,"sys","detail_ufdr_other","ms_port",X_24); | | X_161 := sql.append(X_158,"sys","detail_ufdr_other","server_port",X_25); | | X_163 := sql.append(X_161,"sys","detail_ufdr_other","apn",X_26); | | X_165 := sql.append(X_163,"sys","detail_ufdr_other","sgsn_sig_ip",X_27); | | X_168 := sql.append(X_165,"sys","detail_ufdr_other","ggsn_sig_ip",X_28); | | X_170 := sql.append(X_168,"sys","detail_ufdr_other","sgsn_user_ip",X_29); | | X_172 := sql.append(X_170,"sys","detail_ufdr_other","ggsn_user_ip",X_30); | | X_175 := sql.append(X_172,"sys","detail_ufdr_other","mcc",X_31); | | X_177 := sql.append(X_175,"sys","detail_ufdr_other","mnc",X_32); | | X_179 := sql.append(X_177,"sys","detail_ufdr_other","rat",X_33); | | X_182 := sql.append(X_179,"sys","detail_ufdr_other","lac",X_34); | | X_184 := sql.append(X_182,"sys","detail_ufdr_other","rac",X_35); | | X_187 := sql.append(X_184,"sys","detail_ufdr_other","sac",X_36); | | X_189 := sql.append(X_187,"sys","detail_ufdr_other","ci",X_37); | | X_191 := sql.append(X_189,"sys","detail_ufdr_other","l4_type",X_38); | | X_193 := sql.append(X_191,"sys","detail_ufdr_other","l4_ul_throughput",X_39); | | X_196 := sql.append(X_193,"sys","detail_ufdr_other","l4_dw_throughput",X_40); | | X_198 := sql.append(X_196,"sys","detail_ufdr_other","l4_ul_goodput",X_41); | | X_201 := sql.append(X_198,"sys","detail_ufdr_other","l4_dw_goodput",X_42); | | X_203 := sql.append(X_201,"sys","detail_ufdr_other","network_ul_traffic",X_43); | | X_205 := sql.append(X_203,"sys","detail_ufdr_other","network_dl_traffic",X_44); | | X_207 := sql.append(X_205,"sys","detail_ufdr_other","l4_ul_packets",X_45); | | X_210 := sql.append(X_207,"sys","detail_ufdr_other","l4_dw_packets",X_46); | | X_213 := sql.append(X_210,"sys","detail_ufdr_other","tcp_conn_states",X_47); | | X_215 := sql.append(X_213,"sys","detail_ufdr_other","tcp_conn_times",X_48); | | X_217 := sql.append(X_215,"sys","detail_ufdr_other","tcp_conn_2_failed_times",X_49); | | X_219 := sql.append(X_217,"sys","detail_ufdr_other","tcp_conn_3_failed_times",X_50); | | X_221 := sql.append(X_219,"sys","detail_ufdr_other","flag",X_51); | | X_224 := sql.append(X_221,"sys","detail_ufdr_other","flow_sample_ratio",X_52); | | X_227 := sql.append(X_224,"sys","detail_ufdr_other","tac",X_53); | | X_229 := sql.append(X_227,"sys","detail_ufdr_other","eci",X_54); | | X_231 := sql.append(X_229,"sys","detail_ufdr_other","tcp_syn_time",X_55); | | X_233 := sql.append(X_231,"sys","detail_ufdr_other","tcp_syn_time_msel",X_56); | | X_235 := sql.append(X_233,"sys","detail_ufdr_other","tcp_rtt_step1",X_57); | | X_239 := sql.append(X_235,"sys","detail_ufdr_other","tcp_ul_retrans_withpl",X_58); | | X_241 := sql.append(X_239,"sys","detail_ufdr_other","tcp_dw_retrans_withpl",X_59); | | X_243 := sql.append(X_241,"sys","detail_ufdr_other","tcp_ul_retrans_withoutpl",X_60); | | X_245 := sql.append(X_243,"sys","detail_ufdr_other","tcp_dw_retrans_withoutpl",X_61); | | X_247 := sql.append(X_245,"sys","detail_ufdr_other","tcp_ul_packages_withpl",X_62); | | X_249 := sql.append(X_247,"sys","detail_ufdr_other","tcp_dw_packages_withpl",X_63); | | X_253 := sql.append(X_249,"sys","detail_ufdr_other","tcp_ul_packages_withoutpl",X_64); | | X_255 := sql.append(X_253,"sys","detail_ufdr_other","tcp_dw_packages_withoutpl",X_65); | | X_257 := sql.append(X_255,"sys","detail_ufdr_other","upperlayer_ip_ul_packets",X_66); | | X_259 := sql.append(X_257,"sys","detail_ufdr_other","upperlayer_ip_dl_packets",X_67); | | X_261 := sql.append(X_259,"sys","detail_ufdr_other","downlayer_ip_ul_packets",X_68); | | X_264 := sql.append(X_261,"sys","detail_ufdr_other","downlayer_ip_dl_packets",X_69); | | X_267 := sql.append(X_264,"sys","detail_ufdr_other","upperlayer_ip_ul_fragments",X_70); | | X_269 := sql.append(X_267,"sys","detail_ufdr_other","upperlayer_ip_dl_fragments",X_71); | | X_271 := sql.append(X_269,"sys","detail_ufdr_other","downlayer_ip_ul_fragments",X_72); | | X_273 := sql.append(X_271,"sys","detail_ufdr_other","downlayer_ip_dl_fragments",X_73); | | X_275 := sql.append(X_273,"sys","detail_ufdr_other","service_dalay_msec",X_74); | | X_278 := sql.append(X_275,"sys","detail_ufdr_other","valid_trans_duration",X_75); | | X_281 := sql.append(X_278,"sys","detail_ufdr_other","air_port_duration",X_76); | | X_283 := sql.append(X_281,"sys","detail_ufdr_other","radio_conn_times",X_77); | | X_285 := sql.append(X_283,"sys","detail_ufdr_other","tcp_fst_syn_direction",X_78); | | X_287 := sql.append(X_285,"sys","detail_ufdr_other","ran_ne_id",X_79); | | X_290 := sql.append(X_287,"sys","detail_ufdr_other","ran_ne_user_ip",X_80); | | X_292 := sql.append(X_290,"sys","detail_ufdr_other","tcp_states",X_81); | | X_295 := sql.append(X_292,"sys","detail_ufdr_other","tcp_rtt",X_82); | | X_297 := sql.append(X_295,"sys","detail_ufdr_other","tcp_ul_outofsequ",X_83); | | X_299 := sql.append(X_297,"sys","detail_ufdr_other","tcp_dw_outofsequ",X_84); | | X_301 := sql.append(X_299,"sys","detail_ufdr_other","tcp_win_size",X_85); | | X_304 := sql.append(X_301,"sys","detail_ufdr_other","tcp_mss",X_86); | | X_306 := sql.append(X_304,"sys","detail_ufdr_other","trans_type",X_87); | | X_309 := sql.append(X_306,"sys","detail_ufdr_other","get_num",X_88); | | X_311 := sql.append(X_309,"sys","detail_ufdr_other","get_succeed_num",X_89); | | X_313 := sql.append(X_311,"sys","detail_ufdr_other","get_retrans_num",X_90); | | X_316 := sql.append(X_313,"sys","detail_ufdr_other","get_timeout_num",X_91); | | X_318 := sql.append(X_316,"sys","detail_ufdr_other","post_num",X_92); | | X_320 := sql.append(X_318,"sys","detail_ufdr_other","post_succeed_num",X_93); | | X_323 := sql.append(X_320,"sys","detail_ufdr_other","post_retrans_num",X_94); | | X_325 := sql.append(X_323,"sys","detail_ufdr_other","post_to_num",X_95); | | X_327 := sql.append(X_325,"sys","detail_ufdr_other","service_succeed_flag",X_96); | | X_330 := sql.append(X_327,"sys","detail_ufdr_other","interfaceid",X_97); | | X_332 := sql.append(X_330,"sys","detail_ufdr_other","homemcc",X_98); | | X_334 := sql.append(X_332,"sys","detail_ufdr_other","homemnc",X_99); | | X_337 := sql.append(X_334,"sys","detail_ufdr_other","homeprovince",X_100); | | X_339 := sql.append(X_337,"sys","detail_ufdr_other","homecity",X_101); | | X_342 := sql.append(X_339,"sys","detail_ufdr_other","host",X_102); | | X_344 := sql.append(X_342,"sys","detail_ufdr_other","service_id",X_103); | | X_346 := sql.append(X_344,"sys","detail_ufdr_other","charging_class",X_104); | | X_348 := sql.append(X_346,"sys":str,"detail_ufdr_other","charge_flag",X_105); | | X_351 := sql.append(X_348,"sys":str,"detail_ufdr_other","prepaid_flag",X_106); | | X_352 := aggr.count(X_106); | | sql.affectedRows(X_351,X_352,""); | | end s1_1; | +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! *****************************************************************************************

2 1

Re: MonetDB: DVframework - For 64-bit integers we need to use type l...
by Ying Zhang 06 May '13

06 May '13

Stefan, FYI. Your fixes work. DVframework now also compiles on Lion :) Thanks! Jennie On May 06, 2013, at 19:22, Stefan Manegold wrote: > Changeset: 32b9bc615b3e for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=32b9bc615b3e > Modified Files: > sql/backends/monet5/vaults/mseed.c > Branch: DVframework > Log Message: > > For 64-bit integers we need to use type lng, not type long. > MSRecord.samplecnt is defined as int64_t, i.e., a 64-bit integer; > hence, in MonetDB, we use type lng, not type long. > > PLEASE NOTE: > Ceterum censeo type long MUST NOT be used as it is everything but portable; > for instance, on our beloved Windows platforms, long is always 32-bit, > even on 64-bit systems. > > > diffs (30 lines): > > diff --git a/sql/backends/monet5/vaults/mseed.c b/sql/backends/monet5/vaults/mseed.c > --- a/sql/backends/monet5/vaults/mseed.c > +++ b/sql/backends/monet5/vaults/mseed.c > @@ -116,7 +116,7 @@ str SQLstatementIntern(Client c, str *ex > > #define QRYinsertI "INSERT INTO mseedCatalog(mseed, seqno, dataquality, network, \ > station, location, channel, starttime , samplerate, sampleindex, samplecnt, sampletype, minval,maxval) \ > - VALUES(%d, %d,'%c','%s', '%s','%s','%s','%s',%f,%d,%d,'%s',%d,%d);" > + VALUES(%d, %d,'%c','%s', '%s','%s','%s','%s',%f,%d,"LLFMT",'%s',%d,%d);" > > str > MseedImport(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci) > @@ -174,7 +174,7 @@ MseedImport(Client cntxt, MalBlkPtr mb, > } > t= MS_HPTIME2EPOCH(msr->starttime); > tm = gmtime(&t); > - snprintf(starttime,BUFSIZ,"%d-%02d-%02d %02d:%02d:%02d.%06ld", tm->tm_year +(tm->tm_year > 80?1900:2000), tm->tm_mon+1,tm->tm_mday, tm->tm_hour, tm->tm_min,tm->tm_sec, msr->starttime % HPTMODULUS); > + snprintf(starttime,BUFSIZ,"%d-%02d-%02d %02d:%02d:%02d.%06d", tm->tm_year +(tm->tm_year > 80?1900:2000), tm->tm_mon+1,tm->tm_mday, tm->tm_hour, tm->tm_min,tm->tm_sec, (int) (msr->starttime % HPTMODULUS)); > /* collect the statistics */ > switch(msr->sampletype){ > case 'i': > @@ -185,7 +185,7 @@ MseedImport(Client cntxt, MalBlkPtr mb, > if ( imax < ((int*) msr->datasamples)[j]) imax = ((int*) msr->datasamples)[j]; > } > snprintf(buf,BUFSIZ,QRYinsertI, *vid, msr->sequence_number,msr->dataquality,msr->network, msr->station, msr->location, msr->channel, > - starttime,msr->samprate, sampleindex,msr->samplecnt,"int",imin,imax); > + starttime,msr->samprate, sampleindex,(lng)msr->samplecnt,"int",imin,imax); > break; > case 'a': case 'f': case 'd': > default: > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

1 0

Announcement: New Feb2013-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 29 Apr '13

29 Apr '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>.

1 0

Fix to the typing issue in MAL generation
by Masood Mortazavi 22 Apr '13

22 Apr '13

Attached, please find a fix for the typing issue in MAL generation, reported in http://bugs.monetdb.org/show_bug.cgi?id=3252 The fix was produced on: Feb2013_SP1_release 47060:fc0c9b319d22 We have not done thorough, release testing of this fix but it appears correct, and we have used it successfully. Regards, Masood P.S. By the way, what's the proper way to report or submit minor fixes like this?

1 0

Help:How can i start MonetDB development quickly
by 高小猛 22 Apr '13

22 Apr '13

sir/mam, I am a newbie to MonetDB development . I have read some paper and user guide about MonetDB. And i download the sourse code and want to modify the code, especially the storage part, to test my idea. However, the project is too large to understand quickly. anyone can give me suggestion ? Whether are there documents or structure charts about the project that make newbie to understand quickly. thanks. Andy

2 3

Re: MonetDB: default - Cross-Compile JDBC connector to Java 1.5 and up
by Fabian Groffen 17 Apr '13

17 Apr '13

On 17-04-2013 14:39:48 +0200, Hannes Muehleisen wrote: > Changeset: c81ba7ca1e8c for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=c81ba7ca1e8c > Modified Files: > java/build.xml > Branch: default > Log Message: > > Cross-Compile JDBC connector to Java 1.5 and up IMO this is wrong, please undo it. -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home

1 0

Re: MonetDB: Feb2013 - add temporary to the non-reserved keywords
by Sjoerd Mullender 12 Apr '13

12 Apr '13

On 2013-04-12 21:35, Niels Nes wrote: > Changeset: 2d22867fb284 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=2d22867fb284 > Modified Files: > sql/server/sql_parser.y > Branch: Feb2013 > Log Message: > > add temporary to the non-reserved keywords > > > diffs (11 lines): > > diff --git a/sql/server/sql_parser.y b/sql/server/sql_parser.y > --- a/sql/server/sql_parser.y > +++ b/sql/server/sql_parser.y > @@ -4721,6 +4721,7 @@ non_reserved_word: > | ACCORDING { $$ = sa_strdup(SA, "according"); } > | URI { $$ = sa_strdup(SA, "uri"); } > | FILTER { $$ = sa_strdup(SA, "filter"); } > +| TEMPORARY { $$ = sa_strdup(SA, "temporary"); } > ; > > name_commalist: > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > A Changelog message would not be amiss here. -- Sjoerd Mullender

1 0

Re: MonetDB: default - fixed problem unaligned avg/counts bats.
by Sjoerd Mullender 10 Apr '13

10 Apr '13

Was this the problem we discussed yesterday? If so, shouldn't this be on the Feb2013 branch? On 2013-04-10 11:00, Niels Nes wrote: > Changeset: 16f6734d1286 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=16f6734d1286 > Modified Files: > monetdb5/optimizer/opt_mergetable.c > Branch: default > Log Message: > > fixed problem unaligned avg/counts bats. > > > diffs (28 lines): > > diff --git a/monetdb5/optimizer/opt_mergetable.c b/monetdb5/optimizer/opt_mergetable.c > --- a/monetdb5/optimizer/opt_mergetable.c > +++ b/monetdb5/optimizer/opt_mergetable.c > @@ -633,7 +633,7 @@ mat_aggr(MalBlkPtr mb, InstrPtr p, mat_t > pushInstruction(mb, u); > > /* Filter empty partitions */ > - if (getModuleId(p) == aggrRef) { > + if (getModuleId(p) == aggrRef && !isAvg) { > s = newInstruction(mb,ASSIGNsymbol); > setModuleId(s, algebraRef); > setFunctionId(s, selectNotNilRef); > @@ -710,6 +710,15 @@ mat_aggr(MalBlkPtr mb, InstrPtr p, mat_t > pushInstruction(mb, w); > > r = w; > + > + /* filter nils */ > + s = newInstruction(mb,ASSIGNsymbol); > + setModuleId(s, algebraRef); > + setFunctionId(s, selectNotNilRef); > + getArg(s,0) = newTmpVariable(mb, battp); > + s = pushArgument(mb, s, getArg(r,0)); > + pushInstruction(mb, s); > + r = s; > } > > s = newInstruction(mb,ASSIGNsymbol); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender

1 0

Re: developers-list Digest, Vol 8, Issue 1
by Meenesh sharma 01 Apr '13

01 Apr '13

this is too much , am not getting where to read for answer especially for mine questions ... where where where ????????? do reply for god sake..... On Mon, Apr 1, 2013 at 3:30 PM, <developers-list-request(a)monetdb.org> wrote: > Send developers-list mailing list submissions to > developers-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.monetdb.org/mailman/listinfo/developers-list > or, via email, send a message with subject or body 'help' to > developers-list-request(a)monetdb.org > > You can reach the person managing the list at > developers-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of developers-list digest..." > > Today's Topics: > > 1. Help:How can i start MonetDB development quickly (???) > > > ---------- Forwarded message ---------- > From: 高小猛 <xmgtih(a)gmail.com> > To: developers-list(a)monetdb.org > Cc: > Date: Mon, 1 Apr 2013 16:09:13 +0800 > Subject: Help:How can i start MonetDB development quickly > sir/mam, > I am a newbie to MonetDB development . I have read some paper and user > guide about MonetDB. And i download the sourse code and want to modify the > code, especially the storage part, to test my idea. However, the project > is too large to understand quickly. anyone can give me suggestion ? > Whether are there documents or structure charts about the project that > make newbie to understand quickly. thanks. > > > Andy > > _______________________________________________ > developers-list mailing list > developers-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/developers-list > > -- " HAVE A NICE DAY"

3 2

Re: developers-list Digest, Vol 7, Issue 20
by Meenesh sharma 30 Mar '13

30 Mar '13

sir/mam, I unable to read the contents therefore neither get my query's answer nor able to open it .... . . .. do send way to open it . On Fri, Mar 29, 2013 at 4:30 PM, <developers-list-request(a)monetdb.org>wrote: > Send developers-list mailing list submissions to > developers-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.monetdb.org/mailman/listinfo/developers-list > or, via email, send a message with subject or body 'help' to > developers-list-request(a)monetdb.org > > You can reach the person managing the list at > developers-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of developers-list digest..." > > > Today's Topics: > > 1. Re: how to speed up the loading of data into a table (Niels Nes) > 2. Re: how to speed up the loading of data into a table (Niels Nes) > 3. Re: how to speed up the loading of data into a table > (Stefan Manegold) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 28 Mar 2013 20:17:08 +0100 > From: Niels Nes <Niels.Nes(a)cwi.nl> > To: "Communication channel for developers of the MonetDB suite." > <developers-list(a)monetdb.org> > Subject: Re: how to speed up the loading of data into a table > Message-ID: <20130328191708.GB13470(a)niels.cwi.nl> > Content-Type: text/plain; charset="utf-8" > > On Thu, Mar 28, 2013 at 06:41:08PM +0000, Bouslah Ahmed wrote: > > Good afternoon, > > > > I have 2 questions : > > - 1st question > > I have a csv file with 300000000 million lines. > > > > when I load it in a MonetDB table, it takes more than 16 hours, whereas > > in Oracle it only takes 15 minutes. Where is the problem? > > I use mclient as follows :? > > mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO > > LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\ > > LINEORDER.tbl > > > > ?for information I use : > > > > MonetDB5 > > Windows > > Memory : 16 Go > > There are two constraints : primary key and foreign key on the table > > prior to loading data? > > > > 2nd question : > > how inner joins are made in MongoDB? > typo ? > > Niels > > > > cordialy. > > > _______________________________________________ > > developers-list mailing list > > developers-list(a)monetdb.org > > http://mail.monetdb.org/mailman/listinfo/developers-list > > > -- > Niels Nes, Centrum Wiskunde & Informatica (CWI) > Science Park 123, 1098 XG Amsterdam, The Netherlands > room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl > url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl >

1 0

Re: How would I add a new directory to /monetdb5/extras/
by Stefan Manegold 28 Mar '13

28 Mar '13

It's a shared library that will only be loaded once required at run time. Just say yes when gab asks you if the breakpoint should only be set once the library/code is loaded. -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Reply message ----- From: "Chi-Young Ku" <chi(a)querymetrica.com> To: "Communication channel for developers of the MonetDB suite." <developers-list(a)monetdb.org> Subject: How would I add a new directory to /monetdb5/extras/ Date: Wed, Mar 27, 2013 19:08 Hi Babis, Thanks a lot for your help! I am able to get the foo directory compiled and built lib_foo.la. But, when I attatch GDB to the mserver5 process and try to set a breakpoint in foo.test(), I got an error saying foo.test() is not loaded. Have I missed anything? Chi ________________________________ From: Charalampos Nikolaou <charnik(a)di.uoa.gr> To: Communication channel for developers of the MonetDB suite. <developers-list(a)monetdb.org> Sent: Thursday, March 21, 2013 3:04 PM Subject: Re: How would I add a new directory to /monetdb5/extras/ Hi, you have to edit monetdb5/extras/Makefile.ag appending to variable SUBDIRS the name of the directory you want to add. For example, if you want to add directory "foo", given that the value of SUBDIRS was HAVE_RAPTOR?rdf HAVE_SPHINXCLIENT?sphinx HAVE_JAQL?jaql mal_optimizer_template the new value for SUBDIRS should be HAVE_RAPTOR?rdf HAVE_SPHINXCLIENT?sphinx HAVE_JAQL?jaql mal_optimizer_template foo Of course, this way you should run de-bootstrap and bootstrap again. Notice, that you have to specify also another Makefile.ag in monetdb5/extras/foo for taking care of the compilation of your code. However, I don't see any connection of your question to the message you replied to. Babis On Thu, Mar 21, 2013 at 7:35 PM, Chi-Young Ku <chi(a)querymetrica.com> wrote: > Hi, > > I would like to add a new sub-directory to /monetdb5/extras/ so that: > > 1. the configure process generates Makefiles for it and > > 2. it gets built when "make" is invoke in the top source directory. > > Thanks in advance. > > Chi > > > ________________________________ > From: Stephen P. Morgan <stephen_p_morgan(a)sbcglobal.net> > To: developers-list(a)monetdb.org > Sent: Thursday, March 21, 2013 12:06 PM > Subject: Question in re remote.exec() > > I have a question about remote.exec(). Let's say I have a user function, > s1_1 that takes a BAT as a parameter and returns a BAT. Its signature might > be: > > function user.s1_1(in:bat[:oid,:int]):bat[:oid,:int]; > > How can I call s1_1 using remote.exec to pass in a BAT and to retrieve the > result? I want to do something like > > X_2 := remote.exec(conn, "user", "s1_1", X_1); > > where X_1 and X_2 are both :bat[:oid,:int]. But that doesn't seem to work. > > Thanks, > > Steve Morgan > > _______________________________________________ > developers-list mailing list > developers-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/developers-list > > > > _______________________________________________ > developers-list mailing list > developers-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/developers-list > _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Re: Question about : LOAD DATA
by Stefan Manegold 28 Mar '13

28 Mar '13

Please see my answer I sent to your identical question yesterday. Stefan -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Reply message ----- From: "Bouslah Ahmed" <a.bouslah(a)yahoo.fr> To: "developers-list(a)monetdb.org" <developers-list(a)monetdb.org> Subject: Question about : LOAD DATA Date: Wed, Mar 27, 2013 19:44 Good afternoon, I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows :? mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl for information I use : MonetDB5 Windows Memory : 16 Go There are two constraints : primary key and foreign key on the table prior to loading data? cordialy.

1 0

Question about : LOAD DATA
by Bouslah Ahmed 27 Mar '13

27 Mar '13

Good afternoon, I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows :? mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl for information I use : MonetDB5 Windows Memory : 16 Go There are two constraints : primary key and foreign key on the table prior to loading data? cordialy.

1 0

Question in re remote.exec()
by Stephen P. Morgan 27 Mar '13

27 Mar '13

I have a question about remote.exec(). Let's say I have a user function, s1_1 that takes a BAT as a parameter and returns a BAT. Its signature might be: function user.s1_1(in:bat[:oid,:int]):bat[:oid,:int]; How can I call s1_1 using remote.exec to pass in a BAT and to retrieve the result? I want to do something like X_2 := remote.exec(conn, "user", "s1_1", X_1); where X_1 and X_2 are both :bat[:oid,:int]. But that doesn't seem to work. Thanks, Steve Morgan

3 3

Question
by Bouslah Ahmed 26 Mar '13

26 Mar '13

I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows : mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl cordially.

2 1

reply soon
by Meenesh sharma 24 Mar '13

24 Mar '13

sir, please tell can i add coding for variant stochastic cracking ....so, that i can analysis it with work done previously . . ... do help me -- " HAVE A NICE DAY"

1 0

do reply
by Meenesh sharma 22 Mar '13

22 Mar '13

i am doing my dissertation work on variant stochastic cracking .....so, i want to know which tool should i use to implement and then analysis it for various workload . do reply sir . . . i will very very thank ful to you .. . .please guide me ............. -- " HAVE A NICE DAY"

2 1

Re: MonetDB: default - approved output
by Sjoerd Mullender 20 Mar '13

20 Mar '13

Shouldn't this go to the Feb2013 branch? On 2013-03-20 15:55, Niels Nes wrote: > Changeset: 59ae06013de6 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=59ae06013de6 > Modified Files: > sql/backends/monet5/rel_bin.c > sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > Branch: default > Log Message: > > approved output > fixed bug view-view, bug 3245 > > > diffs (281 lines): > > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -2340,14 +2340,21 @@ rel2bin_select( mvc *sql, sql_rel *rel, > if (!sub) > return NULL; > sub = row2cols(sql, sub); > - } else { > + } > + if (!sub && !predicate) > predicate = rel2bin_predicate(sql); > - } > + else if (!predicate) > + predicate = stmt_const(sql->sa, bin_first_column(sql->sa, sub), stmt_bool(sql->sa, 1)); > if (!rel->exps->h) { > if (sub) > return sub; > return predicate; > } > + if (!sub && predicate) { > + list *l = sa_list(sql->sa); > + append(l, predicate); > + sub = stmt_list(sql->sa, l); > + } > /* handle possible index lookups */ > /* expressions are in index order ! */ > if (sub && (en = rel->exps->h) != NULL) { > @@ -2368,10 +2375,8 @@ rel2bin_select( mvc *sql, sql_rel *rel, > assert(0); > return NULL; > } > - if (s->nrcols == 0){ > - if (!predicate) > - predicate = rel2bin_predicate(sql); > - predicate = stmt_uselect(sql->sa, predicate, s, cmp_equal, NULL); > + if (s->nrcols == 0){ > + sel = stmt_uselect(sql->sa, predicate, s, cmp_equal, sel); > } else if (e->type != e_cmp) { > sel = stmt_uselect(sql->sa, s, stmt_bool(sql->sa, 1), cmp_equal, NULL); > } else { > @@ -2379,14 +2384,6 @@ rel2bin_select( mvc *sql, sql_rel *rel, > } > } > > - if (predicate && sel) { > - sel = stmt_reverse(sql->sa, sel); > - sel = stmt_join(sql->sa, sel, predicate, cmp_all); > - sel = stmt_result(sql->sa, sel, 0); > - predicate = NULL; > - if (!sub) > - predicate = sel; > - } > /* construct relation */ > l = sa_list(sql->sa); > if (sub && sel) { > @@ -2399,22 +2396,6 @@ rel2bin_select( mvc *sql, sql_rel *rel, > col = stmt_project(sql->sa, sel, col); > list_append(l, col); > } > - } else if (sub && predicate) { > - stmt *h = NULL; > - n = sub->op4.lval->h; > - h = stmt_join(sql->sa, column(sql->sa, n->data), predicate, cmp_all); > - h = stmt_result(sql->sa, h, 0); > - for( n = sub->op4.lval->h; n; n = n->next ) { > - stmt *col = n->data; > - > - if (col->nrcols == 0) /* constant */ > - col = stmt_const(sql->sa, h, col); > - else > - col = stmt_project(sql->sa, h, col); > - list_append(l, col); > - } > - } else if (predicate) { > - list_append(l, predicate); > } > return stmt_list(sql->sa, l); > } > diff --git a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > --- a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > +++ b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > @@ -29,17 +29,17 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 117 # length > +% 111 # length > function user.s0_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.likesubselect(X_6,X_3,A0,"",false); > + X_48 := algebra.likesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.likesubselect(r1_9,A0,"",false); > + X_49 := algebra.likesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.likesubselect(X_12,X_3,A0,"",false); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.likesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -55,17 +55,17 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 121 # length > +% 115 # length > function user.s1_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.likesubselect(X_6,X_3,A0,"",true); > + X_48 := algebra.likesubselect(X_6,X_3,A0,"",true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.likesubselect(r1_9,A0,"",true); > + X_49 := algebra.likesubselect(r1_9,A0,"",true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.likesubselect(X_12,X_3,A0,"",true); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.likesubselect(X_12,X_3,A0,"",true); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -81,17 +81,17 @@ end s1_1; > % .explain # table_name > % mal # name > % clob # type > -% 118 # length > +% 112 # length > function user.s2_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.ilikesubselect(X_6,X_3,A0,"",false); > + X_48 := algebra.ilikesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.ilikesubselect(r1_9,A0,"",false); > + X_49 := algebra.ilikesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.ilikesubselect(X_12,X_3,A0,"",false); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.ilikesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -107,17 +107,17 @@ end s2_1; > % .explain # table_name > % mal # name > % clob # type > -% 122 # length > +% 116 # length > function user.s3_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.ilikesubselect(X_6,X_3,A0,"",true); > + X_48 := algebra.ilikesubselect(X_6,X_3,A0,"",true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.ilikesubselect(r1_9,A0,"",true); > + X_49 := algebra.ilikesubselect(r1_9,A0,"",true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.ilikesubselect(X_12,X_3,A0,"",true); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.ilikesubselect(X_12,X_3,A0,"",true); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -153,19 +153,19 @@ end s3_1; > % .explain # table_name > % mal # name > % clob # type > -% 121 # length > +% 115 # length > function user.s4_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_56 := algebra.likesubselect(X_6,X_3,X_53,"",false,false); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_55 := algebra.likesubselect(X_6,X_3,X_52,"",false,false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_57 := algebra.likesubselect(r1_9,X_53,"",false,false); > + X_56 := algebra.likesubselect(r1_9,X_52,"",false,false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_58 := algebra.likesubselect(X_12,X_3,X_53,"",false,false); > - X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > + X_57 := algebra.likesubselect(X_12,X_3,X_52,"",false,false); > + X_14 := sql.subdelta(X_55,X_9,X_56,X_57); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > @@ -181,19 +181,19 @@ end s4_1; > % .explain # table_name > % mal # name > % clob # type > -% 125 # length > +% 119 # length > function user.s5_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_57 := algebra.likesubselect(X_6,X_3,X_53,"",false,true); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_56 := algebra.likesubselect(X_6,X_3,X_52,"",false,true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_58 := algebra.likesubselect(r1_9,X_53,"",false,true); > + X_57 := algebra.likesubselect(r1_9,X_52,"",false,true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_59 := algebra.likesubselect(X_12,X_3,X_53,"",false,true); > - X_14 := sql.subdelta(X_57,X_9,X_58,X_59); > + X_58 := algebra.likesubselect(X_12,X_3,X_52,"",false,true); > + X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > @@ -209,19 +209,19 @@ end s5_1; > % .explain # table_name > % mal # name > % clob # type > -% 122 # length > +% 116 # length > function user.s6_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_57 := algebra.likesubselect(X_6,X_3,X_53,"",true,false); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_56 := algebra.likesubselect(X_6,X_3,X_52,"",true,false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_58 := algebra.likesubselect(r1_9,X_53,"",true,false); > + X_57 := algebra.likesubselect(r1_9,X_52,"",true,false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_59 := algebra.likesubselect(X_12,X_3,X_53,"",true,false); > - X_14 := sql.subdelta(X_57,X_9,X_58,X_59); > + X_58 := algebra.likesubselect(X_12,X_3,X_52,"",true,false); > + X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > @@ -237,19 +237,19 @@ end s6_1; > % .explain # table_name > % mal # name > % clob # type > -% 126 # length > +% 120 # length > function user.s7_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_56 := algebra.likesubselect(X_6,X_3,X_53,"",true,true); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_55 := algebra.likesubselect(X_6,X_3,X_52,"",true,true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_57 := algebra.likesubselect(r1_9,X_53,"",true,true); > + X_56 := algebra.likesubselect(r1_9,X_52,"",true,true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_58 := algebra.likesubselect(X_12,X_3,X_53,"",true,true); > - X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > + X_57 := algebra.likesubselect(X_12,X_3,X_52,"",true,true); > + X_14 := sql.subdelta(X_55,X_9,X_56,X_57); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender

1 0

Re: MonetDB: default - Toptimize doesn't measure anything usefull, i...
by Martin Kersten 20 Mar '13

20 Mar '13

I agree, was looking at it as well. Idea was take the time of all optimizers together. sql>call querylog_enable(); sql>select * from querylog_catalog(); +----+-------+---------+-------+------+-----+----------+ | id | owner | defined | query | pipe | mal | optimize | +====+=======+=========+=======+======+=====+==========+ +----+-------+---------+-------+------+-----+----------+ 0 tuples (1.752ms) sql>call querylog_enable(); sql>select count(*) from tables; +------+ | L1 | +======+ | 33 | +------+ 1 tuple (3.112ms) sql>select * from querylog_catalog(); +-----------+---------+----------------------------+------------------------------+--------------+------+----------+ | id | owner | defined | query | pipe | mal | optimize | +===========+=========+============================+==============================+==============+======+==========+ | 1359324@0 | monetdb | 2013-03-20 13:22:14.000000 | select count(*) from tables; | default_pipe | 54 | 69124831 | +-----------+---------+----------------------------+------------------------------+--------------+------+----------+ 1 tuple (1.740ms) sql>select * from querylog_calls(); +-----------+----------------------------+----------------------------+--------------+--------+------+------+------+------+-------+ | id | start | stop | arguments | tuples | run | ship | cpu | io | space | +===========+============================+============================+==============+========+======+======+======+======+=======+ | 1359324@0 | 2013-03-20 13:22:14.000000 | 2013-03-20 13:22:14.000000 | user.s3_1(); | 1 | 742 | 14 | 0 | 0 | 0 | +-----------+----------------------------+----------------------------+--------------+--------+------+------+------+------+-------+ On 3/20/13 2:21 PM, Niels Nes wrote: > Changeset: 6bd79fe72615 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6bd79fe72615 > Modified Files: > monetdb5/mal/mal_runtime.c > sql/backends/monet5/sql_gencode.c > Branch: default > Log Message: > > Toptimize doesn't measure anything usefull, ie just store 0 for the time > being (solves all problems with changing output). We need a different > solution to store the time spend in the optimizer stack. > > > diffs (34 lines): > > diff --git a/monetdb5/mal/mal_runtime.c b/monetdb5/mal/mal_runtime.c > --- a/monetdb5/mal/mal_runtime.c > +++ b/monetdb5/mal/mal_runtime.c > @@ -172,8 +172,8 @@ runtimeProfileExit(Client cntxt, MalBlkP > if (malProfileMode == 0) > return; /* mostly true */ > > - if (stk != NULL && prof->stkpc >= 0 && mb->profiler != NULL && mb->profiler[stkpc].trace && mb->profiler[stkpc].clk) > - { > + if (stk != NULL && prof->stkpc >= 0 && mb->profiler != NULL && > + mb->profiler[stkpc].trace && mb->profiler[stkpc].clk) { > MT_lock_set(&mal_contextLock, "sysmon"); > gettimeofday(&mb->profiler[stkpc].clock, NULL); > mb->profiler[stkpc].counter++; > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -2408,7 +2408,7 @@ backend_dumpproc(backend *be, Client c, > { > char *t; > InstrPtr q; > - lng Toptimize = GDKusec(); > + lng Toptimize = 0; > > if ( be->q && be->q->codestring) { > t = GDKstrdup( be->q->codestring); > @@ -2421,7 +2421,6 @@ backend_dumpproc(backend *be, Client c, > q->token = REMsymbol; // will be patched > q = pushStr(mb, q, t); > q = pushStr(mb, q, pipe= initSQLoptimizer()); > - Toptimize = GDKusec() - Toptimize; > (void) pushLng(mb, q, Toptimize); > m->Tparse = 0; > GDKfree(pipe); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: MonetDB: default - Reduce critical section in mal_resource
by Sjoerd Mullender 19 Mar '13

19 Mar '13

Why are you messing with this code? What was wrong? The ATOMIC_CAS was the initialization of the variable running. It needs to happen. Why did you move it? The ATOMIC_GET that you first commented out and now removed actually already was just reading the variable without locks on most systems. Why did you replace it? It doesn't make *any* difference on x86_64 systems. Obviously you didn't measure the impact since there cannot have been any. On 2013-03-19 21:27, Martin Kersten wrote: > Changeset: a43f806ebd08 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a43f806ebd08 > Modified Files: > monetdb5/mal/mal_resource.c > Branch: default > Log Message: > > Reduce critical section in mal_resource > > > diffs (31 lines): > > diff --git a/monetdb5/mal/mal_resource.c b/monetdb5/mal/mal_resource.c > --- a/monetdb5/mal/mal_resource.c > +++ b/monetdb5/mal/mal_resource.c > @@ -198,7 +198,6 @@ MALresourceFairness(lng usec) > if ( usec > 0 && ( (usec = GDKusec()-usec)) <= TIMESLICE ) > return; > threads = GDKnr_threads > 0 ? GDKnr_threads : 1; > - ATOMIC_CAS_int(running, 0, threads, runningLock, "MALresourceFairness"); > > /* use GDKmem_cursize as MT_getrss(); is to expensive */ > rss = GDKmem_cursize(); > @@ -210,16 +209,14 @@ MALresourceFairness(lng usec) > clk = usec / 1000; > > if ( clk > DELAYUNIT ) { > + ATOMIC_CAS_int(running, 0, threads, runningLock, "MALresourceFairness"); > PARDEBUG mnstr_printf(GDKstdout, "#delay initial "LLFMT"n", clk); > ATOMIC_DEC_int(running, runningLock, "MALresourceFairness"); > - while (clk > 0) { > + /* always keep one running to avoid all waiting */ > + while (clk > 0 && running >= 2) { > /* speed up wake up when we have memory */ > if (rss < MEMORY_THRESHOLD * monet_memory) > break; > - /* always keep one running to avoid all waiting */ > - if ( running < 2) /* dirty read of shared variable is safe here */ > - //if ( (r =ATOMIC_GET_int(running, runningLock, "MALresourceFairness")) < 2) > - break; > delay = (unsigned int) ( ((double)DELAYUNIT * running) / threads); > if (delay) { > if ( delayed++ == 0){ > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender

1 0

Questions About Generated MAL Code
by Stephen P. Morgan 14 Mar '13

14 Mar '13

Hi, My colleagues and I from Huawei have been experimenting with the MAL code generated by SQL explain. We noticed big differences between the code generated by version 15 and the present top-of-tree. For example, where the level-15 code tended to generate just one barrier-exit block for a simple "select *," the new code generates several. (That might have to do with views--that's my guess, anyway.) Unfortunately, whereas the "old" (level 15) code is manually executable by the MAL interpreter, the new code doesn't run directly. For example, the interpreter complains about uninitialized variables. Of course, this is straightforward to fix manually, though somewhat tedious. But we also find when running multiple-barrier code, we get many complaints about undefined things, e.g., sql.mvc(), sql.bind(), among others. We don't (yet?) know how to overcome this issue. Is this just a temporary situation, or will it be possible at some point again to manually run MAL code? (Is it a policy of the MonetDB project that explained MAL code should be manually executable?) Would it be possible to explain (briefly) the reasons for the differences between the two versions of generated MAL code? We're curious what would prompt such a big change in the generated code. Thank you, Steve Morgan P.S. I've created a version of the mapi.connect() and mapi.reconnect() code that accepts a database name as another parameter. We've used it to send RPCs between systems (Linux containers, actually). We would be happy to donate this code to your organization, if you would like it. I added a new MAL routine for mapi.connect(), i.e., SERVERconnectWithDBname() and similar things for the mapi.reconnect() calls that detects the extra parameter and uses it if it's there. It shouldn't break any existing code, and it allows one to run RPCs while using monetdbd (not just mserver5).

1 0

Mercurial Source Missing opt_history.c
by Stephen P. Morgan 13 Mar '13

13 Mar '13

Hi MonetDB'ers, The present version of the Merucrial source for MonetDB seems not to build. I got the following error from make when I tried, even after a make clean: make[5]: *** No rule to make target `opt_history.c', needed by `liboptimizer_la-opt_history.lo'. Stop. hg log -l1 says: changeset: 47196:ba73b2c5ad5e tag: tip parent: 47194:9f63c9d7da57 parent: 47195:2c80719b8615 user: Stefan Manegold <Stefan.Manegold(a)cwi.nl> date: Wed Mar 13 13:20:55 2013 +0100 summary: Merge with Feb2013 branch. Thanks, Steve

2 2

Re: MonetDB: default - rewrote result parser to make it eat multiple...
by Gijs Molenaar 12 Mar '13

12 Mar '13

Now I think about it a bit more I think this is not a new feature but a bugfix, i can apply this commit also on the feb2013 branch if you guys want. On 11/03/13 18:44, Gijs Molenaar wrote: > Changeset: 39b2905626b8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=39b2905626b8 > Modified Files: > clients/python2/monetdb/sql/cursors.py > clients/python2/test/capabilities.py > clients/python3/monetdb/sql/cursors.py > clients/python3/test/capabilities.py > Branch: default > Log Message: > > rewrote result parser to make it eat multiple resultsets -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

3 2

Re: MonetDB: default - rewrote result parser to make it eat multiple...
by Stefan Manegold 12 Mar '13

12 Mar '13

Dear Gijs, thank you very much for continuously taking care of MonetDB's python interface / API! We very muchappreciate your work and contribution! However, we'd appreciate even more, it you could also keep the stable output up-to-date whenever you change / extend you test or your changes intendedly change the output of your tests. See http://monetdb.cwi.nl/testweb/web/status.php http://monetdb.cwi.nl/testweb/web/testgrid.php?serial=47169:7d1c376e995f&or… http://monetdb.cwi.nl/testweb/web/testgrid.php?serial=47169:7d1c376e995f&or… http://monetdb.cwi.nl/testweb/web/showtestoutput.php?serial=47169:7d1c376e9… http://monetdb.cwi.nl/testweb/web/testchange.php?test=47169:7d1c376e995f/GN… If nothing else, simply go to .../MonetDB/sql/test/mapi/Tests/ and run `Mtest.py` to run (among others) your pyhton test (preferably on a system with both Python2 & python3 installed, and follow the instructions. See also `Mtest.py --help`, `Mapprove.py --help` and http://dev.monetdb.org/hg/MonetDB/file/tip/testing/README Thanks! Stefan On Mon, Mar 11, 2013 at 06:44:43PM +0100, Gijs Molenaar wrote: > Changeset: 39b2905626b8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=39b2905626b8 > Modified Files: > clients/python2/monetdb/sql/cursors.py > clients/python2/test/capabilities.py > clients/python3/monetdb/sql/cursors.py > clients/python3/test/capabilities.py > Branch: default > Log Message: > > rewrote result parser to make it eat multiple resultsets > > > diffs (truncated from 463 to 300 lines): > > diff --git a/clients/python2/monetdb/sql/cursors.py b/clients/python2/monetdb/sql/cursors.py > --- a/clients/python2/monetdb/sql/cursors.py > +++ b/clients/python2/monetdb/sql/cursors.py > @@ -380,133 +380,99 @@ class Cursor(object): > if not block: > block = "" > > - lines = block.split("\n") > - firstline = lines[0] > + for line in block.split("\n"): > + if line.startswith(mapi.MSG_INFO): > + logger.info(line[1:]) > + self.messages.append((Warning, line[1:])) > > - while firstline.startswith(mapi.MSG_INFO): > - logger.info(firstline[1:]) > - self.messages.append((Warning, firstline[1:])) > - lines = lines[1:] > - firstline = lines[0] > + elif line.startswith(mapi.MSG_QTABLE): > + (self.__query_id, rowcount, columns, tuples) = line[2:].split() > > - if firstline.startswith(mapi.MSG_QTABLE): > - (id, rowcount, columns, tuples) = firstline[2:].split() > - columns = int(columns) # number of columns in result > - rowcount = int(rowcount) # total number of rows > - tuples = int(tuples) # number of rows in this set > - rows = [] > + columns = int(columns) # number of columns in result > + self.rowcount = int(rowcount) # total number of rows > + tuples = int(tuples) # number of rows in this set > + self.__rows = [] > > - # set up fields for description > - table_name = [None]*columns > - column_name = [None]*columns > - type_ = [None]*columns > - display_size = [None]*columns > - internal_size = [None]*columns > - precision = [None]*columns > - scale = [None]*columns > - null_ok = [None]*columns > + # set up fields for description > + table_name = [None] * columns > + column_name = [None] * columns > + type_ = [None] * columns > + display_size = [None] * columns > + internal_size = [None] * columns > + precision = [None] * columns > + scale = [None] * columns > + null_ok = [None] * columns > + typesizes = [(0, 0)] * columns > + self.__offset = 0 > + self.lastrowid = None > > - typesizes = [(0,0)]*columns > + elif line.startswith(mapi.MSG_HEADER): > + (data, identity) = line[1:].split("#") > + values = [x.strip() for x in data.split(",")] > + identity = identity.strip() > > - for line in lines[1:]: > - if line.startswith(mapi.MSG_HEADER): > - (data, identity) = line[1:].split("#") > - values = [x.strip() for x in data.split(",")] > - identity = identity.strip() > + if identity == "table_name": > + table_name = values > + elif identity == "name": > + column_name = values > + elif identity == "type": > + type_ = values > + elif identity == "length": > + length = values > + elif identity == "typesizes": > + typesizes = [[int(j) for j in i.split()] for i in values] > + internal_size = [x[0] for x in typesizes] > + for num, typeelem in enumerate(type_): > + if typeelem in ['decimal']: > + precision[num] = typesizes[num][0] > + scale[num] = typesizes[num][1] > + else: > + msg = "unknown header field" > + self.messages.append((InterfaceError, msg)) > + self.__exception_handler(InterfaceError, msg) > > - if identity == "table_name": > - table_name = values # not used > - elif identity == "name": > - column_name = values > - elif identity == "type": > - type_ = values > - elif identity == "length": > - pass # not used > - elif identity == "typesizes": > - typesizes = [[int(j) for j in i.split()] for i in values] > - internal_size = [x[0] for x in typesizes] > - for num, typeelem in enumerate(type_): > - if typeelem in ['decimal']: > - precision[num] = typesizes[num][0] > - scale[num] = typesizes[num][1] > - else: > - self.messages.append((InterfaceError, > - "unknown header field")) > - self.__exception_handler(InterfaceError, > - "unknown header field") > + self.description = list(zip(column_name, type_, display_size, > + internal_size, precision, scale, > + null_ok)) > + self.__offset = 0 > + self.lastrowid = None > > - self.description = list(zip(column_name, type_, > - display_size, internal_size, precision, scale, null_ok)) > > - if line.startswith(mapi.MSG_TUPLE): > - values = self.__parse_tuple(line) > - rows.append(values) > + elif line.startswith(mapi.MSG_TUPLE): > + values = self.__parse_tuple(line) > + self.__rows .append(values) > > - elif line == mapi.MSG_PROMPT: > - self.__query_id = id > - self.__rows = rows > - self.__offset = 0 > + elif line.startswith(mapi.MSG_QBLOCK): > + self.__rows = [] > > - self.rowcount = rowcount > - self.lastrowid = None > - logger.debug("II store result finished") > - return > - > - elif firstline.startswith(mapi.MSG_QBLOCK): > - rows = [] > - for line in lines[1:]: > - if line.startswith(mapi.MSG_TUPLE): > - values = self.__parse_tuple(line) > - rows.append(values) > - elif line == mapi.MSG_PROMPT: > - logger.debug("II store result finished") > - self.__rows = rows > - return > - > - elif firstline.startswith(mapi.MSG_QSCHEMA): > - if lines[1] == mapi.MSG_PROMPT: > + elif line.startswith(mapi.MSG_QSCHEMA): > + self.__offset = 0 > + self.lastrowid = None > self.__rows = [] > - self.__offset = 0 > self.description = None > self.rowcount = -1 > - self.lastrowid = None > - logger.debug("II schema finished") > - return > > - elif firstline.startswith(mapi.MSG_QUPDATE): > - if lines[1] == mapi.MSG_PROMPT: > - (affected, identity) = firstline[2:].split() > + elif line.startswith(mapi.MSG_QUPDATE): > + (affected, identity) = line[2:].split() > + self.__offset = 0 > self.__rows = [] > - self.__offset = 0 > self.description = None > self.rowcount = int(affected) > self.lastrowid = int(identity) > self.__query_id = -1 > - logger.debug("II update finished") > + > + elif line.startswith(mapi.MSG_QTRANS): > + self.__offset = 0 > + self.lastrowid = None > + self.__rows = [] > + self.description = None > + self.rowcount = -1 > + > + elif line.startswith(mapi.MSG_PROMPT): > return > > - elif firstline.startswith(mapi.MSG_ERROR): > - self.__exception_handler(ProgrammingError, firstline[1:]) > - > - elif firstline.startswith(mapi.MSG_QTRANS): > - if lines[1] == mapi.MSG_PROMPT: > - self.__rows = [] > - self.__offset = 0 > - self.description = None > - self.rowcount = -1 > - self.lastrowid = None > - logger.debug("II transaction finished") > - return > - > - elif firstline.startswith(mapi.MSG_PROMPT): > - self.__query_id = -1 > - self.__rows = [] > - self.__offset = 0 > - > - self.rowcount = 0 > - self.lastrowid = None > - logger.debug("II empty response, assuming everything is ok") > - return > + elif line.startswith(mapi.MSG_ERROR): > + self.__exception_handler(ProgrammingError, line[1:]) > > # you are not supposed to be here > self.__exception_handler(InterfaceError, "Unknown state, %s" % block) > diff --git a/clients/python2/test/capabilities.py b/clients/python2/test/capabilities.py > --- a/clients/python2/test/capabilities.py > +++ b/clients/python2/test/capabilities.py > @@ -352,4 +352,16 @@ class DatabaseTest(unittest.TestCase): > t = ["list", "test"] > self.assertRaises(ProgrammingError, self.db_module.monetize.convert, t) > self.db_module.monetize.mapping[list] = str > - self.assertEqual(self.db_module.monetize.convert(t), "['list', 'test']") > \ No newline at end of file > + self.assertEqual(self.db_module.monetize.convert(t), "['list', 'test']") > + > + def test_multiple_queries(self): > + table1 = self.new_table_name() > + table2 = table1[:-1] + 'bla"' > + self.cursor.execute("create table %s (a int)" % table1) > + self.cursor.execute("create table %s (a int, b int)" % table2) > + self.cursor.execute("insert into %s VALUES (100)" % table1) > + self.cursor.execute("insert into %s VALUES (50, 50)" % table2) > + self.cursor.execute('select * from %s; select * from %s;' % > + (table1, table2)) > + result = self.cursor.fetchall() > + self.assertEqual(result, [(50, 50)]) > \ No newline at end of file > diff --git a/clients/python3/monetdb/sql/cursors.py b/clients/python3/monetdb/sql/cursors.py > --- a/clients/python3/monetdb/sql/cursors.py > +++ b/clients/python3/monetdb/sql/cursors.py > @@ -381,133 +381,99 @@ class Cursor(object): > if not block: > block = "" > > - lines = block.split("\n") > - firstline = lines[0] > + for line in block.split("\n"): > + if line.startswith(mapi.MSG_INFO): > + logger.info(line[1:]) > + self.messages.append((Warning, line[1:])) > > - while firstline.startswith(mapi.MSG_INFO): > - logger.info(firstline[1:]) > - self.messages.append((Warning, firstline[1:])) > - lines = lines[1:] > - firstline = lines[0] > + elif line.startswith(mapi.MSG_QTABLE): > + (self.__query_id, rowcount, columns, tuples) = line[2:].split() > > - if firstline.startswith(mapi.MSG_QTABLE): > - (id, rowcount, columns, tuples) = firstline[2:].split() > - columns = int(columns) # number of columns in result > - rowcount = int(rowcount) # total number of rows > - tuples = int(tuples) # number of rows in this set > - rows = [] > + columns = int(columns) # number of columns in result > + self.rowcount = int(rowcount) # total number of rows > + tuples = int(tuples) # number of rows in this set > + self.__rows = [] > > - # set up fields for description > - table_name = [None]*columns > - column_name = [None]*columns > - type_ = [None]*columns > - display_size = [None]*columns > - internal_size = [None]*columns > - precision = [None]*columns > - scale = [None]*columns > - null_ok = [None]*columns > + # set up fields for description > + table_name = [None] * columns > + column_name = [None] * columns > + type_ = [None] * columns > + display_size = [None] * columns > + internal_size = [None] * columns > + precision = [None] * columns > + scale = [None] * columns > + null_ok = [None] * columns > + typesizes = [(0, 0)] * columns > + self.__offset = 0 > + self.lastrowid = None > > - typesizes = [(0,0)]*columns > + elif line.startswith(mapi.MSG_HEADER): > + (data, identity) = line[1:].split("#") > + values = [x.strip() for x in data.split(",")] > + identity = identity.strip() > > - for line in lines[1:]: > - if line.startswith(mapi.MSG_HEADER): > - (data, identity) = line[1:].split("#") > - values = [x.strip() for x in data.split(",")] > - identity = identity.strip() > + if identity == "table_name": > + table_name = values > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 2

mapi.connect / mapi_connect (Second Try)
by Stephen P. Morgan 12 Mar '13

12 Mar '13

I wrote an email a few days ago to the effect that I could not get the mapi.connect() MAL statement to work due to the server side complaining that I had not specified the database to use. This morning I ran one of the sample programs included with MonetDB, clients/examples,C/sample0.c (sample0). It seems very similar as it issues a mapi_connect() with the database field set to NULL. It fails in the same way, i.e., MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !monetdbd: please specify a database Should mapi_connect() work with the database field set to NULL? (If so, then the mapi.connect() MAL statement should work, I think.) Is there a bug in the server that insists on the database being specified? If there is a bug, does anyone have a fix for this? It would be greatly appreciated. Thanks, Steve Morgan

3 7

Coredump caused by function SERVERreconnectWithoutAlias() and its verifies fix
by Fang Cao 12 Mar '13

12 Mar '13

Hi, Today when I try to use mapi.connect() and mapi.reconnect() with mserver5, I found such crash when I tried to : fang@fang-ThinkCentre-M91p:~/Work/DB-Farm$ mserver5 --dbpath=/home/fang/Work/DB-Farm # MonetDB 5 server v11.15.1 "Feb2013" # Serving database 'DB-Farm', using 8 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 3.692 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # MonetDB/JAQL module loaded # MonetDB/SQL module loaded >mid:=mapi.reconnect("localhost",50000,"monetdb","monetdb","mal"); Segmentation fault (core dumped) The root cause is located at: SERVERreconnectWithoutAlias(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci) { int *key =(int*) getArgReference(stk,pci,0); str *host = (str*) getArgReference(stk,pci,1); int *port = (int*) getArgReference(stk,pci,2); str *username = (str*) getArgReference(stk,pci,4); ===>> should be 3 str *password = (str*) getArgReference(stk,pci,5); ===>> should be 4 str *lang = (str*) getArgReference(stk,pci,6); ===>> should be 5 Since we only transfer 5 input parameters, so for the last one “lang”, of course we will coredump the program when try to check 6th. After the modification, everything is fine now, for both mapi.connect() and mapi.reconnect() The reason why this issue is introduced: pattern reconnect(host:str, port:int, usr:str, passwd:str,lang:str):int address SERVERreconnectWithoutAlias comment "Re-establish connection with a remote mserver."; pattern reconnect(host:str, port:int, db_alias:str, usr:str, passwd:str,lang:str):int address SERVERreconnectAlias comment "Re-establish connection with a remote mserver."; So when to implement SERVERreconnectWithoutAlias(), seems following the style of SERVERreconnectAlias(). The latter one has 6 inputs, but the first one has only 5 inputs. So un-needed db_alias was removed, but forgot to adjust the sequence number. Thanks, Fang

2 1

Re: MonetDB: Feb2013 - opt_support.c & opt_wrapper.c: "coercion" -> ...
by Stefan Manegold 12 Mar '13

12 Mar '13

> Changeset: 30c2d470a73d for MonetDB > URL: > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=30c2d470a73d > Modified Files: > monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > monetdb5/optimizer/opt_support.c > monetdb5/optimizer/opt_wrapper.c > sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > Branch: Feb2013 > Log Message: > > opt_support.c & opt_wrapper.c: "coercion" -> "coercions", > i.e., using the correct optimizer name "coercions" > to ensure the optimizer is indeed activated/used. > > Approved related changes in test output (modified MAL plans > due to now active coercions optimizer). > > This should fix bug 3151. ^!^^ Typo: this should read bug 3251 > Thanks to Pete Hollobon for reporting and proving the initial fix! > > > Open question: > > Why is an incorrect / non-exiting optimizer name silently ignored?? > > > diffs (truncated from 413 to 300 lines): > > diff --git > a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > --- > a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > +++ > b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > @@ -61,15 +61,15 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 178 # length > +% 153 # length > function user.s2_1{autoCommit=true}():void; > X_2 := sql.mvc(); > - (X_5:bat[:oid,:int] ,X_6:bat[:oid,:int] ,X_7:bat[:oid,:int] ) := > sql.copy_from("sys":str,"ttt":str,"|":str,"\\n":str,nil:str,"null":str,"/:\\tmp/xyz":str,-1:lng,0:lng,0:int); > - X_9 := sql.append(X_2,"sys","ttt","a",X_5); > - X_11 := sql.append(X_9,"sys","ttt","b",X_6); > - X_15 := sql.append(X_11,"sys","ttt","c",X_7); > - X_16 := aggr.count(X_7); > - sql.affectedRows(X_15,X_16,""); > + (X_13:bat[:oid,:int] ,X_14:bat[:oid,:int] ,X_15:bat[:oid,:int] ) > := > sql.copy_from("sys","ttt","|","\\n",nil:str,"null","/:\\tmp/xyz",-1:lng,0:lng,0); > + X_17 := sql.append(X_2,"sys","ttt","a",X_13); > + X_19 := sql.append(X_17,"sys","ttt","b",X_14); > + X_23 := sql.append(X_19,"sys","ttt","c",X_15); > + X_24 := aggr.count(X_15); > + sql.affectedRows(X_23,X_24,""); > end s2_1; > # optimizer.mitosis() > # optimizer.dataflow() > @@ -89,15 +89,15 @@ end s2_1; > % .explain # table_name > % mal # name > % clob # type > -% 178 # length > +% 153 # length > function user.s5_1{autoCommit=true}():void; > X_2 := sql.mvc(); > - (X_5:bat[:oid,:int] ,X_6:bat[:oid,:int] ,X_7:bat[:oid,:int] ) := > sql.copy_from("sys":str,"ttt":str,"|":str,"\\n":str,nil:str,"null":str,"/:\\tmp/xyz":str,-1:lng,0:lng,0:int); > - X_9 := sql.append(X_2,"sys","ttt","a",X_5); > - X_11 := sql.append(X_9,"sys","ttt","b",X_6); > - X_16 := aggr.count(X_7); > - X_15 := sql.append(X_11,"sys","ttt","c",X_7); > - sql.affectedRows(X_15,X_16,""); > + (X_13:bat[:oid,:int] ,X_14:bat[:oid,:int] ,X_15:bat[:oid,:int] ) > := > sql.copy_from("sys","ttt","|","\\n",nil:str,"null","/:\\tmp/xyz",-1:lng,0:lng,0); > + X_17 := sql.append(X_2,"sys","ttt","a",X_13); > + X_19 := sql.append(X_17,"sys","ttt","b",X_14); > + X_24 := aggr.count(X_15); > + X_23 := sql.append(X_19,"sys","ttt","c",X_15); > + sql.affectedRows(X_23,X_24,""); > end s5_1; > # optimizer.mitosis() > # optimizer.dataflow() > diff --git a/monetdb5/optimizer/opt_support.c > b/monetdb5/optimizer/opt_support.c > --- a/monetdb5/optimizer/opt_support.c > +++ b/monetdb5/optimizer/opt_support.c > @@ -144,7 +144,7 @@ struct OPTcatalog { > {"groups", 0, 0, 0, DEBUG_OPT_GROUPS}, > {"aliases", 0, 0, 0, DEBUG_OPT_ALIASES}, > {"cluster", 0, 0, 0, DEBUG_OPT_CLUSTER}, > -{"coercion", 0, 0, 0, DEBUG_OPT_COERCION}, > +{"coercions", 0, 0, 0, DEBUG_OPT_COERCION}, > {"commonTerms", 0, 0, 0, DEBUG_OPT_COMMONTERMS}, > {"compress", 0, 0, 0, DEBUG_OPT_COMPRESS}, > {"constants", 0, 0, 0, DEBUG_OPT_CONSTANTS}, > diff --git a/monetdb5/optimizer/opt_wrapper.c > b/monetdb5/optimizer/opt_wrapper.c > --- a/monetdb5/optimizer/opt_wrapper.c > +++ b/monetdb5/optimizer/opt_wrapper.c > @@ -84,7 +84,7 @@ struct{ > {"aliases", &OPTaliasesImplementation}, > {"centipede", &OPTcentipedeImplementation}, > {"cluster", &OPTclusterImplementation}, > - {"coercion", &OPTcoercionImplementation}, > + {"coercions", &OPTcoercionImplementation}, > {"commonTerms", &OPTcommonTermsImplementation}, > {"compression", &OPTcompressionImplementation}, > {"costModel", &OPTcostModelImplementation}, > diff --git a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > --- a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > +++ b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > @@ -27,10 +27,10 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 76 # length > +% 72 # length > function user.s0_1{autoCommit=true}(A0:bte,A1:bte):void; > X_4 := udf.fuse(A0,A1); > - > sql.exportValue(1,".","fuse_single_value":str,"smallint",16,0,6,X_4,""); > + > sql.exportValue(1,".","fuse_single_value","smallint",16,0,6,X_4,""); > end s0_1; > # optimizer.mitosis() > # optimizer.dataflow() > @@ -44,10 +44,10 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 71 # length > +% 67 # length > function user.s1_1{autoCommit=true}(A0:sht,A1:sht):void; > X_4 := udf.fuse(A0,A1); > - > sql.exportValue(1,".","fuse_single_value":str,"int",32,0,6,X_4,""); > + sql.exportValue(1,".","fuse_single_value","int",32,0,6,X_4,""); > end s1_1; > #select fuse(1000,2000); > % . # table_name > @@ -59,10 +59,10 @@ end s1_1; > % .explain # table_name > % mal # name > % clob # type > -% 74 # length > +% 70 # length > function user.s2_1{autoCommit=true}(A0:int,A1:int):void; > X_4 := udf.fuse(A0,A1); > - > sql.exportValue(1,".","fuse_single_value":str,"bigint",64,0,6,X_4,""); > + > sql.exportValue(1,".","fuse_single_value","bigint",64,0,6,X_4,""); > end s2_1; > #select fuse(1000000,2000000); > % . # table_name > diff --git a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > --- a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > +++ b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > @@ -48,10 +48,10 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 74 # length > +% 70 # length > function user.s0_1{autoCommit=true}(A0:str):void; > X_3 := udf.reverse(A0); > - > sql.exportValue(1,".","reverse_single_value":str,"clob",0,0,4,X_3,""); > + > sql.exportValue(1,".","reverse_single_value","clob",0,0,4,X_3,""); > end s0_1; > #select reverse('MonetDB'); > % . # table_name > @@ -81,7 +81,7 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 62 # length > +% 60 # length > function user.s3_1{autoCommit=true}():void; > X_2 := sql.mvc(); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_reverse"); > @@ -92,7 +92,7 @@ function user.s3_1{autoCommit=true}():vo > X_15 := algebra.leftfetchjoin(X_3,X_14); > X_16:bat[:oid,:str] := batudf.reverse(X_15); > X_18 := sql.resultSet(1,1,X_16); > - sql.rsColumn(X_18,"sys.","reverse_x":str,"clob",0,0,X_16); > + sql.rsColumn(X_18,"sys.","reverse_x","clob",0,0,X_16); > X_22 := io.stdout(); > sql.exportResult(X_22,X_18); > end s3_1; > diff --git > a/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > b/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > --- > a/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > +++ > b/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > @@ -27,7 +27,7 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 63 # length > +% 59 # length > function user.s0_1{autoCommit=true}(A0:dbl):void; > X_3 := sql.mvc(); > X_4:bat[:oid,:oid] := sql.tid(X_3,"sys","sf_2596114"); > @@ -38,7 +38,7 @@ function user.s0_1{autoCommit=true}(A0:d > X_16 := algebra.leftfetchjoin(X_4,X_15); > X_17:bat[:oid,:dbl] := batmmath.pow(X_16,A0); > X_19 := sql.resultSet(1,1,X_17); > - sql.rsColumn(X_19,"sys.","power_x":str,"double",53,0,X_17); > + sql.rsColumn(X_19,"sys.","power_x","double",53,0,X_17); > X_24 := io.stdout(); > sql.exportResult(X_24,X_19); > end s0_1; > @@ -46,7 +46,7 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 65 # length > +% 61 # length > function user.s1_1{autoCommit=true}():void; > X_2 := sql.mvc(); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","sf_2596114"); > @@ -57,7 +57,7 @@ function user.s1_1{autoCommit=true}():vo > X_15 := algebra.leftfetchjoin(X_3,X_14); > X_16:bat[:oid,:dbl] := batcalc.*(X_15,X_15); > X_18 := sql.resultSet(1,1,X_16); > - sql.rsColumn(X_18,"sys.","sql_mul_x":str,"double",53,0,X_16); > + sql.rsColumn(X_18,"sys.","sql_mul_x","double",53,0,X_16); > X_23 := io.stdout(); > sql.exportResult(X_23,X_18); > end s1_1; > diff --git > a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > --- > a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > +++ > b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > @@ -34,22 +34,22 @@ function user.s0_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_45 := algebra.likesubselect(X_6,X_3,A0,"":str,false); > + X_45 := algebra.likesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_46 := algebra.likesubselect(r1_9,A0,"":str,false); > + X_46 := algebra.likesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_47 := algebra.likesubselect(X_12,X_3,A0,"":str,false); > - X_14 := sql.subdelta(X_45,X_9,X_46,X_47); > - X_16 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > - X_17 := sql.bind(X_2,"sys","functions","func",0); > - (X_19,r1_28) := sql.bind(X_2,"sys","functions","func",2); > - X_21 := sql.bind(X_2,"sys","functions","func",1); > - X_22 := sql.projectdelta(X_14,X_17,X_19,r1_28,X_21); > - X_23 := sql.resultSet(2,1,X_16); > - sql.rsColumn(X_23,"sys.functions","name","varchar",256,0,X_16); > - sql.rsColumn(X_23,"sys.functions","func","varchar",8196,0,X_22); > - X_32 := io.stdout(); > - sql.exportResult(X_32,X_23); > + X_47 := algebra.likesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_45,X_9,X_46,X_47); > + X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > + X_18 := sql.bind(X_2,"sys","functions","func",0); > + (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > + X_22 := sql.bind(X_2,"sys","functions","func",1); > + X_23 := sql.projectdelta(X_15,X_18,X_20,r1_28,X_22); > + X_24 := sql.resultSet(2,1,X_17); > + sql.rsColumn(X_24,"sys.functions","name","varchar",256,0,X_17); > + sql.rsColumn(X_24,"sys.functions","func","varchar",8196,0,X_23); > + X_33 := io.stdout(); > + sql.exportResult(X_33,X_24); > end s0_1; > #explain select name,func from functions where name not like > '%optimizers%'; > % .explain # table_name > @@ -60,22 +60,22 @@ function user.s1_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_45 := algebra.likesubselect(X_6,X_3,A0,"":str,true); > + X_45 := algebra.likesubselect(X_6,X_3,A0,"",true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_46 := algebra.likesubselect(r1_9,A0,"":str,true); > + X_46 := algebra.likesubselect(r1_9,A0,"",true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_47 := algebra.likesubselect(X_12,X_3,A0,"":str,true); > - X_14 := sql.subdelta(X_45,X_9,X_46,X_47); > - X_16 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > - X_17 := sql.bind(X_2,"sys","functions","func",0); > - (X_19,r1_28) := sql.bind(X_2,"sys","functions","func",2); > - X_21 := sql.bind(X_2,"sys","functions","func",1); > - X_22 := sql.projectdelta(X_14,X_17,X_19,r1_28,X_21); > - X_23 := sql.resultSet(2,1,X_16); > - sql.rsColumn(X_23,"sys.functions","name","varchar",256,0,X_16); > - sql.rsColumn(X_23,"sys.functions","func","varchar",8196,0,X_22); > - X_32 := io.stdout(); > - sql.exportResult(X_32,X_23); > + X_47 := algebra.likesubselect(X_12,X_3,A0,"",true); > + X_15 := sql.subdelta(X_45,X_9,X_46,X_47); > + X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > + X_18 := sql.bind(X_2,"sys","functions","func",0); > + (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > + X_22 := sql.bind(X_2,"sys","functions","func",1); > + X_23 := sql.projectdelta(X_15,X_18,X_20,r1_28,X_22); > + X_24 := sql.resultSet(2,1,X_17); > + sql.rsColumn(X_24,"sys.functions","name","varchar",256,0,X_17); > + sql.rsColumn(X_24,"sys.functions","func","varchar",8196,0,X_23); > + X_33 := io.stdout(); > + sql.exportResult(X_33,X_24); > end s1_1; > #explain select name,func from functions where name ilike > '%optimizers%'; > % .explain # table_name > @@ -86,22 +86,22 @@ function user.s2_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_45 := algebra.ilikesubselect(X_6,X_3,A0,"":str,false); > + X_45 := algebra.ilikesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_46 := algebra.ilikesubselect(r1_9,A0,"":str,false); > + X_46 := algebra.ilikesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_47 := algebra.ilikesubselect(X_12,X_3,A0,"":str,false); > - X_14 := sql.subdelta(X_45,X_9,X_46,X_47); > - X_16 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > - X_17 := sql.bind(X_2,"sys","functions","func",0); > - (X_19,r1_28) := sql.bind(X_2,"sys","functions","func",2); > - X_21 := sql.bind(X_2,"sys","functions","func",1); > - X_22 := sql.projectdelta(X_14,X_17,X_19,r1_28,X_21); > - X_23 := sql.resultSet(2,1,X_16); > - sql.rsColumn(X_23,"sys.functions","name","varchar",256,0,X_16); > - sql.rsColumn(X_23,"sys.functions","func","varchar",8196,0,X_22); > - X_32 := io.stdout(); > - sql.exportResult(X_32,X_23); > + X_47 := algebra.ilikesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_45,X_9,X_46,X_47); > + X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > + X_18 := sql.bind(X_2,"sys","functions","func",0); > + (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > + X_22 := sql.bind(X_2,"sys","functions","func",1); > + X_23 := sql.projectdelta(X_15,X_18,X_20,r1_28,X_22); > + X_24 := sql.resultSet(2,1,X_17); > + sql.rsColumn(X_24,"sys.functions","name","varchar",256,0,X_17); > + sql.rsColumn(X_24,"sys.functions","func","varchar",8196,0,X_23); > + X_33 := io.stdout(); > + sql.exportResult(X_33,X_24); > end s2_1; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Running MAL Programs Manually?
by Stephen P. Morgan 12 Mar '13

12 Mar '13

Should it be possible to run MAL programs manually, by reading them from a file? I just tried a simple experiment and got an error message. The experiment: I created a table, "test," with two columns, "field1" and field2." I created a MAL query by issuing the SQL query, "EXPLAIN SELECT * FROM test;" and cut and pasted the output to a file. I then stripped out all of the formatting characters ('|') and extraneous spaces. I then re-entered mclient, choosing the MAL language, and read in the MAL form of the query with "\< junk.mal." The interpreter then complained that two identifiers were not initialized (see output, below). So, is it not possible to issue a MAL query by hand? Or am I missing something? (Again, please forgive me, I am new to MonetDB.) Thanks, Steve Morgan mal>\< junk.mal MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = function user.s1_1{autoCommit=true}():void; barrier X_41 := language.dataflow(); X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","test"); X_6 := sql.bind(X_2,"sys","test","field1",0); (X_9,r1_9) := sql.bind(X_2,"sys","test","field1",2); X_12 := sql.bind(X_2,"sys","test","field1",1); X_14 := sql.delta(X_6,X_9,r1_9,X_12); X_15 := algebra.leftfetchjoin(X_3,X_14); X_16 := sql.bind(X_2,"sys","test","field2",0); (X_18,r1_22) := sql.bind(X_2,"sys","test","field2",2); X_20 := sql.bind(X_2,"sys","test","field2",1); X_21 := sql.delta(X_16,X_18,r1_22,X_20); X_22 := algebra.leftfetchjoin(X_3,X_21); language.pass(X_3); exit X_41; X_23 := sql.resultSet(2,1,X_15); sql.rsColumn(X_23,"sys.test","field1","int",32,0,X_15); sql.rsColumn(X_23,"sys.test","field2","int",32,0,X_22); X_29 := io.stdout(); sql.exportResult(X_29,X_23); end s1_1; ERROR = !TypeException:user.s1_1[16]:'X_15' may not be used before being initialized !TypeException:user.s1_1[17]:'X_15' may not be used before being initialized !TypeException:user.s1_1[18]:'X_22' may not be used before being initialized

1 1

External data atoms Issues
by Mike Gomez 11 Mar '13

11 Mar '13

Hi, i have been trying to expand data types (both core and external atoms).However, after discussions, i understood that addition of core data types will takes years( if i am not exaggerating).So, i started testing up external gdk atoms (monetdb5/modules/atoms). i am creating new atom type say mytype which is int based as below. this atom is pure int ( there are no bitfields and so on) typedef int mytype; atom mytype:int; in mal i have defined the following mal procedures( and C backends) fromstr(), tostr(), mytype(s:str):mytype, new(s:str):mytype, isnil(v:mytype):bit at this moment,it can successfully create tables and do selects. however, selects are not working as they should. these queries work. create table test(i mytype); insert into test values('453535'); select * from test these don't work. select * from test2 where i >= 5 above query gives error "types mytype(0,0) and tinyint(8,0) are not equal for column 'i'" insert into test2 values(453535); gives types int(32,0) and mytype(0,0) are not equal however, this query works !!!!!! select * from test2 where i >= '1' after testing and too many code peeping for above query, i figured out that the above query converts each element of table to string using tostr, and then compares the strings, and takes decision. so, it looks like there can be performance improvements here. However,my question is HOW do i get the queries working such as select * from test2 where i >= 5 am i missing something?? any pointers may be helpful. Thanks. -Mike

1 0

sabdb protocol V2
by Gijs Molenaar 11 Mar '13

11 Mar '13

Hi MonetDB people, I've noticed that there has been a new version (2) of the sabdb protocol introduced. The sabdb protocol is used to send administrative command to a monetdb server. This breaks the Python API, which I just fixed, I'll commit it later. But I've noticed some strange behavior, it looks like monetdb closes the connection in some cases, not all. Specifically when I issue a status command the connection is closed, but in other cases it is not. I've fixed it now by reconnecting before every command, but I just wanted to verify if that is the right way to go. greetings, -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

3 5

mapi.connect() MAL Statement
by Stephen P. Morgan 08 Mar '13

08 Mar '13

I'm in the early stages of trying to write a MAL program to access a remote database using mapi.* modules, mainly to test out the function, but also for future work on distributed query processing. I find that if I issue the mapi,connect() call as follows: mal> ... mal> mid=mapi.connect("localhost",50000,"monetdb","monetdb","mal"); I get the following response: MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = mid:=mapi.connect("localhost",50000,"monetdb","monetdb","mal"); ERROR = !IOException:mapi.connect:Could not connect: monetdbd: please specify a database The "please specify a database" part seems to be coming from tools/merovingian/daemon/client.c, i.e., the connection was attempted but was rejected by monetdbd because a database parameter was not specified. I( noticed that mapi.connect() calls SERVERconnet() which calls SERVERconnectAll() which calls mapi_connect() with NULL as the database parameter. Should SERVERconnect() (i.e., mapi.connect()) be altered to accept another parameter, database, which would be passed through SERVERconnectAll() and mapi_connect()? Or am I missing something altogether about mapi.connect()? P.S. Please excuse me if this is a foolish question. I am new to MonetDB. Thank you, Steve Morgan

1 0

Adding data types (maybe to core), probably BUG
by Mike Gomez 08 Mar '13

08 Mar '13

Hi, i am working on monetdb so as it can be used efficiently by more users(myself included).However, the issue i seem to face ( as maybe others) is the data types. for many weeks now,I have been going through the source code, understanding and learning it.Data types in monetdb can be extended in 2 types, external and core GDK. external method problem ( maybe bug in monetdb) ======================== I have created new data type, say unsigned int 32 bit, using extensions (such as inet,color and so on).However, the bug is monetdb system doesn't use my comparison operation, instead uses the default inbuilt operators. eg. in the uint mal file( MonetDB/monetdb5/modules/atoms/uint.mal), i have declared -------------- atom uint:int; .. ..............skipped source code however, when i sort the table, the system still places negative values ahead, means instead of using my declared comparison algos, it's using internal algos. my question is how do i override that ( or is it even possible to do it). core GDK method =============== After failing that, i thought lets add the core data type uint in the gdk_atoms.c. i modified the parsor(& everything else) and the system recognizes my data type. however,when i run simple insert query such as below, i get insert into test1 values('2'); TypeException:user.s1_1[3]:'calc.uint' undefined in: _5:any := calc.uint(_4:str) however, i have got completely stuck in the MAL operators and language. so, how do i get the algebra, operations and everything right, so it works? Any pointers will help. let me know if something more is required. i am working on this out of my interest,so eventually if i get it to work somehow, the code goes back to project. Thanks for help. mike

2 3

Announcement: New Feb2013-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 07 Mar '13

07 Mar '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Feb 2013-SP1 bugfix release Bug Fixes * 3228: Python client - mapi.Connection.__getbyes should throw exception when 0 bytes received from socket * 3229: GRANT does not work for UPDATE * 3230: GRANT SELECT priveleges not set in SQL FUNCTIONs * 3233: getting a crash of server with segmentation fault on SLES 11 64 SP2 * 3235: MapiPP.pm unquote '0' as 'undef' * 3236: mclient \d produces no output * 3237: Assertion on BATgroup_internal * 3241: SEGFAULT on string processing when using optimizer.reduce()

1 0

Re: MonetDB: Feb2013 - Make MALresource independent of cntxt and mb
by Fabian Groffen 04 Mar '13

04 Mar '13

Any reason for undoing 0f30e64e9bb2? On 26-02-2013 14:52:15 +0100, Martin Kersten wrote: > Changeset: 33a1b7bfcb28 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=33a1b7bfcb28 > Modified Files: > monetdb5/mal/mal_dataflow.c > monetdb5/mal/mal_interpreter.c > monetdb5/mal/mal_resource.c > monetdb5/mal/mal_resource.h > Branch: Feb2013 > Log Message: > > Make MALresource independent of cntxt and mb > > > diffs (87 lines): > > diff --git a/monetdb5/mal/mal_dataflow.c b/monetdb5/mal/mal_dataflow.c > --- a/monetdb5/mal/mal_dataflow.c > +++ b/monetdb5/mal/mal_dataflow.c > @@ -328,13 +328,13 @@ DFLOWworker(void *t) > } > MT_lock_unset(&flow->flowlock, "MALworker"); > > + q_enqueue(flow->done, fe); > if ( fnxt == 0) { > if (todo->last == 0) > profilerHeartbeatEvent("wait"); > else > - MALresourceFairness(flow->cntxt, flow->mb, usec); > + MALresourceFairness(usec); > } > - q_enqueue(flow->done, fe); > } > GDKfree(GDKerrbuf); > GDKsetbuf(0); -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home

4 6

Announcement: New Feb2013 Feature release of MonetDB suite
by Sjoerd Mullender 19 Feb '13

19 Feb '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Feb 2013 feature release Testing Environment * enabled "top-level" Mtest.py So far, while Mtest.py could be called in any subdirectory of the MonetDB source tree (and could then run all tests in the entire sub-tree), it was not possible to call Mtest.py in the top-level MonetDB source directory to run all tests. Instead, to run all tests, Mtest.py had to be called at least 4 times, once in each of these directories: "clients", "monetdb5", "sql", "geom". Now, it is possible to call Mtest.py once in the top-level MonetDB source directory to run all tests in one go. The behaviour of calling Mtest.py in any subdirectory, including the four mentioned above, did not changed, other than that now obsolete command line options "-p / --package <package>" and "-5 / --monetdb5" have been removed. Java Module * merocontrol was changed to return server URIs, and lastStop time. Connections and dbpath were removed. * Mapi protocol v8 support was removed from MapiSocket. Protocol v8 has not been used by the servers any more since Apr2012 release Client Package * Mapi protocol v8 support was removed from all client drivers. Protocol v8 has not been used by the servers any more since Apr2012 release * The tool mnc was removed from installations * msqldump: Implmented an option (--table/-t) to dump a single table. * Changed msqdump's trace option to be in line with mclient. In both cases, the long option is --Xdebug and the short option is -X. MonetDB5 Server * mserver5: The --dbname and --dbfarm options have been replaced by the single --dbpath option. * The scheduler of mserver5 was changed to use a fixed set of workers to perform the work for all connected clients. Previously, each client connection had its own set of workers, easily causing resource problems upon multiple connections to the server. Merovingian * Upgrade support for dbfarms from Mar2011 and Aug2011 was dropped * monetdb status now uses a more condensed output, to cater for the uris being shown, and prints how long a database is stopped, or how long ago it crashed * monetdb status now prints the connection uri for each database, when available. The connections and database path properties have been dropped. * monetdb status now prints last crash date only if the database has not been started since. Bug Fixes * 2291: small doubles end up as NULL after arithmetic * 3215: Calculation Date function using interval and year * 3033: stethoscope needs better documentation * 3084: Timestamp arithmetic very slow (especially on Windows) * 3125: Python tests fail after recent Python API changes * 3172: assertion fails on table function with subselects as parameters * 3178: one scan is enough to implement ALGstdev_@1 in monetdb5/modules/kernel/algebra.mx * 3179: LIKE: batstr.like+algebra.uselect called instead of pcre.like_filter * 3193: Expressions not supported in the GROUP BY or ORDER BY clause. * 3216: "unknown property" error setting format and width in .monetdb file * 3217: gdk_posix fails to compile under NetBSD * 3221: can no execute large statements * 3227: MT_set_lock() call on an non-initialized lock

1 0

Au sujet MonetDB
by Bouslah Ahmed 17 Feb '13

17 Feb '13

Bonsoir, Je travaille depuis peu de temps sur MonetDB et je rencontre des difficultés à comprendre le principe de fonctionnement, pour cela j'ai des questions à vous poser. 1 - Où puis-je trouver le code source; 2 - Comment le compiler; 3 - Pour information j'utilise Windows comme système d'exploitation. Merci de me répondre. Cordialement a.bouslah(a)yahoo.fr

2 2

Re: MonetDB: default - Merge with Feb2013 branch.
by Martin Kersten 14 Feb '13

14 Feb '13

Thx On 2/14/13 11:24 PM, Sjoerd Mullender wrote: > Changeset: f334ca6b1b90 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f334ca6b1b90 > Added Files: > sql/test/Users/Tests/privs.sql.src > sql/test/Users/Tests/privs.stable.err > sql/test/Users/Tests/privs.stable.out > sql/test/Users/Tests/privs2.sql.src > sql/test/Users/Tests/privs2.stable.err > sql/test/Users/Tests/privs2.stable.out > sql/test/Users/Tests/test_privs2_p1.SQL.py > sql/test/Users/Tests/test_privs2_p1.stable.err > sql/test/Users/Tests/test_privs2_p1.stable.out > sql/test/Users/Tests/test_privs2_p2.SQL.py > sql/test/Users/Tests/test_privs2_p2.stable.err > sql/test/Users/Tests/test_privs2_p2.stable.out > sql/test/Users/Tests/test_privs_p1.SQL.py > sql/test/Users/Tests/test_privs_p1.stable.err > sql/test/Users/Tests/test_privs_p1.stable.out > sql/test/Users/Tests/test_privs_p2.SQL.py > sql/test/Users/Tests/test_privs_p2.stable.err > sql/test/Users/Tests/test_privs_p2.stable.out > sql/test/Users/privs.sql > sql/test/Users/privs2.sql > sql/test/Users/test_privs.sql > sql/test/Users/test_privs2.sql > Modified Files: > monetdb5/mal/mal_dataflow.c > monetdb5/mal/mal_interpreter.c > monetdb5/mal/mal_resource.c > sql/backends/monet5/rel_bin.c > sql/backends/monet5/sql_gencode.c > sql/backends/monet5/sql_gencode.h > sql/backends/monet5/sql_scenario.c > sql/common/sql_backend.c > sql/common/sql_backend.h > sql/server/rel_updates.c > sql/test/Users/Tests/All > Branch: default > Log Message: > > Merge with Feb2013 branch. > > > diffs (truncated from 1336 to 300 lines): > > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -1033,8 +1033,6 @@ rel_parse_value(mvc *m, char *query, cha > m->argc = 0; > m->sym = NULL; > m->errstr[0] = '\0'; > - /* via views we give access to protected objects */ > - m->user_id = USER_MONETDB; > > (void) sqlparse(m); /* blindly ignore errors */ > > @@ -2648,8 +2646,6 @@ sql_parse(mvc *m, sql_allocator *sa, cha > m->sym = NULL; > m->errstr[0] = '\0'; > m->errstr[ERRSIZE-1] = '\0'; > - /* via views we give access to protected objects */ > - m->user_id = USER_MONETDB; > > /* create private allocator */ > m->sa = (sa)?sa:sa_create(); > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -54,7 +54,8 @@ > #include <rel_bin.h> > > static int _dumpstmt(backend *sql, MalBlkPtr mb, stmt *s); > -static void backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s); > +static int backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s); > +static int backend_create_func(backend *be, sql_func *f); > > /* > * @+ MAL code support > @@ -261,7 +262,7 @@ relational_func_create_result( mvc *sql, > } > > > -static void > +static int > _create_relational_function(mvc *m, char *name, sql_rel *rel, stmt *call) > { > sql_rel *r; > @@ -314,16 +315,18 @@ _create_relational_function(mvc *m, char > } > } > > - backend_dumpstmt(be, curBlk, s); > + if (backend_dumpstmt(be, curBlk, s) < 0) > + return -1; > /* SQL function definitions meant for inlineing should not be optimized before */ > varSetProp(curBlk, getArg(curInstr, 0), sqlfunctionProp, op_eq, NULL); > addQueryToCache(c); > if (backup) > c->curprg = backup; > + return 0; > } > > /* stub and remote function */ > -static void > +static int > _create_relational_remote(mvc *m, char *name, sql_rel *rel, stmt *call, prop *prp) > { > Client c = MCgetClient(m->clientid); > @@ -346,7 +349,8 @@ _create_relational_remote(mvc *m, char * > /* dirty hack, rename (change first char of name) L->l, local > * functions name start with 'l' */ > name[0] = 'l'; > - _create_relational_function(m, name, rel, call); > + if (_create_relational_function(m, name, rel, call) < 0) > + return -1; > > /* create stub */ > name[0] = old; > @@ -462,18 +466,19 @@ _create_relational_remote(mvc *m, char * > if (backup) > c->curprg = backup; > name[0] = old; /* make sure stub is called */ > + return 0; > } > > -static void > +static int > monet5_create_relational_function(mvc *m, char *name, sql_rel *rel, stmt *call) > > { > prop *p = NULL; > > if (rel && (p = find_prop(rel->p, PROP_REMOTE)) != NULL) > - _create_relational_remote(m, name, rel, call, p); > + return _create_relational_remote(m, name, rel, call, p); > else > - _create_relational_function(m, name, rel, call); > + return _create_relational_function(m, name, rel, call); > } > > /* > @@ -663,7 +668,8 @@ dump_joinN(backend *sql, MalBlkPtr mb, s > InstrPtr q; > int op1, op2, op3 = 0; > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > > @@ -1121,7 +1127,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > node *n; > char *mod, *fimp; > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > > @@ -1632,7 +1639,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > char *mod, *fimp; > int l = _dumpstmt(sql, mb, s->op1); > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > if (s->op1->nrcols && strcmp(fimp, "not_uniques") == 0) { > @@ -1666,7 +1674,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > int l = _dumpstmt(sql, mb, s->op1); > int r = _dumpstmt(sql, mb, s->op2); > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > > @@ -1713,7 +1722,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > /* dump operands */ > _dumpstmt(sql, mb, s->op1); > > - backend_create_func(sql, f->func); > + if (backend_create_func(sql, f->func) < 0) > + return -1; > mod = sql_func_mod(f->func); > fimp = sql_func_imp(f->func); > if (s->nrcols) { > @@ -1769,7 +1779,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > /* dump args */ > if (s->op1) > _dumpstmt(sql, mb, s->op1); > - monet5_create_relational_function(sql->mvc, fimp, rel, s); > + if (monet5_create_relational_function(sql->mvc, fimp, rel, s) < 0) > + return -1; > > q = newStmt(mb, mod, fimp); > q = relational_func_create_result(sql->mvc, mb, q, rel); > @@ -1791,7 +1802,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > int restype = s->op4.aggrval->res.type->localtype; > int complex_aggr = 0; > > - backend_create_func(sql, s->op4.aggrval->aggr); > + if (backend_create_func(sql, s->op4.aggrval->aggr) < 0) > + return -1; > mod = s->op4.aggrval->aggr->mod; > aggrfunc = s->op4.aggrval->aggr->imp; > > @@ -2242,7 +2254,8 @@ static void setCommitProperty(MalBlkPtr > varSetProperty(mb, getArg(getInstrPtr(mb,0),0), "autoCommit","=", &cst); > } > > -static void backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s) > +static int > +backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s) > { > mvc *c = be->mvc; > stmt **stmts = stmt_array(c->sa, s); > @@ -2259,9 +2272,11 @@ static void backend_dumpstmt(backend *be > clear_stmts(stmts); > while(stmts[nr]) { > stmt *s = stmts[nr++]; > - (void)_dumpstmt(be, mb, s); > + if (_dumpstmt(be, mb, s) < 0) > + return -1; > } > - (void)_dumpstmt(be, mb, s); > + if (_dumpstmt(be, mb, s) < 0) > + return -1; > > be->mvc_var = old_mv; > if (c->caching && (c->type == Q_SCHEMA || c->type == Q_TRANS)) { > @@ -2276,9 +2291,10 @@ static void backend_dumpstmt(backend *be > q->barrier = RETURNsymbol; > } > pushEndInstruction(mb); > + return 0; > } > > -void > +int > backend_callinline(backend *be, Client c, stmt *s ) > { > mvc *m = be->mvc; > @@ -2312,8 +2328,10 @@ backend_callinline(backend *be, Client c > } > } > } > - backend_dumpstmt(be, curBlk, s); > + if (backend_dumpstmt(be, curBlk, s) < 0) > + return -1; > c->curprg->def = curBlk; > + return 0; > } > > Symbol > @@ -2336,8 +2354,9 @@ backend_dumpproc(backend *be, Client c, > sql_subfunc *cq = sql_find_func(m->sa, sys, "keepcall", NR_KEEPCALL_ARGS, F_PROC); > > assert(kq && cq); > - backend_create_func(be, kq->func); > - backend_create_func(be, cq->func); > + if (backend_create_func(be, kq->func) < 0 || > + backend_create_func(be, cq->func) < 0) > + return NULL; > /* only needed once */ > m->history = 2; > } > @@ -2383,7 +2402,8 @@ backend_dumpproc(backend *be, Client c, > } > } > > - backend_dumpstmt(be, mb, s); > + if (backend_dumpstmt(be, mb, s) < 0) > + return NULL; > Toptimize = GDKusec(); > Tparse = Toptimize - m->Tparse; > > @@ -2460,53 +2480,6 @@ backend_call(backend *be, Client c, cq * > } > } > > -void > -monet5_create_table_function(ptr M, char *name, sql_rel *rel, sql_table *t) > -{ > - sql_rel *r; > - mvc *m = (mvc*)M; > - Client c = MCgetClient(m->clientid); > - backend *be = (backend *) c->sqlcontext; > - MalBlkPtr curBlk = 0; > - InstrPtr curInstr = 0; > - Symbol backup = NULL; > - stmt *s; > - > - (void)t; > - r = rel_optimizer(m, rel); > - s = rel_bin(m, r); > - > - if (s->type == st_list && s->nrcols == 0 && s->key) { > - /* row to columns */ > - node *n; > - list *l = sa_list(m->sa); > - > - for(n=s->op4.lval->h; n; n = n->next) > - list_append(l, const_column(m->sa, n->data)); > - s = stmt_list(m->sa, l); > - } > - s = stmt_table(m->sa, s, 1); > - s = stmt_return(m->sa, s, 0); > - > - backup = c->curprg; > - c->curprg = newFunction(userRef,putName(name,strlen(name)), FUNCTIONsymbol); > - > - curBlk = c->curprg->def; > - curInstr = getInstrPtr(curBlk, 0); > - > - curInstr = table_func_create_result(curBlk, curInstr, t); > - setVarUDFtype(curBlk,0); > - > - /* no ops */ > - > - backend_dumpstmt(be, curBlk, s); > - /* SQL function definitions meant for inlineing should not be optimized before */ > - varSetProp(curBlk, getArg(curInstr, 0), sqlfunctionProp, op_eq, NULL); > - addQueryToCache(c); > - if (backup) > - c->curprg = backup; > -} > - > int > monet5_resolve_function(ptr M, sql_func *f) > { > @@ -2549,7 +2522,7 @@ monet5_resolve_function(ptr M, sql_func > } > > /* TODO handle aggr */ > -void > +static int > backend_create_func(backend *be, sql_func *f) > { > mvc *m = be->mvc; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: MonetDB: Feb2013 - fixes and test added for bug 3230
by Stefan Manegold 13 Feb '13

13 Feb '13

Niels, could you please also check in the missing file sql/test/Users/privs2.sql Thanks! Stefan On Wed, Feb 13, 2013 at 04:00:35PM +0100, Niels Nes wrote: > Changeset: fffd420f1ee8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fffd420f1ee8 > Added Files: > sql/test/Users/Tests/privs2.sql.src > sql/test/Users/Tests/privs2.stable.err > sql/test/Users/Tests/privs2.stable.out > sql/test/Users/Tests/test_privs2_p1.SQL.py > sql/test/Users/Tests/test_privs2_p1.stable.err > sql/test/Users/Tests/test_privs2_p1.stable.out > sql/test/Users/Tests/test_privs2_p2.SQL.py > sql/test/Users/Tests/test_privs2_p2.stable.err > sql/test/Users/Tests/test_privs2_p2.stable.out > Modified Files: > sql/backends/monet5/rel_bin.c > sql/backends/monet5/sql_gencode.c > sql/backends/monet5/sql_gencode.h > sql/backends/monet5/sql_scenario.c > sql/common/sql_backend.c > sql/common/sql_backend.h > sql/test/Users/Tests/All > Branch: Feb2013 > Log Message: > > fixes and test added for bug 3230 > > > diffs (truncated from 901 to 300 lines): > > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -1124,8 +1124,6 @@ rel_parse_value(mvc *m, char *query, cha > m->argc = 0; > m->sym = NULL; > m->errstr[0] = '\0'; > - /* via views we give access to protected objects */ > - m->user_id = USER_MONETDB; > > (void) sqlparse(m); /* blindly ignore errors */ > > @@ -2741,8 +2739,6 @@ sql_parse(mvc *m, sql_allocator *sa, cha > m->sym = NULL; > m->errstr[0] = '\0'; > m->errstr[ERRSIZE-1] = '\0'; > - /* via views we give access to protected objects */ > - m->user_id = USER_MONETDB; > > /* create private allocator */ > m->sa = (sa)?sa:sa_create(); > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -54,7 +54,8 @@ > #include <rel_bin.h> > > static int _dumpstmt(backend *sql, MalBlkPtr mb, stmt *s); > -static void backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s); > +static int backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s); > +static int backend_create_func(backend *be, sql_func *f); > > /* > * @+ MAL code support > @@ -261,7 +262,7 @@ relational_func_create_result( mvc *sql, > } > > > -static void > +static int > _create_relational_function(mvc *m, char *name, sql_rel *rel, stmt *call) > { > sql_rel *r; > @@ -314,16 +315,18 @@ _create_relational_function(mvc *m, char > } > } > > - backend_dumpstmt(be, curBlk, s); > + if (backend_dumpstmt(be, curBlk, s) < 0) > + return -1; > /* SQL function definitions meant for inlineing should not be optimized before */ > varSetProp(curBlk, getArg(curInstr, 0), sqlfunctionProp, op_eq, NULL); > addQueryToCache(c); > if (backup) > c->curprg = backup; > + return 0; > } > > /* stub and remote function */ > -static void > +static int > _create_relational_remote(mvc *m, char *name, sql_rel *rel, stmt *call, prop *prp) > { > Client c = MCgetClient(m->clientid); > @@ -346,7 +349,8 @@ _create_relational_remote(mvc *m, char * > /* dirty hack, rename (change first char of name) L->l, local > * functions name start with 'l' */ > name[0] = 'l'; > - _create_relational_function(m, name, rel, call); > + if (_create_relational_function(m, name, rel, call) < 0) > + return -1; > > /* create stub */ > name[0] = old; > @@ -462,18 +466,19 @@ _create_relational_remote(mvc *m, char * > if (backup) > c->curprg = backup; > name[0] = old; /* make sure stub is called */ > + return 0; > } > > -static void > +static int > monet5_create_relational_function(mvc *m, char *name, sql_rel *rel, stmt *call) > > { > prop *p = NULL; > > if (rel && (p = find_prop(rel->p, PROP_REMOTE)) != NULL) > - _create_relational_remote(m, name, rel, call, p); > + return _create_relational_remote(m, name, rel, call, p); > else > - _create_relational_function(m, name, rel, call); > + return _create_relational_function(m, name, rel, call); > } > > /* > @@ -663,7 +668,8 @@ dump_joinN(backend *sql, MalBlkPtr mb, s > InstrPtr q; > int op1, op2, op3 = 0; > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > > @@ -1121,7 +1127,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > node *n; > char *mod, *fimp; > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > > @@ -1632,7 +1639,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > char *mod, *fimp; > int l = _dumpstmt(sql, mb, s->op1); > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > if (s->op1->nrcols && strcmp(fimp, "not_uniques") == 0) { > @@ -1666,7 +1674,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > int l = _dumpstmt(sql, mb, s->op1); > int r = _dumpstmt(sql, mb, s->op2); > > - backend_create_func(sql, s->op4.funcval->func); > + if (backend_create_func(sql, s->op4.funcval->func) < 0) > + return -1; > mod = sql_func_mod(s->op4.funcval->func); > fimp = sql_func_imp(s->op4.funcval->func); > > @@ -1713,7 +1722,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > /* dump operands */ > _dumpstmt(sql, mb, s->op1); > > - backend_create_func(sql, f->func); > + if (backend_create_func(sql, f->func) < 0) > + return -1; > mod = sql_func_mod(f->func); > fimp = sql_func_imp(f->func); > if (s->nrcols) { > @@ -1769,7 +1779,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > /* dump args */ > if (s->op1) > _dumpstmt(sql, mb, s->op1); > - monet5_create_relational_function(sql->mvc, fimp, rel, s); > + if (monet5_create_relational_function(sql->mvc, fimp, rel, s) < 0) > + return -1; > > q = newStmt(mb, mod, fimp); > q = relational_func_create_result(sql->mvc, mb, q, rel); > @@ -1791,7 +1802,8 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > int restype = s->op4.aggrval->res.type->localtype; > int sum_or_prod = 0; > > - backend_create_func(sql, s->op4.aggrval->aggr); > + if (backend_create_func(sql, s->op4.aggrval->aggr) < 0) > + return -1; > mod = s->op4.aggrval->aggr->mod; > aggrfunc = s->op4.aggrval->aggr->imp; > > @@ -2241,7 +2253,8 @@ static void setCommitProperty(MalBlkPtr > varSetProperty(mb, getArg(getInstrPtr(mb,0),0), "autoCommit","=", &cst); > } > > -static void backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s) > +static int > +backend_dumpstmt(backend *be, MalBlkPtr mb, stmt *s) > { > mvc *c = be->mvc; > stmt **stmts = stmt_array(c->sa, s); > @@ -2258,9 +2271,11 @@ static void backend_dumpstmt(backend *be > clear_stmts(stmts); > while(stmts[nr]) { > stmt *s = stmts[nr++]; > - (void)_dumpstmt(be, mb, s); > + if (_dumpstmt(be, mb, s) < 0) > + return -1; > } > - (void)_dumpstmt(be, mb, s); > + if (_dumpstmt(be, mb, s) < 0) > + return -1; > > be->mvc_var = old_mv; > if (c->caching && (c->type == Q_SCHEMA || c->type == Q_TRANS)) { > @@ -2275,9 +2290,10 @@ static void backend_dumpstmt(backend *be > q->barrier = RETURNsymbol; > } > pushEndInstruction(mb); > + return 0; > } > > -void > +int > backend_callinline(backend *be, Client c, stmt *s ) > { > mvc *m = be->mvc; > @@ -2311,8 +2327,10 @@ backend_callinline(backend *be, Client c > } > } > } > - backend_dumpstmt(be, curBlk, s); > + if (backend_dumpstmt(be, curBlk, s) < 0) > + return -1; > c->curprg->def = curBlk; > + return 0; > } > > Symbol > @@ -2335,8 +2353,9 @@ backend_dumpproc(backend *be, Client c, > sql_subfunc *cq = sql_find_func(m->sa, sys, "keepcall", NR_KEEPCALL_ARGS, F_PROC); > > assert(kq && cq); > - backend_create_func(be, kq->func); > - backend_create_func(be, cq->func); > + if (backend_create_func(be, kq->func) < 0 || > + backend_create_func(be, cq->func) < 0) > + return NULL; > /* only needed once */ > m->history = 2; > } > @@ -2382,7 +2401,8 @@ backend_dumpproc(backend *be, Client c, > } > } > > - backend_dumpstmt(be, mb, s); > + if (backend_dumpstmt(be, mb, s) < 0) > + return NULL; > Toptimize = GDKusec(); > Tparse = Toptimize - m->Tparse; > > @@ -2459,53 +2479,6 @@ backend_call(backend *be, Client c, cq * > } > } > > -void > -monet5_create_table_function(ptr M, char *name, sql_rel *rel, sql_table *t) > -{ > - sql_rel *r; > - mvc *m = (mvc*)M; > - Client c = MCgetClient(m->clientid); > - backend *be = (backend *) c->sqlcontext; > - MalBlkPtr curBlk = 0; > - InstrPtr curInstr = 0; > - Symbol backup = NULL; > - stmt *s; > - > - (void)t; > - r = rel_optimizer(m, rel); > - s = rel_bin(m, r); > - > - if (s->type == st_list && s->nrcols == 0 && s->key) { > - /* row to columns */ > - node *n; > - list *l = sa_list(m->sa); > - > - for(n=s->op4.lval->h; n; n = n->next) > - list_append(l, const_column(m->sa, n->data)); > - s = stmt_list(m->sa, l); > - } > - s = stmt_table(m->sa, s, 1); > - s = stmt_return(m->sa, s, 0); > - > - backup = c->curprg; > - c->curprg = newFunction(userRef,putName(name,strlen(name)), FUNCTIONsymbol); > - > - curBlk = c->curprg->def; > - curInstr = getInstrPtr(curBlk, 0); > - > - curInstr = table_func_create_result(curBlk, curInstr, t); > - setVarUDFtype(curBlk,0); > - > - /* no ops */ > - > - backend_dumpstmt(be, curBlk, s); > - /* SQL function definitions meant for inlineing should not be optimized before */ > - varSetProp(curBlk, getArg(curInstr, 0), sqlfunctionProp, op_eq, NULL); > - addQueryToCache(c); > - if (backup) > - c->curprg = backup; > -} > - > int > monet5_resolve_function(ptr M, sql_func *f) > { > @@ -2548,7 +2521,7 @@ monet5_resolve_function(ptr M, sql_func > } > > /* TODO handle aggr */ > -void > +static int > backend_create_func(backend *be, sql_func *f) > { > mvc *m = be->mvc; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Coercions optimizer name bug
by Pete Hollobon 12 Feb '13

12 Feb '13

Hi, The name of the coercions optimizer in the codes array in opt_wrapper.c is wrong (currently it's "coercion"), and so it is never run. This patch fixes: diff -r 6e9b305e92e2 monetdb5/optimizer/opt_wrapper.c --- a/monetdb5/optimizer/opt_wrapper.c Sun Feb 10 17:39:23 2013 +0100 +++ b/monetdb5/optimizer/opt_wrapper.c Tue Feb 12 11:33:20 2013 +0000 @@ -84,7 +84,7 @@ {"aliases", &OPTaliasesImplementation}, {"centipede", &OPTcentipedeImplementation}, {"cluster", &OPTclusterImplementation}, - {"coercion", &OPTcoercionImplementation}, + {"coercions", &OPTcoercionImplementation}, {"commonTerms", &OPTcommonTermsImplementation}, {"compression", &OPTcompressionImplementation}, {"costModel", &OPTcostModelImplementation}, Regards, Pete

1 0

No mclient prompt. Connection hangs.
by Tapomay Dey 06 Feb '13

06 Feb '13

I have started facing issues when trying to connect to the DB using mclient remotely/locally. All my services have started failing as the DB is no longer responding(using JDBC). There is hardly any load on the DB - no memory/cpu usage. Although the monetdb status shows a health of only 47%(What does this mean?) When I do a sockstat on the merovingian I can see the client's socket. But when I sockstat on the mserver5 process I don't see that. (FYI- when I am able to connect using mclient I see this socket in mserver's sockstat instead of monetdbd) I do get the mclient prompt sometimes with full functionality. But most of the times it just gets stuck and the prompt never occurs. Any more diagnostics that I can perform to resolve this? When I ctrl+c on the mclient the mclient process stops but socket stays in ESTABLISHED state on server. Verified it by comparign port no. before and after killing mclient and making sure no one else is connected. Regards, Tapomay.

1 1

Python API's: sql.Binary() fix
by Mate Birkas 04 Feb '13

04 Feb '13

Hi, I may have found a bug in sql.Binary(). Here is the patch: --- pythonize.py 2013-02-03 15:22:09.217186598 +0100 +++ pythonize.py.fix 2013-02-03 15:23:34.613187703 +0100 @@ -119,7 +119,7 @@ def Binary(data): """returns binary encoding of data""" - return ''.join([hex(ord(i))[2:] for i in data]).upper() + return ''.join(["%02X" % ord(i) for i in data]) def DateFromTicks(ticks): """Convert ticks to python Date""" The hex() built-in returns three character string for integers lower than 15. This combined with the original code's [2:] results in corrupt string representations of binary objects having bytes with these values. cheers Mate

3 2

BATsubselect: invalid argument: s must be sorted.
by Tapomay Dey 30 Jan '13

30 Jan '13

I keep getting a GDK error as follows: Query succeeds if I change the order of filters in where clause!!! sql>SELECT count(*****_sales) AS "Sales" more>FROM tableName more>WHERE client_id = 53 more>AND vendor_id = 1; GDK reported error. BATsubselect: invalid argument: s must be sorted. sql> sql> sql>SELECT count(*****_sales) AS "Sales" more>FROM tableName more>WHERE vendor_id = 1 more>AND client_id = 53; +--------+ | Sales | +========+ | 109593 | +--------+ 1 tuple (571.649ms) I am running on an older revision of Feb13 branch. Ubuntu 12.10 Attaching the MAL. PS: http://www.mail-archive.com/checkin-list@monetdb.org/msg09267.html Thanks and Regards, Tapomay

1 1

Re: MonetDB: default - try to fix DataCell compilation error, by sim...
by Martin Kersten 28 Jan '13

28 Jan '13

Yes, that is correct. On 1/28/13 5:10 PM, Jennie Zhang wrote: > Changeset: 50111ec8e490 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=50111ec8e490 > Modified Files: > sql/backends/monet5/datacell/basket.c > Branch: default > Log Message: > > try to fix DataCell compilation error, by simply removing the unused variable that causes the problem. > > Mtest shows this change doesn't make the code worse. > > @Martin: please double check! > > > diffs (24 lines): > > diff --git a/sql/backends/monet5/datacell/basket.c b/sql/backends/monet5/datacell/basket.c > --- a/sql/backends/monet5/datacell/basket.c > +++ b/sql/backends/monet5/datacell/basket.c > @@ -41,10 +41,6 @@ str protocolname[4] = { "<unknown>", "TC > > BSKTbasketRec *baskets; /* the datacell catalog */ > int bsktTop = 0, bsktLimit = 0; > -static MT_Lock bsktLock MT_LOCK_INITIALIZER("bsktLock"); > - > -#define lockBSKTbasketCatalog() MT_lock_set(&bsktLock, "basket"); > -#define unlockBSKTbasketCatalog() MT_lock_unset(&bsktLock, "basket"); > > /* We have to obtain the precise wall-clock time > * This is not produced by GDKusec, which returns microseconds > @@ -90,9 +86,6 @@ static int BSKTnewEntry(void) > if (bsktLimit == 0) { > bsktLimit = MAXBSK; > baskets = (BSKTbasketRec *) GDKzalloc(bsktLimit * sizeof(BSKTbasketRec)); > -#ifdef NEED_MT_LOCK_INIT > - MT_lock_init(&bsktLock, "basket"); > -#endif > bsktTop = 1; /* entry 0 is used as non-initialized */ > } else if (bsktTop == bsktLimit) { > bsktLimit += MAXBSK; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: Issues with MonetDB
by Stefan Manegold 23 Jan '13

23 Jan '13

Dear Ashish, thank you very much for your feedback. Did you already try Oct2012-SP2, upcoming Oct2012-SP3 (currently available from http://dev.monetdb.org/downloads/testing/ ; will be formally released soon), or the Feb2013 branch from Mercurial (http://dev.monetdb.org/hg/MonetDB/branches)? To be able to further debug the problem on our side, we would need all the ingredients required to reproduce the problem you're experiencing (code, data, queries, detailed instructions). Best, Stefan Ashish Kumar Singh <ashishkumar.singh(a)altair.com> wrote: Dear Monet DB team, We are facing few very critical issues in monetdb in one of the reporting applications we are trying to put up using it. First of all performance improvements are really great using monetdb but we are facing some issues related to data consistency as described below: 1. Duplicate tuples are getting inserted few times. 2. Updates to database are not available for read immediately , sometimes wrong values are returned by database. After certain number of retires these becomes available though can go to 10 times of retires. We are facing this for Oct 2012 Sp1 build. These data related bug are not acceptable in our usage scenario and this will be the case with other folks too! Question to the list is there any released version backwards / upwards which can be used instead of October 2012 Sp1 which does not have these issues? I have filled a bug also for this at http://bugs.monetdb.org/show_bug.cgi?id=3222 Regards, Ashish

2 2

Announcement: New Oct2012-SP3 Bugfix release of MonetDB suite
by Sjoerd Mullender 22 Jan '13

22 Jan '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP3 bugfix release Bug Fixes * 2291: small doubles end up as NULL after arithmetic * 2579: SQL: incorrect cast from double to int * 2659: tuples INTERSECT vs count (... INTERSECT ...) differs * 2805: row_number doesn't work in complex query * 2886: warning: Catalog parameter '' ignored * 2977: timestamp minus date results in program error * 2978: now() + 1 results in illegal argument * 2999: Up-cast from SQL drops order information? * 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause * 3124: insert into table with two foreign keys crashes mserver5 * 3136: Memleak (in querycache?) while querying * 3143: Type resolution error in SQL procedural code * 3163: mserver5 crashes with large number of concurrent clients * 3168: crash when insert varchar/char/clob with default NULL * 3169: php driver uses call-time pass-by-reference * 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either * 3176: Same query and different results * 3181: Join query returns zero results when there is an index on the tables * 3185: inconsistency between documentation and implementation of command line options * 3186: case evaluted to early * 3188: Storage of client history * 3189: Overflow in calculation * 3190: When subtracting two columns of type "timestamp", no results are returned * 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed. * 3192: "No such column with index" error if CASE in SELECT clause * 3199: Inconspicuous sequence of prepare, execute, commit and close produces error * 3205: Inet datatype handling inconsistent between int, cast( as inet) and string * 3208: Unable to execute prepared statements with a large number of arguments * 3209: exp_bin: Assertion `0' failed * 3212: Query cache does not work properly

1 0

Issues with MonetDB
by Ashish Kumar Singh 22 Jan '13

22 Jan '13

Dear Monet DB team, We are facing few very critical issues in monetdb in one of the reporting applications we are trying to put up using it. First of all performance improvements are really great using monetdb but we are facing some issues related to data consistency as described below: 1. Duplicate tuples are getting inserted few times. 2. Updates to database are not available for read immediately , sometimes wrong values are returned by database. After certain number of retires these becomes available though can go to 10 times of retires. We are facing this for Oct 2012 Sp1 build. These data related bug are not acceptable in our usage scenario and this will be the case with other folks too! Question to the list is there any released version backwards / upwards which can be used instead of October 2012 Sp1 which does not have these issues? I have filled a bug also for this at http://bugs.monetdb.org/show_bug.cgi?id=3222 Regards, Ashish

2 1

Fw: Inconsistency in data selection. count*
by Tapomay Dey 21 Jan '13

21 Jan '13

Forwarding to dev forum. Not appropriate for user list. ----- Forwarded Message ----- From: Tapomay Dey <tapomay(a)yahoo.com> To: Communication channel for Monet DB users <users-list(a)monetdb.org> Sent: Monday, January 21, 2013 2:39 PM Subject: Inconsistency in data selection. count* select count(*) from table; Results: 168466 rows select count(*) from table where column=0; Results: 168466 rows select count(*) from table where column<>0; Results: 168466 rows When I do a "select column from table where client_id=0 limit 5;" I see correct non-zero values in the result although the selection condition is that the column value must be 0. I put the data on another instance/machine - I don't see this issue. Plz find a screenshot@ https://picasaweb.google.com/115470548136322675830/January212013?authuser=0… Another case: sql>select count(*) from table2 where column2=0; | 391326 | sql>select count(*) from table2 where column2<>0; | 141108 | sql>select count(*) from table2; | 399326 | sql>select column2 from table2 where column2=0 limit 5; +-----------+ | column2 | +===========+ | 53 | | 53 | | 53 | | 53 | | 53 | +-----------+ 5 tuples (156.631ms) Regards, Tapomay _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 2

Error in merovingian log while performing a specific select: gdk_batop.c insert_string_bat: Assertion failed
by Tapomay Dey 18 Jan '13

18 Jan '13

ERR msearch_stats_db[4515]: mserver5: gdk_batop.c:175: insert_string_bat: Assertion `v >= ((var_t) (((1<<10) * sizeof(stridx_t)) >> 0))' failed. I see above error in merovingian.log while performing a specific select query and the client says "Connection terminated": select id, col2, col3, col4, col5, timestamp, count(*) from stats_daily where id=38 group by "id", "col2", "col3", "col4", "timestamp", "col5" having count(*) >1; There's a unique key on the table stats_daily on (id,col2,col3, col4, timestamp, col5). However following query succeeds(it just has more selection conditions): select id, col2, col3, col4, col5, timestamp, count(*) from stats_daily where col2 <> '' and col3 <> '' and col4 <> '' and col5 <> '' and id <> 0 group by "id", "col2", "col3", "col4", "timestamp", "col5" having count(*) >1; One thing I know for sure is that there was an error in inserting records into the DB at some point before: Following is an excerpt from the merovingian logs: 2013-01-10 10:40:49 ERR msearch_stats_db[19210]: !mal_mapi.listen: expected filedescriptor, but received something else 2013-01-10 10:40:57 ERR merovingian[8996]: client error: client [XX.XX.XX.XX]:51573 sent challenge in incomplete block: 2013-01-10 10:45:29 ERR msearch_stats_db[19210]: mserver5: bat_storage.c:40: delta_bind_del: Assertion `b' failed. 2013-01-10 10:45:29 ERR msearch_stats_db[19210]: mserver5: bat_storage.c:40: delta_bind_del: Assertion `b' failed. 2013-01-10 23:48:18 ERR msearch_stats_db[31637]: !mal_mapi.listen: expected filedescriptor, but received something else 2013-01-10 23:48:19 ERR msearch_stats_db[31637]: !mal_mapi.listen: expected filedescriptor, but received something else 2013-01-10 23:48:32 ERR msearch_stats_db[31637]: !FATAL: BBPdir: subcommit attempted without backup BBP.dir. 2013-01-11 04:29:31 ERR msearch_stats_db[4480]: mserver5: gdk_batop.c:175: insert_string_bat: Assertion `v >= ((var_t) (((1<<10) * sizeof(stridx_t)) >> 0))' failed. Also, I keep getting "!mal_mapi.listen: expected filedescriptor, but received something else" almost every second(Performing continuous inserts into the db). Also attaching a subset of mervingian log that may contain other kinds of ERR s than above. What other diagnostics can I do? PS: Running on Feb2013 branch. changeset: 38061:d1a99a110b63 Ubuntu 12.10 64 bit. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 3

MonetDB Clustering
by nguewo dominique 15 Jan '13

15 Jan '13

Dear Developers, I have a database in two systems with the same name. They both have a table with the same name. I have set the shared properties as follows. laptop1% monetdb -h laptop2-P mypassphrase set shared=pump/1/ pump laptop1% monetdb -h laptop3-P mypassphrase set shared=pump/2/ pump From a master node om trying to query the database to access data from the two tables but its not working. How t go about it? _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Strange behaviour with MonetDB
by Ashish Kumar Singh 15 Jan '13

15 Jan '13

Hi List , We are facing one strange behavior with Monetdb. We have a simple table called test with following structure: CREATE SEQUENCE "testtable_seq" as bigint START WITH 1; CREATE TABLE testtable (dim_id bigint DEFAULT NEXT VALUE FOR "testtable_seq", dim_type varchar(100) NOT NULL, dim_value varchar(100) NOT NULL, CONSTRAINT unique_dimension UNIQUE(dim_type,dim_value) , CONSTRAINT pk_testtable PRIMARY KEY (dim_id)); We have a program which reads and writes to this table along with other tables using JDBC. We are facing below problems interminently: 1. When we query back data from this table with query like : select dim_id,dim_type,dim_value from testtable where dim_type='user' and dim_value='user1' we are getting back ID for some other value present in testable . 2. We are also seeing duplicate values getting inserted in to this table. Please suggest , we are using Oct SP1 build. Regards, Ashish _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

undefined symbol: msab_init while starting monetdb
by Tapomay Dey 14 Jan '13

14 Jan '13

I keep getting the following error while trying to start monetdb. msearch_stats_db[20642]: /usr/bin/mserver5: symbol lookup error: /usr/bin/mserver5: undefined symbol: msab_init FYI inititally I was getting !FATAL: GDKlockHome: could not create /usr/local/var/monetdb5/dbfarm/demo which I solved by changing the permissions of the specified folder following which I keep getting the above error. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

Bulk Load, Same Table, Sequential COPY INTO, MAPI
by Brandon Jackson 13 Jan '13

13 Jan '13

Dear developers, In the last two weeks we have made a lot of progress on the MonetDB Streaming Bulk Loader Step in the Pentaho Data Integration. Yesterday, I tried to put it into use similar to a larger scale production use and started getting some errors along the lines of: "COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead" Not sure when these clear up. In this case after previous errors like this, I established a new connection and wanted to simply create a table. sql>CREATE TABLE "testing"."fact_cen_hist" ( "CENSUS_SK" INT , "FACILITY_SK" INT , "RESIDENT_SK" INT , "RESIDENT_LOCATION_SK" INT , "DISCHARGED_TO" CLOB , "PAYOR" VARCHAR(20) , "PAYOR_CATEGORY" VARCHAR(20) , "CENSUS_STATUS" VARCHAR(50) , "CENSUS_EVENT" VARCHAR(50) , "IS_ACTIVE" INT , "IS_MEDICAID_DAY" INT , "IS_MEDICARE_DAY" INT , "IS_HOSPICE_DAY" INT , "IS_COMMERCIAL_INSURANCE_DAY" INT , "IS_PRIVATE_PAY_DAY" INT , "IS_MANAGED_CARE_DAY" INT , "IS_MEDICARE_SECOND_PAYOR" INT , "IS_MEDICAID_PENDING_DAY" INT , "DATE_OF_STAY" INT , "NEXT_EVENT_DATE" INT , "DATE_BEFORE_NEXT_EVENT" INT , "CENSUS_EVENT_DATE" INT , "IS_MEMORY_CARE_DAY" INT , "IS_AL_DAY" INT , "IS_SNF_DAY" INT , "IS_DISCHARGED" INT , "IS_RTA" INT , "ROOM_KEY" VARCHAR(5) , "IS_BED_HOLD" INT , "BILLABLE_DAY" INT , "HEAD_IN_BED_DAY" INT , "FINANCIAL_DAY" INT , "IS_ADMISSION" INT , "IS_READMISSION" INT , "IS_EXPIRED" INT , "PAYOR_SK" INT , "IS_SUBMISSION" INT , "SPLIT_CENSUS" INT ) ; operation successful (2.7s) COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead sql> ---- Generally speaking, our bulk loader reads in a chunk of rows into memory say 10k, 20k or even 100k or 1 Million rows, then it will generate a COPY INTO, know how many rows are outbound; stream the rows through a BufferedWriter with all the new lines, and reading from BufferedReader and waiting for prompts etc. Then PDI will fetch another chunk of rows and repeat the process. I noticed that when using the MAPI API and trying to append a COMMIT in the output brought back an error saying that COMMIT is not allowed in auto commit mode, so it looks like after each big COPY INTO statement sends its complete pack of rows, with the new line stuff, should immediately try to COMMIT. What I want to figure out is why after the first COPY INTO of however many rows, 10k for example, would I get a message about failure to commit because of concurrency, ROLLBACK instead. Once that happens, is MonetDB in a funk? Do I need to shut it down and restart? What can I do to provide more helpful information to understand the nature of the problem? I read on: http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/LoadingBulkData That I could supply a guess much larger on the first COPY INTO so MonetDB can prepare itself for a huge influx of rows, despite getting a smaller batch. Example in my mind is know that my table has roughly 6 million rows, so let 6 million be the guess, then have really accurate calls where COPY INTO is told 20k rows are on the way in each subsequent call leading up to the 6 million that I guessed would be necessary. It worries me that this appears to be brittle. We might produce 10 million rows in a source table and want to stream it over. At current, it looks like COMMIT concurrency errors would come up as we sequentially do COPY INTO's building up to finally having the bulk load complete. Any ideas or feedback is very much appreciated. I can carefully provide code, more detail and follow any procedures necessary to help diagnose the issue. Sample data is also available. Thanks ahead of time, Brandon Jackson _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Solution: jdbcclient.jar is always prompting even there is password list in .monetdb
by sigit kartiko 10 Jan '13

10 Jan '13

I use jdbcclient.jar to backup several table periodically with my crontab. I already put .monetdb with my password text there. But there is problem, it' s always prompting for password. I check the source and i make it right with this solution: before : if (user != equals(copts.getOption("user").getArgument()) pass = null; after: if (!user.equals(copts.getOption("user").getArgument())) pass = null; regards _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4

MonetDb Clustering
by nguewo dominique 09 Jan '13

09 Jan '13

Hey guys, I'm trying to do a MonetDb Clustering. I am using two laptops both with ubuntu 12.10 installed. In both laptops, MonetDb is installed. I followed the following steps as described in the cluster management documentation: 1. I created a DB farm and set the passphrase in both laptops worker% monetdbd create /fullpath/to/mydbfarm worker% monetdbd set passphrase=mypassphrase /fullpath/to/mydbfarm worker% monetdbd start /fullpath/to/mydbfarm 2. From one system I'm able to discover the other system guarded with monetdbd willem% monetdb discover 3. Using one system I created a database in the other one willem% monetdb -h cruquius -P mypassphrase create pump Since I only have two laptops I created the same database in both the laptops and I'm trying to set the shared property so that i will be able to refer to the same Database from both the laptops. The goal is to have different dataset(tables) of the same database stored in different machines but being able to query the database as a whole from a system. I tried the following but was unsuccesfull laptop1% monetdb -h laptop2-P mypassphrase set shared=pump/1/ pump laptop1% monetdb -h laptop1-P mypassphrase set shared=pump/2/ pump then laptop1% mclient -d '*/pump/2' laptop1% mclient -d '*/pump/1' How should I go about it? Dominique _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: Feb2013 - Reduce CATNAP in vmtrim
by Sjoerd Mullender 08 Jan '13

08 Jan '13

Please revert. The reason why there is a loop that sleeps 50 ms per iteration is to get a responsive exit. With your change you have to wait up to half a second before the server exits. That is too long. If you want to reduce the frequency with which the vmtrim thread does actual work, you need to change the number of times the loop is run, not the duration of the sleep. On 2013-01-08 01:21, Martin Kersten wrote: > Changeset: 4ca22f5177b6 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=4ca22f5177b6 > Modified Files: > gdk/gdk_utils.c > Branch: Feb2013 > Log Message: > > Reduce CATNAP in vmtrim > The overload on memory is already protected in the dataflow scheduler > and optimizer pipeline, which means we probably can relax the time to > issue the BBPtrim(). > > > diffs (22 lines): > > diff --git a/gdk/gdk_utils.c b/gdk/gdk_utils.c > --- a/gdk/gdk_utils.c > +++ b/gdk/gdk_utils.c > @@ -1034,6 +1034,7 @@ GDKmunmap(void *addr, size_t size) > * Their value is turned into a blanc space. > */ > > +#define CATNAP 500 > int GDKrecovery = 0; > > static MT_Id GDKvmtrim_id; > @@ -1053,8 +1054,8 @@ GDKvmtrim(void *limit) > size_t cursize; > > /* sleep using catnaps so we can exit in a timely fashion */ > - for (t = highload ? 500 : 5000; t > 0; t -= 50) { > - MT_sleep_ms(50); > + for (t = highload ? CATNAP : 10 * CATNAP; t > 0; t -= CATNAP) { > + MT_sleep_ms(CATNAP); > if (GDKexiting()) > return; > } > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 3

Re: MonetDB: Feb2013 - Limit the number of concurrent queries
by Stefan Manegold 04 Jan '13

04 Jan '13

> Changeset: e79b2edeba71 for MonetDB > URL: > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e79b2edeba71 > Modified Files: > monetdb5/mal/mal.c > monetdb5/mal/mal.h > monetdb5/mal/mal_interpreter.c > Branch: Feb2013 > Log Message: > > Limit the number of concurrent queries > The total throughput for small (and large) queries is > improved if we limit the number of concurrent queries > being executed. This patch is geared at SQL, wrapping > callMAL with a semaphore to limit the level of parallelism > to number of physical cores. > > > diffs (59 lines): > > diff --git a/monetdb5/mal/mal.c b/monetdb5/mal/mal.c > --- a/monetdb5/mal/mal.c > +++ b/monetdb5/mal/mal.c > @@ -195,6 +195,7 @@ MT_Lock mal_remoteLock; > MT_Lock mal_profileLock ; > MT_Lock mal_copyLock; > MT_Lock mal_delayLock; > +MT_Sema mal_parallelism; > /* > * Initialization of the MAL context > * The compiler directive STRUCT_ALIGNED tells that the > @@ -233,6 +234,7 @@ int mal_init(void){ > MT_lock_init( &mal_profileLock, "mal_profileLock"); > MT_lock_init( &mal_copyLock, "mal_copyLock"); > MT_lock_init( &mal_delayLock, "mal_delayLock"); > + MT_sema_init( &mal_parallelism, (GDKnr_threads ? GDKnr_threads/2 : 4), "mal_parallelism"); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ With GDKnr_threads == 1 (single threaded execution or single-core system), this results in mal_parallelism == 0, which in turn results in all SQL tests failing (hanging until timeout) with `Mtest.py -n1` (single-threaded). Stefan > > tstAligned(); > MCinit(); > diff --git a/monetdb5/mal/mal.h b/monetdb5/mal/mal.h > --- a/monetdb5/mal/mal.h > +++ b/monetdb5/mal/mal.h > @@ -118,6 +118,7 @@ mal_export MT_Lock mal_remoteLock; > mal_export MT_Lock mal_profileLock ; > mal_export MT_Lock mal_copyLock ; > mal_export MT_Lock mal_delayLock ; > +mal_export MT_Sema mal_parallelism; > > > mal_export int mal_init(void); > diff --git a/monetdb5/mal/mal_interpreter.c > b/monetdb5/mal/mal_interpreter.c > --- a/monetdb5/mal/mal_interpreter.c > +++ b/monetdb5/mal/mal_interpreter.c > @@ -479,7 +479,13 @@ callMAL(Client cntxt, MalBlkPtr mb, MalS > ValPtr lhs; > InstrPtr pci = getInstrPtr(mb, 0); > RuntimeProfileRecord runtimeProfile; > - > + > +/* > + * Control the level of parallelism. The maximum number of > concurrent MAL plans > + * is determined by an environment variable. It is initially set > equal to the > + * number of cores, which may be too coarse. > + */ > + MT_sema_down(&mal_parallelism,"mal_parallelism"); > runtimeProfileInit(mb, &runtimeProfile, cntxt->flags & memoryFlag); > #ifdef DEBUG_CALLMAL > mnstr_printf(cntxt->fdout, "callMAL\n"); > @@ -518,8 +524,10 @@ callMAL(Client cntxt, MalBlkPtr mb, MalS > case PATcall: > case CMDcall: > default: > + MT_sema_up(&mal_parallelism,"mal_parallelism"); > throw(MAL, "mal.interpreter", RUNTIME_UNKNOWN_INSTRUCTION); > } > + MT_sema_up(&mal_parallelism,"mal_parallelism"); > if (cntxt->qtimeout && time(NULL) - stk->clock.tv_usec > > cntxt->qtimeout) > throw(MAL, "mal.interpreter", RUNTIME_QRY_TIMEOUT); > return ret; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 1

Redirect dev.monetdb.org to dev.monetdb.org/hg
by Stefan de Konink 04 Jan '13

04 Jan '13

As $subj, Would probably make it more easy for users to find the hg repositories. Stefan _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Table column type meta-data and the MonetDB API (MAPI)
by Brandon Jackson 03 Jan '13

03 Jan '13

Good day, I am working on improvements to the Pentaho MonetDB Bulk Loader plugin and want to get some advice for how to properly use the MonetDB API to get the MonetDB column data type meta-data for a given table. Here is the use case: A row of data in Pentaho Data Integration is composed of fields which have a basic Java data type and other meta data kept in a helper class that keeps track of format strings, digits of precision and so on. At the current state of development a user has to know exactly which data type was used for a particular column and in which order the columns were defined by the original SQL CREATE statement. In PDI, every field (column) starts life with primitive Java data types without necessarily having a specific format defined when the variable is initialized, a user might send a TIMESTAMP (i.e. 2013-01-03 00:00:00) to MonetDB where really a DATE (i.e 2013-01-03) was expected. What I would like to do is before the bulk load actually runs, ask MonetDB through the API what the meta data is for each of the columns and output to the end user a table that shows the columns reported by MonetDB for a given table, the data type specified in the DDL, and show the user a suggested format string, which they could modify if they know what they are doing. Format strings look like "yyyy-MM-dd HH:mm:ss" and "#,###.#" etc. Generally, what API calls or approach is recommended to accomplish what I am trying to do? As a fall back, I considered querying the meta-data out like so: SELECT name AS column_name, type as column_data_type FROM "sys"."columns" WHERE table_id = ( SELECT id AS TABLE_ID FROM "sys"."tables" WHERE name = 'DIM_DATE' and schema_id = (SELECT id AS SCHEMA_ID FROM "sys"."schemas" WHERE name = 'testing')) ORDER BY number; This brings back two columns, name and type where I can match them to the outgoing fields. Then I can process the result set in the step and update the value meta-data with the appropriate format mask. Running such SQL should be unnecessary but at least I have a fall-back solution. Thanks for the advice, Brandon Jackson _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Re: MonetDB: Feb2013 - Make joinpath signatures specific
by Stefan Manegold 31 Dec '12

31 Dec '12

Martin, this causes tests monetdb5/optimizer/Tests/tst4300.mal and monetdb5/optimizer/Tests/tst4610.mal to fail with !TypeException:user.main[17]:'algebra.joinPath' undefined in: zz:any := algebra.joinPath(b:bat[:oid,:lng], cr:bat[:lng,:oid], d:bat[:oid,:lng]) and !TypeException:user.main[9]:'algebra.joinPath' undefined in: r:any := algebra.joinPath(a:bat[:oid,:int], br:bat[:int,:oid], c:bat[:oid,:int]) Stefan On Mon, Dec 31, 2012 at 04:55:02AM +0100, Martin Kersten wrote: > Changeset: 9ac4a1b83d56 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=9ac4a1b83d56 > Modified Files: > monetdb5/optimizer/optimizer.mal > Branch: Feb2013 > Log Message: > > Make joinpath signatures specific > > > diffs (27 lines): > > diff --git a/monetdb5/optimizer/optimizer.mal b/monetdb5/optimizer/optimizer.mal > --- a/monetdb5/optimizer/optimizer.mal > +++ b/monetdb5/optimizer/optimizer.mal > @@ -280,19 +280,19 @@ pattern optimizer.joinPath(mod:str, fcn: > address OPTwrapper > comment "Join path constructor"; > > -pattern algebra.joinPath(l:bat[:any,:any]...):bat[:any,:any] > +pattern algebra.joinPath(l:bat[:oid,:any]...):bat[:oid,:any] > address ALGjoinPath > comment "Routine to handle join paths. The type analysis is rather tricky."; > > -pattern algebra.leftjoinPath(l:bat[:any,:any]...):bat[:any,:any] > +pattern algebra.leftjoinPath(l:bat[:oid,:any]...):bat[:oid,:any] > address ALGjoinPath > comment "Routine to handle join paths. The type analysis is rather tricky."; > > -pattern algebra.leftfetchjoinPath(l:bat[:any,:any]...):bat[:any,:any] > +pattern algebra.leftfetchjoinPath(l:bat[:oid,:any]...):bat[:oid,:any] > address ALGjoinPath > comment "Routine to handle join paths. The type analysis is rather tricky."; > > -pattern algebra.semijoinPath(l:bat[:any,:any]...):bat[:any,:any] > +pattern algebra.semijoinPath(l:bat[:oid,:any]...):bat[:oid,:any] > address ALGjoinPath > comment "Routine to handle join paths. The type analysis is rather tricky."; > > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Unique key constraint violation violated
by Tapomay Dey 28 Dec '12

28 Dec '12

Is this a known issue that you are working on? If so please give me the bug id where I would know when its fixed. The case is as follows: I have a table with 32 columns with unique key on 3 of them. I am running live ETLs for putting data into monetdb. These ETLs are generating sql insert scripts.(inserts/updates/ddls) Using JDBC and c3p0 connection pooling. Have developed an external wrapper module to achieve master-master replication. One common consistency check would be a count*. Observation: Queries are run in batches of 100 inserts clubbed into a single insert. I do a select count(*) after each batch of 100. After running 9100 queries the count* on one DB was 9100 but in the other was 9200. The second DB had 100 records that were duplicates of 8901-9000. It means when I inserted 9001 to 9100 the previous batch went into the store again with it and somehow surpassed the unique key constraint check. I have also observed another case where count(*) gave less results than actual no. of rows in the DB. REQUEST: Just want to confirm if such kind of issues are part of active development. Otherwise it would be really difficult for me to push monetdb usage into production. Will try again with MonetDataSource instead of c3p0. But I doubt if the issues are client side or connection-related. Plz note that the keys for which this occurs is random and thus I concluded it's not a data issue. Regards, Tapomay Dey. Software Design Engg. Sokrati Inc. Email: tapomayATsokratiDOTcom _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

fast bit integration into monetdb
by Stanislav Seltser 27 Dec '12

27 Dec '12

does anybody know if fastbit integration into scidb was ever completed? _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4

Unique key constraint violation violated
by Tapomay Dey 26 Dec '12

26 Dec '12

Is this a known issue that you are working on? If so please give me the bug id where I would know when its fixed. The case is as follows: I have a table with 32 columns. It has a unique key on 3 of them. I know that it's recommended that you do bulk inserts from csv for performance. But in my case I am running live ETLs for putting data into monetdb. These ETLs are generating sql insert scripts.(inserts/updates/ddls) I use JDBC and c3p0 connection pooling for executing these sqls. I have a failover mechanism where the scripts are executed on multiple DBs and I try to keep their states consistent. One common consistency check would be a count*. I am running the latest stable installation from repo on Ubuntu 10.04 64 bit. Observation: After running 9100 queries the count* on one DB was 9100 but in the other was 9200. I have diagnostics in place to do a select count* after running a batch of 100 queries in a single JDBC transaction. I observed that the count* was consistent until 9000 queries were executed. It failed at 9100. I did a manual check to see if duplicate records exist with same unique key values and they did. Following is the query I used for the same: select ad_group_id, count(*) from ad_groups group by ad_group_id having count(ad_group_id)>1 and client_id=X and account_id=Y; My constraint is as follows: CONSTRAINT "ad_groups_client_id_account_id_ad_group_id_unique" UNIQUE ("client_id", "account_id", "ad_group_id") I found exactly 100 tuples with group_by_count* = 2 and the ad_group_id(unique key) values for these exactly matched queries 8901 to 9000. It means when I inserted 9001 to 9100 the previous batch went into the store again with it and somehow surpassed the unique key constraint check. Please note that I am converting a batch of 100 insert queries into a single insert query and running this single bulk insert in a single JDBC transaction to reduce network traffic. This same issue recurred at 21500 only this time it happened for 300 records with same unique key value being duplicated. And their unique key values were exactly matching the ones in the last 3 batches Ad_group_id s 21103-21202, 21203-21302, 21303-21402 existed twice. 21403-21502 existed exactly once. REQUEST: Can anyone tell me if I can enable query logs - something like what mysql has. I will try re-installing monetdb and see if this problem goes away. :) Merry Christmas and Warm Regards, Tapomay _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 1

Re: MonetDB: Feb2013 - ported fix to the query cache from Oct2012
by Stefan Manegold 23 Dec '12

23 Dec '12

Niels, does this replace a merge of your Oct2012 checkin to the Feb2013 branch? Before this checkin, the merge worked without any conflicts (I did not test, whether/how it works). After this checkin, the merge fails with conflicts (I did not check the details). Thanks! Stefan Niels Nes <commits(a)monetdb.org> wrote: Changeset: ee6e585e72c5 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ee6e585e72c5 Modified Files: sql/backends/monet5/Tests/optimizers.stable.out sql/backends/monet5/UDF/Tests/udf-fuse.stable.out sql/backends/monet5/UDF/Tests/udf-reverse.stable.out sql/server/sql_mvc.c sql/server/sql_parser.y sql/server/sql_qc.c sql/test/ADT2006/Tests/bram.stable.out sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/variable_arg_substring.SF_942981.stable.out sql/test/BugDay_2005-11-09_2.8/Tests/crash_in_case.SF-1097749.stable.out sql/test/BugDay_2005-11-09_2.9.3/Tests/sql_server_crash.SF-1080024.stable.out sql/test/BugDay_2005-11-09_2.9.3/Tests/union-duplicates.SF-1119186.stable.out sql/test/BugTracker-2009/Tests/AVG_of_SQRT.SF-2757642.stable.out sql/test/BugTracker-2009/Tests/IS_NULL_in_CASE.stable.out sql/test/BugTracker-2009/Tests/group_by_on_values.SF-2837561.stable.out sql/test/BugTracker-2009/Tests/not_equal.SF-2678502.stable.out sql/test/BugTracker-2009/Tests/old_views_incorrectly_reused.SF-2863804.stable.out sql/test/BugTracker-2009/Tests/push_select_down_crash.stable.out sql/test/BugTracker-2009/Tests/utf8_bug.SF-2822855.stable.out sql/test/BugTracker-2010/Tests/LIMIT_OFFSET_big-endian.Bug-2622.stable.out sql/test/BugTracker-2010/Tests/crash_on_where_in.Bug-2333.stable.out sql/test/BugTracker-2010/Tests/group-by_ordered_column.Bug-2564.stable.out sql/test/BugTracker-2010/Tests/mat.slice_limit1.Bug-2645.stable.out sql/test/BugTracker-2010/Tests/order_by_on_single_row_view.SF-2950453.stable.out sql/test/BugTracker-2010/Tests/substring_display_width.Bug-2683.stable.out sql/test/BugTracker-2011/Tests/case-when-alias.Bug-2848.stable.out sql/test/BugTracker-2012/Tests/predicate_select.Bug-3090.stable.err sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out sql/test/BugTracker/Tests/assert_fails_on_algebra_version.SF-2019349.stable.out sql/test/BugTracker/Tests/bind_with_cast.SF-1720934.stable.err sql/test/BugTracker/Tests/crash_on_delete.SF-1639108.stable.out sql/test/BugTracker/Tests/crash_on_delete.SF-1651567.stable.out sql/test/BugTracker/Tests/crash_on_insert.SF-1639113.stable.out sql/test/BugTracker/Tests/insert_not_exists.SF-1380287.stable.out sql/test/Dependencies/Tests/Dependencies.stable.out sql/test/Skyserver/Tests/Skyserver.stable.out sql/test/Tests/coalesce.stable.out sql/test/Tests/decimal2.stable.out sql/test/Tests/setoptimizer.stable.out sql/test/VOC/Tests/VOC.stable.out sql/test/bugs/Tests/crash_order_by.stable.out sql/test/bugs/Tests/select_from_values.stable.out sql/test/bugs/Tests/simple_view.stable.out sql/test/bugs/Tests/subselect_multiple_unionall_where_1=1-bug-sf-1005596.stable.out sql/test/bugs/Tests/subselect_single_value-bug-sf-941791.stable.out sql/test/bugs/Tests/subselect_union-bug-sf-941826.stable.out sql/test/bugs/Tests/unicode_varchar-bug-sf-1041324_JdbcClient.stable.out sql/test/bugs/Tests/unicode_varchar-bug-sf-1041324_MapiClient.stable.out sql/test/bugs/Tests/union_all-bug-sf-941788.stable.out sql/test/bugs/Tests/union_all_where_1=1-bug-sf-936386.stable.out sql/test/leaks/Tests/check0.stable.out sql/test/leaks/Tests/check1.stable.out sql/test/leaks/Tests/check2.stable.out sql/test/leaks/Tests/check3.stable.out sql/test/leaks/Tests/check4.stable.out sql/test/leaks/Tests/check5.stable.out sql/test/leaks/Tests/drop3.stable.out sql/test/leaks/Tests/temp1.stable.out sql/test/leaks/Tests/temp2.stable.out sql/test/leaks/Tests/temp3.stable.out sql/test/pg_regress/Tests/char.stable.out sql/test/pg_regress/Tests/strings_concat.stable.out sql/test/pg_regress/Tests/text.stable.out sql/test/pg_regress/Tests/varchar.stable.out sql/test/rdf/Tests/q2_28_v.stable.out sql/test/rdf/Tests/q2_v.stable.out sql/test/rdf/Tests/q3_28_v.stable.out sql/test/rdf/Tests/q3_v.stable.out sql/test/rdf/Tests/q4_28_v.stable.out sql/test/rdf/Tests/q4_v.stable.out sql/test/rdf/Tests/q6_28_v.stable.out sql/test/rdf/Tests/q6_v.stable.out Branch: Feb2013 Log Message: ported fix to the query cache from Oct2012 diffs (truncated from 3392 to 300 lines): diff --git a/sql/backends/monet5/Tests/optimizers.stable.out b/sql/backends/monet5/Tests/optimizers.stable.out --- a/sql/backends/monet5/Tests/optimizers.stable.out +++ b/sql/backends/monet5/Tests/optimizers.stable.out @@ -47,31 +47,31 @@ Ready. #select 'optimizer test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 15, 12 # length [ "optimizer test:", "default_pipe" ] #select 'optimizer off test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 19, 12 # length [ "optimizer off test:", "default_pipe" ] #select 'optimizer on test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 18, 12 # length [ "optimizer on test:", "default_pipe" ] #select 'optimizer <empty> test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 23, 12 # length [ "optimizer <empty> test:", "default_pipe" ] #select 'done'; % . # table_name % single_value # name -% char # type +% varchar # type % 4 # length [ "done" ] diff --git a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out --- a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out +++ b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out @@ -45,12 +45,10 @@ end s0_1; % mal # name % clob # type % 71 # length -function user.s2_1{autoCommit=true}(A0:sht,A1:sht):void; +function user.s1_1{autoCommit=true}(A0:sht,A1:sht):void; X_4 := udf.fuse(A0,A1); sql.exportValue(1,".","fuse_single_value":str,"int",32,0,6,X_4,""); -end s2_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s1_1; #select fuse(1000,2000); % . # table_name % fuse_single_value # name @@ -62,12 +60,10 @@ end s2_1; % mal # name % clob # type % 74 # length -function user.s4_1{autoCommit=true}(A0:int,A1:int):void; +function user.s2_1{autoCommit=true}(A0:int,A1:int):void; X_4 := udf.fuse(A0,A1); sql.exportValue(1,".","fuse_single_value":str,"bigint",64,0,6,X_4,""); -end s4_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s2_1; #select fuse(1000000,2000000); % . # table_name % fuse_single_value # name @@ -97,7 +93,7 @@ end s4_1; % mal # name % clob # type % 60 # length -function user.s11_1{autoCommit=true}():void; +function user.s5_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_fuse"); X_6 := sql.bind(X_2,"sys","udf_fuse","a",0); @@ -115,15 +111,13 @@ function user.s11_1{autoCommit=true}():v sql.rsColumn(X_25,"sys.","fuse_a","smallint",16,0,X_23); X_30 := io.stdout(); sql.exportResult(X_30,X_25); -end s11_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s5_1; #explain select fuse(c,d) from udf_fuse; % .explain # table_name % mal # name % clob # type % 57 # length -function user.s12_1{autoCommit=true}():void; +function user.s6_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_fuse"); X_6 := sql.bind(X_2,"sys","udf_fuse","c",0); @@ -141,15 +135,13 @@ function user.s12_1{autoCommit=true}():v sql.rsColumn(X_25,"sys.","fuse_c","int",32,0,X_23); X_30 := io.stdout(); sql.exportResult(X_30,X_25); -end s12_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s6_1; #explain select fuse(e,f) from udf_fuse; % .explain # table_name % mal # name % clob # type % 58 # length -function user.s13_1{autoCommit=true}():void; +function user.s7_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_fuse"); X_6 := sql.bind(X_2,"sys","udf_fuse","e",0); @@ -167,9 +159,7 @@ function user.s13_1{autoCommit=true}():v sql.rsColumn(X_25,"sys.","fuse_e","bigint",64,0,X_23); X_30 := io.stdout(); sql.exportResult(X_30,X_25); -end s13_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s7_1; #select fuse(a,b) from udf_fuse; % sys. # table_name % fuse_a # name diff --git a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out --- a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out +++ b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out @@ -82,7 +82,7 @@ end s0_1; % mal # name % clob # type % 62 # length -function user.s7_1{autoCommit=true}():void; +function user.s3_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_reverse"); X_6 := sql.bind(X_2,"sys","udf_reverse","x",0); @@ -95,7 +95,7 @@ function user.s7_1{autoCommit=true}():vo sql.rsColumn(X_18,"sys.","reverse_x":str,"clob",0,0,X_16); X_22 := io.stdout(); sql.exportResult(X_22,X_18); -end s7_1; +end s3_1; #select reverse(x) from udf_reverse; % sys. # table_name % reverse_x # name diff --git a/sql/server/sql_mvc.c b/sql/server/sql_mvc.c --- a/sql/server/sql_mvc.c +++ b/sql/server/sql_mvc.c @@ -193,8 +193,6 @@ mvc_trans(mvc *m) qc_clean(m->qc); } } - if (m->session->active) - m->type = Q_TRANS; store_unlock(); } diff --git a/sql/server/sql_parser.y b/sql/server/sql_parser.y --- a/sql/server/sql_parser.y +++ b/sql/server/sql_parser.y @@ -3954,7 +3954,7 @@ literal: string { char *s = sql2str($1); int len = _strlen(s); sql_subtype t; - sql_find_subtype(&t, "char", len, 0 ); + sql_find_subtype(&t, "varchar", len, 0 ); $$ = _newAtomNode( _atom_string(&t, s)); } | HEXADECIMAL { int len = _strlen($1), i = 2, err = 0; diff --git a/sql/server/sql_qc.c b/sql/server/sql_qc.c --- a/sql/server/sql_qc.c +++ b/sql/server/sql_qc.c @@ -143,6 +143,11 @@ param_cmp(sql_subtype *t1, sql_subtype * if (t1->scale != t2->scale) return -1; + + if (t1->type->eclass == EC_NUM && t2->type->eclass == EC_NUM && + t1->digits >= t2->digits) + return 0; + res = is_subtype(t2, t1); if (!res) return -1; @@ -178,7 +183,7 @@ param_list_cmp(sql_subtype *typelist, at if ((!((at->type->eclass == EC_DEC || at->type->eclass == EC_NUM) && tp->type->eclass == EC_FLT)) && - (!(EC_VARCHAR(tp->type->eclass) && + (!(EC_VARCHAR(tp->type->eclass) && EC_VARCHAR(at->type->eclass) && (!tp->digits || tp->digits >= at->digits))) && diff --git a/sql/test/ADT2006/Tests/bram.stable.out b/sql/test/ADT2006/Tests/bram.stable.out --- a/sql/test/ADT2006/Tests/bram.stable.out +++ b/sql/test/ADT2006/Tests/bram.stable.out @@ -458,7 +458,7 @@ Ready. #(SELECT 'Major' AS grade, count(na % ., .inspector # table_name % grade, inspectors # name -% char, wrd # type +% varchar, wrd # type % 9, 1 # length [ "Detective", 3 ] [ "Constable", 3 ] @@ -475,7 +475,7 @@ Ready. # + cast(substring(length from (position('\'' in length) + 1) for (position('"' in length) + 1)) AS integer)) as height from victim v % .L1, . # table_name % L1, average_height # name -% char, char # type +% varchar, varchar # type % 6, 4 # length [ "blonde", "5'7\"" ] [ "brown", "5'5\"" ] diff --git a/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out b/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out --- a/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out +++ b/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out @@ -20,12 +20,12 @@ Ready. % . # table_name % single_value # name -% char # type +% varchar # type % 2 # length [ "aa" ] % . # table_name % single_value # name -% char # type +% varchar # type % 3 # length [ "a'a" ] diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out @@ -101,7 +101,7 @@ Ready. [ "users", "SELECT u.\"t\" AS \"name\", ui.\"fullname\", ui.\"default_schema\" FROM \"sys\".\"db_users\" AS u, \"sys\".\"db_user_info\" AS ui WHERE u.\"h\" = ui.\"id\" AND u.\"h\" NOT IN (SELECT s.\"h\" FROM \"sys\".\"db_scens\" AS s WHERE s.\"t\" NOT LIKE 'sql');", false, true, 0, 0 ] % .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables # table_name % table_cat, table_schem, table_name, table_type, remarks, type_cat, type_schem, type_name, self_referencing_col_name, ref_generation # name -% char, varchar, varchar, char, varchar, char, char, char, char, char # type +% varchar, varchar, varchar, varchar, varchar, varchar, varchar, varchar, varchar, varchar # type % 4, 3, 8, 12, 222, 4, 4, 4, 2, 6 # length [ "null", "sys", "columns", "SYSTEM TABLE", "", "null", "null", "null", "id", "SYSTEM" ] [ "null", "sys", "db_scens", "SYSTEM TABLE", "", "null", "null", "null", "id", "SYSTEM" ] diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out @@ -63,8 +63,9 @@ Ready. [ 1 ] % . # table_name % single_value # name -% char # type -% 1 # length +% varchar # type +% 0 # length +#select * from my_table where 1 = 0; % sys.my_table # table_name % att # name % int # type diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out @@ -31,7 +31,7 @@ Ready. [ 1 ] % sys. # table_name % ifthenelse_isnull_=_i # name -% char # type +% varchar # type % 23 # length [ "base table" ] [ "system table" ] diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out @@ -43,8 +43,8 @@ Ready. #ORDER BY "TABLE_TYPE", % .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables # table_name % TABLE_CAT, TABLE_SCHEM, TABLE_NAME, TABLE_TYPE, REMARKS, TYPE_CAT, TYPE_SCHEM, TYPE_NAME, SELF_REFERENCING_COL_NAME, REF_GENERATION # name -% char, varchar, varchar, char, char, char, char, char, char, char # type -% 0, 0, 0, 12, 0, 0, 0, 0, 2, 6 # length +% char, varchar, varchar, varchar, varchar, char, char, char, varchar, varchar # type +% 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 # length # 11:50:19 > # 11:50:19 > Done. diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out @@ -26,8 +26,8 @@ Ready. % .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables # table_name % TABLE_CAT, TABLE_SCHEM, TABLE_NAME, TABLE_TYPE, REMARKS, TYPE_CAT, TYPE_SCHEM, TYPE_NAME, SELF_REFERENCING_COL_NAME, REF_GENERATION # name -% char, varchar, varchar, char, char, char, char, char, char, char # type -% 4, 3, 4, 20, 0, 0, 0, 0, 5, 6 # length +% varchar, varchar, varchar, varchar, varchar, char, char, char, varchar, varchar # type _______________________________________________ checkin-list mailing list checkin-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/checkin-list _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Announcement: New Oct2012-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Dec '12

21 Dec '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP2 bugfix release SQL * Fixed crash when performing an INSERT on a table with string-like column defaulting to NULL and omitting that column from VALUES, bug #3168 Java Module * Implemented type map support of Connection to allow custom mapping of UDTs to Java classes. By default the INET and URL UDTs are now mapped to nl.cwi.monetdb.jdbc.types.{INET,URL}. Most notably, ResultSet.getObject() and PreparedStatement.setObject() deal with the type map. * Fixed a problem in PreparedStatement where the prepared statement's ResultSetMetaData (on its columns to be produced) incorrectly threw exceptions about non existing columns. Bug #3192 Bug Fixes * 2291: small doubles end up as NULL after arithmetic * 2579: SQL: incorrect cast from double to int * 2659: tuples INTERSECT vs count (... INTERSECT ...) differs * 2805: row_number doesn't work in complex query * 2886: warning: Catalog parameter '' ignored * 2977: timestamp minus date results in program error * 2978: now() + 1 results in illegal argument * 2999: Up-cast from SQL drops order information? * 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause * 3124: insert into table with two foreign keys crashes mserver5 * 3136: Memleak (in querycache?) while querying * 3143: Type resolution error in SQL procedural code * 3163: mserver5 crashes with large number of concurrent clients * 3168: crash when insert varchar/char/clob with default NULL * 3169: php driver uses call-time pass-by-reference * 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either * 3181: Join query returns zero results when there is an index on the tables * 3185: inconsistency between documentation and implementation of command line options * 3186: case evaluted to early * 3188: Storage of client history * 3189: Overflow in calculation * 3190: When subtracting two columns of type "timestamp", no results are returned * 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed. * 3192: "No such column with index" error if CASE in SELECT clause * 3199: Inconspicuous sequence of prepare, execute, commit and close produces error * 3205: Inet datatype handling inconsistent between int, cast( as inet) and string _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

readline and the Mac build
by Masood Mortazavi 21 Dec '12

21 Dec '12

It seems "readline" isn't linked in the Mac build and installation. Configure or package repository don't indicate any issues, and yet history completions fail on mclient's sql prompt. Regards, -m. $ sudo port list | grep readline gauche-readline @0.1 devel/gauche-readline hs-readline @1.0.1.0 devel/hs-readline pure-readline @0.1 devel/pure-readline readline @6.2.000 devel/readline readline-4 @4.3 devel/readline-4 readline-5 @5.2.014 devel/readline-5 ... ... $ more config.log | grep readline configure:18493: checking readline/readline.h usability configure:18493: checking readline/readline.h presence configure:18493: checking for readline/readline.h configure:18495: checking for readline in -lreadline configure:18520: gcc -o conftest -g conftest.c -lreadline >&5 configure:18631: checking for rl_completion_matches in -lreadline configure:18656: gcc -o conftest -g conftest.c -lreadline -lreadline >&5 | #include <readline/readline.h> ac_cv_header_readline_readline_h=yes ac_cv_lib_readline_readline=yes ac_cv_lib_readline_rl_completion_matches=yes _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

4 8

Re: MonetDB: Feb2013 - Properly realloc the namespace
by Stefan Manegold 19 Dec '12

19 Dec '12

Martin, assertions are not the most suitable way to check for malloc failures. Assertions are to *assert* facts that must hold, i.e., the would be a logical error if they do not hold. Consequently, assertions are only enabled during development and testing, but disabled (for among others for performance reasons) in any production builds. If triggered, assertions stop the entire process instantly. Failing mallocs is a runtime exception that can occur at any time for various reasons. The code should check for this and gracefully handle such situations with proper exeption handling. Having said that, in particular is the man namespace is local per query/client, a failure in one query/client should not crash the whole server by an assertion (during development/testing) or a segfault (in production builds), but rather only greacefull trigger and handle an exception for that one query/client. Also, if namespaces are global and a malloc failure prevent the whole server from continuing properly, it should at least (try to) gracefully shut itself down (with a proper error message), rather than crashing instantly with an assertion of segfault. Best, Stefan On Wed, Dec 19, 2012 at 11:53:37AM +0100, Martin Kersten wrote: > Changeset: 2a9bd075c039 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=2a9bd075c039 > Modified Files: > monetdb5/mal/mal_namespace.c > Branch: Feb2013 > Log Message: > > Properly realloc the namespace > The expandName always lead to a different anchoring point of the namespace. > This may leave concurrent threads with dead meat. > This patch reduces the chance on a SEGfault. > > > diffs (37 lines): > > diff --git a/monetdb5/mal/mal_namespace.c b/monetdb5/mal/mal_namespace.c > --- a/monetdb5/mal/mal_namespace.c > +++ b/monetdb5/mal/mal_namespace.c > @@ -66,26 +66,13 @@ typedef struct NAMESPACE{ > static Namespace namespace; > > static void expandNamespace(void){ > - str *nme; > - size_t *length; > - int *link, newsize, incr = 2048; > - > - newsize= namespace.size + incr; > - nme= (str *) GDKmalloc(sizeof(str *) * newsize); > - assert(nme != NULL); /* we cannot continue */ > - link= (int *) GDKmalloc(sizeof(int) * newsize); > - assert(link != NULL); /* we cannot continue */ > - length = (size_t *) GDKmalloc(sizeof(size_t) * newsize); > - assert(length != NULL); /* we cannot continue */ > - > - memcpy(nme, namespace.nme, sizeof(str *) * namespace.nmetop); > - memcpy(link, namespace.link, sizeof(int) * namespace.nmetop); > - memcpy(length, namespace.length, sizeof(size_t) * namespace.nmetop); > - > - namespace.size += incr; > - GDKfree(namespace.nme); namespace.nme= nme; > - GDKfree(namespace.link); namespace.link= link; > - GDKfree(namespace.length); namespace.length= length; > + namespace.size += 2048; > + namespace.nme= (str *) GDKrealloc(namespace.nme, sizeof(str *) * namespace.size); > + assert(namespace.nme != NULL); /* we cannot continue */ > + namespace.link= (int *) GDKrealloc(namespace.link, sizeof(int) * namespace.size); > + assert(namespace.link != NULL); /* we cannot continue */ > + namespace.length = (size_t *) GDKrealloc(namespace.length, sizeof(size_t) * namespace.size); > + assert(namespace.length != NULL); /* we cannot continue */ > } > > void initNamespace(void) { > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: Oct2012 - fixed bug 3209, ie first crossproduct if the ...
by Ying Zhang 16 Dec '12

16 Dec '12

Hai Niels, Your fix works in Feb2013, but not in Oct2012… Jennie On Dec 15, 2012, at 12:29, Niels Nes wrote: > Changeset: 8650dce52dd8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=8650dce52dd8 > Modified Files: > sql/server/rel_exp.c > sql/server/rel_exp.h > Branch: Oct2012 > Log Message: > > fixed bug 3209, ie first crossproduct if the join expression consists > of functions (these should have been pushed down, whenever possible). > > > diffs (59 lines): > > diff --git a/sql/server/rel_exp.c b/sql/server/rel_exp.c > --- a/sql/server/rel_exp.c > +++ b/sql/server/rel_exp.c > @@ -1032,6 +1032,44 @@ exp_is_atom( sql_exp *e ) > } > > static int > +exps_has_func( list *exps) > +{ > + node *n; > + int has_func = 0; > + > + for(n=exps->h; n && !has_func; n=n->next) > + has_func |= exp_has_func(n->data); > + return has_func; > +} > + > +int > +exp_has_func( sql_exp *e ) > +{ > + switch (e->type) { > + case e_atom: > + return 0; > + case e_convert: > + return exp_has_func(e->l); > + case e_func: > + case e_aggr: > + return 1; > + case e_cmp: > + if (e->flag == cmp_or) { > + return (exps_has_func(e->l) || exps_has_func(e->r)); > + } else if (e->flag == cmp_in || e->flag == cmp_notin || e->flag == cmp_filter) { > + return (exp_has_func(e->l) || exps_has_func(e->r)); > + } else { > + return (exp_has_func(e->l) || exp_has_func(e->r) || > + (e->f && exp_has_func(e->f))); > + } > + case e_column: > + case e_psm: > + return 0; > + } > + return 0; > +} > + > +static int > exp_key( sql_exp *e ) > { > if (e->name) > diff --git a/sql/server/rel_exp.h b/sql/server/rel_exp.h > --- a/sql/server/rel_exp.h > +++ b/sql/server/rel_exp.h > @@ -105,6 +105,7 @@ extern int exp_is_eqjoin(sql_exp *e); > extern int exp_is_correlation(sql_exp *e, sql_rel *r ); > extern int exp_is_join_exp(sql_exp *e); > extern int exp_is_atom(sql_exp *e); > +extern int exp_has_func(sql_exp *e); > > extern sql_exp *exps_bind_column( list *exps, char *cname, int *ambiguous); > extern sql_exp *exps_bind_column2( list *exps, char *rname, char *cname); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: default - fixed a low level bug where empty mapi result...
by Sjoerd Mullender 14 Dec '12

14 Dec '12

On 2012-12-14 12:10, Gijs Molenaar wrote: > Changeset: 81f191d61822 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=81f191d61822 > Modified Files: > clients/python2/monetdb/mapi.py > clients/python3/monetdb/mapi.py > Branch: default > Log Message: > > fixed a low level bug where empty mapi results where not properly handled Is this is fix that should have gone to the Oct2012 branch? Please, make sure you fix bugs in the lowest branch where the bug occurs and that is still maintained. Currently the still maintained branches are Oct2012, Feb2013 and default. > > diffs (30 lines): > > diff --git a/clients/python2/monetdb/mapi.py b/clients/python2/monetdb/mapi.py > --- a/clients/python2/monetdb/mapi.py > +++ b/clients/python2/monetdb/mapi.py > @@ -163,9 +163,9 @@ class Connection(object): > self.__putblock(operation) > response = self.__getblock() > if not len(response): > - return True > + return "" > elif response.startswith(MSG_OK): > - return response[3:].strip() or True > + return response[3:].strip() or "" Isn't this equivalent to: return response[3:].strip() > if response == MSG_MORE: > # tell server it isn't going to get more > return self.cmd("") > diff --git a/clients/python3/monetdb/mapi.py b/clients/python3/monetdb/mapi.py > --- a/clients/python3/monetdb/mapi.py > +++ b/clients/python3/monetdb/mapi.py > @@ -163,9 +163,9 @@ class Connection(object): > self.__putblock(operation) > response = self.__getblock() > if not len(response): > - return True > + return "" > elif response.startswith(MSG_OK): > - return response[3:].strip() or True > + return response[3:].strip() or "" > if response == MSG_MORE: > # tell server it isn't going to get more > return self.cmd("") > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

group_concat
by Tapomay Dey 13 Dec '12

13 Dec '12

Almost there. As suggested by Fabian in the following thread I gave up the MX file approach and implemented group_concat as UDF in C. http://sourceforge.net/mailarchive/forum.php?thread_name=mailman.243.135532… When I try to conclude my function UDFBATgroup_concat_(BAT **ret, BAT *b, BAT *g, BAT *h, FILE* fp) //ignore fp as follows: BATaccessBegin(h, USE_HEAD | USE_TAIL, MMAP_SEQUENTIAL); index = 0; BATloop(h, p, q) { ptr hhead = BUNhead(hi, p); fprintf(fp, "%d: hhead=%d", index, hhead); str dst = tailValArray[index]; fprintf(fp, "%d: dstStr=%s# ", index, dst); BUNins(bn, hhead, dst, FALSE); fprintf(fp, "%d: Insert complete\n", index); index++; } I get the following error on the client terminal [tapomay@tapomay-laptop monetdata]$ echo "select house, group_concat(name) from gs group by house;" | mclient -d tapomay GDKerror:!ERROR: BUNins: access denied to tmp_10162, aborting. BUNins: access denied to tmp_10162, aborting. BUNins: access denied to tmp_10162, aborting. The groupings are ready in the tailVarArray. NEED HELP. Thanks and Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Still searching to find the Octopus :-)
by Masood Mortazavi 12 Dec '12

12 Dec '12

I'm still searching for user documentation that would explain MonetDB/octopus usage/feature. Does any exist? Thanks, Masood P.S. My earlier e-mails regarding the octopus documentation was somewhat rushed, it seems. The documentation in the man pages are actually for remtoe access and funnel-multiplex features, which seem entirely different. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

MX file expansion
by Tapomay Dey 12 Dec '12

12 Dec '12

How do I generate the sources from a new mx file. Thanks and Regards, Tapomay _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 5

Re: MonetDB: Oct2012 - configure fixes for if there is no python2 bi...
by Fabian Groffen 11 Dec '12

11 Dec '12

On 11-12-2012 15:59:04 +0100, Sjoerd Mullender wrote: > Changeset: fb7305c917ec for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fb7305c917ec > Modified Files: > configure.ag > Branch: Oct2012 > Log Message: > > configure fixes for if there is no python2 binary, but python is version 2. > Also fixed a syntax error. > +AC_PATH_PROG(PYTHON,python,no,$PATH) > +PYTHON_MAJ= > +if test "x$PYTHON" != xno; then > + AC_MSG_CHECKING([major version of $PYTHON]) > + case "`"$PYTHON" -V 2>&1`" in > + "Python 2."[[67]]*) # older Pythons don't get Python3 syntax > + PYTHON_MAJ=2 > + case "$have_python2" in > + yes|auto) > + have_python2=yes > + PYTHON2="$PYTHON" Because PYTHON2=/path/to/python here, we no longer check for python2 which is preferably over "/path/to/python", since it contains a version. The active python can change over updates or preferences of the user. Perhaps (something like) the following would be enough? diff --git a/configure.ag b/configure.ag --- a/configure.ag +++ b/configure.ag @@ -1164,7 +1164,7 @@ fi if test "x$have_python2" != xno; then if test x$cross_compiling != xyes; then - AC_PATH_PROG(PYTHON2,$PYTHON2,no,$PATH) + AC_PATH_PROG(PYTHON2,python2,no,$PATH) if test "x$PYTHON2" = xno; then if ( test "x$have_python2" = xyes || test "x$have_python2" = xauto ) && test "x$PYTHON_MAJ" = x2; then PYTHON2="$PYTHON" @@ -1220,7 +1220,7 @@ AC_SUBST(PYTHON2_LIBDIR) if test "x$have_python3" != xno; then if test x$cross_compiling != xyes; then - AC_PATH_PROG(PYTHON3,$PYTHON3,no,$PATH) + AC_PATH_PROG(PYTHON3,python3,no,$PATH) if test "x$PYTHON3" = xno; then if ( test "x$have_python3" = xyes || test "x$have_python3" = xauto ) && test "x$PYTHON_MAJ" = x3; then PYTHON3="$PYTHON" -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 3

Re: MonetDB: Feb2013 - dotmonetdb: don't support passwd
by Lefteris 11 Dec '12

11 Dec '12

I was using that:P On Sat, Dec 8, 2012 at 4:20 PM, Fabian Groffen <commits(a)monetdb.org> wrote: > Changeset: c1f634d9db8c for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=c1f634d9db8c > Modified Files: > clients/mapiclient/dotmonetdb.c > Branch: Feb2013 > Log Message: > > dotmonetdb: don't support passwd > > The purposely only option for it ever documented is "password", so no > need to check for more than that. > > > diffs (12 lines): > > diff --git a/clients/mapiclient/dotmonetdb.c > b/clients/mapiclient/dotmonetdb.c > --- a/clients/mapiclient/dotmonetdb.c > +++ b/clients/mapiclient/dotmonetdb.c > @@ -95,7 +95,7 @@ parse_dotmonetdb(char **user, char **pas > if (user) > *user = strdup(q); > q = NULL; > - } else if (strcmp(buf, "password") == 0 || > strcmp(buf, "passwd") == 0) { > + } else if (strcmp(buf, "password") == 0) == 0) { > if (passwd) > *passwd = strdup(q); > q = NULL; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

bat bind Access kind
by Tapomay Dey 10 Dec '12

10 Dec '12

pattern bind(mvc:int, schema:str, table:str, column:str, access:int ):bat[:oid,:any_1] address mvc_bind_wrap comment "Bind the 'schema.table.column' BAT with access kind: 0 - base table 1 - inserts 2 - updates"; What do you mean by access kind here. Appreciate any help. Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

Re: MonetDB: Feb2013 - Introduce a sink point for parallel processing
by Fabian Groffen 10 Dec '12

10 Dec '12

On 10-12-2012 11:49:29 +0100, Martin Kersten wrote: > Changeset: d944146a71ef for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d944146a71ef > Modified Files: > monetdb5/modules/mal/language.c > monetdb5/modules/mal/language.h > monetdb5/modules/mal/language.mal > Branch: Feb2013 > Log Message: > > Introduce a sink point for parallel processing sink noun 1 a fixed basin with a water supply and a drain. 2 a pool or marsh in which a river's water disappears by evaporation or percolation. • technical a body or process that acts to absorb or remove energy or a particular component from a system: a heat sink | the oceans can act as a sink for CO 2. The opposite of source. 3 short for sinkhole. 4 a place of vice or corruption: moralizers complain that the Net is simply a festering sink of iniquity. sync |siNGk|(also synch ) informal noun synchronization: images flash onto your screen in sync with the music. -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: default - use new sub sample function (cleans up some m...
by Sjoerd Mullender 06 Dec '12

06 Dec '12

Why the mix of ptr/BUN/wrd in SAMPLEsubuniform? The function is only called from MAL for a command whose last parameter is declared as s:wrd, so SAMPLEsubuniform can be declared as having a last argument as wrd *s and the cast inside then becomes correct too: (BUN) *s. Remember, wrd and BUN are not *necessarily* the same size. On 2012-12-06 10:25, Niels Nes wrote: > Changeset: 7c832c8b1a70 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=7c832c8b1a70 > Modified Files: > monetdb5/modules/kernel/algebra.mx > monetdb5/modules/mal/sample.c > monetdb5/modules/mal/sample.h > monetdb5/modules/mal/sample.mal > sql/backends/monet5/rel_bin.c > sql/backends/monet5/sql_gencode.c > Branch: default > Log Message: > > use new sub sample function (cleans up some more reverses and marks) > > > diffs (112 lines): > > diff --git a/monetdb5/modules/kernel/algebra.mx b/monetdb5/modules/kernel/algebra.mx > --- a/monetdb5/modules/kernel/algebra.mx > +++ b/monetdb5/modules/kernel/algebra.mx > @@ -282,6 +282,11 @@ comment "Selects all elements that have > command sample ( b:bat[:oid,:any_2], num:int ) :bat[:oid,:any_2] > address ALGsample > comment "Produce a random selection of size 'num' from the input BAT."; > + > +command subsample(b:bat[:oid,:any_1], num:int ) :bat[:oid,:oid] > +address ALGsubsample > +comment "Return the oids of a random selection of size 'num' from the input BAT."; > + > # @+ BAT copying > command copy( b:bat[:any_1,:any_2]) :bat[:any_1,:any_2] > address ALGcopy > @@ -1076,6 +1081,7 @@ algebra_export str ALG@1(int *result, in > @:ALGbinaryExport(tdifference)@ > @:ALGbinaryExport(tdiff)@ > @:ALGbinaryintExport(sample)@ > +@:ALGbinaryintExport(subsample)@ > > algebra_export str ALGtunique(int *result, int *bid); > algebra_export str ALGtsort(int *result, int *bid); > @@ -2765,6 +2771,11 @@ ALGsample(bat *result, bat *bid, int *pa > return ALGbinaryint(result, bid, param, BATsample, "algebra.sample"); > } > > +str > +ALGsubsample(bat *result, bat *bid, int *param) > +{ > + return ALGbinaryint(result, bid, param, BATsample_, "algebra.subsample"); > +} > > /* add items missing in the kernel */ > str > diff --git a/monetdb5/modules/mal/sample.c b/monetdb5/modules/mal/sample.c > --- a/monetdb5/modules/mal/sample.c > +++ b/monetdb5/modules/mal/sample.c > @@ -96,6 +96,23 @@ SAMPLEuniform(bat *r, bat *b, ptr s) { > } > > sample_export str > +SAMPLEsubuniform(bat *r, bat *b, ptr s) { > + BAT *br, *bb; > + > + if ((bb = BATdescriptor(*b)) == NULL) { > + throw(MAL, "sample.subuniform", INTERNAL_BAT_ACCESS); > + } > + br = BATsample_(bb,*(BUN *)s); > + if (br == NULL) > + throw(MAL, "sample.subuniform", OPERATION_FAILED); > + > + BBPunfix(bb->batCacheid); > + BBPkeepref(*r = br->batCacheid); > + return MAL_SUCCEED; > + > +} > + > +sample_export str > SAMPLEuniform_dbl(bat *r, bat *b, ptr p) { > BAT *bb; > double pr = *(double *)p; > diff --git a/monetdb5/modules/mal/sample.h b/monetdb5/modules/mal/sample.h > --- a/monetdb5/modules/mal/sample.h > +++ b/monetdb5/modules/mal/sample.h > @@ -42,6 +42,9 @@ sample_export str > SAMPLEuniform(bat *r, bat *b, ptr s); > > sample_export str > +SAMPLEsubuniform(bat *r, bat *b, ptr s); > + > +sample_export str > SAMPLEuniform_dbl(bat *r, bat *b, ptr p); > > #endif > diff --git a/monetdb5/modules/mal/sample.mal b/monetdb5/modules/mal/sample.mal > --- a/monetdb5/modules/mal/sample.mal > +++ b/monetdb5/modules/mal/sample.mal > @@ -27,6 +27,10 @@ command uniform(b:bat[:oid,:any],s:wrd): > address SAMPLEuniform > comment "Returns a uniform sample of size s" > > +command subuniform(b:bat[:oid,:any],s:wrd):bat[:oid,:oid] > +address SAMPLEsubuniform > +comment "Returns the oids of a uniform sample of size s" > + > command uniform(b:bat[:oid,:any],p:dbl):bat[:oid,:any] > address SAMPLEuniform_dbl > comment "Returns a uniform sample of size = (p x count(b)), where 0 <= p <= 1.0" > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -2677,7 +2677,6 @@ rel2bin_sample( mvc *sql, sql_rel *rel, > sc = column(sql->sa, sc); > sample = stmt_sample(sql->sa, stmt_alias(sql->sa, sc, tname, cname),s); > > - sample = stmt_reverse(sql->sa, stmt_mark_tail(sql->sa, sample, 0)); > for ( ; n; n = n->next) { > stmt *sc = n->data; > char *cname = column_name(sql->sa, sc); > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -983,7 +983,7 @@ _dumpstmt(backend *sql, MalBlkPtr mb, st > case st_sample:{ > int l = _dumpstmt(sql, mb, s->op1); > int r = _dumpstmt(sql, mb, s->op2); > - q = newStmt(mb, "sample", "uniform"); > + q = newStmt(mb, "sample", "subuniform"); > q = pushArgument(mb, q, l); > q = pushArgument(mb, q, r); > s->nr = getDestVar(q); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: Oct2012 - autogen: fix for Python 3
by Stefan Manegold 04 Dec '12

04 Dec '12

This (or previous?) checkin(s) seem to break compilation with python 2.7.3: Making all in python make[4]: Entering directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make all-am make[5]: Entering directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || cp -p '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/setup.py' 'setup.py' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || mkdir -p '['monetdb']' [ '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python' -ef . ] || cp -p '/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/['monetdb']'/*.py '['monetdb']' cp: cannot stat `/ufs/manegold/_/Monet/HG/Oct2012/source/MonetDB/clients/python/[monetdb]/*.py': No such file or directory make[5]: *** [all-local-python_setup] Error 1 make[5]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make[4]: *** [all] Error 2 make[4]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients/python' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients' make[2]: *** [all] Error 2 make[2]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize/clients' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Oct2012/builds/--disable-debug_--enable-assert_--enable-optimize' make: *** [all] Error 2 Stefan On Tue, Dec 04, 2012 at 09:08:46PM +0100, Fabian Groffen wrote: > Changeset: d46e57d76fec for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d46e57d76fec > Modified Files: > buildtools/autogen/autogen.py > buildtools/autogen/autogen/am.py > buildtools/autogen/autogen/codegen.py > buildtools/autogen/autogen/msc.py > buildtools/autogen/autogen/var.py > Branch: Oct2012 > Log Message: > > autogen: fix for Python 3 > [...] -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4

Dear MonetDB development team. I have some question for MonetDB
by 김경호 03 Dec '12

03 Dec '12

Dear MonetDB development team. I am a graduate student attending Korea's Hongik University. I am working on a project that improving query processing performance. I want to add some algorithms in MonetDB source code. Can I ask some questions by this e-mail address? Yours sincerely. fantastictoy님의 블로그 안녕하세요. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 2

Re: MonetDB: default - allow fixed number of partitions using --set ...
by Fabian Groffen 02 Dec '12

02 Dec '12

On 02-12-2012 11:51:30 +0100, Niels Nes wrote: > Changeset: 6e166ce6744e for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6e166ce6744e > Modified Files: > gdk/gdk_utils.c > gdk/gdk_utils.h > monetdb5/optimizer/opt_mitosis.c > Branch: default > Log Message: > > allow fixed number of partitions using --set mito_parts=? isn't this the same as --forcemito + gdk_nr_threads=? -- Fabian Groffen fabian(a)monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Octopus and
by Masood Mortazavi 02 Dec '12

02 Dec '12

Hi, Where would I find any documentation or instructions to try MonetDB's Octopus functionality, assuming it is still active at (or near) the tip? Regards, Masood P.S. I would also be interested in exploring Monet's any other distributed processing functionality, say Monet/Datacyclotron or Monet/MapReduce. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 1

New Feb2013 stable branch created
by Sjoerd Mullender 29 Nov '12

29 Nov '12

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Feb2013. So in each of your stable checkout directories you can do hg pull hg update -rFeb2013 to update to this new branch. -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Announcement: New Oct2012-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Nov '12

21 Nov '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP1 bugfix release Bug Fixes * 3155: Provide an option for setting discovery to false * 3161: invalid tuple received from server, got 138 columns, expected 476, ignoring * 3182: When using aggregate SQL functions, precision & decimal metadata for numeric/decimal type returned is 0,0 _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

[Bug 3178] one scan is enough to implement ALGstdev_@1 in monetdb5/modules/kernel/algebra.mx
by Yinhe Cao 20 Nov '12

20 Nov '12

Hi Fabian, Thank you very much for your email. I had the same concern about the accumulation error of one-scan approach as you do, even though mathematically they should be exactly the same. I remember about 10 years ago, I did some experimentations on both of the approaches and I did not found any noticeable differences between the results, but the sample size I tried then is not huge. For the sample size of the order 1 trillion, I have the same concern even with the two-scan approach. For example, After 500 billion entries have been added to the sum of square, the rest may not be able to be added up anymore, since it may well with the range of accumulation errors! For a huge BAT, if the precision is top priority, we may need to use an hierarchical approach: divide the huge sample into some manageable block, compute it for each block first, and then aggregate the results. Thanks, Yinhe _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: default - Cleanup code upon request.
by Fabian Groffen 15 Nov '12

15 Nov '12

On 15-11-2012 08:31:51 +0100, Martin Kersten wrote: > +/* There is just a single queue for the workers */ > static void > MRqueueCreate(int sz) > { > @@ -54,14 +55,21 @@ MRqueueCreate(int sz) > MT_lock_init(&mrqlock, "q_create"); > MT_lock_set(&mrqlock, "q_create"); > MT_sema_init(&mrqsema, 0, "q_create"); > - sz = ((sz << 1) >> 1); /* we want a multiple of 2 */ > + if ( mrqueue ) { > + GDKerror("One map-reduce queue allowed"); > + return; > + } > + sz *= 2; > mrqueue = (MRqueue *) GDKzalloc(sizeof(MRqueue) * sz); > - assert(mrqueue); > + if ( mrqueue == 0) { > + GDKerror("Could not create the map-reduce queue"); > + return; shouldn't you unset the mrqlock here? > + } _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

BugID=3178: A patch to compute stdev using one scan of BAT is available
by Yinhe Cao 13 Nov '12

13 Nov '12

Hi, The current implementation of ALGstdev_@1 in monetdb5/modules/kernel/algebra.mx takes two scans of the BAT. First scan to compute the sum (average) Second scan to compute stdev. By using a well-known mathematical property of the stdev, one scan is enough: during the scan, the sum of the value and the square of the value can be computed together, and the stdev can be computed then. The new implementation would roughly cut the running time by half for most of meaningful cases, especially when the bat is big. Link to the patch: http://bugs.monetdb.org/attachment.cgi?id=158&action=edit Link to the bug: http://bugs.monetdb.org/show_bug.cgi?id=3178 Any comments or suggestions will be greatly appreciated. Thanks, Yinhe _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Clean install gcc pragma warnings
by Stefan de Konink 10 Nov '12

10 Nov '12

Hi, I just try to install MonetDB on a new system. Used hg and updated to Oct2013. y.tab.c: In function 'sqlparse': y.tab.c:6609:3: error: unknown option after '#pragma GCC diagnostic' kind [-Werror=pragmas] y.tab.c:13496:3: error: unknown option after '#pragma GCC diagnostic' kind [-Werror=pragmas] cc1: all warnings being treated as errors I'm currently running gcc (Gentoo 4.6.3 p1.7, pie-0.5.2) 4.6.3 Is this known? Stefan _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

RubyGem on Jul2012
by Charalampos Nikolaou 08 Nov '12

08 Nov '12

Hi there, it's my first time playing around with the Jul2012 version of MonetDB and during "make install" I encountered the following error: mkdir -p '/Library/Ruby/Site/1.8' gem install --local --install-dir '/Library/Ruby/Site/1.8' --force --rdoc 'activerecord-monetdb-adapter-0.1.gem' ERROR: While executing gem ... (Gem::FilePermissionError) You don't have write permissions into the /Library/Ruby/Site/1.8 directory. As you can see, I am on a Mac (10.7) and Ruby is installed in /Library/Ruby as indicated above, BUT since I work with many versions of MonetDB, I install all of them "locally", in my home directory. Thus, I would prefer not to install the MonetDB gem file in a system directory. Is that possible? By reading the help output of the configuration, I saw that this is not possible, or might be? Moreover, I understood by the commit messages that this addition (Ruby) is needed for providing another client for MonetDB server. While I do not need such a client, is there an option to avoid building/using/producing it in the compilation process? I didn't see any option in the configuration as well. Thanks a lot, Babis _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 5

Re: MonetDB: default - Low-level task scheduler.
by Sjoerd Mullender 07 Nov '12

07 Nov '12

There are a few things wrong with this code: - sz·=·((sz·<<·1)·>>·1); does *not* turn sz into a multiple of two (as suggested by the comment). This statement basically is a no-op. - when you create joinable threads, you should join them. - it's not a great idea to use assert to make sure that GDKmalloc succeeds. Better is to return an error. - The include of monet_options.h should be in the C file, not in the include file (which would be included elsewhere where monet_options.h will already be included). On 2012-11-07 13:03, Martin Kersten wrote: > Changeset: 5ff3c16e865f for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=5ff3c16e865f > Added Files: > gdk/gdk_mapreduce.c > gdk/gdk_mapreduce.h > Modified Files: > gdk/Makefile.ag > monetdb5/modules/mal/groups.c > Branch: default > Log Message: > > Low-level task scheduler. > This module provide a lightweight map-reduce scheduler for multicore systems. > A limited number of workers are initialized upfront, which take the tasks > from a central queue. The header of these task descriptors should comply > with the MRtask structure. > > > diffs (239 lines): > > diff --git a/gdk/Makefile.ag b/gdk/Makefile.ag > --- a/gdk/Makefile.ag > +++ b/gdk/Makefile.ag > @@ -36,7 +36,7 @@ lib_gdk = { > gdk_private.h gdk_delta.h gdk_logger.h gdk_posix.h \ > gdk_system.h gdk_tm.h gdk_storage.h \ > gdk_calc.c gdk_calc.h gdk_calc_compare.h gdk_calc_private.h \ > - gdk_aggr.c gdk_group.c \ > + gdk_aggr.c gdk_group.c gdk_mapreduce.c gdk_mapreduce.h \ > bat.feps bat1.feps bat2.feps \ > libbat.rc > LIBS = ../common/options/libmoptions \ > diff --git a/gdk/gdk_mapreduce.c b/gdk/gdk_mapreduce.c > new file mode 100644 > --- /dev/null > +++ b/gdk/gdk_mapreduce.c > @@ -0,0 +1,141 @@ > +/* > + * The contents of this file are subject to the MonetDB Public License > + * Version 1.1 (the "License"); you may not use this file except in > + * compliance with the License. You may obtain a copy of the License at > + * http://www.monetdb.org/Legal/MonetDBLicense > + * > + * Software distributed under the License is distributed on an "AS IS" > + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > + * License for the specific language governing rights and limitations > + * under the License. > + * > + * The Original Code is the MonetDB Database System. > + * > + * The Initial Developer of the Original Code is CWI. > + * Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > + * Copyright August 2008-2012 MonetDB B.V. > + * All Rights Reserved. > + */ > + > +/* > + * (co) Martin L. Kersten > + * This module provide a lightweight map-reduce scheduler for multicore systems. > + * A limited number of workers are initialized upfront, which take the tasks > + * from a central queue. The header of these task descriptors should comply > + * with the MRtask structure. > + * > + */ > +#include "monetdb_config.h" > +#include "gdk.h" > +#include "gdk_mapreduce.h" > + > +/* each entry in the queue contains a list of tasks */ > +typedef struct MRQUEUE { > + MRtask **tasks; > + int index; /* next available task */ > + int size; /* number of tasks */ > +} MRqueue; > + > +static MRqueue *mrqueue; > +static int mrqsize= -1; /* size of queue */ > +static int mrqlast= -1; > +static MT_Lock mrqlock; /* its a shared resource, ie we need locks */ > +static MT_Sema mrqsema; /* threads wait on empty queues */ > + > + > +static void MRworker(void *); > + > +static void > +MRqueueCreate(int sz) > +{ > + int i; > + MT_Id tid; > + > + MT_lock_init(&mrqlock, "q_create"); > + MT_lock_set(&mrqlock,"q_create"); > + MT_sema_init(&mrqsema, 0, "q_create"); > + sz = ((sz << 1) >> 1); /* we want a multiple of 2 */ > + mrqueue = (MRqueue*)GDKzalloc(sizeof(MRqueue) *sz); > + assert(mrqueue); > + mrqsize = sz; > + mrqlast = 0; > + /* create a worker thread for each core as specified as system parameter*/ > + for ( i =0; i < GDKnr_threads; i++) > + MT_create_thread(&tid, MRworker, (void *) 0, MT_THR_JOINABLE); > + MT_lock_unset(&mrqlock,"q_create"); > +} > + > +static void > +MRenqueue(int taskcnt, MRtask **tasks) > +{ > + assert(taskcnt > 0); > + MT_lock_set(&mrqlock, "mrqlock"); > + if (mrqlast == mrqsize) { > + mrqsize <<= 1; > + mrqueue = (MRqueue*) GDKrealloc(mrqueue, sizeof(MRqueue) * mrqsize); > + } > + mrqueue[mrqlast].index = 0; > + mrqueue[mrqlast].tasks = tasks; > + mrqueue[mrqlast].size = taskcnt; > + mrqlast++; > + MT_lock_unset(&mrqlock, "mrqlock"); > + /* a task list is added for consumption*/ > + while (taskcnt-- > 0) > + MT_sema_up(&mrqsema, "mrqsema"); > +} > + > +static MRtask * > +MRdequeue(void) > +{ > + MRtask *r = NULL; > + int idx; > + > + MT_sema_down(&mrqsema, "mrqsema"); > + assert(mrqlast); > + MT_lock_set(&mrqlock, "mrqlock"); > + if (mrqlast > 0) { > + idx = mrqueue[mrqlast-1].index; > + r = mrqueue[mrqlast-1].tasks[idx++]; > + if ( mrqueue[mrqlast-1].size == idx) > + mrqlast--; > + else > + mrqueue[mrqlast-1].index = idx; > + } > + MT_lock_unset(&mrqlock, "mrqlock"); > + assert(r); > + return r; > +} > + > +static void > +MRworker(void * arg) > +{ > + MRtask *task; > + (void) arg; > + do{ > + task= MRdequeue(); > + (task->cmd)(task); > + MT_sema_up(task->sema, "mrqsema"); > + } while (1); > +} > + > +/* schedule the tasks and return when all are done */ > +void > +MRschedule(int taskcnt, void **arg, void (*cmd)(void*p)) > +{ > + int i; > + MT_Sema sema; > + MRtask **task = (MRtask**) arg; > + > + if ( mrqueue == 0) > + MRqueueCreate(1024); > + > + MT_sema_init(&sema, 0, "q_create"); > + for ( i= 0; i < taskcnt; i++){ > + task[i]->sema = & sema; > + task[i]->cmd = cmd; > + } > + MRenqueue(taskcnt,task); > + /* waiting for all report result */ > + for ( i= 0; i < taskcnt; i++) > + MT_sema_down(&sema, "mrqsema"); > +} > diff --git a/gdk/gdk_mapreduce.h b/gdk/gdk_mapreduce.h > new file mode 100644 > --- /dev/null > +++ b/gdk/gdk_mapreduce.h > @@ -0,0 +1,32 @@ > +/* > + * The contents of this file are subject to the MonetDB Public License > + * Version 1.1 (the "License"); you may not use this file except in > + * compliance with the License. You may obtain a copy of the License at > + * http://www.monetdb.org/Legal/MonetDBLicense > + * > + * Software distributed under the License is distributed on an "AS IS" > + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > + * License for the specific language governing rights and limitations > + * under the License. > + * > + * The Original Code is the MonetDB Database System. > + * > + * The Initial Developer of the Original Code is CWI. > + * Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > + * Copyright August 2008-2012 MonetDB B.V. > + * All Rights Reserved. > + */ > + > +#ifndef _GDK_MAPREDUCE_H_ > +#define _GDK_MAPREDUCE_H_ > + > +#include <monet_options.h> > + > +typedef struct{ > + MT_Sema *sema; /* micro scheduler handle */ > + void (*cmd)(void *); /* the function to be executed */ > +}MRtask; > + > +gdk_export void MRschedule(int taskcnt, void **arg, void (*cmd)(void *p)); > + > +#endif /* _GDK_MAPREDUCE_H_ */ > diff --git a/monetdb5/modules/mal/groups.c b/monetdb5/modules/mal/groups.c > --- a/monetdb5/modules/mal/groups.c > +++ b/monetdb5/modules/mal/groups.c > @@ -66,11 +66,15 @@ GRPmulticolumngroup(Client cntxt, MalBlk > /* sort order may have influences */ > /* SF100 Q16 showed < ordering is 2 times faster as > ordering */ > for ( i = 3; i< pci->argc; i++) > - for ( j = i+1; j<pci->argc; j++) > - if ( sizes[j] < sizes[i]){ > - l = sizes[j]; sizes[j]= sizes[i]; sizes[i]= l; > - bi = bid[j]; bid[j]= bid[i]; bid[i]= bi; > - } > + for ( j = i+1; j<pci->argc; j++) > + if ( sizes[j] < sizes[i]){ > + l = sizes[j]; > + sizes[j]= sizes[i]; > + sizes[i]= l; > + bi = bid[j]; > + bid[j]= bid[i]; > + bid[i]= bi; > + } > /* for (i=2; i<pci->argc; i++) > mnstr_printf(cntxt->fdout,"# after [%d] "LLFMT"\n",i, sizes[i]); */ > > @@ -82,8 +86,6 @@ GRPmulticolumngroup(Client cntxt, MalBlk > i = 4; > if (msg == MAL_SUCCEED && pci->argc > 4 ) > do { > - if (*ext) > - BBPdecref(*ext, TRUE); > /* early break when there are as many groups as histogram entries */ > b = BATdescriptor(*hist); > if ( b ){ > @@ -91,8 +93,8 @@ GRPmulticolumngroup(Client cntxt, MalBlk > BBPreleaseref(*hist); > if ( j) break; > } > - if (*hist) > - BBPdecref(*hist, TRUE); > + BBPdecref(*ext, TRUE); > + BBPdecref(*hist, TRUE); > > /* (grp,ext,hist) := group.subgroupdone(arg,grp) */ > oldgrp= *grp; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Where can I add my join algorithm in MonetDB?
by Xiaojun Chang 05 Nov '12

05 Nov '12

Hi, I am one Ph.D student and right now I am working on IMDB. Could anyone tell me where I can add my join algorithm in MonetDB? I cannot find such document. Thanks. Xiaojun Chang _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 2

Re: cloning a repository from release MonetDB-11.9.7
by Chi-Young Ku 02 Nov '12

02 Nov '12

Hi, I would like to clone a repository from MonetDB-11.9.7. What would be the branch or rev number for: hg clone http://dev.monetdb.org/hg/MonetDB -b Thanks in advance. Chi ________________________________ From: Chi-Young Ku <chi(a)querymetrica.com> To: "developers-list(a)monetdb.org" <developers-list(a)monetdb.org> Cc: Chi- Young Ku <chi(a)querymetrica.com> Sent: Wednesday, October 17, 2012 5:47 PM Subject: Generating gprof report Hi, I used the "--enable_profile" option to configure my MonetDB and ran "make clean; make; make install". I, then, started my MonetDB instance and ran the following query: select count(*) from lineitem where l_partkey > 9000; Lastly, I quit from the "mclient" session and issued "monetdbd stop ...". I observed that there are several gmon.out files in different directories in my top build directory. Therefore, I issued the following gprof command: gprof /usr/local/bin/mserver5 ./sql/backends/monet5/gmon.out ./gdk/gmon.out ./monetdb5/extras/mal_optimizer_template/gmon.out ./monetdb5/modules/atoms/gmon.out ./monetdb5/modules/kernel/gmon.out ./monetdb5/modules/mal/gmon.out ./monetdb5/scheduler/gmon.out ./monetdb5/optimizer/gmon.out ./monetdb5/mal/gmon.out However, the output of the above command did not show any calling sequence containing the execution of MAL instructions. Did I miss anything? Thanks in advance. Chi _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 2

Re: MonetDB: Oct2012 - Ported to Ubuntu Quantal Quetzal.
by Fabian Groffen 01 Nov '12

01 Nov '12

On 01-11-2012 08:53:47 +0100, Sjoerd Mullender wrote: > Changeset: 4c6c6b484d34 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=4c6c6b484d34 > Modified Files: > configure.ag > Branch: Oct2012 > Log Message: > > Ported to Ubuntu Quantal Quetzal. > Whether it is the newer geos library (3.3.3) or something > quantal-specific, I don't know, but we need to use the --clibs option > to geos-config. This option does not exist on our Fedora system. It is an option from newer geos. However, my 3.3.5 has both libs and clibs options. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Re: MonetDB: default - The storage cost calculator
by Stefan Manegold 25 Oct '12

25 Oct '12

This checkin --- strangely dated Sep 21 2012; cf. http://dev.monetdb.org/hg/MonetDB/rev/523120729e8d !?? --- breaks compilation: http://monetdb.cwi.nl/testweb/web/status.php Martin Kersten <commits(a)monetdb.org> wrote: Changeset: 523120729e8d for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=523120729e8d Added Files: sql/scripts/75_storagemodel.sql Modified Files: monetdb5/mal/mal_profiler.c sql/backends/monet5/sql.mx sql/scripts/Makefile.ag Branch: default Log Message: The storage cost calculator This script gives the database administrator insight in the actual footprint of the persistent tables and the maximum playground used when indices are introduced upon them. By chancing the storagemodelinput table directly, the footprint for yet to be loaded databases can be assessed. The actual storage footprint of an existing database can be obtained by the table procuding function storage() It represents the actual state of affairs, i.e. storage on disk of columns and foreign key indices, and possible temporary hash indices. For strings we take a sample to determine their average length. diffs (truncated from 425 to 300 lines): diff --git a/monetdb5/mal/mal_profiler.c b/monetdb5/mal/mal_profiler.c --- a/monetdb5/mal/mal_profiler.c +++ b/monetdb5/mal/mal_profiler.c @@ -182,6 +182,10 @@ profilerEvent(int idx, MalBlkPtr mb, Mal } if (profileCounter[PROFstart].status == 0 && start) return; + if (myname == 0) + myname = putName("profiler", 8); + if (getModuleId(getInstrPtr(mb, pc)) == myname) + return; if (offlineProfiling) offlineProfilerEvent(idx, mb, stk, pc,start); if (cachedProfiling) diff --git a/sql/backends/monet5/sql.mx b/sql/backends/monet5/sql.mx --- a/sql/backends/monet5/sql.mx +++ b/sql/backends/monet5/sql.mx @@ -504,12 +504,13 @@ pattern storage()( schema:bat[:oid,:str], table:bat[:oid,:str], column:bat[:oid,:str], + type:bat[:oid,:str], location:bat[:oid,:str], count:bat[:oid,:lng], - capacity:bat[:oid,:lng], - width:bat[:oid,:int], - size:bat[:oid,:lng], - hashsize:bat[:oid,:lng], + atomwidth:bat[:oid,:int], + columnsize:bat[:oid,:lng], + heap:bat[:oid,:lng], + indices:bat[:oid,:lng], sorted:bat[:oid,:bit]) address sql_storage comment "return a table with storage information "; @@ -7347,13 +7348,13 @@ str SQLoptimizersUpdate(Client cntxt, Ma * Inspection of the actual storage footprint is a recurring question of users. * This is modelled as a generic SQL table producing function. * create function storage() - * returns table ("schema" string, "table" string, "column" string, location string, "count" bigint, capacity bigint, width int, size bigint, hashsize bigint, sorted int) + * returns table ("schema" string, "table" string, "column" string, "type" string, location string, "count" bigint, width int, columnsize bigint, heapsize bigint indices bigint, sorted int) * external name sql.storage; */ str sql_storage(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci) { - BAT *sch, *tab, *col, *loc, *cnt, *cap, *atom, *size, *aux, *sort; + BAT *sch, *tab, *col, *type, *loc, *cnt, *atom, *size, *heap, *indices, *sort; mvc *m = NULL; str msg = getSQLContext(cntxt,mb, &m, NULL); sql_trans *tr = m->session->tr; @@ -7362,13 +7363,14 @@ sql_storage(Client cntxt, MalBlkPtr mb, int *rsch = (int*) getArgReference(stk,pci,0); int *rtab = (int*) getArgReference(stk,pci,1); int *rcol = (int*) getArgReference(stk,pci,2); - int *rloc = (int*) getArgReference(stk,pci,3); - int *rcnt = (int*) getArgReference(stk,pci,4); - int *rcap = (int*) getArgReference(stk,pci,5); + int *rtype = (int*) getArgReference(stk,pci,3); + int *rloc = (int*) getArgReference(stk,pci,4); + int *rcnt = (int*) getArgReference(stk,pci,5); int *ratom = (int*) getArgReference(stk,pci,6); int *rsize = (int*) getArgReference(stk,pci,7); - int *raux = (int*) getArgReference(stk,pci,8); - int *rsort = (int*) getArgReference(stk,pci,9); + int *rheap = (int*) getArgReference(stk,pci,8); + int *rindices = (int*) getArgReference(stk,pci,9); + int *rsort = (int*) getArgReference(stk,pci,10); if (msg) return msg; @@ -7379,31 +7381,34 @@ sql_storage(Client cntxt, MalBlkPtr mb, BATseqbase(tab, 0); col = BATnew(TYPE_void,TYPE_str, 0); BATseqbase(col, 0); + type = BATnew(TYPE_void,TYPE_str, 0); + BATseqbase(type, 0); loc = BATnew(TYPE_void,TYPE_str, 0); BATseqbase(loc, 0); cnt = BATnew(TYPE_void,TYPE_lng, 0); BATseqbase(cnt, 0); - cap = BATnew(TYPE_void,TYPE_lng, 0); - BATseqbase(cap, 0); atom = BATnew(TYPE_void,TYPE_int, 0); BATseqbase(atom, 0); size = BATnew(TYPE_void,TYPE_lng, 0); BATseqbase(size, 0); - aux = BATnew(TYPE_void,TYPE_lng, 0); - BATseqbase(aux, 0); + heap = BATnew(TYPE_void,TYPE_lng, 0); + BATseqbase(heap, 0); + indices = BATnew(TYPE_void,TYPE_lng, 0); + BATseqbase(indices, 0); sort = BATnew(TYPE_void,TYPE_bit, 0); BATseqbase(sort, 0); - if ( sch == NULL || tab == NULL || col == NULL || loc == NULL || sort == NULL || - cnt == NULL || cap == NULL || atom == NULL || size == NULL || aux == NULL){ + if ( sch == NULL || tab == NULL || col == NULL || type == NULL || loc == NULL || sort == NULL || + cnt == NULL || atom == NULL || size == NULL || heap == NULL ||indices == NULL){ if ( sch ) BBPreleaseref(sch->batCacheid); if ( tab ) BBPreleaseref(tab->batCacheid); if ( col ) BBPreleaseref(col->batCacheid); if ( loc ) BBPreleaseref(loc->batCacheid); if ( cnt ) BBPreleaseref(cnt->batCacheid); - if ( cap ) BBPreleaseref(cap->batCacheid); + if ( type ) BBPreleaseref(type->batCacheid); if ( atom ) BBPreleaseref(atom->batCacheid); if ( size ) BBPreleaseref(size->batCacheid); - if ( aux ) BBPreleaseref(aux->batCacheid); + if ( heap ) BBPreleaseref(heap->batCacheid); + if ( indices ) BBPreleaseref(indices->batCacheid); if ( sort ) BBPreleaseref(sort->batCacheid); throw(SQL,"sql.storage", MAL_MALLOC_FAIL); } @@ -7412,7 +7417,8 @@ sql_storage(Client cntxt, MalBlkPtr mb, sql_schema *s= (sql_schema*) nsch->data; if ( isalpha((int)b->name[0]) ) - if (s->tables.set) for(ntab= (s)->tables.set->h ;ntab; ntab= ntab->next){ + if (s->tables.set) + for(ntab= (s)->tables.set->h ;ntab; ntab= ntab->next){ sql_base *bt= ntab->data; sql_table *t= (sql_table*) bt; if (isTable(t)) @@ -7426,28 +7432,49 @@ sql_storage(Client cntxt, MalBlkPtr mb, sch = BUNappend(sch, b->name, FALSE); tab = BUNappend(tab, bt->name, FALSE); col = BUNappend(col, bc->name, FALSE); + type = BUNappend(type, c->type.type->sqlname, FALSE); /*printf(" cnt "BUNFMT, BATcount(bn));*/ sz= BATcount(bn); cnt = BUNappend(cnt, &sz, FALSE); - /*printf(" cap "BUNFMT, BATcapacity(bn));*/ - sz= BATcapacity(bn); - cap = BUNappend(cap, &sz, FALSE); /*printf(" loc %s", BBP_physical(bn->batCacheid));*/ loc = BUNappend(loc, BBP_physical(bn->batCacheid), FALSE); /*printf(" width %d", bn->T->width);*/ w= bn->T->width; + if ( bn->ttype == TYPE_str){ + BUN p,q; + double sum=0; + BATiter bi = bat_iterator(bn); + lng cnt1,cnt2= cnt1=(lng) BATcount(bn); + + /* just take a sample */ + if ( cnt1 > 512) + cnt1 = cnt2 = 512; + BATloop(bn,p,q){ + str s = BUNtail(bi,p); + if( s != NULL && strcmp(s, str_nil)) + sum += (int) strlen(s); + if ( --cnt1 <= 0) + break; + } + if ( cnt2) + w = (int) (sum/cnt2); + } atom = BUNappend(atom, &w, FALSE); - /*printf(" size "BUNFMT, tailsize(bn,BATcount(bn)) + (bn->T->vheap? bn->T->vheap->size:0));*/ - sz = tailsize(bn,BATcapacity(bn)) + (bn->T->vheap? bn->T->vheap->size:0); - sz += headsize(bn,BATcapacity(bn)) + (bn->H->vheap? bn->H->vheap->size:0); + + sz = tailsize(bn,BATcount(bn)); + sz += headsize(bn,BATcount(bn)); size = BUNappend(size, &sz, FALSE); + sz = bn->T->vheap? bn->T->vheap->size:0; + sz += bn->H->vheap? bn->H->vheap->size:0; + heap = BUNappend(heap, &sz, FALSE); + sz = bn->T->hash?bn->T->hash->heap->size:0; sz += bn->H->hash?bn->H->hash->heap->size:0; - aux = BUNappend(aux, &sz, FALSE); - /*printf(" auxsize "BUNFMT, bn->T->hash?bn->T->hash->heap->size:0);*/ + indices = BUNappend(indices, &sz, FALSE); + /*printf(" indices "BUNFMT, bn->T->hash?bn->T->hash->heap->size:0);*/ /*printf("\n");*/ w = BATtordered(bn); @@ -7468,28 +7495,49 @@ sql_storage(Client cntxt, MalBlkPtr mb, sch = BUNappend(sch, b->name, FALSE); tab = BUNappend(tab, bt->name, FALSE); col = BUNappend(col, bc->name, FALSE); + type = BUNappend(type, "oid", FALSE); /*printf(" cnt "BUNFMT, BATcount(bn));*/ sz= BATcount(bn); cnt = BUNappend(cnt, &sz, FALSE); - /*printf(" cap "BUNFMT, BATcapacity(bn));*/ - sz= BATcapacity(bn); - cap = BUNappend(cap, &sz, FALSE); /*printf(" loc %s", BBP_physical(bn->batCacheid));*/ loc = BUNappend(loc, BBP_physical(bn->batCacheid), FALSE); /*printf(" width %d", bn->T->width);*/ w= bn->T->width; + if ( bn->ttype == TYPE_str){ + BUN p,q; + double sum=0; + BATiter bi = bat_iterator(bn); + lng cnt1, cnt2= cnt1 = BATcount(bn); + + /* just take a sample */ + if ( cnt1 > 512) + cnt1 = cnt2 = 512; + BATloop(bn,p,q){ + str s = BUNtail(bi,p); + if( s != NULL && strcmp(s, str_nil)) + sum += (int) strlen(s); + if ( --cnt1 <= 0) + break; + } + if ( cnt2) + w = (int) (sum/cnt2); + } atom = BUNappend(atom, &w, FALSE); /*printf(" size "BUNFMT, tailsize(bn,BATcount(bn)) + (bn->T->vheap? bn->T->vheap->size:0));*/ - sz = tailsize(bn,BATcapacity(bn)) + (bn->T->vheap? bn->T->vheap->size:0); - sz += headsize(bn,BATcapacity(bn)) + (bn->H->vheap? bn->H->vheap->size:0); + sz = tailsize(bn,BATcount(bn)); + sz += headsize(bn,BATcount(bn)); size = BUNappend(size, &sz, FALSE); + sz = bn->T->vheap? bn->T->vheap->size:0; + sz += bn->H->vheap? bn->H->vheap->size:0; + heap = BUNappend(heap, &sz, FALSE); + sz = bn->T->hash?bn->T->hash->heap->size:0; sz += bn->H->hash?bn->H->hash->heap->size:0; - aux = BUNappend(aux, &sz, FALSE); - /*printf(" auxsize "BUNFMT, bn->T->hash?bn->T->hash->heap->size:0);*/ + indices = BUNappend(indices, &sz, FALSE); + /*printf(" indices "BUNFMT, bn->T->hash?bn->T->hash->heap->size:0);*/ /*printf("\n");*/ w = BATtordered(bn); sort = BUNappend(sort, &w, FALSE); @@ -7504,11 +7552,12 @@ sql_storage(Client cntxt, MalBlkPtr mb, BBPkeepref(*rtab = tab->batCacheid); BBPkeepref(*rcol = col->batCacheid); BBPkeepref(*rloc = loc->batCacheid); + BBPkeepref(*rtype = type->batCacheid); BBPkeepref(*rcnt = cnt->batCacheid); - BBPkeepref(*rcap = cap->batCacheid); BBPkeepref(*ratom = atom->batCacheid); BBPkeepref(*rsize = size->batCacheid); - BBPkeepref(*raux = aux->batCacheid); + BBPkeepref(*rheap = heap->batCacheid); + BBPkeepref(*rindices = indices->batCacheid); BBPkeepref(*rsort = sort->batCacheid); return MAL_SUCCEED; } diff --git a/sql/scripts/75_storagemodel.sql b/sql/scripts/75_storagemodel.sql new file mode 100644 --- /dev/null +++ b/sql/scripts/75_storagemodel.sql @@ -0,0 +1,154 @@ +-- The contents of this file are subject to the MonetDB Public License +-- Version 1.1 (the "License"); you may not use this file except in +-- compliance with the License. You may obtain a copy of the License at +-- http://www.monetdb.org/Legal/MonetDBLicense +-- +-- Software distributed under the License is distributed on an "AS IS" +-- basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the +-- License for the specific language governing rights and limitations +-- under the License. +-- +-- The Original Code is the MonetDB Database System. +-- +-- The Initial Developer of the Original Code is CWI. +-- Copyright August 2008-2012 MonetDB B.V. +-- All Rights Reserved. + + +-- Author M.Kersten +-- This script gives the database administrator insight in the actual +-- footprint of the persistent tables and the maximum playground used +-- when indices are introduced upon them. +-- By chancing the storagemodelinput table directly, the footprint for +-- yet to be loaded databases can be assessed. + +-- The actual storage footprint of an existing database can be +-- obtained by the table procuding function storage() +-- It represents the actual state of affairs, i.e. storage on disk +-- of columns and foreign key indices, and possible temporary hash indices. +-- For strings we take a sample to determine their average length. + +create function storage() +returns table ("schema" string, "table" string, "column" string, "type" string, location string, "count" bigint, typewidth int, columnsize bigint, heapsize bigint, indices bigint, sorted boolean) +external name sql.storage; + +-- To determine the footprint of an arbitrary database, we first have +-- to define its schema, followed by an indication of the properties of each column. +-- A storage model input table for the size prediction is shown below: +create table storagemodelinput( + "schema" string, + "table" string, + "column" string, _______________________________________________ checkin-list mailing list checkin-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/checkin-list _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB Development Environment
by Stefan Manegold 24 Oct '12

24 Oct '12

Fedora, joe, gcc S. ----- Reply message ----- From: "Fabian Groffen" <Fabian.Groffen(a)cwi.nl> To: "Communication channel for developers of the MonetDB suite." <developers-list(a)monetdb.org> Subject: MonetDB Development Environment Date: Wed, Oct 24, 2012 10:08 On 23-10-2012 14:19:42 -0700, Masood Mortazavi wrote: > I'm wondering�what development environments (OS, editors, etc.)�or IDEs, if > any, MonetDB development team are using. Vim and Emacs. Fabian _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

MonetDB Development Environment
by Masood Mortazavi 24 Oct '12

24 Oct '12

Hi -- I'm wondering what development environments (OS, editors, etc.) or IDEs, if any, MonetDB development team are using. Regards, Masood _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Announcement: New Oct2012 Feature release of MonetDB suite
by Fabian Groffen 23 Oct '12

23 Oct '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Due to a packaging problem on Debian and Ubuntu, no binary packages for these systems exist. We apologise for the inconvenience. We expect this problem to be resolved by the Oct2012-SP1 release. Oct 2012 feature release Build Environment * Removed --enable-noexpand configure option. Java Module * Fixed problem with PreparedStatements and setXXX() methods using column numbers instead of names, bug #3158 Client Package * mclient now accepts URIs as database to connect to. * all strings returned by python2 are unicode, removed use_unicode option * python2 and 3 type convertion speed improvements * python2 uses new style objects now (bug #3104) * split python2 and python3 MonetDB5 Server * Removed module attach since it wasn't used or even tested. * The MAL-to-C Compiler (mcc) was removed. The code wasn't tested and most likely non-functional. Merovingian * Fixed problem where monetdbd would refuse to startup when discovery was set to false, bug #3155 MonetDB Common * Removed the gdk_embedded (and embedded) option. The code wasn't tested and most likely non-functional. * BAT-of-BATs is no longer allowed. It was already not allowed to make these types of BATs persistent, but now they can't be created at all anymore. Bug Fixes * 3084: Timestamp arithmetic very slow (especially on Windows) * 3125: Python tests fail after recent Python API changes * 3155: Provide an option for setting discovery to false _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

ANSI SQL 2003 Extension OVER() appears to be missing in MonetDB Click to flag this post
by James Becker III 22 Oct '12

22 Oct '12

Hello MonetDB Community. My name is James Becker, my friends call me Jaimi, and I am new to posting messages here, so I hope you will bear with me. I am conducting a POC on MonetDB to see if we can use it to replace Vertica for a large scale read mostly educational reporting application. So far things are showing promise, but the legacy reports lean heavily on OVER(), which doesn't appear to be part of the 11.11.11 release. Am I missing something, or is this the case? If anyone who has run into a similar limitations has any insights as to where I have made a wrong turn, or how to either implement OVER() in MonetDB using either MAL/C, or any suggestions for query work-arounds that would yield similar results, this would be greatly appreciated. Thx. ~jj _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 5

datetime extract function performance
by metin d 19 Oct '12

19 Oct '12

Hey, I'm running a small performance test on MonetDB and Postgres. According to my results "extract" function in monetDB (http://www.monetdb.org/Documentation/SQLreference/Temporal) is nearly 10x slower then Postgres. Do you know the reason, or do you have any idea which can make "extract" function faster in MonetDB ? Thanks, Metin _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

width_bucket function
by metin d 19 Oct '12

19 Oct '12

Hey, In features web page it says that monetDB is based on the SQL 2003 standard, but width_bucket function doesn't work although it is in SQL 2003 standard (http://savage.net.au/SQL/sql-2003-2.bnf.html#width bucket function). Thanks. Metin _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Some questions
by Sid Just 19 Oct '12

19 Oct '12

Greetings all, I have a simple question about MonetDB. I'm evaluating different several column-store db and I have some questions about Monet : * Is MonetDB a column-store database ? If yes I can't understand how I can query it with standard SQL (it'so simple bue it seem to be the ONLY column-store db that support SQL) * Can MonetDB scale-out over several machines (to build a cluster) for the same database ? * There is no .NET provider that I can use to connect to MonetDB for fast accessing from C# instead of using ODBC ?In general I'm looking for a database that can perform realtime queries against milions of records, because we are going to build a web application that need real fast access to the data. What are best practices in desing tables in MonetDB ? Thanks a lot to everyone! S _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Generating gprof report
by Chi-Young Ku 18 Oct '12

18 Oct '12

Hi, I used the "--enable_profile" option to configure my MonetDB and ran "make clean; make; make install". I, then, started my MonetDB instance and ran the following query: select count(*) from lineitem where l_partkey > 9000; Lastly, I quit from the "mclient" session and issued "monetdbd stop ...". I observed that there are several gmon.out files in different directories in my top build directory. Therefore, I issued the following gprof command: gprof /usr/local/bin/mserver5 ./sql/backends/monet5/gmon.out ./gdk/gmon.out ./monetdb5/extras/mal_optimizer_template/gmon.out ./monetdb5/modules/atoms/gmon.out ./monetdb5/modules/kernel/gmon.out ./monetdb5/modules/mal/gmon.out ./monetdb5/scheduler/gmon.out ./monetdb5/optimizer/gmon.out ./monetdb5/mal/gmon.out However, the output of the above command did not show any calling sequence containing the execution of MAL instructions. Did I miss anything? Thanks in advance. Chi _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

A couple of minor fixes to HowToStart.rst
by Masood Mortazavi 12 Oct '12

12 Oct '12

I have a couple of minor fixes to HowToStart.rst. Can I use "hg push" to push these changes back to the repository? Regards, Masood _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Installation issue
by Masood Mortazavi 11 Oct '12

11 Oct '12

Hi, I started with a clone of MonetDB's mercurial repository. I'm using Ubuntu with Linux kernel 3.0.x I followed the instruction in HowToStart.rst bootstrap, config and make moved forward well. There were no complaints in "sudo make install" . . . However, here's what I get: > ~/work/chi-MonetDBConfig$ which mserver5 /usr/local/bin/mserver5 > ~/work/chi-MonetDBConfig$ sudo mserver5 mserver5: error while loading shared libraries: libmonetdb5.so.13: cannot open shared object file: No such file or directory > ls -al /usr/local/lib/libmonetdb5.so lrwxrwxrwx 1 root root 21 2012-10-11 11:46 /usr/local/lib/libmonetdb5.so -> libmonetdb5.so.13.0.0 > ls -al /usr/local/lib/libmonetdb5.so.13.0.0 lrwxrwxrwx 1 root root 21 2012-10-11 11:46 /usr/local/lib/libmonetdb5.so -> libmonetdb5.so.13.0.0 Do I need to add the shared library path explicitly to my LD_LIBRARY_PATH by hand? It seems that it should be added as part of the installation. No? Regards, Masood _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

MonetDB - some questions
by Sid Just 11 Oct '12

11 Oct '12

Greetings all, I have a simple question about MonetDB. I'm evaluating different several column-store db and I have some questions about Monet : * Is MonetDB a column-store database ? If yes I can't understand how I can query it with standard SQL (it'so simple bue it seem to be the ONLY column-store db that support SQL) * Can MonetDB scale-out over several machines (to build a cluster) for the same database ? * There is no .NET provider that I can use to connect to MonetDB for fast accessing from C# instead of using ODBC ? In general I'm looking for a database that can perform realtime queries against milions of records, because we are going to build a web application that need real fast access to the data. What are best practices in desing tables in MonetDB ? Thanks a lot to everyone! S _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: developers-list Digest, Vol 2, Issue 1
by Masood Mortazavi 03 Oct '12

03 Oct '12

Hi Fabian, Does the core MonetDB development team use any public git repository or any other publicly available source-control repository for collaboration purposes? I did see this: https://github.com/search?langOverride=&q=MonetDB&repo=&start_value=1&type=… but it only seems to include more or less ad-hoc work on tooling. Perhaps, I'm missing something qutie obvious. Regards, Masood On Wed, Oct 3, 2012 at 3:00 AM, <developers-list-request(a)monetdb.org> wrote: Hi Masood, On 02-10-2012 10:22:41 -0700, Masood Mortazavi wrote: > I'm trying to search for answers to questions like the following: > > Whether MonetDB has a github (or some other) repository that is maintained > by the developers or owners of the project. > > What is the best way to obtain MonetDB 11.9.7 sources, which I'm trying to > build locally? The search engine of your choice, is your friend for these trivial questions. Best, Fabian _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 2

Mechanism to Search the Archive + Repository, if any
by Masood Mortazavi 02 Oct '12

02 Oct '12

Hi, What is the mechanism for searching the developers-list(a)monetdb.org archive? (A quick review of mailman users documentation didn't reveal that mechanism if one exists.) I'm trying to search for answers to questions like the following: Whether MonetDB has a github (or some other) repository that is maintained by the developers or owners of the project. What is the best way to obtain MonetDB 11.9.7 sources, which I'm trying to build locally? Regards, Masood _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Re: MonetDB: default - Checking the optimizer
by Fabian Groffen 27 Sep '12

27 Sep '12

On 27-09-2012 02:00:41 +0200, Martin Kersten wrote: > Changeset: 38c89881c700 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=38c89881c700 > Modified Files: > monetdb5/optimizer/opt_pipes.c > Branch: default > Log Message: > > Checking the optimizer > This list does not change the number of errors reported on SQL tests. this looks like a mess, was it intended to be committed? > --- a/monetdb5/optimizer/opt_pipes.c > +++ b/monetdb5/optimizer/opt_pipes.c > @@ -64,7 +64,22 @@ struct PIPELINES { > {"default_pipe", > "optimizer.inline();" > "optimizer.remap();" > - "optimizer.deadcode();" > + "optimizer.evaluate();" > + "optimizer.costModel();" > + "optimizer.coercions();" > + "optimizer.emptySet();" > + "optimizer.aliases();" > +// "optimizer.mitosis();" > +// "optimizer.mergetable();" > + "optimizer.deadcode();" > + "optimizer.commonTerms();" > + "optimizer.groups();" > + "optimizer.joinPath();" > +// "optimizer.reorder();" > + "optimizer.deadcode();" > + "optimizer.reduce();" > +// "optimizer.dataflow();" > + "optimizer.history();" > "optimizer.multiplex();" > "optimizer.garbageCollector();", > "stable", NULL, NULL, 1}, _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

[Monetdb-developers] MonetDB project will move to monetdb.org
by Fabian Groffen 26 Sep '12

26 Sep '12

We would like to inform you that as of Januari 2013, the monetdb project on SourceForge will be removed. All services that SourceForge has kindly provided us with in the past will be migrated to our monetdb.org infrastructure. Currently, the only remaining services hosted on SourceForge are the three mailing lists monetdb-announce, monetdb-users and monetdb-developers. As of this message, these lists are migrated to their equivalents on monetdb.org. The new lists are as follows: monetdb-announce(a)lists.sourceforge.net -> announce-list(a)monetdb.org monetdb-users(a)lists.sourceforge.net -> users-list(a)monetdb.org monetdb-developers(a)lists.sourceforge.net -> developers-list(a)monetdb.org We invite you to subscribe to the new lists on monetdb.org at your earliest convenience. Any posts to the old lists are rejected from now on. Posts to the new lists at monetdb.org are forwarded in batches to the old lists until the end of this year. The MonetDB team would like to take this opportunity to thank SourceForge for the services provided over the past years. We hope to welcome all current subscribers on our new lists at monetdb.org, see: http://mail.monetdb.org/mailman/listinfo The MonetDB team

1 0

[Monetdb-developers] MonetDB project will move to monetdb.org
by Fabian Groffen 26 Sep '12

26 Sep '12

We would like to inform you that as of Januari 2013, the monetdb project on SourceForge will be removed. All services that SourceForge has kindly provided us with in the past will be migrated to our monetdb.org infrastructure. Currently, the only remaining services hosted on SourceForge are the three mailing lists monetdb-announce, monetdb-users and monetdb-developers. As of this message, these lists are migrated to their equivalents on monetdb.org. The new lists are as follows: monetdb-announce(a)lists.sourceforge.net -> announce-list(a)monetdb.org monetdb-users(a)lists.sourceforge.net -> users-list(a)monetdb.org monetdb-developers(a)lists.sourceforge.net -> developers-list(a)monetdb.org We invite you to subscribe to the new lists on monetdb.org at your earliest convenience. Any posts to the old lists are rejected from now on. Posts to the new lists at monetdb.org are forwarded in batches to the old lists until the end of this year. The MonetDB team would like to take this opportunity to thank SourceForge for the services provided over the past years. We hope to welcome all current subscribers on our new lists at monetdb.org, see: http://www.monetdb.org/mailman/listinfo The MonetDB team

1 0

[Monetdb-developers] Announcement: New Jul2012-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Sep '12

21 Sep '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2012-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2012-SP2 bugfix release Java Module * Fixed a bug where DatabaseMetaData.getURL() did return null:0 for hostname:port. MonetDB5 Server * Changed the way nclients maximum was calculated to avoid 'out of client slots' errors way before the maximum was reached. Merovingian * Resolved a problem where monetdb could fail to start a database with the message "database 'X' started up, but failed to open up a communication channel". Bug #3134, comment #7. MonetDB Common * Fixed a bug in BATantijoin when either side is a singleton BAT. This fixes bug 3139. Bug Fixes * 3117: test sql/test/BugTracker-2011/Tests/history.Bug-2909.sql produces extra profiling output since changeset 817328b74e2f * 3134: Database gets killed by timeout during startup * 3139: Not equal evaluation does not work * 3144: Memleak when connecting to a database (mclient and mapi) * 3145: mclient error when enabling history * 3150: Multiple string equality tests fail to match (Test_JdbcClient)

1 0

[Monetdb-developers] MonetDB on windows(visual studio 2010)
by 李珀瀚 14 Sep '12

14 Sep '12

Hi : I am a new user and I downloaded the newest version ( MonetDB-11.11.7 ).Because my main work is under windows , I want to test and use MonetDB on windows, too.I know that you supplied MonetDB5-SQL-Installer-x86_64-20120813.msi for users to install on windows and I tried it, it worked. But my question is that I want to explore the source code of MonetDB , I download MonetDB-11.11.7.zip<http://dev.monetdb.org/downloads/sources/Latest/MonetDB-11.11.7.zip> and there is a directory named NT,I entered and then enter the directory install64 , in there I found MonetDB5-SQL-Installer.sln and I I think this is the source which I can use to create a visual studio project. So I used Visual Studio 2012 to open it and then compile. The result is that I failed. So my question is :(1) how to trace MonetDB's source code on windows with VS 2010 ,do you have any suggstion? (2)After I installed MonetDB server and client , how could I debug the server(the problem is pretty much the same as the first)? Thank you! Pohan Li

1 1

[Monetdb-developers] some questions
by guangliang peng 12 Sep '12

12 Sep '12

hi, i am a new learner of monetdb.And i have some questions below. 1 when I ran some queries on monetdb,I found that the memory was automatically released by monetdb.And it needs extra time to load data into memory again.I don't know why and if there are some triggers in monetdb? Thank you for helps. 2 how do i see the join path when i run a query? 3 what's source code structure? i want to read the source code .And any advices for better understanding source code. thank, guangliang peng

2 1

[Monetdb-developers] error: not protecting function: no buffer at least 8 bytes long
by Tapomay Dey 29 Aug '12

29 Aug '12

Hi, I have checked out the july branch. I do bootstrap - > configure -> make make throws following error: cc1: warnings being treated as errors inet.c: In function ‘INET_comp_CW’: inet.c:323: error: not protecting function: no buffer at least 8 bytes long inet.c: In function ‘INETbroadcast’: inet.c:423: error: not protecting function: no buffer at least 8 bytes long inet.c: In function ‘INETnetmask’: inet.c:523: error: not protecting function: no buffer at least 8 bytes long inet.c: In function ‘INETnetwork’: inet.c:593: error: not protecting function: no buffer at least 8 bytes long inet.c: In function ‘INETabbrev’: inet.c:660: error: not protecting function: no buffer at least 8 bytes long make[7]: * [libatoms_la-inet.lo] Error 1 HELP!!! I am using the following hack as of now: Is it a good idea to go to the given locations and change the 4 -size arrays to 8-size.? Regards, Tapomay

2 1

[Monetdb-developers] Migration DW to MonetDB
by R＠quel 27 Aug '12

27 Aug '12

Hi, I have a question, i need comparison the execution query times between MonetDB and SQL 2008 in Data Warehouse (applying a different orientations database systems). But i have a problem, i used the AdventureWorkR2 2008 but i don't know how i can migrate the information to MonetDB. To compare the same data. Someone help me? Please. It's very important. Thanks, Raquel -- View this message in context: http://old.nabble.com/Migration-DW-to-MonetDB-tp34348185p34348185.html Sent from the monetdb-developers mailing list archive at Nabble.com.

1 0

Re: [Monetdb-developers] MonetDB: Jul2012 - Only print runtimeTiming information when in ...
by Stefan Manegold 25 Aug '12

25 Aug '12

THANKS!!! ;-) S. -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Reply message ----- From: "Sjoerd Mullender" <sjoerd(a)acm.org> To: <checkin-list(a)monetdb.org> Subject: MonetDB: Jul2012 - Only print runtimeTiming information when in ... Date: Fri, Aug 24, 2012 17:09 Changeset: 0901d759538f for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=0901d759538f Modified Files: monetdb5/mal/mal_runtime.c Branch: Jul2012 Log Message: Only print runtimeTiming information when in the (mdb) debugger. This fixes bugs 3117 and 3145. diffs (12 lines): diff --git a/monetdb5/mal/mal_runtime.c b/monetdb5/mal/mal_runtime.c --- a/monetdb5/mal/mal_runtime.c +++ b/monetdb5/mal/mal_runtime.c @@ -102,7 +102,7 @@ void runtimeTiming(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci, int tid , MT_Lock *lock, RuntimeProfile prof) { str line; - if( cntxt->flags && stk->cmd != 't' && stk->cmd != 'C'){ + if( cntxt->flags && stk->cmd != '\0' && stk->cmd != 't' && stk->cmd != 'C'){ if (lock) MT_lock_set(&*lock, "timing"); mnstr_printf(cntxt->fdout,"= "); /* single column rendering */ _______________________________________________ Checkin-list mailing list Checkin-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/checkin-list

1 0

[Monetdb-developers] MonetDB manager for Python
by Gijs Molenaar 23 Aug '12

23 Aug '12

Hi MonetDB People, We currently want to manage (create, start, release, etc) MonetDB databases from Python. What do you people think if I make a (thin) wrapper around the 'monetdb' command as a python module (monetdb.manage) and commit this to default? greetings, -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

2 2

[Monetdb-developers] Problems compiling MonetDB-11.11.5 on SuSE SLES-11 SP2
by Jukka Santala 23 Aug '12

23 Aug '12

SuSE Linux Enterprise Server 11 SP2, release in March. As already covered on the bug-tracker etc. old bison won't work, so I had to install latest version, 2.6.2 to build. After that: make[5]: Entering directory `/opt/MonetDB/MonetDB-11.11.5/sql/server' /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../.. -I. -I../include -I./../include -I../common -I./../common -I../storage -I./../storage -I../../clients/mapilib -I./../../clients/mapilib -I../../common/options -I./../../common/options -I../../common/stream -I./../../common/stream -I../../gdk -I./../../gdk -DLIBSQLSERVER -g -O2 -c -o libsqlserver_la-sql_scan.lo `test -f 'sql_scan.c' || echo './'`sql_scan.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I. -I../include -I./../include -I../common -I./../common -I../storage -I./../storage -I../../clients/mapilib -I./../../clients/mapilib -I../../common/options -I./../../common/options -I../../common/stream -I./../../common/stream -I../../gdk -I./../../gdk -DLIBSQLSERVER -g -O2 -c sql_scan.c -fPIC -DPIC -o .libs/libsqlserver_la-sql_scan.o In file included from sql_semantic.h:27, from sql_scan.c:29: sql_parser.h:191: error: conflicting types for 'sqlparse' y.tab.h:678: error: previous declaration of 'sqlparse' was here No idea how to properly fix that, but added "#define YYPARSE_PARAM parm" at the front of sql_scan.c, which forces the prototype to retain the parameter. Next problem, HowToStart.rst states: "Then in any directory (preferably a *new, empty* directory and *not* in the ``MonetDB`` top-level directory) give the command:: .../configure [<options>]", which is indeed recommended practice. Unfortunately, nobody has apparently ever done this. If configure and make are ran in a different directory from the source root, "make check" will fail to find monetdb5/modules/mal/mal_init.mal and who knows what else, giving long time of head-scratching. After re-doing configure and make in the prohibited directory, make check will report: . opt_sql_append : Crashed! * (opt_sql_append.test.out.FILTERED) significantly * (opt_sql_append.test.err.FILTERED) significantly ....................................................................................................................................................................................................................................... * (recycle04.test.out.FILTERED) significantly ........................................................................................................................................................................................................................................................................... !ERROR: Testing FAILED SIGNIFICANTLY (2 out of 499 tests failed) 1 out of 499 tests resulted in a crash extras/mal_optimizer_template/opt_sql_append 1 out of 499 tests produced SIGNIFICANTLY different output mal/recycle04 To be frank, that sounds "significantly" like something translated by Google translator to/from some lesser known language. No idea what it's trying to tell me, or how to proceed from there. Can't find any Google search hits on the problems either. Suggestions? -Jukka Santala

3 4

[Monetdb-developers] Announcement: New Jul2012-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 23 Aug '12

23 Aug '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2012-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2012-SP1 bugfix release Java Module * Fixed adaptive cache size used when retrieving results, not to cause divide by zero errors when memory gets short, bug #3119. Client Package * mclient no longer prints the SQLSTATE at the start of each error returned by the SQL-server. MonetDB5 Server * The server now distinguishes between starting and started states, such that monetdbd can wait for it to finish starting. Merovingian * Starting a server now waits for as long as the server needs to possibly recover, bug #3134. In case of a long wait, the monetdbd logfile gives extra information on what the server is doing to recover. * Fixed a crash of monetdbd when local databases were unshared, bug #3135 * Resolved a problem where automatic starting of a database initiated by multiple clients at the same time could cause failed starts. Bug #3107 Bug Fixes * 3075: inconsistent declaration of algebra.markH with 3 input arguments * 3090: crashed if using single identifier for where condition * 3093: sql fail if use scalar subquery with alias * 3107: concurrent connections to the same stopped database yield in multiple mserver5 starts by monetdbd * 3119: MonetConnection$ResultSetResponse throws java.lang.ArithmeticException: divide by zero * 3132: test/BugTracker-2011/func_iter_vs_bulk.Bug-2826 fails * 3134: Database gets killed by timeout during startup * 3135: monetdbd crash while creating & loading database

1 0

Re: [Monetdb-developers] MonetDB: default - Implemented another trivial case for grouping...
by Stefan Manegold 21 Aug '12

21 Aug '12

;-) Dank je wel! --- Niet alleen voor deze, maar met naam ook voor de hele nieuwe group-by code en al die andere nieuwe schone code!!! Stefan On Tue, Aug 21, 2012 at 12:43:42PM +0200, Sjoerd Mullender wrote: > Changeset: 6f001c03cd8d for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6f001c03cd8d > Modified Files: > gdk/gdk_group.c > Branch: default > Log Message: > > Implemented another trivial case for grouping: all equal values. > > > diffs (122 lines): > > diff --git a/gdk/gdk_group.c b/gdk/gdk_group.c > --- a/gdk/gdk_group.c > +++ b/gdk/gdk_group.c > @@ -44,11 +44,14 @@ > * is always created. In other words, the groups argument may not be > * NULL, but the extents and histo arguments may be NULL. > * > - * There are five different implementations of the grouping code. > + * There are six different implementations of the grouping code. > * > * If it can be trivially determined that all groups are singletons, > * we can produce the outputs trivially. > * > + * If all values in b are known to be equal (both sorted and reverse > + * sorted), we produce a single group or copy the input group. > + * > * If the input bats b and g are sorted, or if the subsorted flag is > * set (only used by BATsubsort), we only need to compare consecutive > * values. > @@ -102,6 +105,7 @@ BATgroup_internal(BAT **groups, BAT **ex > > if (b->tkey || BATcount(b) <= 1 || (g && (g->tkey || BATtdense(g)))) { > /* grouping is trivial: 1 element per group */ > + ALGODEBUG fprintf(stderr, "#BATgroup: trivial case: 1 element per group\n"); > if (BATcount(b) == 1 && b->htype == TYPE_oid) > ngrp = * (oid *) Hloc(b, BUNfirst(b)); > else > @@ -132,6 +136,62 @@ BATgroup_internal(BAT **groups, BAT **ex > } > return GDK_SUCCEED; > } > + if (b->tsorted && b->trevsorted) { > + /* all values are equal */ > + if (g == NULL) { > + /* there's only a single group: 0 */ > + ALGODEBUG fprintf(stderr, "#BATgroup: trivial case: single output group\n"); > + ngrp = 0; > + gn = BATconstant(TYPE_oid, &ngrp, BATcount(b)); > + if (gn == NULL) > + goto error; > + BATseqbase(gn, b->hseqbase); > + *groups = gn; > + if (extents) { > + ngrp = gn->hseqbase; > + en = BATconstant(TYPE_void, &ngrp, 1); > + if (en == NULL) > + goto error; > + BATseqbase(BATmirror(en), ngrp); > + *extents = en; > + } > + if (histo) { > + wrd cnt = (wrd) BATcount(b); > + > + hn = BATconstant(TYPE_wrd, &cnt, 1); > + if (hn == NULL) > + goto error; > + *histo = hn; > + } > + return GDK_SUCCEED; > + } > + if ((extents == NULL) == (e == NULL) && > + (histo == NULL) == (h == NULL)) { > + /* inherit given grouping; note that if > + * extents/histo is to be returned, we need > + * e/h available in order to copy them, > + * otherwise we will need to calculate them > + * which we will do using the "normal" case */ > + ALGODEBUG fprintf(stderr, "#BATgroup: trivial case: copy input groups\n"); > + gn = BATcopy(g, g->htype, g->ttype, 0); > + if (gn == NULL) > + goto error; > + *groups = gn; > + if (extents) { > + en = BATcopy(e, e->htype, e->ttype, 0); > + if (en == NULL) > + goto error; > + *extents = en; > + } > + if (histo) { > + hn = BATcopy(h, h->htype, h->ttype, 0); > + if (hn == NULL) > + goto error; > + *histo = hn; > + } > + return GDK_SUCCEED; > + } > + } > assert(g == NULL || !BATtdense(g)); /* i.e. g->ttype == TYPE_oid */ > bi = bat_iterator(b); > cmp = BATatoms[b->ttype].atomCmp; > @@ -167,6 +227,7 @@ BATgroup_internal(BAT **groups, BAT **ex > (g == NULL || g->tsorted || g->trevsorted)) || > subsorted) { > /* we only need to compare each entry with the previous */ > + ALGODEBUG fprintf(stderr, "#BATgroup: compare consecutive values\n"); > if (grps) > prev = *grps++; > pv = BUNtail(bi, BUNfirst(b)); > @@ -220,6 +281,7 @@ BATgroup_internal(BAT **groups, BAT **ex > /* for each value, we need to scan all previous equal > * values (a consecutive, possibly empty, range) to > * see if we can find one in the same old group */ > + ALGODEBUG fprintf(stderr, "#BATgroup: subscan old groups\n"); > pv = BUNtail(bi, BUNfirst(b)); > ngrps[0] = ngrp; > if (extents) > @@ -284,6 +346,7 @@ BATgroup_internal(BAT **groups, BAT **ex > } > } else if (b->T->hash) { > /* we already have a hash table on b */ > + ALGODEBUG fprintf(stderr, "#BATgroup: use existing hash table\n"); > hs = b->T->hash; > for (r = BUNfirst(b), p = r, q = r + BATcount(b); p < q; p++) { > v = BUNtail(bi, p); > @@ -343,6 +406,7 @@ BATgroup_internal(BAT **groups, BAT **ex > * build an incomplete hash table on the fly--also see > * BATassertHeadProps and BATderiveHeadProps for > * similar code */ > + ALGODEBUG fprintf(stderr, "#BATgroup: create partial hash table\n"); > nme = BBP_physical(b->batCacheid); > nmelen = strlen(nme); > if ((hp = GDKzalloc(sizeof(Heap))) == NULL || > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Re: [Monetdb-developers] MonetDB: default - Hack to circumvent SQL implementation issue
by Stefan Manegold 16 Aug '12

16 Aug '12

If you consider this a bug, please report ;-) Thanks! Stefan On Thu, Aug 16, 2012 at 10:46:53AM +0200, Martin Kersten wrote: > Changeset: 9e78eccb3fb3 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=9e78eccb3fb3 > Modified Files: > sql/backends/monet5/datacell/Tests/dctemper00.sql > sql/backends/monet5/datacell/emitter.c > sql/backends/monet5/datacell/receptor.c > Branch: default > Log Message: > > Hack to circumvent SQL implementation issue > The SQL procedural code can not properly handle the > case where an attribute of a select is also part of > the target table. > > > diffs (52 lines): > > diff --git a/sql/backends/monet5/datacell/Tests/dctemper00.sql b/sql/backends/monet5/datacell/Tests/dctemper00.sql > --- a/sql/backends/monet5/datacell/Tests/dctemper00.sql > +++ b/sql/backends/monet5/datacell/Tests/dctemper00.sql > @@ -12,7 +12,7 @@ create table datacell.temperature( > tmp decimal(4,2) > ); > > -create table datacell.warnings (msg string, ts time with time zone, location string); > +create table datacell.warnings (msg string, ts time with time zone, loc string); > create table datacell.templog( ts timestamp with time zone, cnt integer); > > call datacell.receptor('datacell.temperature','localhost',50550); > diff --git a/sql/backends/monet5/datacell/emitter.c b/sql/backends/monet5/datacell/emitter.c > --- a/sql/backends/monet5/datacell/emitter.c > +++ b/sql/backends/monet5/datacell/emitter.c > @@ -35,7 +35,7 @@ > #include "dcsocket.h" > #include "stream_socket.h" > > -#define _DEBUG_EMITTER_ > +/* #define _DEBUG_EMITTER_ */ > > static Emitter emAnchor = NULL; > > @@ -105,7 +105,6 @@ EMemitterStartInternal(int *ret, str *tb > em->delay = delay; > em->lastseen = *timestamp_nil; > /* > - * @- > * All tables are prepended with a default tick bat. > * It becomes the synchronization handle. > */ > diff --git a/sql/backends/monet5/datacell/receptor.c b/sql/backends/monet5/datacell/receptor.c > --- a/sql/backends/monet5/datacell/receptor.c > +++ b/sql/backends/monet5/datacell/receptor.c > @@ -41,7 +41,7 @@ > #include "stream_socket.h" > #include "mal_builder.h" > > -#define _DEBUG_RECEPTOR_ > +/* #define _DEBUG_RECEPTOR_ */ > > /* default settings */ > #define RCHOST "localhost" > @@ -495,7 +495,6 @@ parse: > #endif > } > /* > - * @- > * A short cut is to generate the events based upon the interpretation > * of a scenario file. Much like the one used in the sensor. > * It is processed multiple times. The header is the delay imposed. > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] New Oct2012 stable branch created
by Sjoerd Mullender 13 Aug '12

13 Aug '12

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Oct2012. So in each of your stable checkout directories you can do hg pull hg update -rOct2012 to update to this new branch. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] MonetDB: default - Removed histogram module: it's not even compi...
by Martin Kersten 09 Aug '12

09 Aug '12

Correct. It was an utility for the sample based bulk-loader, which never materialized. On 8/9/12 2:56 PM, Sjoerd Mullender wrote: > Changeset: 22072dd26a9c for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=22072dd26a9c > Removed Files: > monetdb5/modules/mal/histogram.c > monetdb5/modules/mal/histogram.h > monetdb5/modules/mal/histogram.mal > Modified Files: > monetdb5/modules/mal/Makefile.ag > Branch: default > Log Message: > > Removed histogram module: it's not even compiled, let alone used. > > > diffs (truncated from 606 to 300 lines): > > diff --git a/monetdb5/modules/mal/Makefile.ag b/monetdb5/modules/mal/Makefile.ag > --- a/monetdb5/modules/mal/Makefile.ag > +++ b/monetdb5/modules/mal/Makefile.ag > @@ -39,7 +39,6 @@ lib_mal = { > factories.c factories.h \ > groupby.c groupby.h \ > groups.c groups.h \ > - histogram.h \ > inspect.c inspect.h \ > iterator.c iterator.h \ > joinpath.c joinpath.h \ > @@ -83,6 +82,6 @@ headers_mal = { > calc.mal batcalc.mal > } > > -EXTRA_DIST = attach.mal batExtensions.mal iterator.mal constraints.mal groupby.mal histogram.mal mal_init.mal manual.mal mkey.mal pcre.mal profiler.mal recycle.mal remote.mal sabaoth.mal trader.mal transaction.mal txtsim.mal tablet.mal tablet.h sample.mal mal_mapi.mal mat.mal tokenizer.mal pqueue.mal calc.mal batcalc.mal > +EXTRA_DIST = attach.mal batExtensions.mal iterator.mal constraints.mal groupby.mal mal_init.mal manual.mal mkey.mal pcre.mal profiler.mal recycle.mal remote.mal sabaoth.mal trader.mal transaction.mal txtsim.mal tablet.mal tablet.h sample.mal mal_mapi.mal mat.mal tokenizer.mal pqueue.mal calc.mal batcalc.mal > > EXTRA_DIST_DIR = Tests > diff --git a/monetdb5/modules/mal/histogram.c b/monetdb5/modules/mal/histogram.c > deleted file mode 100644 > --- a/monetdb5/modules/mal/histogram.c > +++ /dev/null > @@ -1,470 +0,0 @@ > -/* > - * The contents of this file are subject to the MonetDB Public License > - * Version 1.1 (the "License"); you may not use this file except in > - * compliance with the License. You may obtain a copy of the License at > - * http://www.monetdb.org/Legal/MonetDBLicense > - * > - * Software distributed under the License is distributed on an "AS IS" > - * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > - * License for the specific language governing rights and limitations > - * under the License. > - * > - * The Original Code is the MonetDB Database System. > - * > - * The Initial Developer of the Original Code is CWI. > - * Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > - * Copyright August 2008-2012 MonetDB B.V. > - * All Rights Reserved. > - */ > - > -/* > - * @f histogram > - * @a Martin Kersten, Stratos Idreos > - * @v 1 > - * @+ Histogram > - * This support module is geared at handling small equi-bin histograms. > - */ > -/* > - * @- > - */ > -#include "monetdb_config.h" > -#include "histogram.h" > - > -Histogram > -HSTnew(int bins, ValPtr minval, ValPtr maxval) > -{ > - Histogram h; > - int i=0; > - > - assert( minval && maxval); > - assert( minval->vtype == maxval->vtype); > - assert( bins > 0); > - > - h = (Histogram) GDKzalloc(sizeof(*h) + sizeof(dbl) * 2 * bins); > - h->low = *minval; > - h->hgh = h->inc = *maxval; > - h->bins = bins; > - h->total = 0; > - for ( i = 0; i < 2*h->bins; i++) > - h->cnt[i]=0; > - > - switch( minval->vtype){ > - case TYPE_bte: > - h->inc.val.btval = (maxval->val.btval - minval->val.btval)/ bins + 1; > - assert(h->inc.val.btval); > - break; > - case TYPE_sht: > - h->inc.val.shval = (maxval->val.shval - minval->val.shval)/ bins + 1; > - assert(h->inc.val.shval); > - break; > - case TYPE_int: > - h->inc.val.ival = (maxval->val.ival - minval->val.ival)/ bins + 1; > - assert(h->inc.val.ival); > - break; > - case TYPE_lng: > - h->inc.val.lval = (maxval->val.lval - minval->val.lval)/ bins + 1; > - assert(h->inc.val.lval); > - break; > - case TYPE_dbl: > - h->inc.val.dval = (maxval->val.dval - minval->val.dval)/ bins + 1; > - assert(h->inc.val.dval); > - break; > - case TYPE_flt: > - h->inc.val.fval = (maxval->val.fval - minval->val.fval)/ bins + 1; > - assert(h->inc.val.fval); > - } > - return h; > -} > - > -/*get the bin we would increment if we would add this value*/ > -int > -HSTgetIndex(Histogram h, ValPtr val){ > - int index =0; > - switch( val->vtype){ > - case TYPE_bte: > - if (val->val.btval > h->hgh.val.btval) > - index= (h->hgh.val.btval - h->low.val.btval) / h->inc.val.btval; > - else > - index= (val->val.btval - h->low.val.btval) / h->inc.val.btval; > - break; > - case TYPE_sht: > - if (val->val.shval > h->hgh.val.shval) > - index= (h->hgh.val.shval - h->low.val.shval) / h->inc.val.shval; > - else > - index= (val->val.shval - h->low.val.shval) / h->inc.val.shval; > - break; > - case TYPE_int: > - if (val->val.ival > h->hgh.val.ival) > - index= (h->hgh.val.ival - h->low.val.ival) / h->inc.val.ival; > - else > - index= (val->val.ival - h->low.val.ival) / h->inc.val.ival; > - break; > - case TYPE_lng: > - if (val->val.lval > h->hgh.val.lval) > - index= (int) ((h->hgh.val.lval - h->low.val.lval) / h->inc.val.lval); > - else > - index= (int) ((val->val.lval - h->low.val.lval) / h->inc.val.lval); > - break; > - case TYPE_dbl: > - if (val->val.dval > h->hgh.val.dval) > - index= (int) ((h->hgh.val.dval - h->low.val.dval) / h->inc.val.dval); > - else > - index= (int) ((val->val.dval - h->low.val.dval) / h->inc.val.dval); > - break; > - case TYPE_flt: > - if (val->val.fval > h->hgh.val.fval) > - index= (int) ((h->hgh.val.fval - h->low.val.fval) / h->inc.val.fval); > - else > - index= (int) ((val->val.fval - h->low.val.fval) / h->inc.val.fval); > - break; > - } > - return index; > -} > - > -int > -HSTincrement(Histogram h, ValPtr val){ > - int index = HSTgetIndex(h,val); > - h->cnt[index]++; > - h->total++; > - h->cnt[h->bins+index]=h->cnt[index]/h->total; > - return index; > -} > - > -void > -HSTdecrement(Histogram h, ValPtr val){ > - int index = HSTgetIndex(h,val); > - h->cnt[index]--; > - h->total--; > - h->cnt[h->bins+index]=h->cnt[index]/h->total; > -} > - > -dbl > -HSTeuclidian(Histogram h1, Histogram h2){ > - dbl sum= 0; > - int i; > - assert(h1->bins == h2->bins); > - for ( i = 0; i < h1->bins; i++) > - sum += (h1->cnt[i] - h2->cnt[i]) * (h1->cnt[i] - h2->cnt[i]); > - return sqrt((double) sum); > -} > - > - > -/* if 1 then adding this value to h2 increases the distance between h1-h2*/ > -int > -HSTeuclidianWhatIf(Histogram h1, Histogram h2, ValPtr val){ > - dbl sum= 0, sumWhatIf=0, cur; > - dbl distance, distanceWhatIf; > - int i, affectedBin; > - assert(h1->bins == h2->bins); > - > - affectedBin = HSTgetIndex(h2,val); > - > - for ( i = 0; i < h1->bins; i++){ > - cur = (h1->cnt[i] - h2->cnt[i]) * (h1->cnt[i] - h2->cnt[i]); > - sum += cur; > - if (i==affectedBin) > - sumWhatIf += (h1->cnt[i] - (h2->cnt[i]+1)) * (h1->cnt[i] - (h2->cnt[i]+1)); > - else > - sumWhatIf += cur; > - } > - > - distance = sqrt((double) sum); > - distanceWhatIf = sqrt((double) sumWhatIf); > - if (distanceWhatIf > distance) return 1; > - return 0; > -} > - > -/* if 1 then moving this value to h2 increases the distance between h1-h2*/ > -int > -HSTeuclidianWhatIfMove(Histogram h1, Histogram h2, ValPtr val){ > - dbl sum= 0, sumWhatIf=0, cur; > - dbl distance, distanceWhatIf; > - int i, affectedBin; > - assert(h1->bins == h2->bins); > - > - affectedBin = HSTgetIndex(h1,val); > - > - for ( i = 0; i < h1->bins; i++){ > - cur = (h1->cnt[i] - h2->cnt[i]) * (h1->cnt[i] - h2->cnt[i]); > - sum += cur; > - if (i==affectedBin) > - sumWhatIf += ((h1->cnt[i]-1) - (h2->cnt[i]+1)) * ((h1->cnt[i]-1) - (h2->cnt[i]+1)); > - else > - sumWhatIf += cur; > - } > - > - distance = sqrt((double) sum); > - distanceWhatIf = sqrt((double) sumWhatIf); > - if (distanceWhatIf > distance) return 1; > - return 0; > -} > - > -dbl > -HSTeuclidianNorm(Histogram h1, Histogram h2){ > - dbl sum= 0; > - int i; > - assert(h1->bins == h2->bins); > - for ( i = h1->bins; i < 2*h1->bins; i++) > - sum += (h1->cnt[i] - h2->cnt[i]) * (h1->cnt[i] - h2->cnt[i]); > - return sqrt((double) sum); > -} > - > -dbl > -HSTeuclidianNormWhatIf(Histogram h1, Histogram h2, ValPtr val, dbl *whatIfDistance){ > - dbl sum= 0, sumWhatIf=0, cur; > - dbl distance, distanceWhatIf; > - int i, affectedBin; > - assert(h1->bins == h2->bins); > - > - affectedBin = HSTgetIndex(h2,val)+h2->bins; > - > - for ( i = h1->bins; i < 2*h1->bins; i++){ > - cur = (h1->cnt[i] - h2->cnt[i]) * (h1->cnt[i] - h2->cnt[i]); > - sum += cur; > - if (i==affectedBin) > - sumWhatIf += (h1->cnt[i] - (((h2->cnt[i-h1->bins]+1))/(h2->total+1))) * (h1->cnt[i] - (((h2->cnt[i-h1->bins]+1))/(h2->total+1))); > - else > - sumWhatIf += cur; > - } > - > - distance = sqrt((double) sum); > - distanceWhatIf = sqrt((double) sumWhatIf); > - *whatIfDistance = distanceWhatIf; > - if (distanceWhatIf > distance) return (distanceWhatIf - distance); > - return 0; > -} > - > -/* if 1 then moving this value to h2 increases the distance between h1-h2*/ > -int > -HSTeuclidianNormWhatIfMove(Histogram h1, Histogram h2, ValPtr val, dbl *whatIfDistance){ > - dbl sum= 0, sumWhatIf=0, cur; > - dbl distance, distanceWhatIf; > - int i, affectedBin; > - assert(h1->bins == h2->bins); > - > - affectedBin = HSTgetIndex(h1,val)+h2->bins; > - > - for ( i = h1->bins; i < 2*h1->bins; i++){ > - cur = (h1->cnt[i] - h2->cnt[i]) * (h1->cnt[i] - h2->cnt[i]); > - sum += cur; > - if (i==affectedBin) > - sumWhatIf += ((((h1->cnt[i-h1->bins]-1))/(h1->total-1)) - (((h2->cnt[i-h1->bins]+1))/(h2->total+1))) * ((((h1->cnt[i-h1->bins]-1))/(h1->total-1)) - (((h2->cnt[i-h1->bins]+1))/(h2->total+1))); > - else > - sumWhatIf += cur; > - } > - > - distance = sqrt((double) sum); > - distanceWhatIf = sqrt((double) sumWhatIf); > - *whatIfDistance = distanceWhatIf; > - if (distanceWhatIf > distance) return 1; > - return 0; > -} > - > - > -dbl > -HSTcityblock(Histogram h1, Histogram h2){ > - dbl sum=0; > - int i; > - assert(h1->bins == h2->bins); > - for ( i = 0; i < h1->bins; i++) > - sum += ABS(h1->cnt[i] - h2->cnt[i]); > - return (dbl) sum; > -} > - > -dbl > -HSTchebyshev(Histogram h1, Histogram h2){ > - dbl max; > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

[Monetdb-developers] undefined symbol: BATaccess
by Stefan de Konink 04 Aug '12

04 Aug '12

Was this already known? # MonetDB 5 server v11.12.0 # This is an unreleased version # Serving database 'kvk-2012-augustus', using 2 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 3.860 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information #WARNING: LoaderException:loadLibrary:failed to load library: /opt/monetdb-augustus/lib/monetdb5/lib_lsst.so: undefined symbol: BATaccess #WARNING: LoaderException:loadLibrary:failed to load library: /opt/monetdb-augustus/lib/monetdb5/lib_lsst.so: undefined symbol: BATaccess # Listening for connection requests on mapi:monetdb://127.0.0.1:60010/ # MonetDB/GIS module loaded # MonetDB/JAQL module loaded # MonetDB/SQL module loaded

3 5

Re: [Monetdb-developers] MonetDB: default - Added missing stuff
by Stefan Manegold 26 Jul '12

26 Jul '12

Just to double check, since your log message does not appear to match your changes: message: "*Added* missing stuff" changes: removed references to "trace" optimizer Where the changes indeed intended as done, "despite" the log message Stefan On Thu, Jul 26, 2012 at 10:39:15AM +0200, Martin Kersten wrote: > Changeset: 7ad5e675d7e4 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=7ad5e675d7e4 > Modified Files: > monetdb5/optimizer/opt_support.c > monetdb5/optimizer/opt_wrapper.c > Branch: default > Log Message: > > Added missing stuff > > > diffs (30 lines): > > diff --git a/monetdb5/optimizer/opt_support.c b/monetdb5/optimizer/opt_support.c > --- a/monetdb5/optimizer/opt_support.c > +++ b/monetdb5/optimizer/opt_support.c > @@ -181,7 +181,6 @@ struct OPTcatalog { > {"selcrack", 0, 0, 0, DEBUG_OPT_SELCRACK}, > {"sidcrack", 0, 0, 0, DEBUG_OPT_SIDCRACK}, > {"strengthreduction", 0, 0, 0, DEBUG_OPT_STRENGTHREDUCTION}, > -{"trace", 0, 0, 0, DEBUG_OPT_TRACE}, > {"centipede", 0, 0, 0, DEBUG_OPT_CENTIPEDE}, > { 0, 0, 0, 0, 0} > }; > diff --git a/monetdb5/optimizer/opt_wrapper.c b/monetdb5/optimizer/opt_wrapper.c > --- a/monetdb5/optimizer/opt_wrapper.c > +++ b/monetdb5/optimizer/opt_wrapper.c > @@ -73,7 +73,6 @@ All Rights Reserved. > #include "opt_reorder.h" > #include "opt_statistics.h" > #include "opt_strengthReduction.h" > -#include "opt_trace.h" > > struct{ > str nme; > @@ -113,7 +112,6 @@ struct{ > {"remoteQueries", &OPTremoteQueriesImplementation}, > {"reorder", &OPTreorderImplementation}, > {"strengthReduction", &OPTstrengthReductionImplementation}, > - {"trace", &OPTtraceImplementation}, > {0,0} > }; > opt_export str OPTwrapper(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr p); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 2

Re: [Monetdb-developers] MonetDB: xid - BATderiveHeadProps & COLsettrivprop: do not set/d...
by Stefan Manegold 25 Jul '12

25 Jul '12

yep. thanks! testing just reminded me as well ;-) fix on its way. Sorry for premature checking! Stefan ----- Original Message ----- > On 2012-07-25 09:24, Stefan Manegold wrote: > > Changeset: 4da710a71d6f for MonetDB > > URL: > > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=4da710a71d6f > > Modified Files: > > gdk/gdk.h > > gdk/gdk_bat.c > > Branch: xid > > Log Message: > > > > BATderiveHeadProps & COLsettrivprop: do not set/derive properties > > for xid-compressed BATs > > their content is known and designed to not match their properties > > see also changesets 805110fd0096 & b33eba6cfb09 > > > > > > diffs (36 lines): > > > > diff --git a/gdk/gdk.h b/gdk/gdk.h > > --- a/gdk/gdk.h > > +++ b/gdk/gdk.h > > @@ -1699,8 +1699,10 @@ gdk_export void GDKqsort_rev(void *h, vo > > #define BATtkey(b) (b->tkey != FALSE || BATtdense(b)) > > > > /* set some properties that are trivial to deduce */ > > +/* but not for xid-compressed BATs */ > > #define COLsettrivprop(b, col) \ > > do { \ > > + if (b->htype == TYPE_oid && b->H->heap.xidcompressed) { \ > > Shouldn't that be: > if (b->htype != TYPE_oid || !b->H->heap.xidcompressed) { > ? > You want the properties to be set if the heap is *not* compressed. > > > if ((col)->type == TYPE_void) { \ > > if ((col)->seq == oid_nil) { \ > > if (!(col)->nil && (b)->batCount >= 1) { \ > > @@ -1772,6 +1774,7 @@ gdk_export void GDKqsort_rev(void *h, vo > > (b)->batDirtydesc = 1; \ > > } \ > > } \ > > + } \ > > } while (0) > > #define BATsettrivprop(b) \ > > do { \ > > diff --git a/gdk/gdk_bat.c b/gdk/gdk_bat.c > > --- a/gdk/gdk_bat.c > > +++ b/gdk/gdk_bat.c > > @@ -3096,6 +3096,10 @@ BATderiveHeadProps(BAT *b, int expensive > > assert(b != NULL); > > if (b == NULL) > > return; > > + if (b->htype == TYPE_oid && b->H->heap.xidcompressed) { > > + /* avoid property deriving on xid-compressed BATs */ > > + return; > > + } > > assert((b->hkey & BOUND2BTRUE) == 0); > > COLsettrivprop(b, b->H); > > cmpf = BATatoms[b->htype].atomCmp; > > _______________________________________________ > > Checkin-list mailing list > > Checkin-list(a)monetdb.org > > http://mail.monetdb.org/mailman/listinfo/checkin-list > > > > > -- > Sjoerd Mullender <sjoerd(a)monetdb.org> > MonetDB: Column store pioneers <http://www.monetdb.org/> > > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: [Monetdb-developers] MonetDB: xid - BATassertHeadProps: avoid property checking on xi...
by Sjoerd Mullender 24 Jul '12

24 Jul '12

On 2012-07-24 22:13, Stefan Manegold wrote: > Changeset: 805110fd0096 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=805110fd0096 > Modified Files: > gdk/gdk_bat.c > Branch: xid > Log Message: > > BATassertHeadProps: avoid property checking on xid-compressed BATs > their content is not plain valid data, > and their properties are those of the uncompressed BAT > > > diffs (13 lines): > > diff --git a/gdk/gdk_bat.c b/gdk/gdk_bat.c > --- a/gdk/gdk_bat.c > +++ b/gdk/gdk_bat.c > @@ -2892,6 +2892,9 @@ BATassertHeadProps(BAT *b) > return; > } > > + > + /* no property checking on xid-compressed BATs */ > + if (b->htype != TYPE_oid || !b->H->heap.xidcompressed) > PROPDEBUG { /* only do a scan if property checking is requested */ > if (b->hsorted || b->hrevsorted || !b->hkey) { > /* if sorted (either way), or we don't have to I really don't like this layout. Please consider using return after the if statement (with complementary test). I would also find that clearer. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] MonetDB: default - Cleanup of logger code to split head and tail...
by Niels Nes 18 Jul '12

18 Jul '12

On Wed, Jul 18, 2012 at 12:55:31PM +0200, Niels Nes wrote: > Changeset: 982fdf69e13b for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=982fdf69e13b > Modified Files: > gdk/gdk_logger.c > gdk/gdk_logger.h > sql/server/rel_optimizer.c > sql/storage/bat/bat_logger.c > sql/storage/restrict/restrict_logger.c > Branch: default > Log Message: > > Cleanup of logger code to split head and tail bat into 2 void headed bats This checkin breaks the current on disk format, ie if your using the defaul branch a reload is needed. Before the (september) release we need to add updating functions for atleast SQL. Niels > > > diffs (truncated from 723 to 300 lines): > > diff --git a/gdk/gdk_logger.c b/gdk/gdk_logger.c > --- a/gdk/gdk_logger.c > +++ b/gdk/gdk_logger.c > @@ -73,7 +73,10 @@ > #include "gdk_logger.h" > #include <string.h> > > - > +static BUN BUNfndT( BAT *b, ptr v) > +{ > + return BUNfnd(BATmirror(b), v); > +} > /* > * @- > * The log record encoding is geared at reduced storage space, but > @@ -218,12 +221,11 @@ la_bat_clear(logger *lg, logaction *la) > BAT *b; > > /* do we need to skip these old updates */ > - if (BATcount(lg->snapshots)) { > - BUN p = BUNfnd(lg->snapshots, &bid); > + if (BATcount(lg->snapshots_bid)) { > + BUN p = BUNfndT(lg->snapshots_bid, &bid); > > if (p != BUN_NONE) { > - BATiter i = bat_iterator(lg->snapshots); > - int tid = *(int *) BUNtloc(i, p); > + int tid = *(int *) Tloc(lg->snapshots_tid, p); > > if (lg->tid <= tid) > return; > @@ -244,16 +246,18 @@ static int > log_read_seq(logger *lg, logformat *l) > { > int seq = l->nr; > - lng id; > + lng val; > + BUN p; > > - if (!mnstr_readLng(lg->log, &id)) > + if (!mnstr_readLng(lg->log, &val)) > return LOG_ERR; > > - if (BUNfnd(lg->seqs, &seq) != BUN_NONE) { > - BUNdelHead(lg->seqs, &seq, FALSE); > + if ((p = BUNfndT(lg->seqs_id, &seq)) != BUN_NONE) { > + BUNdelete(lg->seqs_id, p, FALSE); > + BUNdelete(lg->seqs_val, p, FALSE); > } > - BUNins(lg->seqs, &seq, &id, FALSE); > - > + BUNappend(lg->seqs_id, &seq, FALSE); > + BUNappend(lg->seqs_val, &val, FALSE); > return LOG_OK; > } > > @@ -404,12 +408,11 @@ la_bat_updates(logger *lg, logaction *la > return; /* ignore bats no longer in the catalog */ > > /* do we need to skip these old updates */ > - if (BATcount(lg->snapshots)) { > - BUN p = BUNfnd(lg->snapshots, &bid); > + if (BATcount(lg->snapshots_bid)) { > + BUN p = BUNfndT(lg->snapshots_bid, &bid); > > if (p != BUN_NONE) { > - BATiter i = bat_iterator(lg->snapshots); > - int tid = *(int *) BUNtloc(i, p); > + int tid = *(int *) Tloc(lg->snapshots_tid, p); > > if (lg->tid <= tid) > return; > @@ -477,10 +480,14 @@ la_bat_destroy(logger *lg, logaction *la > log_bid bid = logger_find_bat(lg, la->name); > > if (bid) { > + BUN p; > + > logger_del_bat(lg, bid); > - if (BUNfnd(lg->snapshots, &bid) != BUN_NONE) { > - BUNdelHead(lg->snapshots, &bid, FALSE); > - BUNins(lg->snapshots, &bid, &lg->tid, FALSE); > + if ((p = BUNfndT(lg->snapshots_bid, &bid)) != BUN_NONE) { > + BUNdelete(lg->snapshots_bid, p, FALSE); > + BUNdelete(lg->snapshots_tid, p, FALSE); > + BUNappend(lg->snapshots_bid, &bid, FALSE); > + BUNappend(lg->snapshots_tid, &lg->tid, FALSE); > } > } > } > @@ -564,15 +571,19 @@ la_bat_use(logger *lg, logaction *la) > { > log_bid bid = la->nr; > BAT *b = BATdescriptor(bid); > + BUN p; > > if (!b) { > GDKerror("logger: could not use bat (%d) for %s\n", (int) bid, la->name); > return; > } > logger_add_bat(lg, b, la->name); > - if (BUNfnd(lg->snapshots, &b->batCacheid) != BUN_NONE) > - BUNdelHead(lg->snapshots, &b->batCacheid, FALSE); > - BUNins(lg->snapshots, &b->batCacheid, &lg->tid, FALSE); > + if ((p = BUNfndT(lg->snapshots_bid, &b->batCacheid)) != BUN_NONE) { > + BUNdelete(lg->snapshots_bid, p, FALSE); > + BUNdelete(lg->snapshots_tid, p, FALSE); > + } > + BUNappend(lg->snapshots_bid, &b->batCacheid, FALSE); > + BUNappend(lg->snapshots_tid, &lg->tid, FALSE); > logbat_destroy(b); > } > > @@ -875,17 +886,23 @@ static int > logger_commit(logger *lg) > { > int id = LOG_SID; > + BUN p; > > if (lg->debug & 1) > fprintf(stderr, "logger_commit\n"); > > - BUNdelHead(lg->seqs, &id, FALSE); > - BUNins(lg->seqs, &id, &lg->id, FALSE); > + p = BUNfndT(lg->seqs_id, &id); > + BUNdelete(lg->seqs_id, p, FALSE); > + BUNdelete(lg->seqs_val, p, FALSE); > + BUNappend(lg->seqs_id, &id, FALSE); > + BUNappend(lg->seqs_val, &lg->id, FALSE); > > /* cleanup old snapshots */ > - if (BATcount(lg->snapshots)) { > - BATclear(lg->snapshots, FALSE); > - BATcommit(lg->snapshots); > + if (BATcount(lg->snapshots_bid)) { > + BATclear(lg->snapshots_bid, FALSE); > + BATclear(lg->snapshots_tid, FALSE); > + BATcommit(lg->snapshots_bid); > + BATcommit(lg->snapshots_tid); > } > return bm_commit(lg); > } > @@ -919,34 +936,34 @@ check_version(logger *lg, FILE *fp) > } > > static int > -bm_subcommit(BAT *list, BAT *catalog, BAT *extra, int debug) > +bm_subcommit(BAT *list_bid, BAT *list_nme, BAT *catalog_bid, BAT *catalog_nme, BAT *extra, int debug) > { > BUN p, q; > - BUN nn = 2 + (list->batFirst > list->batDeleted ? list->batFirst - list->batDeleted : 0) + BATcount(list) + (extra ? BATcount(extra) : 0); > + BUN nn = 3 + (list_bid->batFirst > list_bid->batDeleted ? list_bid->batFirst - list_bid->batDeleted : 0) + BATcount(list_bid) + (extra ? BATcount(extra) : 0); > bat *n = (bat*)GDKmalloc(sizeof(bat) * nn); > int i = 0; > - BATiter iter = bat_iterator(list); > + BATiter iter = (list_nme)?bat_iterator(list_nme):bat_iterator(list_bid); > int res; > > n[i++] = 0; /* n[0] is not used */ > > /* first loop over deleted then over current and new */ > - for (p = list->batDeleted; p < list->batFirst; p++) { > - bat col = *(log_bid *) BUNhead(iter, p); > + for (p = list_bid->batDeleted; p < list_bid->batFirst; p++) { > + bat col = *(log_bid *) Tloc(list_bid, p); > > if (debug & 1) > fprintf(stderr, "commit deleted %s (%d) %s\n", > BBPname(col), col, > - (list == catalog) ? BUNtail(iter, p) : "snapshot"); > + (list_bid == catalog_bid) ? BUNtail(iter, p) : "snapshot"); > n[i++] = ABS(col); > } > - BATloop(list, p, q) { > - bat col = *(log_bid *) BUNhead(iter, p); > + BATloop(list_bid, p, q) { > + bat col = *(log_bid *) Tloc(list_bid, p); > > if (debug & 1) > fprintf(stderr, "commit new %s (%d) %s\n", > BBPname(col), col, > - (list == catalog) ? BUNtail(iter, p) : "snapshot"); > + (list_bid == catalog_bid) ? BUNtail(iter, p) : "snapshot"); > n[i++] = ABS(col); > } > if (extra) { > @@ -957,14 +974,16 @@ bm_subcommit(BAT *list, BAT *catalog, BA > if (debug & 1) > fprintf(stderr, "commit extra %s %s\n", > name, > - (list == catalog) ? BUNtail(iter, p) : "snapshot"); > + (list_bid == catalog_bid) ? BUNtail(iter, p) : "snapshot"); > n[i++] = ABS(BBPindex(name)); > } > } > /* now commit catalog, so it's also up to date on disk */ > - n[i++] = ABS(catalog->batCacheid); > + n[i++] = ABS(catalog_bid->batCacheid); > + n[i++] = ABS(catalog_nme->batCacheid); > assert((BUN) i <= nn); > - BATcommit(catalog); > + BATcommit(catalog_bid); > + BATcommit(catalog_nme); > res = TMsubcommit_list(n, i); > GDKfree(n); > return res; > @@ -992,8 +1011,8 @@ logger_new(int debug, char *fn, char *lo > FILE *fp; > char filename[BUFSIZ]; > char bak[BUFSIZ]; > - log_bid seqs = 0; > - bat catalog; > + log_bid seqs_id = 0; > + bat catalog_bid, catalog_nme; > > if (lg == NULL) > return NULL; > @@ -1022,9 +1041,12 @@ logger_new(int debug, char *fn, char *lo > lg->postfuncp = postfuncp; > lg->log = NULL; > lg->end = 0; > - lg->catalog = NULL; > - lg->snapshots = NULL; > - lg->seqs = NULL; > + lg->catalog_bid = NULL; > + lg->catalog_nme = NULL; > + lg->snapshots_bid = NULL; > + lg->snapshots_tid = NULL; > + lg->seqs_id = NULL; > + lg->seqs_val = NULL; > > snprintf(filename, BUFSIZ, "%s%s", lg->dir, LOGFILE); > snprintf(bak, BUFSIZ, "%s.bak", filename); > @@ -1040,10 +1062,10 @@ logger_new(int debug, char *fn, char *lo > } > fp = fopen(filename, "r"); > > - snprintf(bak, BUFSIZ, "%s_catalog", fn); > - catalog = BBPindex(bak); > + snprintf(bak, BUFSIZ, "%s_catalog_bid", fn); > + catalog_bid = BBPindex(bak); > > - if (catalog == 0) { > + if (catalog_bid == 0) { > log_bid bid = 0; > > /* catalog does not exist, so the log file also shouldn't exist */ > @@ -1055,16 +1077,24 @@ logger_new(int debug, char *fn, char *lo > goto error; > } > > - lg->catalog = logbat_new(TYPE_int, TYPE_str, BATSIZE); > + lg->catalog_bid = logbat_new(TYPE_void, TYPE_int, BATSIZE); > + lg->catalog_nme = logbat_new(TYPE_void, TYPE_str, BATSIZE); > if (debug) > fprintf(stderr, "create %s catalog\n", fn); > > - bid = lg->catalog->batCacheid; > + /* Make persistent */ > + bid = lg->catalog_bid->batCacheid; > + BBPincref(bid, TRUE); > + BATmode(lg->catalog_bid, PERSISTENT); > + snprintf(bak, BUFSIZ, "%s_catalog_bid", fn); > + BBPrename(lg->catalog_bid->batCacheid, bak); > > /* Make persistent */ > + bid = lg->catalog_nme->batCacheid; > BBPincref(bid, TRUE); > - BATmode(lg->catalog, PERSISTENT); > - BBPrename(lg->catalog->batCacheid, bak); > + BATmode(lg->catalog_nme, PERSISTENT); > + snprintf(bak, BUFSIZ, "%s_catalog_nme", fn); > + BBPrename(lg->catalog_nme->batCacheid, bak); > > if (!GDKcreatedir(filename)) { > logger_fatal("logger_new: cannot create directory for log file %s\n", > @@ -1082,7 +1112,7 @@ logger_new(int debug, char *fn, char *lo > fclose(fp); > fp = NULL; > > - if (bm_subcommit(lg->catalog, lg->catalog, NULL, lg->debug) != 0) { > + if (bm_subcommit(lg->catalog_bid, lg->catalog_nme, lg->catalog_bid, lg->catalog_nme, NULL, lg->debug) != 0) { > /* cannot commit catalog, so remove log */ > unlink(filename); > goto error; > @@ -1092,8 +1122,11 @@ logger_new(int debug, char *fn, char *lo > require a logical reference we also add a logical > reference for the persistent bats */ > BUN p, q; > - BAT *b = BATdescriptor(catalog); > - BATiter bi = bat_iterator(b); > + BAT *b = BATdescriptor(catalog_bid), *n; > + > + snprintf(bak, BUFSIZ, "%s_catalog_nme", fn); > + catalog_nme = BBPindex(bak); > + n = BATdescriptor(catalog_nme); > > /* the catalog exists, and so should the log file */ > if (fp == NULL) { > @@ -1107,42 +1140,63 @@ logger_new(int debug, char *fn, char *lo > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Niels Nes, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

Re: [Monetdb-developers] MonetDB: default - Partial de-mx batmtime
by Stefan Manegold 18 Jul '12

18 Jul '12

Just wondering ... (and trying to learn): On Wed, Jul 18, 2012 at 10:41:04AM +0200, Martin Kersten wrote: > Changeset: 29b4294d4bea for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=29b4294d4bea > Modified Files: > monetdb5/modules/kernel/batmtime.mx > Branch: default > Log Message: > > Partial de-mx batmtime > Rest should be taken care of by shell scripting > > > diffs (truncated from 316 to 300 lines): > > diff --git a/monetdb5/modules/kernel/batmtime.mx b/monetdb5/modules/kernel/batmtime.mx > --- a/monetdb5/modules/kernel/batmtime.mx > +++ b/monetdb5/modules/kernel/batmtime.mx > @@ -30,28 +30,25 @@ All Rights Reserved. > * In line with the batcalc module, we assume that > * if two bat operands are provided that they are already > * aligned on the head. Moreover, the head of the BATs > - * are limited to :void, which can be cheaply realized using > + * are limited to :oid, which can be cheaply realized using > * the GRPsplit operation. > */ > @= compareOp > command batcalc.@2( l:bat[:oid,:@1], r:bat[:oid,:@1]) :bat[:oid,:bit] > -address MTIMEbat_@1_@3; > - > -command batcalc.@2( l:bat[:void,:@1], r:bat[:void,:@1]) :bat[:void,:bit] > address MTIMEbat_@1_@3 > comment "Compare a bat of @1 against each other"; > command batcalc.@2( l:bat[:oid,:@1], r:@1) :bat[:oid,:bit] > address MTIMEbat_@1_@3_cst; > -command batcalc.@2( l:bat[:void,:@1], r:@1) :bat[:void,:bit] > +command batcalc.@2(r:@1, l:bat[:oid,:@1]) :bat[:oid,:bit] Is the above signature change (swapping left & right operand) intended? If so, what is the reason for it? > address MTIMEbat_@1_cst_@3 > comment "Compare a bat of @1 against a singleton"; > > -command batcalc.@2(a:bat[:void,:bit], l:bat[:void,:@1], r:bat[:void,:@1]) :bat[:void,:bit] > +command batcalc.@2(a:bat[:oid,:bit], l:bat[:oid,:@1], r:bat[:oid,:@1]) :bat[:oid,:bit] > address MTIMEaccum_@1_@3 > comment "Compare a bat of @1 against each other"; > -command batcalc.@2(a:bat[:void,:bit], l:bat[:oid,:@1], r:@1) :bat[:oid,:bit] > +command batcalc.@2(a:bat[:oid,:bit], l:bat[:oid,:@1], r:@1) :bat[:oid,:bit] > address MTIMEaccum_@1_@3_cst; > -command batcalc.@2(a:bat[:void,:bit], l:bat[:void,:@1], r:@1) :bat[:void,:bit] > +command batcalc.@2(a:bat[:oid,:bit], r:@1, l:bat[:oid,:@1]) :bat[:oid,:bit] Is the above signature change (swapping left & right operand) intended? If so, what is the reason for it? > address MTIMEaccum_@1_cst_@3 > comment "Compare a bat of @1 against a singleton"; > > @@ -64,24 +61,35 @@ comment "Compare a bat of @1 against a s > @:compareOp(@1,>,GT)@ > @:compareOp(@1,>=,GE)@ > > -@= unaryISNIL > - command batcalc.isnil(b:bat[:any_1,:@1]):bat[:any_1,:bit] > - address MTIMEunaryISNIL > - comment "Unary check for nil over the tail of the bat"; > - > - command batcalc.isnil(b:bat[:void,:@1]):bat[:oid,:bit] > - address MTIMEunaryISNIL > - comment "Unary check for nil over the tail of the bat"; > - > -@ > @mal > @:compareGrp(date)@ > @:compareGrp(daytime)@ > @:compareGrp(timestamp)@ > > -@:unaryISNIL(date)@ > -@:unaryISNIL(daytime)@ > -@:unaryISNIL(timestamp)@ > +command batcalc.isnil(b:bat[:any_1,:date]):bat[:any_1,:bit] > +address MTIMEunaryISNIL > +comment "Unary check for nil over the tail of the bat"; > + > +command batcalc.isnil(b:bat[:oid,:date]):bat[:oid,:bit] > +address MTIMEunaryISNIL > +comment "Unary check for nil over the tail of the bat"; > + > +command batcalc.isnil(b:bat[:any_1,:daytime]):bat[:any_1,:bit] > +address MTIMEunaryISNIL > +comment "Unary check for nil over the tail of the bat"; > + > +command batcalc.isnil(b:bat[:oid,:daytime]):bat[:oid,:bit] > +address MTIMEunaryISNIL > +comment "Unary check for nil over the tail of the bat"; > + > +command batcalc.isnil(b:bat[:any_1,:timestamp]):bat[:any_1,:bit] > +address MTIMEunaryISNIL > +comment "Unary check for nil over the tail of the bat"; > + > +command batcalc.isnil(b:bat[:oid,:timestamp]):bat[:oid,:bit] > +address MTIMEunaryISNIL > +comment "Unary check for nil over the tail of the bat"; I see a pattern of three pairs of signatures. What is the difference between, say, command batcalc.isnil(b:bat[:any_1,:date]):bat[:any_1,:bit] and command batcalc.isnil(b:bat[:oid,:date]):bat[:oid,:bit] ? Isn't the latter fully covered by the former? Thanks! Stefan ps: I not study the remainder, yet ... > > @h > /* > @@ -112,9 +120,6 @@ comment "Compare a bat of @1 against a s > * head column. This is not checked and may be mis-used to deploy the > * implementation for shifted window arithmetic as well. > */ > -@= chkSize > - if( BATcount(@1) != BATcount(@2) ) > - throw(MAL, "batcalc.@3", ILLEGAL_ARGUMENT " Requires bats of identical size"); > > @= implDef > batmtime_export str MTIMEbat_@1_@2(int *ret, int *l, int *r); > @@ -124,68 +129,18 @@ batmtime_export str MTIMEaccum_@1_@2(int > batmtime_export str MTIMEaccum_@1_@2_cst(int *ret, int *r, int *l, @1 *cst); > batmtime_export str MTIMEaccum_@1_cst_@2(int *ret, int *r, @1 *cst, int *l); > > -@ > - * @- Comparison operator implementation for temporal objects > - * The structure of this code is copied from the arithmetic scheme. > - * @- > -@= voidresultBAT > - bn = BATnew(TYPE_void, @1, BATcount(b)); > - BATseqbase(bn, b->hseqbase); > - if (bn == NULL) { > - throw(MAL, @2, MAL_MALLOC_FAIL); > - } > - bn->hsorted = b->hsorted; > - bn->hrevsorted = b->hrevsorted; > - bn->tsorted = b->tsorted; > - bn->trevsorted = b->trevsorted; > - bn->H->nonil = 1; > +@c > +#define resultBAT(Type,Error)\ > + bn = BATnew(TYPE_void, Type, BATcount(b));\ > + BATseqbase(bn, b->hseqbase);\ > + if (bn == NULL) \ > + throw(MAL, Error, MAL_MALLOC_FAIL);\ > + bn->hsorted = b->hsorted;\ > + bn->hrevsorted = b->hrevsorted;\ > + bn->tsorted = b->tsorted;\ > + bn->trevsorted = b->trevsorted;\ > + bn->H->nonil = 1;\ > bn->T->nonil = b->T->nonil; > -@= unarybaselineCode > -str MTIMEbat_@1_@2_@3(int *ret, int *l) > -{ > - BAT *bn, *b; > - @1 *p, *q, *t; > - bit *o; > - > - if ((b = BATdescriptor(*l)) == NULL) > - throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > - @:voidresultBAT(@3,"batcalc.@2")@ > - > - o = (bit*)Tloc(bn,BUNfirst(bn)); > - p = (@1*)Tloc(b,BUNfirst(b)); > - q = (@1*)Tloc(b,BUNlast(b)); > - t = (@1*)Tloc(right,BUNfirst(right)); > - > - BATaccessBegin(b, USE_TAIL, MMAP_SEQUENTIAL); > - BATaccessBegin(right, USE_TAIL, MMAP_SEQUENTIAL); > - if (b->T->nonil) > - for (;p<q; p++, t++, o++) > - @1@2(o,p,t); > - else > - for (;p<q; p++, t++, o++) > - if ( @1_isnil(*p)) > - *o = bit_nil; > - else @1@2(o,p,t); > - > - BATaccessEnd(right, USE_TAIL, MMAP_SEQUENTIAL); > - BATaccessEnd(b, USE_TAIL, MMAP_SEQUENTIAL); > - if (!(bn->batDirty&2)) bn = BATsetaccess(bn, BAT_READ); > - > - BATsetcount(bn, BATcount(b)); > - if (b->htype != bn->htype) { > - BAT *r = VIEWcreate(b,bn); > - > - BBPreleaseref(bn->batCacheid); > - bn = r; > - } > - > - bn->tsorted = 0; > - bn->trevsorted = 0; > - BATkey(BATmirror(bn),FALSE); > - BBPkeepref(*ret= bn->batCacheid); > - BBPreleaseref(b->batCacheid); > - return MAL_SUCCEED; > -} > > @= baselineCode > str MTIMEbat_@1_@2(int *ret, int *l, int *r) > @@ -201,8 +156,9 @@ str MTIMEbat_@1_@2(int *ret, int *l, int > BBPreleaseref(b->batCacheid); > throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > } > - @:chkSize(b,right,batcalc.@2)@ > - @:voidresultBAT(TYPE_bit,"batcalc.@2")@ > + if( BATcount(b) != BATcount(right) ) > + throw(MAL, "batcalc.@2", ILLEGAL_ARGUMENT " Requires bats of identical size"); > + resultBAT(TYPE_bit,"batcalc.@2") > > o = (bit*)Tloc(bn,BUNfirst(bn)); > p = (@1*)Tloc(b,BUNfirst(b)); > @@ -270,8 +226,10 @@ str MTIMEaccum_@1_@2(int *ret, int *d, i > BBPreleaseref(b->batCacheid);BBPreleaseref(right->batCacheid); > throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > } > - @:chkSize(bn,b,batcalc.@2)@ > - @:chkSize(bn,right,batcalc.@2)@ > + if( BATcount(bn) != BATcount(b) ) > + throw(MAL, "batcalc.@2", ILLEGAL_ARGUMENT " Requires bats of identical size"); > + if( BATcount(bn) != BATcount(right) ) > + throw(MAL, "batcalc.@2", ILLEGAL_ARGUMENT " Requires bats of identical size"); > > o = (bit*)Tloc(bn,BUNfirst(bn)); > p = (@1*)Tloc(b,BUNfirst(b)); > @@ -322,7 +280,7 @@ str MTIMEbat_@1_@2_cst(int *ret, int *l, > > if ((b = BATdescriptor(*l)) == NULL) > throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > - @:voidresultBAT(TYPE_bit,"batcalc.@2")@ > + resultBAT(TYPE_bit,"batcalc.@2") > BATsetcount(bn, BATcount(b)); > if (!(bn->batDirty&2)) bn = BATsetaccess(bn, BAT_READ); > > @@ -371,7 +329,7 @@ str MTIMEbat_@1_cst_@2(int *ret, @1 *cst > > if ((b = BATdescriptor(*l)) == NULL) > throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > - @:voidresultBAT(TYPE_bit,"batcalc.@2")@ > + resultBAT(TYPE_bit,"batcalc.@2") > BATsetcount(bn, BATcount(b)); > > o = (bit*)Tloc(bn,BUNfirst(bn)); > @@ -423,7 +381,8 @@ str MTIMEaccum_@1_@2_cst(int *ret, int * > @4; > throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > } > - @:chkSize(bn,b,batcalc.@2)@ > + if( BATcount(bn) != BATcount(b) ) > + throw(MAL, "batcalc.@2", ILLEGAL_ARGUMENT " Requires bats of identical size"); > > o = (bit*)Tloc(bn,BUNfirst(bn)); > p = (@1*)Tloc(b,BUNfirst(b)); > @@ -464,7 +423,8 @@ str MTIMEaccum_@1_cst_@2(int *ret, int * > BBPreleaseref(bn->batCacheid); > throw(MAL, "batcalc.@2", RUNTIME_OBJECT_MISSING); > } > - @:chkSize(bn,b,batcalc.@2)@ > + if( BATcount(bn) != BATcount(b) ) > + throw(MAL, "batcalc.@2", ILLEGAL_ARGUMENT " Requires bats of identical size"); > > o = (bit*)Tloc(bn,BUNfirst(bn)); > p = (@1*)Tloc(b,BUNfirst(b)); > @@ -493,9 +453,6 @@ str MTIMEaccum_@1_cst_@2(int *ret, int * > return MAL_SUCCEED; > } > > -@ > - * @- > - * The implementation section. > @= implCmpGrpDef > @:implDef(@1,EQ)@ > @:implDef(@1,NEQ)@ > @@ -532,20 +489,6 @@ batmtime_export str MTIMEunaryISNIL(int > @:baselineCode(timestamp,GT)@ > @:baselineCode(timestamp,GE)@ > > -@= accumUnaryISNIL > -{ > - @1 *p = (@1*)Tloc(b,BUNfirst(b)); > - @1 *q = (@1*)Tloc(b,BUNlast(b)); > - BATaccessBegin(b, USE_TAIL, MMAP_SEQUENTIAL); > - if (b->T->nonil){ > - for(; p<q; p++, o++ ) > - *o = FALSE; > - } else > - for(; p<q; p++, o++ ) > - *o = @1_isnil(*p); > - BATaccessEnd(b, USE_TAIL, MMAP_SEQUENTIAL); > -} > -@ > @c > str MTIMEunaryISNIL(int *ret, int *bid) > { > @@ -554,18 +497,41 @@ str MTIMEunaryISNIL(int *ret, int *bid) > > if ((b = BATdescriptor(*bid)) == NULL) > throw(MAL, "batcalc.isnil", RUNTIME_OBJECT_MISSING); > - @:voidresultBAT(TYPE_bit,"batcalc.isnil")@ > + resultBAT(TYPE_bit,"batcalc.isnil") > > o = (bit*)Tloc(bn,BUNfirst(bn)); > + BATaccessBegin(b, USE_TAIL, MMAP_SEQUENTIAL); > if ( BATttype(b) == TYPE_date) { > - @:accumUnaryISNIL(date)@ > + date *p = (date*)Tloc(b,BUNfirst(b)); > + date *q = (date*)Tloc(b,BUNlast(b)); > + if (b->T->nonil){ > + for(; p<q; p++, o++ ) > + *o = FALSE; > + } else > + for(; p<q; p++, o++ ) > + *o = date_isnil(*p); > } else > if ( BATttype(b) == TYPE_daytime) { > - @:accumUnaryISNIL(daytime)@ > + daytime *p = (daytime*)Tloc(b,BUNfirst(b)); > + daytime *q = (daytime*)Tloc(b,BUNlast(b)); > + if (b->T->nonil){ > + for(; p<q; p++, o++ ) > + *o = FALSE; > + } else > + for(; p<q; p++, o++ ) > + *o = daytime_isnil(*p); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 2

[Monetdb-developers] php_monetdb.dll
by suchitradaemon 17 Jul '12

17 Jul '12

Hi I want to be able to use all the SQL related functions using PHP on monetdb. As far as I understand, the newer versions of monetdb state that the PHP interface is completely written in PHP and need not be built. Is this interface same as the php_mapi.in and the php_monetdb.php? If so, I haven't seen any support for all the SQL related function like query_params. Please correct me if I am wrong. I am new to MonetDB. I had initially stumbled upon one of the old server reference manuals which had indicated of an option to build monetdb for php_monetdb.dll php extension. But then I started off with the incorrect version of the source (the tarball) and hit all sorts of errors. Then I was told to use the HG version. At which point I checked the windowsbuild.rst which stated that the PHP need not be built. So I just needed a confirmation whether building php_monetdb.dll is supported even in the latest releases or I need to make use of the php_monetdb.php file or there's any other way of getting all the SQL related functions? If php_monetdb.php is the answer, please let me know how to make use of it. Thank you Suchitra

4 4

[Monetdb-developers] Announcement: New Jul2012 Feature release of MonetDB suite
by Fabian Groffen 17 Jul '12

17 Jul '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2012 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2012 feature release Build Environment * Created seperate RPM and DEB packages for MonetDB/JAQL. * Removed --enable-bits option from configure. * Split the MonetDB-client-ruby RPM package into two and named them in accordance with the Fedora packaging guidelines as rubygem-<gem-name>. SQL * COPY INTO now accepts optional parenthesis for file argument. Binary COPY INTO now requires 'COPY BINARY INTO'. Client Package * ODBC: Fixed a bug where SQLNativeSql expected a statment handle instead of a connection handle. MonetDB5 Server * Crackers code has been removed. Development continues in the holindex branch. Merovingian * Fixed misc memory leaks, which caused monetdbd to grow in memory size over time. * Removed erroneously (re-)added master and slave properties, this functionality is currently not working. MonetDB Common * Fixed intermittent problem that joins and selects return incorrect results and possibly inconsistent databases. The problems only occurred after a series of queries and updates, therefore it was hard to reproduce. * The sorted property, which was used to maintain whether a column in a BAT was sorted or reverse sorted, has been replaced by a pair of properties, sorted and revsorted. These new properties can be set independently (unlike the old sorted property), and so if both are set, the column must be constant. In addition, internal property checking has been overhauled. Now, when a property is set incorrectly, and when assertions are enabled, an assertion will go off. There is also a function which can derive properties. * Implemented proper overflow checking on all arithmetic operations. Bug Fixes * 3016: Overflowing double multiplication leads to incorrect result * 3091: Since Apr2012-SP1 !TypeException:user.s11_1[59]:'bat.reverse' undefined in: _89:any := bat.reverse(_86:wrd) * 3099: ambiguous attributes in query with WITH clause and EXISTS clause * 3100: charset problem with the odbc driver for windows * 3103: display_size and internal_size of Cursor.description or incorrect * 3105: Leaking using Python * 3109: Wrong order in autoload mal script causes warning on startup * 3111: memory leak, monetdb set/get command * 3112: Median over temporal domains * 3113: Cannot set value of indexed column when doing UPDATE * 3114: Foreign Key prblems * 3116: Segfault in SQL

1 0

[Monetdb-developers] Why is the sourcecode 'de-mx-ed'?
by Stefan de Konink 16 Jul '12

16 Jul '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 The MonetDB checkins show over time a lot of de-mx operations. Why is the preprocessor language being abandoned in favor of 'native' macro's? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAlAEDa8ACgkQYH1+F2Rqwn2t3gCgiPrdrWFcxtBLIBWpVi/OTcx2 pywAoIdjsB8qXqGDjQqou0OKs5Esim8Q =xXNR -----END PGP SIGNATURE-----

2 1

Re: [Monetdb-developers] MonetDB: default - Add HEADLESS warning flag
by Stefan Manegold 15 Jul '12

15 Jul '12

FYI: "headless" means *dense* BAT heads (unless they are void-nil). S. ----- Original Message ----- > Changeset: 3be4152afaa3 for MonetDB > URL: > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=3be4152afaa3 > Modified Files: > gdk/gdk.h > gdk/gdk_bat.c > gdk/gdk_bat.h > monetdb5/mal/mal_dataflow.c > monetdb5/mal/mal_interpreter.c > Branch: default > Log Message: > > Add HEADLESS warning flag > During conversion towards :oid and col it would be nice to > trap all cases where an 'alien' bat is created. > A free bit in the debug flag is used for that purpose. > > > diffs (71 lines): > > diff --git a/gdk/gdk.h b/gdk/gdk.h > --- a/gdk/gdk.h > +++ b/gdk/gdk.h > @@ -2334,6 +2334,8 @@ VALptr(const ValRecord *v) > */ > #define PARMASK (1<<7) > #define PARDEBUG if (GDKdebug & PARMASK) > +#define HEADLESSMASK (1<<8) > +#define HEADLESSDEBUG if ( GDKdebug & HEADLESSMASK) > /* TRGMASK not used anymore > #define TRGMASK (1<<8) > #define TRGDEBUG if (GDKdebug & TRGMASK) > diff --git a/gdk/gdk_bat.c b/gdk/gdk_bat.c > --- a/gdk/gdk_bat.c > +++ b/gdk/gdk_bat.c > @@ -89,6 +89,11 @@ BATcreatedesc(int ht, int tt, int heapna > > if (bs == NULL) > return NULL; > + HEADLESSDEBUG { > + if ( ht != TYPE_void && ht != TYPE_oid) > + fprintf(stderr, "#headless violation in BATcreatedesc %d\n", ht); > + > + } > /* > * assert needed in the kernel to get symbol eprintf resolved. > * Else modules using assert fail to load. > @@ -2799,7 +2804,7 @@ BATmode(BAT *b, int mode) > * is not actually used anywhere, but it is checked. */ > > #ifndef NDEBUG > -void > +static void > BATassertHeadProps(BAT *b) > { > BATiter bi = bat_iterator(b); > diff --git a/gdk/gdk_bat.h b/gdk/gdk_bat.h > --- a/gdk/gdk_bat.h > +++ b/gdk/gdk_bat.h > @@ -23,7 +23,6 @@ > gdk_export BUN void_replace_bat(BAT *b, BAT *u, bit force); > gdk_export int void_inplace(BAT *b, oid id, const void *val, bit > force); > gdk_export BAT *BATattach(int tt, const char *heapfile); > -gdk_export void BATassertHeadProps(BAT *b); > > extern int default_ident(char *s); > extern oid MAXoid(BAT *i); > diff --git a/monetdb5/mal/mal_dataflow.c > b/monetdb5/mal/mal_dataflow.c > --- a/monetdb5/mal/mal_dataflow.c > +++ b/monetdb5/mal/mal_dataflow.c > @@ -618,7 +618,7 @@ DFLOWstep(FlowTask *t, FlowStatus fs) > #if 0 /* when needed, enable */ > if (i < pci->retc && bid){ > BAT *b = BATdescriptor(bid); > - BATassertHeadProps(b); > + BATassertProps(b); > BBPunfix(b->batCacheid); > } > #endif > diff --git a/monetdb5/mal/mal_interpreter.c > b/monetdb5/mal/mal_interpreter.c > --- a/monetdb5/mal/mal_interpreter.c > +++ b/monetdb5/mal/mal_interpreter.c > @@ -860,7 +860,7 @@ str runMALsequence(Client cntxt, MalBlkP > #if 0 /* when needed, enable */ > if (i < pci->retc && bid){ > BAT *b = BATdescriptor(bid); > - BATassertHeadProps(b); > + BATassertProps(b); > BBPunfix(b->batCacheid); > } > #endif > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: [Monetdb-developers] MonetDB: default - Debugging routine should be globally known
by Stefan Manegold 15 Jul '12

15 Jul '12

Shouldn't BATassertProps() (have) be(en) used instead of BATassertHeadProps()? The former is already globally known and calls the latter on both, head & tail of a BAT ... Stefan ----- Original Message ----- > Changeset: 6a5b3889c41e for MonetDB > URL: > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6a5b3889c41e > Modified Files: > gdk/gdk_bat.c > gdk/gdk_bat.h > Branch: default > Log Message: > > Debugging routine should be globally known > > > diffs (23 lines): > > diff --git a/gdk/gdk_bat.c b/gdk/gdk_bat.c > --- a/gdk/gdk_bat.c > +++ b/gdk/gdk_bat.c > @@ -2799,7 +2799,7 @@ BATmode(BAT *b, int mode) > * is not actually used anywhere, but it is checked. */ > > #ifndef NDEBUG > -static void > +void > BATassertHeadProps(BAT *b) > { > BATiter bi = bat_iterator(b); > diff --git a/gdk/gdk_bat.h b/gdk/gdk_bat.h > --- a/gdk/gdk_bat.h > +++ b/gdk/gdk_bat.h > @@ -23,6 +23,7 @@ > gdk_export BUN void_replace_bat(BAT *b, BAT *u, bit force); > gdk_export int void_inplace(BAT *b, oid id, const void *val, bit > force); > gdk_export BAT *BATattach(int tt, const char *heapfile); > +gdk_export void BATassertHeadProps(BAT *b); > > extern int default_ident(char *s); > extern oid MAXoid(BAT *i); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: [Monetdb-developers] MonetDB: default - Include debugging
by Stefan Manegold 15 Jul '12

15 Jul '12

In the light of :oid- or even dense-headed BATs (and eventually COLs), what is the reason to only check/assert head properties (trivial once dense), but not tail properties? Stefan ----- Original Message ----- > Changeset: f96a7db93e54 for MonetDB > URL: > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f96a7db93e54 > Modified Files: > monetdb5/mal/mal_dataflow.c > monetdb5/mal/mal_interpreter.c > Branch: default > Log Message: > > Include debugging > We have to trace for illegal property settings. It is an expensive > test, that may isolate a few deeply hidden errors. > > Moreover, initialize the cpu affinity > > > diffs (61 lines): > > diff --git a/monetdb5/mal/mal_dataflow.c > b/monetdb5/mal/mal_dataflow.c > --- a/monetdb5/mal/mal_dataflow.c > +++ b/monetdb5/mal/mal_dataflow.c > @@ -614,6 +614,16 @@ DFLOWstep(FlowTask *t, FlowStatus fs) > if (isaBatType(getArgType(mb, pci, i))) { > bat bid = stk->stk[a].val.bval; > > +#ifndef NDEBUG > +#if 0 /* when needed, enable */ > + if (i < pci->retc && bid){ > + BAT *b = BATdescriptor(bid); > + BATassertHeadProps(b); > + BBPunfix(b->batCacheid); > + } > +#endif > +#endif > + /* Check output properties */ > /* update the bigfoot information only if we need to gc */ > if (cntxt->flags & bigfootFlag) > updateBigFoot(cntxt, bid, TRUE); > @@ -811,8 +821,12 @@ runDFLOWworker(void *t) > Thread thr; > int i, local = 0, last = 0; > long usec=0; > + //int cpu; > > thr = THRnew(MT_getpid(), "DFLOWworker"); > + > + //cpu = MT_set_affinity(task->id); > + > GDKsetbuf(GDKmalloc(GDKMAXERRLEN)); /* where to leave errors */ > GDKerrbuf[0] = 0; > while (task) { > @@ -915,6 +929,7 @@ runDFLOWworker(void *t) > } > GDKfree(GDKerrbuf); > GDKsetbuf(0); > + //MT_unset_affinity(cpu); > THRdel(thr); > } > > diff --git a/monetdb5/mal/mal_interpreter.c > b/monetdb5/mal/mal_interpreter.c > --- a/monetdb5/mal/mal_interpreter.c > +++ b/monetdb5/mal/mal_interpreter.c > @@ -856,6 +856,16 @@ str runMALsequence(Client cntxt, MalBlkP > if (isaBatType(getArgType(mb, pci, i))) { > bat bid = stk->stk[a].val.bval; > > +#ifndef NDEBUG > +#if 0 /* when needed, enable */ > + if (i < pci->retc && bid){ > + BAT *b = BATdescriptor(bid); > + BATassertHeadProps(b); > + BBPunfix(b->batCacheid); > + } > +#endif > +#endif > + > /* update the bigfoot information only if we need to gc */ > if (cntxt->flags & bigfootFlag) > updateBigFoot(cntxt, bid, TRUE); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

[Monetdb-developers] a problem
by wuxiaobing 09 Jul '12

09 Jul '12

hi I hava a problem with my MonetDB. I want to visit MonetDB from other machines, that the document told us to set mapi_open=yes,but I do not know how to set it on windows. thank you for answer me. 2012-07-06 wuxiaobing

2 1

Re: [Monetdb-developers] MonetDB: holindex - Change L1 cache size into the number of inte...
by Stefan Manegold 25 Jun '12

25 Jun '12

On Mon, Jun 25, 2012 at 02:53:54PM +0200, Eleni Petraki wrote: > Changeset: f52327015707 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f52327015707 > Modified Files: > monetdb5/extras/crackers/crackers_selecthol_ops.mx > Branch: holindex > Log Message: > > Change L1 cache size into the number of integers that fit into L1. > > > diffs (12 lines): > > diff --git a/monetdb5/extras/crackers/crackers_selecthol_ops.mx b/monetdb5/extras/crackers/crackers_selecthol_ops.mx > --- a/monetdb5/extras/crackers/crackers_selecthol_ops.mx > +++ b/monetdb5/extras/crackers/crackers_selecthol_ops.mx > @@ -306,7 +306,7 @@ createView: > bit copy=TRUE; > > int countBatElements; > - int L1=8192; /*KiloBytes*/ > + int L1=2048000; /*Number of integers that can fit into L1 cache (size: 8192KB)*/ Wow --- which CPU has 8 MB L1(!) cache? > /*k and randomoid are used for DD1R*/ > @1 k,temp_h; > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] ODBC driver
by Max Kad 25 Jun '12

25 Jun '12

Hi Guys, I've installed MonetDB yesterday on Ubuntu 11.10 and it seems working fine. I've 2 questions though: 1) I'm trying to install MonetDB ODBC driver for UnixODBC following the instructions at (http://www.monetdb.org/Documentation/Manuals/SQLreference/Programming/ODBC ) but for some reason I can't find the Monet ODBC libMonetODBC.so or libMonetODBCs.so driver. ( I found the following files: libmonetdb5.so.11, libmonetdb5.so.11.0.2 but I'm not sure what they are for. ) 2) Are there similar Date functions available on MonetDB as the following MySQL functions: UNIX_TIMESTAMP(), DATE() ) or an epoch function. I couldn't find it on the following link. ( http://www.monetdb.org/Documentation/SQLreference/Temporal ) Thanks in advance, Max

2 1

[Monetdb-developers] ODBC driver
by Max Kad 23 Jun '12

23 Jun '12

Hi Guys, I've installed MonetDB yesterday on Ubuntu 11.10 and it seems working fine. I've 2 questions though: 1) I'm trying to install MonetDB ODBC driver for UnixODBC following the instructions at ( http://www.monetdb.org/Documentation/Manuals/SQLreference/Programming/ODBC ) but for some reason I can't find the Monet ODBC libMonetODBC.so or libMonetODBCs.so driver. ( I found the following files: libmonetdb5.so.11, libmonetdb5.so.11.0.2 but I'm not sure what they are for. ) 2) Are there similar Date functions available on MonetDB as the following MySQL functions: UNIX_TIMESTAMP(), DATE() ) or an epoch function. I couldn't find it on the following link. ( http://www.monetdb.org/Documentation/SQLreference/Temporal ) Thanks in advance, Max

1 0

[Monetdb-developers] Announcement: New Apr2012-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 19 Jun '12

19 Jun '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2012-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Apr 2012-SP2 bugfix release Client Package * Resolved a cosmetical error where tab-characters would cause misalignments in tabular result views. For the time being, tabs are now represented as a single space in tabular view. MonetDB Common * Limit number of detected CPU cores to 16 on 32-bits systems to avoid running quickly out of addressable resources followed by a kill from the OS. Bug Fixes * 2594: NULL cannot automatically be casted to every other type * 3085: assertion fails upon execution of a valid SQL query * 3086: Modulo function does not work in select from table * 3087: query causes "Not yet implemented" message * 3089: mserver5 crashed if join condition has two column and incorrect order * 3090: crashed if using single identifier for where condition * 3096: Type of median error

1 0

Re: [Monetdb-developers] MonetDB: default - attempt to more thoroughly remove opt_accessmode
by Sjoerd Mullender 14 Jun '12

14 Jun '12

On 2012-06-14 10:16, Fabian Groffen wrote: > Changeset: af264c7d6966 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=af264c7d6966 > Modified Files: > NT/installer32/MonetDB5-SQL-Installer.vdproj > NT/installer64/MonetDB5-SQL-Installer.vdproj > clients/Tests/exports.stable.out > monetdb5/modules/mal/mal_init.mal > monetdb5/optimizer/opt_support.c > sql/backends/monet5/Tests/optimizers.sql > sql/backends/monet5/Tests/optimizers.stable.err > Branch: default > Log Message: > > attempt to more thoroughly remove opt_accessmode > > > diffs (163 lines): > > diff --git a/NT/installer32/MonetDB5-SQL-Installer.vdproj b/NT/installer32/MonetDB5-SQL-Installer.vdproj > --- a/NT/installer32/MonetDB5-SQL-Installer.vdproj > +++ b/NT/installer32/MonetDB5-SQL-Installer.vdproj > @@ -2477,26 +2477,6 @@ > "IsDependency" = "11:FALSE" > "IsolateTo" = "8:" > } > - "{1FB2D0AE-D3B9-43D4-B9DD-F88EC61E35DE}:_4704D4539B804719AEB979F67EE20CDE" > - { > - "SourcePath" = "8:lib\\monetdb5\\opt_accessmode.mal" > - "TargetName" = "8:opt_accessmode.mal" > - "Tag" = "8:" > - "Folder" = "8:_B593DAA0787744D6A44D17884E0C3951" > - "Condition" = "8:" > - "Transitive" = "11:FALSE" > - "Vital" = "11:TRUE" > - "ReadOnly" = "11:FALSE" > - "Hidden" = "11:FALSE" > - "System" = "11:FALSE" > - "Permanent" = "11:FALSE" > - "SharedLegacy" = "11:FALSE" > - "PackageAs" = "3:1" > - "Register" = "3:1" > - "Exclude" = "11:FALSE" > - "IsDependency" = "11:FALSE" > - "IsolateTo" = "8:" > - } > "{1FB2D0AE-D3B9-43D4-B9DD-F88EC61E35DE}:_47A76DE07EE648EF9F19C2D1AC42B9EE" > { > "SourcePath" = "8:lib\\monetdb5\\autoload\\80_udf.mal" > diff --git a/NT/installer64/MonetDB5-SQL-Installer.vdproj b/NT/installer64/MonetDB5-SQL-Installer.vdproj > --- a/NT/installer64/MonetDB5-SQL-Installer.vdproj > +++ b/NT/installer64/MonetDB5-SQL-Installer.vdproj > @@ -2937,26 +2937,6 @@ > "IsDependency" = "11:FALSE" > "IsolateTo" = "8:" > } > - "{1FB2D0AE-D3B9-43D4-B9DD-F88EC61E35DE}:_7209273FAD0B4F4ABFF16B6C23681DA5" > - { > - "SourcePath" = "8:lib\\monetdb5\\opt_accessmode.mal" > - "TargetName" = "8:opt_accessmode.mal" > - "Tag" = "8:" > - "Folder" = "8:_B593DAA0787744D6A44D17884E0C3951" > - "Condition" = "8:" > - "Transitive" = "11:FALSE" > - "Vital" = "11:TRUE" > - "ReadOnly" = "11:FALSE" > - "Hidden" = "11:FALSE" > - "System" = "11:FALSE" > - "Permanent" = "11:FALSE" > - "SharedLegacy" = "11:FALSE" > - "PackageAs" = "3:1" > - "Register" = "3:1" > - "Exclude" = "11:FALSE" > - "IsDependency" = "11:FALSE" > - "IsolateTo" = "8:" > - } > "{1FB2D0AE-D3B9-43D4-B9DD-F88EC61E35DE}:_727ED1BD7FF74A91A491B34CC26E6848" > { > "SourcePath" = "8:lib\\monetdb5\\udf.mal" If you do this, also remove the "Entry" with MsmKey equal to the value behind the UUID (so 4704D4539B804719AEB979F67EE20CDE and 7209273FAD0B4F4ABFF16B6C23681DA5 in this case respectively). -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New Apr2012-SP1 Bugfix release of MonetDB suite
by Fabian Groffen 25 May '12

25 May '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2012-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Apr 2012-SP1 bugfix release Build Environment * Windows: we now use OpenSSL 1.0.1b. SQL * Databases that were upgraded from the Aug2011 release have an error in the catalog for SQL procedures. This is now fixed. Merovingian * Fixed a bug where connecting to a stopped multiplex-funnel would result in a 'there are no available connections' error. MonetDB Common * Implemented MT_getrss for Mac OS X systems, this allows the server to know about how much memory is currently in use. Bug Fixes * 2454: SQL: add various statistical functions * 2916: non admin user cannot create temporary tables * 2949: SQL exception during execution of a correct user defined function that just returns a relation table constructed by a simple SQL select query * 2958: Grant does't work with schema syntax * 2960: mclient terminates connection during insertion of a large number of records from a file * 2972: SQL URL functionality contains errors * 2987: Broken type inference when using sql round * 2992: SQL module truncates DECIMALS when passing integers to MAL module * 3000: Inconsistency between read-only and read-write modes after dropping a schema * 3028: Segmentation fault in seqscan_eq_bte_bte_tloc_void_void * 3046: mserver5 terminates with segmentation fault * 3063: datacell functions no longer work after restart of database * 3072: Database upgrade from Dec2011 to Apr2012 broken for storage() * 3073: VERY slow foreign key check (due to missing batcalc.isnil() ?) * 3074: Name resolution in procedures * 3077: No errors on UDF returning table with too many columns * 3079: Converting an Oracle schema DDL for MonetDB fails to create a column named "no".

1 0

[Monetdb-developers] MonetDB on SUSE
by Hualin Wang 22 May '12

22 May '12

Dear All, I have trouble to install the right PCRE lib on my SUSE box. The server has an old PCRE 7.8 installed. I compiled my own PCRE 8.3 version with UTF-8 enabled. Then set up the env variables pcre_LIBS, pcre_CFLAGS, PKG_CONFIG_PATH, but the ./configure sitll complain utf-8 supported no. Which version I should install on the SUSE box? Thanks for any hint/help Best Regards, Hualin

2 2

[Monetdb-developers] New Jun2012 stable branch created
by Sjoerd Mullender 21 May '12

21 May '12

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Jun2012. So in each of your stable checkout directories you can do hg pull hg update -rJun2012 to update to this new branch. -- Sjoerd Mullender

1 1

Re: [Monetdb-developers] MonetDB: Apr2012 - approve output after null-any type changes
by Fabian Groffen 17 May '12

17 May '12

Niels, On 16-05-2012 22:49:30 +0200, Niels Nes wrote: > Changeset: f450300984b8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f450300984b8 > @@ -20,12 +20,12 @@ Ready. > > % . # table_name > % single_value # name > -% char # type > +% any # type > % 0 # length > [ NULL ] If the "any" type now boils up to the clients, we need to fix/change the APIs too, to understand this type and in some cases do something sensical with it. JDBC will barf on it currently, IIRC. Would it be possible to use an existing type for NULL when it has no type at the stage where it is sent back to the client (like exportResult)? Fabian

2 1

Re: [Monetdb-developers] MonetDB: default - Fix compilation.
by Stefan Manegold 09 May '12

09 May '12

FYI, if I recall correctly, on my Fedora 16 desktop, a delta build worked fine, but a build from scratch (both optimized) failed as nightly testing did. Stefan On Wed, May 09, 2012 at 09:36:04AM +0200, Sjoerd Mullender wrote: > Changeset: 5aa6c5cc9702 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=5aa6c5cc9702 > Modified Files: > gdk/gdk_ssort_impl.h > Branch: default > Log Message: > > Fix compilation. > Why didn't gcc on Fedora 16 find these? > > > diffs (161 lines): > > diff --git a/gdk/gdk_ssort_impl.h b/gdk/gdk_ssort_impl.h > --- a/gdk/gdk_ssort_impl.h > +++ b/gdk/gdk_ssort_impl.h > @@ -21,14 +21,28 @@ > * to be defined by the including file, and we expect that the > * combination (T,O) is unique to each inclusion. */ > > -#define binarysort(a,b) binarysort_##a##b > -#define COPY(a) COPY_##a > -#define ISLT(a,b) ISLT_##a##b > -#define gallop_left(a,b) gallop_left_##a##b > -#define gallop_right(a,b) gallop_right_##a##b > -#define merge_at(a,b) merge_at_##a##b > -#define do_ssort(a,b) do_ssort_##a##b > -#define GDKssortimpl(b) GDKssort##b > +/* concatenate two or three tokens */ > +#define CONCAT2(a,b) a##b > +#define CONCAT3(a,b,c) a##b##c > + > +/* these tokens get redefined in each inclusion */ > +#undef binarysort > +#undef ISLT > +#undef COPY > +#undef gallop_left > +#undef gallop_right > +#undef merge_at > +#undef do_ssort > +#undef GDKssortimpl > + > +#define binarysort(a,b) CONCAT3(binarysort_,a,b) > +#define COPY(a) CONCAT2(COPY_,a) > +#define ISLT(a,b) CONCAT3(ISLT_,a,b) > +#define gallop_left(a,b) CONCAT3(gallop_left_,a,b) > +#define gallop_right(a,b) CONCAT3(gallop_right_,a,b) > +#define merge_at(a,b) CONCAT3(merge_at_,a,b) > +#define do_ssort(a,b) CONCAT3(do_ssort_,a,b) > +#define GDKssortimpl(b) CONCAT2(GDKssort,b) > > /* binarysort is the best method for sorting small arrays: it does few > * compares, but can do data movement quadratic in the number of > @@ -709,19 +723,20 @@ do_ssort(T,O)(MergeState *ms, ssize_t nr > n = 1; > } else { > n = 2; > - if (ISLT(T,O)(PTRADD(ms.bh, nlo, ms.hs), > - PTRADD(ms.bh, olo, ms.hs), &ms)) { > + if (ISLT(T,O)(PTRADD(ms->bh, nlo, ms->hs), > + PTRADD(ms->bh, olo, ms->hs), > + ms)) { > descending = 1; > for (olo = nlo++; > nlo < hi; > olo = nlo++, ++n) { > - if (!ISLT(T,O)(PTRADD(ms.bh, > + if (!ISLT(T,O)(PTRADD(ms->bh, > nlo, > - ms.hs), > - PTRADD(ms.bh, > + ms->hs), > + PTRADD(ms->bh, > olo, > - ms.hs), > - &ms)) > + ms->hs), > + ms)) > break; > } > } > @@ -729,32 +744,32 @@ do_ssort(T,O)(MergeState *ms, ssize_t nr > for (olo = nlo++; > nlo < hi; > olo = nlo++, ++n) { > - if (ISLT(T,O)(PTRADD(ms.bh, > + if (ISLT(T,O)(PTRADD(ms->bh, > nlo, > - ms.hs), > - PTRADD(ms.bh, > + ms->hs), > + PTRADD(ms->bh, > olo, > - ms.hs), > - &ms)) > + ms->hs), > + ms)) > break; > } > } > } > } > if (descending) > - reverse_slice(lo, lo + n, &ms); > + reverse_slice(lo, lo + n, ms); > /* If short, extend to min(minrun, nremaining). */ > if (n < minrun) { > ssize_t force = nremaining <= minrun ? nremaining : minrun; > > - binarysort(T,O)(lo, lo + force, lo + n, &ms); > + binarysort(T,O)(lo, lo + force, lo + n, ms); > n = force; > } > /* Push run onto pending-runs stack, and maybe merge. */ > - assert(ms.n < MAX_MERGE_PENDING); > - ms.pending[ms.n].base = lo; > - ms.pending[ms.n].len = n; > - ms.n++; > + assert(ms->n < MAX_MERGE_PENDING); > + ms->pending[ms->n].base = lo; > + ms->pending[ms->n].len = n; > + ms->n++; > { > /* Examine the stack of runs waiting to be merged, merging adjacent > * runs until the stack invariants are re-established: > @@ -765,19 +780,19 @@ do_ssort(T,O)(MergeState *ms, ssize_t nr > * See listsort.txt for more info. > * > * Returns 0 on success, -1 on error. */ > - struct slice *p = ms.pending; > + struct slice *p = ms->pending; > > - while (ms.n > 1) { > - ssize_t i = ms.n - 2; > + while (ms->n > 1) { > + ssize_t i = ms->n - 2; > > if (i > 0 && > p[i - 1].len <= p[i].len + p[i + 1].len) { > if (p[i - 1].len < p[i + 1].len) > --i; > - if (merge_at(T,O)(&ms, i) < 0) > + if (merge_at(T,O)(ms, i) < 0) > return -1; > } else if (p[i].len <= p[i + 1].len) { > - if (merge_at(T,O)(&ms, i) < 0) > + if (merge_at(T,O)(ms, i) < 0) > return -1; > } else > break; > @@ -794,14 +809,14 @@ do_ssort(T,O)(MergeState *ms, ssize_t nr > * one remains. This is used at the end of the mergesort. > * > * Returns 0 on success, -1 on error. */ > - struct slice *p = ms.pending; > + struct slice *p = ms->pending; > > - while (ms.n > 1) { > - ssize_t n = ms.n - 2; > + while (ms->n > 1) { > + ssize_t n = ms->n - 2; > > if (n > 0 && p[n - 1].len < p[n + 1].len) > --n; > - if (merge_at(T,O)(&ms, n) < 0) > + if (merge_at(T,O)(ms, n) < 0) > return -1; > } > } > @@ -916,3 +931,4 @@ GDKssortimpl(O)(void *h, void *t, void * > merge_freemem(&ms); > return result; > } > +#endif /* DEFINE_MAIN_FUNC */ > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] Announcement: New Apr2012 Feature release of MonetDB suite
by Sjoerd Mullender 27 Apr '12

27 Apr '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2012 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Apr 2012 feature release Build Environment * Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case. * On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997. SQL * Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient. * Stripped off implementation-specific parts from error messages before they get presented to the user. Java Module * Password reading by JdbcClient no longer results in strange artifacts * JdbcClient now returns exit code 1 in case of failures * Implemented missing Number types support in PreparedStatement.setObject() * Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument. * Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages Client Package * ODBC: Implemented the SQL_ATTR_CONNECTION_TIMEOUT attribute. * mclient now has a -a (--autocommit) option to turn off autocommit mode. MonetDB5 Server * The type "chr" has been removed. chr has long been superseded by bte for 1 byte arithmetic plus it is pretty useless to hold single characters since we use Unicode and thus only a tiny subset of the supported character set would fit. * Fixed potential crash by dealing better with non-standard types. * A couple of memory leaks have been plugged. Merovingian * The logfile and pidfile monetdbd properties are now displayed with dbfarm path when relative * Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries. * Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982. * Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983. * Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984. * Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984. MonetDB Common * The type "chr" has been removed. chr has long been superseded by bte for 1 byte arithmetic plus it is pretty useless to hold single characters since we use Unicode and thus only a tiny subset of the supported character set would fit. Bug Fixes

1 0

Re: [Monetdb-developers] MonetDB: holindex - Initial step for holistic indexing.
by Stefan Manegold 24 Apr '12

24 Apr '12

Eleni, you might also want to checkin your new files that are required to compile your code ;-) Thanks, Stefan On Tue, Apr 24, 2012 at 10:06:06AM +0200, Eleni Petraki wrote: > Changeset: 71b71ebda4e6 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=71b71ebda4e6 > Modified Files: > monetdb5/extras/crackers/60_crackers.mal > monetdb5/extras/crackers/Makefile.ag > monetdb5/extras/crackers/crackers.mx > monetdb5/optimizer/opt_pipes.c > monetdb5/optimizer/opt_support.c > monetdb5/optimizer/opt_support.h > Branch: holindex > Log Message: > > Initial step for holistic indexing. > I implemented an operator that initializes with zero the weights in a list. > > > diffs (175 lines): > > diff --git a/monetdb5/extras/crackers/60_crackers.mal b/monetdb5/extras/crackers/60_crackers.mal > --- a/monetdb5/extras/crackers/60_crackers.mal > +++ b/monetdb5/extras/crackers/60_crackers.mal > @@ -19,5 +19,6 @@ > library crackers; > include opt_selcrack; > include opt_selcrackst; > +include opt_selcrackhol; > include crackers; > > diff --git a/monetdb5/extras/crackers/Makefile.ag b/monetdb5/extras/crackers/Makefile.ag > --- a/monetdb5/extras/crackers/Makefile.ag > +++ b/monetdb5/extras/crackers/Makefile.ag > @@ -35,9 +35,13 @@ lib_crackers = { > crackers_index.mx \ > crackers_core_unordered.mx \ > crackers_select_ops.mx \ > + crackers_holistic.c \ > + crackers_holistic.h \ > crackers_selectst_ops.mx \ > + crackers_selecthol_ops.mx \ > opt_selcrack.mx \ > - opt_selcrackst.mx > + opt_selcrackst.mx \ > + opt_selcrackhol.mx > LIBS = ../../tools/libmonetdb5 \ > ../../../gdk/libbat \ > ../../../common/stream/libstream \ > @@ -49,7 +53,8 @@ headers_mal = { > DIR = libdir/monetdb5 > SOURCES = crackers.mx \ > opt_selcrack.mx \ > - opt_selcrackst.mx > + opt_selcrackst.mx \ > + opt_selcrackhol.mx > } > > headers_crackersautoload = { > diff --git a/monetdb5/extras/crackers/crackers.mx b/monetdb5/extras/crackers/crackers.mx > --- a/monetdb5/extras/crackers/crackers.mx > +++ b/monetdb5/extras/crackers/crackers.mx > @@ -239,6 +239,55 @@ address CRKthetaselectst_@2 > comment "Retrieve the subset using a cracker > index producing preferably a BATview."; > > + > +command selecthol(b:bat[:oid,:@2],l:@2,h:@2):bat[:oid,:@2] > +address CRKselecthol_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command selecthol(b:bat[:oid,:@2],l:@2):bat[:oid,:@2] > +address CRKselectholValue_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command selecthol(b:bat[:oid,:@2],l:@2,h:@2,li:bit,hi:bit):bat[:oid,:@2] > +address CRKselectholBounds_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command uselecthol(b:bat[:any_1,:@2],l:@2,h:@2):bat[:any_1,:void] > +address CRKuselecthol_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command uselecthol(b:bat[:any_1,:@2],l:@2):bat[:any_1,:void] > +address CRKuselectholValue_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command uselecthol(b:bat[:any_1,:@2],l:@2,h:@2,li:bit,hi:bit):bat[:any_1,:void] > +address CRKuselectholBounds_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command thetauselecthol(b:bat[:any_1,:@2],v:@2,op:str):bat[:any_1,:void] > +address CRKthetauselecthol_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command thetaselecthol(b:bat[:any_1,:@2],v:@2,op:str):bat[:any_1,:@2] > +address CRKthetaselecthol_@2 > +comment "Retrieve the subset using a cracker > + index producing preferably a BATview."; > + > +command initFrequencyStruct(b:bat[:any_1,:@2]):void > +address CRKinitFrequencyStruct_@2 > +comment "Initialize the struct(lifo) with zero values."; > + > +command selectNode(l:int,h:int):void > +address CRKselectNode > +comment "Select node/BAT to crack (randomly)."; > + > @ > @= Index > command printCrackerIndexBATpart(b:bat[:any_1,:@2]):void > @@ -296,12 +345,15 @@ comment "Break a BAT into three pieces w > #include "crackers_index.h" > #include "crackers_select_ops.h" > #include "crackers_selectst_ops.h" > +#include "crackers_selecthol_ops.h" > +#include "crackers_holistic.h" > #include "crackers_core_unordered.h" > #include "crackers_AVL_tree.h" > > #ifdef LIBCRACKERS > extern int maxCrackMap; > extern CrackerIndexNode *CrackerIndex; > +extern FrequencyNode *FrequencyStruct; > extern int IndexEntries; > extern int IndexSize; > extern int IndexStop; > @@ -323,6 +375,7 @@ extern struct storageManagement *Storage > int maxCrackMap = 0; > > CrackerIndexNode *CrackerIndex; > +FrequencyNode *FrequencyStruct; > > struct storageManagement *StorageInfo = NULL; > > diff --git a/monetdb5/optimizer/opt_pipes.c b/monetdb5/optimizer/opt_pipes.c > --- a/monetdb5/optimizer/opt_pipes.c > +++ b/monetdb5/optimizer/opt_pipes.c > @@ -296,6 +296,28 @@ struct PIPELINES { > "optimizer.multiplex();" > "optimizer.garbageCollector();", > "experimental", "OPTselcrackst", 0}, > +/*The HOLISTIC indexing pipeline*/ > + {"crackerhol_pipe", > + "optimizer.inline();" > + "optimizer.remap();" > + "optimizer.evaluate();" > + "optimizer.costModel();" > + "optimizer.coercions();" > + "optimizer.emptySet();" > + "optimizer.aliases();" > + "optimizer.selcrackhol();" > + "optimizer.deadcode();" > + "optimizer.commonTerms();" > + "optimizer.groups();" > + "optimizer.joinPath();" > + "optimizer.reorder();" > + "optimizer.deadcode();" > + "optimizer.reduce();" > + "optimizer.dataflow();" > + "optimizer.history();" > + "optimizer.multiplex();" > + "optimizer.garbageCollector();", > + "experimental", "OPTselcrackhol", 0}, > > > {"sidcrack_pipe", > diff --git a/monetdb5/optimizer/opt_support.c b/monetdb5/optimizer/opt_support.c > --- a/monetdb5/optimizer/opt_support.c > +++ b/monetdb5/optimizer/opt_support.c > @@ -603,6 +603,7 @@ struct OPTcatalog { > {"replication", 0, 0, 0, DEBUG_OPT_REPLICATION}, > {"selcrack", 0, 0, 0, DEBUG_OPT_SELCRACK}, > {"selcrackst", 0, 0, 0, DEBUG_OPT_SELCRACKST}, > +{"selcrackhol", 0, 0, 0, DEBUG_OPT_SELCRACKHOL}, > {"sidcrack", 0, 0, 0, DEBUG_OPT_SIDCRACK}, > {"singleton", 0, 0, 0, DEBUG_OPT_SINGLETON}, > {"strengthreduction", 0, 0, 0, DEBUG_OPT_STRENGTHREDUCTION}, > diff --git a/monetdb5/optimizer/opt_support.h b/monetdb5/optimizer/opt_support.h > --- a/monetdb5/optimizer/opt_support.h > +++ b/monetdb5/optimizer/opt_support.h > @@ -93,6 +93,7 @@ > #define DEBUG_OPT_HEURISTIC 49 > #define DEBUG_OPT_CENTIPEDE 50 > #define DEBUG_OPT_SELCRACKST 51 > +#define DEBUG_OPT_SELCRACKHOL 52 > > #define DEBUG_OPT(X) ((lng) 1 << (X)) > opt_export lng optDebug; > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] help ! Why can't i analyze schema ?
by hunglmk 20 Apr '12

20 Apr '12

* * * * * * * * * * * * * * * * * * * * Sio Lâm Minh Khánh Hùng * * * * * * * * * * * * * * * * * * * * * ----- Original Message ----- From: hunglmk To: monetdb-developers(a)lists.sourceforge.net Sent: Wednesday, April 04, 2012 8:07 chieu Subject: Why can't i analyze schema ? Good morning . I'm using monetdb with proximity and when i was trying to [analyze schema] the database but nothing happened thought i did exactly like the tutorial . here what i got from the command promt ================================================================== 0 [main] WARN kdl.prox3.util.Util - * log config file not found (using defaults ): D:\prx\proximity\prox.lcf' Exception in thread "AWT-EventQueue-0" java.lang.IllegalAccessError: class javax .swing.$Proxy1 cannot access its superinterface javax.swing.TransferHandler$HasG etTransferHandler at java.lang.reflect.Proxy.defineClass0(Native Method) at java.lang.reflect.Proxy.getProxyClass(Unknown Source) at java.lang.reflect.Proxy.newProxyInstance(Unknown Source) at spin.Spin.getProxy(Spin.java:190) at spin.Spin.over(Spin.java:223) at spin.off.ListenerSpinOver.spinOver(ListenerSpinOver.java:104) at spin.off.ListenerSpinOver.intercept(ListenerSpinOver.java:64) at spin.Spin$AbstractInvocationHandler.invoke(Spin.java:411) at $Proxy0.addPropertyChangeListener(Unknown Source) at kdl.prox3.gui2.SchemaAnalysisJFrame.analyze(SchemaAnalysisJFrame.java :98) at kdl.prox3.gui2.SchemaAnalysisJFrame.access$000(SchemaAnalysisJFrame.java:57) at kdl.prox3.gui2.SchemaAnalysisJFrame$1.actionPerformed(SchemaAnalysisJFrame.java:121) at javax.swing.AbstractButton.fireActionPerformed(Unknown Source) at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.setPressed(Unknown Source) at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source) at java.awt.Component.processMouseEvent(Unknown Source) at javax.swing.JComponent.processMouseEvent(Unknown Source) at java.awt.Component.processEvent(Unknown Source) at java.awt.Container.processEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Window.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$000(Unknown Source) at java.awt.EventQueue$1.run(Unknown Source) at java.awt.EventQueue$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue$2.run(Unknown Source) at java.awt.EventQueue$2.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source) ================================================================== I'm using window XP SP2 and JDK 1.6 , JRE 1.6 , monetdb 4.6.2 for windows . Do i need to install any software or library . Any advices will be great help . Thanks for your time Have a nice day * * * * * * * * * * * * * * * * * * * * Sio Lâm Minh Khánh Hùng * * * * * * * * * * * * * * * * * * * * *

1 0

[Monetdb-developers] Why can't i analyze schema ?
by hunglmk 04 Apr '12

04 Apr '12

Good morning . I'm using monetdb with proximity and when i was trying to [analyze schema] the database but nothing happened thought i did exactly like the tutorial . here what i got from the command promt ================================================================== 0 [main] WARN kdl.prox3.util.Util - * log config file not found (using defaults ): D:\prx\proximity\prox.lcf' Exception in thread "AWT-EventQueue-0" java.lang.IllegalAccessError: class javax .swing.$Proxy1 cannot access its superinterface javax.swing.TransferHandler$HasG etTransferHandler at java.lang.reflect.Proxy.defineClass0(Native Method) at java.lang.reflect.Proxy.getProxyClass(Unknown Source) at java.lang.reflect.Proxy.newProxyInstance(Unknown Source) at spin.Spin.getProxy(Spin.java:190) at spin.Spin.over(Spin.java:223) at spin.off.ListenerSpinOver.spinOver(ListenerSpinOver.java:104) at spin.off.ListenerSpinOver.intercept(ListenerSpinOver.java:64) at spin.Spin$AbstractInvocationHandler.invoke(Spin.java:411) at $Proxy0.addPropertyChangeListener(Unknown Source) at kdl.prox3.gui2.SchemaAnalysisJFrame.analyze(SchemaAnalysisJFrame.java :98) at kdl.prox3.gui2.SchemaAnalysisJFrame.access$000(SchemaAnalysisJFrame.java:57) at kdl.prox3.gui2.SchemaAnalysisJFrame$1.actionPerformed(SchemaAnalysisJFrame.java:121) at javax.swing.AbstractButton.fireActionPerformed(Unknown Source) at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.setPressed(Unknown Source) at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source) at java.awt.Component.processMouseEvent(Unknown Source) at javax.swing.JComponent.processMouseEvent(Unknown Source) at java.awt.Component.processEvent(Unknown Source) at java.awt.Container.processEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Window.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$000(Unknown Source) at java.awt.EventQueue$1.run(Unknown Source) at java.awt.EventQueue$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue$2.run(Unknown Source) at java.awt.EventQueue$2.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source) ================================================================== I'm using window XP SP2 and JDK 1.6 , JRE 1.6 , monetdb 4.6.2 for windows . Do i need to install any software or library . Any advices will be great help . Thanks for your time Have a nice day * * * * * * * * * * * * * * * * * * * * Sio Lâm Minh Khánh Hùng * * * * * * * * * * * * * * * * * * * * *

2 1

[Monetdb-developers] (no subject)
by sylver_b 02 Apr '12

02 Apr '12

<a href="http://babyproduct.co.in/wp-content/plugins/extended-comment-options/02efpk…"> http://babyproduct.co.in/wp-content/plugins/extended-comment-options/02efpk…</a>

1 0

[Monetdb-developers] (no subject)
by sylver_b 02 Apr '12

02 Apr '12

<a href="http://netbooksindia.com/wp-content/plugins/extended-comment-options/02efpk…"> http://netbooksindia.com/wp-content/plugins/extended-comment-options/02efpk…</a>

1 0

Re: [Monetdb-developers] MonetDB: default - Merge with default branch.
by Stefan Manegold 17 Mar '12

17 Mar '12

oop, sorry, message should have been "Merge with Apr2012 branch." Stefan On Sat, Mar 17, 2012 at 03:20:38PM +0100, Stefan Manegold wrote: > Changeset: 498e01451664 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=498e01451664 > Modified Files: > clients/mapiclient/mclient.c > monetdb5/mal/mal_resolve.c > sql/backends/monet5/UDF/udf.c > sql/backends/monet5/UDF/udf.h > sql/test/bugs/Tests/crash_order_by.stable.out > tools/merovingian/daemon/controlrunner.c > tools/merovingian/daemon/merovingian.c > Branch: default > Log Message: > > Merge with default branch. > > > diffs (truncated from 413 to 300 lines): > > diff --git a/clients/mapiclient/mclient.c b/clients/mapiclient/mclient.c > --- a/clients/mapiclient/mclient.c > +++ b/clients/mapiclient/mclient.c > @@ -654,7 +654,9 @@ CSVrenderer(MapiHdl hdl) > s = mapi_fetch_field(hdl, i); > if (s == NULL) > s = nullstring == default_nullstring ? "" : nullstring; > - if (strchr(s, *sep) != NULL || strchr(s, '\n') != NULL || strchr(s, '"') != NULL) { > + if (strchr(s, *sep) != NULL || > + strchr(s, '\n') != NULL || > + strchr(s, '"') != NULL) { > mnstr_printf(toConsole, "%s\"", > i == 0 ? "" : sep); > while (*s) { > @@ -675,8 +677,6 @@ CSVrenderer(MapiHdl hdl) > mnstr_write(toConsole, "\"\"", 1, 2); > break; > default: > - if (*s == *sep) > - mnstr_write(toConsole, "\\", 1, 1); > mnstr_write(toConsole, s, 1, 1); > break; > } > diff --git a/monetdb5/mal/mal_resolve.c b/monetdb5/mal/mal_resolve.c > --- a/monetdb5/mal/mal_resolve.c > +++ b/monetdb5/mal/mal_resolve.c > @@ -68,7 +68,7 @@ > * These examples give little room to achieve our prime objective, i.e. > * a fast and early type resolution scheme. Any non-polymorphic function > * can be type checked and marked type-safe upon completion. > - * Type checking polymorphic functions are post-poned until a concrete > + * Type checking polymorphic functions are postponed until a concrete > * type instance is known. It leads to a clone, which can be type checked > * and is entered into the symbol table. > * The type resolution status is marked in each instruction. > diff --git a/sql/backends/monet5/UDF/udf.c b/sql/backends/monet5/UDF/udf.c > --- a/sql/backends/monet5/UDF/udf.c > +++ b/sql/backends/monet5/UDF/udf.c > @@ -3,26 +3,26 @@ > * Version 1.1 (the "License"); you may not use this file except in > * compliance with the License. You may obtain a copy of the License at > * http://www.monetdb.org/Legal/MonetDBLicense > - * > + * > * Software distributed under the License is distributed on an "AS IS" > * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > * License for the specific language governing rights and limitations > * under the License. > - * > + * > * The Original Code is the MonetDB Database System. > - * > + * > * The Initial Developer of the Original Code is CWI. > * Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > * Copyright August 2008-2012 MonetDB B.V. > * All Rights Reserved. > -*/ > - > + */ > > #include "monetdb_config.h" > #include "udf.h" > > /* Reverse a string */ > -str UDFreverse ( str *ret , str *arg ) > +str > +UDFreverse(str *ret, str *arg) > { > size_t len = 0; > str src = NULL, dst = NULL; > @@ -37,7 +37,8 @@ str UDFreverse ( str *ret , str *arg ) > if (src == NULL || strcmp(src, str_nil) == 0) { > *ret = GDKstrdup(str_nil); > if (*ret == NULL) > - throw(MAL, "udf.reverse", "failed to create copy of str_nil"); > + throw(MAL, "udf.reverse", > + "failed to create copy of str_nil"); > > return MAL_SUCCEED; > } > @@ -46,7 +47,8 @@ str UDFreverse ( str *ret , str *arg ) > len = strlen(src); > *ret = dst = GDKmalloc(len + 1); > if (dst == NULL) > - throw(MAL, "udf.reverse", "failed to allocate string of length " SZFMT, len+1); > + throw(MAL, "udf.reverse", > + "failed to allocate string of length " SZFMT, len + 1); > > /* copy characters from src to dst in reverse order */ > dst[len] = 0; > @@ -62,7 +64,8 @@ str UDFreverse ( str *ret , str *arg ) > * Generic "type-oblivious" version, > * using generic "type-oblivious" BAT access interface. > */ > -str UDFBATreverse ( bat *ret , bat *bid ) > +str > +UDFBATreverse(bat *ret, bat *bid) > { > BATiter li; > BAT *bn = NULL, *left = NULL; > @@ -78,7 +81,8 @@ str UDFBATreverse ( bat *ret , bat *bid > /* check tail type */ > if (left->ttype != TYPE_str) { > BBPreleaseref(left->batCacheid); > - throw(MAL, "batudf.reverse", "tail-type of input BAT must be TYPE_str"); > + throw(MAL, "batudf.reverse", > + "tail-type of input BAT must be TYPE_str"); > } > > /* allocate result BAT */ > @@ -98,6 +102,7 @@ str UDFBATreverse ( bat *ret , bat *bid > /* the core of the algorithm, expensive due to malloc/frees */ > BATloop(left, p, q) { > str tr = NULL, err = NULL; > + > /* get original head & tail value */ > ptr h = BUNhead(li, p); > str t = (str) BUNtail(li, p); > @@ -106,7 +111,8 @@ str UDFBATreverse ( bat *ret , bat *bid > err = UDFreverse(&tr, &t); > if (err != MAL_SUCCEED) { > /* error -> bail out */ > - BATaccessEnd(left, USE_HEAD | USE_TAIL, MMAP_SEQUENTIAL); > + BATaccessEnd(left, USE_HEAD | USE_TAIL, > + MMAP_SEQUENTIAL); > BBPreleaseref(left->batCacheid); > BBPreleaseref(*ret); > return err; > @@ -116,7 +122,7 @@ str UDFBATreverse ( bat *ret , bat *bid > assert(tr != NULL); > > /* insert original head and reversed tail in result BAT */ > - /* BUNins() takes care of all necessary admininstration */ > + /* BUNins() takes care of all necessary administration */ > BUNins(bn, h, tr, FALSE); > > /* free memory allocated in UDFreverse() */ > @@ -155,9 +161,9 @@ str UDFfuse_##in##_##out ( out *ret , in > return MAL_SUCCEED; \ > } > > -UDFfuse_scalar_impl(bte,unsigned char ,sht, 8) > -UDFfuse_scalar_impl(sht,unsigned short,int,16) > -UDFfuse_scalar_impl(int,unsigned int ,lng,32) > +UDFfuse_scalar_impl(bte, unsigned char, sht, 8) > +UDFfuse_scalar_impl(sht, unsigned short, int, 16) > +UDFfuse_scalar_impl(int, unsigned int, lng, 32) > > > /* BAT fuse */ > @@ -165,8 +171,8 @@ UDFfuse_scalar_impl(int,unsigned int ,l > * Type-expanded optimized version, > * accessing value arrays directly. > */ > -str > -UDFBATfuse ( bat *ires , bat *ione , bat *itwo) > + str > +UDFBATfuse(bat *ires, bat *ione, bat *itwo) > { > BAT *bres = NULL, *bone = NULL, *btwo = NULL; > bit two_tail_sorted_unsigned = FALSE; > @@ -185,89 +191,102 @@ UDFBATfuse ( bat *ires , bat *ione , bat > } > > /* check for dense & aligned heads */ > - if (!BAThdense(bone) || !BAThdense(btwo) || BATcount(bone) != BATcount(btwo) || bone->hseqbase != btwo->hseqbase) { > + if (!BAThdense(bone) || > + !BAThdense(btwo) || > + BATcount(bone) != BATcount(btwo) || > + bone->hseqbase != btwo->hseqbase) { > BBPreleaseref(bone->batCacheid); > BBPreleaseref(btwo->batCacheid); > - throw(MAL, "batudf.fuse", "heads of input BATs must be aligned"); > + throw(MAL, "batudf.fuse", > + "heads of input BATs must be aligned"); > } > > /* check tail types */ > if (bone->ttype != btwo->ttype) { > BBPreleaseref(bone->batCacheid); > BBPreleaseref(btwo->batCacheid); > - throw(MAL, "batudf.fuse", "tails of input BATs must be identical"); > + throw(MAL, "batudf.fuse", > + "tails of input BATs must be identical"); > } > > /* advice on sequential scan */ > BATaccessBegin(bone, USE_TAIL, MMAP_SEQUENTIAL); > BATaccessBegin(btwo, USE_TAIL, MMAP_SEQUENTIAL); > > -#define UDFBATfuse_TYPE(in,uin,out,shift) \ > -{ /* type-specific core algorithm */ \ > - in *one = NULL, *two = NULL; \ > - out *res = NULL; \ > - BUN i, n = BATcount(bone); \ > - \ > - /* allocate result BAT */ \ > - bres = BATnew(TYPE_void, TYPE_##out, n); \ > - if (bres == NULL) { \ > - BBPreleaseref(bone->batCacheid); \ > - BBPreleaseref(btwo->batCacheid); \ > - throw(MAL, "batudf.fuse", MAL_MALLOC_FAIL); \ > - } \ > - \ > - /* get direct access to the tail arrays */ \ > - one = (in *) Tloc(bone, BUNfirst(bone)); \ > - two = (in *) Tloc(btwo, BUNfirst(btwo)); \ > - res = (out*) Tloc(bres, BUNfirst(bres)); \ > - \ > - /* is tail of right/second BAT sorted, also when cast to unsigned type, */ \ > - /* i.e., are the values either all >= 0 or all < 0? */ \ > - two_tail_sorted_unsigned = BATtordered(btwo)&1 && ( two[0] >= 0 || two[n-1] < 0 ); \ > - \ > - /* iterate over all values/tuples and do the work */ \ > - for (i = 0; i < n; i++) \ > - if (one[i] == in##_nil || two[i] == in##_nil) \ > - /* NULL/nil in => NULL/nil out */ \ > - res[i] = out##_nil; \ > - else \ > - /* do the work; watch out for sign bits */ \ > - res[i] = ( ((out)((uin)one[i])) << shift ) | ((uin)two[i]); \ > - \ > - /* set number of tuples in result BAT */ \ > - BATsetcount(bres, n); \ > -} > +#define UDFBATfuse_TYPE(in,uin,out,shift) \ > +do { /* type-specific core algorithm */ \ > + in *one = NULL, *two = NULL; \ > + out *res = NULL; \ > + BUN i, n = BATcount(bone); \ > + \ > + /* allocate result BAT */ \ > + bres = BATnew(TYPE_void, TYPE_##out, n); \ > + if (bres == NULL) { \ > + BBPreleaseref(bone->batCacheid); \ > + BBPreleaseref(btwo->batCacheid); \ > + throw(MAL, "batudf.fuse", MAL_MALLOC_FAIL); \ > + } \ > + \ > + /* get direct access to the tail arrays */ \ > + one = (in *) Tloc(bone, BUNfirst(bone)); \ > + two = (in *) Tloc(btwo, BUNfirst(btwo)); \ > + res = (out*) Tloc(bres, BUNfirst(bres)); \ > + \ > + /* is tail of right/second BAT sorted, also when cast to \ > + * unsigned type, */ \ > + /* i.e., are the values either all >= 0 or all < 0? */ \ > + two_tail_sorted_unsigned = BATtordered(btwo) & 1 && \ > + (two[0] >= 0 || two[n - 1] < 0); \ > + \ > + /* iterate over all values/tuples and do the work */ \ > + for (i = 0; i < n; i++) \ > + if (one[i] == in##_nil || two[i] == in##_nil) \ > + /* NULL/nil in => NULL/nil out */ \ > + res[i] = out##_nil; \ > + else \ > + /* do the work; watch out for sign bits */ \ > + res[i] = ((out) (uin) one[i] << shift) | (uin) two[i]; \ > + \ > + /* set number of tuples in result BAT */ \ > + BATsetcount(bres, n); \ > +} while (0) > > /* type expansion for core algorithm */ > switch (bone->ttype) { > case TYPE_bte: > - UDFBATfuse_TYPE(bte,unsigned char ,sht, 8) > + UDFBATfuse_TYPE(bte, unsigned char, sht, 8); > break; > + > case TYPE_sht: > - UDFBATfuse_TYPE(sht,unsigned short,int,16) > + UDFBATfuse_TYPE(sht, unsigned short, int, 16); > break; > + > case TYPE_int: > - UDFBATfuse_TYPE(int,unsigned int ,lng,32) > + UDFBATfuse_TYPE(int, unsigned int, lng, 32); > break; > + > default: > - throw(MAL, "batudf.fuse", "tails of input BATs must be one of {bte, sht, int}"); > + throw(MAL, "batudf.fuse", > + "tails of input BATs must be one of {bte, sht, int}"); > } > > BATaccessEnd(bone, USE_TAIL, MMAP_SEQUENTIAL); > BATaccessEnd(btwo, USE_TAIL, MMAP_SEQUENTIAL); > > /* set result properties */ > - bres->hdense = TRUE; /* result head is dense */ > + bres->hdense = TRUE; /* result head is dense */ > BATseqbase(bres, bone->hseqbase); /* result head has same seqbase as input */ > - bres->hsorted = GDK_SORTED; /* result head is sorted */ > - BATkey(bone, TRUE); /* result head is key (unique) */ > + bres->hsorted = GDK_SORTED; /* result head is sorted */ > + BATkey(bres, TRUE); /* result head is key (unique) */ > > - /* Result tail is sorted, if the left/first input tail is sorted and key (unique), > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Re: [Monetdb-developers] MonetDB: default - Activating the new implementation of BATsample.
by Stefan Manegold 16 Mar '12

16 Mar '12

lsidir, (;-)) since it now seems to be dead code (correct me if I'm wrong), is there any reason to keep the "deprecated" implementation of BATsample around, or shouldn't we better clean up our code and remove the old BATsample (now BATsample_deprecated) entirely? Thanks, Stefan On Fri, Mar 16, 2012 at 12:51:52PM +0100, lsidir wrote: > Changeset: 6985c995bf66 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6985c995bf66 > Modified Files: > gdk/gdk.h > gdk/gdk_batop.mx > gdk/gdk_sample.c > monetdb5/modules/mal/sample.c > Branch: default > Log Message: > > Activating the new implementation of BATsample. > > > diffs (57 lines): > > diff --git a/gdk/gdk.h b/gdk/gdk.h > --- a/gdk/gdk.h > +++ b/gdk/gdk.h > @@ -3129,7 +3129,7 @@ gdk_export int ALIGNsetH(BAT *b1, BAT *b > gdk_export void PROPdestroy(PROPrec *p); > gdk_export PROPrec * BATgetprop(BAT *b, int idx); > gdk_export void BATsetprop(BAT *b, int idx, int type, void *v); > -gdk_export BAT *BATsample(BAT *b, BUN n); > +gdk_export BAT *BATsample_deprecated(BAT *b, BUN n); > gdk_export BAT *BAThistogram(BAT *b); > gdk_export int BATtopN(BAT *b, BUN topN); /* used in monet5/src/modules/kernel/algebra.mx */ > > @@ -3368,7 +3368,7 @@ gdk_export int BATcalcavg(BAT *b, dbl *a > * @tab BATsample (BAT *b, n) > * @end multitable > */ > -gdk_export BAT *BATsample1(BAT *b, BUN n); > +gdk_export BAT *BATsample(BAT *b, BUN n); > > /* generic n-ary multijoin beast, with defines to interpret retval */ > #define MULTIJOIN_SORTED(r) ((char*) &r)[0] > diff --git a/gdk/gdk_batop.mx b/gdk/gdk_batop.mx > --- a/gdk/gdk_batop.mx > +++ b/gdk/gdk_batop.mx > @@ -1374,7 +1374,7 @@ BATtopN(BAT *b, BUN topN) > * @- Random Selections > */ > BAT * > -BATsample(BAT *b, BUN size) > +BATsample_deprecated(BAT *b, BUN size) > { > BUN cnt, i, r = 0, n, j; > BUN *choice = NULL, *dst; > diff --git a/gdk/gdk_sample.c b/gdk/gdk_sample.c > --- a/gdk/gdk_sample.c > +++ b/gdk/gdk_sample.c > @@ -47,7 +47,7 @@ > */ > > BAT * > -BATsample1(BAT *b, BUN n) > +BATsample(BAT *b, BUN n) > { > BAT *bn; > BUN cnt; > diff --git a/monetdb5/modules/mal/sample.c b/monetdb5/modules/mal/sample.c > --- a/monetdb5/modules/mal/sample.c > +++ b/monetdb5/modules/mal/sample.c > @@ -85,7 +85,7 @@ SAMPLEuniform(bat *r, bat *b, ptr s) { > if ((bb = BATdescriptor(*b)) == NULL) { > throw(MAL, "sample.uniform", INTERNAL_BAT_ACCESS); > } > - br = BATsample1(bb,*(BUN *)s); > + br = BATsample(bb,*(BUN *)s); > if (br == NULL) > throw(MAL, "sample.uniform", OPERATION_FAILED); > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 2

[Monetdb-developers] Announcement: New Dec2011-SP2 Bugfix release of MonetDB suite
by Fabian Groffen 16 Mar '12

16 Mar '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2011-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2011-SP2 bugfix release Java Module * Implemented missing Number types support in PreparedStatement.setObject() Bug Fixes * 2915: Query (mclient or jdbc) with 2 outer joins appears to crash connectionor server * 2947: Queries where a "is null" clause is combined with two equals clauses fail to produce the expected result. * 2968: Memory Leaks * 2974: thetajoin produces invalid/inconsistent bat * 2988: Subquery in select part goes wrong * 2998: Equivalent multi-way join queries give different results * 3002: String plus interval result in program errors upon comparison * 3011: count(*) in subquery results in 1 for query without results * 3012: bitwise comparison fails after a cast * 3029: test sql/test/Dependencies/Tests/Dependencies.sql fails after recent changes * 3031: Virtually all SQL tests fail on Windows * 3036: optimizers are not getting set correctly through set optimizer * 3038: ~190 SQL tests fail on Darwin, FreeBSD & Solaris (only) since introduction of "(extended) relational algebra" * 3043: WITH clause is not allowed in derived table * 3045: algebra.project(0@0,bat) fails at propcheck * 3052: Wrong Error Code on Rollback Exception

1 0

Re: [Monetdb-developers] MonetDB: default - Remainder of group optimizer additions.
by Sjoerd Mullender 12 Mar '12

12 Mar '12

The new opt_group optimizer defines a group.new function that is not compatible with the existing group.new functions. In group.mx we have the definitions command new(b:bat[:any_1,:any_2], start:int, incr:int, grpsize:int) :bat[:any_1,:int] command new(attr:bat[:any_1,:any_2] ) (histo:bat[:any_1,:wrd], grp:bat[:any_1,:void]) command new(attr:bat[:any_1,:any_2] ) (histo:bat[:any_1,:wrd], grp:bat[:any_1,:oid]) command new(attr:bat[:any_1,:any_2], N:int, rng:int) (histo:bat[:any_1,:wrd],grp:bat[:any_1,:oid]) If called with a single :bat[:oid,:oid] bat as argument (as sql.zero_or_one does), it returns two bats with types :bat[:oid,:wrd] and :bat[:oid,:oid] The group.new in the new opt_group optimizer is defined as pattern group.new(l:bat[:oid,:any]...)(grp:bat[:oid,:any],ext:bat[:oid,:any]) so if this one is called with a single :bat[:oid,:oid] bat as argument, it returns two bats with types :bat[:oid,:oid] and :bat[:oid,:oid]. Of course, we now get the one sql.zero_or_one does *not* expect (since it is defined later). This causes a bunch of tests to fail. On 2012-03-12 12:52, Martin Kersten wrote: > Changeset: 89489d21d9bb for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=89489d21d9bb > Added Files: > monetdb5/optimizer/opt_groups.mx > Removed Files: > monetdb5/optimizer/Tests/derivePath00.mal > monetdb5/optimizer/Tests/groupderive.mal > monetdb5/optimizer/Tests/groupderive.stable.err > monetdb5/optimizer/Tests/groupderive.stable.out > monetdb5/optimizer/opt_derivepath.mx > Modified Files: > monetdb5/optimizer/Tests/All > sql/test/Tests/setoptimizer.stable.err > sql/test/Tests/setoptimizer.stable.out > Branch: default > Log Message: > > Remainder of group optimizer additions. > > > diffs (truncated from 654 to 300 lines): > > diff --git a/monetdb5/optimizer/Tests/All b/monetdb5/optimizer/Tests/All > --- a/monetdb5/optimizer/Tests/All > +++ b/monetdb5/optimizer/Tests/All > @@ -74,7 +74,7 @@ mitosis00 > > joinpath > leftjoinpath > -groupderive > +groups00 > > ifthencst > CXerror1 > diff --git a/monetdb5/optimizer/Tests/derivePath00.mal b/monetdb5/optimizer/Tests/derivePath00.mal > deleted file mode 100644 > --- a/monetdb5/optimizer/Tests/derivePath00.mal > +++ /dev/null > @@ -1,19 +0,0 @@ > --- select distinct(A) from T > --- only needs the group extend table, which means we can use a kunique. > -function tst(); > -b:= bat.new(:oid,:str); > - > -bat.append(b,"hello"); > -bat.append(b,"hello"); > -bat.append(b,"world"); > -br:= bat.reverse(b); > -k:= algebra.kunique(br); > -kr:= bat.reverse(k); > -io.print(kr); > -(e,g):= group.new(b); > -io.print(e); > -end tst; > - > -user.tst(); > -optimizer.derivePath("user","tst"); > -mdb.List("user","tst"); > diff --git a/monetdb5/optimizer/Tests/groupderive.mal b/monetdb5/optimizer/Tests/groupderive.mal > deleted file mode 100644 > --- a/monetdb5/optimizer/Tests/groupderive.mal > +++ /dev/null > @@ -1,10 +0,0 @@ > -function tst(); > - b:= bat.new(:oid,:int); > - c:= bat.new(:oid,:str); > - d:= bat.new(:oid,:flt); > - (ext1,grp1) := group.new(b); > - (ext2,grp2) := group.derive(ext1,grp1, c); > - (ext3,grp3) := group.derive(ext2,grp2, d); > -end tst; > -optimizer.derivePath("user","tst"); > -mdb.list("user","tst"); > diff --git a/monetdb5/optimizer/Tests/groupderive.stable.err b/monetdb5/optimizer/Tests/groupderive.stable.err > deleted file mode 100644 > --- a/monetdb5/optimizer/Tests/groupderive.stable.err > +++ /dev/null > @@ -1,80 +0,0 @@ > -stderr of test 'groupderive` in directory 'optimizer` itself: > - > - > -# 09:23:28 > > -# 09:23:28 > mserver5 "--config=/ufs/mk/monet5//Linux/etc/monetdb5.conf" --debug=10 --set gdk_nr_threads=0 --set "monet_mod_path=/ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin" --set "gdk_dbfarm=/ufs/mk/monet5//Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/sql_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=30087 --set xrpc_port=43368 --set xrpc_docroot=/ufs/mk/monet5/package/MonetDB5 --set monet_prompt= --trace --dbname=mTests_src_optimizer groupderive.mal > -# 09:23:28 > > - > -# builtin opt gdk_arch = 64bitx86_64-unknown-linux-gnu > -# builtin opt gdk_version = 1.28.1 > -# builtin opt monet_pid = 10370 > -# builtin opt prefix = /ufs/mk/monet5//Linux > -# builtin opt exec_prefix = ${prefix} > -# builtin opt gdk_dbname = tst > -# builtin opt gdk_dbfarm = ${prefix}/var/MonetDB > -# builtin opt gdk_debug = 8 > -# builtin opt gdk_mem_bigsize = 262144 > -# builtin opt gdk_alloc_map = yes > -# builtin opt gdk_mem_pagebits = 14 > -# builtin opt gdk_vmtrim = yes > -# builtin opt monet_admin = adm > -# builtin opt monet_prompt = > > -# builtin opt monet_welcome = yes > -# builtin opt monet_mod_path = ${exec_prefix}/lib/MonetDB:${exec_prefix}/lib/bin > -# builtin opt monet_daemon = yes > -# builtin opt host = localhost > -# builtin opt mapi_port = 50000 > -# builtin opt mapi_noheaders = no > -# builtin opt mapi_debug = 0 > -# builtin opt mapi_clients = 2 > -# builtin opt sql_debug = 0 > -# builtin opt sql_logdir = ${prefix}/var/MonetDB > -# builtin opt xquery_logdir = ${prefix}/var/MonetDB > -# builtin opt standoff_ns = http://monetdb.cwi.nl/standoff > -# builtin opt standoff_start = start > -# builtin opt standoff_end = end > -# config opt prefix = /ufs/mk/monet5//Linux > -# config opt config = ${prefix}/etc/monetdb5.conf > -# config opt prefix = /ufs/mk/monet5//Linux > -# config opt exec_prefix = ${prefix} > -# config opt gdk_dbfarm = ${prefix}/var/MonetDB5/dbfarm > -# config opt gdk_dbname = demo > -# config opt gdk_alloc_map = no > -# config opt gdk_embedded = no > -# config opt gdk_debug = 0 > -# config opt monet_mod_path = ${exec_prefix}/lib/MonetDB5:${exec_prefix}/lib/MonetDB5/lib:${exec_prefix}/lib/MonetDB5/bin > -# config opt monet_daemon = no > -# config opt monet_welcome = yes > -# config opt mero_msglog = ${prefix}/var/log/merovingian.log > -# config opt mero_errlog = ${prefix}/var/log/merovingian.log > -# config opt mero_timeinterval = 600 > -# config opt mero_pidfile = ${prefix}/var/run/merovingian.pid > -# config opt mero_exittimeout = 7 > -# config opt mero_doproxy = yes > -# config opt mero_discoveryttl = 600 > -# config opt mal_init = ${exec_prefix}/lib/MonetDB5/mal_init.mal > -# config opt mal_listing = 2 > -# config opt checkpoint_dir = ${prefix}/var/MonetDB5/chkpnt > -# config opt mapi_port = 50000 > -# config opt mapi_open = false > -# config opt sql_logdir = ${prefix}/var/MonetDB5/sql_logs > -# config opt sql_init = ${exec_prefix}/lib/MonetDB5/sql_init.sql > -# cmdline opt config = /ufs/mk/monet5//Linux/etc/monetdb5.conf > -# cmdline opt gdk_nr_threads = 0 > -# cmdline opt monet_mod_path = /ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin > -# cmdline opt gdk_dbfarm = /ufs/mk/monet5//Linux/var/MonetDB5/dbfarm > -# cmdline opt sql_logdir = /ufs/mk/monet5//Linux/var/MonetDB5/sql_logs > -# cmdline opt mapi_open = true > -# cmdline opt xrpc_open = true > -# cmdline opt mapi_port = 30087 > -# cmdline opt xrpc_port = 43368 > -# cmdline opt xrpc_docroot = /ufs/mk/monet5/package/MonetDB5 > -# cmdline opt monet_prompt = > -# cmdline opt gdk_dbname = mTests_src_optimizer > -#warning: please don't forget to set your vault key! > -#(see /ufs/mk/monet5//Linux/etc/monetdb5.conf) > - > -# 09:23:28 > > -# 09:23:28 > Done. > -# 09:23:28 > > - > diff --git a/monetdb5/optimizer/Tests/groupderive.stable.out b/monetdb5/optimizer/Tests/groupderive.stable.out > deleted file mode 100644 > --- a/monetdb5/optimizer/Tests/groupderive.stable.out > +++ /dev/null > @@ -1,38 +0,0 @@ > -stdout of test 'groupderive` in directory 'optimizer` itself: > - > - > -# 09:23:28 > > -# 09:23:28 > mserver5 "--config=/ufs/mk/monet5//Linux/etc/monetdb5.conf" --debug=10 --set gdk_nr_threads=0 --set "monet_mod_path=/ufs/mk/monet5//Linux/lib/MonetDB5:/ufs/mk/monet5//Linux/lib/MonetDB5/lib:/ufs/mk/monet5//Linux/lib/MonetDB5/bin" --set "gdk_dbfarm=/ufs/mk/monet5//Linux/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/mk/monet5//Linux/var/MonetDB5/sql_logs" --set mapi_open=true --set xrpc_open=true --set mapi_port=30087 --set xrpc_port=43368 --set xrpc_docroot=/ufs/mk/monet5/package/MonetDB5 --set monet_prompt= --trace --dbname=mTests_src_optimizer groupderive.mal > -# 09:23:28 > > - > -# MonetDB server v5.10.1, based on kernel v1.28.1 > -# Serving database 'mTests_src_optimizer', using 4 threads > -# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > -# Copyright (c) 1993-July 2008 CWI. > -# Copyright (c) August 2008- MonetDB B.V., all rights reserved > -# Visit http://monetdb.cwi.nl/ for further information > -# Listening for connection requests on mapi:monetdb://eir.ins.cwi.nl:30087/ > -function user.tst():void; > - b := bat.new(:oid,:int); > - c := bat.new(:oid,:str); > - d := bat.new(:oid,:flt); > - (ext1,grp1) := group.new(b); > - (ext2,grp2) := group.derive(ext1,grp1,c); > - (ext3,grp3) := group.derive(ext2,grp2,d); > -end tst; > -function user.main():void; > - mdb.list("user","tst"); > -end main; > -function user.tst():void; > - b := bat.new(:oid,:int); > - c := bat.new(:oid,:str); > - d := bat.new(:oid,:flt); > - (ext1,grp1) := group.derivePath(b); > - (ext2,grp2) := group.derivePath(b,c); > - (ext3,grp3) := group.derivePath(b,c,d); > -end tst; > - > -# 09:23:28 > > -# 09:23:28 > Done. > -# 09:23:28 > > - > diff --git a/monetdb5/optimizer/opt_derivepath.mx b/monetdb5/optimizer/opt_derivepath.mx > deleted file mode 100644 > --- a/monetdb5/optimizer/opt_derivepath.mx > +++ /dev/null > @@ -1,239 +0,0 @@ > -@/ > -The contents of this file are subject to the MonetDB Public License > -Version 1.1 (the "License"); you may not use this file except in > -compliance with the License. You may obtain a copy of the License at > -http://www.monetdb.org/Legal/MonetDBLicense > - > -Software distributed under the License is distributed on an "AS IS" > -basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > -License for the specific language governing rights and limitations > -under the License. > - > -The Original Code is the MonetDB Database System. > - > -The Initial Developer of the Original Code is CWI. > -Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > -Copyright August 2008-2012 MonetDB B.V. > -All Rights Reserved. > -@ > - > -@f opt_derivepath > - > -@c > -/* > - * @- Group derive paths > - * The routine @sc{optimizer.derivepaths()} > - * walks through the program looking for grouping operations > - * and cascades them into a single multiple group paths. > - * To illustrate, consider > - * @verbatim > - * (ext1,grp1) := group.new(b); > - * (ext2,grp2) := group.derive(ext1,grp1, c); > - * (ext3,grp3) := group.done(ext2,grp2, d); > - * @end verbatim > - * The result becomes. > - * @verbatim > - * (ext3,grp3) := group.derivepath(b,c,d); > - * @end verbatim > - * > - * The implementation of this operator can freely re-order the BATs > - * for reduced intermediate results or as a basis for parallel scanning > - * all BATs involved to derive their group id. > - * > - * The collection can be extended to immediately aim for group counting > - * or summation, avoiding the materialisation of the group id table. > - */ > -@mal > -pattern optimizer.derivePath():str > -address OPTderivePath; > -pattern optimizer.derivePath(mod:str, fcn:str):str > -address OPTderivePath > -comment "Join path constructor"; > - > -pattern group.derivePath(l:bat[:oid,:any]...)(grp:bat[:oid,:any],ext:bat[:oid,:any]) > -address ALGderivePath > -comment "Derive a group index."; > - > -pattern group.deriveCount(l:bat[:oid,:any]...):bat[:oid,:wrd] > -address ALGderiveCount > -comment "Derive a group count."; > - > -pattern group.deriveSum(s:bat[:oid,:int],l:bat[:oid,:any]...):bat[:oid,:int] > -address ALGderiveSum > -comment "Derive a group sum."; > -pattern group.deriveSum(s:bat[:oid,:lng],l:bat[:oid,:any]...):bat[:oid,:lng] > -address ALGderiveSum > -comment "Derive a group sum."; > -pattern group.deriveSum(s:bat[:oid,:flt],l:bat[:oid,:any]...):bat[:oid,:flt] > -address ALGderiveSum > -comment "Derive a group sum."; > -pattern group.deriveSum(s:bat[:oid,:dbl],l:bat[:oid,:any]...):bat[:oid,:dbl] > -address ALGderiveSum > -comment "Derive a group sum."; > -@h > -#ifndef _OPT_DERIVEPATH_ > -#define _OPT_DERIVEPATH_ > -#include "opt_prelude.h" > -#include "opt_support.h" > -#include "mal_interpreter.h" > - > -opt_export str ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci); > -opt_export str ALGderiveCount(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci); > -opt_export str ALGderiveSum(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci); > - > -#define OPTDEBUGderivePath if ( optDebug & ((lng)1 <<DEBUG_OPT_DERIVEPATH) ) > - > -@c > -#include "monetdb_config.h" > -#include "opt_derivepath.h" > -#include "group.h" > - > -static int > -OPTderivePathImplementation(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr p) > -{ > - int i, actions=0; > - int *pc; > - InstrPtr q; > - InstrPtr *old; > - int limit,slimit; > - Lifespan span; > - > - > - > - (void) cntxt; > - (void) stk; > - span= setLifespan(mb); > - if( span == NULL) > - return 0; > - if (varGetProp(mb, getArg(mb->stmt[0], 0), inlineProp) != NULL) > - return 0; > - > - /* beware, new variables and instructions are introduced */ > - pc= (int*) GDKzalloc(sizeof(int)* mb->vtop * 2); /* to find last assignment */ > - if ( pc == NULL) > - return 0; > - > - old= mb->stmt; > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] MonetDB: sciql - Be more correct with the "fixed" property of th...
by Stefan Manegold 07 Mar '12

07 Mar '12

Jennie, this checkin appear to trigger different output for the following tests: sql/test/BugDay_2005-11-09_2.8/Tests/ORDER_BY_evaluation_error.SF-1023658 sql/test/BugTracker-2009/Tests/join_topn.SF-2654133 sql/test/BugTracker-2010/Tests/offset_limited_32bit.SF-2950579 sql/test/BugTracker/Tests/like_exp.SF-1613949 sql/test/bugs/Tests/select_orderby_alias-bug-sf-1024615 Could you please check, whether the new output is the desired and correct one, and if so, approve it, or otherwise check whether you can fix the code? No hurry. I just happend to notice this and hence do report it ;-) Stefan On Tue, Mar 06, 2012 at 05:42:18PM +0100, Jennie Zhang wrote: > Changeset: a27b334ae151 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a27b334ae151 > Modified Files: > sql/backends/monet5/sql.mx > sql/server/rel_schema.c > sql/server/sql_mvc.c > sql/storage/sql_storage.h > sql/storage/store.c > sql/test/sciql/Tests/01_create_01.sql > sql/test/sciql/Tests/01_create_01.stable.out > Branch: sciql > Log Message: > > Be more correct with the "fixed" property of the SQL catalog > > renamed "fixed" into "fixed_array" in the SQL catalog and set it value to FALSE > (i.s.o. TRUE) for all tables. > > added tests to check if "fixed_array" and "nr_dimensions" in the SQL catalog > are computed correctly. apporved new stable.out > > > diffs (251 lines): > > diff --git a/sql/backends/monet5/sql.mx b/sql/backends/monet5/sql.mx > --- a/sql/backends/monet5/sql.mx > +++ b/sql/backends/monet5/sql.mx > @@ -1881,7 +1881,7 @@ create_table_or_view( mvc *sql, char *sn > } > } > > - nt = sql_trans_create_table(sql->session->tr, s, t->base.name, t->query, t->type, t->system, temp, t->commit_action, t->sz, &t->fixed, &t->ndims); > + nt = sql_trans_create_table(sql->session->tr, s, t->base.name, t->query, t->type, t->system, temp, t->commit_action, t->sz, t->fixed, t->ndims); > > for (n = t->columns.set->h; n; n = n->next) { > sql_column *c = n->data; > diff --git a/sql/server/rel_schema.c b/sql/server/rel_schema.c > --- a/sql/server/rel_schema.c > +++ b/sql/server/rel_schema.c > @@ -729,7 +729,9 @@ create_column(mvc *sql, symbol *s, sql_s > cs->dim->step = GDKstrdup(""); > cs->dim->stop = GDKstrdup(""); > } > - t->fixed = isFixedDim(cs->dim); > + if (!(isFixedDim(cs->dim))) > + t->fixed = 0; > + > /* TODO: the case "ARRAY dim_range_list" is not dealt with */ > } > if (column_options(sql, opt_list, ss, t, cs) == SQL_ERR) > @@ -992,6 +994,8 @@ rel_create_table(mvc *sql, sql_schema *s > int i = 0, j = 0, cnt = 0, *N = NULL, *M = NULL; > lng cntall = 1; > > + if (isArray(t)) > + t->fixed = 1; > for (n = columns->h; n; n = n->next) { > symbol *sym = n->data.sym; > int res = table_element(sql, sym, s, t, 0); > diff --git a/sql/server/sql_mvc.c b/sql/server/sql_mvc.c > --- a/sql/server/sql_mvc.c > +++ b/sql/server/sql_mvc.c > @@ -87,7 +87,7 @@ mvc_init(char *dbname, int debug, store_ > mvc_create_column_(m, t, "system", "boolean", 1); > mvc_create_column_(m, t, "commit_action", "smallint", 16); > mvc_create_column_(m, t, "readonly", "boolean", 1); > - mvc_create_column_(m, t, "fixed", "boolean", 1); > + mvc_create_column_(m, t, "fixed_array", "boolean", 1); > mvc_create_column_(m, t, "nr_dimensions", "int", 32); > mvc_create_column_(m, t, "temporary", "smallint", 16); > > @@ -124,7 +124,7 @@ mvc_init(char *dbname, int debug, store_ > mvc_create_column_(m, t, "system", "boolean", 1); > mvc_create_column_(m, t, "commit_action", "smallint", 16); > mvc_create_column_(m, t, "readonly", "boolean", 1); > - mvc_create_column_(m, t, "fixed", "boolean", 1); > + mvc_create_column_(m, t, "fixed_array", "boolean", 1); > mvc_create_column_(m, t, "nr_dimensions", "int", 32); > mvc_create_column_(m, t, "temporary", "smallint", 16); > if (catalog_version) { > @@ -1029,13 +1029,13 @@ mvc_create_table(mvc *m, sql_schema *s, > tt == tt_table? "TABLE":(tt == tt_array? "ARRAY":"OTHER_TT"), > s->base.name, name, tt, system, persistence, commit_action); > > - /* FIXME: are the NULLs correct values for 'fixed' and 'ndims'? */ > if (persistence == SQL_DECLARED_TABLE && (!s || strcmp(s->base.name, dt_schema))) { > /* declared tables should not end up in the catalog */ > - t = create_sql_table(m->sa, name, tt, system, persistence, commit_action, NULL, NULL); > + /* actual values of 'fixed' and 'ndims' are computed later */ > + t = create_sql_table(m->sa, name, tt, system, persistence, commit_action, 0, 0); > t->s = s; > } else { > - t = sql_trans_create_table(m->session->tr, s, name, NULL, tt, system, persistence, commit_action, sz, NULL, NULL); > + t = sql_trans_create_table(m->session->tr, s, name, NULL, tt, system, persistence, commit_action, sz, 0, 0); > } > return t; > } > @@ -1049,11 +1049,11 @@ mvc_create_view(mvc *m, sql_schema *s, c > fprintf(stderr, "#mvc_create_view %s %s %s\n", s->base.name, name, sql); > > if (persistence == SQL_DECLARED_TABLE) { > - t = create_sql_table(m->sa, name, tt_view, system, persistence, 0, NULL, NULL); > + t = create_sql_table(m->sa, name, tt_view, system, persistence, 0, 0, 0); > t->s = s; > t->query = sa_strdup(m->sa, sql); > } else { > - t = sql_trans_create_table(m->session->tr, s, name, sql, tt_view, system, SQL_PERSIST, 0, 0, NULL, NULL); > + t = sql_trans_create_table(m->session->tr, s, name, sql, tt_view, system, SQL_PERSIST, 0, 0, 0, 0); > } > return t; > } > @@ -1067,11 +1067,11 @@ mvc_create_remote(mvc *m, sql_schema *s, > fprintf(stderr, "#mvc_create_remote %s %s %s\n", s->base.name, name, loc); > > if (persistence == SQL_DECLARED_TABLE) { > - t = create_sql_table(m->sa, name, tt_remote, 0, persistence, 0, NULL, NULL); > + t = create_sql_table(m->sa, name, tt_remote, 0, persistence, 0, 0, 0); > t->s = s; > t->query = sa_strdup(m->sa, loc); > } else { > - t = sql_trans_create_table(m->session->tr, s, name, loc, tt_remote, 0, SQL_REMOTE, 0, 0, NULL, NULL); > + t = sql_trans_create_table(m->session->tr, s, name, loc, tt_remote, 0, SQL_REMOTE, 0, 0, 0, 0); > } > return t; > } > @@ -1084,7 +1084,7 @@ mvc_create_generated(mvc *m, sql_schema > if (mvc_debug) > fprintf(stderr, "#mvc_create_generated %s %s %s\n", s->base.name, name, sql); > > - t = sql_trans_create_table(m->session->tr, s, name, sql, tt_generated, system, SQL_PERSIST, 0, 0, NULL, NULL); > + t = sql_trans_create_table(m->session->tr, s, name, sql, tt_generated, system, SQL_PERSIST, 0, 0, 0, 0); > return t; > } > > diff --git a/sql/storage/sql_storage.h b/sql/storage/sql_storage.h > --- a/sql/storage/sql_storage.h > +++ b/sql/storage/sql_storage.h > @@ -314,7 +314,7 @@ extern void reset_functions(sql_trans *t > extern sql_schema *sql_trans_create_schema(sql_trans *tr, char *name, int auth_id, int owner); > extern void sql_trans_drop_schema(sql_trans *tr, int id, int drop_action); > > -extern sql_table *sql_trans_create_table(sql_trans *tr, sql_schema *s, char *name, char *sql, int tt, bit system, int persistence, int commit_action, int sz, bit *fixed, int *ndims); > +extern sql_table *sql_trans_create_table(sql_trans *tr, sql_schema *s, char *name, char *sql, int tt, bit system, int persistence, int commit_action, int sz, bit fixed, int ndims); > extern sql_table *sql_trans_add_table(sql_trans *tr, sql_table *mt, sql_table *pt); > extern sql_table *sql_trans_del_table(sql_trans *tr, sql_table *mt, sql_table *pt, int drop_action); > > @@ -368,7 +368,7 @@ extern int sql_trans_connect_catalog(sql > extern int sql_trans_disconnect_catalog(sql_trans *tr, char *db_alias); > extern int sql_trans_disconnect_catalog_ALL(sql_trans *tr); > > -extern sql_table *create_sql_table(sql_allocator *sa, char *name, sht type, bit system, int persistence, int commit_action, bit *fixed, int *ndims); > +extern sql_table *create_sql_table(sql_allocator *sa, char *name, sht type, bit system, int persistence, int commit_action, bit fixed, int ndims); > extern sql_column *create_sql_column(sql_allocator *sa, sql_table *t, char *name, sql_subtype *tpe); > extern sql_ukey *create_sql_ukey(sql_allocator *sa, sql_table *t, char *nme, key_type kt); > extern sql_fkey *create_sql_fkey(sql_allocator *sa, sql_table *t, char *nme, key_type kt, sql_key *rkey, int on_delete, int on_update ); > diff --git a/sql/storage/store.c b/sql/storage/store.c > --- a/sql/storage/store.c > +++ b/sql/storage/store.c > @@ -578,7 +578,7 @@ load_table(sql_trans *tr, sql_schema *s, > t->cleared = 0; > v = table_funcs.column_find_value(tr, find_sql_column(tables, "readonly"),rid); > t->readonly = *(bit *)v; _DELETE(v); > - v = table_funcs.column_find_value(tr, find_sql_column(tables, "fixed"),rid); > + v = table_funcs.column_find_value(tr, find_sql_column(tables, "fixed_array"),rid); > t->fixed = *(bit *)v; _DELETE(v); > v = table_funcs.column_find_value(tr, find_sql_column(tables, "nr_dimensions"),rid); > t->ndims = *(int *)v; _DELETE(v); > @@ -1127,7 +1127,7 @@ bootstrap_create_column(sql_trans *tr, s > } > > sql_table * > -create_sql_table(sql_allocator *sa, char *name, sht type, bit system, int persistence, int commit_action, bit *fixed, int *ndims) > +create_sql_table(sql_allocator *sa, char *name, sht type, bit system, int persistence, int commit_action, bit fixed, int ndims) > { > sql_table *t = SA_ZNEW(sa, sql_table); > > @@ -1149,9 +1149,9 @@ create_sql_table(sql_allocator *sa, char > t->pkey = NULL; > t->sz = COLSIZE; > t->cleared = 0; > - /* 'fixed' and 'ndims' are ONLY used for arrays. Pass NULLs to get their defaults. */ > - t->fixed = fixed ? *fixed : 1; /* should be set to 0 if an unbounded dimension is found. */ > - t->ndims = ndims ? *ndims : 0; > + /* 'fixed' and 'ndims' are ONLY used for arrays. For tables, their values arre always 0. */ > + t->fixed = isArray(t) ? fixed : 0; /* should be set to 1 for fixed arrays. */ > + t->ndims = isArray(t) ? ndims : 0; > t->s = NULL; > return t; > } > @@ -1188,7 +1188,7 @@ sql_table * > dup_sql_table(sql_allocator *sa, sql_table *t) > { > node *n; > - sql_table *nt = create_sql_table(sa, t->base.name, t->type, t->system, SQL_DECLARED_TABLE, t->commit_action, &t->fixed, &t->ndims); > + sql_table *nt = create_sql_table(sa, t->base.name, t->type, t->system, SQL_DECLARED_TABLE, t->commit_action, t->fixed, t->ndims); > > for (n = t->columns.set->h; n; n = n->next) > dup_sql_column(sa, nt, n->data); > @@ -1223,7 +1223,7 @@ bootstrap_create_table(sql_trans *tr, sq > int istmp = isTempSchema(s); > int persistence = istmp?SQL_GLOBAL_TEMP:SQL_PERSIST; > sht commit_action = istmp?CA_PRESERVE:CA_COMMIT; > - sql_table *t = create_sql_table(tr->sa, name, tt_table, 1, persistence, commit_action, NULL, NULL /* we never have arrays here */); > + sql_table *t = create_sql_table(tr->sa, name, tt_table, 1, persistence, commit_action, 0, 0 /* we never have arrays here */); > > if (bs_debug) > fprintf(stderr, "#bootstrap_create_table %s\n", name ); > @@ -1411,7 +1411,7 @@ store_init(int debug, store_type store, > bootstrap_create_column(tr, t, "system", "boolean", 1); > bootstrap_create_column(tr, t, "commit_action", "smallint", 16); > bootstrap_create_column(tr, t, "readonly", "boolean", 1); > - bootstrap_create_column(tr, t, "fixed", "boolean", 1); > + bootstrap_create_column(tr, t, "fixed_array", "boolean", 1); > bootstrap_create_column(tr, t, "nr_dimensions", "int", 32); > > t = bootstrap_create_table(tr, s, "_columns"); > @@ -3863,7 +3863,7 @@ sql_trans_del_table(sql_trans *tr, sql_t > } > > sql_table * > -sql_trans_create_table(sql_trans *tr, sql_schema *s, char *name, char *sql, int tt, bit system, int persistence, int commit_action, int sz, bit *fixed, int *ndims) > +sql_trans_create_table(sql_trans *tr, sql_schema *s, char *name, char *sql, int tt, bit system, int persistence, int commit_action, int sz, bit fixed, int ndims) > { > sql_table *t = create_sql_table(tr->sa, name, tt, system, persistence, commit_action, fixed, ndims); > sql_schema *syss = find_sql_schema(tr, isGlobal(t)?"sys":"tmp"); > diff --git a/sql/test/sciql/Tests/01_create_01.sql b/sql/test/sciql/Tests/01_create_01.sql > --- a/sql/test/sciql/Tests/01_create_01.sql > +++ b/sql/test/sciql/Tests/01_create_01.sql > @@ -1,5 +1,16 @@ > -- use the [size] shortcut for integer type dimensions > CREATE ARRAY ary(x TINYINT DIMENSION[4], y BIGINT DIMENSION[-5], v FLOAT DEFAULT 3.7); > SELECT * FROM ary; > + > +CREATE ARRAY ary1(x TINYINT DIMENSION, y BIGINT DIMENSION, v FLOAT DEFAULT 3.7); > +SELECT * FROM ary1; > + > +CREATE ARRAY ary2(x TINYINT DIMENSION[4], y BIGINT DIMENSION, z BIGINT DIMENSION[-5], v FLOAT DEFAULT 3.7); > +SELECT * FROM ary2; > + > +SELECT * FROM _tables WHERE type = 7 OR fixed_array = TRUE; > + > DROP ARRAY ary; > +DROP ARRAY ary1; > +DROP ARRAY ary2; > > diff --git a/sql/test/sciql/Tests/01_create_01.stable.out b/sql/test/sciql/Tests/01_create_01.stable.out > --- a/sql/test/sciql/Tests/01_create_01.stable.out > +++ b/sql/test/sciql/Tests/01_create_01.stable.out > @@ -49,6 +49,26 @@ Ready. > [ 3, -2, 3.7 ] > [ 3, -3, 3.7 ] > [ 3, -4, 3.7 ] > +#CREATE ARRAY ary1(x TINYINT DIMENSION, y BIGINT DIMENSION, v FLOAT DEFAULT 3.7); > +#SELECT * FROM ary1; > +% sys.ary1, sys.ary1, sys.ary1 # table_name > +% x, y, v # name > +% tinyint, bigint, double # type > +% 1, 1, 24 # length > +#CREATE ARRAY ary2(x TINYINT DIMENSION[4], y BIGINT DIMENSION, z BIGINT DIMENSION[-5], v FLOAT DEFAULT 3.7); > +#SELECT * FROM ary2; > +% sys.ary2, sys.ary2, sys.ary2, sys.ary2 # table_name > +% x, y, z, v # name > +% tinyint, bigint, bigint, double # type > +% 1, 1, 1, 24 # length > +#SELECT * FROM _tables WHERE type = 7 OR fixed_array = TRUE; > +% sys._tables, sys._tables, sys._tables, sys._tables, sys._tables, sys._tables, sys._tables, sys._tables, sys._tables, sys._tables # table_name > +% id, name, schema_id, query, type, system, commit_action, readonly, fixed_array, nr_dimensions # name > +% int, varchar, int, varchar, smallint, boolean, smallint, boolean, boolean, int # type > +% 4, 4, 4, 0, 1, 5, 1, 5, 5, 1 # length > +[ 6461, "ary", 2000, NULL, 7, false, 0, false, true, 2 ] > +[ 6466, "ary1", 2000, NULL, 7, false, 0, false, false, 2 ] > +[ 6472, "ary2", 2000, NULL, 7, false, 0, false, false, 3 ] > #DROP ARRAY ary; > > # 13:50:46 > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 1

[Monetdb-developers] New Apr2012 stable branch created
by Sjoerd Mullender 29 Feb '12

29 Feb '12

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Apr2012. So in each of your stable checkout directories you can do hg pull hg update -rApr2012 to update to this new branch. -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New Dec2011-SP1 Bugfix release of MonetDB suite
by Fabian Groffen 29 Feb '12

29 Feb '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2011-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2011-SP1 bugfix release Build Environment * Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case. * On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997. SQL * Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient. * Stripped off implementation-specific parts from error messages before they get presented to the user. Java Module * Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument. * Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages MonetDB5 Server * Fixed potential crash by dealing better with non-standard types. Merovingian * Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries. * Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982. * Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983. * Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984. * Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984. Bug Fixes * 2862: foreign key self references cause violation upon alter table add constraint... * 2933: "TypeException: algebra.join undefined" when evaluating EXISTS (SELECT ...) predicate * 2946: Invalid BAT if left join condition has only columns from right table * 2964: LEFT OUTER JOIN with constants returns errors * 2969: Result precision issue with Decimal data type * 2973: Date coercion functionality tests * 2979: misc merovingian control problems * 2982: monetdbd status and mserver properties have random values * 2983: monetdbd attempts to stop non-existing database 'control' * 2984: monetdb create/release/start/stop returns error "OK" for remote connections * 2995: Test geom/sql/Tests/example.sql crashes since changeset 6399077ff8a4 * 2997: monetdb-python packages installed in wrong dir on Ubuntu 10.04 LTS (lucid) * 3004: Incorrect evaluation of conditions in WHERE part of SQL statement * 3009: Segmentation fault on simple query * 3013: calc.and is not yet defined for dbl * 3014: dayofweek comment is wrong in mtime.mx:700 * 3017: rel_bin.c:611: exp_bin: Assertion `0' failed. with between on multicolumns * 3020: Assertion error in debugging mode * 3034: INNER JOIN with Subquery produces mal errors * 3041: WHERE condition ignored after 2 LEFT JOINs with subqueries

1 0

Re: [Monetdb-developers] MonetDB: default - make sure we reset the point_query flag
by Sjoerd Mullender 23 Feb '12

23 Feb '12

Would this also be appropriate for the Dec2011 branch? On 2012-02-23 14:22, Niels Nes wrote: > Changeset: 656d66d54802 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=656d66d54802 > Modified Files: > sql/backends/monet5/sql.mx > Branch: default > Log Message: > > make sure we reset the point_query flag > > > diffs (11 lines): > > diff --git a/sql/backends/monet5/sql.mx b/sql/backends/monet5/sql.mx > --- a/sql/backends/monet5/sql.mx > +++ b/sql/backends/monet5/sql.mx > @@ -1694,6 +1694,7 @@ sqlcleanup(mvc *c, int err) > if (err < 0) > c->session->status = err; > c->label = 0; > + c->point_query = 0; > scanner_query_processed(&(c->scanner)); > return err; > } > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

2 2

Re: [Monetdb-developers] MonetDB: sciql - rel_select.c, rel_arraytiling(): we do not (yet...
by Ying Zhang 23 Feb '12

23 Feb '12

On Thu, Feb 23, 2012 at 11:06:55AM +0100, Stefan Manegold wrote: > Changeset: 869144f7d945 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=869144f7d945 > Modified Files: > sql/server/rel_select.c > Branch: sciql > Log Message: > > rel_select.c, rel_arraytiling(): we do not (yet?) support tiling on subset of dimensions No, it's not yet supported. > > > diffs (12 lines): > > diff --git a/sql/server/rel_select.c b/sql/server/rel_select.c > --- a/sql/server/rel_select.c > +++ b/sql/server/rel_select.c > @@ -1910,6 +1910,8 @@ rel_arraytiling(mvc *sql, sql_rel **rel, > idx_exps = tile_def->data.lval->h->next->data.lval; > if (dlist_length(idx_exps) > a->ndims) > return sql_error(sql, 02, "SELECT: #dimensions (%d) in array tiling larger than #dimensions (%d) in the array", dlist_length(idx_exps), a->ndims); > + if (dlist_length(idx_exps) < a->ndims) > + return sql_error(sql, 02, "SELECT: #dimensions (%d) in array tiling fewer than #dimensions (%d) in the array --- not supported (yet?)", dlist_length(idx_exps), a->ndims); > if (dlist_length(idx_exps) > ARRAY_TILING_MAX_DIMS) > return sql_error(sql, 02, "SELECT: TODO: array tiling over >%d dimensions", ARRAY_TILING_MAX_DIMS); > for (n = idx_exps->h; n; n = n->next) { > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

1 0

Re: [Monetdb-developers] MonetDB: default - test opt_sql_append: trying to upgrade to new...
by Martin Kersten 18 Feb '12

18 Feb '12

The new optimizer infrastructure returns the name of the pipe, not its content. The definitions can be obtained from the system view: select * from optimizers(); can be used to get the expanded form. On 2/18/12 6:11 PM, Stefan Manegold wrote: > Changeset: 62a52aaf4241 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=62a52aaf4241 > Modified Files: > monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.sql > monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > Branch: default > Log Message: > > test opt_sql_append: trying to upgrade to new optimizer framework > > However, setting SQL variable "optimizer" to a MAL snippert > rather than an optimizer pipeline name does not seem to work (yet?) ... > > > diffs (39 lines): > > diff --git a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.sql b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.sql > --- a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.sql > +++ b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.sql > @@ -1,7 +1,9 @@ > create table ttt (a int, b int, c int); > select optimizer; > explain copy into ttt from '/tmp/xyz'; > -set optimizer = substring(optimizer,0,length(optimizer)-length('garbageCollector')) || 'sql_append,garbageCollector'; > +declare opt_pipe_name string; set opt_pipe_name = ( select optimizer ); > +declare opt_pipe_def string; set opt_pipe_def = ( select def from optimizers() where name = opt_pipe_name ); > +set optimizer = substring(opt_pipe_def,0,length(opt_pipe_def)-length('optimizer.garbageCollector();')) || 'optimizer.sql_append();optimizer.garbageCollector();'; > select optimizer; > explain copy into ttt from '/tmp/xyz'; > drop table ttt; > diff --git a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > --- a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > +++ b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > @@ -49,8 +49,8 @@ Ready. > % . # table_name > % single_value # name > % varchar # type > -% 188 # length > -[ "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,multiplex,accumulators,garbageCollector" ] > +% 12 # length > +[ "default_pipe" ] > #explain copy into ttt from '/tmp/xyz'; > % .explain # table_name > % mal # name > @@ -69,8 +69,8 @@ end s2_1; > % . # table_name > % single_value # name > % varchar # type > -% 199 # length > -[ "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,multiplex,accumulators,sql_append,garbageCollector" ] > +% 452 # length > +[ "optimizer.inline();optimizer.remap();optimizer.evaluate();optimizer.costModel();optimizer.coercions();optimizer.emptySet();optimizer.aliases();optimizer.mitosis();optimizer.mergetable();optimizer.deadcode();optimizer.commonTerms();optimizer.joinPath();optimizer.reorder();optimizer.deadcode();optimizer.reduce();optimizer.dataflow();optimizer.history();optimizer.multiplex();optimizer.accumulators();optimizer.sql_append();optimizer.garbageCollector();" ] > #explain copy into ttt from '/tmp/xyz'; > % .explain # table_name > % mal # name > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

2 1

[Monetdb-developers] How to embed MonetDB
by Florin Ciodaru 17 Feb '12

17 Feb '12

Hi all, First I want to congratulate all of you for this work. Now, we are trying to build a "real-time" like application that on one side periodically receives sets of matrices and on the other side receives query to apply on this matrices - Each set of matrices can have thousands of elements o each of them belongs to a certain class, hence groups of them will need to be inserted in the same monetdb table o each matrix has ~20 columns and the number of rows can grow to 128k - The queries can be o One-time o or , continuous (to be applied on each new set of matrices) Looking for stream processing engine, we reached DataCell and MonetDB. We downloaded the Dec2011 source code from Mercurial depot. Unfortunately, we could not yet figure out how to bring DataCell up and running, although we successfully built it. Maybe you can gives us more pointers than what already exists on the site. In parallel we are trying to experiment with MonetDB functionality as a library to be embedded in our own host (e.g. windows application). The basic functions we want from this library are - Fast low level API's to create tables and populate them with values - High level API's to execute SQL queries For the insertions we figure out the API's in sql_storage.h are close to what we want. However we could not yet find API's that allow us to execute SQL queries (we did find SQLstatementIntern but cannot yet figure out how to retrieve the results of query execution). Your help will be really appreciated, Florin

1 0

Re: [Monetdb-developers] MonetDB: gdk-calc - gdk_calc.c, mod_typeswitchloop(): initialize...
by Stefan Manegold 16 Feb '12

16 Feb '12

Standard gcc on Fedora 16; MonetDB configured with --disable-debug --enable-optimize --disable-assert -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Reply message ----- From: "Sjoerd Mullender" <sjoerd(a)acm.org> To: <monetdb-developers(a)lists.sourceforge.net> Subject: [Monetdb-developers] MonetDB: gdk-calc - gdk_calc.c, mod_typeswitchloop(): initialize... Date: Thu, Feb 16, 2012 09:18 On 2012-02-15 20:12, Stefan Manegold wrote: > Changeset: fc0d3fad41c2 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fc0d3fad41c2 > Modified Files: > gdk/gdk_calc.c > Branch: gdk-calc > Log Message: > > gdk_calc.c, mod_typeswitchloop(): initialize "nils" (hopefully correctly) Which sucky compiler was this? There is no code path where nils was not initialized. (This also means that the value you choose here doesn't matter.) > > diffs (12 lines): > > diff --git a/gdk/gdk_calc.c b/gdk/gdk_calc.c > --- a/gdk/gdk_calc.c > +++ b/gdk/gdk_calc.c > @@ -6157,7 +6157,7 @@ mod_typeswitchloop(const void *lft, int > void *dst, int tp, BUN cnt, > int abort_on_error, const char *func) > { > - BUN nils; > + BUN nils = BUN_NONE; > > switch (ATOMstorage(tp1)) { > case TYPE_bte: > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] MonetDB: gdk-calc - gdk_calc.c, mod_typeswitchloop(): initialize...
by Sjoerd Mullender 16 Feb '12

16 Feb '12

On 2012-02-15 20:12, Stefan Manegold wrote: > Changeset: fc0d3fad41c2 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fc0d3fad41c2 > Modified Files: > gdk/gdk_calc.c > Branch: gdk-calc > Log Message: > > gdk_calc.c, mod_typeswitchloop(): initialize "nils" (hopefully correctly) Which sucky compiler was this? There is no code path where nils was not initialized. (This also means that the value you choose here doesn't matter.) > > diffs (12 lines): > > diff --git a/gdk/gdk_calc.c b/gdk/gdk_calc.c > --- a/gdk/gdk_calc.c > +++ b/gdk/gdk_calc.c > @@ -6157,7 +6157,7 @@ mod_typeswitchloop(const void *lft, int > void *dst, int tp, BUN cnt, > int abort_on_error, const char *func) > { > - BUN nils; > + BUN nils = BUN_NONE; > > switch (ATOMstorage(tp1)) { > case TYPE_bte: > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

1 1

Re: [Monetdb-developers] MonetDB: default - More advice on the optimizer template.
by Stefan Manegold 11 Feb '12

11 Feb '12

On Sat, Feb 11, 2012 at 02:06:17PM +0100, Martin Kersten wrote: > > > On 2/11/12 11:03 AM, Stefan Manegold wrote: > >On Wed, Feb 08, 2012 at 10:27:11AM +0100, Martin Kersten wrote: > >>Changeset: 67c12a700166 for MonetDB > >>URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=67c12a700166 > >>Modified Files: > >> monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > >>Branch: default > >>Log Message: > >> > >>More advice on the optimizer template. > >> > >> > >>diffs (140 lines): > >> > >>diff --git a/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx b/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > >>--- a/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > >>+++ b/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > >[...] > >>@@ -39,6 +39,8 @@ All Rights Reserved. > >> * i.e., an sql.append() statement that is eventually followed by some other > >> * statement later on in the MAL program that uses the same v0 BAT as > >> * argument as the sql.append() statement does, > >>+ * Do you assume a single re-use of the variable v0? > > > >No. Why? > Use assign-once and use-many-times policy. It can improve parallel > processing > and simplifies scope analysis. v0 is (as far as I know) created (assigned) once (by Niels, or preceeding optimizers). If it is used only once (only by sql_append), my optimizer does not (have to) do anything. Otherwise, it replaces one use v0 (by sql_append) by a view of v0. That's the very purpose of this optimizer. > >>+ * Do you assume a non-nested MAL block ? > > > >Not necessarily. > > > Analysis may become complex if you have something like > > V0:= expr > barrier E1:=expr > V0:= expr2 > exit E1 > now V0 depends on runtime use > > > same holds for > barrier E1:= expr > V0:=expr > exit E1 > z:= f(V0) > > will be flagged as an error because V0 may be uninitialized > > >I must admit, that I do not know how the oprimizer framework handles nested > >MAL blocks, and what an optimizer needs to do to be aware of nested MAL > >blocks and to handle them correctly. > Preferrably the MAL blocks are linear programs (until you reach the > dataflow optimizer). How do I know / see that in my optimizer? Do I have to check for barrier / exit statements / constructs myself? > > > >In the sample optimizer, for now, I'd be fine if there are no > >false-positives, i.e., the optimizer triggers in case it should not trigger > >or in cases it cannot handle correctly. > >I can accept false-negatives, i.e., not triggering in all case it could handle > >correctly. > > > >> * > >> * and transform them into > >> * > >>@@ -52,6 +54,7 @@ All Rights Reserved. > >> * > >> * i.e., handing a BAT view v2 of BAT v0 as argument to the sql.append() > >> * statement, rather than the original BAT v0. > >>+ * My advice, always use new variable names, it may capture some easy to make errors. > > > >I/my optimizer does use new variables for all new statements/results. > >I/my optimizer re-use variable names only for identical results. > > > >> * > >> * As a refinement, patterns like > >> * > >[...] > >>@@ -181,13 +195,17 @@ OPTsql_appendImplementation(Client cntxt > >> pushInstruction(mb, q); > >> q1 = q; > >> i++; > >>- actions++; > >>+ actions++; /* to keep track if anything has been done */ > >> } > >> } > >> > >>- /* look for > >>+ /* look for > >> * v5 := ... v0 ...; > >> */ > >>+ /* an expensive loop, better would be to remember that v0 has a different role. > >>+ * A typical method is to keep a map from variable -> instruction where it was > >>+ * detected. The you can check each assignment for use of v0 > >>+ */ > > > >This is general support functionality. > >Is this already available in the optimizer framework? > I try to use single pass algorithms in the optimizers. > Even in the case of commonterms optimizer, we may have to > traverse the history. This can become a n^2 process > > >If so, where is it and how can I use it? > Mimic how it is done in other optimizers (e.g. opt_reorder). > Typically, a buffer is maintained per variable to keep > optimization properties around. > > >If not, where/how could we add it? > > > >> for (j = i+1; !found&& j< limit; j++) > >> for (k = old[j]->retc; !found&& k< old[j]->argc; k++) > >> found = (getArg(old[j], k) == getArg(p, 5)); > >>@@ -202,6 +220,8 @@ OPTsql_appendImplementation(Client cntxt > >> > >> /* push new v1 := aggr.count( v0 ); unless already available */ > >> if (q1 == NULL) { > >>+ /* use mal_buil.mx primitives q1 = newStmt(mb, aggrRef,countRef); setArgType(mb,q1,TYPE_wrd) */ > >>+ /* it will be added to the block and even my re-use MAL instructions */ > > > >Is this (supposed to be) documentation of the existing code below, > >or rather advice how to implement the below functionality differently? > Use the mal_builder to simplify your code base. > > > > >> q1 = newInstruction(mb,ASSIGNsymbol); > >> getArg(q1,0) = newTmpVariable(mb, TYPE_wrd); > >> setModuleId(q1, aggrRef); > >>@@ -211,6 +231,7 @@ OPTsql_appendImplementation(Client cntxt > >> } > >> > >> /* push new v2 := algebra.slice( v0, 0, v1 ); */ > >>+ /* use mal_buil.mx primitives q1 = newStmt(mb, algebraRef,sliceRef); */ > > > >Is this (supposed to be) documentation of the existing code below, > >or rather advice how to implement the below functionality differently? > > > >> q2 = newInstruction(mb,ASSIGNsymbol); > >> getArg(q2,0) = newTmpVariable(mb, TYPE_any); > >> setModuleId(q2, algebraRef); > >>@@ -240,6 +261,7 @@ OPTsql_appendImplementation(Client cntxt > >> for(i++; i<limit; i++) > >> if (old[i]) > >> pushInstruction(mb, old[i]); > >>+ /* any remaining MAL instruction records are removed */ > >> for(; i<slimit; i++) > >> if (old[i]) > >> freeInstruction(old[i]); > >>@@ -253,6 +275,9 @@ OPTsql_appendImplementation(Client cntxt > >> return actions; > >> } > >> > >>+/* optimizers have to be registered in the optcatalog in opt_support.c. > > > >Why? > SQL needs a place to pick up all optimizers known. You may also have > to extend the optimizer pipeline validity code. > > >If at all possible, I'd prefer to be able to add a new optimizer without the > >need to change existing code ... > yes understood, but you have to patch Makefile.ag, youroptimizer.mx, and > opt_support. Possibly, you may have to extend opt_prelude as well > > > > >>+ * you have to path the file accordingly. > "path" > > ^^^^ > >parse? > > > >What does this mean? What am I supposed to do in detail? > > > >>+ */ > >> @include ../../optimizer/optimizerWrapper.mx > >> @c > >> #include "opt_statistics.h" > >>_______________________________________________ > >>Checkin-list mailing list > >>Checkin-list(a)monetdb.org > >>http://mail.monetdb.org/mailman/listinfo/checkin-list > >> > > > >Thanks! > > > >Stefan > > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 1

Re: [Monetdb-developers] MonetDB: default - More advice on the optimizer template.
by Martin Kersten 11 Feb '12

11 Feb '12

>>> + * you have to path the file accordingly. > "path" >> ^^^^ >> parse? patch

1 0

Re: [Monetdb-developers] MonetDB: default - More advice on the optimizer template.
by Stefan Manegold 11 Feb '12

11 Feb '12

On Wed, Feb 08, 2012 at 10:27:11AM +0100, Martin Kersten wrote: > Changeset: 67c12a700166 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=67c12a700166 > Modified Files: > monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > Branch: default > Log Message: > > More advice on the optimizer template. > > > diffs (140 lines): > > diff --git a/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx b/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > --- a/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > +++ b/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx [...] > @@ -39,6 +39,8 @@ All Rights Reserved. > * i.e., an sql.append() statement that is eventually followed by some other > * statement later on in the MAL program that uses the same v0 BAT as > * argument as the sql.append() statement does, > + * Do you assume a single re-use of the variable v0? No. Why? > + * Do you assume a non-nested MAL block ? Not necessarily. I must admit, that I do not know how the oprimizer framework handles nested MAL blocks, and what an optimizer needs to do to be aware of nested MAL blocks and to handle them correctly. In the sample optimizer, for now, I'd be fine if there are no false-positives, i.e., the optimizer triggers in case it should not trigger or in cases it cannot handle correctly. I can accept false-negatives, i.e., not triggering in all case it could handle correctly. > * > * and transform them into > * > @@ -52,6 +54,7 @@ All Rights Reserved. > * > * i.e., handing a BAT view v2 of BAT v0 as argument to the sql.append() > * statement, rather than the original BAT v0. > + * My advice, always use new variable names, it may capture some easy to make errors. I/my optimizer does use new variables for all new statements/results. I/my optimizer re-use variable names only for identical results. > * > * As a refinement, patterns like > * [...] > @@ -181,13 +195,17 @@ OPTsql_appendImplementation(Client cntxt > pushInstruction(mb, q); > q1 = q; > i++; > - actions++; > + actions++; /* to keep track if anything has been done */ > } > } > > - /* look for > + /* look for > * v5 := ... v0 ...; > */ > + /* an expensive loop, better would be to remember that v0 has a different role. > + * A typical method is to keep a map from variable -> instruction where it was > + * detected. The you can check each assignment for use of v0 > + */ This is general support functionality. Is this already available in the optimizer framework? If so, where is it and how can I use it? If not, where/how could we add it? > for (j = i+1; !found && j < limit; j++) > for (k = old[j]->retc; !found && k < old[j]->argc; k++) > found = (getArg(old[j], k) == getArg(p, 5)); > @@ -202,6 +220,8 @@ OPTsql_appendImplementation(Client cntxt > > /* push new v1 := aggr.count( v0 ); unless already available */ > if (q1 == NULL) { > + /* use mal_buil.mx primitives q1 = newStmt(mb, aggrRef,countRef); setArgType(mb,q1,TYPE_wrd) */ > + /* it will be added to the block and even my re-use MAL instructions */ Is this (supposed to be) documentation of the existing code below, or rather advice how to implement the below functionality differently? > q1 = newInstruction(mb,ASSIGNsymbol); > getArg(q1,0) = newTmpVariable(mb, TYPE_wrd); > setModuleId(q1, aggrRef); > @@ -211,6 +231,7 @@ OPTsql_appendImplementation(Client cntxt > } > > /* push new v2 := algebra.slice( v0, 0, v1 ); */ > + /* use mal_buil.mx primitives q1 = newStmt(mb, algebraRef,sliceRef); */ Is this (supposed to be) documentation of the existing code below, or rather advice how to implement the below functionality differently? > q2 = newInstruction(mb,ASSIGNsymbol); > getArg(q2,0) = newTmpVariable(mb, TYPE_any); > setModuleId(q2, algebraRef); > @@ -240,6 +261,7 @@ OPTsql_appendImplementation(Client cntxt > for(i++; i<limit; i++) > if (old[i]) > pushInstruction(mb, old[i]); > + /* any remaining MAL instruction records are removed */ > for(; i<slimit; i++) > if (old[i]) > freeInstruction(old[i]); > @@ -253,6 +275,9 @@ OPTsql_appendImplementation(Client cntxt > return actions; > } > > +/* optimizers have to be registered in the optcatalog in opt_support.c. Why? If at all possible, I'd prefer to be able to add a new optimizer without the need to change existing code ... > + * you have to path the file accordingly. ^^^^ parse? What does this mean? What am I supposed to do in detail? > + */ > @include ../../optimizer/optimizerWrapper.mx > @c > #include "opt_statistics.h" > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > Thanks! Stefan -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Re: [Monetdb-developers] MonetDB: jacqueline - mal: allow command calls up to 16 arguments
by Martin Kersten 28 Jan '12

28 Jan '12

at some point commands should become patterns. There is no limitation on the number of arguments per se. On 1/27/12 10:14 PM, Fabian Groffen wrote: > Changeset: a9fa1c35a273 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a9fa1c35a273 > Modified Files: > monetdb5/mal/mal_interpreter.mx > Branch: jacqueline > Log Message: > > mal: allow command calls up to 16 arguments > > future functions will need more arguments in total (in particular > because each json argument or return requires 7 separate variables) > > > diffs (76 lines): > > diff --git a/monetdb5/mal/mal_interpreter.mx b/monetdb5/mal/mal_interpreter.mx > --- a/monetdb5/mal/mal_interpreter.mx > +++ b/monetdb5/mal/mal_interpreter.mx > @@ -2053,6 +2053,72 @@ safeguardStack(Client cntxt, MalBlkPtr m > (ptr)getArgReference(stk, pci, 10), > (ptr)getArgReference(stk, pci, 11)); > break; > + case 13: ret = (str)(*pci->fcn)( > + (ptr)getArgReference(stk, pci, 0), > + (ptr)getArgReference(stk, pci, 1), > + (ptr)getArgReference(stk, pci, 2), > + (ptr)getArgReference(stk, pci, 3), > + (ptr)getArgReference(stk, pci, 4), > + (ptr)getArgReference(stk, pci, 5), > + (ptr)getArgReference(stk, pci, 6), > + (ptr)getArgReference(stk, pci, 7), > + (ptr)getArgReference(stk, pci, 8), > + (ptr)getArgReference(stk, pci, 9), > + (ptr)getArgReference(stk, pci, 10), > + (ptr)getArgReference(stk, pci, 11), > + (ptr)getArgReference(stk, pci, 12)); > + break; > + case 14: ret = (str)(*pci->fcn)( > + (ptr)getArgReference(stk, pci, 0), > + (ptr)getArgReference(stk, pci, 1), > + (ptr)getArgReference(stk, pci, 2), > + (ptr)getArgReference(stk, pci, 3), > + (ptr)getArgReference(stk, pci, 4), > + (ptr)getArgReference(stk, pci, 5), > + (ptr)getArgReference(stk, pci, 6), > + (ptr)getArgReference(stk, pci, 7), > + (ptr)getArgReference(stk, pci, 8), > + (ptr)getArgReference(stk, pci, 9), > + (ptr)getArgReference(stk, pci, 10), > + (ptr)getArgReference(stk, pci, 11), > + (ptr)getArgReference(stk, pci, 12), > + (ptr)getArgReference(stk, pci, 13)); > + break; > + case 15: ret = (str)(*pci->fcn)( > + (ptr)getArgReference(stk, pci, 0), > + (ptr)getArgReference(stk, pci, 1), > + (ptr)getArgReference(stk, pci, 2), > + (ptr)getArgReference(stk, pci, 3), > + (ptr)getArgReference(stk, pci, 4), > + (ptr)getArgReference(stk, pci, 5), > + (ptr)getArgReference(stk, pci, 6), > + (ptr)getArgReference(stk, pci, 7), > + (ptr)getArgReference(stk, pci, 8), > + (ptr)getArgReference(stk, pci, 9), > + (ptr)getArgReference(stk, pci, 10), > + (ptr)getArgReference(stk, pci, 11), > + (ptr)getArgReference(stk, pci, 12), > + (ptr)getArgReference(stk, pci, 13), > + (ptr)getArgReference(stk, pci, 14)); > + break; > + case 16: ret = (str)(*pci->fcn)( > + (ptr)getArgReference(stk, pci, 0), > + (ptr)getArgReference(stk, pci, 1), > + (ptr)getArgReference(stk, pci, 2), > + (ptr)getArgReference(stk, pci, 3), > + (ptr)getArgReference(stk, pci, 4), > + (ptr)getArgReference(stk, pci, 5), > + (ptr)getArgReference(stk, pci, 6), > + (ptr)getArgReference(stk, pci, 7), > + (ptr)getArgReference(stk, pci, 8), > + (ptr)getArgReference(stk, pci, 9), > + (ptr)getArgReference(stk, pci, 10), > + (ptr)getArgReference(stk, pci, 11), > + (ptr)getArgReference(stk, pci, 12), > + (ptr)getArgReference(stk, pci, 13), > + (ptr)getArgReference(stk, pci, 14), > + (ptr)getArgReference(stk, pci, 15)); > + break; > default: > ret = createScriptException(mb, stkpc, MAL, NULL, > "too many arguments for command call"); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

2 1

[Monetdb-developers] Announcement: New Dec2011 Feature release of MonetDB suite
by Sjoerd Mullender 26 Jan '12

26 Jan '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2011 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2011 feature release SQL * mclient: The csv output format can now also be of the form csv=c and csv+c where c is the column separator. The form with csv+ produces a single header line with column names. * Changed a bug where the sign() function returned the same type as its argument instead of always an INTEGER. * Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements. * Fixed a bug when using default values for interval columns. Bug 2877. * A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882. * A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883. Java Module * Implemented getMetaData() method of PreparedStatement. * Fixed an AssertionError for special results from e.g. EXPLAIN queries. * Fixed overflow error when batching large statements, bug #2952 * Resolved a concurrency problem where ResultSet's date-related getters could cause odd stack traces when used by multiple threads at the same time, bug #2950. * JDBC now implements JDBCv4.1 which makes it possible to be built with Java 7 and up. JDBCv4.1 is a maintenance release of JDBCv4, and hence can be compiled with Java 6, although the added methods obviously are not part of the java.sql interfaces. * SQLExceptions thrown now carry a SQLSTATE. Until the server starts sending correct SQLSTATEs for all errors, server originated errors without SQLSTATE are considered generic data exceptions (22000). * Fixed a bug where closing ResultSets and PreparedStatements could lead to errors on concurrent running queries using the same Connection due to a race condition. * Changed version scheme of JDBC driver and MCL jar to be more standard, from monetdb-X.Y-<thing>.jar to monetdb-<thing>-X.Y.jar, bug #2943 * The embedded monet instance capability of MonetConnection was removed. * Bump JDBC version to 2.0 (Liberica). JDBC now implements JDBCv4 which makes it possible to be built with Java 6 and up. Java 5 and before are no longer supported, and can use the 1.x releases of the driver. * INTERVAL columns are now treated as decimals, since they can have sub-second precision. Client Package * Fix making connections with stethoscope to hosts without monetdbd. Bug #2944. * Fixed some bugs in the PHP interface affecting the mapi_ping(), monetdb_insert_id() and auto_commit() functions. Bugs #2936, #2937, #2938. * ODBC: Fixed SQLNumResultCols and SQLDescribeCol to return useful information after a call to SQLPrepare and before any SQLExecute. * mclient: The exact interpretation of the -i (--interactive) option and the - filename argument have changed. The - filename argument means read from standard input and no longer implies that no \ interpretation is done on the input. Instead, \ interpretation is done if either standard input is a terminal, or if the -i option is given. The -i option no longer causes mclient to read from standard input. It only means to do \ interpretation when reading from standard input. Use the - filename argument to read from standard input. Note that if no -s option is specified and no filename arguments are present, mclient still reads from standard input. * ODBC: Improved internal query for SQLSpecialColumns. Before, the query returned all columns taking part in a PRIMARY KEY *and* all columns taking part in a UNIQUE constraint. Now it returns only one or the other set. * ODBC: The database name is now used as the catalog name throughout. Functions that return a catalog name return the database name, and functions that match on catalog name match it with the database name. * ODBC: Implemented an easier way to create a log file of interactions with the ODBC driver. You can now add a connection attribute "LOGFILE=filename" to the connection string parameter of SQLBrowseConnect and SQLDriverConnect, and to the relevant part of the Windows registry or odbc.ini file. This value is only used if there is no environment variable ODBCDEBUG. * Quoting of object names for mclient's \d command is now more flexible and consistent with standard SQL quoting rules, bug #2846. * ODBC: Implemented SQLColumnPrivileges function. * Install new program, mnc, which provides netcat functionality, but based on MonetDB's communication libraries * Report full URI to database instead of just database when available in mclient. * The time format of the timer output can now be controlled with an optional argument to the -i option. ms, s and m force the time to be formatted as milliseconds, seconds or minutes + seconds respectively. * ODBC: Implemented a workaround in SQLTables for bug 2908. * Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability. * Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901 * mclient now checks the result of encoding conversions using the iconv library. * Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv). * Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois. * Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output * Fixed Perl DBD rowcount for larger results, bug #2889 * Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema). * Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois. * Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois. * mclient: fix display of varchar columns with only NULL values. * Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables. MonetDB5 Server * A couple of memory leaks have been plugged. * Introduced new variable max_clients that allows to define how many concurrent connections are allowed to be made against the database. * mal_interpreter.mx Protect against concurrent exceptions If multiple parallel blocks access the exception variables and perform GDKfree on old messages, then we may enter a case that one thread attempts a second free. Simple lock-based protection is the first line of defense. * bpm and partitions have been moved to the attic. It is replaced by the partition optimizer, still under construction. * The MAL debugger next/step operations semantics have been fixed. The profiler also now clearly shows entrance/exit of a MAL function. * The dataflow optimizer uses a less strict side-effect test for BAT new operations and better recognizes trivial plans. * Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888. Merovingian * Fixed crash in monetdb when an invalid property was retrieved using the get command, bug #2953. * monetdb get output is now grouped by database instead of by property * Unlike in previous releases, 'monetdbd get mserver' now returns the path to the mserver5 binary only for dbfarms that have a running monetdbd, instead of only for those that are not served by a monetdbd. * Add nclients property that controls the maximum number of concurrent clients allowed to access the database * Starting monetdbd without any arguments or without dbfarm is no longer supported. A dbfarm now must be provided for each command. * The control passphrase has been turned into a hash of the password, for a more stronger authorisation model as used by mclients. On upgrade, any existing passphrase is converted to the hashed version of the passphrase. * The monetdbd controlport option has been removed in favour of a boolean control option. On upgrade, when controlport was set to a non-zero value and a passphrase was set, control is set to true, or false otherwise. * The monetdbd discoveryport option has been removed in favour of a boolean discovery option. On upgrade, when discoveryport was set to a non-zero value, discovery is set to true, or false otherwise. * Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port. * Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped. * monetdbd get status now also reports the version of the running monetdbd * Improved response time of 'monetdb start' when the database fails to start. * Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database. * To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client. MonetDB Common * Fixed memory detection on 32-bits Solaris systems with more memory than can be addressed in 32-bits mode, bug #2935 Bug Fixes

1 0

Re: [Monetdb-developers] MonetDB: default - Plug memory leak.
by Stefan Manegold 14 Jan '12

14 Jan '12

(on my 64-bit Fedora 14 desktop) this checkin seems cause several tests to fail or even crash: testing status with changeset 42821:3967ec4a8aef: monetdb5: !ERROR: Testing FAILED SIGNIFICANTLY (4 out of 494 tests failed) 1 out of 494 tests ran into timeout 3 out of 494 tests produced SIGNIFICANTLY different output sql: !ERROR: Testing FAILED SIGNIFICANTLY (4 out of 1161 tests failed) 15 out of 1161 tests could not be executed 1 out of 1161 tests produced slightly different output 3 out of 1161 tests produced SIGNIFICANTLY different output geom: !ERROR: Testing FAILED SIGNIFICANTLY (1 out of 16 tests failed) 1 out of 16 tests resulted in a crash testing status with changeset 42825:f15583fdc322 monetdb5: !ERROR: Testing FAILED SIGNIFICANTLY (21 out of 494 tests failed) 1 out of 494 tests ran into timeout 20 out of 494 tests produced SIGNIFICANTLY different output sql: !ERROR: Testing FAILED SIGNIFICANTLY (909 out of 1161 tests failed) 248 out of 1161 tests could not be executed 6 out of 1161 tests ran into timeout 900 out of 1161 tests resulted in a crash 3 out of 1161 tests produced SIGNIFICANTLY different output geom: !ERROR: Testing FAILED SIGNIFICANTLY (16 out of 16 tests failed) 16 out of 16 tests resulted in a crash I have not yet looked into the details. Stefan On Fri, Jan 13, 2012 at 06:08:12PM +0100, Sjoerd Mullender wrote: > Changeset: f15583fdc322 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f15583fdc322 > Modified Files: > clients/Tests/exports.stable.out > gdk/gdk.mx > gdk/gdk_atoms.mx > monetdb5/mal/mal_atom.c > monetdb5/mal/mal_interpreter.mx > monetdb5/mal/mal_resolve.mx > monetdb5/modules/atoms/blob.c > monetdb5/modules/atoms/blob.mal > monetdb5/modules/mal/Tests/inspect05.stable.out > Branch: default > Log Message: > > Plug memory leak. > Variables of types other than the standard built-in types could not be > freed since the interpreter didn't know what to do. We now have a > new function pointer in BATatoms that the interpreter can use to > destroy an atom. Typically the function pointer will be NULL, but for > e.g. BLOBs it should be GDKfree. This function pointer can be set > using the new > command destroy() address XXX; > construct in MAL. > > > diffs (278 lines): > > diff --git a/clients/Tests/exports.stable.out b/clients/Tests/exports.stable.out > --- a/clients/Tests/exports.stable.out > +++ b/clients/Tests/exports.stable.out > @@ -954,6 +954,7 @@ str BLOBblob_blob(blob **d, blob **s); > str BLOBblob_fromstr(blob **b, str *d); > void BLOBconvert(blob *b, int direction); > void BLOBdel(Heap *h, var_t *index); > +void BLOBdestroy(blob *b); > str BLOBeoln(char *src, char *end); > str BLOBfromblob(str *retval, blob **b); > str BLOBfromidx(str *retval, blob **binp, int *index); > diff --git a/gdk/gdk.mx b/gdk/gdk.mx > --- a/gdk/gdk.mx > +++ b/gdk/gdk.mx > @@ -2125,6 +2125,7 @@ typedef struct { > void (*atomConvert) (ptr v, int direction); > int (*atomFix) (ptr atom); > int (*atomUnfix) (ptr atom); > + void (*atomDestroy) (ptr atom); > > /* varsized atom-only ADT functions */ > var_t (*atomPut) (Heap *, var_t *off, ptr src); > diff --git a/gdk/gdk_atoms.mx b/gdk/gdk_atoms.mx > --- a/gdk/gdk_atoms.mx > +++ b/gdk/gdk_atoms.mx > @@ -2350,64 +2350,44 @@ atomDesc BATatoms[MAXATOMS] = { > (int (*)(ptr, ptr)) lngCmp, > (BUN (*)(ptr)) lngHash, 0, > #endif > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"bit", TYPE_chr, 1, sizeof(bit), sizeof(bit), 0, 0, (ptr) &chr_nil, > (int (*)(str, int *, ptr *)) bitFromStr, (int (*)(str *, int *, ptr)) bitToStr, > (void *(*)(void *, stream *, size_t)) bitRead, (int (*)(void *, stream *, size_t)) bitWrite, > (int (*)(ptr, ptr)) chrCmp, > (BUN (*)(ptr)) chrHash, 0, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"chr", TYPE_chr, 1, sizeof(chr), sizeof(chr), 0, 0, (ptr) &chr_nil, > (int (*)(str, int *, ptr *)) chrFromStr, (int (*)(str *, int *, ptr)) chrToStr, > (void *(*)(void *, stream *, size_t)) chrRead, (int (*)(void *, stream *, size_t)) chrWrite, > (int (*)(ptr, ptr)) chrCmp, > (BUN (*)(ptr)) chrHash, 0, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"bte", TYPE_bte, 1, sizeof(bte), sizeof(bte), 0, 0, (ptr) &bte_nil, > (int (*)(str, int *, ptr *)) bteFromStr, (int (*)(str *, int *, ptr)) bteToStr, > (void *(*)(void *, stream *, size_t)) bteRead, (int (*)(void *, stream *, size_t)) bteWrite, > (int (*)(ptr, ptr)) bteCmp, > (BUN (*)(ptr)) bteHash, 0, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"sht", TYPE_sht, 1, sizeof(sht), sizeof(sht), 0, 0, (ptr) &sht_nil, > (int (*)(str, int *, ptr *)) shtFromStr, (int (*)(str *, int *, ptr)) shtToStr, > (void *(*)(void *, stream *, size_t)) shtRead, (int (*)(void *, stream *, size_t)) shtWrite, > (int (*)(ptr, ptr)) shtCmp, > (BUN (*)(ptr)) shtHash, (void (*)(ptr, int)) shtConvert, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"BAT", TYPE_int, 1, sizeof(bat), sizeof(bat), 0, 0, (ptr) &int_nil, > (int (*)(str, int *, ptr *)) batFromStr, (int (*)(str *, int *, ptr)) batToStr, > (void *(*)(void *, stream *, size_t)) batRead, (int (*)(void *, stream *, size_t)) batWrite, > (int (*)(ptr, ptr)) intCmp, > (BUN (*)(ptr)) intHash, (void (*)(ptr, int)) intConvert, > (int (*)(ptr)) batFix, (int (*)(ptr)) batUnfix, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0}, > {"int", TYPE_int, 1, sizeof(int), sizeof(int), 0, 0, (ptr) &int_nil, > (int (*)(str, int *, ptr *)) intFromStr, (int (*)(str *, int *, ptr)) intToStr, > (void *(*)(void *, stream *, size_t)) intRead, (int (*)(void *, stream *, size_t)) intWrite, > (int (*)(ptr, ptr)) intCmp, > (BUN (*)(ptr)) intHash, (void (*)(ptr, int)) intConvert, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"oid", > #if SIZEOF_OID == SIZEOF_INT > TYPE_int, 1, sizeof(oid), sizeof(oid), 0, 0, (ptr) &int_nil, > @@ -2422,10 +2402,7 @@ atomDesc BATatoms[MAXATOMS] = { > (int (*)(ptr, ptr)) lngCmp, > (BUN (*)(ptr)) lngHash, (void (*)(ptr, int)) lngConvert, > #endif > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"wrd", > #if SIZEOF_WRD == SIZEOF_INT > TYPE_int, 1, sizeof(wrd), sizeof(wrd), 0, 0, (ptr) &int_nil, > @@ -2440,10 +2417,7 @@ atomDesc BATatoms[MAXATOMS] = { > (int (*)(ptr, ptr)) lngCmp, > (BUN (*)(ptr)) lngHash, (void (*)(ptr, int)) lngConvert, > #endif > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"ptr", > #if SIZEOF_VOID_P == SIZEOF_INT > TYPE_ptr, 1, sizeof(ptr), sizeof(ptr), 0, 0, (ptr) &ptr_nil, > @@ -2458,43 +2432,31 @@ atomDesc BATatoms[MAXATOMS] = { > (int (*)(ptr, ptr)) lngCmp, > (BUN (*)(ptr)) lngHash, (void (*)(ptr, int)) lngConvert, > #endif > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"flt", TYPE_flt, 1, sizeof(flt), sizeof(flt), 0, 0, (ptr) &flt_nil, > (int (*)(str, int *, ptr *)) fltFromStr, (int (*)(str *, int *, ptr)) fltToStr, > (void *(*)(void *, stream *, size_t)) fltRead, (int (*)(void *, stream *, size_t)) fltWrite, > (int (*)(ptr, ptr)) fltCmp, > (BUN (*)(ptr)) intHash, (void (*)(ptr, int)) intConvert, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"dbl", TYPE_dbl, 1, sizeof(dbl), sizeof(dbl), 0, 0, (ptr) &dbl_nil, > (int (*)(str, int *, ptr *)) dblFromStr, (int (*)(str *, int *, ptr)) dblToStr, > (void *(*)(void *, stream *, size_t)) dblRead, (int (*)(void *, stream *, size_t)) dblWrite, > (int (*)(ptr, ptr)) dblCmp, > (BUN (*)(ptr)) lngHash, (void (*)(ptr, int)) lngConvert, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"lng", TYPE_lng, 1, sizeof(lng), sizeof(lng), 0, 0, (ptr) &lng_nil, > (int (*)(str, int *, ptr *)) lngFromStr, (int (*)(str *, int *, ptr)) lngToStr, > (void *(*)(void *, stream *, size_t)) lngRead, (int (*)(void *, stream *, size_t)) lngWrite, > (int (*)(ptr, ptr)) lngCmp, > (BUN (*)(ptr)) lngHash, (void (*)(ptr, int)) lngConvert, > - 0, 0, > - 0, 0, > - 0, 0, > - 0, 0}, > + 0, 0, 0, 0, 0, 0, 0, 0, 0}, > {"str", TYPE_str, 1, sizeof(var_t), sizeof(var_t), 0, 1, (ptr) str_nil, > (int (*)(str, int *, ptr *)) strFromStr, (int (*)(str *, int *, ptr)) strToStr, > (void *(*)(void *, stream *, size_t)) strRead, (int (*)(void *, stream *, size_t)) strWrite, > (int (*)(ptr, ptr)) strCmp, > (BUN (*)(ptr)) strHash, 0, > - 0, 0, > + 0, 0, GDKfree, > (var_t (*)(Heap *, var_t *, ptr)) strPut, 0, > (int (*)(ptr)) strLen, strHeap, > (void (*)(Heap *, int)) 0, 0}, > diff --git a/monetdb5/mal/mal_atom.c b/monetdb5/mal/mal_atom.c > --- a/monetdb5/mal/mal_atom.c > +++ b/monetdb5/mal/mal_atom.c > @@ -118,6 +118,11 @@ int malAtomProperty(MalBlkPtr mb, InstrP > setAtomName(pci); > return 1; > } > + if (idcmp("destroy", name) == 0 && pci->argc == 1) { > + BATatoms[tpe].atomDestroy = (void (*)(void *))pci->fcn; > + setAtomName(pci); > + return 1; > + } > break; > case 'c': > if (idcmp("cmp", name) == 0 && pci->argc == 1) { > diff --git a/monetdb5/mal/mal_interpreter.mx b/monetdb5/mal/mal_interpreter.mx > --- a/monetdb5/mal/mal_interpreter.mx > +++ b/monetdb5/mal/mal_interpreter.mx > @@ -2442,9 +2442,7 @@ void garbageElement(Client cntxt, ValPtr > v->val.sval = NULL; > } > v->len = 0; > - return; > - } > - if (v->vtype == TYPE_bat) { > + } else if (v->vtype == TYPE_bat) { > /* > * @- > * All operations are responsible to properly set the > @@ -2466,6 +2464,8 @@ void garbageElement(Client cntxt, ValPtr > if (cntxt && cntxt->flags & bigfootFlag) > updateBigFoot(cntxt, bid, FALSE); > BBPdecref(bid, TRUE); > + } else if (BATatoms[v->vtype].atomDestroy) { > + (*BATatoms[v->vtype].atomDestroy)(v->val.pval); > } > } > /* > diff --git a/monetdb5/mal/mal_resolve.mx b/monetdb5/mal/mal_resolve.mx > --- a/monetdb5/mal/mal_resolve.mx > +++ b/monetdb5/mal/mal_resolve.mx > @@ -529,7 +529,8 @@ findFunctionType(Module scope, MalBlkPtr > isaBatType(@1) || > findGDKtype(@1) == TYPE_str || > (!isPolyType(@1) && @1 < TYPE_any && > - @1 >= 0 && ATOMstorage(@1) == TYPE_str)) { > + @1 >= 0 && (ATOMstorage(@1) == TYPE_str || > + BATatoms[(a)1].atomDestroy))) { > getInstrPtr(@3, 0)->gc |= GARBAGECONTROL; > setVarCleanup(mb, getArg(p, @2)); > p->gc |= GARBAGECONTROL; > @@ -783,7 +784,9 @@ void typeChecker(Module scope, MalBlkPtr > int tpe = getArgType(mb,p,k); > if (findGDKtype(tpe) == TYPE_bat || > findGDKtype(tpe) == TYPE_str || > - (!isPolyType(tpe) && tpe< TYPE_any && ATOMstorage(tpe) == TYPE_str)) > + (!isPolyType(tpe) && tpe< TYPE_any && > + (ATOMstorage(tpe) == TYPE_str || > + BATatoms[tpe].atomDestroy))) > setVarCleanup(mb,getArg(p,k)); > } > } > diff --git a/monetdb5/modules/atoms/blob.c b/monetdb5/modules/atoms/blob.c > --- a/monetdb5/modules/atoms/blob.c > +++ b/monetdb5/modules/atoms/blob.c > @@ -68,6 +68,7 @@ blob_export int BLOBnitems(int *ret, blo > blob_export int BLOBget(Heap *h, int *bun, int *l, blob **val); > blob_export blob * BLOBread(blob *a, stream *s, size_t cnt); > blob_export int BLOBwrite(blob *a, stream *s, size_t cnt); > +blob_export void BLOBdestroy(blob *b); > > blob_export str BLOBblob_blob(blob **d, blob **s); > blob_export str BLOBblob_fromstr(blob **b, str *d); > @@ -741,6 +742,12 @@ BLOBheap(Heap *heap, size_t capacity) > blob_heap(heap, capacity); > } > > +void > +BLOBdestroy(blob *b) > +{ > + GDKfree(b); > +} > + > var_t > BLOBput(Heap *h, var_t *bun, blob *val) > { > diff --git a/monetdb5/modules/atoms/blob.mal b/monetdb5/modules/atoms/blob.mal > --- a/monetdb5/modules/atoms/blob.mal > +++ b/monetdb5/modules/atoms/blob.mal > @@ -31,6 +31,7 @@ command put() address BLOBput; > command del() address BLOBdel; > command length() address BLOBlength; > command heap() address BLOBheap; > +command destroy() address BLOBdestroy; > > command blob(s:blob):blob > address BLOBblob_blob > diff --git a/monetdb5/modules/mal/Tests/inspect05.stable.out b/monetdb5/modules/mal/Tests/inspect05.stable.out > --- a/monetdb5/modules/mal/Tests/inspect05.stable.out > +++ b/monetdb5/modules/mal/Tests/inspect05.stable.out > @@ -3399,6 +3399,7 @@ end main; > [ "blob", "command", "blob", "(s:str):blob ", "BLOBblob_fromstr;" ] > [ "blob", "command", "blob", "(s:blob):blob ", "BLOBblob_blob;" ] > [ "#convert", "command", "blob", "():void ", "BLOBconvert;" ] > +[ "#destroy", "command", "blob", "():void ", "BLOBdestroy;" ] > [ "#del", "command", "blob", "():void ", "BLOBdel;" ] > [ "#fromstr", "command", "blob", "():void ", "BLOBfromstr;" ] > [ "#heap", "command", "blob", "():void ", "BLOBheap;" ] > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 2

Re: [Monetdb-developers] MonetDB: sciql - Add two new functions to create an sql_exp* for...
by Fabian Groffen 12 Jan '12

12 Jan '12

On 11-01-2012 23:22:23 +0100, Jennie Zhang wrote: > Changeset: d42756602492 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d42756602492 > Modified Files: > sql/server/rel_exp.c > sql/server/rel_exp.h > Branch: sciql > Log Message: > > Add two new functions to create an sql_exp* for flt and dbl type atoms. > > This change should be propagated to the default branch, but how? Not propagated, but backported. You can use hg transplant --log d42756602492 on the default branch.

1 0

[Monetdb-developers] Testing code
by Charalampos Nikolaou 11 Jan '12

11 Jan '12

Hi, I have written some code and now I would like to make changes and still be sure that the new changes are correct. So, I have written some scenarios in mal and put it in a Tests directory. I have read the README file under testing/ directory, but still haven't understood how I could generate the <scenario>.stable.out file. How could that file be generated? Even if I run Mtest.py <mytest>, then Mapprove.py the previously "wrong" output, and then again Mtest.py the test fails. Thanks, Babis

3 3

Re: [Monetdb-developers] MonetDB: default - Fix for bug 2965
by Sjoerd Mullender 05 Jan '12

05 Jan '12

Shouldn't this have gone to the Dec2011 branch? On 2012-01-04 22:30, Martin Kersten wrote: > Changeset: fa4034576f02 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=fa4034576f02 > Modified Files: > monetdb5/optimizer/opt_joinpath.mx > Branch: default > Log Message: > > Fix for bug 2965 > Typechecking should be included as extra defense line. > Furthermore, we need to remove reference counts during > error handling. > > > diffs (74 lines): > > diff --git a/monetdb5/optimizer/opt_joinpath.mx b/monetdb5/optimizer/opt_joinpath.mx > --- a/monetdb5/optimizer/opt_joinpath.mx > +++ b/monetdb5/optimizer/opt_joinpath.mx > @@ -580,7 +580,7 @@ ALGjoinPathBody(Client cntxt, int top, B > * BEWARE. you may not use a size estimation, because it > * may fire a BATproperty check in a few cases. > * In case a join fails, we may try another order first before > - * abandoning the task. > + * abandoning the task. It can handle cases where a Cartesian product emerges. > * > * A left-join sequence only requires the result to be sorted > * against the first operand. For all others operand pairs, the cheapest join suffice. > @@ -599,12 +599,20 @@ ALGjoinPathBody(Client cntxt, int top, B > b = BATsemijoin(joins[j], joins[j + 1]); > } > if (b==NULL){ > + if ( postpone[j] && postpone[j+1]){ > + for( --top; top>=0; top--) > + BBPreleaseref(joins[top]->batCacheid); > + GDKfree(postpone); > + return NULL; > + } > postpone[j] = TRUE; > postpone[j+1] = TRUE; > postponed = 0; > for( k=0; k<top; k++) > postponed += postpone[k]== TRUE; > if ( postponed == top){ > + for( --top; top>=0; top--) > + BBPreleaseref(joins[top]->batCacheid); > GDKfree(postpone); > return NULL; > } > @@ -624,6 +632,12 @@ ALGjoinPathBody(Client cntxt, int top, B > mnstr_printf(cntxt->fdout, "#joinPath %d:= join(%d,%d)\n", > b->batCacheid, joins[j]->batCacheid, joins[j + 1]->batCacheid); > > + if ( b == 0 ){ > + for( --top; top>=0; top--) > + BBPreleaseref(joins[top]->batCacheid); > + GDKfree(postpone); > + return 0; > + } > BBPdecref(joins[j]->batCacheid, FALSE); > BBPdecref(joins[j+1]->batCacheid, FALSE); > joins[j] = b; > @@ -645,17 +659,25 @@ ALGjoinPath(Client cntxt, MalBlkPtr mb, > BAT *b, **joins = (BAT**)GDKmalloc(pci->argc*sizeof(BAT*)); > str joinPathRef = putName("joinPath",8); > str leftjoinPathRef = putName("leftjoinPath",12); > + int error = 0; > > if ( joins == NULL) > throw(MAL, "algebra.joinPath", MAL_MALLOC_FAIL); > (void)mb; > for (i = pci->retc; i < pci->argc; i++) { > bid = (int *) getArgReference(stk, pci, i); > - if ((b = BATdescriptor(*bid)) == NULL) { > + b = BATdescriptor(*bid); > + if ( b && top ) { > + if (TYPEerror(joins[top-1]->ttype, b->htype) ) { > + b= NULL; > + error = 1; > + } > + } > + if ( b == NULL) { > for( --top; top>=0; top--) > BBPreleaseref(joins[top]->batCacheid); > GDKfree(joins); > - throw(MAL, "algebra.joinPath", INTERNAL_BAT_ACCESS); > + throw(MAL, "algebra.joinPath", error? SEMANTIC_TYPE_MISMATCH: INTERNAL_BAT_ACCESS); > } > joins[top++] = b; > } > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

1 0

[Monetdb-developers] group by on BATs
by Charalampos Nikolaou 04 Jan '12

04 Jan '12

Hi, I have three BATs and I would like to create groups of pairs from the first two BATs and take the bitwise and for the tail values of each group according to the third BAT. I saw that there is a module for grouping operations in monetdb5/modules/kernel/group.mx. Is it the right one for my purpose? For the case of bitwise and is there any function that I could use or I should write my own like for example the GRPaggr_count() function in that file. Thanks, Babis

1 1

[Monetdb-developers] /usr/local/bin/mserver5: error while loading shared libraries: libmonetdb5.so.8: cannot open shared object file: No such file or directory
by Mark Kerzner 04 Jan '12

04 Jan '12

Hi, sorry to bother, but in building Dec2011 branch, I get the libmonetdb5.so.8 not found error, and in building the default branch, I get a crash. Both errors are attached. Thank you. Sincerely, Mark

2 1

[Monetdb-developers] MX files
by David Pennell 02 Jan '12

02 Jan '12

Where can I learn more about mx files? In particular, in the following code, it looks like @2 is derived from the type expands this into a multitude of type specific functions. I can't figure out what @3 is. Is there a list of these? SORTfndlast_@2(BAT *b, ptr v) { BUN diff, end = BUNlast(b), cur = end; BUN lo = BUNfirst(b), hi = end; int cmp = 1; BATiter bi = bat_iterator(b); if (lo >= hi || @3_CMP(BUNt@1(bi, cur - 1), v, @4)<=0) { /* shortcut: if BAT is empty or last (and hence all) tail * value is <= v, we're done. */ return cur; } while (lo < hi) { BUN mid = (lo + hi) >> 1; cur = mid; cmp = @3_CMP(BUNt@1(bi, cur), v, @4); if (cmp < 0) { lo = ++mid; cur++; } else if (cmp > 0) { hi = mid; } else { break; } } if (cmp == 0 && b->tkey == 0) { /* shift over multiple equals */ for(diff = (end-cur)>>1; diff; diff >>= 1) { while ((cur+diff) < end && @3_EQ(BUNt@1(bi, cur+diff), v, @4)) cur += diff; } } if (cmp == 0) cur++; return cur; }

2 1

[Monetdb-developers] Extending TimeStamp and Time to support 24+ hour days
by Stefan de Konink 15 Dec '11

15 Dec '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 In some fields the usage of 32 or 36 hours in a day are common. The extra time after midnight is used to indicate that a certain operation is linear in time. My particular use case is public transport. Is there any other way that using TIMESTAMP to get the 'features' of TIME, but than in a 32h or 36h day? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk7pYboACgkQYH1+F2Rqwn3CQgCfU0CCgnJgxJj0lFW9RC+qiyz+ Pj0An1Vn8oZETQTCCeIFv+LgWOAx7ERQ =wSgl -----END PGP SIGNATURE-----

2 1

[Monetdb-developers] monetdbd takes too long to start
by Charalampos Nikolaou 12 Dec '11

12 Dec '11

Hi, in Ubuntu 10.04 (as far as I remember it happens in other releases too) monetdbd takes about 10 seconds to start as opposed to mac os x 10.6 and 10.7 in which it starts instantaneously. Inspecting connections.c, the delay is due to the call on gethostbyaddr(). Reading the respective man page, getting the hostname involves a number of steps ranging from looking at /etc/hosts.conf, /etc/hosts, as well as consulting named for resolving the ip address. What's more, the man page states explicitly that gethostbyaddr() is obsolete and suggests the use of getnameinfo() instead. Correspondingly, the man page in os x recommends the use of getnameinfo() as well, but without stating any explanation for this. Have you investigated this issue? Do you know what causes such a delay? Is it present in other OSs too? Thanks, Babis

5 13

[Monetdb-developers] 3D gis support
by luca paganotti 09 Dec '11

09 Dec '11

Hi, I've read in the docs that one of the planned future features of monetdb will be 'support for 3D types'. - Do you mean full 3D support or 2.5D ? - Will be these 3D spatial extensions also integrated in sciQL? - Have you scheduled this feature release? Thanks in advance for any answer. Luca Paganotti

1 0

[Monetdb-developers] bat.inplace() strange exception
by Charalampos Nikolaou 07 Dec '11

07 Dec '11

Hello, considering the following interaction $ mclient -lmal -d test Welcome to mclient, the MonetDB interactive terminal (unreleased) Type \q to quit, \? for a list of available commands mal>sql.init(); mal>_mvc := sql.mvc(); mal>abat:bat[:int,:int] := bat.new(:int, :int); mal>bat.insert(abat, 1, 4); mal>bat.insert(abat, 1, 4); mal>bat.insert(abat, 1, 1); mal>bat.insert(abat, 1, 1); mal>bat.insert(abat, 0, 4); mal>bat.inplace(abat, 0, 5); MAPI = (monetdb) /tmp/.s.monetdb.50000 ACTION= read_line QUERY = bat.inplace(abat, 0, 5); ERROR = !Connection terminated Is this a bug or not? The log of merovingian is the following: 2011-12-07 13:01:51 ERR test[15564]: mserver5: ../../MonetDB/gdk/gdk_bat.mx:1917: void_inplace: Assertion `b->H->type == 0' failed. 2011-12-07 13:01:51 MSG merovingian[14343]: database 'test' (15564) was killed by signal SIGABRT However, the bat5.mx declares the following prototype: command inplace( o:bat[:any_1,:any_2], id:any_1, t:any_2) :bat[:any_1,:any_2] address BKCbun_inplace comment "inplace replace values on the given locations"; Regards, Babis

1 0

[Monetdb-developers] Problem (and possible fix!) with monetdbd.conf path as specified in Makefile.in
by Eric G 29 Nov '11

29 Nov '11

Hello list, I'm currently trying to use the OpenSUSE build service (http://build.openeuse.org) to build packages RPMs for for CentOS and RedHat distributions, and i think I've found a bug in a Makefile for MonetDB. Essentially it looks like the explicit path /etc/tmpfiles.d/monetdbd.conf is used in the Makefile located here: MonetDB-11.5.9/buildtools/conf/Makefile.in ...and the RPM build process doesn't like that. OpenSUSE essentially implicitly enables the "norootdir" option in the spec file (regardless of if it's actually present or not) because RPMs are built is special purpose-built VMs that are provisioned when a "job" to build an RPM is submitted, and the build process doesn't run as root. See here and here for more discussion on this problem (it's a common problem apparently): http://wiki.sipfoundry.org/display/sipXecs/Building+RPM+tips http://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#Permission_de… I think the "fix" is as simple as inserting "$(prefix)" into the line in Makefile.in, so that the build path is used to create the temporary folder instead of attempting to writing to the system-wide /etc folder (the build fails on the Makefile's install phase with permission denied errors because of this static path). Am I on to something here? If I'm right, can the Makefile.in file be patched please, adding $(prefix)? -- Thanks, Eric

3 3