- developers-list - monetdb.org

fast bit integration into monetdb
by Stanislav Seltser 27 Dec '12

27 Dec '12

does anybody know if fastbit integration into scidb was ever completed? _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

3 4

Unique key constraint violation violated
by Tapomay Dey 26 Dec '12

26 Dec '12

Is this a known issue that you are working on? If so please give me the bug id where I would know when its fixed. The case is as follows: I have a table with 32 columns. It has a unique key on 3 of them. I know that it's recommended that you do bulk inserts from csv for performance. But in my case I am running live ETLs for putting data into monetdb. These ETLs are generating sql insert scripts.(inserts/updates/ddls) I use JDBC and c3p0 connection pooling for executing these sqls. I have a failover mechanism where the scripts are executed on multiple DBs and I try to keep their states consistent. One common consistency check would be a count*. I am running the latest stable installation from repo on Ubuntu 10.04 64 bit. Observation: After running 9100 queries the count* on one DB was 9100 but in the other was 9200. I have diagnostics in place to do a select count* after running a batch of 100 queries in a single JDBC transaction. I observed that the count* was consistent until 9000 queries were executed. It failed at 9100. I did a manual check to see if duplicate records exist with same unique key values and they did. Following is the query I used for the same: select ad_group_id, count(*) from ad_groups group by ad_group_id having count(ad_group_id)>1 and client_id=X and account_id=Y; My constraint is as follows: CONSTRAINT "ad_groups_client_id_account_id_ad_group_id_unique" UNIQUE ("client_id", "account_id", "ad_group_id") I found exactly 100 tuples with group_by_count* = 2 and the ad_group_id(unique key) values for these exactly matched queries 8901 to 9000. It means when I inserted 9001 to 9100 the previous batch went into the store again with it and somehow surpassed the unique key constraint check. Please note that I am converting a batch of 100 insert queries into a single insert query and running this single bulk insert in a single JDBC transaction to reduce network traffic. This same issue recurred at 21500 only this time it happened for 300 records with same unique key value being duplicated. And their unique key values were exactly matching the ones in the last 3 batches Ad_group_id s 21103-21202, 21203-21302, 21303-21402 existed twice. 21403-21502 existed exactly once. REQUEST: Can anyone tell me if I can enable query logs - something like what mysql has. I will try re-installing monetdb and see if this problem goes away. :) Merry Christmas and Warm Regards, Tapomay _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 1

Re: MonetDB: Feb2013 - ported fix to the query cache from Oct2012
by Stefan Manegold 23 Dec '12

23 Dec '12

Niels, does this replace a merge of your Oct2012 checkin to the Feb2013 branch? Before this checkin, the merge worked without any conflicts (I did not test, whether/how it works). After this checkin, the merge fails with conflicts (I did not check the details). Thanks! Stefan Niels Nes <commits(a)monetdb.org> wrote: Changeset: ee6e585e72c5 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ee6e585e72c5 Modified Files: sql/backends/monet5/Tests/optimizers.stable.out sql/backends/monet5/UDF/Tests/udf-fuse.stable.out sql/backends/monet5/UDF/Tests/udf-reverse.stable.out sql/server/sql_mvc.c sql/server/sql_parser.y sql/server/sql_qc.c sql/test/ADT2006/Tests/bram.stable.out sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out sql/test/BugDay_2005-10-06_2.9.3/Tests/variable_arg_substring.SF_942981.stable.out sql/test/BugDay_2005-11-09_2.8/Tests/crash_in_case.SF-1097749.stable.out sql/test/BugDay_2005-11-09_2.9.3/Tests/sql_server_crash.SF-1080024.stable.out sql/test/BugDay_2005-11-09_2.9.3/Tests/union-duplicates.SF-1119186.stable.out sql/test/BugTracker-2009/Tests/AVG_of_SQRT.SF-2757642.stable.out sql/test/BugTracker-2009/Tests/IS_NULL_in_CASE.stable.out sql/test/BugTracker-2009/Tests/group_by_on_values.SF-2837561.stable.out sql/test/BugTracker-2009/Tests/not_equal.SF-2678502.stable.out sql/test/BugTracker-2009/Tests/old_views_incorrectly_reused.SF-2863804.stable.out sql/test/BugTracker-2009/Tests/push_select_down_crash.stable.out sql/test/BugTracker-2009/Tests/utf8_bug.SF-2822855.stable.out sql/test/BugTracker-2010/Tests/LIMIT_OFFSET_big-endian.Bug-2622.stable.out sql/test/BugTracker-2010/Tests/crash_on_where_in.Bug-2333.stable.out sql/test/BugTracker-2010/Tests/group-by_ordered_column.Bug-2564.stable.out sql/test/BugTracker-2010/Tests/mat.slice_limit1.Bug-2645.stable.out sql/test/BugTracker-2010/Tests/order_by_on_single_row_view.SF-2950453.stable.out sql/test/BugTracker-2010/Tests/substring_display_width.Bug-2683.stable.out sql/test/BugTracker-2011/Tests/case-when-alias.Bug-2848.stable.out sql/test/BugTracker-2012/Tests/predicate_select.Bug-3090.stable.err sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out sql/test/BugTracker/Tests/assert_fails_on_algebra_version.SF-2019349.stable.out sql/test/BugTracker/Tests/bind_with_cast.SF-1720934.stable.err sql/test/BugTracker/Tests/crash_on_delete.SF-1639108.stable.out sql/test/BugTracker/Tests/crash_on_delete.SF-1651567.stable.out sql/test/BugTracker/Tests/crash_on_insert.SF-1639113.stable.out sql/test/BugTracker/Tests/insert_not_exists.SF-1380287.stable.out sql/test/Dependencies/Tests/Dependencies.stable.out sql/test/Skyserver/Tests/Skyserver.stable.out sql/test/Tests/coalesce.stable.out sql/test/Tests/decimal2.stable.out sql/test/Tests/setoptimizer.stable.out sql/test/VOC/Tests/VOC.stable.out sql/test/bugs/Tests/crash_order_by.stable.out sql/test/bugs/Tests/select_from_values.stable.out sql/test/bugs/Tests/simple_view.stable.out sql/test/bugs/Tests/subselect_multiple_unionall_where_1=1-bug-sf-1005596.stable.out sql/test/bugs/Tests/subselect_single_value-bug-sf-941791.stable.out sql/test/bugs/Tests/subselect_union-bug-sf-941826.stable.out sql/test/bugs/Tests/unicode_varchar-bug-sf-1041324_JdbcClient.stable.out sql/test/bugs/Tests/unicode_varchar-bug-sf-1041324_MapiClient.stable.out sql/test/bugs/Tests/union_all-bug-sf-941788.stable.out sql/test/bugs/Tests/union_all_where_1=1-bug-sf-936386.stable.out sql/test/leaks/Tests/check0.stable.out sql/test/leaks/Tests/check1.stable.out sql/test/leaks/Tests/check2.stable.out sql/test/leaks/Tests/check3.stable.out sql/test/leaks/Tests/check4.stable.out sql/test/leaks/Tests/check5.stable.out sql/test/leaks/Tests/drop3.stable.out sql/test/leaks/Tests/temp1.stable.out sql/test/leaks/Tests/temp2.stable.out sql/test/leaks/Tests/temp3.stable.out sql/test/pg_regress/Tests/char.stable.out sql/test/pg_regress/Tests/strings_concat.stable.out sql/test/pg_regress/Tests/text.stable.out sql/test/pg_regress/Tests/varchar.stable.out sql/test/rdf/Tests/q2_28_v.stable.out sql/test/rdf/Tests/q2_v.stable.out sql/test/rdf/Tests/q3_28_v.stable.out sql/test/rdf/Tests/q3_v.stable.out sql/test/rdf/Tests/q4_28_v.stable.out sql/test/rdf/Tests/q4_v.stable.out sql/test/rdf/Tests/q6_28_v.stable.out sql/test/rdf/Tests/q6_v.stable.out Branch: Feb2013 Log Message: ported fix to the query cache from Oct2012 diffs (truncated from 3392 to 300 lines): diff --git a/sql/backends/monet5/Tests/optimizers.stable.out b/sql/backends/monet5/Tests/optimizers.stable.out --- a/sql/backends/monet5/Tests/optimizers.stable.out +++ b/sql/backends/monet5/Tests/optimizers.stable.out @@ -47,31 +47,31 @@ Ready. #select 'optimizer test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 15, 12 # length [ "optimizer test:", "default_pipe" ] #select 'optimizer off test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 19, 12 # length [ "optimizer off test:", "default_pipe" ] #select 'optimizer on test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 18, 12 # length [ "optimizer on test:", "default_pipe" ] #select 'optimizer <empty> test:',optimizer; % ., . # table_name % single_value, single_value # name -% char, varchar # type +% varchar, varchar # type % 23, 12 # length [ "optimizer <empty> test:", "default_pipe" ] #select 'done'; % . # table_name % single_value # name -% char # type +% varchar # type % 4 # length [ "done" ] diff --git a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out --- a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out +++ b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out @@ -45,12 +45,10 @@ end s0_1; % mal # name % clob # type % 71 # length -function user.s2_1{autoCommit=true}(A0:sht,A1:sht):void; +function user.s1_1{autoCommit=true}(A0:sht,A1:sht):void; X_4 := udf.fuse(A0,A1); sql.exportValue(1,".","fuse_single_value":str,"int",32,0,6,X_4,""); -end s2_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s1_1; #select fuse(1000,2000); % . # table_name % fuse_single_value # name @@ -62,12 +60,10 @@ end s2_1; % mal # name % clob # type % 74 # length -function user.s4_1{autoCommit=true}(A0:int,A1:int):void; +function user.s2_1{autoCommit=true}(A0:int,A1:int):void; X_4 := udf.fuse(A0,A1); sql.exportValue(1,".","fuse_single_value":str,"bigint",64,0,6,X_4,""); -end s4_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s2_1; #select fuse(1000000,2000000); % . # table_name % fuse_single_value # name @@ -97,7 +93,7 @@ end s4_1; % mal # name % clob # type % 60 # length -function user.s11_1{autoCommit=true}():void; +function user.s5_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_fuse"); X_6 := sql.bind(X_2,"sys","udf_fuse","a",0); @@ -115,15 +111,13 @@ function user.s11_1{autoCommit=true}():v sql.rsColumn(X_25,"sys.","fuse_a","smallint",16,0,X_23); X_30 := io.stdout(); sql.exportResult(X_30,X_25); -end s11_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s5_1; #explain select fuse(c,d) from udf_fuse; % .explain # table_name % mal # name % clob # type % 57 # length -function user.s12_1{autoCommit=true}():void; +function user.s6_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_fuse"); X_6 := sql.bind(X_2,"sys","udf_fuse","c",0); @@ -141,15 +135,13 @@ function user.s12_1{autoCommit=true}():v sql.rsColumn(X_25,"sys.","fuse_c","int",32,0,X_23); X_30 := io.stdout(); sql.exportResult(X_30,X_25); -end s12_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s6_1; #explain select fuse(e,f) from udf_fuse; % .explain # table_name % mal # name % clob # type % 58 # length -function user.s13_1{autoCommit=true}():void; +function user.s7_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_fuse"); X_6 := sql.bind(X_2,"sys","udf_fuse","e",0); @@ -167,9 +159,7 @@ function user.s13_1{autoCommit=true}():v sql.rsColumn(X_25,"sys.","fuse_e","bigint",64,0,X_23); X_30 := io.stdout(); sql.exportResult(X_30,X_25); -end s13_1; -# optimizer.mitosis() -# optimizer.dataflow() +end s7_1; #select fuse(a,b) from udf_fuse; % sys. # table_name % fuse_a # name diff --git a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out --- a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out +++ b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out @@ -82,7 +82,7 @@ end s0_1; % mal # name % clob # type % 62 # length -function user.s7_1{autoCommit=true}():void; +function user.s3_1{autoCommit=true}():void; X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_reverse"); X_6 := sql.bind(X_2,"sys","udf_reverse","x",0); @@ -95,7 +95,7 @@ function user.s7_1{autoCommit=true}():vo sql.rsColumn(X_18,"sys.","reverse_x":str,"clob",0,0,X_16); X_22 := io.stdout(); sql.exportResult(X_22,X_18); -end s7_1; +end s3_1; #select reverse(x) from udf_reverse; % sys. # table_name % reverse_x # name diff --git a/sql/server/sql_mvc.c b/sql/server/sql_mvc.c --- a/sql/server/sql_mvc.c +++ b/sql/server/sql_mvc.c @@ -193,8 +193,6 @@ mvc_trans(mvc *m) qc_clean(m->qc); } } - if (m->session->active) - m->type = Q_TRANS; store_unlock(); } diff --git a/sql/server/sql_parser.y b/sql/server/sql_parser.y --- a/sql/server/sql_parser.y +++ b/sql/server/sql_parser.y @@ -3954,7 +3954,7 @@ literal: string { char *s = sql2str($1); int len = _strlen(s); sql_subtype t; - sql_find_subtype(&t, "char", len, 0 ); + sql_find_subtype(&t, "varchar", len, 0 ); $$ = _newAtomNode( _atom_string(&t, s)); } | HEXADECIMAL { int len = _strlen($1), i = 2, err = 0; diff --git a/sql/server/sql_qc.c b/sql/server/sql_qc.c --- a/sql/server/sql_qc.c +++ b/sql/server/sql_qc.c @@ -143,6 +143,11 @@ param_cmp(sql_subtype *t1, sql_subtype * if (t1->scale != t2->scale) return -1; + + if (t1->type->eclass == EC_NUM && t2->type->eclass == EC_NUM && + t1->digits >= t2->digits) + return 0; + res = is_subtype(t2, t1); if (!res) return -1; @@ -178,7 +183,7 @@ param_list_cmp(sql_subtype *typelist, at if ((!((at->type->eclass == EC_DEC || at->type->eclass == EC_NUM) && tp->type->eclass == EC_FLT)) && - (!(EC_VARCHAR(tp->type->eclass) && + (!(EC_VARCHAR(tp->type->eclass) && EC_VARCHAR(at->type->eclass) && (!tp->digits || tp->digits >= at->digits))) && diff --git a/sql/test/ADT2006/Tests/bram.stable.out b/sql/test/ADT2006/Tests/bram.stable.out --- a/sql/test/ADT2006/Tests/bram.stable.out +++ b/sql/test/ADT2006/Tests/bram.stable.out @@ -458,7 +458,7 @@ Ready. #(SELECT 'Major' AS grade, count(na % ., .inspector # table_name % grade, inspectors # name -% char, wrd # type +% varchar, wrd # type % 9, 1 # length [ "Detective", 3 ] [ "Constable", 3 ] @@ -475,7 +475,7 @@ Ready. # + cast(substring(length from (position('\'' in length) + 1) for (position('"' in length) + 1)) AS integer)) as height from victim v % .L1, . # table_name % L1, average_height # name -% char, char # type +% varchar, varchar # type % 6, 4 # length [ "blonde", "5'7\"" ] [ "brown", "5'5\"" ] diff --git a/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out b/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out --- a/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out +++ b/sql/test/BugDay_2005-10-06_2.8/Tests/string_quotes.SF-919539.stable.out @@ -20,12 +20,12 @@ Ready. % . # table_name % single_value # name -% char # type +% varchar # type % 2 # length [ "aa" ] % . # table_name % single_value # name -% char # type +% varchar # type % 3 # length [ "a'a" ] diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/UNION_JOIN_vs_plain_set_UNION-2.SF-920585.stable.out @@ -101,7 +101,7 @@ Ready. [ "users", "SELECT u.\"t\" AS \"name\", ui.\"fullname\", ui.\"default_schema\" FROM \"sys\".\"db_users\" AS u, \"sys\".\"db_user_info\" AS ui WHERE u.\"h\" = ui.\"id\" AND u.\"h\" NOT IN (SELECT s.\"h\" FROM \"sys\".\"db_scens\" AS s WHERE s.\"t\" NOT LIKE 'sql');", false, true, 0, 0 ] % .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables # table_name % table_cat, table_schem, table_name, table_type, remarks, type_cat, type_schem, type_name, self_referencing_col_name, ref_generation # name -% char, varchar, varchar, char, varchar, char, char, char, char, char # type +% varchar, varchar, varchar, varchar, varchar, varchar, varchar, varchar, varchar, varchar # type % 4, 3, 8, 12, 222, 4, 4, 4, 2, 6 # length [ "null", "sys", "columns", "SYSTEM TABLE", "", "null", "null", "null", "id", "SYSTEM" ] [ "null", "sys", "db_scens", "SYSTEM TABLE", "", "null", "null", "null", "id", "SYSTEM" ] diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/basic_arithmetics.SF-922252.stable.out @@ -63,8 +63,9 @@ Ready. [ 1 ] % . # table_name % single_value # name -% char # type -% 1 # length +% varchar # type +% 0 # length +#select * from my_table where 1 = 0; % sys.my_table # table_name % att # name % int # type diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/case.SF-929151.stable.out @@ -31,7 +31,7 @@ Ready. [ 1 ] % sys. # table_name % ifthenelse_isnull_=_i # name -% char # type +% varchar # type % 23 # length [ "base table" ] [ "system table" ] diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/jdbc_crashes_on_getTables.SF_944316.stable.out @@ -43,8 +43,8 @@ Ready. #ORDER BY "TABLE_TYPE", % .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables # table_name % TABLE_CAT, TABLE_SCHEM, TABLE_NAME, TABLE_TYPE, REMARKS, TYPE_CAT, TYPE_SCHEM, TYPE_NAME, SELF_REFERENCING_COL_NAME, REF_GENERATION # name -% char, varchar, varchar, char, char, char, char, char, char, char # type -% 0, 0, 0, 12, 0, 0, 0, 0, 2, 6 # length +% char, varchar, varchar, varchar, varchar, char, char, char, varchar, varchar # type +% 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 # length # 11:50:19 > # 11:50:19 > Done. diff --git a/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out b/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out --- a/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out +++ b/sql/test/BugDay_2005-10-06_2.9.3/Tests/simple_union.SF-1005596.stable.out @@ -26,8 +26,8 @@ Ready. % .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables, .tables # table_name % TABLE_CAT, TABLE_SCHEM, TABLE_NAME, TABLE_TYPE, REMARKS, TYPE_CAT, TYPE_SCHEM, TYPE_NAME, SELF_REFERENCING_COL_NAME, REF_GENERATION # name -% char, varchar, varchar, char, char, char, char, char, char, char # type -% 4, 3, 4, 20, 0, 0, 0, 0, 5, 6 # length +% varchar, varchar, varchar, varchar, varchar, char, char, char, varchar, varchar # type _______________________________________________ checkin-list mailing list checkin-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/checkin-list _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Announcement: New Oct2012-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Dec '12

21 Dec '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2012-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Oct 2012-SP2 bugfix release SQL * Fixed crash when performing an INSERT on a table with string-like column defaulting to NULL and omitting that column from VALUES, bug #3168 Java Module * Implemented type map support of Connection to allow custom mapping of UDTs to Java classes. By default the INET and URL UDTs are now mapped to nl.cwi.monetdb.jdbc.types.{INET,URL}. Most notably, ResultSet.getObject() and PreparedStatement.setObject() deal with the type map. * Fixed a problem in PreparedStatement where the prepared statement's ResultSetMetaData (on its columns to be produced) incorrectly threw exceptions about non existing columns. Bug #3192 Bug Fixes * 2291: small doubles end up as NULL after arithmetic * 2579: SQL: incorrect cast from double to int * 2659: tuples INTERSECT vs count (... INTERSECT ...) differs * 2805: row_number doesn't work in complex query * 2886: warning: Catalog parameter '' ignored * 2977: timestamp minus date results in program error * 2978: now() + 1 results in illegal argument * 2999: Up-cast from SQL drops order information? * 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause * 3124: insert into table with two foreign keys crashes mserver5 * 3136: Memleak (in querycache?) while querying * 3143: Type resolution error in SQL procedural code * 3163: mserver5 crashes with large number of concurrent clients * 3168: crash when insert varchar/char/clob with default NULL * 3169: php driver uses call-time pass-by-reference * 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either * 3181: Join query returns zero results when there is an index on the tables * 3185: inconsistency between documentation and implementation of command line options * 3186: case evaluted to early * 3188: Storage of client history * 3189: Overflow in calculation * 3190: When subtracting two columns of type "timestamp", no results are returned * 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed. * 3192: "No such column with index" error if CASE in SELECT clause * 3199: Inconspicuous sequence of prepare, execute, commit and close produces error * 3205: Inet datatype handling inconsistent between int, cast( as inet) and string _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

readline and the Mac build
by Masood Mortazavi 21 Dec '12

21 Dec '12

It seems "readline" isn't linked in the Mac build and installation. Configure or package repository don't indicate any issues, and yet history completions fail on mclient's sql prompt. Regards, -m. $ sudo port list | grep readline gauche-readline @0.1 devel/gauche-readline hs-readline @1.0.1.0 devel/hs-readline pure-readline @0.1 devel/pure-readline readline @6.2.000 devel/readline readline-4 @4.3 devel/readline-4 readline-5 @5.2.014 devel/readline-5 ... ... $ more config.log | grep readline configure:18493: checking readline/readline.h usability configure:18493: checking readline/readline.h presence configure:18493: checking for readline/readline.h configure:18495: checking for readline in -lreadline configure:18520: gcc -o conftest -g conftest.c -lreadline >&5 configure:18631: checking for rl_completion_matches in -lreadline configure:18656: gcc -o conftest -g conftest.c -lreadline -lreadline >&5 | #include <readline/readline.h> ac_cv_header_readline_readline_h=yes ac_cv_lib_readline_readline=yes ac_cv_lib_readline_rl_completion_matches=yes _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

4 8

Re: MonetDB: Feb2013 - Properly realloc the namespace
by Stefan Manegold 19 Dec '12

19 Dec '12

Martin, assertions are not the most suitable way to check for malloc failures. Assertions are to *assert* facts that must hold, i.e., the would be a logical error if they do not hold. Consequently, assertions are only enabled during development and testing, but disabled (for among others for performance reasons) in any production builds. If triggered, assertions stop the entire process instantly. Failing mallocs is a runtime exception that can occur at any time for various reasons. The code should check for this and gracefully handle such situations with proper exeption handling. Having said that, in particular is the man namespace is local per query/client, a failure in one query/client should not crash the whole server by an assertion (during development/testing) or a segfault (in production builds), but rather only greacefull trigger and handle an exception for that one query/client. Also, if namespaces are global and a malloc failure prevent the whole server from continuing properly, it should at least (try to) gracefully shut itself down (with a proper error message), rather than crashing instantly with an assertion of segfault. Best, Stefan On Wed, Dec 19, 2012 at 11:53:37AM +0100, Martin Kersten wrote: > Changeset: 2a9bd075c039 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=2a9bd075c039 > Modified Files: > monetdb5/mal/mal_namespace.c > Branch: Feb2013 > Log Message: > > Properly realloc the namespace > The expandName always lead to a different anchoring point of the namespace. > This may leave concurrent threads with dead meat. > This patch reduces the chance on a SEGfault. > > > diffs (37 lines): > > diff --git a/monetdb5/mal/mal_namespace.c b/monetdb5/mal/mal_namespace.c > --- a/monetdb5/mal/mal_namespace.c > +++ b/monetdb5/mal/mal_namespace.c > @@ -66,26 +66,13 @@ typedef struct NAMESPACE{ > static Namespace namespace; > > static void expandNamespace(void){ > - str *nme; > - size_t *length; > - int *link, newsize, incr = 2048; > - > - newsize= namespace.size + incr; > - nme= (str *) GDKmalloc(sizeof(str *) * newsize); > - assert(nme != NULL); /* we cannot continue */ > - link= (int *) GDKmalloc(sizeof(int) * newsize); > - assert(link != NULL); /* we cannot continue */ > - length = (size_t *) GDKmalloc(sizeof(size_t) * newsize); > - assert(length != NULL); /* we cannot continue */ > - > - memcpy(nme, namespace.nme, sizeof(str *) * namespace.nmetop); > - memcpy(link, namespace.link, sizeof(int) * namespace.nmetop); > - memcpy(length, namespace.length, sizeof(size_t) * namespace.nmetop); > - > - namespace.size += incr; > - GDKfree(namespace.nme); namespace.nme= nme; > - GDKfree(namespace.link); namespace.link= link; > - GDKfree(namespace.length); namespace.length= length; > + namespace.size += 2048; > + namespace.nme= (str *) GDKrealloc(namespace.nme, sizeof(str *) * namespace.size); > + assert(namespace.nme != NULL); /* we cannot continue */ > + namespace.link= (int *) GDKrealloc(namespace.link, sizeof(int) * namespace.size); > + assert(namespace.link != NULL); /* we cannot continue */ > + namespace.length = (size_t *) GDKrealloc(namespace.length, sizeof(size_t) * namespace.size); > + assert(namespace.length != NULL); /* we cannot continue */ > } > > void initNamespace(void) { > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: Oct2012 - fixed bug 3209, ie first crossproduct if the ...
by Ying Zhang 16 Dec '12

16 Dec '12

Hai Niels, Your fix works in Feb2013, but not in Oct2012… Jennie On Dec 15, 2012, at 12:29, Niels Nes wrote: > Changeset: 8650dce52dd8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=8650dce52dd8 > Modified Files: > sql/server/rel_exp.c > sql/server/rel_exp.h > Branch: Oct2012 > Log Message: > > fixed bug 3209, ie first crossproduct if the join expression consists > of functions (these should have been pushed down, whenever possible). > > > diffs (59 lines): > > diff --git a/sql/server/rel_exp.c b/sql/server/rel_exp.c > --- a/sql/server/rel_exp.c > +++ b/sql/server/rel_exp.c > @@ -1032,6 +1032,44 @@ exp_is_atom( sql_exp *e ) > } > > static int > +exps_has_func( list *exps) > +{ > + node *n; > + int has_func = 0; > + > + for(n=exps->h; n && !has_func; n=n->next) > + has_func |= exp_has_func(n->data); > + return has_func; > +} > + > +int > +exp_has_func( sql_exp *e ) > +{ > + switch (e->type) { > + case e_atom: > + return 0; > + case e_convert: > + return exp_has_func(e->l); > + case e_func: > + case e_aggr: > + return 1; > + case e_cmp: > + if (e->flag == cmp_or) { > + return (exps_has_func(e->l) || exps_has_func(e->r)); > + } else if (e->flag == cmp_in || e->flag == cmp_notin || e->flag == cmp_filter) { > + return (exp_has_func(e->l) || exps_has_func(e->r)); > + } else { > + return (exp_has_func(e->l) || exp_has_func(e->r) || > + (e->f && exp_has_func(e->f))); > + } > + case e_column: > + case e_psm: > + return 0; > + } > + return 0; > +} > + > +static int > exp_key( sql_exp *e ) > { > if (e->name) > diff --git a/sql/server/rel_exp.h b/sql/server/rel_exp.h > --- a/sql/server/rel_exp.h > +++ b/sql/server/rel_exp.h > @@ -105,6 +105,7 @@ extern int exp_is_eqjoin(sql_exp *e); > extern int exp_is_correlation(sql_exp *e, sql_rel *r ); > extern int exp_is_join_exp(sql_exp *e); > extern int exp_is_atom(sql_exp *e); > +extern int exp_has_func(sql_exp *e); > > extern sql_exp *exps_bind_column( list *exps, char *cname, int *ambiguous); > extern sql_exp *exps_bind_column2( list *exps, char *rname, char *cname); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

1 0

Re: MonetDB: default - fixed a low level bug where empty mapi result...
by Sjoerd Mullender 14 Dec '12

14 Dec '12

On 2012-12-14 12:10, Gijs Molenaar wrote: > Changeset: 81f191d61822 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=81f191d61822 > Modified Files: > clients/python2/monetdb/mapi.py > clients/python3/monetdb/mapi.py > Branch: default > Log Message: > > fixed a low level bug where empty mapi results where not properly handled Is this is fix that should have gone to the Oct2012 branch? Please, make sure you fix bugs in the lowest branch where the bug occurs and that is still maintained. Currently the still maintained branches are Oct2012, Feb2013 and default. > > diffs (30 lines): > > diff --git a/clients/python2/monetdb/mapi.py b/clients/python2/monetdb/mapi.py > --- a/clients/python2/monetdb/mapi.py > +++ b/clients/python2/monetdb/mapi.py > @@ -163,9 +163,9 @@ class Connection(object): > self.__putblock(operation) > response = self.__getblock() > if not len(response): > - return True > + return "" > elif response.startswith(MSG_OK): > - return response[3:].strip() or True > + return response[3:].strip() or "" Isn't this equivalent to: return response[3:].strip() > if response == MSG_MORE: > # tell server it isn't going to get more > return self.cmd("") > diff --git a/clients/python3/monetdb/mapi.py b/clients/python3/monetdb/mapi.py > --- a/clients/python3/monetdb/mapi.py > +++ b/clients/python3/monetdb/mapi.py > @@ -163,9 +163,9 @@ class Connection(object): > self.__putblock(operation) > response = self.__getblock() > if not len(response): > - return True > + return "" > elif response.startswith(MSG_OK): > - return response[3:].strip() or True > + return response[3:].strip() or "" > if response == MSG_MORE: > # tell server it isn't going to get more > return self.cmd("") > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

group_concat
by Tapomay Dey 13 Dec '12

13 Dec '12

Almost there. As suggested by Fabian in the following thread I gave up the MX file approach and implemented group_concat as UDF in C. http://sourceforge.net/mailarchive/forum.php?thread_name=mailman.243.135532… When I try to conclude my function UDFBATgroup_concat_(BAT **ret, BAT *b, BAT *g, BAT *h, FILE* fp) //ignore fp as follows: BATaccessBegin(h, USE_HEAD | USE_TAIL, MMAP_SEQUENTIAL); index = 0; BATloop(h, p, q) { ptr hhead = BUNhead(hi, p); fprintf(fp, "%d: hhead=%d", index, hhead); str dst = tailValArray[index]; fprintf(fp, "%d: dstStr=%s# ", index, dst); BUNins(bn, hhead, dst, FALSE); fprintf(fp, "%d: Insert complete\n", index); index++; } I get the following error on the client terminal [tapomay@tapomay-laptop monetdata]$ echo "select house, group_concat(name) from gs group by house;" | mclient -d tapomay GDKerror:!ERROR: BUNins: access denied to tmp_10162, aborting. BUNins: access denied to tmp_10162, aborting. BUNins: access denied to tmp_10162, aborting. The groupings are ready in the tailVarArray. NEED HELP. Thanks and Regards, Tapomay. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1

Still searching to find the Octopus :-)
by Masood Mortazavi 12 Dec '12

12 Dec '12

I'm still searching for user documentation that would explain MonetDB/octopus usage/feature. Does any exist? Thanks, Masood P.S. My earlier e-mails regarding the octopus documentation was somewhat rushed, it seems. The documentation in the man pages are actually for remtoe access and funnel-multiplex features, which seem entirely different. _______________________________________________ developers-list mailing list developers-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list

2 1