- developers-list - monetdb.org

Questions About Generated MAL Code
by Stephen P. Morgan 14 Mar '13

14 Mar '13

Hi, My colleagues and I from Huawei have been experimenting with the MAL code generated by SQL explain. We noticed big differences between the code generated by version 15 and the present top-of-tree. For example, where the level-15 code tended to generate just one barrier-exit block for a simple "select *," the new code generates several. (That might have to do with views--that's my guess, anyway.) Unfortunately, whereas the "old" (level 15) code is manually executable by the MAL interpreter, the new code doesn't run directly. For example, the interpreter complains about uninitialized variables. Of course, this is straightforward to fix manually, though somewhat tedious. But we also find when running multiple-barrier code, we get many complaints about undefined things, e.g., sql.mvc(), sql.bind(), among others. We don't (yet?) know how to overcome this issue. Is this just a temporary situation, or will it be possible at some point again to manually run MAL code? (Is it a policy of the MonetDB project that explained MAL code should be manually executable?) Would it be possible to explain (briefly) the reasons for the differences between the two versions of generated MAL code? We're curious what would prompt such a big change in the generated code. Thank you, Steve Morgan P.S. I've created a version of the mapi.connect() and mapi.reconnect() code that accepts a database name as another parameter. We've used it to send RPCs between systems (Linux containers, actually). We would be happy to donate this code to your organization, if you would like it. I added a new MAL routine for mapi.connect(), i.e., SERVERconnectWithDBname() and similar things for the mapi.reconnect() calls that detects the extra parameter and uses it if it's there. It shouldn't break any existing code, and it allows one to run RPCs while using monetdbd (not just mserver5).

1 0

Mercurial Source Missing opt_history.c
by Stephen P. Morgan 13 Mar '13

13 Mar '13

Hi MonetDB'ers, The present version of the Merucrial source for MonetDB seems not to build. I got the following error from make when I tried, even after a make clean: make[5]: *** No rule to make target `opt_history.c', needed by `liboptimizer_la-opt_history.lo'. Stop. hg log -l1 says: changeset: 47196:ba73b2c5ad5e tag: tip parent: 47194:9f63c9d7da57 parent: 47195:2c80719b8615 user: Stefan Manegold <Stefan.Manegold(a)cwi.nl> date: Wed Mar 13 13:20:55 2013 +0100 summary: Merge with Feb2013 branch. Thanks, Steve

2 2

Re: MonetDB: default - rewrote result parser to make it eat multiple...
by Gijs Molenaar 12 Mar '13

12 Mar '13

Now I think about it a bit more I think this is not a new feature but a bugfix, i can apply this commit also on the feb2013 branch if you guys want. On 11/03/13 18:44, Gijs Molenaar wrote: > Changeset: 39b2905626b8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=39b2905626b8 > Modified Files: > clients/python2/monetdb/sql/cursors.py > clients/python2/test/capabilities.py > clients/python3/monetdb/sql/cursors.py > clients/python3/test/capabilities.py > Branch: default > Log Message: > > rewrote result parser to make it eat multiple resultsets -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

3 2

Re: MonetDB: default - rewrote result parser to make it eat multiple...
by Stefan Manegold 12 Mar '13

12 Mar '13

Dear Gijs, thank you very much for continuously taking care of MonetDB's python interface / API! We very muchappreciate your work and contribution! However, we'd appreciate even more, it you could also keep the stable output up-to-date whenever you change / extend you test or your changes intendedly change the output of your tests. See http://monetdb.cwi.nl/testweb/web/status.php http://monetdb.cwi.nl/testweb/web/testgrid.php?serial=47169:7d1c376e995f&or… http://monetdb.cwi.nl/testweb/web/testgrid.php?serial=47169:7d1c376e995f&or… http://monetdb.cwi.nl/testweb/web/showtestoutput.php?serial=47169:7d1c376e9… http://monetdb.cwi.nl/testweb/web/testchange.php?test=47169:7d1c376e995f/GN… If nothing else, simply go to .../MonetDB/sql/test/mapi/Tests/ and run `Mtest.py` to run (among others) your pyhton test (preferably on a system with both Python2 & python3 installed, and follow the instructions. See also `Mtest.py --help`, `Mapprove.py --help` and http://dev.monetdb.org/hg/MonetDB/file/tip/testing/README Thanks! Stefan On Mon, Mar 11, 2013 at 06:44:43PM +0100, Gijs Molenaar wrote: > Changeset: 39b2905626b8 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=39b2905626b8 > Modified Files: > clients/python2/monetdb/sql/cursors.py > clients/python2/test/capabilities.py > clients/python3/monetdb/sql/cursors.py > clients/python3/test/capabilities.py > Branch: default > Log Message: > > rewrote result parser to make it eat multiple resultsets > > > diffs (truncated from 463 to 300 lines): > > diff --git a/clients/python2/monetdb/sql/cursors.py b/clients/python2/monetdb/sql/cursors.py > --- a/clients/python2/monetdb/sql/cursors.py > +++ b/clients/python2/monetdb/sql/cursors.py > @@ -380,133 +380,99 @@ class Cursor(object): > if not block: > block = "" > > - lines = block.split("\n") > - firstline = lines[0] > + for line in block.split("\n"): > + if line.startswith(mapi.MSG_INFO): > + logger.info(line[1:]) > + self.messages.append((Warning, line[1:])) > > - while firstline.startswith(mapi.MSG_INFO): > - logger.info(firstline[1:]) > - self.messages.append((Warning, firstline[1:])) > - lines = lines[1:] > - firstline = lines[0] > + elif line.startswith(mapi.MSG_QTABLE): > + (self.__query_id, rowcount, columns, tuples) = line[2:].split() > > - if firstline.startswith(mapi.MSG_QTABLE): > - (id, rowcount, columns, tuples) = firstline[2:].split() > - columns = int(columns) # number of columns in result > - rowcount = int(rowcount) # total number of rows > - tuples = int(tuples) # number of rows in this set > - rows = [] > + columns = int(columns) # number of columns in result > + self.rowcount = int(rowcount) # total number of rows > + tuples = int(tuples) # number of rows in this set > + self.__rows = [] > > - # set up fields for description > - table_name = [None]*columns > - column_name = [None]*columns > - type_ = [None]*columns > - display_size = [None]*columns > - internal_size = [None]*columns > - precision = [None]*columns > - scale = [None]*columns > - null_ok = [None]*columns > + # set up fields for description > + table_name = [None] * columns > + column_name = [None] * columns > + type_ = [None] * columns > + display_size = [None] * columns > + internal_size = [None] * columns > + precision = [None] * columns > + scale = [None] * columns > + null_ok = [None] * columns > + typesizes = [(0, 0)] * columns > + self.__offset = 0 > + self.lastrowid = None > > - typesizes = [(0,0)]*columns > + elif line.startswith(mapi.MSG_HEADER): > + (data, identity) = line[1:].split("#") > + values = [x.strip() for x in data.split(",")] > + identity = identity.strip() > > - for line in lines[1:]: > - if line.startswith(mapi.MSG_HEADER): > - (data, identity) = line[1:].split("#") > - values = [x.strip() for x in data.split(",")] > - identity = identity.strip() > + if identity == "table_name": > + table_name = values > + elif identity == "name": > + column_name = values > + elif identity == "type": > + type_ = values > + elif identity == "length": > + length = values > + elif identity == "typesizes": > + typesizes = [[int(j) for j in i.split()] for i in values] > + internal_size = [x[0] for x in typesizes] > + for num, typeelem in enumerate(type_): > + if typeelem in ['decimal']: > + precision[num] = typesizes[num][0] > + scale[num] = typesizes[num][1] > + else: > + msg = "unknown header field" > + self.messages.append((InterfaceError, msg)) > + self.__exception_handler(InterfaceError, msg) > > - if identity == "table_name": > - table_name = values # not used > - elif identity == "name": > - column_name = values > - elif identity == "type": > - type_ = values > - elif identity == "length": > - pass # not used > - elif identity == "typesizes": > - typesizes = [[int(j) for j in i.split()] for i in values] > - internal_size = [x[0] for x in typesizes] > - for num, typeelem in enumerate(type_): > - if typeelem in ['decimal']: > - precision[num] = typesizes[num][0] > - scale[num] = typesizes[num][1] > - else: > - self.messages.append((InterfaceError, > - "unknown header field")) > - self.__exception_handler(InterfaceError, > - "unknown header field") > + self.description = list(zip(column_name, type_, display_size, > + internal_size, precision, scale, > + null_ok)) > + self.__offset = 0 > + self.lastrowid = None > > - self.description = list(zip(column_name, type_, > - display_size, internal_size, precision, scale, null_ok)) > > - if line.startswith(mapi.MSG_TUPLE): > - values = self.__parse_tuple(line) > - rows.append(values) > + elif line.startswith(mapi.MSG_TUPLE): > + values = self.__parse_tuple(line) > + self.__rows .append(values) > > - elif line == mapi.MSG_PROMPT: > - self.__query_id = id > - self.__rows = rows > - self.__offset = 0 > + elif line.startswith(mapi.MSG_QBLOCK): > + self.__rows = [] > > - self.rowcount = rowcount > - self.lastrowid = None > - logger.debug("II store result finished") > - return > - > - elif firstline.startswith(mapi.MSG_QBLOCK): > - rows = [] > - for line in lines[1:]: > - if line.startswith(mapi.MSG_TUPLE): > - values = self.__parse_tuple(line) > - rows.append(values) > - elif line == mapi.MSG_PROMPT: > - logger.debug("II store result finished") > - self.__rows = rows > - return > - > - elif firstline.startswith(mapi.MSG_QSCHEMA): > - if lines[1] == mapi.MSG_PROMPT: > + elif line.startswith(mapi.MSG_QSCHEMA): > + self.__offset = 0 > + self.lastrowid = None > self.__rows = [] > - self.__offset = 0 > self.description = None > self.rowcount = -1 > - self.lastrowid = None > - logger.debug("II schema finished") > - return > > - elif firstline.startswith(mapi.MSG_QUPDATE): > - if lines[1] == mapi.MSG_PROMPT: > - (affected, identity) = firstline[2:].split() > + elif line.startswith(mapi.MSG_QUPDATE): > + (affected, identity) = line[2:].split() > + self.__offset = 0 > self.__rows = [] > - self.__offset = 0 > self.description = None > self.rowcount = int(affected) > self.lastrowid = int(identity) > self.__query_id = -1 > - logger.debug("II update finished") > + > + elif line.startswith(mapi.MSG_QTRANS): > + self.__offset = 0 > + self.lastrowid = None > + self.__rows = [] > + self.description = None > + self.rowcount = -1 > + > + elif line.startswith(mapi.MSG_PROMPT): > return > > - elif firstline.startswith(mapi.MSG_ERROR): > - self.__exception_handler(ProgrammingError, firstline[1:]) > - > - elif firstline.startswith(mapi.MSG_QTRANS): > - if lines[1] == mapi.MSG_PROMPT: > - self.__rows = [] > - self.__offset = 0 > - self.description = None > - self.rowcount = -1 > - self.lastrowid = None > - logger.debug("II transaction finished") > - return > - > - elif firstline.startswith(mapi.MSG_PROMPT): > - self.__query_id = -1 > - self.__rows = [] > - self.__offset = 0 > - > - self.rowcount = 0 > - self.lastrowid = None > - logger.debug("II empty response, assuming everything is ok") > - return > + elif line.startswith(mapi.MSG_ERROR): > + self.__exception_handler(ProgrammingError, line[1:]) > > # you are not supposed to be here > self.__exception_handler(InterfaceError, "Unknown state, %s" % block) > diff --git a/clients/python2/test/capabilities.py b/clients/python2/test/capabilities.py > --- a/clients/python2/test/capabilities.py > +++ b/clients/python2/test/capabilities.py > @@ -352,4 +352,16 @@ class DatabaseTest(unittest.TestCase): > t = ["list", "test"] > self.assertRaises(ProgrammingError, self.db_module.monetize.convert, t) > self.db_module.monetize.mapping[list] = str > - self.assertEqual(self.db_module.monetize.convert(t), "['list', 'test']") > \ No newline at end of file > + self.assertEqual(self.db_module.monetize.convert(t), "['list', 'test']") > + > + def test_multiple_queries(self): > + table1 = self.new_table_name() > + table2 = table1[:-1] + 'bla"' > + self.cursor.execute("create table %s (a int)" % table1) > + self.cursor.execute("create table %s (a int, b int)" % table2) > + self.cursor.execute("insert into %s VALUES (100)" % table1) > + self.cursor.execute("insert into %s VALUES (50, 50)" % table2) > + self.cursor.execute('select * from %s; select * from %s;' % > + (table1, table2)) > + result = self.cursor.fetchall() > + self.assertEqual(result, [(50, 50)]) > \ No newline at end of file > diff --git a/clients/python3/monetdb/sql/cursors.py b/clients/python3/monetdb/sql/cursors.py > --- a/clients/python3/monetdb/sql/cursors.py > +++ b/clients/python3/monetdb/sql/cursors.py > @@ -381,133 +381,99 @@ class Cursor(object): > if not block: > block = "" > > - lines = block.split("\n") > - firstline = lines[0] > + for line in block.split("\n"): > + if line.startswith(mapi.MSG_INFO): > + logger.info(line[1:]) > + self.messages.append((Warning, line[1:])) > > - while firstline.startswith(mapi.MSG_INFO): > - logger.info(firstline[1:]) > - self.messages.append((Warning, firstline[1:])) > - lines = lines[1:] > - firstline = lines[0] > + elif line.startswith(mapi.MSG_QTABLE): > + (self.__query_id, rowcount, columns, tuples) = line[2:].split() > > - if firstline.startswith(mapi.MSG_QTABLE): > - (id, rowcount, columns, tuples) = firstline[2:].split() > - columns = int(columns) # number of columns in result > - rowcount = int(rowcount) # total number of rows > - tuples = int(tuples) # number of rows in this set > - rows = [] > + columns = int(columns) # number of columns in result > + self.rowcount = int(rowcount) # total number of rows > + tuples = int(tuples) # number of rows in this set > + self.__rows = [] > > - # set up fields for description > - table_name = [None]*columns > - column_name = [None]*columns > - type_ = [None]*columns > - display_size = [None]*columns > - internal_size = [None]*columns > - precision = [None]*columns > - scale = [None]*columns > - null_ok = [None]*columns > + # set up fields for description > + table_name = [None] * columns > + column_name = [None] * columns > + type_ = [None] * columns > + display_size = [None] * columns > + internal_size = [None] * columns > + precision = [None] * columns > + scale = [None] * columns > + null_ok = [None] * columns > + typesizes = [(0, 0)] * columns > + self.__offset = 0 > + self.lastrowid = None > > - typesizes = [(0,0)]*columns > + elif line.startswith(mapi.MSG_HEADER): > + (data, identity) = line[1:].split("#") > + values = [x.strip() for x in data.split(",")] > + identity = identity.strip() > > - for line in lines[1:]: > - if line.startswith(mapi.MSG_HEADER): > - (data, identity) = line[1:].split("#") > - values = [x.strip() for x in data.split(",")] > - identity = identity.strip() > + if identity == "table_name": > + table_name = values > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 2

mapi.connect / mapi_connect (Second Try)
by Stephen P. Morgan 12 Mar '13

12 Mar '13

I wrote an email a few days ago to the effect that I could not get the mapi.connect() MAL statement to work due to the server side complaining that I had not specified the database to use. This morning I ran one of the sample programs included with MonetDB, clients/examples,C/sample0.c (sample0). It seems very similar as it issues a mapi_connect() with the database field set to NULL. It fails in the same way, i.e., MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !monetdbd: please specify a database Should mapi_connect() work with the database field set to NULL? (If so, then the mapi.connect() MAL statement should work, I think.) Is there a bug in the server that insists on the database being specified? If there is a bug, does anyone have a fix for this? It would be greatly appreciated. Thanks, Steve Morgan

3 7

Coredump caused by function SERVERreconnectWithoutAlias() and its verifies fix
by Fang Cao 12 Mar '13

12 Mar '13

Hi, Today when I try to use mapi.connect() and mapi.reconnect() with mserver5, I found such crash when I tried to : fang@fang-ThinkCentre-M91p:~/Work/DB-Farm$ mserver5 --dbpath=/home/fang/Work/DB-Farm # MonetDB 5 server v11.15.1 "Feb2013" # Serving database 'DB-Farm', using 8 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # Found 3.692 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # MonetDB/JAQL module loaded # MonetDB/SQL module loaded >mid:=mapi.reconnect("localhost",50000,"monetdb","monetdb","mal"); Segmentation fault (core dumped) The root cause is located at: SERVERreconnectWithoutAlias(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci) { int *key =(int*) getArgReference(stk,pci,0); str *host = (str*) getArgReference(stk,pci,1); int *port = (int*) getArgReference(stk,pci,2); str *username = (str*) getArgReference(stk,pci,4); ===>> should be 3 str *password = (str*) getArgReference(stk,pci,5); ===>> should be 4 str *lang = (str*) getArgReference(stk,pci,6); ===>> should be 5 Since we only transfer 5 input parameters, so for the last one “lang”, of course we will coredump the program when try to check 6th. After the modification, everything is fine now, for both mapi.connect() and mapi.reconnect() The reason why this issue is introduced: pattern reconnect(host:str, port:int, usr:str, passwd:str,lang:str):int address SERVERreconnectWithoutAlias comment "Re-establish connection with a remote mserver."; pattern reconnect(host:str, port:int, db_alias:str, usr:str, passwd:str,lang:str):int address SERVERreconnectAlias comment "Re-establish connection with a remote mserver."; So when to implement SERVERreconnectWithoutAlias(), seems following the style of SERVERreconnectAlias(). The latter one has 6 inputs, but the first one has only 5 inputs. So un-needed db_alias was removed, but forgot to adjust the sequence number. Thanks, Fang

2 1

Re: MonetDB: Feb2013 - opt_support.c & opt_wrapper.c: "coercion" -> ...
by Stefan Manegold 12 Mar '13

12 Mar '13

> Changeset: 30c2d470a73d for MonetDB > URL: > http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=30c2d470a73d > Modified Files: > monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > monetdb5/optimizer/opt_support.c > monetdb5/optimizer/opt_wrapper.c > sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > Branch: Feb2013 > Log Message: > > opt_support.c & opt_wrapper.c: "coercion" -> "coercions", > i.e., using the correct optimizer name "coercions" > to ensure the optimizer is indeed activated/used. > > Approved related changes in test output (modified MAL plans > due to now active coercions optimizer). > > This should fix bug 3151. ^!^^ Typo: this should read bug 3251 > Thanks to Pete Hollobon for reporting and proving the initial fix! > > > Open question: > > Why is an incorrect / non-exiting optimizer name silently ignored?? > > > diffs (truncated from 413 to 300 lines): > > diff --git > a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > --- > a/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > +++ > b/monetdb5/extras/mal_optimizer_template/Tests/opt_sql_append.stable.out > @@ -61,15 +61,15 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 178 # length > +% 153 # length > function user.s2_1{autoCommit=true}():void; > X_2 := sql.mvc(); > - (X_5:bat[:oid,:int] ,X_6:bat[:oid,:int] ,X_7:bat[:oid,:int] ) := > sql.copy_from("sys":str,"ttt":str,"|":str,"\\n":str,nil:str,"null":str,"/:\\tmp/xyz":str,-1:lng,0:lng,0:int); > - X_9 := sql.append(X_2,"sys","ttt","a",X_5); > - X_11 := sql.append(X_9,"sys","ttt","b",X_6); > - X_15 := sql.append(X_11,"sys","ttt","c",X_7); > - X_16 := aggr.count(X_7); > - sql.affectedRows(X_15,X_16,""); > + (X_13:bat[:oid,:int] ,X_14:bat[:oid,:int] ,X_15:bat[:oid,:int] ) > := > sql.copy_from("sys","ttt","|","\\n",nil:str,"null","/:\\tmp/xyz",-1:lng,0:lng,0); > + X_17 := sql.append(X_2,"sys","ttt","a",X_13); > + X_19 := sql.append(X_17,"sys","ttt","b",X_14); > + X_23 := sql.append(X_19,"sys","ttt","c",X_15); > + X_24 := aggr.count(X_15); > + sql.affectedRows(X_23,X_24,""); > end s2_1; > # optimizer.mitosis() > # optimizer.dataflow() > @@ -89,15 +89,15 @@ end s2_1; > % .explain # table_name > % mal # name > % clob # type > -% 178 # length > +% 153 # length > function user.s5_1{autoCommit=true}():void; > X_2 := sql.mvc(); > - (X_5:bat[:oid,:int] ,X_6:bat[:oid,:int] ,X_7:bat[:oid,:int] ) := > sql.copy_from("sys":str,"ttt":str,"|":str,"\\n":str,nil:str,"null":str,"/:\\tmp/xyz":str,-1:lng,0:lng,0:int); > - X_9 := sql.append(X_2,"sys","ttt","a",X_5); > - X_11 := sql.append(X_9,"sys","ttt","b",X_6); > - X_16 := aggr.count(X_7); > - X_15 := sql.append(X_11,"sys","ttt","c",X_7); > - sql.affectedRows(X_15,X_16,""); > + (X_13:bat[:oid,:int] ,X_14:bat[:oid,:int] ,X_15:bat[:oid,:int] ) > := > sql.copy_from("sys","ttt","|","\\n",nil:str,"null","/:\\tmp/xyz",-1:lng,0:lng,0); > + X_17 := sql.append(X_2,"sys","ttt","a",X_13); > + X_19 := sql.append(X_17,"sys","ttt","b",X_14); > + X_24 := aggr.count(X_15); > + X_23 := sql.append(X_19,"sys","ttt","c",X_15); > + sql.affectedRows(X_23,X_24,""); > end s5_1; > # optimizer.mitosis() > # optimizer.dataflow() > diff --git a/monetdb5/optimizer/opt_support.c > b/monetdb5/optimizer/opt_support.c > --- a/monetdb5/optimizer/opt_support.c > +++ b/monetdb5/optimizer/opt_support.c > @@ -144,7 +144,7 @@ struct OPTcatalog { > {"groups", 0, 0, 0, DEBUG_OPT_GROUPS}, > {"aliases", 0, 0, 0, DEBUG_OPT_ALIASES}, > {"cluster", 0, 0, 0, DEBUG_OPT_CLUSTER}, > -{"coercion", 0, 0, 0, DEBUG_OPT_COERCION}, > +{"coercions", 0, 0, 0, DEBUG_OPT_COERCION}, > {"commonTerms", 0, 0, 0, DEBUG_OPT_COMMONTERMS}, > {"compress", 0, 0, 0, DEBUG_OPT_COMPRESS}, > {"constants", 0, 0, 0, DEBUG_OPT_CONSTANTS}, > diff --git a/monetdb5/optimizer/opt_wrapper.c > b/monetdb5/optimizer/opt_wrapper.c > --- a/monetdb5/optimizer/opt_wrapper.c > +++ b/monetdb5/optimizer/opt_wrapper.c > @@ -84,7 +84,7 @@ struct{ > {"aliases", &OPTaliasesImplementation}, > {"centipede", &OPTcentipedeImplementation}, > {"cluster", &OPTclusterImplementation}, > - {"coercion", &OPTcoercionImplementation}, > + {"coercions", &OPTcoercionImplementation}, > {"commonTerms", &OPTcommonTermsImplementation}, > {"compression", &OPTcompressionImplementation}, > {"costModel", &OPTcostModelImplementation}, > diff --git a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > --- a/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > +++ b/sql/backends/monet5/UDF/Tests/udf-fuse.stable.out > @@ -27,10 +27,10 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 76 # length > +% 72 # length > function user.s0_1{autoCommit=true}(A0:bte,A1:bte):void; > X_4 := udf.fuse(A0,A1); > - > sql.exportValue(1,".","fuse_single_value":str,"smallint",16,0,6,X_4,""); > + > sql.exportValue(1,".","fuse_single_value","smallint",16,0,6,X_4,""); > end s0_1; > # optimizer.mitosis() > # optimizer.dataflow() > @@ -44,10 +44,10 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 71 # length > +% 67 # length > function user.s1_1{autoCommit=true}(A0:sht,A1:sht):void; > X_4 := udf.fuse(A0,A1); > - > sql.exportValue(1,".","fuse_single_value":str,"int",32,0,6,X_4,""); > + sql.exportValue(1,".","fuse_single_value","int",32,0,6,X_4,""); > end s1_1; > #select fuse(1000,2000); > % . # table_name > @@ -59,10 +59,10 @@ end s1_1; > % .explain # table_name > % mal # name > % clob # type > -% 74 # length > +% 70 # length > function user.s2_1{autoCommit=true}(A0:int,A1:int):void; > X_4 := udf.fuse(A0,A1); > - > sql.exportValue(1,".","fuse_single_value":str,"bigint",64,0,6,X_4,""); > + > sql.exportValue(1,".","fuse_single_value","bigint",64,0,6,X_4,""); > end s2_1; > #select fuse(1000000,2000000); > % . # table_name > diff --git a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > --- a/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > +++ b/sql/backends/monet5/UDF/Tests/udf-reverse.stable.out > @@ -48,10 +48,10 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 74 # length > +% 70 # length > function user.s0_1{autoCommit=true}(A0:str):void; > X_3 := udf.reverse(A0); > - > sql.exportValue(1,".","reverse_single_value":str,"clob",0,0,4,X_3,""); > + > sql.exportValue(1,".","reverse_single_value","clob",0,0,4,X_3,""); > end s0_1; > #select reverse('MonetDB'); > % . # table_name > @@ -81,7 +81,7 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 62 # length > +% 60 # length > function user.s3_1{autoCommit=true}():void; > X_2 := sql.mvc(); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","udf_reverse"); > @@ -92,7 +92,7 @@ function user.s3_1{autoCommit=true}():vo > X_15 := algebra.leftfetchjoin(X_3,X_14); > X_16:bat[:oid,:str] := batudf.reverse(X_15); > X_18 := sql.resultSet(1,1,X_16); > - sql.rsColumn(X_18,"sys.","reverse_x":str,"clob",0,0,X_16); > + sql.rsColumn(X_18,"sys.","reverse_x","clob",0,0,X_16); > X_22 := io.stdout(); > sql.exportResult(X_22,X_18); > end s3_1; > diff --git > a/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > b/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > --- > a/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > +++ > b/sql/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > @@ -27,7 +27,7 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 63 # length > +% 59 # length > function user.s0_1{autoCommit=true}(A0:dbl):void; > X_3 := sql.mvc(); > X_4:bat[:oid,:oid] := sql.tid(X_3,"sys","sf_2596114"); > @@ -38,7 +38,7 @@ function user.s0_1{autoCommit=true}(A0:d > X_16 := algebra.leftfetchjoin(X_4,X_15); > X_17:bat[:oid,:dbl] := batmmath.pow(X_16,A0); > X_19 := sql.resultSet(1,1,X_17); > - sql.rsColumn(X_19,"sys.","power_x":str,"double",53,0,X_17); > + sql.rsColumn(X_19,"sys.","power_x","double",53,0,X_17); > X_24 := io.stdout(); > sql.exportResult(X_24,X_19); > end s0_1; > @@ -46,7 +46,7 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 65 # length > +% 61 # length > function user.s1_1{autoCommit=true}():void; > X_2 := sql.mvc(); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","sf_2596114"); > @@ -57,7 +57,7 @@ function user.s1_1{autoCommit=true}():vo > X_15 := algebra.leftfetchjoin(X_3,X_14); > X_16:bat[:oid,:dbl] := batcalc.*(X_15,X_15); > X_18 := sql.resultSet(1,1,X_16); > - sql.rsColumn(X_18,"sys.","sql_mul_x":str,"double",53,0,X_16); > + sql.rsColumn(X_18,"sys.","sql_mul_x","double",53,0,X_16); > X_23 := io.stdout(); > sql.exportResult(X_23,X_18); > end s1_1; > diff --git > a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > --- > a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > +++ > b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > @@ -34,22 +34,22 @@ function user.s0_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_45 := algebra.likesubselect(X_6,X_3,A0,"":str,false); > + X_45 := algebra.likesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_46 := algebra.likesubselect(r1_9,A0,"":str,false); > + X_46 := algebra.likesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_47 := algebra.likesubselect(X_12,X_3,A0,"":str,false); > - X_14 := sql.subdelta(X_45,X_9,X_46,X_47); > - X_16 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > - X_17 := sql.bind(X_2,"sys","functions","func",0); > - (X_19,r1_28) := sql.bind(X_2,"sys","functions","func",2); > - X_21 := sql.bind(X_2,"sys","functions","func",1); > - X_22 := sql.projectdelta(X_14,X_17,X_19,r1_28,X_21); > - X_23 := sql.resultSet(2,1,X_16); > - sql.rsColumn(X_23,"sys.functions","name","varchar",256,0,X_16); > - sql.rsColumn(X_23,"sys.functions","func","varchar",8196,0,X_22); > - X_32 := io.stdout(); > - sql.exportResult(X_32,X_23); > + X_47 := algebra.likesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_45,X_9,X_46,X_47); > + X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > + X_18 := sql.bind(X_2,"sys","functions","func",0); > + (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > + X_22 := sql.bind(X_2,"sys","functions","func",1); > + X_23 := sql.projectdelta(X_15,X_18,X_20,r1_28,X_22); > + X_24 := sql.resultSet(2,1,X_17); > + sql.rsColumn(X_24,"sys.functions","name","varchar",256,0,X_17); > + sql.rsColumn(X_24,"sys.functions","func","varchar",8196,0,X_23); > + X_33 := io.stdout(); > + sql.exportResult(X_33,X_24); > end s0_1; > #explain select name,func from functions where name not like > '%optimizers%'; > % .explain # table_name > @@ -60,22 +60,22 @@ function user.s1_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_45 := algebra.likesubselect(X_6,X_3,A0,"":str,true); > + X_45 := algebra.likesubselect(X_6,X_3,A0,"",true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_46 := algebra.likesubselect(r1_9,A0,"":str,true); > + X_46 := algebra.likesubselect(r1_9,A0,"",true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_47 := algebra.likesubselect(X_12,X_3,A0,"":str,true); > - X_14 := sql.subdelta(X_45,X_9,X_46,X_47); > - X_16 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > - X_17 := sql.bind(X_2,"sys","functions","func",0); > - (X_19,r1_28) := sql.bind(X_2,"sys","functions","func",2); > - X_21 := sql.bind(X_2,"sys","functions","func",1); > - X_22 := sql.projectdelta(X_14,X_17,X_19,r1_28,X_21); > - X_23 := sql.resultSet(2,1,X_16); > - sql.rsColumn(X_23,"sys.functions","name","varchar",256,0,X_16); > - sql.rsColumn(X_23,"sys.functions","func","varchar",8196,0,X_22); > - X_32 := io.stdout(); > - sql.exportResult(X_32,X_23); > + X_47 := algebra.likesubselect(X_12,X_3,A0,"",true); > + X_15 := sql.subdelta(X_45,X_9,X_46,X_47); > + X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > + X_18 := sql.bind(X_2,"sys","functions","func",0); > + (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > + X_22 := sql.bind(X_2,"sys","functions","func",1); > + X_23 := sql.projectdelta(X_15,X_18,X_20,r1_28,X_22); > + X_24 := sql.resultSet(2,1,X_17); > + sql.rsColumn(X_24,"sys.functions","name","varchar",256,0,X_17); > + sql.rsColumn(X_24,"sys.functions","func","varchar",8196,0,X_23); > + X_33 := io.stdout(); > + sql.exportResult(X_33,X_24); > end s1_1; > #explain select name,func from functions where name ilike > '%optimizers%'; > % .explain # table_name > @@ -86,22 +86,22 @@ function user.s2_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_45 := algebra.ilikesubselect(X_6,X_3,A0,"":str,false); > + X_45 := algebra.ilikesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_46 := algebra.ilikesubselect(r1_9,A0,"":str,false); > + X_46 := algebra.ilikesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_47 := algebra.ilikesubselect(X_12,X_3,A0,"":str,false); > - X_14 := sql.subdelta(X_45,X_9,X_46,X_47); > - X_16 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > - X_17 := sql.bind(X_2,"sys","functions","func",0); > - (X_19,r1_28) := sql.bind(X_2,"sys","functions","func",2); > - X_21 := sql.bind(X_2,"sys","functions","func",1); > - X_22 := sql.projectdelta(X_14,X_17,X_19,r1_28,X_21); > - X_23 := sql.resultSet(2,1,X_16); > - sql.rsColumn(X_23,"sys.functions","name","varchar",256,0,X_16); > - sql.rsColumn(X_23,"sys.functions","func","varchar",8196,0,X_22); > - X_32 := io.stdout(); > - sql.exportResult(X_32,X_23); > + X_47 := algebra.ilikesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_45,X_9,X_46,X_47); > + X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > + X_18 := sql.bind(X_2,"sys","functions","func",0); > + (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > + X_22 := sql.bind(X_2,"sys","functions","func",1); > + X_23 := sql.projectdelta(X_15,X_18,X_20,r1_28,X_22); > + X_24 := sql.resultSet(2,1,X_17); > + sql.rsColumn(X_24,"sys.functions","name","varchar",256,0,X_17); > + sql.rsColumn(X_24,"sys.functions","func","varchar",8196,0,X_23); > + X_33 := io.stdout(); > + sql.exportResult(X_33,X_24); > end s2_1; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

Running MAL Programs Manually?
by Stephen P. Morgan 12 Mar '13

12 Mar '13

Should it be possible to run MAL programs manually, by reading them from a file? I just tried a simple experiment and got an error message. The experiment: I created a table, "test," with two columns, "field1" and field2." I created a MAL query by issuing the SQL query, "EXPLAIN SELECT * FROM test;" and cut and pasted the output to a file. I then stripped out all of the formatting characters ('|') and extraneous spaces. I then re-entered mclient, choosing the MAL language, and read in the MAL form of the query with "\< junk.mal." The interpreter then complained that two identifiers were not initialized (see output, below). So, is it not possible to issue a MAL query by hand? Or am I missing something? (Again, please forgive me, I am new to MonetDB.) Thanks, Steve Morgan mal>\< junk.mal MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = function user.s1_1{autoCommit=true}():void; barrier X_41 := language.dataflow(); X_2 := sql.mvc(); X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","test"); X_6 := sql.bind(X_2,"sys","test","field1",0); (X_9,r1_9) := sql.bind(X_2,"sys","test","field1",2); X_12 := sql.bind(X_2,"sys","test","field1",1); X_14 := sql.delta(X_6,X_9,r1_9,X_12); X_15 := algebra.leftfetchjoin(X_3,X_14); X_16 := sql.bind(X_2,"sys","test","field2",0); (X_18,r1_22) := sql.bind(X_2,"sys","test","field2",2); X_20 := sql.bind(X_2,"sys","test","field2",1); X_21 := sql.delta(X_16,X_18,r1_22,X_20); X_22 := algebra.leftfetchjoin(X_3,X_21); language.pass(X_3); exit X_41; X_23 := sql.resultSet(2,1,X_15); sql.rsColumn(X_23,"sys.test","field1","int",32,0,X_15); sql.rsColumn(X_23,"sys.test","field2","int",32,0,X_22); X_29 := io.stdout(); sql.exportResult(X_29,X_23); end s1_1; ERROR = !TypeException:user.s1_1[16]:'X_15' may not be used before being initialized !TypeException:user.s1_1[17]:'X_15' may not be used before being initialized !TypeException:user.s1_1[18]:'X_22' may not be used before being initialized

1 1

External data atoms Issues
by Mike Gomez 11 Mar '13

11 Mar '13

Hi, i have been trying to expand data types (both core and external atoms).However, after discussions, i understood that addition of core data types will takes years( if i am not exaggerating).So, i started testing up external gdk atoms (monetdb5/modules/atoms). i am creating new atom type say mytype which is int based as below. this atom is pure int ( there are no bitfields and so on) typedef int mytype; atom mytype:int; in mal i have defined the following mal procedures( and C backends) fromstr(), tostr(), mytype(s:str):mytype, new(s:str):mytype, isnil(v:mytype):bit at this moment,it can successfully create tables and do selects. however, selects are not working as they should. these queries work. create table test(i mytype); insert into test values('453535'); select * from test these don't work. select * from test2 where i >= 5 above query gives error "types mytype(0,0) and tinyint(8,0) are not equal for column 'i'" insert into test2 values(453535); gives types int(32,0) and mytype(0,0) are not equal however, this query works !!!!!! select * from test2 where i >= '1' after testing and too many code peeping for above query, i figured out that the above query converts each element of table to string using tostr, and then compares the strings, and takes decision. so, it looks like there can be performance improvements here. However,my question is HOW do i get the queries working such as select * from test2 where i >= 5 am i missing something?? any pointers may be helpful. Thanks. -Mike

1 0

sabdb protocol V2
by Gijs Molenaar 11 Mar '13

11 Mar '13

Hi MonetDB people, I've noticed that there has been a new version (2) of the sabdb protocol introduced. The sabdb protocol is used to send administrative command to a monetdb server. This breaks the Python API, which I just fixed, I'll commit it later. But I've noticed some strange behavior, it looks like monetdb closes the connection in some cases, not all. Specifically when I issue a status command the connection is closed, but in other cases it is not. I've fixed it now by reconnecting before every command, but I just wanted to verify if that is the right way to go. greetings, -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

3 5