- developers-list - monetdb.org

[Monetdb-developers] New serialization routine
by Jan Rittinger 16 May '06

16 May '06

In the current version serialization takes 10 seconds for a 10 MB file (http://www-db.in.tum.de/~rittinge/files/mixed.xml) which contains only slightly more nodes (517448) then the auction xml file of the same size (472681). The difference however is the number of attribute nodes which is much higher in mixed.xml (517447 vs. 38266). While the serialization of doc("auction10MB.xml") takes 2.72 seconds, doc("mixed.xml") requires 10.34 seconds!! Removing the attributes in mixed.xml (by deleting all entries from the attr_own bat) speeds up the serialization to complete in 4.6 seconds. Some more calculations reveal that for the auction file we are able to serialize 187 nodes/sec (nodes = elements+textnodes+attributes) while for mixed.xml the serialization function (with and without attributes) was only able to generate about 100 nodes/sec. In all 3 cases the serialization seems way to slow... (at least for me). As in principle the serialization is only a dump of the tables (even in table order) and other approaches are really faster we should be able to come up with an appropriate fast serialization routine. I see at least three possibilities to increase the performance: * serialize.mx uses about 50 function calls until one node without attributes is serialized * serialize.mx does collect the nodes using random access with a lot of indirections * serialize.mx uses stream_printf which is probably slower than a fixed size print routine like stream_write. In the following two weeks (before the feature freeze) I will try to come up with a new prototypical print routine that should handle the serialization more efficiently. This prototype will be a proof of concept and probably will fall back to current routine whenever some conditions are not fulfilled (e.g. wrong serialization type or multiple namespaces). Let's see wether we can get a faster serialization routine for the average query result! I will probably need help for some gdk internals, changes introduced by the update facility (e.g. indirections), and for understanding parts of the current serialize.mx. I hope for your support :-) More information on the new serialization routine will be available at the pathfinder wiki. Cheers, Jan -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

2 1

[Monetdb-developers] Bug in MonetDBv4.10.2
by Bart Scheers 11 May '06

11 May '06

Beste lezer, Ik gebruik monetdb v 4.10.2 met de bijbehorende jdbc driver. Na het creëren van een tabel met kolom van type 'double' kan ik via een PreparedStatement geen setDouble(d) aanroepen. Ik krijg de foutmelding: SQLException Message: wrong argument for prepared statement (expected double instead of decimal) en het is toch echt een double (te zien via SQL front end met \d). Als ik de DBMetaData ophaal (ik dacht slim te zijn door hiet het DATA_TYPE op te halen) krijg ik een error: SQLException: syntax error, unexpected IDENT in: "select "sqlname" as "TYPE_NAME", cast(case "sqlname" when 'timetz' then 92 when 'table' then 2003 when 'blob' then 2004 when 'bigint' then -5 when 'time' then 92 when 'varchar' then 12 when 'clob' then 2005 when 'int' then 4 when 'timestamptz' then 93 when 'date' then 91 when 'sec_interval' then -5 when 'boolean' then 16 when 'decimal' then 3 when 'double' then 8 when 'char' then 1 when 'month_interval' then 4 when 'oid' then 1111 when 'smallint' then 5 when 'timestamp' then 93 when 'real' then 7 else 1111 end as smallint) as "DATA_TYPE", "digits" as "PRECISION", cast(case when "systemname" = 'str' then cast('\' AS char) ELSE cast(NULL AS char) END) AS "LITERAL_PREFIX", cast(CASE WHEN "systemname" = 'str" Haakje of kwootje verkeerd? Al met al betekent het dat ik dan geen PreparedStatement kan uitvoeren, wat het wel sneller zou maken. Is dit bekend bij jullie? Groeten Bart Scheers

2 1

Re: Re: [Monetdb-developers] MapiClient connet problem
by Fabian Groffen 10 May '06

10 May '06

Hi, I seems that the mil support is not started by default. You can do this by typing on the server prompt: mapi_start(); After this command MapiClient should work. Beware that mil mode (the default) does not allow you to easily express XQuery queries; instead use the -lxquery flag. I don't know how this is done in the Windows version, but unfortunately MapiClient can't do an interactive session with xquery, so you have to give the file you want to execute as argument. If someone using Windows knows how this can be done the best way, please fill me in. Regards On 10-05-2006 18:34:35 +0800, zhangxin525 wrote: > Hi, > Thanks for your reply! > > First, sorry for my careless. I did not give you an exact copy of the error message. > The exact error message is : > > !ERROR mapi_reconnect: connect: error 10061 > MAPI = anonymous@localhost:50000 > ACTION= mapi_reconnect > ERROR = Setup connection failed > > > I start the MonetDB/XQuery server, the message of the server windows: > > # Monet Database Server V4.10.2 > # Copyright (c) 1993-2006, CWI. All rights reserved. > # Compiled for i386-pc-mingw32/32bit with 32bit OIDs; dynamically linked. > # Visit http://monetdb.cwi.nl/ for further information. > MonetDB> > > I think the server is correct. > Other, when I install MonetDB/XQuery, I chose the user for everynoe. > > Thanks for your patient help again. > > > > > zhangxin525 > 2006-05-10 > > > > 发件人： Fabian Groffen > 发送时间： 2006-05-10 16:40:59 > 收件人： zhangxin525 > 抄送： monetdb-developers > 主题： Re: [Monetdb-developers] MapiClient connet problem > > Hi, > > Thanks for your interest in MonetDB/XQuery. > > On 10-05-2006 16:08:26 +0800, zhangxin525 wrote: > > Hello! > > > > I install MonetDB/XQuery in WindowsXP, and I have start the XUery Server, but when I start MapiClient through command, always get the message: > > > > !ERROR mapi_reconnect: connect: > > MAPI = anonymous@localhost:5000 > > Is this an exact copy of the error message? In case it is, there is a 0 > missing here, the port should be 50000 not 5000. I think you can change > that somewhere in the .bat script that's called from the start menu. > > Maybe this is just a copy/paste error, so if this is not the problem, > can you copy and paste what's in the MonetDB/XQuery server window? > > Regards

1 0

[Monetdb-developers] MapiClient connet problem
by zhangxin525 10 May '06

10 May '06

Hello! I install MonetDB/XQuery in WindowsXP, and I have start the XUery Server, but when I start MapiClient through command, always get the message: !ERROR mapi_reconnect: connect: MAPI = anonymous@localhost:5000 ACTION= mapi_reconnect ERROR = Setup connection failed why? and how to solve it? Thank you very much! zhangxin525 2006-05-10

2 1

[Monetdb-developers] MapiClient connet problem
by zhangxin525 10 May '06

10 May '06

Hello! I install MonetDB/XQuery in WindowsXP, and I have start the XUery Server, but when I start MapiClient through command, always get the message: !ERROR mapi_reconnect: connect: error 10061 MAPI = anonymous@localhost:50000 ACTION= mapi_reconnect ERROR = Setup connection failed why? and how to solve it? Thank you very much! zhangxin525 2006-05-10

1 0

[Monetdb-developers] hashing algorithms usage in MonetDB/Five
by Fabian Groffen 29 Apr '06

29 Apr '06

Hi all, FYI Sourceforge isn't really cooperating, but I implemented MD5 and SHA1 hash algorithms support during challenge response in MAPI. I extended the JDBC driver to be able to use SHA1 or MD5 next to plain (crypt is not easily available in Java). I will check in my changes as soon as 1) SourceForge lets me, and 2) My physical state + location lets me I also added a little option to JdbcClient to force usage of a specified hash algorithm, using -Xhash. I conducted a little experiment to see the effect of connecting with a hash or plain. The results for my system is as follows: (I ran every call 10 times and took the average) % time java -jar jars/jdbcclient-1.4.jar -Xhash SHA1 < /dev/null real 0.29 % time java -jar jars/jdbcclient-1.4.jar -Xhash MD5 < /dev/null real 0.29 % time java -jar jars/jdbcclient-1.4.jar -Xhash plain < /dev/null real 0.23 The precision of time is seconds here, so the difference in time (or the cost of both hash implementations) is 0.06 seconds. This includes the time for the server to do the same of course. All in all, not really shocking IMHO. Please note that the JdbcClient does a lot of other stuff like querying the database for its version number and such. When not doing that the time will be probably lower. Also, the wallclock time does also include the full JVM startup. The C implementation of the hashes might be more efficient, I can only tell something useful about that once MapiClient will also support hash algorithms.

2 4

[Monetdb-developers] OpenSSL default detection
by Fabian Groffen 28 Apr '06

28 Apr '06

In conf/monet.m4 openssl detection is currently set to default to "no". Since I want to make use of the SHA1 and MD5 hashes from openssl and think that this should be enabled by default, I was wondering if there are objections agains setting "have_openssl=auto", such that auto-detection is done, and OpenSSL will be used whenever available. If I see no objections within a few days, I will assume it is ok to make this change to conf/monet.m4.

1 0

[Monetdb-developers] meeting notes
by Niels Nes 28 Apr '06

28 Apr '06

MonetDB developers Today another MonetDB Meeting was held at the CWI. In this meeting we decided to aim for another MonetDB{SQL/XQuery} release on June 25. To reach this goal we are going for a Feature freeze (ie the usual release candidates and release branch) on June 1. So if you have any (well tested) interesting features too add please commit them before June 1. The we are aiming for an Release with many new features. See http://monetdb.cwi.nl/Development/Roadmap/index.html for full details. Among these features are hopefully 'Xquery updates' and 'SQL triggers'. Before this MonetDB 4 release we are going for an alpha release via a 'tarball' of the MonetDB 5 development on May 05. This will be the first public release of the MonetDB 5 code base. Niels Nes

4 5

[Monetdb-developers] Re: AquaStudio en Windows versie
by Martin Kersten 28 Apr '06

28 Apr '06

Fabian wrote: > http://www.apple.com/downloads/macosx/development_tools/dbvisualizer.html > Ok, downloading requires your credentials to be left behind and some hints what to fill in where. Could you copy the doc/Assets/AquaDataStudio.texi file and enumerate the steps. Then I will try to re-do them following the documentation.

3 5

[Monetdb-developers] Re: m 5 documentation
by Martin Kersten 27 Apr '06

27 Apr '06

Niels, The documentation for M5 is self-contained. The overlapping pieces (e.g. HowToInstall's) are hand synchronized. The GDK and module layers are not touched. I will go through the Website and identify pieces to be shared. The latest draft versions are available at: http://www.cwi.nl/~mk/MonetDB http://www.cwi.nl/~mk/manual.pdf Comments on its structure and content are highly appreciated. Niels Nes wrote: > Martin > > Hoe gaan we de documentatie van M5 mergen met de MonetDB4,SQL,Xquery docs? > > Niels >

1 0