June 2008 - developers-list

[Monetdb-developers] Did something with Copy Into and nulls change in nightly stable?
by Rt Ibmer 14 Jun '08

14 Jun '08

I found that the nightly stable build of MonetDB5/sql apparently does not like empty values in tbl text records for importing when using Copy Into, even though the field is defined to allow nulls. This is different behavior compared to the RELEASE version, which did allow this. Please allow me to explain... I have a text file like this: field1|field2|| My db has defined field 1 and 2 as varchar null, and field 3 as INT null. When I run the Copy Into command on the nightly stable, it errors out with this: ' from line 0 field 3 not inserted, expecting type int !SQLException:importTable:failed to import table As a quick test I edited the first line of my text file to insert a fake integer into the 3rd field like this: field1|field2|12345| Then I reran the Copy Into command, this time it errored out on the SECOND line (even those it says line 1 it is really 0 index based so its line 2): ' from line 1 field 3 not inserted, expecting type int !SQLException:importTable:failed to import table So it was happy when I inserted the fake data for the field and does not like rows that are empty. Again, this column is defined as int NULL. Was this change intentional? If not I think there is a bug there. How can I important my text file that has many nulls in it, short of going back to the released version, which I cannot do due to problems with that earlier version. Thank you.

2 1

[Monetdb-developers] server locks up on reboot attempt anytime after running mclient
by Rt Ibmer 13 Jun '08

13 Jun '08

I am running a server instance under Amazon EC2 with MonetDB 5 (latest released version) installed. If I boot up the instance and do NOT start or use Monetdb, then I can issue a reboot command at any time, and the system reboots just fine and comes back up. However if I issue the commands: mkdir -p /mnt/MonetDB5/dbfarm merovingian monetdb create demo monetdb start demo mclient -lsql --time --database=demo sql> \q Then when I go to reboot I get the following output in the console and then things hang, and the instance never reboots. Here is the console output I get (more notes on the issue following this output): INIT: Switching to runlevel: 6 INIT: Sending processes the TERM signal Stopping ConsoleKit: [ OK ] Stopping sshd: [ OK ] Stopping crond: [ OK ] Stopping system message bus: [ OK ] Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] Shutting down interface eth0: [ OK ] Shutting down loopback interface: [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Unloading modules: [ OK ] Starting killall: [ OK ] Sending all processes the TERM signal... ------------[ cut here ]------------ kernel BUG at include/linux/tracehook.h:369! invalid opcode: 0000 [#1] SMP last sysfs file: /class/net/sit0/address Modules linked in: sit(U) tunnel4(U) fuse(U) ipv6(U) dm_mirror(U) dm_multipath(U) dm_mod(U) pcspkr(U) ext3(U) jbd(U) mbcache(U) uhci_hcd(U) ohci_hcd(U) ehci_hcd(U) xenblk(U) xennet(U) CPU: 0 EIP: 0061:[<c10254df>] Not tainted VLI EFLAGS: 00210087 (2.6.21.7-2.fc8xen #1) EIP is at release_task+0x38/0x2f1 eax: c12ff000 ebx: c2b9f450 ecx: f5416000 edx: 00000000 esi: c2b9f450 edi: 00000000 ebp: 00000001 esp: ed207e68 ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0069 Process mserver5 (pid: 1124, ti=ed207000 task=c1404910 task.ti=ed207000) Stack: 00000020 c1404910 00000000 ed7c15b0 c1026918 00000009 00000008 0007b6c0 c1404d64 c1404d64 c14049c4 00000000 c14f30e4 c188bac0 00000000 ed207fb8 c10269fb c14f30e4 00000009 b438a364 c102dcb2 00000000 00000000 00000000 Call Trace: [<c1026918>] do_exit+0x6d6/0x730 [<c10269fb>] sys_exit_group+0x0/0xd [<c102dcb2>] get_signal_to_deliver+0x3d2/0x414 [<c1004be7>] do_notify_resume+0x8c/0x6e6 [<c12038ae>] do_page_fault+0x7a1/0xc24 [<c10e48a8>] copy_to_user+0x3c/0x50 [<c107d766>] sys_select+0x161/0x187 [<c1005765>] work_notifysig+0x13/0x1a ======================= Code: 98 04 00 00 00 74 07 89 f0 e8 e6 98 02 00 8b 86 80 01 00 00 90 ff 48 04 b8 00 f0 2f c1 e8 c5 c7 1d 00 83 be 90 00 00 00 20 74 04 <0f> 0b eb fe 83 be 98 04 00 00 00 74 27 8d 86 a8 04 00 00 e8 ee EIP: [<c10254df>] release_task+0x38/0x2f1 SS:ESP 0069:ed207e68 Fixing recursive fault but reboot is needed! --------------------------------------------------- Note that if I run these same commands mkdir -p /mnt/MonetDB5/dbfarm merovingian monetdb create demo monetdb start demo # OMIT THIS TIME mclient -lsql --time --database=demo But if I DO NOT run mclient, then the problem does not occur and it will reboot fine. However once I run mclient then I am guaranteed to lock up on a reboot with the console output as shown above. What is really bad about this issue in particular is that on EC2 if an instance will not reboot it needs to be terminated. And when terminated all data on the instance is completely detroyed! So I do not get a second chance - once the server is locked out like this it has to be destroyed and a new one built. This is running on 32 bit Fedora Core 8. What is causing this and how can I fix it? Thanks.

2 2

Re: [Monetdb-developers] [MonetDB-users] MonetDB Server dies on connect
by Martin Kersten 13 Jun '08

13 Jun '08

Sjoerd, I was planning to try out the installers on my Vista machine. The previous version is v5.4.0 installed Uninstalling it gives the following error: There is a problem with this Windows Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor. You want specific information, or should i manually remove the C:\Program Files\CWI/MonetDB5 Sjoerd Mullender wrote: > Niels Nes wrote: > >> On Fri, Jun 13, 2008 at 09:41:50AM +0200, Robert Schumacher wrote: >> >>> I am using MonetDB server v5.4.0, based on kernel v1.22.0. >>> >>> Deleting those directories worked, thanks. >>> >> We are very interested in learning if this bug is fixed in >> the (tobe released) version (MonetDB v5.6.0 / sql 2.24 / kernel 1.24). >> >> Sjoerd are there allready windows installers for XP x64 to be tested? >> > > The following files are available for download. Note that these > installers are *completely untested*. > > Since the 64-bit version has a new so-called Upgrade Code (the code that > Windows uses to determine whether a package is an upgrade of an already > installed package), you need to first uninstall the old version and then > install the new. > > Note that the ODBC driver is no longer packaged in the MonetDB5-SQL > installer but only as a separate installer. Also note that you should > install both 32-bit and 64-bit versions of the ODBC installer on a > 64-bit platform. > > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB5-SQL-Installer-x86_64-200… > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB5-SQL-Installer-i386-20080… > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB-ODBC-Installer-x86_64-200… > http://homepages.cwi.nl/~sjoerd/downloads/MonetDB-ODBC-Installer-i386-20080… > > The files with x86_64 in the names are 64-bit, the files with i386 in > the names are 32 bit. > > >> Niels >> >>> Robert >>> >>> -----Ursprüngliche Nachricht----- >>> Von: monetdb-users-bounces(a)lists.sourceforge.net [mailto:monetdb-users-bounces@lists.sourceforge.net] Im Auftrag von Niels Nes >>> Gesendet: Freitag, 13. Juni 2008 09:37 >>> An: Communication channel for MonetDB users >>> Betreff: Re: [MonetDB-users] MonetDB Server dies on connect >>> >>> On Tue, Jun 10, 2008 at 02:51:22PM +0200, Robert Schumacher wrote: >>> >>>> Hello, >>>> >>>> i was evaluating MonetDB. Got the Server running finr on Windows XP x64. I created about 12 Tabled by using a JDBC Client, then the Server died instantly. Since then when I start the Server it dies asap when I either start the Windows Client or perform a JDBC connection. I get no error log, no message, nothing. Has anybody an idea how to get MonetDB to stop crashing on connect? >>>> >>>> >>> Which version of MonetDB are you using? As you seem to have hit a bug, >>> the only solution out of the crash seems to be to remove the dbfarm (and >>> xquery_logs/sql_logs) directories. >>> >>> Niels >>> >>>> Thanks, >>>> Robert >>>> >>>> Robert Schumacher, Senior Software Developer >>>> >>>> prevero Systems GmbH >>>> Landsberger Straße 154 >>>> 80339 München >>>> >>>> Tel +49 89 455 77 - 0 >>>> Fax +49 89 455 77 - 200 >>>> >>>> Registergericht: Amtsgericht München, HRB 143809 -- Geschäftsführer: Matthias Thurner >>>> >>>> Bitte beachten Sie unsere Veranstaltungshinweise unter www.prevero.de >>>> >>>> >>>> >>>> ------------------------------------------------------------------------- >>>> Check out the new SourceForge.net Marketplace. >>>> It's the best place to buy or sell services for >>>> just about anything Open Source. >>>> http://sourceforge.net/services/buy/index.php >>>> _______________________________________________ >>>> MonetDB-users mailing list >>>> MonetDB-users(a)lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>>> >>> -- >>> >>> Niels Nes, Centre for Mathematics and Computer Science (CWI) >>> Kruislaan 413, 1098 SJ Amsterdam, The Netherlands >>> room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 >>> url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl >>> >>> ------------------------------------------------------------------------- >>> Check out the new SourceForge.net Marketplace. >>> It's the best place to buy or sell services for >>> just about anything Open Source. >>> http://sourceforge.net/services/buy/index.php >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >>> ------------------------------------------------------------------------- >>> Check out the new SourceForge.net Marketplace. >>> It's the best place to buy or sell services for >>> just about anything Open Source. >>> http://sourceforge.net/services/buy/index.php >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > ------------------------------------------------------------------------ > > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users >

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql_optimizer.mx, , 1.183, 1.184
by Romulo Goncalves 12 Jun '08

12 Jun '08

Milena Ivanova wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29777 > > Modified Files: > sql_optimizer.mx > Log Message: > Switch off the recycler. Why? We activated it to test it during night on other platforms. It helps to see if there is any problem from one day to another and it also helps to detect problems on the code. If you made a change and you broke tests, you should not commit until until you make sure that the existent tests (SQL or M5) are green. It is with these steps that we can get stable code and avoid bugs... Romulo > > > U sql_optimizer.mx > Index: sql_optimizer.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/sql_optimizer.mx,v > retrieving revision 1.183 > retrieving revision 1.184 > diff -u -d -r1.183 -r1.184 > --- sql_optimizer.mx 6 Jun 2008 20:25:15 -0000 1.183 > +++ sql_optimizer.mx 12 Jun 2008 15:33:12 -0000 1.184 > @@ -455,9 +455,10 @@ > "joinPath," > "mergetable," > "commonTerms," > +/* "replicator," */ > "accumulators," > "deadcode," > - "recycle," > +/* "recycle," */ > "reduce," > "garbageCollector," > "multiplex"; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins

2 1

Re: [Monetdb-developers] Works fine once, then get Invalid credentials for user monetdb
by Rt Ibmer 12 Jun '08

12 Jun '08

Its been a day of great frustration trying to get this going so I hope someone will have some ideas here. I decided to take a shot at upgrading just by running yum again pointing at the stable night - the upgrade process appeared to run just fine. However it was all downhill from there. Once I had that installed, I then tried to start merovingian and upon doing so received the following error message: merovingian cannot open config file /var/tmp/monet.GNU.32.32.d.5761/MonetDB5/.GNU.32.32.d-Fedora8/etc/monetdb5.conf I have no idea why it would be looking at that path. So anyway just to try and make progress I then created the above directory and put a copy of monetdb5.conf in there - and then the error message went away. However, when I try to run mclient I am now getting this 'sql' not initialized mclient -lsql --time --database=demo !MALException:setScenario:Scenario 'sql' not initialized Before all I had to do was run merowingian and then I could use the above mclient command ok, but not any longer. Then I decided to try with mserver5 and it cannot find the sql libraries so it seems: # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'demo' # Compiled for i686-redhat-linux-gnu/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ #WARNING: LoaderException:loadLibrary:sql: loader error lib_logger.so.0: cannot open shared object file: No such file or directory !MALException:MAL.getAddress:address of 'sql.SQLsession' not found !MALException:MAL.getAddress:address of 'sql.SQLsession2' not found .... snip (many more error lines like the above) So then I decided to try everything from scratch. I built a new server, installed the stable nightly from scratch, and then I wound up with the exact same as everything I wrote above. What am I doing wrong? Thanks!

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/common stream.mx, , 1.161, 1.162
by Sjoerd Mullender 12 Jun '08

12 Jun '08

Was this in the correct branch? Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/common > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31463/common > > Modified Files: > stream.mx > Log Message: > fix compilation > > > U stream.mx > Index: stream.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/common/stream.mx,v > retrieving revision 1.161 > retrieving revision 1.162 > diff -u -d -r1.161 -r1.162 > --- stream.mx 9 Jun 2008 05:25:25 -0000 1.161 > +++ stream.mx 12 Jun 2008 11:02:14 -0000 1.162 > @@ -2964,7 +2964,7 @@ > ns->errnr = OPEN_ERROR; > /* blocksizes have a fixed little endian byteorder */ > #ifdef WORDS_BIGENDIAN > - s->byteorder = 3412; /* simply != 1234 */ > + ns->byteorder = 3412; /* simply != 1234 */ > #endif > ns->type = s->type; > ns->access = s->access; > @@ -3057,7 +3057,7 @@ > return NULL; > /* blocksizes have a fixed little endian byteorder */ > #ifdef WORDS_BIGENDIAN > - s->byteorder = 3412; /* simply != 1234 */ > + ns->byteorder = 3412; /* simply != 1234 */ > #endif > ns->type = orig->type; > ns->access = orig->access; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] Works fine once, then get Invalid credentials for user monetdb
by Rt Ibmer 12 Jun '08

12 Jun '08

OK this is very odd... I do this: mkdir -p /mnt/MonetDB5/dbfarm merovingian monetdb create demo monetdb start demo and then start mclient: mclient -lsql --time --database=demo at that point everything is just fine and I can enter sql commands and all works great. I am using the default monetdb/monetdb for uname/pass. However once I exit out of this (\q) and try to go back in using the same command: mclient -lsql --time --database=demo It then comes back with: InvalidCredentialsException:checkCredentials:Invalid credentials for user 'monetdb' no matter how many times I try. So it works the very first time, but then not again. It seems like the very first onc to get a connection to the monetdb server is ok, but then any time after that it gets locked out somehow. What can be causing this? Also on a related note what is the monetdb5.secret file actually used for? I am also confused by this: # If you use Merovingian, a per-database vault key is used and this # setting ignored. #monet_vault_key=/etc/monetdb.secret So where then does the per-database vault key reside? And do I even need to mess with this? Thanks!

2 1

[Monetdb-developers] My little adventure using Sun Studio 12
by Fabian Groffen 11 Jun '08

11 Jun '08

With Sun Studio 12 cc compiler you' won't get far on MonetDB's sources. I had to unset all flags. Just a FYI: /opt/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/ufs/fabian/scratch/monetdb/stable/buildtools/burg -c gram.c "/ufs/fabian/scratch/monetdb/stable/buildtools/burg/zalloc.c", line 14: warning: statement not reached /opt/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/ufs/fabian/scratch/monetdb/stable/common/src/gdk -I../.. -I/ufs/fabian/scratch/monetdb/stable/common/src/gdk -I../common -I/ufs/fabian/scratch/monetdb/stable/common/src/gdk/../common -DLIBGDK -g -D_REENTRANT -c gdk_bbp.c -KPIC -DPIC -o .libs/libbat_la-gdk_bbp.o "gdk_posix.h", line 102: warning: macro redefined: __USE_BSD "/ufs/fabian/scratch/monetdb/stable/common/src/gdk/gdk_bbp.mx", line 1497: identifier redeclared: unshare current : static function(int) returning void previous: function(int) returning int : "/usr/include/bits/sched.h", line 79 (The Linux installation is somewhere on volund (/tmp/Linux), and the CDROM is on my desk, if someone is interested.)

1 0

[Monetdb-developers] crackers on the stable branch
by Romulo Goncalves 11 Jun '08

11 Jun '08

Lately I had to compile the stable branch several times in the optimized mode. Crackers is not a feature to be used in the stable branch, so why do we have crackers there? Its compilation takes too long. Could we remove it? Stratos you are the one who should decide... ;) Thanks, Romulo

2 2

[Monetdb-developers] Increasing speed of inserts?
by Rt Ibmer 10 Jun '08

10 Jun '08

I have a Java app that needs to write single inserts into the db in an extremely fast manner (it is receiving data in real time, and buffering it, then doing single inserts to feed the data into the db as fast as it will insert it). I am doing this using JDBC and this resides on the same physical box as MonetDB 5 latest version. Short of increasing/changing my hardware or changing my code around to bulk write inserts (which I assume would help?) is there anything I can do to improve the speed at which MonetDB can insert records into the db? For this one particular table it is about 160 columns. Maybe only about 25% of the columns are written to in any particular insert. This is literraly the type of thing where the requests are coming in so fast that it needs the db to keep up. I'm sure there is room for improvement in terms of the hardware and even the way we are writing the single inserts (i.e. maybe writing like 10 at a time or something). But short of that, is there any configuration type of options or optimization settings within MonetDB that could help increase the speed over what I am getting now? Thanks!

2 1