- developers-list - monetdb.org

[Monetdb-developers] New stable branch created
by Sjoerd Mullender 29 Jan '09

29 Jan '09

I have created new stable branches for the upcoming release. From now on *all* bug fixes and *only* bug fixes *must* go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. All packages use the same name for the branch: Feb2009. So in each of your stable checkout directories you can do cvs update -rFeb2009 to update to the new branch. I have created branches for the following packages: MonetDB MonetDB4 MonetDB5 TestTools buildtools clients geom java pathfinder sql template testing. Note that datacell is missing from this list. It is not yet released and will not be released in this round. Also note that two packages now have stable branches that didn't use to have them: java and testing. Testing is a new package (split off from MonetDB) and java is now also incorporated in the release cycle. -- Sjoerd Mullender

1 0

[Monetdb-developers] char *mapi_get_type(MapiHdl hdl, int fnr)
by Stefan de Konink 25 Jan '09

25 Jan '09

What is the reason that $subj is not a const char *? Stefan

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/server rel_semantic.mx, , 1.35, 1.36 sql_parser.mx, , 1.299, 1.300 sql_psm.mx, , 1.65, 1.66 sql_semantic.mx, , 1.194, 1.195 sql_updates.mx, , 1.161, 1.162
by Sjoerd Mullender 23 Jan '09

23 Jan '09

Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/server > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv13513/src/server > > Modified Files: > rel_semantic.mx sql_parser.mx sql_psm.mx sql_semantic.mx > sql_updates.mx > Log Message: > added binary import support (using new BATattach) > > > Index: sql_parser.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/server/sql_parser.mx,v > retrieving revision 1.299 > retrieving revision 1.300 > diff -u -d -r1.299 -r1.300 > --- sql_parser.mx 17 Jan 2009 18:17:01 -0000 1.299 > +++ sql_parser.mx 22 Jan 2009 21:58:06 -0000 1.300 > @@ -2400,6 +2401,12 @@ > append_list(l, $2); > append_string(l, $8); > $$ = _symbol_create_list( SQL_COPYFROM, l ); } > +/* binary copy from */ > + | COPY INTO qname FROM '(' string_commalist ')' > + { dlist *l = L(); > + append_list(l, $3); > + append_list(l, $6); > + $$ = _symbol_create_list( SQL_BINCOPYFROM, l ); } > | COPY select_no_parens_orderby INTO string opt_seps opt_null_string > { dlist *l = L(); > append_symbol(l, $2); This appears to break the syntax copy into a from '/tmp/i' using delimiters '\t'; (!syntax error, unexpected STRING, expecting '(' in: "explain copy into a from '/tmp/i'"). It can be salvaged by adding opt_nr like the two COPY statements before this, but then you'd have to check "by hand" that opt_nr returns NULL. -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New bug fix release of MonetDB suite
by Sjoerd Mullender 23 Jan '09

23 Jan '09

The MonetDB team at CWI is pleased to announce the Nov2008-SP2 bug fix release of the MonetDB suite of programs. This is a bug fix release of the Nov2008 release. More information on this release can be found at <http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Nov2008/#Nov200…>. The sources and binaries are available at <http://monetdb.cwi.nl/projects/monetdb/Download/>. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime serialize.mx, , 1.119, 1.120 xrpc_server.mx, , 1.85, 1.86
by Stefan Manegold 22 Jan '09

22 Jan '09

On Thu, Jan 22, 2009 at 03:06:44PM +0100, Ying Zhang wrote: > On Thu, Jan 22, 2009 at 12:30:21PM +0000, Ying Zhang wrote: > > Update of /cvsroot/monetdb/pathfinder/runtime > > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv16608 > > > > Modified Files: > > serialize.mx xrpc_server.mx > > Log Message: > > Added a new env. var. to make the root directory, from which the XRPC HTTP > > server serves files, configurable. > > > > This was (mainly) needed to get the xrpc admin tests run with RunMtest, since RunMtest doesn't have monet installed yet (i.e., there is no $prefix/share/ MonetDB/xrpc/ yet), so we need a way to tell the xrpc server to serve file from > > a different directory, in this case, from $srcdir/runtime/xrpc. > > Sorry, I forgot to mention that this checkin requires an update and a > rebuild of MonetDB4, because changes in MonetDB4/conf/MonetDB.conf.in > are needed. question is why are they needed? or differently, why is the default (only) set in MonetDB.conf, rather then where it is (first) used in Pathfinder/XRPC? I would have expected that the newly added code in Pathfinder/XRPC, just checks, whether the new environment variable is set; if so, it uses that value; otherwise, it uses a default (defined there). In case your code does already exactly this, the changes in MonetDB.conf, should not be required for the new PF/XRPC code to work, rather, they mainly document the new variable and its default value. Stefan > Jennie > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime serialize.mx, , 1.119, 1.120 xrpc_server.mx, , 1.85, 1.86
by Stefan Manegold 22 Jan '09

22 Jan '09

On Thu, Jan 22, 2009 at 12:30:21PM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv16608 > > Modified Files: > serialize.mx xrpc_server.mx > Log Message: > Added a new env. var. to make the root directory, from which the XRPC HTTP > server serves files, configurable. > > This was (mainly) needed to get the xrpc admin tests run with RunMtest, since RunMtest doesn't have monet installed yet (i.e., there is no $prefix/share/ MonetDB/xrpc/ yet), so we need a way to tell the xrpc server to serve file from > a different directory, in this case, from $srcdir/runtime/xrpc. > [...] > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.85 > retrieving revision 1.86 > diff -u -d -r1.85 -r1.86 > --- xrpc_server.mx 8 Jan 2009 16:54:18 -0000 1.85 > +++ xrpc_server.mx 22 Jan 2009 12:30:19 -0000 1.86 > @@ -89,6 +89,10 @@ > xrpc_trusted.append(prefix); > } > > +proc set_xrpc_docroot(str new_docroot) : void { > + > +} > + Jennie, what is the purpose of the above *empty* MIL proc? Stefan > proc get_xrpc_open() : bit { > if (monet_environment.exist("xrpc_open")){ > return bit(monet_environment.find("xrpc_open")); [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Results of FastBit Experiment
by Ross Bates 21 Jan '09

21 Jan '09

A question for any of the core team who were involved.... what resulted when FastBit was plugged into the backend of MonetDB? -Ross

1 0

[Monetdb-developers] mclient running out of memory, crashing mserver5
by sylver_b 21 Jan '09

21 Jan '09

Hi, I've finally found a better way to bulk load records via perl & odbc - a lot more stable. However, I've noticed many times mclient crashing mserver5 when loading records via the interactive interface. For instance, I tried to load about 130 inserts from a sql file using this command : mclient -lsql -d mydb < public_sysuser.sql After few seconds I started to see : #BBPTRIM: memtarget=17978753 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=273741960 [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] #BBPTRIM_ENTER: memsize=275281456,vmsize=275166768 #BBPTRIM: memtarget=19273201 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=275158816 [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] #BBPTRIM_ENTER: memsize=276161752,vmsize=276047064 #BBPTRIM: memtarget=20153497 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=276056368 [ 1 ] #BBPTRIM_ENTER: memsize=276322608,vmsize=276207920 #BBPTRIM: memtarget=20314353 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=276198064 Top is showing : top - 23:26:14 up 249 days, 9:57, 5 users, load average: 0.00, 0.20, 0.45 Tasks: 96 total, 1 running, 95 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 4025600k total, 2433324k used, 1592276k free, 191788k buffers Swap: 9502408k total, 92k used, 9502316k free, 1993392k cached I don't think i'm running out of memory. I had to stop and restart mserver5, flush the table, then re-run the command - all the records were inserted. Also I noticed this type of messages on the mserver5 console when piping records through the mclient (over 1million records) . It seems that the mclient is quiet instable (cf my post about "COPY, terminating connection" few months ago). So, is there a way to increase the memory limit or make sure mserver5 won't crash when running bash inserts via a cron job ? Please advise, Thank you SB

3 9

[Monetdb-developers] MonetDb comparison Data Load
by Prometheus WillSurvive 19 Jan '09

19 Jan '09

Dear Developers-Users, First of all I would like to thanks all of your afforts to birng us these DB technologies and a platform for openly discussion. We are consulting big sized customers and deploying their data search technologies. Total size is around 1 petabytes daily . (total of our customers number ) . Their data size starts 1 TB (9.000.000.000 lines cvs) to 100 TB. so we have different size of customer base. We are looking opensource db solutions for a while due to lack of support and technological cumbersome of the current commercial solutions. We have 3 test customer now : 1-1 TB data dailiy read only purposes. 9.000.000.000 cvs file should be load to database . Data should be load to the database max 2 hours. after that comlex crm analitic queries. max 10 concurrent user/queries. database will be daily and wil not grow dailiy. so limit is 1 TB data load. 2-10 TB data daily. same as above .. 3-50 TB data stabil. will not grow. So we are looking for a solutions: *Fast data loading. How fast we load data to the MonetDB ? Is it could be multithreaded data loading possible ? *Is it possible to MPP ? can we add more server to share cpu resources and decrease the load time ? what is the best HW solution for above first customer ? Do we need still fat enterrprise servers or what ? We are also looking for InfoBright, Vertica and also Luciddb. VErtica is not opensource so out of the list. Infobright seems not giving its full power in the opensource.. Also we heard about the MonetDB X100 and interested. After 3 months succefully test we will support monetDB developer if its solved our problems. We can pay . Best Regards W.S

1 0

[Monetdb-developers] MonetDB Comparison and Power
by willsurvive 19 Jan '09

19 Jan '09

Dear Developers-Users, First of all I would like to thanks all of your afforts to birng us these DB technologies and a platform for openly discussion. We are consulting big sized customers and deploying their data search technologies. Total size is around 1 petabytes daily . (total of our customers number ) . Their data size starts 1 TB (9.000.000.000 lines cvs) to 100 TB. so we have different size of customer base. We are looking opensource db solutions for a while due to lack of support and technological cumbersome of the current commercial solutions. We have 3 test customer now : 1-1 TB data dailiy read only purposes. 9.000.000.000 cvs file should be load to database . Data should be load to the database max 2 hours. after that comlex crm analitic queries. max 10 concurrent user/queries. database will be daily and wil not grow dailiy. so limit is 1 TB data load. 2-10 TB data daily. same as above .. 3-50 TB data stabil. will not grow. So we are looking for a solutions: *Fast data loading. How fast we load data to the MonetDB ? Is it could be multithreaded data loading possible ? *Is it possible to MPP ? can we add more server to share cpu resources and decrease the load time ? what is the best HW solution for above first customer ? Do we need still fat enterrprise servers or what ? We are also looking for InfoBright, Vertica and also Luciddb. VErtica is not opensource so out of the list. Infobright seems not giving its full power in the opensource.. Also we heard about the MonetDB X100 and interested. After 3 months succefully test we will support monetDB developer if its solved our problems. We can pay . Best Regards W.S -- View this message in context: http://www.nabble.com/MonetDB-Comparison-and-Power-tp21541998p21541998.html Sent from the monetdb-developers mailing list archive at Nabble.com.

1 0