- users-list - monetdb.org

[MonetDB-users] Query failure with 'recycle' in optimizer
by Alex Bo. 06 May '09

06 May '09

Hi, I have a query that runs without any problems when the default optimizer settings are used. But once I add 'recycle' to the existing optimizer modules, the query doesn't return. Mserver seems to be stuck in an infinite loop with 50% CPU usage. I am running Monet compiled from CVS on Windows. This is the query: select sum(t1."AddressID"), "hour"(t1."ModifiedDate") as aa from "schema1"."CustomerAddress" as t1 inner join "schema1"."CustomerAddress"as t2 on t1."CustomerID" = t2."CustomerID" group by aa This is what I get from 'select opimizer': inline,remap,evaluate,costModel,coercions,emptySet,mitosis,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex,recycle Thanks. -- View this message in context: http://www.nabble.com/Query-failure-with-%27recycle%27-in-optimizer-tp23389… Sent from the monetdb-users mailing list archive at Nabble.com.

5 5

[MonetDB-users] Compiling with optimizer
by Alex Bo. 30 Apr '09

30 Apr '09

Hi, I have been compiling MonetDB successfuly under Windows so far. Now I have tried compiling as I have done before, but with "optimize=yes". Now when I start the server, I get this message: !FATAL: BBPinit: BBP.dir is empty What does this mean? Am I doing something wrong? Thanks. -- View this message in context: http://www.nabble.com/Compiling-with-optimizer-tp23022199p23022199.html Sent from the monetdb-users mailing list archive at Nabble.com.

3 8

[MonetDB-users] TPC-H for MonetDB
by mayank＠cs.wisc.edu 30 Apr '09

30 Apr '09

Hi, I'm trying to run TPC-H benchmark on MonetDB. I downloaded the package from the TPC-H site but it needs some configuration for it to be able to generate DBGEN and QGEN. I was wondering if anyone can advise me what needs to be changed in makefile, tpcd.h and for 64-bit support. I'm running this on a Dell Intel 32-bit architecture. Thanks. Mayank

3 2

[MonetDB-users] connection terminated
by djdjoko 27 Apr '09

27 Apr '09

hi, i started to have suddenly following problem with the M5Server. I can start the server and get the following screen which seems fine: # MonetDB server v5.10.2, based on kernel v1.28.2 # Serving database 'demo', using 1 thread # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008- MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\Program Files\MonetDB\MonetDB5\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.28.2 loaded When i start the client I get following message and the server shuts down: Connection terminated Press any key to continue . . . I have tried to reinstall monetDb but it does not help. can anyone point me into the right direction? Thanks -- View this message in context: http://www.nabble.com/connection-terminated-tp23260811p23260811.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0

[MonetDB-users] Announcement: New bug fix release of MonetDB suite
by Sjoerd Mullender 27 Apr '09

27 Apr '09

The MonetDB team at CWI is pleased to announce the Feb2009-SP2 bug fix release of the MonetDB suite of programs. This is the second bug fix release of the Feb2009 release. More information on this release can be found at <http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Feb2009/#Feb200…>. The sources and binaries are available at <http://monetdb.cwi.nl/projects/monetdb/Download/>. -- Sjoerd Mullender

1 0

[MonetDB-users] Vista M5Server Crashes
by gsd 24 Apr '09

24 Apr '09

Hi all, My monetdb server (MonetDB server v5.10.2, based on kernel v1.28.2) crashes on me in a reproducible fashion. I'm generally (but not always) accessing it via JDBC. I have 100 tables, each containing just less than 4 million records. Crash 1: - I'm testing retrieval speeds. When I hit table 17 or 18 the server crashes. Crash 2: - I'm testing insertion speeds. Again when I hit table 17 or 17 the server crashes. I'm only inserting about 1000 records per table. I've tried it via "COPY INTO" and via batch INSERT statments. Crash 3: - Using the Mclient.exe application I manually selected 5000 rows from each table. When I hit the 17th table the server crashed again. I accessed the tables in a different order to the above programmatic crashes to try and eliminate this being due to a particular table. Also, this crash also eliminates this being a JDBC-related issue. So, I'm wondering if this may be a known windows issue, or perhaps a Vista issue? Could it be due to some server internal limit being hit as it loads and caches tables? Are there any server logs I can examine? Any ideas? Thanks, Scott -- View this message in context: http://www.nabble.com/Vista-M5Server-Crashes-tp23218455p23218455.html Sent from the monetdb-users mailing list archive at Nabble.com.

4 3

[MonetDB-users] Windows JDBC Issues
by gsd 24 Apr '09

24 Apr '09

Hello (again!), I have MonetDB running on a 32-bit Windows Vista box. When I start the server it says it's "MonetDB server v5.10.2, based on kernel v1.28.2". I'm using JDBC batch updates to populate about 100 tables with approximately 4 million records per table. I've a couple of issues: 1) My batch updates fail from time-to-time. On average this might be every 10,000 batch updates with each batch containing 100 insertions. I get an exception with the following message: "Error(s) occurred while executing the batch, see next SQLExceptions for details" If I close the connection then open another one I can continue quite happily. If I try to reuse the existing connection with a cleared batch I get the same error every time until I do close the connection. I think I may not be the only person with this issue: http://wiki.pentaho.com/display/EAI/MonetDB http://wiki.pentaho.com/display/EAI/MonetDB 2) After a very large amount of insertions (more like 100+ million records, across numerous tables) the insertion simply hangs (a commit usually takes about a second and I've left it 30+ minutes). After killing the client the server becomes impossible to connect to. Even after a reboot I get the following messages: SERVER: >!ERROR: Incompatible database version 000000, this server supports version 050000 !ERROR: Please move away c:\data\monetdbdata\MonetDB5\sql_logs\demo\sql\ and its corresponding dbfarm. !mvc_init: unable to create system tables CLIENT: !SQLException:SQLinit:Catalogue initialization failed JDBC CLIENT: Connection reset (Mserver still alive?) If, however, I delete/move the sql_logs directory then restart things then I can connect and my data is still there. The last time I did this one of the log files (log.97) was ~250mb in size. Any help, advice or comments would be much appreciated. I'm going to spend this afternoon trying out the bulk COPY INTO functionality. Thanks, Scott -- View this message in context: http://www.nabble.com/Windows-JDBC-Issues-tp23196456p23196456.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 3

Re: [MonetDB-users] Tweaking performance
by Garrett Smith 23 Apr '09

23 Apr '09

----- "Martin Kersten" <Martin.Kersten(a)cwi.nl> wrote: > Garrett Smith wrote: >> Does anyone have some guidance on approaches for optimizing database >> performance? >> >> I realize this is a pretty broad question :) >> >> More specifically, I'm running a simple query >> >> select max(date) from metric >> >> on a table with 21 million rows (or whatever you call them in >> Monet). >> It's taking quite a bit longer than I'd expect, given a typically >> indexed column. > Given the fact that there is no (b-tree) index, the underlying > engine has to perform a scan to find the maximum. > This entails a linear scan over 21M elements in the current > implementation. Thanks! After going back and reading the docs on MonetDB, I realized my question was pretty naive. The SQL interface triggered a Pavlovian response to think in terms of a relational database with b-tree indexes. It's pretty amazing how fast that linear scan is. Still, my application calls for something else. I took a look at FastBit, which is insanely adept at selects into huge data sets, but requires a relatively expensive index update to get that performance. As I have routine updates to deal with, that doesn't seem a good fit. We'll probably stick with MySQL+MyISAM for this particular component. Garrett

1 0

[MonetDB-users] COPY INTO with dates
by gsd 23 Apr '09

23 Apr '09

Hello, I'm looking into the COPY INTO bulk insert functionality but was wondering how I should format DATEs and TIMESTAMPs? Thanks, Scott -- View this message in context: http://www.nabble.com/COPY-INTO-with-dates-tp23194496p23194496.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

Re: [MonetDB-users] Tweaking performance
by Martin Kersten 23 Apr '09

23 Apr '09

Garrett Smith wrote: > 123456789012345678901234567890123456789012345678901234567890123456789012 > > Does anyone have some guidance on approaches for optimizing database > performance? > > I realize this is a pretty broad question :) > > More specifically, I'm running a simple query > > select max(date) from metric > > on a table with 21 million rows (or whatever you call them in Monet). > It's taking quite a bit longer than I'd expect, given a typically > indexed column. Given the fact that there is no (b-tree) index, the underlying engine has to perform a scan to find the maximum. This entails a linear scan over 21M elements in the current implementation. > > On an "explain" I see that there's no difference between the plan with > or without the presence of an index on 'date'. Per the SQL index docs, I > see that indexes are more of a hint to Monet and they're often ignored. > > Cool. > > But what happens when I want to fix what seems to be a configuration > problem with my database/schema? > > Thanks! > > Garrett > > ------------------------------------------------------------------------------ > Stay on top of everything new and different, both inside and > around Java (TM) technology - register by April 22, and save > $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. > 300 plus technical and hands-on sessions. Register today. > Use priority code J9JMT32. http://p.sf.net/sfu/p > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users

1 0

[MonetDB-users] Tweaking performance
by Garrett Smith 22 Apr '09

22 Apr '09

123456789012345678901234567890123456789012345678901234567890123456789012 Does anyone have some guidance on approaches for optimizing database performance? I realize this is a pretty broad question :) More specifically, I'm running a simple query select max(date) from metric on a table with 21 million rows (or whatever you call them in Monet). It's taking quite a bit longer than I'd expect, given a typically indexed column. On an "explain" I see that there's no difference between the plan with or without the presence of an index on 'date'. Per the SQL index docs, I see that indexes are more of a hint to Monet and they're often ignored. Cool. But what happens when I want to fix what seems to be a configuration problem with my database/schema? Thanks! Garrett

1 0

Re: [MonetDB-users] the x100 project
by Peter Boncz 21 Apr '09

21 Apr '09

Hi Tom, Thanks for your interest in X100. The project is very much alive in a recent CWI spin-off company called vectorwise. X100 is currently being developed into a fully functional database engine that will fly in datawarehousing and BI environments. Much more than that I cannot say about it now. More news will follow, certainly within a year. It is not yet open source (but in a future that is not off the cards), but if you need that now, you could try MonetDB. Peter > Hello, > I have been asked to evaluate MonetDB. We have interest in the x100 project, is this project still on going... is there a download for it? If > not, is there a plan to release it? > > thanks > tom

1 0

[MonetDB-users] monetdb-client ubuntu package missing binary ?
by Franck Routier 18 Apr '09

18 Apr '09

Hi, I have tried to install and test monetdb using ubuntu repository. But when it comes to using mclient, no binary seems to be installed. Indeed, looking at the package, there seems to be only docs in it. So, am I looking at the wrong place and missing some obvious point, or is the package broken ? (notice that the debian package seems to be similar) Thanks, Franck

2 4

[MonetDB-users] the x100 project
by tom edgar 17 Apr '09

17 Apr '09

Hello, I have been asked to evaluate MonetDB. We have interest in the x100 project, is this project still on going... is there a download for it? If not, is there a plan to release it? thanks tom

1 0

[MonetDB-users] loading big table
by dariuszs 16 Apr '09

16 Apr '09

Hi, I've got a table with 130 fields. When I load 130 million records into that table (about 50G) - it's not a problem, but when I try to load 160 million records (about 60G of data) into it this thing just sits for hours with no error messages. Can somebody help me? Thanks. Dariusz.

5 16

[MonetDB-users] MonetDB XQuery in Production
by Max Fedorov 14 Apr '09

14 Apr '09

The product looks very promising on paper, although not that many people are talking about it and I came across MonetDB by accident. Is anyone using MonetDB in production? Is the product stable enough? I do not feel like spending lots of time testing this XQuery server if nobody runs it in production. I loaded several million records and tested sample queries without issues for the most part, but then once the server just vanished (running latest 32bit version on WindowsXP). I restarted and everything is working again, but now I am very scared. Nothing like this ever happened when using MySQL, MS SQL Server or Oracle. Nothing ever just dies without a trace. Thank you. -Max

4 5

[MonetDB-users] timer option in monetdb
by Mayank Maheshwari 14 Apr '09

14 Apr '09

Hi, I'm working on a project comparing the performance of column-store database systems using MonetDB with the common row-stores for different query scenarios. While running the --timer option on monetDB, I was wondering if it includes the time for printing the result and any other I/O time as well. I'm trying to get the break-up of where the system spends its time while processing the query. Is there a way I could find that out? Also is it possible to find out whats the compression scheme the database is using to store the data? I'm storing a 1 million row table with 6 attributes. Thanks. Mayank Maheshwari mayank(a)cs.wisc.edu

1 0

[MonetDB-users] Fast Loading
by Alex Bo. 14 Apr '09

14 Apr '09

Hi, I came across chapter 3.2.8 in the documentation regarding 'Fast Loading'. I am trying to use it in order to improve the performance I get compared to loading tables using 'COPY INTO'. However, I am not sure how to approach this: "Each file is produced by a program that writes the binary image of the BAT directly, i.e. a binary dump of an C-array" How exactly can I produce these binary images from my source columns? How exactly are they expected by MonetDB to look like? A little more info could be very helpful. Thanks. -- View this message in context: http://www.nabble.com/Fast-Loading-tp23009932p23009932.html Sent from the monetdb-users mailing list archive at Nabble.com.

5 4

[MonetDB-users] mysql vs. monetdb
by Yue Sheng 08 Apr '09

08 Apr '09

Hi, we installed MonetDB on a Linux box, and did some comparison between MySQL and MonetDB. The result seems off by a large magnitude. Has anyone encountered similar problem please? Thanks. For MonetDB: ql>select count(ins_key) from bigtable where ins_key = 15012; +---------+ | L1 | +=========+ | 1931541 | +---------+ 1 tuple Timer 260771.478 msec 1 rows For MySQL: select count(ins_key) from bigtable where ins_key=15012; +----------------+ | count(ins_key) | +----------------+ | 1931548 | +----------------+ real 0m0.989s user 0m0.002s sys 0m0.003s Indices: MonetDB: None MySQL: index on ins_key Thanks.

3 4

[MonetDB-users] Admin GUI not working on Vista 64
by Max Fedorov 08 Apr '09

08 Apr '09

Installed 64 bit version of MoneyDB on Vista. Server and client are working fine, but Admin GUI is coming up with 404. Thanks. -Max

2 4

[MonetDB-users] MonetDB Server logs and location on Windows
by Max Fedorov 07 Apr '09

07 Apr '09

Hello. I have MonetDB installed on Windows XP. The server crashed several times for me during shredding of several files (1mill records each). I am running this Window XP instance in VM, so maybe disk space or memory was an issue. 1. Where are the logs? 2. Also, what is the physical location of the database itself? 3. What is the best strategy to prevent data loss if server crashes. Do I need to backup files or backup through Admin GUI? Thanks a lot!

2 1

[MonetDB-users] XQuery Suggestions
by Max Fedorov 07 Apr '09

07 Apr '09

Hello, please advise why these 2 queries do not work. Samples would be nice. module namespace foo = "xrpc-test-function"; declare function foo:countQueueStats($docName as xs:string) as xs:integer { count(doc($docName)/qflex:queueStats/qflex:queueStat/qflex:queueName) }; declare function foo:addQueueStat($docName as xs:string, $myElement as element()) { do insert $myElement as last into doc($docName)/queueStats }; Thank you!

4 7

[MonetDB-users] Mserver error
by Stefan Klinger 02 Apr '09

02 Apr '09

Hi. I just updated to the 'current' branch via the provided 'monetdb-install.sh'. Now I get an error message when starting Mserver: >>>>> sk@phobos18:~> Mserver # MonetDB Server v4.29.0 # based on GDK v1.29.0 # Copyright (c) 1993-July 2008, CWI. All rights reserved. # Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved. # Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: CMDfind: value not found. !ERROR: CMDfind: operation failed. !ERROR: interpret_params: int(param 1): evaluation error. MonetDB> <<<<< Even worse, I cannot use Pathfinder any more: >>>>> sk@phobos18:~> Mserver --dbinit='module(pathfinder);mil_start();' # MonetDB Server v4.29.0 # based on GDK v1.29.0 # Copyright (c) 1993-July 2008, CWI. All rights reserved. # Copyright (c) August 2008-2009, MonetDB B.V.. All rights reserved. # Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: CMDfind: value not found. !ERROR: CMDfind: operation failed. !ERROR: interpret_params: int(param 1): evaluation error. # PF/Tijah module v0.9.0 loaded. http://dbappl.cs.utwente.nl/pftijah !ERROR: interpret: unknown variable 'REMAP_PAGE_SIZE'. !ERROR: interpret_params: BAT(param 3): evaluation error. !ERROR: interpret_params: rename(param 1): evaluation error. MonetDB> <<<<< Any help wouldbe appreciated. Kind regards, Stefan -- Stefan Klinger o/klettern /\/ bis zum send plaintext only - max size 32kB - no spam \ Abfallen http://stefan-klinger.de

2 1

[MonetDB-users] MonetDB SQL 5.10.3 Unable to restore dump from previous version
by McKennirey.Matthew 31 Mar '09

31 Mar '09

In our test environment we frequently dump MonetDB databases using the mclient -D command and restore using mclient passing in the sql file created by the dump. We decided to try out the latest release (5.10.3 , February2009-SP1) We installed MonetDB from the monetdb-install script using our customary --enable-sql --nightly==stable settings. Installation appeared to complete normally. Our dump was from MonetDB server v5.4.1 (64-bit), based on kernel v1.22.1 (64-bit oids) Copyright (c) 1993-2007 CWI, all rights reserved Visit http://monetdb.cwi.nl/ for further information Configured for prefix: /monetdb Libraries: libpcre: 7.2 2007-06-19 (compiled with 7.2) openssl: OpenSSL 0.9.8e 23 Feb 2007 (compiled with OpenSSL 0.9.8e 23 Feb 2007) Compilation: gcc -g -O2 -std=c99 -fgnu89-inline Linking : ld -IPA -m elf_x86_64 We attempted to load into MonetDB server v5.10.3 (64-bit), based on kernel v1.28.3 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Configured for prefix: /monetdb Libraries: libpcre: 7.2 2007-06-19 (compiled with 7.2) openssl: OpenSSL 0.9.8e 23 Feb 2007 (compiled with OpenSSL 0.9.8e 23 Feb 2007) Compilation: gcc -O2 -std=c99 -fgnu89-inline Linking : ld -IPA -m elf_x86_64 We create our databases by > merovingnian > monetdb create DATABASE > monetdb start DATABASE The SQL error we get when attempting the load is ERROR = !SQLException:sql:missing sep line 0 field 0 !SQLException:importTable:failed to import table There are no errors in the merovingian log. We have tried 3 installs on 2 different machines - each install succeeds, but we have never been able to restore from the dump from 5.4.1 Our tables are 'wide and sparse', that is many columns will contain NULL for many OIDs We have no trouble creating databases on 5.10.3 and populating them with SQL files made up of INSERT statements. We tried doing a more manual load of test data (INSERT), then doing a dump (from 5.10.3), and then doing a restore (to 5.10.3) from that dump and that works.

2 1

[MonetDB-users] server crush
by dariuszs 30 Mar '09

30 Mar '09

Hi, I was selecting 1 record from a table with 100 fields and the server crashed following message: MAPI=monetdb@localhost:50000; ACTION=read_line QUERY=select * from table1; ERROR=connection terminated Press any key to continue... What does it mean? Thanks. Dariusz.

2 3