- users-list - monetdb.org

[MonetDB-users] (no subject)
by Richard Wesley 20 Oct '08

20 Oct '08

Hi All - I am trying to evaluate MonetDB on Windows by loading up some large data sets. I'm not having much luck and I'm hoping someone can help me out. The first set of problems revolve around the ODBC driver. It seems to not be able to report errors or even return the results of queries reliably. Most errors come back as a generic error message although we occasionally run into strange messages involving UTF-16 conversions when no string columns are involved (e.g. a count distinct of a numeric column). We are using the July 2008 version from the web site. Are these known issues and is there a more recent build we could try? This is not in production yet, we are simply trying to evaluate the system. I have managed to work around the ODBC problems very slowly and have got to a point where I can load small data sets. But when I then try to do a large insert using the COPY FROM syntax, my client got a "connection terminated" error. When I take the COPY FROM statement and paste it into the client, the MonetDB server crashes. The only log files I can find have a few (mostly unprintable) characters in them. Restarting the server and trying again produces the same problem. The insert query is COPY 6568253 RECORDS INTO "Airlines"."_2005_On_Time_Performance" FROM 'C:\\DOCUME~1\\hawkfish\\LOCALS~1\\Temp\\1\\7A.tmp\\table.tbl' USING DELIMITERS '\t', '\n', '\''; Is there some limit on the COPY INTO syntax? Is there some log file somewhere that could point me in the right direction? TIA, ________________________________________________________ Richard Wesley Senior Software Developer Tableau Software Visit: http://www.trytableau.com/now.html

4 13

[MonetDB-users] MonetDB with Pentaho and Kettle
by Tom Barber 17 Oct '08

17 Oct '08

Hi Guys, As of 3.1 Kettle now has a connector built in for Monet so I thought I'd give it another go after not getting too far earlier in the year. Anyway I have it up and running on a Quad Core Xeon box running Centos x86_64 and it seems pretty happy, but when I try to copy sizable amounts of data into it, kettle stops with an error: 2008/10/17 12:29:22 - Table output.0 - Prepared statement : INSERT INTO voc.d_budgets3 (entity_anl_code, trad_code, Cost, Sales, CostMTD, SalesMTD) VALUES ( ?, ?, ?, ?, ?, ?) 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : Because of an error, this step can't continue: 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : org.pentaho.di.core.exception.KettleException: 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : Error inserting row into table [d_budgets3] with values: [19], [A09], [0], [0], [0], [0] 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : Error inserting/updating row 2008/10/17 12:29:27 - Table output.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:32:36) : current transaction is aborted (please ROLLBACK) But to my untrained eye the data looks fine(coming from a mysql database) and I don't know where to start debugging this or what settings I could change. The monet table is 2 Strings, 2 Int, 2 Big Num and it always inputs a load of data before stopping. What can I look at to resolve the issues? otherwise, rocking database and keep up the good work. Thanks Tom CONFIDENTIALITY NOTICE This e-mail is confidential and may also be privileged. Please notify us immediately if you are not the intended recipient. You should not copy it, forward it or use it for any purpose or disclose the contents to any person. If you have received this e-mail in error, please notify us as soon as possible and destroy your copy. Unless otherwise expressly stated, this e-mail is not intended to constitute a business letter, order form or other offer or invitation to you, nor does this e-mail form the basis of any contract. Contracts may not be concluded by e-mail and any contract or agreement attached is subject to contract and shall not and is not intended to create a legally binding relationship. This e-mail is sent by or on behalf of Encon Limited or the relevant subsidiary undertaking of Encon Limited with which you are dealing. Encon Limited is registered in England and Wales with company number 3411533 registered office at The Wolseley Center, Harrison Way, Leamington Spa, CV31 3HH, United Kingdom. VAT No 362 0233 93. Encon Ireland is registered in Ireland with company number 262605 registered office at Wolseley Ireland Moydrum Road, Athlone, Co Westmeath. Save a Tree ! - Please think before you print.

3 3

[MonetDB-users] Commercial Support
by Rick Spickelmier 17 Oct '08

17 Oct '08

Hi! Are there companies that are providing commercial support for MonetDB? The home page for MonetDB seems to imply that. If so, can somebody send me a pointer and does anybody have any experience with it? Thanks! Rick Rick Spickelmier | CTO | Success Metrics, Inc. | w (415) 738-2078 | ricks(a)successmetricsinc.com <mailto:ricks@successmetricsinc.com> | www.successmetricsinc.com | www.birst.com

3 2

[MonetDB-users] CREATE INDEX syntax?
by Richard Wesley 14 Oct '08

14 Oct '08

Hi Sjored - I have a lot of questions, so I thought I start a few threads if that is all right. What is the CREATE INDEX syntax? According to the manual it is CREATE [UNIQUE] INDEX <index_name> ON <table_name> ( <column_name>, ...) ; But whey I use this syntax, I get strange errors: sql>CREATE INDEX "_tidx_usernames_2a" ON "Performance"."usernames" ("username"); !CREATE INDEX: no such table 'usernames' sql>select count(*) from "Performance"."usernames"; +--------+ | count_ | +========+ | 5 | +--------+ sql> ________________________________________________________ Richard Wesley Senior Software Developer Tableau Software Visit: http://www.trytableau.com/now.html

1 0

[MonetDB-users] MonetDB on Windows
by bryan rasmussen 07 Oct '08

07 Oct '08

Hi, I haven't looked at monetdb for a while, but now for a new project I have I think it would be a really good solution. I downloaded the current windows installers yesterday (I want to develop this project on Windows for various reasons, specifically related to demos of another component part of the project I need to get up and running quickly that will need to run on Windows) anyway the installers did not install. I don't necessarily want to compile the source etc. since I didn't want to spend a lot of time on getting the database to work in the demo phase, so the question is: what is the last working Windows MSI version so I can download that version and worry about building a new one later? The main problem in the installed servers are a hard coded path to a config file, according to the following error message I get: Could not open file c:\Documents and Settings\sjoerd\My Documents\src\stable\vs32\NT32\etc\monetdb5.conf I guess I could create that path for the purposes of testing but I would rather get a working version. Thanks, Bryan Rasmussen

3 6

[MonetDB-users] Reusing index on another computer
by jz＠uva 07 Oct '08

07 Oct '08

Hi all, I have built an index on a server. I would like to reuse that index on my desktop for testing purposes. However, when I copy the demo directory (consisting of the index), and I restart MServer Mserver --dbinit='module(pathfinder);' --set xquery_backend=milprint_summer then I discover this error: !ERROR: database created with incompatible Mserver So my question is: can I, and if yes, how do I reuse/transfer an index from one computer to another one? Thanks, junte

6 13

[MonetDB-users] allocate timing and deallocate timing of memory
by gashi gashi 07 Oct '08

07 Oct '08

Hi, All I'm working with MonetDB Server v5.6.0, based on kernel 1.24.0 over RedHat Linux Enterprise 5.2. Would you tell me timing which in the memory is allocated and timing in which the memory is deallocated, or is there an instruction that deallocate the memory? Regards, gashi

2 1

[MonetDB-users] merovingian locking up - port issue?
by Ross Bates 07 Oct '08

07 Oct '08

I'm having an issue with merovingian locking up. This is the typical scenario. - Start mero and my db with ./merovingian sobi - Everything works great for anywhere between 50 & 100 queries, then mero stops responding (I'm using python, jdbc & php) - I can still connect to the db from the command line using ./mclient -d sobi -lsql -p 50001 - both merovingian and mserver5 processes continue to run, but mero will end up dying within 1-2 minutes by itself leaving the mserver5 process running - even when mero dies, it appears that all the ports are still in use. This is the result of netstat -a AFTER mero is dead: tcp 0 0 *:50000 *:* LISTEN tcp 0 0 *:50001 *:* LISTEN tcp 0 0 mayfair.local:50000 mayfair.local:48973 ESTABLISHED tcp 0 0 mayfair.local:48973 mayfair.local:50000 ESTABLISHED udp 0 0 *:50000 *:* - when I try to restart mero to reattach it to the database it fails with this error: ./merovingian: binding to stream socket port 50000 failed: Address already in use - stopping the db doesn't work becase mero isn't running. ./monetdb stop sobi warning: MonetDB Database Server is not running stop: cannot perform: MonetDB Database Server (merovingian) is not running - the only thing left to do is kill the mserver5 process and restart both mero and the db. Unfortunately the merovingian.log file doesn't tell me anything. Any other tips for helping troubleshoot this issue? The version is 5.7.0 and was built from the Aug 23rd nightly. Many Thanks, Ross

2 8

[MonetDB-users] Can I copy a local txt file into a remote MonetDB table
by Li Xiaohua 07 Oct '08

07 Oct '08

Hi: I want to copy a txt file into a remote MonetDB table. When I use the copy into command. what should I use after the "into"? using servername.database.schema.tablename? does it work?

1 0

[MonetDB-users] running the monetdb MSI installers on Windows
by bryan rasmussen 06 Oct '08

06 Oct '08

Hi, Is there anyone that has managed to run any particular Monet MSI installer with a resulting successful install of MonetDB (successful implying the database server will start afterwards)? If yes, which version did you use? The problems I have run into so far with the current stable release: Can not find DLLs when starting. Dlls do not seem to have proper entry points (when I try to run them) the server is expecting to find a config file under documents and Settings\Sjoerd\..... I tried uninstalling and installing one release earlier for the sql database, got the first problem again, didn't check if it had all the other problems. Best Regards, Bryan Rasmussen

1 0

Re: [MonetDB-users] best practices with connection pool
by gashi gashi 01 Oct '08

01 Oct '08

Hi,All I found this subject in the archives, and I am having the same problem. ---------------------------------------------------------------------- >> What is currently the best practice of connecting to MonetDB5/SQL? Is >> there much overhead in the connection phase (thus: cache the connection) >> or can a developer just reconnect each time the database connection is >> required without much overhead? >> >> The target platform is extremely demanding in requests and memory, so I >> would like to design it good from the start. >Then go for connection pooling. Logging in requires a handshake, which >is expensive on large scale. Beware that using >1 connection at the >same time to an SQL server is a challenge that is only minimally tested. >My practical experiences with that boils down to signal 11. ---------------------------------------------------------------------- Would you let me know more detail? Your meaning of "Beware that using >1 connection at the same time..." is which case? Case 1. : 2 or more process try to connect at the same. Case 2. : 1 process is connecting, and other process try to connect. I appreciate your kindly support. gashi

2 3

[MonetDB-users] billions rows ?
by eric Gtep 28 Sep '08

28 Sep '08

Hello, I'm testing for store large collections of genotypes on Monetdb. In few words, a genotype is a little piece of information ( about 3 or 4 char ) related to an individual and a dna marker. So a genotypes looks like contents of cell of sheet. Sometime we need to access them by individuals (cross marker), sometime by dna marker (cross individual). Today genotyping technologies provide to get 600, 000,0000 genotypes by run. Is MonetDB able to manage efficiently tables with several billions of rows ? have you any example of application with lot ( > 1 billion) of rows in only one table ? I' ve compiled and installed monetDB/SQL on a Dell PE2950 with 2 quadcores intel xeon 2.66 Ghz and 4 GB RAM with success I 've created a very basic table to store genotype : Create table genotypes ( ind char(10), mark char(10), alleles(char3) ) after I've populated this table with the "copy into table" statement About 370 millions rows have been loaded after 7 minutes. I haven't defined any index. >From mclient I sent the query below : select * from genotypes where alleles ="A A"; Immediatly the server became frozen and after about ten minutes a w unix command showed : load average 16 !!!! I stopped the query Could you explain to me what has appended ? Is this behaviour normal ? Is it possible to restrict the cpu ressources allowed to monetDB ? Thank you in advance for your advices and your help Eric

4 3

[MonetDB-users] How to declare a bug ?
by Pierre BOIZOT 27 Sep '08

27 Sep '08

Hi All, Somebody can explain me how i can submit a bug ? Regards. Pierre -- Pierre Boizot Chemin des Ecureuils, 8 CH-1009 PULLY Mobile : +41 795713660

2 1

[MonetDB-users] Perl DBD
by Pierre BOIZOT 26 Sep '08

26 Sep '08

Hi all, I install on a ubuntu monetdb , i began to test sql , insert , update and select . very nice performance. But when i try to use Perl DBD to access at the database . nothing work :( the CPAN is not update. the perl DBD in the source have no test no example :( what programming interface work with monetdb ? perl ? perhaps i jump something. python ? Where can i find small examples? Thank you for your help Regards. Pierre - Pierre Boizot Rue des philosophes , 13 1400 Yverdon les bains Mobile : +41 795713660

3 2

[MonetDB-users] Installation Problems
by Scott Stephens 25 Sep '08

25 Sep '08

I've been attempting to install MonetDB by compiling the Aug. 2008 superball. My system is 64-bit linux, specifically the Ubuntu Intrepid Ibex alpha . It appears that the installation is failing while compiling the JDBC driver, with the error message: BUILD FAILED java.lang.UnsupportedClassVersionError: Bad version number in .class file I'm using the Sun Java 1.5.0 JRE and JDK. You can look at my full installation log here: http://docs.google.com/Doc?id=dhqbmd8x_6fqdnn7c3 Can somebody help me out? I'm hoping I've just made an easily corrected newbie mistake, but I'm out of things to try. Thanks, Scott

3 8

[MonetDB-users] basic math error?
by Kirk Abbott 24 Sep '08

24 Sep '08

Hello All, sql>select 585.0 / 635.0; +----------------------+ | sql_div_single_value | +======================+ | 0 | +----------------------+ and sql>select cast(585.0 / 635.0 as numeric(10,4)); +----------------------+ | sql_div_single_value | +======================+ | 0.0000 | +----------------------+ Is not what I was expecting. Ideas? Kirk.

4 5

[MonetDB-users] Bug....
by Pierre BOIZOT 23 Sep '08

23 Sep '08

Hi all, a test with the gold data .... no errors on the load if i stop and restart the database errors is allways here. sql>select age,gender, count(*), avg(price) from car group by age, gender; +-----+-------+-------------+-----------------------+ | age |gender |count_gender |avg_price | +=====+=======+=============+=======================+ | 19 |f | 110025 | -1449.6788093615087 | | 41 |m | 97200 | -6411.6489300411522 | ........ 52 |f | 95940 | -6467.6865332499483 | +-----+-------+-------------+-----------------------+ Timer 2694.008 msec 92 rows sql>select count(*) from car where price = 0; +--------+ | count_ | +========+ | 0 | +--------+ Timer 2502.145 msec 1 rows sql>select count(*) from car where price < 0; +--------+ | count_ | +========+ | 0 | +--------+ Timer 148.724 msec 1 rows see the trace file . Regards Pierre -- Pierre Boizot Rue des philosophes , 13 1400 Yverdon les bains Mobile : +41 795713660

1 0

[MonetDB-users] Several newbie questions: Performance and plans
by Mark V 23 Sep '08

23 Sep '08

Hi, I am more of an end user, new to column store DB's in general and MonetDB in particular. I'm thinking of using MonetDB in a context where CRUD would be done using SQL statements (as much as possible). I have some questions that I'd appreciate some feedback on: It seems that in column stores single insert/update/delete are much slower than reads. In LucidDB this can be in the order of seconds. Q1) In MonetDB are single insert/update/delete in the order of seconds? Comparable to LucidDB? Q2) If so, when might this change/improve and by how much? On comparative performance. I've seen the Goggle tech talk (2008/01) on MonetDB/X100 indicating some incredible performance: Q3) When might those improvements be incorporated into the stable/released MonetDB? Q4) How does MonetDB compare to Kdb and Vertica? Finally, I'm toying with writing a MonetSQL adapter for Ruby's Sequel ORM, hence the reliance on SQL statements. I have seen the MonetDB ActiveRecord library, which uses SWIG to make the Ruby MonetDB-mapi library. Q5) Will MonetDB be implementing something like MySQL's Information_Schema? Q6) What is the relative performance of the JDBC and SWIG-MAPI interfaces? Thanks Mark

2 1

[MonetDB-users] Ubuntu packages and launchpad PPA for MonetDB
by Gijs Molenaar 12 Sep '08

12 Sep '08

Hello everyone, I'm working on precompiled Ubuntu packages for MonetDB, which can be found here: https://launchpad.net/~monetdb/+archive These packages are meant for Ubuntu users who don't want to compile monetdb on every Ubuntu system where they want to use this software. You can install the packages and automatically update to new released packages by adding the following to /etc/apt/sources.list: --- 8< cut here --- deb http://ppa.launchpad.net/monetdb/ubuntu hardy main deb-src http://ppa.launchpad.net/monetdb/ubuntu hardy main ------------------- These packages are quite 'fresh' and not fully tested and completed, but the basics should work fine. If you want to test them out or/and you have some comments, feel free to contact me. Note that these packages are not signed due to a limitation in launchpad. -- Gijs Molenaar http://gijs.pythonic.nl

3 5

Re: [MonetDB-users] I'm looking for a fast way to write data to the DB from a scripting language.
by Michalis Polakis 07 Sep '08

07 Sep '08

Good morning Jay, you can use the Ruby MAPI Wrapper => MAPI calls through Ruby. It's elegant and simple and was used to implement the Ruby on Rails adapter for MonetDB. Plus, from what I see in the SWIG input file, the insert and copy_from functions should be implemented in the Ruby MAPI Wrapper. If you want to go the Ruby way, you may find the files in the MonetDB clients package or at http://rubyforge.org/projects/monetdb-ror/ as part of the Ruby on Rails MonetDB adapter. Regards, Michalis ----------------------------------------------------------------------------------------------- Hi, I have 50 million rows and 15 columns that I'd like to put into one table. My first attempt to transfer them was through PHP using the monetdb_query method. This is causing a lot of disk thrashing and my throughput is about 100 rows/sec. I get an "undefined function" error for both monetdb_insert and monetdb_copy_from. I would like to find a fast method, but from inside a common scripting language. Thanks, Jay

3 3

[MonetDB-users] I'm looking for a fast way to write data to the DB from a scripting language.
by JJ Jakosky 07 Sep '08

07 Sep '08

Hi, I have 50 million rows and 15 columns that I'd like to put into one table. My first attempt to transfer them was through PHP using the monetdb_query method. This is causing a lot of disk thrashing and my throughput is about 100 rows/sec. I get an "undefined function" error for both monetdb_insert and monetdb_copy_from. I would like to find a fast method, but from inside a common scripting language. Thanks, Jay

1 0

[MonetDB-users] Dual instances of MonetDB for 1 dbfarm - is it possible?
by McKennirey.Matthew 04 Sep '08

04 Sep '08

3 4

[MonetDB-users] Monetdb - User defined functions
by Kirk Abbott 04 Sep '08

04 Sep '08

Hello, We are trying to move some data over to Monetdb and are finding that we are missing some of our favorite formatting functions (to_date, to_char etc.). Does Monetdb have a user defined functions extension framework? If so, we would try to mimic these functions there. Thanks, Kirk.

3 8

Re: [MonetDB-users] ERROR 404 /~monet/tars/latest/stable/MonetDB.tar.gz.md5sum
by McKennirey.Matthew 04 Sep '08

04 Sep '08

3 2

[MonetDB-users] Incorrect SQL update result
by Christophe Galerne 03 Sep '08

03 Sep '08

Hi, I am running: # MonetDB server v5.6.0, based on kernel v1.24.0 # Serving database 'clg' # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved # Visit http://monetdb.cwi.nl/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module v2.24.0 loaded I am copying some sample SQL which illustrates what I believe is a bug: ################################################################################ CREATE TABLE SOURCE ( TOTO VARCHAR(5), TITI VARCHAR(5), VALUE DOUBLE ); INSERT INTO SOURCE VALUES('1', '1', 5.0) , ('1', '2', 4.0), ('2', '1', 5.0), ('2', '2', 5.0), ('2', '3', 5.0); INSERT INTO SOURCE VALUES('1', '1', 5.0) , ('1', '2', 4.0), ('2', '1', 5.0), ('2', '2', 5.0), ('2', '3', 5.0); CREATE TABLE TARGET ( TOTO VARCHAR(5), TITI VARCHAR(5), VALUE DOUBLE ); INSERT INTO TARGET VALUES('1', '1', 0) , ('1', '2', 0), ('2', '1', 0), ('2', '2', 0), ('2', '3', 0); UPDATE TARGET SET VALUE = ( SELECT SUM(VALUE) FROM SOURCE WHERE TOTO=SOURCE.TOTO and TITI=SOURCE.TITI); CREATE TABLE TARGETOS ( TATA VARCHAR(5), TUTU VARCHAR(5), VALUE DOUBLE ); INSERT INTO TARGETOS VALUES('1', '1', 0) , ('1', '2', 0), ('2', '1', 0), ('2', '2', 0), ('2', '3', 0); UPDATE TARGETOS SET VALUE = ( SELECT SUM(VALUE) FROM SOURCE WHERE SOURCE.TOTO=TATA and SOURCE.TITI=TUTU); ################################################################################ The expected results is for the tables TARGET and TARGETOS to contain the same result since they only differ by the name of their columns. It seems that monetdb gets confused by the fact that table SOURCE and table TARGET have the column names. Thanks, Christophe

2 2