- users-list - monetdb.org

Re: [MonetDB-users] best practices with connection pool
by gashi gashi 01 Oct '08

01 Oct '08

Hi,All I found this subject in the archives, and I am having the same problem. ---------------------------------------------------------------------- >> What is currently the best practice of connecting to MonetDB5/SQL? Is >> there much overhead in the connection phase (thus: cache the connection) >> or can a developer just reconnect each time the database connection is >> required without much overhead? >> >> The target platform is extremely demanding in requests and memory, so I >> would like to design it good from the start. >Then go for connection pooling. Logging in requires a handshake, which >is expensive on large scale. Beware that using >1 connection at the >same time to an SQL server is a challenge that is only minimally tested. >My practical experiences with that boils down to signal 11. ---------------------------------------------------------------------- Would you let me know more detail? Your meaning of "Beware that using >1 connection at the same time..." is which case? Case 1. : 2 or more process try to connect at the same. Case 2. : 1 process is connecting, and other process try to connect. I appreciate your kindly support. gashi

2 3

[MonetDB-users] billions rows ?
by eric Gtep 28 Sep '08

28 Sep '08

Hello, I'm testing for store large collections of genotypes on Monetdb. In few words, a genotype is a little piece of information ( about 3 or 4 char ) related to an individual and a dna marker. So a genotypes looks like contents of cell of sheet. Sometime we need to access them by individuals (cross marker), sometime by dna marker (cross individual). Today genotyping technologies provide to get 600, 000,0000 genotypes by run. Is MonetDB able to manage efficiently tables with several billions of rows ? have you any example of application with lot ( > 1 billion) of rows in only one table ? I' ve compiled and installed monetDB/SQL on a Dell PE2950 with 2 quadcores intel xeon 2.66 Ghz and 4 GB RAM with success I 've created a very basic table to store genotype : Create table genotypes ( ind char(10), mark char(10), alleles(char3) ) after I've populated this table with the "copy into table" statement About 370 millions rows have been loaded after 7 minutes. I haven't defined any index. >From mclient I sent the query below : select * from genotypes where alleles ="A A"; Immediatly the server became frozen and after about ten minutes a w unix command showed : load average 16 !!!! I stopped the query Could you explain to me what has appended ? Is this behaviour normal ? Is it possible to restrict the cpu ressources allowed to monetDB ? Thank you in advance for your advices and your help Eric

4 3

[MonetDB-users] How to declare a bug ?
by Pierre BOIZOT 27 Sep '08

27 Sep '08

Hi All, Somebody can explain me how i can submit a bug ? Regards. Pierre -- Pierre Boizot Chemin des Ecureuils, 8 CH-1009 PULLY Mobile : +41 795713660

2 1

[MonetDB-users] Perl DBD
by Pierre BOIZOT 26 Sep '08

26 Sep '08

Hi all, I install on a ubuntu monetdb , i began to test sql , insert , update and select . very nice performance. But when i try to use Perl DBD to access at the database . nothing work :( the CPAN is not update. the perl DBD in the source have no test no example :( what programming interface work with monetdb ? perl ? perhaps i jump something. python ? Where can i find small examples? Thank you for your help Regards. Pierre - Pierre Boizot Rue des philosophes , 13 1400 Yverdon les bains Mobile : +41 795713660

3 2

[MonetDB-users] Installation Problems
by Scott Stephens 25 Sep '08

25 Sep '08

I've been attempting to install MonetDB by compiling the Aug. 2008 superball. My system is 64-bit linux, specifically the Ubuntu Intrepid Ibex alpha . It appears that the installation is failing while compiling the JDBC driver, with the error message: BUILD FAILED java.lang.UnsupportedClassVersionError: Bad version number in .class file I'm using the Sun Java 1.5.0 JRE and JDK. You can look at my full installation log here: http://docs.google.com/Doc?id=dhqbmd8x_6fqdnn7c3 Can somebody help me out? I'm hoping I've just made an easily corrected newbie mistake, but I'm out of things to try. Thanks, Scott

3 8

[MonetDB-users] basic math error?
by Kirk Abbott 24 Sep '08

24 Sep '08

Hello All, sql>select 585.0 / 635.0; +----------------------+ | sql_div_single_value | +======================+ | 0 | +----------------------+ and sql>select cast(585.0 / 635.0 as numeric(10,4)); +----------------------+ | sql_div_single_value | +======================+ | 0.0000 | +----------------------+ Is not what I was expecting. Ideas? Kirk.

4 5

[MonetDB-users] Bug....
by Pierre BOIZOT 23 Sep '08

23 Sep '08

Hi all, a test with the gold data .... no errors on the load if i stop and restart the database errors is allways here. sql>select age,gender, count(*), avg(price) from car group by age, gender; +-----+-------+-------------+-----------------------+ | age |gender |count_gender |avg_price | +=====+=======+=============+=======================+ | 19 |f | 110025 | -1449.6788093615087 | | 41 |m | 97200 | -6411.6489300411522 | ........ 52 |f | 95940 | -6467.6865332499483 | +-----+-------+-------------+-----------------------+ Timer 2694.008 msec 92 rows sql>select count(*) from car where price = 0; +--------+ | count_ | +========+ | 0 | +--------+ Timer 2502.145 msec 1 rows sql>select count(*) from car where price < 0; +--------+ | count_ | +========+ | 0 | +--------+ Timer 148.724 msec 1 rows see the trace file . Regards Pierre -- Pierre Boizot Rue des philosophes , 13 1400 Yverdon les bains Mobile : +41 795713660

1 0

[MonetDB-users] Several newbie questions: Performance and plans
by Mark V 23 Sep '08

23 Sep '08

Hi, I am more of an end user, new to column store DB's in general and MonetDB in particular. I'm thinking of using MonetDB in a context where CRUD would be done using SQL statements (as much as possible). I have some questions that I'd appreciate some feedback on: It seems that in column stores single insert/update/delete are much slower than reads. In LucidDB this can be in the order of seconds. Q1) In MonetDB are single insert/update/delete in the order of seconds? Comparable to LucidDB? Q2) If so, when might this change/improve and by how much? On comparative performance. I've seen the Goggle tech talk (2008/01) on MonetDB/X100 indicating some incredible performance: Q3) When might those improvements be incorporated into the stable/released MonetDB? Q4) How does MonetDB compare to Kdb and Vertica? Finally, I'm toying with writing a MonetSQL adapter for Ruby's Sequel ORM, hence the reliance on SQL statements. I have seen the MonetDB ActiveRecord library, which uses SWIG to make the Ruby MonetDB-mapi library. Q5) Will MonetDB be implementing something like MySQL's Information_Schema? Q6) What is the relative performance of the JDBC and SWIG-MAPI interfaces? Thanks Mark

2 1

[MonetDB-users] Ubuntu packages and launchpad PPA for MonetDB
by Gijs Molenaar 12 Sep '08

12 Sep '08

Hello everyone, I'm working on precompiled Ubuntu packages for MonetDB, which can be found here: https://launchpad.net/~monetdb/+archive These packages are meant for Ubuntu users who don't want to compile monetdb on every Ubuntu system where they want to use this software. You can install the packages and automatically update to new released packages by adding the following to /etc/apt/sources.list: --- 8< cut here --- deb http://ppa.launchpad.net/monetdb/ubuntu hardy main deb-src http://ppa.launchpad.net/monetdb/ubuntu hardy main ------------------- These packages are quite 'fresh' and not fully tested and completed, but the basics should work fine. If you want to test them out or/and you have some comments, feel free to contact me. Note that these packages are not signed due to a limitation in launchpad. -- Gijs Molenaar http://gijs.pythonic.nl

3 5

Re: [MonetDB-users] I'm looking for a fast way to write data to the DB from a scripting language.
by Michalis Polakis 07 Sep '08

07 Sep '08

Good morning Jay, you can use the Ruby MAPI Wrapper => MAPI calls through Ruby. It's elegant and simple and was used to implement the Ruby on Rails adapter for MonetDB. Plus, from what I see in the SWIG input file, the insert and copy_from functions should be implemented in the Ruby MAPI Wrapper. If you want to go the Ruby way, you may find the files in the MonetDB clients package or at http://rubyforge.org/projects/monetdb-ror/ as part of the Ruby on Rails MonetDB adapter. Regards, Michalis ----------------------------------------------------------------------------------------------- Hi, I have 50 million rows and 15 columns that I'd like to put into one table. My first attempt to transfer them was through PHP using the monetdb_query method. This is causing a lot of disk thrashing and my throughput is about 100 rows/sec. I get an "undefined function" error for both monetdb_insert and monetdb_copy_from. I would like to find a fast method, but from inside a common scripting language. Thanks, Jay

3 3