- users-list - monetdb.org

Re: [MonetDB-users] Using monetdb and sqlalchemy
by Sabst 23 May '12

23 May '12

Still stuck. If you know the answer please share it. Also if you think the two are not meant to work together, share it too. Anyone? > ----- Message transféré de sabst(a)free.fr ----- > Date : Thu, 10 May 2012 15:07:17 +0200 > De : sabst(a)free.fr > Adresse de retour :sabst@free.fr > Sujet : Using monetdb and sqlalchemy > À : monetdb-users(a)lists.sourceforge.net > > Hi, > I am just starting using monetdb and, even if I could find discussions about > Python/SqlAlchemy integration, I could not make it work. > > I installed python-monetdb module and have a running monetdb server. > > Now I expect my client application to work with monetdb as it does with > postgresql and instead of passing a connection string like: > postgres://user:password@somehost:port/db > I use: > monetdb://user:password@somehost/db > > But I get this error: > ArgumentError: Could not determine dialect for 'monetdb'. > > What are the steps required to integrate monetdb into SqlAlchemy? > > Certainly missing the obvious but I'm currently stuck. > > Thank you in advance for any help, > Stephane. > > ----- Fin du message transféré ----- > > > >

2 4

[MonetDB-users] Monet on SUSE
by Hualin Wang 20 May '12

20 May '12

Dear All, I have trouble compliing the Monet on my SUSE 11 box. The server has an old PCRE installed, like 7.8. I download my own version of pcre 8.3, and complie it with utf-8 enabled. I set up the following three environment vars: pcre_LIBS, pcre_CFLAGS, PKG_CONFIG_PATH to point to the right place. But the Monet ./configure still complain utf-8 support no. By the way, the pcre check is yes. Which version of PCRE package I need install on SUSE? By the way, what this sentence means from monetdb.org "Make sure you install the xxx-dev versions on binary distributions of the necessary packages as well!" Thanks for any help/hints! Best Regards, Hualin

2 1

[MonetDB-users] Bad SELECT performance after deletion of 1 record
by suncpp 17 May '12

17 May '12

Hi everyone, I had added about 30,000,000 records in one MonetDB table. The performance of executing SELECT-queries was really great. But after the deletion of only 1 record from the table, the performance had slowed down considerably. I have seen the topics here with alike problem but I have not found the solution. Could anyone help me to understand what the problem is? -- View this message in context: http://old.nabble.com/Bad-SELECT-performance-after-deletion-of-1-record-tp3… Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] strange trigger behaviour
by Majid Azimi 16 May '12

16 May '12

Hi guys, I have created a table named users: CREATE TABLE names ( name VARCHAR(255) NOT NULL ); another table named names2 which is same as users. only table names are different. I have created a after insert trigger like this: CREATE TRIGGER trg_names_ai AFTER INSERT ON names REFERENCING NEW ROW AS new_row FOR EACH ROW WHEN new_row.name = '<some name>' INSERT INTO names2 VALUES(new_row.name); The problem is it doesn't care about WHEN clause. Any INSERT query will appear in names2. Can anyone help?

2 1

[MonetDB-users] timeout while waiting for database 'xxx' to initialise the sql scenario
by Brian J. Cardiff 11 May '12

11 May '12

Hi all, I'm trying monetdb among with other vertical databases. After some experiments I am unable to start the database again. I'm not sure what the things I did cause the issue, but happens twice already. I have restart monetdbd and server without luck. Is there any way to solve this issue? $ monetdb start xxx starting database 'xxx'... FAILED start: starting 'xxx' failed: timeout while waiting for database 'xxx' to initialise the sql scenario The experiments consist on: 1. Bulk load .csv file with ~ 20*10^6 records 2. Do some aggregate functions with "SELECT AVG(..) FROM TBL WHERE ID IN (...)" with different amount of ids in list (10 - 5000) (yeap, ugly). 3. Add columns to TBL, Massive update columns in TBL, Drop columns in TBL. [version] MonetDB Database Server Toolkit v1.0 (Apr2012) MonetDB Database Server v1.6 (Apr2012) [morovingian.log] 2012-05-11 15:05:55 MSG xxx[637]: # Listening for UNIX domain connection requests on mapi:monetdb:///Users/bcardiff/Work/Xxx/db/xxx/.mapi.sock 2012-05-11 15:05:55 MSG xxx[637]: # MonetDB/GIS module loaded 2012-05-11 15:05:55 MSG xxx[637]: # MonetDB/SQL module loaded 2012-05-11 15:06:05 MSG merovingian[588]: sending process 637 (database 'xxx') the TERM signal 2012-05-11 15:06:05 MSG merovingian[588]: database 'xxx' has shut down 2012-05-11 15:06:05 MSG merovingian[588]: database 'xxx' (637) has exited with exit status 0 2012-05-11 15:06:05 ERR control[588]: (local): failed to fork mserver: timeout while waiting for database 'xxx' to initialise the sql scenario -- Brian J. Cardiff - Manas Technology Solutions [ar.phone] 4796.0232 #BCR(227) [us.phone] 312.612.1050 #BCR(227) [email] bcardiff(a)manas.com.ar [web] www.manas.com.ar [weblog] http://weblogs.manas.com.ar/bcardiff/

1 0

[MonetDB-users] Using monetdb and sqlalchemy
by sabst＠free.fr 10 May '12

10 May '12

Hi, I am just starting using monetdb and, even if I could find discussions about Python/SqlAlchemy integration, I could not make it work. I installed python-monetdb module and have a running monetdb server. Now I expect my client application to work with monetdb as it does with postgresql and instead of passing a connection string like: postgres://user:password@somehost:port/db I use: monetdb://user:password@somehost/db But I get this error: ArgumentError: Could not determine dialect for 'monetdb'. What are the steps required to integrate monetdb into SqlAlchemy? Certainly missing the obvious but I'm currently stuck. Thank you in advance for any help, Stephane.

1 0

Re: [MonetDB-users] Strange things...
by Alexey Mints 07 May '12

07 May '12

Dear All! My old problem http://sourceforge.net/mailarchive/message.php?msg_id=29168305 strikes back on the new version. The situation is just the same - combination of two conditions gives empty result (that is wrong), while each condition alone gives correct resultset. The problem shows up only in the SECOND run of quite a huge set of queries (the first one runs Ok). Even more, there has to be some delay (~10 sec) before the second run, otherwise everything works. Deleting all records does not help, problem disappears only if I recreate the table. What on Earth can this be? It seems rather hard to pin down the problem with SQL-methods only. Just in case - I attach traces of correct and incorrect queries. Regards, Alexey Mints

1 0

[MonetDB-users] problem after upgrading (probably)
by Gijs Molenaar 04 May '12

04 May '12

Hi MonetDB people, greetings from the other side of the street. Long time no see. I just upgraded my monetdb installation to the April release. I use the Ubuntu repo. my database is running, but when I try to connect with the Python API or mclient I get the error: CREATE AGGREGATE: name 'median' (tinyint(8)) already in use Any suggestion how to figure out how to solve this problem? greetings, -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

5 15

[MonetDB-users] bug tracker problems?
by Gijs Molenaar 04 May '12

04 May '12

Me again, I have some issues with the ubuntu packages that I would like to report, but I can't get access to the bug tracker. First of all, the machines seems to respond not really fast. But when I open a new account after the token verification the webserver gives an empty response (according to my browser). I tried this a couple of times now. Is it me? -- Gijs Molenaar http://www.astro.uva.nl/people/gijs-molenaar/

2 2

[MonetDB-users] MonetDB clustering
by Maik Strewe 03 May '12

03 May '12

Hi and good morning from Germany. My name is Maik Strewe and I want to test the performance of the monetDB in a cluster. My application needs millions of rows but few columns. I have to set up an monetDB cluster on 2 servers at an external server hoster. The single instances on the hosts are installed and running. Then I tried to use the "monetdb discover" command but the servers cant see each other. As far as I know the discovery is based on a UDP broadcast, right? Is it possible to configure the cluster members in a configuration file so that the monetdb instance can discover the cluster member by configuration?

4 5

[MonetDB-users] MonetDB clustering
by Maik Strewe 03 May '12

03 May '12

Hi and good morning from Germany. My name is Maik Strewe and I want to test the performance of the monetDB in a cluster. My application needs millions of rows but few columns. I have to set up an monetDB cluster on 2 servers at an external server hoster. The single instances on the hosts are installed and running. Then I tried to use the "monetdb discover" command but the servers cant see each other. As far as I know the discovery is based on a UDP broadcast, right? Is it possible to configure the cluster members in a configuration file so that the monetdb instance can discover the cluster member by configuration?

1 0

Re: [MonetDB-users] Strange things...
by Alexey Mints 02 May '12

02 May '12

Problem solved itself (for now) with the upgrade to the latest version.

1 0

[MonetDB-users] Selecting cracker update algorithms
by Stef Sijben 02 May '12

02 May '12

Hello, We are a group of students trying to repeat the experiments described in the paper Updating a Cracked Database [1]. We know how to use the cracking optimizer, but what we can't figure out is how to select one of the described algorithms to be used for updates. We found implementations of all the algorithms in the source code and I have been looking in documentation and the source code for a considerable time without any luck. Can someone please point us in the right direction? Thanks, Stef Sijben References [1] S. Idreos, M.L. Kersten, and S. Manegold. Updating a cracked database. In Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pages 413–424. ACM, 2007.

1 0

[MonetDB-users] Announcement: New Apr2012 Feature release of MonetDB suite
by Sjoerd Mullender 27 Apr '12

27 Apr '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2012 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Apr 2012 feature release Build Environment * Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case. * On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997. SQL * Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient. * Stripped off implementation-specific parts from error messages before they get presented to the user. Java Module * Password reading by JdbcClient no longer results in strange artifacts * JdbcClient now returns exit code 1 in case of failures * Implemented missing Number types support in PreparedStatement.setObject() * Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument. * Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages Client Package * ODBC: Implemented the SQL_ATTR_CONNECTION_TIMEOUT attribute. * mclient now has a -a (--autocommit) option to turn off autocommit mode. MonetDB5 Server * The type "chr" has been removed. chr has long been superseded by bte for 1 byte arithmetic plus it is pretty useless to hold single characters since we use Unicode and thus only a tiny subset of the supported character set would fit. * Fixed potential crash by dealing better with non-standard types. * A couple of memory leaks have been plugged. Merovingian * The logfile and pidfile monetdbd properties are now displayed with dbfarm path when relative * Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries. * Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982. * Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983. * Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984. * Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984. MonetDB Common * The type "chr" has been removed. chr has long been superseded by bte for 1 byte arithmetic plus it is pretty useless to hold single characters since we use Unicode and thus only a tiny subset of the supported character set would fit. Bug Fixes

1 0

Re: [MonetDB-users] Strange things...
by Alexey Mints 26 Apr '12

26 Apr '12

Hi Stefan! condition1 is like this: exists (select 1 from temp_assoc_counts tc where tc.xtrsrc_id = temp_associations.xtrsrc_id and tc.assoc_count = 1) condition 2 is like: kind is null I have also bumped in the same situation, when condition2 was like "kind = 2". But when "2" was replaced to a column name (with all values in this column were 2) - it worked fine. I.e. there is something wrong when constant appears in the condition. Traces can be seen in attachment.

1 0

[MonetDB-users] Strange things...
by Alexey Mints 23 Apr '12

23 Apr '12

Hello! I have a strange behavior of my MonetDB database. After some actions I run into situation, when queries select * from table1 where {condition1}; and select * from table1 where {condition2}; return the same results (same rows), but select * from table1 where {condition1} and {condition2}; is empty, which is definitely wrong. I cannot see any problems from SQL-side. I am still struggling to get a minimal code producing this bug. Even worse - it is not always occurring under the same conditions... How can I get more information on what's happened? Alexey

2 1

[MonetDB-users] Performance Degredation
by Joseph Brower 13 Apr '12

13 Apr '12

We discussed this a little bit before, but I was called away. Now that I have some time, I'm hoping to discuss this again and hopefully find resolution. Normally our MonetDB instance is running happily and doesn't have any problems. We've limited the number of queries being performed concurrently to the number of cores in the machine, and we do all of our inserts using COPY INTO statements. The problem is that, for no reason that we can see, suddenly MonetDB starts to perform incredibly slowly. This causes queries to stop returning, or, what used to return after less than a second starts to take over 30 seconds, and causes a timeout on our system that produces the graphs based on this information. We restart MonetDB and the problem goes away for a while. Sometimes it drops again after a short time (as little as a few hours) other times it will run fine for a few days. Does anyone have any ideas? What other information can I provide to get a closer look at why it is behaving this way? Thanks, Joseph Brower

1 0

[MonetDB-users] MonetDB driver for node.js?
by Robert Kosara 05 Apr '12

05 Apr '12

Hello, I am looking for a better database for a project where I am dealing with large numbers of data points, and MonetDB seems like a really interesting choice for that. This is for a web application that is written in JavaScript/node.js, though, so I'm wondering if anybody might be working on a MonetDB driver for node. I know that your focus is on scientific data, and JavaScript might not be a big factor there, but node is gaining a lot of momentum for web applications. Also, is there a way to search the mailing list archive? I can't seem to find a way to do that on the sourceforge website. Best, Robert

3 2

[MonetDB-users] Funnel
by Joseph Brower 02 Apr '12

02 Apr '12

We're looking at setting up the funnel. We've got 2 database servers that we're planning on using, and we simply want to balance the requests between the two. We don't want any of the multiplexing to take effect. I'm not seeing much as far as guides or how to set this up in the man pages. Does anyone have any good pointers on how to set things up this way? Thanks, Joseph Brower

3 7

[MonetDB-users] MonetDB - Mondrian connection
by Pedro Salgueiro 30 Mar '12

30 Mar '12

Hi guys! I have been trying MonetDB and it's awesome. I tried to connect MonetDB to Mondrian using olap4j. I managed to establish the connection using the MonetDB JDBC, but I have a problem: There are no errors, but it presents the wrong results. Every MDX query returns the right tables, but with all values "zeroed", something like this: ------------------------ Unit Sales ========== 0 ------------------------ I tried to make a simple JDBC connection, and it worked out ok. I am using the following connection string: String connectionProperties = "jdbc:mondrian:JdbcDrivers=nl.cwi.monetdb.jdbc.MonetDriver;" + "Jdbc=jdbc:monetdb://localhost/foodmart;" + "JdbcUser=monetdb;" + "JdbcPassword=monetdb;" + "Catalog=FoodMart.xml;"; Does anyone knows if I am doing anything wrong? Has anyone managed to make a working connection between Mondrian and Monet? If so, did you have to make something special? It would be great to have a working connection between Mondrian and MonetDB, since its much faster than MySQL or LucidDB, which I have already tried. Best regards, Pedro

2 5

[MonetDB-users] Fwd: Re: DB Corruption
by Joseph Brower 27 Mar '12

27 Mar '12

This is what one of my coworkers reports about the problem. both columns are varchar(24). We have confirmed that there are no null values. We have a theory… Approximately every 6 hours, monetdb stops responding to queries and starts thrashing the disk. This behavior has been confirmed without any inserts. Because of this behavior, we routinely stop and start the service. After a restart, the service returns to a normal state. This appears to be the core problem. Our guess is that we restarted the service in the middle of a massive import. We will make sure this doesn't happen again, however, we still cannot find any sort of resolution as to why monetdb routinely starts thrashing the disk (on the order of writing 150MB/s) without any inserts, only select queries. Sorry to change the topic, but we're moving on the assumption we corrupted our db by restarting the service in the middle of a massive "INSERT INTO FROM" query. Can you think of any reason why monetdb would need to write to the disk in such an aggressive manner while ONLY serving select queries? This appears to be the core problem. Thanks On Mar 26, 2012, at 9:23 AM, Joseph Brower wrote: > What would you like me to tell them? > > > -------- Original Message -------- > Subject: Re: [MonetDB-users] DB Corruption > Date: Sun, 25 Mar 2012 09:15:24 +0200 > From: Martin Kersten <Martin.Kersten(a)cwi.nl> > Reply-To: Martin.Kersten(a)cwi.nl, Communication channel for MonetDB > users <monetdb-users(a)lists.sourceforge.net> > To: Communication channel for MonetDB users > <monetdb-users(a)lists.sourceforge.net> > > > > What is the column type? > Is the situation repeatable using a fresh database? > > regards, Martin > > On 3/25/12 6:34 AM, Joseph Brower wrote: > > We just checked. There are no nulls in the database at all. > > Additionally, there is only one column that has a different count. > > > > Thanks, > > > > Joseph Brower > > > > On 03/24/2012 09:58 AM, Stefan Manegold wrote: > >> Joseph, > >> > >> since you did not specify your queries in more detail, I assume they are as > >> simple as > >> (1) select count(*) from<your_table>; > >> and > >> (2) select count(single_column) from<your_table>; > >> > >> In that case, you could try the following sanity check: > >> > >> (3) select count(*) from<your_table> where single_column is null; > >> > >> and check whether that returns the difference between your counts. > >> > >> If that indeed returns the difference between your global count(*) and > >> count(single_column), this would confirm the existence of NULL values, and > >> you'd need to invest which of the values in your CVS file have been > >> interpreted as NULL and why. > >> > >> In case the result of queries (2) and (3) do not add up to the number > >> returned by query (1), you might want to check the output(s) of > >> > >> TRACE select count(*) from<your_table>; > >> TRACE select count(single_column) from<your_table>; > >> TRACE select count(*) from<your_table> where single_column is null; > >> > >> (or have us check them, provided/once we find the time). > >> > >> Stefan > >> > >> On Sat, Mar 24, 2012 at 09:37:38AM -0600, Joseph Brower wrote: > >>> There shouldn't be any nulls. The records were all added by a "Copy > >>> Into" command. The files that we imported had no nulls in it at > >>> all. > >>> > >>> On 03/24/2012 01:57 AM, Sjoerd Mullender wrote: > >>>> On 2012-03-24 00:14, Joseph Brower wrote: > >>>>> We were troubleshooting some odd results coming back from our database, > >>>>> and found that it's missing some entries from a column. This appears to > >>>>> be causing some sort of offset. Does anyone have any ideas on what can > >>>>> be done to resolve it or what may have caused it? Is there any > >>>>> information we could send that would be helpful in troubleshooting this > >>>>> or for a bug report? > >>>>> > >>>>> If we do a count(*) and then a count('single_column') we are getting two > >>>>> different results (when they should be indentical) > >>>> count (*) counts all rows, whereas count(single_column) only count > >>>> non-null values. Are there any nulls in the single_column? > >>>> > >>>> > >>>> > >>>> > >>>> ------------------------------------------------------------------------------ > >>>> This SF email is sponsosred by: > >>>> Try Windows Azure free for 90 days Click Here > >>>> http://p.sf.net/sfu/sfd2d-msazure > >>>> > >>>> > >>>> _______________________________________________ > >>>> MonetDB-users mailing list > >>>> MonetDB-users(a)lists.sourceforge.net > >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users > >>> ------------------------------------------------------------------------------ > >>> This SF email is sponsosred by: > >>> Try Windows Azure free for 90 days Click Here > >>> http://p.sf.net/sfu/sfd2d-msazure > >>> _______________________________________________ > >>> MonetDB-users mailing list > >>> MonetDB-users(a)lists.sourceforge.net > >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users > >> > > > > > > ------------------------------------------------------------------------------ > > This SF email is sponsosred by: > > Try Windows Azure free for 90 days Click Here > > http://p.sf.net/sfu/sfd2d-msazure > > _______________________________________________ > > MonetDB-users mailing list > > MonetDB-users(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-users > > ------------------------------------------------------------------------------ > This SF email is sponsosred by: > Try Windows Azure free for 90 days Click Here > http://p.sf.net/sfu/sfd2d-msazure > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users

2 3

[MonetDB-users] DB Corruption
by Joseph Brower 26 Mar '12

26 Mar '12

We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report? If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical) Thanks, Joseph Brower

5 6

[MonetDB-users] Example of GUI on top of MonetDB
by Ryad Ben-El-Kezadri 22 Mar '12

22 Mar '12

Hello, We've updated our Java GUI for analyzing network traces on top of MonetDB A HD video trailer is available here : http://www.youtube.com/watch?v=P8EEhttP-NU GPL Source is available here: https://sites.google.com/site/yavista/ Unfortunately it only works on top of MonetDB Xquery but we hope to port it on MonetDB SQL Regards Ryad

1 0

[MonetDB-users] Failure reading in negative bigint
by Nikhil Padmanabhan 22 Mar '12

22 Mar '12

Hello, I have recently started trying out monetdb and have been very impressed by its speed so far. However, I am hitting a failure trying to read in a large negative number (bitmask) into a bigint field; sql>\<test.sql operation successful value '-9223372036854775808' from line 1 field 17 not inserted, expecting type bigint failed to import table I've attached the sql commands to create the table and a test file to this message. As far as I can tell, this should be fine for being represented by a 64 bit integer.... I am running this on an OS X 10.6 64-bit system, this is the Dec-2011 SP2 version of the 64 bit binaries dowloaded from www.monetdb.org. Any help is much appreciated. Thanks in advance, -- Nikhil --------------------------------- Nikhil Padmanabhan nikhil.padmanabhan(a)yale.edu

2 1

[MonetDB-users] Problem with installation from sources
by Stathis Gkotsis 17 Mar '12

17 Mar '12

Hi all, trying to install the latest version of monetdb from sources, I just did: * ./configure --enable-debug --enable-monetdb5 --enable-sql * make and the make install I created a database and now that I try to connect to it, in the log I get: 2012-03-16 17:33:44 MSG merovingian[1729]: starting a proxy failed: cannot get address for hostname '10-04-64bit': Unknown host 2012-03-16 17:33:44 ERR merovingian[1729]: client error: cannot get address for hostname '10-04-64bit': Unknown host I tried defining 10-04-64bit in /etc/hosts as localhost / IP of the machine, but then I get: "Too many redirects" error. Any ideas on how to solve this? BR, Stathis

2 1