- users-list - monetdb.org

QueryTimeout Question
by Sebastián Acevedo 09 Feb '17

09 Feb '17

Hi, there is any way to chnage the default querytimeout for a user?... i need to use this to make a restriction for long querys, if i run call sys.settimeout(1) this change is only apllied to the current session and if i login again with the same user the timeout is not longer applied. +---------+-----------------+------+------------------+------+-------+ | user | login | sess | lastcommand | quer | activ | : : : iont : : ytim : e … [View More]

3 4

Python Aggregate functions
by Anderson, David B 06 Feb '17

06 Feb '17

Mark, I think there is a bug in the implementation of the PYTHON_MAP aggregate functions. The return value of a PYTHON_MAP aggregate function can change if there are other functions in the query. The return value of my function weighted_percentile_0 (definition below) changes if I include the function median() in my query. The results are incorrect if median() isn't in the query, but are correct if median() is in the query. In this test case, the weights (v2) are all 1 and the values (v1) … [View More]are uniformly sampled integers from [1,10]. Number of rows for each fctr is ~1000, and my function should reduce to min(v1) in this case. Is some information bleeding between threads? Should I submit a bug report? This is on the default branch. Thanks, Dave sql>select fctr,weighted_percentile_0(v1,v2),min(v1) from mini where fctr < 10 group by fctr order by fctr; +------+--------------------------+------+ | fctr | L1 | L2 | +======+==========================+======+ | 1 | 3 | 1 | | 2 | 2 | 1 | | 3 | 2 | 1 | | 4 | 1 | 1 | | 5 | 3 | 1 | | 6 | 3 | 1 | | 7 | 2 | 1 | | 8 | 1 | 1 | | 9 | 2 | 1 | +------+--------------------------+------+ 9 tuples (126.928ms) sql>select fctr,weighted_percentile_0(v1,v2),min(v1),median(v1) from mini where fctr < 10 group by fctr order by fctr; +------+--------------------------+------+------+ | fctr | L1 | L2 | L3 | +======+==========================+======+======+ | 1 | 1 | 1 | 5 | | 2 | 1 | 1 | 5 | | 3 | 1 | 1 | 5 | | 4 | 1 | 1 | 5 | | 5 | 1 | 1 | 5 | | 6 | 1 | 1 | 5 | | 7 | 1 | 1 | 5 | | 8 | 1 | 1 | 5 | | 9 | 1 | 1 | 5 | +------+--------------------------+------+------+ 9 tuples (519.195ms) sql> CREATE AGGREGATE weighted_percentile_0(a DOUBLE, w DOUBLE) RETURNS DOUBLE LANGUAGE PYTHON_MAP { import numpy as np # Standardize and sort based on values in a q = np.array([0]) / 100.0 idx = np.argsort(a) a_sort = a[idx] w_sort = w[idx] # Get the cumulative sum of weights ecdf = np.cumsum(w_sort) # Find the percentile index positions associated with the percentiles p = q * (w_sort.sum() - 1) # Find the bounding indices (both low and high) idx_low = np.searchsorted(ecdf, p, side='right') idx_high = np.searchsorted(ecdf, p + 1, side='right') idx_high[idx_high > ecdf.size - 1] = ecdf.size - 1 # Calculate the weights weights_high = p - np.floor(p) weights_low = 1.0 - weights_high # Extract the low/high indexes and multiply by the corresponding weights x1 = np.take(a_sort, idx_low) * weights_low x2 = np.take(a_sort, idx_high) * weights_high wp = np.add(x1,x2) return(wp[0]) }; [View Less]

3 4

Virtual Column
by imad hajj chahine 01 Feb '17

01 Feb '17

Hi, How far is the feature of supporting virtual column, with the possibility to persist it as a physical column, from the future release of monetdb? This feature is supported in mysql starting the 5.7 version. https://dev.mysql.com/doc/refman/5.7/en/create-table-generated-columns.html Thank you.

1 0

SQL grammar
by imad hajj chahine 01 Feb '17

01 Feb '17

Hi, I am interested to know how actually monetdb parse an SQL expression and generate the expression tree, are you using lex/yacc similar grammar tool. I am currently using antlr for some basic parsing tasks, but my objective is to be able to generate a complete expression tree from an SQL expression. Thank you.

1 0

Installing Monetdb in on Ubuntu 16.04.1 LTS via repo
by Yan Grange 01 Feb '17

01 Feb '17

Hi there, I'm trying to install monetdb via the instructions on the website inside a Docker container derived from the Ubuntu 16.04.1 LTS image. I get the following error message: root@ContainerHost:/local# sudo apt-get update Ign:1 http://dev.monetdb.org/downloads/deb xenial InRelease Get:2 http://dev.monetdb.org/downloads/deb xenial Release [3878 B] Get:3 http://dev.monetdb.org/downloads/deb xenial Release.gpg [163 B] Hit:4 http://archive.ubuntu.com/ubuntu xenial InRelease Hit:5 http://… [View More]

2 1

problem stopping/starting monetdbd
by Lynn Carol Johnson 30 Jan '17

30 Jan '17

Hi all - Yesterday our monetdb instance seemed to be running slowing, perhaps due to user queries that had been killed but maybe not cleaned up. We decided to stop the dbs, then start/stop monetdbd. I executed the following commands, but was unable to restart the daemon: sudo monetdbd stop /local/monetdb/dbfarmBL sudo monetdbd start /local/monetdb/dbfarmBL We had our admin reboot the machine, thinking this would clear up everything. This morning I tried again to start monetdbd. The … [View More]command comes back with no messages, but I get a connection refused message when trying to access a database: could not connect to localhost:50000: Connection refused Get all shows this: [lcj34@cbsudc01 anp68_ije_Locus9Combined_Result]$ monetdbd get all /local/monetdb/dbfarmBL property value hostname cbsudc01 dbfarm /local/monetdb/dbfarmBL status no monetdbd is serving this dbfarm mserver unknown (monetdbd not running) logfile /local/monetdb/dbfarmBL/merovingian.log pidfile /local/monetdb/dbfarmBL/merovingian.pid sockdir /tmp listenaddr 128.84.3.206 port 50000 exittimeout 60 forward proxy discovery yes discoveryttl 600 control no passphrase <unknown> mapisock /tmp/.s.monetdb.50000 controlsock /tmp/.s.merovingian.50000 [lcj34@cbsudc01 anp68_ije_Locus9Combined_Result]$ The log shows the following message: 2017-01-27 07:53:26 MSG maizeFullGenomeDB[14286]: # Listening for UNIX domain connection requests on mapi:monetdb:///local/monetdb/dbfarmBL/maizeFullGenomeDB/.mapi.sock 2017-01-27 07:53:26 MSG maizeFullGenomeDB[14286]: # MonetDB/SQL module loaded 2017-01-27 07:53:29 ERR merovingian[13705]: client error: cannot connect: Connection refused 2017-01-27 07:55:41 MSG merovingian[13705]: caught SIGTERM, starting shutdown sequence 2017-01-27 07:55:50 MSG merovingian[15395]: Merovingian 1.7 (Jun2016-SP2) starting 2017-01-27 07:55:50 MSG merovingian[15395]: monitoring dbfarm /local/monetdb/dbfarmBL 2017-01-27 07:55:50 MSG merovingian[15395]: accepting connections on TCP socket cbsudc01:50000 2017-01-27 07:55:50 MSG merovingian[15395]: accepting connections on UNIX domain socket /tmp/.s.monetdb.50000 2017-01-27 07:55:50 ERR merovingian[15395]: binding to datagram socket port 50000 failed: no available address 2017-01-27 07:55:50 MSG merovingian[15395]: Merovingian 1.7 stopped 2017-01-27 07:55:50 ERR merovingian[15395]: fatal startup condition encountered, aborting startup Any ideas what to try? Thanks - Lynn [View Less]

3 7

Re: running make command gives error on installation
by Pham Duc 29 Jan '17

29 Jan '17

Hi Miguel, I guess that you tried MonetDB 11.2.0 as it is mentioned in the "Heuristics-based Query Optimisation for SPARQL" paper (https://openproceedings.org/2012/conf/edbt/TsialiamanisSFCB12.pdf) Did you check the instruction provided by the ICS-FORTH researchers ( http://www.ics.forth.gr/isl/index_main.php?l=e&c=646 ) for installing MonetDB? If not, please follow that instruction. If it still does not work, you will need to contact ICS-FORTH researchers (please see the contact at … [View More]the end of the instruction) as that RDF version of MonetDB 11.2.0 was specifically developed for their research with the support of our group members (e.g., Lefteris). Best regards, Minh Duc ----- Original Message ----- From: "Martin Kersten" <martin.kersten(a)cwi.nl> To: "Communication channel for MonetDB users" <users-list(a)monetdb.org>, "P Boncz" <P.Boncz(a)cwi.nl>, "Pham Minh Duc" <P.Minh.Duc(a)cwi.nl> Sent: Sunday, January 29, 2017 8:41:12 AM Subject: Re: running make command gives error on installation Hi Indeed, this is a branch used by Pham Duc during his PhD research. The code base has not been kept in sync with the main branches and you should contact him/his advisor for further support. regards, Martin On 28/01/2017 21:59, Miguel Mármol wrote: > Hi, > I am trying to install MonetDB5 11.2.0 (The version of MonetDB is not an official MonetDB release. This version includes an RDF frontend that is used to parse RDF datasets and load them in MonetDB.). > > I get the following error when I run the make command: > > > miguel@miguel-VirtualBox:~/Downloads/MonetDB$ sudo make > make all-recursive > make[1]: Entering directory '/home/miguel/Downloads/MonetDB' > Making all in buildtools > make[2]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools' > make all-recursive > make[3]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools' > Making all in Mx > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' > make[5]: Nothing to be done for 'all-am'. > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' > Making all in conf > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/conf' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/conf' > make[5]: Nothing to be done for 'all-am'. > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/conf' > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/conf' > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools' > make[4]: Nothing to be done for 'all-am'. > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools' > make[3]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools' > make[2]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools' > Making all in common > make[2]: Entering directory '/home/miguel/Downloads/MonetDB/common' > make all-recursive > make[3]: Entering directory '/home/miguel/Downloads/MonetDB/common' > Making all in stream > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common/stream' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/common/stream' > make[5]: Nothing to be done for 'all-am'. > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/common/stream' > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common/stream' > Making all in options > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common/options' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/common/options' > make[5]: Nothing to be done for 'all-am'. > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/common/options' > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common/options' > Making all in utils > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common/utils' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/common/utils' > make[5]: Nothing to be done for 'all-am'. > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/common/utils' > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common/utils' > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common' > make[4]: Nothing to be done for 'all-am'. > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common' > make[3]: Leaving directory '/home/miguel/Downloads/MonetDB/common' > make[2]: Leaving directory '/home/miguel/Downloads/MonetDB/common' > Making all in clients > make[2]: Entering directory '/home/miguel/Downloads/MonetDB/clients' > make all-recursive > make[3]: Entering directory '/home/miguel/Downloads/MonetDB/clients' > Making all in mapilib > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapilib' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapilib' > make[5]: Nothing to be done for 'all-am'. > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapilib' > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapilib' > Making all in mapiclient > make[4]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' > make all-am > make[5]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' > /bin/bash ../../libtool --tag=CC --mode=link gcc -Wall -Wextra -g -pedantic -Wno-long-long -Wno-declaration-after-statement -D_REENTRANT -export-dynamic -o mclient mclient.o ReadlineTools.o libmcutil.la ../mapilib/libmapi.la > ../../common/stream/libstream.la -lcrypt -pthread > libtool: link: gcc -Wall -Wextra -g -pedantic -Wno-long-long -Wno-declaration-after-statement -D_REENTRANT -o .libs/mclient mclient.o ReadlineTools.o -pthread -Wl,--export-dynamic ./.libs/libmcutil.a ../mapilib/.libs/libmapi.so > ../../common/stream/.libs/libstream.so -lcrypt -pthread -Wl,-rpath -Wl,/usr/local/monet/lib > ../mapilib/.libs/libmapi.so: undefined reference to `SHA256' > ../mapilib/.libs/libmapi.so: undefined reference to `SHA384' > ../mapilib/.libs/libmapi.so: undefined reference to `RIPEMD160' > ../mapilib/.libs/libmapi.so: undefined reference to `MD5' > ../mapilib/.libs/libmapi.so: undefined reference to `SHA1' > ../mapilib/.libs/libmapi.so: undefined reference to `SHA512' > ../mapilib/.libs/libmapi.so: undefined reference to `SHA224' > collect2: error: ld returned 1 exit status > Makefile:666: recipe for target 'mclient' failed > make[5]: *** [mclient] Error 1 > make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' > Makefile:568: recipe for target 'all' failed > make[4]: *** [all] Error 2 > make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' > Makefile:764: recipe for target 'all-recursive' failed > make[3]: *** [all-recursive] Error 1 > make[3]: Leaving directory '/home/miguel/Downloads/MonetDB/clients' > Makefile:709: recipe for target 'all' failed > make[2]: *** [all] Error 2 > make[2]: Leaving directory '/home/miguel/Downloads/MonetDB/clients' > Makefile:600: recipe for target 'all-recursive' failed > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory '/home/miguel/Downloads/MonetDB' > Makefile:530: recipe for target 'all' failed > make: *** [all] Error 2 > > Please any help will be really appreciated, thanks. > > Miguel > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list > [View Less]

1 0

running make command gives error on installation
by Miguel Mármol 29 Jan '17

29 Jan '17

Hi, I am trying to install MonetDB5 11.2.0 (The version of MonetDB is not an official MonetDB release. This version includes an RDF frontend that is used to parse RDF datasets and load them in MonetDB.). I get the following error when I run the make command: miguel@miguel-VirtualBox:~/Downloads/MonetDB$ sudo make make all-recursive make[1]: Entering directory '/home/miguel/Downloads/MonetDB' Making all in buildtools make[2]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools' … [View More]make all-recursive make[3]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools' Making all in Mx make[4]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' make[5]: Nothing to be done for 'all-am'. make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/Mx' Making all in conf make[4]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/conf' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools/conf' make[5]: Nothing to be done for 'all-am'. make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/conf' make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools/conf' make[4]: Entering directory '/home/miguel/Downloads/MonetDB/buildtools' make[4]: Nothing to be done for 'all-am'. make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools' make[3]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools' make[2]: Leaving directory '/home/miguel/Downloads/MonetDB/buildtools' Making all in common make[2]: Entering directory '/home/miguel/Downloads/MonetDB/common' make all-recursive make[3]: Entering directory '/home/miguel/Downloads/MonetDB/common' Making all in stream make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common/stream' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/common/stream' make[5]: Nothing to be done for 'all-am'. make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/common/stream' make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common/stream' Making all in options make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common/options' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/common/options' make[5]: Nothing to be done for 'all-am'. make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/common/options' make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common/options' Making all in utils make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common/utils' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/common/utils' make[5]: Nothing to be done for 'all-am'. make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/common/utils' make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common/utils' make[4]: Entering directory '/home/miguel/Downloads/MonetDB/common' make[4]: Nothing to be done for 'all-am'. make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/common' make[3]: Leaving directory '/home/miguel/Downloads/MonetDB/common' make[2]: Leaving directory '/home/miguel/Downloads/MonetDB/common' Making all in clients make[2]: Entering directory '/home/miguel/Downloads/MonetDB/clients' make all-recursive make[3]: Entering directory '/home/miguel/Downloads/MonetDB/clients' Making all in mapilib make[4]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapilib' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapilib' make[5]: Nothing to be done for 'all-am'. make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapilib' make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapilib' Making all in mapiclient make[4]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' make all-am make[5]: Entering directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' /bin/bash ../../libtool --tag=CC --mode=link gcc -Wall -Wextra -g -pedantic -Wno-long-long -Wno-declaration-after-statement -D_REENTRANT -export-dynamic -o mclient mclient.o ReadlineTools.o libmcutil.la ../mapilib/libmapi.la ../../common/stream/libstream.la -lcrypt -pthread libtool: link: gcc -Wall -Wextra -g -pedantic -Wno-long-long -Wno-declaration-after-statement -D_REENTRANT -o .libs/mclient mclient.o ReadlineTools.o -pthread -Wl,--export-dynamic ./.libs/libmcutil.a ../mapilib/.libs/libmapi.so ../../common/stream/.libs/libstream.so -lcrypt -pthread -Wl,-rpath -Wl,/usr/local/monet/lib ../mapilib/.libs/libmapi.so: undefined reference to `SHA256' ../mapilib/.libs/libmapi.so: undefined reference to `SHA384' ../mapilib/.libs/libmapi.so: undefined reference to `RIPEMD160' ../mapilib/.libs/libmapi.so: undefined reference to `MD5' ../mapilib/.libs/libmapi.so: undefined reference to `SHA1' ../mapilib/.libs/libmapi.so: undefined reference to `SHA512' ../mapilib/.libs/libmapi.so: undefined reference to `SHA224' collect2: error: ld returned 1 exit status Makefile:666: recipe for target 'mclient' failed make[5]: *** [mclient] Error 1 make[5]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' Makefile:568: recipe for target 'all' failed make[4]: *** [all] Error 2 make[4]: Leaving directory '/home/miguel/Downloads/MonetDB/clients/mapiclient' Makefile:764: recipe for target 'all-recursive' failed make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory '/home/miguel/Downloads/MonetDB/clients' Makefile:709: recipe for target 'all' failed make[2]: *** [all] Error 2 make[2]: Leaving directory '/home/miguel/Downloads/MonetDB/clients' Makefile:600: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/home/miguel/Downloads/MonetDB' Makefile:530: recipe for target 'all' failed make: *** [all] Error 2 Please any help will be really appreciated, thanks. Miguel [View Less]

2 1

apt having trouble installing deb packages
by Clayton A Davis 27 Jan '17

27 Jan '17

Ubuntu seems to be having an issue with the deb packages. On 16.04, it complains about the "Valid-Until" entry in the Release file, and on 16.10 it complains about the "Date" entry and the security. Here are the error messages from 16.04: W: http://dev.monetdb.org/downloads/deb/dists/xenial/Release.gpg: Signature by key 213E64DCD5DDC2C063CC39FD053C3ED40583366F uses weak digest algorithm (SHA1) W: Invalid 'Date' entry in Release file /var/lib/apt/lists/dev.monetdb.… [View More]

1 0

Announcement: New Dec2016-SP1 Bugfix release of MonetDB suite
by Panagiotis Koutsourakis 26 Jan '17

26 Jan '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2016-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2016-SP1 bugfix release Bug Fixes * 4039: Slow mserver5 start after … [View More]

1 0