- users-list - monetdb.org

COPY INTO from LZ4 file: bug or missing feature?
by Ying Zhang 04 Sep '17

04 Sep '17

Dear all, Is COPY INTO from an LZ4 file supported by MonetDB? When I try to load data from an LZ4 file, I always get the error: Failed to import table 'pt', record too long (EOS found) Is this an error or a missing feature? Additional info: - MonetDB version Jul2017 (Mercurial source) - Mac OS (Sierra) - I checked the config.log that my lz4 lib was found - The LZ4 file is valid (I decompressed it, then I could load the data). Thanks in advance! Jennie

1 1

Execute MAL queries
by Dennis Fassl 04 Sep '17

04 Sep '17

Dear all, I am trying to write and execute MAL queries. The overall goal is to translate Cypher to MAL and to use Cypher as an input language for MonetDB. I am using the "explain" statement to get some example MAL queries. This is an output of explain (without the optimizer stuff at the end): function user.s4_1():void; X_0:void := querylog.define("explain select source, target from graph where source = \\'A\\';","default_pipe",14:int); X_30 := bat.new(nil:str); X_36 := bat.new(nil:int); X_34 := bat.new(nil:int); X_33 := bat.new(nil:str); X_32 := bat.new(nil:str); X_3 := sql.mvc(); X_7:bat[:str] := sql.bind(X_3,"sys","graph","source",0:int); C_4:bat[:oid] := sql.tid(X_3,"sys","graph"); C_18 := algebra.thetasubselect(X_7,C_4,"A","=="); X_21:bat[:str] := sql.bind(X_3,"sys","graph","target",0:int); X_28 := algebra.projection(C_18,X_21); X_20 := algebra.projection(C_18,X_7); X_37 := bat.append(X_30,"sys.graph"); X_39 := bat.append(X_32,"source"); X_40 := bat.append(X_33,"varchar"); X_42 := bat.append(X_34,31:int); X_44 := bat.append(X_36,0:int); X_46 := bat.append(X_37,"sys.graph"); X_47 := bat.append(X_39,"target"); X_48 := bat.append(X_40,"varchar"); X_49 := bat.append(X_42,31:int); X_50 := bat.append(X_44,0:int); sql.resultSet(X_46,X_47,X_48,X_49,X_50,X_20,X_28); end user.s4_1; But when I try to execute this query I get errors like that: ERROR = !TypeException:user.main[8]:'sql.tid' undefined in: C_4:bat[:oid] := sql.tid("sys":str,"sys":str,"graph":str); I don't know why the "X_3" is replaced by "sys". I see changes like that in the queries regularly. Does someone know what is wrong or missing in this query? Thanks a lot in advance! Best regards, Dennis

4 5

Possible bug with table generating functions
by Mark Clements 29 Aug '17

29 Aug '17

Is there a restriction on the use of the same table generating function in the same select statement? This relates to the following possible bug: -- reimplement generate_series for MonetDBLite CREATE FUNCTION my_generate_series(start INT, finish INT) RETURNS TABLE (value INT) BEGIN DECLARE TABLE tmp_generate_series(value INT); DECLARE i INT; SET i = start; WHILE (i <= finish) DO INSERT INTO tmp_generate_series VALUES(i); SET i = i + 1; END WHILE; RETURN tmp_generate_series; END; --- This seems to work quite nicely: select * from my_generate_series(1,10); -- and it's fast: select count(*) from my_generate_series(1,100000); -- however, I get odd and non-deterministic counts for the following: select count(*) from my_generate_series(1,100) as t1, my_generate_series(1,100) as t2; This is using MonetDB v11.23.3 (Jun2016) that was compiled with embedded R, but not run with --set embedded_r=true. I got similar errors with MonetDBLite. Kindly, Mark.

2 2

Looking for Admin UI / Tools / Resources
by Cota, Luis 28 Aug '17

28 Aug '17

Can anyone recommend any tools (commercial or open source) which assist in administering a MonetDB instance? Things I'd like to do are: Create tables from existing data files (CSV, TXT, HDF5) Import or query Oracle databases and either create tables, or populate tables, with the results of those queries Query existing monetdb instances Thanks, Luis ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

1 0

Error after updating monetdb
by Mahmoud Mohsen 19 Aug '17

19 Aug '17

Hello, I have updated monetdb from Dec2016-SP5 to version v11.27.5 "Jul2017-SP1, but after this when trying to accessing the database via mclient -u monetdb -d tpch2 I am getting this error: internal error while starting mserver 'database 'tpch2' started up, but failed to open up a communication channel', please refer to the logs looking at the logs, 2017-08-18 22:42:02 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:02 MSG merovingian[10645]: database 'voc' (10660) has exited with exit status 0 2017-08-18 22:42:02 MSG merovingian[10645]: starting database 'voc', up min/avg/max: 0s/0s/0s, crash average: 0.00 0.00 0.00 (1-1=0) 2017-08-18 22:42:02 MSG merovingian[10645]: sending process 10663 (database 'voc') the TERM signal 2017-08-18 22:42:02 MSG voc[10663]: arguments: /usr/bin/mserver5 --dbpath=/home/mohsen/mfarm/voc --set merovingian_uri=mapi:monetdb://u145-16:50000/voc --set mapi_open=false --set mapi_port=0 --set mapi_usock=/home/mohsen/mfarm/voc/.mapi.sock --set monet_vault_key=/home/mohsen/mfarm/voc/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2017-08-18 22:42:02 MSG voc[10663]: # MonetDB 5 server v11.27.5 "Jul2017-SP1" 2017-08-18 22:42:02 MSG voc[10663]: # Serving database 'voc', using 8 threads 2017-08-18 22:42:02 MSG voc[10663]: # Compiled for x86_64-pc-linux-gnu/64bit with 128bit integers 2017-08-18 22:42:02 MSG voc[10663]: # Found 251.889 GiB available main-memory. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) 1993-July 2008 CWI. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved 2017-08-18 22:42:02 MSG voc[10663]: # Visit https://www.monetdb.org/ for further information 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:^LoaderException:loadLibrary:Loading error failed to open library sql (from within file '/usr/lib/monetdb5/lib_sql.so'): /usr/lib/monetdb5/lib_sql.so: undefined symbol: mnstr_writeFlt 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError: ^';' expected 2017-08-18 22:42:03 MSG merovingian[10645]: database 'voc' has shut down 2017-08-18 22:42:03 ERR control[10645]: !monetdbd: an internal error has occurred 'database 'voc' started up, but failed to open up a communication channel' 2017-08-18 22:42:07 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:07 MSG merovingian[10645]: database 'voc' (10663) has exited with exit status 0 Thank you a lot in advance, Best regards, Mahmoud Mohsen

2 3

Re: Re: failed to import record too long: Loading 1.5 Terabyte table
by Stefan Manegold 18 Aug '17

18 Aug '17

You need to use the correct record delimiter i.e. newline instead of space. -------- Original message --------From: Mahmoud Mohsen <mmohsen35(a)gmail.com> Date: 8/18/17 19:40 (GMT+01:00) To: users-list(a)monetdb.org Subject: Re: Re: failed to import record too long: Loading 1.5 Terabyte table Dear Stefan, Thank you very much for your quick reply, so this is the first line of the file "lineitem.tbl", so no need to use an offset I think 1|310378689|15378705|1|17|29786.89|0.04|0.02|N|O|1996-03-13|1996-02-12|1996-03-22|DELIVER IN PERSON|TRUCK|egular courts above the| as for the version, I am using: MonetDB Database Server Toolkit v1.1 (Dec2016-SP5). So can this may be the cause of the error... an old version? I am looking forward hearing from you, Best regards, Mahmoud Mohsen

2 2

failed to import record too long: Loading 1.5 Terabyte table
by Mahmoud Mohsen 18 Aug '17

18 Aug '17

Hello, I'm trying to load a table with 1.5 Terabytes of data, the command I'm using COPY INTO lineitem FROM '/vol1/tpch-dbgen/2000/lineitem.tbl' USING DELIMITERS '|', ' ' LOCKED; The physical memory I'm using is 256G, the swap is 2.2 terabytes, the dbfarm is located within area has 4 Terabytes of free storage. The message I'm getting: "Failed to import table 'lineitem', record too long" So what could be a possible solution to this problem, Thank you a lot in advance, Mahmoud Mohsen

3 3

(lastest version) monetdb source compilation and installation problem
by 张馨匀 17 Aug '17

17 Aug '17

Hi All, Could you please help me to solve the following error? When I try to install monetdb I have the following error: * fatal error: *geos_c.h: No such file or directory #include <geos_c.h> *My steps used to compile monetdb src and to install monetdb are:* a. Environment setup: $ export MonetDB_rel=July2017 $ export MonetDB_ver=11.27.5 $ export MonetDB_base_dir=$HOME/MonetDB-$MonetDB_rel $ export MonetDB_source_dir=$MonetDB_base_dir/MonetDB-$MonetDB_ver $ export MonetDB_build_dir=$MonetDB_base_dir/BUILD $ export MonetDB_prefix_dir=$MonetDB_base_dir $ mkdir $MonetDB_base_dir $ cd $MonetDB_base_dir b. Download the latest version: $ wget https://dev.monetdb.org/downloads/sources/Jul2017-SP1/MonetDB-11.27.5.tar.xz $ tar xf MonetDB-$MonetDB_ver.tar.xz c. Build: $ mkdir $MonetDB_build_dir $ cd $MonetDB_build_dir $ MonetDB_source_dir/configure --prefix=$MonetDB_prefix_dir d. Install: $ make -j 20 (error mentioned above occurs here) $ make -j 20 install Thanks,

2 1

Memory bloat on concurrency - Any fix to release memory post query run
by Sharma, Sreejith 11 Aug '17

11 Aug '17

Hi, I have been using monetdb (11.25.3) in CentOS for one of application. I have been working on a PSR testing for this application and have noticed on high concurrency and on same pattern of query running again and again causing memory to bloat up and doesn’t release once the query run completes. I understand that the data is lifted into memory and the result set is also cached, however I assume this should be release once the system is idle which I don’t see it as of now. Is there any way or parameter to release the unused memory blocks? I have asked this before, but couldn't get to the root to fix this. Regards, Sreejith

3 4

Troubles with Python API
by 倪令斌 09 Aug '17

09 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

2 3

Troubles with Python API
by 倪令斌 08 Aug '17

08 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

1 0

Mdiff outputs unexpected result
by Zhouyang Jia 05 Aug '17

05 Aug '17

Hi, I'm new to MonetDB, and I find that the component Mdiff would generate unexpected results if inputted wrong files, e.g., ./Mdiff wrong_file_1 wrong_file1. Mdiff will output that these two files are equal, whereas the expected result shoud be "files not exist". Attached please find the patch against version MonetDB-11.27.5. Hopefully, it can solve this problem. Best, Zhouyang

1 0

Announcement: New Jul2017-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 04 Aug '17

04 Aug '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP1 bugfix release Build Environment * The Debian and Ubuntu installers have been fixed: there was a file missing in the Jul2017 release. * Added a new RPM called MonetDB-selinux which provides the SELinux policy required to run MonetDB under systemd, especially on Fedora 26. * The Windows installers (*.msi files) are now created using the WiX Toolset. * The Windows binaries are now built using Visual Studio 2015. Because of this, you may need to install the Visual C++ Redistributable for Visual Studio 2015 before being able to run MonetDB. Merovingian * monetdbd was leaking open file descriptors to the mserver5 process it started. This has been fixed. MonetDB Common * Many functions in GDK are now annotated with the GCC attribute __warn_unused_result__ meaning that the compiler will issue a warning if the result of the function (usually an indication of an error) is not used. Bug Fixes * 6325: Merge table unusable in other connections * 6328: Transactional/multi-connection issues with merge tables * 6336: VALUES multiple inserts error * 6339: Mserver5 crashes on nested SELECT * 6340: sample operator takes effect after the execution of the query, expected before * 6341: MERGE TABLE issue: Cannot register * 6342: MERGE TABLE issue: hang * 6344: Spurious errors and assertions (SQLsmith) * 6375: MAL profiler truncates JSON objects larger than 8192 characters

3 2

Error during installation (make)
by Dennis Fassl 31 Jul '17

31 Jul '17

Hi, I want to install MonetDB from the source files. (Version 11.27.1). I am following the guide "HowToStart.rst" in the source directory. But I get an error when executing make. Please find the output below. Do you have a hint for me how to solve this error? Many thanks and kind regards, Dennis make all-recursive make[1]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir' Making all in buildtools make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' Making all in conf make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' Making all in common make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common' Making all in stream make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' Making all in options make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' Making all in utils make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common' Making all in clients make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' Making all in mapilib make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' Making all in mapiclient make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' Making all in examples make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' Making all in C make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' Making all in php make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' Making all in perl make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' Making all in gdk make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' make all-am make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' Making all in monetdb5 make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' Making all in mal make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' Making all in modules make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' Making all in atoms make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' Making all in kernel make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' Making all in mal make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' Making all in optimizer make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' Making all in scheduler make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' Making all in tools make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' Making all in extras make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' Making all in mal_optimizer_template make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' Making all in rapi make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' Making all in sql make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' Making all in include make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' Making all in common make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' Making all in storage make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' Making all in bat make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' Making all in server make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' Making all in backends make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' Making all in monet5 make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make all-recursive make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' Making all in vaults make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make all-recursive make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[10]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[10]: Nothing to be done for `all-am'. make[10]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' Making all in UDF make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make all-recursive make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' Making all in cudf make[10]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make all-am make[11]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make[11]: Nothing to be done for `all-am'. make[11]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make[10]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make[10]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make[10]: Nothing to be done for `all-am'. make[10]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' Making all in LSST make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' make all-am make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' make[9]: Nothing to be done for `all-am'. make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' Making all in generator make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make all-am make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make[9]: Nothing to be done for `all-am'. make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[8]: Nothing to be done for `all-am'. make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' Making all in scripts make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' Making all in tools make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' Making all in mserver make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' Making all in merovingian make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' Making all in utils make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' Making all in client make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' Making all in daemon make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' /bin/bash ../../../libtool --tag=CC --mode=link gcc -g -O2 -export-dynamic -o monetdbd merovingian.o argvcmds.o client.o connections.o controlrunner.o discoveryrunner.o forkmserver.o handlers.o multiplex-funnel.o proxy.o ../utils/libmeroutil.la ../../../common/utils/libmutils.la ../../../monetdb5/tools/libmonetdb5.la -lssl -lcrypto -pthread libtool: link: gcc -g -O2 -o .libs/monetdbd merovingian.o argvcmds.o client.o connections.o controlrunner.o discoveryrunner.o forkmserver.o handlers.o multiplex-funnel.o proxy.o -pthread -Wl,--export-dynamic ../utils/.libs/libmeroutil.a ../../../common/utils/.libs/libmutils.a ../../../monetdb5/tools/.libs/libmonetdb5.so -lssl -lcrypto -pthread /usr/bin/ld: controlrunner.o: undefined reference to symbol 'GDKinit' //home/dennis/MonetDB-11.27.1/buildDir/gdk/.libs/libbat.so.15: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make[7]: *** [monetdbd] Error 1 make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' make[6]: *** [all] Error 2 make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' make[4]: *** [all] Error 2 make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir' make: *** [all] Error 2

2 1

C (or Java) API to insert blob via prep statement
by Anton Kravchenko 26 Jul '17

26 Jul '17

Is blob insertion (via prepared statement) supported by C (or Java) MonetDB API as of now?

1 0

COPY INTO doesn't recognise sorted varchar column
by Sridhar Eaturu 25 Jul '17

25 Jul '17

Hi, I imported some csv data using COPY INTO to a table. The data is sorted by the first column "sym", which is of VARCHAR type. Upon importing, the COPY INTO is not recognising the sortedness of the "sym" column. On the other hand, another test I ran with a sorted DATE column gets marked as sorted as expected. I'm using MonetDB v11.25.23 (Dec2016-SP5), and the steps demonstrating this are below. Is this expected? sql>CREATE TABLE tempquote (sym VARCHAR(3),time0 TIME,cond1 VARCHAR(1),bid INTEGER,bidqty INTEGER,ask INTEGER,askqty INTEGER,cond2 VARCHAR(1)); operation successful (2.430ms) sql>COPY INTO tempquote from 'tempquote.csv' USING DELIMITERS ',','\n' NULL AS ''; 5 affected rows (31.972ms) sql>select * from tempquote; +------+----------+-------+------+--------+------+--------+-------+ | sym | time0 | cond1 | bid | bidqty | ask | askqty | cond2 | +======+==========+=======+======+========+======+========+=======+ | ABY | 09:30:03 | F | 1489 | 50 | 1872 | 44 | I | | ACG | 09:30:00 | M | 1419 | 27 | 1542 | 43 | A | | AHK | 09:30:00 | G | 1094 | 74 | 1808 | 56 | F | | AXN | 09:30:01 | I | 1022 | 26 | 1541 | 88 | E | | BEV | 09:30:00 | L | 1377 | 38 | 1353 | 77 | M | +------+----------+-------+------+--------+------+--------+-------+ 5 tuples (1.131ms) sql>select table,column,sorted from storage() where "table"='tempquote'; +-----------+--------+--------+ | table | column | sorted | +===========+========+========+ | tempquote | sym | null | | tempquote | time0 | null | | tempquote | cond1 | null | | tempquote | bid | null | | tempquote | bidqty | null | | tempquote | ask | null | | tempquote | askqty | null | | tempquote | cond2 | null | +-----------+--------+--------+ 8 tuples (5.090ms) sql> Thanks, Sridhar

1 0

Is there any parameter to release memory? I see memory bloats on high concurrency and continuous runs
by Sharma, Sreejith 25 Jul '17

25 Jul '17

Hi, Is there any parameter or any options available for MonetDB to release the memory consumed? As one of the application I tested would run on high concurrency and may run SQL overnight. In such cases I see a memory bloat which may come up to 200gb and will never come down. Please advise? Regards, Sreejith

2 3

Troubles with the MonetDB
by 倪令斌 25 Jul '17

25 Jul '17

Dear Sir/Madam， I have encountered a problem that it has troubled me for several days and I can not continue the next job.I found it online for a long time, but I did not find a solution.Hope you can give me some inspiration: what's the matter with my PC? PS:The OS is Ubuntu16.04. Thanks a lot! Cheers, Lingbin

2 2

Copy into with DEFAULT values
by Ariel Abadi 24 Jul '17

24 Jul '17

Hi Im trying to upload a FILE into one table. The FILE has all the fields of the table, except for the last field which is the *updated *time. For that reason I have created the table with the field: "nrm_now" TIMESTAMP(0) DEFAULT current_timestamp() BUT when im trying to insert the FILE (COY INTO test_20170711 from FILE .... ), there is an error: Failed to import table 'test_20170711', Column value 143 missing Does anybody can help me? Am I doing something wrong? Which should be the best way to implement this? Thnks in advance Ariel

2 2

Monetdb Installation
by Amit Ambekar 24 Jul '17

24 Jul '17

Hi, July 2017 release is not working good . I am getting error while login using terminal in Ubuntu OS. When new release will be done ??

2 1

Bug on timestamp ?
by Ariel Abadi 21 Jul '17

21 Jul '17

Hi! May be someone can help me, Im facing the following problem. Im trying to pull some information from the field nrm_connect_time, I tried almost all the date/timestamp functions, but all of them failed. Finally, I decided to CAST as STRING, and see what happends... If I do a SELECT DISTINCT .. then I have the correct information, if I dont include the DISTINCT what I see has nonsense... sql>SELECT nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10) FROM NRM_20170609 LIMIT 5; +----------------------+-----------------------+ | nrm_connect_time | left_nrm_connect_time | +======================+=======================+ | 2017-06-09 00:05:00 | -1-01-01 0 | | 2017-06-09 00:05:00 | -1-01-01 0 | | 2017-06-09 00:05:01 | -1-01-01 0 | | 2017-06-09 00:05:03 | -1-01-01 0 | | 2017-06-09 00:05:03 | -1-01-01 0 | +----------------------+-----------------------+ 5 tuples (29.802ms) sql>*SELECT DISTINCT* nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10) FROM NRM_20170609 LIMIT 5; +----------------------+------------+ | nrm_connect_time | L2 | +======================+============+ | 2017-06-09 00:05:00 | 2017-06-09 | | 2017-06-09 00:05:01 | 2017-06-09 | | 2017-06-09 00:05:03 | 2017-06-09 | | 2017-06-09 00:05:04 | 2017-06-09 | | 2017-06-09 00:05:05 | 2017-06-09 | +----------------------+------------+ 5 tuples (424.306ms) *The "nrm_connect_time" is defined as TIMESTAMP(0).* Let me show you a differente example: sql>*SELECT DISTINCT *nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10), timestamp_to_str(nrm_connect_time, '%Y') FROM NRM_20170609 LIMIT 5; +----------------------+------------+------+ | nrm_connect_time | L2 | L3 | +======================+============+======+ | 2017-06-09 00:05:00 | 2017-06-09 | 2017 | | 2017-06-09 00:05:01 | 2017-06-09 | 2017 | | 2017-06-09 00:05:03 | 2017-06-09 | 2017 | | 2017-06-09 00:05:04 | 2017-06-09 | 2017 | | 2017-06-09 00:05:05 | 2017-06-09 | 2017 | +----------------------+------------+------+ 5 tuples (707.217ms) sql>*SELECT *nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10), timestamp_to_str(nrm_connect_time, '%Y') FROM NRM_20170609 LIMIT 5; TypeException:user.s16_55[123]:'mtime.timestamp_to_str' undefined in: mtime.timestamp_to_str(X_196:oid,A1:str); program contains errors Regards Ariel

3 3

Re: Re: Add entry in a table only if that entry does not already exist
by Stefano Piani 21 Jul '17

21 Jul '17

Hello, sorry for my late replay, I have been really busy in the last few days. > Hello Stefano, > > Which MonetDB version do you use? On which OS? Sorry, I forgot to mention this. I am using ubuntu 16.04 and I am running MonetDB inside a Docker container (which also runs ubuntu). >> On 4 Jul 2017, at 16:32, Stefano Piani <stefano.piani(a)exact-lab.it> wrote: >> >> Dear monetdb users, >> >> I started to use MonetDB a few weeks ago and, until now, I have been >> able to use it proficiently. Unfortunately, I have faced a small issue >> that I have not been able to figure out how to fix. >> >> I collect data generated by different users. A unique id is associated >> to each user. I have a specific table (called "user") where I save the >> name and some other information about the user. In this table, the ID >> is a primary key. >> When I collect the data, I want to check if the specific user who >> generated that data is already saved in the user table. If this is not >> the case, >> I want to add a row to the user table for this user. >> Unfortunately, I have not been able to accomplish this operation in >> one SQL query and, therefore, I have been forced to query for the user >> id and, >> if the answer is negative, add the user using another query. The >> following is the python script that perform the operations. >> >> def save_user_in_database(user, cursor): >> db_line = 'SELECT 1 FROM users WHERE user_id = {}'.format(user.id) >> LOGGER.info('Executing on DB: %s', db_line) >> cursor.execute(db_line) >> >> answer = cursor.fetchall() >> LOGGER.info('Get this answer %s', answer) >> if len(answer) == 0: >> LOGGER.info('Inserting new user') >> >> db_line = sql_cmd_from_template( >> USER_TABLE_INSERT_TEMPLATE, >> user.id, >> user.name, >> user.description, >> user.location >> ) >> LOGGER.info('Executing on DB: %s', db_line) >> cursor.execute(db_line) > >Do you execute the queries in a different schema than the default ?sys? schema (which is recommended for database users)? >Because ?users? is a special VIEW in the ?sys? schema. > Yes, I should be in a different schema. In any case, I changed the name of the table just to be on the safe way. >> >> This function is executed inside a single threaded application. >> Moreover, there are no "DELETE" or similar operations inside this >> script (it only adds rows to the database). >> No user (human or software) but this script connects to my MonetDB instance. >> >> Unfortunately, after I while that the script is running (let's say one >> or two days), something unexpected happens. For example, this is the >> log generated for user 42847029 >> (there are a lot of other lines in between that I removed). As you can >> see, after a while, my check fails and I don't understand why. >> >> 2017/06/13 09:30:01 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 09:30:01 - INFO - save_user_in_database: Get this answer [(1,)] >> >> 2017/06/13 10:00:04 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 10:00:04 - INFO - save_user_in_database: Get this answer [(1,)] >> >> 2017/06/13 10:02:33 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 10:02:33 - INFO - save_user_in_database: Get this answer [(1,)] >> >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Get this answer [] >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Inserting new user > >This shouldn?t happen, as is shown by the error below that a user_id of 42847029 still exists; also, as you mentioned, there hasn?t been any delete queries. > >This might be a bug in MonetDB. > >Is this problem repeatable? If yes, can you give us a sequence of queries to reproduce this problem. We?d like to look into it. > I will try to reproduce the problem with some scripts as soon as possible. Unfortunately, I am quite busy in this period and therefore this will require some time. In any case, this happens only when the system is busy and the table is big enough. I think that the key to reproduce the problem is to stress the database with several operations per second. In this situation, I have noticed that the db starts to use a lot of RAM (about 15 GB for a database that occupies 3 GB on the disk). At that point, the db becomes extremely slow (for example, for a query that usually takes less than 10 ms I have to wait for more than 2 seconds) and the unique key problem start to appear. > >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Executing on DB: >> INSERT INTO users(user_id, user_name, user_description, user_location) >> VALUES (42847029, 'XXXX', 'XXXX', 'XXXX') >> 2017/06/13 10:30:36 - WARNING - on_data: Error elaborating data >> Traceback (most recent call last): >> File "data_collector.py", line 267, in on_data >> return self.elaborate_data(data) >> File "data_collector.py", line 354, in elaborate_data >> save_user_in_database(user, cursor) >> File "data_collector.py", line 234, in save_user_in_database >> cursor.execute(db_line) >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/sql/cursors.py", >> line 164, in execute >> block = self.connection.execute(query) >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/sql/connections.py", >> line 133, in execute >> return self.command('s' + query + ';') >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/sql/connections.py", >> line 138, in command >> return self.mapi.cmd(command) >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/mapi.py", >> line 235, in cmd >> raise exception(string) >> pymonetdb.exceptions.IntegrityError: INSERT INTO: PRIMARY KEY >> constraint 'users.users_user_id_pkey' violated >> >> Have you any advice? Is there a way to perform the same check in just >> one SQL line? Is this way atomic (if a would ever move to a >> multithreading script)? > >It is not atomic, but you can make it atomic by wrapping both queries into one transaction. > >> Am I doing something >> wrong with my current code? Please, take into account that I do not >> have a deep experience with other SQL tools so it is perfectly >> possible that I miss something >> that for you is extremely obvious. > >No, you?re not doing anything wrong. But there are better alternatives: > >1. If this is a standalone action (i.e. not part of a transaction), the check for existing user_id is not necessary. The simplest and most efficient way is to always do the insert and ignore the ?primary key constraint ? violated? >error (e.g. use a try-catch block). > >2. If this insert is going to be part of a (big, long) transaction, then you won?t want the insert to fail. In that case, you can use a stored procedure: > >create procedure insert_new (id int, name varchar(10), description varchar(10), location varchar(10)) >begin > declare cnt int; > set cnt = (select count(*) from myusers where user_id=id); > > if (cnt = 0 ) > then > insert into myusers values (id, name, description, location); > end if; >end; > >Then call the procedure: call insert_new (42847029, 'XXXX', 'XXXX', 'XXXX?) > >I hope this helps. > II tried the solution number 2, but unfortunately the problem is still there. I get an Integrity error even calling the procedure that you described before. What is really strange is that, after a while, execute the procedure that you have showed me requires more than 3 seconds on my system. If I restart the monetdb instance, it takes a few milliseconds again. I think that maybe there is something broken in my setup. It could be that I have misconfigured something or I have a problem with my machine. Unfortunately, taking also into account that I am using Docker (which adds a layer of complexity), there are plenty of things that could not work as expected. Thank you for your help! >Regards, >Jennie > >> >> Thank you in advance! >> >> Best regards, >> >> Stefano Piani

2 1

MonetDB :: Memory Blots Up On High Concurrency
by Sharma, Sreejith 21 Jul '17

21 Jul '17

Hi, Does anyone encountered memory blot issue on high concurrency? Also, I don't see it coming down when idle. When its blots up, it's slows down the response time. Regards, Sreejith

3 6

Monetdb Installation
by amit＠g-square.in 21 Jul '17

21 Jul '17

Hi, how to install monetdb's old version June 2017 for ubuntu ??

2 1

Error trying to login
by amit＠g-square.in 20 Jul '17

20 Jul '17

Hi, I am using Ubuntu OS. I have created dbpharm and database. Now i am trying to login mclient using terminal but it gives following error. Please reply asap. root@mp-VirtualBox:/var/lib/monetdb# mclient -u monetdb -d narrator password: TypeException:user.sql.init[5]:'sqlcatalog.create_function' undefined in: sqlcatalog.create_function(X_1:str, X_3:str, X_5:ptr); MALException:optimizer.MALoptimizer:Start with inconsistent MAL plan

2 1