- users-list - monetdb.org

COPY INTO from LZ4 file: bug or missing feature?
by Ying Zhang 04 Sep '17

04 Sep '17

Dear all, Is COPY INTO from an LZ4 file supported by MonetDB? When I try to load data from an LZ4 file, I always get the error: Failed to import table 'pt', record too long (EOS found) Is this an error or a missing feature? Additional info: - MonetDB version Jul2017 (Mercurial source) - Mac OS (Sierra) - I checked the config.log that my lz4 lib was found - The LZ4 file is valid (I decompressed it, then I could load the data). Thanks in advance! Jennie

1 1

Execute MAL queries
by Dennis Fassl 04 Sep '17

04 Sep '17

Dear all, I am trying to write and execute MAL queries. The overall goal is to translate Cypher to MAL and to use Cypher as an input language for MonetDB. I am using the "explain" statement to get some example MAL queries. This is an output of explain (without the optimizer stuff at the end): function user.s4_1():void; X_0:void := querylog.define("explain select source, target from graph where source = \\'A\\';","default_pipe",14:int); X_30 := bat.new(nil:str); X_36 := bat.new(nil:int); X_34 := bat.new(nil:int); X_33 := bat.new(nil:str); X_32 := bat.new(nil:str); X_3 := sql.mvc(); X_7:bat[:str] := sql.bind(X_3,"sys","graph","source",0:int); C_4:bat[:oid] := sql.tid(X_3,"sys","graph"); C_18 := algebra.thetasubselect(X_7,C_4,"A","=="); X_21:bat[:str] := sql.bind(X_3,"sys","graph","target",0:int); X_28 := algebra.projection(C_18,X_21); X_20 := algebra.projection(C_18,X_7); X_37 := bat.append(X_30,"sys.graph"); X_39 := bat.append(X_32,"source"); X_40 := bat.append(X_33,"varchar"); X_42 := bat.append(X_34,31:int); X_44 := bat.append(X_36,0:int); X_46 := bat.append(X_37,"sys.graph"); X_47 := bat.append(X_39,"target"); X_48 := bat.append(X_40,"varchar"); X_49 := bat.append(X_42,31:int); X_50 := bat.append(X_44,0:int); sql.resultSet(X_46,X_47,X_48,X_49,X_50,X_20,X_28); end user.s4_1; But when I try to execute this query I get errors like that: ERROR = !TypeException:user.main[8]:'sql.tid' undefined in: C_4:bat[:oid] := sql.tid("sys":str,"sys":str,"graph":str); I don't know why the "X_3" is replaced by "sys". I see changes like that in the queries regularly. Does someone know what is wrong or missing in this query? Thanks a lot in advance! Best regards, Dennis

4 5

Possible bug with table generating functions
by Mark Clements 29 Aug '17

29 Aug '17

Is there a restriction on the use of the same table generating function in the same select statement? This relates to the following possible bug: -- reimplement generate_series for MonetDBLite CREATE FUNCTION my_generate_series(start INT, finish INT) RETURNS TABLE (value INT) BEGIN DECLARE TABLE tmp_generate_series(value INT); DECLARE i INT; SET i = start; WHILE (i <= finish) DO INSERT INTO tmp_generate_series VALUES(i); SET i = i + 1; END WHILE; RETURN tmp_generate_series; END; --- This seems to work quite nicely: select * from my_generate_series(1,10); -- and it's fast: select count(*) from my_generate_series(1,100000); -- however, I get odd and non-deterministic counts for the following: select count(*) from my_generate_series(1,100) as t1, my_generate_series(1,100) as t2; This is using MonetDB v11.23.3 (Jun2016) that was compiled with embedded R, but not run with --set embedded_r=true. I got similar errors with MonetDBLite. Kindly, Mark.

2 2

Looking for Admin UI / Tools / Resources
by Cota, Luis 28 Aug '17

28 Aug '17

Can anyone recommend any tools (commercial or open source) which assist in administering a MonetDB instance? Things I'd like to do are: Create tables from existing data files (CSV, TXT, HDF5) Import or query Oracle databases and either create tables, or populate tables, with the results of those queries Query existing monetdb instances Thanks, Luis ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

1 0

Error after updating monetdb
by Mahmoud Mohsen 19 Aug '17

19 Aug '17

Hello, I have updated monetdb from Dec2016-SP5 to version v11.27.5 "Jul2017-SP1, but after this when trying to accessing the database via mclient -u monetdb -d tpch2 I am getting this error: internal error while starting mserver 'database 'tpch2' started up, but failed to open up a communication channel', please refer to the logs looking at the logs, 2017-08-18 22:42:02 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:02 MSG merovingian[10645]: database 'voc' (10660) has exited with exit status 0 2017-08-18 22:42:02 MSG merovingian[10645]: starting database 'voc', up min/avg/max: 0s/0s/0s, crash average: 0.00 0.00 0.00 (1-1=0) 2017-08-18 22:42:02 MSG merovingian[10645]: sending process 10663 (database 'voc') the TERM signal 2017-08-18 22:42:02 MSG voc[10663]: arguments: /usr/bin/mserver5 --dbpath=/home/mohsen/mfarm/voc --set merovingian_uri=mapi:monetdb://u145-16:50000/voc --set mapi_open=false --set mapi_port=0 --set mapi_usock=/home/mohsen/mfarm/voc/.mapi.sock --set monet_vault_key=/home/mohsen/mfarm/voc/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2017-08-18 22:42:02 MSG voc[10663]: # MonetDB 5 server v11.27.5 "Jul2017-SP1" 2017-08-18 22:42:02 MSG voc[10663]: # Serving database 'voc', using 8 threads 2017-08-18 22:42:02 MSG voc[10663]: # Compiled for x86_64-pc-linux-gnu/64bit with 128bit integers 2017-08-18 22:42:02 MSG voc[10663]: # Found 251.889 GiB available main-memory. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) 1993-July 2008 CWI. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved 2017-08-18 22:42:02 MSG voc[10663]: # Visit https://www.monetdb.org/ for further information 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:^LoaderException:loadLibrary:Loading error failed to open library sql (from within file '/usr/lib/monetdb5/lib_sql.so'): /usr/lib/monetdb5/lib_sql.so: undefined symbol: mnstr_writeFlt 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError: ^';' expected 2017-08-18 22:42:03 MSG merovingian[10645]: database 'voc' has shut down 2017-08-18 22:42:03 ERR control[10645]: !monetdbd: an internal error has occurred 'database 'voc' started up, but failed to open up a communication channel' 2017-08-18 22:42:07 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:07 MSG merovingian[10645]: database 'voc' (10663) has exited with exit status 0 Thank you a lot in advance, Best regards, Mahmoud Mohsen

2 3

Re: Re: failed to import record too long: Loading 1.5 Terabyte table
by Stefan Manegold 18 Aug '17

18 Aug '17

You need to use the correct record delimiter i.e. newline instead of space. -------- Original message --------From: Mahmoud Mohsen <mmohsen35(a)gmail.com> Date: 8/18/17 19:40 (GMT+01:00) To: users-list(a)monetdb.org Subject: Re: Re: failed to import record too long: Loading 1.5 Terabyte table Dear Stefan, Thank you very much for your quick reply, so this is the first line of the file "lineitem.tbl", so no need to use an offset I think 1|310378689|15378705|1|17|29786.89|0.04|0.02|N|O|1996-03-13|1996-02-12|1996-03-22|DELIVER IN PERSON|TRUCK|egular courts above the| as for the version, I am using: MonetDB Database Server Toolkit v1.1 (Dec2016-SP5). So can this may be the cause of the error... an old version? I am looking forward hearing from you, Best regards, Mahmoud Mohsen

2 2

failed to import record too long: Loading 1.5 Terabyte table
by Mahmoud Mohsen 18 Aug '17

18 Aug '17

Hello, I'm trying to load a table with 1.5 Terabytes of data, the command I'm using COPY INTO lineitem FROM '/vol1/tpch-dbgen/2000/lineitem.tbl' USING DELIMITERS '|', ' ' LOCKED; The physical memory I'm using is 256G, the swap is 2.2 terabytes, the dbfarm is located within area has 4 Terabytes of free storage. The message I'm getting: "Failed to import table 'lineitem', record too long" So what could be a possible solution to this problem, Thank you a lot in advance, Mahmoud Mohsen

3 3

(lastest version) monetdb source compilation and installation problem
by 张馨匀 17 Aug '17

17 Aug '17

Hi All, Could you please help me to solve the following error? When I try to install monetdb I have the following error: * fatal error: *geos_c.h: No such file or directory #include <geos_c.h> *My steps used to compile monetdb src and to install monetdb are:* a. Environment setup: $ export MonetDB_rel=July2017 $ export MonetDB_ver=11.27.5 $ export MonetDB_base_dir=$HOME/MonetDB-$MonetDB_rel $ export MonetDB_source_dir=$MonetDB_base_dir/MonetDB-$MonetDB_ver $ export MonetDB_build_dir=$MonetDB_base_dir/BUILD $ export MonetDB_prefix_dir=$MonetDB_base_dir $ mkdir $MonetDB_base_dir $ cd $MonetDB_base_dir b. Download the latest version: $ wget https://dev.monetdb.org/downloads/sources/Jul2017-SP1/MonetDB-11.27.5.tar.xz $ tar xf MonetDB-$MonetDB_ver.tar.xz c. Build: $ mkdir $MonetDB_build_dir $ cd $MonetDB_build_dir $ MonetDB_source_dir/configure --prefix=$MonetDB_prefix_dir d. Install: $ make -j 20 (error mentioned above occurs here) $ make -j 20 install Thanks,

2 1

Memory bloat on concurrency - Any fix to release memory post query run
by Sharma, Sreejith 11 Aug '17

11 Aug '17

Hi, I have been using monetdb (11.25.3) in CentOS for one of application. I have been working on a PSR testing for this application and have noticed on high concurrency and on same pattern of query running again and again causing memory to bloat up and doesn’t release once the query run completes. I understand that the data is lifted into memory and the result set is also cached, however I assume this should be release once the system is idle which I don’t see it as of now. Is there any way or parameter to release the unused memory blocks? I have asked this before, but couldn't get to the root to fix this. Regards, Sreejith

3 4

Troubles with Python API
by 倪令斌 09 Aug '17

09 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

2 3