- users-list - monetdb.org

Excessive slowness in a query
by Alfio Bettini 14 Dec '17

14 Dec '17

Hi all, we're testing MonetDB performances in order to understand if it could be possible to use it as data mart in our BI software. We're considering a set of queries executed in our test environment (CentOS 7, 32 GB RAM, 4 cpu). All the executed queries have quite good performances (between 10 and 45 seconds) except the following one that is taking several minutes in order to be executed: SELECT id, count(distinct value) FROM table GROUP BY id; The table contains 100 millions of data. It has about 130 columns (varchar, int, bigint, smallint, tinyint, timestamp, datetime). We have no BLOB/CLOB/Text types. The fields used in the query have these data types: id: int value: bigint Can you suggest us any optimizations in order to have this query run faster? Do you know some possible reasons why it's so slow? Kind Regards, Alfio Bettini The information contained in this message, together with any attachments, may be privileged and confidential and is intended for the use of the addressee(s) only. Any review, copying, use or distribution of this email by others (including any of its attachments) is strictly prohibited. If you are not the intended recipient, please destroy this email and its attachments and contact the sender immediately. This email was sent by EidosMedia SpA<http://www.eidosmedia.com>

1 0

Re: algebra.join heapextend crash
by Stefan Manegold 13 Dec '17

13 Dec '17

Allocating "only" about 2.8 GB does not ring alarms that the plan night be degenerated - assuming your data is not tiny. Could you share:OS, full query, data sizes, free disk space at time of failure, mserver5 process size at time of failure? Thanks,Stefan -------- Original message --------From: Anthony Damico <ajdamico(a)gmail.com> Date: 11/22/17 12:13 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: algebra.join heapextend crash hi, i'm using monetdblite-r 0.5.0 on a confidential dataset. my syntax worked on monetdblite 0.3.1 on the same machine ftruncate: Invalid argument Error in .local(conn, statement, ...) : Unable to execute statement 'CREATE TABLE ccaed123_b_svcmon_xwalk AS ( SELECT a.enrolid , a.yr , a.dataty...'. Server says 'MALException:algebra.join:GDK reported error. !ERROR: GDKextendf: c !ERROR: HEAPextend: failed to extend to 2890530816 for 02\57\25766.tail: GDKmre map() failed '. it's a local drive, the hard disk isn't close to full and the machine has 700gb of ram, but the table it's trying to create might be bigger than that. dunno if related to https://www.monetdb.org/bugzilla/show_bug.cgi?id=3791 thanks

2 4

MonetDB Blob Inserts
by Brian Hood 07 Dec '17

07 Dec '17

Hi Guys, I don't seem to be able to INSERT into a blob field is there any information how to achieve this. Regards, Brian Hood

3 6

What is better JSON or TEXT field in terms of performance ???
by Ariel Abadi 07 Dec '17

07 Dec '17

Hi All! Does anybody has worked with json fields? We are planning to use json fields with very large text on it (the hierarchy of our json would be plain (only one level), we are doing some tests over 7MM records, it took like 1m30s to retrieve the information. In this case to get the information we used the function json.filter(name_of_field, '$..xxxx') We have also tested using a TEXT field (and using some string functions) we parsed the information inside the field and we found that sometimes it was quicker it took from 1m30s to 35s. I dont know exactly why the difference on that delta. We need to implement the processes, and need to decide how to move forward.... Does anybody can help me on the decision ??? We know and is a fact, that Monet can beat any DB in terms of performance, and that is why we decided to use it, but now Im worried about this type of queries (against json/text fields) and how long this queries would take if we increase the amount of records.... What would you suggest ?? Thanks in advance! Rgds Ariel

3 3

mserver5 crash on startup in Ubuntu Xenial
by Manuel Pagliai 06 Dec '17

06 Dec '17

Hello, I get this error when I try to start monetdb on Ubuntu Xenial 16.04.03 mserver5 visokio # MonetDB 5 server v11.27.9 "Jul2017-SP2" # Serving database 'demo', using 12 threads # Compiled for x86_64-pc-linux-gnu/64bit with 128bit integers # Found 7.702 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved # Visit https://www.monetdb.org/ for further information !SyntaxException:parseError:include sqlcatalog; !SyntaxException:parseError:^LoaderException:loadLibrary:Loading error failed to open library sql (from within file '/usr/lib/monetdb5/lib_sql.so'): /usr/lib/monetdb5/lib_sql.so: undefined symbol: mnstr_writeFlt !SyntaxException:parseError:include sqlcatalog; !SyntaxException:parseError: ^';' expected Segmentation fault (core dumped) any ideas ? Thanks, Best regards, *Manuel Pagliai* Visokio Ltd www.visokio.com +44 20 7351 4646

1 0

Issue with SUM(DISTINCT) in 11.27.9
by Sharma, Sreejith 05 Dec '17

05 Dec '17

All, We have noticed when SQL pattern has SUM(DISTINCT... ) there is a degradation of SQL timings when compared to 11.27.5 build. Anyone is aware of this? Regards, Sreejith

1 0

Re: MAL query "sql.resultSet"
by dennis.fassl 01 Dec '17

01 Dec '17

Hi all, can you help me with the below query? Is there an other way to return query results using the jdbc interface (without using sql.resultSet)? Thanks in advance! Best regards,Dennis -------- Ursprüngliche Nachricht --------Von: Dennis Fassl <dennis.fassl(a)uni-konstanz.de> Datum: 25.11.17 16:55 (GMT+01:00) An: users-list(a)monetdb.org Betreff: MAL query "sql.resultSet" Hi all, I am trying to execute the MAL query below. But the command "sql.resultSet" is terminating mclient. When using the JDBC interface I get "SQLException: Connection to server lost! (mserver still alive?)". Do you have an idea why this is happening? Thanks and regards, Dennis sql.init(); table_names := bat.new(nil:str); bat.append(table_names, "t0"); bat.append(table_names, "t1"); table_attrs := bat.new(nil:str); bat.append(table_attrs, "int_col"); bat.append(table_attrs, "str_col"); table_types := bat.new(nil:str); bat.append(table_types, "int"); bat.append(table_types, "str"); table_lens := bat.new(nil:int); bat.append(table_lens, 1); bat.append(table_lens, 1); table_scales := bat.new(nil:int); bat.append(table_scales, 0); bat.append(table_scales, 0); ints := bat.new(nil:int); bat.append(ints, 42); bat.append(ints, 23); strs := bat.new(nil:str); bat.append(strs, "foo"); bat.append(strs, "bar"); sql.resultSet(table_names, table_attrs, table_types, table_lens, table_scales, ints, strs); _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

4 6

MAL query "sql.resultSet"
by Dennis Fassl 25 Nov '17

25 Nov '17

Hi all, I am trying to execute the MAL query below. But the command "sql.resultSet" is terminating mclient. When using the JDBC interface I get "SQLException: Connection to server lost! (mserver still alive?)". Do you have an idea why this is happening? Thanks and regards, Dennis sql.init(); table_names := bat.new(nil:str); bat.append(table_names, "t0"); bat.append(table_names, "t1"); table_attrs := bat.new(nil:str); bat.append(table_attrs, "int_col"); bat.append(table_attrs, "str_col"); table_types := bat.new(nil:str); bat.append(table_types, "int"); bat.append(table_types, "str"); table_lens := bat.new(nil:int); bat.append(table_lens, 1); bat.append(table_lens, 1); table_scales := bat.new(nil:int); bat.append(table_scales, 0); bat.append(table_scales, 0); ints := bat.new(nil:int); bat.append(ints, 42); bat.append(ints, 23); strs := bat.new(nil:str); bat.append(strs, "foo"); bat.append(strs, "bar"); sql.resultSet(table_names, table_attrs, table_types, table_lens, table_scales, ints, strs);

1 0

Number of instance
by Sharma, Sreejith 23 Nov '17

23 Nov '17

All, May be a dump question, but curious to know. What is the maximum number of MonetDB instance anyone has setup in a single server? And what level of concurrency has been tested? Regards, Sreejith

1 0

MonetDB - Data Compression, is it possible?
by Sharma, Sreejith 23 Nov '17

23 Nov '17

All, Is there any data compression methods available in MonetDB? What I understand from I read is, by default MonetDB doesn't have a compression methods to avoid the overhead of uncompression while loading the files into memory mapped files. Is that true? If there is a compression mechanism (alike QUERY HIGH Compression in Oracle) what is it in here? Regards, Sreejith

2 2

algebra.join heapextend crash
by Anthony Damico 22 Nov '17

22 Nov '17

hi, i'm using monetdblite-r 0.5.0 on a confidential dataset. my syntax worked on monetdblite 0.3.1 on the same machine ftruncate: Invalid argument Error in .local(conn, statement, ...) : Unable to execute statement 'CREATE TABLE ccaed123_b_svcmon_xwalk AS ( SELECT a.enrolid , a.yr , a.dataty...'. Server says 'MALException:algebra.join:GDK reported error. !ERROR: GDKextendf: c !ERROR: HEAPextend: failed to extend to 2890530816 for 02\57\25766.tail: GDKmre map() failed '. it's a local drive, the hard disk isn't close to full and the machine has 700gb of ram, but the table it's trying to create might be bigger than that. dunno if related to https://www.monetdb.org/bugzilla/show_bug.cgi?id=3791 thanks

1 0

Patch for backreferences in PCRE replace
by Roberto Cornacchia 16 Nov '17

16 Nov '17

Hi, Following up on this thread ( https://www.monetdb.org/pipermail/users-list/2016-April/009107.html) from last year, the issue was that the pcre-based regex replacement did not implement back references in the replacement string. In need for this feature, I so far used the pcre implementation via R function. It works well, but using R just for this is a bit odd, especially because R-core has a pretty dumb dependency on the whole TeX distribution, and building MonetDB docker images that need to include R and TeX just because of regex isn't a neat option. I have also tried to use both re and regex packages via python, but this is very slow compared to pcre. So I finally decided to put a few hours on the MonetDB implementation and added support for back references. They can be indicated with both \1 and $1 syntax. Back-reference 0 is the whole match. Out-of-bound back-references are empty strings. While at it: - pcre.replace replaces all matches, so I made a variant pcre.replacefirst to replace only the first match. - the bat implementation was ignored because of a missing "module batpcre" in pcre.mal, it always resorted to manifold on the string version. Fixed that. Also, I'm not sure why there don't seem to exist SQL functions declared for this. I use the following: CREATE FUNCTION pcre_replace(s string, pattern string, repl string, flags string) RETURNS string EXTERNAL NAME pcre."replace"; CREATE FUNCTION pcre_replacefirst(s string, pattern string, repl string, flags string) RETURNS string EXTERNAL NAME pcre."replace_first"; Perhaps they could come by default? Please feel free to use and modify the patch (Jul2017) in attachment if you think it's useful. I've tested it quite a bit, no issues found. Here are some examples: sql>select name, pcre_replace(name, '([a-z]+)_([a-z]+)', '[\\1]-[\\2]', '') as replaced from sys.functions limit 2; +-------------+---------------------------------------------+ | name | replaced | +=============+=============================================+ | mbr_overlap | [mbr]-[overlap] | | mbr_overlap | [mbr]-[overlap] | +-------------+---------------------------------------------+ 2 tuples (4.650ms) sql>select name, pcre_replacefirst(name, '([a-z])', '[\\1]', '') as replaced from sys.functions limit 2; +-------------+---------------------------------------+ | name | replaced | +=============+=======================================+ | mbr_overlap | [m]br_overlap | | mbr_overlap | [m]br_overlap | +-------------+---------------------------------------+ 2 tuples (4.728ms) sql>select name, pcre_replace(name, '([a-z])', '[\\1]', '') as replaced from sys.functions limit 2; +-------------+-----------------------------------------------------------------------------------------------------+ | name | replaced | +=============+=====================================================================================================+ | mbr_overlap | [m][b][r]_[o][v][e][r][l][a][p] | | mbr_overlap | [m][b][r]_[o][v][e][r][l][a][p] | +-------------+-----------------------------------------------------------------------------------------------------+ 2 tuples (10.360ms)

1 1

UDF function with >3 parameters cannot benefit from using PYTHON_MAP
by Hanfeng Chen 15 Nov '17

15 Nov '17

Hi, I created a table with 9 columns and passed 6 columns from the table to a UDF. This UDF is fully vectorizable. However, we found only single thread was used during the execution. Then, we came to think about what the problem behind and we created the two simple functions python_min_map_3 and python_min_map_4 which takes 3 and 4 parameters separately. The MonetDB Linux version we used was July 2017. If PYTHON_MAP is enabled, it would return the results of each segment processed by parallel code. The number of results depends on how many threads are used. Otherwise, a single value should be returned (e.g. because of numpy.min). The result is - python_min_map_3 returns a couple of numbers - python_min_map_4 returns a single number Even when we increased the number of the arguments, a single number was always returned. Is it a restriction in using PYTHON_MAP when the number of arguments should be no more than 3? Here is our example code below. CREATE FUNCTION python_min_map_3(x0 FLOAT, x1 FLOAT, x2 FLOAT) RETURNS FLOAT LANGUAGE PYTHON_MAP { return numpy.min(x0) }; select python_min_map_3(x0, x1, x2) from table_0; CREATE FUNCTION python_min_map_4(x0 FLOAT, x1 FLOAT, x2 FLOAT, x3 FLOAT) RETURNS FLOAT LANGUAGE PYTHON_MAP { return numpy.min(x0) }; select python_min_map_4(x0, x1, x2,x3) from table_0; NumPy in MonetDB reference: https://www.monetdb.org/blog/embedded-pythonnumpy-monetdb Best regards, Hanfeng Chen

1 0

read BLOB column on tail heap and loop over rows
by Anton Kravchenko 13 Nov '17

13 Nov '17

I wonder how one would read BLOB column and loop over its rows? BAT *b = table_funcs.full_column(m->session->tr, blob_col_name); //blob *readblobrows = (blob*) b->T.heap.base; //unsigned char *readblobrows = (unsigned char*) b->T.heap.base; p.s. For INT column it would be go like this: int * readintrows = (int *) b->T.heap.base; for (irow=0; irow < nrow; irow++) { intvalue = *readintrows++; } Thanks, Anton

1 1

Querying JSON arrays
by Brian Ploetz 09 Nov '17

09 Nov '17

Hi, Suppose I have a table with a "tags" column that is of the JSON datatype ( https://www.monetdb.org/Documentation/Manuals/SQLreference/Types/JSON) The column will contain a JSON array of Strings. For example: ["foo","bar","baz"] Is it possible to query for rows that contain a specific value in the tags array? For example, if I wanted to find all of the rows where tags contains "foo", my query would look something like: select count(*) from my_table where tags ???? 'foo'; But it's not clear to me what ???? should be (or if it's even possible) based on the JSON documentation above. Thanks in advance. BP

2 3

large virtual memory spike on BLOB column select
by Anton Kravchenko 09 Nov '17

09 Nov '17

I am using MonetDB v11.27.9 on cent7. 1) 'select v_blob from blob_table' produces a virtual memory spike ~800Gb [using a 'limit' reproduces virtual memory spike too] >From 'select * from storage()' it looks a blob column size is ~25Gb. 2) And 'create table new_blob_table as (select v_blob from blob_table) with data;' produces only ~50Gb of virtual memory. 3) Also for a table with non-blob column both 'select * from nonblob_table' and 'create new_nonblobtable as (select * from nonblob_table) with data;' produce the ~same virtual memory consumption. p.s. Here are steps to reproduce this issue # to generate date by using Python hex_blob='47'*2000 hex_blob_chunk=(hex_blob+'\n')*1000 f=open('blob_data.txt','w') for irow in range(10000): f.write(hex_blob_chunk) f.close() # to load data into Monet blob_table create table blob_table(v_blob blob); COPY 10000000 RECORDS INTO blob_table FROM '/home/blob_data.txt' using delimiters ',','\n' NULL AS '' LOCKED; # to produce virtual memory spike of ~800GB select * from blob_table limit 50000; # to produce virtual memory of ~50Gb create table new_blob_table as (select * from blob_table) with data; Thanks, Anton

2 1

Problems creating json files with more than 10^3 table entries
by Giacomo Bergami 09 Nov '17

09 Nov '17

I am using MonetDB-11.27.9 on ubuntu 17.04. The client was compiled from the source code. I am trying to run the following query on your database select distinct T.src as author, T1.src as coauthor, json.tojsonarray(T1.dst) as papers from edges1000 as T, edges1000 as T1 where T.dst = T1.dst AND T.src <> T1.src group by T.src, T1.src ; When I run this query over a table with at most 10^4 entries, the mclient client quits. If I try to execute the same qury via JDBC, I receive an error that tells me "maybe mserver closed unexpectedly". On the other hand, the server is still alive and responsive if I afterwards run the query either oover mclient or JDBC over a table having less data (edges10). Is that a problem of memory allocation ? Is there a way to solve this issue? I'm using the default configuration, and compiled the source code using "autoreconf -if" first. Thanks in advance for your support, Bergami G

1 0

latest version question
by Lynn Carol Johnson 02 Nov '17

02 Nov '17

Hello monetdb team – We upgraded this afternoon to the latest version of monetdb (Jul2017-SP2). When testing access for my lab, I noticed the general login could no longer perform the command: \d <table_name> The errors were related to missing group access to sys.objects, sys.keys and sys.idxs. When, as monetdb admin, I granted “select” on these tables, the “\d <table_name>” command then worked. Is this something new, or does it indicate a problem with the db ? It was not necessary to grant general users permission to these system tables previously. Thanks - Lynn

2 4

Problem in installation
by Amit Ambekar 27 Oct '17

27 Oct '17

Hi, Sorry sending this email again because of limit in message body is 40 kb. I had installed monetdb and now wanted to start mclient. So first i started Mserver5.bat and then when i click on mclient.bat that time "api-ms-win-crt-runtime-l1-1-0.dll is missing issue" this error is coming in alert box. Is monetdb is only usefull for windows server 2003,vista and Windows 7 or it can be used with Windows 2012 server also? Installed Visual C++ Redistributable for Visual Studio 2015 , then too error coming . Please check attached screenshot. this error coming when tried to start mclient

3 4

Setting Up Tools
by Sandy 26 Oct '17

26 Oct '17

Just getting back into trying MonetDB after many years. Got the environment set up pretty well with the test DB `VOC` loaded up and able to do all expected in the command line tools. So I figured I would give a couple of the mentioned GUI tools a shot and see how they work against the server I just set up on a virtual box. I picked up the JDBC drivers and installed on my Window's 10 install of SQuirrel SQL and DBeaver. I got both of the drivers works sort of... On DBeaver I can execute and see queries in the tool but can't do much more than that. Overall looks like an issue with accessing the system table `sys.environment`, not sure if anything related to set up or rights. Using the `voc` user as created with the example db in the getting started. Any help or thought on where to start? Thanks! Sandy -------- Errors * DBeaver 4.0.8 On inspection of some of the logs I see things like - `Error loading object metadata from database` `SQL Error [22000]: SELECT: access denied for voc to table 'sys.environment' java.sql.SQLException: SELECT: access denied for voc to table 'sys.environment'` * SQuirreL SQL Client 3.7.1 java.sql.SQLException: SELECT: access denied for voc to table 'sys.environment' -------- Driver and DB versions I'm using this JAR for the drivers - C:\Users\Squidward\Downloads\monetdb-jdbc-2.25.jar, 2.25 (Liberica 20170330 based on MCL v1.15) MonetDB version - MonetDB v11.25.23 (Dec2016-SP5), 'mapi:monetdb://sandy-VirtualBox-18:50000/voc'

3 6

Announcement: New Jul2017-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 26 Oct '17

26 Oct '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP2 bugfix release (11.27.9) MonetDB Common * A serious bug, possibly resulting in database corruption, having to do with appending data to a string BAT was fixed. Bug Fixes * 4017: server crashes when executing particular loopback query in embedded python * 6207: identifier ambiguous when grouping and selecting the same column twice * 6239: Incorrect profiling * 6261: New handling of delta tables hurts badly reusage of bats * 6287: should the CORR function return some numeric type that allows fractions? * 6321: Two-column aggregation on join result extremely slow. * 6335: Sqlitelogictest crash in complex SQL query * 6343: MERGE TABLE issue: unable to find * 6348: Interference of procedure/table name * 6350: Carriage return and form feed in TEXT fields are returned as 'r' and 'f' in jdbcclient and ResultSets * 6352: Scope resolution problem (sqlsmith) * 6353: implicit NULL value not propagated in distributed/remote query * 6374: Wrong answer from merge table after content changes * 6379: Table UDF: SEGV raised when invoking a non existing function * 6380: unable to create new databases from clean installation * 6381: Parser misses error messages in conditional * 6382: Can't set JSON fields via PreparedStatement * 6384: crash when setting a wrong listenaddr * 6385: AGGREGATE UDFs with more than 2 parameters incorrectly processed * 6386: Unexpected error from server for query with long floats * 6387: Performance degradation on multi column sort * 6388: JDBC Connection via user voc produces errors when fetching certain meta data information * 6392: SELECT EXISTS (empty table) returns 'true' * 6395: BAT leak of scalar result sets * 6397: Isolation of generating functions not correct * 6398: Null Matches in outer join are not supported * 6399: UDF crashes when subquery and scalar values are passed as pameters * 6400: getCharacterStream() currently not supported * 6404: COPY INTO crashes if table has primary key or foreign key constraint * 6405: Compilation error if DISABLE_PARENT_HASH not defined * 6409: sqllogictest crash on aggregation query with NOT IN clause in HAVING clause * 6410: Sqlitelogictest crash on aggregation query with IN clause * 6411: Sqlitelogictest crash in aggregation query * 6412: Overflow in sys.epoch * 6413: Wrong answer for TPC-H Q17 * 6414: Using RPAD returns: could not allocate space * 6416: Sqlitelogictest crash in aggregation query with a NOT IN clause * 6417: Segfault encountered (sqlsmith) * 6418: Segfault in renaming (sqlsmith) * 6419: segfault in rel_optimizer (sqlsmith) * 6420: Assertion error in mergetable task (sqlsmith) * 6422: Another assertion error in rel_or (sqlsmith) * 6423: Dereference null pointer (sqlsmith) * 6424: Assertion error in rel_rename_expr (sqlsmith) * 6425: Assertion error in exp_bin (sqlsmith) * 6426: Assertion error in rel_find_exp_ (sqlsmith) * 6427: Assertion error in eq_typeswitchloop (sqlsmith) * 6428: Sqlitelogictest crash in aggregation query * 6430: Assertion raised in another eq_typeswitch error (sqlsmith) * 6431: Sqlitelogictest crash in aggregation query with a long having clause * 6432: Assertion error in exp_bin (sqlsmith) * 6433: Sqlitelogictest crash in complex SELECT query with IN operator * 6435: Sqlitelogictest crash in simple select query * 6437: System schemas "profiler" and "json" shouldn't be allowed to be dropped. * 6438: Implement functionality to enforce the restrict option in: DROP SCHEMA xyz RESTRICT; * 6440: Faulty plan generated. Query returns more rows than expected or existing in the view sys.tables.

1 0

Proper syntax for a windowing function
by Omri Schwarz 25 Oct '17

25 Oct '17

Hi, all. I'm trying to figure out the proper syntax for queries such as a moving average over a window in MonetDB. THis works: sql>select avg(energy) from TABLENAME; +--------------------------+ | L3 | +==========================+ | 658.5049567388747 | +--------------------------+ This fails: select avg(energy) OVER () from TABLENAME; SELECT: function 'avg' not found (Welcome to mclient, the MonetDB/SQL interactive terminal (Jul2017-SP1) Database: MonetDB v11.27.5 (Jul2017-SP1), 'mapi:monetdb://SERVER2:50000/scada' ) How do I write such a query? Many thanks.

4 5

Transaction Replication Question
by Sharma, Sreejith 25 Oct '17

25 Oct '17

All, I wanted to have "Transaction Replication" setup for getting an load balanced environment. It's mentioned in the documentation that below property needs to be setup for replication or shipping of the transaction log. How and where should I setup this? Is there anything else I should keep in mind for transaction replication? gdk_keep_persisted_log_files gdk_shared_logdir gdk_shared_drift_threshold Also, I wanted to know if we can have similar approach as below. Can we have different tables that are part of the same schema from different disks/mounts in MonetDB? Table 1 from Mount X Table 2 from Mount Y Regards, Sreejith

2 1

Yacc bug in sql_parser.y ?
by Omri Schwarz 24 Oct '17

24 Oct '17

Hi, all. Still trying to get rolling averages out of my MonetDB database, and I ran into this: Line 3954 of sql_parser.y, window_frame_end: UNBOUNDED FOLLOWING { $$ = _symbol_create_int(SQL_FRAME, -1); } | window_frame_following { $$ = $1; } | CURRENT ROW { $$ = _symbol_create_int(SQL_FRAME, 0); } ; window_frame_following: value_exp PRECEDING { $$ = $1; } ; Should that not be "value_exp FOLLOWING "?

1 0

RE: Transaction Replication Question - how to setup gdk_keep_persisted_log_files?
by Sharma, Sreejith 24 Oct '17

24 Oct '17

Also, below parameters. gdk_shared_logdir gdk_shared_drift_threshold From: Sharma, Sreejith Sent: Tuesday, October 24, 2017 4:01 PM To: Communication channel for MonetDB users Subject: Transaction Replication Question - how to setup gdk_keep_persisted_log_files? All, I wanted to have "Transaction Replication" setup for getting an load balanced environment. It's mentioned in the documentation that below property needs to be setup for replication or shipping of the transaction log. How and where should I setup this? Is there anything else I should keep in mind for transaction replication? gdk_keep_persisted_log_files Regards, Sreejith

1 0

Transaction Replication Question - how to setup gdk_keep_persisted_log_files?
by Sharma, Sreejith 24 Oct '17

24 Oct '17

All, I wanted to have "Transaction Replication" setup for getting an load balanced environment. It's mentioned in the documentation that below property needs to be setup for replication or shipping of the transaction log. How and where should I setup this? Is there anything else I should keep in mind for transaction replication? gdk_keep_persisted_log_files Regards, Sreejith

1 0

IMPORTANT: serious possibility for database corruption
by Sjoerd Mullender 23 Oct '17

23 Oct '17

We just found (and fixed) a serious problem in the Jul2017 releases that can result in database corruption. So far we have only seen the corruption occur after a database (mserver5) restart. We have not yet seen a problem with a server that continues running (but unfortunately no guarantees). We will create a new release candidate with this bug fixed as soon as possible (that will be version 11.27.9, i.e. anything in the 11.27.X series older than this is affected). Currently known work around is to *not* restart the server, but keep it running, and creating a dump of the database using msqldump. This dump will then have to be restored into a fresh database in a server that is not affected by the bug (Dec2016 release or the upcoming Jul2017-SP2). -- Sjoerd Mullender

1 0

Problem in installation
by Amit Ambekar 23 Oct '17

23 Oct '17

Hi Stefan, I had installed monetdb and now wanted to start mclient. So first i started Mserver5.bat and then when i click on mclient.bat that time "api-ms-win-crt-runtime-l1-1-0.dll is missing issue" this error is coming in alert box. Is monetdb is only usefull for windows server 2003,vista and Windows 7 or it can be used with Windows 2012 server also? On Monday 23 October 2017 03:30 PM, users-list-request(a)monetdb.org wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. Why does corr(int,int) return int? (Stefan Manegold) > 2. Re: Why does corr(int,int) return int? (Anthony Damico) > 3. Problem in installation (Amit Ambekar) > 4. Re: Why does corr(int,int) return int? (Stefan Manegold) > 5. Re: Problem in installation (Stefan Manegold) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 22 Oct 2017 19:48:27 +0200 (CEST) > From: Stefan Manegold <Stefan.Manegold(a)cwi.nl> > To: Communication channel for MonetDB users <users-list(a)monetdb.org> > Subject: Why does corr(int,int) return int? > Message-ID: <1997044866.6459538.1508694507470.JavaMail.zimbra(a)cwi.nl> > Content-Type: text/plain; charset=utf-8 > > > Hi, > > could some one possible explain to me > why the MonetDB corr() aggregate function, > that IMHO is supposed to calculate the correlation between two attributes, > does return the same type as (the larger of) its input type(s)? > > IMHO the correlation between two attributes (or (samples of) random variables > is a factional value between -1 and 1. > > Consequently, MonetDB corr() on two integer columns returns an integer results, > i.e., one of {-1,0,1} ... > > Shouldn't corr() --- much like avg(), var_pop(), var_samp(), stddev_samp(), stddev_pop() > always return type double ? > > Thanks! > > Best, > Stefan > -- Amit Ambekar Technology Associate Office: Ist floor, Poddar Chambers, Mathruradas Mill Compound, Lower Parel, Mumbai 400013 | Landline: +91 22 43470408 Cell: +91-7741823310 | Email: amit(a)g-square.in| Website: www.g-square.in cid:image003.png@01D2A947.80D76E60

2 1

Problem in windows OS
by Amit Ambekar 23 Oct '17

23 Oct '17

Hi, I am using monetdb since long time with linux OS, i just tried it with Windows 7 OS. But in windows after shut down machine, database is crashing. I am not able to use any query in mclient now for that databases. What is problem ?

2 2

Why does corr(int,int) return int?
by Stefan Manegold 23 Oct '17

23 Oct '17

Hi, could some one possible explain to me why the MonetDB corr() aggregate function, that IMHO is supposed to calculate the correlation between two attributes, does return the same type as (the larger of) its input type(s)? IMHO the correlation between two attributes (or (samples of) random variables is a factional value between -1 and 1. Consequently, MonetDB corr() on two integer columns returns an integer results, i.e., one of {-1,0,1} ... Shouldn't corr() --- much like avg(), var_pop(), var_samp(), stddev_samp(), stddev_pop() always return type double ? Thanks! Best, Stefan -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

2 2

Is there kind of tablespace concepts?
by Sharma, Sreejith 18 Oct '17

18 Oct '17

Hi All, Is there any possibility of retrieving the same table from 2 different sources (as similar to 2 tablespaces)? Regards, Sreejith

2 5

MonetDB build failure on Alpine Linux
by Artem Klevtsov 14 Oct '17

14 Oct '17

Hi, I tried to build MonetDB from source on the Alpine Linux. But build process was stopped with the following message: merovingian.c: In function 'main': merovingian.c:835:51: error: 'S_IRWXO' undeclared (first use in this function) (e = openConnectionUNIX(&unsock, control_usock, S_IRWXO, _mero_ctlout)) == NO_ERR) { ^~~~~~~ merovingian.c:835:51: note: each undeclared identifier is reported only once for each function it appears in merovingian.c:862:9: error: 'S_IRWXG' undeclared (first use in this function) umask(S_IRWXG | S_IRWXO); ^~~~~~~ Steps to reproduce (inside the alpine:latest docker container): apk update apk add musl-dev gcc bison make python libressl-dev libxml2-dev xz-dev bzip2-dev lz4-dev readline-dev snappy-dev curl-dev pcre-dev unixodbc-dev libatomic_ops-dev wget wget https://www.monetdb.org/downloads/sources/Jul2017-SP1/MonetDB-11.27.5.tar.xz tar xvf MonetDB-11.27.5.tar.xz cd MonetDB-11.27.5/ ./configure make I found that S_IRWXO and S_IRWXG macro defined in /usr/include/sys/stat.h: egrep '(S_IRWXO|S_IRWXG)' /usr/include/sys/stat.h #define S_IRWXG 0070 #define S_IRWXO 0007 I found also that sys/stat.h included in merovingian.c: grep sys/stat.h /MonetDB-11.27.5/tools/merovingian/daemon/merovingian.c #include <sys/stat.h> /* stat */ -- Regards Artem Klevtsov

1 0

RE: users-list Digest, Vol 62, Issue 5
by Warren Master 13 Oct '17

13 Oct '17

Restarting the server resolved the issue. I did not look at the plans.. There still remains the isse that tables can be added multiple times to the same merge table causing duplicate rows to be returned In some cases we were unable to remove a table from a merge table without first dropping the merge table Additionally we also have issues with bulk copy into tables of merge tables GDKerror:!ERROR: Incompatible operands. - this occurs now and then.. the only fix is to drop the merge table , drop the affecting table that fails to bulk copy to , recreate it and recreate the merge table. When I tried to drop the table after removing from the merge table , the server would core dump., I was only able to drop the bad table after dropping the merge table, the alter table Merge Table drop table Sub Table did not work Warren S. Master Chief Technology Officer, TRG Management LP 280 Park Avenue - 30th floor/West - New York, NY 10017 Phone 212.984.3504 Email warren.master(a)rohatyngroup.com Web www.rohatyngroup.com Linkedin www.linkedin.com/in/warrenmaster -----Original Message----- From: users-list [mailto:users-list-bounces+warren.master=rohatyngroup.com@monetdb.org] On Behalf Of users-list-request(a)monetdb.org Sent: Friday, October 13, 2017 6:00 AM To: users-list(a)monetdb.org Subject: users-list Digest, Vol 62, Issue 5 Send users-list mailing list submissions to users-list(a)monetdb.org To subscribe or unsubscribe via the World Wide Web, visit https://www.monetdb.org/mailman/listinfo/users-list or, via email, send a message with subject or body 'help' to users-list-request(a)monetdb.org You can reach the person managing the list at users-list-owner(a)monetdb.org When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..." Today's Topics: 1. Re: Merge Table Issues (Niels Nes) ---------------------------------------------------------------------- Message: 1 Date: Thu, 12 Oct 2017 20:39:51 +0200 From: Niels Nes <Niels.Nes(a)cwi.nl> To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: Re: Merge Table Issues Message-ID: <20171012183951.GA8701(a)niels.cwi.nl> Content-Type: text/plain; charset=utf-8 On Thu, Oct 05, 2017 at 07:29:08PM +0000, Warren Master wrote: > all_vw is a merge table with three tables all with identical structure Warren Do the plan's look correct, ie include the proper member tables? Niels > > > > intraday > > history > > currentyear > > ---------------------------------------------------------------------- > ----------------------- > > > > My merge tables do not return consistent results > > > > I have tried to refresh statistics > > > > ANALYZE sys. intraday (asofdate) MINMAX > > ANALYZE sys. history (asofdate) MINMAX > > ANALYZE sys. currentyear (asofdate) MINMAX > > > > still same issue > > > > the issue exists for a few dates that I checked, there are many days > that do work on the merge table , all dates work on the view and the > individual tables. > > I thought it might be an issue with statistics then the conversion > function made it work on the merge table , so now at a loss. > > > > > > > > the following query on the merge table return no rows > > ---------------------------------------------------------------------- > ----------------------- > > > > select fundid,sum(curdlttot) as fxt > > from vw where asofdate='2017-10-02' > > group by fundid > > > > The following does return data from the merge table albeit, much > slower ? after converting date value > > ---------------------------------------------------------------------- > ----------------------- > > select fundid,sum(curdlttot) as fxt > > from vw where asofdate='2017-10-02D' > > group by fundid > > > > select fundid,sum(curdlttot) as fxt > > from vw where asofdate= date_to_str(asofdate, '%Y-%m-%d') = > '2017-10-02' > > group by fundid > > > > > > the followings query on the subtable of the merge table returns data > with no issues or performance > > ---------------------------------------------------------------------- > ----------------------- > > select fundid,sum(curdlttot) as fxt > > from currentyear where asofdate='2017-10-02' > > group by fundid > > > > creating a view on the union of all three tables returns data as > expected > > > > create view test_all as > > select * from intraday > > union all > > select * from history > > union all > > select * from currentyear > > > > > > select fundid,sum(curdlttot) as fxt > > from test_all where asofdate='2017-10-02' > > group by fundid > > > > > > > > > > > > Warren S. Master > > Chief Technology Officer, CISO > > TRG Management LP > > 280 Park Avenue - 30th floor/West - New York, NY 10017 > > Phone 212.984.3504 > > Email warren.master(a)rohatyngroup.com > > Web www.rohatyngroup.com > > Linkedin www.linkedin.com/in/warrenmaster > > > > This e-mail communication and any attachments may contain confidential > and privileged information and are for use by the intended > addressee(s) only. If you are not the intended addressee, you have > received this communication in error. Any dissemination or > reproduction of this email or its contents is strictly prohibited and > may be unlawful. If you suspect you have received this communication > in error, please notify us immediately by replying to this message and > delete it from your computer.Thank you. > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list -- Niels Nes, Manager ITF, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: https://www.cwi.nl/people/716e-mail: Niels.Nes(a)cwi.nl ------------------------------ Subject: Digest Footer _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list ------------------------------ End of users-list Digest, Vol 62, Issue 5 ***************************************** This e-mail communication and any attachments may contain confidential and privileged information and are for use by the intended addressee(s) only. If you are not the intended addressee, you have received this communication in error. Any dissemination or reproduction of this email or its contents is strictly prohibited and may be unlawful. If you suspect you have received this communication in error, please notify us immediately by replying to this message and delete it from your computer.Thank you.

1 0

Merge Table Issues
by Warren Master 12 Oct '17

12 Oct '17

all_vw is a merge table with three tables all with identical structure intraday history currentyear --------------------------------------------------------------------------------------------- My merge tables do not return consistent results I have tried to refresh statistics ANALYZE sys. intraday (asofdate) MINMAX ANALYZE sys. history (asofdate) MINMAX ANALYZE sys. currentyear (asofdate) MINMAX still same issue the issue exists for a few dates that I checked, there are many days that do work on the merge table , all dates work on the view and the individual tables. I thought it might be an issue with statistics then the conversion function made it work on the merge table , so now at a loss. the following query on the merge table return no rows --------------------------------------------------------------------------------------------- select fundid,sum(curdlttot) as fxt from vw where asofdate='2017-10-02' group by fundid The following does return data from the merge table albeit, much slower - after converting date value --------------------------------------------------------------------------------------------- select fundid,sum(curdlttot) as fxt from vw where asofdate='2017-10-02D' group by fundid select fundid,sum(curdlttot) as fxt from vw where asofdate= date_to_str(asofdate, '%Y-%m-%d') = '2017-10-02' group by fundid the followings query on the subtable of the merge table returns data with no issues or performance --------------------------------------------------------------------------------------------- select fundid,sum(curdlttot) as fxt from currentyear where asofdate='2017-10-02' group by fundid creating a view on the union of all three tables returns data as expected create view test_all as select * from intraday union all select * from history union all select * from currentyear select fundid,sum(curdlttot) as fxt from test_all where asofdate='2017-10-02' group by fundid Warren S. Master Chief Technology Officer, CISO TRG Management LP 280 Park Avenue - 30th floor/West - New York, NY 10017 Phone 212.984.3504 Email warren.master(a)rohatyngroup.com<mailto:warren.master@rohatyngroup.com> Web www.rohatyngroup.com<http://www.rohatyngroup.com/> Linkedin www.linkedin.com/in/warrenmaster<http://www.linkedin.com/in/warrenmaster> This e-mail communication and any attachments may contain confidential and privileged information and are for use by the intended addressee(s) only. If you are not the intended addressee, you have received this communication in error. Any dissemination or reproduction of this email or its contents is strictly prohibited and may be unlawful. If you suspect you have received this communication in error, please notify us immediately by replying to this message and delete it from your computer.Thank you.

2 1

mal function returns clob datatype
by Gatis Ozoliņš 10 Oct '17

10 Oct '17

I have string aggregation mal function with this definition: function group_concat(b:bat[:str],s:bat[:str]):str; It was working fine on Dec 2016-SP3, however we upgraded to Jul 2017-SP1 and now function is returning clob datatype. I looked at Changelog and code changes and couldn't find anything that would explain this. Is it feature or bug? Gatis

1 0

make[9]: *** No rule to make target `../../../../sql/backends/monet5/lib_sql.la', needed by `lib_myudfname.la'. Stop.
by Anton Kravchenko 09 Oct '17

09 Oct '17

Hi there, I will try to map my issue to something that you are familiar with. For example "MonetDB-11.23.3/sql/backends/monet5/vaults" module. 1) It has Makefile.ag which has (among other lines): lib__vault = { MODULE DIR = libdir/monetdb5 SOURCES = vault.c LIBS = ../../../../monetdb5/tools/libmonetdb5 \ ../../../../gdk/libbat $(curl_LIBS) } And that means that "libmonetdb.la" has to be created before "vaults" is built. 2) Let's say I have added "../../../../sql/backends/monet5/lib_sql" dependency lib__vault = { MODULE DIR = libdir/monetdb5 SOURCES = vault.c LIBS = ../../../../monetdb5/tools/libmonetdb5 \ * ../../../../sql/backends/monet5/lib_sql \* ../../../../gdk/libbat $(curl_LIBS) } As you can see "vaults" is leaving inside "MonetDB-11.23.3/sql/backends/monet5" directory. 3) Now my question is: How can one propagate into MonetDB-11.23.3/sql/backends/monet5/Makefile.am that "lib_sql.la" has to be created before "vaults" is built? p.s. On Windows, there is MonetDB-11.23.3/sql/backends/monet5/Makefile.msc used (instead Makefile.am). And I would just change order in "all-recursive all-msc" to "all-msc all-recursive": # build-all: $(BUILT_SOURCES) all-recursive all-msc build-all: $(BUILT_SOURCES) all-msc all-recursive Thank you, Anton

2 4

Error starting monetdb
by Corbit, Dann 06 Oct '17

06 Oct '17

When M5server.bat runs, it outputs this message: !FATAL: BBPinit: first != 0 (ID = 72). And exits. I do not have any data I need to save in my database. Is there a way to drop the old database and create a new one, so that I can again run tests?

1 0

How to improve query performance on a single node in MonetDB?
by Tomas Repik | Instarea 04 Oct '17

04 Oct '17

Hello list, recently I came across MonetDB as a columnar database and I have some issues with getting it perform well. I posted my question on StackOverflow <https://stackoverflow.com/questions/46558842/how-to-improve-query-performan…>, idealy post your answers there but, mailing it to the list is fine as well. Thank you Tomas > TL;DR > I've installed the latest (MonetDB 5 server v11v.27.5 "Jul2017-SP1") on > Windows 2012 Server and I'm trying to query large table 1,4 billion rows in > a reasonable time 2-3s. > Is this even possible with MonetDB? What could I do to improve the > performance? > Detailed description of what I've done so far: > Created table: > CREATE TABLE t939ba ( id INT, xa INT, xb INT, ya INT, yb INT, a1 TINYINT, > a2 TINYINT, a3 TINYINT, a4 TINYINT, a5 TINYINT, a6 TINYINT, a7 TINYINT, a8 > TINYINT, a9 TINYINT); > Loaded the data: > COPY 1450000000 OFFSET 2 RECORDS INTO tbl FROM 'D:\\es_export\\file.csv' > USING DELIMITERS ',' NULL AS '' LOCKED; > Run the query: > SELECT COUNT(DISTINCT id) FROM tbl WHERE a1=22 > AND xb>=143455 AND yb>=90911 AND xa<=143615 AND ya<=91007 > AND a2 IN (2, 3, 4) AND a3 IN (0, 1, 2, 3, 4) AND a4 IN (0, 1, 2) > AND a5 IN (-1, 1, 2, 3, 4, 5, 6, 7) AND a6 IN (-1, 11, 12, 13, 14); > It took *14m 52s* to get the result.

3 2

build error MonetDBv11.27.5 on Linux
by Anton Kravchenko 19 Sep '17

19 Sep '17

Hi there, I am able to build and install MonetDB v11.21.19 "Jul2015-SP4" by following and example from https://www.monetdb.org/wiki/MonetDB:Building_from_sources But when I try build MonetDB v11.27.5 on same machine I get this error (for a full log info see https://drive.google.com/drive/folders/0B7dMycp0Ku-NTlRsZW95R0xCNmM?usp=sha… ): /bin/sh ../../../../../libtool --tag=CC --mode=link gcc -DLIBPYAPI -g -O2 -D_REENTRANT -module -avoid-version -o lib_pyapi.la -rpath /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib/monetdb5 lib_pyapi_la-pyapi.lo lib_pyapi_la-unicode.lo lib_pyapi_la-pytypes.lo lib_pyapi_la-type_conversion.lo lib_pyapi_la-formatinput.lo lib_pyapi_la-connection.lo lib_pyapi_la-pyloader.lo lib_pyapi_la-emit.lo lib_pyapi_la-conversion.lo ../../../../../monetdb5/tools/libmonetdb5.la ../../../../../gdk/libbat.la -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic libtool: link: gcc -shared -fPIC -DPIC .libs/lib_pyapi_la-pyapi.o .libs/lib_pyapi_la-unicode.o .libs/lib_pyapi_la-pytypes.o .libs/lib_pyapi_la-type_conversion.o .libs/lib_pyapi_la-formatinput.o .libs/lib_pyapi_la-connection.o .libs/lib_pyapi_la-pyloader.o .libs/lib_pyapi_la-emit.o .libs/lib_pyapi_la-conversion.o -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/monetdb5/tools/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs ../../../../../monetdb5/tools/.libs/libmonetdb5.so /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs/libbat.so /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/ clients/mapilib/.libs/libmapi.so -lpcre -lxml2 ../../../../../gdk/.libs/libbat.so /home/SECDEV.LOCAL/akravchenko /MonetDB-Jul2017-SP1/BUILD/common/stream/.libs/libstream.so -lsnappy -lssl -lcrypto -lz -lpython2.7 -lpthread -ldl -lutil -lm -g -O2 -Wl,-export-dynamic -pthread -Wl,-soname -Wl,lib_pyapi.so -o .libs/lib_pyapi.so /usr/bin/ld: cannot find -lpython2.7 collect2: ld returned 1 exit status make[11]: *** [lib_pyapi.la] Error 1 make[11]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi' make[10]: *** [all] Error 2 make[10]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi' make[9]: *** [all-recursive] Error 1 make[9]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF' make[8]: *** [all] Error 2 make[8]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF' make[7]: *** [all-recursive] Error 1 make[7]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5' make[6]: *** [all] Error 2 make[6]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends' make[4]: *** [all] Error 2 make[4]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD' make: *** [all] Error 2 Thank you, Anton

2 4

Embedded Python is enabled but an error was thrown during initialization.
by Anton Kravchenko 15 Sep '17

15 Sep '17

Hi there, sql>create or replace function f1() returns table(v1 INT) more>LANGUAGE PYTHON { more> return [1,2,3] more>}; operation successful (3.658ms) sql>select * from f1(); Embedded Python is enabled but an error was thrown during initialization. Does one have to setup another python env variables (like it's done on MonetDB for Windows: PYTHONHOME, PYTHONPATH) on Linux? p.s. Here are is how I setup environment 1) installed MonetDB v11.27.5 (Jul2017-SP1) binaries on centos-release-7-4.1708.el7.centos.x86_64: sudo yum install https://dev.monetdb.org/downloads/epel/MonetDB-release-epel.noarch.rpm sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-stream-11.27.5-2017… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-client-11.27.5-2017… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB5-server-11.27.5-201… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-SQL-server5-11.27.5… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-python2-11.27.5-201… 2) installed Anaconda2 Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 export PATH="/home/akravchenko/anaconda2/bin:$PATH" 3) created and started Monet database: monetdbd create hnode1_dbfarm monetdbd set port=50001 hnode1_dbfarm monetdbd set control=yes hnode1_dbfarm monetdbd set passphrase=monetdb hnode1_dbfarm monetdbd start hnode1_dbfarm monetdb create hnode1_dbfarm_db1 *monetdb set embedpy=true hnode1_dbfarm_db1* monetdb start hnode1_dbfarm_db1 monetdb release hnode1_dbfarm_db1 4)monetdbd get all hnode1_dbfarm property value hostname somehostname dbfarm hnode1_dbfarm status monetdbd[31089] 1.7 (Jul2017-SP1) is serving this dbfarm mserver /usr/bin/mserver5 logfile hnode1_dbfarm/merovingian.log pidfile hnode1_dbfarm/merovingian.pid sockdir /tmp listenaddr localhost port 50001 exittimeout 60 forward proxy discovery yes discoveryttl 600 control yes passphrase {SHA512}a73f1d86383446438ac64f56e15ada38b41fbb18f029d2181723aeb2acac6a831f60e5fdbd64ac2c8c70e035dd44cbbe3b45565ef2d58feb2821a2078c7fad35 mapisock /tmp/.s.monetdb.50001 controlsock /tmp/.s.merovingian.50001 5) monetdb get all hnode1_dbfarm_db1 name prop source value hnode1_dbfarm_db1 name - hnode1_dbfarm_db1 hnode1_dbfarm_db1 type default database hnode1_dbfarm_db1 shared default yes hnode1_dbfarm_db1 nthreads default 32 hnode1_dbfarm_db1 optpipe default default_pipe hnode1_dbfarm_db1 readonly default no hnode1_dbfarm_db1 embedr default no hnode1_dbfarm_db1 embedpy local yes hnode1_dbfarm_db1 embedpy3 default no hnode1_dbfarm_db1 nclients default 64 hnode1_dbfarm_db1 dbextra default <unknown> Thank you, Anton

1 2

SQL_CALC_FOUND_ROWS equivalent?
by Alberto Ferrari 15 Sep '17

15 Sep '17

Hi, is there any function or trick in Monet equivalent to mysql "SELECT SQL_CALC_FOUND_ROWS ..." to get de total rows when limit is present ? ((pagination) So I could get de TOTAL rows/pages number without exec another query everytime I need paginate results. Thanks!

2 2

Re: Column rename issue
by Stefan Manegold 15 Sep '17

15 Sep '17

Hi, Add new column, copy/cast data from old to new column, drop old column. Stefan -------- Original message --------From: Amit Ambekar <amit(a)g-square.in> Date: 9/15/17 12:14 (GMT+01:00) To: users-list(a)monetdb.org Subject: Column rename issue Hi, how to update already created column name and type in monetdb ? _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

Column rename issue
by Amit Ambekar 15 Sep '17

15 Sep '17

Hi, how to update already created column name and type in monetdb ?

1 0

large scale hardware configuration for MonetDB
by Anton Kravchenko 06 Sep '17

06 Sep '17

Hi there, >From https://www.monetdbsolutions.com/news/monetdb-numascale : "The system is fine-tuned to exploit the large main memories of modern computer systems effectively and has proven to be highly efficient in data warehouse query processing. NumaQ can easily scale to thousands of cores and terabytes of memory for data-intensive." I wonder if there is a public documentation on how to configure large scale hardware for optimal MonetDB performance. So one could try setup NumaQ-like machine by using what you already learned. Also from https://stackoverflow.com/questions/7435235/is-it-worth-trying-monetdb : "MonetDB is developed/tuned under two main assumptions: 1. Your workload is analytical, i.e., you have lots of (grouped) aggregations and the like. 2. Even more important: your hot dataset (the data that you actually work with) fits into the main memory of your system. MonetDB does not have it's own Buffer Manager but relies on the OS to handle disk I/O. Since the OS (especially windows but Linux too) is sometimes very dumb about disk swapping that may become a problem (especially for joins that run out of memory)." Are there any official benchmarks for Windows vs Linux MonetDB performance (especially for queries that require disk swapping) ? Thank you, Anton

2 1

default_pipe optimizer slowness during concurrency
by Sharma, Sreejith 06 Sep '17

06 Sep '17

Hi, I have a scenario where if I run the a SQL with 10 user concurrency in a recursive mode, the execution time is significantly slower. The SQL which completes in .8sec will end up in completing 5-10hrs during concurrency. This behavior is seen when the optimizer is set to "default_pipe" , however when set to "sequential_pipe" the timing has been drastically reduced and it end ups in seconds. How is "sequential_pipe" optimizer works here, is it to do anything with concurrency. NOTE - this happens when there is DISTINCT pattern in SQL. Regards, Sreejith

2 1

Autoload scripts from MonetDB/sql/scripts
by Lefteris 05 Sep '17

05 Sep '17

After playing around with some udfs and such, it seems to me that the autoload feature of the scripts in directory MonetDB/sql/scripts only run if I start monetdb with the following command mserver5 --dbpath=... and then when I connect with mclient all these messages about loading the scripts appear at the mserver5 console. However, if I create a dbfarm with monetdbd create path/ monetdbd start path/ and then a database monetdb create dbname monetdb release dbname and connect with mclient -lsql -d dbname all the goodies from the script dir do not appear to be there. Is there another trigger for this case that runs autoload? thanx

3 5

How to get Memory Dump. Is there any way to eliminate Memory leaks.
by Sharma, Sreejith 05 Sep '17

05 Sep '17

All, Is there any way to get the memory dump from MonetDB? The memory consumed by merver5 post concurrent query execution are not coming down. Need to investigate why memory leak is happening here. Regards, Sreejith

2 2

Monetdb on windows with embedded python 3.6
by imad hajj chahine 05 Sep '17

05 Sep '17

Hi, I have been successfully using monetdb (11.25.3) on windows with embedded python 2.7, recently I needed a library that is only supported on python 3.6, is it possible somehow to point monetdb to the python 3.6 folder? I know this is possible on linux, but my dev environment is on windows. Thank you.

3 6

Defining a table with a sorted column
by Cota, Luis 05 Sep '17

05 Sep '17

Is it possible to define a table such that the server maintains the table sorted with each new row? I'm dealing with observations on market data, and while market data for a specific stock symbol will be sorted, I will be adding multiple symbols to the same table in chunks, so that each symbol is contained within its own load file, and is sorted within that file. How can I enforce this constraint? - Luis ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

2 1

Function with table as argument that return a table
by imad hajj chahine 05 Sep '17

05 Sep '17

Hi, I want to create a Python function that accept a query as argument, do some processing and return another table, In addition to the table columns I want to have some additional scalar parameters and i am getting the following error: function requires a single sub query. CREATE function f1(col1 string, col2 int, col3 double, scalar1 int) RETURNS TABLE(col1 string, col2 double, col3 double, col4 double) LANGUAGE PYTHON Is this the normal behavior of Table function, as in aggregate function I can combine columns with scalar parameters, any workaround for this issue. Thank you.

2 3

COPY INTO from LZ4 file: bug or missing feature?
by Ying Zhang 04 Sep '17

04 Sep '17

Dear all, Is COPY INTO from an LZ4 file supported by MonetDB? When I try to load data from an LZ4 file, I always get the error: Failed to import table 'pt', record too long (EOS found) Is this an error or a missing feature? Additional info: - MonetDB version Jul2017 (Mercurial source) - Mac OS (Sierra) - I checked the config.log that my lz4 lib was found - The LZ4 file is valid (I decompressed it, then I could load the data). Thanks in advance! Jennie

1 1

Execute MAL queries
by Dennis Fassl 04 Sep '17

04 Sep '17

Dear all, I am trying to write and execute MAL queries. The overall goal is to translate Cypher to MAL and to use Cypher as an input language for MonetDB. I am using the "explain" statement to get some example MAL queries. This is an output of explain (without the optimizer stuff at the end): function user.s4_1():void; X_0:void := querylog.define("explain select source, target from graph where source = \\'A\\';","default_pipe",14:int); X_30 := bat.new(nil:str); X_36 := bat.new(nil:int); X_34 := bat.new(nil:int); X_33 := bat.new(nil:str); X_32 := bat.new(nil:str); X_3 := sql.mvc(); X_7:bat[:str] := sql.bind(X_3,"sys","graph","source",0:int); C_4:bat[:oid] := sql.tid(X_3,"sys","graph"); C_18 := algebra.thetasubselect(X_7,C_4,"A","=="); X_21:bat[:str] := sql.bind(X_3,"sys","graph","target",0:int); X_28 := algebra.projection(C_18,X_21); X_20 := algebra.projection(C_18,X_7); X_37 := bat.append(X_30,"sys.graph"); X_39 := bat.append(X_32,"source"); X_40 := bat.append(X_33,"varchar"); X_42 := bat.append(X_34,31:int); X_44 := bat.append(X_36,0:int); X_46 := bat.append(X_37,"sys.graph"); X_47 := bat.append(X_39,"target"); X_48 := bat.append(X_40,"varchar"); X_49 := bat.append(X_42,31:int); X_50 := bat.append(X_44,0:int); sql.resultSet(X_46,X_47,X_48,X_49,X_50,X_20,X_28); end user.s4_1; But when I try to execute this query I get errors like that: ERROR = !TypeException:user.main[8]:'sql.tid' undefined in: C_4:bat[:oid] := sql.tid("sys":str,"sys":str,"graph":str); I don't know why the "X_3" is replaced by "sys". I see changes like that in the queries regularly. Does someone know what is wrong or missing in this query? Thanks a lot in advance! Best regards, Dennis

4 5

Possible bug with table generating functions
by Mark Clements 29 Aug '17

29 Aug '17

Is there a restriction on the use of the same table generating function in the same select statement? This relates to the following possible bug: -- reimplement generate_series for MonetDBLite CREATE FUNCTION my_generate_series(start INT, finish INT) RETURNS TABLE (value INT) BEGIN DECLARE TABLE tmp_generate_series(value INT); DECLARE i INT; SET i = start; WHILE (i <= finish) DO INSERT INTO tmp_generate_series VALUES(i); SET i = i + 1; END WHILE; RETURN tmp_generate_series; END; --- This seems to work quite nicely: select * from my_generate_series(1,10); -- and it's fast: select count(*) from my_generate_series(1,100000); -- however, I get odd and non-deterministic counts for the following: select count(*) from my_generate_series(1,100) as t1, my_generate_series(1,100) as t2; This is using MonetDB v11.23.3 (Jun2016) that was compiled with embedded R, but not run with --set embedded_r=true. I got similar errors with MonetDBLite. Kindly, Mark.

2 2

Looking for Admin UI / Tools / Resources
by Cota, Luis 28 Aug '17

28 Aug '17

Can anyone recommend any tools (commercial or open source) which assist in administering a MonetDB instance? Things I'd like to do are: Create tables from existing data files (CSV, TXT, HDF5) Import or query Oracle databases and either create tables, or populate tables, with the results of those queries Query existing monetdb instances Thanks, Luis ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

1 0

Error after updating monetdb
by Mahmoud Mohsen 19 Aug '17

19 Aug '17

Hello, I have updated monetdb from Dec2016-SP5 to version v11.27.5 "Jul2017-SP1, but after this when trying to accessing the database via mclient -u monetdb -d tpch2 I am getting this error: internal error while starting mserver 'database 'tpch2' started up, but failed to open up a communication channel', please refer to the logs looking at the logs, 2017-08-18 22:42:02 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:02 MSG merovingian[10645]: database 'voc' (10660) has exited with exit status 0 2017-08-18 22:42:02 MSG merovingian[10645]: starting database 'voc', up min/avg/max: 0s/0s/0s, crash average: 0.00 0.00 0.00 (1-1=0) 2017-08-18 22:42:02 MSG merovingian[10645]: sending process 10663 (database 'voc') the TERM signal 2017-08-18 22:42:02 MSG voc[10663]: arguments: /usr/bin/mserver5 --dbpath=/home/mohsen/mfarm/voc --set merovingian_uri=mapi:monetdb://u145-16:50000/voc --set mapi_open=false --set mapi_port=0 --set mapi_usock=/home/mohsen/mfarm/voc/.mapi.sock --set monet_vault_key=/home/mohsen/mfarm/voc/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2017-08-18 22:42:02 MSG voc[10663]: # MonetDB 5 server v11.27.5 "Jul2017-SP1" 2017-08-18 22:42:02 MSG voc[10663]: # Serving database 'voc', using 8 threads 2017-08-18 22:42:02 MSG voc[10663]: # Compiled for x86_64-pc-linux-gnu/64bit with 128bit integers 2017-08-18 22:42:02 MSG voc[10663]: # Found 251.889 GiB available main-memory. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) 1993-July 2008 CWI. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved 2017-08-18 22:42:02 MSG voc[10663]: # Visit https://www.monetdb.org/ for further information 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:^LoaderException:loadLibrary:Loading error failed to open library sql (from within file '/usr/lib/monetdb5/lib_sql.so'): /usr/lib/monetdb5/lib_sql.so: undefined symbol: mnstr_writeFlt 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError: ^';' expected 2017-08-18 22:42:03 MSG merovingian[10645]: database 'voc' has shut down 2017-08-18 22:42:03 ERR control[10645]: !monetdbd: an internal error has occurred 'database 'voc' started up, but failed to open up a communication channel' 2017-08-18 22:42:07 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:07 MSG merovingian[10645]: database 'voc' (10663) has exited with exit status 0 Thank you a lot in advance, Best regards, Mahmoud Mohsen

2 3

Re: Re: failed to import record too long: Loading 1.5 Terabyte table
by Stefan Manegold 18 Aug '17

18 Aug '17

You need to use the correct record delimiter i.e. newline instead of space. -------- Original message --------From: Mahmoud Mohsen <mmohsen35(a)gmail.com> Date: 8/18/17 19:40 (GMT+01:00) To: users-list(a)monetdb.org Subject: Re: Re: failed to import record too long: Loading 1.5 Terabyte table Dear Stefan, Thank you very much for your quick reply, so this is the first line of the file "lineitem.tbl", so no need to use an offset I think 1|310378689|15378705|1|17|29786.89|0.04|0.02|N|O|1996-03-13|1996-02-12|1996-03-22|DELIVER IN PERSON|TRUCK|egular courts above the| as for the version, I am using: MonetDB Database Server Toolkit v1.1 (Dec2016-SP5). So can this may be the cause of the error... an old version? I am looking forward hearing from you, Best regards, Mahmoud Mohsen

2 2

failed to import record too long: Loading 1.5 Terabyte table
by Mahmoud Mohsen 18 Aug '17

18 Aug '17

Hello, I'm trying to load a table with 1.5 Terabytes of data, the command I'm using COPY INTO lineitem FROM '/vol1/tpch-dbgen/2000/lineitem.tbl' USING DELIMITERS '|', ' ' LOCKED; The physical memory I'm using is 256G, the swap is 2.2 terabytes, the dbfarm is located within area has 4 Terabytes of free storage. The message I'm getting: "Failed to import table 'lineitem', record too long" So what could be a possible solution to this problem, Thank you a lot in advance, Mahmoud Mohsen

3 3

(lastest version) monetdb source compilation and installation problem
by 张馨匀 17 Aug '17

17 Aug '17

Hi All, Could you please help me to solve the following error? When I try to install monetdb I have the following error: * fatal error: *geos_c.h: No such file or directory #include <geos_c.h> *My steps used to compile monetdb src and to install monetdb are:* a. Environment setup: $ export MonetDB_rel=July2017 $ export MonetDB_ver=11.27.5 $ export MonetDB_base_dir=$HOME/MonetDB-$MonetDB_rel $ export MonetDB_source_dir=$MonetDB_base_dir/MonetDB-$MonetDB_ver $ export MonetDB_build_dir=$MonetDB_base_dir/BUILD $ export MonetDB_prefix_dir=$MonetDB_base_dir $ mkdir $MonetDB_base_dir $ cd $MonetDB_base_dir b. Download the latest version: $ wget https://dev.monetdb.org/downloads/sources/Jul2017-SP1/MonetDB-11.27.5.tar.xz $ tar xf MonetDB-$MonetDB_ver.tar.xz c. Build: $ mkdir $MonetDB_build_dir $ cd $MonetDB_build_dir $ MonetDB_source_dir/configure --prefix=$MonetDB_prefix_dir d. Install: $ make -j 20 (error mentioned above occurs here) $ make -j 20 install Thanks,

2 1

Memory bloat on concurrency - Any fix to release memory post query run
by Sharma, Sreejith 11 Aug '17

11 Aug '17

Hi, I have been using monetdb (11.25.3) in CentOS for one of application. I have been working on a PSR testing for this application and have noticed on high concurrency and on same pattern of query running again and again causing memory to bloat up and doesn’t release once the query run completes. I understand that the data is lifted into memory and the result set is also cached, however I assume this should be release once the system is idle which I don’t see it as of now. Is there any way or parameter to release the unused memory blocks? I have asked this before, but couldn't get to the root to fix this. Regards, Sreejith

3 4

Troubles with Python API
by 倪令斌 09 Aug '17

09 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

2 3

Troubles with Python API
by 倪令斌 08 Aug '17

08 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

1 0

Mdiff outputs unexpected result
by Zhouyang Jia 05 Aug '17

05 Aug '17

Hi, I'm new to MonetDB, and I find that the component Mdiff would generate unexpected results if inputted wrong files, e.g., ./Mdiff wrong_file_1 wrong_file1. Mdiff will output that these two files are equal, whereas the expected result shoud be "files not exist". Attached please find the patch against version MonetDB-11.27.5. Hopefully, it can solve this problem. Best, Zhouyang

1 0

Announcement: New Jul2017-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 04 Aug '17

04 Aug '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP1 bugfix release Build Environment * The Debian and Ubuntu installers have been fixed: there was a file missing in the Jul2017 release. * Added a new RPM called MonetDB-selinux which provides the SELinux policy required to run MonetDB under systemd, especially on Fedora 26. * The Windows installers (*.msi files) are now created using the WiX Toolset. * The Windows binaries are now built using Visual Studio 2015. Because of this, you may need to install the Visual C++ Redistributable for Visual Studio 2015 before being able to run MonetDB. Merovingian * monetdbd was leaking open file descriptors to the mserver5 process it started. This has been fixed. MonetDB Common * Many functions in GDK are now annotated with the GCC attribute __warn_unused_result__ meaning that the compiler will issue a warning if the result of the function (usually an indication of an error) is not used. Bug Fixes * 6325: Merge table unusable in other connections * 6328: Transactional/multi-connection issues with merge tables * 6336: VALUES multiple inserts error * 6339: Mserver5 crashes on nested SELECT * 6340: sample operator takes effect after the execution of the query, expected before * 6341: MERGE TABLE issue: Cannot register * 6342: MERGE TABLE issue: hang * 6344: Spurious errors and assertions (SQLsmith) * 6375: MAL profiler truncates JSON objects larger than 8192 characters

3 2

Error during installation (make)
by Dennis Fassl 31 Jul '17

31 Jul '17

Hi, I want to install MonetDB from the source files. (Version 11.27.1). I am following the guide "HowToStart.rst" in the source directory. But I get an error when executing make. Please find the output below. Do you have a hint for me how to solve this error? Many thanks and kind regards, Dennis make all-recursive make[1]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir' Making all in buildtools make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' Making all in conf make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools/conf' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/buildtools' Making all in common make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common' Making all in stream make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/stream' Making all in options make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/options' Making all in utils make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common/utils' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/common' Making all in clients make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' Making all in mapilib make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapilib' Making all in mapiclient make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/mapiclient' Making all in examples make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' Making all in C make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/C' Making all in php make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/php' Making all in perl make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples/perl' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients/examples' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/clients' Making all in gdk make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' make all-am make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/gdk' Making all in monetdb5 make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' Making all in mal make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/mal' Making all in modules make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' Making all in atoms make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/atoms' Making all in kernel make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/kernel' Making all in mal make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules/mal' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/modules' Making all in optimizer make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/optimizer' Making all in scheduler make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/scheduler' Making all in tools make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/tools' Making all in extras make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' Making all in mal_optimizer_template make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/mal_optimizer_template' Making all in rapi make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras/rapi' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5/extras' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/monetdb5' Making all in sql make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' Making all in include make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/include' Making all in common make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/common' Making all in storage make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' Making all in bat make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage/bat' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/storage' Making all in server make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/server' Making all in backends make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' Making all in monet5 make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make all-recursive make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' Making all in vaults make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make all-recursive make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[10]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[10]: Nothing to be done for `all-am'. make[10]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/vaults' Making all in UDF make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make all-recursive make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' Making all in cudf make[10]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make all-am make[11]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make[11]: Nothing to be done for `all-am'. make[11]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make[10]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF/cudf' make[10]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make[10]: Nothing to be done for `all-am'. make[10]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/UDF' Making all in LSST make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' make all-am make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' make[9]: Nothing to be done for `all-am'. make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/LSST' Making all in generator make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make all-am make[9]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make[9]: Nothing to be done for `all-am'. make[9]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5/generator' make[8]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[8]: Nothing to be done for `all-am'. make[8]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends/monet5' make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/backends' Making all in scripts make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql/scripts' make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/sql' Making all in tools make[2]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' make all-recursive make[3]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' Making all in mserver make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' make all-am make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/mserver' Making all in merovingian make[4]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' make all-recursive make[5]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' Making all in utils make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/utils' Making all in client make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/client' Making all in daemon make[6]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' make all-am make[7]: Entering directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' /bin/bash ../../../libtool --tag=CC --mode=link gcc -g -O2 -export-dynamic -o monetdbd merovingian.o argvcmds.o client.o connections.o controlrunner.o discoveryrunner.o forkmserver.o handlers.o multiplex-funnel.o proxy.o ../utils/libmeroutil.la ../../../common/utils/libmutils.la ../../../monetdb5/tools/libmonetdb5.la -lssl -lcrypto -pthread libtool: link: gcc -g -O2 -o .libs/monetdbd merovingian.o argvcmds.o client.o connections.o controlrunner.o discoveryrunner.o forkmserver.o handlers.o multiplex-funnel.o proxy.o -pthread -Wl,--export-dynamic ../utils/.libs/libmeroutil.a ../../../common/utils/.libs/libmutils.a ../../../monetdb5/tools/.libs/libmonetdb5.so -lssl -lcrypto -pthread /usr/bin/ld: controlrunner.o: undefined reference to symbol 'GDKinit' //home/dennis/MonetDB-11.27.1/buildDir/gdk/.libs/libbat.so.15: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make[7]: *** [monetdbd] Error 1 make[7]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' make[6]: *** [all] Error 2 make[6]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian/daemon' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' make[4]: *** [all] Error 2 make[4]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools/merovingian' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir/tools' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/dennis/MonetDB-11.27.1/buildDir' make: *** [all] Error 2

2 1

C (or Java) API to insert blob via prep statement
by Anton Kravchenko 26 Jul '17

26 Jul '17

Is blob insertion (via prepared statement) supported by C (or Java) MonetDB API as of now?

1 0

COPY INTO doesn't recognise sorted varchar column
by Sridhar Eaturu 25 Jul '17

25 Jul '17

Hi, I imported some csv data using COPY INTO to a table. The data is sorted by the first column "sym", which is of VARCHAR type. Upon importing, the COPY INTO is not recognising the sortedness of the "sym" column. On the other hand, another test I ran with a sorted DATE column gets marked as sorted as expected. I'm using MonetDB v11.25.23 (Dec2016-SP5), and the steps demonstrating this are below. Is this expected? sql>CREATE TABLE tempquote (sym VARCHAR(3),time0 TIME,cond1 VARCHAR(1),bid INTEGER,bidqty INTEGER,ask INTEGER,askqty INTEGER,cond2 VARCHAR(1)); operation successful (2.430ms) sql>COPY INTO tempquote from 'tempquote.csv' USING DELIMITERS ',','\n' NULL AS ''; 5 affected rows (31.972ms) sql>select * from tempquote; +------+----------+-------+------+--------+------+--------+-------+ | sym | time0 | cond1 | bid | bidqty | ask | askqty | cond2 | +======+==========+=======+======+========+======+========+=======+ | ABY | 09:30:03 | F | 1489 | 50 | 1872 | 44 | I | | ACG | 09:30:00 | M | 1419 | 27 | 1542 | 43 | A | | AHK | 09:30:00 | G | 1094 | 74 | 1808 | 56 | F | | AXN | 09:30:01 | I | 1022 | 26 | 1541 | 88 | E | | BEV | 09:30:00 | L | 1377 | 38 | 1353 | 77 | M | +------+----------+-------+------+--------+------+--------+-------+ 5 tuples (1.131ms) sql>select table,column,sorted from storage() where "table"='tempquote'; +-----------+--------+--------+ | table | column | sorted | +===========+========+========+ | tempquote | sym | null | | tempquote | time0 | null | | tempquote | cond1 | null | | tempquote | bid | null | | tempquote | bidqty | null | | tempquote | ask | null | | tempquote | askqty | null | | tempquote | cond2 | null | +-----------+--------+--------+ 8 tuples (5.090ms) sql> Thanks, Sridhar

1 0

Is there any parameter to release memory? I see memory bloats on high concurrency and continuous runs
by Sharma, Sreejith 25 Jul '17

25 Jul '17

Hi, Is there any parameter or any options available for MonetDB to release the memory consumed? As one of the application I tested would run on high concurrency and may run SQL overnight. In such cases I see a memory bloat which may come up to 200gb and will never come down. Please advise? Regards, Sreejith

2 3

Troubles with the MonetDB
by 倪令斌 25 Jul '17

25 Jul '17

Dear Sir/Madam， I have encountered a problem that it has troubled me for several days and I can not continue the next job.I found it online for a long time, but I did not find a solution.Hope you can give me some inspiration: what's the matter with my PC? PS:The OS is Ubuntu16.04. Thanks a lot! Cheers, Lingbin

2 2

Copy into with DEFAULT values
by Ariel Abadi 24 Jul '17

24 Jul '17

Hi Im trying to upload a FILE into one table. The FILE has all the fields of the table, except for the last field which is the *updated *time. For that reason I have created the table with the field: "nrm_now" TIMESTAMP(0) DEFAULT current_timestamp() BUT when im trying to insert the FILE (COY INTO test_20170711 from FILE .... ), there is an error: Failed to import table 'test_20170711', Column value 143 missing Does anybody can help me? Am I doing something wrong? Which should be the best way to implement this? Thnks in advance Ariel

2 2

Monetdb Installation
by Amit Ambekar 24 Jul '17

24 Jul '17

Hi, July 2017 release is not working good . I am getting error while login using terminal in Ubuntu OS. When new release will be done ??

2 1

Bug on timestamp ?
by Ariel Abadi 21 Jul '17

21 Jul '17

Hi! May be someone can help me, Im facing the following problem. Im trying to pull some information from the field nrm_connect_time, I tried almost all the date/timestamp functions, but all of them failed. Finally, I decided to CAST as STRING, and see what happends... If I do a SELECT DISTINCT .. then I have the correct information, if I dont include the DISTINCT what I see has nonsense... sql>SELECT nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10) FROM NRM_20170609 LIMIT 5; +----------------------+-----------------------+ | nrm_connect_time | left_nrm_connect_time | +======================+=======================+ | 2017-06-09 00:05:00 | -1-01-01 0 | | 2017-06-09 00:05:00 | -1-01-01 0 | | 2017-06-09 00:05:01 | -1-01-01 0 | | 2017-06-09 00:05:03 | -1-01-01 0 | | 2017-06-09 00:05:03 | -1-01-01 0 | +----------------------+-----------------------+ 5 tuples (29.802ms) sql>*SELECT DISTINCT* nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10) FROM NRM_20170609 LIMIT 5; +----------------------+------------+ | nrm_connect_time | L2 | +======================+============+ | 2017-06-09 00:05:00 | 2017-06-09 | | 2017-06-09 00:05:01 | 2017-06-09 | | 2017-06-09 00:05:03 | 2017-06-09 | | 2017-06-09 00:05:04 | 2017-06-09 | | 2017-06-09 00:05:05 | 2017-06-09 | +----------------------+------------+ 5 tuples (424.306ms) *The "nrm_connect_time" is defined as TIMESTAMP(0).* Let me show you a differente example: sql>*SELECT DISTINCT *nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10), timestamp_to_str(nrm_connect_time, '%Y') FROM NRM_20170609 LIMIT 5; +----------------------+------------+------+ | nrm_connect_time | L2 | L3 | +======================+============+======+ | 2017-06-09 00:05:00 | 2017-06-09 | 2017 | | 2017-06-09 00:05:01 | 2017-06-09 | 2017 | | 2017-06-09 00:05:03 | 2017-06-09 | 2017 | | 2017-06-09 00:05:04 | 2017-06-09 | 2017 | | 2017-06-09 00:05:05 | 2017-06-09 | 2017 | +----------------------+------------+------+ 5 tuples (707.217ms) sql>*SELECT *nrm_connect_time, LEFT(CAST(nrm_connect_time AS STRING),10), timestamp_to_str(nrm_connect_time, '%Y') FROM NRM_20170609 LIMIT 5; TypeException:user.s16_55[123]:'mtime.timestamp_to_str' undefined in: mtime.timestamp_to_str(X_196:oid,A1:str); program contains errors Regards Ariel

3 3

Re: Re: Add entry in a table only if that entry does not already exist
by Stefano Piani 21 Jul '17

21 Jul '17

Hello, sorry for my late replay, I have been really busy in the last few days. > Hello Stefano, > > Which MonetDB version do you use? On which OS? Sorry, I forgot to mention this. I am using ubuntu 16.04 and I am running MonetDB inside a Docker container (which also runs ubuntu). >> On 4 Jul 2017, at 16:32, Stefano Piani <stefano.piani(a)exact-lab.it> wrote: >> >> Dear monetdb users, >> >> I started to use MonetDB a few weeks ago and, until now, I have been >> able to use it proficiently. Unfortunately, I have faced a small issue >> that I have not been able to figure out how to fix. >> >> I collect data generated by different users. A unique id is associated >> to each user. I have a specific table (called "user") where I save the >> name and some other information about the user. In this table, the ID >> is a primary key. >> When I collect the data, I want to check if the specific user who >> generated that data is already saved in the user table. If this is not >> the case, >> I want to add a row to the user table for this user. >> Unfortunately, I have not been able to accomplish this operation in >> one SQL query and, therefore, I have been forced to query for the user >> id and, >> if the answer is negative, add the user using another query. The >> following is the python script that perform the operations. >> >> def save_user_in_database(user, cursor): >> db_line = 'SELECT 1 FROM users WHERE user_id = {}'.format(user.id) >> LOGGER.info('Executing on DB: %s', db_line) >> cursor.execute(db_line) >> >> answer = cursor.fetchall() >> LOGGER.info('Get this answer %s', answer) >> if len(answer) == 0: >> LOGGER.info('Inserting new user') >> >> db_line = sql_cmd_from_template( >> USER_TABLE_INSERT_TEMPLATE, >> user.id, >> user.name, >> user.description, >> user.location >> ) >> LOGGER.info('Executing on DB: %s', db_line) >> cursor.execute(db_line) > >Do you execute the queries in a different schema than the default ?sys? schema (which is recommended for database users)? >Because ?users? is a special VIEW in the ?sys? schema. > Yes, I should be in a different schema. In any case, I changed the name of the table just to be on the safe way. >> >> This function is executed inside a single threaded application. >> Moreover, there are no "DELETE" or similar operations inside this >> script (it only adds rows to the database). >> No user (human or software) but this script connects to my MonetDB instance. >> >> Unfortunately, after I while that the script is running (let's say one >> or two days), something unexpected happens. For example, this is the >> log generated for user 42847029 >> (there are a lot of other lines in between that I removed). As you can >> see, after a while, my check fails and I don't understand why. >> >> 2017/06/13 09:30:01 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 09:30:01 - INFO - save_user_in_database: Get this answer [(1,)] >> >> 2017/06/13 10:00:04 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 10:00:04 - INFO - save_user_in_database: Get this answer [(1,)] >> >> 2017/06/13 10:02:33 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 10:02:33 - INFO - save_user_in_database: Get this answer [(1,)] >> >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Executing on DB: >> SELECT 1 FROM users WHERE user_id = 42847029 >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Get this answer [] >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Inserting new user > >This shouldn?t happen, as is shown by the error below that a user_id of 42847029 still exists; also, as you mentioned, there hasn?t been any delete queries. > >This might be a bug in MonetDB. > >Is this problem repeatable? If yes, can you give us a sequence of queries to reproduce this problem. We?d like to look into it. > I will try to reproduce the problem with some scripts as soon as possible. Unfortunately, I am quite busy in this period and therefore this will require some time. In any case, this happens only when the system is busy and the table is big enough. I think that the key to reproduce the problem is to stress the database with several operations per second. In this situation, I have noticed that the db starts to use a lot of RAM (about 15 GB for a database that occupies 3 GB on the disk). At that point, the db becomes extremely slow (for example, for a query that usually takes less than 10 ms I have to wait for more than 2 seconds) and the unique key problem start to appear. > >> 2017/06/13 10:30:36 - INFO - save_user_in_database: Executing on DB: >> INSERT INTO users(user_id, user_name, user_description, user_location) >> VALUES (42847029, 'XXXX', 'XXXX', 'XXXX') >> 2017/06/13 10:30:36 - WARNING - on_data: Error elaborating data >> Traceback (most recent call last): >> File "data_collector.py", line 267, in on_data >> return self.elaborate_data(data) >> File "data_collector.py", line 354, in elaborate_data >> save_user_in_database(user, cursor) >> File "data_collector.py", line 234, in save_user_in_database >> cursor.execute(db_line) >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/sql/cursors.py", >> line 164, in execute >> block = self.connection.execute(query) >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/sql/connections.py", >> line 133, in execute >> return self.command('s' + query + ';') >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/sql/connections.py", >> line 138, in command >> return self.mapi.cmd(command) >> File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1. 0.4-py3.6.egg/pymonetdb/mapi.py", >> line 235, in cmd >> raise exception(string) >> pymonetdb.exceptions.IntegrityError: INSERT INTO: PRIMARY KEY >> constraint 'users.users_user_id_pkey' violated >> >> Have you any advice? Is there a way to perform the same check in just >> one SQL line? Is this way atomic (if a would ever move to a >> multithreading script)? > >It is not atomic, but you can make it atomic by wrapping both queries into one transaction. > >> Am I doing something >> wrong with my current code? Please, take into account that I do not >> have a deep experience with other SQL tools so it is perfectly >> possible that I miss something >> that for you is extremely obvious. > >No, you?re not doing anything wrong. But there are better alternatives: > >1. If this is a standalone action (i.e. not part of a transaction), the check for existing user_id is not necessary. The simplest and most efficient way is to always do the insert and ignore the ?primary key constraint ? violated? >error (e.g. use a try-catch block). > >2. If this insert is going to be part of a (big, long) transaction, then you won?t want the insert to fail. In that case, you can use a stored procedure: > >create procedure insert_new (id int, name varchar(10), description varchar(10), location varchar(10)) >begin > declare cnt int; > set cnt = (select count(*) from myusers where user_id=id); > > if (cnt = 0 ) > then > insert into myusers values (id, name, description, location); > end if; >end; > >Then call the procedure: call insert_new (42847029, 'XXXX', 'XXXX', 'XXXX?) > >I hope this helps. > II tried the solution number 2, but unfortunately the problem is still there. I get an Integrity error even calling the procedure that you described before. What is really strange is that, after a while, execute the procedure that you have showed me requires more than 3 seconds on my system. If I restart the monetdb instance, it takes a few milliseconds again. I think that maybe there is something broken in my setup. It could be that I have misconfigured something or I have a problem with my machine. Unfortunately, taking also into account that I am using Docker (which adds a layer of complexity), there are plenty of things that could not work as expected. Thank you for your help! >Regards, >Jennie > >> >> Thank you in advance! >> >> Best regards, >> >> Stefano Piani

2 1

MonetDB :: Memory Blots Up On High Concurrency
by Sharma, Sreejith 21 Jul '17

21 Jul '17

Hi, Does anyone encountered memory blot issue on high concurrency? Also, I don't see it coming down when idle. When its blots up, it's slows down the response time. Regards, Sreejith

3 6

Monetdb Installation
by amit＠g-square.in 21 Jul '17

21 Jul '17

Hi, how to install monetdb's old version June 2017 for ubuntu ??

2 1

Error trying to login
by amit＠g-square.in 20 Jul '17

20 Jul '17

Hi, I am using Ubuntu OS. I have created dbpharm and database. Now i am trying to login mclient using terminal but it gives following error. Please reply asap. root@mp-VirtualBox:/var/lib/monetdb# mclient -u monetdb -d narrator password: TypeException:user.sql.init[5]:'sqlcatalog.create_function' undefined in: sqlcatalog.create_function(X_1:str, X_3:str, X_5:ptr); MALException:optimizer.MALoptimizer:Start with inconsistent MAL plan

2 1

Re: how to know imprints take effect?
by integrity 20 Jul '17

20 Jul '17

Hi Stefan, Nice talking to you again! Sorry my query in the image didn't show up, so I paste it in text now. The query is a simple select with equals. ======================================== sql>select * from id14 where id=1461; +------+-------+------+ | id | label | cid | +======+=======+======+ | 1461 | false | 0 | +------+-------+------+ 1 tuple (104.549ms) sql> sql> sql>select row_number() over (),* from storage() where "table"='id14'; +------+--------+-------+--------+---------+----------+----------+---------+-----------+------------+----------+----------+-------+----------+--------+-----------+--------+----------+------+ | L5 | schema | table | column | type | mode | location | count | typewidth | columnsize | heapsize | hashes | phash | imprints | sorted | revsorted | unique | orderidx | L5 | +======+========+=======+========+=========+==========+==========+=========+===========+============+==========+==========+=======+==========+========+===========+========+==========+======+ | 1 | sys | id14 | id | bigint | writable | 12/1244 | 2811606 | 8 | 22492848 | 0 | 28049448 | false | 0 | false | false | null | 0 | 1 | | 2 | sys | id14 | label | boolean | writable | 02/254 | 2811606 | 1 | 2811606 | 0 | 0 | false | 0 | null | null | null | 0 | 2 | | 3 | sys | id14 | cid | bigint | writable | 12/1231 | 2811606 | 8 | 22492848 | 0 | 0 | false | 0 | false | false | null | 0 | 3 | +------+--------+-------+--------+---------+----------+----------+---------+-----------+------------+----------+----------+-------+----------+--------+-----------+--------+----------+------+ 3 tuples (10.090ms) ===================================================== Best, Meng ------------------ Original ------------------ From: "Stefan Manegold";<Stefan.Manegold(a)cwi.nl>; Date: Jul 19, 2017 To: "Communication channel for MonetDB users"<users-list(a)monetdb.org>; Subject: Re: how to know imprints take effect? Hi Meng, most important question: From what kind of queries do you expect millisecond responses over a table of 15,000,000 rows? Thanks! Best, Stefan ----- On Jul 19, 2017, at 12:37 PM, integrity 357416268(a)qq.com wrote: > Hi all, > > I am back, and I am using MonetDB Dec2016-SP5 with "enable-optimize" to do graph > computing. I remember that imprints are suitable for all numeric types, > however, after querying on the bigint column for many times, I found there is > no imprints. > > > > What I want to achieve is to "make" MonetDB respond in milliseconds, although I > know that in MonetDB it is not necessary to create index like imprints. > So, on a table of 15,000,000 rows, can I expect to get response in milliseconds? > and whether imprints will help with this? > > Any comments is appreciated! > > Best regards, > Meng > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

2 1

how to know imprints take effect?
by integrity 19 Jul '17

19 Jul '17

Hi all, I am back, and I am using MonetDB Dec2016-SP5 with "enable-optimize" to do graph computing. I remember that imprints are suitable for all numeric types, however, after querying on the bigint column for many times, I found there is no imprints. What I want to achieve is to "make" MonetDB respond in milliseconds, although I know that in MonetDB it is not necessary to create index like imprints. So, on a table of 15,000,000 rows, can I expect to get response in milliseconds? and whether imprints will help with this? Any comments is appreciated! Best regards, Meng

2 1

Announcement: New Jul2017 Feature release of MonetDB suite
by Sjoerd Mullender 18 Jul '17

18 Jul '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017 feature release MonetDB5 Server * The "sub" prefix of many functions, both at the MAL and the C level, has been removed. * Changed the interfaces of the AUTH* functions: pass values, not pointers to values. * Removed calc.setoid(). * group.subgroup is now called group.group if it is not refining a group. Both group.group and group.subgroup now also have variants with a candidate list. * The allocation schemes for MAL blocks and Variables has been turned into block-based. This reduces the number of malloc()/free() calls. . * Added a new server-side protocol implementation. The new protocol is backwards compatible with the old protocol. Clients can choose whether they want to use the old or the new protocol during the initial handshake with the server. The new protocol is a binary column-based protocol that is significantly faster than the old protocol when transferring large result sets. In addition, the new protocol supports compression using Snappy or LZ4. * Moved the sphinx extension module to its own repository. See https://dev.monetdb.org/hg/MonetDB-sphinx/. * Removed GSL module: it's now a separate (extension) package. See https://dev.monetdb.org/hg/MonetDB-gsl/. * The PCRE library is now optional for systems that support POSIX regular expressions. * Added 5 new sys schema tables: function_languages, function_types, key_types, index_types and privilege_codes. They are pre-loaded with static content and contain descriptive names for the various integer type and code values. See also sql/scripts/51_sys_schema_extension.sql Merovingian * Added handling of a dbextra property per database at the daemon level. The user can set the dbextra property for a database using the command: $ monetdb set dbextra=<path> <database> and the daemon will make sure to start the new server using the correct --dbextra parameter. Client Package * The mclient and msqldump programs lost compatibility with old mserver5 versions (pre 2014) which didn't have a "system" column in the sys.schemas table. * The mclient and msqldump programs lost compatibility with ancient mserver5 versions (pre 2011) which didn't have the sys.systemfunctions table. * Removed the "array" and "quick" functions from the mapi library. To be precise, the removed functions are: mapi_execute_array, mapi_fetch_field_array, mapi_prepare_array, mapi_query_array, mapi_quick_query, mapi_quick_query_array, and mapi_quick_response. * Added a more elaborate \help command for SQL expressions. MonetDB Common * Improved error checking in the logger code (dealing with the write-ahead log); changed return types a several functions from int to gdk_return (i.e., they now return GDK_SUCCEED or GDK_FAIL). The logger no longer calls GDKfatal on error. Instead the caller is responsible for dealing with errors. * BATsort may now create an order index as a by product. * Quantile calculations now use the order index if available (and use BATsort otherwise, producing an order index). * Quantiles calculate a position in the sorted column. If this position is not an integer, we now choose the nearest position, favoring the lower if the distance to the two adjacent positions is equal (round down to nearest integer). * Removed function BATprintf. Use BATprint or BATprintcolumns instead. * Removed BATsave from the list of exported functions. * Replaced BBPincref/BBPdecref with BBPfix/BBPunfix for physical reference count and BBPretain/BBPrelease for logical reference count maintenance. * Removed automatic conversion of 32-bit OIDs to 64 bits on 64-bit architectures. * Removed functions OIDbase() and OIDnew(). * Removed talign field from BAT descriptor. * BATappend now takes an optional (NULL if not used) candidate list for the to-be-appended BAT. * New function BATkeyed(BAT *b) that determines (possibly using a hash table) whether all values in b are distinct. SQL * Made the operator precedence of % equal to those of * and /. All three are evaluated from left to right. * Removed table sys.connections. It was a remnant of an experimental change that had already been removed in 2012. * Protect against runaway profiler events If you hit a barrier block during profiling, the JSON event log may quickly become unwieldy. Event production is protected using a high water mark, which ensures that never within the single execution of MAL block the instruction causes excessive event records. Bug Fixes * 3465: Request: add support for CREATE VIEW with ORDER BY clause * 3545: monetdb commands don't work with -h -P -p options (locally and remotely) * 3996: select * from sys.connections always returns 0 rows. Expected to see at least one row for the active connection. * 6187: Nested WITH queries not supported * 6225: Order of evaluation of the modulo operator * 6289: Crashes and hangs with remote tables * 6292: Runaway SQL optimizer in too many nested operators * 6310: Name resolution error (sqlsmith) * 6312: Object not found in LIMIT clause (sqlsmith) * 6313: Null type resolution in disjunction fails (sqlsmith) * 6319: Server crash on LATERAL (sqlsmith) * 6322: Crash on disjunction with LIMIT (sqlsmith) * 6323: Deadlock calling sys.bbp() * 6324: Sqlitelogictest crash in a IN query (8th) * 6327: The daemon does not respect the actual name of the mserver5 executable * 6330: Sqlitelogictest crash on a complex SELECT query * 6331: sys.statistics column "nils" always contains 0. Expected a positive value for columns that have one or more nils/NULLs * 6332: Sqlitelogictest crash related to an undefined MAL function

3 3

possible to change the decimal character from "." to ", " for the COPY INTO command?
by Anthony Damico 18 Jul '17

18 Jul '17

hi, i'm curious if this is possible to implement? not seeing it in the docs.. for example, in R you can use read.csv() which uses , as the delimiter and . as the decimal or you can use read.csv2 which uses ; as the delimiter and , as the decimal thanks all

1 0

Add entry in a table only if that entry does not already exist
by Stefano Piani 14 Jul '17

14 Jul '17

Dear monetdb users, I started to use MonetDB a few weeks ago and, until now, I have been able to use it proficiently. Unfortunately, I have faced a small issue that I have not been able to figure out how to fix. I collect data generated by different users. A unique id is associated to each user. I have a specific table (called "user") where I save the name and some other information about the user. In this table, the ID is a primary key. When I collect the data, I want to check if the specific user who generated that data is already saved in the user table. If this is not the case, I want to add a row to the user table for this user. Unfortunately, I have not been able to accomplish this operation in one SQL query and, therefore, I have been forced to query for the user id and, if the answer is negative, add the user using another query. The following is the python script that perform the operations. def save_user_in_database(user, cursor): db_line = 'SELECT 1 FROM users WHERE user_id = {}'.format(user.id) LOGGER.info('Executing on DB: %s', db_line) cursor.execute(db_line) answer = cursor.fetchall() LOGGER.info('Get this answer %s', answer) if len(answer) == 0: LOGGER.info('Inserting new user') db_line = sql_cmd_from_template( USER_TABLE_INSERT_TEMPLATE, user.id, user.name, user.description, user.location ) LOGGER.info('Executing on DB: %s', db_line) cursor.execute(db_line) This function is executed inside a single threaded application. Moreover, there are no "DELETE" or similar operations inside this script (it only adds rows to the database). No user (human or software) but this script connects to my MonetDB instance. Unfortunately, after I while that the script is running (let's say one or two days), something unexpected happens. For example, this is the log generated for user 42847029 (there are a lot of other lines in between that I removed). As you can see, after a while, my check fails and I don't understand why. 2017/06/13 09:30:01 - INFO - save_user_in_database: Executing on DB: SELECT 1 FROM users WHERE user_id = 42847029 2017/06/13 09:30:01 - INFO - save_user_in_database: Get this answer [(1,)] 2017/06/13 10:00:04 - INFO - save_user_in_database: Executing on DB: SELECT 1 FROM users WHERE user_id = 42847029 2017/06/13 10:00:04 - INFO - save_user_in_database: Get this answer [(1,)] 2017/06/13 10:02:33 - INFO - save_user_in_database: Executing on DB: SELECT 1 FROM users WHERE user_id = 42847029 2017/06/13 10:02:33 - INFO - save_user_in_database: Get this answer [(1,)] 2017/06/13 10:30:36 - INFO - save_user_in_database: Executing on DB: SELECT 1 FROM users WHERE user_id = 42847029 2017/06/13 10:30:36 - INFO - save_user_in_database: Get this answer [] 2017/06/13 10:30:36 - INFO - save_user_in_database: Inserting new user 2017/06/13 10:30:36 - INFO - save_user_in_database: Executing on DB: INSERT INTO users(user_id, user_name, user_description, user_location) VALUES (42847029, 'XXXX', 'XXXX', 'XXXX') 2017/06/13 10:30:36 - WARNING - on_data: Error elaborating data Traceback (most recent call last): File "data_collector.py", line 267, in on_data return self.elaborate_data(data) File "data_collector.py", line 354, in elaborate_data save_user_in_database(user, cursor) File "data_collector.py", line 234, in save_user_in_database cursor.execute(db_line) File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1.0.4-py3.6.egg/pymonetdb/sql/cursors.py", line 164, in execute block = self.connection.execute(query) File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1.0.4-py3.6.egg/pymonetdb/sql/connections.py", line 133, in execute return self.command('s' + query + ';') File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1.0.4-py3.6.egg/pymonetdb/sql/connections.py", line 138, in command return self.mapi.cmd(command) File "/opt/Python-3.6.0/lib/python3.6/site-packages/pymonetdb-1.0.4-py3.6.egg/pymonetdb/mapi.py", line 235, in cmd raise exception(string) pymonetdb.exceptions.IntegrityError: INSERT INTO: PRIMARY KEY constraint 'users.users_user_id_pkey' violated Have you any advice? Is there a way to perform the same check in just one SQL line? Is this way atomic (if a would ever move to a multithreading script)? Am I doing something wrong with my current code? Please, take into account that I do not have a deep experience with other SQL tools so it is perfectly possible that I miss something that for you is extremely obvious. Thank you in advance! Best regards, Stefano Piani

2 3

Drop a local temporary table within a function
by Luis Larios 14 Jul '17

14 Jul '17

Hi, all, Consider the following scenario. I have a bunch of user defined functions, f_i, that create, populate, and use a local temporary table with the same name "res": create function f_i() returns table( ... ) begin # Result stats table. create local temporary table res( param1 int, param2 int ); /* Populate res */ /* Use res in some queries */ return table( /* Query involving res */ ); end; My first question is: Does "create local temporary table x" create a table that is disposed/dropped as soon as the function where it was declared finishes execution? I assumed this was the case, but I'm now skeptical as we are running into all sorts of concurrency issues that point to the "res" table not being cleared out properly from memory. This takes me to the second question: How do I drop a temporary table if such table exists? The reason for this is to prevent "table create" exceptions (which we are currently running into). I'd like to achieve something along these lines: create function f_i() returns table( ... ) begin # Drop local temporary table if such table exists. drop local temporary table res; <-------- How do I do this in MonetDB? # Result stats table. create local temporary table res( param1 int, param2 int ); /* Populate res */ /* Use res in some queries */ return table( /* Query involving res */ ); end; Where, as you can see, I want to explicitly drop the local temporary table "res" if such exists. If I leave the "local temporary" markers in the drop statement, MonetDB complains that it found unexpected "LOCAL" or "TEMPORARY". I wouldn't like to remove these markers because a statement like: drop table res; will drop any table in the default function schema with that name. Any help and hints are well appreciated! Thanks for the help in advance ~ Luis Angel The information transmitted, including any attachments, is intended only for the individual or entity to which it is addressed, and may contain confidential and/or privileged information. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by individuals or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you have received this communication in error, please delete the information from any computer and notify the sender.

3 2

Jul2015-SP4 vs. Dec2016-SP5
by Anderson, David B 13 Jul '17

13 Jul '17

All, I know I've skipped multiple releases so this is a difficult question to answer, but should I expect the Dec2016-SP5 version of MonetDB to be faster than/equal to the Jul2015-SP4 version? I have a parallel set of servers (on the same box, 24 cores 192GB ram, listening to different ports) operating on identical data sets, and the Dec2016 version is slower than the Jul2015 version (~15-20%). My queries are basic joins between two tables with grouping. I have top running on the same box, and I can see that the Jul2015 server uses all 24 cores, but the Dec2016 server barely cracks 2-3. The MAL plans obtained by explain are totally different (which partly explains the cores being utilized because I can see blocks of 24 commands) and the length of the Dec2016 one is a fraction of the Jul2015 version. I guess the older version was inefficiently using all 24 cores, but the new version efficiently uses a smaller set of cores?? Thanks, Dave

2 2

script-friendly prepare statement
by Roberto Cornacchia 13 Jul '17

13 Jul '17

Hi there, I'm trying to script the execution of (generated) prepared queries. It does not look possible, because the identifier to use for the EXEC statement is returned dynamically by the PREPARE statement: sql>prepare select 1; execute prepared statement using: EXEC 73(...) This can obviously not be scripted. If I tried to parse the output of the prepare statement to retrieve the identifier, this would split PREPARE and EXEC in two different mclient sessions, losing the prepared statement. As far as I know, there is at the moment no syntax available for providing the identifier, something like: sql>prepare myid as select 1; (that wouldn't be a bad idea, would it?) Besides syntax extensions, I was wondering if there is any trick I could use at runtime with the current syntax to retrieve that identifier. Like, is it stored in some system table? Is it available as any undocumented variable? I could not find anything like that, but I might have missed it. Cheers, Roberto

3 4

adding an "insert into" completely changes output from explain
by Anderson, David B 13 Jul '17

13 Jul '17

All, Is it expected behavior if adding "insert into xxxx" at the beginning of a sql query completely alters the output of explain? Without the "insert into", explain returns 1490 tuples, but when I add it, explain only returns 88. I was expecting that the number would go up, i.e. that the original plan would be wrapped in some additional commands which would implement the "insert into" part. Here is my sql: explain insert into aggr select 'CHANNEL', s.product, s.coupon, s.productionyear, s.tpoflag, sum(1), sum(d.scheduled)/1000000, sum(d.realized )/1000000, round(100*(1-power(sum(d.realized)/sum(d.scheduled),12)),3) as cpr from stt s, dyn d where s.loanseqnum = d.loanseqnum group by s.product, s.coupon, s.productionyear, s.tpoflag having sum(d.scheduled) > 0 ;

2 1

Extremely slow order by queries
by Manish gupta 13 Jul '17

13 Jul '17

Hi, I have a scenario where data size is ~3M rows. My query is select C1, C2, C3 from T1 group by C1,C2,C3 This returns result in 30 Sec ( number of groups around 1 M ) While, if I modify the query to select C1, C2, C3 from T1 group by C1,C2,C3 Order by C1, C2 on the same dataset, it takes ~ 1 hour to get result. The platforms I tried is both widows 64 bit and Linux 64 bit, with latest monet installed. Both the machines have 4 cores. And the behavior is same on both the machines. I am really surprised to see overhead of second query. One more observation is that this query occupies only single core, while monet is free to utilize 4 cores. Regards, Manish

2 1

Mserver returns with "memory exhausted in" error if query size exceeds 1.2MB
by Sharma, Sreejith 08 Jul '17

08 Jul '17

Hi, I have encountered scenario where SQL generated exceeds 3mb and it's failing in mserver with an error "memory exhausted in select .......". On further checks I have noticed that the SQL will go through only if SQL size is less than 1.2MB. Has anyone encountered this before? Or is anyone aware of any SQL size limitation in MonetDB? Please help. Regards, Sreejith

1 0

access MapiHdl by using qtag value from C API
by Anton Kravchenko 26 Jun '17

26 Jun '17

Is there a way to access MapiHdl of the query by using its qtag value?

1 0

Re: MonetDB taking lot of storage space
by Stefan Manegold 17 Jun '17

17 Jun '17

Typo in formula:does * columns * (average) width per columnmust readrows * columns * (average) width per column, -------- Original message --------From: Stefan Manegold <stefan.manegold(a)cwi.nl> Date: 6/17/17 12:33 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: Re: MonetDB taking lot of storage space Hi Amir, May I assume "cr" stands for "crore" (c.f.., https://en.m.wikipedia.org/wiki/Crore) I.e. 1 cr = 10 million = 10^7 = 10,000,000 ? The plain expected size of the database assuming only fixed-width column types and ignoring automatically generated indexes, etc.Can be assumed as does * columns * (average) width per column, e.g.,Assuming all 4 byte numerical columns:(4+5) * 10^7 * 10 * 4 byte = 36 * 10^8 ~= 3.6 GBor twice as much for all 8 byte columns. Your 196 GB is far from that. Are you sure there is no other left-over data in your database?As Jennie suggested, did you do a lot of update (insert, delete) operations on these tables?Which (top-level) directories in your dbfarm are the largest? Best,Stefan -------- Original message --------From: Ying Zhang <Y.Zhang(a)cwi.nl> Date: 6/17/17 10:35 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: Re: MonetDB taking lot of storage space Hai, MonetDB doesn’t provide any user definable compressions. What do you mean with “5cr rows”? Compressed Rows? 198GB is indeed pretty big, so it’s expected to contain a lot of table data. If you don’t have variable length columns, it’s easy to compute the expected storage space: #rows * #columns * (sum of the length of each column). Are your database fresh or a lot of insert and delete have been conducted on them? Then, the deletes might have leaved garbages. Did you happen to have put larges BLOBs or CLOBs in some columns? Which MonetDB version and OS do you use? Regards, Jennie > On 17 Jun 2017, at 09:38, Amit Ambekar <amit(a)g-square.in> wrote: > > Hi, > > i have 2 databases in monetdb. first one has 5cr rows and second has 4 cr rows with average 10 columns per table. It takes 196 GB storage space. is it normal ?? most of data consists of numeric values. And How i can compress database size in MonetDB ?? > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

Re: MonetDB taking lot of storage space
by Stefan Manegold 17 Jun '17

17 Jun '17

Hi Amir, May I assume "cr" stands for "crore" (c.f.., https://en.m.wikipedia.org/wiki/Crore) I.e. 1 cr = 10 million = 10^7 = 10,000,000 ? The plain expected size of the database assuming only fixed-width column types and ignoring automatically generated indexes, etc.Can be assumed as does * columns * (average) width per column, e.g.,Assuming all 4 byte numerical columns:(4+5) * 10^7 * 10 * 4 byte = 36 * 10^8 ~= 3.6 GBor twice as much for all 8 byte columns. Your 196 GB is far from that. Are you sure there is no other left-over data in your database?As Jennie suggested, did you do a lot of update (insert, delete) operations on these tables?Which (top-level) directories in your dbfarm are the largest? Best,Stefan -------- Original message --------From: Ying Zhang <Y.Zhang(a)cwi.nl> Date: 6/17/17 10:35 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: Re: MonetDB taking lot of storage space Hai, MonetDB doesn’t provide any user definable compressions. What do you mean with “5cr rows”? Compressed Rows? 198GB is indeed pretty big, so it’s expected to contain a lot of table data. If you don’t have variable length columns, it’s easy to compute the expected storage space: #rows * #columns * (sum of the length of each column). Are your database fresh or a lot of insert and delete have been conducted on them? Then, the deletes might have leaved garbages. Did you happen to have put larges BLOBs or CLOBs in some columns? Which MonetDB version and OS do you use? Regards, Jennie > On 17 Jun 2017, at 09:38, Amit Ambekar <amit(a)g-square.in> wrote: > > Hi, > > i have 2 databases in monetdb. first one has 5cr rows and second has 4 cr rows with average 10 columns per table. It takes 196 GB storage space. is it normal ?? most of data consists of numeric values. And How i can compress database size in MonetDB ?? > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

MonetDB taking lot of storage space
by Amit Ambekar 17 Jun '17

17 Jun '17

Hi, i have 2 databases in monetdb. first one has 5cr rows and second has 4 cr rows with average 10 columns per table. It takes 196 GB storage space. is it normal ?? most of data consists of numeric values. And How i can compress database size in MonetDB ??

2 1

SQLsetTrace assumes in-place append
by Roberto Cornacchia 14 Jun '17

14 Jun '17

Background: I'm working on a side-effect-free branch of MonetDB. No in-place operations allowed. With bat.append() already changed this way, everything seemed to work so far, except TRACE. Even TRACE SELECT 1 would fail. I looked at it and the reason is that SQLsetTrace assumed in-place appends, thus first adding new empty bats to resultset, then filling in those bats. Everywhere else this is not assumed. The return bat of append is always created and used, even if it is the same of the first operand. The diff in attachment makes it work for both branches and is in my opinion more correct and consistent with the rest of the code. Best, Roberto

1 0

Announcement: New Dec2016-SP5 Bugfix release of MonetDB suite
by Sjoerd Mullender 12 Jun '17

12 Jun '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Dec2016-SP5 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Dec 2016-SP5 bugfix release Bug Fixes * 6290: Crash (and assertion failure) with a correlated subquery with NOT IN in the select-list * 6291: crash if executes a function with sample operator * 6294: Sqlitelogictest crash * 6296: Another sqllitelogictest crash :( * 6297: 7th sqllitelogictest crash :( * 6300: Protect against missing BATs (sqlsmith) * 6314: Lateral crash report (sqlsmith) * 6315: Exist operator on type bigint missing (sqlsmith) * 6316: Coalesc and limit error (sqlsmith) * 6317: Two-column integer aggregation extremely slow * 6318: Daemon crashes if .merovignian_properties for a database contains more than 42 entries * 6320: The daemon ignores all but the last entry in the .merovignian_properties file when spawning mserver5

1 0

How to use prepared statements within a function
by Luis Larios 08 Jun '17

08 Jun '17

Hi, We’d like to know if there is a way to create and execute prepared statements within an SQL function. In particular, once that we use: “prepare select * from table” How can I retrieve the prepared statement ID to use in an exec()? Thanks, Luis Ángel. The information transmitted, including any attachments, is intended only for the individual or entity to which it is addressed, and may contain confidential and/or privileged information. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by individuals or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you have received this communication in error, please delete the information from any computer and notify the sender.

2 1

Cannot execute embedded R in MonetDB
by Luis Larios 25 May '17

25 May '17

Hi, We are having issues when executing embedded R in a MonetDB SQL function. We have compiled and configured MonetDB accordingly, so as to be able to run embedded R; however, the simple example in the documentation fails to execute. This is the function we have: create function rapi00() returns table( d int ) language R { return seq(1,10); }; Having just "seq(1,10)" doesn't make any difference either. When this function gets created, we don't get any error; however, when we do the following: select d from rapi00() as r where d > 5; we get the this error: Query failed: TypeException:user.s6_3[3]:'rapi.eval' undefined in: X_4:bat[:int] := rapi.eval(7fdb9830e210:ptr,"{\n\treturn seq(1,10);\n};":str); We don't know how to tackle this issue. We have followed the process from https://www.monetdb.org/content/embedded-r-monetdb. Do you have any suggestion to make this work? Thanks! Luis Ángel. The information transmitted, including any attachments, is intended only for the individual or entity to which it is addressed, and may contain confidential and/or privileged information. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by individuals or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you have received this communication in error, please delete the information from any computer and notify the sender.

2 1

binary bulk loading improvements?
by Lynn Carol Johnson 22 May '17

22 May '17

Hello Stefan and developers – Has any more thought been given to performance issues on binary bulk loading of large tables? I’m hoping to hear the “steal” option will be available in an upcoming release. Thanks - Lynn

1 0

Distributed Join and Replica Tables
by Kelly Burkhart 11 May '17

11 May '17

Can someone describe how exactly replica tables are used to push joins to worker nodes? Assume a master node and two worker nodes. I assume my reference table would go on my master node, and on my worker nodes I would have my large data table and replicas of the reference tables from the master node. Is that the general strategy? Thanks, -K

4 6

RE: Queries regarding embedded python
by Ganesh 08 May '17

08 May '17

Hi there, hope you are doing good. We were trying to implement Monetdb/python i.e. embedded python and tried the following code but its giving an error. (We have tried it on Linux) CREATE FUNCTION python_times_two(i INTEGER) RETURNS INTEGER LANGUAGE PYTHON { return i * 2 }; CREATE TABLE integers(i INTEGER); INSERT INTO integers VALUES (1), (2), (3), (4), (5); SELECT python_times_two(i) AS result FROM integers; Error: TypeException:user.s12_1[10]:'pyapi.eval' undefined in: X_18:int := pyapi.eval(0x7f051006a340:ptr,"{\n return i * 2\n};":str,X_17:int); Please help us on what should be done in order to code to work. Also need clarification on following points: 1. Does embedded R or embedded python work on windows? 2. Does it take data in memory when we use machine learning with Numpy and Scikitlearn? Awaiting for your response. Warm Regards, Ganesh Anbhule Data Scientist

2 1

Retrieving all schemata for a user
by Luis Larios 07 May '17

07 May '17

Hi, I wonder if anyone knows how I can retrieve all schemata on which a user has been granted permissions/authorization. I have checked most of the hinting tables I get from querying >> select id, name, schema_id, query from sys.tables where system = true; but I have had no luck with that. I only get the default schema for every user when I perform the following: >> select * from sys.users Any ideas or suggestions are welcome. Thanks! Luis Ángel. The information transmitted, including any attachments, is intended only for the individual or entity to which it is addressed, and may contain confidential and/or privileged information. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by individuals or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you have received this communication in error, please delete the information from any computer and notify the sender.

2 1

Error while installing Monetdb
by Amit Ambekar 06 May '17

06 May '17

Hi, How much time it will take for new release ? On Wednesday 03 May 2017 03:30 PM, users-list-request(a)monetdb.org wrote: > Error while installing Monetdb

4 5

Problem with release
by Amit Ambekar 03 May '17

03 May '17

Hi, New release of monetdb is not installing on ubuntu 16.04 LTS. I think problem will be your new release which is happened on 1 May 2017. Change https://dev.monetdb.org/downloads/deb/dists/xenial/monetdb/binary-amd64/Rel…: Valid-Until: Tue, 30 Nov 2038 00:00:00 JST this is not working. JST to some other proper format

1 0