- users-list - monetdb.org

Help with performance issue
by Roberto Cornacchia 26 Jan '18

26 Jan '18

Hi there, Would someone like to pick up this wonderful opportunity to tell me that I'm doing something silly? Please do. In this loop, the cleanup take 4/5 of the total time. Is there a better way of doing this? loop(...) { BAT *gn, *en, *hn; BAT *idxn, *tn, *pn; // tokenize, group, transform, append // this takes 1/5 of the total loop time if (BATutf8_tokenize(&tokens, s, delims, min_tok_len) != GDK_SUCCEED) goto fail; if (BATgroup(&gn, &en, &hn, tokens, NULL, NULL, NULL, NULL) != GDK_SUCCEED) goto fail; idxn = BATconstant(0, TYPE_int, &idx, BATcount(en), TRANSIENT); tn = BATproject(en,tokens); pn = BATconvert(hn, NULL, TYPE_dbl, TRUE); if (BATappend(br1,idxn,NULL,FALSE) != GDK_SUCCEED) goto fail; if (BATappend(br2,tn,NULL,FALSE) != GDK_SUCCEED) goto fail; if (BATappend(br3,pn,NULL,FALSE) != GDK_SUCCEED) goto fail; // this takes 4/5 of the total loop time BBPreclaim(idxn); BBPreclaim(tn); BBPreclaim(pn); BBPreclaim(hn); BBPreclaim(gn); BBPreclaim(en); BBPreclaim(tokens); }

3 6

"brew install monetdb" failed due to "libxml2 library not found"
by Ying Zhang 25 Jan '18

25 Jan '18

Dear all, I try to install MonetDB with homebrew, but it failed with the error: configure: error: libxml2 library not found but required for MonetDB5 But as far as I can tell, libxml2 is already installed, e.g. $ brew list libxml2 /usr/local/Cellar/libxml2/2.9.7/bin/xml2-config /usr/local/Cellar/libxml2/2.9.7/bin/xmlcatalog /usr/local/Cellar/libxml2/2.9.7/bin/xmllint /usr/local/Cellar/libxml2/2.9.7/include/libxml2/ (47 files) /usr/local/Cellar/libxml2/2.9.7/lib/libxml2.2.dylib /usr/local/Cellar/libxml2/2.9.7/lib/cmake/libxml2/libxml2-config.cmake /usr/local/Cellar/libxml2/2.9.7/lib/pkgconfig/libxml-2.0.pc /usr/local/Cellar/libxml2/2.9.7/lib/python2.7/ (6 files) /usr/local/Cellar/libxml2/2.9.7/lib/ (3 other files) /usr/local/Cellar/libxml2/2.9.7/share/aclocal/libxml.m4 /usr/local/Cellar/libxml2/2.9.7/share/doc/ (153 files) /usr/local/Cellar/libxml2/2.9.7/share/gtk-doc/ (55 files) /usr/local/Cellar/libxml2/2.9.7/share/man/ (4 files) $ brew install libxml2 Warning: libxml2-2.9.7 already installed Is there anything I can do to fix this? Thanks a lot in advance! Jennie =============== Complete output of “brew install monetdb” =============== $ brew install monetdb Warning: You are using macOS 10.13. We do not provide support for this pre-release version. You may encounter build failures or other breakages. Please create pull-requests instead of filing issues. ==> Using the sandbox ==> Downloading https://www.monetdb.org/downloads/sources/Jul2017-SP3/MonetDB-11.27.11.tar.… Already downloaded: /Users/jennie/Library/Caches/Homebrew/monetdb-11.27.11.tar.xz ==> ./configure --prefix=/usr/local/Cellar/monetdb/11.27.11 --enable-debug=no --enable-assert=no --enable-optimi Last 15 lines from /Users/jennie/Library/Logs/Homebrew/monetdb/01.configure: checking how to link with libiconv... -liconv checking for iconv declaration... extern size_t iconv (iconv_t cd, char * *inbuf, size_t *inbytesleft, char * *outbuf, size_t *outbytesleft); checking for library containing uuid_generate... none required checking for CC_MD5_Update... yes checking for CC_RIPEMD160_Update... no checking for CC_SHA1_Update... yes checking for CC_SHA224_Update... yes checking for CC_SHA256_Update... yes checking for CC_SHA384_Update... yes checking for CC_SHA512_Update... yes checking for pcre... yes checking whether pcre comes with UTF-8 support... yes checking for libxml2... no configure: error: libxml2 library not found but required for MonetDB5 READ THIS: https://git.io/brew-troubleshooting If reporting this issue please do so at (not Homebrew/brew): https://github.com/Homebrew/homebrew-core/issues Warning: You are using macOS 10.13. We do not provide support for this pre-release version. You may encounter build failures or other breakages. Please create pull-requests instead of filing issues.

2 2

Re: MonetDB - question on rename table
by David Jankoski 25 Jan '18

25 Jan '18

hey sreejith, maybe this SO answer will be helfpul to you: https://stackoverflow.com/questions/19661538/how-to-rename-a-table-in-monet… hth. david On 25 January 2018 at 08:56, <users-list-request(a)monetdb.org> wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. MonetDB-extend clang error: ld: unknown option: -soname > (Ying Zhang) > 2. Re: MonetDB-extend clang error: ld: unknown option: -soname > (Sjoerd Mullender) > 3. "brew install monetdb" failed due to "libxml2 library not > found" (Ying Zhang) > 4. MonetDB - question on rename table (Sharma, Sreejith) > 5. Re: MonetDB - question on rename table (Stefan Manegold) > 6. RE: MonetDB - question on rename table (Sharma, Sreejith) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 24 Jan 2018 15:56:19 +0100 > From: Ying Zhang <Y.Zhang(a)cwi.nl> > To: Communication channel for MonetDB users <users-list(a)monetdb.org> > Subject: MonetDB-extend clang error: ld: unknown option: -soname > Message-ID: <1DD5C9B8-D559-4CD2-89E0-35CA729C9B67(a)cwi.nl> > Content-Type: text/plain; charset=utf-8 > > Dear all, > > I was trying to compile the regexp example in the MonetDB-extend repo on a Mac OS 10.13, but got an ?unknown option? error: > > $ make > cc -fPIC -DPIC -o lib_regexp.so -shared regexp.o -L/usr/local/opt/openssl/lib -L/Users/jennie/scratch/monet-install/Jul2017/debug/lib -L/usr/local/Cellar/pcre/8.41/lib -lmonetdb5 -lbat -lpcre -Wl,-soname -Wl,lib_regexp.so > ld: unknown option: -soname > clang: error: linker command failed with exit code 1 (use -v to see invocation) > make: *** [lib_regexp.so] Error 1 > > I looked around a bit. There are many discussions about clang error ld unknown option, but nothing specific for the ?-soname? option. Simply remove this option is not an option. > > Does anyone know how to fix/workaround this? > > Thanks a lot! > > Jennie > > ------------------------------ > > Message: 2 > Date: Wed, 24 Jan 2018 16:34:38 +0100 > From: Sjoerd Mullender <sjoerd(a)acm.org> > To: users-list(a)monetdb.org > Subject: Re: MonetDB-extend clang error: ld: unknown option: -soname > Message-ID: <78539081-3ab0-3b89-d22d-dbd12af4a6fa(a)acm.org> > Content-Type: text/plain; charset="utf-8" > > On 24/01/18 15:56, Ying Zhang wrote: >> Dear all, >> >> I was trying to compile the regexp example in the MonetDB-extend repo on a Mac OS 10.13, but got an ?unknown option? error: >> >> $ make >> cc -fPIC -DPIC -o lib_regexp.so -shared regexp.o -L/usr/local/opt/openssl/lib -L/Users/jennie/scratch/monet-install/Jul2017/debug/lib -L/usr/local/Cellar/pcre/8.41/lib -lmonetdb5 -lbat -lpcre -Wl,-soname -Wl,lib_regexp.so >> ld: unknown option: -soname >> clang: error: linker command failed with exit code 1 (use -v to see invocation) >> make: *** [lib_regexp.so] Error 1 >> >> I looked around a bit. There are many discussions about clang error ld unknown option, but nothing specific for the ?-soname? option. Simply remove this option is not an option. >> >> Does anyone know how to fix/workaround this? >> >> Thanks a lot! >> >> Jennie > > The question really is, how does one create a dynamically loaded module > on MacOS? The file will be loaded dynamically by mserver, not linked > into a program. > > > -- > Sjoerd Mullender > >

1 0

Re: MonetDB - question on rename table
by Stefan Manegold 25 Jan '18

25 Jan '18

Yep: that feature is not implemented. -------- Original message --------From: "Sharma, Sreejith" <Sreejith.Sharma(a)harman.com> Date: 1/25/18 07:45 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: MonetDB - question on rename table All, Any reason why we can’t do RENAME of table in MonetDB? Regards, Sreejith

2 1

MonetDB - question on rename table
by Sharma, Sreejith 25 Jan '18

25 Jan '18

All, Any reason why we can't do RENAME of table in MonetDB? Regards, Sreejith

1 0

MonetDB-extend clang error: ld: unknown option: -soname
by Ying Zhang 24 Jan '18

24 Jan '18

Dear all, I was trying to compile the regexp example in the MonetDB-extend repo on a Mac OS 10.13, but got an “unknown option” error: $ make cc -fPIC -DPIC -o lib_regexp.so -shared regexp.o -L/usr/local/opt/openssl/lib -L/Users/jennie/scratch/monet-install/Jul2017/debug/lib -L/usr/local/Cellar/pcre/8.41/lib -lmonetdb5 -lbat -lpcre -Wl,-soname -Wl,lib_regexp.so ld: unknown option: -soname clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [lib_regexp.so] Error 1 I looked around a bit. There are many discussions about clang error ld unknown option, but nothing specific for the “-soname” option. Simply remove this option is not an option. Does anyone know how to fix/workaround this? Thanks a lot! Jennie

2 1

Question on MonetDB Source Code Compilation
by Sharma, Sreejith 24 Jan '18

24 Jan '18

All, Here is a quick question. I have compiled "MonetDB v11.27.9 (Jul2017-SP2)" version of source code and while running below sample SQL am finding an issue. Could anyone help me to trace issue? Is this due to the compiler version? Issue Details: While using SUM() or any function without an alias the header datatype returned is showing as HUGEINT for base INTEGER column, where as it should have been BIGINT. This is causing a failure in my BI application as it's having issues in identifying HUGEINT. Below are the details. Source Code Version : MonetDB v11.27.9 (Jul2017-SP2) Linux OS Version : CentOS release 6.3 (Final) C Compile Version : gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC) Also, I see below note in MonetDB documentation. Which version of GCC should be appropriate for compilation with __128 bit integer. Note: HUGEINT is only available since the Jul2015 release and on platforms with a C-compiler that supports the __int128 or __int128_t data type (e.g., recent gcc, clang, & icc on Linux or MacOS X) Sample Table DDL : CREATE TABLE INT_VALID (COL_INT INTEGER, COL_BIGINT BIGINT, COL_HUGEINT HUGEINT, COL_DOUBLE DOUBLE, COL_NUMERIC NUMERIC(10)); Compiled Version sql>SELECT SUM(col_int) FROM INT_VALID; mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID; fetch next block: start at:10093 got next block: length:105 text:&1 4 1 1 1 2452 % DM_POS_TSV_MRT_P6A.L4 # table_name % L3 # name % hugeint # type % 5 # length [ 11110 ] Binary Version sql>SELECT SUM(col_int) FROM INT_VALID; mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID; fetch next block: start at:1168 got next block: length:105 text:&1 4 1 1 1 13998 % DM_POS_TSV_MRT_P6A.L4 # table_name % L3 # name % bigint # type % 5 # length [ 11110 ] +---------+------------+--------------------------+-------------+ | col_int | col_bigint | col_double | col_numeric | +=========+============+==========================+=============+ | 1111 | 1234567 | 1234.44 | 1111 | | 2222 | 2345678 | 34355.66 | 2222 | | 3333 | 3456789 | 66423.66 | 3333 | | 4444 | 45678912 | 8765.66 | 4444 | +---------+------------+--------------------------+-------------+ Regards, Sreejith

2 3

Memory leak in MonetDB
by Sergii Kliuchnyk 23 Jan '18

23 Jan '18

Hello, my name Sergii. I hope this is right place for this message. In our company we using MonetDB near 1 year and we have only one problem with it - memory leak. We found only one dependence - if we making queries then memory starts growing. I read "Memory footprint" article that MonetDB excessively uses main memory for processing, but I thought it should stop allocate memory after sometime but it's not. And looks like this allocated memory is not gives some performance. What we do now is restarting monet server every month. I found on mserver5 man-page parameter "gdk_vmtrim" but I'm not 100% sure is it what we need and what correct way to apply this parameter. Maybe MonetDB SQL has some function to unload no needed memory? Maybe it's php/nodejs clients (from github.com/MonetDB repo) somehow cause this memory leaks? I will appreciate any help

1 1

BUG: on CREATE TABLE (like)
by Ariel Abadi 18 Jan '18

18 Jan '18

Hi! It seems to be, that there is a BUG on the statement CREATE TABLE.. It is not being consider the *DEFAULT property.* Take a look on the following example: sql>CREATE TABLE test (updated TIMESTAMP(0) DEFAULT current_timestamp()); operation successful (1.433ms) sql>\d test CREATE TABLE "sb_traf"."test" ( "updated" TIMESTAMP(0) DEFAULT current_timestamp() ); sql>CREATE TABLE test2 (LIKE test); operation successful (1.102ms) sql>\d test2 CREATE TABLE "sb_traf"."test2" ( "updated" TIMESTAMP(0) ); May be Im missing something, if thats the case, please let me know! Rgds Ariel

2 2

Embedded Python/NumPy
by Ariel Abadi 18 Jan '18

18 Jan '18

Hi! Someone can help please!?!?!!! I have created a function in python (see attached the definition), every time I try to use it against a "big table" monetdb is hanging out. What I found is that instead of sending the value of each nrm_json (to the function) , monet is sending the full column!!!.. This means that if the table has 10MM rows, then python is receiving ALL the records instead of the nrm_json STRING: SELECT sb_glob.getjsonfield_python(nrm_json, 'conference_id') FROM nrm_20180107 LIMIT 3; CREATE FUNCTION sb_glob.getjsonfield_python(nrm_json STRING, nrm_field STRING) RETURNS STRING LANGUAGE PYTHON { import ast a=ast.literal_eval(str(nrm_json.encode('utf-8'))) if not a.has_key(nrm_field): a[nrm_field] = '' return str(a[nrm_field]) }; I hope you understand, and guide me in solving this problem Thks in advance Ariel

1 0

program contains errors occured
by Amit Ambekar 18 Jan '18

18 Jan '18

Hi, I have big table around 15 mln records. getting error as "program contains errors" when i tried to fetch data using select query

2 1

Python Functions
by Ariel Abadi 11 Jan '18

11 Jan '18

Hi! Maybe someone can help me. Im trying to connect to a MySQL table to get some data and use it within MonetDB. Eveything works fine, TILL... the query Im running on MySQL is bringing NO DATA. As an example I put it as LIMIT 0 so I dont receive any row. Eventhough I have considered that situation and have defined the array with null values, Im still getting an error. *Could not create a Numpy array from the return type.* Does anybody has an idea how to solve this issue? Thks in advance! Ariel ----------------------------------------------------- DROP FUNCTION connect_to_mysql() ; CREATE FUNCTION connect_to_mysql() RETURNS TABLE(agr_id integer, agr_name string ) LANGUAGE PYTHON { import datetime import mysql.connector cnx = mysql.connector.connect(user='XXXX', password='XXXX',host='192.168.15.5',database='XXXXX',use_pure=False) cursor = cnx.cursor() query = "SELECT agr_id, agr_name FROM AGR_Agreement LIMIT 0 " cursor.execute(query) allrows = cursor.fetchall() ncols = len(cursor.description) colnames = [cursor.description[i][0] for i in range(ncols)] result = dict() for col in range(ncols): if len(allrows) == 0: result[colnames[col]] = [None] for row in range(len(allrows)): if allrows[row][col] == None: valor = None else: valor = str(allrows[row][col]) try: result[colnames[col]].append(valor) except KeyError: result[colnames[col]] = [valor] cursor.close() cnx.close() return result }; sql>SELECT * FROM connect_to_mysql() ; Could not create a Numpy array from the return type.

1 0

DEC2016-SP2 and BINARY bulk load
by Lynn Carol Johnson 11 Jan '18

11 Jan '18

Hi all – I have always used the COPY BINARY INTO … commands to load my 2.0 Billion row genetic data into a monetdb table. With 135 columns, it has been blindingly fast. Last week I moved from the June2016-SP2 release to dec2016-SP2. My binary loads are taking WAY longer. I killed one after 3 hours (via “call sys.stop(pid)” so it could clean up properly). I then started the load again, thinking perhaps the problem was related to the new columns I was adding. I have since dropped the table and remade it using the same data and scripts that worked in just over 3 minutes in February on the jun2016-SP2 load. It is really chugging along – I’m up to 30 minutes and counting. I don’t have access to the sql log files, but the Merovingian.log shows nothing. I do notice that previously the binary files, once loaded, were removed from the loading directly. This does not happen now. Were these files previously “moved” and now they are copied? Has anyone see this performance issue with Dec2016-SP2 COPY BINARY INTO …. Commands? Thanks - Lynn

5 17

RE: MonetDB - Column Header Question
by Sharma, Sreejith 09 Jan '18

09 Jan '18

I see below issue is happening only if base column datatype is integer. All other types working fine. From: Sharma, Sreejith Sent: Saturday, January 06, 2018 8:24 PM To: users-list(a)monetdb.org Subject: MonetDB - Column Header Question All, Quick question - For a specific scenario I would like to understand below - If we don't specify column header in SQL or alias, how is MonetDB picks up the column header? And what is the datatype for it? I saw it's hge (scalar type) is it so? Also, I would like to know where in source code it's happening. Example: SELECT SUM(COLUMN_NAME) FROM TABLE_NAME; ------------------+ | L3 | ------------------+ |123456678 | ------------------+ Regards, Sreejith

2 3

Bulk data upload through JDBC
by suvendu chakra 09 Jan '18

09 Jan '18

Hello, I am following SQLCopInto.java to upload bulk data into MonetDB but seems to be not working. I am even using mclient command to upload data but the command expects the input file to be accessible by the DB server, is there any alternative to this. Please suggest a best way to do through JDBC Thanks

3 3

MonetDB - Column Header Question
by Sharma, Sreejith 06 Jan '18

06 Jan '18

All, Quick question - For a specific scenario I would like to understand below - If we don't specify column header in SQL or alias, how is MonetDB picks up the column header? And what is the datatype for it? I saw it's hge (scalar type) is it so? Also, I would like to know where in source code it's happening. Example: SELECT SUM(COLUMN_NAME) FROM TABLE_NAME; ------------------+ | L3 | ------------------+ |123456678 | ------------------+ Regards, Sreejith

1 0

SyntaxException:parseError:Xclose
by Dennis Fassl 06 Jan '18

06 Jan '18

Hi all, I am generating MAL queries in Java and I am using the JDBC interface to execute them. This works fine for small tables. But if the resultSet has more than 100 entries MonetDB is printing the first 100 rows and then gives me the following error: ... [ 19, "undergraduateDegreeFrom", 1509 ] [ 19, "memberOf", 613 ] [ 19, "takesCourse", 556 ] [ 19, "advisor", 600 ] [ 20, "undergraduateDegreeFrom", 1493 ] [ 20, "memberOf", 613 ] [ 20, "takesCourse", 529 ] [ 20, "takesCourse", 578 ] [ 20, "takesCourse", 557 ] [ 20, "advisor", 1315 ] MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = sql.resultSet(table_names, table_attrs, table_type, table_lens, table_scales, start_id, type, end_id); ERROR = !SyntaxException:parseError:Xclose 0 !SyntaxException:parseError: ^';' expected Could you help me to avoid this error? This is the full query: sql.init(); tbd := sql.mvc(); start_id:bat[:int] := sql.bind(tbd,"sys","relations","start_id",0:int); type:bat[:str] := sql.bind(tbd,"sys","relations","type",0:int); end_id:bat[:int] := sql.bind(tbd,"sys","relations","end_id",0:int); table_names := bat.new(nil:str); table_names := bat.append(table_names, "start_id"); table_names := bat.append(table_names, "type"); table_names := bat.append(table_names, "end_id"); table_attrs := bat.new(nil:str); table_attrs := bat.append(table_attrs, "str"); table_attrs := bat.append(table_attrs, "str"); table_attrs := bat.append(table_attrs, "str"); table_type := bat.new(:str); table_type := bat.append(table_type, "int"); table_type := bat.append(table_type, "varchar"); table_type := bat.append(table_type, "int"); table_lens := bat.new(nil:int); table_lens := bat.append(table_lens, 1); table_lens := bat.append(table_lens, 1); table_lens := bat.append(table_lens, 1); table_scales := bat.new(nil:int); table_scales := bat.append(table_scales, 0); table_scales := bat.append(table_scales, 0); table_scales := bat.append(table_scales, 0); sql.resultSet(table_names, table_attrs, table_type, table_lens, table_scales, start_id, type, end_id); I am using "MonetDB Database Server v1.7 (Dec2016-SP5)" on Ubuntu 14.04. What am I trying to do: The goal of the project is to use Cypher as an input language for MonetDB. I have a Cypher optimizer written in Java. The output of this optimizer should be used as input for MonetDB. In the end we want to compare the the runtime of a query in MonetDB and in Neo4J. Thanks and regards, Dennis

1 1

Re: MonetDB header Datatype Question
by Stefan Manegold 04 Jan '18

04 Jan '18

What's the (SQL-)type of your column COLUMN_NAME?Which version of MonetDB are you using?Which version of the MonetDB ODBC driver are you using?Which compiler did you use to compile MonetDB on which OS?What did configure say about what the largest supported integer type is? -------- Original message --------From: "Sharma, Sreejith" <Sreejith.Sharma(a)harman.com> Date: 12/31/17 18:33 (GMT+01:00) To: users-list(a)monetdb.org Subject: RE: MonetDB header Datatype Question All, Here is the specifics of error – typeODBC: SQL_UNKNOWN_TYPE mDataType: HG_NULL_VALUE(hgColumnType) precision: 0 scale: 0 nullable: true Column name: L6. Regards, Sreejith From: Sharma, Sreejith Sent: Sunday, December 31, 2017 11:21 AM To: users-list(a)monetdb.org Subject: MonetDB header Datatype Question All, I would like to know what is the MonetDB header datatype if no alias or column name provided? In my BI application, am getting an error while using SELECT SUM(DISTINCT COLUMN_NAME) FROM TABLE_NAME. I see it works in 11.27.9 build when installed in CentOS. However, when i download the source tarball and recompile the code, this fox seems to be lost. Is there anywhere this can be identified? Or fixed? Regards, Sreejith

3 5

Re: MonetDB header Datatype Question
by Stefan Manegold 04 Jan '18

04 Jan '18

Looks fine to me, but of course you're free to try a different compiler of your choice. Stefan -------- Original message --------From: "Sharma, Sreejith" <Sreejith.Sharma(a)harman.com> Date: 1/4/18 10:31 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: RE: MonetDB header Datatype Question Thanks Stefan. However am suspecting if it's to do with the compiler. Did you got a chance to look at the details I sent across for the questions. configure:5039: gcc --version >&5 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16) Regards, Sreejith -----Original Message----- From: users-list [mailto:users-list-bounces+sreejith.sharma=harman.com@monetdb.org] On Behalf Of Stefan Manegold Sent: Wednesday, January 03, 2018 2:11 PM To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: [EXTERNAL] Re: MonetDB header Datatype Question I was mislead by the "HG" prefix in the error message. I now realize that the error message does not seem to be a MonetDB error message. Hence, I'm afraid we cannot help much. You said is works/worked with a binary installation of Jul2017-SP2 (11.27.9), but does not with a self-compiled installation of the very same release. You'd need to double-check you compilation and installation of the self-compiled release. You can also try with the latest Jul2017-SP3 (11.27.11) release --- binary and/or self-compiled --- but as said, since the error message is not from MonetDB, you'd need to search for the problem (also) in the client software / application you're using. Best, Stefan ----- On Jan 1, 2018, at 5:55 PM, Sharma, Sreejith Sreejith.Sharma(a)harman.com wrote: > Hi Stefan, > > > > Here are the details you had asked for. Any help here soon is appreciated. > > > > What's the (SQL-)type of your column COLUMN_NAME? – > > INTEGER > > > > Which version of MonetDB are you using? – > > MonetDB 5 server v11.27.9 "Jul2017-SP2" (64-bit, 128-bit integers) > > > > Which version of the MonetDB ODBC driver are you using? > > ODBC Driver - /usr/lib64/libMonetODBC.so (11.27.9 and tried 11.27.5 > updated) > > > > Which compiler did you use to compile MonetDB on which OS? > > configure:5030: checking for C compiler version > > configure:5039: gcc --version >&5 > > gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16) > > > > #define PACKAGE_NAME "MonetDB" > >| #define PACKAGE_TARNAME "MonetDB" > >| #define PACKAGE_VERSION "11.27.9" > >| #define PACKAGE_STRING "MonetDB 11.27.9" > >| #define PACKAGE_BUGREPORT "https://clicktime.symantec.com/a/1/js6skxjfSiwdIJPQwPy1ginHSudH8-MwrUFzw00H…" > >| #define PACKAGE_URL "https://clicktime.symantec.com/a/1/ZNQS7LXKm8wqT3QEExvIAnZ3Vk2CRq8bspHohtAV…" > >| #define PACKAGE "MonetDB" > >| #define VERSION "11.27.9" > >| #define HOST "x86_64-pc-linux-gnu" > >| #define MONETDB_RELEASE "Jul2017-SP2" > > > > What did configure say about what the largest supported integer type is? > > hge_bits='128' > > largest integer type: ${hge_bits} bits" > > > > > > Regards, > > Sreejith > > > > > From: users-list > [mailto:users-list-bounces+sreejith.sharma=harman.com@monetdb.org] On > Behalf Of Stefan Manegold > Sent: Monday, January 01, 2018 7:05 AM > To: Communication channel for MonetDB users <users-list(a)monetdb.org> > Subject: [EXTERNAL] Re: MonetDB header Datatype Question > > > > > > > > > What's the (SQL-)type of your column COLUMN_NAME? > > > Which version of MonetDB are you using? > > > Which version of the MonetDB ODBC driver are you using? > > > Which compiler did you use to compile MonetDB on which OS? > > > What did configure say about what the largest supported integer type is? > > > > > > > > > -------- Original message -------- > > > From: "Sharma, Sreejith" < Sreejith.Sharma(a)harman.com > > > > Date: 12/31/17 18:33 (GMT+01:00) > > > To: users-list(a)monetdb.org > > > Subject: RE: MonetDB header Datatype Question > > > > > > All, > > > > Here is the specifics of error – > > > > typeODBC: SQL_UNKNOWN_TYPE mDataType: HG_NULL_VALUE(hgColumnType) > precision: 0 > scale: 0 nullable: true Column name: L6. > > > > Regards, > > Sreejith > > > > > From: Sharma, Sreejith > Sent: Sunday, December 31, 2017 11:21 AM > To: users-list(a)monetdb.org > Subject: MonetDB header Datatype Question > > > > > All, > > > > > > I would like to know what is the MonetDB header datatype if no alias > or column name provided? In my BI application, am getting an error > while using SELECT SUM(DISTINCT COLUMN_NAME) FROM TABLE_NAME. I see it > works in 11.27.9 build when installed in CentOS. However, when i > download the source tarball and recompile the code, this fox seems to be lost. Is there anywhere this can be identified? > Or fixed? > > > Regards, > > > Sreejith > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://clicktime.symantec.com/a/1/EiR5ebgK8iUa4CLKJS5IhZJEZAKRFGZRDFU > FUCIYntY=?d=l99sUyvGli3efslX3N2ZiOXRT2oUDQGPF4w16I0hM9zUMoOkJQMri6AAcO > e4XlOqB4NFs5YhzH2NIIET0LyAMDvzRQkCYnLv2cWSJsLXuGRkmOTx5BxDA1nwzncmE0Ex > YksGCv1OTBY5h8YekFgUFxCvagkJjjUM6qSdc8J_3jJdMcNfFBT-6B5sCtZZWTBqGFRvVr > mq4XFYO_7DYsI9LdIJf7p8tqhBL-xSVWB3pCxqFx46Iyjz8nidH-BXBjg-pRI2Gk56GQ4p > kiPFSFlud_PVfn48XxozGVFit5p6x7F-AVEgKrooyJUyAchXTTYPKW5qN60JUu-5toDlGZ > U0tcE6B8a_-BAZmcVxJiZTehQx5mL1hAm0sBK2BdVeLD5BkXQ_JG8gnBigTg0N4E9I6b5C > geHSGmxUBIKMr_4WO21MsBtAqElF9oRpXg%3D%3D&u=https%3A%2F%2Fwww.monetdb.o > rg%2Fmailman%2Flistinfo%2Fusers-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list(a)monetdb.org https://clicktime.symantec.com/a/1/EiR5ebgK8iUa4CLKJS5IhZJEZAKRFGZRDFUFUCIY… _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

MonetDB header Datatype Question
by Sharma, Sreejith 31 Dec '17

31 Dec '17

All, I would like to know what is the MonetDB header datatype if no alias or column name provided? In my BI application, am getting an error while using SELECT SUM(DISTINCT COLUMN_NAME) FROM TABLE_NAME. I see it works in 11.27.9 build when installed in CentOS. However, when i download the source tarball and recompile the code, this fox seems to be lost. Is there anywhere this can be identified? Or fixed? Regards, Sreejith

1 1

Monetdb date column issue
by Amit Ambekar 27 Dec '17

27 Dec '17

Hi, i am using MonetDB v11.23.7 (Jun2016-SP1) version. in that when is reboot my ubuntu machine, my date columns in all tables becoming null . is there any issue related to version? Warm Regards, *Amit Suryakant Ambekar*** */Technology Associate/* Office: Ist floor, Poddar Chambers, Mathruradas Mill Compound, Lower Parel, Mumbai 400013 | Landline: +91 22 43470408 Cell: +91 7741823310 | Email: amit(a)g-square.in | Website:www.g-square.in

1 0

SELECT: too many nested operators
by Sharma, Sreejith 19 Dec '17

19 Dec '17

All, I have a valid scenario where our application generates a SQL which is around 4.4MB and contains 75K of combination of AND and OR operators. This SQL fails with "SELECT: too many nested operators" error while executing this SQL. While I went through the code, I noticed any thread size above THREAD_STACK_SIZE - 16 * 1024 is getting aborted. Is there any possibility that we can change this? Any reason why it's set to 16MB? As server our server has enough memory and storage and we need to understand if this can be handled in code or via parameter as this is valid scenario. Also, we can't split the SQL too. Regards, Sreejith

4 4

too many nested operators - Query size 4MB
by Sharma, Sreejith 16 Dec '17

16 Dec '17

All, Have a quick question- I have been trying to perform a test on bigger SQL and I see SQL's are failing which are above 4 MB with error "too many nested operators". There are many AND and OR operators. Not sure if there is any limitations. Regards, Sreejith

1 0

Announcement: New Jul2017-SP3 Bugfix release of MonetDB suite
by Sjoerd Mullender 14 Dec '17

14 Dec '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP3 bugfix release (11.27.11) MonetDB Common * Reimplemented summing of a column of floating point (flt and dbl) values. The old code could give wildly inaccurate results when adding up lots and lots of values due to lack of precision. Try SELECT sum(c) FROM t; where t is 100,000,000 rows, c is of type REAL and all values are equal to 1.1. (The old code returned 33554432 instead of 1.1e8.) Bug Fixes * 3898: Deadlock on insertion * 6429: ROUND produces wrong data type * 6436: Query sequence with 2x ifthenelse() and next nullif() causes mserver5 Segmentation fault * 6437: System schemas "profiler" and "json" shouldn't be allowed to be dropped. * 6439: Invalid references to sys.columns.id from sys.statistics.column_id * 6442: SEGFAULT with COPY INTO BEST EFFORT and skipping input columns * 6443: complex(?) query forgets(?) column name * 6444: Using 'with' keyword with table returning function crashes monetdb * 6445: Sqlitelogictest crash in MySQL query * 6446: sql_parser.y bug? * 6448: 'insert into' with multiple rows containing subqueries crashes * 6449: Assertion error in rel_dce_refs (sqlsmith) * 6450: Assertion error in exp_bin (sqlsmith) * 6451: Assertion error in sql_ref_dec (sqlsmith) * 6453: Assertion error in rel_rename_exps (sqlsmith) * 6454: SQL lexical error * 6455: Assertion error in rel_apply_rewrite (sqlsmith) * 6456: NULL becomes 0 in outer join * 6459: Assertion error in exp_bin (sqlsmith) * 6462: large virtual memory spike on BLOB column select * 6465: appending to variables sized atom bats other than str bats with force flag may result in corrupted heap * 6467: date_to_str formatter is wrong * 6470: mitosis gets in the way of simple select * 6471: calls to sys.generate_series should auto-convert arguments * 6472: Assertion failure in rel_rename (Sqlsmith) * 6477: assertion eror rel_push_project_up (sqlsmith) * 6478: Crash with nested order by/ limit offset * 6479: Mserver receives an assertion error on a procedure call * 6480: Segfault in mvc_find_subexp (sqlsmith)

1 0

Excessive slowness in query
by Alfio Bettini 14 Dec '17

14 Dec '17

Hi all, we're testing MonetDB performances in order to understand if it could be possible to use it as data mart in our BI software. We're considering a set of queries executed in our test environment (CentOS 7, 32 GB RAM, 4 cpu). All the executed queries have quite good performances (between 10 and 45 seconds) except the following one that is taking several minutes in order to be executed: SELECT id, count(distinct value) FROM table GROUP BY id; The table contains 100 millions of data. It has about 130 columns (varchar, int, bigint, smallint, tinyint, timestamp, datetime). We have no BLOB/CLOB/Text types. Can you suggest us any optimizations in order to have this query run faster? Do you know some possible reasons why it's so slow? Kind Regards, Alfio Bettini The information contained in this message, together with any attachments, may be privileged and confidential and is intended for the use of the addressee(s) only. Any review, copying, use or distribution of this email by others (including any of its attachments) is strictly prohibited. If you are not the intended recipient, please destroy this email and its attachments and contact the sender immediately. This email was sent by EidosMedia SpA<http://www.eidosmedia.com>

1 0