- users-list - monetdb.org

FW: Cancel query
by Zingman, Jonathan 13 Jun '19

13 Jun '19

Yes, sys.queue(), sorry. A couple of clarifications. The cookbook shows mclient, but we're using the API and we assume you can do the same thing from the API as well. Also, we use autogenerated queries that can be hundreds if not thousands of characters long. Matching in a case independent way would be very resource intensive. It's fairly clear from the documentation that MonetDB doesn't stop immediately in the middle of whatever it is doing, so that's understood. But cancelling queries has to be part of a robust database system. My model of how this might work is that the handle that’s returned when we issue a query could be used to associate with some data returned by sys.queue(), preferably tag or start time since those should be unique and fairly short. Then we stash then in some structure, and if a cancel request comes in, we do a much simpler search of the sys.queue() table that's guaranteed to work. JONATHAN ZINGMAN | Manager – TrueCall Data | DIRECT: 510.809.0423 | CELL: 510.823.5638 | www.netscout.com -----Original Message----- From: users-list <users-list-bounces+jonathan.zingman=netscout.com(a)monetdb.org> On Behalf Of Martin Kersten Sent: Tuesday, June 11, 2019 1:56 PM To: users-list(a)monetdb.org Subject: Re: Cancel query This message originated outside of NETSCOUT. Do not click links or open attachments unless you recognize the sender and know the content is safe. On 11/06/2019 22:38, Ying (Jennie) Zhang wrote: > > >> On 11 Jun 2019, at 00:58, Zingman, Jonathan <Jonathan.Zingman(a)netscout.com> wrote: >> >> In the cookbook, you give a good way to find out the status of queries by using the sys.query() call, > > Hai, Jonathan, > > MonetDB doesn’t have a function called sys.query(). Did you mean sys.queue() instead? > >> which returns the list of queries tagged in various ways. Our question is whether there is any way to access, e.g., the tag value. In our application, we need to occasionally cancel queries from our UI. The UI has no direct knowledge of the query. While we might be able to find the query we want to cancel by comparing the query text, our automatically generated queries can be very long, and if there’s some internal reformatting by Monetdb, we might not get an exact match. So is there anything in the handle or elsewhere that we can use to identify the query we want to cancel? > > Correct me if I’m wrong, but I’m not aware of a feature with which you can query the qtag of a query. Some code study will be needed to see if it’s feasible to add such feature, and how. > > You can probably get away with string matching, since MonetDB doesn’t do much/any reformatting next to to-lowercase. Not sure how much text the “query” field of sys.queue() holds. > > In addition, please be aware that > i) sys.queue() is originally develop for debugging purpose… > ii) the minimal cancellation entity is a function, i.e., when MonetDB is busy executing a very long running function, a cancellation will only take effect *after* that function has finished => then MonetDB won’t continue executing the next function. Small correction here, with function we mean a MAL function call. So, if the system is executing a very time consuming join, it will *not* be interrupted, but rather at the end of it the query in which is rans is terminated. > > Regards, > Jennie > >> >> JONATHAN ZINGMAN >> Manager – TrueCall Data >> >> <image001.png> >> 2855 Telegraph Ave, Suite 200 >> Berkeley, CA, 94705-1169 >> >> DIRECT: 510.809.0423 >> CELL: 510.823.5638 >> http://www.netscout.com >> >> >> _______________________________________________ >> users-list mailing list >> users-list(a)monetdb.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.monetdb.org_mailma… > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.monetdb.org_mailma… > _______________________________________________ users-list mailing list users-list(a)monetdb.org https://urldefense.proofpoint.com/v2/url?u=https-3A__www.monetdb.org_mailma…

2 1

Cancel query
by Zingman, Jonathan 11 Jun '19

11 Jun '19

In the cookbook, you give a good way to find out the status of queries by using the sys.query() call, which returns the list of queries tagged in various ways. Our question is whether there is any way to access, e.g., the tag value. In our application, we need to occasionally cancel queries from our UI. The UI has no direct knowledge of the query. While we might be able to find the query we want to cancel by comparing the query text, our automatically generated queries can be very long, and if there's some internal reformatting by Monetdb, we might not get an exact match. So is there anything in the handle or elsewhere that we can use to identify the query we want to cancel? JONATHAN ZINGMAN Manager - TrueCall Data [logos] 2855 Telegraph Ave, Suite 200 Berkeley, CA, 94705-1169 DIRECT: 510.809.0423 CELL: 510.823.5638 www.netscout.com<http://www.netscout.com/>

3 2

MonetDB - Case Sensitive
by Suresh Devarakonda 11 Jun '19

11 Jun '19

Hi, I am new to the MonetDB and interested to know about its case sensitivity. We noticed that objects are being created in lower case if they are not specified in quotes so it appears to me that default case for objects created in Monet is LOWERCASE. But, we got a requirement that objects should be created in upper case even though they are not specified in quotes in DDL statements. In other words, can we make the default to UPPERCASE for all the objects when they created. Please let us know if that is possible. Your early response is very much appreciated. ~Suresh D

2 1

Installation error on AWS Linux
by Gerald Sangudi 04 Jun '19

04 Jun '19

Hello, I'm trying to install MonetDB on AWS Linux. The first command succeeds: sudo yum install https://dev.monetdb.org/downloads/epel/MonetDB-release-epel.noarch.rpm The second command fails: sudo yum install MonetDB-SQL-server5 MonetDB-client Error: *[ec2-user@ip-172-31-23-149 ~]$ sudo yum install MonetDB-SQL-server5 MonetDB-clientLoaded plugins: extras_suggestions, langpacks, priorities, update-motdhttps://dev.monetdb.org/downloads/ <https://dev.monetdb.org/downloads/>epel/2/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not FoundTrying other mirror. One of the configured repositories failed (MonetDB 2 - x86_64), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:* It seems to me there's a reference to epel/2 which is out of date. Should that be epel/7? Is it possible to get an updated rpm? Thanks, Gerald

2 2

REPLICA, CLUSTER
by Ariel Abadi 29 May '19

29 May '19

Hi! Does anybody implemented a REPLICA DB, or a CLUSTER with several nodes for monetdb? I couldnt find too much information in how to implement it. Any information you can provide, will help a lot! Thks in advance Ariel

1 0

MonetDB performance on large table with 20K columns
by Malware Research 16 May '19

16 May '19

Hi, I posted a question on StackOverflow <https://stackoverflow.com/questions/55951563/monetdb-performance-on-a-large…> regarding s specific performance issue. I’m not sure if the right people are monitoring S/O from time to time so I thought I’ll post it here too. Here it is in case you do not want to follow the link: I'm testing MonetDB as a solution for a data-science project. I have a table of 21K columns - all but three are features described as float (32bit) and 6.5M rows (which may or may not become larger, perhaps up to 20M rows). My aim is to use the integrated Python on MonetDB to achieve the ability to train without exporting the data from the DB every time. In addition, queries on specific columns are necessary so the columnar storage can be a significant advantage. I have compiled MonetDB 11.31.13 to gain the embedded Python support. OS is CentOS 7. Storage is not SSD. 48 core server with ~300GB of memory. I created an (unique) index on the table (without analyze). I noticed that when I SELECT * FROM [TABLE_NAME] SAMPLE 50; it takes a long long time to complete. I then tried: SELECT f1, f2, ..., f501 from [TABLE_NAME] SAMPLE 50; SELECT f1, f2, ..., f1001 from [TABLE_NAME] SAMPLE 50; SELECT f1, f2, ..., f2001 from [TABLE_NAME] SAMPLE 50; ... SELECT * from [TABLE_NAME] SAMPLE 50; I ran the queries locally with mclient and used time to measure the amount of time it took and I noticed two things: 1. There is a period where a single core is taking 100% CPU. The more columns the longer it takes to complete. Only when it finishes I can see all cores working, data being consumed, etc... In addition, during that time, the query does not appear in the result of select * from sys.queue(); Eventually, the time needed to get 50 rows from the table was almost 4 hours. 2. The amount of columns is doubled but between each step in the test the amount of time it takes to get a result is tripled. So my questions is: Is this behaviour expected or does it reflect something I did wrong? The data requested from the table should be around 4MB (50 * 21000 * 4Bytes), so this reflects a significant time waiting for such a small amount of data. Help is appreciated! Help is very much appreciated!

2 1

MonetDB mercurial source doesn't compile on macOS Mojave
by Ying Zhang 16 May '19

16 May '19

Dear all, When trying to compile the MonetDB mercurial source on macOS Mojave, it ends with the error: /Users/jennie/monet/Apr2019/common/stream/stream.c:819:10: error: implicit declaration of function 'fdatasync' is invalid in C99 [-Werror,-Wimplicit-function-declaration] && fdatasync(fileno(fp)) < 0 ^ 1 error generated. Does anyone know a fix or workaround for this? Thanks a lot! Jennie

1 1

BPP version problem
by Corbit, Dann 02 May '19

02 May '19

Mdblaunch.bat seems to work: C:\Program Files\MonetDB\MonetDB5>mserver5 --dbpath=dbfarm\voc\ --set mapi_port=50000 # MonetDB 5 server v11.33.3 (Apr2019) # Serving database 'voc', using 12 threads # Compiled for x86_64-pc-winnt/64bit # Found 31.927 GiB available main-memory. # Copyright (c) 1993 - July 2008 CWI. # Copyright (c) August 2008 - 2019 MonetDB B.V., all rights reserved # Visit https://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # MonetDB/SQL module loaded > But neither of these batch files work. C:\Program Files\MonetDB\MonetDB5>MSQLserver.bat MonetDB/Python Disabled: Python 3.7 installation not found. !FATAL: BBPinit: incompatible BBP version: expected 061041, got 061033. This database was probably created by a too old version of MonetDB. Press any key to continue . . . C:\Program Files\MonetDB\MonetDB5>M5server.bat MonetDB/Python Disabled: Python 3.7 installation not found. !FATAL: BBPinit: incompatible BBP version: expected 061041, got 061033. This database was probably created by a too old version of MonetDB. Press any key to continue . . . They complain about the server version. I tried deleting the database folder and restarting. The database was rebuilt. But I still get the errors. How do I fix this error? I do not have real database systems. I only have the voc sample data.

3 3

Trying to link IBM Cognos and MonetDB via ODBC
by SERET Paul 25 Apr '19

25 Apr '19

Dears, I get the following error while trying to link IBM Cognos Analytics software to MonetDB via ODBC (sorry, the error message is in french, I translate below) : [cid:image001.png@01D4FB81.C8C4CA40] QE-DEF-0285 The logon failed QE-DEF-0325 The logon failed for the following reason: RQP-DEF-0068 Unable to connect to at least one database during a multi-database attach to 1 database(s) in: 'testDataSourceConnection' UDA-SQL-0031 Unable to access the 'testDataSourceConnection' database. Check that the connection parameters to the database are configured correctly. For example, ensure that the data source connection contains the signon information, such as a password, to connect to the database. UDA-SQL-0107 A general exception has occurred during the operation 'attach(error code -49)'. The specified driver has not been loaded because of system error 127 : The specified procedure cannot be found. (MonetDB ODBC Driver, C:\Program Files (x86)\MonetDB\MonetDB ODBC Driver\lib\libMonetODBC.dll). The last section of the message sounds strange to me, because the DLL seems to be correctly installed from the most recent ODBC driver installer (2019/01/04) : [cid:image002.png@01D4FB84.B9159040] It looks like there is an issue inside the DLL program. Is there a known issue with this DLL or the driver ? IBM Cognos mentions that the ODBC driver must be compliant with the 3.5 standard. I guess this is the case for MonetDB ODBC driver, isn't it ? Could you please have any clue to get this sorted out ? Thanks in advance, best regards Paul SERET

1 0

How can I get a parameterized bulk version of a scalar C UDF to work in SQL?
by aris 25 Apr '19

25 Apr '19

Hi, Say that I have a family of simple scalar transforming function f(x ,p) => y parameterized by p which are implemented as C UDF's and exposed as SQL UDF's. Is it possible to create a bulk version of this? So the scalar C signature is simplified to something like char* f(int* output, int* input, const char** func_parameter); the mal stuff module m; command f(input:int, func_parameter:str):int address f; and in sql we have CREATE FUNCTION f(input INT, func_param STRING) RETURNS INT EXTERNAL NAME m.f; Now for the bulk version I would like to use in C: char* f(bat* output, bat* input, const char** func_parameter); in MAL: module batm; command f(input:bat[:int], func_parameter:str):bat[:int] address f; in SQL: reuse the scalar SQL signature. But that doesn't seem to work. When I compile this and call the udf in a sql query on an actual column, the plan applies the scalar version on each tuple instead of calling the bulk version once. However if I remove the parametrization, i.e. removing the func_parameter from the signature, the bulk version is correctly called. Aris

1 0